October 2013 This publication was produced at the request of the United States Agency for International Development. It was prepared independently by Moneval Solutions Ltd (www.moneval-solutions.com) under contract to ACDI VOCA and its Sub-Contract Partners: JSI, PCI and Making Cents International. Mid-Term Evaluation Report: Liberian Agricultural Upgrading Nutrition and Child Health Implementation Contractors: Evaluation undertaken by: PHOTO CREDIT Cover photo and all subsequent photos used in the report were taken by Daphyne Williams, Team Member during the course of evaluation, 2013. MID-TERM EVALUATION LIBERIAN AGRICULTURAL UPGRADING NUTRITION AND CHILD HEALTH October 11, 2013 DISCLAIMER The author’s views expressed in this publication do not necessarily reflect the views of the United States Agency for International Development or the United States Government. LAUNCH MTE – Final Report ©Moneval Solutions 2013 Page i of 113 CONTENTS 1. EXECUTIVE SUMMARY.....................................................................................................1 1.1 EVALUATION PURPOSE AND EVALUATION QUESTIONS ..........................................................................1 1.2 PROJECT BACKGROUND...............................................................................................................................1 1.3 METHODOLOGY.............................................................................................................................................1 1.4 MAIN FINDINGS ..............................................................................................................................................1 1.4.1 SO1: Food Access and Availability................................................................................................................1 1.4.2 SO2: Reduced Chronic Malnutrition of Vulnerable Women and Children.......................................................4 1.4.3 SO3: Increased Access to Education Opportunities.......................................................................................4 1.4.4 Commodity Management ...........................................................................................................................6 1.5 PRIORITY RECOMMENDATIONS ...................................................................................................................7 1.5.1 Recommendations by Strategic Objective.....................................................................................................7 1.5.2 Recommendations for Cross Cutting Themes.............................................................................................10 2. INTRODUCTION ................................................................................................................12 3. MTE METHODOLOGY.......................................................................................................14 3.1 EVALUATION QUESTIONS.............................................................................................................................15 3.2 LIMITATIONS AND RESTRICTIONS................................................................................................................15 4. SO1: EVALUATION FINDINGS ........................................................................................17 4.1 DESIGN AND PLANNING...............................................................................................................................17 4.1.1 Integration of Project Objectives................................................................................................................17 4.1.2 LAUNCH Strategies and Activities.............................................................................................................17 4.2 IMPLEMENTATION...........................................................................................................................................23 4.2.1 Effectiveness ............................................................................................................................................23 4.2.2 Efficiency .................................................................................................................................................37 5. SO2: EVALUATION FINDINGS ........................................................................................39 5.1 DESIGN AND PLANNING...............................................................................................................................39 5.2 INTEGRATION OF PROGRAM OBJECTIVES...................................................................................................40 5.3 LAUNCH STRATEGIES AND ACTIVITIES.....................................................................................................41 5.3.1 Target Groups..........................................................................................................................................41 5.3.2 Government Engagement .........................................................................................................................41 5.3.3 Government Health Center Capacity Building ............................................................................................42 5.3.4 Care Group and Lead Mother Organization...............................................................................................43 5.4 IMPLEMENTATION...........................................................................................................................................45 5.4.1 Effectiveness ............................................................................................................................................45 5.4.2 Efficiency .................................................................................................................................................49 6. SO3: EVALUATION FINDINGS ........................................................................................51 6.1 DESIGN AND PLANNING...............................................................................................................................51 6.1.1 Integration of Project Objectives................................................................................................................51 6.1.2 LAUNCH Strategies and Activities.............................................................................................................53 6.2 IMPLEMENTATION...........................................................................................................................................54 6.2.1 Effectiveness ............................................................................................................................................54 6.2.2 Efficiency .................................................................................................................................................58 7. CROSS CUTTING THEMES...............................................................................................60 7.1 GENDER...........................................................................................................................................................60 7.2 SUSTAINABILITY ..............................................................................................................................................61 LAUNCH MTE – Final Report ©Moneval Solutions 2013 Page ii of 113 7.3 COORDINATION AND INTEGRATION.........................................................................................................63 7.4 MONITORING AND EVALUATION................................................................................................................64 7.5 DISASTER REDUCTION AND EARLY WARNING SYSTEM...........................................................................65 7.6 INFRASTRUCTURE ...........................................................................................................................................66 8. COMMODITY MANAGEMENT.........................................................................................67 8.1 COMMODITIES RECEIPTS AND STORAGE ....................................................................................................67 8.2 TRANSPORTATION AND FOOD DISTRIBUTION MANAGEMENT .............................................................68 8.3 COMMODITIES, BELLMON ANALYSIS AND MONETIZATION ....................................................................68 9. LESSONS LEARNED...........................................................................................................70 10. RECOMMENDATIONS.......................................................................................................72 10.1 PRIORITY RECOMMENDATIONS ...................................................................................................................72 10.2 SECONDARY RECOMMENDATIONS .............................................................................................................75 10.3 CROSS CUTTING RECOMMENDATIONS......................................................................................................79 10.3.1.1 Priority Recommendations ....................................................................................................................79 10.3.1.2 Secondary Recommendations ...............................................................................................................79 ANNEXES - ANNEX I: EVALUATION STATEMENT OF WORK - ANNEX II: EVALUATION METHOD LAUNCH MTE – Final Report ©Moneval Solutions 2013 Page iii of 113 ACRONYM LIST ACF) African Conservation Foundation AEA Agricultural Extension Agents ANC Antenatal Care APR Annual Project Reports ARR Annual Results Review ART Annual Resource Requests BCC Behavior Change Communication CAC County Agriculture Coordinator CDCS Country Development Cooperation Strategy CFNFS Comprehensive Food and Nutrition Survey (Liberia) CHC Community Health Committees CHILD Children in Local Development CHT Community Health Team CIMCI Community Integrated Management of Childhood Illness CMAM Community Management of Acute Malnutrition COP Chief of Party CSB Corn Soy Blend CU2 Children under Two years old DEO District Education Officer DCOP Deputy Chief of Party DHOs District Health Officers DMC Disaster Management Committees DMO District Medical Officer EBF Exclusive Breast Feeding ECD Early Childhood Development ENA Essential Nutrition Actions EQ Evaluation Question FaaB Farming as a Business FANTA Food and Nutrition Technical Assistance FDP Food Distribution Points FEW Field Extension Worker FFP Food for Peace FGIs Focus Group Interviews LAUNCH MTE – Final Report ©Moneval Solutions 2013 Page iv of 113 FSNS Food Security and Nutrition Strategy FY Fiscal Year GCHV Government Community Health Volunteers GoL Government of Liberia HAZ Height for Age HCA Host Country Agreement IGA Income Generating Activities IYCN Infant and Young Child Nutrition (USAID project) IPTT Indicator Performance Tracking Table IT/GIS Information Technology/Geographic Information JSI John Snow, Inc. LAUNCH Liberia Agricultural Upgrading, Nutrition and Child LMEP Liberia Monitoring and Evaluation Program M&E Monitoring and Evaluation MCI Making Cents International MoA Ministry of Agriculture MOCI Ministry of Commerce and Industry MoE Ministry of Education MOHSW Ministry of Health and Social Welfare. MT Metric Ton MTE Mid-term evaluation NGO Non-Governmental Organization OICI Opportunities Industrialization Centers International PCI Project Concern International PERSUAP Pesticide Evaluation Report and Safe Use Action Plan PHH Post harvest handling PIDS Performance Indicator Database System PLW Pregnant and Lactating Women PM2A Preventing Malnutrition in Children under Two Approach PMP Performance Monitoring Plan PREPs Pipeline and Resource Estimate Proposals PTA Parents Teachers Association RF Results Framework SG Savings Group SOW Scope of Work SUSU Savings Club LAUNCH MTE – Final Report ©Moneval Solutions 2013 Page v of 113 TBA Traditional Birth Attendants TTM Trained Traditional Midwives USAID United States Agency for International Development WASH Water, Sanitation and Hygiene WAZ Weight for Age LAUNCH MTE – Final Report ©Moneval Solutions 2013 Page 1 of 113 1. EXECUTIVE SUMMARY 1.1 Evaluation Purpose and Evaluation Questions It was agreed in May 2012 to carry out a Mid-term Evaluation (MTE) to assess program performance towards meeting its objectives and to help LAUNCH and partners realign activities and effort with program strategy, objectives and targets. The evaluation therefore focused on the impact of LAUNCH activities, Design, Planning, and Implementation and four cross cutting themes of Gender, Monitoring and Evaluation, DRR and early warning System and Sustainability. In addition the MTE sought to identify areas for improvement, lessons learned and actionable recommendations that drive performance improvement and ultimately deliver increased value to the beneficiary. 1.2 Project Background In 2010, USAID awarded ACDI/VOCA a five-year USD 40 million Title II Multi Year Assistance Program in Liberia. The “Liberia Agricultural Upgrading, Nutrition and Child Health” (LAUNCH) program aims to improve food security of vulnerable people in Bong and Nimba Counties through an integrated approach to implementing three Strategic Objectives (SOs). ACDI VOCA is the lead organization and is responsible for SO1 ‘Increased Availability of and Access to Food of Vulnerable Rural Populations’ while SO2 ‘Reduced Chronic Malnutrition of Vulnerable Women and Children’ and SO3 ‘Increased Access to Education Opportunities’ are being implemented by Concern International, John Snow International Incorporated (JSI) and Making Cents International (MCI). LAUNCH targeted 10,800 farmers in agriculture, 10,281 pregnant and lactating women (PLW), 16,770 children 6-23 months, as well as 98,094 family members. In addition, the program aims to reach out to as many as 13,800 school children through institutional assistance to local schools and training in rural entrepreneurship to out-of-school youth. This allows targeted communities to benefit from sustainable livelihoods interventions as well as health and nutrition and education services. The corresponding performance indicators (Intermediate Results 1.1 through to 3.2) are defined in an Indicator Performance Tracking Table (IPTT). 1.3 Methodology The methodology adopted for the MTE used three complementary methods these being analysis of available documentation, interviews and discussions with stakeholders and LAUNCH participants, and field visits and discussions with direct beneficiaries in areas where the program activities are implemented. The MTE team presented preliminary findings to the LAUNCH team in Monrovia following the field work. 1.4 Main Findings The main MTE findings are summarized in the sections below. The findings include a mixture of achievements and challenges for the LAUNCH program and are considered to be the most relevant for LAUNCH. Please refer to the main report for the full set of MTE findings. 1.4.1 SO1: Food Access and Availability SO1: “Increased Availability of and Access to Food of Vulnerable Rural Populations” is fully aligned with the “Feed the Future” initiative and also forms part of the agricultural policy according to the LAUNCH MTE – Final Report ©Moneval Solutions 2013 Page 2 of 113 Vice Minister for Planning and Development. The strategy of LAUNCH is well adapted to local conditions and food security objectives and goals. Achievements: Considering the most recent figures1 for Fiscal Year 20122 there are 4,311 farmers in 220 Focus Groups (FGs) who have been reached by LAUNCH, together with their families. They have learned some improved production techniques and the majority of FG members have harvested a first crop on their demonstration plot. Incremental volume of the demonstration plot production (IR1.1) is still low with some 12 m/tons of cassava, rice, or vegetables in total for the time being3. The majority of farmers met during the MTE, however, promised that they are planning to adopt the acquired techniques which will add an additional benefit to IR1.1 and its contribution to SO1. Considering the qualitative component of the SO1 indicators, it is noted that the project has opened perspectives and initiated first steps towards more stable livelihood conditions. Farmers have been introduced into more systematic food production practices and new farming practices. With respect to IR1.2 (“increased livelihood opportunities”) LAUNCH has made progress with some FGs specializing in vegetable crops. Two of the groups visited had quite successfully started to build up commercial operations. The majority of the cassava groups have, so far not yet established commercial links but the rice groups visited had a first harvest from the block farm plots in line with the planned goals. Some of the crop will be provided to the participants and the remainder will be stored with the intention of selling the crop once rice prices become more favorable. Technical Field Staff and training: LAUNCH extension strategy is implemented by an extension team of some 30 Agricultural Extension Agents (AEAs) plus 8 subject matter specialists covering livelihood, “Farming as Business” (FaaB), post harvest management and coordination. This structure provides good support for the project and the extension strategy for year three is considered adequate for 130 farmers groups and 3,231 members in six districts of Bong and Nimba County. However, the implementation of the strategy has not been effective as anticipated by LAUNCH. This is evident by the low production levels and that lack of tangible results at this stage in the program. AEAs have good agricultural backgrounds and were able to answer questions regarding production issues related to the three focus crops. Most training was practical and took place on block farms in line with the activities defined in the LAUNCH Strategy Document. Farming as a Business: FaaB has been considered in the LAUNCH Strategy Document as a training option for FGs after successfully completing activities on demonstration plots. The Strategy 1 Figures been provided by ACDI/VOCA on June 30th, 2013. FY 2012 data were: 3,231 farmers in 130 FGs. 2 See report: LAUNCH _PREP_FY12_Narrative_12 16 2011_FINAL_approved. 3 Not all yield data have been collected so far as some plots have not yet been harvested. LAUNCH MTE – Final Report ©Moneval Solutions 2013 Page 3 of 113 Paper has identified all farming members of a FG as potential trainees. Training comprises of budgeting, work planning, cash flow, record keeping and other activities. Care needs to be taken to ensure the training is aligned with the skill level of the trainees as the trainees have little business or administration experience and the majority do not meet basic requirements regarding literacy, numeracy skills etc. to be businesswomen (or businessmen). Investment in production and processing has not been considered within the LAUNCH Strategy Document as an integrated instrument to enhance FG development. However, it does form part of the long term strategy of the saving groups. Credit and Saving Groups (SGs): Some smaller groups depend entirely on member contributions. Assets may be sufficient for small trade credits, but these will not generate sufficient funds for investment projects during the lifetime of LAUNCH. The SGs which only consist of FG members have the best prospects for becoming sustainable, especially if affiliated with a vegetable FG, for example: there was a case of an SG which received a deposit of more that USD 1,200 from vegetable sales which could be used as working capital or for investment purposes for the FG. Management and Coordination: Program implementation is mainly in the hands of the two County offices (Totota and Saclepea) who organize and coordinate AEAs and subject matter specialists. The head office is responsible for strategic issues as well as monitoring of activities and results which ensure efficient implementation of most day-to-day activities. The organizational structure is considered to be efficient and well adapted to the geographical conditions. The Annual Monitoring Survey provides useful data regarding overall food availability and access. Based on project records reviewed during the MTE the quality of monitoring data for food production needs to be improved. The monitoring data should include information on food production development of individual plots with the corresponding information collected by AEAs. Conclusions: In assessing the overall performance the following conclusions have been made:  Some of the targets under IR1.1 have been achieved; actual progress versus IR1.2 is still lagging behind although overall coordination is decentralized and efficient.  Program effectiveness is still limited by the shortfall in numbers of FG and members during the first two years and the low attention which the project gives to the food production on individual farmer plots. The promotion of vegetable production seems to have the best prospects to improve household food security. For the time being there have not been reports on cassava shortages in the regions. Consequently effects of improved cassava cultivation techniques will not be visible as long as no feasible marketing opportunities have been identified nor investment in processing has been approved. This would also imply need for stronger integration of the SGs within the FG.  Prospects for sustainability of the results at FGs and at farm level (with respect to food security) are satisfactory and will improve further, once the bottlenecks regarding marketing and processing are solved4. The program does not work within an existing institutional 4USAID (pdf.usaid.gov/pdf_docs/pnabg576.pdf;(DW Brinkerhoff – 1990): Institutional sustainability is achieved, in case an institution is able to recover part of their cost and/or provide a continuous s flow of benefits and/or survive over time as identifiable unit. LAUNCH MTE – Final Report ©Moneval Solutions 2013 Page 4 of 113 arrangement and institutional sustainability of the services has not been fully considered. Consequently there is a risk that institutional sustainability will be limited. 1.4.2 SO2: Reduced Chronic Malnutrition of Vulnerable Women and Children SO2 interventions aim to decrease malnutrition in pregnant and lactating women and children under two by implementing the ‘Preventing Malnutrition in Children under two Approach’ (PM2A). It is widely felt by the LAUNCH staff that malnutrition in Liberia is largely behavioral (both in diversity and practice), and due to the utilization of available foods, especially during lean seasons. The implementation of LAUNCH is in line with the PM2A and Care Group methodology in targeted population and Care Group geographic distribution. Health Knowledge: LAUNCH shows success in the promotion and uptake of exclusive breastfeeding as evidenced through focus group discussions with staff, lead mothers and household mother s and supported by the LAUNCH FG discussants (Lead Mothers, Household Mothers, District Health and Medical Officers) were knowledgeable of Exclusive Breast Feeding (EBF) practices. In Nimba, 4 out of 6 Neighborhood and Lead Mother FGDs reported that since the commencement of LAUNCH, they have changed a number of traditional practices and stopped discarding colostrum after learning of its importance in the groups. Lead Mothers in these communities stated that trained midwives are now also promoting the use of colostrum. The Lead Mothers were, however, not able to articulate why this change occurred. Additional investigation and documentation of the success in the adoption and uptake of colostrum promoted behavior is necessary to confirm this important change in traditional beliefs. Target Groups: Care Groups are appropriately distributed both geographically and in size, in line with the methodology. Lead Mothers are instructed in health education strategies that are in line with Liberian and international standards. Lead Mothers do not report difficulty in conducting health education sessions with neighborhood mothers. Focus of LAUNCH: FGDs conducted with households and Lead Mothers indicated that they believe the purpose of LAUNCH is to make children healthier, provide rations and to educate mothers in improved health behavior. Mothers attributed health improvements in their children to the ration, not to the improved nutrition or their other actions such as breastfeeding or Antenatal Care (ANC) visits. 1.4.3 SO3: Increased Access to Education Opportunities LAUNCH intends to increase access to education opportunities through the achievement of two intermediate results (IRs): Increased community capacity to support education infrastructure and systems; and increased access to livelihood-based education. Implementation of Education Strategy: To achieve these results LAUNCH planned to engage school authorities, community leaders, and Parents Teachers Associations (PTAs) to support local education. Specifically, LAUNCH’s strategy has been to work closely with school administrators and PTA members to: 1) establish, or reactivate, local PTAs with clear roles and duties; and 2) identify critical factors impacting a schools’ performance. LAUNCH’s intervention in schools and communities has helped them take greater ownership of schools and they better placed to ensure their sustainability. LAUNCH MTE – Final Report ©Moneval Solutions 2013 Page 5 of 113 Staff Structure and Responsibilities: SO3 is managed by two separate managers - one for each intermediate result. The support provided to schools and communities under the first intermediate objective has proven to be of immense help for the running of education activities in targeted schools and communities. For example, all Parents Teachers Associations (PTAs) in communities visited are trained and taking ownership of community schools. PTAs now call regular scheduled meetings to discuss activities concerning their respective schools and to resolve related issues. Additionally teacher attendance has improved due to tracking by school authorities. Record keeping and the tracking of activities are now common in all schools visited while PTAs and other community members are now constructing annexes using local material gathered by them. However, education officers are not fully involved in the implementation of SO3 activities as there appears to be a lack of consistency of their involvement at district level. For example, in Bong, the Sanoyea District Education Officer stated that he was un-informed of LAUNCH activities and that he had never been involved in any activities carried out by LAUNCH. While the Salala District Education Officer articulated his delight in LAUNCH’s intervention in 24 out of 28 schools. Education officers in Nimba stated that they were aware of LAUNCH activities and had participated in some key activities such as school selection and trainings for PTAs and teachers. Youth: The youth component has had a positive impact in the targeted communities. Youth who benefit from the entrepreneurial training are now capable of initiating business activities on their own. The training and mini-grants have provided an opportunity for them to learn and practice what they have learned thus contributing to their livelihood pursuits. Youth groups are now giving loans to other youth to start businesses and there have not been any problems with loans being repaid. Youth clearly show understanding about trustworthiness and performance as a basis for receiving mini-grants. Youth are involved in establishing “table markets” composed of a table and assorted materials which are for sale such as sweets, “chicklets”, biscuits. Some even engage in money exchange. School Gardens: The weakest component of strategic objective 3 is the school gardens as they have delivered little value in the two counties. In Sanoyea District, Bong County, discussants stated that the intervention had ended when locations were identified for the garden. In Nimba, not much was mention about school gardens but they were aware of farming groups. Integration: There is dire need for the integration of all of LAUNCH activities. This will ensure maximum use of resources with greater impact on communities. Integration will ensure communities benefit from all of LAUNCH’s activities thereby making monitoring more holistic. Apart from the limited integration with SO1 for the school gardens there also exist opportunity for integration with SO2 through the establishment of health or hygiene clubs in schools. Student participation in WASH activities would see a positive impact on their health at home and in school. Coordination: Centralized procurement has impacted negatively on timely implementation of project activities. In Nimba the construction of annexes are at a standstill as communities await their fifty bags of cement promised by LAUNCH. Annexes under construction were seen in every community in the districts visited in Nimba County. LAUNCH MTE – Final Report ©Moneval Solutions 2013 Page 6 of 113 1.4.4 Commodity Management5 Commodity Import System: All food commodities are received at the port of Monrovia. While there have been incidences of pilferage at the port in the past, measures have been taken to ensure this is not repeated. The administrative formalities for clearing and discharge of commodities have been dealt with through meetings and orientation provided by LAUNCH staff to selected clearing agents and stevedores. Warehouse storage adheres to ideal warehouse management principles of proper commodity spacing and stacking security and safety including pest control (approved Pesticide Evaluation Report and Safe Use Action Plan (PERSUAP) guidelines). Food distribution to beneficiaries is well managed with a system allowing for beneficiaries travelling from a greater distance to have priority versus beneficiaries travelling a shorter distance. Commodity Value and Monetization: The quantities of commodities including vegetable oil, bulgur wheat and split yellow peas are within the recommended levels as per Bellmon analysis. These items have no impact on local production and market and pose no pressure on public storage capacity. With respect to rice, in terms of yearly monetization (during year 3 of the project’s implementation) the monetized quantity of 4,540 m. tons of rice represents approximately 1.5 per cent of total anticipated commercial imports. This figure is well within the quantities that could create market distortions. However with the Ministry of Commerce has exercising strict control over the price of rice which has resulted in an average cost recovery of 78% against an anticipated recovery of 88%. Cost recovery based on market prices was, approximately 62% as against an anticipated 78% for vegetable oil and approximately 71% with respect to wheat. The proceeds are thus substantially less than anticipated and although with the roll-over and inflow of Year four funds 5 The SOW for the evaluation did not allow an in depth assessment of commodity management. The conclusions are based on a cursory assessment being made within the time frame and data available. Photo: Daphyne Williams, 2013 Challenges of food distribution in Liberia LAUNCH MTE – Final Report ©Moneval Solutions 2013 Page 7 of 113 the project might manage, there is the possibility of it facing a problem of resources to continue implementation. Added to the above issue has been the fact that Indian production and sales of par boiled rice has entered the market at about half the USAID prices. In addition, it has been reported that Japan has also donated a substantial amount of rice to Liberia. Although the program has found cost savings, with no changes expected in the pricing of rice, the program could face a problem of resource availability to cover its expenditures without a resolution of the price anomaly or without an infusion of additional financial resources to meet operational expenditures. 1.5 Priority Recommendations The priority MTE recommendations are detailed in the two sub-sections below covering each SO as well as Cross Cutting Themes and are considered a priority for the LAUNCH project. Please refer to the main report for the full set of MTE recommendations including additional secondary recommendations. 1.5.1 Recommendations by Strategic Objective SO1 Recommendation: Extension of project intervention area (USAID, ACDI/VOCA, LAUNCH Management) LAUNCH Management should analyze whether it is feasible to increase the number of villages attended and/or the number of supported FGs and beneficiaries of the project in the next two years. This is intended to compensate for the most likely small benefit generated per head of FG member who is participating in the program (please refer to section 4.2.1.4 for details). Likewise alternative measures should be included into the yearly planning in order to attract more women as members of FGs as they have a stabilizing effect on those groups. SO1 Recommendation: Hands on management (ACDI/VOCA, LAUNCH Management): A more hands-on management of component SO1 is recommended by looking for executive staff with farming backgrounds and knowledge about shifting cultivation techniques as well as risks, in view of the problems with low productivity of food producing farms in Nimba and Bong. (Please refer to chapter 4.2) SO1 Recommendation: Launch Agricultural Extension Strategy Document (Agricultural Extension team): A comprehensive extension strategy needs to be developed defining: a) the type and content of LAUNCH assistance to the FG after year one; b) the type and volume of assistance provided to individual farmers plots of FG members; and c) the possible support to commercial farmers associated with a FG as well as support to processors and traders associated with FG (Please refer to 4.2.1.10). For the time being extension activities are limited to FG plots and FG activities and with respect to Saving Groups it needs to be decided whether or not savings groups shall be an independent instrument or whether they should be integrated into FG as it can lead to conflicts if non FG members are deciding about the savings of the FG. The Extension Team should look for assistance in the area of agricultural production systems (for example through cooperation with a University) in order to investigate additional crops and production techniques suitable to be introduced in the project area with higher return on farm labor input (Please refer to 4.2.1.3). It is suggested that some AEAs receive special short term training regarding medium term farm planning under shifting cultivation conditions, management of food production risks (please refer to section 4.2.1.10 for details) SO1 Recommendation: Funds for food processing and marketing (USAID, ACDI/VOCA and LAUNCH Management): Considerably higher funds should be budgeted for investments in processing equipment or agricultural machinery services or investment in material for chicken LAUNCH MTE – Final Report ©Moneval Solutions 2013 Page 8 of 113 cooperatives. It is suggested that an investment budget of some USD 1000-2000 be foreseen for each group which has successfully completed the year 1 exercise (please refer to section 4.2.1.10 for details). Part of the funds assigned d to the design and implementation of some of the monitoring activities (such as creation of jobs, total food production etc.) could rather be assigned to finance FG small investment projects in order to assure more effectiveness. SO1 Recommendation: Food processing and marketing (LAUNCH extension team): More processing, marketing and management skills will be needed in the extension team in order to provide follow up assistance to the FG after year one. It is suggested that some AEAs will receive special short term training (Please refer to 4.2.1.10). Training program for interested FGs on food processing and marketing should be initiated immediately in addition to those for agricultural extension, savings and credit groups and post harvest handling. A suitable schedule will also be needed as FG leaders are required to take important decisions such as what to do with the harvest. Corresponding training has to be given well in advance in order to provide FGs with the necessary skills to make informed decisions (Please refer to 4.2.1.7). SO2 Health Training: LAUNCH Health and Nutrition and Education staff should be trained in the importance of ECD and ways to engage mothers in ECD, including how nutrition impacts brain development long-term. LAUNCH should incorporate mother-led ECD strategies into the Care Group sessions, articulating why this is important in a way beneficiaries can understand and adopt. Strategies include: The importance of communication and interaction with infants and children; Ways to interact with children and infants for brain development and growth (for example promoting singing to infants, talking to them, helping children count or match items, and so on); and How nutrition and brain development are interconnected and critical for future achievement and development. SO2: Behavioral Change: Neighborhood mothers attribute changes and improvement in their children’s health to CSB, few linking the progress to their own actions and appear doubtful that they would be able to make changes solely using locally available foods. In the time remaining in the project, LAUNCH Health and Nutrition staff must help mothers to connect their actions with their children’s improved health. They must help the mothers to understand why this age group is critical and to disassociate to some extent with the focus on the CSB but rather on the improved nutrition and one’s own actions. This requires additional education sessions, a consistent message and strategy to be developed by the Health and Nutrition staff and management. This message should be developed immediately, taking into consideration best practices and lessons learned in the program as well as from other programs identified in the IYCN project. Health and nutrition staff should be immediately trained by management to monitor, identify, document and solve problems as they arise. Staff should be held accountable and recognized for problem solving as well as documentation of successes so that the message can continue to be used by other health workers. SO2 Lead Mother Retention/Motivation: In some implementation areas, Lead Mother participation is already waning. LAUNCH should incentivize Lead Mother participation with regular and ongoing recognition, sufficient working tools and other intangible incentives, such as community recognition and personal pride; this is of particular importance as Lead Mothers begin to graduate LAUNCH MTE – Final Report ©Moneval Solutions 2013 Page 9 of 113 from food benefits. Health and nutrition staff should begin to routinely ask what Lead Mothers have accomplished that has made them proud during routine monitoring. This strategy will help Lead Mothers to recognize their achievements, helping to reinvigorate and maintain their participation as Lead Mothers, while allowing health and nutrition staff to monitor and document success stories. In particular, LAUNCH should work with local communities to ensure that the communities are aware of the contribution of Lead Mothers in improving the health of their children. As the government is reworking the role of TBAs, LAUNCH may be able to assist in the identification of a long term role for lead mothers, in supporting the work of the government. Lead Mothers and/or their households should be included as part of the regular program implementation in farmer groups and other activities. Provision of working tools: Lead Mothers have been told by LAUNCH they will receive working tools such as lappas and t-shirts to indicate they work with LAUNCH. Identifying items are both a working tool and an incentive as it helps the Lead Mother’s efforts be recognized by the community. Delivery of these working tools should be expedited and Lead Mothers informed of the expected timing of delivery. Health and nutrition management should ensure that they prioritize this roll-out and hold staff accountable for the timely delivery of such materials. SO2 Care Group Lessons/Improvement: Health and Nutrition staff should immediately begin to educate Lead Mothers as to why CU2 and their nutrition are important. Without proper understanding as to why this age group is important, the cut-off seems arbitrary and unfair and compromises the likelihood of long-term sustainability of the achievements. Health and nutrition management staff should ensure that staff, have a consistent understanding and message on the importance of this age group based on evidence the “First 1,000 Days Approach” and how to plan the intervention exit. Care groups lessons should include how mothers can sustain the children’s physical achievements after they have graduated from food rations. SO2 Exclusive Breastfeeding: FGDs indicate EBF is widely practiced. Mothers report changing behavior and beliefs about EBF. By the end of 2013, LAUNCH should begin to verify, document and analyze in depth the changes in breastfeeding practices, especially the changes in colostrum use. Complementary Feeding: LAUNCH appears to have had significant impact on promoting and normalizing EBF. LAUNCH has taught women the components of appropriate complementary foods in the event that CSB is not available for the 6 month to 2 year olds. However mothers appear not to have the self-efficacy to have a greater impact on their child’s nutrition. LAUNCH should immediately expand its efforts to focus nutrition education and self-efficacy on years 2–5, so that program achievements are sustained and the program goal is realized. Among activities to focus on, LAUNCH health and nutrition, staff must increase household mother (and father) understanding as to why this age group is important. They should also aim to increase their self-efficacy to make positive changes in their child’s nutritional intake and to promote the growth and utilization of additional diverse foods, through activities such as household gardening. Strategies to increase the adoption of this behavior may include:  Identification of barriers and helpers to adopting the behavior, including cultural, monetary and social. Using findings and recommendations from the integrated social behavior change strategy developed in May 2012. Based on these barriers and helpers, the health and nutrition management should initiate a strategy to train, monitor and document health and nutrition staff implementation experiences of the strategy; LAUNCH MTE – Final Report ©Moneval Solutions 2013 Page 10 of 113  Inclusion of pictorial representation of promoted behavior including/emphasizing a session in the Care Group such as the promotion of dietary diversity using locally available foods could be considered;  Cooking demonstrations for proper feeding of 2-5 year olds;  Increase education on the importance of proper feeding for this group and connecting it with future achievement, which appears to have influenced household mothers changes in EBF and complementary feeding; and  LAUNCH health and nutrition management and team leaders should begin a dialogue with the local health authorities and the Ministry of Health to develop common terminology to address the confusion between fortified/vitamin rich foods and medicine. This is critical to both the program as well as the future efforts of the local health authorities to combat under-nutrition and stunting in the areas. Rectifying this misunderstanding is likely to influence the self-efficacy of mothers to change their child’s nutritional status. SO3 Management modalities: Field coordinators should be empowered with full oversight responsibilities and direct implementation of all SO3 activities. The strategic objective should be managed by one manager in coordination with county coordinators. To support this, one manager has to also be placed at central level at LAUNCH headquarters to be referred to as Education Manager for both IRs. A single manager for the SO will enhance coordination of field activities. This will also provide clarity on the activities and expectations of the field coordinators. Additionally, accurate planning and reporting will be enhanced thus supporting timely and effective decision making. SO3 Strengthen collaboration with local and national education authorities: The PTA coordination unit at the MOE needs to be fully involved in the planning of activities with PTAs from the start. With the pending implementation of the Education Reform Act of 2011, collaboration will assist project implementation to be in line with national education goals and objectives. For example, there is a structure for PTA coordination at both national and local levels which can be tapped into to successfully work with PTAs. In addition, the collaboration with local education authorities will support the decentralization process that the MOE is currently engaged in. The MOE already has five county staff in various positions in each county performing M&E, education planning, finance personnel and human resource management roles. SO3 School Improvement Plans: To continue the support provided to schools LAUNCH staff should work with communities to produce plans for school improvement with further support on implementing the plan. There are plans by the Ministry of Education with support from the World Bank to provide grants to all schools by 2013. The utilization of such grants will depend on the ability of school authorities and communities to produce a school improvement plan. LAUNCH could begin the process of training communities to develop school improvement plans to help prepare them for successful grant applications. 1.5.2 Recommendations for Cross Cutting Themes Commodity Management: Continued vigilance and improvement in the security of the commodities particularly at the port as well as during transport from the warehouse to the distribution points needs to be emphasized. LAUNCH MTE – Final Report ©Moneval Solutions 2013 Page 11 of 113 Commodity Management: Relying on better coordination of imports with other FFP and/or Title II programs funded by other donors is unlikely to resolve monetization problem of commodity sale. It is therefore suggested that USAID permit ACDI/VOCA to look at the possibility of third country monetization to meet operational costs. Disaster Risk Reduction (DRR): As the District Management Committees (formerly DRR Committees) have proved to be neither relevant nor effective with respect to food availability and access. Consequently it is suggested not to support these committees under the SO1 component anymore and given LAUNCH is not an institutional strengthening program this really falls to other development initiatives. Integration to Deliver as one Program: There is a real need for greater integration of all of LAUNCH activities across the three SOs. This will ensure maximum use of resources with greater impact on communities. Beginning with joint assessment involving all SOs, interventions can be planned to enable a single vehicle to carry a team comprising of all SOs for various intervention in a community. Currently communities for SO3 interventions are selected based on the existence of other SOs while Extension workers are carrying messages for all SOs in communities they work. This good practice that can be used as opportunity for integration of strategic objective. Discussants even intimated that graduates of Care Givers should move from receiving food to join farmer groups. Overall, field staff planning together taking into consideration available resources is the beginning of integration. LAUNCH MTE – Final Report ©Moneval Solutions 2013 Page 12 of 113 2. INTRODUCTION Subsequent to the end of the civil war in 2003 that left Liberia devastated with a barely functioning economy, infrastructure and an impoverished food insecure population, the Government has undertaken a number of development initiatives to address the situation. Despite this, food insecurity is still prevalent, with high rates of stunting (39%) and underweight (19%) among children. The challenge has been how to reduce food insecurity without undermining market oriented development efforts or creating dependencies among beneficiary populations. In 2010, USAID awarded ACDI/VOCA a five-year USD 40 million Title II Multi Year Assistance Program in Liberia. The program aims to improve food security of vulnerable rural people in Bong (Salala and Sanoyea districts) and Nimba (Gbor, Wee Gbey Mahn, Zoe Gbao and Yarpea Mahn districts). LAUNCH has implemented an integrated approach to the three Strategic Objectives targeting 10,800 farmers in agriculture, 10,281 pregnant and lactating women (PLW), 16,770 children 6-23 months, as well as 98,094 family members. In addition, the program aims to reach as many as 13,800 school children through institutional assistance to local schools and training in rural entrepreneurship to out-of-school youth. This allows targeted communities to benefit from sustainable livelihoods interventions as well as health and nutrition and education services. ACDI VOCA is the lead organization and is responsible for SO1 ‘Increased Availability of and Access to Food of Vulnerable Rural Populations’. SO2 ‘Reduced Chronic Malnutrition of Vulnerable Women and Children’ and SO 3 ‘Increased Access to Education Opportunities’ are being implemented by Concern International, John Snow International Incorporated (JSI) and Making Cents International (MCI). Results Framework: LAUNCH has adopted a three pronged strategy which provides a logical flow of how activities, interventions and outputs will contribute to the anticipated outcomes and desired impacts within USAID’s overall country strategy for Liberia. The framework is as a result of discussions with USAID/Food for Peace (FFP) and the Food and Nutrition Technical Assistance-2 project (FANTA-2) following approval of ACDI/VOCA’s initial proposal. LAUNCH is applying an integrated approach to food security by targeting interventions across three Strategic Objectives (SO) and varying Intermediate Results (IRs):  SO1 aims at increased availability and access to food for vulnerable rural populations. Interventions are anticipated to result in: o IR 1.1 – Improved smallholder farm management, production, and post￾harvest handling practices; o IR 1.2 – Increased market linkages; and o IR 1.3 – Improved feeder road infrastructure6. 6 Activities aiming at IR1.3 are being dealt with in LAUNCH reports under IR1.2 (market linkages). There have not been any specific IR1.3 performance indicators defined. LAUNCH MTE – Final Report ©Moneval Solutions 2013 Page 13 of 113  SO2 aims at reducing chronic malnutrition of vulnerable women and children. Using the PM2A approach, the interventions are anticipated to result in: o IR 2.1 – Strengthened community capacity to engage in health, nutrition and hygiene interventions; o IR 2.2 – Reduced malnutrition in children under two and improved maternal nutrition; o IR 2.3 – Improved prevention and treatment of maternal and child illness; and o IR 2.4 – Improved water, hygiene and sanitation access and practices.  SO3 aims at increased access to education opportunities for the rural poor. The interventions are anticipated to result in: o IR 3.1 – Increased community capacity to support education infrastructure and systems; and o IR 3.2 – Increased access to livelihoods-based education. LAUNCH also includes four cross cutting themes of Gender, Monitoring and Evaluation, DRR and early warning System and Sustainability across the three SOs. The program is implemented by ACDI/VOCA in partnership with Project Concern International (PCI), John Snow International Incorporated (JSI) and Making Cents International (MCI). ACDI/VOCA’s specific responsibility is for implementing SO1 through an adoption of a value chain approach and commodity management. PCI’s specific responsibility is for implementing the health and nutrition components as well as the education component with support from JSI under SO2 MCI is developing innovative curricula in entrepreneurship for youth and women also under SO3. LAUNCH MTE – Final Report ©Moneval Solutions 2013 Page 14 of 113 3. MTE METHODOLOGY The methodology adopted for the review was pre-determined in the SOW and used three complementary methods: (i) analysis of available documentation; (ii) interviews and discussions with stakeholders and LAUNCH participants, and (iii) field visits and discussions with direct beneficiaries in areas where the project activities are implemented. The MTE concentrated on the validation and clarification on the ‘whys’ of the quantitative results from the Annual Results Survey (ARS) 2012. The evaluation approach focused on discussions with beneficiary groups and key informant interviews (KIIs) with participants, beneficiaries and agencies to ensue coverage across all the Strategic Objective interventions. Documents Analysis: The analysis of documents included the Multi Year Assistance Program technical proposal, quarterly and Annual Project Reports (APR), the Annual Results Review (ARR) and the Annual Resource Requests (ARTs) (all part of the Pipeline and Resource Estimate Proposals (PREPs)) for the program. It also included the Performance Monitoring Plan (PMP), Results Framework (RF), and the Indicator Performance Tracking Table (IPTT), minutes of technical meetings, reports on project activities, relevant national policy documents, trip reports and other relevant documents. Stakeholder Discussions: Discussions with stakeholders included ACDI/VOCA staff, staff of Partner NGOs (PCI, JSI and MCI) and technical staff at the field level as well as government staff. Site Visits and Focus Group Discussions: LAUNCH staff determined the focus group participant selection and site locations as agreed in the SOW. The focus group participants provided ‘grass roots’ validation of the impact achieved thus far by the LAUNCH program. Site visits included food warehouses, food distribution points, community health centers, schools and PTAs. Focus Group Discussions (FGDs) The MTE conducted the following FGDs:  SO1: (126 men and 127 women) with 18 groups (including 2 rice FGs, 2 vegetable FGs, 6 cassava FGs, 3 FGs with other crops and 6 Saving Groups) and 3 DRCs;  SO2: 13 FGDs with 136 lead mothers and household mothers (lead mothers were selected by program staff, household mothers were selected by lead mothers); and  SO3: 7 PTAs and 4 youth groups. with total of 147 discussants. Site visits:  SO1: 6 block farms and 6 individual farmers’ plots;  SO2: 2 Home visits, 4 Clinic visits, 4 WASH projects and 1 food distribution point; and  SO3: Visits to 6 schools. Register review (SO2):  Lead Mother registers; and LAUNCH MTE – Final Report ©Moneval Solutions 2013 Page 15 of 113  Child health books and Pregnant Women Books, where available. Key Informant Interviews (KIs) including Government officials  SO1: 9 KIIs with Government Agriculture officials (District Agricultural Officers (DAO) and Country Agricultural Coordinators(CAC)), also with regional LAUNCH staff;  SO2:18 KIIs with Local Government Health Officials, GCHVs, Water Committees and county LAUNCH field staff;  SO3: 4 DEOs, 2 CEOs; and  ACDI/VOCA, PCI, JSI and Making Cents. 3.1 Evaluation Questions The evaluation questions were detailed in the SOW as determined by the program and formed the core of the questions to be discussed during the FGDs. MTE used these questions as the basis to create FGD discussion guides to be used with community participants. The FGD discussion guides were reviewed and commented upon by program staff in the field and at LAUNCH headquarters. The final set of evaluation questions and topics was reviewed and approved by the LAUNCH team at HQ and in Monrovia prior to the MTE team leaving for the field. 3.2 Limitations and Restrictions There were a number of limitations and restrictions encountered during the MTE. These should be noted when interpreting the finding and recommendations of the MTE report. The key limitations and restrictions are summarized below. Quantitative Survey: the SOW did not include a quantitative study of the LAUNCH program. The MTE was a qualitative evaluation with the focus being on the validation and clarification on the ‘why's’ of the quantitative results from the ARS 2012. Selection of beneficiaries, informants and locations: The methodology was based on interviews at locations of pre selected beneficiaries through FGDs and key informants. The MTE team was not involved in the process and did not have any input into the selection criteria of the participants or of their location except that it covered all activities and districts. This was due to the fact that, it would have been impractical for the MTE team to undertake the selection within the time available for the MTE. Consequently, there is the possibility of a bias and a limitation in the results. However, based on the FGDs conducted and interactions with beneficiaries the MTE team has concluded that the selection of participants and locations was made objectively. There was no evidence or indication of a bias towards a particular type of beneficiary or location. The team attempted to mitigate the potential bias to some degree by informing the participants that the MTE team was not part of the LAUNCH project and that it was important for the success of the MTE and of the LAUNCH program that the participants were open and objective in their discussions. The MTE team is confident that evaluation findings and recommendations would have been similar even in the case of a more refined sampling and data collection procedure. LAUNCH MTE – Final Report ©Moneval Solutions 2013 Page 16 of 113 New entrants: The evaluation in its methodology aimed at excluding new entrants in the selection of beneficiaries. This was based on the argument that program implementation is dependent on the extent of time that beneficiaries have been exposed to the program. It was important to establish a cut-off point so that a level playing field among the participants would allow an explicit substantive assessment of the program and its components. The MTE team felt there would be value in terms of useful findings and recommendations by focusing on participants who have been exposed to the project for a reasonable period. In point of fact, more than ten of the 18 groups under SO1 visited, were relatively freshly established and had not finished a whole agricultural cycle at the time of the field visit. Only two groups were able to describe tangible effects due to the project (one rice group and one vegetable group). The rest of the groups considered themselves in one or another “waiting state”, for example: “waiting for seed”, “waiting for harvesting” or “waiting for selling” etc. and unable to specify the incremental benefits of the project intervention. There was also a mixture of “new” and “old” participants included in the SO2 and SO3 groups. The amalgam of participants of varying periods of exposure to program activities therefore is a limitation in the evaluation. Rural development and benefit change communication (BCC) measures, implicit in agriculture and nutrition interventions, will have different adoption rates. It would have been better to exclude the impact of time by comparing participants who have been exposed to the programme for a reasonable period. Commodity Management: The SOW did not allow for an in depth assessment of commodity management and should be taken as a limitation of the evaluation. The conclusions reflected in this report however are those resulting from a “cursory assessment of the Commodity Management component t within the timeframe and data made available. Control group: The SOW did not allow the MTE team to visits to reference groups or project sites, which might have added further insight for the evaluation. Translators: Although the MTE team included a local expert for SO3 the group interviews and discussions were conducted via a translator, which were subject to interpretation by the translator. Given the variety of experience in qualitative research and translation abilities, this should be considered a limitation. Timeframe: the timeframe for the MTE was challenging with 13 days of field work to cover 48 locations with 42 discussion groups consisting of 536 participants, 37 interviews and 29 sites. The MTE plan included 14 days for report writing. LAUNCH MTE – Final Report ©Moneval Solutions 2013 Page 17 of 113 4. SO1: EVALUATION FINDINGS 4.1 Design and Planning 4.1.1 Integration of Project Objectives The project objectives are aimed at inclusive agriculture growth, with:  Improved agricultural productivity;  Expanded markets and trade;  Increased investment in agriculture and nutrition related activities; and  Increased employment opportunities in targeted value chains. Considering the LAUNCH SO1 envisaged results, it can be concluded that the program is fully embedded into the “Feed the Future” initiative. The Government of Liberia (GoL) priorities in the agricultural sector also emphasize food security issues as it has been established within the Food and Agriculture Policy Strategy of 2008. This includes the following goal regarding Improved Food Security and Nutrition: “Safe and nutritious foods are available in sufficient quantity and quality at all times to satisfy the nutrition needs for optimal health of all Liberians, throughout their life cycles”. The established objective still forms part of agricultural policy according to the Vice Minister for Planning and Development. 4.1.2 LAUNCH Strategies and Activities EQ: Evaluation Question (EQ): Are LAUNCH strategies and activities appropriate and relevant to the achievement of the project’s intended results? LAUNCH results with respect to SO1 have been stipulated as follows:  IR 1.1 – Improved smallholder farm management, production, and post-harvest handling practices;  IR 1.2 – Increased market linkages; and  IR 1.3 – Improved feeder road infrastructure. The importance of agricultural infrastructure has also been considered under Support Services, Markets and Trade Policies and Strategies as “only about 4.6 % of Liberian villages have access to functional markets“. This demonstrates the relevance and appropriateness of the strategy of SO1 for the development of the food sector in Liberia. It can be concluded that the achievement of the results IR1.1-1.3 is considered appropriate and relevant in order to reach SO1 aims. Comparing LAUNCH activities with its intended results, it can be noted that the value chains of cassava, upland rice and vegetables had already been identified as the most relevant ones for the development of food production in Liberia (FAO’s Comprehensive Assessment of the Agricultural LAUNCH MTE – Final Report ©Moneval Solutions 2013 Page 18 of 113 Sector 2007). LAUNCH´s strategic document for project implementation, which was elaborated in April 2012, has given priority to the following training topics:  Lowland rice production;  Cassava production;  Vegetable production;  Farming as a Business; and  Post Harvest handling. This clearly indicates that LAUNCH’s strategy to concentrate on rice, cassava and vegetables is completely in line with the priorities set by GoL. It is considered appropriate to the market conditions in Liberia as well as considering its importance as part of the consumption patterns of the rural community. There has not been, however, an ex-ante assessment of the economic benefits of such promotional activities which would provide a rough estimate of the expected farm income increase to both the extension services and the farmer. Such information would be useful in order to decide whether the introduction of improved farm management techniques would in fact have any significant income effect at village or county level. FG members´ initiatives to look for increased family income are widespread. It has been observed that roughly half of the farm households participating in the FGD also try to increase their income outside agriculture, with tree crops (rubber and palm oil), charcoal, or other cash crops (pineapple, watermelon). These observations are in line with the preliminary findings of the Comprehensive Food and Nutrition Survey (CFNFS 2012) which concludes that food, tree and cash crop production are the most important livelihood activities of the rural population. Regarding the relevance of the SO1 Strategy Document (April 2012) and in particular the chapter on “Farming as a Business”, it has to be noted that the vast majority of farmers who participated in the interviews felt quite uncomfortable calling themselves “professional farmers” or business oriented persons. The majority preferred to be described as “traditional farmers”. There was a general feeling between the participants that there is a need to change behavior and farming systems quite drastically in order to improve their livelihood situation. The inclusion of a higher share of professionally oriented farmers in the FG with market linkages in Bong and Nimba, and a prospect to become commercial would certainly have a positive effect on IR1.1 and 1.2 and improve the relevance of the LAUNCH program. In addition to the problem of quantifying the effects of improved crop management practices there is also little ex-ante knowledge regarding the average benefit of a farm run as a business or of the potential benefits of improved post harvest management. Observations in the villages have shown that crop spoilage due to improper storing of crops is not considered a major source of losses by farmers. Regarding IR 1.3 “Improved feeder road infrastructure,” it is widely accepted that in Africa, rural road construction has lead to significant increases in agricultural production. This is especially true for non-food export crops, expanded use of agricultural credit, improved land productivity, proliferation of small shops and expansion of rural markets (Anderson et al., 1982). The consequence of poor roads and few transport services is that transport costs are high in targeted villages, particularly during the rainy season and especially on poor-quality roads. In spite of the LAUNCH MTE – Final Report ©Moneval Solutions 2013 Page 19 of 113 importance of improved road infrastructure the LAUNCH program design has given priority to IR1.1 and 1.2 and no strategy has been elaborated by the program with respect to IR1.3 as feeder road construction lies in the hands of the Liberian Government. The government is not particularly supportive of NGOs building roads connecting “their villages”. Consequently LAUNCH activities have been restricted to support GoL initiatives. Concentration on those activities may be appropriate but cannot be considered relevant to the achievement of IR1.3. 4.1.2.1 Target Groups Vulnerability EQ: Is LAUNCH strategy for SO1 aiming to reach the most vulnerable in target geographic areas? Launch is aiming at the most vulnerable in target areas: Farmers in rural villages of the project area show relatively homogenous characteristics. None of the targeted communities visited had electricity and not all villages had a pump well. Only a few latrines were installed and housing standards are modest. The majority of the huts still have straw thatched roofs. There was no indication that village dwellers in the targeted communities off the main road own trucks or even personal vehicles. Road access in most of the villages is very difficult and almost impossible during the rainy season. No regular public transport exists. Variation in income of the rural households does not seem to be very high within the targeted families. There are some differences due to volume of agricultural production sales or deriving small extra income made by off farm activities. According to recent census data the average household size in Nimba and Bong is six persons which is slightly above national average. It is noted, however, that it is common for two or more households occupy one house. According to the information provided by village chiefs, and extension staff, there are often 10-20 people living in one house, which indicates that the poverty levels in the project region are high. According to the 2012 census a typical household spends more than two-thirds of its income on the provision of food for the family, leaving little for basic investments, education, health care and leisure. FGDs held by the MTE confirmed census data as almost all beneficiaries were highly dependent on their own food production for their livelihood and thus vulnerable to any risks Photo: Daphyne Williams, 2013 A food distribution point showing delivery of food allowances form the area. LAUNCH MTE – Final Report ©Moneval Solutions 2013 Page 20 of 113 affecting their food crops. There are no practical indicators to distinguish between small-scale farming and subsistence-based farming, with little surplus available for sale in either case. It is recognized that the criterion of “vulnerability” has been transformed by LAUNCH into an easily applicable and practical catalogue of indicators and questions. However, at the time of identification little information is collected addressing the potential of the specific community and FG to adopt “LAUNCH” initiated practices. A short term rural sociologist should be contracted to investigate this further. It should be noted that women in rural areas are not necessarily the most vulnerable persons. Liberia is one of few countries in which the poverty rate of female-headed households is lower than the rate of male-headed households7. Agro-ecological and Socio-Cultural environment – Most of the land in Bong and Nimba is cultivated under customary occupation, which is a prevalence of traditional shifting cultivation and which does not favor high investment for land clearing and/or mechanization. The use of agricultural inputs is almost non-existent. Tradition in Libera is that farmers have to look for new farm plots every 4-5 years, sometimes as far as two hours walking distance from their homes. A widely acknowledged fact which can also be confirmed for Bong and Nimba counties is the fact that traditional farming systems have minimal negative effects such as land degradation and loss of biodiversity. It can be concluded that minor improvements within the traditional system - as recommended by LAUNCH - would not change the positive balance as long as soil erosion, resulting from poor land use practices can be avoided and the fallow period not becoming too short. Most of the shortcomings observed in agricultural production and most of the problems of food availability, however, stem from the types of land holding in Liberia (see box 1). While there are differences in tenure security among the different types of holding, all farmers suffer poor tenure security. Addressing the land holding issue would have improved LAUNCHs’ prospects for long term impact on food production but it is also noted that land ownership problems are not only affecting food availability but the entire issue of rural development. Box 1: COMPREHENSIVE ASSESSMENT OF THE AGRICULTURE SECTOR IN LIBERIA (CAAS￾Lib), Ministry of Agriculture 2007: In the smallholder sector there are five broad types of land holding, with different levels of tenure security:  Deed holders (or holders of other documents) with a comparatively high degree of tenure security;  Customary occupation without a deed, which results in relative security within the customary domain; 7 According to UNDP (2001), the proportion below the poverty line was 79% for male-headed households compared with 68% for female-headed households (respective figures for severe poverty were 55% and 42%). LAUNCH MTE – Final Report ©Moneval Solutions 2013 Page 21 of 113  Rental or leasing of land with lower security;  ‘Strangers’ or ‘borrowers’ of land who are not from a local area and do not rent, but who are allowed very temporary and insecure access to land, and must supply a token amount of crop produce to the owner to acknowledge that the land is owned by another – in essence acknowledging that the land is being loaned; and  Squatters, who, although they can be evicted at any time if they are discovered by the owner, are also the most aggressive about attempting to claim land by planting tree crops and through forms of adverse possession. Cropping Patterns According to discussions with farmers there is usually one “farm” per family or household under cultivation. If conditions and access permits farmers can also run a lowland farm, mainly for rice production. In Liberia, it is estimated that women contribute 36% of the total labor in rice and cassava production and men contribute 64%8. Men provide most of the labor for clearing and preparing the land, while women do most of the weeding and harvesting of the crop. Little data exists regarding the labor requirements for land clearance, cultivation and crop harvesting. In order to select the most suitable crops and production techniques and to enhance productivity, comparative figures for labor inputs per ton of production would be beneficial. LAUNCH’s strategy is to focus on the main food crops and their value chains. Data drawn from the CFSNS (2006) shows that:  For Rice: 63% of households are fully reliant on upland rice techniques, while 17% opted for swampland and 21% used a mixture of both. Liberian upland rice farm sizes averages 1.1 ha, with rice yields between 0.5 and 1.1 mt/ha.  For cassava, the second most important food crop the annual production estimated is 250,000 tons9. Its advantages are that it can be planted all year round, the time of harvest is not critical, and it can be stored in the ground. It is therefore very important for food contingency, especially before the rice harvest. It is often planted as a follow-on crop after upland rice is harvested. In addition, cassava leaves are an important vegetable, although harvesting of leaves affects tuber yield (this effect is reduced in the rainy season). The crop area is around 0.5 ha, and yields are estimated to be between 6 and 10 mt/ha on upland farms. Cassava is grown on the flat and is usually intercropped with maize and often with sweet potato and pepper. Tubers tend to be small and may be broken when harvested, which reduces shelf life. 8 Comprehensive Assessment of The Agriculture Sector In Liberia (CAAS-Lib), Ministry of Agriculture 2007 Volume 2.2 - Sub-Sector Reports Liberia 2007 9 From the latest CNFS. LAUNCH MTE – Final Report ©Moneval Solutions 2013 Page 22 of 113  Vegetables, the third major crop includes pepper and bitter balls (garden eggplants), as well as groundnuts which all have a ready local market. It was a good decision by LAUNCH to concentrate on these three major food crops in order to address the maximum number of beneficiaries and to maximize impact. Promoted activities by LAUNCH have certainly considered prevailing agricultural practices based on a certain crop rotation typical for the upland. The cultivation techniques are low intensity and do not cause a hazard to the environment. The schemes of LAUNCH however aim to increase the yield per hectare. The production on a given plot and the spacing of plants is changed to allow more plants to grow. LAUNCH could carry out an analysis in order to compare labor requirements for both the traditional and the improved techniques including land clearing and time required for weeding. This would give an a priori estimate for the project benefit per block farm or even FG member. 4.1.2.2 Linkages and Relationships with Government EQ: Are linkages and relationships being established and strengthened between the project and the Government of Liberia in terms of food security? Based on the MTE it can be concluded that the linkages and relationships between the project and the Government of Liberia are being established in terms of food security. This is supported by the evidence highlighted below. Linkages with the GoL have been established both at national and regional level. There are monthly coordination meetings with the Country Agricultural Officers and District Officers both in Nimba and Bong with all major projects and NGOs operating the Counties. Local coordination, especially with respect to possible overlapping of program activities is dealt with at County level. Intra-sectoral issues are brought forward to the County Development Steering Committee (CDSC) to take decisions. A second coordination level exists in Monrovia involving the MoA and all donors in the agricultural working group. LAUNCH management reported activities being carried out jointly by LAUNCH HQ and Government staff as public funding for initiatives at the regional and district level was negligible. At county level, authorities expect better reporting in order to strengthen the linkages, providing the type of support and benefits provided to the communities as a result of the program intervention. However, the protocol is for the Ministries to request data directly from USAID rather than from LAUNCH. 4.1.2.3 Selection Criteria EQ: Are the criteria for beneficiary selection appropriate? Targeted FGs in Nimba and Bong show many similarities and their members seem to represent the vast majority of the rural population. There were some visible differences (including improved roofs, better clothing etc.) between those villages and communities living close to the Monrovia-Gbarnga road and those living off the main road, such as the FG close to Piata situated almost in Lofa County. Community and FG selection has been carried out on the basis of regional pre-selection made by the GoL, which is also assessing the degree of vulnerability of the population. The subsequent assessment by LAUNCH also involved an inventory of various GoL and NGO-run programs LAUNCH MTE – Final Report ©Moneval Solutions 2013 Page 23 of 113 operating in the areas and their sources of funding to ensure better coordination and greater synergies. LAUNCH also considers gender aspects in their selection criteria. LAUNCH has aimed to find a balance between communities with satisfactory access via feeder roads and those villages without reasonable options to market their products. The LAUNCH criteria for beneficiary selection cover a variety of socio-economic and farm related aspects which are generally considered relevant and sufficient to identify future members of the FG. The adoption of a range of selection criteria is appropriate as it assures a certain degree of homogeneity within the groups and allows appropriate targeting. There is always a risk however, that potential beneficiaries will not be considered if more comprehensive selection criteria is applied. Moreover, more human resources from the project would be required to verify the eligibility of potential beneficiaries. It should be kept in mind that FG membership figures have a tendency to decrease during the process as participants look for other opportunities. No systematic assessment has been made regarding the reasons for not continuing with the FG which could possibly indicate weaknesses in the LAUNCH approach and thereby identifying opportunities for improvement. There has not been a mechanism set in place to replenish FG with new members. There seem to be several villages in Bong and Nimba without program support for the time being. It is well understood that other development projects are also active in both counties. The risk of having two or more projects operating in one community is not considered to be a major threat to effectiveness as it is always possible that extensionists of different projects coordinate their visits at village level. Exclusivity of regions to one donor or project does not seem to be an appropriate strategy as long as deficiencies in the communities are widespread and that the support provided does not include major investments but mainly consists of extension and training. The program has managed to involve a good number of female participants in the FGs10 including in leading positions which reflects the actual involvement of women in the cultivation of crops. 4.2 Implementation 4.2.1 Effectiveness 4.2.1.1 Progress Towards Strategic Objective EQ: As defined and measured, do the performance indicators provide useful and reliable data on program progress? EQ: Has the program made progress in achieving its strategic objective? SO1 indicators relate to increased availability of and access to food of vulnerable rural populations and are defined in the Indicator Performance Tracking Table (IPTT). The MTE comments on the usefulness of each indicator are included in the table below: 10 1533 females out of 3231 organized farmers (FY 2012 Annual Report). LAUNCH MTE – Final Report ©Moneval Solutions 2013 Page 24 of 113 Table 1: LAUNCH Performance Tracking Table (IPTT) Strategic Objective 1: Increased Availability of and Access to Food of Vulnerable Rural Populations Code Indicator Comments FFP IM.1 Average number of months of adequate household food provisioning (FFP Impact Indicator #1) Impact indicator serves as Decision making tool for donor agency FFP IM.2 Average household dietary diversity score (FFP Impact Indicator #2) Impact indicator serves as Decision Making tool for donor agency FFP MN.2 # of beneficiaries reached, by sector (FFP Monitoring Indicator #2, Output) Does provide data on participants in training events but does not give an information on active FG membership after having been trained FFP MN.5 # of assisted communities with disaster early warning and response systems, in place as a result of project assistance (FFP Monitoring Indicator #5, Output) Corresponding result not very meaningful as it does not have a major effect on SO1. Corresponding indicator tries to measure the early warning effect which can only be properly assessed once an early warning situation will occur. The existence of a DRR system does not mean that it is effective. MIS EG.1 # of rural households benefiting directly from USG interventions (agriculture) (Mission Output Indicator, Economic Growth) Benefit not very clearly defined, even more difficult to distinguish between LAUNCH benefits and those originating from private initiatives. MIS EG.2 # of person-days of employment created within targeted value-chains (Mission Outcome Indicator, Economic Growth) Employment would be indirect effect; most likely numbers will be very insignificant. It unlikely this will be really achieved given the current activities. AV OC 1.1 Change in rural assets Highly complicated to assess, changes in rural assets may be an effect of other interventions or private initiatives as well. Some direct measurement of income may be needed. LAUNCH MTE – Final Report ©Moneval Solutions 2013 Page 25 of 113 Indicators 6 and 7 above show weakness, as there is a risk that effects originating from other interventions might be counted as LAUNCH effects. More detailed data on indicators 3 to 5 may be relatively easy to collect by using extension service staff to create monthly reports. Indicator 6 seems to be very ambitious and extremely complicated to assess, especially with respect to showing the project’s contribution. Indicator 7 was based on a representative and comprehensive ex ante assessment of rural assets in the region. It needs subsequent follow-up using representative samples of beneficiaries and non beneficiaries. The indicators defined above theoretically provide a complete picture regarding the availability of, and access to, food but their verification would require specific surveys with household interviews of LAUNCH beneficiaries and non beneficiaries. Considering the relative low level of intensity of benefits at household level, it is considered highly unlikely that robust and positive performance data will be produced by such surveys. A recent, but still as yet unpublished, “Liberia Comprehensive Food Security and Nutrition Survey” carried out by WFP based on 15,840 households in Liberia (CFSNS) revealed the following food consumption status in Liberia11: Table 2: Food Consumption by Urban and Rural Populations Food Consumption (%) Rural Urban Total Poor 7.9 1.5 4.2 Borderline 47.2 17.8 30.1 Acceptable 44.9 80.8 (in Monrovia 97%) 65.8 The overall situation regarding food security in Bong and Nimba County (FY 2012 Annual Results Report a separately survey by LAUNCH for its target districts) has recently been monitored by LAUNCH staff using the FANTA household hunger scale. The results reveal that severe hunger occurs only in 1% of the surveyed households, whilst 37% show “moderate hunger” symptoms and 62% suffer only little or no hunger. This data is a snapshot of the actual situation and could serve as a baseline for a corresponding survey at the end of the program. Both set of statistics, although based on reasonably sound survey practices, show significant differences which demonstrate the difficulty of properly assessing food insecurity or food consumption status, especially with respect to the percentage of households experiencing hunger or deficient food consumption. Both sets of surveys are however not sufficient for making assessments 11 Food security was measured using household food consumption as a proxy indicator. This was determined by analyzing household dietary diversity and frequency of consumption compared against a Food Security Score standard calibrated to suit the Liberian setting. Survey was carried out in lean season 2012. LAUNCH MTE – Final Report ©Moneval Solutions 2013 Page 26 of 113 on whether the project has already made a significant contribution to SO1 at the regional level. Even at district level project effects on food production and availability are modest for the time being as incremental crop production produced on each of the demonstration plots are only a few tons12. Comments received by participants during FGDs, indicated that the vast majority were able to produce enough food to feed their families and that there were occasional food shortages in the lean season between May and September before harvest. These shortages did not impose major threats, as there was a widespread opinion that regional food substitutes to the diet were available. The FY 2012 Annual Report states that there are 3,231 farmers in 130 FGs who have been reached by LAUNCH13, together with their families, during Implementation Year three. Farmers have learned some improved production techniques; the majority of FG members have harvested a first crop on their demonstration plot and in total, some dozen tons of cassava, rice, or vegetables. Per participant the amounts are small and the production value per household would hardly exceed a two digit USD14 figure15. Most of the groups have less than 30 members and it would be a great surprise if their food availability situation had changed as a consequence of the intervention. It needs to be taken into account that these figures do not reflect the “without project” control group. Most of the farmers reached would have carried out other agricultural activities instead in the traditional manner, not necessarily with less production value. Considering the qualitative component of the SO1 indicators, it has to be noted that the project has opened perspectives and initiated first steps towards more stable livelihood conditions. This has been achieved by introducing the farmers to more systematic work and “opening their eyes” to new opportunities. These qualitative achievements, however, cannot be considered as tangible benefits with a contribution towards SO1. Gender specific data collection does not seem to give much additional insight as it was mentioned that male and female household members more or less encountered the same problems regarding food availability and access. As long as overall progress towards food security is slow there will not be a high need for gender specific performance data. Achievements: Achievement of results and objectives has been linked to the following indicators as shown in table 3 below: 12 According to the FY 2012 Annual Report the production was 1.2 t of bitter balls 2.5 t of water melon, 2.4 t of rice and 4.7 t of vegetables. Moreover, a total of 13.6 t of cassava had been harvested by the FG. Some of the figures include also production on private plots. 13 Recent ACDI/VOCA figures indicate that the numbers have increased to 4,311 farmers in 220 FGs. 14 The best performing FG (12 members) in Gbor sold their demonstration plot vegetable production for US$ 980. All the others had much lower production volumes per capita. 15 It is recognized that not all the data has been collected for the survey. LAUNCH MTE – Final Report ©Moneval Solutions 2013 Page 27 of 113 Table 3: Result oriented indicators (IPTT) for SO1 Code Indicator Comment IR 1.1 Improved Smallholder Production FFP MN.4 % of smallholders using at least 3 recommended sustainable agronomic technologies (FFP Monitoring Indicator #4, Outcome) The indicator is considered to be useful as a way of showing diversification practices, skill enhancement and environmental awareness. Sub-IR 1.1.1: Improved Farm Management Practices Adopted MIS EG.3 % change in yield of specific food crops [rice] (Mission Output Indicator, Economic Growth) Yield per ha does not seem to be relevant in order to measure increases in production and productivity in Nimba and Bong. Local cropping systems seem to be adjusted rather to labor requirements. Data should be collected by comparing plots with improved techniques to those subject to "traditional" practices to measure which is more efficient and effective. MIS EG. 3.1 % change in yield of specific food crops [cassava] (Mission Output Indicator, Economic Growth) Ok. MIS EG.4 # of individuals who have received USG supported short-term Agricultural sector productivity training (Mission Output Indicator, Economic Growth) Valid only if the quality of the "significant knowledge or skills" is sufficient to improve farm management, to be collected by AEA MIS EG.5 # of additional hectares under improved technologies or management practices as a result of USG assistance (Mission Output Indicator, Economic Growth) it is suggested to measure the additional volume in food production, but only the part which is a direct effect of LAUNCH, to be monitored by AEA AV OP 1.1.1.1 # of farmer Training Groups formed Number of farmer groups is crucial to measure “coverage” of the project. AV OP 1.1.1.2 % of women participating in farmer training Ok LAUNCH MTE – Final Report ©Moneval Solutions 2013 Page 28 of 113 groups AV OP 1.1.1.3 # demonstration plots used/established Ok AV OP 1.1.1.4 # of women's groups members trained in small livestock and poultry practices activity has not yet started Sub-IR 1.1.2: Improved Smallholder Access to Agricultural Inputs AV OP 1.1.2.1 % of smallholder households diversifying crops cultivated Ok, but has to be a result of LAUNCH intervention AV OP 1.1.2.2 Volume of seeds and other inputs distributed to women and smallholder farms Ok IR 1.2 : Increased Rural Household Livelihood Opportunities MIS IM.1 # of rural jobs created (Mission Impact Indicator) Not a very meaningful indictor as the number of jobs created as result of LAUNCH intervention will be very small and most likely not reach the 550 targeted. MIS EG.6 Increased sales of selected commodities and products (Mission Output Indicator, Economic Growth) Unlikely to be established within project framework, even with sophisticated statistical methods. Farmers generally do not recall volumes of production, prices and sales after a short time (a few seasons). High risks to obtain biased information as farmers are unlikely to give information which is accurate and therefore likely to be quite wrong. Sub-IR 1.2.1: Improved Market Linkages MIS EG.7 # of people benefitting from USG sponsored transportation infrastructure projects (Mission Outcome Indicator, Economic Growth) No program effects envisaged AV OP 1.2.1.1 % of smallholder households engaged in bulk marketing Corresponding activities have not yet been started on a significant level even if the around 12% have been reported as such this is too small to see significant impact. Individual marketing not LAUNCH MTE – Final Report ©Moneval Solutions 2013 Page 29 of 113 considered Sub-IR 1.2.2: Improved Smallholder Access to Financial Resources The indicator could measure: No of smallholder farmers in Bong and Nimba who received credit (applied for credit), Volume of short and long term credits received by those smallholders. AV OP 1.2.2.1 % smallholder households with access to cash savings and/or credit (through community associations or formal financial institutions) Ok, but Saving Groups are not a valid source for credit for the time being. AV OP 1.2.2.2 # agribusiness grant recipients For the time being there are no cases of FGs which have received an agribusiness credit although there, may well have been other grants issued. AV OP 1.2.2.3 # of agribusinesses trained in enterprise management Most of the individual farmers will not be able to effectively apply “enterprise management” skills into their business operations for the time being. Training should be foreseen to FGs and possibly a few “commercially” oriented farmers evolving from LAUNCH support to the FGs. The set of indicators seems to be appropriate to measure the progress against LAUNCH results. It has to be noted that the complete set of indicators is both ambitious in view of its established targets as well as in view of the effort to verify the degree of achievement. No indicators have been formulated with respect to IR1.3. Performance against indicators has recently been investigated during the LAUNCH Annual Monitoring Survey. Positive performance has been reported on AV OP 1.1.2.1 and AV MTE based findings on KIIs and field visits show that the achievement of IR 1.1 is still lagging behind. This is due to the cultivation practices and diversification measures are not as easily adopted by farmers on their individual farms as anticipated by LAUNCH. Some reasons for this include the following:  Shifting cultivation requires crop rotation until the soil is depleted. This limits the selection of food crops to be cultivated and limits farmer options to introduce new crops or techniques in any given year;  Many farmers wait for the outcome of the first demonstration plot harvest before using the “mounds and ridges” technique on their own farm;  Only a very limited number of individual farmers (mainly female) have successfully started a cash crop farm (pepper, pineapple) on their private land in close cooperation with AEAs; and LAUNCH MTE – Final Report ©Moneval Solutions 2013 Page 30 of 113  Participants seem to be more interested in vegetable FGs than groundnut and cassava FGs. This is most probably because of the higher income derived from it and the potential to feed the family. Two individual vegetable FG farmers started a new vegetable farm in close proximity to their community. One rice FG has created a comparison plot using “traditional” measures to compare its performance with the plot using improved techniques. No systematic assessment of the crop has been made so far. With respect to IR2 (increased livelihood opportunities) the project has made some progress with two vegetable FGs which have sold their products and have started to build up closer links with commercial traders. There may be a chance for some employment and that credit will be made available for investment. Cassava groups and their members have so far not yet achieved any tangible benefit as per indicators MIS EG.7, AV OP1.2.1.1 or AV OP 1.2.2.1. However, the rice groups which had been visited seem to have performed better versus Sub-IR 1.2.1 (AV OP 1.2.1.1). There was a first harvest from block farm plots in line with planned goals. Some of the crop will be provided to the participants and the remainder will be stored with the intention of selling the crop once rice prices become more favorable. LAUNCH contribution towards IR3 was minimal as no road infrastructure has been improved or rehabilitated for the time being. 4.2.1.2 Farmer Productivity EQ: Are the improved technologies and practices promoted by LAUNCH appropriate for improving farmer’s productivity in the project’s targeted areas? The land issue is considered to be the single most important bottleneck towards higher agricultural production and productivity. (Please refer to box 2 below). Box 2: The various types of land holding provide different incentives for undertaking agricultural investments (C. Deininger, 2003). For smallholders the prospects of technology adoption, such as planting of tree crops, and investments such as soil conservation, terraces, or other long-term strategies differ with the different occupancy types noted above. Deed holders face two difficulties in this regard: the issue of multiple transactions over time (including fraud), and the designation of boundaries. In the case of the former, the current surge in cases of land and property dispute in all forms of courts that relate to various problems with deeds means that deed holders who are involved in a dispute, or think that others might in any way have a counterclaim, may be unwilling or less willing to adopt long-term technologies such as tree planting or investments associated with longer term strategies. Based on the observation of the MTE, it seems safe to say that:  Farmers’ productivity regarding cassava will slightly increase as LAUNCH training mainly deals with the improvement of the yields without properly keeping a record of farmers’ additional labor requirements or savings. Yields and overall production is the main concern of the promotional activities although many farmers have to walk more than one hour to LAUNCH MTE – Final Report ©Moneval Solutions 2013 Page 31 of 113 reach their individual farms. They would also most likely want to know production techniques which would reduce overall labor requirements to manage their crops. Cassava groups in communities with poor road access have reported to have great difficulty getting benefits from increased production. Two groups during the MTE insisted that they needed a cassava grinding machine before harvesting their crop;  There are comments that farmers productivity of lowland rice farming seem to significantly increase; and  The highest potential productivity increase can be expected from improved vegetable production but more prerequisites have to be met by FGs regarding suitable plots and qualifications of the participants. It should be noted that for the time being participating farmers only benefitted from some implements 16and from improved seed provided by LAUNCH. So far no fertilizer has been provided. Compost making activities which might have a positive effect on crop yield have apparently not been taken up by the farming community. For the time being only a small fraction of farmers and FGs has been able to increase their productivity as a result of LAUNCH promotional work. A more significant increase in productivity is hampered by a lack of on-farm and post harvest processing machinery availability. It is doubtful that much additional production potential will be tapped by communities without opting for mechanization in cultivation and processing. The reluctance of LAUNCH regarding investments within the FG and on private plots is well understood given the risks that could occur if those benefits are not well maintained and/or misused. Considering the actual cost of extension per farmer or FG and the effects generated as a result of LAUNCH intervention, it seems less risky to allocate a volume of USD 1,000 to USD 2,000 per FG in the project budget to be used as investment to improve mechanization, processing, storage, transport etc. The total amount would be a fraction of the total project budget and would certainly improve productivity of the FG significantly, especially if provided during the lifetime of LAUNCH and not at the program end. There is a lack of understanding on behalf of LAUNCH management regarding the socio-economic conditions. There is also a lack of knowledge regarding agricultural productivity and practices in the region and a lack of own ideas about effective means to improve current productivity and food security problems. 4.2.1.3 Block Farm and Farmer Groups EQ: Is the block farm and farmer group approach appropriate and effective in enhancing adoption of improved farm management practices? LAUNCH has from the beginning focused on FGs as recipients of program training and beneficiaries of support. Furthermore a Block Farm approach has been applied by LAUNCH using demonstration plots for specific crops and their management. The approach is considered appropriate considering the need of the program to reach a large number of farmers and FGs. At least during the first year 16 Mainly cutlasses, shovels, wheelbarrows etc owned by the FG. LAUNCH MTE – Final Report ©Moneval Solutions 2013 Page 32 of 113 there are no alternatives when it comes to building up a FG other than to train members working in groups according to pre-established schedules. This also allows the demonstration of new crop management cultivation techniques. LAUNCH has carried out intensive practical training for specialized FGs (and their crop) on demonstration plots (up to 2 sessions weekly) covering all major steps from site selection to harvesting. The following observations have been made with respect to training on Block Farms:  The vast majority of the participants recall the main elements of the practical training and the “improved production techniques” on the demonstration plots. However, a minority (especially cassava farmers) were critical about the added value of the acquired skills compared to the time they had to invest in doing this and also in participating in the training;  The number of farmers participating in the demonstration plot exercise and practical training is declining with very few farmers receiving training from fellow farmers;  Systematic training of individual farmers through the groups they are a member of apart from the block farm exercises has not been provided by LAUNCH. In most communities individual farmers were given advice by the AEAs regarding the management of food crops. However, there is a lack of knowledge on behalf of the farmers and the AEAs regarding longer term farm planning and food production under shifting cultivation conditions and management of food production risks;  Whilst the reports of AEAs concentrate on activities they do not sufficiently focus on the progress of the individual FG towards SO1 and the performance indicators;  What is generally missing on the Block Farms is the installation of a “traditionally managed” piece of land on the demonstration plot which will not be treated according to the new methods but rather, dealt with in the traditional way. Only one FG reported that they had decided to leave 1 out of the 20 rice training plots of the Block Farm under the traditional rice cultivation technique. It was reported that this would help trainees to compare the differences between the two methods with respect to labor, inputs and yield. The majority of FGs are still at a very initial stage without a clear agenda and a clear program. Most of them indicated that they were not carrying out any activities at the time of the MTE, but were rather in “waiting modus” waiting for the planting season, harvesting season or delivery of inputs. After year one, few follow-up activities have been carried out by LAUNCH on demonstration plots, except for post harvest handling of rice and nursery activities which may partially be taking place. All other activities foreseen in the Strategy Document (FaaB, post harvest losses, saving groups) will be better carried out in the community. The FaaB training material is in the process of being reformulated however. It can be concluded that the block farm and FG approach is an effective method to introduce innovative production techniques and measures. It has to be noted, though, that the proposed innovations have to guarantee immediate benefits after year one. If these cannot be guaranteed there should be a further project support to selected FG members regarding marketing and storing. 4.2.1.4 Farming as a Business (FaaB) EQ: Are the methodology and approach under farming as a business appropriate and effective? LAUNCH MTE – Final Report ©Moneval Solutions 2013 Page 33 of 113 There is a chapter on “Farming as Business” as a training topic within the Strategy Document for So1. FaaB covers aspects of management, business plans and record keeping. No specific methodology and no comprehensive approach for FaaB in Bong and Nimba has been devised for the time being. There are differences regarding the marketing of the food crops: On average, in Nimba 17% of their production of rice was sold, this was less in Bong. Selling of vegetables dominates in Nimba (72%). At the national level cassava was mainly consumed by the producer (57%), although households were more likely to market cassava than rice (35% versus 7%17. Recently LAUNCH has started focusing more closely on the marketing aspects which is considered indispensable in order to respond to the needs of the farmers and FGs. An overall methodology how to deal with marketing has not been elaborated by LAUNCH. It has been foreseen, however, to provide the AEAs with more training on business and marketing issues. The FaaB training does not sufficiently consider the framework conditions and the specific needs of the FG members. It is well understood, by LAUNCH that the AEAs and subject matter specialists are learning from their experience. They are focused on providing service to interested individual farmers in the communities, thus constantly improving the present FaaB approach. The approach is appropriate but needs to be detailed better. Currently it is not as effectiveness as it could be and whilst the principle (i.e. the approach) remains sound, implementation is weak. Identification of training candidates: Farming as a Business has been considered a subsequent training subject after the FG has successfully completed year 1 after activities on the demonstration plot. The Strategy Paper has identified all farming members of a FG as potential trainees. Training comprises planning, budgeting, work plan, cash flow and record keeping which seems to be too ambitious considering the trainees who, one year before, were selected as most vulnerable and who on the whole do not meet basic requirements to be business people. It needs to be decided whether FaaB extension and support activities should be directed towards the management of the FG, to all of its members, or even towards individual farmers of the community. Training content: The incentives provided by the program have a value of less than USD 30 per participating FG member. This shows that the initial motivation of the farmers is high to learn and improve their livelihood situation. The training provided by LAUNCH does not seem to be designed according to the needs of the FG and the farmers as individuals. Most of the members expressed that they would rather receive more practical advice regarding medium term planning of their own individual farm and regarding marketing (such as volumes, prices and demand). Training of farmers is not systematically oriented towards agricultural processing or marketing of crops (cassava and vegetables). There is too little training given to the participants with respect to management of agricultural or processing equipment after year one. It is not clear whether 17 Program Statistics. LAUNCH MTE – Final Report ©Moneval Solutions 2013 Page 34 of 113 processing forms part of “Farming as a Business” training or if it should be covered by post harvest management. The content of the FaaB concept as documented in the Strategy Document (April 2012) seems to be ambitious. This is based on the fact that most of the members of the FGs are illiterate and hardly in a position to present viable alternative projects for generating income. Savings Groups: The LAUNCH training concept for credit and savings groups is not well embedded within the FG training as part of “Farming as a Business”. It does not state when to start the training or how it should be linked with other training delivered to each FG. Regarding the Credit and Saving Groups (SG) the following specific observations were made:  Some smaller groups seem to depend entirely on member contributions (usually USD 1-2 per month) without affiliation to a FG. Typically, total savings are less than USD 200. Assets may be sufficient for small trade credits, but will not generate sufficient funds for investment projects during the lifetime of LAUNCH;  Some SGs consisted of both FG and non-FG members which will most likely create future conflicts of interest when FGs deposit their farm income with the SG as the group may favor agricultural lending over non-agricultural lending, for example;  SGs which only consist of FG members have better prospects of becoming sustainable than those described under points 1 and 2 above, especially if they are affiliated with a vegetable FG. This is illustrated by one SG which received a deposit of more that USD 1,200 from vegetables sales and which could effectively be used as working capital or for investment purposes for the FG;  All groups presented their books and were able to give actual figures about the total savings and number of credits they maintained;  In only a few examples was credit was provided (mainly at interest of 20-25% per month) to women to finance trade operations. This covered only a fraction of the total savings available; and  None of the groups received training regarding the distribution and management of saving books. Investment: LAUNCH program management has, in most cases, successfully resisted FG demands to create more incentives and investments. After the first year of LAUNCH assistance the options for investments with each FG should be discussed. Typical investments would be for motor tillers (less than USD 400 investment) or processing equipment (USD 1000-2000) or storage facilities. Chickens: LAUNCH is preparing to support chicken raising ventures within their extension strategy and is a viable option for those FGs which do not meet the prerequisite for increased cassava, rice or vegetable production. No activities in the field have yet been initiated for the time being pending approval from USAID. Conclusion: A comprehensive strategy for FaaB has not been elaborated by LAUNCH. For the time being only very general FaaB training is provided which does not sufficiently consider the LAUNCH MTE – Final Report ©Moneval Solutions 2013 Page 35 of 113 framework conditions and the specific needs of the FG members (for example, processing). Effectiveness is hampered by the shortcomings of the trainers, and the lack of supporting investment. 4.2.1.5 Post Harvest Management EQ: Are the methodology and approach under post-harvest handling techniques appropriate and effective? Post harvest loss is a major problem for cassava given it is extremely perishable and must be processed quickly after harvest. It has been noted that the majority of cassava FGs are reluctant to harvest their crop as they have not been able to find buyers for their demonstration plot harvest. They state that either the program should find a buyer to pay a reasonable price or facilitate a cassava grinding machine in order to process the crop and thus counter it perishability. The program has not foreseen to the need to address cassava grinding and processing as part of the post harvest management and the corresponding extension module. Post harvest management extension occupies a relatively small area of extension work (except for cassava) as few problems have been reported regarding post harvest losses in rice and vegetables. Extension work is mainly concentrating on rice storage, for which “rice kitchens” have been established and seem to function very well. 4.2.1.6 Technical Field Staff EQ: Is the technical field staff adequately trained and supervised? The LAUNCH extension strategy is implemented by the extension team of some 30 AEAs plus 8 subject matter specialists (covering livelihood, FaaB, post harvest management and coordination) for the 130 farmer groups (with 3,231 members) in the 6 districts of Bong and Nimba County18. This reflects an average ratio of 85 farmers to one extensionist, and which is comparable to projects at grass root level dealing with individual smallholders in regions with a low population density. The major work of the extensionists is focused on activities carried out on Block Farms. Some of the FGs have received twice weekly visits of the AEA which seems to be too much in view of the range of measures which the AEA is expected to deliver. Extension staff have good agricultural backgrounds and were able to answer questions regarding production issues related to the three focus crops. Farmers confirmed that there were hardly any questions asked by them which the extensionists or the LAUNCH specialist could not answer. FG members who were interviewed during the MTE did not bring forward any complaint regarding the qualification of the AEAs. There were, however, complaints regarding, the provision of inputs or missing abilities of the specialist in identifying buyers for the FG crop. Planning of AEA visits is mainly left to the staff. There seems to be little supervision and control of their performance and little regulation regarding the coordination between AEA and the “FaaB” Specialist. Monthly reports mainly summarize the activities carried out but do not refer to the achievements. Regular in-service training is provided, but there are no regular and formal meetings to discuss bottlenecks and exchange experiences being properly documented as lessons learnt. 18 FY 2012 figures. LAUNCH MTE – Final Report ©Moneval Solutions 2013 Page 36 of 113 Some AEAs complained about the absence of respect for their time consuming work and asked for better working conditions and computers to improve their reporting. 4.2.1.7 Training EQ: Are the training and BCCs materials used by LAUNCH appropriate and effective in project’s targeted areas? Most of the training was practical and took place on Block Farms in line with the activities defined in the LAUNCH Strategy Document. For each of the training topics a series of training sessions has been designed based on FAO and GoL recommendations. Most of beneficiaries are illiterate and could not, during the MTE, recall having received any written training material. The training they received has been well adjusted for each of the three block farming modules and covered crop management issues. Most of the participants remembered some of the training subjects on plant management, mainly technical and hands-on issues such as: How to prepare ridges and mounds, preparation of nurseries, proper weeding. A minority of farmers were critical about the added value of the acquired skills compared to the time they had to invest. Marketing and processing issues are frequently dealt with by project staff but this comes often too late such as at harvest time. This leaves little time for the FGs to decide on the utilization of the crop and the long term direction for the development of their group. FG members, as well as some LAUNCH staff, have difficulties when it comes to making estimates about expected yields, quantities and prices in order to calculate entrepreneurial risks. This leaves a high degree of uncertainty of future activities. The strategy also provides training modules for “Farming as a Business” which is not clearly structured and mixes teaching contents for “normal” members of FGs with training topics which are only required for managers. These are therefore aimed towards the leaders of the FG or to some “professional farmers” in the community with individual plots and strong private interest in the subject. It is feared, however, that those persons would have little interest in training modules around crop management. LAUNCH has not yet established a comprehensive extension strategy defining training content, achievements and suitable modules for the different groups of farmers. Suggestions include:  Traditional farmers being FG members willing to learn new crop management techniques;  “Professional Farmers” with basic commercial qualifications and motivation to do farming as individual businesses (also able to take risks); and  FG leaders who need to be trained to organize FGs as well as in business skills including book keeping, market analysis and financial management etc. Considering the training needs and the demand for extension, it seems that the present composition of LAUNCH extension personnel reflects the needs of FGs in year one only rather than anything beyond (there is an emphasis on crop cultivation techniques only). Considering all the skills to be acquired by beneficiaries in subsequent years (farm management, post harvest management, processing and marketing) it seems that more subject matter specialists such as tropical agronomists, agricultural extensionists, marketing and small business support will be needed in the extension team. LAUNCH MTE – Final Report ©Moneval Solutions 2013 Page 37 of 113 Farmers have not received any individual training material (written), but FGs have received agricultural/planting kits. It is concluded that the training methodology is appropriate but effectiveness is still hampered by deficiencies in the contents of the training. 4.2.2 Efficiency 4.2.2.1 Bottlenecks Impacting Implementation Program implementation is mainly in hands of to the two County Offices (Totota and Saclepea) who organize and coordinate the AEA and subject matter specialist input. The head office is responsible for strategic issues as well as the monitoring of activities and results. This ensures efficient implementation of most day-to-day activities. The organizational structure is considered to be efficient and well adapted to the geographical conditions. There is a lack of in-depth knowledge and practical experience within LAUNCH management with respect to prevailing problems of agricultural production in the region. This makes it difficult to decide about suitable implementation strategies. The assignment of AEAs to the FG show some deficiencies though, as the time spent per FG is significant. According to the purpose FGs are visited by various extension agents or specialists. With a higher degree of specialization within the extension team towards their crops (including production, processing and marketing aspects) the number of visits to FGs could be reduced. Part of the effort assigned to improve the monitoring of the IPTTs seems to be unbalanced as long as it is quite clear that no significant progress can be expected. 4.2.2.2 Coordination On request of the GoL the project has participated in coordination meetings with other donor projects mainly to avoid geographical overlapping of activities. Little has been done so far at the regional level regarding areas for cooperation with other programs (for example, graduation of groups), benefitting from other project equipment, exchange of seeds, joint research on alternative value chains or linking producers to benefit from economies of scale. 4.2.2.3 Issues and Constraints EQ: What are the key constraints that have been encountered during program implementation? A number of the key constraints with respect to effectiveness have been dealt with in the preceding chapters. Some crucial problems related to generating additional farm income from food production stem from framework conditions which the project cannot fully deal with, including:  Poor road access: Many farmers mentioned the lack of road access, especially during the harvesting season, as the most severe bottleneck towards more food production (especially in remote areas like Piata (Bong) and Yarpea Mahn (Nimba);  Access to individual plots: Many farmers have to walk more than one hour to reach their individual farm; LAUNCH MTE – Final Report ©Moneval Solutions 2013 Page 38 of 113  Lack of capital was mentioned by many farmers as a bottleneck to increased production, mainly to pay for agricultural inputs, farm labor or even processing machinery;  Lack of easily applicable and low input production techniques for food crops which provide higher production or require less attention or farm work; and  Risks with respect to crop losses: Farmers identified animal pests as a major constraint, such as ‘groundhog attacks’, referring to various types of bush animals who eat crops standing in the field, (reported by more of than half interviewed of the households in Bong, but less in Nimba). It could also be confirmed that bird attacks were a threat to crops, which had been reported by 17% of farming households according to CFNFS data). The LAUNCH Support Strategy is based on the following principles and assumptions:  Group based approach for extension work in order to reduce extension and travel time and provide cost effective training;  Standardized practical training and work on block farm with demonstration plots;  Limited funds for investment and farm credit; and  Replication of improved production systems will also take place on private farms belonging to FG members. The LAUNCH Extension Strategy limits the instruments for effective implementation as proposed farming practices on the one hand need to be adapted to the realities faced by farmers and on the other envisaged results should be tangible and significant in relation to food production. Comments from resource persons and MTE observations indicate that the gap between farmers reality and tangible benefits has not yet been narrowed sufficiently in order to become effective with respect to improvements in the food situation (SO1) or farmers behavioral changes (Please refer to IR1.1. and IR1.2). Experience with cassava shows that many farmers are still reluctant to apply improved management techniques on their private plots. This is because they are not convinced that additional production volumes are worth changing their traditional agricultural practices. This may be a matter of time before adoption rates increase but current indications are that there is a reluctance to become too adventurous in the short term. Very few observations made of lowland rice show that there might be a need to establish costly water retention schemes or other infrastructure in order to significantly increase rice production in the LAUNCH villages. Increase in vegetable production is highly dependent on the initiative of one or more strongly motivated and entrepreneurial spirited farmers in the communities. The success of market oriented vegetable production depends on the availability and readiness of these people. It is concluded from FDGs that crop management improvements are not sufficiently obvious or readily achievable, as originally envisaged by the LAUNCH team. This is based on farmer risk aversion and the real perceived prospects of failure. LAUNCH MTE – Final Report ©Moneval Solutions 2013 Page 39 of 113 5. SO2: EVALUATION FINDINGS 5.1 Design and Planning LAUNCH has four intermediate results with respect to SO2 which aims reduce chronic malnutrition of vulnerable women and children under two. These are as follows:  IR 2.1 – Strengthened community capacity to engage in health, nutrition and hygiene interventions;  IR 2.2 – Reduced malnutrition in children under two and improved maternal nutrition;  IR 2.3 – Improved prevention and treatment of maternal and child illness; and  IR 2.4 – Improved water, hygiene and sanitation access and practices. Table 4: Result oriented indicators (IPTT) for SO2 Code Indicator Comment IR 1.1 FFP IM.3 Percentage of underweight (WAZ<-2) children aged 0-59 months (FFP Impact Indicator #3) LAUNCH appears to be making significant changes in the numbers of underweight children in their implementation area. FFP IM.4 Percentage of stunted (HAZ<-2) children aged 6-59 months (FFP Impact Indicator #4) Growth monitoring is reportedly indicating changes in the percentage of stunted children. FFP MN.2 # of beneficiaries reached, by sector (FFP Monitoring Indicator #2, Output) LAUNCH appears to be on target to reach the # of beneficiaries outlined. FFP MN.3.2 % of children 6-23 months of age who receive a minimum acceptable diet (FFP Monitoring Indicator #3, Outcome) MTE was not able to ascertain this particular outcome. However, FGDs indicate that reported changes in caregiver behavior will lead to the achievement of this outcome. FFP MN.3.2 % of children 6-23 months of age who receive a minimum acceptable diet (FFP Monitoring Indicator #3, Outcome) MTE was not able to ascertain this particular outcome. However, FGDs indicate that reported changes in caregiver behavior will lead to the achievement of this outcome. LAUNCH MTE – Final Report ©Moneval Solutions 2013 Page 40 of 113 FFP MN.3.8 % of caregivers demonstrating proper food hygiene behaviors (FFP Monitoring Indicator #3, Outcome) MTE did not assess the demonstration of proper food hygiene behaviors, but FGDs indicated that caregivers know proper food hygiene behaviors in great detail. FFP MN.3.9 % of caregivers demonstrating proper water hygiene behaviors (FFP Monitoring Indicator #3, Outcome) MTE did not assess the demonstration of proper water hygiene behaviors, but FGDs indicate that caregivers know proper water hygiene behaviors in great detail. MIS HE.1 # of children reached by USG-supported nutrition programs (Mission Output Indicator, Health) This indicator will be met. AV OP 2.1.1 # of pregnant and lactating women receiving food rations This indicator will be met. AV OP 2.1.2 # of children aged 6-23 months receiving food rations This indicator will be met. AV IM 2.2.1 # of health facilities that meet minimum GoL standards Not assessed by MTE. MIS HE.2 # of people trained in child health and nutrition through USG-supported health area (Mission Output Indicator, Health) [Clinic Staff] While MTE assessed if clinics had received training, an assessment of the number trained against the indicator was not assessed by MTE. MIS HE.3 # of people trained in maternal and/or newborn health and nutrition care through USG-supported programs Clinical, community and caregivers have been trained. This is likely to be met. AV OP 2.2.1.1 # of medical service providers trained in targeted areas on CMAM This indicator will likely be met. AV OP 2.2.1.2 # of facility-based staff trained on nutritional counseling, anthropometry, and effective utilization of referrals systems This indicator will likely be met. 5.2 Integration of Program Objectives SO2 is generally not integrated with other the other SOs of LAUNCH, for example, Lead Mothers and Neighborhood mothers (or their household members) may be part of farmer groups, but their participation is not systematic and part of the program design. There is very little overlap save the LAUNCH MTE – Final Report ©Moneval Solutions 2013 Page 41 of 113 same groups being targeted. In the LAUNCH Annual Monitoring Survey –FY 12, 23% of beneficiaries benefitted from SO1 and SO2, with the rest benefiting from one or the other SOs. SO2 interventions are aimed to decrease malnutrition in PLW and CU2 through the implementation of the Preventing Malnutrition in Children under two Approach (PM2A). This is a “food-assisted” approach to reduce the prevalence of child malnutrition by targeting a package of health and nutrition interventions to all pregnant women, mothers of children 0-23 months, and children under two in food insecure program areas, regardless of nutritional status19”. In Liberia, 20% of children under five are underweight and 40% suffer from stunting. To address the combination of inadequate food and less than optimal food usage, LAUNCH complements PM2A activities with the Care Group methodology. “A Care Group is a group of 10-15 volunteers, community-based health educators who regularly meet together with project staff for training and supervision. Each volunteer is responsible for regularly visiting 10-15 of her neighbors (at least monthly), sharing what she has learned and facilitating behavior change at the household level20”. The Care Group model is intended to be implemented by participant selected lead mothers who receive training on health and nutrition messaging and in turn conduct sessions with their neighborhood group. A Care Group is the geographically defined group of lead mothers. Each lead mother has an assigned group of household mothers, who benefit from the food distribution. 5.3 LAUNCH Strategies and Activities 5.3.1 Target Groups LAUNCH distributes food rations to pregnant and lactating women (PLW), children 6-23 months and households during the lean season regardless of nutritional status or economic situation. This has nothing to do with farming activities under the program. The household ration is intended to prevent misuse of the regular PLW and Children under two (CU2) rations. LAUNCH is using a “one-size fits all” ration for the child, PLW and household ration. To receive the food ration, PLW beneficiaries must be members of a Care Group, receive a beneficiary number and card, present records of their pregnancy or child’s birth verified through the child or woman’s health card book used at clinics and proof of identity. 5.3.2 Government Engagement EQ: Is the level of effort among different interventions and activities relevant to the MCHN problems facing the community? The MTE was unable to participate in discussions with national Ministry of Health (M0H) officials, as a meeting could not be secured. Though local government awareness varied slightly, the MTE found local government officials were aware of the food ration, nutrition education, training of TBAs/TTMs and the Care Group model. This was especially true of clinic and government officials. Additionally, nutrition officers were very aware of the nutrition activities of LAUNCH. Maternal and Child health 19 Title II Technical Reference Materials. TRM-01: Preventing Malnutrition in Children Under 2 Approach (PM2A): A Food￾Assisted Approach. Revised November 2010. Washington DC: AED 2010. 20 http://www.caregroupinfo.org/blog/criteria as accessed May 27, 2013. LAUNCH MTE – Final Report ©Moneval Solutions 2013 Page 42 of 113 coordinators were less informed in general, but all were aware that LAUNCH provided food rations. Only one local government official, a community coordinator, was informed of the other non-health interventions. All KIIs with government concluded that LAUNCH is implemented in line with the GoL strategies. All local government staff report believing the health education sessions will continue at the end of LAUNCH, although they speculate there will be fewer numbers of participants and possibly fewer topics covered. However, there was no indication of a concrete strategy for GoL to assume responsibility for the activities. One government worker stated that a benefit of the food ration is that households then have to spend less money on food. The MTE only conducted one KII with TBAs. However, through KIIs with clinic staff and the TBAs/TTMs, trained by LAUNCH, it appears TBAs/TTMs are knowledgeable of ENA messaging and indicate changes in the messages they promote as a result. In some communities, the clinic vaccinators do outreach in the community to increase vaccinations in addition to clinic vaccinations during routine clinic visits. In one community, the local government clinic vaccinator took the initiative to liaise with the program to participate in the FDP to perform their outreach vaccinations. While the program knew about this initiative, there was no systematic promotion of this as an adopted best practice. LAUNCH has funded large scale complementary inputs at the local health centers. These include mother waiting shelters, toilets, and hand washing stations. However, during site visits, while these inputs were complete, they were not yet in use. Health center staff report waiting for a formal opening ceremony (known locally as “dedication”) before allowing these inputs to be used. This did not allow the MTE to assess the impact of these inputs on the health center services. Through its work with household mothers providing education and food rations, LAUNCH is also making progress towards reducing malnutrition in children under 5 and PLW. 5.3.3 Government Health Center Capacity Building EQ: How successful has the activity been in increasing access to government preventative and curative health and related social services? GoL requires facility based growth monitoring, which does not allow for community based growth monitoring. LAUNCH bases its community promoted activities on the promotion of ANC visits, vaccinations and growth monitoring, while building the capacity of government health centers through supportive supervision, training and growth monitoring materials. Working within the GoL strategies, LAUNCH has initiated training on how to properly complete the pregnant woman and child health books and offers routine monitoring and visitation to selected clinics. Interaction with clinic staff and a review of child health books of beneficiaries, indicate that training is effective and helpful to staff. One District Medical Officer (DMO) commented that staff appreciate the child health books (and pregnant women books) because they offer a readily available history of the services the individual has received. However, household mothers report their child health books are being completed but they do not know what they are writing because they cannot read or what the charts mean for their child’s health. A review of growth monitoring books brought by FGD participants indicate that growth monitoring charts are being completed and that LAUNCH participants are going for ANC visits. Through review of randomly chosen growth monitoring charts during FGDs and clinic visits, the growth charts are being completed accurately, however, mothers are not told why the books are necessary or what the notations mean. LAUNCH MTE – Final Report ©Moneval Solutions 2013 Page 43 of 113 5.3.4 Care Group and Lead Mother Organization EQ: Do field staff and community volunteers have appropriate BCC materials that focus on a limited number of harmonized messages for optimal infant and young child feeding? EQ: Do those responsible for promoting health and nutrition behavior change have SBC materials that are appropriate – tailored to the user, actionable, accurate and linked to growth promotion messages? Which materials need strengthening, if any, and how? Are users of these materials able to select the appropriate messages and provide effective counseling? If not, what skills need strengthening? Care Groups are organized into geographically acceptable boundaries within groups of 10 -15 Lead Mothers. Lead mothers did not report finding distances to attend Care Groups difficult, nor did they report challenges with the distance of the neighborhood mothers. Content and Instruction: Lead Mothers have received four Health Modules which cover all information on health promoted by LAUNCH. The training module is conducted by government health promoters and monitored by LAUNCH Health and Nutrition Staff. Lead Mothers did not report challenges with the material. However, through FGDs there was an indication that there may be a need for more emphasis in two areas these being self-efficacy around the provision of appropriate complimentary feeding with locally available foods and the importance of nutrition for children aged 2-5 years. Lead Mothers: There was no report or indication that Lead Mothers face geographic or distance challenges in the implementation of neighborhood education sessions. This is in line with the Care Group methodology. Although Lead Mothers were very knowledgably of the various health messages in the modules, when asked what questions they had of the program, every Lead Mother FGD, asked why the program only fed children under 2 years old (CU2). Lead Mothers reported never having received information on why this age group is the focus. Lead mothers were asked what questions they still wanted answered of the program. In every FGD with Lead Mothers, they asked why the program only fed children under two. When asked why, they would say “…because the program says so”. This becomes problematic; as the child beneficiaries are obviously bigger and healthier the mothers want the same for their other children. When asked what LAUNCH does, household mothers would indicate making children healthier. Often a mother would hold a big baby and say “LAUNCH”, clearly indicating that mothers see the changes in their children as a result of the program. However without proper understanding as to why this age group is important, the cutoff Photo: Daphyne Williams, 2013 Neighbourhood Mother Group meeting is concluded, Santa Village, Liberia. LAUNCH MTE – Final Report ©Moneval Solutions 2013 Page 44 of 113 seems arbitrary and unfair, and will likely lead to them sharing CSB resulting in all of their children being improperly fed. Lead Mothers are trained by LAUNCH Health and Nutrition staff and monitored by the staff and Government Community Health Volunteers (GCHVs). Lead Mothers have received laminated health education modules for use with neighborhood mothers. FGDs with lead mothers and neighborhood mothers indicate that the lead mothers all have access and use the health education modules. The MTE did not find any Lead Mother or receive reports from Household Mothers that Lead Mothers were without health education materials. Graduation and Incentivizing Lead Mother Participation: After 2.5 years of implementation, LAUNCH is reaching the stage where some Lead Mothers will be graduated from the program, i.e. their children are 2 years old. There does not appear to be a consistent plan to maintain Lead Mother participation or motivation. For example, in Bong, Lead Mothers report some are beginning to decrease their participation and activities as Lead Mother, after they graduated. In one community in Bong, Lead Mothers have organized themselves to motivate those who are not participating with the previous level of effort. Furthermore, the lack of distribution of promised materials, may be contributing to the decrease in participation. In Bong, LAUNCH selected high performing Lead Mothers to receive improved cooking stoves and other materials such as lappas. However, the process was highly centralized and the responsibility of one staff member. When that staff member resigned, distribution of the stoves was stopped because the other staff reported not knowing who and how recipients were to be selected. There are well over 50 stoves in the office storeroom in Bong awaiting distribution as a result of this confusion. Staff report waiting for instructions as how to proceed in this distribution. At various points during the MTE, Lead mothers, GCHV, clinic staff and household mothers inquired when Lead Mothers would receive the lappas and other working tools as indicated by the program. EQ: Does the community have a clear understanding about the services offered by the project and who are eligible to receive them? EQ: Has the program made progress in implementing intervention as designed to reduce Chronic Malnutrition among Vulnerable Women and Children? Neighborhood Mothers: Neighborhood Mothers have great trust and faith in LAUNCH and the information provided by Lead Mothers, going so far as to attribute benefits to LAUNCH that other organizations have provided, including bed net distribution. In FGDs they report taking the messages to heart and adopting the promoted behavior. Reported changes that were frequently mentioned were; practice of EBF, hand washing and knowledge of family planning options. This is believed to due in part to the very visible differences in children benefiting from LAUNCH, as well as their belief that since LAUNCH began fewer children are dying in their community. This has created immense social credibility and positions the program to make significant impact in the promotion of proper feeding for the 2-5 year olds. However, many attribute the benefits and changes in their children solely to the utilization of CSB, post EBF. There appears to be poor understanding that the changes are due to improved nutrition, healthcare and hygiene. LAUNCH MTE – Final Report ©Moneval Solutions 2013 Page 45 of 113 5.4 Implementation 5.4.1 Effectiveness 5.4.1.1 Progress Towards Strategic Objective The implementation of SO2 appears to be quite effective in that it is actually reaching its targets and implementation objectives. Core results for the percentage of underweight (WAZ<-2) children aged 0-59 months (FFP Impact Indicator #3) and (HAZ<-2) children aged 6-59 months (FFP Impact Indicator #4) show a positive change in the percentage of stunted children. In addition LAUNCH appears to be making significant changes in the numbers of underweight children in their implementation area and the numbers of beneficiaries reached by sector (FFP Monitoring Indicator #2, Output) also appears to be also good. Other areas of quantitative effectiveness were not so easily observed by the MTE team. Qualitatively however there has been good uptake by target groups involved in terms of both practice and behavior change anticipated. There is a danger that there will be a dependency created on LAUNCH by communities which will undermine longer term desired effects of independence and the creation of demand for a reliable and fully serviced health system for families. IR 2.1 – Strengthened community capacity to engage in health, nutrition and hygiene interventions; and IR 2.2 – Reduced malnutrition in children under two and improved maternal nutrition. The MTE team believes that LAUNCH is implemented in line with PM2A and that PLW are realizing the improved nutrition as expected from the proper implementation of PM2 and that LAUNCH is implementing the Care Group in line with the designed methodology. The groups are of appropriate size and geographical location. The MTE team found no indication that the models were improperly organized, poorly geographically distributed or included inappropriate beneficiaries. LAUNCH also work with Community Health Centers (CHCs) and Community Health District Centers (CHDCs). SO 2 activities consist of a complementary package of services which will lead to long term behavior change including:  Training of Lead Mothers in Essential Nutrition Actions (ENA), messaging and the LAUNCH Mother and Child Health Education Flip Charts. This includes information on mother and child health, optimal infant feeding, child spacing, male involvement and hygiene promotion;  Training of traditional birth attendants (TBAs)/trained traditional midwives (TTMs) in ENA messaging and facility based delivery;  Promotion of facility based births;  Promotion of antenatal care (ANC) visits, clinic growth monitoring and vaccination;  Supportive supervision and trainings for government clinic staff;  Conditional food ration provision; and  Water hardware improvement and/or installation. LAUNCH MTE – Final Report ©Moneval Solutions 2013 Page 46 of 113 5.4.1.2 Exclusive Breastfeeding EQ: Has the program made progress in reducing Chronic Malnutrition among Vulnerable Women and Children? EQ: Is LAUNCH’s Care Group approach appropriate and effective in improving Nutrition, Feeding and Care Practices among PLWs and Children under 2; EQ: Which MCHN strategies have been more effective so far and why? What new knowledge have clients acquired and what knowledge is lacking? Has new knowledge translated into new practices and recommended behaviors? If yes, what are the contributing factors to this achievement? If no, does the program understand the obstacles to optimal behavior and have a plan to address? In the LAUNCH Annual Survey Report – FY12, 81% were found to practice exclusive breastfeeding. This is supported by the MTE findings as indicated in FGDs. LAUNCH appears to have made significant progress in the promotion of exclusive breastfeeding (EBF) and the utilization of colostrum. All (100%) of Lead and Household mothers focus groups report practicing EBF from birth until the child is 6 months old. In each group, household mothers could articulate the importance of EBF and believe it makes their children stronger and prevents sickness. All (100%) beneficiary FGDs could articulate how and when to practice EBF, when to stop, as well as proper hygiene practices for breastfeeding. Household mothers were asked questions to verify their understanding of EBF during FGDs, including whether it is appropriate to give a three month old child water and if it is appropriate to give a child under six months old water if sick with fever. All answered correctly. The benefit of EBF appears to be widely accepted and the practice normalized amongst the household mothers. In Nimba, several communities (~50% of FGDs) reported that since the beginning of LAUNCH, they have changed the traditional practice of discarding colostrum after learning of its benefits in the neighborhood groups. Household mothers not only reported the use of colostrum, but also promoted the belief that colostrum is beneficial to babies. Many described colostrum as a vaccine with one calling it “the first vaccine”. Lead Mothers in these communities indicated that the TBAs/TTMs also now promote the use of colostrum. TBAs/TTMs reported they now promote and understand the importance of colostrum due to LAUNCH training they received. 5.4.1.3 Complementary Feeding Ration Use: FGDs indicate that household and lead mothers understand the intended use of the ration that is for pregnant and lactating women and children between 6 - 24 months. Lead and Household Mothers report proper use of the food ration as intended by the program and as instructed in neighborhood groups. When prompted, 90% Household Mother FGDs gave a detailed explanation of the proper and hygienic preparation, storage and use of CSB for children over 6 months. Additionally, household mothers were able to articulate promoted complementary food for children 6 months to 2 years using diverse locally and readily available food. Household mothers report gaining this knowledge from LAUNCH. The children of LAUNCH beneficiaries (from newborns to 2 year olds) are visibly large and plump. When asked what LAUNCH does, neighborhood mothers in several groups presented a big baby and simply said “LAUNCH”. This is a clear indication that women see the changes in their children since LAUNCH and attribute these changes to the program. There is therefore widespread belief that the rations are helping children. However, there appears to be poor understanding that the nutritional achievements are due to both the proper utilization of the CSB and the mothers own LAUNCH MTE – Final Report ©Moneval Solutions 2013 Page 47 of 113 efforts. Despite seeing the changes in the size and health of their children, ~ 50% of Lead Mothers and household mothers indicated that they do not attribute the improvement in their children to their actions. This line of questioning often spurred a great debate as to whether locally available foods were as good as the CSB. Some FGDs argued local food mixtures can have the same impact on the children as the CSB. One household mother FGD suggested that the program puts medicine in the food to make the children healthier. Some discussants indicated that they believed that there are medicines in the ration to make the children healthy. This is likely to result in both the utilization of the ration as well as increase the likelihood of sharing the ration with other children. When the issue of women believing medicine is in CSB was raised with staff during debriefing discussions, staff indicated that the statement (that they put medicine in the food), could actually be a result of translation constraints. According to staff, there is no term in the local dialects for “vitamin” and therefore Program Staff have translated the fortified/vitamin rich foods with the same or similar term used to indicate vitamins from the clinic, such as vitamin A promoted in the ENA education. Rather than there being a fundamental misunderstanding about the food, there may be a translation error that could be causing beneficiaries to mistakenly attribute additional benefits to the CSB. While LAUNCH is having obvious immediate effects on the size and health of children in the PM2A program in complementary feeding. However, the attribution of these achievements solely to CSB and not to the mother’s own action is a cause for concern. IR 2.3 - Improved prevention and treatment of maternal and child illness EQ: Are the health and nutrition change and communication (BCC) material used by LAUNCH appropriate? EQ: Through what process were the health and nutrition BCC materials developed, tested and applied? LAUNCH Behavior Change Communication (BCC) strategy is built on the provision of ENA, health and key hygiene messages primarily delivered through the Care Group methodology using trained Lead Mothers to conduct education sessions which lead to the increased uptake of preventative and curative services as far as the MTE could tell. LAUNCH staff and GCHVs monitor the delivery, timing and accuracy of education sessions. While the groups vary greatly, it appears there is at least minimal understanding as to why ENA messages were important during each FGD. Among the messages that appear to particularly resonate with mothers are hygiene, especially hand washing, and EBF. In each FGD, these three topics were mentioned as important to child health. Hygiene promotion education appears to have had a large impact on the actions of the women. When asked “what is the biggest difference made by LAUNCH, after the food rations?” increased and improved hand washing was repeated most often by lead and neighborhood mothers, with nearly every FGD mentioning hand washing. The women report changes in the frequency, method (using clean water and/or not using one bowl and the same water for hand washing) and timing of hand washing, after the toilet and before eating. In Nimba, Lead Mothers, after learning about the importance of hand washing developed a hand washing station using locally available resources. However, these changes and impact are independent of the water, sanitation and hygiene (WASH) strategies and WASH committee activities and predominately a result of the health education messaging received in the neighborhood discussions. It should be noted that while the FGDs LAUNCH MTE – Final Report ©Moneval Solutions 2013 Page 48 of 113 reported knowledge of the importance of hygiene and some groups demonstrated their knowledge, in the LAUNCH Annual Survey, 52% of caregivers demonstrated proper hygiene behavior. With the combined and complementary efforts of the government promotion of facility based delivery, LAUNCH promotion of facility-based deliveries with mothers and education of TBAs/TTMs as collaborators in delivering ENA messages, appear to have significantly increased ANC visits, facility based births and well-baby visits, and the knowledge of the importance of these visits. Though household mothers state the number of promoted ANC visits ranged from three, to once a week for four months, all FGDs reported attending ANC visits in their last pregnancy and have participated in growth monitoring. In KII sessions of the MTE, clinic staff and District Health Officers (DHOs) attributed increases in ANC visits and facility-based childbirths to LAUNCH. 5.4.1.4 WASH IR 2.4 – Improved water, hygiene and sanitation access and practices EQ Are WASH and DRR components appropriate and relevant in attaining SO2? EQ: What strategies are employed to influence water, sanitation and hygiene (WASH) behaviors of men, women and children? What is the level of successes of these strategies in influencing WASH behaviors of target population? What is the quality (i.e. interactive sessions, length of the session, topic selection, quality and use of visual aids, quality and use of flipcharts, participation, and demonstrated knowledge of the facilitator on the topic) of behavior change sessions? How it can be improved? EQ: What is the level of training of staff to promote health and hygiene interventions? Is there knowledge about behavioral change and its links to project hierarchy adequate? What additional training, if any, would be recommended for project staff? WASH activities in LAUNCH are two-pronged and distinct relating to the construction or rehabilitation of water points and the promotion of hygiene behaviors including the construction of latrines. WASH committees are tasked with leading community hygiene efforts, including the construction or rehabilitation of water points, hardware and promotion of hygiene activities including the construction of hand-washing stations. WASH committee members are selected from the community and are often also beneficiaries (or their partners) of other LAUNCH activities, i.e. Lead Mothers who are also members of the WASH committees. LAUNCH has established WASH committees in Bong. The WASH committees and activities in Nimba have not yet been initiated, so WASH activities beyond the Care Group lessons were not in evidence. WASH Hardware Construction: WASH committees are contributing to the construction of water points in the community by rallying and organizing the community to complete a number of tasks to ensure water point construction or pump rehabilitation as directed by LAUNCH. The MTE included KIIs with WASH committee members in Bong and visits to LAUNCH constructed water points. No LAUNCH constructed water point was observed functioning as they all were awaiting an opening ceremony (known locally as a “dedication”). The impact of the water point construction or rehabilitation on the community could therefore not be assessed. WASH Hygiene Promotion: While the mandate for the WASH committees covers structural water point access and maintenance, they also promote hygiene integration and rolling out “tippy taps” and other hand washing stations. However, the WASH committee promoted behavior is separate and distinct from the health and nutrition activities and insufficiently integrated with the hygiene promotion activities in the Care Groups. WASH staff reported having inadequate LAUNCH MTE – Final Report ©Moneval Solutions 2013 Page 49 of 113 information on the hygiene activities promoted in the Care Groups to allow them to provide technical assistance or to appropriately and sufficiently complement these Care Group activities. LAUNCH reports the intention to promote and use “tippy taps”. However the “tippy tap” construction and rollout is slow and insufficient. Staff report they are waiting to see community demonstration/adoption of the “tippy taps” uptake and promoted hygiene behaviors before introducing it widely to communities which is not happening currently. When asked, staff could not articulate what “uptake” looked like or what criteria were required before rolling out the methodology. Despite considering the handout as a challenge to program implementation, WASH staff report planning to provide 5 liter containers for the construction of household “tippy taps”. One main benefit of the tippy tap is the ease of construction with ready availability of materials. 5.4.2 Efficiency 5.4.2.1 PM2A Food Collection EQ: Is the commodity and food distribution system efficient and well set-up? Despite the proper implementation of PM2A, PLW experience several challenges in receiving the food rations. This includes distances travelled to food distribution points, long waits for distribution and at times they do not receive rations before the commodity distribution ends and must return the next day. Site visits to a food distribution point indicate that commodity management staff prioritize women who have traveled the farthest for the first distribution. Food distribution ends at 4:30 or 5:00 pm depending on the distance the food delivery trucks travel to the food distribution point, so that the truck can be parked by 6:00 pm per organizational rules. However, the time food recipient beneficiaries spend waiting for food distribution or the time and distance travelling to the food distribution sites is not considered in the timing of the end of the distribution. The end result is that at times beneficiaries are making repeated trips to food distribution points to receive rations. Food distribution management is highly centralized. Health and Nutrition and Commodity Staff are present during distribution. In Bong, staff indicate there is a complaints table so that disputes can be addressed during the distribution. However, despite the presence of Health and Nutrition Staff and Food Monitors, Commodity Staff in Monrovia solve all disputes regarding ration eligibility, resulting in delays and occasional improper beneficiary ejection from the rations. LAUNCH has initiated food distribution committees to alleviate issues at the food distribution points, however the MTE did not visit a site with a food distribution committee. FGDs with household and Lead Mothers indicated a concern/disbelief that food is not being divided according to program direction. Signs at food distribution points detail the ration size and division requirements, although 60% of Liberian women are illiterate21 and likely higher in rural areas these signs are written and numeric. Despite LAUNCH health and nutrition staff encouragement and promotion to bring along a family member to assist in carrying the ration home, some beneficiaries still come alone even late in their pregnancy or shortly after giving birth. 21 http://www.feedthefuture.gov/country/liberia as accessed May 29, 2013. LAUNCH MTE – Final Report ©Moneval Solutions 2013 Page 50 of 113 5.4.2.2 Issues and Constraints EQs: Are staff qualified and knowledgeable of the purpose and methods used in the program? EQs: What is being done to improve the capabilities of the staff and local partners to respond to community needs and meet the objectives of the program? Core issues and constraints of delivery of SO2 often apply to qualified technical staff as well as the infrastructure of Liberia which hinder implementation. Staff Technical Capacity: LAUNCH Health and Nutrition staff have a varied background and capacity. However, the program seems to be running effectively with no major challenges in terms of staff ability to support the health education training of Lead Mothers or to conduct support supervision visits at community health facilities. Staff would benefit from regular and ongoing training on nutrition strategies as well as training on documentation and data use for program improvement. Geographic and road conditions: Geographic distances and road conditions pose challenges for staff in the monitoring of some communities at great distances and with PM2A food recipients who travel long distances to the food distribution points. However, Care Groups and Lead Mother led health education sessions are organized so that there are few distance barriers to participation. Lead Mothers have little basic education which presents limitations in terms of documentation, reporting and assessment of the impact of the program. Monitoring tools, such as the QIVC for Health Education Sessions Given by H&N Staff and the Team Leader Checklist for Supervising H&N Staff monitor the accuracy and effectiveness of the information provided to Neighborhood mothers by Lead Mothers and Health and Nutrition Staff. This helps to mitigate the limitations of the less educated lead mothers. LAUNCH MTE – Final Report ©Moneval Solutions 2013 Page 51 of 113 6. SO3: EVALUATION FINDINGS 6.1 Design and Planning 6.1.1 Integration of Project Objectives SO3 aims at increasing access to education opportunities for the rural poor. The interventions are anticipated to result in:  IR 3.1 – Increased community capacity to support education infrastructure and systems; and  IR 3.2 – Increased access to livelihoods-based education. All of the schools and communities visited were in GOL and international NGO neglected areas. This was evidenced by the numerous accessibility/geographic challenges in reaching communities and schools by the MTE team. For example reaching Mao in Yarpeah Mah District, Nimba County was hampered by bad road condition and broken bridges. In addition, Wraputa in the Salala District and Boyea in the Sanoyea District of Bong are examples of county neglected schools of neglected communities, which LAUNCH is currently intervening. Strategic Objective 3 seeks to address education infrastructure, get students back to school, improve the standards of teachers and improve the performance of Parents Teachers Association. Activities employed to attain the objectives include working with schools, teacher and communities to build capacity in an improved record system in schools, tracking boys and girls enrolment, intake and dropout and reason for drop out. EQ: How does the SO3 link to other LAUNCH objectives? EQ: Are beneficiaries also benefitting from other activities under other objectives? Strategic Objective 3 is linked more indirectly to food security through the two Intermediate Results of SO3. On the other hand activities carried out especially WASH are usually carried out in communities only where students are not direct beneficiaries. The education IRs’ are relevant to food security through the establishment of school gardens. School gardens are essential to address food security situations of students thus reducing burdens placed on families. The IRs on youth empowerment contributes largely to the food security situation of youths. Discussions with youth groups showed that the entrepreneurial training contributes largely to their livelihood pursuit and most said that they feed themselves from proceeds of their sales. EQ: What does SO3 seek to address? This discussion question was posed to staff after a review of IPTT. The ultimate objective of this discussion was to gauge knowledge of staff in terms of expected results of the strategic objective. There is some confusion on the performance tracking table as the objective and the two intermediate objectives have been rephrased. However, using the initial set of indicators regarding SO3 implementation, the MTE comments on indicator usefulness are provided in the table below. LAUNCH MTE – Final Report ©Moneval Solutions 2013 Page 52 of 113 Table 5: Result oriented indicators (IPTT) for SO3 Strategic Objective 3: Improved Educational Opportunities for Children and Youth Code Indicator Comment AV IM 3.1 Promotion Rate This indicator needs to fit into the national Education Management Information System (EMIS) to avoid duplication with national education statistics. FFP MN.2 # of beneficiaries reached, by sector (FFP Monitoring Indicator #2, Output) This is okay as the evidence for increase in enrolment due to LAUNCH can be tracked AV IM 3.2 Continuation Rate This indicator needs to fit into the national EMIS to avoid duplication with national statistics on education. IR 3.1 Improved Quality of Primary School and Livelihoods-based Education for Youth MIS ED.1 # of learners enrolled in USG supported primary school or equivalent non-school-based settings (Mission Output Indicator, Education) This is okay as the evidence can be tracked by LAUNCH MIS ED.2 # of learners enrolled in USG supported secondary school or equivalent non-school-based settings (Mission Output Indicator, Education) (LAUNCH - Entrepreneurship Promotion Programs) This is okay as the evidence can be tracked by LAUNCH AV OC 3.1.1 # of activities initiated by grant recipients to improve education, health, nutrition (e.g. demonstration gardens, social awareness campaigns, etc) This is on track as evidenced in Sanoyea and Salala. This is okay as the evidence can be tracked by LAUNCH AV OP 3.1.1 # of community-led educational grants disbursed This is okay as the evidence can be tracked by LAUNCH AV OP 3.1.2 # of entrepreneurship grants to youth Okay and on track AV OP 3.1.3 % of grants disbursed to women An improvement needed in participation of boys. This is okay as the evidence can be tracked by LAUNCH grant AV OP 3.1.4 # of businesses started by youths who participated in Entrepreneurship Programs This okay and on track LAUNCH MTE – Final Report ©Moneval Solutions 2013 Page 53 of 113 IR 3.2 Improved Management of Schools/Education Programs MIS ED.3 # of Parent-Teacher Associations or similar ‘school’ governance structures supported (Mission Outcome Indicator, Education) This is okay and on track. PTAs are supporting school as a result of LAUNCH intervention. This is okay as the evidence can be tracked by LAUNCH AV OC 3.2.1 # educational institutions and organizations that have instituted programs to address the needs of youth AV OC 3.2.2 # of communities with developed learning/educational/school plans This is okay as visited schools had tracking boards displayed with plans. This is okay as the evidence can be tracked by LAUNCH EQ: How does the LAUNCH SO3 objective link to Government education sector objectives? LAUNCH’s two-pronged approach to SO3 intends to increase access to education opportunities for targeted youth by developing community capacity to support their education. Such activities should involve key government agencies including the Ministry of Education. This has not been the case for SO3. In discussions with County Education Officers they intimated that they were aware of the interventions of LAUNCH in their respective counties although not all officers were clear about LAUNCH’s education objectives. According to Education Officers this is largely due to LAUNCH doing their own assessments without their participation. At the district level, there were contradicting reactions to the program’s intervention. In Bong, the Sanoyea District Education Officer stated that he was uninformed of LAUNCH activities and that he had never been involved in planning any school activity carried out by LAUNCH. The Salala District Education Officer by contrast had stated that he was very happy with LAUNCH’s intervention in 24 out of 28 schools in his region22. In Nimba, all Education Officers were aware of the interventions by LAUNCH. 6.1.2 LAUNCH Strategies and Activities 6.1.2.1 Target Groups In line with LAUNCH’s objective under SO3, Government schools are the primary target of IR 3.1. The schools were selected based on the several key factors, including: needs within the school; limited previous support from NGOs; the willingness of the local Parents and Teachers Association (PTA) to participate; and the readiness of school principals to collaborate with the program. 22 “It is LAUNCH that makes me proud to visit my schools” (Philip F. Mulbah, DEO Salala District, Bong County). LAUNCH MTE – Final Report ©Moneval Solutions 2013 Page 54 of 113 LAUNCH considers communities with the presence of one of the two other initiatives before selection of the school. The target groups under the two IRs are found in vulnerable areas. 6.1.2.2 Selection Criteria EQ: How do you ensure that the criteria for beneficiary selection are appropriate? Selection of communities for SO3 activities is dependent on activities of the other two SOs followed by an assessment for its inclusion. There is conflicting feedback about the involvement of education authorities in LAUNCH between LAUNCH staff and education authorities of the project. 6.2 Implementation EQ: Share some key lessons you have learned during the implementation of LAUNCH SO3 On the whole there is sufficient gain in the attainment of result for the strategic objective. However, the implementation of activities under the SO3 is divided between two sets of staff with each set being responsible for one IR. Making Cents and PCI manage the attainment of these IRs respectively. Therefore, there are two managers at the implementation level supervising each set of staff. Even at the field level, it was difficult for the County Coordinators to precisely articulate managerial modalities for the staff involved in the implementation of the overall SO in a uniform manner. The result is that the implementation of SO3 is varied given the two implementation agents. According to field coordinators, it was hampering delivery of service especially procurement. 6.2.1 Effectiveness EQ: How has the implementation of LAUNCH changed communities and schools? The primary schools in LAUNCH’s target areas are in need of support. Most of the primary schools were in poor condition, managed by under qualified principals who were supervising poorly trained teachers. In addition, schools also lacked ‘learning materials’ and teacher/student contact hours were limited. Following an assessment of the schools in the districts where LAUNCH operates, LAUNCH concluded that it would support both school administrators and fledgling PTAs to identify low cost interventions to improve both access and education opportunities. LAUNCH has overall, responded effectively to date to the problems identified during design. The various support measures have made a contribution to the schools and communities in the following ways:  The materials provided have helped to increase enrolment in all the schools that LAUNCH works with There is evidence of an incremental enrolment in every school visited. For example between 2010 and 2012 school enrollment grew from 125 to 275 students in Wraputa and Salala Districts in Bong County. Collaboration with LAUNCH began in earnest in 2011;  Accurate record keeping and improved lesson planning for teachers are supportive for efficient administration of schools by principals. This will, it is hoped lead to sustainable change and continue beyond the end of project; and LAUNCH MTE – Final Report ©Moneval Solutions 2013 Page 55 of 113  There is now increased coordination between schools and PTAs at least in those schools visited. Support of this nature has made school administration and PTAs more efficient and therefore effective in attaining the goals. 6.2.1.1 Progress Towards Strategic Objective EQ: In what way do you think the activities set can achieve the set objectives? IR 3.1 - Increased community capacity to support education infrastructure and system The activities under IR 3.1 are intended to support Government effort in creating improved community access to education and improving the quality of the education provided. Activities have included: provision of inputs required for teaching and learning such as chalk and chalkboards among others. Additionally physical renovation exercises were carried out and furniture provided to support activities have been undertaken. The training of PTAs and provision of materials have also contributed highly to the quality of instruction in schools. LAUNCH has built the capacity of school administrators to better manage their schools. During the MTE, principals and teachers stated that the training in record keeping has improved their ability to track activities in their respective schools. They suggested that this has:  Led to the sustainability of these activities in schools even after the program ends, primarily due to improvements in the schools, better classroom management and active PTAs.  All teachers felt that they are fully equipped with knowledge to prepare lesson plans for daily instruction and that lesson planning made teaching easier.  Led to proper lesson planning, which according to all teachers during FGDs has contributed to the quality of instruction in LAUNCH school communities.  Led to greater accuracy in record keeping for school administration and pedagogical training, which have strengthened quality teaching and learning. Strengthening PTAs EQ: What was the level of organization of this PTA before LAUNCH? EQ: What supports are provided the PTA by LAUNCH? EQ: How does your PTA contribute towards increase in the enrolment of students? EQ: Are the support provide useful for your PTA? Explain By and large PTAs were not as organized and focused before their association with LAUNCH. In all FGDs and KIIs, it was expressed that even if PTAs were organized, LAUNCH collaboration has empowered them to better help school principals manage their schools. This component is perceived to be very successful for the following reasons:  Discussants stated that PTAs are now taking a lead role in school/community coordination as a direct result of LAUNCH. Every PTA visited hold regular meetings to discuss school related matters; LAUNCH MTE – Final Report ©Moneval Solutions 2013 Page 56 of 113  Discussants stated that they can distinguish a LAUNCH supported school from those not supported by LAUNCH. In all communities visited, discussant articulated that it is the PTAs that are now calling regular meetings;  There is evident increase in coordination between schools and PTAs as a result of LAUNCH intervention; and  The involvement of PTAs in school activities has contributed to the increase in school enrolment. Key Informants stated that they are involved in encouraging parents to send their children to school. Some PTA members stated that they were thinking about imposing fines on parents who did not attend. Provision of education supplies EQ: What can you say are the achievements of this PTA due to LAUNCH? EQ: How has the support made you efficient in carrying out your roles?  LAUNCH stationeries and other material support to schools are of immense help to the smooth running of schools. All FGDs participants articulated that material provided were those needed by schools to run their programs. They intimated that LAUNCH actually filled the gap created by the absence of supplies from the government;  They cited the supply of Early Childhood Development (ECD) kits as an example of supplies for children and the school. The support according to discussants has contributed to more effective delivery of quality education; and  There is delay in the supply of LAUNCH renovation materials in every community in Nimba visited. The delay in delivery of inputs has stalled community efforts in completing their projects. In all communities in Nimba, the supply of cement was still outstanding during the MTE visit. School Gardens: Discussants in all FGDs stated that this was part of the intervention by LAUNCH. Every school in one way or another has planned for the cultivation of crops. IR 3.2 - Increased access to livelihood-based education Activities under this IR include training of youth in entrepreneurship and providing grants to engage them in businesses. These two key activities can be summed up as improving access to livelihood pursuits for youth. Business Training for Youth Groups EQ: How can you describe the LAUNCH activities in working with you? EQ: How much has the LAUNCH program done for you? EQ: Which activities contributed most to your eventual livelihood pursuits? This activity is said to be successful and on track for the following reasons: LAUNCH MTE – Final Report ©Moneval Solutions 2013 Page 57 of 113  Discussants reported that the program intends to train them in their own business, by and for youth, and to teach the youth how to manage a business. They reported that initially as a group they were given the chance to practice what they learned. Furthermore, according to discussants, the better performing youth could receive additional support;  Over 95% of all discussants during the MTE reported that the two activities which contributed most to their livelihood pursuits were business training first and provision of grants second;  98% stated that training contributes more and with adequate training they believed they could pursue a business after the closure of LAUNCH23; and  Every discussant wished for additional support for business planning before receiving an initial mini-grant. They report knowing what sells better and at which time of the year would be useful and therefore a marketing support may be important to consider. 6.2.1.2 Technical Field Staff EQ: What do you think about the technical field staff training to supervise the implementation of the program? Achievement of results under SO3 relies heavily on the quality and quantity of staff involved in implementation. In each field office there are three staff implementing activities of under SO3. Two staff work on the primary education while the third concentrates on youth activities. From all indications, staff appear to be overstretched as the number of schools increases . In addition to the above, interaction with field staff indicates that further capacity building is required to improve their performance. Education Officers stated that field staff need some training in pedagogy to effectively monitor teaching and learning in schools covered by the project 6.2.1.3 Training EQ: Which activities have been most effective as perceived by the community in support of education? EQ: Which activities were not successful and why? How might these activities be improved? EQ: Which activities contributed most to your eventual livelihood pursuits? SO3 has been training teachers, PTA members and youth groups. Although trainings have an individual objective, participants were of the opinion that their present performance was better due to the training they received. During discussions with teachers, the training was thought to be very helpful. The benefits they saw included the following:  Improved skills in accurate record keeping in schools;  Their classroom management abilities had improved immensely; 23 “Without even grants, the training prepares us to do business using business chart“(Sanoyea Youth Group) said one interviewee. LAUNCH MTE – Final Report ©Moneval Solutions 2013 Page 58 of 113  Lesson planning for teachers had improved; and  ECD training had a positive impact on the communities by helping to change the behavior of parents towards their children. The training of PTAs was helpful in the following ways:  It increased coordination between the school and PTA members; PTAs are paying regular visits to schools while school authorities are now consulting PTAs in schools visited.  The PTA could now guide better the operations of the school and monitor attendance of teachers and students;  It improved ownership of the school and a sense of pride; and  The PTA was now able to coordinate activities better between schools and communities in which they were located. For example issues of low attendance and corporal punishment are now discussed with school authorities at PTA meetings. According to youth groups interviewed, training prepared them for the following:  Meeting business challenges faced by micro-and small enterprises in terms of profit and loss and forecasting of their sales to ensure they are successful; and  Identifying and developing approaches to proactively managing risks faced by businesses such as managing supplies, cash flow and business diversity. Every indication shows that training activities have contributed largely to the financial performance of the targeted groups. 6.2.2 Efficiency 6.2.2.1 Issues and Constraints EQ: What are some of the challenges you faced during the implementation of the project? EQ: What can you say are things that need to improve while working with LAUNCH? EQ: What are the key issues and constraints that have been encountered during program implementation and how can they be addressed? There are some overarching constraints inherent in the design, implementation and management of SO3. While there are field managers in each county, two additional managers for each IR does not afford the field manager to direct and monitor activities for fear of counter reaction from the two IR Managers. This issue has to be addressed if staff are to support each other effectively. Field coordinators have to be empowered to oversee and direct implementation of all SO3 activities. To support this, one manager has to also be placed at central level at LAUNCH headquarters to be referred to as education manager for both IRs. Several issues were raised as constraints in achieving results under SO3 at both field and community levels, these are: LAUNCH MTE – Final Report ©Moneval Solutions 2013 Page 59 of 113 Field Level: At the field office level, staff members involved in the implementation of activities stated the following:  Heavy centralization of procurement procedures have caused delay in the provision of inputs such as cement for school expansion;  Limited capacity of existing staff implementing the program;  Inadequate number of staff at field level. (there are simply not enough staff to work in say 40 schools in each county which would be a requirement); and  Most teachers in vulnerable schools are volunteers and are not paid. From discussions with County Education Officers, other generalized constraints identified included:  Slow delivery of inputs to schools by LAUNCH;  Lack of involvement of education officers by field staff;  Education Officers are often asked for information for use by LAUNCH without education district priorities. Schools which are priorities for the District Education Officer (DEO) are not considered by LAUNCH field staff;  According to DEO, external opinion formers, they are never part of any planning of activities carried out in schools;  Varying technical competence of LAUNCH field staff in the field of education;  No two-way communication with field managers but only field workers; and  The Tracking Board provided for schools by LAUNCH for exhibiting plans for schools is new and useful but needs rewording for clarity of messages. This will curtail ambiguity in information transmitted. Community Level: At the community school level, discussants during FGDs identified the following as constraints although these appear to contradict some of the general findings from the MTE:  Delay in providing inputs;  Poor (irregular) visitation by LAUNCH staff; and  No consultation of rejected items by communities. Items provided are sometimes rejected for quality reasons by communities in some cases and no further consultations takes place for replacement. This was articulated during PTA discussion in Kpakolokoya-Ta referring to their supplies for the renovation of their schools. Cutting across all discussions with communities and education officers, were issues of slow delivery and irregular visitation by field staff. For field offices, procurement procedures have been identified as a main cause of slow delivery of inputs to schools and communities. LAUNCH MTE – Final Report ©Moneval Solutions 2013 Page 60 of 113 7. CROSS CUTTING THEMES 7.1 Gender EQ: Has the program adequately integrated gender in its implementation? LAUNCH has successfully responded to ensuring that gender sensitivity is well captured in SO1, So2 and So3. The MTE team did not receive any negative comments nor were there any observations demonstrating a lack of gender awareness. Within its Strategic Document, LAUNCH has given attention to agricultural extension activities which are interesting and attractive for both women and men. FG membership figures show an almost equal share between female and male members. EQ: Is the program collecting and reporting gender disaggregated data? Some disaggregated performance data have been collected. It is doubtful that disaggregated monitoring data would provide added value at this stage. Tangible effects of the project are still insignificant both for men and women. There will be a need to collect this data once better market linkages are established and both significant farm and more improved farm management practices are adopted. There are some indications that women are quicker in starting cash crop farms. Disaggregated data regarding FG membership, however, should continue to be collected in order to monitor the share of women in FG and in leadership positions. EQ: What have been the successes and challenges in promoting women's leadership in farmers groups? Female AEAs have contributed to having a balanced share of women actively participating in the FG and also in leadership positions. It should be noted, however, that the main deficiencies towards SO1 are not primarily related to gender issues. Male members of FG were also underlining the positive role played by women in FGs. They seem to provide additional stability to the group. EQ: To what extent are women leaders of farmers groups exerting their leadership? FGs are still quite loose organizations, but women members are considered to provide strength and cohesion to their group. Most SUSUs (Credit and Saving Groups) are headed by women who demonstrated a careful management of their resources. Women seem to be the most active in the SGs, also with respect to credit, although the volume of savings and credits in those groups is still very low. In all FGs visited, women formed part of the leadership structure occupying positions such as vice-chairperson and secretary. EQ: What is the quantity and quality of women's leadership roles and capacities in these LAUNCH activities? In all FGs visited there was at least one woman in a leadership position in a SG group, four out of six FGs visited were headed by a woman. So far there are only a few major decisions taken by a FG, which makes it difficult to assess the role of women. Lead Mothers, GCHVs and health and nutrition staff utilize various strategies to engage men in the improvement of child health and nutrition. One strategy that is widely used is to invite and engage men during neighborhood mother discussions that are greatly impacted and influenced by the attitudes and behaviors of men, such as family planning. This is a strategic and effective approach to prevent discord at home and to increase the likelihood of uptake. It is not clear if this is a LAUNCH MTE – Final Report ©Moneval Solutions 2013 Page 61 of 113 programmatic strategy and promoted throughout the program or something that individual Care Groups are doing as a result of experience. Most Lead Mothers report inviting men to neighborhood discussions, although, it is not clear how their participation is documented or reported. FGDs with household mothers indicated that women are sharing the information that they learn from the neighborhood lessons with their husbands. When asked, how they know the men are listening, the women report a variety of examples including: their husbands are asking why they have changed the way they are doing things, that men are washing their hands more and men, who are more likely to be literate, read the baby health books provided to clinics by LAUNCH. Some report men setting aside money for delivery of the baby and encouraging wives to take sick children to the clinic. A MTE review of Lead Mother registers revealed there are men who are participating in the lead mother led neighborhood meetings regularly, both unofficially when invited and officially as members. There appears to be no mechanism to document or report this participation as part of the regular monitoring system. Conversations among staff indicate that female staff members may not have the opportunity to practice EBF while they are working as they promote to others due to work requirements. Lower level and non-health staff may be less informed as to the benefits of EBF and that proper nutrition is important for long-term child development. There is gender sensitivity within employment in LAUNCH. However at the field level, SO3 staff are all men. According to discussants at field office level, there is no document on gender in use, yet. Females were found working in the field but were working with other SOs. In their work, staff said they encourage participation of women in community work. Gender balance is ensured in PTAs through awareness raising by staff. This is so far the only attempt is for reaching for the participation of women at the implementation of SO3 activities. Most women occupy nominal positions in the school system. For all PTAs there were only two instances, one in Bong and Nimba each where the head of the PTAs was a woman. Women and men, according to discussants, are encouraged to work together but women play the role of advisors to community members to encourage them to send their children to school. For youth groups, LAUNCH ensures equal participation of both boys and girls for enrolment into the program. There is high retention for girls in the program. Discussants articulated however, that more boys dropped out after enrolment. There is no empirical reason for the reason of drop out that was articulated during the FGDs. However, a number of discussants pointed to the lack of interest and high un-managed expectation by boys as possible reasons for the dropout rate. There is a need to ensure the participation of boys firstly by conducting in-depth discussion with boys. 7.2 Sustainability EQ: Are the outcomes related to the adoption of better practices sustainable i.e. participants are likely to continue after the project ends? The practices introduced by the project regarding improved crop management and those to be introduced in the future regarding marketing and post harvest management will most likely be continued after the end of the project. These components are well adapted to the farmers’ reality. LAUNCH MTE – Final Report ©Moneval Solutions 2013 Page 62 of 113 EQ: Which outcomes are likely or unlikely to be sustainable, and why? The following risks regarding sustainability were identified and which contrast with crop management activities:  FGs may not be sustainable as they do not have a clear, common goal which provides tangible benefits to its members after the end of LAUNCH;  SUSU existed before LAUNCH; many are just seeking support from LAUNCH although they are still vulnerable. SGs need to reach a certain level of savings and generated some earnings before a statement regarding its sustainability can be made; and  DMCs do not stand a chance of being sustained if they are not officially recognized by local authorities and also not regularly supported by development projects. EQ: Are the initiated activities likely to continue after donor funding ends? It has to be recalled that LAUNCH is a stand-alone project which is not embedded in GoL structures. For the time being there is little prospect that LAUNCH extension work will be continued by GoL staff after the end of the project. It is quite likely, though, that a good number of AEAs will be contracted by some other project which will be active in the region after the end of LAUNCH. Some of the SG may well continue beyond the life of the LAUNCH although it is unlikely that farmers who were trained will continue to train other farmers in better farming practices and techniques. The LAUNCH strategy is well adapted to local conditions and is in line with GoL food security objectives and goals. Some of the targets under IR1.1 have been achieved but actual progress versus IR1.2 is still lagging behind. For the time being project effectiveness is still limited by the shortfall in numbers of FGs and members, and the low attention which the program gives to private farmers’ plots. The promotion of vegetable production seems to have the best prospect whereas effects of improved cassava cultivation techniques are not yet visible. This is due to the lack of feasible marketing opportunities and investment in processing has to be approved by LAUNCH/USAID. This would also imply stronger integration of SGs within the FG. Effectiveness of DMCs with respect to SO1 is negligible. Day-to-day project coordination is decentralized and efficient. Prospects for the sustainability of results at FG and farm level (with respect to food security) are good, once the bottlenecks regarding effectiveness are solved. Institutional sustainability of the provided services has not been foreseen and will most likely not be achieved. In the short term, LAUNCH will achieve its goal of improving the nutrition of PLW and CU2. The adoption and uptake of EBF as well as PLW knowledge of the benefits of using CSB indicate LAUNCH will achieve its goal. The mothers attribute many positive changes in their children to their participation in the program. PLW report changes in their preventative and curative clinical services and even report changes in the behavior of their partners. However, a concern for the long term uptake of behavior, specifically the complementary feeding of children six months to two years, as well as the importance of the nutritional needs of children between ages two and five years are concerning. The poor understanding of the impact of the mother’s change in behavior and feeding practices is of particular concern. Additionally, when women report adoption of the promoted LAUNCH MTE – Final Report ©Moneval Solutions 2013 Page 63 of 113 behaviors, it is as if it is done for the benefit of LAUNCH and they are doing LAUNCH a service, rather than for the improvement of their children’s health. Although there were some hurdles during community and school collaboration with LAUNCH, discussants expressed several good practices and have learned much. PTAs and school authorities stated that:  PTAs are now in charge of calling regular meetings of their members instead of just by the principal;  The knowledge of ECD has helped communities to respect children;  People and communities now do things for themselves which never worked before; and  The collaboration between PTAs and school authorities has improved to the extent that members make regular visits to the schools. Beneficiaries are of the opinion that several activities will continue even when the funding ends. Some of the key activities include:  Record keeping and tracking will continue in all schools;  PTAs will continue to perform their roles with regular meetings of PTAs to discuss school matters even after LAUNCH ends;  Proper care for children within LAUNCH communities will continue due to the ECD training; and  Lesson planning by teachers will continue to be practiced. PTAs (especially in Nimba) were proud to say that the new annex constructed with support from LAUNCH will be a mark of remembrance for the organization. Youth groups also said that they will continue to put into practice the knowledge and skills acquired from the various training they underwent. 7.3 Coordination and Integration EQ: How are activities of this objective related to those of other objectives? From a food security perspective integration across SOs has been inadequately planned and implemented. There appears to have been little consideration of nutrition in the selection of crops. Health and nutrition staff report conducting health education sessions with farmer groups, but the timing, coordination with Care Groups or general structure is not clear and appears to be uncoordinated. There appears to be no interaction between education efforts and health and nutrition efforts. In particular, there is no inclusion of ECD activities beyond nutrition in the Care Groups. Household mothers do not have sufficient information on the connection between nutrition and their child’s long-term mental development. LAUNCH MTE – Final Report ©Moneval Solutions 2013 Page 64 of 113 While there has been training on documenting success stories, few staff report attending these and there appears to have been an inadequate cascade of the training. Staff report having had little training on how to identify, document and share innovations or best practices. EQ: What effort should be made to ensure communities benefit from all of LAUNCH activities? EQ: What can be done to further integrate the education component with the other two components? EQ: What opportunities are there for further integration of program components to maximize impact? There are attempts to ensure that SO3 integrates with the other SOs. The first attempt is that communities for S03 intervention are selected based on the presence of other SOs. Field staff agree that there are numerous opportunities for integration. There have been many instances when staff of other SOs are invited to attend workshops conducted by other SOs as well as to conducting a joint assessment in communities identified by LAUNCH24. Additionally, the future implementation of the School Feeding Policy of the Ministry of Education will provide an opportunity for addressing issues of stunting, food and retention in schools as envisaged by the policy. 7.4 Monitoring and Evaluation EQ: Is the M&E system organized and effective? The M&E system is well organized and is in a position to cover all monitoring tasks. Its effectiveness cannot be rated yet as long as project progress is still slow and widespread tangible benefits have not been monitored yet. Its efficiency with respect to S01 could be improved if data collection of agricultural production data would be left entirely in the hands of the AE staff. EQ: Are M&E data used for management and reporting purposes? The Performance Monitoring Plan (PMP) is considered a comprehensive document with indicators meeting the criteria of being specific, measurable and also mostly relevant (SMART). Some of the indicators which are reported upon, however, do not seem to be achievable as they are too ambitious (see 4.2.1.1). PMP indicators related to the household food situation will only be collected at the end of project and will not be useful for making management decisions. Other indicators are updated annually or quarterly and mostly provide useful information to take corrective action by LAUNCH management as well as for regional coordination. The project has set up a monitoring unit at HQ level and has three people in each of the County offices. This is considered fully sufficient to meet the monitoring requirements and provide useful information to management and ACDI/VOCA HQ. 24 ‘‘The best way to integrate is to conduct joint assessment in communities identified for launch intervention‘‘, (Nimba County Coordinator). LAUNCH MTE – Final Report ©Moneval Solutions 2013 Page 65 of 113 A review of the tools used to monitor health and education activities was carried out. The tools provide a great deal of information on the skills needed to conduct a participatory learning session, including engagement of participants, eye contact, respectful administration of the lessons, but very little on the accuracy of the information given. It is unclear how the tools might be used for program improvement or identification of challenging module components. LAUNCH health and nutrition M&E forms completed during module training, monitoring visits and Lead Mother performance visits are used to collect information on the timing, administration of the sessions as intended. However, mechanisms to collect and report programmatic impact and behavior change are less clearly outlined in the forms. Additionally, the forms do not allow sufficiently for the assessment of the quality and accuracy of the content. It is not clear how the data collected would be used to improve the content and accuracy of the lessons. It is unlikely that the forms would allow data collectors to assess the changes in the accuracy of training over time. EQ: As defined and measured, do the performance indicators provide useful and reliable data on program progress and outputs? The quality of the Annual Monitoring Survey is adequate with respect to the achievement of indicators and in providing useful data regarding the food availability and access. Some additional information regarding FG participation and performance (MIS EG.5), especially on individual plots, needs to be collected more often than anticipated in the PMP and corresponding information should be gathered by the AEAs. This data could lead to corrective action on behalf of LAUNCH management regarding their annual plans and selection of communities. 7.5 Disaster Reduction and Early Warning System EQ: Has the program allocated enough resources for the DRR component of the program? The MTE team visited three Disaster Management Committees (DMC) in Bong County. These groups consisted of some five members who received specific training from LAUNCH, mainly on household waste disposal issues and mitigating mosquito breeding points. They also consider themselves responsible for monitoring drinking water issues. In view of the low cost effectiveness of the DMCs there is no need to allocate additional resources to these committees. It was highly ineffective to allocate resources from the SO1 budget to the DRR component. This is because as long as the committees are neither relevant as they still don’t count with community appraisal nor do they have legal status, nor would they contribute to improved food access and availability. EQ: Are DRR committees that LAUNCH has helped to set up functional? EQ: Is the early warning system relevant and appropriate to the current food security context? DMCs do not have any recognized mandate regarding early warning food shortages. The groups are not trained to initiate measures in order to effectively reduce damage to food production (e.g. from an invasion of ground hogs, goats, birds and insects), neither do the members have the capacity to detect them. The only methods in place for dealing with ground hogs and goats are fences and against bird attacks and people use slings. No innovative methods have been developed yet. Almost no crop damages have been reported across the program region due to flooding, draughts, locusts, insect or other plant diseases. According to their understanding, DMCs have an advisory role only LAUNCH MTE – Final Report ©Moneval Solutions 2013 Page 66 of 113 regarding the aforementioned threats. They and need to have well established links with local authorities in order to get help in eradicating potential health hazards such as mosquito breeding pits, inappropriate household waste disposal. Good relations with local chiefs have not always been established. Some additional resource allocation (such as through training) may help the DMC become better prepared although overall this areas is not a core element within any LAUNCH SO. The MTE has not been informed about any major food security problem which has been or could be solved by an early warning system managed by the DMCs. The DMC contribution regarding the achievement of SO1 remains marginal to building the capacity of focus communities. The Committees are rather contributing to SO2 EQ: How can we improve this allocation? DMC members mentioned that they would like to receive training in health issues and would like to provide medicine to their communities instead of having more training on food related issues. 7.6 Infrastructure LAUNCH received approval from the Ministry of Public Works and USAID to rehabilitate a 10 km road linking Zukarzue to Mao in Nimba County. However, the program was later notified by the Ministry that another partner received a higher level of funding to rehabilitate this road beyond the stretch proposed by the LAUNCH program. In coordination and collaboration with the Nimba county resident engineer LAUNCH has identified 3 bridges to be rehabilitated in Year 3 of implementation and initiated sensitization meetings with all targeted communities. MOUs are going to be signed in 2013.The problem of bad road access was always mentioned in the FGDs as one of the major obstacles to rural development in both counties. But it seems doubtful that LAUNCH will be able to get any effects from its infrastructure support during the project lifetime. LAUNCH MTE – Final Report ©Moneval Solutions 2013 Page 67 of 113 8. COMMODITY MANAGEMENT LAUNCH targets the direct distribution of 7,700 MT of rice under Title II including vegetable oil, split yellow peas, bulgur and CSB as food rations to PM2A families. During the same period the program envisages the monetization of 34,430 tons of commodities during its life25. The amounts have been based on technical justifications provided by FANTA26 and estimated requirements for implementation by LAUNCH management. All Title II commodities bound for Liberia are shipped to Monrovia which has adequate facilities for the discharge of such cargoes. Management and distribution of commodities involves complexities that are not entirely within the control of LAUNCH management and is manifested on several dimensions27 . 8.1 Commodities Receipts and Storage Prior to the arrival of the commodities at the port, ACDI/VOCA meets with the clearing agents and their recruited stevedores to provide an orientation for all the necessary paper work and authorization. ACDI/VOCA conducts Discharge Surveys for both Direct Distribution and Monetization commodities to account for any discrepancies between the Bill of Lading and the amount discharged. Security at the port however has been a problem with a high propensity towards theft and the loss of a substantial amount during the first year of imports. Commodities for monetization are handed over to buyers directly at the port on the basis of an at￾sight Letter of Credit or an irrevocable financial guarantee as means for payment. This reduces the risks associated with the discharge of monetization commodities. Although there is no direct Government involvement in actual monetization, the Ministry of Commerce exercises a strict control on rice prices and permission has to be secured before the commodity arrives in the country. Given the variability of the sales price, ACDI/VOCA has held meetings with other organizations planning monetization sales, including Food for Progress Award recipients, to coordinate call forward dates. Commodities for direct distribution are transported to one central warehouse in Monrovia for storage until direct onward distribution to beneficiaries. Warehouse storage adheres to ideal warehouse management principles of proper commodity spacing and stacking, security and safety including pest control. Commodity storage follows approved PERSUAP guidelines with two additional conditions pertaining to monitoring of gas and appropriate training of staff. Losses from stored commodities are reported to be minimal. 25 The amounts reported here reflect the quantities requested in the Technical Proposal of the project to USAID. 26 FANTA-2, ‘’Title II Technical Reference Material: TRM – 01: Preventing Malnutrition in Children Under 2 Approach (PM2A): A Food assisted Approach (Revised November 2010) 27 The SOW for the evaluation did not allow an in depth assessment of commodity management with conclusions resulting from only a “rapid” assessment being made within the time frame and data available. LAUNCH MTE – Final Report ©Moneval Solutions 2013 Page 68 of 113 8.2 Transportation and Food Distribution Management The project has no secondary warehouses in the field. Containers at each Food Distribution Point (FDP) serve as a field storage facility and efforts are made to keep the amount at these facilities at a minimum given the susceptibility of losses. Commodities for direct distribution are transported by trucks once a month for distribution to the 44 FDPs in the two counties (20 in Bong and 24 in Nimba). The transportation is undertaken in LAUNCH owned trucks but private trucks can be used when necessary. While there are no major problems during the dry season, the rainy season is a major constraint for food commodity deliveries as roads become impassable. It is also a constraint for beneficiaries who walk to the FDPs. Undistributed commodities at each centre are brought back to the field storage facility. Food distribution at the 44 FDPs in the two counties is coordinated through a Food Distribution Committee. Registration and distribution of food commodities to beneficiaries has been facilitated by the use of electronic data collection through smart phones at these points. A selection system among beneficiaries based on a number of parameters including the beneficiary condition and the distance they have walked to the distribution centers, among others, provided a degree of need based system of allocating priority. While this has worked in general, there have been cases of violations and/or stoppages of distribution before the scheduled time. To ensure reliability in data collection, especially for registration for food rations, LAUNCH has implemented a commendable electronic data collection system. This was previously this was paper based, and subject to challenges of lost information, data entry errors. In March 2012 LAUNCH has, implemented a smart phone based transfer of data through preset forms on mobile phones directly to the internet for immediate view and use. Food distribution is managed through a complement of approximately 40 food distribution staff divided into two sets. One set deals with just the hand-over of commodities and the other deals with beneficiaries. The quality of staff has been difficult for the project and all were provided training. Systematic and routine monitoring has been a regular feature with a quarterly review of activities to assess status. This is supplemented by end use monitoring (each December/January). For instance, CSB was found in the markets of Nimba. It is however suspected that it could be from distributions made by the WFP. 8.3 Commodities, Bellmon Analysis and Monetization Of the three main staples, cassava, rice and bananas, rice is the most traded crop. Rice production however is unable to meet current market demand and overall rice consumption is heavily dependent on imports. Based on market analysis undertaken by ACDI/VOCA, local production reached 121,755 m/tons (in 2008) representing approximately 37% of total consumption. The analysis suggests that over 10,500 m/tons of rice could be monetized per year without creating market distortions. Reports published by the Ministry of Commerce and Industry (MoCI) indicate that commercial imports in Liberia were at about 292,000 m/tons of rice in 2011 of which about 175,000 m/tons were consumed leaving the remainder for market sale in 2012. In terms of yearly monetization during year 3 of the program’s implementation, the monetized quantity of 4,540 m/tons of rice represents about 1.5% of total anticipated commercial imports. The quantities of commodities including vegetable oil, bulgar wheat and split yellow peas are within the recommended levels as per Bellmon analysis. These items have no impact on local production and market and pose no pressure on public storage capacity. LAUNCH MTE – Final Report ©Moneval Solutions 2013 Page 69 of 113 ACDI/VOCA uses competitive tenders for the monetization of rice from a list of licensed importers provided by MoCI while vegetable oil is tendered to the general public. However, the Ministry exerts a tight control on the quantities and sales price of rice, essentially providing a ceiling price for different rice qualities to avoid social upheaval as a consequence of high prices. Vegetable oil, on the contrary, while widely consumed, is less tightly controlled and not subject to a ceiling price. Wheat is not a regulated commodity and prices generally reflect international prices. The dictated sale price by MoCI results in the fact that the average rice cost recovery was approximately 78% against an anticipated cost average recovery of 88%. Similarly, vegetable oil could generate a cost recovery of only 61.48% against an anticipated 78% cost recovery28. Only wheat was able to bring in the anticipated cost recovery of 71%. The proceeds are thus substantially less than anticipated and the project faces a problem of resources to continue implementation. Although the project has found cost savings, with no changes expected in the pricing of rice, the program could face a problem of resource availability to cover its expenditures. Added to the above issue has been the fact that Indian production and sales of par boiled rice has entered the market at about half the USAID prices of US rice. In addition, it has been reported that Japan has also donated a substantial amount of rice to Liberia. The political economy of rice in Liberia indicates no change in the Government policy of dictated or ‘suggested’ prices in the immediate and medium term future. The program consequently faces a real issue of resource availability to cover its expenditures without a resolution of the price anomaly or without an infusion of additional financial resources to meet operational expenditures. 28 The program achieved a 74.78% and 70.63 % cost recovery on sales of rice during the first year. In the second year, the program achieved 82.84 % cost recovery on rice sales. Similarly, with respect to vegetable oil, in the first year, the program achieved a 61.48% cost recovery while in the second year the program achieved a 69.91 % cost recovery. LAUNCH MTE – Final Report ©Moneval Solutions 2013 Page 70 of 113 9. LESSONS LEARNED The following key lessons have been identified through the MTE: Lesson 1: Farmers are keen to Improve their Smallholding Practices The vast majority of small farmers in the region show great interest in adopting new techniques for their food crops and have a strong interest to change their cropping patterns. It has been learned, however, that the traditional techniques are well adapted to the local conditions. They are hard to compete with by improved management practices unless investments will be made available for land preparation, cultivation, and harvest or processing. Lesson 2: Farmers are optimizers rather than maximizers Farmers in both Counties seem to look to optimize their labor input when considering starting a new farm or to start growing new crops rather than trying to achieve a maximum production per hectare. Lesson 3: Female involvement is very high in the LAUNCH program, higher than similar food security projects in Asia There is a much stronger and more active involvement of women within FGs in Liberia compared to FGs related to similar food security interventions in Asia. Lesson 4: Monitoring performance data needs to be balanced against the performance The effort for monitoring performance data needs to be balanced against the performance progress which is being monitored. This means looking at balancing what is effectively high resource allocation against the benefits of the program. Lesson 5: Cost recovery for monetized commodities Where the price of monetized commodities is controlled or highly influenced by Government there should be provisions for alternative disposal sale of commodities. This may include the sale of the commodities in third countries; this should be actively monitored and implemented when necessary. This will serve as a source of raising resources to meet operational expenditures. Lesson 6: Integration of implementation activities with other interventions of other agencies County offices should collect information on other organizations working in the same location area including organizations such as WFP, Africare and African Conservation Foundation (ACF). County management staff should begin developing relationships with these organizations to identify opportunities to leverage and/or complement LAUNCH activities across sectors and benefit from shared resources and information where possible. Lesson 7: Independence of the evaluators: It was pointed out by USAID staff in Monrovia that there is a need to reorganize the USAID contracting and implementation procedures for MTEs to ensure the independence of the evaluation LAUNCH MTE – Final Report ©Moneval Solutions 2013 Page 71 of 113 team. The fact that the program implementing agency is also the contracting agency creates a potential conflict of interest both within project management and the MTE team. LAUNCH MTE – Final Report ©Moneval Solutions 2013 Page 72 of 113 10. RECOMMENDATIONS The following recommendations are based on the country work of the MTE. The recommendations have been categorized as ‘priority’ and ‘secondary’ for each SO and Cross Cutting Themes. Recommendations are by and large budget neutral and should be implemented as soon as possible. The exception is recommendation 3, 7, 17 which may have budget implications. The parties responsible for taking action are identified within the description. 10.1 Priority Recommendations SO1 Recommendation: Extension of project intervention area (USAID, ACDI/VOCA, LAUNCH Management) LAUNCH Management should analyze whether it is feasible to increase the number of villages attended and/or the number of supported FGs and beneficiaries of the project in the next two years. This is intended to compensate for the most likely small benefit generated per head of FG member who is participating in the program (please refer to section 4.2.1.4 for details). Likewise alternative measures should be included into the yearly planning in order to attract more women as members of FGs as they have a stabilizing effect on those groups. SO1 Recommendation: Hands on management (ACDI/VOCA, LAUNCH Management): A more hands-on management of component SO1 is recommended by looking for staff with farming backgrounds and knowledge about extension work, shifting cultivation techniques as well as risks, in view of the problems with low productivity of food producing farms in Nimba and Bong. (Please refer to chapter 4.2) SO1 Recommendation: Launch Agricultural Extension Strategy Document (Agricultural Extension team): A comprehensive extension strategy needs to be developed defining: a) the type and content of LAUNCH assistance to the FG after year one; b) the type and volume of assistance provided to individual farmers plots of FG members; and c) the possible support to commercial farmers associated with a FG as well as support to processors and traders associated with FG (Please refer to 4.2.1.10). For the time being extension activities are limited to FG plots and FG activities. With respect to Saving Groups it needs to be decided whether or not savings groups shall be an independent instrument or whether they should be integrated into FG as it can lead to conflicts if non FG members are deciding about the savings of the FG. The Extension Team should look for assistance in the area of agricultural production systems (for example through cooperation with a University) in order to investigate additional crops and production techniques suitable to be introduced in the project area with higher return on farm labor input (Please refer to 4.2.1.3). It is suggested that some AEAs receive special short term training regarding medium term farm planning under shifting cultivation conditions, management of food production risks (please refer to section 4.2.1.10 for details) SO1 Recommendation: Funds for food processing and marketing (USAID, ACDI/VOCA and LAUNCH Management): Considerably higher funds should be budgeted for investments in processing equipment or agricultural machinery services or investment in material for chicken cooperatives. It is suggested that an investment budget of some USD 1000-2000 be foreseen for each group which has successfully completed the year 1 exercise (please refer to section 4.2.1.10 for details). LAUNCH MTE – Final Report ©Moneval Solutions 2013 Page 73 of 113 Part of the funds assigned to the design and implementation of some of the monitoring activities (such as creation of jobs, total food production etc.) could rather be assigned to finance FG small investment projects in order to assure more effectiveness. SO Recommendation: Food processing and marketing (LAUNCH extension team): More processing, marketing and management skills will be needed in the extension team in order to provide follow up assistance to the FG after year one. It is suggested that some AEAs will receive special short term training (Please refer to 4.2.1.10). Training programs for interested FGs on food processing and marketing should be initiated immediately in addition to those for agricultural extension, savings and credit groups and post harvest handling. A suitable schedule will also be needed as FG leaders are required to take important decisions such as what to do with the harvest. Corresponding training has to be given well in advance in order to provide FGs with the necessary skills to make informed decisions (Please refer to 4.2.1.7). SO2 Recommendation: Health Training: LAUNCH Health and Nutrition and Education staff should be trained in the importance of ECD and ways to engage mothers in ECD, including how nutrition impacts brain development long-term. LAUNCH should incorporate mother-led ECD strategies into the Care Group sessions, articulating why this is important in a way beneficiaries can understand and adopt. Strategies include:  The importance of communication and interaction with infants and children;  Ways to interact with children and infants for brain development and growth (for example promoting singing to infants, talking to them, helping children count or match items, and so on); and  How nutrition and brain development are interconnected and critical for future achievement and development. SO2 Recommendation: Behavioral Change: Neighborhood mothers attribute changes and improvement in their children’s health to CSB, few linking the progress to their own actions and appear doubtful that they would be able to make changes solely using locally available foods. In the time remaining in the program, LAUNCH Health and Nutrition staff must help mothers to connect their actions with their children’s improved health. They must help the mothers to understand why this age group is critical and to disassociate to some extent with the focus on the CSB but rather on the improved nutrition and one’s own actions. This requires additional education sessions, a consistent message and strategy to be developed by the Health and Nutrition staff and management. This message should be developed immediately, taking into consideration best practices and lessons learned in the program as well as from other programs identified in the IYCN project. Health and nutrition staff should be immediately trained by management to monitor, identify, document and solve problems as they arise. Staff should be held accountable and recognized for problem solving as well as documentation of successes so that the message can continue to be used by other health workers. SO2 Recommendation: Lead Mother Retention/Motivation: In some areas, Lead Mother participation is waning. LAUNCH should incentivize Lead Mother participation with regular and ongoing recognition, sufficient working tools and other intangible incentives, such as community recognition and personal pride; this is of particular importance as Lead Mothers begin to graduate LAUNCH MTE – Final Report ©Moneval Solutions 2013 Page 74 of 113 from food benefits. Health and nutrition staff should begin to routinely ask what Lead Mothers have accomplished that has made them proud during routine monitoring. This strategy will help Lead Mothers to recognize their achievements, helping to reinvigorate and maintain their participation as Lead Mothers, while allowing health and nutrition staff to monitor and document success stories. In particular, LAUNCH should work with local communities to ensure that the communities are aware of the contribution of Lead Mothers in improving the health of their children. As the government is reworking the role of TBAs, LAUNCH may be able to assist in the identification of a long term role for lead mothers, in supporting the work of the government. Lead Mothers and/or their households should be systematically included in farmer groups and other activities. Provision of working tools: Lead Mothers have been told by LAUNCH they will receive working tools such as lappas and t-shirts to indicate they work with LAUNCH. Identifying items are both a working tool and an incentive as it helps the Lead Mother’s efforts be recognized by the community. Delivery of these working tools should be expedited and Lead Mothers informed of the expected timing of delivery. Health and nutrition management should ensure that they prioritize this roll-out and hold staff accountable for the timely delivery of such materials. SO2 Recommendation: Care Group Lessons/Improvement: Health and Nutrition staff should immediately begin to educate Lead Mothers as to why CU2 and their nutrition are important. Without proper understanding as to why this age group is important, the cut-off seems arbitrary and unfair and compromises the likelihood of long-term sustainability of the achievements. Health and nutrition management staff should ensure that staff, have a consistent understanding and message on the importance of this age group based on evidence the “First 1,000 Days Approach” and how to plan the intervention exit. Care groups lessons should include how mothers can sustain the children’s physical achievements after they have graduated from food rations. SO2 Recommendation: Appropriate Feeding: Exclusive Breastfeeding: FGDs indicate EBF is widely practiced. Mothers report changing behavior and beliefs about EBF. By the end of 2013, LAUNCH should begin to verify, document and analyze in depth the changes in breastfeeding practices. In particular, this should include the changes in colostrum use to highlight the achievements of the program as well as to help other organizations and individuals learn from LAUNCH achievements. Complementary Feeding: LAUNCH appears to have had significant impact on promoting and normalizing EBF. LAUNCH has taught women the components of appropriate complementary foods in the event that CSB is not available for the 6 month to 2 year olds, however mothers appearing not to have the self-efficacy to have a greater impact on their child’s nutrition. LAUNCH should immediately expand its efforts to focus nutrition education and self-efficacy on years 2 – 5, so that program achievements are sustained and the program goal is realized. Among activities to focus on, LAUNCH health and nutrition staff must increase household mother (and father) understanding as to why this age group is important, increase her self-efficacy to make a positive change in her child’s nutritional intake and to promote the growth and utilization of additional diverse foods, through activities such as household gardening. Strategies to increase the adoption of this behavior may include:  Identification of barriers and helpers to adopting the behavior, including cultural, monetary and social. Using findings and recommendations from the integrated social behavior change strategy developed in May 2012. Based on these barriers and helpers, the health and LAUNCH MTE – Final Report ©Moneval Solutions 2013 Page 75 of 113 nutrition management should initiate a strategy to train, monitor and document health and nutrition staff implementation experiences of the strategy;  Inclusion of pictorial representation of promoted behavior including/emphasizing a session in the Care Group such as the promotion of dietary diversity using locally available foods could be considered;  Cooking demonstrations for proper feeding of 2-5 year olds;  Increase education on the importance of proper feeding for this group and connecting it with future achievement, which appears to have influenced household mothers changes in EBF and complementary feeding; and  LAUNCH health and nutrition management and team leaders should begin a dialogue with the local health authorities and the Ministry of Health to develop common terminology to address the confusion between fortified/vitamin rich foods and medicine. This is critical to both the program as well as the future efforts of the local health authorities to combat under-nutrition and stunting in the areas. Rectifying this misunderstanding could impact the self-efficacy of mothers to change their child’s nutritional status. SO3 Recommendation: Management Modalities: Field coordinators should be empowered with full oversight responsibilities to oversee and direct implementation of all SO3 activities. The SO should be managed by one manager in conjunction with county coordinators. To support this, one manager has to also be placed at central level at LAUNCH headquarters to be referred to as Education Manager for both IRs. A single manager for the SO will enhance coordination of field activities. This will also provide clarity on the activities and expectations of the field coordinators. Additionally, accurate planning and reporting will be enhanced thus supporting timely and effective decision making. SO3 Recommendation: School Improvement Plans: To continue the support provided to schools LAUNCH staff should work with communities to produce plans for school improvement with further support on implementing the plan. There are plans by the MoE with support from the World Bank to provide grants in all schools by 2013. The utilization of such grant will depend on ability of school authorities and communities to produce a school improvement plan. LAUNCH could begin the process of training communities to develop school improvement plans to help prepare them for successful grant applications. SO3 Recommendation: The ECD training had a positive impact on the communities by helping to change the behavior of parents towards their children. Corporal punishment is reduced al all communities visited. They are now communicating harmful effect of child abuse. This gain can be sustained by the full implementation of the code of conduct in schools. LAUNCH should take advantage of the existence of the Code of Conduct for teachers developed by the Ministry of Education and her partners. 10.2 Secondary Recommendations SO1 Recommendation: Revision of the IPTT (USAID, ACDI/VOCA and LAUNCH Management): It is proposed that IR1.3 should be eliminated from the results framework as it is misleading. It might be included in the IPTT as indicator for IR1.2 (“increased market linkages”) as it LAUNCH MTE – Final Report ©Moneval Solutions 2013 Page 76 of 113 can be assumed that only minor deliverables will be contributed by the project with respect to an improvement in infrastructure (Please refer to chapters 4.2.1.1 and 4.2.1.2). A modification to the IPTT and PMP regarding IR1.1 and IR1.2 should be considered once a decision has been made regarding So1 recommendation 1-5, also considering the time horizon. There are some modifications suggested with respect to the actual set of indicators in the PMP (Please refer to 4.2.1.2) in order to provide meaningful monitoring and performance data:  FFP MN.5 (“# of assisted communities with disaster early warning and response systems in place as a result of project assistance”) might be revised as it is not meaningful with respect to SO1 and “early warning effects” cannot be properly assessed; no indicator will be required under SO1 in case LAUNCH management decides to provide support to the DMCs under SO2.  AV OC 1.1 should be substituted with an indicator which is composed of a few typical assets and which can be more easily measured. The M&E team should define proxy indicators together with the field staff who have a good view on the items, which demonstrate relative wealth in both counties (e.g. mobile phones, type of roofing, and ownership of agricultural implements) SO1 Recommendation 7: Food production data: Monitoring department should analyze how to obtain more realistic data on food production in the project area using the experience of the AE staff. SO1 Recommendation: Rural sociologist: A short term rural sociologist should be contracted by the Agricultural Extension team to investigate the “potential” of selected communities and FG to adopt “LAUNCH” initiated practices and to provide a catalogue of selection criteria. SO2 Recommendation: PM2A: Commodity and health program staff need to have a deeper understanding of why the program is providing rations. This should include clarifying if the purpose is the program’s objective is driven by commodities or if commodities are driven by the provision of nutritional supplementation for the most vulnerable. This in turn will help them make informed decisions on eligibility and problem solving during ration distribution that considers the goal of the program as well as commodity accountability. Commodity and health program staff need regular and on-going training on the use and purpose of the rations. The JSI nutritionist has created a presentation on “Why Nutrition Matters: Integrating Nutrition into all LAUNCH Activities”. All staff should receive this training within the next quarter and be held accountable by LAUNCH management to ensure that the recommendations contained within are implemented. Commodity and health program staff need to work more closely together to solve discrepancies in eligibility and general problem solving issues. Staff should immediately embark on collaborative problem solving, especially for exceptional situations. LAUNCH management should ensure a standard operating manual for maintaining and documenting these procedures and that there is a transparent mechanism for resolutions. Food distribution points should be used as opportunities to reinforce the promoted health behaviors and messages. In the next quarter, WASH staff should consider the feasibility of travelling hand￾washing stations to reinforce the promoted behavior and to act as demonstration opportunities. LAUNCH MTE – Final Report ©Moneval Solutions 2013 Page 77 of 113 LAUNCH currently uses written signs to explain ration divisions, while accurate, such signs are ineffective education tools for a largely illiterate recipient group. LAUNCH should immediately print pictorial signage that specifies the distribution divisions for rations. SO2 Recommendation: Standardize the integration of outreach vaccinators: As the government is already doing vaccination outreach, where feasible, LAUNCH should standardize the integration of outreach vaccinators into the food distribution days. This will increase integration with local government efforts. Health and nutrition management should consider the frequency, timing and locations should be determined by appropriateness and feasibility. By the next quarter, plans to include additional vaccinators should be finalized and rolled-out. SO2 Recommendation: Water, Sanitation & Hygiene: FGDs indicate that LAUNCH is educating mothers about sanitation and hand washing. LAUNCH should roll-out the tippy tap demonstration and other hand washing station strategies immediately and should include Care Groups and FGs. WASH staff should then monitor the uptake of tippy taps, as well as the construction of other hand washing stations, including the reed method developed in Nimba. By the beginning of the next quarter, WASH and health and nutrition staff should be routinely monitoring, documenting and reporting on the uptake of hand washing stations by the community, beneficiaries and non￾beneficiaries. LAUNCH health and nutrition management should reconsider the plan to provide containers for household tippy taps. LAUNCH should instead provide instruction and encourage community members to provide their own, readily available containers and building materials. This is more indicative of understanding of the importance of the behavior change. Within the next two quarters, tippy taps and other hand washing stations should be demonstrated to all relevant local health officials such as the district health officer, county nutrition officer and maternal and health coordinator. This will increase the promotion of hand-washing and to increase the general awareness of low-cost and effective hand washing strategies, Within the next two quarters, demonstration tippy taps should be constructed at clinics and food distribution points for additional promotion outlets. Immediately, WASH and Health and Nutrition staff should jointly plan complementary activities to promote WASH in groups. WASH staff should immediately be educated on the hygiene behaviors promoted in Care Groups. The office in Nimba should immediately construct a hand washing station to reinforce the promoted behaviors. SO3 Recommendation: Field level involvement: Field Offices, especially for SO3 staff, should begin their involvement in schools by holding discussions with CEOs, DEOs and principals from the start. This comes from the numerous complaints from CEOs that they are not usually involved in planning activities for schools. SO3 Coordinators should always work with CEOs to identify professionals that will be involved in teacher orientation workshops. SO3 Recommendation: Decentralization of Procurement: Senior Management should consider the decentralization of procurement procedures to enhance faster delivery of inputs. It was an overwhelming agreement among field staff that procurement at central level is the main cause for delays in the delivery of inputs to beneficiaries. LAUNCH MTE – Final Report ©Moneval Solutions 2013 Page 78 of 113 SO3 Recommendation: Consultation with Youth: Field staff should always consult youth groups on the types of commodity items supported through the mini-grants. Youth groups know exactly commodities and products that sell in their areas. There were numerous complaints that some items bought by LAUNCH staff were “useless” and could not be sold at all during the first grant round. SO3 Recommendation: Avoid raising expectations: Field staff should avoid raising expectations of communities. There was one incident in one community in Nimba County where field staff had promised to deliver sports materials to students if the students could dig a pit large enough to bury garbage around the school environment. The delay in meeting the demands of the students was discussed during FGD Korsein Town, Gbor District. Students and school authorities were still awaiting response from LAUNCH field staff up to the time of the MTE visitation. SO3 Recommendation: Improve School Gardens: Greater focus on school gardens is essential as this can lead to improved food security. School gardens will ensure meals for students while in school. This is particularly relevant as the World Food Program does not cover every school (such as the three schools in Salala, Bong County). It has been observed that providing food increases attendance and retention levels, especially for vulnerable communities. Most students in these communities go to school hungry and addressing some of their food needs through their own efforts will complement their parents’ efforts. Providing agricultural inputs through extension workers to schools will improve this component of the program. SO3 coordinators should revisit school garden interventions along with S01 coordinators. There needs to be some clarity as to the level of involvement of SO1 as currently SO3 staff pay less attention to school gardens. This would lead to greater integration between the two SOs. SO3 Recommendation: Education Skills of LAUNCH Staff: LAUNCH management should make sure that SO3 staff have sufficient training in education, especially in pedagogy. Most County Education Officers felt that LAUNCH staff are not well equipped to work in the education sector. This permeates also to school principals who mostly listen passively to LAUNCH field staff as they are not viewed as educators by most school principals. It was discovered that most field staff implementing activities under SO3 have little knowledge of the field of education. Consequently, they will be of little or no help to teachers and education officers. SO3 Recommendation: Quarterly Reviews of SO3: LAUNCH management should conduct quarterly reviews of the SO3 plan to ensure that implementation of SO3 is on track. This review can be increased to semi-annual reviews as the capacity of staff and their understanding of the program develops. SO3 Recommendation: Field Coordinators should ensure the development and implementation of regular field monitoring visits. Mao community members complained of the long absence of staff from their communities before evaluators arrived. This community was one of the hard to reach sites visited during the MTE. SO3 Recommendation: Support to PTAs and Teacher Training: Provide continuous support to PTA and training of teachers. Support for PTAs will increase enrolment and ensure retention while the training of teachers and supply of inputs to schools will support the quality component and will eventually lead to an increase in the promotion rates. For PTAs, training in the development of school improvement plans and the provision of inputs to support such plans are essential activities to strengthen PTAs. At the same time, teacher training has to continue regularly as most teachers lack the opportunity to attend formal teacher training programs. LAUNCH MTE – Final Report ©Moneval Solutions 2013 Page 79 of 113 10.3 Cross Cutting Recommendations 10.3.1.1 Priority Recommendations Recommendation Commodity Monetization: Addressing the monetization problem of commodity sales through better coordination of imports with other FFP and/or Title II programs and other donors is unlikely to resolve the issue at hand. It is therefore suggested that USAID permit ACDI/VOCA to look at the possibility of third country monetization to meet operational costs. DRR Recommendation: Non-Active DMCs: As the DMCs (formerly DRR Committees) have proved to be neither relevant nor effective with respect to food availability and access. There was no interaction observed in the MTE between the DRR committees and the WASH committees. Consequently it is suggested not to support those committees under the SO1 component. M&E Recommendation with respect to SO3: LAUNCH Management should clarify what the SO3 seeks to achieve consistently. This will ensure clarity on what is reported on both quarterly and annually. The M&E Unit should develop realistic monitoring indicators to track the following:  Community capacities to support education;  Barriers to enrollment and retention in selected schools and communities;  Community solutions of problems in schools using local resources;  Youth entrepreneurship training outcomes in terms of business plan development;  Employability of youth that have skills for employment;  Creation of jobs for and by youth through entrepreneurship (Note: In Sanoyea youth groups give out loans to other youth as business startups); and Impact of training and mini-grants on youth livelihoods will require marketing support from LAUNCH. 10.3.1.2 Secondary Recommendations Recommendation: Integration to Deliver as one Program: There is dire need for the integration of all of LAUNCH activities. This will ensure maximum use of resources with greater impact on communities. Beginning with joint assessment involving all SOs, interventions can be planned to enable a single vehicle to carry a team comprising of all SOs for various intervention in a community. SO1 Recommendation: Coordination: In case of a positive decision to continue LAUNCH program there should be a strategy on how Farmer groups could benefit from SO2 activities and be provided with basic nutritional information on crops. This should also include the importance of dietary diversity to child health and growth in order to expand their knowledge on food and nutrition. Corresponding activities should be carried out once more progress has been made LAUNCH MTE – Final Report ©Moneval Solutions 2013 Page 80 of 113 towards SO1 objectives. Achieving this will require a long term arrangement with closer coordination and integration between S01 and SO2. Sustainability Recommendation: Work with government transitioning: LAUNCH health and nutrition management and team leaders should immediately increase their involvement with government structures, including reinforcing the purpose of supplemental food rations with government staff. Additionally, Health and nutrition management should begin reiterating the end of LAUNCH. With local field staff, health and nutrition management should begin to develop handover and transition plans. In particular, help the clinics develop a realistic strategy to take over the health education messages and strategize how Lead Mothers could be utilized in the future. Sustainability Recommendation: Long term success of LAUNCH: County offices should collect information on other organizations working in the area and what they are working on and where they are working including, WFP, Africare and ACF. County management staff should begin developing relationships with these organizations and identify opportunities to leverage and/or complement LAUNCH activities across sectors. Program planning should include strategies to leverage the activities of these programs for the long term success of LAUNCH. Sustainability SO3 Recommendation: Strengthen collaboration with local and national education authorities: The PTA coordination unit at the Ministry of Education (MoE) needs to be fully involved in planning of various activities with PTAs from the start. With the coming into effect of the Education Reform Act of 20111, collaboration will assist project implementation to be in line with national education goals and objectives. For example, there is a structure for PTA coordination at both national and local levels which can be tapped into to successfully work with PTAs. Also, the collaboration with local education authorities will support the decentralization process that the Ministry of education is currently engaged with in the effort of decentralization of education programs. The MOE already has five county staff in various positions in each county performing M&E, education planning, finance, personnel and human resource management roles. Gender Recommendation: M&E for Gender: LAUNCH health and nutrition management, in coordination with M&E, should develop a uniform reporting/collecting strategy to capture male attendance in Care Groups. LAUNCH health and nutrition management should implement strategies for health and nutrition staff to document the various methods Lead Mothers, health and nutrition staff and GCHVs use to engage men. Health and nutrition management, with M&E, should then evaluate, document and promote the methods most likely to have an impact on program beneficiaries. Health and nutrition management should work with M&E to develop strategies to collect and report perceived changes in male behavior. In line with its Gender Integration Policy, LAUNCH female staff members should have opportunities to engage in optimal breastfeeding for the health of their babies and to demonstrate to communities that EBF is the best option for all. LAUNCH management should evaluate and implement strategies that support optimal breastfeeding for staff which although is policy is nevertheless not so apparent in practice. M&E Recommendation with respect to SO2: LAUNCH Health and Nutrition management should expedite the modification of monitoring and evaluation forms to include documentation of promoted BCC messaging. This will improve monitoring of the impact of the program, BCC changes that could be documented include: LAUNCH MTE – Final Report ©Moneval Solutions 2013 Page 81 of 113  Men attending lead mother led neighborhood education sessions.  During home visits, presence of hand washing stations;  Water storage containers with lids; and  Promising strategies in behavior change should be shared widely in both counties and promoted to increase learning and the adoption of successful behavior change strategies. LAUNCH MTE – Final Report ©Moneval Solutions 2013 Page 82 of 113 ANNEXES LAUNCH MTE – Final Report ©Moneval Solutions 2013 Page 83 of 113 ANNEX I: EVALUATION STATEMENT OF WORK Consultancy Title: Mid-Term Evaluation Consultant Project Name: Liberian Agricultural Upgrading Nutrition and Child Health Location: Liberia Timeframe: March – April 2013 Level of effort: Up to 36 work days; number of work days to be extended only upon approval from project supervisor I) Introduction The Liberian Agricultural Upgrading Nutrition and Child Health, (LAUNCH), Title II MYAP program will conduct a mid-term evaluation of its five year Title II food security program in Liberia for submission to USAID. II) Purpose of the Mid-Term Evaluation The purpose of the evaluation is to assess progress toward meeting the LAUNCH program’s objectives. Findings from the MTE will be used to make mid-course corrections pertaining to program strategy and implementation. Specifically, the mid-term evaluation will fulfill the following objectives.  Asses the project’s progress in meeting its expected results and the likelihood of attaining its intended and unintended outcomes;  Identify areas for improvement and make recommendations to improve the implementation and the monitoring and evaluation of the project;  Identify and make recommendations about administrative, operational issues that could hinder or enhance the project’s performance including staffing coordination and technical support between the various offices (i.e. Headquarters, National office and County offices)  Help improve the project’s performance by highlighting and identifying ways, lessons learnt and best practices;  Make suggestions and recommendations to inform the post-evaluation action plan that will be prepared by the project team to guide the project in its remaining years. The evaluator will carry out evaluation activities and produce a concise, readable report that assesses and documents the impact of LAUNCH activities both expected and unexpected with respect to project objectives. Additionally, the report should highlight the relevance, performance and accomplishments of the program’s interventions components. The report will be presented to USAID after review by the LAUNCH implementing partners. III) Description of the project At the end of the 14-year civil war in August 2003, Liberia was left devastated with a barely functioning economy, destroyed infrastructure and an impoverished and food-insecure population that had suffered devastating losses, displacement and trauma. Despite numerous efforts by the Government of Liberia, its development partners and stakeholders to improve the situation, food insecurity is still prevalent with high rates of stunting (39 percent) underweight (19 percent) among children. The challenge has been how to reduce food insecurity without undermining market oriented development efforts or creating dependencies among beneficiary populations. In 2010, USAID awarded ACDI/VOCA a five-year $40 million Title II MYAP in Liberia. The program aims to improve food security of vulnerable rural people in Bong (Salala and Sanoyea districts) and Nimba (Gbor, Wee GbeyMahn, Zoe Gbao and Yarpea Mahn districts). LAUNCH implements an integrated approach targeting 10,800 farmers in agriculture, 10,281 pregnant and lactating women (PLW), LAUNCH MTE – Final Report ©Moneval Solutions 2013 Page 84 of 113 16,770 children 6-23 months, as well as 98,094 family members with health and nutrition services. In addition, the program aims to reach out to as many as 13,800 school children beneficiaries through institutional assistance to local schools and training in rural entrepreneurship to out-of-school youth where targeted communities benefit from sustainable livelihoods intervention as well as health and nutrition services. Following is the program’s results framework and strategic objectives: Table 1: LAUNCH Results Framework Strategic objective Intermediate result Lead partner Objective 1: Increased Availability of and Access to Food of Vulnerable Rural Populations IR 1.1 – Improved smallholder farm management, production, and post-harvest handling practices IR 1.2 – Increased market linkages IR 1.3 – Improved feeder road infrastructure ACDI/VOCA Objective 2: Reduced Chronic Malnutrition of Vulnerable Women and Children IR 2.1 – Strengthened community capacity to engage in health, nutrition and hygiene interventions IR 2.2 – Reduced malnutrition in children under two and improved maternal nutrition IR 2.3 – Improved prevention and treatment of maternal and child illness IR 2.4 – Improved water, hygiene and sanitation access and practices PCI (with technical support from JSI) Objective 3: Increased Access to Education Opportunities IR 3.1 – Increased community capacity to support education infrastructure and systems IR 3.2 – Increased access to livelihoods-based education PCI & MCI Cross-cutting Gender equity Disaster risk reduction and early warning systems PCI & AV IV) Key evaluation questions The evaluation and report shall address the following key components of the program: a) General questions on Project planning and implementation  Are project activities appropriate in addressing the food security problems identified in the targeted areas? Has the program made progress in achieving its three strategic objectives?  Are LAUNCH strategies and activities (for all SOs) appropriate and relevant to the achievement of the project’s intended results? What changes could be made in current activities that would enhance their contribution to the fulfillment of objectives, taking into account funding availability? What improvements can be made to the design of the program to improve results?  What are the key issues and constraints that have been encountered during program implementation and how can they be addressed?  Are the program’s strategic objectives well integrated? Are there opportunities for further integration of program components to maximize impact?  Is the technical field staff adequately trained and supervised? LAUNCH MTE – Final Report ©Moneval Solutions 2013 Page 85 of 113  Are linkages and relationships being established and strengthened between the project the Government of Liberia in terms of food security? What are additional ways to improve coordination with other food security partners in Bong and Nimba?  Are the criteria for beneficiary selection appropriate? Is the Title II program reaching the most vulnerable in target geographic areas?  Is the transfer of knowledge between LAUNCH staff (especially AEAs (agricultural extension agents), health/nutrition officers, Education Coordinators) and farmer group, Care group members, PTAs respectively, efficient? What results did these groups realize from the technical trainings?  Are the training and BCCs materials used by LAUNCH appropriate and effective in project’s targeted areas? Are they well suited to the local agro ecological environments?  Is the commodity and food distribution system efficient and well set-up? How can the program reduce food returns and losses? b) Cross cutting themes Gender  Has the program adequately integrated gender in its implementation? How effective is the program at reaching women? What could be done to improve women’s participation in the program?  Is the program collecting and reporting gender disaggregated data? Monitoring and Evaluation  Is the M&E system organized and effective? How can it be improved?  Are M&E data used for management and reporting purposes? How can M&E data be used better for program management?  As defined and measured, do the performance indicators provide useful and reliable data on program progress and outputs? DRR and early warning System  Has the program allocated enough resources for the DRR component of the program? How can we improve this allocation?  Are DRR committees that LAUNCH has helped to set up functional?  Is the early warning system relevant and appropriate to the current food security context? Sustainability  Are the outcomes related to adoption of better practices sustainable i.e. participants are likely to continue after the project ends? Which outcomes are likely or unlikely to be sustainable, and why? Are the initiated activities likely to continue after donor funding ends? What can be done to increase the sustainability? V) Evaluation Methodology The evaluation will be a two-step process whereby data gathering is carried out followed by expert analysis of the information gathered. The evaluation will make use of available quantitative and qualitative data gathered through field visits, review of project documents and reports, meetings with stakeholders, focus groups with beneficiaries, and discussions with project staff. The mid-term evaluation will examine the role of the stakeholders and ascertain their opinions about the activities and grantees work. A final report will be the main deliverable of this activity and will be shared with project stakeholders and approved by ACDI/VOCA and USAID. The consultant will work with LAUNCH staff to determine a feasible and cost effective methodology at the beginning of the evaluation. LAUNCH expects the evaluator to implore and recommend LAUNCH MTE – Final Report ©Moneval Solutions 2013 Page 86 of 113 qualitative and quantitative techniques (mixed method) to comprehensively respond to the evaluation questions. The quantitative survey will serve to supplement some of the findings of the annual survey that was conducted in July 2012. ACDI/VOCA recommends the use of the LQAS methodology to sample representatively among the beneficiaries. Table 2: Indicators for the quantitative study Indicator Baseline Collected at annual survey Remarks SO1 Household hunger scale No Yes c) % change in yield of specific food crops [rice & cassava] Yes No d) % of smallholders households diversifying crops cultivated Yes Yes but not well reported % of smallholders using at least 3 recommended sustainable agronomic technologies (disaggregated by gender) No Yes % smallholder households with access to cash savings and/or credit (through, or adopted by, community associations or formal financial institutions) Yes Yes SO2 % of mothers of children 0-23.9 mo who had at least 4 prenatal care visits by a trained provider during their last delivery No No % of currently married/in union women 15-49 y using a modern family planning method No No The qualitative part will include, but not necessarily be limited to, the following methods: a) Desk review of relevant documents (project documents, quarterly and annual project reports, minutes of technical meetings, reports on project activities, relevant national policy documents etc.); b) Individual and/or group interviews with LAUNCH staff (both in the field and headquarters) and government and other NGO counterparts; c) Individual or group interviews with representatives of partners and additional stakeholders in country; d) Interviews (focus groups – women only and men only) with project beneficiaries; e) Meetings with representatives of USAID mission in Liberia; f) Field visits to services developed/supported under the project. The evaluation consultant is expected to submit a detailed statement of proposed evaluation methodology and implementation plan that will be considered for approval. Participatory Nature of the Evaluation As a lesson learned from previous evaluations, the mid-term evaluation will be carried out in a participatory fashion, forming a team that, in various places and times, includes a range of managers, implementers, community leaders, partner agency staff and stakeholders. The role of the MTE is to re-direct the implementation. The mid-term evaluation is also an opportunity to use the knowledge LAUNCH MTE – Final Report ©Moneval Solutions 2013 Page 87 of 113 gained during the years of implementation to validate assumptions made in the original analysis and to adjust accordingly. Teams may need to fine tune activities and outputs to achieve the project’s intended outcomes on schedule. An evaluation helps determine the best course of action. The study will note the views of the target groups with regard to their respective projects, paying particular attention to any significant gender-based differences in those views. It will also provide any other information that may further support or clarify the impact of the program. The process and findings are expected to enable ACDI/VOCA and its partners to clearly and easily evaluate the quality of programming over the last two and a half years. VI) Outputs/Deliverables 1. Below are the main deliverables of the MTE process: 2. Presentation of detailed Methodology (both qualitative and quantitative portions of the exercise) to carry out the survey 3. Detailed implementation plan that include a logistic plan 4. Progress report/ presentation of preliminary findings after completing field-work 5. Evaluation first draft report 6. Training: The awardee will provide training to the local LAUNCH monitoring and evaluation staff. The training will be focused on general M&E principals as well as M&E as it relates to Title II Projects. 7. Presentation: The awardee will present to the Liberia staff initial findings (using powerpoint) for discussion and feedback prior to development of the draft report. The awardee will also be prepared to present to the host government, USAID and/or others as requested by ACDI/VOCA prior to leaving Liberia. 8. Evaluation final report: the report content and structure must comply with current USAID’s Evaluation policy (http://transition.usaid.gov/evaluation/USAIDEvaluationPolicy.pdf) and FFP guidance and relevant guidelines. It should follow any applicable mid-term evaluation report guidelines from USAID’s Office of Food for Peace. The final report will have the following components:  Cover Page  Table of Contents  List of Acronyms and Abbreviations  Executive Summary including major recommendations  Introduction  Background  Findings  Strategic Objective 1 – Evaluation questions/results  Strategic Objective 2 – Evaluation questions/results  Strategic Objective 3 – Evaluation questions/results  Assessment (as relate to food security, institutional and impact sustainability, and target/actual achievements) for each Objective/IR and the combined impact  Conclusions / Recommendations  Annexes  Bibliography The final report should be submitted both in hard copy and electronic formats to the donor and one for each partner (ACDI/VOCA, JSI, PCI and Making Cents). The evaluation must also provide a CD of all the datasets (including raw data in SPSS/Excel file), the analysis syntax used in the evaluation as well as the meetings and interviews notes/records. List of documents to be provided to the evaluator(s) by the project team: LAUNCH MTE – Final Report ©Moneval Solutions 2013 Page 88 of 113  MYAP proposal  Performance Monitoring Plan  Results framework  Indicator Performance Tracking Table (IPTT)  Annual Results Reports for the program  Annual Resource Requests (PREPs) for the program  Quarterly Reports  Relevant trip reports VII. Evaluation team The team will be multidisciplinary as the mid-term evaluation is an opportunity to adjust the direction of all program components. To minimize costs and maximize the potential for institutional capacity building, the mid-term will use a combination of external experts and internal resources. An external consultant will be hired to coordinate and lead the mid-term survey. Local consultants will be hired to collect, enter and process the data as well as for fieldwork supervision and to ensure coordination between field sites. General supervision and quality assurance will be provided by LAUNCH management. The team will be composed of: 1. A team leader with extensive experience in evaluation coupled with profound knowledge of nutrition and/or development programs. S/he, in coordination with the LAUNCH team, will be responsible for planning and organizing the overall evaluation. The team leader will lead the qualitative survey team and may also undertake one of the following roles (with exception of quantitative survey Coordinator): 2. A Health/Nutrition specialist 3. An Agricultural/Livelihood/Value Chain Specialist 4. A Quantitative survey Coordinator 5. Enumerators Below is the desired profile of the lead consultant:  Five to ten years experience in project evaluation. Specific experience evaluating or working with Title II PL480 programming and monetization activities is preferred;  Academic background in Agriculture and Development or other related field;  Extensive professional experience in rural agriculture development and/or food aid programs and vulnerability assessments;  Excellent verbal and written communication in English required;  Previous experience working in Liberia or West Africa region is preferred;  Minimum of graduate degree in relevant field required;  Strong critical analysis and report-writing skills required (a writing sample may be requested);  Ability to provide technical lead and work in a team and meet deadlines;  Demonstrated experience and capacity to design and implement qualitative and quantitative research methods for the purpose of project evaluation of USAID projects;  Significant experience with the development of data collection methods and tools (forms, formats, questionnaires) as well as systems for the entry (e.g. CSPRO, mobile phone technology), analysis (e.g. SPSS, STATA) and storage of data and reporting writing;  Experience with rural agriculture training, health, nutrition, DRM and education activities as part of an integrated food distribution program;  Extensive experience in conducting mid-term evaluations for food security programs in Africa for USAID projects, preferably Title II programs;  Previous work experience in Liberia preferred;  Experience working with USAID reporting and assessment tools; LAUNCH MTE – Final Report ©Moneval Solutions 2013 Page 89 of 113  Ability to interact with host government , USAID and/or others as requested by ACDI/VOCA  Strong organizational and reporting skills, presentation skills, attention to detail, ability to meet deadlines, and proficiency in Microsoft Office. VIII. Travel, Expenditures and logistical support Travel to Bong and Nimba counties is anticipated for field data collection. A driver and car will be provided for travel outside of Monrovia. The consultant will be eligible to receive per diem (lodging and meals/incidentals) for time spent in Liberia as well as for time spent traveling to and from Monrovia. Additional expenditures will be reimbursed on the basis of expense reports backed by receipts. Allowable expenditures consist of phone/fax charges related to the consultancy, photocopying, inner city transport to meetings, and travel (based on approved itineraries). No danger pay will be provided. The Consultant will provide their own laptop computer and will provide all documents in Microsoft Word and Excel formats and hard copies of receipts as necessary. IX. Proposed timeline Level of Effort: Up to 36 days. The final version of the mid-term evaluation should be available [tbd] in order to allow project partners to adjust implementation plans if needed. The evaluator will travel to Bong and Nimba counties to visit program activities. Upon returning to Monrovia, the evaluator will prepare the draft report which will be presented to the program’s senior management team and selected project partners before departing the country. Table 3: Timeline Activity Deadline Submit a proposal to the project team March 12, 2013 Approval of proposal March 14, 2013 Submit the mid-term evaluation work plan March 18, 2013 Literature review and Preparation for field data collection TBD Data collection TBD Data entry TBD Analysis of data, preparation and submission of the preliminary report TBD Feedback on draft report by LAUNCH TBD Submission of 2nd draft of the report TBD Debriefing meeting with LAUNCH management team and selected project partners TBD Submission of the final report TBD Final Presentation to host government/USAID Mission TBD Submission to DEC and USAID Mission TBD Clarifications and Amendments Following discussions with the LAUNCH team and USAID in Monrovia (29 April – 01 May, 2013) the following clarifications and amendments were agreed: LAUNCH MTE – Final Report ©Moneval Solutions 2013 Page 90 of 113 Survey: while the initial SOW talked about a quantitative study and an evaluation, the final SOW clarifies it primarily as an evaluation exercise. However, to avoid the possibility of any misunderstanding it was agreed that the MTE team will concentrate on the evaluation only with the focus being on the validation and clarification on the ‘why's’ of the quantitative results from the ARS 2012. The evaluation approach was not to undertake a formal survey but a reasonably comprehensive coverage of beneficiary responses from the field with the objective of confirming and validating the results of the annual survey and seeking clarifications (the explanations as to “why things are as they are”). Under the conditions that we are planning to do the field work, the evaluation approach would be to discuss with beneficiary groups and undertake key informant interviews with involved people and agencies. To make more effective use of time, the LAUNCH team will make a pre￾selection of the beneficiaries and the MTE team will hold discussions with all the different group categories. It was important to secure a complete coverage of all categories of beneficiaries across all the Strategic Objective interventions. We also think it would be advisable to select people who have been in the program for a while rather than new groups so that we are able to identify the areas of success and inadequacies in the approach. Given that we have six districts to cover, approximately two days per district, to cover all categories of beneficiaries should be adequate with the prospect of adjusting as the field work progresses. SO3: there appeared to be a discrepancy in the description of the IRs for SO3 in dealing with Education in the original proposal, the draft SOW, and the final SOW. It was agreed that the MTE team will refer to the IRs as reported in the final SOW and the indicators as reported in the latest version of the IPTT. Evaluation Questions: the team presented the FGD and KII questions to the LAUNCH team in Monrovia and benefited from clarifications and suggestions that resulted in the reformulation of some of these. A general agreement on the approach and the questions was reached to the satisfaction of all prior to the team leaving for the field. It was recognized that time and environment would, to some extent, dictate the execution of the field visit. It was also emphasized that a primary objective of the FGDs was to confirm, possibly clarify and seek further analysis to some of the quantitative results from the 2012 survey. The evaluation questions were discussed with LAUNCH at the Bong field office on the 2nd of May and were updated on the 6th of May following the first five days of field work. Please refer to Annex 1 for the details of the evaluation questions. The field schedule: detailed discussion was undertaken not only on the logistics of the field visits but in the approach that each of the specialists would adopt to undertake the evaluation of the different elements of the project. Commodity Management: although the SOW did not require Moneval to respond to the issue of commodity management, it was suggested that some reporting also be done on that given that the issue was raised in our meeting with USAID. The MTE team have agreed to undertake the responsibility of evaluating the commodity management dimension of the project in addition to the activities described under the SOW. The main aspect of the section dealing with commodity management would aim to:  Assess the effectiveness of commodity management (including warehousing and distribution); LAUNCH MTE – Final Report ©Moneval Solutions 2013 Page 91 of 113  Evaluate the standard operating procedures to ensure: o timely commodity delivery; o commodity quality control; and o minimize losses. The issue of timely delivery and quality control will be assessed from both the end user and from the aggregate planning perspectives. This will include questions relating to the requirements for requesting the release of both categories, for direct distribution and for monetization, the delivery at the port, delivery to warehouse, management at the warehouse and delivery to the food distribution points within the six districts. LAUNCH MTE – Final Report ©Moneval Solutions 2013 Page 92 of 113 ANNEX II: EVALUATION METHODS The work was based on a 15 April start date with the country visits, data collection and presentation of the preliminary finding between the end of April and mid May. The draft final evaluation reports will be prepared between the end of May and the first week in June. The work plan has been divided into the following three phases: Phase 1 –15 April – 26 April: Preparation Phase 2 – 29 April – 18 May: Data collection and preliminary analysis:  29 April – 01 May: Monrovia  01 May – 13/14 May: Field work  14 May: Formulate findings  15 May: Discuss findings with LAUNCH and final data collection  16 May: Presentation and discussion of preliminary findings LAUNCH  17 May: Presentation and discussion of preliminary findings USAID/others  18 May: International Team depart Liberia Phase 3 - 20 May – 05 July: Interpretation and Reporting  14 June: Draft report  10 July: Final report The work plan is illustrated in the diagrams below. Overall Work Plan LAUNCH MTE – Final Report ©Moneval Solutions 2013 Page 93 of 113 Country Work Plan Field Work Schedule – Bong LAUNCH MTE – Final Report ©Moneval Solutions 2013 Page 94 of 113 LAUNCH MTE – Final Report ©Moneval Solutions 2013 Page 95 of 113 Field Work Schedule – Nimba Friday 10/5 Mboweh: Kwakeh vegetable group Logistical Arrangements The evaluation team was supported by 2 sets of interpreters in both Bong and Nimba to accommodate the varying local languages. The team was also accompanied and guided by local LAUNCH staff although not be part of the beneficiary interviews themselves. Two vehicles and drivers were available for the team during the field work and their time in Monrovia. Accommodation in Monrovia and the field was arranged by LAUNCH. LAUNCH MTE – Final Report ©Moneval Solutions 2013 Page 96 of 113 Key Evaluation Questions Area Question Project Planning and Implementation Are project activities appropriate in addressing the food for security problems identified in the target areas? Has the program made progress in achieving its three strategic objectives? Are LAUNCH strategies and activities (for all SOs) appropriate and relevant to the achievement of the project’s intended results? What changes could be made in current activities that would enhance their contribution to the fulfillment of objectives, taking into account funding availability What improvements can be made to the design of the program to improve results? What are the key issues and constraints that have been encountered during the program implementation and how can they be addressed Are the program’s strategic objectives well integrated? Are there opportunities for further integration of program components to maximize impact? Is the technical field staff adequately trained and supervised Are linkages and relationships being established and strengthened between the project and the Government of Liberia in terms of food security? What are additional ways to improve coordination with other food security partners in Bong and Nimba? Are the criteria for beneficiary selection appropriate? Is the Title II program reaching the most vulnerable in the target geographic areas? Is the transfer of knowledge between LAUNCH staff (especially AEAs (agricultural extension agents), health/nutrition officers, Education Coordinators) and farmer group, Care Group members, PTAs respectively, efficient? What results did these groups realize from the technical trainings? Are the BCC materials used by LAUNCH appropriate and effective in project’s targeted areas? Are they well suited to the local ecological environments? Is the commodity and food distribution system efficient and well set up? How can the program reduce food returns and losses? Cross cutting: Gender Has the program adequately integrated gender in its implementation? How effective is the program at reaching women? What could be done to improve women’s participation in the program? LAUNCH MTE – Final Report ©Moneval Solutions 2013 Page 97 of 113 Is the program collecting and reporting gender disaggregated data? What have been the successes and challenges in promoting women’s leadership in farmers groups, school PTAs, and mother Care Groups? What is the quantity and quality of women’s leadership roles and capacities in these LAUNCH activities To what extent are women leaders of farmers groups effectively asserting their leadership? Cross cutting: Monitoring and Evaluation Is the M&E system organized and effective? How can it be improved? Are M&E data used for management and reporting purposes? How can M&E data be used better for program management? As defined and measured, do the performance indicators provide useful and reliable data on program progress and outputs? Cross cutting: DDR and early warning system Has the program allocated enough resources for the DDR component? How can we improve this allocation? Are the DDR committees that LAUNCH has helped to set up functional? Is the early warning system relevant and appropriate to the current food security context? Sustainability Are the outcomes related to the adoption of better practices sustainable i.e. participants are likely to continue after the project ends? Which outcomes are likely or unlikely to be sustainable and why? Are the initiated activities likely to continue after donor funding ends? SO1 and S02 Evaluation Questions Area Question General - village characteristics How many inhabitants? How many houses? Relevance - food security problems in the community Do you produce enough own food for you and your family? When do you usually purchase extra food? How much? Are you able to sell part of your crop? What are you spending the money for? What are the Members are doing with surplus money? How much money do you spend every year for input seeds, fertilizer, chemicals, external labor? LAUNCH MTE – Final Report ©Moneval Solutions 2013 Page 98 of 113 Why don’t you produce more food? Do you consider yourself a professional farmer? How would you describe your knowledge about agriculture and food production? Or do you have other sources of income outside agriculture? Which? Relevance - group characteristics Since when does the group exist? How many lead farmers? How many members? How many present? Relevance and Effectiveness - Is the block farm and farmer group approach appropriate and effective in enhancing adoption of improved farm management practices? Do you prefer individual or group farming? Do you prefer to receive training and advice on the demonstration plot or on a private farm? Why have you joined the group? How is the group composed? Male/female-young /experienced In which area is your group active?  Best practices?  Credit and Savings (SuSU)?  Post harvest?  Farming as Business)  Other? Also supported by LAUNCH Effectiveness - Is the technical field staff adequately trained and supervised? How many visits by extensionists? What can you tell about the qualification of your extensionist? Is she/he knowledgeable? Punctual and reliable? Does she/he answer well questions? What do you think he does well or not so well? Has there been training kit? Did you keep it? What is the value for you individually? Effectiveness - Are the training and BCCs materials used by LAUNCH appropriate and effective in project’s targeted areas? Are they well suited to the local agro ecological environments? Did the training consider your specific agro ecological zone and its problems? Regarding droughts, floods, pests etc.? Did you receive specific training material adopted to your conditions Effectiveness – best practices How many of you learned about best practices in the last year? What exactly? What is the advantage of applying it? Do you think it will show better results on your individual farms as well? Which? LAUNCH MTE – Final Report ©Moneval Solutions 2013 Page 99 of 113 Effectiveness - crop diversification How many of you are growing more or new crops since start of the project? Which new crops? Effectiveness - extended cropping area How many of you did increase the cropped area as result of the project intervention? Last year? By How much? Effectiveness - Are the methodology and approach under post-harvest handling techniques appropriate and effective? What improvements can be done? How many of you have learned about post harvest handling? What exactly did you learn? What have you applied? What is your experience? Which type of losses did you reduce? Effectiveness - credit and saving groups Since when does the SUSU group exist? Constitution? Have you had prior experience with credit and savings? Loan practices? How many of you received a credit? What amount? What has it been used for? How many of you have made savings? Have you formed a saving group with saving cards? What will you be doing in the future? Effectiveness - Are the methodology and approach under farming as a business appropriate and effective? What improvements can be done What kind of assistance did you get so far? Have you learned to estimate cropping areas and yields? What about: Business Plans? Cost analysis and profit? Product quality? Selling and marketing? Effectiveness - market linkages Have you an idea to whom to sell your products? Which market? Local? Trader? Farmgate? Other What quantities will you be able to sell and at which price? What other problems prevail when considering selling? Transport and Access? Low prices, low demand? Sustainability - outcomes sustainable Which of the practices the project has introduced do you feel to be able to continue after the end of project support? Impact - Main project effects regarding beneficiaries livelihood, assets or income Which effects has the project made so far? What do you expect after two more years Higher agricultural income? Less work on the field or easier work? Will there be more women involved in making decisions? Will there be more work in groups or more individual work? What will be the most important change at the end of the assistance? LAUNCH MTE – Final Report ©Moneval Solutions 2013 Page 100 of 113 Cross cutting - Is the DRR system relevant and appropriate to the current food security context? Since when does the DMC exist? How many are participating in the disaster risk reduction group? Which mini-sort disasters have affected the community in the last 20 years? natural? crops? health? How many households have been severely affected? How does your group help? who has become more knowledgeable; what skills have been learned; what participants are doing with this knowledge and skill Has anything been done by the government or any other institution since this disaster to reduce the impact of future disasters? If so, could you describe? What plans does the DMC have for the future? cross cutting - gender Are there women in leadership position? What is the advantage of having women in the team? cross cutting - integration How many of you are also participating in WASH or CARE activities? SO2 Evaluation Questions Members that lead mothers lead. Area Question General Ask the number of members who are part of farm groups (or their husbands) to compare those groups with more dual members and those with less Program implementation What is the purpose of LAUNCH? What does LAUNCH do? Why is this LAUNCH activity important? This is working with Lead Mothers with the two “baby mothers” group. How often does your Care Group meet? Before this meeting, when was the last time you attended a group? Why do you want to improve the nutrition of your child? What is the purpose of LAUNCH? What does LAUNCH do? Why is this LAUNCH activity important? This is working with Lead Mothers with the two “baby mothers” group. Rations What do you have to do to receive rations? When do rations end? What will you do when your child is 2 years (24 months) old? Is 1 year, 9 months, 2 years? When did you last receive your ration? When (EMPHASIZE NOT A DATE but short time or long time) you registered, how long did it take for you to receive your ration? Have you have trouble receiving a ration? If so what? How do you use the ration? How do you prepare CSB? If you don’t have CSB what should you feed the baby? Will this help LAUNCH MTE – Final Report ©Moneval Solutions 2013 Page 101 of 113 your baby grow strong like CSB? Did you have to learn how to cook the ration? Did you learn this from the project? Do you use the ration in the non-lean season? Why or why not? What are some ways you can improve the nutrition of your child? WASH (will also observe): Quantify before and after LAUNCH I’ve asked this of the WATSAN committees How do you ensure water is safe? Did you ensure your water was safe before LAUNCH? If you do not have a covered water container, what do you do? If someone in your community has a child with diarrhea, what should they do? (Specific hygiene practices) Did you learn this from LAUNCH? This doesn’t work with “baby mothers.” Growth Monitoring/Clinic interactions (review of health cards) How far do you have to travel to get to the clinic? Does your have child health cards (road to health cards)? (Is there a difference between community with clinics and those without?) Has the care they receive at clinics changed? How so? Is this because of LAUNCH? Thinking about the last time you took your child under 2 to the clinic, were they weighed? Did they write it down? Why is this important to know? Has your child been vaccinated? Do you know how many vaccinations your child needs to be completely vaccinated? Are your children healthier? Is it because of LAUNCH? Why? Utilization/Adoption When last pregnant, did you complete your ANC visits? Why or why not? (is there a difference between those who live near clinics) If your child is over 6 months, when did you introduce complementary foods? What kinds of foods are ok to introduce? Do you and your partner know how to space the births of your children? Are you? What can you do to make your child healthier? Did you learn this from LAUNCH? What do you use from the education from LAUNCH ENA to improve your child’s health? What is exclusive breastfeeding? In your last pregnancy, did you exclusively breastfeed? Is it ok if I give my 3 month old baby water? Why not? What do you tell a friend who is feeding her baby less than 6 months? Why do some women not breastfeed? How could you encourage them to breastfeed? LAUNCH MTE – Final Report ©Moneval Solutions 2013 Page 102 of 113 Is it important to talk and play with your baby? Why? Do you tell your husband what you’ve learned? Integration How has LAUNCH education impacted these decisions? If the decision maker is male Do any of you or your husband’s participate in farmers groups? Of the households in farmers groups, do you eat the crops grown? Why or why not? What do you do with the money made from the crops sold? Challenges & Solutions Do your husband’s/partners help your efforts to improve your children’s nutrition? What do they do to support you? Why or why not? Based on what they have identified earlier (challenges to rations, clinic access etc), what could LAUNCH do to make it easier? Am not asking this of baby mothers. I’ve asked of Lead mothers. Most important change LAUNCH had on you and your child’s health? Lead mothers/father (KII): (I’ve added some questions from the first group) What is exclusive breastfeeding? Are women in your groups’ exclusive breastfeeding? Why do you think they are so successful? Do beneficiaries have trouble receiving rations? If so what? Why? What are challenges your groups discuss? This does not work. What have you done to increase men’s participation in improving the nutrition of their children? Is it working? If yes, describe a success story? What do you do when you have success stories? If not, why? When LAUNCH program staff come to the program what happens? Tell me something while working in LAUNCH that makes you proud. (This will work if the success story doesn’t work) Field Staff: Review supportive supervision forms, IEC materials. Area Question General Why is nutrition important in PLW and CU2? What are the essential nutrition actions? Program Implementation What is the registration process for new ration recipients? How long does it take from registration to receiving the food? Are there challenges in beneficiaries getting food? What are those? What steps have you taken to address those challenges? How does LAUNCH integrate with government entities? Vaccinators? What LAUNCH trainings have been implemented in your community? Has it been stepped down? What were the challenges, successes and lessons learned in the step down process? LAUNCH MTE – Final Report ©Moneval Solutions 2013 Page 103 of 113 When do women graduate? What happens if the card ends and the child is 1 year, 9 months? What do you during supportive supervision? What if there is a problem? What if there is a success? County supervisor What is the community visitation schedule? How is it tracked? Where is this information recorded? How is the WASH tolling out? Most major inputs are waiting for dedication, when will that happen? Challenges Staff knowledge/skills? Uptake of behavior? Utilization/Adoption How do know households have adopted behavior? How do you know a household is using this behavior? How do you report/record this info? Integration What are some strategies you have used to integrate Agriculture and Education activities into SO2? What are other ways you might integrate Agriculture and Education activities? What other organizations are implementing in this area? How does LAUNCH interact with them? How do you integrate/engage men in child nutrition? Challenges & Solutions What are some challenges faced while implementing LAUNCH? What have you/the program tried to address them? Did it work? Why or why not? What is one positive/success story of LAUNCH? Have you shared this story? Clinic Staff/CGHVs/ (TBAs/TTNs (BONG CLINIC) : Review Supportive supervision forms and growth charts (AFTER); health education materials, Visit to mother’s waiting room and MNCH space. Area Question Program Implementation What is the LAUNCH program? Who is eligible to receive benefits? Do you have nutrition/MNCH education materials? Where did the materials come from? LAUNCH or another project? Have you been trained to use them? Do you need training? Has LAUNCH staff visited the clinic? When? What happened? Are there other organizations working in your community on child nutrition? How do you coordinate activities of LAUNCH and that program? How do you coordinate activities of LAUNCH and that program? Challenges & Solutions Some clinics have had challenges completely the child health cards. Why do you think that is? How can this be resolved? When the LAUNCH is finished, what will you continue to use LAUNCH MTE – Final Report ©Moneval Solutions 2013 Page 105 of 113 How has the implementation of LAUNCH change communities and schools? Good practices What are some things that were done differently that are improved now due to LAUNCH? What makes LAUNCH communities and schools different from other communities without the program? Are there issues with the way LAUNCH targets the beneficiaries? If yes list them? Integration How are activities of this objective related to those of other objectives? What effort should be made to ensure communities benefit from all of LAUNCH activities? Transfer of knowledge What can you say that communities have shown in terms of new knowledge gained as a result of LAUNCH? What skills have communities gained during program implementation? Gender: participation How involved are women in the implementation of Launch activities? How is gender mainstreamed in all activities of LAUNCH? Gender: access to education for girls How are activities structured to ensure equal access by both girls and boys? Are there special considerations to ensure girls have access to LAUNCH? Sustainability What efforts are underway to ensure continuation of some activities after LAUNCH leaves? What activities are bound to continue even after LAUNCH? If these activities are bound to continue, what roles with authorities of the county need to play? Members: PTA (principal/SCHOOL ADMIN and officials of the PTA), Teachers, Communities Themes Questions Before and after LAUNCH (achievements, appropriateness, relevance, efficiency) What was the level of organization of this PTA before LAUNCH? What support is provided the PTA by LAUNCH? Are the support provide useful for your PTA? What can you say are the achievements of this PTA due to LAUNCH? How have the support made you efficient in carrying on your roles? How does your PTA contribute towards increase in the enrolment of students? Things to improve (issues, challenges in implementing the project and achieving LAUNCH’s targets What are some of the challenges you faced during the implementation of the project? What can you say are things that need to improve while working with LAUNCH? LAUNCH MTE – Final Report ©Moneval Solutions 2013 Page 104 of 113 from the LAUNCH? Why? What do you think will not continue? Why? Success Stories How has LAUNCH helped the women in your community? Can you give a specific example? TTMs/TMs: Area Question General What is LAUNCH? What does LAUNCH do? Are the women and babies in your community different after LAUNCH? How? How has your role changed after LAUNCH? CHTs/District Leaders: Area Question General What does LAUNCH do? Who is eligible to receive benefits? Why is nutrition for children important? How does improved nutrition in children, help your community? How do you support LAUNCH improve the nutrition of children in your community? Are men involved in improving the nutrition of their children? Is there something else they could be doing? How do we get more men involved? SO3 Evaluation Questions Program coordinator and Team Leaders Themes Discussion questions Appropriateness What does SO3 seek to address? In what way do you think the activities set can achieve the set objectives? Are there problems associated with the set activities? If so identify them? Linkages and relationship Are beneficiaries also benefitting from other activities under other objectives? How does SO3 link to other LAUNCH objectives? How does the LAUNCH SO3 link to other education sector objectives? Lesson learnt during implementation Are there activities under this objective that could be done differently for satisfactory result? Share some key lessons you have learned during the implementation LAUNCH MTE – Final Report ©Moneval Solutions 2013 Page 106 of 113 Are there issues with the way LAUNCH identify beneficiaries? If yes list them? Good practices and lessons learned What can you say are new knowledge you have gained as a result of LAUNCH? What things that were done that can be done differently today? What makes your PTA different from others due to LAUNCH intervention? Sustainability How can we make sure that activities continue after the LAUNCH project? How prepare are members of the PTA and community to contribute towards activities continuing after LAUNCH? What structures are there, that can be used to continue the program? Integration What activities under LAUNCH are being carried out in this community? What are the similar activities that are carried out in both schools and communities? Gender (participation, inclusive access to education) How involved are women in the implementation of LAUNCH activities? How are women helping for girls to enter and remain in school? What have LAUNCH to ensure increase in girl’s enrolment? How has LAUNCH contributed to increase in women participation or leadership in the PTA? LAUNCH MTE – Final Report ©Moneval Solutions 2013 Page 108 of 113 Winifred Salir Clera Sherman Joe Martin Shirley Cooper Philomena Flomo DHO, Sanoyea GOL Harris Dahn DHO, Gbor and Zoe Gbao Christopher Ayettey F. Nimene Sonpon Martin Boakai Sonnie Blamah Grace Gee Thomas Boyeh Nimba Field Staff Health and Nutrition Field Staff Hennis Bong County Manger ACDI/VOCA  Bibliography of Documents Reviewed  LAUNCH Compiled Responses to USAIDS Questions on FY12 ARR  LAUNCH ARR FY12 Attachment IPTT FINAL  LAUNCH CA Attachment B Technical  LAUNCH_F11 Q3 Quarterly Report Final 072911 SUBMITTED  LAUNCH_FY11 ARR Narrative FINAL  LAUNCH FY11 Q2 Progress Report 051011 SUBMITTED  LAUNCH FY12 ARR Narrative FINAL  LAUNCH PREP FY11 rev2 Narrative_011311_APPROVED.pdf  LAUNCH PREP FY12 Narrative 12 16 2011 FINAL approved  LAUNCH Quarterly Report April to June 2012 Final submitted  LAUNCH Quarterly Report January to March 2012 043012  LAUNCH USAID FY10 Q4 Progress Report 102410 SUBMITTED  Farmer Group Selection Guide Dec2012  USAID Evaluation Guidelines PNADO818  USAID EVALUATION POLICY  LAUNCH Gender Policy  Liberia Comprehensive Food Security and Nutrition Survey (March 2013)  USAID Strategy Document For Project Implementation (Produced by Agricultural Extension team, April 2012) LAUNCH MTE – Final Report ©Moneval Solutions 2013 Page 109 of 113  LAUNCH Annual Monitoring Survey FY 2012  Feed the Future. Liberia. Retrieved May 28, 2013, from http://www.feedthefuture.gov/country/liberia  Feed the Future. Liberia FY 2010 Implementation Plan. Retrieved May 24, 2013 from http://www.feedthefuture.gov/sites/default/files/country/strategies/files/FTF_2010_Implement ation_Plan_Liberia.pdf.  Guyon, AB and Quinn, VJ. Booklet on Key Essential Nutrition Actions Messages. Core Group, Washington, D.C., January 2011  UNICEF. Early Childhood, Early Learning. Retrieved May 28, 2013 from http://www.unicef.org/earlychildhood/index_40747.html.  UNICEF. Early Childhood Development Kit: Guideline for caregivers. Retrieved May 26, 2013 from: http://www.unicef.org/chinese/earlychildhood/files/GuidelineforECDKitcaregivers.pdf.  UN Liberia. EMPOWERING WOMEN IN LIBERIA. Retrieved May 27, 2013 from http://www.unliberia.org/doc/genderemail.pdf.  WHO. Integrating Early Childhood Development (ECD) activities into Nutrition Programmes in Emergencies. Why, What and How. Retrieved May 26, 2013 from http://www.who.int/mental_health/emergencies/ecd_note.pdf. U.S. Agency for International Development 1300 Pennsylvania Avenue, NW Washington, DC 20523