April 10, 2012 This publication was produced for review by the United States Agency for International Development (USAID) under the terms of Cooperative Agreement USAID-FFP-A-09-00001. It was prepared by Mr. Mike DeVries, Program Design, Monitoring and Evaluation Specialist, Dr. Judiann McNulty, Maternal and Child Health and Nutrition Specialist, and Mr. Golam Kabir, Program Design & Evaluation Consultant. EVALUATION Wellness and Agriculture for Life Advancement (WALA) Mid-Term Evaluation Report Wellness and Agriculture for Life Advancement (WALA) Title II, Multi-Year Assistance Program CRS Malawi Cooperative Agreement #: AID-FFP-A-09-00001 Mid-Term Evaluation Report 10 April 2012 WALA Mid-Term Evaluation Report 18 March 2012 Acknowledgement The members of the evaluation team would like to gratefully acknowledge the support provided by the staff of the WALA Program. Special thanks go to the CATCH staff who went out of their way to provide logistical support to enable the team to implement a complicated evaluation schedule during a period of significant fuel shortage and relatively heavy rains. We would also like to thank the staff from all of the implementing and technical partners for supporting the field work, participating in various events during the evaluation, and providing insights while answering our multitudinous questions. Innumerable program participants and beneficiaries have also provided useful responses to our questions and participated in stimulating discussions. Our gratitude goes out to them as well. Hopefully we've listened well, our observations are grounded in reality, our assessment is accurate, and our recommendations are useful. Sincerely, Mr. Golam Kabir, Program Design & Evaluation Consultant Dr. Judiann McNulty, Maternal and Child Health and Nutrition Specialist Mr. Mike DeVries, Program Design, Monitoring and Evaluation Specialist This report was made public by support from the U.S. Agency for International Development (USAID) under the terms of Cooperative Agreement USAID-FFP-A-09-00001. The authors’ views expressed in this publication do not necessarily reflect the views of USAID or the Government of the United States of America WALA Mid-Term Evaluation Report 18 March 2012 FY 2009-2014 Multi-Year Assistance Program Wellness and Agriculture for Life Advancement (WALA) Catholic Relief Services Malawi MID-TERM EVALUATION Page I. EXECUTIVE SUMMARY 1 II. BACKGROUND 3 A. Overview of WALA Strategy B. Program History and Operating Context C. Evaluation Methodology III. ACTIVITIES AND OUTPUTS PRODUCED UNDER EACH STRATEGIC OBJECTIVE 4 A. Strategic Objective 1: Maternal and Child Health and Nutrition 4 1. IR 1.1: 170,724 vulnerable households have increased use of quality maternal and child health and nutrition services Activity 1.1.1: Development of behavior change communication strategy Activity 1.1.2: Formation of Care Groups Activity 1.1 3: Conducting Community Complementary Feeding and Learning Sessions 2. IR1.2: 170,724 vulnerable households have increased use of quality maternal and child health and nutrition services. Activity 1.2 1: Support to the Ministry of Health to roll out C-IMCI: Activity 1.2.2: Reinforce community-based growth monitoring and promotion Activity 1.2.3: Designing and implementing quality improvement and verification guidelines Activity 1.2.4: Provision of targeted supplementary feeding program (SFP) to malnourished children and pregnant and lactating women. 3. IR 1.3: 2,148 community groups have enhanced capacity to promote health and nutrition services for 170,724 targeted households. Activity 1.3.1 Training for community groups in management and sustainability. Activity 1.3.2 Establish linkages to district development authorities. 4. General comments on SO1 B. Strategic Objective 2: Agriculture and Natural Resource Management 14 1. IR 2.1. 147, 500 smallholder farming households apply improved crop production practices. Activity 2.1.1 Capacity Building of sustainable extension services to improve production Activity 2.1.2 Promotion of optimal crop production technologies Conservation Agriculture Small-Scale Irrigation Post-Harvest Handling Water Shed Development Animal Husbandry (Nor Specified in Proposal) Integrated Pest Management (Not Specified in Proposal) Activity 2.1.3 Support to local seed systems 2. IR 2.2 103,400 households have increased their use of financial and related services through Village Savings and Loan (VSL) groups. Activity 2.2.1 Mobilization and group formation Activity 2.2.2 Capacity building in internal savings and loan operations Activity 2.2.3 Promotion of economic development Activity 2.2.4 Introduction of a commercial sustainable implementation approach WALA Mid-Term Evaluation Report 18 March 2012 3. IR 2.3 20, 600 smallholder farming households have increased income and/or assets from commercial marketing. Activity 2.3.1 Value chain analysis and capacity building of marketing staff Activity 2.3.2 Marketing group training in value chain integration and market entry Activity 2.3.3 Sustainable market linkages and business development services 4. General comments on SO2 D. Strategic Objective 3: Disaster Risk Reduction 32 1. IR 3.1 273 communities have strengthened disaster preparedness, contingency and response capacity. Activity 3.1.1 Training in disaster management Activity 3.1.2. Linkages with formal disaster management mechanisms Activity 3.1.3. Integration of environmental protection in SO1 and SO2 2. IR 3.1 21,203 food insecure households have enhanced capacity to withstand shocks and stress. Activity 3.2.1 Enhance safety nets through rations Activity 3.2.2. Support asset building and protection through Food for Work (FFW) Activity 3.2.3. Rebuild assets lost by disasters through Food for Assets (FFA) 3. IR 3.3 8,002 community-led groups have practiced good governance principles. 4. General Comments on SO3 IV. CROSS-CUTTING PROGRAM ACTIVITIES 32 A. Governance B. Gender C. HIV/AIDS D. Environmental Monitoring and Impact Mitigation V. PROGRAM TARGETING AND IMPACT 42 A. Targeting - Who Is Benefitting from Program Activities? B. Where is Significant Impact Being Generated by WALA? C. Impact at the Program Goal Level VI. PROGRAM SYSTEMS 45 A. Program Management B. Partnerships C. Knowledge Management 1. Knowledge In 2. Monitoring and Evaluation Systems 3. Knowledge Out. D. Program Integration and Complementarity E. Financial Resource Management F. Commodity Management G. Human Resource Management VI. PROGRAM- LEVEL CHALLENGES 51 A. Resource Constraints 1. Staff Recruitment and Retention. 2. Devaluation 3. Fuel Shortages B. Sustainability Plans and Exit Strategies C. The Hand-Out Syndrome VII. CONCLUDING REMARKS 53 WALA Mid-Term Evaluation Report 18 March 2012 ANNEXES Annex A Abbreviations and Acronyms 52 Annex B Mid-Term Evaluation Recommendations 59 Annex C Key Dates in the Life of WALA 68 Annex D Mid-Term Evaluation Terms of Reference 69 Annex E Evaluation Plan 85 Annex F Interviews and Site Visits during the Evaluation 124 Annex G SO1 Statistical Summary 137 Annex H SO2 Statistical Summary 139 Annex I External Technical Support for WALA 141 Annex J Suggestions for the IPTT/ITT 143 Annex K: Summary Tables on Finance, Commodities and Human Resources 146 TABLES Table 1. Resource Summary 4 Table 2. Outcome Indicators for SO1 and Accomplishment* at End of FY 2011 4 Table 3. WALA Human Resources for SO1 7 Table 4. Outcome Indicators for SO2 and Achievements* by the End of FY 2011 14 Table 5. Outcome Indicators for SO3 and Accomplishment at End of FY 2011 32 Table 6. WALA Participant Summary as of December 2011 43 Table 7. Where is Impact Being Generated? 44 Table 8. Budget and Cash Expenditure Summary Through December 2011 (US$) 49 ABBREVIATIONS AND ACRONYMS Due to the length of the list of acronyms and abbreviations, these have been attached in an Annex (See Annex A). WALA Mid-Term Evaluation Report 18 March 2012 1 FY 2009-2014 Multi-Year Assistance Program Wellness and Agriculture for Life Advancement (WALA) Catholic Relief Services Malawi MID-TERM EVALUATION I. EXECUTIVE SUMMARY Catholic Relief Services (CRS) commissioned a Mid-Term Evaluation of the Title II Multi-Year Assistance Program (MYAP) entitled Wellness and Agriculture for Life Advancement (WALA) being implemented by a consortium of seven implementing partners under the leadership of CRS in eight districts of southern Malawi. The program has three components, (1) a maternal and child health and nutrition component that is focused on pregnant and lactating women and children under the age of five years, (2) an agriculture and natural resource management component targeting smallholder farmers with landholdings less than one hectare and (3) a disaster risk reduction component building capacities for preparing for and mitigating natural disasters while also strengthening household coping capacities through food distributions. The total program cost to Food for Peace at approval was estimated to reach US$ 80,925,9001 with 105,990 MT of commodities for monetization and distribution. A Cost-Sharing commitment of US$ 886,595 was made at the time of the approval. The program targeted having impact on 214,974 chronically food insecure households over a period of five years from 8 May 2009 through 30 June 2014. The Mid-Term Evaluation was conducted by a team of three development professionals2 over the period 14 January through 19 February, 2012, in Malawi. The team reviewed existing secondary sources of information, reviewed available quantitative information and used qualitative survey methods to obtain information to understand the progress and impact achieved by WALA toward formulating recommendations for the remaining life of the program. Relative to the outputs being produced by the program, the following highlights emerged from the evaluation.  13,517 Care groups for MCHN have been formed, trained in three topics, and are successfully promoting behavior change in sanitation  Community Complementary Feeding and Learning Sessions (CCFLSs) remain too focused on recuperation, do not reach all target households, and groups are much too large for skills acquisition.  As planned, Community-Integrated Management of Child Illness (C-IMCI) training has not yet been implemented and the model for improving Growth Monitoring and Promotion is still in the pilot stage.  Supplemental feeding of malnourished children and pregnant and lactating women delivered through government Community Therapeutic Centers and Nutrition Rehabilitation Units is having good impact; but because of government definitions of child malnutrition, the program is not reaching enough children and not enough women. 1 From the Award Letter received 3 June 2009: Total C&F Commodity Value = $64,331,000, Total ITSH Value = $10,276,300, and Total 202e Value = $6,318,500 . Note that Amendment 1 dated 7 July 2009 corrected the sum to USD 80,925,900. 2 Mike DeVries, Program Design, Monitoring & Evaluation Specialist, Judiann McNulty, Maternal and Child Health and Nutrition Specialist, and Golam Kabir, Program Design and Evaluation Specialist WALA Mid-Term Evaluation Report 18 March 2012 2  Village Health Committees (VHCs) have been formed or revitalized but need training in the key messages, especially on C-IMCI.  There has been limited real adoption of conservation agriculture techniques being extended through 253 community-based Farmer Extension Facilitators to 5,972 producer groups, but the techniques are appropriate, feasible and likely to be adopted over time.  36 irrigation schemes using gravity-fed or treadle pump technologies have been constructed, however, more attention needs to be placed on capacity building with Water User Groups.  The Watershed Development approach being piloted in thirteen sites is demonstrating some useful ideas but needs to target smaller watersheds and be more fully integrated with the conservation agriculture and irrigation components of the program.  Interventions targeting livestock and aquaculture, seed supply systems, post-harvest storage and handling, and integrated pest management have started late mainly because of staff recruitment and turnover challenges.  A total of 37,607 participants with estimated equity of approximately US$ 1,061,750 million are realizing substantial benefits from participation in Village Savings and Loans (VSL) Groups.  A total of 25,962 farmers have joined marketing clubs in 246 locations and have started realizing benefits from collective marketing and out-grower contracts.  A total of 33 Area Civil Protection Committees (ACPCs) and around 150 Village Civil Protection Committees (VCPCs) have been established or reformed.  Training on Disaster Risk Management and strengthening of vertical linkages for emergency response have begun with 85 VCPCs.  In 2011, WALA implemented an emergency food distribution for 8,858 drought-affected households in the program area.  A total of 8,197 households with chronically ill members or caring for orphaned children have received safety net distributions of food.  A total of 6,740 Food for Work participants have constructed or rehabilitated roads, watershed development structures, irrigation systems and a few other unique community assets in over 130 sites. The program has made progress on nearly all proposed activities, and the evaluation did not find that any parts of the proposed WALA Program strategy had become irrelevant due to changes in the context. While the program has just passed the midpoint in its life, it is already achieving some notable impact, especially with the Care groups, the supplemental feeding program, the Village Savings and Loans component, and the safety net food distributions. The Mid-Term Evaluation has formulated ninety–four recommendations for the remaining life of the program. None of these recommendations are suggesting major changes in the program strategy. They are, rather, "tweaking" of the strategy in order to enhance impact or ensure sustainability of the impact being achieved. Some of these have been designated as "PRIORITY", meaning that they should be given priority attention in the remaining life of the program. The full list of recommendations is provided in Annex B. The WALA Program, at this point in time, is a sound program. It has the potential to become a great program, if it can make some adjustments and effectively meet the challenges it currently faces, or may face in the near future. These major challenges include (a) resource constraints related to staff recruitment and retention, persistent fuel shortages, and the likely devaluation of WALA Mid-Term Evaluation Report 18 March 2012 3 the Malawian Kwacha, (b) the need for more comprehensive sustainability plans and exit strategies, and (c) addressing community expectations for hand–outs. II. BACKGROUND A. Overview of WALA Strategy The final goal of the WALA Program is to reduce food insecurity for 214,974 chronically food insecure households in 39 Traditional Authorities within five livelihoods zones of Southern Malawi. The program has seven intermediate results under three strategic objectives as shown below. Strategic Objective 1 (Maternal and Child Health and Nutrition): 170,724 vulnerable households have improved maternal and child health, and nutrition status.  Intermediate Result 1.1: 170,724 vulnerable households have improved maternal and child health, and nutrition practices.  Intermediate Result 1.2: 170,724 vulnerable households have increased use of quality maternal and child health, and nutrition services. Strategic Objective 2 (Agriculture & Natural Resource Management): 147,500 smallholder farming households have improved livelihood status.  Intermediate Result 2.1: 147,500 smallholder farming households have improved crop production practices.  Intermediate Result 2.2: 103,400 smallholder farming households have increased use of financial services.  Intermediate Result 2.3: 20,600 smallholder farming HHs have engaged in commercial marketing. Strategic Objective 3 (Disaster Risk Reduction): 273 targeted communities have improved capacities to withstand shocks and stress.  Intermediate Result 3.1: 273 communities have strengthened mechanisms for disaster preparedness, response and mitigation  Intermediate Result 3.2: 21,203 food insecure households have accessed transitional food rations  Intermediate Result 3.3: 8,002 community-led groups have practices good governance principles. Table 1 summarizes the approved resources and expenditures through December 2011. B. Program History and Operating Context The WALA Program was designed as a follow-on to the Improving Livelihoods through Increasing Food Security (I-LIFE) Program. While I-LIFE operated in the lower half of the country, WALA shifted the focus to the eight southernmost districts, not covered by I-LIFE, where food insecurity is more acute. Four3 of the current WALA Partners were also members of the I-LIFE Consortium, and a number of approaches in WALA were piloted and refined under I￾LIFE, including the Village Savings and Loans approach, the Care Group approach and the interventions associated with irrigation. While the program, for the most part, started up and 3 Africare, Emmanuel International, Save the Children US and World Vision International WALA Mid-Term Evaluation Report 18 March 2012 4 began implementation in a relatively timely way, there were delays with some components as a result of staff recruitment and turn-over. In the two and a half years since implementation began, significant changes in the context that have affected program implementation include civil disturbances in 2011 and on-going fuel shortages which began to intensify early in 2011, reaching a peak in the last quarter of calendar year 2011. As a result, the WALA program had to slowdown implementation in 2011 and cancel some technical support visits by outsiders. Annex C provides a table of key dates in the life of the WALA Program. Table 1. Resource Summary Resource Initial Agreement Approval Revised Agreement Total Actual Expenditures as of Dec. 2011 Distribution Commodities 16,200 MT 16,200 MT 6,024 MT Monetization Commodities 89,790 MT 89,790 MT 43,349 MT Monetization Proceeds $34,886,842 $33,506,639 $12,319,604 202e $10,276,263 $11,656,500 $6,871,551 ITSH $6,318,531 $6,318,531 $2,558,380 Cost Share Contribution $886,596 $886,596 $404,655 Total Program Cost $80,929,900 $80,925,900 $22,154,190 Direct Beneficiaries Target 214,974 HH 214,974 HH Not Available4 Life of Activity May 8, 2009 –June 30, 2014 May 8, 2009 – June 30, 2014 C. Evaluation Methodology As a formative evaluation, the purpose of the Mid-Term Evaluation of the WALA program Project was to use qualitative methods as well as analysis of secondary, baseline and annual survey data to document the activities, outputs and impact of the program, review the processes used to implement or support implementation, and formulate recommendations for the remaining life of the program. The Terms of Reference for the evaluation are included in Annex D. Annex E contains the Evaluation Plan, schedule, and tools used. Annex F contains the persons interviewed and sites visited. In general, the evaluation team was confident with the statistical data provided by the CATCH M&E unit, who were consistently attentive to ensuring the quality of information reported by the program. The main limitation of the evaluation was insufficient time combined with the rainy season which constrained access to more remote locations. III. ACTIVITIES AND OUTPUTS PRODUCED UNDER EACH STRATEGIC OBJECTIVE A. Strategic Objective 1: Maternal and Child Health and Nutrition (MCHN) SO1: 170,724 vulnerable households have improved maternal child health and nutrition status. Table 2. Outcome Indicators for SO1 and Accomplishment* at End of FY 2011 Indicator Baseline FY2011 % Children 0-5 months of age exclusively breastfed within the past 24 hours 65% 67% 4 WALA's current M&E system does not track the number of beneficiaries under IR 1.2 and also cannot yet eliminate overlap between SOs. The program is working on resolving these challenges. WALA Mid-Term Evaluation Report 18 March 2012 5 % children 6 -23 months who receive a minimum acceptable diet 12% 22% % of caregivers of children 0-59 months demonstrating proper environmental hygiene 6% 17% % of caregiver of children 0-59 months demonstrating use of fuel-saving technologies 2% 4% % of caregiver of children 0-59 months regularly attending growth monitoring 41% 51% % of mothers of children 0-11 months who received VAS within 8 weeks of delivery 64% 69% % of children 0-11 months old whose births were attended by a skilled attendant 78% 86% *Note that the comparison is illustrative rather than direct since different sampling methods were used: baseline sample was drawn from a population based sample frame, the FY11 survey was drawn from a participants sample frame. The complete FY 11 survey report was previously submitted to USAID. The maternal child health and nutrition (MCHN) activities were initiated quickly with carry-over of many I-LIFE staff and similar activities. WALA has been fortunate in hiring very competent experienced coordinators. CATCH developed an MCHN strategy and standards for the key activities within the year, but dissemination of these and development of other materials and training has been hindered by high turn-over of MCHN staff in CATCH. In spite of this, MCHN activities are mostly on-schedule and most are of high quality. Aside from issues with Activity 3 and recommendations on improving dietary diversity, most other observations during the evaluation are related to technical details rather than overall performance. Having laid the groundwork, results for MCHN will be very apparent in the next annual survey and will have the most significant impact of any project activities on reducing chronic and acute malnutrition. WALA had already surpassed the goal of reaching 130,000 households with MCHN by the end of FY 11 with a total of 138,609 households reached. Annex G provides a statistical summary of MCHN achievements. In designing WALA, concessions had to be made to government policies. While the partners would prefer to be targeting children under two and using food rations in a preventive manner, government insists on including children up to five years and focusing supplemental feeding only for children identified with high levels of wasting that are not quite severe. WALA has engaged in discussion with the government around these policies, but pressure from USAID and other donors as well as the recently initiated Scaling Up Nutrition (SUN) movement will have far more success in achieving changes in these policies. WALA has been engaged in national level efforts including participation in the revision of the national micronutrient strategy and in the development and launch of the National Nutrition Plan. WALA is coordinating with the Office of the President and Cabinet Health and Nutrition Section in development of national level materials including a recipe book which will feature the WALA recipes. 1. IR1.1: 170,724 vulnerable households have increased use of quality maternal and child health and nutrition services. Activity 1.1.1: Development of behavior change communication strategy. Behavior Change Strategy. Technical staff from CRS headquarters led a workshop on barrier analysis early in the project. The workshop included field work in sites in Zomba District to study a couple of behaviors. After the training, partners were on their own to conduct the barrier analysis studies on behaviors and use the findings in developing messages. Some partners conducted barrier analysis on a few behaviors, mostly related to sanitation and hygiene, but no reports are available, and there was no further development of a behavior change strategy across the project. WALA Mid-Term Evaluation Report 18 March 2012 6 The USAID-funded Infant and Young Child Nutrition (IYCN) Project, with additional funding from the World Bank, conducted extensive qualitative research on complementary feeding practices in Malawi, including the use of Trials for Improved Practices, to identify acceptable food mixtures to improve nutrient intake of young children. The IYCN study was conducted in the north, but many of the results may be applicable to WALA districts. Perhaps, because of turnover in the MCHN TQC position, WALA did not take advantage of this study in designing the complementary feeding module and flipcharts. BCC Materials. WALA adopted the training module and flip chart from I-LIFE on hygiene and sanitation. The flip chart clearly conveys all the generic messages about desired practices in hygiene and sanitation. In addition to using the flip charts for home visits, volunteers also conducted demonstrations on how to make dish drying racks, clotheslines, and hand washing stations using local materials. The training modules and flipcharts developed by WALA for breastfeeding and complementary feeding were field tested for comprehension and contain generic information that would be acceptable for all of Malawi. They are not specific to the context and key issues of the target area. Contextualization during promoter training varies by field supervisor or coordinator, but is not based on any particular formative research. Officials from the MOH and Office of the President and Cabinet (OPC) have initiated discussions with WALA about using the flip charts as a basis for national standardized materials. There are two strategies for promoting behavior change. The overarching strategy is the use of Care Groups to reach all target families directly. Care Groups will be discussed in the following section. The other strategy for promoting behavior change is the Community Complementary Feeding and Learning Sessions (CCFLS) which are described in detail under Activity 3 of SO1. CCFLS is conducted as one of the Care Group activities. Gender. While there are almost twice as many male promoters as females and over 1,100 male Care Group Volunteers, there has not been a strategy for engaging men, particularly the fathers of young children and husbands of pregnant women in MCHN activities. The contact with men has been through home visits when they happen to be present. At a higher level, WALA convened a “Couples Conference” in Blantyre as a major event to raise awareness about the role men can play in assuring optimal nutrition for women and young children. The event brought couples together ranging from high-ranking government officials to community leaders. The intent was for this to be replicated at more local levels, but that has yet to be rolled out. During the evaluation, men who were interviewed consistently expressed their desire and willingness to learn more about nutrition and health. Other than with promotion of orange￾fleshed sweet potatoes, it appears that nutrition is not brought into discussions with farmers in relation to the crops they are planting. Dietary Diversity. Increasing dietary diversity is a critical element for improving nutritional status in Malawi by reducing barriers to availability. The WALA baseline shows that the consumption of animal products, oils, and Vitamin A source foods is particularly low. There is an over-reliance on green leaves of various kinds as the main source of both iron and vitamin A. Iron in this form is not bio-available unless consumed with a source of heme iron (animal food) and the amount of Vitamin A is significantly lower than in dark yellow vegetables or fruit. Families have very few chickens, which are left to find all their own food, and flocks are frequently decimated by epidemics of Newcastle disease. The poorest families do not own goats WALA Mid-Term Evaluation Report 18 March 2012 7 and, families who own them rarely eat them, preferring to sell the meat. WALA has an agreement with Bunda College to develop and test recipes for complementary foods using locally available ingredients. Chancellor College is providing technical assistance on food processing and preservation with an end to assuring that families have nutrient-rich foods all year around. WALA will have to verify that the outputs of these collaborative efforts are indeed feasible and appropriate to recommend to the poorest families. RECOMMENDATIONS - BCC 1. PRIORITY Contextualized Messages. During training for new modules, give WALA staff concrete examples of how to contextualize the training of promoters and Care Group Volunteers to priority messages for the situation. 2. PRIORITY Activities for Men. Plan activities for men at least quarterly making them fun and limited to key messages that will enable men to understand their role in improving nutrition for their wife and children. 3. PRIORITY Sweet Potatoes, Papaya, Groundnuts & Oilseeds. With the technical working group for agriculture, define plans to expand introduction of sweet potatoes (particularly in Nsanje and Chikwawa), distribute papaya seedlings with technical assistance, increase production of ground nuts for home consumption, and investigate production of oilseeds and local presses for increasing local consumption. 4. PRIORITY Poultry . Link poultry activities to Care Groups, focusing on training and sourcing for Newcastle disease and other vaccinations, access to locally produced chicken feed as a supplement for free-range chickens, and production of a one-page flyer on caring for local chickens for use by Care Group Volunteers. 5. Impact on Diet Diversity. In the next annual survey, compare data on food groupings with the baseline and recent survey to assess impact of dietary diversity messages and activities. Activity 1.1. 2: Formation of Care Groups Care Groups are a behavior change methodology originally developed by World Relief for Child Survival and later adapted by Food for the Hungry (FH) for Title II programs. Due to the success of the model, which was widely publicized by FH, the I-LIFE Program adopted the model in the last years and carried it over into the design of WALA. The model enables a limited number of staff to work through two layers of volunteers to reach many households through individual contact. As of the end of FY 11, WALA had 30 field supervisors (sometimes called facilitators) working with 178 promoters (volunteer but paid a stipend) who were training over 13,500 Care Group Volunteers (CGVs), that were transmitting the messages to nearly 140,000 households. The keys to Care Group success include the one-to-one contact with each targeted household, which greatly enhances comprehension and positive reinforcement, and inclusion of women who otherwise might not participate in group sessions. Further, the CGVs are “insiders”, selected by the women in their cluster. Organization of Care Groups. After conducting a complete census of each target village, WALA convened a meeting, often in collaboration with the local chief, of all families with children under five, pregnant women, and sometimes women from other vulnerable households Table 3. WALA Human resources for SO1 (all partners) Position Total Male Female Coordinators 7 3 4 Field supervisors/facilitators 30 14 16 Promoters 178 96 82 Care Group Volunteers 13,610 1,149 12,461 WALA Mid-Term Evaluation Report 18 March 2012 8 Sustained behavior change During the torrential rains which occurred in January, 2012, some latrines suffered structural damage. When asked what these families were doing without a latrine, CGVs reported that they are using neighbors’ latrines rather than returning to old habits of going in the bush, and they already have plans for rebuilding their latrine. such as those with orphans or the chronically ill. At the meeting, all the families were divided into groups of ten, usually based on geographic proximity, but sometimes through self-selection. These clusters then elected one of their group members to become the CGV based on her/his ability to read and write, her/his leadership, and other characteristics such as empathy or humility. It is these volunteers who form the Care Group (CG). The project most often refers to the families in the clusters as beneficiaries. Nearly ten percent of the CGVs are men. One WALA partner says a lesson learned is that the Care Groups are more dynamic when there are some male members. Performance. The CGVs meet for two hours every two weeks with the promoter to learn a particular topic, which they are to transmit to the cluster members, preferably through home visits. In reality, many CGVs have shifted to holding group meetings to transmit the messages, which is easier for them. Although they report attendance is good and that they make home visits to anyone who does not attend, there are concerns about this self-modification. First of all, the flip charts are too small to use effectively in a group since they were designed for individual contacts, and the CGVs have not been trained to facilitate group sessions. Secondly, in the group, shy women may not get their questions asked or concerns addressed. Another major concern is that the group meetings preclude contact with other family members, particularly husbands, who are often engaged during household visits. There may be times, such as during periods of intense farming activity, that group meetings would be acceptable in order for both the CGV and her cluster members to save time. In general, the many CGVs interviewed during the evaluation, have attained a great deal of knowledge through the training they have received on hygiene and sanitation, breastfeeding, and complementary feeding. Importantly, they understand the rationale behind many of the behaviors they are promoting such as personal hygiene, exclusive breastfeeding, and frequency of feeding young children, which enables them to persuade others to adopt these behaviors. The CGVs, however, have not retained some very key messages within the many they have learned. For example, only one or two knew that mothers have to breastfeed more frequently to increase milk production. The training modules they receive do not bring the most critical messages for their context to the fore, but rather, include many generic messages. While the promoter starts each session with a review of the previous one, there has not been a review of key content of previous modules for those CGVs who started their training over two years ago. There seemed to be minimal distortion of messages in this chain from facilitator to promoter to CGV to household. During the evaluation, interviewees from each level were asked the same three technical questions and the end result was that mothers were getting the clear message. For example, one of the questions was “What is the porridge like that is the first food?” This question about consistency elicited the same response from all levels that the porridge has to be thick enough not to flow off a spoon. The evaluation revealed that promoters seldom accompany CGVs on their home visits or while they are conducting group sessions. CGVs said that they can call upon the promoter to accompany them to a household with a problem. Some promoters reported that they make spot-check visits to households to see if the CGV has been there recently. There is no system in place to monitor the performance of the CGVs other than by the numbers they WALA Mid-Term Evaluation Report 18 March 2012 9 present in monthly reports or to provide on-the-job mentoring. Achievements. The first module for the Care Groups was Sanitation and Hygiene. This material was adopted directly from I-LIFE and hence, ready for use as soon as the CGVs were selected. The module includes not only promoting behaviors and latrine construction, but also objects that facilitate hygiene and environmental hygiene. These include dish drying racks, clothes lines, and hand washing stations (tippy taps) near the latrine and cooking area. Filled with enthusiasm for their new role, the CGs have been very successful in getting families to adopt all of these item and to construct latrines. The CGs visited during the evaluation had succeeded in getting around 100% coverage of latrines in their communities, not just among the families in their cluster. When asked how they succeeded when the MOH through the VHC and other NGOs have been promoting latrines for years without reaching such a high level of coverage, the CGVs said it was because of the direct contact with each family to apply peer pressure rather than someone conveying the message in a general community meeting. They also pointed out specific pictures in the flip chart which helped them persuade families. The sanitation module served to engage men from target households because of the construction activities. The annual survey conducted in September, 2011 showed that 79% of households with children under five now have latrines. In the baseline in 2009, this was not calculated for this denominator, but will be calculated in time to compare with the next annual survey. It seems probable that the next annual survey will also show improvements in the indicators for exclusive breastfeeding and complementary feeding. The evaluation observed that children born to CGVs since they were selected are notably larger and healthier than other children of the same age, and mothers affirmed that they have internalized and applied the practices they are teaching their peers. The CGVs began promoting fuel efficient stoves after the September survey, but they report rapid adoption by WALA beneficiaries and beyond. Motivation and Incentives. Starting the CGVs with the hygiene and sanitation module was motivating because they could quickly see tangible results in families adopting dish racks, hand washing stations, and latrines. They have mentally tracked how many families had latrines when they started promoting them and how many more families have adopted them. The results of the breastfeeding and complementary feeding modules are less tangible to them. If they understood that the results are visible in such things as more children maintaining normal weight for age, fewer cases of malnutrition, lower rates of diarrhea due to exclusive breastfeeding, and so on, they could monitor these outcomes in simple pie charts or graphs to share with the Village Health Committee (VHC), chiefs, the Health Surveillance Assistant (HSA) and others. Seeing these changes could be quite motivating, particularly if they articulated goals with the VHC and chief such as to reduce cases of malnutrition by half. To date, CGVs have received minimal incentives such as wrappers and t-shirts. Staff report that some are dropping out or becoming inactive because they feel they need more incentives. During the focus group discussions in the evaluation, no strong demand for incentives emerged. Some CGVs said they need stationary items to keep their records or an umbrella for the rainy season, and others requested a trip out of their villages for training. Further probing revealed that they think such a trip would (a) increase their credibility because community members would perceive that as more significant training than the current bi-weekly meeting, (b) motivate them simply by getting to leave their village, and (c) provide some cash per diem which would be well-received by husbands who would then more value their volunteer work. WALA Mid-Term Evaluation Report 18 March 2012 10 Future incentives, whether a trip to a district level training, a WALA t-shirt, or a certificate from the chief, could be tied to performance. Performance would be measured not only by the number of home visits, but by accomplishments such as latrine coverage, increase in the number of normal weight children in their cluster, and so on. Sustainability. CGVs can be considered as a means to an end, that is, a channel for promoting behavior change in much the same way some programs use mass media. If the target population adopts the new behaviors at a high rate, then, social norms will have been changed; and sustaining the CGVs may not be necessary, at least not unless they are given new practices to promote. Most will remain in their communities with their knowledge and flip charts in case families want to consult them for advice or they have the personal motivation to seek out newly pregnant women or young mothers. If the social norm has changed, these young mothers will be learning desirable health and nutrition practices from their own family members. If the MOH wishes for the structure to continue, they will have to expect the HSA to fill the role of the promoter. With the MCHN+ funding, this is being piloted in four districts with five HSAs. HSAs will need the training modules, if not the training, to be able to train CGVs. Some HSAs are quite enthusiastic about taking on Care Groups while others are adamant that they do not have the time. One concern is that the District Health Officers (DHOs) in the pilot districts would like to see the CGs work with the entire population on a wide range of health issues rather than focusing on children under five and their mothers. There is some potential for VHCs and existing Care Groups to mutually support each other. They could identify specific issues to address, make an annual plan of action, and monitor their accomplishments. This would likely be a less intensive effort than currently, but would keep some activities going, if only on an irregular basis. RECOMMENDATIONS - Care Groups 6. PRIORITY Home Visits Versus Group Sessions. Emphasize home visits over group sessions 7. PRIORITY Review Sessions. Select priority key messages and conduct a review session for all CGVs 8. PRIORITY Participatory M&E for CARE Group Volunteers. Help CGVs start their own monitoring system with goals for key behaviors 9. PRIORITY Incentives for CGVs. Arrange incentives that are performance-based and low-cost or non-monetary 10. PRIORITY Linkages Build stronger linkages between Care Groups and health staff and VDCs. Activity 1.1 3: Conducting Community Complementary Feeding and Learning Sessions (CCFLS) The CCFLS sessions are a modification of Positive Deviance/Hearth which was implemented in the early years of I-LIFE. The key modification was supposed to be inclusion of all mothers to participate in the hands-on learning of skills during 12 daily sessions. Other adaptations included using the sessions to teach food processing and gardening. Otherwise, key concepts of PD/Hearth remained: having mothers bring all the ingredients and materials, sessions facilitated by care group volunteers (CGVs), nutrient-dense menus of locally available foods, and weighing the children the first and last day. WALA Mid-Term Evaluation Report 18 March 2012 11 In reality, the focus has been recuperation of moderately malnourished children. To that end, CCFLS has been quite successful. Although data had not yet been compiled, a review of several registers shows that most participating children have gained an average of around 500g, which is slightly more than the usually recommended PD/Hearth graduation rate of 400g. Some of the partners have done positive deviance inquiries at least once, but it is unclear what the findings were or how they were applied to the CCFLS. The seasonal calendars that were developed with the communities have been used in determining locally available foods at different times of the year, but were not always complete. The major challenge has been having the mothers contribute food, particularly during the lean season. This may indicate that recipes were not adapted to food availability during that season or that mothers are being asked to bring more than is needed for the exact number of participant children. There are two major issues of concern. The first is that, contrary to the standards, most sessions only include mothers of malnourished children with possibly a few other “interested” mothers. Some partners said that other mothers were invited but did not come. One partner has the mothers of non-malnourished children come for only six days of the twelve, which is not enough time for them to practice new skills. The most concerning issue is group size. In the standards and the training, coordinators were told that it was acceptable to have many participants as long as there were different cooking pots. CCFLS sessions observed during the evaluation had from 25 to 233 participants. If there were multiple pots, they were crowded into a small area making it impossible for mothers to get close to participate in cooking or observe. Most of the actual food preparation in some sites was done by the CGVs with the mothers simply observing or waiting out of the way to feed their child. The whole concept of participatory hands-on learning was lost. Research shows that skills acquisition declines sharply as group size increases above eight members. The ideal would be for each CGV to conduct CCFLS at her own house for her own group. This had happened quite successfully in one village. Discussion with the coordinators revealed that the motive for having large groups or multiple groups at one site is the role of the promoter in supervision. In most villages, the promoter would be able to circulate between the different houses where CCFLS would be held. If he/she has trained the CGVs by walking them through sessions, and they are clear on the steps to be implemented and recipes, they may not need much supervision. Another idea is to stagger the start by one week, that is, have five CGVs start in week one with help from the other five and the promoter. In the second week, the five “helpers” start CCFLS sessions for their group on their own, each at her own house. When mothers and CGVs were asked what they learned at CCFLS, the response was always “porridge recipes”. CCFLS is intended to teach many skills including active feeding, safe food handling, hygiene, environmental sanitation, as well as concepts such as feeding frequency, consistency, and amount. The sessions could also include some simple early child development (ECD) activities with the children while they are waiting for the food to be served. Mothers should take turns at different activities such as preparing food, helping children with hygiene, and clean-up on different days. RECOMMENDATIONS - CCFLS 11. PRIORITY Group Size. Limit groups to no more than 10 mothers in separate places. WALA Mid-Term Evaluation Report 18 March 2012 12 12. Seasonal Participation. Invite all mothers to participate at least twice in different seasons. 13. Food Processing. Save the food processing lessons for harvest time outside CCFLS. 14. PRIORITY Recipes. Make sure the recipes are truly local using affordable ingredients and do not call for processes that burden the mother with extra time spent on food preparation. 15. PRIORITY Feeding. Highlight active feeding, consistency, and amount. 16. Field Supervisor Checklist. Field supervisors need to use the Quality Improvement and Verification Checklist to assure the standards for CCFLS, including small group size, are being met. 17. Followup Training for Field Supervisors. Field supervisors may need more training on the key concepts to strengthen this with the promoters. 18. Identifying Malnourished Children. Weigh in at the first session. Screen with MUAC to make referrals. Weigh only the malnourished children after the last session and refer those who did not gain weight. 19. Reporting Burden. Reduce the reporting burden by randomly selecting only a few sites in each district to report only the weight gains and attendance of malnourished children. 2. IR1.2: 170,724 vulnerable households have increased use of quality maternal and child health and nutrition services. Activity 1.2.1: Support to the Ministry of Health to roll out C-IMCI Roll-out of C-IMCI is programmed for FY13. The national materials need to be adapted to the CGV level. WALA staff have the capacity to make the appropriate adaptations. These materials can also be used to train the village health committees. The government, with the support of other agencies, began to train HSAs in “hard-to-reach” areas in Community Case Management (CCM) to give them the skills to diagnose and treat prevalent illnesses such as diarrhea, malaria, and pneumonia. Those trained are to be supplied by the DHOs with the needed supplies and drugs. Since the training did not reach all HSAs, WALA discussed training HSAs in the remaining hard-to-reach areas. More recently, WALA staff learned that there is a problem of HSAs being trained but either leaving or refusing to be posted in the hard-to-reach areas. WALA may reconsider the CCM training after more discussion with the DHO. Activity 1.2.2: Reinforce community-based growth monitoring and promotion (GMP) With the MCHN+ funding, WALA is using a methodology called Partnership Defined Quality (PDQ) which brings community members and health workers together to determine how to improve GMP and how this will be done. As a pilot, staff members were trained in PDQ in four districts and are facilitating the process at the community level. PDQ will next be introduced in the remaining districts. One issue being identified and addressed is lack of shelter, latrines and hand washing stations at the site, and this is being addressed by community mobilization to construct shelters. Another issue identified is the limited number of staff to conduct GMP which results in lack of time for counseling or plotting. This is being resolved by expanding the role of CHVs to assist with counseling. Other issues which the DHO plans to resolve include training of HSAs and provision of an adequate amount of supplies. Activity 1.2.3: Designing and implementing a quality improvement and verification checklist WALA Mid-Term Evaluation Report 18 March 2012 13 To date, the Quality Improvement and Verification Checklist (QIVC) has been developed only to monitor WALA program activities in comparison to the standards for each activity. These are just recently finalized and being introduced. WALA is considering how best to integrate quality guidelines into the C-IMCI and to improve on some existing guidelines used for growth monitoring. Activity 1.2.4: Provision of targeted supplementary feeding program (SFP) to malnourished children and pregnant and lactating women. The SFP is implemented in accordance with the Government of Malawi policy and in coordination with the DHO, particularly the Nutrition Rehabilitation Units (NRU). During the community growth monitoring, any children detected with growth failure or inadequate weight for age are then screened by the HSA with a measure tape for Mid-Upper-Arm Circumference (MUAC). Children detected with severe acute malnutrition (SAM) or moderate acute malnutrition (MAM) are referred to the NRU. Children with SAM and complications are admitted to the NRU, those with uncomplicated SAM are enrolled in Community Therapeutic Care (CTC)5 to receive ready-to-use-therapeutic food (RUTF). Among the children with MAM, only those with the highest level of malnutrition (MUAC between 11 and 11.9 cm) are enrolled in SFP to receive rations from WALA for four months. Recent monitoring data for SFP indicates that most partners are meeting targets for food distribution for malnourished children. Malnourished women are identified as those with a MUAC of less than 22 cm. Most are identified during antenatal care, but at least one partner is also doing mass screening quarterly. The malnourished women receive a food ration for one year. WALA has had difficulty meeting the target for food distribution to malnourished women in some districts. This may be due to the fact that there is not universal coverage of ANC, which is quite low in some of the target districts, and women are not detected by another means. On the other hand, there may not be as many malnourished pregnant women as WALA originally anticipated. WALA continues to advocate with the government to increase the MUAC cutoff for children eligible for SPF. It now appears that government may agree to change the cut-off to 12.5 cm, which would make many more moderately malnourished children eligible to receive SFP. RECOMMENDATIONS - SFP 20. PRIORITY Integration. CGVs should provide close follow-up to SFP-enrolled families to assure that they are integrated into other WALA activities such as gardens, poultry, VSL, and agriculture, as appropriate. 21. ANC Attendance CGVs can promote and monitor ANC attendance 22. PRIORITY Safety Net. Consider shift of food resources to safety net if targets for women and children cannot be met. 3. IR 1.3: 2,148 community groups have enhanced capacity to promote health and nutrition services for 170,724 targeted households. Activity 1.3.1 Training for community groups in management and sustainability. 5 CTC is now known globally as Community Management of Acute Malnutrition (CMAM), but the acronym CTC is still widely used in Malawi by government and NGOs. WALA Mid-Term Evaluation Report 18 March 2012 14 This activity overlaps to a certain degree with the cross-cutting theme of governance. Some of the CGs have received training in governance resulting in their elaboration of internal rules. Many more have received training in group dynamics which is critical in sustaining group cohesion. Village Health Committees have not, as yet, received this type of training. Activity 1.3.2 Establish linkages to district development authorities. To date, the relationships are primarily between WALA staff and district authorities as will be described later in this document. The kind of relationships that exist between health staff and CGVs is highly variable across WALA. Village health committees do not seem to be connected to village development committees or traditional leadership structures. RECOMMENDATIONS - Community Capacities 23. Staff Responsibilities. WALA will need to clarify which staff member is responsible for forging the relationships between community groups and district development authorities before this can move forward. 24. Advocacy on Nutrition. It may be more crucial to focus on advocacy with district development authorities to add nutrition to their priorities. 4. General Comments on SO1 OVERALL RECOMMENDATIONS FOR SO1 25. PRIORITY Positive Reinforcement. Shift the focus to positive reinforcement: Reward mothers or parents whose children are normal weight at certain ages with community recognition or another non-monetary incentive. 26. PRIORITY HSA Capacities. Train HSAs to the same level as the health promoters. 27. Performance Indicators for SO1. Some performance indicators can be revised for clarity as described in the knowledge management section. B. Strategic Objective 2: Agriculture and Natural Resource Management (ANRM) SO2: Livelihood capacities of 147, 500 food insecure households are protected and enhanced. Table 4. Outcome Indicators for SO2 and Achievements* by the End of FY 2011 Indicator Baseline FY 2011 % of beneficiary households using 3 out of 5 WALA promoted sustainable crop cultivation technologies (quality seeds, crop rotation, intercropping, minimum tillage, or mulching) 27% 30% % of beneficiary households using two of the three WALA promoted soil conservation technologies (fertilizer or leguminous trees, contour ridges, box ridges and bunds, or vetiver grass) 12% 16% % of beneficiary households using post–harvest handling and post–harvest storage technologies 34% 41% % of beneficiaries (individual farmers) using at least 2 of 4 WALA promoted Integrated Pest Management technologies (mechanical, cultural, crop rotation, nonchemical or organic pest control products) 9% 10% % of irrigation sites installing two or more WALA promoted design considerations (lined canals, drip systems, adequate field drainage structures, vegetative cover and shutoff valves) 0% 100% % of water users committees using two or more WALA promoted operational practices (user fees, high frequency low flow watering schedules, structure maintenance clearing of weeds and sediments, periodic drainage of waterlogged fields, periodic flushing out of canals to remove 0% 42% WALA Mid-Term Evaluation Report 18 March 2012 15 snails) Number (cumulative) of hectares of land brought under irrigation by WALA activities 0 220 Total Savings (cumulative) deposits held by USG–assisted micro–finance institutions 0 US$755,420 Percentage of Village Savings and Loan members who used loans or savings to establish productive investments (agro–enterprise, other micro–enterprise, or to purchase agricultural inputs) 0% 53% Percentage of beneficiaries (individual farmers) who cultivated at least two of five WALA– promoted priority products (rice, beans, groundnuts, pigeon peas, poultry or fish) 38% 40% *Note that the comparison is illustrative rather than direct since different sampling methods were used: baseline sample was drawn from a population based sample frame, the FY11 survey was drawn from a participants sample frame. Under the Agriculture and Natural Resource Management (ANRM) objective, the original design of the WALA Program targeted increasing skills and knowledge of smallholder farmers on conservation agriculture, improving the availability of and access to quality seed for staple food crops, increasing food production through irrigation, strengthening governance and financial management skills using a Village Savings and Loan (VSL) approach and enhancing business and marketing skills of a sub–set of smallholder farmers and successfully linking them to domestic and regional markets. Table 4 compares performance on SO2 outcome indicators listed in the IPTT using baseline data and data from the 2011 Annual Survey conducted in September and October 2011. In general, across these indicators, the program has shown little significant impact, apart from those outcome indicators associated with VSL. The evaluation, however, observed that the interventions being implemented are appropriate, and the program is poised to achieve significant progress in its remaining life, as described in the sections which follow. 1. IR 2.1. 147, 500 smallholder farming households apply improved crop production practices. Activity 2.1.1 Capacity Building of sustainable extension services to improve production The approach being used by WALA for developing an extension service delivery system begins with identifying Farmer Extension Facilitators (also called Farmer Extension Volunteers, hereafter referred to as FEFs) who reside in the communities6 being targeted by the program. The selection criteria for these persons are that they must be farmers themselves, have some education, and be respected in their communities. The program builds their capacities around the key technical ideas and approaches being promoted and provides logistical support in the form of a bicycle and work supplies. Depending on the implementing partner, some of the FEFs receive a cash operations and maintenance allowance for their bicycle, some receive a volunteer stipend and some receive no cash support. FEFs are expected to pass on their knowledge and training to selected farmers designated as lead farmers who are nominated from farmers within each village in which the program is working based on their farming skills and respect in the community. Each lead farmer works with a producers group of around twenty other farmers from the village. The FEFs provide technical training to lead farmers around a demonstration of the different ideas and technologies being promoted. The program provides some input support for the demonstrations, primarily hybrid seed or other improved seed, to also demonstrate these. Lead farmers are then expected to pass on their knowledge to the farmers in their producer group through bi–weekly meetings and on– 6 In WALA, the concept of "community" refers usually to a Group Village Headman (GVH), a unit of local government that encompasses a small cluster of villages WALA Mid-Term Evaluation Report 18 March 2012 16 An Innovative Lead Farmer One lead farmer interviewed during the evaluation has four plots of maize, planted at the same time in one location. In three of these, he is experimenting with different combinations of the ideas being promoted by the program. The fourth is a traditionally cultivated crop of maize for comparison farm visits. Standard Operating Guidelines for the Agriculture component of the program are currently being developed to guide implementation7 . The latest information available through December 2011 indicates that the program has recruited 253 community–based Farmer Extension Facilitators and has facilitated the establishment of 5,972 producer groups encompassing 116,175 farmers. This represents 78% of the LOA target for participating farmers. Annex H contains a statistical summary of this information by implementing partner. The WALA Program is working in 218 Group Village Headman (GVHs) at the present time, which means that, on average, there is at least one FEF per GVH. Implementing partners, however, are using different approaches, ranging from nearly five FEFs per GVH on one district to one FEF serving two GVHs in another district. The average number of farmers supported indirectly by an FEF is around 460 across the program, but this ranges from around 230 farmers per FEF for three partners to a high of over 980 farmers served indirectly by an FEF for another partner. As will be explained in the next section, there has not yet been extensive real adoption of the technologies being promoted by the program, but qualitative interviews and observations did not reveal any great differences between partners on adoption rates despite the wide range of extensionist to farmer contact. In general, the selection of farmer extension facilitators and lead farmers by the program appears to be effective. Those interviewed during the evaluation were real farmers; most were recognized as leaders; and some were real innovators. In at least one location, the FEF function has been merged with the Agribusiness Community Agent (ACA) function in one person, since the implementing partner found it difficult in this relatively remote location to find enough qualified candidates to fill both positions. One advantage to this arrangement is that the conservation agriculture techniques and other technical support provided through the FEF can be applied to other crops being grown for marketing, such as chillie peppers. The risk in this arrangement is that the work of the FEF or the work of the ACA does not achieve a satisfactory level of quality because (a) the individual does not have the right skill sets to do both jobs or (b) he or she is expected to cover too many groups. Field observations and discussions with FEFs and field staff during the evaluation consistently indicated that the job of establishing the new ideas being promoted by WALA will certainly not be completed in the life of the program. There are only two more primary growing seasons after this year before the program finishes, and it is unlikely that the ideas will have become established firmly enough to have reached the point at which they will diffuse more widely on their own. RECOMMENDATIONS – Extension Services 28. FEF to Farmer Ratio: In the next annual survey, a large enough sample should be selected to obtain data to be able to compare information between PVOs on adoption outcome indicators to determine whether there is link between adoption and the FEF to farmer ratio and a recommended ratio should be agreed upon. 7 There has been significant turnover in the position of TQC agriculture over the life of the program which is a significant reason for the delay of the development of Standard Operating Guidelines.. WALA Mid-Term Evaluation Report 18 March 2012 17 Conservation Agriculture in WALA Marker Ridges: Used to indicate contours Ridge Realignment: Aligning ridges along contours indicated by marker ridges Compost Manure: Using vegetative matter and animal manure to make compost Pit/Basin Planting: Using compost, planting continuously in small pits or basins rather than rows to capture moisture Sasakawa (One–One): A maize plant spacing technique which increases plant density Mixed Cropping: At the moment in WALA, focused on maize and pigeon pea Mixed Crops & Trees: Planting leguminous trees such as Faidherbia albida (Msangu), Tephrosia vogelii, or Sesbania sesban in maize fields. Mulching: Using maize crop residues or other vegetation as mulch. Burying Crop Residue: Burying green crop residues to accelerate decomposition. Box Ridges: Cross–ridges between rows to capture and hold water in the row. 29. PRIORITY Standard Operating Guidelines. The extension component of WALA is in need of a clear, contemporary, context appropriate vision. The Standard Operations Guidelines should be completed as soon as possible since this will operationalize the program's vision. 30. PRIORITY FEF Sustainability Strategy. The program should develop a strategy for enabling FEFs to continue to serve in this function after the program ends. Elements of this strategy would include: • Stronger links between them and sources of new ideas and technologies in government, the NGO sector and the private sector • Stronger links specifically with the Department of Agricultural Extension, so that by the end of the program, FEFs are perceived by government as an extension of their systems. • Identifying services that they can provide to farmers from which FEFs can generate income and be motivated to maintain contact with farmers. 31. Systematically Monitor the Effectiveness of the Combined FEF/ACA. While TQCs from both the agribusiness and extension components of the program have visited the location where the functions of the FEF and ACA have been combined, a more rigorous and systematic analysis should be conducted to determine whether this is a viable model for facilitating sustained extension and marketing services. Activity 2.1.2 Promotion of optimal crop production technologies In this set of activities, the WALA Program intended to deliver key new ideas and technologies on conservation agriculture and post–harvest storage and handling through the FEF/Lead Farmer/Producer Group structure, to develop sustainable irrigation systems with Water Users Groups and to strengthen local seed supply systems. Conservation Agriculture. The concept of conservation agriculture used by the WALA Program revolves around three core principles: minimum tillage, mixed cropping using legumes and soil protection. The specific techniques being promoted by WALA that were observed during the evaluation are shown in the box. Technical support on conservation agriculture has been provided annually by the Conservation Farming Unit from Zambia. In most areas, the program is still at the stage of demonstrating new ideas and technologies. There appears to be relatively little wide–scale adoption. The evaluation observed some “fake” adoption in which respondents were trying the techniques primarily to please the program to see what other benefits could be obtained. The evaluation also observed, however, some true adoption in which the adopters were really testing the ideas for themselves. The concepts being promoted are not complex, and many participating farmers already know about legumes and compost. Many of the new ideas and technologies WALA Mid-Term Evaluation Report 18 March 2012 18 being promoted have the potential to have measurable impact on soil quality and farm production over the long–term and do not require external inputs, e.g., basin planting, mulching, ridge realignment, and box ridges. They can fairly easily be adopted when farmers see benefit. Farmers, however, need to be reassured that the techniques being promoted by the program are going to be beneficial for them in terms of increasing production or reducing input costs, including labor. Effective demonstrations will be critical to convincing farmers The coverage of the demonstrations in WALA, however, is relatively small, and one has to search in a particularly village to find the location of a WALA demonstration. The demonstrations that have been established, however, are sited well, along roads or paths where they could be visible, but information about the treatments or varieties being demonstrated was missing or incomplete. None of the demonstrations observed during the evaluation offered a structured comparison between the new approaches being promoted and traditional cultivation techniques. The concept is that the new techniques will achieve greater production, under the same conditions, as traditional cultivation, but the proof was not being demonstrated. Sometimes, by coincidence, a traditional field would be near the demonstration plot, a comparison could be made, and the demonstration was persuasive. Other times, the nearby field of traditional maize would clearly be out–performing the demonstration because it was planted earlier (or later in some cases). Because there was no traditional plot included as part of the demonstration to show how the new techniques perform better, under the same conditions, these demonstrations were hardly persuasive. All demonstrations of conservation agriculture that were observed were being applied around maize, although the techniques are certainly valid for other crops, including drought–resistant crops such as millet and sorghum. Government input subsidies are supporting maize cultivation, however, making it challenging for the program to promote other risk reduction crops such as sorghum and millet. In order to persuade a participating farmer to set aside some of his or her land for demonstrations, the program provides seed for the demonstration. Often this seed is hybrid seed which certainly outperforms traditional varieties. However, highly food insecure households producing maize primarily for consumption may find it difficult to purchase hybrid seed. Also, using only hybrid seed in demonstrations may cultivate the perception that the conservation Persuasive Demo: WALA on the right, traditional on the left. Unconvincing Demo: WALA in the foreground, traditional in the rear WALA Mid-Term Evaluation Report 18 March 2012 19 agriculture techniques only work with hybrid seed. At least some demonstrations in each location should use local varieties of seed. The WALA Program is advocating for use of compost manure, but there is little support for expanding manure production, which is a problem for those households with few animals. The program does not have systems in place for monitoring how farmers are adapting the ideas in their own fields or for identifying farmer innovations that could be extended through the program. RECOMMENDATIONS – Conservation Agriculture 32. PRIORITY Quality of Demonstrations. All future demonstrations should clearly indicate using on–site sign boards in Chichewa what is being demonstrated and should have an area set aside showing traditional cultivation practices for comparison. 33. PRIORITY Additional Demonstrations. In order to increase the visibility of the techniques being promoted (and encourage adoption), the program should take advantage of opportunities to show the techniques by preparing small signboards in Chichewa to be placed in fields of early adopters who have convincing demonstrations. 34. PRIORITY Farmer–Driven Innovation. Most of the new ideas and technologies being promoted by WALA are coming from the government research or extension systems and in some cases the private sector. The program should put in place a system for encouraging lead farmers, FEFs, and front–line staff to find new ideas being tested and adapted by farmers themselves. 35. Advocate for Government Input Subsidies for Drought–Resistant Food Crops. Food crops such as millet and sorghum are much more appropriate for some parts of the WALA program area than maize, but since government support is focused now on maize and cotton, farmers are pulled toward production of these crops. Given its scale, WALA has some clout that could be brought to bear in an advocacy strategy to increase government support for drought–resistant food crops. Small-Scale Irrigation. In this component of the WALA Program, investments are being made in small–scale irrigation systems. A micro–granting process and/or Food–for–Work, in some cases, is being used to build irrigation structures under the technical guidance of Agricane, an international irrigation consulting firm wth an office in Malawi, which has assigned four full– time staff to the seven implementing partners. The program organizes a Water Users Group around each system and provides training on leadership, group dynamics, water scheduling and resource mobilization. An Irrigation Field Manual and Standard Operations Guidelines have been developed under the guidance of the Technical Quality Coordinator for Irrigation. A separately-funded activity called Wellness in Irrigation for Life Advancement (WILA) is being implemented alongside the WALA irrigation activities using the same strategy. At the time of the evaluation, thirty–six schemes were under development with a total catchment area of 220 hectares farmed by 2118 households. The area covered represents 44 percent of the LOA area target. Fourteen of these systems are gravity–fed river diversion A Simple River Diversion System WALA Mid-Term Evaluation Report 18 March 2012 20 systems, and the remaining twenty–two are using treadle pumps. None of the systems are completely finished although many of them were able to produce a crop under irrigation last dry season, although it is not clear how much of the 220 hectares has been irrigated. The LOA target for irrigation is focused on area covered, and since river diversion/gravity–fed systems can cover a wider area than a treadle pump, the trend in the program has been to move away from treadle pumps and toward river diversion. Further complicating this is that in areas where there is a significant opportunity cost for labor required to pump the pumps, participants appear to be losing interest. Every irrigation site has its unique design challenges, and for the most part the program is meeting these technical challenges fairly well. Most of the systems observed were functional and able to handle the water load during the wet season although the peak in some areas is yet to come. The irrigation systems so far appear to only be used during the dry season, even though there are intermittent dry periods during the rainy season that stress crops. Sometimes this is related to land holding arrangements in which members of the water user groups are given access to land for irrigation during the dry season, but then the land reverts back to the original land owner during the rainy season. Sometimes it seems to be related to a preference for farming traditionally in the rainy season, even though irrigation is available. In any case, the systems are not being used as much as they could be; and consequently, the full value of the systems are not being realized by Water Users Groups. When these beneficiaries realize the full value, i.e., water available for irrigation year round for both food and cash crops, they will have a much stronger sense of ownership in the systems. There appear to be minimal threats at this point to land grabbing or otherwise losing access to land for irrigation. For all systems observed during the evaluation, access to land has been either given or witnessed by GVHs, Traditional Authorities and District Commissioners. Some of the systems are too small to be of interest to very powerful people. Water User Groups have been established and trained around all sites. The training provided has covered topics related to group organization and dynamics, water scheduling, and resource mobilization. Given the importance of establishing functional and sustainable management systems around the irrigation systems, however, far less time has been spent on this "software", than on the "hardware" of the irrigation system itself. For only two of the seven systems observed were resources being mobilized. While it is early to expect much resource mobilization since the systems have been functional in part for only one season, the signs of good resource mobilization and management are not promising. In one case the funds that had been raised were extended out in loans to members with just a list of loan amounts by member's names and only the secretary knew who had repaid loans. Members of Water Users Groups have a comparative advantage in having access to irrigation for crop production in the dry season. In most cases, at least some members of Water Users Groups were also members of Marketing Clubs, but not in the same club. This makes it difficult for A Complex River Diversion System WALA Mid-Term Evaluation Report 18 March 2012 21 them to use their comparative advantage since they have to persuade other members of their marketing club who do not have access to irrigation to consider producing and marketing crops most appropriate for irrigation in the dry season. The majority of Water Users Groups have a marketing committee, but for the most part, these committees do not interact with or participate in training activities being implemented under the agribusiness component of WALA. The systems that have been put in place are relatively high input, one off systems, not likely to be replicated. The primary focus of WALA so far has been on river diversion and treadle pump systems. There has been some investigations into the potential for irrigated kitchen gardens (keyhole gardens) and drip irrigation, and, apparently one partner is piloting the use of small motorized pumps. There have been debates within the program on the use of Food for Work (FFW) for constructing irrigation schemes. On one side, the FFW certainly contributes to the hand–out syndrome and may diminish the sense of ownership in the system. On the other hand, while villagers could certainly dig canals and do the other work being supported by FFW, they would have to do this work when their opportunity cost is lowest, essentially extending the time taken to build the system. In a five year irrigation program, it is critical that most of the time is spent doing the capacity building with Water Users Groups to ensure that the systems are managed and maintained well. This means getting the irrigation structure in place, building management capacities and then disengaging to see how well the capacity building has been done, so that remedial training can be provided where required. Using FFW accelerates getting the structure in place to enable the software development investments to be done well. The sustainability of program impact around irrigation will almost entirely depend on the quality of the software investments, including (a) maximizing returns from the irrigation structure through increasing production and strengthening marketing committees and (b) strengthening the Water User Group management systems, including resource mobilization and management, water scheduling and governance to be able to minimize and manage disputes. RECOMMENDATIONS – Irrigation 36. Irrigation Area Targets. Program implementation is being driven to meet an area target. The principle that the program should follow in doing this is to achieve the target with the maxmimum number of beneficiaries using the most appropriate and acceptable technology for the specific context. 37. PRIORITY Capacity Building of Marketing Committees: The program should include all of the Water Users Groups Marketing Committees of the agribusiness component. 38. PRIORITY Increasing Production. Once marketing strategies are developed targeting specific crops, facilitate linkages to FEFs or other sources of technical advice on the crops selected and establish demonstrations on high value irrigated crops.. 39. PRIORITY Cash Resource Mobilization and Management. Provide more intensive training on cash resource mobilization and management. 40. PRIORITY Governance. Intensify governance training, since representation, accountability and transparency will be critical for resolving disputes over water between members or between the WUG and outsiders. WALA Mid-Term Evaluation Report 18 March 2012 22 Watershed Development in WALA Watershed Development Committees: Capacity building on the concept of watershed development and formulation of watershed development plans Marker Ridges: Used to indicate contours Ridge Realignment: Aligning ridges along contours indicated by marker ridges Contour Trenches: Construction of trenches on hillsides along contours to capture and direct water into the ground. Stone Bunds: Construction of large bunds made of stone along contours to arrest erosion and capture soil Gully Checks & Dams: Stone structures built in gullies to arrest erosion and capture water in some cases for irrigation. Use of Vetiver Grass for Soil Stabilization: Outplanting of this deep-rooted species in market ridges to stabilize soil. Agroforestry: Planting of leguminous and timber trees such as (tree type) on hillsides and along river banks to control erosion and restore soil quality. Pest Control in Stored Maize In a focus group discussion in an area where an agroforestry project had been implemented, respondents reported that they were using neem for pest control in stored maize. One approach was to mix the dried neem leaves with the grains, which was not preferred because of taste implications. A second approach was to mix the leaves with water and soak the storage bags in this solution, which seemed to work. Post-Harvest Handling. Given reported post-harvest losses as a high as 40 percent8 , the WALA Program has sought to identify appropriate interventions for improving storage of maize. A training was provided in 2011 to all implementing partners on a variety of different storage technologies. Since then the program has implemented demonstrations of metal silos being promoted by CIMMYT and sealable plastic bags being promoted by Chemicals and Marketing Limited. Both of these technologies have problems with mold or mildew if the grain being stored is not dry enough. In addition, the grain silos are expensive. They are far too expensive, in fact, for a poor family to purchase, and the volume is suitable for one or two families at most. The program has also presented the option of using Actellic for protection of stored grain against pests. There are alternatives to these (see text box) technologies that are less costly and appropriate for households with limited disposable cash income. All of these activities are in their early stages, so there has been little impact or adoption by participants. RECOMMENDATIONS – Post–Harvest Storage 41. PRIORITY Additional Options. The program should explore and test additional options for pest control in stored maize that are more appropriate for households with limited cash resources. 42. Estimating Storage Losses. Given the wide disparity between estimates of storage losses, WALA should use the opportunity of the next Annual Survey to obtain information from participants on storage losses. Water Shed Development. A Watershed Development Approach was proposed as part of the original WALA Program proposal to be used to guide planning around irrigation schemes. In actual implementation, training and FFW are used to support pilot watershed development activities as a separate activity, apart from the irrigation schemes that have been developed. Key elements of the intervention are shown in the text box. The program is working in a total of thirteen sites covering an estimated 800 hectares. FFW is used to support construction of demonstrations of various structures to capture and direct water resources as well as to control soil erosion. Technical support has been provided annually by the Water Organization Trust from India. The program does not have specific output indicators for watershed development. 8 This number was cited by a number of people, but the evaluation could not find a concrete reference or source. WALA Mid-Term Evaluation Report 18 March 2012 23 The evaluation observed that the understanding of the concept of watershed development seems pretty good at higher levels in the program but fairly weak at the frontlines. Consequently, some activities being undertaken are relatively irrelevant for a particular context, such as constructing contour trenches on relatively flat land where ridge realignment may serve the same purpose with much less labor; and opportunities for other watershed development activities are overlooked, such as using roads to capture and direct water. A basic element of watershed development is capturing and using water resources coming into the watershed for irrigation where possible. With some partners, irrigation technical staff are providing support and even taking the lead in working with Watershed Development Groups. However, with some partners, irrigation technical staff in WALA, including Agricane, are not providing this technical support and as a result the program is not capitalizing on opportunities to expand land under irrigation. Some of the thirteen sites selected for pilot watershed development activities were small mini– watersheds with clear boundaries and a small enough area in which to be able to show some impact on soil and water resources. Other watersheds, however, were very large, encompassing multiple mini–watersheds making it very difficult to show much impact over a three or four year period. One of the challenges for the program is to demonstrate that watershed development, given the relatively slow rate of impact on water and soil resources, will increase food production or income. In one area, the watershed management committee is implementing an activity to monitor changes in the water resources in the watershed by measuring water flow annually in the stream flowing through the watershed. While there are other factors that will influence this water flow, the concept of participatory monitoring and evaluation like this could be very useful to enable participants to be able to see changes in water and soil resources. The program has also established tree nurseries by providing seed and polybags. Species being grown are predominantly leguminous and timber species. Fruit tree seedlings are not being produced although many participants reported that there is demand for these trees, and there is seed available. The nurseries are not viewed as something that needs to be continued after the program ends, even though the need for tree seedlings will still exist. A number of vetiver nurseries were seen during the evaluation but there was only one out–planting that A Contour Trench on Flat Ground Stone Bunds Provide a Foundation for Future Terracing Road Rehabilitation with Drainage Structures (an exception, not the rule) WALA Mid-Term Evaluation Report 18 March 2012 24 was observed along an irrigation canal, not even part of a watershed development plan. This species is being promoted primarily for its soil stabilization benefits especially for steep slopes. But animals rarely eat it in Malawi since there are many preferred alternatives. It is not consumed by people, and it is not easy to walk on. While it has great benefits on steep slopes with poor soils, it should not be blindly promoted everywhere. The program should be looking for alternatives, such as napier grass or pineapples, for less sloped areas that have more benefits for people and will still serve the purpose of protecting soil. The program has also implemented a large number of road rehabilitation Food for Work schemes, but these do not appear to be tied to a watershed development plan. The technical quality of the road rehabilitation being supported is fairly basic, involving clearing vegetation and rebuilding the road surface. In most cases drainage structures are not part of the design, but even in those few instances where some structures, the structures are very basic without either gabions or concrete. Roads passing through the pilot watersheds have not been included as part of watershed development plans, despite the opportunities they present for controlling and diverting water. The greatest impact so far from watershed development activities in WALA has been the impact of the food received as FFW (see Section III.D.3 for more detail). The impact from watershed development interventions, in terms of retained surface and subsurface water and restored soil fertility, takes time. There is likely to be relatively little observed impact on food security within the life of WALA. This impact could be amplified if the watershed development were more systematically integrated with conservation agriculture. Because FFW has been necessary to construct demonstrations, replication, especially of larger structures, is limited. Nevertheless, there appear to be a few cases of participants replicating some watershed development activities on their own land, mainly ridge realignment and smaller contour trenches. The program is building interest in agro–forestry but not putting in place services that will exist after the program ends to meet demand. RECOMMENDATIONS – Watershed Development 43. PRIORITY Monitor Adoption: The program should monitor more closely the few cases where participants are replicating watershed technologies and modify the WALA strategy to incorporate these. 44. PRIORITY Agro–Forestry Nurseries: The program should develop a strategy for sustaining nursery services after the program ends, most likely by finding a way to convert the nurseries to self–sustaining income–generating activities. 45. PRIORITY Participatory Monitoring & Evaluation: The program should build on the experience of some consortium partners to expand use of PME to enable watershed management committees to monitor impact on the watershed. (The risk, of course, is that over the life of WALA, they fail to measure any impact and get discouraged). 46. Including Roads in Watershed Development Plans. The program should look for appropriate ways to use roads to capture and direct water in watershed development strategies. Animal Husbandry. The WALA Program proposal does not describe a specific set of activities around aquaculture or small animal husbandry nor are there any output indicators in the WALA Mid-Term Evaluation Report 18 March 2012 25 performance monitoring plan. The program has had a position designated for Technical Quality Coordinator for Aquaculture and Small Animal Husbandry (Livestock). This position was only recently filled, however, and the limited achievements made by the time of the evaluation were primarily a result of the initiatives of implementing partners. This component of the program has worked through existing MCHN and ANRM groups (including VSL) with a variety of activities, including a small ruminants pass–on approach with goats, a poultry pass–on approach with crossbreeding improved breeds (black australorp) with local breeds, fish pond construction or rehabilitation ponds and livestock technical support for broilers or goats. At the time of the evaluation, the program had aquaculture or animal husbandry activities with 59 program groups, including twenty–one poultry projects (broilers and australorps cross–breeding), twenty aquaculture projects (three active and seventeen waiting for fingerlings), sixteen goat projects and two pig projects. A Livestock Production Strategy Paper has also been developed. There seems to be significant demand for support for livestock development, including small ruminants, poultry and fish, although the demand seems to be greatest where there are, or have been, other projects in the area that have already been doing livestock development through pass–on approaches. The pass–on approach seems useful, but it takes time and also may contribute to cultivating the hand–out syndrome in communities. The first goat distribution in WALA was April 2011, but to date no goats have yet been passed on to other participants. Some very interesting projects were observed with two VSL groups who had invested their own capital in a livestock project. One of these was a fish pond in which FFW was used to construct the pond while the group covers all other expenses. The other was a goat project in which all costs were covered by the VSL group. In both of these groups, the sense of ownership in the activity was very high. Confined animals require more care than free–range animals, and in some of the projects, the training provided on confined animal care seemed insufficient. In one of the goat projects observed, for example, the hooves of the nanny provided by the project badly needed trimming, but nobody present from the village nor from project staff knew how to do this. RECOMMENDATIONS – Livestock 47. PRIORITY Livestock Focus. If funding allows, livestock activities should be scaled up as much as possible with special attention to food insecure participants who are involved in SO1 (for nutrition knowledge), VSL (for capital) and production groups (for using manure). 48. PRIORITY Pass–On Approaches. Research should be undertaken on pass–on approaches to determine (a) how the process can be accelerated and (b) how the approach can be delivered in ways that mitigate contributions to the hand–out syndrome 49. Animal Husbandry Training. The program needs to enhance the animal husbandry training to ensure that participants are fully capable of caring for confined animals. Integrated Pest Management. The WALA performance monitoring plan has a specific outcome indicator associated with integrated pest management; however, no significant activities have been implemented systematically by the program to achieve this outcome. The program has taken advantage of opportunities to provide simple training to program staff by asking researchers and other technical trainers to include any information on IPM that they may have when they provide technical training on other topics. It is unlikely, however, that the achievements made on this outcome as shown in Table 4 are associated with this training. The WALA Mid-Term Evaluation Report 18 March 2012 26 change in the outcome indicator is more likely to be the result of other features of the context, including perhaps interventions by other organizations on IPM type practices that farmers already know. RECOMMENDATIONS – IPM 50. IPM Performance Indicator. It is too late for WALA to implement a full IPM strategy as suggested by the performance indicator in the IPTT (understanding pest environment, mechanical control, biological control, crop rotation and patterns, appropriate chemical control), so the program should discuss with FFP the need to continue with this indicator. Activity 2.1.3 Support to local seed systems The WALA proposal described two types of activities to strengthen local seed systems. One set of activities intended to work with MCHN groups to expand the supply of planting material for targeted nutritious crops. A second set of activities was designed to ultimately enable marketing groups to become commercial suppliers of certified seed. The program has made some progress on the former but no progress on the latter set of activities, since most of the focus in the agribusiness component has been on marketing training and development of market linkages. There seems to be potential, however, to at least pilot in the remaining life of WALA the concept of marketing clubs becoming seed suppliers to meet local demand. Planting material has been obtained from various sources, including Bvumbwe, Chitedze, and Lufuwu research stations, farmer training centers and the private sector. The improved varieties promoted include both open pollinated varieties of maize, hybrid maize varieties various yellow and orange fleshed sweet potato, high yielding and mosaic resistant cassava, early maturing and high yielding pigeon pea varieties, amaranthus, mustard (greens), tomato and rice. Combination demonstration and multiplication sites have been established with the strategy to pass on seeds and cuttings from the sites. A consistent pattern was observed in the evaluation that, while some sites may have had signs indicating what was being demonstrated, for the most part, these were not clear. In the picture shown for example, the sign board indicates that this is a pigeon pea variety demonstration, but there is no information on which varieties and where they have been planted in the demonstration. The evaluation had a chance to visit a grain bank activity that was described as a seed supply intervention. The one grain bank observed, however, was less of a bank and more of a grain reserve to enable people to cope with emergencies. At harvest every household makes a contribution of maize, and this stock is disbursed over the year mainly during funerals or to assist orphans. In a few cases, maize is borrowed as seed, and repayment of the loan is double the amount received. However, this was considered far less important to participants than being able Pigeon Pea Variety Demonstration and Seed Multiplication Site Demonstration WALA Mid-Term Evaluation Report 18 March 2012 27 to use the grain to support vulnerable households in the village. The main contribution from the WALA Program was group organization and provision of a storage bag, and the success of the grain bank seemed to be mainly a function of a strong village headman. RECOMMENDATIONS – Seed Supply Systems 51. PRIORITY Grain Banks. This intervention represents an opportunity to strengthen a community–based safety net that could be sustained beyond the life of WALA and the program should consider incorporating this in the strategy under SO3 for addressing the food insecurity of OVCs, PLHIV and other chronically ill, currently benefitting from food distributions. 52. PRIORITY Seed Selection and Storage. Apparently the program has undertaken some very basic seed selection and storage training for participants planning to carry over seed. This training activity should be expanded and accelerated, since it is very appropriate for the most vulnerable households with limited cash resources. 53. Community–Based Seed Suppliers. The concept original proposed for facilitating marketing groups to become certified seed suppliers should be revived and piloted in the remaining life of WALA. 2. IR 2.2 103,400 households have increased their use of financial and related services through Village Savings and Loan (VSL) groups. In this component of the program, community–based Community Agents (CAs) are trained by the program on VSL group organization, bookkeeping, and management of VSL funds. These CAs then form groups of participants within their GVHs and provide training and guidance on establishing and managing savings and loan funds. The program has also started the process of providing economic activity selection, planning and management training to VSL groups to build their capacities to make wise decisions on using loans and share out funds. Standard Operating Guidelines have been developed to guide implementation of the VSL component of WALA. Finally, as part of a sustainability strategy, the program has begun a process of rolling out a Private Service Provider (PSP) model for sustaining the services of CAs after the program ends. Activity 2.2.1 Mobilization and group formation As of the end of December 2011, a total of 304 community–based Community Agents (CAs) have been trained to support 2,187 VSL groups with a total membership of 37,607, representing around 36% of the LOA target. On average, one CA is working with around seven VSL groups. The guidelines for group formation indicate that the maximum group size should be no more than twenty–five members. The available statistics show that on average the group size is around seventeen members. Annex H provides more detail on ratios of CA to groups and average group sizes. While over 82% of VSL members are women, significant numbers of men are participating indirectly by giving wives cash for buying shares and participating in decision–making on loans and Some Members of a VSL Group WALA Mid-Term Evaluation Report 18 March 2012 28 share–outs. Nearly all women participants who responded during focus group discussions described joint household decision–making as far as using loans and share–outs. Other qualitative data collected in discussions with MCHN groups also participating in VSL indicated, however, that men were making decisions in some cases. The support provided by PVOs for their Community Agents varies from those providing a bicycle and other material support to other PVOs doing this and providing a monetary allowance either as a bicycle maintenance fund or a stipend. A number of Community Agents indicated that they are providing support to other new VSL groups outside of those specifically formed for WALA. They indicated that sometimes these additional VSL groups are reported, but in some areas they have not been reported. Activity 2.2.2 Capacity–building in internal savings and loan operations As of the end of December 2011, current member equity for VSL group members in the form of group investments in economic activities (2.8%), savings shares purchased (71%) or interest earned on loans by the VSL (26.2%) is estimated at approximately USD 1,061,750. A total of 1,084 VSL groups had completed at least one cycle and some of these have competed two cycles. In general, the number and size of VSL groups are expanding. Respondents during the evaluation indicated that typically between cycles, a few members who have had difficulty with loans are dropped from groups and large numbers of new members are joining. When the number exceeds the twenty–five member limit, new groups are spun off. While there are certainly challenges with loan repayment, funds management and protection of funds when they accumulate shortly before the share–out, the training provided by the program has been very effective. Many of the focus group discussions held during the evaluation were with participants who are in a number of different groups in the program. When asked which activity under WALA has been most useful for them, ninety percent of the time with these groups, VSL is declared most useful. Many VSL groups have purchased their own uniforms to distinguish their VSL identities. Activity 2.2.3 Promotion of economic development The Economic Activity Selection, Planning and Management (EASPM) training process in WALA is still underway, and there is little evidence that the training has been used by participants. Nevertheless, the VSL component of WALA is already having significant impact.  Loans are used for various purposes, most of which have livelihoods impact  Share–outs are also used for various purposes and some groups have shared–out twice  Investments in IGAs or other productive activities made from loans or share–outs will continue to generate income  The social fund managed by each VSL group is meeting emergency needs  Social capital benefits from being in a VSL group are starting to emerge, particularly around helping other group members during funerals or other household crises.  Women participants are reporting increased levels of confidence from participating in a group and having money WALA Mid-Term Evaluation Report 18 March 2012 29  In discussions around dependence on ganyu 9 , many VSL participants indicated that they were no longer dependent on ganyu and some were even now hiring people to work for them. Two VSL groups have invested part of their share out funds in a common income generating activity, a goat project in one case and a fisheries project in the other. In the case of the former, each member of the VSL group contributed around 5,000 Kw (around half of their share–out) to build the pen, purchase ten nannies and cover operations costs. The manure produced by the goats will be shared among members, and the kids will be distributed among members as they come (none have dropped yet). The program has only provided technical support and there is a government veterinary agent who lives in the village and works with the group. In the case of the fisheries project, FFW was used to construct the fish pond, and each member contributed 1,000 Kw for the purchase of fingerlings, brood stock and food (mixed locally). There has been no harvest yet from the pond. Activity 2.2.4 Introduction of a commercial sustainable implementation approach In discussions around the need for CA services after the program ends, all respondents in the evaluation indicated that the CA function is still required. The calculation of share–outs still requires CA guidance, and VSL members (or others in the community who have sufficient knowledge of the VSL approach) have not yet started forming their own VSL groups without CA assistance. There is also high demand for forming new VSL groups both inside the GVH where WALA is being implemented as well as in neighboring GVHs. To address this post–program need, WALA has already begun the process of converting CAs to become Private Service Providers (PSPs) on a pilot basis with one implementing partner. The key elements of this process are to certify the quality of the services provided by an individual CA and to identify markets for their services from which they can generate fees or be able to continue services. As mentioned some CAs are already working with new non–WALA VSLs. Some CAs and VSL groups also reported that the groups are expressing “appreciation” to their CAs, especially during the share out, in the form of cash, something in–kind, or occasionally even a bicycle. This seemed to be more prevalent in areas where CAs do not receive a program stipend. RECOMMENDATIONS – VSL 54. PRIORITY PSP Roll–Out: The roll–out of the PSP model should be accelerated...some partners and CAs are ready to go...so that there is time left in the life of the program to address any constraints that may arise. 55. Reporting Diffusion: Diffusion of the VSL approach is an important impact and frontline life should be encouraged to identify and report when this is happening. 56. VSL Group Investments: Group–managed businesses have a higher rate of failure so promotion of group–managed IGAs should be treated with caution. Nevertheless, existing VSL group–managed IGAs should be monitored (none as yet have produced benefits) and if they prove successful, VSL groups should be exposed to the concept 9 Ganyu is the local term for piecemeal work or daily wage labor. Sometimes there is legitimate need for labor, but other times ganyu is requested or offered when a household is in desperate need of support. WALA Mid-Term Evaluation Report 18 March 2012 30 3. IR 2.3 20, 600 smallholder farming households have increased income and/or assets from commercial marketing. In this component of the program, program staff sensitize communities to the strategy of this component and the opportunity to form marketing clubs and marketing clusters. Interested participants register as marketing clubs of fifteen to twenty–five members. All of the marketing clubs in an area are aggregated into a marketing cluster which has its elected leadership and a marketing committee. A community–based Agri–Business Community Agent (ACA) is selected from the cluster and trained by the program on group organization, market opportunities analysis, collective marketing and bulk purchase of inputs. Membership in a marketing club requires payment of a nominal membership fee of between 200 and 500 kwacha per year. The program has also started the process of providing more detailed marketing training in the form of a set of six modules around Farming as a Business (FAAB), an approach that has been developed and used elsewhere by ACDI–VOCA, who provides technical oversight for this component of WALA. To complement the training that is being provided, the program is also facilitating the development of market linkages and expanding access to other market services including price information services. Standard Operating Guidelines have been developed to guide implementation of the agribusiness component of WALA. Activity 2.3.1 Value chain analysis and capacity building of marketing staff In the original design of WALA, a value chain analysis approach was proposed to identify key value chains to target in this component of the program. A basic value chain analysis has been undertaken on the pigeon sub–sector, including implementation of a stakeholder workshop to discuss value chain development. Preliminary analyses were also undertaken on cowpeas and sesame, and studies are planned for cassava, chillies and groundnuts. Comprehensive value chain analyses will typically map out prices and quantities along each marketing channel in a sub–sector to identify constraints and opportunities for expanding the value chain. Since the focus of WALA is at the community level and the program feels that it does not have the capacity to address higher level constraints or opportunities downstream in a value chain, the program has made a strategic decision to focus on local market opportunities analyses, rather than on comprehensive value chain development. Activity 2.3.2 Marketing group training in value chain integration and market entry The latest information from the program, through December 2011, shows that a total of 25,962 farmers have been formed into marketing clubs in 246 market clusters. Each cluster is served by an Agribusiness Community Agent. The LOA target for this component of the program is 20,600 smallholder farmers. Although the current number of participants exceeds the target number, not all of these participants are smallholder farmers. The program is working on establishing a system for confirming the actual number of smallholder participants who are in marketing clubs (see Section VI.C.2) . MKw 34 million (approximately USD 200,000) worth of commodities have been sold collectively by marketing clusters. Bulk purchase of inputs has also been undertaken in the program area on a small–scale. Farmers selling collectively reported that they received a better price and since the buyer came to the village, they avoided paying transport costs. Almost equally important, collective marketing ensured proper weighing of commodities and better prices for members who are vulnerable to exploitation by buyers when they must sell as individuals. WALA Mid-Term Evaluation Report 18 March 2012 31 Finding a Market for Pigeon Pea In one cluster, the person designated to use cluster fees to search for a market for pigeon peas was unsuccessful in finding a buyer in Blantyre. On his way back to his village, he posted a flyer at a couple of road junctions indicating that the cluster had pigeon peas for sale with his phone number listed. Shortly thereafter they received a call from a buyer and the cluster was able to negotiate an acceptable price. Activity 2.3.3 Sustainable market linkages and business development services Relative to building market linkages, one round of marketing fairs with events in three locations in the program area has been organized and implemented by the program. The marketing fairs were often cited by respondents as the primary means they had used for identifying buyers. Marketing committees have been organized in each cluster, and the membership fees (and penalty fees in some groups) are used to send designated persons to find markets as per marketing strategies developed by these committees. Sometimes the designated person to find the market is the ACA, sometimes someone else from the cluster, and sometimes it is two or three persons from the committee who are made responsible. Out–grower contracts with ExAgris were piloted by one of the current WALA partners in the previous I–LIFE Program, and this activity has been expanded to all WALA partners in the current growing season. The ExAgris out–grower relationship seems reasonably fair and solid, although negotiations on the commission fee that ExAgris provides to agents assembling produce from local farmers for purchase by ExAgris have indicated some reluctance to fully commit to providing the bonus to WALA ACAs. In the current season, ExAgris has provided subsidized seed and technical advice on production of birdseye chillies, and the out–grower contract stipulates a minimum price they will pay at harvest, with the opportunity to negotiate a higher price if the market price rises substantially over the minimum. The program has also begun working on a facilitating a similar out–grower relationship with Chibuku Products for sorghum. Sorghum seed has also been provided to a few producers as an informal out–grower arrangement. The WALA Program also has developed a relationship with Chemicals and Marketing Limited for proper training on chemical application, and business development service linkages are being developed with Esoko and the Agricultural Commodities Exchange (ACE), Farmers Voice Radio and Opportunities International Bank of Malawi (OIBM). The services provided by Esoko oriented around using SMS cell phone technology to access price information on key commodities is interesting, but it is still early in the process of making it work with ACAs. . In discussions around the need for ACA services after the program ends, most respondents reported that the need will continue since market opportunities will change. There is an important role for someone (possibly an ACA) to continue to find markets and assemble commodities after the program ends, and there is potential to support this service with commissions. RECOMMENDATIONS – AgriBusiness 57. PRIORITY Marketing Fairs: Find a long–term managing agency for this activity and work toward establishing this as a sustained service. 58. PRIORITY ACAs as PSPs: Develop and implement a strategy for sustaining the services of ACAs after the program ends through commissions for linking producers and buyers or provision of other paid marketing services. Given the short time remaining in the program to put in place a system , test it and provide remedial support to address any problems that may emerge, WALA Mid-Term Evaluation Report 18 March 2012 32 59. Scaling-Up Business Linkages: Participating farmers are learning about marketing both conceptually through the Farming–as–a–Business training as well as through acquring experience in business relationships faciltated by the marketing fairs and the linkages being developed by the program. Learning–by–doing is generally more effective at acquiring practical skills, and the program should scale up this element of its agri–business capacity building strategy, i.e., expand existing mechanisms such as marketing fairs and out–grower relations and look for other ways to facilitate business relationships. 60. TQC Agri–Business: Given the recommendations above as well as continuing to build marketing capacities among targeted impact groups which cannot be completed within the next six months when the current TQC position is supposed to phase out, the program should continue to look for a means to retain a TQC capacity in CATCH through the end of the program. 4. General Comments on SO2 The evaluation team investigated the level of integration within SO2 since there are opportunities for a household to participate in a producers group, a VSL group, a marketing club, a Water Users Group and a Watershed Management Committee. While the program's M&E systems do not yet provide quantitative information on this overlap, qualitative interviews during the evaluation indicated that very few participating households are involved in only one program activity under SO2 (usually VSL). The vast majority of households are involved in two groups, and as many as thirty to forty percent are participating in a producer group, VSL group and marketing club. Despite participation in both a producer group and a marketing club, many participants are not using conservation agriculture in their fields, such as chillies, which have been planted for a marketing opportunity. In general, there appears to be not much real synergy being developed between components within SO2. D. Strategic Objective 3: Disaster Risk Reduction (DRR) SO3: The resiliency of 273 communities is protected and enhanced. Table 5. Outcome Indicators for SO3 and Achievements by the End of FY 2011 Indicator Baseline FY2011 Number (cumulative) of project assisted communities that had disaster early warning and response systems in place. 0 75 Number (cumulative) of project assisted communities that had improved physical infrastructure (roads, water management systems, etc.) to mitigate the impact of shocks. 0 45 Number (cumulative) of people benefitting from USG supported social assistance programming disaggregated by gender F=0 M=0 F=11,256 M= 5799 In this objective of the program, WALA is following the guiding principles for three key dimensions of disaster management – preparedness (awareness), mitigation (contingency) and response (assistance). The following activities are being implemented under SO3:  Training of community led-groups in formal disaster management  Linkages with WALA partners, communities, local government institutions and government structures  Building knowledge and awareness in environmental protection  Enhance safety nets through rations WALA Mid-Term Evaluation Report 18 March 2012 33  Support asset building and protection through Food for Work  Rebuild assets lost by disasters through Food for Assets  Good governance training  Scorecard and Civil Society Index to promote good governance  Promotion of increased community participation in decentralized GoM The original design of the program had included additional activities under SO3 for environmental protection and good governance; but following a visit by FANTA, these were converted to cross-cutting themes and are discussed in Section IV. 1. IR 3.1 273 communities have strengthened disaster preparedness, contingency and response capacity. Activity 3.1.1 Training in disaster management Communities targeted by WALA are threatened by recurrent natural disasters like floods and drought. In addition, the livelihoods of these people are challenged by shifts in market prices of staple and other key commodities. Over its life, WALA has planned to provide training to 273 communities at the Group Village Headman (GVH) level in disaster risk management to reinforce community resiliency to external shocks and to build coping capacities for managing increased prices of staple food. The approach that the program follows to train communities are to (1) revamp/establish the government–mandated Area Civil Protection Committees (ACPCs) at the Traditional Authority (TA) level and Village Civil Protection Committees (VCPCs) at the GVH level, (2) provide Training of Trainers (TOT) for ACPC members in formal disaster management and (3) monitor ACPCs as they train the VCPCs. A user–friendly Disaster Risk Reduction Training Manual has been developed in collaboration with the District Civil Protection Committees from the WALA program districts. The implementation of this activity has lagged behind other activities of WALA. The low progress is primarily due to delay in the hiring of the senior DRR staff by CATCH. The DRR TQC (Technical Quality Coordinator) joined CATCH in September, 2011. The activity is also affected by high rate of staff turn–over with some implementing partners. As of December 2011, program has conducted training sessions for 85 VCPCs in Disaster Risk Management, representing 31% of the LOA target of 273. The implementing partner DRR Coordinators facilitated the development of Community-led Disaster Management Plans which include an emergency preparedness action plan, mapping of prioritized community infrastructure, types of hazards and their possible threats. The plans were tested and found to be fully functional in the communities that were affected by droughts in 2011 and floods most recently. The program is making good progress toward revamping/establishing ACPCs and VCPCs. As of December 2011, members of over 200 VCPCs and 39 ACPCs have received orientation in their roles and responsibilities. Most importantly, the program facilitated formalization of these committees with the appropriate representation of community and government service providers according to the GoM policy and guidance. Before WALA, these committees were very much irregular in size and representation, and the members were unclear on their functions. The program has also facilitated development and improvement of Disaster Contingency Plans for four out of the eight WALA program districts. The members of the District Civil Protection Committees who oversee emergency response in their respective district expressed for the quality and usefulness of these plans. WALA Mid-Term Evaluation Report 18 March 2012 34 The implementing partner DRR coordinators received training in Participatory Capacity and Vulnerability Assessment (PCVA), and they have trained members of 100 VCPCs as of December 2011. The PCVA was conducted to analyze a community’s exposure to disaster vulnerability, livelihood strategies, and coping mechanisms using their own resources. Using mobile phone SMS technology, the program has piloted a tool for collection of Household Hunger Scale (HHS) data. The HHS tool, which is a form of trigger indicator monitoring, was developed by FANTA and is found to be effective in rapid assessment, critical to timely emergency responses. The evaluation observed that the VCPCs who had been trained were found to be effective in:  Assessing risks and disseminating information about risks  Inventorying resources available in affected communities  Developing coping mechanisms for disaster response  Coordinating community resources to respond in emergencies  Shifting the focus of vulnerable families from dependency on relief and handouts. The DRR training, including formation of the fully functional ACPCs and VCPCs was tested in the recent floods in Chikwawa and Nsanje Districts. The GVHs visited in these districts during the evaluation acknowledged that WALA-trained VCPCs produced high quality flood assessment reports and made timely submissions to the ACPC at the Traditional Authority level. The VCPCs who submitted the assessment reports, however, expressed frustration with too many layers, from the GVH to the TA to the District and finally the National Assembly, delaying the emergency response, despite their hard work. Other observations from the evaluation include the following.  There have been no assessments of the emergency response that have been implemented by WALA (After Action Reviews).  While households may have their own individual mechanisms for coping with rising prices of food staples, there are no other community level or higher level mechanisms, such as community safety nets, to support particularly vulnerable households.  Women lack knowledge about safety and rescue during emergencies. This is a problem particularly for those women who are the head of the family or have elderly or sick adults and young children in the household.  WALA can divert up to 10% of in-country Title II commodity stocks for an emergency response (without reimbursement), but there is no provision of cash in the budget for an emergency response.  The program is capable of responding to emergencies outside of the WALA TAs in other non-WALA TAs within the WALA districts without adding significant costs to the program budget.  Government representatives at the TA and District levels were surprisingly familiar with WALA activities whether or not they were participating in the program. Appreciation was expressed for the soil and water conservation activities that the program is promoting.  Planting trees around the homestead is a viable mitigation measure against strong wind. Long term sustained disaster risk management is always threatened by a lack of commitment of policymakers and frontline workers, and preparation elements of disaster risk reduction need to be integrated in mainstream development planning. WALA Mid-Term Evaluation Report 18 March 2012 35 RECOMMENDATIONS - DRR Capacities 61. PRIORITY DRR Training. Expedite training of outstanding VCPCs, ACPCs and DCPCs in DRR to make them well prepared to respond to disasters by the end of the program implementation year 3. 62. Early Warning on Food Prices. Emphasize early warning in DRR training of GVHs on how to monitor prices of staple food, so that households may have more time to strengthen their coping capacities. 63. Women's Needs in Emergency Response. Include a special module in the DRR training on safety and rescue of women during an emergency response. 64. PRIORITY After Action Reviews. Conduct “After Action” reviews including assessment, reporting, proposal writing and lessons learned from disaster response by WALA and use the information to revise DRR training modules. 65. PRIORITY Budget for Emergency Response. Make a provision in the annual budget for an emergency fund to enable a timely emergency response. This budget can be reimbursed if external donor resources are obtained, as happened with the recent floods in the Lower Shire Valley. 66. Emergency Response Protocol. Define a decision-making protocol for Civil Protection Committees at different levels to facilitate more rapid processing and approval of emergency assistance. 67. Scope of Emergency Response. Expand the WALA catchment area for disaster response to respond quickly and appropriately in a wider area, without adding significant costs to the program budget. Activity 3.1.2. Linkages with formal disaster management mechanisms This activity aims to improve disaster risk management and strengthen early warning systems by enhancing relationships between program partners, communities, and existing and emerging structures in government. The chain of relationships encompasses the levels described in the previous section from GVH to District as well as with national level stakeholders such as the Malawian Vulnerability Assessment Committee (MVAC), Department of Disaster Management Affairs (DODMA), the National Disaster Preparedness and Relief Committee (NDPR) and the Famine Early Warning Systems Network (FEWSNET). WALA has done well in facilitating linkages between the program's implementing partners and the Civil Protection Committees at different levels. These linkages were established through the activities described in the previous section. Linkages between these integral program participants and the higher level organizations are more informal. The linkages activity has not specified output targets or outcome indicator in the IPTT. There are other opportunities for building linkages for disaster risk reduction. In SO2, for example, the program is facilitating a linkage between WALA and Farmers Voice Radio (FVR)10 Malawi for making agribusiness information available over the radio or via SMS. The FVR has a facility to send SMS text messages to the farmers at free of cost. At present the messages are 10 More information on the FVR Project is found at http://www.air.org/focus-area/international￾development/index.cfm?fa= viewContent&contentid=606. WALA Mid-Term Evaluation Report 18 March 2012 36 “Please thank America for the ART and the food. Now, I have energy to go work in my fields again. I can provide for my family with what I grow there and in my garden and, through the marketing group, I am going to sell pigeon peas. I don’t need the [supplemental] food anymore. You can give it to someone else who needs it. I have started my life over.” - PLWHA in Chikwawa District. about tips on agronomic practices and weather patterns that affect agriculture production. In future, this facility can be used to send predictions of adverse weather conditions, for example, threats from strong winds, droughts and floods. RECOMMENDATIONS - DRR Linkages 68. PRIORITY Farmers Voice Radio. The program should negotiate with Farmers Voice Radio to also include disaster early warning messages along with the agribusiness information currently being discussed with them. 69. Output Targets. The program should define output targets for the higher level linkages required to enhance disaster risk management capacities at the district level and below. Activity 3.1.3. Integration of environmental protection in SO1 and SO2 Specific activities and recommendations associated with environmental protection are described in Section IV on crosscutting themes. 2. IR 3.1 21,203 food insecure households have enhanced capacity to withstand shocks and stress. Activity 3.2.1 Enhance safety nets through rations WALA distributes food rations to households eligible for safety nets from among the most food insecure in the community. The first consideration for identifying beneficiaries is identifying poor and very poor households with limited food availability and access who meet three out of the six food insecurity vulnerability criteria set by the program. The Village Development Committees then prioritize households from this pool by selecting those which meet one or more of the following: households caring for orphaned children: child-headed households; elderly headed households; household ds with chronically ill (HIV or AIDS affected) members; female-headed households; households with two or more years of successive crop failure; or households with children receiving supplementary or therapeutic feeding. This selection process is similar to the guidelines developed by WFP Malawi in conjunction with the Joint Emergency Food Aid Program for HIV Targeting. Over the life of the program, WALA is targeting distribution of 9,703 MT of food to 8,197 beneficiaries annually for five years11. The caloric value of the monthly food rations of 5 kgs. of pinto beans, 15 kgs. of corn soy blend (CSB) and 3.67 kgs. of vegetable oil is 641.76 kcal per day, representing approximately 31% of the recommended daily intake. Targeted households are entitled to receive rations for a twelve month period. At the end of the twelve month period, all participants are deregistered, and a new regstration process is initiated. If a recipient from 11 Some beneficiaries may receive food in more than one year, and the WALA M&E systems are working on being able to provide a cumulative number of food recipient beneficiaries. Safety Net Rations An elderly widow raising orphans says the supplemental food provided by WALA for a year in 2010 has given her a new lease on life. Before, she spent every day trying to find enough food to eat and feed the 3 children. She was physically able to farm only a very little of the land she owned. With the USAID food providing meals for about 3 weeks of every month, she suddenly had some time to invest in petty trade. That income led to her joining the VSL from which she took a loan to pay laborers to farm all her land. She is now food secure and has a source of cash income to buy additional food and pay for schooling. WALA Mid-Term Evaluation Report 18 March 2012 37 the previous year qualifies, he or she may be registered and receive rations for another twelve month period. The program is right on track having reached the target of 8,197 households in each of the first two years. In addition, 8,858 drought-affected families received emergency rations for two months in 2011 as part of an emergency response. The evaluation found that the majority of the beneficiaries are chronically ill followed in sequence by women-headed households, households caring for orphaned children, households with ill adults and children, and elderly-headed households. The impact of the food has been substantial. Beneficiaries interviewed in Njereza GVH of Kasisi TA in Chikwawa District stated that they are willing to sacrifice their entitlements for other people in their community who are now more deserving. Many beneficiaries reported that WALA was the first program providing food rations to them. The food rations have boosted most of the CI beneficiaries interviewed both physically and emotionally. The beneficiaries reported that the food rations have transformed their bodies, enabling them to engage in agriculture labor and other physical work including taking care of their children. RECOMMENDATIONS - Safety Net Distributions 70. PRIORITY Integration of Safety Net Beneficiaries. The program should ensure that safety net beneficiaries have opportunities to be included in SO2 activities as appropriate as well as in SO3 activities to enhance their coping strategies to be able to protect the new assets they are able to accrue. Activity 3.2.2. Support asset building and protection through Food for Work (FFW) & Activity 3.2.3. Rebuild assets lost by disasters through Food for Assets (FFA) The original WALA program proposal made a distinction between FFW used for various watershed development structures, footpaths and roads and FFA used to rebuild household and community assets lost through natural disasters. In fact, this distinction has been lost in WALA and depending on which implementing agency one is speaking to, the definition varies. The master table kept in the commodities unit listing the types of FFW/FFA activities shows a total of 101 projects, but the total number of projects reported by implementing partners is much higher, at least 130 sites. FFW/FFA projects include irrigation structures, road rehabilitation, afforestation, watershed development structures and some unique projects including two river dyke rehabilitation projects and construction of an electric fence on the boundary of a national park to keep elephants out of farms. The approach used for identifying types of FFW projects is perceived by some participants as being fairly restrictive, not allowing communities to explore alternatives to roads, watershed structures and irrigation. As already mentioned, there have been some rather unique FFW/FFA projects undertaken, such as fencing along a national park boundary, a fish pond for a VSL group and two river dike rehabilitations. Driven to achieve food distribution targets, strategic decisions on the best investments to make with FFW/FFA food have taken a back seat to achieving distribution targets. As a result, roads are prioritized and the location of watershed development structures such as contour trenches in relatively flat terrain. While the program gives due diligence to the technical quality of some of WALA Mid-Term Evaluation Report 18 March 2012 38 the FFW/FFA investments such as watershed structures and irrigation systems, the work done on roads is pretty basic, not much new, with a few exceptions. From the FFW recipient perspective, commodities comprising 15 kgs. of pinto beans and 3.67 kgs. of vegetable oil are being paid as wages for a month’s work under FFW/FFA activities. The program has targeted distribution of a total of 2,892 MT to 25,809 beneficiaries at an average of 8,603 beneficiaries annually beginning in Year 2 through Year 4 of WALA. The FFW/FFA distributions are progressing well, benefitting 6,740 workers in October through December, 2011. Criteria set by the program for selection of FFW/FFA beneficiaries are not strictly followed in many instances. FFW/FFA reports output (the number of persons receiving ration) not outcome (person days of employment creation). Beneficiaries seemed happy with the pinto beans and vegetable oil as wages but requested provision of CSB in the package, since others in the community are receiving CSB with other food distribution activities. FFW/FFA beneficiaries reported that the food received allowed them to use cash for other purposes that would normally be used to purchase food, and they also reported that the food is consumed. Appreciation was expressed mostly for the vegetable oil since oil is not readily available locally and is expensive when it is available. RECOMMENDATIONS - FFW/FFA 71. FFW/FFA Distinction. There seems to be no real benefit in trying to distinguish between FFW and FFA, so find a term that people are comfortable with and use a single term. 72. PRIORITY Selection Criteria. The program should review the targeting of FFW beneficiaries to ensure that the food is going to chronically food insecure households with some labor capacity. The value of the ration should be analyzed and compared to the local wages paid for ganyu so that the ration itself is an effective targeting tool. 73. PRIORITY Assets Built with FFW: The program should give more attention to strategic decisions around assets to be created by FFW/FFA to ensure that the program capitalizes on the best opportunities to use this food resource effectively. 74. Person-Days. Person-days of employment created by FFW/FFA should be included as an outcome indicator in the ITT. 3. IR 3.3 8,002 community-led groups have practiced good governance principles. Following a visit by FANTA, this component of the program was revised to become a cross cutting theme rather than an Intermediate Result. Observations and recommendations on governance are covered in the next section. 4. General Comments on SO3 The target for SO3 on food distribution recipients is 21,023 food insecure households per year. The program is not likely to achieve this target, however, since the target for safety net distributions is 8,197 households and the target for FFW/FFAs is 8,603 households for a total of 16,800 households. The commodities available are sufficient for the latter number. WALA Mid-Term Evaluation Report 18 March 2012 39 Governance as Cross-Cutting Theme or an Intermediate Result The designers of WALA wanted importance given to inducing good governance behavior in participating groups. Consequently, this intervention was elevated to an Intermediate Result level, even though it goes beyond Disaster Risk Reduction (the SO under which the IR was placed) and can clearly be defined as a cross-cutting theme. Cross cutting themes, however, tend to be relegated to "add-ons" in program strategies, often not getting enough attention. WALA seems to have struck a balance. There is a TQC for Governance and output targets for governance have been specified. Progress is being made, but perhaps not to the level envisioned by the original program designers. IV. CROSS-CUTTING PROGRAM ACTIVITIES A. Governance Interventions in WALA that are focused on building capacities for good governance were moved from the Intermediate Result level to become a cross-cutting theme following a FANTA review of the WALA M&E systems in 2010. In the original design, these interventions intended to (a) foster an understanding and use of the governance principles of accountability and transparency and (b) strengthen linkages between participating groups and decentralized local government structures. The program was to provide training to Care Groups, groups of PLHIV, Village Health Committees, Village Savings and Loan Groups, Marketing Clubs, Producer Groups, Water User Groups and Watershed Development Groups. The program also intended to introduce tools for monitoring progress toward good governance such as the Civil Society Index (CSI) and Food Security Coping Capacity Index (FSCCI). Finally, in the original design of WALA, the program intended to promote community participation in democratic elections of the Village Development Committees and Area Development Committees. WALA conducted governance trainings to nearly 2,500 groups and committees, representing 31% the LOA target. These trainings focused on group dynamics, community mobilization, communication skills, conflict management and leadership. Other key activities undertaken by WALA staff implementing governance activities included working with the irrigation sector in the development of MOUs on land tenure issues, supporting the rolling out of Participatory Planning, Monitoring and Evaluation (PPM&E) training to Village Development Committees, and development of Standard Operating Guidelines for Governance. As a result of governance activities in WALA, various participating groups under SO2 had constitutions or by-laws to govern their activities and decision-making, leadership in Traditional Authorities and GVHs interviewed reported that they would be willing to advocate for their constituents for alternative, drought-resistant crops to maize in response to requests from their constituents, and communities requested full details on the final selection of beneficiaries in targeted food ration activities. The evaluation observed that, even though the program has a Technical Quality Coordinator (TQC) for Governance, the ability of the program to expand and accelerate governance interventions is constrained by the TQC position residing outside CATCH and being part-time; and implementation staff assigned governance responsibilities also have other implementation responsibilities. WALA Mid-Term Evaluation Report 18 March 2012 40 RECOMMENDATIONS - Governance 75. PRIORITY Governance Focus. In the remaining life of WALA, the program should focus on the following governance principles:  Transparency – Leaders informing constituents on the rationale and processes used for making decisions......for example, ensuring that the selection process and criteria for receiving inputs or food are widely known.  Acknowledgment – Provide recognition and provision of non-monetary incentives and rewards for good community work and leadership.  Empowerment and Accountability - Implement some form of social audit as part of PPM&E process so that leadership are held accountable in identification, preparation and implementation of development projects in their communities. 76. PRIORITY Governance TQC. Since governance is a cross cutting theme that must be incorporated across all components of the program, the Governance TQC should be relocated to the CATCH office to facilitate more continuous interaction with other TQCs and implementing partners. B. Gender The WALA proposal contained a well thought-out gender strategy as an annex which coincided well with current USAID emphasis and thinking on gender. The gender strategy laid out a plan for a complete gender analysis to be undertaken by WALA before starting up project activities. Unfortunately, CATCH struggled to fill a gender position or identify a point person for gender so the gender analysis was never done. The DRR TQC is now responsible for overseeing gender which may enable WALA to consider gender and equity more during the remaining life of the program. Currently, the numbers of participants and staff do show very good gender balance, but WALA must look deeper at how gender considerations are taken into account in planning activities, for example in increasing women’s workload or multiple meetings diverting their attention from child care. It will also be worth further examining how participation in various WALA activities is increasing women’s ability and power to make decisions in the family. Without thinking with a gender lens, WALA is doing many activities which are empowering women. For example, women who sell their product through marketing groups are not subject to the price bias that a woman selling on her own would likely face. The marketing groups also solve the issue of limited mobility which often forces women to sell locally at low prices. Female participants in VSL groups spoke often during interviews about how their participation in the group has shown them how to work within a group as well as giving them confidence to express their opinions. WALA is also implementing innovations to reduce women’s workload. These include the fuel￾efficient stoves, fireless cookers, wash lines, post-harvest storage, conservation farming, and irrigation technology. For the latter, women now benefitting from WALA irrigation schemes mentioned the hours they used to spend carrying water to irrigate during the dry season and the previous need to plant vegetables on the far side of a field nearest the water source, which meant farther to carry them home. On the other hand, the work of maintaining a sanitary environment around the house, which is a new behavior promoted by WALA, has fallen to women, who are also spending many hours in various WALA meetings and activities. Treadle pumps also increase work for women, as does food processing technology. This has to be balanced against WALA Mid-Term Evaluation Report 18 March 2012 41 the eventual benefit of these types of activities. As already mentioned, the DRR planning has not taken into account the differing risk women and children face in times of emergency. In SO1, a specific strategy to engage men is lacking. This was more fully described above in the description of SO1 activities. In SO1, specifically reaching out to men is not only about gender but also critical to facilitating behavior change where gender dynamics often form a barrier to adopting healthy practices. RECOMMENDATIONS - Gender 77. Gender Strategy. Program management and TQCs should review the gender strategy and update the operational plans in the strategy for the remaining life of WALA. 78. PRIORITY Gender Mainstreaming. Technical Working Groups in all components should spend more time reflecting on gender in relation to the on-going and planned activities to identify where there is positive impact and where adaptation is needed to mitigate negative impact. 79. Gender Success Stories. Identify and document the success stories on gender to include in the next annual report. C. HIV/AIDS When the WALA proposal was written, HIV AIDS was one of the cross-cutting themes to be addressed through all SOs. SO1 was to focus on preventive nutrition and health practices for PLWHA and SO2 was to focus on increasing their agricultural productivity and income, while SO3 was to provide targeted food assistance to food insecure households affected by HIV. WALA used the safety net food distribution of SO3 as the entry point to reach these households. Project data shows clearly that these households have indeed been well integrated into other WALA activities including VSL and agriculture along with focused SO1 activities to improve hygiene, sanitation, food processing and preparation, fuel efficient technologies and home gardens. Those interviewed during the evaluation expressed with great emotion how they have regained their health, energy, and sense of worth. Before they go off food assistance after one year, they have a “graduation plan” which shows how they will be able to feed themselves and their families with their new resources. Integration of messages on prevention of HIV and seeking voluntary counseling and testing have not yet been shared across all WALA groups. In spite of very close collaboration in general, there is some uncertainty between WALA and the companion project IMPACT on who is responsible for rolling this out. IMPACT staff and the MCHN TQC are now discussing how to move forward. One possibility is to adopt the community reflection methodology from CRS Ethiopia called “We Stop AIDS”. D. Environmental Monitoring and Impact Mitigation WALA has implemented a number of activities to mitigate and prevent environmental degradation resulting from program activities related to watershed development, conservation agriculture, and irrigation as specified in the Initial Environmental Examination (IEE) undertaken during the proposal submission process. The program has also worked toward integrating environmental protection in all SOs as a cross-cutting theme. In 2011, the program invited external technical assistance from CRS Madagascar to enhance capacities for environmental monitoring in WALA. The Environmental Status Report for FY 2011 submitted WALA Mid-Term Evaluation Report 18 March 2012 42 with the FY 2012 PREP describes mitigation measures that the program implemented during the reporting year, including monitoring of water logging from irrigation systems, reducing soil erosion a key element of watershed development, planting of trees along riverbanks to minimize river erosion and discouraging cultivation on river banks. In general, the program has been performing well on monitoring environmental impact identified during the IEE. The program is less diligent, however, on monitoring emerging environmental impact such as smoke risk from using fuel efficient stoves without chimneys and changes in micro-environment around conservation agriculture, for example, the potential for mulching to create a conducive environment for crop pests. The evaluation also observed that no Environmental Management Plans (EMPs) have been prepared for roads rehabilitated under FFW. The IEE also did not review the commodity warehouse fumigation plans which are normally included in the examination. CRS has reported that they have pre-authorization from USAID for fumigating commodity warehouses, but the documentation to verify this could not be found. RECOMMENDATIONS - Environment 80. Emerging Environmental Impact. The program should expand the monthly and quarterly reporting systems to include sections that allow (and encourage) implementing staff to report on their observations of emerging environmental impact. 81. PRIORITY Environmental Management Plans (EMPs). Program should develop and implement EMPs for roads rehabilitated under FFW. 82. PRIORITY Smoke from Fuel Efficient Stoves. The program should identify modifications that can be made to the fuel efficient stoves being promoted to reduce the risk from smoke inhalation. V. PROGRAM TARGETING AND IMPACT A. Targeting - Who Is Benefitting from Program Activities At the core of identifying program impact is to determine who is benefitting from program activities and how well they conform with the targeted impact group, which for the WALA Program are "214,974 chronically food insecure households". There is a bit of tension in the targeting in WALA, since different targeting strategies are being used for different Strategic Objectives, and not all of these specifically focus on "chronically food insecure households". In SO1, for example, WALA targets pregnant and lactating women and children under five years of age, regardless whether they are from chronically food insecure households. In SO2, WALA targets smallholder farmers defined as those with less than 1 hectare of land, which in some areas, such as the Lower Shire Valley, would exclude many households who are highly food insecure but have landholdings greater than one hectare. The one hectare limit is not rigidly followed by the program in any case12 . Under SO3, the WALA targeting of food recipients for safety net food distributions is following fairly well-defined criteria. The targeting for FFW/FFA participants, however, includes anyone who is a member of a particular program group being supported with FFW/FFA. Targeting drift13 has clearly occurred in WALA with significant 12 In the 2011 Annual Survey, the estimate of the percentage of participants with less than one hectare was 72 percent. 13 Targeting drift refers to the situation in which after a program gets underway the characteristics of actual beneficiaries has drifted away from the definition of the impact group specified at the beginning of the program. WALA Mid-Term Evaluation Report 18 March 2012 43 numbers of beneficiaries who are not defined as chronically food insecure or who do not meet the targeting criteria set for SO2 and FFW. The program has recognized this and is putting in place a system to verify conformance with targeting criteria and plans to report more clearly the number of program beneficiaries who actually fall within the definitions of the impact group. The WALA program is currently working extensively in a total of 215 GVHs from a total LOA target of 273. In interviews held during the evaluation with GVH headmen, most confirmed that there are few people in the GVH who are not participating in WALA in one way or another. They said that those not participating have chosen not to participate ("because they are lazy” or “just don’t like to work in groups”). The evaluation observed that few young male adults were participating in the program and the reason given is that they are looking for more permanent work or migrating for work on plantations, on estates and in urban areas. Others in the community with no labor capacities to participate in FFW and without sources of cash income to participate in VSL are nearly totally dependent on other households in the community. Usually, the “host” households for these people are participating in the program, so even these poorest people are benefitting indirectly from the program. The evaluation did not find significant numbers of people who should have been included in the program but who had been excluded for some reason. Table 6 summarizes the total number of participants for each component and for the program as a whole. As the table shows, the program is making good progress toward achieving the proposed total number of beneficiaries. There are weaknesses, however, in the program's reporting systems in being able to report on the number of people who are benefitting from some program activities. The program also needs to distinguish between beneficiaries who benefit significantly from more than one program intervention and those who benefit from only one intervention. The CACH M&E staff developed a system called the Consortium Management Information System (C-MIS) to be able to estimate this number. The basic system is in place, and data is being collected and entered with the full system expected to be operational within a couple of months. Focus group discussions during the evaluation suggest that a significant proportion of participants are benefitting from multiple interventions. Table 6. WALA Participant Summary as of December 2011 IR or SO Type of Participant Total Participants Thru FY 2011 LOA Target IR 1.1 Practices Households with a child under five or pregnant woman 138,609 170,724 IR 1.2 Services Not Available 170,724 SO 1 MCHN 138,609 170,724 IR 2.1 Crop Production Smallholder farm households with less than one hectare of land 60,097 147,500 IR 2.2 Financial Services 23,363* 103,400 IR 2.3 Marketing 18,330 20,600 SO2 ANRM 60,097 147,500 IR 3.1 DRR Individuals trained 4,058 7,560 IR 3.2 Food Distribution Individual food recipients 17,055 8,197 SO 3 DRR Communities (GVHs) 85 273 Program Total Chronically food insecure households Not Available 214,974 *Adjusted to reflect only beneficiaries with less than one hectare of land WALA Mid-Term Evaluation Report 18 March 2012 44 The evaluation observed a pattern of apparent greater impact in GVHs which were more inaccessible and less exposed to the giveaway programs of past NGOs. People in these communities were showing more self- initiative and interest in participating in WALA activities RECOMMENDATIONS - Targeting 83. PRIORITY Targeting Tension between SOs and Goal. The program should develop context specific criteria for determining who qualifies as "chronically food insecure" and apply these criteria to current and new participants to be able to clearly document beneficiaries from the intended impact groups. 84. PRIORITY Selection of New GVHs. In selecting the remaining fifty-eight GVHs for the program, consider as much as possible within the fuel shortages constraint, those that are more remote where program impact is likely to be greater. B. Impact Being Generated by WALA Table 7 summarizes information on program outputs currently being produced14 in terms of what is working well in achieving expected impact and what needs to be improved in order to enhance impact. Table 7. Where is Impact Being Generated in WALA? Activities Achieving Good Impact Activities Achieving Impact but More Expected Activities Having Potential - Need to be Enhanced Care Groups - Significant numbers of beneficiaries and good impact, especially in sanitation CCFLS - Too focused on recuperation, do not reach all target households, and groups are much too large for skills acquisition. Watershed Development - Relatively few participants, needs better integration with other components, and needs conceptualization at the frontlines strengthened. VHCs - Village Health Committees have been formed or revitalized and are positioned to begin achieving impact in the second half. Supplemental Feeding - Significant numbers of children beneficiaries and good impact. Lead Farmer/Producer Groups - Very limited true adoption so far, but the interventions being promoted can have good impact. Livestock, Storage and IPM - Livestock just getting underway, little progress on storage or IPM. Irrigation (including FFW) - Irrigation structures are well designed, but more attention needs to be given to the software capacity building. Village Savings and Loans - Extraordinary, diverse impact with numbers of participants beginning to grow rapidly. Marketing Clusters and Clubs - Participation numbers on target and some linkage activities beginning to have impact. DRR Linkages - Linkages from villages to District established but not systematically developed to higher levels. Market Linkages - Some year three linkages established and the program in discussion on further linkages. . Safety Net Food Distributions - Excellent impact for significant numbers DRR Capacity-Building - Civil Protection Committees strengthened and DRR training done with 31% of the local instutions targeted. C. Impact at the Program Goal Level At the goal level, the program is expected to reduce food insecurity for chronically food insecure households. The program Performance Monitoring Plan does not specify impact indicators at the 14 Some activities, such as CIMCI and GMP under SO@ are being implemented in the second half of the program. WALA Mid-Term Evaluation Report 18 March 2012 45 goal level, so the only information available at present to capture impact information is from the qualitative information gathering. While it is too early to expect life changing impact, some notable impact observed by the evaluation team included the extent of the success being achieved by Care Groups on sanitation, the impact of food distributions on the households with chronically ill members, and the multiple impact being achieved with Village Savings and Loan Groups. VI. PROGRAM SYSTEMS A. Program Management Program management investigations during evaluations typically look at management structures, program vision and leadership, program planning, problem–solving and decision–making and communications. The evaluation found that the Consortium Administration and Technical Capacity Hub (CATCH) has faced significant staffing challenges over the life of the program. Transitions have occurred with key positions, including the COP, Deputy COP for Program, M&E Coordinator, Agriculture TQC, and MCHN TQC; and the program also faced significantly delayed recruitment of the DRR and Livestock TQC positions. Implementation in some components is behind schedule as a result. Even at present, at least two positions in CATCH are vacant or are about to become vacant. Despite the staffing problems, CATCH represents a "best practice". Key features are that (1) the vision and leadership responsibilities for the program clearly rest with one partner15 who is the grant recipient, (2) compliance as a theme for CATCH is less important than facilitating technical support and coordination, and (3) CATCH is housed in one location, close to the field. All in all, there is a general feeling of satisfaction across the program with management at the consortium level, particularly relative to vision, leadership, and communications. In interviews during the evaluation around management topics, nothing was consistently mentioned as a significant problem. Management issues that were mentioned included: • Limited opportunities for discussions on the bigger picture of the WALA strategy. • Management practices in some partners are too relaxed and responses to queries and requests sometimes take too long; but on the other side, sometimes not enough time is given to review documents that have been received. • Decision–making is sometimes too bureaucratic involving too many layers on issues/processes that should be resolved more quickly, especially if these issues/processes need to be referred to head offices. • Procurement is cited as an area which has been problematic. Finally, the evaluation observed that there were few maps of the program area, which are useful tools for planning, in any of the WALA offices. B. Partnerships Partnership investigations during evaluations examine the effectiveness of the relationships between partners to identify best practices or issues that are affecting implementation. In WALA, the evaluation observed that implementing partners are working within the framework of a common strategy, but they are allowed to deviate in order to capitalize on an organizational 15 In the previous ILIFE program, the leadership was split between one organization that held the COP position and another organization which held the DCOP position and was the grant recipient. WALA Mid-Term Evaluation Report 18 March 2012 46 comparative advantage or to test a new approach. The program also has mechanisms, primarily the Technical Working Groups and the quarterly managers' meetings, for airing problems and learning from each other. Implementation partners include a mix of international organizations and national NGOs. All are treated as equal partners in WALA. The three new partners in the consortium have all developed competencies for commodity management. The smallest (and newest) partner working in one of the most difficult parts of the country is meeting its WALA targets. While each partner has its strengths and weaknesses, no major differences were observed in program quality between implementing partners. Partnerships in large programs can be fraught with tension, but there have been relatively few substantial clashes of organizational cultures in WALA. The major source of frustration is around different financial management systems, and the program has already explored solutions, including developing individualized reporting schedules for each partner. Since these systems are part of worldwide financial systems for each partner, there appears to be not much more that can be done to address this tension. Minor clashes have also occurred over the life of WALA around initial implementing agreements, recruitment of TQCs, and recruitment of staff from WALA for other programs being implemented by partners. The program has long–term technical partnerships with Agricane for irrigation and ACDI– VOCA for agri–business. Both of these are functioning well and producing results. All in all, partnership relationships are functioning well, and organizations are learning from each other while also have some flexibility in approaches. C. Knowledge Management Knowledge management refers to how knowledge is brought into a program, how it is generated and used within a program through M&E systems, and how it is generated and disseminated outside of a program. 1. Knowledge In Relative to bringing in new ideas, the WALA Program has obtained an admirable level of external technical support from a variety of sources. Annex I contains a list of information sources that have been tapped by WALA. Some additional specific technical assistance needs identified during the evaluation that may need to be addressed include concepts of farmer-driven innovation, participatory monitoring and evaluation approaches, and technical support for the design of road rehabilitation FFW projects. 2. Monitoring and Evaluation Systems Staff turnover with M&E positions in CATCH and with some partners has affected progress on designing and implementing M&E systems. The program has begun work on a number of improvements to the systems including (1) installing a Consortium Management Information System (CMIS) which will track a wider range of performance indicators as well as be able to determine households that are benefitting from multiple interventions, (2) implementation of Quality Improvement Verification Checklists for MCHN, Agriculture, Irrigation, VSL and Agri￾Business, (3) implementation of a Knowledge Management Strategy, and (4) expansion of the pilot system for gathering Household Hunger Scale Data using SMS, a trigger indicator WALA Mid-Term Evaluation Report 18 March 2012 47 monitoring system. The need for these improvements was identified last year but because of staff turnover, as well as fuel shortages, it is taking longer than anticipated to get these in place. The evaluation observed that the IMPACT Program appears to be demanding a disproportionate amount of the time of M&E and management staff of implementing partners. Participatory Monitoring and Evaluation in which program participants define their own performance indicators and assess progress against these, is not commonly done across the program. It is being done as part of the Participatory Planning Monitoring and Evaluation process used in SO3, and it is also being done in spots around the program. It would be a very useful approach for helping communities monitor the nutritional status of children and for Watershed Development Groups to monitor the impact of watershed development activities, but it can also be useful for any of the groups formed by the program. Indicator Performance Tracking Table. A program needs to have a good tool that that can be used to monitor progress against all outputs and provides indicators for assessing impact. Because of the scope of WALA and the limitations on size, the Indicator Performance Tracing Table (IPTT) does not serve as an effective tool at the moment. The IPTT, for example, is missing some key outputs indicators associated with watershed development, livestock activities, assets created through FFW/FFA, community-based service providers, linkages with formal DRR structures, and environmental protection. It is also difficult to judge progress against some participation targets specified in Intermediate Results, for example, participants in IR 2.1 on access to MCHN services. The program has already negotiated some changes to outcome indicators, but there are still some indicators that are inconsistent. For example, Indicator 2.1.1a an outcome indicator to capture adoption of sustainable agricultural technologies is defined in terms of some conservation agriculture technologies, including intercropping, crop rotation, minimum tillage and mulching which are soil conservation techniques which should be included under Indicator 2.1.1b which is focused on soil conservation. Finally, there are no indicators to capture other outputs and impact of the program related to the diffusion of innovation, such as for example, with fuel efficient stoves and VSL, other than using impact indicators with a population-based sample at the end of the program. WALA recognized many of these problems and consequently began developing a comprehensive Indicator Tracking Tool (ITT) which will guide information to be entered into the CMIS. Annex J contains suggestions compiled by the evaluation team on changes and additions to the performance indicators. WALA is achieving significant impact in some areas that is at risk of not being captured during the final evaluation because of the reduced number of impact indicators. There is little point in changing outcome and impact indicators now, since baseline information on any new indicators may not have been gathered during the Baseline Survey. In similar circumstances, other programs have developed and implemented strategies to obtain information using ad hoc surveys sometimes with a control-treatment methodology to ensure that the information is available for the summative impact evaluation. RECOMMENDATIONS - M&E 85. CMIS/Indicator Tracking Tool. The program already has a time schedule for rolling out the CMIS/ITT system. One last review of the ITT list should be made to ensure that all outputs are covered. 86. Participatory Monitoring & Evaluation (PME). The program should use expertise from within the consortium (or outside if necessary) to build capacities for using PME effectively WALA Mid-Term Evaluation Report 18 March 2012 48 across all SOs. 87. PRIORITY Final Evaluation Preparation. In preparation for the Final Evaluation, the program should (1) review the program document to identify proposed impact (e.g., improved access to non-timber forest products) that is not being achieved because there are no outputs being implemented to achieve these outcomes and begin discussions with USAID to ensure that the program is not held accountable against these, and (2) develop and implement a strategy for obtaining information to fill in the gaps on important outcomes and impact that the program is achieving. 3. Knowledge Out. In general, not many documents have been disseminated very widely outside of the program apart from the Baseline Report and some program monitoring reports. CATCH has one position which is focused on this dimension of knowledge management, but the position was only recently filled and is about to be vacated again. Despite this, the program has contributed to extending knowledge outside in other ways. The VSL component, for example, has made contributions to the design of the National Social Protection Policy; and as members of various technical networks, WALA contributes knowledge and experience on issues discussed. WALA has also recently begun inviting representatives from other non-participating TAs to attend some events in order to expose them to the WALA strategy. Assuming the knowledge management position in CATCH will be filled and become fully functional, the evaluation does not have any specific recommendations for strengthening the knowledge out function. C. Program Integration and Complementarity Within WALA, there is integration in the sense that many households are participating in multiple WALA activities, although the current M&E systems are not able to capture this. The improved C-MIS, once it is fully operational in a few months, however, will make it possible to identify not only participants who benefit from multiple interventions but also participants who are not participating in an activity in which they should, such as a household meeting the MCHN criteria but not reached by a Care Group Volunteer. While for the most part, overlap across components is not systematically captured, the participation of safety net beneficiaries in other WALA activities has been documented, showing excellent integration. While the evaluation did observe that participants were engaged in different components of WALA, this was more coincidental than part of a strategy to facilitate greater integration across components. Even when a household is participating in multiple groups, there is not always integration of activities. For example, conservation agriculture techniques are rarely used with crops being produced for the market such as chillies, even though the farmer is a member of a Producers Group and a Marketing Club. Relative to integration of WALA with the structures and strategies of government, integration of WALA under SO1 with the MOH varies. There has been very good collaboration at the national level as described earlier, and district health officers are well-informed about WALA but this does not hold true in many cases for District Health Office functionaries such as nutrition and MCHN coordinators. In the field, the coordination with Health Surveillance Assistants is very strong in some places and less so in others. In SO2, while coordination with government agriculture agents is not systematized, there is excellent coordination with the agriculture research centers and increasing integration with the private sector in building market linkages. The DRR work under SO3 is well-integrated with the government from the District level down. WALA Mid-Term Evaluation Report 18 March 2012 49 There are many other recent or current development initiatives targeting the same districts as WALA whose presence is coordinated by the Development Executive Committee (DEC) of each district. The WALA representative who attends these meetings is generally quite well-informed about what the other agencies are doing, at least in a broad sense. Among field-level staff, there is much less knowledge of these initiatives and where collaboration might be useful. RECOMMENDATIONS - Integration 88. PRIORITY SFP Beneficiaries. As with safety net food recipients, the program should make a determined effort to engage those families receiving SFP whose children have repeated malnutrition in other WALA activities to increase their food production or income. 89. PRIORITY Strategic Integration Across SOs. The program should generally plan more strategically to facilitate integration across components, for example:  SO2 should promote production of groundnuts to support the promotion of ground nuts for child nutrition in SO1  Similarly, SO2 should give more attention to poultry, oilseeds, and fruit trees to support nutrition messages under SO1.  Before share out, VSL members should be asked to reflect on how they might use the funding in relation to what they are learning in SO1, for example, to use some of the cash to purchase soap to improve family hygiene.  Community-based disaster management plans developed under SO3 should incorporate the watershed development plans developed under SO2. 90. PRIORITY Other Complementary Programs. An inventory should be made of other programs and projects in each district with interventions similar to those of WALA, and WALA field staff should be encouraged to seek out field staff of these other programs to informally share lessons learned, technologies, and assess potential for collaboration. E. Financial Resource Management The total cash budget for the WALA Program of $52,368,266 includes monetization proceeds, 202(e) and ITSH. As of December 2011, total expenditures against this budget amounted to USD 22,154,190, representing 42.3% of the LOA budget. December 2011 represents 51.6% of the program life, so the program appears to be somewhat underspent on its budget, although the annual budgets for years two, three and four are higher than for years one and five. The low burn rate is attributed to a high rate of staff turnover in Year 2 which slowed program implementation. WALA is more fully staffed at the present time and it is expected that the program will be accelerated in the coming year. At the implementing level, expenditures against the approved LOA budget through December 2011 range from 33% to 58%. Table 8 provides a summary of budget and expenditures. A more complete table is provided in Annex K. Over the life of the program, there have been no major budgeting or cash flow problems as a result of delayed monetization or fund transfers from the donor. There have been occasional instances of short term cash flow breaks due to lengthy financial reporting policies and procedures, submission of incorrect reports, and decision-makers being located in diverse locations inside or outside of Malawi. Some implementing partners reported that there have been improvements on cash flow in the last year. The evaluation observed that for some partners, managers have not been delegated much authority for approving financial transactions. Either WALA Mid-Term Evaluation Report 18 March 2012 50 Table 8. Budget and Cash Expenditure Summary Through December 2011 (US$) Budget and Expenses Monetization Proceeds 202e ITSH Cost Share Total TOTAL LOA Budget at Time of Approval 34,886,839 10,276,263 6,318,531 886,596 52,368,229 Current Amended LOA Budget 33,506,639 11,656,500 6,318,531 886,596 52,368,266 TOTAL Expenses through December 2011 12,319,604 6,871,551 2,558,380 404,655 22,154,190 Percent of Current LOA Budget Spent by through December 2011 37% 59% 40% 46% 42% Percent of Current LOA Implementing Partner Budgets Spent by through December 2011 34% 49% 35% 38% 38% they have not been given the authority to sign checks, or the limit that they have on checks that they can sign is relatively low. This weak delegation of authority increases processing time on financial transactions. Internal audits conducted annually have not identified any major findings relative to financial management systems. RECOMMENDATIONS - Financial Management 91. PRIORITY Delegation of Authority. All partners should review their delegations of authority for financial transactions to look for opportunities to make financial transactions more efficient without threatening high levels of accountability. F. Commodity Management The approved LOA commodity levels for WALA are 89,790 MT of commodities for monetization and 16,200 MT for distribution through safety net distributions, supplemental feeding, and food for work. Details on commodities, including monetization cost recovery and commodity losses are included in Annex K. As of December 2011, the program has monetized six consignments totaling 43,349 MT of commodities including 39,300 MT of wheat and 3,650 MT of Crude Degummed Soybean Oil (CSDO) with an average cost recovery of 76% of the C&F value. The average anticipated cost recovery for these shipments was 73%. Relative to distribution commodities, as of December 2011, the program has received 9,761 MT of commodities (CSB, pinto beans and vegetable oil) representing 60% of the LOA approved level and 6,024 MT, representing 37% of the total LOA approved level, has been distributed to program participants. Relative to commodity losses, total losses to date amount to slightly less than .3% of total commodities called forward, with all of these losses occurring during ocean transport. Despite the challenges of operating in a landlocked country, commodity distribution, transportation, security, tracking and reporting are functioning well in the WALA Program. The only weakness observed during the evaluation was poor ventilation in a few partner warehouses. WALA Mid-Term Evaluation Report 18 March 2012 51 RECOMMENDATIONS - Commodity Management 92. PRIORITY CATCH Commodity Manager. The expatriate commodity manager position in WALA is due to be phased out at the end of this fiscal year. Because of recruitment challenges, however, the program is not prepared to turn over responsibilities to a local staff member, therefore, the current commodity manager should be retained until a competent successor has been fully trained. 93. Warehouse Ventilation. The program should ensure that all warehouses have enough ventilation for storage of commodity for a long period (more than three months). G. Human Resource Management The WALA Program has encountered substantial challenges in recruiting and retaining staff for the program, particularly in CATCH and with some implementing partners. The program, at the moment, is as fully staffed as it has been over its life, but there are still 33 positions, or nearly 8%, of a total of 428 positions in the program that are vacant. The main challenge is a small pool of qualified staff in-country relative to the rising demand for this staff from new programs and projects starting up. There are some WALA partners, however, who have not faced high levels of staff turnover; and they attribute this to satisfactory compensation and benefits packages and a satisfying work environment that empowers staff without imposing a workload that exceeds the capacity of the position. Relative to other observations on human resources during the evaluation, as already mentioned, the location of the TQC for Governance at the partner project office rather than in the CATCH office has constrained implementation, and there are no full-time positions within WALA that have specific responsibilities for ensuring that gender is addressed as a cross cutting theme in the program. RECOMMENDATIONS - Human Resources 94. PRIORITY Expanding the HR Pool. The program should strategize on ways to expand the pool of candidates for positions in WALA, for example, through internship programs with local universities or finding ways to attract the Malawian diaspora back to Malawi. VI. PROGRAM- LEVEL CHALLENGES During the evaluation, a number of major challenges were identified that will likely continue to affect the ability of the program to achieve its goal. The program has already begun developing strategies to deal with these, but the evaluation team would like to make some additional suggestion around refining the strategies that are evolving. The challenges identified include (a) resource constraints related to staff recruitment and retention, persistent fuel shortages, and the likely devaluation of the kwacha, (b) the need for more comprehensive sustainability plans and exit strategies, and (3) addressing community expectations for hand–outs. A. Resource Constraints 1. Staff Recruitment and Retention. As mentioned in the previous section, the size of the pool of people from which to recruit staff for the program continues to constrain recruitment and retention, especially in CATCH and some partners. Related to this is that a number of positions are scheduled to be phased out at the end of this project year, including the M&E Coordinator, the Commodity Manager, and the Agri– WALA Mid-Term Evaluation Report 18 March 2012 52 Business TQC. Given the recruitment challenges, the program would likely falter if these positions are totally phased out this year. 2. Devaluation While it is not clear when a devaluation will occur, most analysts believe a devaluation is inevitable. Some theorize that the devaluation will not occur before the next elections in 2014. However, should it occur, the impact on WALA could be substantial, since the largest component of the budget is monetization funds which are received in local currency. CRS is already exploring ways to bring the local currency balance down, looking into ways to receive monetization funds in a currency other than kwacha, and having discussions with FFP on contingency plans. The program may also want to systematically prioritize activities at different levels to determine those that can be reduced or eliminated, should devaluation severely affect program resources. 3. Fuel Shortages The program has been facing serious fuel shortages for almost a year now with significant impact on the program’s activities, especially on monitoring field activities and rolling out new activities. Each partner is using its own capacities and relationships to stock fuel when it becomes available, and CRS/CATCH is taking into account WALA needs in developing their contingency plans. Other smaller activities that the program might consider to mitigate the impact of fuel shortages are to (1) address some significant fuel consumption patterns such as requiring WVI and TLC, for example, to drive to the east bank in the Lower Shire and having CKD go through Blantyre to reach Chabvala, and (2) train drivers on improved driving habits, for example, turning vehicles off when parked and reducing air-conditioner use. B. Sustainability Plans and Exit Strategies Apart from the PSP model for VSL, WALA has not systematically developed sustainability plans with exit strategies to ensure that impact is sustained and replicated after the program ends. With less than 2 ½ years remaining in the life of the program, there is not a lot of time to get measures in place to mitigate threats to sustaining impact. The program should undertake a systematic process to develop additional sustainability plans and exit strategies. Typically this process involves: 1. Defining sustainability objectives which are organizations, services, or relationships that need to continue beyond the life of the program to maintain or expand impact 2. Identifying threats to these objectives, and 3. Formulating program activities that can be undertaken in the remaining life of WALA to mitigate these threats which then becomes the foundation for an exit strategy. C. The Hand-Out Syndrome WALA needs to ensure as much as possible that it does not contribute to the hand–out mentality evident in communities that have a history of working with NGOs, especially those that have responded to emergencies. A set of principles and practices should be developed to guide activities to mitigate this impact. Examples of some principles and practices relevant for WALA would include the following.  Adopt the position that anything that villagers can do themselves, they should be encouraged to do. WALA Mid-Term Evaluation Report 18 March 2012 53 Flooding in Mbangu GVH The Village of Zimu in Mbangu GVH, didn't wait for external assistance to come in order to enable them to survive the recent floods in the Lower Shire Valley. They organized themselves and excavated a river channel and planted trees to effectively cope with the floods.  Before distributing an input, consider implications on other components that may be working with the same inputs but in other ways. In WALA, for example, a livestock pass-on approach should not be used in the same villages where VSL groups are purchasing their livestock.  If inputs must be provided by the program, this should be done as much as possible through the supplier of the input, to build linkages as well as to detach the program from being perceived as giving away materials.  When food or inputs are provided, stress transparency so that other participants know the criteria that were used to determine eligibility.  Develop policies for ensuring that FFW is used mainly to demonstrate new ideas that participants do not know how to do.  Implement a low-cost campaign around the message that depending on NGOs for assistance is not in the best interests of Malawian villages.  Reward villages that show significant self–initiative, such as the case in the text box on the right. VII. CONCLUDING REMARKS As with every program that is reviewed midway through its life, the WALA Mid -Term Evaluation found activities that are working well in producing planned outputs and activities that need to be improved in order to result in outputs more likely to produce intended outcomes and impact. In some cases, evaluations may even find activities that should be discontinued because they are not likely to have impact due to contextual changes or other reasons. In the WALA Program, the evaluation found that the program logic is rational. The planned activities and outputs are likely to result in the assumed outcomes which will ultimately produce the intended impact on the food security of targeted impact groups. The evaluation also found that no parts of the strategy appeared to be irrelevant at this point. The WALA Program is implementing many activities that were already inducing behavioral change, and the evaluation found relatively fewer activities that needed to be improved. All in all, the WALA Program has established itself as a sound program, with the potential to become a great program, if it can make some adjustments and meet the challenges effectively. WALA Mid -Term Evaluation Report Annex A 54 ABBREVIATIONS & ACRONYMS AC Agribusiness Coordinator ACA Agribusiness Change Agents or Agri-business Community Agent ACDI-VOCA Agricultural Cooperative Development International and Volunteers in Overseas Cooperative Assistance ACE Agricultural Commodity Exchange ACPC Area Civil Protection Committee ACTESA Alliance for Commodity Trade in Eastern and Southern Africa ADB African Development Bank ADC Area Development Committee ADD Agriculture Development Division ADP Area Development Programme or Agricultural Development Program AEDO Agricultural Extension Development Officer AELA Agro-Enterprise Learning Alliance AEO Agriculture Extension Officer AFASS Acceptable, Feasible, Affordable, Sustainable and Safe AIDS Acquired Immuno-Deficiency Syndrome ALIDRAA Ask, Listen, Identify, Discuss options, Recommend and negotiate, Agree and repeat agreed-upon action, follow-up Appointment ARI Acute Respiratory Illness ART Anti-Retroviral Therapy BCC Behavioral Change Communication BDS Business Development Services BMGF Bill & Melinda Gates Foundation BMI Body Mass Index BSDI Beneficiary and Service Delivery Indicators C&F Cost and Freight C-FISH Captive Fisheries for Income and Strengthened Households C-IMCI Community Integrated Management of Childhood Illness CA Conservation Agriculture or Community Agent CAADP Comprehensive Africa Agricultural Development Programme CATCH Consortium Administration Technical Coordination Hub CBCC Comunity-Based Child Care Center CBES Community-Based Extension CBO Community-Based organization CCFLS Community-led Complementary Feeding and Learning Sessions CCT Continuous Contour Trenches CDSO Crude De-gummed Soybean Oil CFU Conservation Farming Unit CG Care Group CGV Care Group Volunteer CI Chronically Ill CIMMYT Centro Internacional de Mejoramiento de Maíz y Trigo (International Maize and Wheat Improvement Center) CKD Chikwawa Diocese WALA Mid -Term Evaluation Report Annex A 55 CLCFLS Community-Led Complementary Feeding and Learning Sessions CM Consortium Member CMAM Community-BAsed Management of Acute Malnutrition CMD Cassava Mosaic Disease CMIS Consortium Management Information System CNFA Citizens Network for Foreign Affairs CONGOMA Council for Non-Government Organizations in Malawi COVO Community Voice Organization CPC Civil Protection Committee CRS Catholic Relief Services CSB Corn Soya Blend CSFLS Community Supplementary Feeding & Learning Session CSI Civil Society Index CTC Community Therapeutic Centers or Community Therapeutic Care CTU Community Therapeutic Care Unit CU Concern Universal DAC District AIDS Coordinator DAP Development Assistance Program DADO District Agriculture Development Officer DC District Commissioner DCT District Coordinating Team DDRMP District Disaster Risk Management Plan DEC District Executive Committee DEHO District Environmental Health Office DF Development Facilitator DHMT District Health Management Team DHO District Health Office DIP Development (or Detailed) Implementation Plan DME Design, Monitoring and Evaluation DNH Department of Nutrition and Health DN-OPC Department of Nutrition in the Office of the President and Council DPD Department of Planning and Development DPDMA Department of Poverty and Disaster Management Affairs DPM Deputy Program Manager DRR Disaster Risk Reduction DSWO District Social Welfare Office EASPM Economic Activity Selected Planning and Management EBF Exclusive Breast Feeding EI Emmanuel International ENA Essential Nutrition Actions EPA Extension Planning Area EU European Union FaaB Farming as Business FANTA Food and Nutrition Technical Assistance FAO Food and Agriculture Organization FDP Food Distribution Point WALA Mid -Term Evaluation Report Annex A 56 FEF Farmer Extension Facilitator FEV Farmer Extension Volunteer FEWSNET Famine Early Warning Systems Network FFA Food for Assets FFP Food for Peace FFW Food for Work FGD Focus Group Discussion FO Field Officer FY Fiscal Year GDA Global Development Alliance GM Growth Monitoring GMP Growth Monitoring and Promotion GoM Government of Malawi GVH Group Village Headman HBC Home-Based Care HF Health Facilitator HH Household HHS Household Hunger Scale HIV Human Immuno-deficiency Virus HP Health Promoter HRWW Hard Red Winter Wheat HSA Health Surveillance Assistant HTC HIV Testing and Counseling I-LIFE Improving Livelihoods through Increased Food Security IB Intermediate Buyer ICRISAT International Cros Researcj Institute for the Semi-Arid Tropics ICT Information Communication Technology IEC Information, Education and Communication IFAD International Fund for Agricultural develoment IGA Income-Generating Activity IITA International Institute of Tropical Agriculture I-LIFE Improving Livelihoods through Increasing Food Security ILO International Labor Organization IMPACT Integrated Mitigation HIV/AIDS Effect and Positive Action for Community Transformation IPTT Indicator Performance Tracking Table IR Intermediate Result IYCF Infant and Young Child Feeding K&L Knowledge and Learning LF Lead Farmer LFG Lead Farmer Group LOA Life of Activity LQA Lot Quality Assurance LV Lead Volunteers M&E Monitoring and Evaluation MCHN Maternal and Child Health and Nutrition WALA Mid -Term Evaluation Report Annex A 57 MDHS Malawi Demographic Health Survey MDGs Millennium Development Goals MGDS Malawi Growth Development Strategy MIS Management Information System MOI Market Opportunity Identification MoH Ministry of Health MT Metric Ton MYAP Multi Year Assistance Program MoAFS Ministry of Agriculture and Food Security MoH Ministry of Health MT Metric Ton MYAP Multi-Year Assistance Program NAPHAM National Association of People Living with HIV and AIDS in Malawi NARS National Agriculture Research Systems NASFAM National Smallholder Farmers' Association of Malawi NASSPA National Smallholder Seed Producers Association NFI Non Food Items NGO Non-Governmental Organization NRM Natural Resource Management NRU Nutritional Rehabilitation Unit NSR Night Storage Reservoir OFSP Orange-Fleshed Sweet Potato OIBM Opportunity International Bank in Malawi OPC-DNHA Office of the President and Cabinet-Department of Nutrition and HIV and AIDS OPV Open-Polinated Varieties (of maize) OTP Outpatient Therapeutic Care OVC Orphans and Vulnerable Children PCI Project Concern International PCVA Participatory Capacity Vulnerability Assessment PDI Positive Deviance Inquiry PDQ Partner Defined Quality PG Producer Group PIMS Project Information Management System PLHIV People Living with HIV PLW Pregnant and Lactating Women PMTCT Prevention of Mother to Child Transmission PPM&E Participatory Planning, Monitoring and Evaluation PREP Pipeline Resource Estimate Proposal PSP Private Service Provider PVO Private Voluntary Organization QIVC Quality Improvement Verification Checklist QPM Quality Protein Maize RUTF Ready to Use Therapeutic Food SAVE Strengthening Agriculture Value Chains and Enterprises SC Save the Children SFP Supplementary Feeding Program WALA Mid -Term Evaluation Report Annex A 58 SO Strategic Objective SOG Standard Operating Guideline SWC Soil and Water Conservation SWOT Strengths, Weaknesses, Opportunities and Threats TA Traditional Authority TLC Total Land Care TNP Targeted Nutrition Program ToT Trainer of Trainers TQC Technical Quality Coordinator TWG Technical Working Group UNECA United Nations Economic Commission for Africa USAID United States Agency for International Development VCPC Village Civil Protection Committee VDC Village Development Committee VHC Village Health Commitee VSL Village Savings and Loans VUC Village Umbrella Committee WALA Wellness and Agriculture for Life Advancement WFP World Food Program WILA Wellness in Irrigation for Life Advancement WHO World Health Organization WOTR Watershed Organization Trust WUC Water User Committee WUG Water User Groups WVI World Vision International WALA Mid-Term Evaluation Report Annex B 59 MID-TERM EVALUATION RECOMMENDATIONS The following Recommendations have been made in the Mid-Term Evaluation of WALA SO1 - MCHN Behavioral Change Communications 1. PRIORITY Contextualized Messages. During training for new modules, give WALA staff concrete examples of how to contextualize the training of promoters and Care Group Volunteers to priority messages for the situation. 2. PRIORITY Activities for Men. Plan activities for men at least quarterly making them fun and limited to key messages that will enable men to understand their role in improving nutrition for their wife and children. 3. PRIORITY Sweet Potatoes, Papaya, Groundnuts & Oilseeds. With the technical working group for agriculture, define plans to expand introduction of sweet potatoes (particularly in Nsanje and Chikwawa), distribute papaya seedlings with technical assistance, increase production of ground nuts for home consumption, and investigate production of oilseeds and local presses for increasing local consumption. 4. PRIORITY Poultry. Link poultry activities to Care Groups, focusing on training and sourcing for Newcastle disease and other vaccinations, access to locally produced chicken feed as a supplement for free-range chickens, and production of a one-page flyer on caring for local chickens for use by Care Group Volunteers. 5. Impact on Diet Diversity. In the next annual survey, compare data on food groupings with the baseline and recent survey to assess impact of dietary diversity messages and activities. Care Groups 6. PRIORITY Home Visits Versus Group Sessions. Emphasize home visits over group sessions 7. PRIORITY Review Sessions. Select priority key messages and conduct a review session for all CGVs 8. PRIORITY Participatory M&E for CARE Group Volunteers. Help CGVs start their own monitoring system with goals for key behaviors 9. PRIORITY Incentives for CGVs. Arrange incentives that are performance-based and low￾cost or non-monetary 10. PRIORITY Linkages. Build stronger linkages between Care Groups and health staff and VDCs. CCFLS 11. PRIORITY Group Size. Limit groups to no more than 10 mothers in separate places. 12. Seasonal Participation. Invite all mothers to participate at least twice in different seasons. 13. Food Processing. Save the food processing lessons for harvest time outside CCFLS. WALA Mid-Term Evaluation Report Annex B 60 14. PRIORITY Recipes. Make sure the recipes are truly local using affordable ingredients and do not call for processes that burden the mother with extra time spent on food preparation. 15. PRIORITY Feeding. Highlight active feeding, consistency, and amount. 16. Field Supervisor Checklist. Field supervisors need to use the Quality Improvement and Verification Checklist to assure the standards for CCFLS, including small group size, are being met. 17. Followup Training for Field Supervisors. Field supervisors may need more training on the key concepts to strengthen this with the promoters. 18. Identifying Malnourished Children. Weigh in at the first session. Screen with MUAC to make referrals. Weigh only the malnourished children after the last session and refer those who did not gain weight. 19. Reporting Burden. Reduce the reporting burden by randomly selecting only a few sites in each district to report only the weight gains and attendance of malnourished children. Supplemental Feeding 20. PRIORITY Integration. CGVs should provide close follow-up to SFP-enrolled families to assure that they are integrated into other WALA activities such as gardens, poultry, VSL, and agriculture, as appropriate. 21. ANC Attendance CGVs can promote and monitor ANC attendance 22. PRIORITY Safety Net. Consider shift of food resources to safety net if targets for women and children cannot be met. Community Capacities 23. Staff Responsibilities. WALA will need to clarify which staff member is responsible for forging the relationships between community groups and district development authorities before this can move forward. 24. Advocacy on Nutrition. It may be more crucial to focus on advocacy with district development authorities to add nutrition to their priorities. General Recommendations for SO1 25. PRIORITY Positive Reinforcement. Shift the focus to positive reinforcement: Reward mothers or parents whose children are normal weight at certain ages with community recognition or another non-monetary incentive. 26. PRIORITY HSA Capacities. Train HSAs to the same level as the promoters. 27. Performance Indicators for SO1. Some performance indicators can be revised for clarity as described in the knowledge management section below. SO2 - ANRM Extension Services WALA Mid-Term Evaluation Report Annex B 61 28. FEF to Farmer Ratio: In the next annual survey, a large enough sample should be selected to obtain data to be able to compare information between PVOs on adoption outcome indicators to determine whether there is link between adoption and the FEF to farmer ratio and a recommended ratio should be agreed upon. 29. PRIORITY Standard Operating Guidelines. The extension component of WALA is in need of a clear, contemporary, context appropriate vision. The Standard Operations Guidelines should be completed as soon as possible since this will operationalize the program's vision. 30. PRIORITY FEF Sustainability Strategy. The program should develop a strategy for enabling FEFs to continue to serve in this function after the program ends. Elements of this strategy would include: • Stronger links between them and sources of new ideas and technologies in government, the NGO sector and the private sector • Stronger links specifically with the Department of Agricultural Extension, so that by the end of the program, FEFs are perceived by government as an extension of their systems. • Identifying services that they can provide to farmers from which FEFs can generate income and be motivated to maintain contact with farmers. 31. Systematically Monitor the Effectiveness of the Combined FEF/ACA. While TQCs from both the agribusiness and extension components of the program have visited the location where the functions of the FEF and ACA have been combined, a more rigorous and systematic analysis should be conducted to determine whether this is a viable model for facilitating sustained extension and marketing services. Conservation Agriculture 32. PRIORITY Quality of Demonstrations. All future demonstrations should clearly indicate using on–site sign boards in Chichewa what is being demonstrated and should have an area set aside showing traditional cultivation practices for comparison. 33. PRIORITY Additional Demonstrations. In order to increase the visibility of the techniques being promoted (and encourage adoption), the program should take advantage of opportunities to show the techniques by preparing small signboards in Chichewa to be placed in fields of early adopters who have convincing demonstrations. 34. PRIORITY Farmer–Driven Innovation. Most of the new ideas and technologies being promoted by WALA are coming from the government research or extension systems and in some cases the private sector. The program should put in place a system for encouraging lead farmers, FEFs, and front–line staff to find new ideas being tested and adapted by farmers themselves. 35. Advocate for Government Input Subsidies for Drought–Resistant Food Crops. Food crops such as millet and sorghum are much more appropriate for some parts of the WALA program area than maize, but since government support is focused now on maize and cotton, farmers are pulled toward production of these crops. Given its scale, WALA has some clout that could be brought to bear in an advocacy strategy to increase government support for drought–resistant food crops. Irrigation WALA Mid-Term Evaluation Report Annex B 62 36. Irrigation Area Targets. Program implementation is being driven to meet an area target. The principle that the program should follow in doing this is to achieve the target with the maxmimum number of beneficiaries using the most appropriate and acceptable technology for the specific context. 37. PRIORITY Capacity Building of Marketing Committees: The program should include all Water Users Groups Marketing Committees in the agribusiness component. 38. PRIORITY Increasing Production. Once marketing strategies are developed targeting specific crops, facilitate linkages to FEFs or other sources of technical advice on the crops selected and establish demonstrations on high value irrigated crops.. 39. PRIORITY Cash Resource Mobilization and Management. Provide more intensive training on cash resource mobilization and management. 40. PRIORITY Governance. Intensify governance training, since representation, accountability and transparency will be critical for resolving disputes over water between members or between the WUG and outsiders. Post–Harvest Storage 41. PRIORITY Additional Options. The program should explore and test additional options for pest control in stored maize that are more appropriate for households with limited cash resources. 42. Estimating Storage Losses. Given the wide disparity between estimates of storage losses, WALA should use the opportunity of the next Annual Survey to obtain information from participants on storage losses. Watershed Development 43. PRIORITY Monitor Adoption: The program should monitor more closely the few cases where participants are replicating watershed technologies and modify the WALA strategy to incorporate these. 44. PRIORITY Agro–Forestry Nurseries: The program should develop a strategy for sustaining nursery services after the program ends, most likely by finding a way to convert the nurseries to self–sustaining income–generating activities. 45. PRIORITY Participatory Monitoring & Evaluation: The program should build on the experience of some consortium partners to expand use of PME to enable watershed management committees to monitor impact on the watershed. (The risk, of course, is that over the life of WALA, they fail to measure any impact and get discouraged). 46. Including Roads in Watershed Development Plans. The program should look for appropriate ways to use roads to capture and direct water in watershed development strategies. Livestock 47. PRIORITY Livestock Focus. If funding allows, livestock activities should be scaled up as much as possible with special attention to food insecure participants who are involved in SO1 (for nutrition knowledge), VSL (for capital) and production groups (for using manure). WALA Mid-Term Evaluation Report Annex B 63 48. PRIORITY Pass–On Approaches. Research should be undertaken on pass–on approaches to determine (a) how the process can be accelerated and (b) how the approach can be delivered in ways that mitigate contributions to the hand–out syndrome 49. Animal Husbandry Training. The program needs to enhance the animal husbandry training to ensure that participants are fully capable of caring for confined animals. IPM 50. IPM Performance Indicator. It is too late for WALA to implement a full IPM strategy as suggested by the performance indicator in the IPTT (understanding pest environment, mechanical control, biological control, crop rotation and patterns, appropriate chemical control), so the program should discuss with FFP the need to continue with this indicator. Seed Supply Systems 51. PRIORITY Grain Banks. This intervention represents an opportunity to strengthen a community–based safety net that could be sustained beyond the life of WALA and the program should consider incorporating this in the strategy under SO3 for addressing the food insecurity of OVCs, PLHIV and other chronically ill, currently benefitting from food distributions. 52. PRIORITY Seed Selection and Storage. Apparently the program has undertaken some very basic seed selection and storage training for participants planning to carry over seed. This training activity should be expanded and accelerated, since it is very appropriate for the most vulnerable households with limited cash resources. 53. Community–Based Seed Suppliers. The concept original proposed for facilitating marketing groups to become certified seed suppliers should be revived and piloted in the remaining life of WALA. VSL 54. PRIORITY PSP Roll–Out: The roll–out of the PSP model should be accelerated...some partners and CAs are ready to go...so that there is time left in the life of the program to address any constraints that may arise. 55. Reporting Diffusion: Diffusion of the VSL approach is an important impact and frontline life should be encouraged to identify and report when this is happening. 56. VSL Group Investments: Group–managed businesses have a higher rate of failure so promotion of group–managed IGAs should be treated with caution. Nevertheless, existing VSL group–managed IGAs should be monitored (none as yet have produced benefits) and if they prove successful, VSL groups should be exposed to the concept Agri–Business 57. PRIORITY Marketing Fairs: Find a long–term managing agency for this activity and work toward establishing this as a sustained service. 58. PRIORITY ACAs as PSPs: Develop and implement a strategy for sustaining the services of ACAs after the program ends through commissions for linking producers and buyers or WALA Mid-Term Evaluation Report Annex B 64 provision of other paid marketing services. Given the short time remaining in the program to put in place a system , test it and provide remedial support to address any problems that may emerge, 59. Farming as a Business: Participating farmers are learning about marketing both conceptually through the Farming–as–a–Business training as well as through acquring experience in business relationships faciltated by the marketing fairs and the linkages being developed by the program. Learning–by–doing is generally more effective at acquiring practical skills, and the program should scale up this element of its agri–business capacity building strategy, i.e., expand existing mechanisms such as marketing fairs and out–grower relations and look for other ways to facilitate business relationships. 60. TQC Agri–Business: Given the recommendations above as well as continuing to build marketing capacities among targeted impact groups which cannot be completed within the next six months when the current TQC position is supposed to phase out, the program should continue to look for a means to retain a TQC capacity in CATCH through the end of the program. SO3 - DRR DRR Capacities 61. PRIORITY DRR Training. Expedite training of outstanding VCPCs, ACPCs, and DCPCs in DRR to make them well prepared to respond to disasters by the end of the program implementation year 3. 62. Early Warning on Food Prices. Emphasize early warning in DRR training of GVHs on how to monitor prices of staple food, so that households may have more time to strengthen their coping capacities. 63. Women's Needs in Emergency Response. Include a special module in the DRR training on safety and rescue of women during an emergency response. 64. PRIORITY After Action Reviews. Conduct “After Action” reviews including assessment, reporting, proposal writing and lessons learned from disaster response by WALA and use the information to revise DRR training modules. 65. PRIORITY Budget for Emergency Response. Make a provision in the annual budget for an emergency fund to enable a timely emergency response. This budget can be reimbursed if external donor resources are obtained, as happened with the recent floods in the Lower Shire Valley. 66. Emergency Response Protocol. Define a decision-making protocol for Civil Protection Committees at different levels to facilitate more rapid processing and approval of emergency assistance. 67. Scope of Emergency Response. Expand the WALA catchment area for disaster response to respond quickly and appropriately in a wider area, without adding significant costs to the program budget. DRR Linkages 68. PRIORITY Farmers Voice Radio. The program should negotiate with Farmers Voice Radio to also include disaster early warning messages along with the agribusiness information currently being discussed with them. WALA Mid-Term Evaluation Report Annex B 65 69. Output Targets. The program should define output targets for the higher level linkages required to make enhance disaster risk management capacities at the district level and below. Safety Net Distributions 70. PRIORITY Integration of Safety Net Beneficiaries. The program should ensure that safety net beneficiaries have opportunities to be included in SO2 activities as appropriate as well as in SO3 activities to enhance their coping strategies to be able to protect the new assets they are able to accrue. FFW/FFA 71. FFW/FFA Distinction. There seems to be no real benefit in trying to distinguish between FFW and FFA, so find a term that people are comfortable with and use a single term. 72. PRIORITY Selection Criteria. The program should review the targeting of FFW beneficiaries to ensure that the food is going to chronically food insecure households with some labor capacity. The value of the ration should be analyzed and compared to the local wages paid for ganyu so that the ration itself is an effective targeting tool. 73. PRIORITY Assets Built with FFW: The program should give more attention to strategic decisions around assets to be created by FFW/FFA to ensure that the program capitalizes on the best opportunities to use this food resource effectively. 74. Person-Days. Person-days of employment created by FFW/FFA should be included as an outcome indicator in the ITT. Cross-Cutting Themes Governance 75. PRIORITY Governance Focus. In the remaining life of WALA, the program should focus on the following governance principles:  Transparency – Leaders informing constituents on the rationale and processes used for making decisions......for example, ensuring that the selection process and criteria for receiving inputs or food are widely known.  Acknowledgment – Provide recognition and provision of non-monetary incentives and rewards for good community work and leadership.  Empowerment and Accountability - Implement some form of social audit as part of PPM&E process so that leadership are held accountable in identification, preparation and implementation of development projects in their communities. 76. PRIORITY Governance TQC. Since governance is a cross cutting theme that must be incorporated across all components of the program, the Governance TQC should be relocated to the CATCH office to facilitate more continuous interaction with other TQCs and implementing partners. Gender WALA Mid-Term Evaluation Report Annex B 66 77. Gender Strategy. Program management and TQCs should review the gender strategy and update the operational plans in the strategy for the remaining life of WALA. 78. Gender Mainstreaming. Technical Working Groups in all components should spend more time reflecting on gender in relation to the on-going and planned activities to identify where there is positive impact and where adaptation is needed to mitigate negative impact. 79. Gender Success Stories. Identify and document the success stories on gender to include in the next annual report. Environment 80. Emerging Environmental Impact. The program should expand the monthly and quarterly reporting systems to include sections that allow (and encourage) implementing staff to report on their observations of emerging environmental impact. 81. PRIORITY Environmental Mitigation Plans (EMPs). Program should develop and implement EMPs for roads rehabilitated under FFW. 82. PRIORITY Smoke from Fuel Efficient Stoves. The program should identify modifications that can be made to the fuel efficient stoves being promoted to reduce the risk from smoke inhalation. Program Systems Targeting 83. PRIORITY Targeting Tension between SOs and Goal. The program should develop context specific criteria for determining who qualifies as "chronically food insecure" and apply these criteria to current and new participants to be able to clearly document beneficiaries from the intended impact groups. 84. PRIORITY Selection of New GVHs. In selecting the remaining fifty-eight GVHs for the program, consider as much as possible within the fuel shortages constraint, those that are more remote where program impact is likely to be greater. M&E 85. C-MIS/Indicator Tracking Tool. The program already has a time schedule for rolling out the CMIS/ITT system. One last review of the ITT list should be made to ensure that all outputs are covered. 86. Participatory Monitoring & Evaluation (PME). The program should use expertise from within the consortium (or outside if necessary) to build capacities for using PME effectively across all SOs. 87. PRIORITY Final Evaluation Preparation. In preparation for the Final Evaluation, the program should (1) review the program document to identify proposed impact (e.g., improved access to non-timber forest products) that is not being achieved because there are no outputs being implemented to achieve these outcomes and begin discussions with USAID to ensure that the program is not held accountable against these, and (2) develop and implement a strategy for obtaining information to fill in the gaps on important outcomes and impact that the program is achieving. WALA Mid-Term Evaluation Report Annex B 67 Integration and Complementarity 88. PRIORITY SFP Beneficiaries. As with safety net food recipients, the program should make a determined effort to engage those families receiving SFP whose children have repeated malnutrition in other WALA activities to increase their food production or income. 89. PRIORITY Strategic Integration Across SOs. The program should generally plan more strategically to facilitate integration across components, for example:  SO2 should promote production of groundnuts to support the promotion of ground nuts for child nutrition in SO1  Similarly, SO2 should give more attention to poultry, oilseeds, and fruit trees to support nutrition messages under SO1.  Before share out, VSL members should be asked to reflect on how they might use the funding in relation to what they are learning in SO1, for example, to use some of the cash to purchase soap to improve family hygiene.  Community-based disaster management plans developed under SO3 should incorporate the watershed development plans developed under SO2. 90. PRIORITY Other Complementary Programs. An inventory should be made of other programs and projects in each district with interventions similar to those of WALA, and WALA field staff should be encouraged to seek out field staff of these other programs to informally share lessons learned, technologies, and assess potential for collaboration. Financial Management 91. PRIORITY Delegation of Authority. All partners should review their delegations of authority for financial transactions to look for opportunities to make financial transactions more efficient without threatening high levels of accountability. Commodity Management 92. PRIORITY CATCH Commodity Manager. The expatriate commodity manager position in WALA is due to be phased out at the end of this fiscal year. Because of recruitment challenges, however, the program is not prepared to turn over responsibilities to a local staff member, therefore, the current commodity manager should be retained until a competent successor has been fully trained. 93. Warehouse Ventilation. The program should ensure that all warehouses have enough ventilation for storage of commodity for a long period (more than three months). Human Resources 94. PRIORITY Expanding the HR Pool. The program should strategize on ways to expand the pool of candidates for positions in WALA, for example, through internship programs with local universities or finding ways to attract the Malawian diaspora back to Malawi. WALA Mid-Term Evaluation Report ANNEX C 68 Key Dates in the Life of the WALA Program Date Event 8 May 2009 Effective Start Date 8 May 2009 First Obligation of 202e & ITSH 3 July 2009 Sub-Award Agreement Signed with Africare 8 Aug 2009 Sub-Award Agreement Signed with Emmanuel International 3 Sep 2009 Sub-Award Agreement Signed with Project Concern International 21 Aug 2009 Sub-Award Agreement Signed with Save the Children 9 July 2009 Sub-Award Agreement Signed with the Chikwawa Diocese 15 Feb 2010 Sub-Award Agreement Signed with Total Land Care 9 Oct 2009 Sub-Award Agreement Signed with World Vision July 2010 Sub-Award Agreement Signed with ACDI-VOCA October 2009 First Monetization Sale February 2010 First Arrival of Commodities for Distribution October & November 2009 Baseline Survey June 2010 IMPACT Program begins implementation October 2010 to January 2011 Transition in WALA Chief of Party Position August 2011 to January 2012 Transition in WALA Deputy Chief of Party Program Position July to September 2010 Transition in M&E Coordinator Position April & May 2011 Significant Staff Turnover in CATCH April 2011 Fuel Shortages begin to intensify October through December 2011 Fuel shortages peak, severely curtailing some program activities July to September 2011 Civil Disturbances Disrupt Implementation September & October 2011 Annual Survey January and February 2012 Mid–Term Evaluation 30 June 2014 Official End Date WALA Mid-Term Evaluation Report ANNEX D 69 Mid-Term Evaluation of WALA Program 2009-2014, Malawi (USAID Title II MYAP) Scope of Work and Terms of Reference December 2011 WALA Mid-Term Evaluation Report ANNEX D 70 Contents 1. Introduction ............................................................................................................................... 70 1.1 Objective of the mid-term evaluation................................................................................. 70 1.2 Description of the WALA program ................................................................................... 70 1.3 Key WALA interventions .................................................................................................. 71 1.4 Geographical coverage of the program .............................................................................. 72 1.5 Key partners and how activities are coordinated ............................................................... 73 1.6 Implementation history and issues to date ......................................................................... 73 2. Composition of Evaluation team ............................................................................................... 73 3. Team and individual team member SOWs................................................................................ 74 3.1 SOW for the overall team and the team leader .................................................................. 75 3.2 SOW for SO1: Health and Nutrition team member........................................................... 76 3.3 SOW for SO2: Agriculture, Irrigation, NRM and Agribusiness team member................. 77 3.4 SOW for SO3: Commodities, DRR and Good Governance .............................................. 78 4. Description of team members’ activities and deliverables........................................................ 78 5. Ownership of Research Data/Findings...................................................................................... 80 6. Timeframe ................................................................................................................................. 80 Annex-1: Suggested outline for the Mid-term evaluation report...................................................... 82 Annex- 2: IPTT (in separate file)...................................................................................................... 82 WALA Mid-Term Evaluation Report ANNEX D 71 1. Introduction 1.1 Objective of the mid-term evaluation To assess the progress of the WALA program implementation in the first two and half years in achieving its three strategic objectives and guide the program team in making necessary course corrections in achieving its set LoAs/goals. The sub objectives are: 1. To review the progress towards achievement of stated WALA program objectives and intermediate results. 2. To carry out a comparative analysis of baseline and 2011 annual survey results, and suggest revisions in performance indicators if necessary. 3. To identify program strategies and interventions that contributed to or impeded the achievement of intended impact of program interventions and establish plausible links between inputs and impacts at mid-term. 4. To assess the effectiveness and efficiency of technical, managerial and resource management strategies, structures and systems established to support program implementation at the consortium and PVO levels in terms of their impact on program results. 5. To assess the effectiveness of WALA program strategies and interventions in contributing to the USAID/Malawi Food Security Strategy. 6. Make specific recommendations on how WALA can improve its strategies and program interventions to enhance its performance with respect to the above mentioned objectives. 7. To assess the synergy between various WALA program components including linkages with Government of Malawi and other development programs, and its effectiveness in enhancing the program performance. 1.2 Description of the WALA program Goal of the program, including Strategic Objectives and Intermediate Results The overall goal, strategic objectives (SOs) and intermediate results (IRs) are stated below: Goal: Reduced food insecurity of 214,974 chronically food insecure households in 39 Traditional Authorities within five livelihood zones in Southern Malawi by 2014. Maternal, Child Health and Nutrition SO1: 170,724 vulnerable households have improved maternal and child health, and nutrition status. Intermediate Results (IR): IR 1.1: 170,724 vulnerable households have improved maternal and child health, and nutrition practices. IR 1.2: 170,724 vulnerable households have increased use of quality maternal and child health, and nutrition services. Agriculture, NRM, Irrigation and Economic Activity SO2: 147,500 smallholder farming households have improved livelihood status WALA Mid-Term Evaluation Report ANNEX D 72 Intermediate Results (IR): IR 2.1: 147,500 smallholder farming households have improved crop production practices. IR 2.2: 103,400 smallholder farming households have increased use of financial services. IR 2.3: 20,600 smallholder farming HHs have engaged in commercial marketing. Disaster Risk Reduction SO 3: 273 targeted communities have improved capacity to withstand shocks and stresses Intermediate Results (IR): IR 3.1: 273 communities have strengthened mechanisms for disaster preparedness, response and mitigation IR 3.2: 21,203 most food insecure households have accessed transitional food rations Crosscutting IR: HIV Mitigation, Gender Equality, Environmental Protection and Good Governance The Indicator Performance Tracking Table (IPTT) listing the indicators, baseline, annual targets and life of activity targets and achievements are given in Annex-1. 1.3 Key WALA interventions WALA is a five year (2009-2014) $80 million Title-II development assistance program, funded by USAID. The program commenced on 1 July 2009. This is the continuation of the five-year I-LIFE Program (2004-2009), though the WALA area of operation is different with the exception of one district Thyolo in which both I-Life and WALA are operating. SO1:  Care Group Model is a community based health service provision strategy employed to increase the coverage and quality of health/nutrition services. All health and nutrition interventions below are implemented through the model.  Community Complementary Feeding and Learning Sessions (CCFLS) approach is applied to enhance the nutritional skills of mothers of children under five and pregnant and lactating women.  Strengthening of the ministry of health through capacity building, provision of resources, and collaboration in undertaking key activities.  Strengthening the capacity of community based organizations (CBOs) to undertake and sustain development activities, such as village health committees etc. SO2:  Demonstration sites approach to enhance agricultural production and promotes a number of appropriate improved farming practices e.g. crop diversification.  Small-scale irrigation, focusing on high quality and nutritious crops has been scaled up and integrated with other WALA components. Stream diversions for gravity systems and shallow wells for treadle pump systems are commonly utilized.  Village savings and loan (VS&L) has been employed to increase household incomes and facilitate linkages with micro-enterprises – in order to boost economic development. WALA Mid-Term Evaluation Report ANNEX D 73  Farming as a business has been promoted by strengthening linkages between small-scale farmers and the private sector. The collective marketing of produce by farmers has increasing their bargaining power and increased marketing options.  Strengthening the capacity of community based organizations (CBOs) to undertake and sustain development activities, such as formation of water users committees, marketing clubs, etc. SO3:  Food safety net: Provision of food aid to chronically ill beneficiaries, who are targeted for other WALA interventions.  Empowerment of communities on Disaster Risk Reduction and mitigation.  Good Governance elements such as the participatory planning monitoring and evaluation (PPME) exercises, conflict management. Cross Cutting Theme  Expanded knowledge management.  Mainstreaming of HIV/AIDS into all WALA key activities.  Gender mainstreaming in all key WALA activities.  Environment protection. 1.4 Geographical and beneficiary targeting Malawi is highly susceptible to food insecurity, and the Southern Region is the most vulnerable. Underlying causes include frequent natural disasters, pervasive poverty, volatile prices and the HIV pandemic, all of which diminish rural communities’ capacity to adequately reduce their risks. The WALA program is operating in eight districts of Malawi (Balaka, Machinga, Zomba, Chiradzulu, Mulanje, Thyolo, Chikwawa and Nsanje) that were identified as among the most vulnerable and food insecure in the country. WALA targets the most vulnerable communities and households, ensuring holistic provision of services to the selected groups. For the most part, targeted groups are comprised of households that: have small and marginal farms, female-headed, hosting chronically ill persons (TB and HIV/AIDS), food insecure, and hosting orphans. 1.5 Management of WALA program The program is led by Catholic Relief Services (CRS). To effectively manage overall program coordination, CRS have established an independently housed Consortium Administration and Technical Capacity Hub (CATCH), which is based in Blantyre. Along with CRS, an additional seven partner private voluntary organisations (PVOs) Africare, Chikwawa Diocese, Emmanuel International, Project Concern Internationl, Save the Children, Total Land Care and World Vision. Apart from the above mentioned seven PVOs, ACDI/VOCA provides technical support in the Agribusiness component. The PVOs are working with the CATCH to implement activities in each of the eight target districts. Four of the above mentioned consortium members were also partner organizations worked in the I￾Life program, though in different districts except for World Vision (Thyolo district). The program aims to promote learning within consortium members and other similar organizations, as well as foster good programming practices through: The establishment and coordination of technical working groups (TWGs). The TWGs, through the leadership of Technical Leads ensure technical soundness of strategies employed by WALA to achieve WALA Mid-Term Evaluation Report ANNEX D 74 expected results. The identification, documentation and dissemination of lessons learnt and best practices will contribute to the replication and scale-up of best practices identified. 1.6 WALA monitoring and evaluation plan WALA monitors the program progress towards achieving its annual indicator targets through PVO quarterly report annual compilation, annual surveys, and crop data collection of rainfed and winter cropping from secondary sources such as National Statistical Office, Ministry of Agriculture, etc. The Indicator Performance Tracking Table (IPTT) of WALA to FFP/Washington is attached (Annex-2). The first annual survey is scheduled during July￾August 2011, the annual survey design will be similar to that of baseline, except for the coverage limiting to beneficiary households. Since the annual survey and mid-term evaluation are close to each other (gap of less than 6 months), we are not proposing to do another quantitative survey for the mid-term. 1.7 Implementation history and progress to date WALA Program started in June 2009 with four of the seven implementing partners commencing key activities in the first quarter itself. Other three partners were slow in their start-up; however they have picked-up momentum from second quarter. The key challenges faced by the consortium members were the availability of transport for the field staff in the initial one and half years of its implementation coupled with some delays in the quarterly payments to the PVOs, e.g. Agribusiness trainings and household listing trainings were delayed in some PVOs. Another challenge faced was the drought in 2009-10, when most of the districts experienced varying degrees of crop loss resulting in food shortage. The extent of this problem was addressed by providing drought resistant crops such as sweet potato vines and cassava cuttings to small holder farmers in the affected areas. Further, to reduce the impact of shocks, additional food was distributed to households who were engaged in the Food for Asset (FFA) activities. Some of the PVOs also faced challenges with setting up of offices, slow pace of staff recruitments and turn over. However, the situation has improved notably during the second half of year two of implementation. It is also worth noting that due to the moving out of Salvation Army operations from Malawi, the Total Land Care (TLC) was selected as one of the PVOs. The process of identifying the new PVO was done in consultation with USAID/FFP. 2. Composition of Evaluation team The evaluation will be carried out by an external evaluation team. One team leader having expertise in agriculture and livelihood is supported by a health and nutrition expert and a person with experience in good governance, disaster risk reduction (DRR) and commodity management. The Head of M&E and Knowledge Management (KM) of WALA program will coordinate the evaluation activities, with technical assistance from CRS regional and HQ M&E units. The team leader will oversee the mid-term evaluation and provide leadership to health and nutrition, and good governance, disaster risk reduction (DRR) and commodity management expert. S/he will have additional responsibilities of working with WALA Head of M&E and KM on logistics, ensuring that the other team member fulfills her/his obligations, organizing WALA Mid-Term Evaluation Report ANNEX D 75 and directing team interactions, planning briefing meetings with CATCH Leadership team and other staff stakeholders, etc. S/he will also consolidate the final report and ensure its quality and timeliness. The team leader and the other two team members, respectively, should have the following qualifications and experience: Team Leader-cum- agriculture and livelihood expert  In-depth knowledge of USAID evaluation requirements preferably for Title II programs  Previous experience in leading a multi-sectoral Title II programs addressing food security  Strong program evaluation experience in NGO sector, consortium program experience or exposure  Strong management and administrative skills with experience of evaluations looking at a large number of cross-cutting issues (e.g., questions concerning overall program administration or how to better integrate the technical components of programs)  Master’s degree in development studies or other relevant field of study such as agriculture and livelihood management  Demonstrated knowledge of regional development and more than 10 years of work experience in Southern Africa, preferably in Malawi  More than ten years’ working experience in the development field  Strong analytical skills  Proven leadership skills  Sound training and facilitation skills  Excellent writing and document presentation skills  Demonstrated ability to meet deadlines Team Member (Health and Nutrition)  Experience of Title II MCHN programs  Strong program evaluation experience in NGO sector, consortium program experience or exposure  Master’s degree in public health or nutrition  Demonstrated knowledge of regional development and more than 10 years of work experience in Southern Africa, preferably in Malawi  Good analytical skills  Good writing and document presentation skills  Demonstrated ability to meet deadlines Team Member (Commodities, Governance and DRR expert)  Experience of Title II programs addressing food security  Strong program evaluation experience in NGO sector, consortium program experience or exposure  Master’s degree in development studies or other relevant field of study  Strong DRR, commodity management and governance experience  Demonstrated knowledge of regional development and more than 10 years of work experience in Southern Africa, preferably in Malawi  Good analytical skills  Good writing and document presentation skills  Demonstrated ability to meet deadlines WALA Mid-Term Evaluation Report ANNEX D 76 3. Team and individual team member SOWs Data from the FY’10 annual survey (due in July-August) will provide information on IPTT indicators to the evaluation team. This data will be complemented by information from the CATCH-MIS, and PVO quarterly reports on process and output indicators. In addition, to answer evaluation questions, the team shall conduct qualitative exercises/studies such as meetings/discussion, FGD’s, in-depth interviews, record/document reviews, etc. The final methodology and design shall be fine-tuned by the external evaluation team, after they are hired and formed. 3.1 SOW for the overall team and the team leader The MTE team is expected to answer the following cross cutting questions in addition to the specific questions in each sectoral areas listed in the following sections, but shall be discussed and refined during a stakeholder workshop up on their arrival in Malawi. Design, Implementation and achievements:  To what extent has the expected change in the indicators (see Annex 1) been achieved compared to the baseline levels at the end of year 2 (mid-term)?  To what extent have internal factors (interventions, structures and systems) positively and negatively influenced achievement of program?  To what extent have external factors affected the achievement of program?  Are there any negative impacts or unintended consequences of the program that need to be addressed, so that the WALA’s LoA can be achieved?  How effective is the program in reaching women and addressing gender issues?  To what extent have the WALA consortium (CATCH TQC’s, Technical Working Group & monthly meetings, etc) and PVO structures/approaches affected the implementation of the program?  Are the sectoral strategies appropriate to address the food security problems identified in the selected target areas?  As defined and measured, do the performance indicators provide useful and reliable data on program progress and impacts?  To what extent is the monitoring system operational? How manageable, valid, reliable, and useful is data generated by the system?  How well integrated is the WALA program i.e. extent of beneficiaries benefiting from services from each of the strategic objectives?  What is the extent of replicating WALA’s best practices/lessons learnt? Are there any examples that worked well?  What program activities are sustainable by communities without WALA program support and why? What program activities do not appear to be sustainable and why? Are there specific activities that can be phased over to government support? Behavior Change:  Are beneficiaries adopting desired practices or behaviors promoted by the program?  Are the program interventions addresses the barriers to the desired behaviours?  What is their primary source of information concerning practices and behaviors? What are other key channels of information?  Which practices have beneficiaries been more inclined to adopt, and why?  What strategies have been effective in facilitating behavior change? How can the program be modified to address constraints to behavior change? WALA Mid-Term Evaluation Report ANNEX D 77 Capacity Strengthening:  Are the training materials appropriate for the participants? Are the training materials state-of-the art? If necessary, how can the materials are improved to better meet the objectives of the training?  Which training strategies have been effective in imparting the desired knowledge and skills?  Is WALA’s capacity building efforts working well? If yes, explain its potential. If no, explain the reasons.  Is the technical field staff well trained and supervised? What areas, if any, need strengthening?  Is the program effectively developing the capacity of PVOs? If not, how could the design or implementation be altered to improve capacity strengthening?  Is the program effectively enabling, or developing the capacity of, beneficiaries? If not, how could the design or implementation be altered to improve capacity strengthening? Community/local institutions strengthening:  Has the capacity of community and local institutions to protect and enhance food security is improved?  What factors influenced improving or not improving their capacity?  What processes have been undertaken to improve the capacity of community and local institutions?  What is the scale and level of community participation and community support to the WALA promoted activities in the program villages?  How transparent is the management (democratic representation, by-laws etc.) of community organizations and its roles and responsibilities?  Do the community organizations and institutions have proven capacity to analyze, plan, implement, monitor and evaluate to address the community needs?  What actions do the community organizations and institutions take to protect and enhance the food security in their communities?  How can the community organizations and institutions be further developed? Sustainability and exit strategy:  Are the program results sustainable?  Are the outcomes related to adoption of better practices sustainable, i.e., participants are likely to continue with the lessons learned? Which outcomes are likely or unlikely to be sustainable, and why? What can be done to increase the sustainability?  What is the extent of WALA’s collaboration with the relevant government ministries? Are beneficiaries able to receive follow-up technical support from their respective government Extension Workers?  Are the sustainability strategy and their interventions in the right direction?  Are the community organizations and institutions and their activities sustainable after the project exit? 3.2 SOW for SO1: Health and Nutrition team member Potential Questions to Answer  How successful has the program been in protecting the nutrition/health status of specific groups (such as young children, mothers, pregnant women, the chronically) WALA Mid-Term Evaluation Report ANNEX D 78  Which strategies have been more effective in SO1 so far and why?  Does the intervention reflect the MCHN problems facing the community?  Are there clear linkages and coordination with public and private health and social services in the community?  How successful has the activity been able to leverage government preventative and curative health, water and sanitation and related social services?  To what extent do coordination committees ensure people’s involvement in the program with due consideration of gender, age, ethnicity, and socio-economic status?  Are there clearly written and applied guidelines for targeted feeding including appropriate breast and young child feeding?  Are there appropriate and applied protocols for referral and treatment of severely malnourished children?  Are staff qualified and aware of the purpose and methods used in the program?  What is being done to improve the capabilities of the staff and local partners to respond to community needs and meet the objectives of the program?  Are staff responsible for nutrition and health assessments and care trained and supervised in the necessary techniques for children and mothers?  Are health and nutrition BCC materials appropriate – tailored to the user, actionable, accurate and linked to growth promotion messages? Which materials need strengthening, if any, and how?  Is there a care group volunteer attrition problem? If so, why do they drop out and what can be done to reduce the rate of attrition? 3.3 SOW for SO2: Agriculture and Livelihood expert (Team leader) Potential Questions to Answer Agriculture, Irrigation and NRM:  What is the extent of adoption of the promoted agricultural techniques by farmers?  Are the technologies and practices being promoted established and suitable to the local agro-ecological environments?  Did the use of food for work for participation in agricultural production related activities act as an incentive/disincentive to improving productivity, and how?  Are farmers able to obtain improved and recommended inputs without program assistance (free or subsidized inputs)? If not, what would be required in order that they could do so?  Are farmers and other community groups able to maintain new productive infrastructure on their own? If not, why and what could be done to address this limitation?  Do demonstration plots accurately reflect the real conditions facing farmers?  Are WALA activities linked to the Ministry of Agriculture or the national or international agricultural research centers and has a communication protocol been established? Are the linkages sustainable? If not, why and suggest solutions.  Is the increased agricultural production sustainable?  Has the small scale irrigation interventions enhanced agricultural productivity and food availability in the targeted communities?  Do the small scale irrigation structures pose any negative environment effect in the community and in the surroundings?  Have the communities supported by small scale irrigation interventions established systems to raise and manage funds for operation and maintenance of the irrigation schemes?  Do the irrigation scheme farmers have the knowledge of irrigation crop sequencing and water management? WALA Mid-Term Evaluation Report ANNEX D 79  Is there a market for farmer produce?  Do the established farmers groups have the capacity to sustain the established relationships with the private sector?  What are the greatest access opportunities – increasing the volume of sales, improving the quality of produce, building business expertise, etc.? Does the program take adequate advantage of these opportunities? Village Savings & Loans and microenterprises:  Are credit programs (VS&L – Village Savings and Loans) designed and implemented according to standard best practices? If not, why and how can credit practices be improved?  What is the repayment rate for loans and what are the design elements contributing to this repayment rate? If the repayment rate is low, what can be done to improve it?  Are there certain groups/individuals within the target population better able to access loans, and why? If there are groups/individuals who are unable to access loans, should the program be broadened to include these groups and how can the program be modified to incorporate them?  Is there an accessible market for the products or services produced by the microenterprises promoted by WALA program?  Are these microenterprises likely to sustain its operations without WALA support? If not, why and what can be done to enhance their sustainability?  Have microenterprise/microfinance activities contributed directly to household food security, and how?  Are there ways to enhance the effect of income-generating activities on household food security? 3.4 SOW for SO3: Commodities, DRR and Good Governance Potential Questions to Answer Commodities:  Whether the commodity movement was managed according to the FFP guidelines at all levels?  Did the use of food for work for participation in agricultural production related activities act as an incentive/disincentive to improving productivity, and how? Infrastructure (Community assets through Food for Work and Food for Asset):  Has the rehabilitation of rural roads had an effect on seasonal road use and transportation time and costs for the targeted beneficiaries? Has it had an effect on seasonal availability of food and other important commodities?  Are there additional direct or indirect benefits derived from infrastructure construction or rehabilitation that are not currently being captured?  Are there any unintended negative environmental impacts stemming from infrastructure activities? If so, are there sustainable mitigation measures being implemented? What additional measures can be implemented?  What is the impact of the rural infrastructure/roads on the marketing of agricultural produce? DRR and Good Governance:  Whether the DRR activities carried out under the WALA Program are sustainable? If yes, what are the key sustainable elements we could high light. If not, what are the areas need improvement. WALA Mid-Term Evaluation Report ANNEX D 80 4. Description of team members’ activities and deliverables 4.1 Activities In collaboration with CATCH Head of M&E and KM and the mid-term evaluation team, the experts will undertake the following activities:  Conduct review of data sources and relevant literature according to list provided  E-mail/telephone communications with CATCH Head of M&E and KM for detailed briefing  Finalize appropriate mid-term evaluation methodology and design  Finalize mid-term evaluation plan  Review and input on annual survey data analysis results tables  Field work; meet with key stakeholders including beneficiaries, PVO representatives as well as WALA-CATCH staff.  De-brief the CATCH and mid-term evaluation team  Synthesize findings  Conduct stakeholder meetings to present and discuss preliminary findings  Produce draft report including meeting feedback (a sample basic outline of the report is presented in Annex-2)  Incorporate comments from WALA in draft report and produce final report. 4.2 Deliverables The consultants will be responsible for the following deliverables: Output Due Date Review 2011 annual survey (quantitative) data analysis results tables and share inputs to CATCH Head of M&E and KM on any additional analysis to answer mid-term evaluation questions 16 Jan 2012 Draft mid-term evaluation plan/design 13 Jan 2012 Internal stakeholder meetings and information collection (in Malawi) 12 Jan 2012 Information analysis and PowerPoint presentation of key findings and recommendations to stakeholders (in Malawi) 16 Feb 2012 Submit PowerPoint presentation of key findings and recommendations (in Malawi) (soft copies) 17 Feb 2012 Submit draft mid-term evaluation report to CATCH (soft copies) 17 Feb 2012 Submission of final mid-term evaluation report (soft copies) incorporating comments on draft coordinated by CATCH Head of M&E and KM 15 Mar 2012 4.3 Team members’ other responsibilities The members shall be responsible for the following:  Adhering to all terms/conditions stipulated in their contracts  Validity of their passports and other relevant travel documentation within Malawi  Obtaining their health insurance  Conducting themselves in a respectful manner, while undertaking assignment, which includes not making any commitments to communities and any other persons, on behalf of WALA  Adhering to the agreed time-frames with regard to all activities outlined in the timeline  Consultant’s own laptop to be used during the assignment 4.4 WALA CATCH Responsibilities WALA Mid-Term Evaluation Report ANNEX D 81 The CATCH will be responsible for the following:  Ensure effective coordination of the mid-term evaluation logistics to facilitate the consultants in undertaking their assignment including their travel, stay, per-diem and visa fee  Provide consultants with literature review materials/necessary documentations  Link consultants to partner PVOs and other relevant stakeholders  Carry out additional data analysis of FY10 annual survey data suggested by external experts before their arrival in Malawi  Buy and deliver air tickets for team members’ travel to Malawi  Communicate with the team members that their travel, hotel, per-diem, and visa fee is born by CRS/Malawi as per its policies 5. Ownership of Research Data/Findings All data collected for this study shall remain the property of the WALA-CATCH. Any work product resulting from this study must cite the participating PVOs and USAID as well as include WALA staff as a primary or contributing author. WALA Mid-Term Evaluation Report ANNEX D 82 6. Timeframe The consultancy will commence on 23 December 2011 and proceed until 15 March, 2012 as per the timeline outlined below: Date Day Activity Paid Work Days Team Questions/Comments Leader MCHN SO2 DRR SO3 By 23 December Friday Preliminary literature review and Draft Evaluation Plan sent to CRS Malawi 2 2 2 CRS Malawi needs to send IPTT, annual and quarterly implementation reports (including commodity reports), program manuals, and other relevant background documents to evaluation team By January 12 Thursday Review of Annual Survey Data and Results tables with input sent to Malawi for additional analysis 1 1 1 CRS Malawi needs to send survey data and results to evaluation team 16 January Monday Mike, Judiann and Golam arrive in￾country 2 TD 2TD 2TD Arriving in Blantyre by January 16 17 January Tuesday MORNING: Evaluation Team meeting to fine-tune evaluation methodology and agree on responsibilities. 1 1 1 CATCH is welcome to participate in this meeting AFTERNOON: Programme Orientation - CATCH provides an overview of the programme for the evaluation team 18-19 January Wednesday - Thursday Evaluation Plan revision (based on meetings previous day), field visit schedule finalized, tools development, and review of existing quantitative data sets 2 2 2 Actual analysis of existing quantitative data sets from impact surveys and monitoring system will not be done at this time, but plans will be developed, including specification of information needed, for doing the analysis. A short meeting will be scheduled with CATCH and USAID/FFP to present an overview of the Evaluation Plan 20 January - 2 February Leave on a Friday, Finish on a Thursday Field Work - Collection of qualitative data through interviews with impact groups, intermediaries, program implementation staff and partner leadership staff 14 14 14 Assume that on average will spend two days with each of seven partners. However, this may vary if some partners are covering a wider area or otherwise have a larger role in the program. 3-5 February Friday - Sunday Information Processing and Preparation for the Ground Truthing Workshop 3 3 3 Evaluation team members will work individually to prepare for GT Workshop. WALA Mid-Term Evaluation Report ANNEX D 83 Date Day Activity Paid Work Days Team Questions/Comments Leader MCHN SO2 DRR SO3 6 - 7 February Monday - Tuesday Ground-Truthing Workshop in which the evaluation team presents to implementing staff their draft observations and recommendations for discussion. 2 2 2 Participants in this workshop should be restricted to the evaluation team and program implementation staff. It should not include anyone who is not intimately familiar with at least some part of the program. 8 -9 February Wednesday -Thursday Post-GT Workshop processing and debriefing preparations. 2 2 2 The GT Workshop will produce clarifications on observations, practical revisions to preliminary recommendations and identification of gray areas that need additional investigation. 10 February Friday MORNING: Stakeholder Debriefing to present an overview of the primary content that will be included in the evaluation report and that will be presented to USAID/FFP 1 1 1 This is the dress rehearsal for the presentation that will be made to USAID/FFP, usually a half day workshop. Participants include CATCH and leadership from implementing partners. 11 - 16 February Saturday - Thursday Preparation of Draft Report (in-country) 6 5 5 Judiann and Golam will be finished by 15 February and can depart at that point 14 February (Or any time 13-16 Feb) Tuesday USAID/FFP Debriefing -- -- -- Evaluation team with CRS Country Representation and Programme Manager, no more than two hours, in Blantyre. 17 February Friday AFTERNOON: Exit Debriefing and turnover of Draft Report to CRS Malawi 1 --- --- 18 February Saturday Departure of Evaluation Team 1 TD -- 1 TD This is the date the team leader departs, other team members will have departed earlier By 7 March Wednesday CRS Malawi compiles and sends comments on draft report to the Team Leader --- --- --- By 15 March Thursday Final report submitted to CRS Malawi 2 1 1 TOTAL PAID WORK AND TRAVEL (TD) DAYS 37 3 TD 34 2 TD 34 3 TD Total of 105 work days plus 8 travel days. WALA Mid-Term Evaluation Report ANNEX D 84 Annex-1: Suggested outline for the Mid-term evaluation report (Maximum 50 pages, in Times New Roman 12pt font in single space, excluding annexes) Title page with date Executive summary Introduction Objective of SOW Brief description of project Technical Sector I – SO1: MCHN Brief description of interventions Implementation progress and achievement of results Meeting targets Other achievements Discussion of generic evaluation questions Discussion of specific technical sector SOW questions Technical Sector II – SO2: Livelihood Brief description of interventions Implementation progress and achievement of results Meeting targets Other achievements Discussion of generic evaluation questions Discussion of specific technical sector SOW questions Technical Sector III – SO3: Commodities, DRR, and Good governance Brief description of interventions Implementation progress and achievement of results Meeting targets Other achievements Discussion of generic evaluation questions Discussion of specific technical sector SOW questions Cross-cutting issues Brief description of cross-cutting issues Discussion of specific cross-cutting SOW questions, including management, finance, M&E and learning Summary Recommendations (by sector and cross cutting) Annexes Evaluation SOW Composition of the team Methods List of sites visited List of key informants References Indicator performance tracking tables (IPTT) Annual survey and evaluation data collection tools List of acronyms Annex- 2: IPTT (in separate file) WALA Mid-Term Evaluation Report Annex E 85 Catholic Relief Services in Malawi Wellness and Agriculture for Life Advancement (WALA) Mid-Term Evaluation EVALUATION PLAN Catholic Relief Services has been leading a consortium composed of nine partners (including CRS) to implement a Multi-Year Assistance Program entitled Wellness and Agriculture for Life Advancement (WALA). With a final goal to reduce food insecurity for 214,974 chronically food insecure households in 39 Traditional Authorities within five livelihoods zones of Southern Malawi, the program has seven intermediate results under three strategic objectives as shown below. Strategic Objective 1 (Maternal and Child Health and Nutrition): 170,724 vulnerable households have improved maternal and child health, and nutrition status.  Intermediate Result 1.1: 170,724 vulnerable households have improved maternal and child health, and nutrition practices.  Intermediate Result 1.2: 170,724 vulnerable households have increased use of quality maternal and child health, and nutrition services. Strategic Objective 2 (Agriculture, Natural Resource Management, Irrigation and Economic Activity): 147,500 smallholder farming households have improved livelihood status.  Intermediate Result 2.1: 147,500 smallholder farming households have improved crop production practices.  Intermediate Result 2.2: 103,400 smallholder farming households have increased use of financial services.  Intermediate Result 2.3: 20,600 smallholder farming HHs have engaged in commercial marketing. Strategic Objective 3 (Disaster Risk Reduction): 273 targeted communities have improved capacities to withstand shocks and stress.  Intermediate Result 3.1: 273 communities have strengthened mechanisms for disaster preparedness, response and mitigation  Intermediate Result 3.2: 21,203 food insecure households have accessed transitional food rations  Intermediate Result 3.3: 8,002 community-led groups have practices good governance principles. The program is midway through year three of its planned five-year life, scheduled to end in June 2014, and CRS has commissioned a Mid-Term Evaluation to review progress and impact toward formulating recommendations for the remaining life of the program. This document describes the plan for implementing the evaluation in Malawi over the period January 16 through February 18. WALA Mid-Term Evaluation Report Annex E 86 Evaluation Purpose The Mid-Term Evaluation of the WALA is a formative evaluation intended to assess the progress of the program, the effects and impact of program activities, and the quality of various processes being used to implement the program to (1) identify lessons learned and (2) formulate recommendations to be implemented in the remaining life of the program. Evaluation Objectives of specific interest to WALA are: 1. To review the progress towards achievement of stated WALA program objectives and intermediate results. 2. To carry out a comparative analysis of baseline and 2011 annual survey results, and suggest revisions in performance indicators if necessary. 3. To identify program strategies and interventions that contributed to or impeded the achievement of intended impact of program interventions and establish plausible links between inputs and impacts at mid-term. 4. To assess the effectiveness and efficiency of technical, managerial and resource management strategies, structures and systems established to support program implementation at the consortium and PVO levels in terms of their impact on program results. 5. To assess the effectiveness of WALA program strategies and interventions in contributing to the USAID/Malawi Food Security Strategy. 6. Make specific recommendations on how WALA can improve its strategies and program interventions to enhance its performance with respect to the above mentioned objectives. 7. To assess the synergy between various WALA program components including linkages with Government of Malawi and other development programs, and its effectiveness in enhancing the program performance. Information to be Collected The evaluation will investigate the following features of WALA: Program History & Context – This part of the investigation will examine the history of the program, particularly how it has evolved since inception and the critical features of the operating environment that have affected, positively or negatively, program implementation and the impact that has been achieved. This information will be analyzed for lessons learned as well as for possible recommendations for responding to features of the current context and trends. Program Outputs - The outputs produced under each intermediate result will be quantified and analyzed to identify what the program has actually accomplished on the ground. The processes used to produce outputs as well as the quality of the outputs in terms of achieving impact will be investigated. Lessons learned will be extracted on what has worked well and what may not have worked well in producing program outputs. The evaluation team will analyze the information toward formulating recommendations for the remaining life of the program. WALA Mid-Term Evaluation Report Annex E 87 Program Impact – Both qualitative and quantitative information will be used to assess the impact of the program at different levels, including impact at the intermediate result level for the seven Intermediate Results, at the strategic objective level for the three Strategic Objectives and at the program goal level. At every level, the evaluation will assess the sustainability of the impact that has been observed. Lessons learned associated with program logic or other dimensions of the impact that has been identified will be extracted, and the evaluation team will analyze the information toward formulating recommendations for the remaining life of the program. The evaluation will investigate the key areas of expected impact as per the program document. These will be discussed further with CRS early in the evaluation since decisions are made with the donor over the course of implementing the program that may have shifted the focus of the program. Discussions will also be held to identify new areas of expected impact that may not have been defined in the program document. Program Processes and Cross-Cutting Themes – The processes used to implement the program and cross-cutting themes prioritized by the program will be investigated to identify how these contributed to program impact as well as to extract positive and negative lessons learned from program implementation. The processes and cross-cutting themes to be investigated include program management, partnerships, targeting, knowledge management including monitoring and evaluation, HIV/AIDS mainstreaming, gender mainstreaming, program integration and complementarity, resource management, and environmental monitoring and impact mitigation. The evaluation team will formulate recommendations for the remaining life of the program for enhancing the effectiveness or efficiency of these processes and cross-cutting themes. Methodology The Mid-Term Evaluation will use a mixture of qualitative and quantitative methods, using existing data sets, as described below for each of the different areas of investigation. Program History and Context. Included in Annex A are tables that will be completed describing the key events in the history of the program (Table 1), a summary of resources used in the program (Table 2) and a summary of reported beneficiaries (Table 3). Information for these tables, as well as for understanding the history of the program and contextual factors affecting implementation will be obtained from program monitoring and financial reports reports (ARRs and Quarterly Reports) and as well as through interviews with program implementation staff. Program Outputs and Impact. Qualitative information on program outputs and the impact of these outputs will be obtained through reviews of program monitoring reports, reviews of key secondary data, interviews/focus group discussions with program participants, and interviews with program implementation staff. A comparison of the most recent Annual Survey data to comparable information from the program's baseline will provide quantitative information to complement and clarify the qualitative information obtained. Some program components, such as the Village Savings and Loans component, also have data sets that will be mined for useful information. The following outputs under each SO will be the primary focus of the evaluation. Discussions will be held, however, with the program implementation team to ensure that these WALA Mid-Term Evaluation Report Annex E 88 cover all outputs being produced by the program. Additional outputs may be added following these discussions. Under SO1 (MCHN)  The Care Group Model  Community Complementary Feeding and Learning Sessions Approach  Capacity Building of the Ministry of Health  Capacity Building of Community-Based Organizations Under SO2 (ANR)  Farm Production  Small-Scale Irrigation  Village Savings and Loans (VSL)  Agri-Business  Capacity Building of Community-Based Organizations Under SO3 (DRR)  Food Safety Nets  Capacity Building of Communities on DRR  Good Governance In principle, the mid-term evaluation will gather information around all of the major outputs produced by the program. Given the time available, the evaluation team may not visit every type of program activity, and discussions will be held with program implementation staff upon arrival in Malawi to agree on the most important outputs to be observed. The following information gathering activities are planned to obtain information on outputs and impact under each SO. For SO 1 MCHN o Focus group discussions with Care Group Volunteers, mothers targeted by Care Group Volunteers, mothers participating in Community-led Complementary Feeding and Learning Sessions, and Village Health Committees. o Interviews with Health Promoters, Health Surveillance Assistants, mothers receiving rations for a malnourished child, and families receiving safety net rations. o Observations of Community Complementary Feeding and Learning Sessions o Analysis of quantitative data from Annual Surveys o Interviews with staff implementing SO1 activities For SO 2 ANR o Focus group discussions with Lead Farmer Producer Groups, Village Savings and Lending Groups, Marketing Cubs, Watershed Management Committees and Water User Committees (for all of these groups will ensure that significant numbers of PLHIV participants, seed subsidy recipients and women participants are included in the discussions). WALA Mid-Term Evaluation Report Annex E 89 o Interviews with Farmer Extension Volunteers (FEVs), VSL Community Agents, and Agri-Business Community Agents o Observations of a sample of demonstration plots, small-scale irrigation systems, and FFW sites. o Analysis of quantitative data from Annual Surveys and VSL records o Interviews with program Technical Quality Control staff (from Africare for Agriculture, from World Vision for Aquaculture and Animal Husbandry, from ACDI-VOCA for Agro-Enterprise, and from PCI for aquaculture, private sector linkages and agro-enterprise) o Review of reporting, MOUs and other documentation associated with program linkages to research institutes, government ministries, local authorities and private sector input suppliers and marketing agencies, including NASFAM (regrettably, there appears to be insufficient time to conduct interviews). o Interviews with staff implementing SO2 activities For SO 3 DRR o Focus group discussions with District Civil Protection Committees, Area Development Committees, Area Civil Protection Committees and Village Civil Protection Committees. o Focus group discussions with safety net food recipients, and Food for Work/Food for Assets participants o Interviews with leaders of District Executive Committees, Traditional Authorities, and Group Village Headmen. o Observations of community-level disaster preparedness plans o Observation of FFW and FFA work sites. o Analysis of quantitative data from Annual Surveys o Interviews with staff implementing SO3 activities Table 4 in Annex A will be used to document the observations of the evaluation team on quality of outputs and impact at different levels. Program Processes and Cross-Cutting Themes. Information on program processes will be obtained from program monitoring reports and key informant interviews/focus group discussions with program implementation staff, staff from CATCH, directors from each of the consortium partners and senior staff from the CRS country office. Topical outlines attached in Annex C indicate the key areas of investigation for each of the program processes and cross-cutting themes. Tables 5 through 8 in Annex A will be used to summarize information on financial and commodity resource management. Implementation Schedule The implementation schedule for the evaluation is shown in Annex B. Field work in Malawi will commence on the 16th of January 2012 and is expected to be completed by 17 February. More detail on some of the key events in the process is provided below. WALA Mid-Term Evaluation Report Annex E 90 Program Orientation Meeting, January 17 (Half Day). In this meeting, WALA implementation staff will present an orientation to the program for the evaluation team. The purpose of this meeting is to finalize the evaluation plan with clarity on the types of outputs produced by the program, the stakeholders that need to be interviewed to understand the impact of these outputs, and additional data sources for information to support the evaluation. Discussions will also be held around the sample frames available for site selection for the qualitative interviews. Attendees to this meeting include the CATCH team, selected representatives from implementing partners, and anyone else from the program likely to be involved in implementing or supporting the mid-term evaluation. Following this meeting, the evaluation team will specify the sample of villages, CBOs, and other program participants that will be interviewed in the qualitative survey. Final revisions will be made to this Evaluation Plan following the meeting. Field Data Collection, January 20 - February 4 (Sixteen Days). Review of secondary data, key informant interviews, focus group discussions and direct observations to obtain qualitative data will be undertaken by the evaluation team over the period January 20 through February 4. A detailed itinerary for the field work is included in Annex B. Ground-Truthing Workshop, February 8-9 (Two Days). During and following the field work, the evaluation team will have started formulating preliminary observations on outputs produced by WALA, the impact, lessons learned and recommendations for the remaining life. These will be shared in the Ground Truthing Workshop with WALA implementation staff and discussed further to ensure that they reflect reality and are described appropriately. Three major outputs are targeted for the workshop. These are (1) agreement on key observations assembled so far from the evaluation on the outputs, effects and impact of the program, (2) identification of lessons learned, and (3) agreement on major recommendations for the remaining life of the program. Participants in the workshop will be program implementation staff and the evaluation team. Stakeholder Debriefing, February 13 (Half Day). Following the Ground-Truthing Workshop, the evaluation team will continue to process information toward developing a final product which will document observations, lessons learned and recommendations. A draft summary of this product in Powerpoint will be presented to the leadership of the consortium as well as representatives from other program stakeholders, such as government agencies or technical partners, for final discussions. This is the final opportunity for implementation stakeholders to provide input into the evaluation before the final product is produced. Participants in this debriefing are CATCH, the leadership from consortium partners, and representatives from other agencies having active roles in the program. USAID/FFP Debriefing, Sometime February 15 (Two Hours). Following the Stakeholder Debriefing, final changes will be made to the evaluation product, and a summary of the final product in Powerpoint will be presented to USAID/FFP. Participants in this meeting will be CRS leadership, WALA leadership and representatives from USAID/FFP. Sampling Approach Qualitative Information Gathering. Table 1 indicates the different types of program participants in each SO, the number of participants in the program, and the number of these participants that WALA Mid-Term Evaluation Report Annex E 91 will be interviewed during the qualitative assessment. In consultation with WALA consortium partners, the evaluation team will select the villages from which to obtain information from the types of participants listed. Since this is a Mid-Term Evaluation, the sample will not be chosen randomly, but rather strategically, so that the evaluation team can observe what is working and what is not working, as well as any particularly innovative approaches. The sample will cover all implementation partners in the consortium. Table 1. Qualitative Interviews for Each SO Type of Participant Number of Participants in the Program Planned Number in FGDs or Individual Interviews SO1: MCHN Health Promoters 128 28 Care Group Volunteers 13,610 280 Mothers Targeted by Care Group Volunteers 138,609 280 Health Surveillance Assistants 424 28 Mothers Participating in CCFLS TBO 70 Mothers Receiving Rations for Malnourished Children 4516 7 Families Receiving Safety Net Rations 8,853 7 Village Health Committees TBO 7 SO2: ANR Lead Farmer Producer Groups 4905 14 Village Savings and Loan Groups (Completing at least one cycle) 443 14 Marketing Clubs 1260 14 Water User Committees 36 14 Farmer Extension Facilitators/Volunteers 182 14 Community Agents (Paid and not paid) 258 14 Agri-Business Community Agents 252 14 Watershed Management Committees/Technical Committees 7 7 Private Service Providers (only in Chikwawa) 30 10 Demonstration Plots (Farm Production including crops, small ruminants, poultry, and fish, Conservation Agriculture, and Post-Harvest Storage and Handling) 250 35 Small-Scale Irrigation Systems (River diversion & treadle pumps) 36 14 FFW/FFA Projects 102 35 SO 3: DRR Recipients of Safety Net Rations 8853 70 Recipients of FFW/FFA Rations 2204 140 Village Civil Protection Committees 100 21 Leaders of Traditional Authorities 39 7 Group Village Headmen in DRR GVHs 95 7 Area Development Committee 39 7 Area Civil Protection Committees 39 7 District Executive Committees 8 7 District Civil Protection Committees 8 7 WALA Mid-Term Evaluation Report Annex E 92 Survey Instruments Topical outlines will be used to guide the discussions around qualitative information gathering. Annex C contains the working draft topical outlines that will be used to start the field work. Evaluation Team The evaluation team will be composed of the following: Mike DeVries, Team Leader and ANR – Responsible for coordinating implementation of the evaluation, investigations around SO2, program process investigations as shown in Table 2, facilitation of meetings and workshops, preparation of debriefings, and completion of the final report. Judiann McNulty, MCHN) – Responsible for investigations around SO1, program process investigations as shown in Table 2, and the writing of sections of the final report on these investigations. Golam Kabir (DRR) – Responsible for investigations around SO3, program process investigations as shown in Table 2, and the writing of sections of the final report on these investigations. WALA Staff - Various WALA staff will join the evaluation at key points to provide the evaluation team with pertinent information and other support. A summary of the assignment of topical areas for investigation to evaluation team members is shown in Table 2. Table 2. Evaluation Team Member Assignments INVESTIGATION TOPIC LEAD TEAM MEMBER I. Program History and Context A. Program History Mike B. Operating Environment (including Assumptions & Risks) Mike II. Outputs, Intermediate Results and Strategic Objectives A. SO1 MCHN Judiann B. SO2 ANR Mike C. SO3 DRR Golam III. Overall Impact A. Focus of the Evaluation Mike B. Targeting Mike C. Impact at the Goal Level Mike IV. Program Processes A. Program Management (CATCH & Partners) Mike B. Partnership Judiann C. Knowledge Management (including M&E) Mike D. HIV/AIDS Mainstreaming Judiann E. Gender Mainstreaming Judiann F. Program Integration and Complementarity Judiann WALA Mid-Term Evaluation Report Annex E 93 G. Financial Resource Management Golam H. Commodity Management Golam I. Human Resource Management Golam J. Environmental Monitoring and Impact Mitigation Golam Final Report A draft report following the outline provided in Annex D will be presented to CRS Malawi in the exit debriefing scheduled for 17 February. CRS will return comments to the Team Leader by March 7, and the final version of the report responding to comments will be submitted to CRS Malawi by March 15. WALA Mid-Term Evaluation Report Annex E 94 ANNEX A – Summary Tables Included in Evaluation Report Key Dates for the WALA Program – ANNEX C Resource Summary – Page 4 Reported Beneficiaries by Intermediate Result and Strategic Objective by Fiscal Year – Page 42 Summary Assessment of Achievements at Different Levels (Modified from Orginal Plan) – Page 43 Cash Expenditure Summary Projected Through December 2011 (US$) – Page 49 LOA Commodity Summary (MT) – ANNEX J Monetization Cost Recovery – ANNEX J Commodity Loss Summary (MT) – ANNEX J WALA Mid–Term Evaluation Report Annex E 95 Evaluation Schedule WALA Mid-Term Evaluation Date Day Activity Paid Work Days Team Questions/Comments Leader MCHN SO2 DRR SO3 By 23 December Friday Preliminary literature review and Draft Evaluation Plan sent to CRS Malawi 2 2 2 CRS Malawi needs to send IPTT, annual and quarterly implementation reports (including commodity reports), program manuals, and other relevant background documents to evaluation team By January 12 Thursday Review of Annual Survey Data and Results tables with input sent to Malawi for additional analysis 1 1 1 CRS Malawi needs to send survey data and results to evaluation team 16 January Monday Mike, Judiann and Golam arrive in-country 2 TD 2TD 2TD Arriving in Blantyre by January 16 17 January Tuesday MORNING: Evaluation Team meeting to fine-tune evaluation methodology and agree on responsibilities. 1 1 1 CATCH is welcome to participate in this meeting AFTERNOON: Programme Orientation - CATCH provides an overview of the programme for the evaluation team 18-19 January Wednesday - Thursday Evaluation Plan revision (based on meetings previous day), field visit schedule finalized, tools development, and review of existing quantitative data sets 2 2 2 Actual analysis of existing quantitative data sets from impact surveys and monitoring system will not be done at this time, but plans will be developed, including specification of information needed, for doing the analysis. A short meeting will be scheduled with CATCH and USAID/FFP to present an overview of the Evaluation Plan 20 January - 4 February Leave on a Friday, Finish on Saturday Field Work - Collection of qualitative data through interviews with impact groups, intermediaries, program implementation staff and partner leadership staff 14 14 14 Assume that on average will spend two days with each of seven partners. However, this may vary if some partners are covering a wider area or otherwise have a larger role in the program. 5-7 February Sunday - Tuesday Information Processing and Preparation for the Ground Truthing Workshop 3 3 3 Evaluation team members will work individually to prepare for GT Workshop. 8 - 9 February Wednesday - Thursday Ground-Truthing Workshop in which the evaluation team presents to implementing staff their draft observations and 2 2 2 Participants in this workshop should be restricted to the evaluation team and program implementation staff. It should not include anyone who is not intimately familiar with at least some WALA Mid–Term Evaluation Report Annex E 96 recommendations for discussion. part of the program. 10-12 February Friday - Sunday Post-GT Workshop processing, debriefing preparations and report writing. 2 2 2 The GT Workshop will produce clarifications on observations, practical revisions to preliminary recommendations and identification of gray areas that need additional investigation. 13 February Monday MORNING: Stakeholder Debriefing to present an overview of the primary content that will be included in the evaluation report and that will be presented to USAID/FFP 1 1 1 This is the dress rehearsal for the presentation that will be made to USAID/FFP, usually a half day workshop. Participants include CATCH and leadership from implementing partners. 13 - 16 February Monday - Thursday Preparation of Draft Report (in-country) 6 5 5 Judiann and Golam will be finished by 15 February and can depart at that point 15 February Wednesday USAID/FFP Debriefing -- -- -- Evaluation team with CRS Country Representation and Programme Manager, no more than two hours, in Blantyre. 17 February Friday AFTERNOON: Exit Debriefing and turnover of Draft Report to CRS Malawi 1 --- --- 18 February Saturday Departure of Evaluation Team 1 TD -- 1 TD This is the date the team leader departs, other team members will have departed earlier By 7 March Wednesday CRS Malawi compiles and sends comments on draft report to the Team Leader --- --- --- By 15 March Thursday Final report submitted to CRS Malawi 2 1 1 TOTAL PAID WORK AND TRAVEL (TD) DAYS 37 3 TD 34 2 TD 34 3 TD Total of 105 work days plus 8 travel days. WALA Mid–Term Evaluation Report Annex E 97 WALA Mid-Term Evaluation Focus Group Discussions, Individual Interviews and Site Visits during Each PVO Field Trip Over the course of the two-day field visit with each WALA implementing partner as shown in the table below, the WALA Mid-Term Evaluation team would like to have the following interviews, focus group discussions and site visits. FIELD VISIT SCHEDULE DATES LOCATION PVO CATCH STAFF SUPPORT 16-20 January Blantyre CATCH All 20-21 January Zomba & Chiradzulu SAVE Dorothy (DRR), David (Livestock), Jay (M&E) 23-24 January Nsanje TLC David (Livestock), Catherine (MCHN), Owen (M&E) 25-26 January Chikwawa Chikwawa Diocese Gitau (Agri-business), Solani (VSL), Jonathan (Agriculture) 27-28 January Thyolo World Vision Juma (Irrigation), David (Livestock), Gitau (Agri-buisness), Dorothy (DRR) 30-31 January Mulanje Africare Solani (VSL), Juma (Irrigation), Catherine (MCHN), Abel (M&E) 1-2 February Zomba & Machinga EI Gitau (Agri-business), Dorothy (DRR), Jam (Governance), Isaac (M&E) 3-4 February Balaka & Machinga PCI Jonathan (Agriculture), Solani (VSL), Jay (M&E) 5-15 February Blantyre CATCH All Upon arrival:  Brief Overview provided by PVO staff (1 hour, including questions. In addition to clarification on project activities, the MTE team will obtain information on changes in the context, environmental monitoring and human resource management in this session.) For SO1 (Judiann) In each village to be visited (at least four villages):  Individual Interview with a Health Promoter (30 minutes)  Focus Group Discussion with Care Group Volunteers (no more than ten participants, 1 hour)  Focus Group Discussion with mothers targeted by Care Group Volunteers (no more than ten participants, 1 hour)  Individual Interview with a Health Surveillance Assistant (1 hour) Over the course of the two days: WALA Mid–Term Evaluation Report Annex E 98  Focus Group Discussion with mothers participating in CCFLS (no more than ten participants, 1 hour)  Individual Interview with a mother receiving rations for a malnourished child (1 hour)  Individual Interview with a family receiving a safety net ration (1 hour)  Two site visits to see a normally scheduled CCFLS  Focus Group Discussion with a Village Health Committee (1 hour) For SO2 (Mike) In at least two villages:  Focus Group Discussion with Lead Farmer/Producer Group (1.5 hours)  Focus Group Discussion with a VSL Group that has completed at least one cycle (1.5 hours)  Focus Group Discussion with a Marketing Club (1.5 hours)  Individual Interview with a Farmer Extension Facilitator (1 hour)  Individual Interview with a Community Agent (1 hour)  Individual Interview with an Agri-business Community Agent (1 hour)  Focus Group Discussion with a Water Users Committee(1.5 hours) Over the course of the two days:  Focus Group Discussion with a Watershed Management Committee (1.5 hours)  Site Visits to two irrigation sites  Site Visits to five demonstration sites showing different technologies  Site visits to five FFW project sites (can be combined with irrigation site visits and can also be covered with short stops enroute between villages) For SO3 (Golam) In each village to be visited (two to three villages):  Individual Interviews or Focus Group Discussions with recipients of safety net rations (1 hour)  Focus Group Discussion with recipients of FFW/FFA rations (no more than ten participants, 1 hour)  Focus Group Discussion with a Village Civil Protection Committee (1 hour) Over the course of the two days:  Individual Interview with the chief of a Traditional Authority (30 minutes)  Individual Interview with the headman of a GVH (30 minutes)  Focus Group Discussion with an Area Development Committee (1 hour)  Interview with one or two representatives of a District Executive Committee (30 minutes)  Focus Group Discussion with a District Civil Protection Committee (1 hour)  Focus Group Discussion with a Area Civil Protection Committee (1 hour) For Program Processes (MTE team member WALA Mid–Term Evaluation Report Annex E 99  Interview with the Project Manager, Country Representative or Country Director and any other senior staff from the partner headquarters (Mike, 1 hour)  Interview with M&E Staff (Mike, 1 hour)  Site visits to PVO commodity management facilities and interviews with commodity management staff (Golam, 1 hour)  Interview with PVO financial management staff (Golam, 1 hour) At the end of the field visit, the MTE team will conduct a brief debriefing for staff to explain the next steps in the process and answer any questions. WALA Mid–Term Evaluation Report Annex E 100 ANNEX C: TOPICAL OUTLINES FOR QUALITATIVE DATA COLLECTION SO1: MCHN Community Complementary Feeding and Learning Session – observe in one or two partners Objectives: a. Discern adherence to implementation standards b. Assess CGV understanding of key concepts and capacity to independently facilitate effective CCFLS c. Assess effectiveness as a behavior change strategy (combined with questions for participants) d. Determine potential for replication or scaling up in other Title II programs Number of mothers in attendance______ How many of the children are malnourished?________ Who is facilitating the session?________________________________ What is the role of WALA staff? ____________________________________________________ Yes No Comments 1. Did mothers bring all the materials? 2. Are all mothers actively engaged? 3. ECD activities for children? 4. Menu is diverse and nutrient-dense 5. Hygiene before cooking 6. Hygiene before eating 7. Active feeding 8. Portion size is appropriate for child’s age 9. Mothers assist with clean-up 10. Food safety practiced 11. Mothers can articulate menu content 12. Water source and latrine available Questions for the CGV facilitating the session: 1. Why do we ask the mothers to bring the foods? Has this been a problem? Your solution? 2. Would you show me or explain the different menus? Do you sometimes change the menu? Why? 3. How do you know if a malnourished child is recovering? 4. Do you invite all mothers in your charge? If not, why not? Do you invite others? Do the mothers who are invited come every day? 5. Are the fathers of the children invited? Do they come? Do they encourage their wives to come? 6. Does anyone in the community other than these families provide any materials or support? 7. Do you think you can continue to hold these sessions when WALA no longer comes to the community? Will the mothers come? Village Health Committees in every village Objectives: a. Ascertain knowledge and involvement in WALA activities b. Assess current level of support for volunteers c. Learn of other ways the VHC may be able to support improved nutrition and health d. Discuss their role in sustainability of activities after WALA WALA Mid–Term Evaluation Report Annex E 101 Number of VHC members____ Number present____ Names on separate sheet 1. How long has this VHC existed? 2. How long have you been members? How were you selected? Do members ever change? 3. What is purpose of the VHC? What did you do before WALA? 4. What activities are you doing now as VHC? 5. How many of you are also CGVs? Are you a volunteer in another WALA activity? 6. What do the CGVs do? What are mothers learning from them? 7. What can you tell me about the CCLFS? (probe for any support from VHC to CCLFS) 8. Besides the CGVs, the CCLFS, the gardens, what else is needed to prevent malnutrition here? 9. What role do men/fathers have in helping prevent malnutrition? 10. In the future, when WALA is no longer helping, what can you do to keep the CGVs active? 11. How will you continue motivating families to practice the healthy behaviors they have learned? Promoters – in every site Objectives: a. Assess understanding of basic MCHN concepts and of approaches (CGs and CCLFS). b. Evaluate capacity to train and motivate CGVs c. Assess commitment and motivation for purposes of sustainability. Name_________________ Number of years as a promoter_____ 1. What training have you received from WALA? Which topic has been most useful? 2. Please tell me how you found the CGVs and organized the group? Who trains them? 3. What are the CGVs supposed to be doing? How often do you accompany each one? 4. How do they identify the households they will visit or be responsible for? 5. Have all the CGVs been the same or have there been some changes? Why? 6. How much time do the CGVs spend each week? Why are they willing to give this time? 7. Can you explain to me what is CCFLS and what happens there? Who trained you in CCFLS? What is the purpose of CCFLS? Who is invited? Where did the menus come from? 8. (Ask questions on concepts from CGV interview guide on CCFLS on observation sheet and note here which concepts are clearly understood or not.) 9. What challenges have you encountered with the CCFLS sessions? Solutions? 10. What other activities do you lead or what other responsibilities do you have as a promoter? 11. What additional support do you need from WALA to do your work as promoter? 12. What is your relationship with the HAS? Does the HAS or health center support you? How? 13. When WALA no longer supports you, will you be able to continue to support the CGVs? Care Group Volunteers – in every site Objectives: a. Assess capacity to promote behavior change through their activities – content and methods. b. Ascertain motivation and potential for sustainability c. Appraise WALA partner capacity to implement the CG model WALA Mid–Term Evaluation Report Annex E 102 Number of CGVs present in FGD________ Record names on separate sheet. 1. One by one ask how long each has been a CGV. Why are you a CGV? 2. How much time to you spend on CGV activities every week? Does this ever become difficult? 3. What kinds of activities do the CGVs carry out? Which activities do you most like to do? (probe garden) 4. Who trains you? What topics have you learned? What is the most interesting thing you learned? 5. How do you pass this learning to the mothers? (probe methods and frequency of contact) 6. What have you learned that has helped you in your own family? 7. How did you select the households you cover? What age are the youngest children? Are all households in this village covered? Why or why not? Coverage of new households (PLW) 8. How often do you meet with the promoter? Does he/she accompany you on home visits? How often? WALA staff? 9. Have you received flip charts to use? Which topics do they cover? 10. Who can tell me all the messages about breast feeding? Which one is most important? Why? 11. Why do you think some mothers give their baby water or nsima before they are 6 months old? 12. What foods should a mother first give to her baby at six months? Why? Consistency? How often? (probe for barriers the mothers face and CGV solutions) 13. Do you ever talk to the fathers? What topics? 14. What would make it possible for you to continue visiting mothers if there were no meetings with the promoter or WALA staff? Mothers who are targets of CGs – in every site of every partner Number in attendance_____ Obejectives: a. Assess the efficacy of the project approaches (CG, CCLFS, Impact) on bringing about behavior change. b. Evaluate the need for any improvements in implementation of the approaches to improve impact. Each mother introduces herself and tells us the name and age of her youngest child. 1. Does someone from the Care Group visit you at home? What is her name? Did you know her before? 2. How often does she visit? How long does she stay there? What do you discuss during those visits? 3. Is your husband ever home when she visits? Does he listen or do you tell him what she said? 4. Do you know women with small children who are not being visited by the CGL? Why not? 5. Do the CGLs conduct some group activities? Please tell me about those activities. Probe: What is the purpose? How often? What happens there? How many women come? How long does the session last? 6. Does the CGL show you pictures or posters? Do you remember any of the pictures? 7. How much food does a child who is one year old need to eat? How often? Why? 8. What are the best first foods to give a child? (describe fully) 9. At what age do most mothers start giving their baby food? Water or tea? 10. What problems do women have with breastfeeding? Who can help? 11. Have you learned anything about gardening? How many have a new garden from WALA effort? WALA Mid–Term Evaluation Report Annex E 103 Mothers who have participated in CCFLS – one GVH per partner Objectives: a. Assess effectiveness of CCFLS in promoting behavior change. b. Discern acceptability of CCFLS to participants and challenges to their participation. 1. In how many sessions have you participated? 2. What do you like about the sessions? 3. What have you learned? 4. Why do you contribute all the food? Is this very difficult? 4 a. CCFLS is 12 days in a row. Can you go every day? Why or why not? (probe about hours spent there) 5. Which of the menus have you repeated at home? Did the family like them? 6. Which ones were you not able to make at home? Why not? 7. How much food does a child who is one year old need to eat? How often? Why? 8. What are the best first foods to give a child? (describe fully) 9. For how long should a child receive only breast milk? Why do some mothers start giving tea too soon? 10. At CCFLS, how often do you wash your hands? Why? Can you also do this at home? Why or why not? Mothers/families who are receiving rations for MAM – one per partner Name_______________ Name of child_____________ Age of child______ 1. What foods are you receiving? Who eats the food? How do you prepare the CSB? 2. Do you know why you are receiving this food? 3. What else are you doing to help your child recover? 4. Do you know the Care Group Volunteer? Does he/she visit you? 5. Do you attend sessions to learn more about caring for your child and about health? 6. What is the most interesting or useful thing you have learned there? 7. How often do you take your child to be weighed? Where? 8. Ask to see the growth card and ask her to explain what the plotted line means. 9. Observe or ask about garden and poultry. 10. Do you participate in a VSL group? 11. Does someone in the family participate in another WALA activity? 12. If your family has some problem like lack of food due to crop failure or lack of money or another reason, who would you ask for help? Mothers/families who are receiving safety net rations – one per partner Name_______________ Name of child_____________ Age of child______ 1. What foods are you receiving? Who eats the food? How do you prepare the CSB? 2. Do you know why you are receiving this food? 3. Do you know the Care Group Volunteer? Does he/she visit you? 4. Do you attend sessions to learn more about caring for your children and about health? 5. What is the most interesting or useful thing you have learned there? WALA Mid–Term Evaluation Report Annex E 104 6. Observe or ask about garden and poultry. Has WALA helped you with the garden? 7. Do you participate in a VSL group? 8. Does someone in the family participate in another WALA activity? 9. If your family has some problem like lack of food due to crop failure or lack of money or another reason, who would you ask for help? HSAs wherever available in GVH 1. How long have you worked with this GVH? What activities do you here on a regular basis? 2. What are the main health problems facing women? Children? 3. What do you think are the main causes of acute malnutrition in children? How many cases of SAM did you find here in 2011? 4. What is the cause of chronic malnutrition/stunting in this area of Malawi? 5. What is the role of the promoter? How do you interact or coordinate with him/her? 6. Do you know about the Care Groups? What do you think of the Care Group approach? 7. What are the CGLs doing? What are they teaching? 8. Do you know about other activities of WALA in this GVH? Which of those do you think will improve health and nutrition of women and children? Which ones will improve food security? 9. Do you know if WALA staff meet with health facility/clinic staff? How often? What do they discuss? 10. What difficulties are health centers facing now with supplies? Transportation for outreach and supervision? WALA Mid–Term Evaluation Report Annex E 105 Topical Outline - SO2 - Agriculture and Natural Resources The purpose of the investigation using the following questions is to obtain information to: (1) assess the quality of the outputs produced by the program (2) assess the impact of these outputs on the food security of target populations (3) assess the quality of the approaches used by the program (4) judge the potential sustainability of the impact that has been achieved. Investigation will pay particular attention to identifying problem areas for which recommendations can be formulated for the remaining life of the program and best practices that should be scaled up within the program. Information will be gathered through key informant interviews, focus group discussions or large group discussions with program participants, project implementation staff, and program managers. Topics to be Covered with Program Beneficiaries from Targeted Impact Groups Beneficiaries in Lead Farmer Groups or Producer Groups  Group purpose, vision, and values  Type of members (sex and age)  Group member selection process  Group member poverty/food insecurity status  Member benefits (type, equitable distribution within group, relative to costs)  Others in the community who want to participate, why not?  Participation in the program - trainings, distributions of seed, livestock and other inputs, others  Best impact, kinds of impact (life changing), why?  Sustainability of impact - what happens after project ends  Low impact, why?  Cross-cutting Themes - Governance, HIV/AIDS, Gender, Environment  Other Targeting Questions - Participating in other interventions  Recommendations related to current activities  Recommendations for additional activities Beneficiaries in VSL Groups  Group purpose, vision, and values  Type of members (sex and age)  Group member selection process  Group member poverty/food insecurity status  Others in the community who want to participate, why not?  Participation in the program - trainings, distributions, others  Best impact, kinds of impact (life changing, member benefits, distribution of benefits within group, benefits vs. costs), why? WALA Mid–Term Evaluation Report Annex E 106  Sustainability of impact - what happens after project ends, additional cycles, PSP  Low impact, why?  Cross-cutting Themes - Governance, HIV/AIDS, Gender, Environment  Other Targeting Questions - Participating in other interventions  Recommendations related to current activities  Recommendations for additional activities Beneficiaries in Marketing Groups  Group purpose, vision, and values  Type of members (sex and age)  Group member selection process  Group member poverty/food insecurity status  Others in the community who want to participate, why not?  Participation in the program - trainings, distributions, others  Best impact, kinds of impact (life changing, member benefits, distribution of benefits within group, benefits vs. costs), why?  Sustainability of impact - what happens after project ends, maintaining business relationshis  Low impact, why?  Cross-cutting Themes - Governance, HIV/AIDS, Gender, Environment  Other Targeting Questions - Participating in other interventions  Recommendations related to current activities  Recommendations for additional activities Beneficiaries in Water Users Committees  Group purpose, vision, and values  Type of members (sex and age)  Group member selection process  Group member poverty/food insecurity status  Others in the community who want to participate, why not?  Participation in the program - trainings, FFW, distributions, others  Best impact, kinds of impact (life changing, member benefits, distribution of benefits within group, benefits vs. costs), why?  Sustainability of impact - maintaining irrigation system, what happens after project ends  Low impact, why?  Cross-cutting Themes - Governance, HIV/AIDS, Gender, Environment  Other Targeting Questions - Participating in other interventions  Recommendations related to current activities  Recommendations for additional activities Topics to be Covered with Program Intermediaries Farmer Extension Facilitators (FEFs), Community Agents (CAs), and Agri-business Community Agens (ACAs) WALA Mid–Term Evaluation Report Annex E 107  Function of position, vision and expectations  Characteristics (sex, age, food security status)  Types of support received from WALA  Activities  Working well, why?  Not working well, why?  Sustainability, what happens after project ends  Recommendations Lead Farmers  Lead farmer function  How selected (innovative, curious, respected, representative)  Types of support received from WALA  Benefits from the position  Activities  Working well, why?  Not working well, why?  Sustainability, what happens after project ends  Recommendations Watershed Management Committee  Committee purpose  Type of members  Interface with other community groups (VDC, producer clubs, VSL groups, health clubs, etc.)  Types of support received from WALA  Activities  Working well, why?  Not working well, why?  Sustainability, what happens after project ends  Recommendations Watershed Community Technical Committees  Committee purpose  Type of members  Types of support received from WALA  Activities  Working well, why?  Not working well, why?  Sustainability, what happens after project ends  Recommendations Seed Bank Committees (Mulanje only)  Committee purpose WALA Mid–Term Evaluation Report Annex E 108  Type of members  Type of support received from WALA  Activities  Working well, why?  Not working well, why?  Sustainability, what happens after project ends  Recommendations Private Service Providers (Chikwawa only)  PSP function  How selected  Types of support received from WALA  Expected benefits from the position  Types of activities expected to be undertaken  Potential sustainability Topics to be Covered with Program Implementation Staff and Program Managers  Farm Interventions (Watershed Planning Approach, Technologies & Demonstrations, Small-Scale Irrigation, Seed, Livestock, Poultry, and Aquaculture) Strategy and its evolution Activities (trainings, demonstrations, distributions, FFW and other physical investments) Implementation challenges Best impact, what kind of impact, and why? Sustainability of impact Low impact, why? Replicability Recommendations (fixing problems, scaling up best practices)  Village Savings and Loans Model being used and its evolution (PSP strategy, staggered targeting, exit strategy, IGA SPM) Implementation challenges (holding cash, record-keeping, male participation) Best impact, types of impact, who benefits Low impact, why? Sustainability Replicability Recommendations (fixing problems or scaling up best practices  Agri-Business Interventions Approach - ABAs, Marketing clubs, clusters, FaaB, fairs and value chain stakeholder workshops, agro-processing, out-grower relationships Implementation challenges - membership fees, WALA Mid–Term Evaluation Report Annex E 109 Value chain analyses, choice of value chains, quality of analysis Accessing large local markets (rice in Blantyre) and export markets Sustainability and Replicability Recommendations (fixing problems, scaling up best practices)  Integration with other Components in the Program (MCHN, DRR) Examples of Integration Implementation challenges Recommendations  Collaboration with Other Stakeholders Processors in Blantyre for using surplus OFSP Quality Protein Maize (susuma) from CIMMYT CMD-resistant cassava from IITA Pigeon Pea from ICRISAT Agricane technical partner for irrigation Cornell University on marketing Collaboration with NASFAM Collaboration with MoAFS Others (NGOs, government and private sector)? Recommendations Topics to be Covered with Site Visits Farm Technologies (excluding irrigation which is dealt with separately) Types of technologies being promoted: Improved Crop Varieties: Orange-Fleshed Sweet Potato, tomatoes, amaranth, pigeon pea Conservation Agriculture: Basin planting, minimum tillage, compost manure, contour marker ridging, and ridge re-alignment, crop rotation/intercropping, stone bunds, water absorption trenches, continuous contour trench Post-Harvest Storage and Handling: hermetic bags, silos Animal Husbandry: Small ruminants, poultry, aquaculture  How identified (farmer-driven, technologically appropriate, economically feasible)  Extension approaches (location for demonstrations, demonstrations replicable, follow-up, cross-visits, field days)  Best impact, why? Evidence of adoption  Low impact, why?  Sustainability of impact (what happens after project ends)  Potential for additional innovation after program ends  Recommendations (fixing problems, scaling up best practices) Small-Scale Irrigation Systems  Identification of sites WALA Mid–Term Evaluation Report Annex E 110  Technical quality  Who benefits?  Subsidization, replicability  Operations and maintenance  Recommendations (fixing problems, scaling up best practices) Seed/Seedling Multiplication (pigeon peas, soybeans, groundnuts, sweet potatoes, agro-forestry species)  Site selection  Seed quality assurance  Sustainability, what happens after project ends, management, subsidization levels  Impact on existing seed suppliers  Recommendations (fixing problems, scaling up best practices) FFW Investments (night reservoirs, check dams, planting basins, contour ridges, contour trenches, nursery fences, irrigation canals, infiltration pits, afforestation, roads, water harvesting structures, fish ponds)  Site selection  Design  Subsidization, cost-share  Who benefits  Operations and maintenance  Sustainability, what happens after project ends  Recommendations (fixing problems, scaling up best practices) WALA Mid–Term Evaluation Report Annex E 111 Topical Outline – SO3 – Disaster Risk Reduction The following topics should be covered in interviews with project implementation staff and targeted beneficiaries. WALA-Facilitated Disaster Systems (Target Interviewees – Communities having WALA￾Facilitated DRR systems meaning the communities with disaster early warning and response systems in place that includes Disaster management Plan)  What are the early warning measures you have learned from WALA?  How are they different from what you were aware of or practicing before the WALA?  What are disaster response measures you have learned from WALA?  What are the response measures you used to do prior to the training from WALA?  How are the Early Warning and Response Systems documented?  Did you and who else from your community participated in the development of disaster response and response systems?  How functional are these documents (Disaster Management Plan)?  Have there been any disasters since the disaster early systems and response systems were in place? Did you use these plans in the aftermath of the disasters?  How often you revisit the disaster management plans?  What are the changes you think in the context to make these more relevant? ToT in Disaster Management (Target Interviewees – PVO DRR Coordinators and Members of VCPC)  Have you received any training? What are they? What training activities seem most irrelevant now in terms of having impact on your lives? Why are these irrelevant? What are the changes you think in the context to make these more irrelevant? Food Rations Distribution (Target Interviewees – PVO Managers and Beneficiaries of the Safety-net and Emergency Relief)  What are the criteria used for selection of beneficiaries?  Who participates in the selection of beneficiaries?  What roles the communities play in the selection?  What are the changes/benefits you (safety net food rations recipients) see from the food assistance under the safety net program? What do you do with the food rations you receive from the program?  What you (emergency relief recipients) would have done if you had not received food from the program? Did you receive any assistance from any one else other than the WALA assistance? Food for Work and Food for Assets Activities (Target Interviewees – FFW/FFA Beneficiaries and Communities)  What are the criteria used for selection of FFW/FFA interventions (schemes) and beneficiaries?  Who participates in the selection of interventions and beneficiaries? WALA Mid–Term Evaluation Report Annex E 112  What roles the communities play in the selection of FFW/FFA interventions and beneficiaries?  What are the changes/benefits you (FFW/FFA communities) see from the FFW/FFA interventions?  What do you do with the food rations you receive from the program?  What you (emergency relief recipients) would have done if you had not received food from the program? Did you receive any assistance from anyone else other than the WALA assistance? At the end of discussion of each of the above subjects ask - What has been most useful? What has been least useful? Any recommendations on any of these subjects? WALA Mid–Term Evaluation Report Annex E 113 Topical Outline – Context & Operating Environment The following topics should be covered in interviews with project implementation staff and targeted beneficiaries.  In the last three years, what changes have occurred in the operating context (major events or slow onset) that have affected program implementation either positively or negatively?  Which program activities/outputs seem most irrelevant now in terms of having impact on the lives of targeted impact groups? Why are these irrelevant? What has changed in the context to make these more irrelevant?  What was the most important information that emerged from the baseline survey that should be taken into account in formulating recommendations for the remaining life of the project? Topical Outline – Targeting Topics for Programme Managers and Implementation Staff ∞ Intended Beneficiaries. Who are the intended beneficiaries of program activities? ∞ Targeting Systems. What systems were used to identifying program participants? What worked well? What did not work well? ∞ Actual Beneficiaries. Who are the actual beneficiaries of program activities and how do they compare to the intended beneficiaries? ∞ Targeting Drift. How does the program monitor and manage targeting drift? ∞ Unintended Beneficiaries. How would you describe those beneficiaries who fall outside of the targeted impact group in terms of food insecurity or vulnerability? ∞ Level of Benefits. How would you classify beneficiaries in terms of level of benefits? For example, there will be some beneficiaries who benefit from all components of the program while others may only benefit from one activity. What proportion of beneficiaries fall into the different classifications? Topics for Participants and Intermediaries ∞ Intended Beneficiaries. What proportion of actual program beneficiaries fall within the definition of the impact group for the program "chronically food insecure households"? ∞ Unintended Beneficiaries. Who else, apart from these intended beneficiaries, has benefitted from the program and how have they benefitted? How should we describe these unintended beneficiaries in terms of food insecurity? ∞ Level of Benefits. What proportion of any particular group of participants are benefitting from multiple interventions? What proportion of the group are only benefitting from a single program intervention? WALA Mid–Term Evaluation Report Annex E 114 Topical Outline – Program Management ∞ What is the history of the management structure for the program, including turnover of any key positions, restructuring or other changes? ∞ Who is responsible for the vision for the program at the consortium level and within each partner? How well has the vision for the program been articulated? How effectively has this vision been imparted to program staff within the different agencies? ∞ How are strategic and operational plans developed for the program? What has worked well in this process? What has not worked well? ∞ How are implementation problems identified, analyzed and solved? How is information generated by the program's M&E systems used for decision-making? What has worked well in terms of problem-solving? What has not worked well? ∞ How well does information get communicated throughout the program, including within the program, with implementing partners at different levels, with external contacts and donors? What has worked well? What has not worked well? ∞ How well has the program communicated with stakeholders outside of the program? What has worked well? What has not worked well? ∞ Specific topics: Advisory body (Directors), CATCH, different management and technical support structures with partners, governance IR vs. governance cross-cutting, technical working groups ∞ What changes would you propose to improve program management in WALA? ∞ What are the most important lessons learned from WALA relative to management of the program? Cross-Cutting Themes: Questions to be asked by each of MTE members with all participant groups A. Gender Mainstreaming: Observe how many women are in the FGD and how actively they are participating. B. Targeting: Please refer to questions under Topical Outline for Targeting C. Environmental Protection: WALA has to-date implemented a number of activities to mitigate/prevent environmental degradation resulting from activities and general community practices such the Watershed Development/management, Conservation Agriculture and Irrigation and drainage network construction. WALA claims significant cooperation among all stakeholders in the watershed work together to improve the watershed they live in and their entire quality of life through various activities.  Ask the community or individual visited what really impacted their communities that WALA claims.  How they see that the interventions helped mitigate negative environmental impacts.  What are the changes the community/individual has observed to date?  Have you received any training from WALA related to environmental protection? WALA Mid–Term Evaluation Report Annex E 115 D. Good Governance: WALA facilitated a number of trainings for PVO staff in an effort to support community capacities to practice good governance and strengthen the different groups formed under the WALA. Training topics included: Participatory Planning, Monitoring and Evaluation (PPME), community mobilization, group formation and dynamics.  Ask the FGD participants what was their role in the formation of different groups under WALA. How transparent was the groups formation? What were the process followed in the gropu formation?  How they see that the training helped improve their skilss in participaroy decision making?  Are they involved in any decision making since the training?  What are the changes the community/individual has observed to date? Topical Outline - Partnership We investigate the quality of partnership in terms of how well this program is using partnership relationships to enhance program effectiveness and efficiency. The purpose is to identify good practices and lessons learned. Following are the key areas of investigation. CRS, Implementing Partner and Technical Partner Perspectives ∞ How have implementing partners in the program participated in developing strategic and operational plans over the last year? What has worked well in this process? What has not worked well? ∞ How have partner budgets been prepared, monitored and reported on? What works well? What does not work well? ∞ How have problems with partners been identified and solved? What has worked well in terms of problem solving? What has not worked well? ∞ How does information get communicated to partners? What has worked well? What has not worked well? ∞ What are the most importance lessons learned from the project relative to partnership? ∞ What should be done differently in future programs similar to this one? GoM, Private Sector and Other Stakeholder Perspectives ∞ How familiar are you with the WALA Program? How much do you know about the purpose and strategy of the program? ∞ How has your organization been engaged by the program? What has worked well? What has not worked well? ∞ How well has the program communicated with you? What has worked well? What has not worked well? WALA Mid–Term Evaluation Report Annex E 116 Topical Outline - Knowledge Management (including M&E) Knowledge management refers to how knowledge is brought into a program, how it is used within a program and how it is captured and disseminated outside of a program. New ideas and approaches that are brought in, tested and adapted by a program represent “knowledge in”. M&E systems are designed to obtain and use information within the program to make decisions to improve the efficiency or effectiveness of the program, and systems for capturing best practices and lessons learned represent “knowledge out”. Key Questions - Knowledge In ∞ Who is responsible for bringing new ideas and approaches into the program from outside sources? How is this responsibility formalized and monitored? ∞ What new ideas and approaches have been brought into the program from outside the program? Who can be credited for bringing these into the program? Which of these have been most useful? Which have not been very useful? Why? o From other programs with CRS or WALA partners o From Technical Consultants o Through Cross-Visits with other organizations o Brought by new staff o Obtained from the internet or other documentation o Other sources? ∞ What should the program do to be better able to bring in good ideas and approaches from outside of the program? Key Questions - M&E Systems ∞ Who is responsible for overseeing the M&E functions in the program? ∞ Please describe the annual planning process or other processes used to develop program implementation plans. What has worked well? What has not worked so well? Why? ∞ How does the program monitor progress against the implementation plans? What has worked well? What has not worked so well? Why? o Systems - Staff and Tools o Progress monitoring reports o Field monitoring visits o Other monitoring activities ∞ Who uses information generated through the program's progress monitoring systems? What has worked well? What has not worked so well? o Quality of information o Timeliness ∞ How has the program strategy or approach changed as a result of information generated by the program's M&E systems? ∞ How is feedback provided back down the reporting chain on program monitoring reports? What has worked well? What has not worked so well? ∞ Specific topics: Consortium-MIS, WALA KM Strategy, Quality Improveent Verification Checklists, SMS Technology for Trigger Indicators, Joint Data QualityAssessments, effectiveness f the M&E TWG, linkages with MoAFS WALA Mid–Term Evaluation Report Annex E 117 ∞ What changes would you recommend to make the program's monitoring systems more effective or efficient? Specific issues - Staff turnover, field staff workload, working with the GoM, ambitious planning, political environment ∞ Relative to the baseline survey, what worked well in getting this organized, implemented and completed? What did not work so well? Why? ∞ How was the Baseline Survey Report used by the program? ∞ What other types of impact assessment tools does the program use? What works well? What has not worked so well? Why? ∞ What lessons learned relative to M&E have emerged from the experience of the WALA Program up to now? Key Questions - Knowledge Out ∞ Who is responsible for identifying, documenting and disseminating knowledge out of the program? ∞ How does the program identify best practices and lessons learned? Please provide some examples. ∞ How do these get documented and disseminated? ∞ What is working well as far as identifying, documenting and disseminating lessons learned and best practices from the WALA experience? What has not worked so well? Why? ∞ How can the program do a better job of capturing and disseminating lessons learned and best practices being generated by the project? Topical Outline - Program Integration & Complementarity We investigate the quality of integration in terms of how well this program is capitalizing on opportunities for synergistic or complementary impact as well as in terms of reducing duplication to promote efficient use of resources. Following are the key areas of investigation. With the GoM (Poverty Reduction Strategy and Government Planning) ∞ How well does the strategy of the MYAP align with government strategies as reflected in the PRSP? ∞ Which activities in the MYAP may be creating or strengthening obstacles to achieving progress described in the PRSP? ∞ How does the MYAP coordinate with relevant government ministries at different levels? ∞ How do MYAP activities interface with other government systems? How effective is this? How could this be changed to make these links more effective? With USAID Country Strategy ∞ How well does the strategy of the MYAP align with the development strategy of the USG? ∞ Which activities are making the largest contributions to the USG strategy? ∞ Which activities in the MYAP may be creating or strengthening obstacles to achieving progress described in the USG strategy? ∞ What other program activities funded by the USG are being implemented in DAP program areas? How does the MYAP interface with these other activities? How should this be changed? WALA Mid–Term Evaluation Report Annex E 118 Within each MYAP Partner ∞ How does the MYAP fit within the programs of the partners? ∞ What are the biggest contributions that the MYAP makes to the development strategy of the partners? ∞ What other programs are being implemented by the partner in the same geographic areas as the MYAP? How does the MYAP interface with these? ∞ What other programs are being implemented by the partner in the same sectors as the DAP but in different geographic areas? How does the MYAP interface with these? How do experiences get exchanged across different projects in the same sector? Within the MYAP ∞ Roughly what proportion of the program's beneficiaries are common beneficiaries (participating in more than one program activity) who have received support to address multiple food security issues? ∞ What potential synergies within the program are not being capitalized upon in terms of complementary program activities that could be extended to the same beneficiaries? Topical Outline - Financial Resources The following topics should be covered in interviews with CRS (COP, DCOP – Finance, Compliance and Administration) and CATCH Finance and Administration Staff. A. CRS and FFP Cooperative Agreement Budget Questions ∞ What is the current LOP approved budget? How has this been revised since project start-up? ∞ What are expenditures through the most recent reporting period in FY '12? ∞ What percentage of the approved budget will likely be spent by the end of the program in June 2014? ∞ How do the changes in MYAP foodgrain prices in the US affect your budget and obligation? Resource Flow ∞ How often and regular funds being obligated by USAID? ∞ Have there been any significant delays in resource flow from USAID to CRS, CRS to Partner PVOs? ∞ What was the cause of the delays? What changes were made in managing cash flow? Reporting (Financial) ∞ How do financial reports for the program get prepared? ∞ What problems have occurred with financial reporting and how have these been resolved? Cost Share ∞ How is the cost-share commitment being met? WALA Mid–Term Evaluation Report Annex E 119 ∞ What percentage of the cost-share commitment has been achieved by the end of FY 11? ∞ What percentage of the cost-share commitment will be achieved by the end of the program? ∞ What other forms of cost-share have arisen since the program was initiated? How are these being reported? Audits ∞ What audits have been completed on project funding since the program was initiated? ∞ What have been the audit findings? ∞ How have these been addressed? At the end of discussion of each of the above subjects ask - What has been most useful? What has been least useful? Any recommendations on any of these subjects? B. CRS and Partner PVOs Sub-award Agreements Budget Questions ∞ What is the current LOP approved budget? Has this been revised since project start-up? ∞ What are expenditures through the most recent reporting period in FY '12? ∞ What percentage of the approved budget will likely be spent by the end of the program in June 2014? Cash Flow ∞ How do you receive cash for project implementation from CRS? How frequent is the cash flow? ∞ How effective has cash flow been managed in the program? ∞ Have there been any significant delays in cash flow from CRS to your organization, your organization to local implementing partners? What was the cause of the delays? What changes were made in managing cash flow? Reporting ∞ How do financial reports for the program get prepared? ∞ What problems have occurred with financial reporting and how have these been resolved? Cost Share ∞ How is the cost-share commitment being met? ∞ What percentage of the cost-share commitment has been achieved by the end of FY 11? ∞ What percentage of the cost-share commitment will be achieved by the end of the sub￾award? ∞ What other forms of cost-share have arisen since the sub-award agreement was signed? How are these being reported to CRS? Audits ∞ What audits have been completed on project funding since the sub-award signed? ∞ What have been the audit findings? ∞ How have these been addressed? WALA Mid–Term Evaluation Report Annex E 120 At the end of discussion of each of the above subjects ask - What has been most useful? What has been least useful? Any recommendations on any of these subjects? Topical Outline - Commodities The following topics should be covered in interviews with CRS (Commodity and Monetization Staff) and CATCH (Commodity Logistics Staff) . A. CRS Commodity and Monetization Staff Pipeline ∞ How does the food pipeline work? Who is responsible for call forwards, port off-loading and transport to the port warehouse, inland transport, and distribution to end users? ∞ What problems or changes have occurred with call forwards and how were these managed? ∞ What lessons have you learned with you commodity management in the MYAP? Commodity Storage and Handling (Direct Distribution Commodity) ∞ What are the arrangements for storage and handling of distribution commodities? ∞ How many warehouses you use for storage of the commodity? Where are they located? What are their storage capacities? ∞ How do you do the warehouse maintenance? Do you fumigate the warehouse? What are the fumigants you use? ∞ What problems have occurred with port commodity management and how were these managed? ∞ What problems have occurred with inland transport and how were these managed? ∞ What problems have occurred with warehousing/final distribution and how were these managed? ∞ How do you transfer commodity from the PDP and other temporary warehouses to the warehouses at the field? ∞ What are the terms and conditions with transport owners if commodities are lost while at their disposal? ∞ Have there been any losses of the commodity during storage and transfers? How the incidence of were reported to USAID? How the losses are being recovered? ∞ What was the impact of the losses on the program as well as on subsequent call forwards? What changes were made in commodity management systems in response to the losses? Monetization ∞ What has been the history of monetization in the MYAP? ∞ How is monetization currently being done? What are the types of commodities being monetized? ∞ What problems have occurred with monetization and how were these managed? ∞ What is the anticipated cost recovery rate on monetization over the life of the project? What is the cost recovery rate on the most recent monetization? ∞ How is monetization being used to support market development? WALA Mid–Term Evaluation Report Annex E 121 Reporting ∞ How are commodity storage, handling and distribution, and monetization reports prepared? ∞ What problems have occurred in report preparation and how have these been managed? ∞ What has been the response from the donor on commodity reports? Rations ∞ How are ration composition and quantities calculated? ∞ How have they changed over the life of the program? ∞ How have rations been used by recipients? ∞ What changes should be considered relative to ration composition or quantities? Why should these changes be considered? B. CATCH Partners Commodity Logistics Staff Pipeline ∞ How often do you receive commodity from CRS? How does the supply from CRS work? Have you had any supply issues? ∞ Who is responsible for receiving the commodity from CRS? Commodity Storage and Handling ∞ What are the arrangements for storage and handling of distribution commodities? ∞ How many warehouses you use for storage of the commodity? Where are they located? What are their storage capacities? ∞ How do you handle the distribution to beneficiaries? ∞ What problems have occurred with warehousing/final distribution and how were these managed? ∞ What lessons have you learned with your commodity management in the MYAP? ∞ What losses have occurred and how did these occur? What was the impact of the losses on the distribution to beneficiaries? What changes were made in commodity management systems in response to the losses? ∞ How do you do the warehouse maintenance? Do you fumigate the warehouse? What are the fumigants you use? ∞ How do you receive commodity from CRS warehouses to your warehouses? ∞ What are the terms and conditions with transport owners if commodities are lost while at their disposal? ∞ Have there been any losses of the commodity during storage and transfers? How the incidence of were reported to CRS? How the losses are being recovered? Reporting ∞ How are commodity reports prepared? ∞ What problems have occurred in report preparation and how have these been managed? ∞ What has been the response from CRS on commodity reports? Rations ∞ How are ration composition and quantities calculated? ∞ How have they changed over the life of the program? WALA Mid–Term Evaluation Report Annex E 122 ∞ How have rations been used by recipients? ∞ What changes should be considered relative to ration composition or quantities? Why should these changes be considered? Topical Outline - Human Resource Management The following topics should be covered in interviews with CRS (COP, DCOP and HR) and CATCH HR Staff. A. CRS Staff ∞ What are the total positions approved under the Cooperative Agreement between CRS and USAID? How many of the total positions are key positions? ∞ Have any positions been vacant for extended periods and, if so, why, and how has the program been affected? How soon the vacant positions will be filled in? ∞ In general, what has worked well and what has not worked well relative to staffing in the program? B. CATCH Partners Staff ∞ What are the total positions approved under the sub-award agreement with CRS? How many of the total positions are key positions? ∞ Have any positions been vacant for extended periods and, if so, why, and how has the program been affected? How soon the vacant positions will be filled in? ∞ In general, what has worked well and what has not worked well relative to staffing in the program? Topical Outline - Environmental Monitoring and Impact Mitigation Please refer to the WALA MYAP Initial Environmental Examination Environmental Mitigation Plan (Table 3 – Identifying Potential Impacts and Mitigation Measures). The Mitigation Plan described some specific potential environmental impact that may need to be monitored. The ESR for FY 2011 submitted along with the FY 2012 PREP provides a summary of the mitigation measures that the program implemented during the reporting year. The following topics should be covered in interviews with CRS, CATCH staff responsible for environmental monitoring and impact mitigation. ∞ What emerging or potential environmental impact is there from program activities? ∞ How does/will the program monitor this potential environmental impact? ∞ What capacity building activities have been undertaken with program participants or implementing partners around monitoring and mitigating environmental impact? WALA Mid–Term Evaluation Report Annex E 123 ANNEX D: Evaluation Report Format See the actual Report and Table of Contents WALA Mid-Term Evaluation Report Annex F 124 Mid-Term Evaluation Methodology for SO1 The methodology was qualitative, combining observation with focus group discussions (FGD) and key informant interviews. Findings were summarized at the end of each day and plans made to assure triangulation in other sites. Final analysis was conducted with MCHN staff participating. After studying the results of the annual survey some additional quantitative analysis validated certain findings. The qualitative methods and the objectives of each are listed below. Community Complementary Feeding and Learning Session – Observation Objectives: a. Discern adherence to implementation standards b. Assess CGV understanding of key concepts and capacity to independently facilitate effective CCFLS c. Assess effectiveness as a behavior change strategy (combined with questions for participants) d. Determine potential for replication or scaling up in other Title II programs Promoters – Individual interviews in every site Objectives: a. Assess understanding of basic MCHN concepts, messages and of approaches (CGs and CCLFS). b. Evaluate capacity to train and motivate CGVs c. Assess commitment and motivation for purposes of sustainability. Care Group Volunteers – FGD in every site Objectives: a. Assess capacity to promote behavior change through their activities – content and methods. b. Ascertain motivation and potential for sustainability c. Appraise WALA partner capacity to implement the CG model d. Assess understanding of basic MCHN messages Mothers who are targets of CGs – FGD in every site Objectives: a. Assess the efficacy of the project approaches (CG, CCLFS) on bringing about behavior change. b. Assess understanding of basic MCHN messages c. Evaluate the need for any improvements in implementation of the approaches to improve impact. Mothers who have participated in CCFLS – one FGD GVH per partner Objectives: a. Assess effectiveness of CCFLS in promoting behavior change. b. Discern acceptability of CCFLS to participants and challenges to their participation. Individual interviews as each site with HSAs, WALA field supervisors, families receiving food rations – at each site where available Observations of outputs: gardens, latrines, hand-washing stations, fuel efficient stoves The MCHN evaluator was accompanied on each site visit by the partner MCHN coordinator and field supervisors and, in one instance, by the program manager and, in another, by the M&E coordinator. Partner staff did not participate directly in the interviews or focus groups to prevent bias, but were invaluable in organizing villagers. MCHN staff from CATCH alternated in serving as translators, which was a helpful opportunity for them to hear directly from the beneficiaries and volunteers. Data Collection for SO1 took place over a period of 14 days encompassing a total of 32 WALA Mid-Term Evaluation Report Annex F 125 villages. WALA partners selected the sites after being advised to select some of their best and lower￾performing sites. Only in two sites was there overlap with the teams for the other SOs. January 20 and 21, 2012 Save the Children Districts: Zomba and Chiradzulu GVH and Village Method Participants Nakanda - Mlumbe Promoter Dorothy Msinkhu FGD Care Group Volunteers 13 volunteers, Dinah, Lorey, Chrissy Kande, Doreen Makwinja, Maureen Nsanama, Catherine Jailosi, Filesi, Rose Beyard, Sigele, June, Narjine, Lydia Yusuf, Rose Dama FGD Mothers benefitting from Care Groups 9 mothers: Catherine, Miriam, Sofina, Doreen, Lucy, Laine, Padroma, Mercy, Dorica HSA Lonely Phiri Katunga, Katunga FGD Care Group Volunteers 3 volunteers; Christina Chilamwe, Alice Moya, Dorothy Kaunda FGD Mothers benefitting from Care Groups 5 mothers: Beatrice Tausi, Edda Chinasi, Linez, Joyce Lydia Interview with promoter James Njowe Village Health Committee 8 of 10 members: Enifa Mpeta, Janet Lazalo, Mable Mbela, Joseph Phiri, Lucius White, Nicolos Gloss, Edda Chenache, Beatrice Tausi Interview HSA Agnes Zondenje Maleta - Nkhamula Interview Promoter Ruth Byson FGD Care Group Volunteers 4 volunteers: Yasinta Tayali, Veronica Ndala, Annie Chikombole, Mwandida Lino FGD Mothers benefitting from Care Groups 9 mothers: Lusia Kaliati, Maculata Misomali, Lucia Regina Said, Sakina Musa, Maria, Esther, Catherine, Esnot Interview Mother receiving Safety Net Ration Christina Nachuma Interview Mother receiving SFP Ration Katalina Clement Balakasi - Bakakasi Observation of fuel efficient stoves, fireless cooker, Solar dryer, garden FGD Care Group Volunteers 10 volunteers: Emma Mulima, Jelina Tamani, Edda Makina, Fainess Kaphera, Gladies Khulazingwe, Charity Katseko, Lucy Kamowa, Iness Manyamula, Merisy Sungani, Stella Makiyi FGD Village Health Committee 6 members: Henderson Maloko, Linda Mpawa, Loyde Maseya, Stella Makiyi, Anny Mphale, Grace Ali January 23 and 24 Total Land Care Nsanje District GVH and Village Method Participants Kalumbi, Kalumbi Interview of HSA Grace Nagamera FGD Village Health Committee 7 of 10 members: Mazoe, Somo, Cidric,Thigowen, Jussab, Rose, Foster, Noria, Verena, Severia Interview with promoter Bridget Master Interview with field supervisor Fwasani Silungwe FGD Care Group Volunteers 9 members: Ester Kandeya, Grace Zuze, Mary Kizito, Cecilia Masikochi, Sunganani Mindozo,Patricia Geoffrey, Eluby Chimwemwe, Hanna Kalumbi, Mary Makina FGD Mothers benefitting from Care Groups Sixteen mothers Observation of CCFLS session Eleven mothers participating FGD Mothers participating in CCFLS Eleven mothers; Elizabeth, Ivy, Marida, Lucia, Mary, Tadela, Ruth, Esther, Mary Berida, Maria Mbazo, Gatoma, Chapinga FGD Village Health Committee 8 of 10 members: Elizabeth, Ruth, Alcayanjono, Whitman, Violet, Esnot, Mary, Goodson WALA Mid-Term Evaluation Report Annex F 126 Interview with promoter Rose Makiyi Interview with HSA Naison Navindiyo FGD with Care Group Volunteers from 3 villages 23 volunteers from Bazo, Dembe, and Gatoma FGD Mothers benefitting from Care Groups 11 mothers: Lydia, Gladys, Myhastso, Mercy,, Mercy, Elizabeth, Dorin, Mwalwawo, Ruth, Patricia, Joyce FGD with mothers receiving SFP rations Eight mothers: Rose, Florida, Lydia, Mary Joyce, Fanny, Esther, Ferema, Ainafe Chapinga, Chapinga Care Group Volunteers/Support Group leaders 7 volunteers: Getrude Sikelo, Wonderford Wayawaya, Nema Maneya, Gladys Zakana, Chrissy Kapopo, Jennifer Botomu, Anastasia Lankeni FGD Village Health Committee 5 of 10 members: Patrick, Ericks, Bernadeta, Lameck, Stella Interview with field supervisor Emma Banda Interview with promoter Cecilia Mofolo January 25 and 16, 2010 Chikwawa Diocese Chikwawa District GVH and Village Method Participants Jombo, Hannock FGD Village Health Committee 3 of 10 members FGD Mothers who participated in CCFLS Six mothers: Elizabeth, Miriam, Martha, Mavis, Margaret, Mary Interview with HSA Peterkins Kabotolo Interview with promoter for Hannock and Washen Thomas Gunsaru FGD Care Group Volunteers 5 volunteers: Mary Mangani, Joyce Zuze, Leah Benito, Christina Mpungula, Geoffrey Kalidoza FGD Mothers benefitting from Care Groups 7 mothers: Joyce, Sofia, Dorothy, Mary, Kerita, Marita, Howa Jombo, Washen FGD Care Group Volunteers 5 volunteers: Kennedy Mbereko, Alfred Chadauka, Naomi, Fanny Goba, Felia Ngwali Observation of CCFLS 13 participants, 9 CGVs FGD Mothers benefitting from Care Groups 6 mothers: Beatrice, Ruth, Florence, Catherine, Irena Discussion with 2 fathers Kingsley, Patrick Njereza Sekera/Chipula Interview with recipient of Safety Net Rations Lucias Arason FGD Village Health Committee Six of ten members: Njawo, Harold, Emmanuel, Bernardo, Christina, Elizabeth Interview with mother receiving SFP rations Lucia Interview with HSA Alexander Chiwanda Interview with promoter Samson Beyard FGD Care Group Volunteers 11 volunteers: Victoria Thomas, Peter Kabwazu, Joyce Stanley, Delifa Jesitere, Annie, Josephine Tasala, Michael , Adam Njerekezi, Esta Lusiyasi, Irene Chaguluku FGD Mothers benefitting from Care Groups 12 mothers:Labeka, Martha, Monica, Luciana, Martina, Grace, Lucia, Evere, Rhoda, Magdalena, Maggie, Lamouse Observation CCFLS 10 mothers of malnourished children CKD Interviews with field supervisors Aaron, Banda, Brenda Mbayimibayi January 27 and 28, 2012 World Vision Thyolo District GVH and Village Method Participants Mkusa, Ligwelema Interview with promoter Kenneth Makaika FGD Care Group Volunteers 11 volunteers: Esther, Alice Kutani, Dorothy Mosolini, Tichichitenji Banda, Brenda Tipoti, Mary Mlauzi, Edina WALA Mid-Term Evaluation Report Annex F 127 Thumba Annie Chitaukali, Grace Majawa, Maphatso Laikedi FGD Mothers benefitting from Care Groups Nine mothers: Ruth, Jessie, Christina Mkanamwano, Evelin Kokoliko, Doreen, Grace Mafati, Elbe, Annie Listoni Individual interview with HSA Rose Mitengo Gunde, Bvumbwe FGD Village Health Committee Six members: Edina Likwenga, Terezina Magetsi, Emma Kachere, Winikesi Mayoyo, Flossy Sefasi, Catherine Chitsulo, FGD Care Group Volunteers 11 volunteers: Mary Naligonje, Catherine Chitsulo, Christina Masangwi, Elina Mpeketula, Edina Maulana, Tiyese Manuel, Chikondi Chiromo, Jessie Kamwendo, Rachael Kumsonga,, Jessie Kamwendo, Esnart Nyali FGD with CCFLS participating mothers Five mothers: Memory, Aisha, Memory, Gladys, Shadia Kabambe Chelewani Interview with HSA Magusaza Mangochi Interview with promoter Mabvuto Kambula FGD Care Group Volunteers 11 volunteers: Patricia, Eliza Bizwick,Tyanjani Halmitoni, Stella Mkolokosa, Grace Zakeyo, Ellen Kaulesi, Memory Langwani, Ellen Kaulesi, Grace Wellington, Dorothy, Samsoni Brazio FGD Mothers benefitting from Care Groups 16 mothers: Grace, Annie, Teresa, Catherine, Mary, Madalena, Martha, Felista, Mary, Dorothy, Alice, Mary, Aida, Brenda, Effa, Cecelia Kamazdi, Mberenga FGD Village Health Committee 4 members:: Violet Kachimanga, Mastanziyo Makwacha, Presley Zalira,Time Payere FGD Care Group Volunteers 12 volunteers: Richard Chiwaya , Celesta, Steria Nolomani, Lydia Kulinje, Mercy Beni, Janet Matimati, Mercy Stenala, Margaret Mmwala, Margaret, Eda Buleya, Lekeleni Thangala, , Annie Singano Observation CCFLS 233 participants (not a typo) FGD with CGVs facilitating CCFLS 11 of 28 CGVs January 30 and 31 Africare Malanje District GVH and Village Method Participants Chibade Namasalima Interview with promoter Jones Mambala FGD Care Group Volunteers 7 volunteers: Enefesi Chifunga, Rose Lupala, Zaina Lihoma, Caroline William, Florence Baison, Gertrude Lipato, Treza Ndini FGD Village Health Committee 9 members: Bernard, Lekuwa, Aida Mailosi, Alefa Daniel, Reseby Chimwaza, Gertrude Maulidi,Mercy Magulu, Leason Kathonje, Mary Mulanje, Odetta Tsamila FGD Mothers benefitting from Care Groups 9 mothers: Prisca Chikungwa, Mercy Wanda, Stella Muliba, Alefa Mamba, Grace, Thonyiwa, Mercy Frank, Catherine Makwete, Fanny Khasu Duswa Kaliza FGD Mothers benefitting from Care Groups 10 mothers: Violet Bamu, Julita, Khaiya, Odetta Nasolo, Mary Pensulo,Lucy Afick, ElubyNnamidiya, Stella Gwagwala, Maloya Mafoya, Stella Sozibefe, Enifa Magombo FGD Care Group Volunteers 11 volunteers: Ethel Bondo, Beauty Saopa, Edina Lundo, Felista Kaunde, Mercy Katalama, Olive Makwete, Joyce Ngalande, Luwiza Kumsuku, Lucy Kopolandi, Bertha Ngalande Stella Mangala Interview with promoter Dawson Manja Interview with HSA Charles Muhiwa Majiya Mthilamanja FGD Mothers who participated in CCFLS 10 mothers: Chrissie Lameck, Rose Alfred, Malita Jemusi, Irine Dayson, Margaret Kennedy, Modesta WALA Mid-Term Evaluation Report Annex F 128 Blackstone, Evelyn Maliama, Eliza walunda, Alinafe Jefrey, Edina Mwehiwa FGD Care Group Volunteers 10 volunteers: Mary Mdoka, Alice Kilini, Cecilia Amosi, Esnat Kongola, Bertha Likeke, Abgirl Uledi, Margaret White, Patuma Mofati, Patricia Mbozi, Rose Mbondo Interview with HSA Konelio Chistsamlu Interview with promoter Angela Nmangisa FGD with SFP beneficiaries Falesi Nathaya, Zinemnani Dube, Lizineti Eneya, Aida Muliya, Ndaziona Charles, Aida Mwamyali, Monica Chtseko, Anne Fnima FGD with Safety Net beneficiaries Lailo Charles (OVC HH), Monica Chisekwe (PLWA), Falesi Nathaya (elderly) FGD Village Health Committee 7 members of 10: Humphrey Mbozi, Lucius Rodrick, Jenet Stephano, Lone Taulo, Rose Jemusi, Agnes Mateyu, Daniel Masikini Chikumbu Namputo FGD Care Group Volunteers 10 volunteers: Mary Beneti, Rose Nkondowe, Catherine Roben, Nolia Sitole, Jenifer Nansongole, Treza Jackson, Fanny Adson, Agnes Sambani, Ndaona Kalimba, Tryness Phapha Health Promoter Gertrude Pangani FGD Mothers benefitting from Care Groups 11 participants: Malita Dangwe, Hilda Nangwiya, Martha Nagoli, Olive Josaya, Esnat Chilomo, Rose Namaona, Sevelia Bizwick, Eliza Kankhomba, Agnes Mbendela, Fanny Muyela, Catherine Bizwick FGD Village Health Committee 12 members: Rosemary Salima, Rose Majanja, Sungeni Maseya, Mercy Limphoni, Enifa Qwatira, Nelli Limphom, Fanny Saidi, Margret Mikwamba, Lufaness Gawani, Allan Wisca, Vasco Mondiwa, Livingston Better Interview with HSA Peter Joseph February 1 and 2, 2012 Emmanuel International Zomba and Machinga Districts GVH and Village Method Participants Chitenjere, Zomba Interview with 2 HSAs Chifundo Bobo, Modesta Chikwanje FGD with Village Health Committee 9 members: Elise Kasupe, Estery wanja, Agatha Kalonga, Dyna Kaunda, Ismmael Daisa, Esnart Milanzie, Douglas Mtapila, Enifa Salade, Hilda Phiri Interview with promoter Mina Bamsi Seasonal calendar process Lumbe, Machinjiri Zomba FGD with Care Group Volunteers 9 volunteers: Margret Msoma, Esnart Aufi, Edina Sinoya, Eunice Madoni, Esterly Pichesi, Patuma Dailesi, Rose Kaombe FGD with participants in CCFLS Sara Mphepo, Margret Green, Dyna Maganga, HannifaMagombo, Miriam Maloya, Mal Jafali, Patuma Banda, Linesi Yusufu, Charity Gilimoni, Alinafe Amidu Observation of CCFLS FGD Safety Net 4 chronically ill and 3 elderly women Idali Kawinga, Makwemba, Machinga FGD Care Group Volunteers 10 volunteers: Mdhepo Luwa, Angelina Chigwere, Pakombwere Sugar, Mary Siyamo, Fairness, Juma, Patuma Jefule, Esagit Audiolhi, Luckia Symani, Malegur Kenneth, Zione Simioni, Interview with HSA Nelso Kapiti Observation of CCFLS FGD with CCFLS participants 15 participants: Mina Yusufu, Alesi Sinoden, Esnart Meja, Hamida Yusufu, Leaveness Imulani, Maliam Laisan, Margret Sinoya, Jenet Alfred, Edina Chidothi, Jenfa Manesi, Mbirire Steven, labia Makufi, Solofe Ojesi, WALA Mid-Term Evaluation Report Annex F 129 Divasyeje Faison, Loster Mwente Interview with two promoters Dison Danja, Luke Muloma FGD with Safety Net Mina Yusufu, Enelesi Yamikani, Fidesi Mahata, Felia Mauro, Ulerina Nsonkho, Tuweni Makiyi ( most are widows) FGD Mothers benefitting from Care Groups 11 women: Jenifa, Jolichi, Maliam Jafali, Asiatu Banet, Mercy Fanesi, Ester Siani, Asiatu Belo, Elliot Meenhula, Annie Gayesi, Dolofe, Chalamaoa, Esnart James, Violet Tibu (men were gone to lake to fish or ganyu) Ling’ole, Kawinga, Machinga FGD with Care Group Volunteers 9 volunteers: Catherine Yakobe, Emily Chikopa, Martha Labisoni, Ethel John, Kenneth Ronald edith Kazembe, Florence Matiki, Elube Benison, Sumaisi, Kachingwe FGD Mothers benefitting from Care Groups 8 mothers: Lukia Chikumba, Patuma Omali, Zalia Kasimu, Aisha Sinoya, Margret Liwonde, Rhoda Lyson, Esimme Smart, Fatima Lameki FGD Village Health Committee 8 members: Lawrance Major, Juweni Roan, Agnes Paulo, Kenneth Ronald, Marth Rabson, Emily Chuopa, Ethel John, Catherine Yakdbe Interview with promoter Sinoya Twarich February 3 and 4, Project Concern International, Balaka and Machinga Districts GVH and Village Method Participants Mawila, Kamboure, Balaka Interview with HSA Gabriel Banda FGD Mothers benefitting from Care Groups M. Willo, M. Mpinji, M. Mahikelon, M. Rajabu, M.Jjosephy, Dolophy Adamm, Mariam Hajahon, Esme Kulemeks, M Lipenga, M. Mkwanda FGD with Care Group Volunteers from six different villages Elizabeth Tenge, Linily Mbededza, Esterly Ulanda, Suzan Kilout, Esnert Mataka, Edmes Kakhomba, Lucy Kachak, Hilda Bima, Fanny Fubianio, Ellen Kambuku Interview with health promoter Gerald Mwarabo FGD Village Health Committee May Mowecha, Ermie Kuloneke, Mullium Lajob, Elizabeth Tenge, Elizabeth Mwolbi, Aniya Bwonali Zalengera Balaka Interview with promoter Onex Bwanari FGD with Care Group Volunteers 10 volunteers: Meria Maonga, Mery Kamanlia, Severia Anthuachinu, Lovenes Kamena, Margret Jamaci, Snale Pankuku, Rhoda Frarion, Jenet Jonas, Agnes Chabwela, Edna Mbenaera FGD Mothers benefitting from Care Groups Miriam Chikapusa, Rhda Kutsata, maria Waiti, Mary Komakoma, Alice Roben, Falesi Jonasi, Estin Lyson, Lonny Soko, Synolia Pias, Sirireni Kamwendo, Kumbukani Jimu, Emeresi, Zak aria, Iness Peter, Hawa Kayenda Interview with SFP beneficiary Dalesi Katundu Interview with Safety Net beneficiary Stevena Bokosala – elderly with OVC FGD with Village Health Committee 7 members: Shaleleni Kanyakata, Olive Kasambere, Grace jenns, Fyness Kambiri, Jenipher Palankuku, Cathereen Nanjiri, Annet Kapusa Nkalo, Mkabwire, Machinga Interview with HSA Loveness Matipuiri Interview with promoter Annie Biriyati FGD Care Group Volunteers 10 volunteers: Eluby Bakali, Tereza Caponda, Esnart Taibu, Gladys Mtambo, Samuel Linera, Violet Gomani, Doreen Gracium, Estebe Chikaonda, Mina John, Zinenani Stanley FGD Village Health Committee 5 members: Anderson Nyambalo, Ajiki Mpasidwa, Doreen Gracium, Violet Gomani, Edson Chatayika FGD Mothers benefitting from Care Groups Tsala Sumayo, Chimwenwe William, Severia Adamu, WALA Mid-Term Evaluation Report Annex F 130 Rose Matenje, Viana Kholomana, Lidia Twaibu, Chisomo Mantu, Matilda Makwinja, Ellena Samson Makabwire, Machinga Observation of CCFLS M’manga Simbora Marketing group 5 members FGD Care Group Volunteers 7 volunteers: Sinofi Mtendoe, Ellen Kawenga, Florence Winadoni, Mike Kambalazaza, Esnart Jimu, Patricia NKhalango, Aaziana Jorname FGD Mothers benefitting from Care Groups Mary chimwere, Faines Ngoma, Doris Gunda, Amina Juma, Sevelia Parl, Emile Austin, Agnes Hattami Interview with Promoter who works with both SO1 and SO2 Chrissy Kameyana February 7: Analysis, Validation and Action Planning Participants: Catherine Chiphazi, Caeser Kachale, Kanji Nyambo and Hazel Simpson – CATCH MCHN coordinators: Joylet Grenda – Chikwawa, Wakisa Kachale –TLC, Doris Mphande –SC, Hellen Phalaza – Africare, Scholastica Mkandawire - PCI Others: TLC Program Manager Angela, World Vision - Kennedy WALA Mid-Term Evaluation Report Annex F 131 SO2 - ANRM. Program Management, Knowledge Management Methodological Changes from the Evaluation Plan In the evaluation plan (see Annex E), it was envisioned that the following process would be used for two days of SO2 qualitative data collection with each implementing partner: In at least two villages:  Focus Group Discussion with Lead Farmer/Producer Group (1.5 hours)  Focus Group Discussion with a VSL Group that has completed at least one cycle (1.5 hours)  Focus Group Discussion with a Marketing Club (1.5 hours)  Individual Interview with a Farmer Extension Facilitator (1 hour)  Individual Interview with a Community Agent (1 hour)  Individual Interview with an Agri-business Community Agent (1 hour)  Focus Group Discussion with a Water Users Committee(1.5 hours) Over the course of the two days:  Focus Group Discussion with a Watershed Management Committee (1.5 hours)  Site Visits to two irrigation sites  Site Visits to five demonstration sites showing different technologies  Site visits to five FFW project sites (can be combined with irrigation site visits and can also be covered with short stops enroute between villages) The actual process used for each implementing partners was as follows:  Focus Group Discussion in two to four villages with a group of SO2 participants, including a Lead Farmer/Producer Group, a VSL Group that has completed at least one cycle and a Marketing Club  Joint Interviews with a Farmer Extension Facilitator, a Community Agent and an Agri￾business Community Agent  Focus Group Discussion with a Water Users Committee and visit to an irrigation system  Focus Group Discussion with a Watershed Management Committee and a visit to a watershed development site  Site Visits to three demonstration sites showing different technologies  Site visits to three FFW project sites (usually combined with irrigation site visits and watershed development site visits) The following specific locations were visited: Nkanga Village, Nkanda GVH, Mlumbe TA, Zomba District (Save the Children) Discussion with twenty-five participants all female, representing four Producer Groups, four VSL Groups and six Marketing Clubs Interviews with an 3 FEFs, 2 CAs and an ACA Site visit to see conservation agriculture demonstrations Mbeluwa Village, Mbeluwa GVH, , Mlumbe TA, Zomba District (Save the Children) WALA Mid-Term Evaluation Report Annex F 132 Discussion with thirty-four participants, representing Producer Groups, VSL Groups, Marketing Clubs, and a Water Users Groups with all respondents also participating in FFW Interviews with an FEF, CA and ACA Site visit to see conservation agriculture demonstrations and early adopters Unable to visit irrigation sit due to time and bad weather Chiriwani Village, Chiradzulu District (Save the Children) Discussion with sixteen participants, representing Producer Groups, VSL Groups and Marketing Clubs, with all respondents also participating in FFW Interview with an FEF Site visit to see conservation agriculture demonstrations Balakasi Village, Balakasi GVH, Ntchema TA, Chiradzulu District (Save the Children) Discussion with thirty-seven participants, representing Producer Groups, VSL Groups and Marketing Clubs, and a Water Users Group Interview with an GVH Headman Interviews with an FEF and ACA Site visit to see conservation agriculture demonstrations Mbangu Village, Mbangu GVH, Malemia TA, Nsanje District (Total Land Care) Discussion with thirty-two participants, representing Producer Groups, VSL Groups, Marketing Clubs, and a Water Users Group Interviews with an FEF, CA and ACA Site visit to see conservation agriculture demonstrations Site visit to an irrigation site Gatoma GVH, Mlolo TA, Nsanje District (Total Land Care) Discussion with a Watershed Development Group Discussion with eighteen participants, representing Producer Groups, VSL Groups and Marketing Clubs Interviews with an FEF CA and ACA Site visit to see conservation agriculture demonstrations Site visit to a watershed development site Mbelengezi Village, Nyambalo GVH, Makhwira TA, Chikwawa District (Chikwawa Diocese) Discussion with seven participants, representing Producer Groups, VSL Groups and Marketing Clubs Interviews with an FEF CA and ACA Site visit to see conservation agriculture demonstrations Mtambo Village, Jana GVH, Makhwira TA, Chikwawa District (Chikwawa Diocese) Discussion with thirty-one participants, representing Producer Groups, VSL Groups, Marketing Clubs and a Water Users Group Interviews with an FEF, two CAs and two ACAs A separate interview with four CAs participating in the PSP activity Interview with a Village Headman Site visit to see conservation agriculture demonstrations Palato Village, Chavala GVH, Chavala TA, Chikwawa District (Chikwawa Diocese) WALA Mid-Term Evaluation Report Annex F 133 Discussion with sixty-two participants, representing Producer Groups, VSL Groups, Marketing Clubs, a Watershed Development Group and a Water Users Group Interviews with three combined FEF/ACA and a CA Site visit to see conservation agriculture demonstrations Site visit to see watershed development demonstrations Unable to visit irrigation site due to time Nsudwa Village, Kwethemule TA, Thyolo District (World Vision) Discussion with fourteen participants from a Water Users Group, but also representing Producer Groups, VSL Groups, Marketing Clubs, and a Watershed Development Group Site visit to an irrigation site Discussion with approximately eighty participants, representing Producer Groups, VSL Groups, Marketing Clubs, and a Watershed Development Group Interviews with a GVH Headman and Head of a CBO established with thesupport of the Tea estate Site visit to see conservation agriculture demonstrations Kumadzi Village, Kapichi TA, Thyolo District (World Vision) Discussion with twelve participants from a Water Users Group, but also representing Producer Groups, VSL Groups, and Marketing Clubs Site visit to an irrigation site Kabambe Village, Kabambe GVH, Mchoramwela TA, Thyolo District (World Vision) Discussion with sixty-three participants representing Producer Groups, VSL Groups, and Marketing Clubs Interviews with five FEVs, two ACAs and three CAs Site visit to see conservation agriculture demonstrations Mthilamanja GVH, Mthilamana TA, Mulanje District (Africare) Discussion with approximately seventy participants, representing Producer Groups, VSL Groups, and Marketing Clubs Interview with a GVH Headman Interviews with one FEF, one ACA and two CAs Site visit to see conservation agriculture demonstrations Mbewa Village, Tchete GVH, Mabuka TA, Mulanje District (Africare) Discussion with approximately forty participants, primarily from a ater Users Groups but also representing Producer Groups, VSL Groups, and Marketing Clubs Interview with a GVH Headwoman Interviews with an ACA and a CA Site visit to see conservation agriculture demonstrations Site visit to an irrigation site Robeni GVH, Chikumbu TA, Mulanje District (Africare) Discussion with fifteen participants from a Watershed Management Committee Site visit to see watershed development demonstrations Mwamadi Village, Robeni GVH, Chikumbu TA, Mulanje District (Africare) WALA Mid-Term Evaluation Report Annex F 134 Discussion with tweny-one participants from a Village Grain Bank Groups, including the Village Headman. Interviews with two FEVs, four ACAs and two CAs Nyamuka Village, Togo GVH, Zomba District (Emanuel International) Discussion with nineteen participants from a Watershed Management Committee Site visit to see watershed development demonstrations Chilonga Village, Malemia TA, Zomba District (Emanuel International) Discussion with twenty-two participants from a VSL Group Site visit to see a VSL FFP fishpond Interviews with two FEFs, an ACA and a CA Chibwana Village, Chibwana GVH, Mlomba TA, Machinga District (Emanuel International) Discussion with fifteen participants representing VSL groups, producer groups, markeing clubs and a water users group Interview with a GVH headman Interviews with an FEF, an ACA and a CA Site visit to see conservation agriculture demonstrations and demonstrations of new varieties Masambaka Village, Mtadila GVH, Machinga District (Emanuel International) Discussion with twenty-two participants from a Water Users Group Site visit to an irrigation site Makhoye Village, Mbosongwe GVH, Kawinga TA, Machinga District (Emanuel International) Discussion with approximately fifty participants representing a VSL group, producer groups, and marketing clubs Interview with a GVH headman and a village headman Interviews with an FEF, an ACA and two CAs Site visit to a VSL goat project Sungaleke Village, Balaka District (Project Concern Internaional) Discussion with approximately thirty participants representing Producers Groups, VSL Groups, Marketing Clubs and a Watershed Development Committee. Interview with a village headman Site visit to a conservation agriculture demonstration Site visit to watershed development demonstrations Mtumbwe GVH, Kachenga TA, Balaka District (Project Concern Internaional) Discussion with approximately thirty participants representing Producers Groups, VSL Groups and Marketing Clubs. Interview with a GVH headman Interviews with an FEF , an ACA and a CA. Site visit to a conservation agriculture demonstration and demonstrations of new varieties Kumwima Village, Malajira GVH, Nkula TA, Machinga District (Project Concern Internaional) Discussion with approximately forty participants representing Producers Groups, VSL Groups, Marketing Clubs and a Water Users Group Interview with a GVH headman and three village headmen WALA Mid-Term Evaluation Report Annex F 135 Interviews with an FEF , an ACA and a CA. Site visit to a conservation agriculture demonstration Site visit to an irrigation site Peter Kasanga Village, Mkweta GVH, Kalembo TA, Balaka District (Project Concern Internaional) Discussion with twenty-three members of a VSL Group In addition, the following additional interviews were conducted sometimes formally and other times enroute to village visits: Save the Children: Matthew, James & Emmanuel TLC: Zwide, Angela, Gift, Ayton and Stanley Chikwawa Diocese: Father Mateo, Nicholas, Lingston, Loyce World Vision: Marko, Hilda, Thokozani, Gideon & Mayeso Africare: Hyghten, Maggie, Absalom, Innocent, Warings, Leo & Madalitso Emmanuel International: Jones, Thomas, Yobu, Luke & Simon Project Concern International: Tim, Gideon, Bahati, Damson & Imlan CATCH: Shane, Jonathan, Juma, Solani, Gitau, David M, David (new staff), Jay, Isaac, Owen and Abel CRS Leadership: Amy Agricane: Matthew, Oliver & Alex ExAgris: Charles WALA Mid-Term Evaluation Report Annex F 136 SO3 - DRR. Governance, Environment, Finance, Commodities & HR Interviews were held at the district level with representatives of the District Executive Committees and District Civil Protection Committees in the following districts Zomba District Chiradzulu District Nsanje District Chikwawa District Thyolo District Mulanje District Machinga District Balaka District Interviews were held with TA Chiefs and Area Civil Protection Committees in the following Traditional Authorities Mlumbe TA Ntchema TA Mlolo TA Malemia TA Makhwira TA Kwethemule TA Namasalima TA Mthilamanja TA Kawinga TA Mpilisi TDC Interviews were held with Village Civil Protection Committees and Focus Group Discussions were held with food recipients in the following Villages or GVHs Nkanga Village Mbeluwa Village Chiriwani Village Balakasi Village Gatoma Village Mbangu Village Mtambo Village Njereza Village Palato Village Kwethemule Village Amos Village Kabambe Village Mthilamanja Village Robeni Village Kachingwe Village Nyamuka Village Lumbe Village William Village WALA Mid-Term Evaluation Report Annex F 137 In addition, the following interviews were held: District Agricultural Development Officer and District Health Officer, Nsanje District Finance managers with each partner where available Commodity managers and warehouse managers, with site visits to warehouses Human resource managers were possible CATCH Finance and Commodity Management Teams WALA Mid-Term Evaluation Report Annex G 138 MCNH Summary Table (Cumulative totals) by Partner (September 2011) PVO TAs GHVs Villages Field Super￾visors Promoters CGs CGVs MCHN Beneficiary households SFP Benefi￾ciaries Africare 4 32 156 4 21 120 1,590 15,900 318 CKD 6 37 228 5 25 161 1,764 20,143 49 EI 8 33 317 3 30 214 1,900 15,465 838 PCI 5 36 371 4 28 222 2,107 22,897 1,735 SAVE 5 30 396 4 28 161 1,769 19,173 943 TLC 5 39 320 5 23 155 1,640 15,930 1,202 WVM 7 20 109 5 35 213 2,840 29,101 1,267 TOTAL 40 227 1,897 30 178 1,246 13, 610 138,609 6,352 WALA Mid-Term Evaluation Report ANNEX H 139 Statistics by Partner for SO2 Table H-1. Farmer Extension Facilitators, Lead Farmers and Producer Groups as of the End of December 2011 Partner GVHs FEFs or FEVs FEF per GVH Lead Farmers &Producer Groups Groups per FEF Farmers Average Group Size Farmer per FEF Africare 33 35 1.1 855 24 17,771 21 508 Chikwawa Diocese 27 23 0.9 850 37 17,850 21 776 Emmanuel International 33 30 0.9 516 17 8,202 16 273 Project Concern International 36 24 0.7 1,478 84 23, 648 16 985 Save the Children 30 19 0.6 727 38 17,523 24 922 Total Land Care 39 31 1.3 768 25 14,843 16 479 World Vision 20 91 4.6 778 9 16,338 21 180 Program 218 253 1.2 5,972 24 116,175 19 459 Table H-2. Community Agents & VSL Groups as of the end of December 2011 Partner GVHs CAs CAs per GVH VSL Groups Groups per CA VSL Members Average Group Size Member per CA Africare 33 59 1.8 272 5 5054 19 86 Chikwawa Diocese 27 35 1.3 354 10 6,667 19 190 Emmanuel International 33 47 1.4 293 7 6,077 19 138 Project Concern International 36 27 0.8 296 11 5,548 19 205 Save the Children 30 38 1.3 249 7 4,219 17 111 Total Land Care 39 31 1.3 246 8 4,177 17 135 World Vision 20 67 3.4 477 7 5,865 12 88 Program 218 304 1.4 2187 7 37,607 17 124 WALA Mid-Term Evaluation Report ANNEX H 140 Table H-3. Agri-Business Community Agents & Marketing Clusters Partner GVHs ACAs ACAs per GVH Market Clusters Clusters per ACA Cluster Members Average Group Size Member per ACA Africare 33 35 1.1 35 1 4,329 124 124 Chikwawa Diocese 27 30 1.1 30 1 2,262 75 75 Emmanuel International 33 35 1.1 35 1 4,448 127 127 Project Concern International 36 17 0.5 17 1 1,834 108 108 Save the Children 30 45 1.5 45 1 4,752 106 106 Total Land Care 39 37 1.3 45 1 4,150 92 92 World Vision 20 39 2.0 39 1 4,187 107 107 Program 218 238 1.1 246 1 25,962 105 109 WALA Mid-Term Evaluation Report Annex I 141 Knowledge In - Partners and Consultancy Support Received by WALA No. Partner/Consultant Nature of work done/support provided Sector Involved 1 Bunda College/Chancellor College Develop training manuals for food processing and preservation and facilitate nutrition trainings and demonstrations; Provision of consultancy on seed systems MCHN & Agriculture & NRM 2 Department of Agricultural Research Services (Bvumbwe, Makoka, Lifuwu and Chitedze research stations) Developing technologies for vegetables, sweet potatoes, pigeon peas (Bvumbwe); cassava (Makoka) and rice (Lifuwu); joint seed security systems assessment with Seed Services at Chitedze & government Extension staff Agriculture and NRM 3 Chemicals and Marketing Limited Conduct post-harvest & Safe use of chemical trainings for WALA technical staff Agriculture and NRM & Agribusiness 4 Conservation Farming Unit of Zambia (CFU) Provides technical support and staff trainings on conservation agriculture. Agriculture and NRM 5 Water Organisation Trust of India (WOTR) Provide technical assistance on micro-watershed development, facilitate exchange visits and training of technical staff on watershed development Agriculture and NRM 6 Agricane Help assess the suitability of identified sites for irrigation and design suitable irrigation systems for the sites. Provide technical guidance to PVOs during scheme construction. Irrigation 7 Department of Animal Health and Livestock Development Develop a paravet training manual as well as conducting livestock trainings Livestock 8 National Aquaculture Centre/Department of Fisheries Develop the fisheries training manual and facilitate aquaculture trainings in the PVOs Livestock 9 Department of Disaster Management Affairs (DoDMA) Participate in DoDMA meetings and share any DRR related information DRR 10 Malawi Vulnerability Assessment Committee (MVAC) WALA is a member of MVAC DRR 11 Famine Early Warning Systems Network (FEWS NET) Participate in FEWS NET meetings and share information DRR 12 Esoko & Agricultural Commodity Exchange (ACE) Provide market related information to farmers Agribusiness 13 International Potato Centre (CIP) Developing technologies for orange fleshed sweet potatoes including seed multiplication and distribution, & processing and value addition Agriculture and NRM 14 C:AVA Project at Chancellor College Cassava processing, value addition, & marketing Agriculture and NRM & Agribusiness 15 Pesticides Board of Malawi Training on safe use of agricultural chemicals Agriculture and NRM 16 Ministry of Agriculture, Department of Agricultural Working together at PVO level to deliver messages; & involved in Agriculture and NRM WALA Mid-Term Evaluation Report Annex I 142 Extension Services planning meetings, trainings etc 17 Exagris Africa Ltd Extension support for chillies outgrowers Agribusiness & Agriculture 18 Chibuku Products Seed for red sorghum outgrowers Agribusiness 19 Farmers Voice Radio Extension support through radio and sms text messaging Agribusiness 20 Universal Farming & Milling Ltd Technical support for development of dry cassava buying quality guide Agribusiness 21 Dr. TD Jose, CRS, EARO Development of Quality Improvement Verification Checklist (QIVC) Monitoring, Evaluation and Knowledge Management 22 FANTA Finalization of IPTT and PMP Monitoring, Evaluation and Knowledge Management 23 Mr Melkamu Dereb, CRS, Burundi Training on commodity reporting software (Food Log) CATCH and PVO commodity accountants 24 Mr Roges Chilemba Development of Maternal Nutrition Module imagery MCHN 25 District Health Offices MCHN MCHN 26 Office of the President and Cabinet HIV/AIDS Advisor HIV/AIDS MCHN 27 International Cross Visit to Zambia Conservation Agriculture Agriculture 28 CRS Zimbabwe Commodities and HR Commodities and HR 29 CRS Madagascar Environmental monitoring Environment 30 Martin Hartney Training on Title II Resource Management Finance & Program Managers 31 MANGO Financial Management Program Managers WALA-Mid Term Evaluation Report Annex J 143 OBSERVATIONS AND SUGGESTIONS FOR THE IPTT AND ITT The following sections offer some suggestions for improving the performance indicators that are shown in either the Indicator Performance Tracking Table or the new Indicator Tracking Table. Indicators Related to SO1 • 1.1.1 Revise wording to reflect data collection during the past 24 hours • 1.1.2 Good indicator but also look at changes in consumption for each of the 12 food categories • 1.1.3 Eliminate correct use of LLIT bed nets as an option. This is not environmental sanitation and is being promoted by others in the target area rather than WALA. It could be a separate indicator in the ITT • 1.1.4 Eliminate solar dryers as an option because this is not a fuel efficient technology. • 1.2.1 Define regularly attending in the indicator or explanation of the n. • 1.2.2 Explain whether this is mother’s recall or card. • 1.2.2. If the government is going to drop providing Vitamin A supplements to post-partum women per WHO guidance issued in March 2011, then, this indicator should be dropped. • 1.2.3 This indicator could be eliminated. WALA does not have a focus on maternal health so achievement cannot be attributed to WALA but rather to government policy. Indicators Related to SO2 Indicator 2.2 on Diet Diversity, there are other factors that affect performance on this indicator in addition to agricultural production, finance and marketing, notably the quality of the nutrition education under SO1, which is why many programs place this indicator at the goal level. IPTT Indicators 2.1.1a and 2.1.1b on sustainable crop cultivation technologies and soil conservation technologies, it is not clear why there should be two indicators since they are so interrelated....sustainable crop cultivation requires soil conservation. In addition, for the five components in the sustainable crop cultivation indicator, WALA is doing very little on seed supply, many farmers already know about crop rotation particularly using legumes, similarly, pigeon pea and ground nuts are commonly intercropped in traditional cultivation systems, minimum tillage can be many things and mulching is one part of minimum tillage. Similarly for the three soil conservation technologies, fertilizer trees is okay, a farmer should not have to do all of contour ridges, box ridges and bunds (stone or contour ridges?) to qualify and vetiver grass may not be the best intervention. The idea is not that farmers are doing exactly what the program tells them (especially if what we tell them has only limited benefit such as vetiver grass), but that they understand the concepts well enough to apply them. The key concepts for sustainable crop cultivation are that they are using contours to capture and hold water, mitigating soil erosion and restoring soil fertility. The program should decide which techniques represent good practice in these areas, and not just focus on those being promoted by the program. If the program promotes the idea and farmers understand the idea well enough to even find their own technique, then that should be considered a success. IPTT Indicator 2.1.1.c. on integrated pest management, since the program is not really doing much to present new ideas and techniques on mechanical control of pests, cultivation control of pests, crop rotation forpest control, use of organic pest control products, any movement on this indicator is likely to WALA-Mid Term Evaluation Report Annex J 144 be the result of other interventions, not WALA. The program should either scale up IPM or negotiate this indicator out of the IPTT. IPTT Indicator 2.1.3. on irrigation sites is described as an outcome indicator implying that it captures behavioral change induced by the program. The elements of the indicator, however, are all structures that the program provides support to build. The only element that may have behavioral implications is vegetative cover which requires pariciants to plant on their own. This indicator should be either dropped r made into an output indicator. The next indicator (2.1.4.) actually captures the outcome of the program. Missing Output Indicators: The ITT does a good job of including additional output indicators related to producers groups and water users groups, but the program should also include similar output indicators for watershed development groups and livestock activities. In the IPTT, the savings indicator 2.2.1 captures only a part of the impact of the program. The VSL daabase has good information on member equity that captures a lot more of the impact from group investments and shareouts. In the ITT under VSL, there should be output and outcome indicators related to the Private Service Provider activity, including the number of PSPs who have been certified (output) and the number actually providing services to VSL groups as PSPs (outcome). The ITT should also include an indicator to capture other VSL groups that have formed with CA assistance but outside of the official WALA groups, i.e., the diffusion of the intervention. Indicator 2.3.1. is a somewhat odd indicator to capture the outcome of WALA's marketing interventions. It implies that the program is doing the marketing analyses to identify what farmers should produce and then success is measured by whether the farmers are producing these crops. That is not really the current strategy of WALA which is oriented more around forming groups, training them on markets and market opportunities analysis, training them to mobilize and manage cash resources, and enabing them to develop their own plans to find markets. The new indicators in the ITT (2.3.2 and 2.3.3 are aligned more with the WALA strategy. Indicator 2.3.1. is not really measuring what the program wants to do. Success is measured not by farmers producing that the program tells them to, but by farmers developing and implementing their own marketing plans. Indicator B-5 in the ITT which describes total sales from collective marketing, may not be a very useful indicator for being able to assess program impact. The benefit from collective marketing comes from better weighing of produce and better negotiation of prices. The amount a group has sold collectively is less important than the number of farmers who sell collectively and the difference in the price received, adjusting for transport costs. Ten large farmers, for example, can sell a huge amount collectively but realize little marginal benefit fromthe program since they may be getting a fair price in any case and, because they are large, can negotiate effectively. Another group of fifty farmers, for example, sell much less, but because they are small farmers with little negotiation power, the impact ofthe program on them is much greater. When the final evaluation is undertaken, the evaluators may review the program document to understand what the program proposed to do and then compare that to what the program has actually done. A review of the program proposal from this perspective highlights the following as potential WALA-Mid Term Evaluation Report Annex J 145 problem areas for WALA because either there are no activities planned in WALA to achieve the impact or there are no indicators in the ITT to capture impact being achieved:  Expanded lead farmer capacities to propagate new ideas and technologies (listed as a technical area in which the program would work)  Strengthened linkages with input suppliers, especially seed and fertilizer (listed as both a technical area & an asset accumulation barrier that would be addressed by WALA)  Improved access to natural resources, especially timber and non-timber forest products (listed as an asset accumulation barrier)  Reduced reliance on ganyu as a major source of income (listed as an asset accumulation barrier) The program is certainly having impact on dependence on ganyu, but the M&E systems are not capturing these changes. Indicators Related to SO3 Indicator 3.1.2 on project communities with improved infrastructure is presumably supposed to capture the impact of FFW activities. It is not at all clear where the targets or performance numbers are coming from since there are at least 130 FFW/FFA sites in many more than 45 communities. This indicator should be revised so that it conforms more closely with the WALA records on FFW. As recommended in the main text, output indicators should be developed to be able to monitor the higher level linkages being developed by WALA (see page 35). Indicator 3.2.1 describes food recipients, but there are many different kinds of food recipients in WALA, including those receiving food through the SFP, those receiving safety net food and those receiving FFW food. It would be useful to have this indicator disaggregated into these different groups. WALA Mid-Term Evaluation Report Annex K 146 Resource Summary Tables Table K-1. Cash Expenditure Summary Projected Through December 2011 (US$) Cost Center Monetization Proceeds 202e ITSH Cost Share Total Personnel salary and Benefits 1,692,745 1,187,450 472,926 10,031 3,363,152 Capital Equipment over $500 145,425 541,690 33,528 --- 720,643 Materials under $5000 326,514 253,542 98,327 32,353 710,736 Consultancy & Technical Assistance 1,952,783 174,539 (1,561) 29,496 2,155,257 Travel& Transport 167,253 35,734 4,313 22,254 229,554 Training & Workshops 65,845 10,748 1,648 21,571 99,812 Subcontract/Sub grants 5,642,206 3,971,795 1,359,148 238,525 11,211,674 Warehousing Expenses 458,985 458,985 Other Direct Costs 1,426,429 248,713 131,066 29,882 1,836,090 Total Direct Costs 11,419,201 6,424,212 2,558,380 384,112 20,785,905 NICRA for CRS 900,403 447,339 --- 20,543 1,368,285 TOTAL Expenses through December 2011 12,319,604 6,871,551 2,558,380 404,655 22,154,190 TOTAL LOA Budget at Time of Approval 34,886,839 10,276,26 3 6,318,531 886,596 52,368,229 Current Amended LOA Budget 33,506,639 11,656,50 0 6,318,531 886,596 52,368,266 Percent of Current LOA Budget Spent by through December 2011 37% 59% 40% 46% 42% Table K -2. LOA Commodity Summary (MT) Distribution Monetization Total FY 09 (Actual) 0 13,242 13,242 FY 10 (Actual) 964 13,086 14,050 FY 11 (Actual) 4,064 17,021 21,085 FY ‘12 (Actual through Dec 2011) 739 0 739 Cumulative disbursements through December 2011 5,767 43,349 49,146 Original Proposed LOA Quantities 16,200 89,790 105,990 Revised Projected LOA Quantities 16,200 89,790 105,990 Percentage of LOA Projection Achieved Through Dec 2011 35.6% 48.3% 46.3% WALA Mid-Term Evaluation Report Annex K 147 Table K -3: Monetization Cost Recovery FISCAL YEAR COMMODITY ACTUAL QUANTITY ( MT) C&F ($/T) SALE PRICE ($/MT) COST RECOVERY FY 09 Wheat 11,692 395 280 71% FY 09 CDSO 1,550 1,245 950 76% FY 10 Wheat 11,586 358 265 74% FY 10 CDSO 1,500 1,299 935 72% FY 11 Wheat 16,021 524 430 82% FY 11 CDSO 1,000 2,019 1,345 67% TOTAL 43,349 532* 403* 76% *Weighted averages by volume Table K -4. Commodity Loss Summary (MT) Fiscal Year Amount Purchased Ocean Losses Amount Received in Country Inland Losses Percent Lost Monetized Commodities FY 09 13,380 138 13,242 0 1.0 FY 10 13,000 0 13,086 0 - FY 11 17,020 0 17,021 0 - FY 12 Qtr 1 - - - - - TOTAL 43,400 138 43,349 0 0.3 Distributed Commodities FY 09 0 0 0 0 - FY 10 1,438.68 2.60 1,436.08 0 0.18 FY 11 4,164.62 15.66 4,148.96 0 0.38 FY 12 Qtr 1 4,177.72 1.73 4,175.99 0 0.041 TOTAL 9,781.02 19.98 9,761.04 0 0.205 WALA Mid-Term Evaluation Report Annex K 148 Table K-5. WALA Staff Positions – As of December 31, 2011 CATCH or PVO Approved Positions Filled Positions Vacancies Comment Africare 58 58 0 Chikwawa Diocese 48 47 1 Emmanuel International 54 46 8 Vacant positions include frontline/field positions Project Concern International 147 124 23 Vacant positions include promoters; Vacant since Jul’11; Expected fill-in by April’12 Save the Children 38 37 1 VSL Assistant position Total Land Care 40 40 0 World Vision 40 40 0 ACDI-VOCA 3 3 0 CATCH 38 37 1 PROGRAM TOTAL 466 432 34 Percent of Approved Positions 7.3%