Institutional Capacity Building Grant AFP-A-00-03-00039-00 Midterm Review Submitted to the Hunger and Malnutrition Unit Save the Children Federation, Inc. July 28, 2006 Ashley Aakesson Richard Dixon Sonia Khush List of Acronyms BCA Behavior Centered Approach CAT Capacity Assessment Tool CCM Community Case Management CEP Capacity Enhancement Plan CEC Children in Emergencies and Crisis CHW Community Health Worker CO Country Office COMMIS Commodity Management Information System CORE Child Survival Collaboration and Resources Group CS Cooperating Sponsors CTC Community Therapeutic Care DAP Development Assistance Program ECB Emergency Capacity Building Network EPU Emergencies and Protection Unit FACTS Food and Commodity Tracking System FAM Food Aid Management FANTA Food and Nutrition Technical Assistance Project FFP Office of Food for Peace HIV/AIDS Human Immunodeficiency Virus/Acquired Immune Deficiency Syndrome HMU Hunger and Malnutrition Unit (previously the Food Security Unit) ICBG Institutional Capacity Building Grant IFAD International Fund for Agricultural Development IFPRI International Food Production Research Institute IPTT Indicator Performance Tracking Table IR Intermediate Result LAC Latin America and Caribbean LQAS Lot Quality Assurance Sampling M & E Monitoring and Evaluation MOS Minimum Operating Standards MSU Management Support Unit MTE Midterm Evaluation M4M Monitoring for Management NGO Non Governmental Organization OH Office of Health OVC Orphans and Vulnerable Children PLG Program Learning Group POP Program Operating Plan PVO Private Voluntary Organization SAM Severe Acute Malnutrition SC Save the Children SO Strategic Objective TA Technical Assistance USAID United States Agency for International Development WFP World Food Organization WWO Washington and Westport Offices of Save the Children ii Table of Contents Acknowledgements………………………………………………………iv I. Executive Summary…………………………………………………...1 II. Goals and Process of ICBG Midterm Review....................................3 II. A. Goals…………………………………………………………………3 II. B. Team Composition…………………………………………………..3 II. C. Process of MTE………………………………………………………3 III. Review Findings………………………………………………………5 III. A. Key Achievements to Date…………………………………………5 III. B. IR1. Increased field level management and technical capacity to plan and implement high-impact programs that reduce food insecurity………………………………………………………9 III. C. IR2. Enhanced technical leadership and innovation in addressing food insecurity…………………………………………………14 III. D. IR3. Increased strategic collaboration to promote effective food security strategies…………………………………………...18 III.E. Cross Cutting Issues …………………………………………………19 IV. Recommendations…………………………………………………….20 Annexes…………………………………………………………………….27 A. Scope of Work for Midterm Review……………………………………28 B. List of Technical Assistance Trips, FY 04-May FY 06………………...32 C. Interview Guides…………………………………………………………40 D. Risk Management Framework………………………………………….43 E. Gender Sensitization Guidelines.……………………………………….44 F. PLG Agendas……………………………………………………………..47 iii Acknowledgements Many people have generously given their time and insights in the course of the review of the Hunger and Malnutrition Unit’s Institutional Capacity Building Grant from USAID’s Office of Food for Peace. We would like to thank key informants from Save the Children’s Country Offices and Washington and Westport Offices. All were very open and keen to discuss the issues. We would also like to thank all of the of the team members of the Hunger and Malnutrition Unit: Ina Schonberg, Ron Shaw, Bill Fiebig, Toby Stillman, Heather Danton, TS Ramachandran, Paige Harrigan, Katie O’Connor, and Tandice Ghajar. Their feedback and participation were greatly appreciated throughout the process. Thanks to everyone for being so open and candid. Bill Fiebig of the Hunger and Malnutrition Unit has been extremely helpful with preparations for the review, including developing the scope of work for the review team, and supporting the document review. We would like to thank Katie O’Connor and Tandice Ghajar of the Hunger and Malnutrition Unit for their help in organizing travel and accommodation for team members during the review and for assistance with the document review. iv I. Executive Summary The findings of the midterm review indicate that the Hunger and Malnutrition Unit is on track for achieving the intermediate results and strategic objective of the Institutional Capacity Building Grant (ICBG), increasing the capacity of Save the Children (SC) to implement effective programs that reduce food insecurity. An extensive document review shows that the Hunger and Malnutrition Unit (HMU) has met specific targets set for each fiscal year, with one exception: other cooperating sponsors (CS’s) adopting SC identified best practices for use in their food security programs. Interviews with SC staff in other technical units (such as the Office of Health), and in country and regional offices, show that HMU staff are perceived as being responsive to field needs, as providing high quality technical assistance and capacity building, and as effectively testing and adapting innovative approaches to SC’s principles and programs. Documentation and interviews also show that the HMU has been a driver of productive collaborations within and outside of SC, more than under past Institutional Support Agreements with the Office of Food for Peace (FFP). Internal to SC, HMU staff have been instrumental in the establishment of three cross-agency working groups, focusing on Nutrition, Vulnerability, and Behavior Centered Approaches to programming. Externally, HMU staff have participated actively in FANTA’s efforts to validate and disseminate tools for improving the way food security programs measure access to food, in efforts of CS’s to form a cross-agency food security forum to replace some of the functions of FAM, in the CORE group, and other external fora. One of the most important achievements of the HMU during the first half of this ICBG has been the development of the Capacity Assessment Tool and Enhancement Plans (CAT and CEP), annual self assessment and planning tools for country staff and managers. The CAT has been useful for the HMU to measure growth in capacity in key management and technical areas. As yet, the potential of the CAT and the CEP as tools for country managers and HMU staff to systematically build country staff capacity in a collaborative way has not been realized. Regarding the CAT and CEP tools, the midterm review team recommends that: • The HMU, as a team, improves the process of feedback and dialogue with country programs on their individual CATs and CEPs during the FY 06 round; and • The HMU refines the tools in FY07, after improving the overall CAT and CEP process, based on what has worked well and what has been challenging for the first few rounds of the CAT. The HMU has committed to building capacity within SC in diverse areas under this ICBG, from looking at programs and staff structures through a gender lens, to strengthening program design, monitoring, and evaluation skills in the field. Three areas in particular have made significant progress, and will continue to be important throughout the rest of this ICBG and beyond. They are: • Building capacity in the Management of Severe Acute Malnutrition (SAM), including Community Therapeutic Care (CTC), in emergency and development contexts; • Developing SC specific tools in Behavior Centered Approaches to Programming (BCA) and incorporating them into new and ongoing programs; and 1 • Developing and disseminating the child-centered Risk Management Framework for analyzing the types and magnitudes of risk to children in a given context, as well as the assets available to manage those risks. See Annex C for the Framework. Regarding these three areas, the midterm review team recommends the following: • Management of Severe Acute Malnutrition: the HMU should continue to validate and document the impacts of SC’s field activities, and contribute to the global evidence base regarding the management of SAM in emergency, transitional, and development contexts; • Behavior Centered Approaches: the HMU should continue to support country level staff who are experienced in behavior centered methodologies, by facilitating additional training opportunities and regular cross country TA visits; and • Risk Management Framework: the HMU should develop and disseminate situational assessment and project design tools that operationalize the concepts of risk, assets, and vulnerability in the framework. Other key recommendations include: • Increasing emphasis on defining best practices in technical areas for which field, HMU, and WWO staff expressed demand, such as cash programming and household/community safety nets, market based agriculture and non-agriculture interventions that maximize outcomes for children, and management of childhood illnesses, including community case management where appropriate; and • Making minor changes to the organization of the results framework for the ICBG at the activity level, to more accurately describe the nature of activities, and for ease of monitoring and reporting. These changes would not necessitate any changes to targets or indicators. 2 II. Goals and Process of ICBG Midterm Review II.A. Goals The goals of the midterm review were to assess progress in achieving planned results and refine program activities and targets accordingly. Specifically the midterm offered the opportunity to: • Assess progress towards achieving targets; • Refine targets if needed; • Review program strategies and their effectiveness in addressing food security issues; • Identify constraints and successes; • Make recommendations to improve ICBG performance; and • Suggest modifications of activities under the grant as appropriate. The midterm review offered the HMU an opportunity to assess project management strengths and weakness. The monitoring and evaluation system was also reviewed to ensure that it is able to track process and outcome indicators. II.B. Team Composition The assessment team comprised three members: Ashley Aakesson, team leader and member of the HMU; Richard Dixon, Food Security Program Manager working in the Mozambique Country Office (CO); and Sonia Khush, member of the Emergencies and Protection Unit (EPU) in the Washington, DC office, who works closely with the HMU. Although all of the review team members currently work for SC, each one offers a different perspective regarding the HMU contribution to effective food security programming at SC. These perspectives are: from the field; from a Unit which works closely with the HMU; and, from someone who is familiar with the internal workings (operation and activities) of the HMU. II.C. Process of the Midterm Review The assessment team used a process evaluation method, meaning that, at this halfway point in the implementation of the grant, the team assessed whether the HMU was on track to meet established targets and achieve the intermediate results leading to the strategic objective of the ICBG. The process followed by the team is described below. 1. Planning the Review Process: A number of team meetings were held to plan the review. During these meetings, team members: • Compiled the list of project documents to review; • Developed a list of people to interview from SC’s Westport and Washington offices (WWO), Country and Area (Regional) Offices, and the HMU; and • Developed interview guides for the three different groups of key informants. See Annex B for the guides. 3 2. Review of Documents: The documents that the midterm review team examined included: • The ICBG proposal; • Annual performance reports; • Annual work plans and narrative; • Monitoring and Evaluation plans and the IPTT; • Technical program reports; • Previous Institutional Support Agreement evaluations; • Documents generated by HMU staff, including policy documents, manuals and training materials, technical notes, and assessment tools; • Proceedings from Program Learning Groups and other trainings/workshops; and • Other documents including Country Food Security Program reports, FS Program Capacity Enhancement Plans, and articles related to program approaches. 3. Individual and group interviews: The review team conducted in-depth interviews with three distinct groups within SC: (i) food security program staff from SC COs as well as regional staff; (ii) WWO staff; and (iii) HMU staff. Interviews consisted of open ended questions that allowed key informants to express their opinions and perceptions in a conversational format. Interviews were conducted with a total of 33 key informants within the three different groups. From the field, ten different COs and one Area Office were selected and contacted. This was an important group as they are the HMU’s ICBG beneficiaries. The team wanted to cover a wide range of different types of food security programs from developmental, transitional and emergency programs as well as programs with: large food aid components; a mix of food aid and monetization; and one program with one hundred per cent monetization. Thirteen key informants at varying levels of management were contacted. Some were interviewed in person but most were contacted by phone. Usually all of the review team members were present. From the WWO staff key informants were interviewed from several units and offices within SC’s technical and management divisions. These units and offices work directly with the HMU in varying degrees of collaboration. Three Vice Presidents were interviewed as well. This gave a good cross section of personnel within SC’s WWO with varied perspectives and interests. All HMU technical or management staff were interviewed to assess perceptions of the unit’s strengths, weaknesses, and overall performance under the ICBG. 4. Data analysis: The interview data from each group were analyzed using separate matrices. Frequently made comments and important points were then drawn out and summarized in a power point presentation separated by Intermediate Result and interview groups. Findings from the document review were incorporated into this presentation as well. 5. Sharing and refining the Review Report: The team presented the review findings to the HMU team on May 12, for feedback and questions. After the two hour feedback session recommendations were drawn up. The first draft of the report, including recommendations, was circulated to the HMU for comments. HMU staff provided written comments to the team. These comments have been incorporated into this document as the midterm review team felt 4 appropriate. This step has ensured that the findings and recommendations are clear to the HMU. The content of findings and recommendations remain the review team’s own. 6. Externally facilitated HMU workshop: On June 9, the HMU, including three new staff members, spent a day processing the midterm review findings and recommendations in a meeting facilitated by Spee Braun, a long term SC consultant. Among other things, the team agreed on specific actions related to the recommendations, and some changes in language in the results framework (RF) from the original ICBG proposed RF, all of which are explained below. III. Review Findings III.A. Key Achievements to Date This section summarizes some milestones during the first half of the ICBG. Items were included as key achievements if they were repeatedly mentioned favorably in interviews or if they were instrumental in accomplishing a target in the Indicator Performance Tracking Table (IPTT). 1. February 2004: Food Security Program Learning Group (PLG) workshop, Turkey Key technical and management staff from SC’s FS programs worldwide shared best practices and lessons learned and discussed innovative food security program initiatives. Sectoral working groups agreed on key competencies for each functional area of the CAT. Specific topics addressed during the PLG included: • Behavior Centered Approaches; • Community Therapeutic Care; • Risk and Vulnerability; • Livelihood Activities; • Effective Targeting; • Advocacy; • Commodity Management; and • Developing M & E Systems. 2. FY 2004: Development of the Capacity Assessment Tool (CAT) The CAT is a competency based assessment tool that measures technical and managerial capacity among food security staff in COs. The tool defines necessary competencies to meet minimum standards across project management, monitoring and evaluation, agriculture, health and nutrition, and commodity management. Based on the results of the tool application, a Capacity Enhancement Plan (CEP) for each food security program is developed that identifies strengths in staff capacity and areas that need further improvement in order to meet SC standards. 3. September 2004: Start-up Workshop, Turkey This workshop, which was funded from the HMU’s private budget, provided training in Title II grant management, financial management tracking and reporting, commodity management, and other grant reporting requirements. Participants included new FS program managers (for start-up 5 programs as well as newly recruited managers for on-going programs), current field office finance and administration staff, and program technical staff. 4. September 2004: Development of the Commodity Management Information System (COMMIS) COMMIS was developed by the SC FS program in Bolivia. The system proved to be robust, user friendly, and capable of generating reports that meet USAID’s requirements. In FY05 the HMU leveraged private funds to translate and standardize the COMMIS system for dissemination to all SC FS programs managing food aid. The process of disseminating COMMIS and building staff capacity in the System is ongoing and will be complete by the end of the ICBG. 5. September 2004/March 2005: Gender Sensitization Guielines developed and disseminated at Regional Gender Workshop and PLG in the Latin America/Caribbean region (LAC) During the regional workshop the gender sensitization guidelines were presented and country program managers agreed to go through an exercise with their project staff to assess how gender relations might impact their food security programs. Please see Annex D for the gender sensitization guidelines tool. 6. FY 2005: Commodity Management Review Trainings Trainings were conducted in Malawi, Nicaragua, Haiti and Guatemala. They increased staff capacity and knowledge of commodity management practices and Title II commodity management regulations. They also laid the groundwork for participants to conduct periodic audits for other programs in their region. 7. March 2005: Save the Children hosted Interagency CTC Meeting, Washington D.C. CTC is an innovative approach that links therapeutic feeding with supplementary feeding and bases it at the community level, allowing for broader reach. The meeting was organized in collaboration with FANTA and Concern Worldwide, and brought together key CTC stakeholders from throughout the world to review approaches, develop a formalized coordination mechanism between CTC implementing agencies, and outline next steps with regard to developing the CTC evidence base. 8. FY 2005: The SC Behavior Centered Approaches training manual was field tested and disseminated The manual was based on action-research initiatives in Ethiopia, Haiti, Mozambique and Uganda. Further field work was conducted in FY05 in Bangladesh, Guatemala, and Honduras in FY05 and the manual has been revised to incorporate these findings. The manual has been disseminated to country programs electronically and via TA visits. This manual has since been used by a cross sectoral working group within SC to develop an SC-wide Behavior Centered Approaches training package. 6 9. February 2005: Food Security & HIV/AIDS PLG workshop, Malawi One objective of this workshop was to improve the integration between food security programs and HIV/AIDS programs and to share lessons learned from the Title II programs in Uganda and Malawi. Topics covered included: • Capacity Building; • Gender and HIV/AIDS; • Risk and vulnerability, especially related to orphans and vulnerable children; • HIV/AIDS and food security; • Nutritional needs of people living with HIV/AIDS; and • Increasing food security in high HIV prevalence contexts. 10. March 2005: Regional Food Security PLG workshop, Nicaragua One of the main themes of this meeting was agricultural commercialization in the LAC region. PLG participants were given an introduction to business planning and vulnerability assessments. The PLG provided an additional opportunity for the HMU staff to spend time with each CO discussing country-specific programs. HMU and CO staff followed up specifically on the implementation of the recommendations from the Gender Analysis workshop and the use of the CAT. 11. July 2005: Asia PLG and Nutrition Meeting, Thailand Co-hosted by the HMU and the Office of Health (OH), this meeting was attended by health and nutrition technical staff from eight Asian countries as well as the Directors of the HMU and the OH, and key WWO technical personnel. The meeting focused on health and nutrition technical interventions that cross programming sectors. During the final session, staff from health and food security programs worked together to frame an Asia-wide strategy for health and nutrition programming. 12. September 2005: ‘Monitoring for Management’ (M4M) workshop, Guatemala The LAC M4M workshop included an overview of M&E and examined the use of information systems for management. Participants focused on better understanding and use of Results Frameworks, using a Results Monitoring Matrix to track project results, using an Activity Monitoring Matrix to track the implementation of project activities, and using a Quality Monitoring Matrix to track the quality of project activities. At the close of the workshop each CO developed a set of next steps outlining how they plan to apply lessons learned at the workshop to management of their food security programs. 13. FY 2005: Regional Commodity Management Trainings These three regional workshops brought together commodity and finance managers from each region and focused on increasing commodity management accountability through increased collaboration and knowledge management between commodity and finance staff. In addition, best practices in commodity management were reviewed and COs were provided with tools to conduct regular self assessments in commodity management. The standardized commodity management information system software (COMMIS) was also introduced and distributed to workshop participants. These workshops were held in Honduras (LAC), Uganda (Africa) and Indonesia (Asia). 7 14. March 2006: Asia and Africa M4M workshop, Kenya This workshop used the same training tools as the LAC M4M workshop. See above 15. March 2006: Global Food Security PLG, Kenya. A number of important topics were covered: (i) Understanding how the results frameworks, conceptual frameworks, and food security assessments work together in the program design. (ii) The introduction of the risk and vulnerability framework and discussion about how existing programs might approach targeting, activity design or implementation differently had they been designed using this framework. (iii) The introduction to successful cash based and safety net programs discussion centered around, when are these approaches are appropriate and feasible. (vi) How to target vulnerable groups. (v) CTC: comparing emergency and non-emergency interventions and understanding the different ways to incorporate “emergency” elements into programs. (vi) Looking at different ways to incorporate market interventions into programs. 16. June 2006: CTC: A Case Study, by Miriam Chaiken, Hedwig Deconinck, and Tedbabe Degefie published This article will be published in the Food and Nutrition Bulletin, June 2006. Dr. Chaiken collected qualitative data and quantitative project data from an SC impact area in Ethiopia in May 2005. The authors analyzed these data sets to assess the impact of the CTC interventions on children in the target communities of the project area. The conclusion of the article is that CTC is an important tool to effectively address nutritional emergencies and may be a valuable entry point for long term development. 8 III.B. IR 1: Increased field level management and technical capacity to plan and implement high impact programs that reduce food insecurity III.B.1. Technical Assistance Please see Annex B for a complete list of technical assistance (TA) visits completed by HMU staff, or facilitated by HMU staff. Facilitation of TA visits usually involved an HMU member developing a scope of work with project staff , identifying the appropriate person within or outside of SC to provide the TA, supporting logistics for the visit, and following up on recommendations from the trip report after the visit. CO staff gave very positive feedback on the responsiveness of HMU in providing or facilitating technical assistance, and on the high quality of that assistance. WWO interviewees perceive that the HMU is engaged with the field in a positive way that results in country buy-in for broader agency level food security TA agendas. Regional food security advisors have been a successful way to provide more locally appropriate TA in local languages, especially in LAC. In addition, CO staff found cross-country TA visits, facilitated and sometimes financially supported by the HMU, to be useful for both COs. Interview participants, including HMU staff, have found that consistent follow up on actions decided during TA visits has occasionally been problematic, largely due to time/human resource constraints within a problem with follow-up from HMU. III.B.2. Capacity Assessment Tool/Capacity Enhancement Plans The CAT was developed during FY 2004 with input from FS managers and staff from COs. The first round of CATs was completed by CO project staff in late FY 2004 for existing projects, and in early FY 2005 for projects started in FY 2004. The HMU considers this first round a baseline of sorts for the ICBG, against which future annual outputs of the CAT process can be compared. At the regional PLGs in 2005, the CAT tool and the processes for completing the CAT and developing an annual CEP based on the CAT were officially “rolled out” to participants. Participants were asked to replicate this roll out with their FS project staff and managers. In the FY 2005 round of CATs, 31% of programs reported a score of at least 3 out of 4 in all functional areas of the CAT, and 75% of programs reported improvements from the previous round, meeting or exceeding the CAT related targets in the ICBG M & E plan. In FY 2006 some FS projects have submitted their CATs. The Bangladesh DAP has combined the CAT process with an intensive program review by three HMU staff. HMU staff will meet in August 2006 to jointly process the CATs and provide feedback to help FS programs develop country specific CEPs. 9 The consensus among interview participants from COs, the LAC Area Office, and the HMU is that, while the CAT could be a useful tool, the process of dialogue between the HMU and COs to develop a CEP based on a CAT is flawed. Country project staff are concerned that the main measurement of project quality should remain project outcomes, and would like more guidance on how to integrate the CAT/CEP process with other planning and assessment processes, such as project M & E plans, staff performance reviews, and Program Operating Plans at the CO level. III.B.3. Commodity Management The HMU has made important strides toward meeting its ICBG target of all SC food security programs using a standard commodity management system by using private funds to translate and adapt COMMIS from the Bolivia CO, and beginning to disseminate and build capacity in COMMIS at CO and regional levels. As of the FY 2005 performance report, 31% of country food security programs were using COMMIS, though with varying levels of competence. In addition to the work with COMMIS, the HMU has been working with the Management Support Unit (MSU), which sits in the same division of SC as the HMU, to conduct regular Commodity Management Reviews in COs. In FY 2004, reviews were conducted in Malawi, Nicaragua, Haiti, and Guatemala. In FY 2005, reviews were conducted in Indonesia, Ethiopia, and Sudan. In addition, there were three regional Commodity Management Trainings, in Africa, Asia, and LAC. As of June 2006, reviews have been conducted in Bangladesh and Indonesia. During the life of the ICBG, HMU staff have completed Bellmon analyses and monetization plans in Bangladesh and Mozambique, and submitted a report on monetization in Bolivia to USAID and a consortium of CSs. The consensus among interview participants is that HMU support and capacity building in commodity management, in collaboration with the MSU, is very useful for effectively implementing food security projects. Comments about COMMIS were generally positive, although additional support is needed to fully integrate the system and adapt it to specific project or CO needs. Especially effective and popular has been supporting, and partially funding, CO staff to participate in commodity reviews in other countries in their region. See the table below for a summary of commodity reviews and the status of COMMIS installation in COs, as of the time this report was written. 10 SC Country Office Commodity Reviews Done in 2004/2005 and Plan for 2006/2007 Field Office Reviewed in Proposed review in 2006 Proposed review in 2007 Installing COMMIS 2004 2005 Month Dates Month Dates Month Dates INDONESIA X March 17 - 25 March 12 - 18 BANGLADESH Mar/Apr 26 - 5 February 12 - 16 KC CONFERENCE April 25 - 27 UGANDA X May 15 - 26 April 9 - 16 ETHIOPIA (Emerg) June/July 08 - 07 Feb Com.Rev. June 17 - 24 TAJIKISTAN X August 14 - 25 August 2 - 8 HAITI X Sept 11 - 22 BOLIVIA Oct 16 - 27 installed SUDAN Nov 13 - 22 July 16 - 22 MALAWI X Dec 11 - 21 not yet MOZAMBIQUE March Com.Rev. not yet NICARAGUA X April Com.Rev. no visit needed WEST BANK/GAZA May Com.Rev. MALI July Com.Rev. PAKISTAN HONDURAS Sept Com.Rev. GUATAMALA Oct Com.Rev. no visit needed 11 III.B.4. Workshops and Trainings Workshops, including annual Program Learning Groups (PLGs), and trainings, are two key mechanisms that the HMU employs for field to field, field to headquarters, and headquarters to field exchange of new ideas, lessons learned, and sharing of best practices. The HMU builds capacity of country staff through formal trainings, such as the Commodity Management training, Behavior Centered Approaches training, or Monitoring for Management training, but also by transferring skills and concepts informally during TA visits. Qualitatively, country and area level interviewees agreed that the global and regional PLGs were productive and enjoyable. They felt that PLGs are an effective way to share lessons learned and best practices across country programs, and to update key SC food security staff on new developments in the field. Country staff appreciated skills based trainings such as the M4M and BCA trainings. Alternating regional and global PLGs was popular, particularly in the LAC region, as regional PLGs allow deeper country participation and more discussion in local languages. For descriptions of illustrative trainings and workshops, see Section III.A. above. III.B.5. Monitoring and Evaluation Much of the direct technical assistance that HMU sectoral experts give to specific projects is in the area of M &E, from M & E system design, to using monitoring data for mid course corrections, to developing scopes of work and recruiting consultants for midterm and final evaluations. In addition, the HMU has been closely engaged with an agency wide effort to standardize M & E systems and to build capacity in key competencies at country, regional, and headquarters levels. The Monitoring for Management training mentioned above incorporates many concepts and some tools from this initiative. III.B.6. Behavior Centered Approaches In FY 2004 the HMU supported field testing of BC tools and methodologies in Ethiopia, Haiti, and Mozambique, and facilitated cross program TA from Ethiopia to Uganda. The HMU and Ethiopia Country Office supported the development of a training manual for Behavior Centered Approaches. The target audience for this manual is field based technical staff. In FY 2005, the BCA Manual was finalized and disseminated electronically and through TA visits. A cross agency BCA Working Group was established, largely due to efforts from HMU staff. Collaboration of this working group resulted in a “Save the Children” training package, combining elements of the BCA manual with CORE’s BEHAVE training. Throughout the life of the ICBG to date, HMU staff have built the capacity of country level staff, some of whom have been identified as regional resource people in BCA. HMU staff have been 12 instrumental in building BCA into new program designs and supporting country staff to operationalize those designs effectively. There is a consensus among interview participants at all levels that BCA has been important for making SC’s food security programs effective, and will continue to be important for securing grants and implementing projects in the future. III.B.7. Community Mobilization The BCA Manual incorporates key concepts and methods from SC’s approach to CM, and the leading CM expert at SC sits on the BCA Working Group. Aspects of community mobilization continue to be incorporated into SC’s food security projects as cross cutting strategies, and HMU technical staff provide ongoing TA as projects implement, monitor, and evaluate their CM strategies. III.B.8. HIV/AIDS and Food Security Throughout the first half of the ICBG, the HMU has been coordinating TA efforts with SC’s Office of HIV/AIDS in selected African food security programs, including Malawi, Mozambique, and Uganda. In FY 2005 the Africa regional PLG was jointly organized by the HMU and Office of HIV/AIDS. In FY 2004 the Uganda DAP staff completed an assessment of HIV/AIDS affected households. An HMU supported consultancy examined the validity of the methodology used for the Uganda assessment as well as the effectiveness of the project’s monitoring plan for measuring reduction in vulnerability. In FY 2005 the HMU leveraged private funding to hire FIFSA to conduct a study and make recommendations about how SC could better use food aid in projects with HIV affected populations. Title II and other food security program staff contributed to the study as well as HIV program staff and managers. The final report from the study has not been completed. The fact that there has not been an Africa based Food Security Advisor since September 2004 has impacted the level of effort the HMU has been able to invest in this area. As of June 2006, two Africa based positions have been filled, one focusing on Health and Nutrition and the other on Livelihoods. III.B.9. Integrating Gender into Food Security Programs In FY 2005, the HMU developed Gender Sensitization Guidelines which were disseminated at a LAC Gender workshop and again at the LAC regional PLG in 2005. Each food security project in agreed to conduct gender assessments of food security projects and staffing structures. The assessments were completed, but follow up has depended on food security managers. 13 In FY 2006, the HMU has facilitated contracting the International Center for Research on Women to conduct a gender analysis of SC’s Bangladesh DAP. Lessons learned from this analysis will be shared with other COs. There is consensus among country, area, and HMU interviewees that the gender guidelines presented by the HMU were a good start, but that more focused TA and capacity building is needed to take this initiative forward in other regions as well as LAC. III.B.10. Integrating agriculture and nutrition The HMU has taken this initiative forward largely through TA for new project design as well as improving the integration of agriculture and nutrition projects in existing projects. Countries that have been particularly successful at increasing synergies among project activities across sectors have included Guatemala, Haiti, Mozambique, Nicaragua, Malawi, Bangladesh, and Uganda. In addition, HMU staff participate in IFPRI and IFAD working groups on this topic. III.B.11. Partnerships In FY 2004, the HMU completed a review of quality of partnerships between SC food security programs and local/national NGOs, as well as how the HMU can best support partnerships in country programs. Taking the lessons learned from this review, principles of good partnerships continue to be designed into new projects and supported in existing ones. III.C. IR 2: Enhanced technical leadership and innovation in addressing food insecurity III.C.1. Management of Severe Acute Malnutrition During FY 2004, HMU staff participated in joint planning with FANTA and Concern Worldwide for a multi agency meeting that took place in FY 2005. The HMU continued collaboration with Valid to support the Ethiopia CTC program as it moved from emergency response to a transitional project. During FY 2005, SC’s CTC programs in Ethiopia and Sudan expanded, and in Malawi and Bangladesh, management of SAM was incorporated into Title II food security programs. In FY 2006, SC participated, by invitation, in a WHO informal consultation on SAM. The HMU is currently chairing SC’s cross agency Nutrition/CTC Working Group. HMU staff have collaborated in the development of the FANTA funded CTC Manual which is soon to be released, and are engaged in the development of the associated training package. The consensus among interview participants is that building capacity in effective approaches for managing SAM, including community based approaches like CTC, will continue to be important in the coming years. Field interviewees were positive about the TA that the HMU has provided and appreciative of private funds for piloting CTC programs that the HMU helped to leverage. 14 Country level participants noted that as there has been rapid growth in demand for TA in this area, HMU staff have not always been able to respond to TA needs in as timely a manner as desired. External consultants that the HMU has identified to fill the gap have worked well, but have brought their own challenges, such as being very tied to one model of community based management of SAM. III.C.2. Child Centered, Risk Management Framework Under the ICBG, the HMU committed to developing a child centered food security framework. An external consultant completed a first draft of this framework, which took a life cycle approach. At the same time, the HMU was looking at ways to help programs analyze risk and vulnerability in order to more effectively manage risk in SC’s food security programs. A Risk and Vulnerability working group was formed with other SC units, field staff, and external members. It was supported by a technical assistance group, comprised of one HMU staff person, a Tulane PhD student and a Tufts PhD student. This technical assistance group did the majority of the work leading to the first and subsequent drafts of the risk and vulnerability framework. Over time, as understanding of and clarity within the framework developed, it became known as the draft Risk Management Framework. The framework has remained child centered, but has become more broadly applicable across sectors, and in emergency as well as development contexts. In FY 2006 the current version of the Risk Management Framework and explanatory narrative were completed and disseminated at the HMU PLG and other SC meetings. Dissemination continues; the A Risk and Vulnerability working group has become a sub-group of the SC-wide Program Development, Monitoring, and Evaluation working group. Links to an SC-wide Avian and Pandemic Influenza working group have also been established. The framework and narrative have also been disseminated to other CSs through the Emergency Capacity Building (ECB) Network. Key tasks that are currently under way are the completion of an annotated bibliography of risk and vulnerability literature, and the development of a risk and vulnerability assessment tool kit to help HMU and country/regional staff to begin applying the concepts in the framework on the ground. Interview participants at country, regional, and HMU level agreed that the Risk Management Framework has the potential to improve the way SC’s programs address risk and vulnerability, but that field testing and validation of the framework is needed. III.C.3. Management of Childhood Illnesses While this is not an area specifically mentioned in the ICBG agreement, elements of improving the management of childhood illnesses at the household and community levels have usually been incorporated into SC’s food security programs. This is increasingly the case, and interview participants at all levels expressed an interest at becoming more effective at integrating the management of childhood illnesses into food security programs. 15 Some examples of this trend include: • The Bangladesh Title II program has included Community Case Management (CCM) of ARI and diarrhea. The Bangladesh Ministry of Health has given authorization and committed to participate. Roll out of CCM will begin in the final quarter of FY 2006. • The Title II program in Uganda has actively supported CCM of malaria in the project area through community distribution agents, in collaboration with the Ministry of Health and Roll Back Malaria. The HMU in collaboration with SC’s Office of Health has submitted a proposal for funding to pilot CCM of ARI in the same project area. • Funds have been secured for pilot CCM activities in conjunction with the Nicaragua Title II program. This resulted from a joint HMU/OH visit to LAC to assess the potential for CCM in existing country programs. The HMU and OH are pursuing funding opportunities for linking CCM activities with Title II programs in Honduras and Guatemala in the future. III.C.4. Adoption of Best Practices in Country Programs The HMU assesses this by reviewing annual country program reports. In FY 2004, Haiti, Guatemala, and Nicaragua reported best practices in food systems management, and Mozambique reported best practices in Behavior Centered Approaches for Health and Nutrition. In FY 2005, Ethiopia, Bangladesh, Haiti, Mozambique, Uganda, Malawi, and Nicaragua reported best practices in BCA and food systems management and extension through model homesteads. III.C.5. Documenting and disseminating best practices The HMU has documented and disseminated best practices to SC staff and other CSs through a variety of media, including technical notes, tools and manuals, newsletters, and external publications. A list of each type of medium which has been completed to date follows: 1. Technical notes: • Developing minimum standards for agricultural programs; • Integrating agricultural marketing plans into food security programming; • Gender and food security: Women feeding the world; • Farmer field schools: a group extension process for food security programs; and • Use of LQAS surveys for monitoring in Haiti’s food security program (Agriculture and Health/Nutrition sectors). 2. Tools and Manuals: • Gender Sensitization Guidelines; • BCA Manual 3. Newsletters: The HMU has distributed three newsletters since FY2004 to SC food security program staff and other units within SC. This is just short of the plan for biannual newsletters. 16 4. External publications: • Chaiken/Degefie/Deconinck: CTC: A Case Study, Food and Nutrition Bulletin, June 2006; • Aakesson/Swedberg: “Supporting caregiver behaviors”, a chapter in BASICS Field Guide to CCM of Childhood Illness, FY 2006; • Shaw: Guidance on Local/Regional food procurement, 2006; • Fiebig, Quarterly UN Chronicle, March-May 2005; • Shaw: Bolivian monetization data analysis: 2001-2005; • Stillman: Emergency Nutrition Network special supplement on CTC, 2005; • Stillman/Kiernan:Wall Street Journal article on CTC; • Stillman/Kiernan: Philadelphia Inquirer article on CTC; and • Deconinck: Adopting CTC from scratch in Ethiopia, Emergency Nutrition Network, Special Supplement, November 2004. CO and Regional interview participants appreciate the efforts of the HMU to disseminate the latest technical guidance on food security issues, whether produced by SC or other organizations. They said that occasionally, more explanation about how that guidance might affect specific projects would be useful. The HMU has supported some country programs to document lessons learned and innovations driven by country program staff, but CO interviewees said that more specific guidance on targeting different audiences and formats for different types of documents would help them to more effectively do this. CO and HMU interviewees said that the HMU should be more strategic about supporting outside experts to validate project results for formal journal publications, while acknowledging the cost implications of this. III.C.6. Participating in External Fora HMU staff participate regularly in external technical and policy fora. Two particularly intensive commitments have been the FANTA working group on developing improved indicators for access to food, and the ECB Network. SC has participated in developing assessment and monitoring tools for FANTA’s dietary diversity, hunger gap, and food insecurity index indicators, and has volunteered to test them in some upcoming midterm evaluations. One of HMU’ s Food Security Advisors is technical advisor to the Disaster Risk Reduction initiative of the ECB. Participating in ECB has enabled SC to network with other NGOs and learn from best practices in the field of risk and vulnerability assessment and emergency response. The ECB has also provided a forum for the HMU to share the Risk Management Framework and get feedback from other CSs. Illustrative one time or ongoing fora that HMU staff have participated in are: • The CORE group, including working group and annual meetings; • International Rural Development Meeting in Nairobi, June 2004; • Food Aid Management (FAM) working groups and meetings; • World Vision-US’s GWISER workshop; • UN Standing Committee on Nutrition, NYC, March 2004; 17 • UN Standing Committee on Nutrition, Geneva, March 2006; • IFPRI/Gov’t of Uganda conference on Assuring Food and Nutrition Security in Africa by 2020, Kampala, April 2004; • United Nations Security Council presentation, July 2005; and • Global Health Council Meetings, FY 04-06. III.C.7. Best practices being used by other CS’s in Title II programs The HMU set a target of two CSs per fiscal year using an SC developed tool for applying best practices in food security programs. This is the only target that is under achieved at the midway point in the life of the ICBG. HMU staff have contributed to the dissemination of best practices in cross agency groups in the following ways: • The SC BCA training package will be reviewed by CORE’s Social and Behavior Change working group in FY 2006/2007 as it develops it’s revised version of the BEHAVE training package; and • The HMU has contributed to the development of new IEE guidelines with the FFP/CS Environmental Working Group. These guidelines are still under development. IV.D. IR 3: Increased strategic collaboration to promote effective food security practices III.D.1. Cross unit/office collaboration within Save the Children One area in which internal collaboration has increased is in joint or cross unit TA to country programs. In FY 2004, the HMU facilitated six TA visits to food security projects from experts in other offices within SC. In FY 2005 that number increased to five. In addition, the HMU is increasingly facilitating, and sometimes financially supporting, cross country TA. Two examples of this are TA in BCA from Ethiopia to Uganda, and TA from Pakistan to Indonesia in Commodity Management. Another area of increased collaboration is in participating in other units’ PLGs, and inviting participation from other units in HMU PLGs. Since FY 2004 the HMU has hosted two PLGs jointly; one with the Office of HIV/AIDS in Malawi, and one with the Office of Health in Thailand. HMU staff have helped to establish cross agency working groups in BCA, Nutrition/CTC, and Risk and Vulnerability. In addition, HMU staff participate in SC’s M & E, Technical Assistance, and Avian and Pandemic Influenza working groups. Qualitatively, interview participants in CO and regional offices, HMU and WWO are very positive about the increasingly productive and cost effective collaboration across units and offices within SC. Participants appreciate the leadership role HMU staff have played in initiating and maintaining collaborative relationships. 18 III.D.2. Building Relationships with Universities and Action Research Projects For the last several years, the HMU has built relationships with faculty at universities with strong public health and food security programs. Key faculty members at Emory, Tufts, Tulane, and University of Florida agreed to serve as informal technical advisors to the HMU. In addition, Tufts and Tulane students participated in the Risk and Vulnerability Technical Advisory Group that developed the Risk Management Framework. One of the most rewarding outcomes of building these relationships, from both CO and HMU perspectives has been a series of summer interns that have completed high quality projects such as action research to validate program results, and qualitative assessments of project service provision. Other action research projects that the HMU has supported include project assessments in Ethiopia and Mozambique. The Ethiopia assessment resulted in the article, CTC: A Case Study, which will be published in Food and Nutrition Bulletin in June 2006. In addition, an action research project linked GMP interventions with weight gain in Nicaragua, and the Uganda food security staff submitted a proposal to the IFPRI RENEWAL program that made it to “finalist”, but was ultimately not funded. In the area of management of severe malnutrition, SC continues to validate best practices through action research in conjunction with Title II projects, in Ethiopia, Malawi, and Bangladesh. In Malawi SC is piloting the treatment of adult HIV with Ready to Use Therapeutic Food, and at the relevance of supplementary feeding for treatment of early stage kwashiorkor. In Bangladesh CHWs have been involved in delivering CTC on a pilot basis. III.D.3. PVO Participation in ICB Supported Workshops In FY 04, the HMU participated in an Africare hosted workshop on Developmental Relief, and in the FAM consultancy on lessons learned from Food Aid Consortia. In FY 05, SC/US co￾facilitated a workshop on PD/Hearth and CTC for Title II managers in which 10 CS’s participated. In addition, the HMU participated with other CS’s in revising the “Cooperating Sponsor’s Field Guide to USAID Environmental Compliance Procedures”, including examples from specific countries. The HMU has also been working with other CSs to establish a follow up mechanism for cooperation and sharing technical knowledge after FAM dissolution. IV.E. Cross Cutting Issues IV.E.1. HMU Role in Emergencies and Transitional Programming The HMU is currently supporting a large scale review of feeding in emergencies, and also an HMU Technical Advisor taught courses in Selective Feeding in Emergencies at Columbia and Johns Hopkins Universities in FY 2006. 19 The perception of field staff and WWO staff interviewed for the midterm review is that the main support from HMU is related to commodity management and distribution. They feel that other technical assistance from the HMU has been good in some cases, particularly for CTC, but inconsistent due to unclear roles and insufficient staff. HMU interview participants agree that there is a need for a comprehensive strategy for how the HMU supports SC emergency and safety net programs. IV.E.2. Communication and General Management MTE Team findings: It is clear that in general ICBG management is effective, and communications among the HMU team themselves, as well as among HMU staff and other colleagues within SC are positive and productive. As mentioned in the executive summary, the HMU is on track to achieve its commitments under the ICBG. The HMU staff made themselves available and supported the MTE process generously, but there were information and time management issues that made the process difficult This finding supports numerous mentions in the interviews from country program staff, the HMU, and WWO colleagues, of these issues. Regarding information management, there is as yet no central system for easily retrieving data on HMU activities, country programs, or technical documents and reports. The document review required at least two days of effort from administrative staff to bring together hard copies in one place for the convenience of the midterm review team. As one example, there is no one place, in paper or electronic media, that lists every TA trip HMU staff have made, every training or workshop HMU members have organized or participated in, or every document that HMU staff have produced. The information was gleaned through annual reports and individual accounts. Under time management, there is a general tendency not to set specific goals for meetings or phone calls, or, if goals have been set, not to stick with them. The MTE team has observed that most members of the team contribute to this issue at different times. IV. Recommendations It is clear that Save the Children has a strong and viable program that is attempting to increase Save the Children’s capacity to implement effective Title II activities that decrease food insecurity. The establishment of minimum standards and key competencies for technical and managerial quality through the provision of TA is working well. Some of the recommendations included here are suggestions to modify some of the initiatives currently included in the ICBG. The team has also included recommendations to add additional components to the program in response to impending changes in the operating environment that were not anticipated during the development of the original proposal. The recommendations are organized into the sections listed below and not per IR because most of them are cross cutting issues relevant to all of the IRs. 20 1. Clarifying priorities for capacity building and technical innovation In the final two and half years of the grant the review team recommends that the HMU continue to focus on building field capacity and disseminating tools and lessons learned in the following areas: Assessment, M&E; BCA; Using the child centered Risk Management framework; and Management of acute malnutrition and food security in high HIV prevalence settings. A great deal has been learned in the first half of the ICBG and the HMU is now in a good position to move forward and build capacity in the field in these programming areas, as well as playing a leadership role among other PVOs. Increase focus on some areas, including: • Adapting and applying best practices for programming in transitional contexts such as HH and community social safety nets and use of cash resources as an alternative to food; • Management of childhood illnesses; and • Increasing market-based considerations into all agricultural and non-agricultural activities to ensure child centered livelihoods programming. Looking ahead, these areas are thought by field staff and the HMU team to be important to incorporate into existing programs where appropriate and to build into the design of future programs. The HMU should continue to flesh out and define a child-centered livelihoods approach, building on both the current evidence base and on experience gained through existing field programs. (Note, slightly inconsistent with the summary found above…maybe you do want to identify 4 areas instead of 3 to expand more on CCM (as separate from SAM). Continue to facilitate TA as requested by individual food security programs. The HMU has achieved success in building capacity in the areas of gender analysis, community mobilization, and better integration of agriculture and nutrition in during the first half of the grant. In addition, issues of gender and community mobilization have been incorporated into the Behavior Centered Approaches manual. As discussed above, it may prudent to shift focus to the topics identified in the previous paragraphs. Ensure that individual work plans reflect the above priorities, and plan for strengthening the capacities within existing or new staff that will be necessary to succeed in these areas. HMU management, with input from all of the HMU staff, should assess necessary skill sets and whether they exist in the team. If not, put action plan in place to build skill sets and secure new staffing, if necessary. 2. Revise the Results Framework of the ICBG at Activity Level To simplify monitoring and reporting on the ICBG, the MTE team suggests minor revisions to the Results Framework. The grant was designed so that each IR is related to and supports the others, the team feels that assigning specific initiatives (such as BCA) under one IR will more accurately reflect the nature of the work in those areas, and ease information tracking and reporting. The suggested revised RF is below. The changes suggested would not necessitate changes in the IPTT, which is also included below. 21 22 SO: Increase SC’s capacity to implement effective Title II activities that decrease food insecurity. Save the Children Institutional Capacity Building Grant Revised Results Framework Activity 1.1 Ensure standardized management and technical quality of development and developmental relief programs. Activity 1.2 Improve application of best practices in: • Assessments and Project Design; • Behavior Centered Approaches, including Community Mobilization; • Monitoring and Evaluation; and • Gender and Food Security. Activity 1.3 Improve SC capacity to reduce food insecurity in high HIV/AIDS prevalence settings. Activity 2.1 Develop a child￾centered framework for food security. Support its application in SC’s programs. Activity 2.2 Develop and test approaches and methodologies in: • Management of Acute Malnutrition, including CTC; • Community based management of childhood illnesses; • Risk assessment and project design, including best targeting of food and cash resources in developmental relief; and • Livelihoods interventions that maximize benefits to children. Activity 2.3 Document and disseminate experiences and best practices. Activity 3.1 Enhance the knowledge base for food security programs by supporting effective organizational learning at SC, partners, and cross PVO institutions. Activity 3.2 Enhance partner and SC’s food security capacities through partnerships. Activity 3.3 Improve inter-PVO coordination and cooperation. IR 1: Increased field-level management and technical capacity to plan and implement high impact programs that reduce food insecurity. IR 3: Increased strategic collaboration to promote effective food security practices. IR 2: Enhanced technical leadership and innovation in addressing food insecurity. 3. Specific initiatives: Provide meaningful feedback, as a unit, to each food security program’s CAT and CEP. A concerted and timely effort to systematically review and provide feedback on each CO plan by all members of the HMU is needed. Country program staff stated in the interviews that the CAT and CEP tools would be more valuable to them if HMU staff gave concrete feedback on the CEP in particular, and used the CEP to guide discussions with country offices on felt needs for specific HMU support for TA and capacity building. The Capacity Assessment Tool (CAT) is a good competency based assessment tool, which is able to measure technical and managerial capacity enhancement among the food security staff. The tool identifies competencies across project management; monitoring and evaluation; agriculture; health and nutrition; and, commodity management. If FS programs receive good constructive feedback from the HMU after the second round of the CAT process (May 06), they will be able to truly evaluate the usefulness of the tool and should then be asked to suggest improvements to refine the tool. The development of this type of tool is a dynamic process. Country office staff have suggested that the tool should be simplified, harmonized better across sectors, and made to fit better with their other planning obligations within COs, as well as with M&E plans and staff performance reviews. Clarification is needed that the tool is not meant for comparison between country programs. Roll out the next phase of the risk management framework which specifies practical applications through tools and approaches. Field staff have been introduced to the risk framework and now need to understand its practicalities. How can they incorporate the framework into new program design? The Framework is extremely important, given the focus on vulnerability and safety-netting for future programs. At present, CO food security staff do not fully understood the way in which the framework can be practically applied. Complete the plan to transfer Behavioral Change Approach (BCA) capacity to the field and circulate the BCA manual with follow-on BCA training. Specific country offices have staff with strong capacity to apply BCA and train others. These staff should be supported to become regional resource people in BCA, to provide training and TA to other country programs in their region as needed. Collaboration with the Office of Health to develop an SC-wide BCA manual has gone well, but the HMU should continue to disseminate the existing BCA Manual from Ethiopia rather than wait for the SC wide tool to be finalized. Maintain emphasis on program innovation in the Management of Severe Acute Malnutrition (SAM) in emergency and development contexts, and document the field based evidence. There is a growing interest in the management of acute malnutrition (global and severe) in both development and emergency contexts. This is as a result of new developments in the treatment of acute malnutrition. Supported by the ICBG, SC is emerging as a technical leader in this area, notably under the field application of the CTC approach; a promising community based method to manage SAM, in appropriate contexts. In the next phase of the grant SC will further document field based innovations in promising approaches to better manage acute malnutrition, in particular at the community level. 23 4. Continue the things that the HMU is doing well. Continue to provide the high quality of TA presently being given. Both external consultants and in-house technical assistance provided to COs has been of high quality. TA appears to work best when the request has come from the field with clear scope of work. TA facilitated by the HMU between country offices has also worked well. Continue to organize and facilitate relevant, skills based workshops. Workshops have been appreciated and well received by field staff. The workshops have been of high quality and covered useful, appropriate themes. Continue the commodity management reviews and training for each country office HMU has gone a long way to standardize the COMMIS system throughout all of the FS programs. The commodity management reviews bring together field staff, home office staff and financial managers which has created a deeper understanding of commodity management for everyone concerned. 6. Documentation and Communication with Field Staff Communicate to the CO Food Security programs the objectives of the institutional capacity building grant, the HMU role, and the benefits to the FS programs in achieving the ICBG objectives. Field programs need to know more about HMU activities. It would be useful for COs to know the areas of expertise that exist within HMU and how these can be accessed. It is recommended that the HMU develop a one page profile of its work, structure and relationship with other units within SC for dissemination to the field. Develop technical notes for BCA, CTC, and Assessing program impact on Hunger Gaps (Moz). Request feed back from the field for existing technical notes as to their usefulness. A number of technical notes have already been developed for agriculture, gender, farmer field schools and LQAS. Interviews indicate that technical notes are most appreciated when they clearly show how to do a particular intervention with references to more in depth material, rather than a discussion paper about a particular intervention. Re-examine the purpose of the HMU Newsletter. The team suggests the HMU produce a concise, quarterly newsletter which includes HMU global activities, short stories from the field that are directly related to successful new approaches/interventions, and latest thinking on relevant food security issues. The field needs to be ‘in touch’ with what is going on in the wider world of food security and increase their ability to access information about relevant food security issues. Analyze the different types of documentation required for journals, donors, in-house (SC) and websites and develop a strategy for collecting it. The team suggests that the HMU take the lead with regards to documentation of food security program successes and innovations, rather than expecting CO staff to assume responsibility for this task. For example, the HMU can contract an independent specialist to visit a specific CO and document a successful approach. HMU has a wide network of University contacts that would be helpful in this regard, including for cost sharing arrangments. For short stories, from the field, for circulation in-house and/or for posting on websites, develop a simple format with guidelines for field staff to enable them to write 24 stories. Alternatively, produce a short questionnaire to collect information which can then be used by HMU staff to write short stories for distribution. This process of documentation of success approaches is extremely important. Continue with the present format for Program Learning Group (PLG’s) workshops. Field staff greatly appreciate this chance to get together and exchange ideas. Many felt that this opportunity to have face to face contact is the best venue to communicate new approaches and put forward new concepts. 7. Information management and dissemination Refine the current information collection system to better understand: beneficiary numbers and how they are counted; cost per beneficiary; proportion of the population reached; program geographical size; budgets, and, number of different interventions being implemented by food security programs. Train people on the system as appropriate. This will provide important information both for Save the Children management, other units working with the HMU and for donors. Revise the current central filing system, including shared hard drive and SaveNet materials. Keep a current annotated bibliography, available to field and WWO staff, of all assessment tools, articles and reports written by the HMU or produced by food security programs-through SaveNet or another medium. It will be important to then provide training in the use of the system for HMU staff to effectively contribute to and maintain the system, and for field staff to contribute to and have access to this information. Adapt a formal protocol from other SC units for email use, and train the HMU team in it. It appears from a number of comments that the email system is overused. For example, emails are often cc’ed to people to whom the message is not directly relevant. This creates an information overload and brings extra people into a conversation that can easily be streamlined for efficiency and timely resolution. It is important to have more structured, regular staff meetings with a set agenda and rotating facilitation. The MTE team suggests that responsibility for setting an agenda and facilitating the meeting rotates among all the HMU staff. If the person whose “turn” is coming for the next meeting would be traveling, it would be that person’s responsibility to switch turns with someone else in the HMU. This will help to make meetings more productive and less time consuming, as well as to divide the responsibility for high quality meetings more equitably among the team. An agenda set in advance will ensure that meeting objectives are clearly understood and met. Put up an HMU travel board to post personnel travel overseas and to meetings. This would assist HMU schedule trips and see at a glance where and when staff are committed. 8. Collaboration Continue the good collaboration between the HMU and other units. Collaboration has gone well with the Office of Health (OH) particularly when developing strategies for Management of 25 SAM, CCM and BCA. Collaboration with the HIV/AIDS unit and the EPU has also been productive. Develop a formal agreement format for requesting technical assistance for food security programs from other units/offices within Save the Children. An agreement will demonstrate firm commitment from other units to provide TA for FS programs and will also formalize requests from the HMU to other units for TA in a particular area or country. Some frustration exists on both sides with regards the informal arrangement for TA provision to FS programs. 26 Annexes 27 Annex A. SOW for Midterm Review AFP-A-00-03-00039-00 Institutional Capacity Building Grant Mid-Term Evaluation Scope of Work Save the Children June, 2005 1. Management: 1.1 Team composition: SC submitted a proposed budget that would have allowed recruiting an outside consultant as team leader but this line item was eliminated during the proposal approval process. SC plans to look in-house for an evaluator from the SC Program Development Unit or Field Office and if private funds are available, the FSU will recruit an external team leader. It is also expected that one other SC staff member will be asked to participate in the evaluation. A third member of the evaluation team has yet to be identified but will be a mid-level internal staff member who can assist the team to collect and analyze data, schedule interviews, prepare surveys, assist with any travel arrangements, and facilitate with the compilation of the report. 1.2 Timeframe The report will take approximately three weeks to complete. Two weeks will be devoted to an internal review of documentation and interviews and the final week for the report preparation. The evaluation is currently scheduled for the month of July 2006. 2. Methodology In order to provide some basis for comparison between the ISA final evaluation and the ICB mid-term, the two evaluations will use a similar methodology that includes a five-step process. This includes team planning, review of documents, interviews, analysis of findings and the final report preparation. 2.1 Team Planning The team will begin with a two-day exercise. The team planning exercise will set the groundwork for how the evaluation team will function, and create in each team member a similar frame of reference to work on the project. Taking the time to go through this process was key to the integrity of the ISA final evaluation. Some of the items to be covered during the team planning exercise will include: • Understanding the ICB and its history • Understanding the clients for the report • Operationalizing the SOW • Developing a work plan for the evaluation • Understanding team member working styles • Developing questionnaires and instruments to query field offices 28 During this two-day process, as well as during the interview process, evaluators will look at how the ICB currently fits within the organization. The review will look at how the ICB has built on the ISA, and if any of the underlying assumptions or objectives of the ICB have changed since its inception. It will also examine SC’s definition of its ICB “beneficiary population” and how that group has been served thus far under the ICB. 2.2 Review of Documents The team will begin by reviewing three types of documents. The first group are classified as background documents related to the policy, planning, implementation, and monitoring & evaluation of Title II programs. These will be such things as the most recent DAP/MYAP guidelines, the Monetization Field Manual, the Monitoring & Evaluation guidelines, the USAID Food Security Policy Paper and the FFP Strategic Plan for FY06-10. The second group of documents are SC documents specifically related to its work with Title II and the ICB. These would include policy related documents; manuals and training materials that have been produced by the headquarters based team as a result of the ICB. It would include databases, capability statements or reports that have been prepared about SC’s institutional capacity in food security. The third set of documents would be documents prepared by the “beneficiaries” of the ICB and would include Field Office reports, CATs and other country specific materials that show the results of capacity building on specific country programs. 2.3 Interviews The evaluation will include a series of interviews. Although the evaluation team will have the final determination as to who will be interviewed, it is expected that a number of groups will be included. Similar to the ISA final evaluation, it is expected that staff members within Children in Emergencies and Crisis will be interviewed as well as other SC staff members in other Departments, including senior executives. However, it is expected that the interviews will be targeted to the “beneficiary population” which includes field staff from a number of SC field offices. It is expected that a focus group format will probably be used with Headquarters-based staff and that a questionnaire format will be used with senior level field based staff. The evaluation team will have determined the best method for surveying field staff that have received training from the ICB. 2.4 Data Collection/Report Preparation Data collection will include both secondary data (document review) and primary data (interviews focus groups and site visit). The data collection and analysis will be done by the two-team members, with the assistance of the third team member. SC’s Food Security Unit staff will assure that all secondary data to be reviewed is compiled and available to the team on a timely basis. The Food Security Unit will also facilitate the collection of primary data as requested by the team. The team is responsible for analyzing the data collected from interviews, focus groups and surveys. 29 The report preparation will be the shared responsibility of both team members, with the Team Leader responsible for the final product. The Director of the FSU requests a report outline prior to the team’s departure for the field site. The staff of the Food Security Unit will meet with the evaluators for a presentation of the final results of the report before it goes to Food for Peace, and have the right to comment on any factual errors or misconceptions contained in the report. 3. Evaluation Points of Emphasis Although the evaluation team will use the ICB grant document as their guide for the evaluation, there are a number of points of emphasis that the Food Security Unit would like the evaluators to give special attention. 3.1 Quality In Save the Children’s ICB submission, our first objective was to improve our existing programs (DAPs) in order to build depth and breadth before attempting to expand. How has our strong emphasis on training in the first two and half years contributed to this objective? At what point time and in what ways has our emphasis on M&E impacted on the quality of our programs? The midterm will examine how the ICB has contributed to specific aspects of quality such as better understanding of food security concepts, the application of state-of-the-art program activities, the flow of technical support from the home office and regional technical presence, and overall improved logistical capacity. 3.2 Expansion SC has embarked on assertive food security program expansion since the ISA going from 8 to 13 programs in FY05. At this point, it is important to conduct both a self-evaluation and an overall assessment of the food security policy environment: What else does SC need to do to accomplish this objective in a timely manner? Will original targets of program growth have to be revised as a result of USAID policy changes? For example, our initial proposal saw monetization as a tool for expansion, yet changes in policies towards monetization now put that strategy in jeopardy. Likewise, SC wants to know how its improved ability to do food security problem analysis through RSFAs and other assessment tools impacted our ability to write better DAP/MYAPs for new programs. 3.3 Coordination SC champions the dissemination and open discussion of successful food-assisted programming endeavors. During the midterm evaluation, SC wants to know if its efforts of sharing best practices with other NGOs helped us to build stronger in-house capacity? Areas to be investigated might include: • Reg 216 trainings • M&E working group • Local capacity building • Hearth Model/Positive Deviance methodology • Community-based M&E and mobilization strategies • BC approaches and trainings • CTC innovations 30 3.4 IPTT & M&E Assessing progress in institutional capacity building at the midterm is crucial in order to make crucial adjustments or to strengthen certain aspects of the ICB. In particular, SC wants to know how it is doing in its monitoring and evaluation of the ICB? How could its ICB M&E plan be improved and what accomplishments can be measured to date? The mid-term evaluation team will be responsible for recommending updates for the IPTT table and will make recommendations on the revision of indicators, targets or annual monitoring tools should this be required. 31 Annex B. List of Technical Assistance Trips, FY 04 to May 06 Country and Primary objective of visit Relevant technical or managerial areas Outputs from the trip Non SC or non food security staff in country* Trainings or workshops conducted during visit (if not listed below)? HEATHER DANTON Angola- RFSA and develop MYAP Title II MYAP Training workshop for staff in RFSA tools, methodology Bangladesh – DAP preparation (2 visits) Title II DAP HKI DAP prep/design workshop including understanding Results Frameworks Uganda- DAP preparation Title II DAP Sudan- Strategic planning for Nuba Mtns. program transition Transitional programming First draft results framework Um Ruwaba program manager – Education sector Internal strategic planning workshop Pakistan- Rapid livelihoods assessment Livelihoods and emergencies Livelihoods Assessment report Emergency staff, including Team Leader and Site managers Ethiopia- Development of Safety Net proposal (2 visits) Safety nets, Title II Concept Paper and proposal Ethiopia- Lessons learned from DAP II Title II Trip report, summary of staff interviews Other sector staff, Health, Education, HIV/AIDS 2-hour workshop on understanding concepts of risk and vulnerability Indonesia- Attend AERA semi-annual review meeting, meet AERA livelihoods and food distribution staff and meet DAP staff Emergencies and transitions; Title II Trip report Emergency staff, SC Alliance members, other CEC and DPC staff, Country Office leadership and key sector staff, outside facilitators Guatemala- Start-up workshop for MYAP prep Title II MYAP development plan/schedule Outside consultants 2-Day MYAP Preparation workshop 32 Country and Primary objective of visit Relevant technical or managerial areas Outputs from the trip Non SC or non food security staff in country* Trainings or workshops conducted during visit (if not listed below)? RON SHAW Bangladesh – Assist in preparation of DAP monetization components & prepare distribution program Bellmon Analysis (Sept￾Oct 03) Analysis of risks for producer disincentive and market disruption associated with distribution of Title II commodities. Monetization sales design. Monetization sections of DAP Distribution program Bellmon Analysis DAP manager participation Yes Bolivia – Conduct Title II monetization commodity selection, pricing and sales methodology study for USAID & CS Consortium (Nov 03) Study/evaluation of ongoing Title II monetization program re commodity selection, sales methodology, minimum price management, etc. Report, recommendations and options for future of program. Commodity manager participation No Mozambique – Prepare Bellmon Analysis Update for Title II monetization program (Jan-Feb 04) Analysis of risks for producer disincentive and market disruption associated with monetization of Title II commodities Bellmon Analysis Update None No Haiti – Prepare Title II emergency program proposal (Mar 04) Design emergency urban based FFW program. Project proposal for For urban FFW project. Program manager participation No Kosovo – Prepare CARE USDA monetization proposal (Jun 04). Note: Non/FFP Commodity selection & sales implementation design for Section 416(b) monetization. None No Bangladesh – Conduct Study/evaluation of potential for Report on potential for Title II None No 33 Country and Primary objective of visit Relevant technical or managerial areas Outputs from the trip Non SC or non food security staff in country* Trainings or workshops conducted during visit (if not listed below)? Title II monetization private sector sales study (Dec 04) shifting Title II monetization sales from government to private sector buyers. monetization sales to private sector buyers with implementation recommendations Mozambique – Prepare update of Bellmon Analysis for Title II monetization program (Jan-Feb (05) Analysis of risks for producer disincentive and market disruption associated with monetization of Title II commodities Bellmon Analysis Update None No Niger- Emergency Program start-up (Aug. 05) Note: Non/FFP activity WFP contracting. Start-up of emergency food distribution WFP Contract Program plan SC/UK FOD Participation No TOBIAS STILLMAN Facilitated TA LAC - Regional Community Case Management Assessment – OH, FSU, Johns Hopkins University Health/Nutrition Visit Report, program proposals FS and Health staff None Haiti – Support to Health/Nutrition sector (2-3 trips) - Swedberg Health/Nutrition and M&E Trip reports In one trip Eric worked with staff from co-located MSH project in addition to our staff Bangladesh – Support to GMP program - Bolles Health/Nutrition Powerpoint slide show including recommendations CTC pilot strategy and training plan All FS staff GMP Bangladesh – Facilitate CTC Pilot Training – Valid International Health/Nutrition Training completed, final report pending All FS staff CTC 34 Country and Primary objective of visit Relevant technical or managerial areas Outputs from the trip Non SC or non food security staff in country* Trainings or workshops conducted during visit (if not listed below)? Bangladesh – Review CCM programming and make recommendations for Title II roll out – Eric Starbuck Health/Nutrition Trip Report FS and OH staff Bolivia – GBGP training and country roll out – Ofilio Mayorga Health/Nutrition FS Staff and other CS GBGP Guatemala – PD/hearth Training – Facilitated attendance of Claudia Nieves to attend workshop in USA Health/Nutrition FS Staff PD/Hearth Uganda – Informal mid￾term review of health sector – Consultant Health/Nutrition Visit Report FS staff Adult education methodologies Ethiopia – CTC program review – C. Tanner Health/Nutrition Visit Report Emergency/Health Staff Ethiopia – Ongoing Technical Support for CTC – Valid International Health/Nutrition Visit Reports Emergency/Health staff CTC Malawi – Crisis response program design – C. Tanner Health/Nutrition Visit Report, Gates Proposal FS and Emergency Staff CTC Malawi – CTC technical assistance Health/Nutrition Visit report FS and emergency staff CTC Malawi – Ongoing technical support for CTC – Valid International Health/Nutrition Visit Reports FS and emergency staff CTC Tajikistan – Title II SHN set up support – Karin Lapping (2x) Health/Nutrition Visit Report FS staff 35 Country and Primary objective of visit Relevant technical or managerial areas Outputs from the trip Non SC or non food security staff in country* Trainings or workshops conducted during visit (if not listed below)? Trips Guatemala (3x) – Amendment development, TA, TA Health/Nutrition, M&E Trip Reports Amendment FS Staff Results frameworks IMCI Honduras (1x) – Start-up workshop Health/Nutrition – FS Workshop report FS staff, cooperating partners Use of food Behavior Change Etc. IMCI Haiti (1x) – Emergency response and review programming Health/Nutrition, Emergency FS staff Nicaragua – Midterm evaluation Health/Nutrition, M&e Trip Report FS staff Bolivia (1x) – TA Health/Nutrition Trip Report FS staff BC IMCI Uganda (1x) – Startup support – IMCI Health/Nutrition Trip report FS staff Results frameworks IMCI Ethiopia – CTC program review Health/Nutrition FS and emergency health staff Angola – program design Health/Nutrition DAP FS staff and health staff Sudan – program review and TA Health/Nutrition Powerpoint presentation FS and Health staff Results frameworks Bangladesh (3X) – Program design, TA Health/Nutrition Proposal, trip reports FS and health staff Results frameworks BC IMCI PAIGE HARRIGAN April 2005 Cross visit with members of Guatemalan DAP team to Nicaragua; AIN-C field visit Opportunity to learn about AIN-C implementation in Nicaragua. AIN￾C is an integrated prevention based health and nutrition program located at the community level. A Direct capacity building of GT team members and application of principles in Guatemalan DAP, where AIN-C is also implemented by the MOH The TA visit to Bolivia by Dr. Ofilio Mayorga in June/July 2005 benefited not only SC, but also the other TII CSs in Bolivia. 36 Country and Primary objective of visit Relevant technical or managerial areas Outputs from the trip Non SC or non food security staff in country* Trainings or workshops conducted during visit (if not listed below)? great focus is on monthly community based growth monitoring and promotion activities. nationwide. Facilitated training visit of Dr. Ofilio Mayorga (Nicaragua DAP manager) to the Bolivian country office to teach the SC team and other CSs about AIN￾C and application of CBGP in Bolivia. I developed a detailed SOW for the TA visit. July 2005 Community Case Management Exploratory visit to Guatemala, Honduras, Nicaragua investigate country level possibilities to integrate CCM into FS activities. Community case management￾CCM is a delivery strategy designed to bring simple, curative interventions to the community for common infant and childhood infections, such as diarrhea, pneumonia, malaria, and newborn sepsis. Key interventions are inexpensive and can include oral rehydration and zinc for diarrhea; antibiotics for pneumonia, dysentery and newborn sepsis, and antimalarials. Yes, a concept paper and proposal are being developed for “pilot” CCM activities in Nicaragua. At this time, the location for the pilot would not fall directly within FS project target areas, but FS staff are key implementers. Yes, served as organizer of the visit with E Bocaletti (LAC Health Advisor) and other members of the team were D. Marsh (OH), K. Bolles (OH), and Ellie Leonstini (sp?)- JHUSPH. Made presentation introducing CCM delivery strategy overall and the findings to date to USAID/Guatemala representatives. Made similar presentation in Nicaragua to SC staff. August 2005 Bolivia H&N, CBGP, Monitoring and Evaluation, review of Ag commercialization activities, and orientation visit. Visit strongly informed development of Sept 2005 Monitoring for Managers workshop. No Sept 2005 TA visit to Haiti Participated in preparation of 2006 DIP and, work with M&E team, and work with new BCC coordinator and H&N team to review current performance of Hearth nutrition Developed a detailed SOW for Eric Swedberg (OH) TA visit in February 2006. The delay was due in part to national elections in Haiti. SC Partners. Yes, Presentation of Community based therapeutic care (based on materials prepared by 37 Country and Primary objective of visit Relevant technical or managerial areas Outputs from the trip Non SC or non food security staff in country* Trainings or workshops conducted during visit (if not listed below)? rehabilitation activities And informed discussions with Tstillman for proposed TA visit to Haiti for needs assessment for CTC in Haiti. Tstillman) and discussed February 2005 TA to GT country office During DAP final evaluation. Truly this was an ongoing exercise from August 2005 to Feb 2005. in preparation for and carrying out joint final evaluation. The evaluation itself was in Feb 2006. I participated in data collection the second week fo Feb 2006 and served as a representative from SC to the eval team throughout the process. March/April 2006 have included review of preliminary evaluation findings Assisted DAP team in preparing useful reports that summarized program and indicator results from SC’s DAP experience for the final evaluation team. Helped SC DAP team review and assemble tables of quantitative results from the DAP for the final evaluation team. Joint TII evaluation. As a result, participation from USAID, CRS, SHARE, and CARE. Consulting firm, TANGO carried out the evaluation. Capacity building exercise for team. Feb 2005 Nicaragua Participated in finalization of questionnaire for RFSA (March/April 2006), followed up on “next steps” CCM and documenting impressive nutrition results in Nicaraguas FS project, and covered monitoring and evaluation. SC Nica Prepared very brief presentation on how to improve presentation of data in reports and powerpoint presentations. Feb 2006 TA visit to Honduras Received detailed presentation of baseline survey results, worked with team members on next steps in finalizing monitoring and evaluation system, discussed BCC strategy development. Honduras team to develop SOW for consultant to carry out development of a BCC strategy (in process) Annie Foster, LAC regional director Not a presentation, but facilitated a useful technical discussion with team members on identifying findings from the baseline that should be prioritized in development of BCC strategy. March 2006 to present Numerous meetings/ presentations for MYAP development. 38 Country and Primary objective of visit Relevant technical or managerial areas Outputs from the trip Non SC or non food security staff in country* Trainings or workshops conducted during visit (if not listed below)? BILL FIEBIG Angola CDRA program visit, review agric program Recommended actions; diversify production systems Guatemala Visit impact area Recommended agric program activities Haiti Annual strategic planning workshops; visited impact areas Recommended program activities in agric sector Malawi Annual strategic planning workshops; visits to program impact areas; Recommendations; assisted in baseline survey analysis Mali Designed and implemented new food security program; developed results framework, visited impact area in northern Mali Recommended program activities Mozambique Visited impact areas; conducted study of local seed supply systems; Provided recommendations for community-based gardening programs Nicaragua Visited impact areas Recommended agric strategies, marketing strategies; Sudan Visited program impact area in Nuba Mtns.; Recommended activities for agric sector development program Assisted with initial strategic planning process for development programs Tajikistan Visited impact areas Recommended activities for agric sector development program Uganda Visited impact areas Recommended activities for agric sector development program Participated in DAP start￾up workshop; provided training in sustainable agriculture and diversification of farming systems 39 Annex C. Interview Guides for ICB Midterm ICB Midterm Evaluation May 2006 Question guide for Country Program Staff Purpose of interview: part of a midterm assessment of the progress towards meeting HMU obligations under the ICB Grant from FFP. Explain SO and IRs. Process focused evaluation￾need field staff input to see what has been working so far, what hasn’t, and to get suggestions for getting better results for the remaining life of the grant. (to be sent in the scheduling email and re-iterated at the beginning of the phone interview). A. General ICB and Communications 1. Have your program managers and staff gotten messages or requests for activities from the HMU related to the ICB grant? (after initial answer can probe: documenting successful activities and lessons learned, testing innovative approaches, using standard design, reporting, and management tools (including the CAT)? 2. What has been the level of clarity in HMU communications to you and your staff about what the HMU needs from program staff to fulfill the ICB and why? 3. What actions have you or your staff taken based on these requests? 4. How can HMU improve the communications process about a. the objectives of the ICB, b. the benefits of the ICB for country programs, and c. how country programs contribute to meeting those objectives? 5. What is the level of understanding among your staff of the role the HMU should be playing in terms of program quality, technical excellence, and leadership in the field? How could this role be made more clear to field staff? B. Standardized tools for program quality Add commodity management and M&E 6. Have the CATs helped managers and staff think about the needed capacities and competencies for quality programs? Have they helped managers and staff to assess strengths and weaknesses in staff capacity and plan to fill them? 7. Did you and your staff use the CAT to inform last year’s annual planning process? Why or why not? 8. What has been the quality of feedback from the HMU on your program’s CAT process? What has been the quality of assistance from the HMU to follow up with capacity assessment? How could the HMU provide more effective feedback and support to programs to make the CAT process more effective? 9. What other tools have members of the HMU provided you or your staff for other aspects of quality programs, such as: program design, implementation, monitoring, evaluation, or documentation? 40 C. Testing Innovative Approaches and Documenting and Disseminating Lessons Learned 10. Do you feel the HMU supported your program staff to test new approaches or document and disseminate effective approaches? If yes, did your staff or the HMU initiate this process? How did HMU provide this support? If no, why not? How can the HMU support testing new approaches and documenting/disseminating lessons learned from approaches that work in programs? 11. Has your program staff received information from HMU about new approaches that have worked well in other SC programs? What about successful approaches from other agencies? 12. Have you gotten clear guidance from HMU about HMU needs for documenting and disseminating information about successful approaches in your program? If yes, please describe them. What information from HMU would be helpful for you to do these things more effectively for your own and HMU needs (for example, formats, info on target audiences)? D. Strategic Collaboration 13. Has HMU provided your program with guidance or support to partner more effectively with other agencies in country, including SC Alliance members? If yes, please describe that support. If no, why not? 14. Has HMU provided your program with guidance or support to develop and implement a strategy for leveraging comparative advantages with other actors, (for example, SC Alliance members, Universities, Advocacy Groups) to achieve objectives, particularly related to advocacy and technical leadership? If yes, please describe. If no, why not? E. Emergency response 1. Have you worked with HMU staff during an emergency response? If so, which one (s)? 2. What was HMU’s role in the response? How was this role determined? 3. Did HMU add value to the response? If so – how – management wise? Operational support? technical support and direction? Was this sufficient, in your opinion, or should the Unit have been involved in other ways? 4. What HMU tools/innovations do you know of? Have you used any of these during an emergency? If so, were they helpful? 5. Have HMU staff engaged you (or vice versa) to co-develop tools that may be useful for both emergencies and developmental settings? 6. Have you used any rapid food security assessment tools in the course of a response? Did you use this with an HMU staff present? If so, how was the collaboration and sharing of expertise? If no HMU staff was present, did HMU provide technical support or guidance remotely? How did you decide which tool to use? 41 ICB Midterm Evaluation May 2006 Question guide for Westport/Washington Based Staff Interview background (to be sent in the scheduling email and reiterated at the beginning of the interview or group discussion). This interview is part of a midterm assessment of the progress towards meeting HMU obligations under the ICB Grant from FFP. Explain SO and IRs. Process focused evaluation-need WWO staff input to see what has been working so far, what hasn’t, and to get suggestions for getting better results for the remaining life of the grant. HMU is also interested in assessing how well the objectives of the ICB correspond to HMU and SC-wide strategic priorities in the current, rapidly changing funding and operational environment. Your ideas will help us refocus our energies for the rest of the grant and position ourselves better to continue our work after the grant. A. Role of HMU in SC and Communications 1. Have you worked directly with HMU staff? What types of work have you done together? (proposal development, program design, grant administration, assessments, monitoring and evaluation, technical assistance) 2. In your experience, how effective are HMU staff in communicating about the HMU’s strategic and programming priorities? How could communication about priorities be more clear, consistent, and compelling in future? 3. In your experience, how effective are HMU staff in communicating program impacts and successful programming approaches? a. Have you received HMU documentation of program impacts or successful approaches (for example, final reports or lessons learned? If yes, please describe them. b. Have you received any tools from HMU for standardizing technical or managerial aspects of programs? (for example, assessment tools, M & E tools, training manuals)? If yes, please describe. 4. In your opinion, how effective has the HMU been at helping SC make lasting changes in the lives of children and their families through: c. Developing and implementing high quality programs? d. Ensuring technical excellence, and leadership in the field? e. Leveraging complementary advantages through effective partnerships at local, national, and international levels? B. Strategic Collaboration 5. In the collaboration(s) you mentioned above, what has worked well and what has been difficult about working with HMU staff? How could the process be improved? 6. Are you aware of any technical working groups within or outside of Save the Children that HMU staff participate in? If so, what are these? Are there any working groups you think HMU staff should be participating in, but aren’t? 7. In your opinion, has the HMU been effective at drawing on outside partnerships (for example, with SC Alliance members, Universities, or Advocacy Groups)where necessary for expertise, support, and to pursue funding opportunities? How could the HMU be more effective at this? 42 Annex D. Child Centered Risk Management Framework Natural Risk Social Risk Health Risk Lifecycle Risk Financial Risk Political Risk Environment al Risk C LIM A T E O F RIS K Financial Assets Social Assets Natural Assets Physical Assets Human Assets Shock occurs DEVELOPMENT PHASE TRANSITIONAL PROGRAMMING RELIEF PHASE Preparedness strategies: Risk Mitigation Risk Reduction Shock response Coping strategies attempt to deal with losses that are realized from the shock Outcome Change or no change in child’s wellbeing EXTERNAL AGENTS 43 Annex E. Gender Sensitization Guide for Food Security Programs BACKGROUND One of Save the Children’s program quality considerations is the importance of understanding gender implications in our programs. An understanding of and respect for the roles that women and men play in their daily livelihood and care-giving activities is critical to SC’s ability to address child malnutrition and hunger in a sustainable manner. In addition, gender equity (define…) is an important value held by Save the Children, and we strive to reflect this in our programming. In order to support the quality of existing programs and the appropriate design of future food security programs, the FSU is piloting a household and community based approach to assessing the capacities, motivations and barriers that exist for and among men and women for participation in the types of behavior change activities that characterize most of SC’s current food security programs. SC’s food security program, staff ranging from front line community workers to sectoral and program managers, make different types of decisions every day related to program implementation. For example, while it is easy to assume that we know the motivators and challenges faced by men and women in our target program areas, until we take the time to ask target beneficiaries, themselves, the types of barriers they face in doing what they do every day, it is not possible to make well-informed decisions that create positive change without causing harm or threatening cultural values. This approach consists of one day of project participant focus group inquiry, followed by staff examination and discussion of results. OBJECTIVES 1) Sensitize staff to gender issues 2) Examine program-based gender considerations from the ground up 3) Consider implications of gender for program implementation and decision-making OUTLINE OF ACTIVITIES Getting Started: - Identify participant focus group facilitators: Program managers should find out from ALL FS staff how many (and who) among them have participated with and/or have been trained in facilitating focus group discussions. One to two staff should be designated as lead facilitators. This person must have an open mind and an ability as well as a willingness to continue to ask questions of focus group members within one or two target communities to ensure that the questions are answered as thoroughly (with as much detail) as possible. - Select focus group locations and participants: Separate focus groups for men and women should have no more than 10 -15 people and a comfortable environment in which 44 all members feel safe to express their thoughts, ideas and views. To the extent possible, focus group members should be representative of the target HHs served by the FS program. - Conduct focus groups: A 2-3 hour session should be sufficient, pending on the size of the group, breadth of SO or number of activities analyzed. The questions should be asked in such a way that a specific activity under one or more IRs of a SO is the point of reference. Therefore, the FS program may decide to (initially) assess gender under only one SO, rather then the entire program. However, this process can (and should) be repeated for all activities under all program SOs. Community focus groups: Discussion Guide: The Focus Group (FG) Discussion facilitator should ask FG members two basic questions: 1) What do men do related to the activity in question and why (or for what purpose) or why not? 2) What do women do related to the activity in question and why (or for what purpose) or why not? However, facilitators will want to ensure that the following information is elicited in discussion, posing additional questions as needed: a) Within this particular activity area (e.g. the agricultural production cycle), who does what? In other words, what is the division of labor between/among men and women for this particular activity? Time requirements/demands and expectations of spouses, relatives, friends, community for each role played (disaggregated by gender). b) Who (men or women) have access to the key assets or resources required to undertake this activity? c) Who benefits from the activity (men, women, children?) and how (provide some additional probing questions on the how)? d) Who within the HH or the community maintains the information, skills, capacities needed for the activity to be successful? e) Who makes the decisions (or has the power) related to the control of both using and obtaining assets/resources related to this activity? Answers to the questions provided by each group covering all aspects of a – e should be recorded carefully. Don’t forget to note which group it was (village location, men’s group or women’s group, date of discussion). A matrix for doing this is attached as Exhibit A. Using the information: Focus Group Discussion facilitators and program managers should call a meeting of all FS staff to review the findings/answers from the focus group discussions. Using this information, re￾examine the approaches currently used and the expected outputs related to the activities discussed with the groups. What decisions, suggestions or ideas for improving implementation approaches can be gleaned from the FG members’ answers? This can be discussed in the context 45 46 of understanding current (or perceived) roles of men and women and what we might view as their “ideal” roles for ensuring decreased malnutrition and hunger among children and decreased food insecurity within the HH. Using the matrix attached as Exhibit A, detail the current (as expressed by FG members) and ideal roles (per program goals/objectives and indicators) for both men and women and list the barriers faced by each in achieving the ideal. How might the barriers identified by both men and women be addressed? What steps will be taken to remove or lower these barriers? Depending on the number of staff involved, breakout groups may be appropriate to ensure full engagement of staff at all levels. Exhibit A Gender Assessment Focus Group Discussion Record Province/District/Village________________________________ Enumerator Name or No.__________________ Group Type___________________________________________ Date____________________________________ Objective/Strategy Behaviors Current Roles Ideal Roles Barriers How Address Barriers? Men Women Men Women Men Women Men Women 47 Annex F. Illustrative HMU Program Learning Group Agenda FOOD SECURITY PROGRAM LEARNING GROUP MARCH 13 - 16, 2006 AGENDA Monday, March 13 Session 1: Introductions, Expectations & Purpose of PLG Facilitator: Ina Schonberg Objectives: Participants meet, share expectations, and understand the purpose of the PLG 8:30 – 9:00 Welcome & Introductions 9:00 – 9:20 Sharing of Expectations 9:20 – 9:30 Review of Agenda Materials: Agenda; PLG Purpose Statement; Summary of 2005 PLG recommendations and follow up actions Session 2: Opportunities and Challenges for SC Hunger and Malnutrition Programs Facilitator: Ina Schonberg Objectives: Reflect on changes in the funding and operational environments that affect day to day work and strategic planning; Obtain field and FSU perspectives to frame the rest of the PLG sessions 10:00-11:15 Materials: Session 3: Presentation about Save UK’s Hunger Reduction Programming Facilitator: Ina Schonberg Objectives: Learn how Save UK is responding to a changing environment; Understand the overlapping and complementary capacities of US and UK 11:30-12:30 12:30- 1:30 Lunch Break Session 4: Standardizing Program Design I: Common Results Framework Facilitator: Toby Stillman Objectives: Understand how the results frameworks, conceptual frameworks, and food security assessments work together in the program design; Discuss commonalities between current project results and frameworks; Get feedback on proposed common results framework 48 1:30-3:00 Materials: Session 5: Standardizing Program Design II: Risk and Vulnerability Framework Facilitator: Heather Danton Objectives: Become familiar with the risk and vulnerability framework; Discuss how existing programs might approach targeting, activity design or implementation differently had they been designed using this framework 3:15- 5:00 Standardizing Program Design II: Risk and Vulnerability Framework Materials: Session 6: Wrap Up Facilitator: TBD Objectives: 5:00-5:30 Materials: 49 Tuesday, March 14 Session 7: Standardizing Program Design III: Food Security Assessments Facilitator: Heather Objectives: Brainstorm steps in a food security assessment, including tools and processes Compare sample food security assessment for emergency response with one for a development program; Plan to build capacity in food security assessment for emergency response, transitional, and developmental programming 9:00 - 11:00 Materials: 11:00 – 11:15 Break Session 8: New Approaches for SC/US Facilitator: TBD Objectives: Become familiar with successful cash based and safety net programs; Discuss when these approaches are appropriate and feasible 11:15 - 12:30 Materials: 12:30 – 1:30 Lunch Break Session 9: Panel Discussion on Targeting Vulnerable Groups Facilitator: TBD Objectives: Understand processes to identify groups as more vulnerable; Understand program mechanisms to determine membership of individuals in these groups; Understand how program activities and results differ as a result of alternate targeting choices 1:30 – 3:15 Uganda: food insecure and HIV affected households Ethiopia: those at the “edge” of pastoralism Nicaragua: poor vs. ultra poor households Indonesia: targeting in post emergency context Materials: Executive summary on content and use of training manual Session 10: Community Management of Severe Malnutrition Facilitator: Hedwig and/or Toby Objectives: Compare emergency and non-emergency interventions; Can discuss when community management interventions are feasible and desirable 3:30 – 4:30 Materials: Pubs on CTC Session 11: Wrap Up Facilitator: TBD 4:30 – 5:00 50 Wednesday, March 15 Session 12: Panel Discussion on Emergency Preparedness and Response Facilitator: Bill, Rama, Ashley Objectives: Understand different ways to incorporate “emergency” elements into programs; Share program perspectives on coordination issues for emergency response and preparedness 9:00 – 11:00 Emergency preparedness: Bangladesh Non food aid response: Malawi & Indonesia National surveillance: Malawi & Mozambique Materials: Session 13: Poster Gallery – Successful Approaches Facilitator: Ashley, Katie Objectives: Familiarize with successful aspects of approaches or aspects of program implementation; Celebrating the good work that SC does to bring lasting and positive change to the lives of children! 11:00 – 12:00 Materials: 12:00 – 1:00 Lunch Break Session 14: Panel Discussion of Market Based Approaches to Livelihoods Facilitator: TBD Objectives: Understand different ways to incorporate market interventions into programs; Relate this to new opportunities for SC/US 1:00 – 4:00 Ag marketing in Bolivia Ag marketing in Nicaragua Linking pastoralists to markets in Ethiopa Ag marketing in Uganda Materials: Session 15: Concurrent, Voluntary Small Group Meetings Facilitator: Adrian Cullis Objectives: Allow topics to be covered that may not be relevant for all participants 4:00 – 5:00 Nutrition in SC programs HIV and Food Security Lessons learned from SC Emergency Response experience Materials: Thursday, March 16 Session 16: Recap of Opportunities and Challenges Discussion from Day 1 Facilitator: Ina Schonberg Objectives: Summarizing and refreshing the issues to prepare for next session 9:00 – 9:30 Materials: Session 27: Strategic Planning for Hunger & Malnutrition Facilitator: Ina Schonberg Objectives: Conduct SWOT analysis by region; Identify priority programming areas, opportunities, donors, challenges; Provide input for new strategy articulation 9:30 – 11:15 Materials: 11:15 – 11:30 Break Session 17: Managing for Program Quality Facilitator: Bill, Ina Objectives: Share “findings” from quality planning and assessment jars; Discuss annual planning processes 11:30 – 12:30 Materials: 12:30 – 1:30 Lunch Break Session 18: Panel Discussion on Innovative Monitoring Approaches Facilitator: Bill, Ashley Objectives: Understanding of tools and approaches discussed; Focusing on how programs are using the data gathered to improve quality of projects 1:30 – 3:30 Lot quality assurance techniques in Haiti and Uganda Monitoring natural resources management in Bolivia Annual survey analysis in Mozambique HEA as an evaluative tool Materials: Session 19: Closing Session Facilitator: TBD Objectives: Action Items, Poster Awards, Evaluation 3:30 – 4:30 51