September 2013 This publication was produced at the request of the United States Agency for International Development and the Canadian International Development Agency (CIDA). It was prepared independently by Dr. Elvira Beracochea, Dr. Sarah Tisch, Sarah Weber, Deborah Bickel, Deric Zanera, and Arnold Mndalira of Social Impact, Inc. FINAL EVALUATION OF THE COMMUNITY￾BASED THERAPEUTIC CARE INSTITUTIONALIZATION IN MALAWI (CTCIM) ACKNOWLEDGMENTS The final evaluation team wishes to thank Ms. Janet Guta, Deputy Director Nutrition Management of the Department of Nutrition and HIV/AIDS, and Ms. Dalitso Dembo Kang’ombe, Chief Nutritionist of the Ministry of Health for their leadership in the conduct of this evaluation. The evaluation team also wishes to thank Mr. Chimwemwe Chitsulo and Ms. Violet Orchardson of USAID for their technical guidance throughout the implementation of this evaluation and in the preparation of this report. The Concern Worldwide team of Mairead Petersen, Megan Christensen, and Noel Molony also deserves our appreciation for their help in the conduct of the evaluation even after the CWW project had ended. All the CMAM stakeholders, particularly Ms. Julita Manda of CIDA Malawi, deserve thanks for sharing their experience and for their commitment to ending malnutrition in Malawi. The team also wants to specially acknowledge the support of the SSDI project team in facilitating the participation of District staff in the Findings Workshop, and all the people that generously gave their time and shared their wisdom through key informant interviews, survey responses and focus group discussions. Photo: Provided by USAID/Malawi. FINAL EVALUATION OF THE COMMUNITY-BASED THERAPEUTIC CARE INSTITUTIONALIZATION IN MALAWI (CTCIM) FINAL REPORT September 26, 2013 AID-612-TO-13-000004 This document was submitted by Social Impact, Inc., with MIDEGO Inc., to the United States Agency for International Development. Additional information can be obtained from: Social Impact, Inc. 2300 Clarendon Boulevard Arlington, VA, 22201 Tel: (703) 465-1884 Fax: (703) 465-1888 info@socialimpact.com DISCLAIMER The author’s views expressed in this publication do not necessarily reflect the views of the United States Agency for International Development or the United States Government. TABLE OF CONTENTS Executive Summary................................................................................................................................................................................. 1 Evaluation Purpose & Evaluation Questions .................................................................................................................................... 5 Project Background ................................................................................................................................................................................ 7 Evaluation methods & limitations ........................................................................................................................................................ 9 Findings, conclusions & Recommendations ....................................................................................................................................13 Annexes ...................................................................................................................................................................................................34 Annex I: Evaluation Statement of Work .........................................................................................................................................35 Annex II: Summary of CTCIM Project Objectives and Indicators........................................................................................... 42 Annex III: Evaluation Planning Matrix............................................................................................................................................... 44 Annex IV: Attendees for the Findings Workshop ........................................................................................................................48 Annex V: Gender Analysis ..................................................................................................................................................................50 Annex VI: Key Informants Interviewed ........................................................................................................................................... 69 Annex VII: Additional Analysis Stakeholder Projects, Capacity Building, and Nutrition ....................................................70 Annex VIII: Documents Reviewed and Referenced ..................................................................................................................... 75 Table 1: Data by Source and Sex ........................................................................................................................................................ 9 Table 2: Selected Districts Rated as per Sphere Indicators from the CMAM Database (2008-2012) ........................... 10 Table 3: CMAM Facility Capacity Summary ...................................................................................................................................14 Table 4: Challenges Reported by Visited Facilities....................................................................................................................... 14 Table 5: Reported CAS Project Main Successes ........................................................................................................................... 16 Table 6: Reported CAS Project Weaknesses................................................................................................................................ 16 Table 7: Summary of Operational Plan Commitments................................................................................................................ 22 Table 8: CAS Project Perceived Institutionalization Effectiveness ........................................................................................... 23 Table 9: How do caregivers react if their children are found to be malnourished? ............................................................ 51 Table 10: Do female caregivers react differently than male caregivers? Can you describe any differences? ................ 52 Table 11: Do you have a routine way to identify every malnourished child? ........................................................................ 52 Table 12: Does this routine differ for male and female malnourished children? .................................................................. 52 Table 13: What do you think the main impact of training men in nutrition would be on children's nutrition? .......... 53 Table 14: Gender-Related Evaluation Questions .......................................................................................................................... 57 Table 15: Key Informant Interview Summary Gender Questions ............................................................................................ 62 Table 16: Gender Relations, Constraints, and Opportunities ................................................................................................... 65 Table 17: Gender Analysis Evaluation Questions ......................................................................................................................... 67 Table 18: Impact of Training Men on Nutrition on Children’s Nutrition ..............................................................................68 Figure 1: Topics Covered in Nutrition Trainings.......................................................................................................................... 71 Figure 2: Challenges in Delivering CMAM Services at Facilities ............................................................................................... 72 Box 1: USAID Investment Award Dates ......................................................................................................................................... 7 Box 2: Institutionalization Metric ........................................................................................................................................................ 9 Box 3: Summary of Enabling Factors ................................................................................................................................................19 Box 4: Stakeholder Enabling Factors ................................................................................................................................................23 Box 5: Gender Sub-questions ............................................................................................................................................................ 25 Box 6: Identification of Boys and Girls ............................................................................................................................................ 25 Box 7: KII Views of Male Roles ......................................................................................................................................................... 26 Box 8: Gender Roles and Nutrition ................................................................................................................................................. 54 Box 9: GBV and Stunting ..................................................................................................................................................................... 55 Box 10: Household Nutrition ............................................................................................................................................................ 56 Box 11: MoH Job Descriptions ......................................................................................................................................................... 71 ACRONYMS BS Bachelor of Science CAS CMAM Advisory Services CDC Center for Disease Control CHAI Clinton Health Access Initiative CIDA Canadian International Development Agency CMAM Community-based Management of Acute Malnutrition CTCIM Community-based Therapeutic Care Institutionalization in Malawi CWW Concern Worldwide DHMTS District Health Management Teams DHO District Health Office DONUTS Donors in Nutrition DHIS2 District Health Information Software 2 DHMT District Health Management Team DHS Demographic and Health Survey DIP District Implementation Plan DNHA Department of Nutrition, HIV and AIDS EHP Essential Health Package EPI Environmental Health Officers FGD Focus Group Discussions GoM Government of Malawi HMIS Health Management Information System HSA Health Surveillance Assistances IEC Information Education and Communication KII Key Informant Interviews M&E Monitoring and Evaluation MCH Maternal and Child Health MoH Ministry of Health MUAC Mid-upper arm circumference NECS National Nutrition Education and Communication Strategy NGO Non-Governmental Organization NRU Nutrition Rehabilitation Unit OTP Outpatient Therapeutic Program PHC Primary Health Centers RUTF Ready-to-use Therapeutic Food SAM Severe Acute Malnutrition SFP Supplementary Feeding Program SI Social Impact SOW Scope of work SOP Standard Operating Procedure SSDI Support for Service Delivery Integration SWAp Sector Wide Approach SUN Scaling Up Nutrition UNICEF United Nations International Children’s Emergency Fund USAID U.S. Agency for International Development VCT Voluntary Counseling and Testing WB World Bank WFP World Food Programme WHO World Health Organization CTCIM Final Performance Evaluation - Final Report 1 EXECUTIVE SUMMARY EVALUATION PURPOSE AND EVALUATION QUESTIONS There were three overarching purposes of this final performance evaluation of the USAID/Malawi Community-based Therapeutic Care Institutionalization (CTCIM) project: to measure the extent to which the CTCIM project institutionalized Community-Based Management of Acute Malnutrition or (CMAM) at national and district health offices; to evaluate the capacity of health providers to implement CMAM; and to provide lessons learned and best practices to the Government of Malawi (GoM), USAID/Malawi, and other stakeholders on how best to scale up and institutionalize CMAM in order to combat acute malnutrition in Malawi. The evaluation questions in the scope of work for the assignment mandate an evaluation of how effectively the CMAM Advisory Services Project (CAS) institutionalized CMAM at various levels, to what extent CAS strengthened the monitoring of the CMAM program, the extent to which CMAM was incorporated into the Essential Health Package (EHP), whether stakeholders fulfilled their commitments to the CMAM National Nutritional Operational Plan, and what role gender analysis played in CMAM. PROGRAM BACKGROUND The CTCIM project was implemented by Concern World Wide (CWW) under Cooperative Agreement No: AID-674-A-00-10-00011. CMAM implementation within Malawi began in 2002 and was supported through a series of mechanisms/projects by different donors, including the U.S. government. USAID’s most recent Cooperative Agreement with CWW operated from March 1, 2010 – March 31, 2013 with an estimated total cost of $1,144,019. The core mandate of the CTCIM project was to scale up and institutionalize the CMAM program within the Ministry of Health (MoH) and was guided by four objectives in Malawi: 1. Ensure that Health Service Providers have the capacity to implement and manage CMAM; 2. CMAM is implemented in at least 80% of the Health Facilities in every district; 3. Existence of a sustainable and standardized CMAM Monitoring and Evaluation system; 4. Evidence for support of CMAM is generated through documentation and sharing of best practices and lessons learnt and CMAM is incorporated within the Pre Service Training for Health Service providers. The CTCIM project, also known as the CMAM Advisory Services or CAS, was the next stage in a long￾term collaboration between the MoH of Malawi, CWW, an Irish NGO, and USAID since 2006. There was no mid-term evaluation of the project. EVALUATION QUESTIONS, DESIGN, METHODS AND LIMITATIONS The final evaluation of the CTCIM Project was conducted from June 15 to September 30, 2013. This evaluation was designed to achieve the three objectives and answer the five main questions detailed above. Before the team embarked on the field portion of the evaluation, the team conducted a comprehensive desk review and reviewed quantitative and qualitative project documents provided in part by USAID/Malawi; documents included the MoH health sector strategic plan, the national nutrition policy and strategic plan, and the National Nutrition Education and Communication Strategy (NECS), part of the country’s scaling-up nutrition movement (SUN), and the USAID/Malawi Country Development Cooperation Strategy. Upon arriving in the field, the evaluation team conducted key informant interviews with major stakeholders in Lilongwe and sent out an email survey to the remaining CMAM partners. Data collection was conducted in 10 districts where the team interviewed community key informants, facility staff and CTCIM Final Performance Evaluation - Final Report 2 frontline health providers, and conducted focus group discussions (FGD) with various CMAM beneficiaries. The team held a Findings Workshop towards the end of their fieldwork in Malawi and disseminated the evaluation’s findings and results to key stakeholders. Careful attention to gender was paid throughout the design of the team’s data collection instruments and during fieldwork, and sex disaggregated data was collected and analyzed when possible. An Institutionalization Metric was also created by the SI team, and helped guide the creation of data collection instruments by keeping in mind the various measures that go into sustaining the CMAM program. There are a number of limitations to be considered when looking at the results of this evaluation. First, some factors and lessons learned may not have been identified given the cross-sectional nature of the evaluation. Second, the team was unable to ascertain the many factors that contributed to the project’s results, particularly because the project evolved over time from a rapid humanitarian response to a food crisis into a nationwide program that responded to various challenges and opportunities throughout the years. Third, the quantitative findings are representative of a small purposive sample of facilities and may not represent the situation in all of the country’s facilities. Therefore, the facility findings are indicative of issues that the zonal and districts teams may wish to explore further during supervisory visits. Additionally the results from the team’s email survey were helpful in revealing key information, but given the survey’s low response rate, the responses cannot be viewed as statistically representative of all stakeholders’ views. Fourth, the main findings of this evaluation depend heavily on the reports of key informants who are part of CMAM, and are in one way or another responsible for its performance and effectiveness. Lastly, the evaluation team was not able to speak to several key informants whose views could have provided additional context and information to the findings of this evaluation. The evaluation team controlled for informant recall bias by asking participants questions about their current experiences with CMAM rather than focusing on past experiences with the program. The subjectivity of the interviewed informants was controlled by triangulating the data from various informants and data sources at national, district, facility and community levels. In spite of these measures, qualitative data may not represent the views of all or even the majority of the CMAM stakeholders and beneficiaries, but indicates emergent issues and views. And lastly, CMAM is a large program and many health system strengths and weaknesses affect its performance. When possible, the team tried to ascertain these factors but a health system performance assessment was beyond the scope of this evaluation. FINDINGS AND CONCLUSIONS The findings demonstrate that the CTCIM project met all of its objectives and handed over its deliverables as planned. The project developed capacity to implement CMAM services in 81% of the facilities and created its own monitoring and evaluation (M&E) tool, which for a long time was the only M&E tool available to the MoH to monitor malnutrition at national and district levels. The CAS model was reported to be an effective way to assist national and district level teams and mentor staff. CAS also documented and shared best practices through Learning Forums that gathered district staff and helped them learn from one another. All interviewed health staff reported that these forums were very useful and would be missed if they were not continued. CMAM guidelines allow for the equal treatment of male and female children. The sex-disaggregated data collected by the districts was useful for understanding more about the beneficiary population, but it did not help with case management since each child’s status cannot be followed up after discharge from a nutrition rehabilitation unit (NRU). As a result, providers do not know if a child improved or survived after treatment. CTCIM Final Performance Evaluation - Final Report 3 In an end of project summary document, CWW identified important lessons learned on ownership, keeping an exit strategy in mind from start of the project and coordinating with all stakeholders. Key informant interviews emphasized the need for country ownership given the numerous stakeholders. This was also confirmed by the participants of the Findings Workshop. Overall, the CAS model helped build capacity and scaled up CMAM services at the national level and handed over its activities, but did not fully institutionalize CMAM. Objective 1.To what extent has the CAS model been effective to institutionalize CMAM? The final evaluation determined that although many CAS activities built capacity and was transferred to the MoH at national and district levels, institutionalization was not completed. The handover schedule was followed with a few delays but the organizational development required for the MoH to assume the project’s advisory and technical functions at the zonal level was not part of the project design and did not take place. In the case of the MoH Nutrition Unit, the transfer was not effective due to the lack of alignment of CAS activities with the decentralization process. The MoH Nutrition Unit should not be expected by donors to manage implementation of the nutrition program but to oversee it. CAS’s technical functions may have been better assumed by zonal teams and the CMAM coordinators themselves in every district. The transfer of the CMAM M&E tool will require more time as the District Health Information System 2 (DHIS2) will assume some of the program information needs. It has not been decided how DHIS2 will be modified to do this. At the facility level, health staff requires case management information for which there is no tool yet in place. Objective 2. What factors enabled and inhibited the effective institutionalization of CMAM? The evaluation identified enabling and inhibiting factors that influenced the institutionalization process. Among the former is the presence of a clear commitment of the Government of Malawi (GoM) to improve the nutritional status of children and pregnant and lactating women, and increase the care given to people living with HIV/AIDS. These are the GoM’s two main priorities, and have been elevated to the Cabinet level. Additionally other enabling factors are the existence of a clear national nutrition policy, a gender policy as well as the dedicated CAS technical advisors that built capacity, created program guidelines and conducted Learning Forums to share best practices. Districts and facilities have trained staff, and frontline workers were reported to deliver services according to CMAM guidelines and to use the CMAM M&E tool. One of the main inhibiting factors identified through key informant interviews and health provider surveys is the weak coordination of all stakeholders at the national and district levels. There is a donor coordinating group called Donors in Nutrition (DONUT), which aims to streamline the coordination process, but it is not chaired by the MoH but by donors on a rotating basis. This group seems to have a ―traditional campaign‖ approach to addressing malnutrition if the NECS is a reflection of their collective focus. It was not clear if other donors were also aligned with the institutionalization agenda of the CTCIM project and with the ongoing decentralization process of the GoM. Providing assistance at the national level with organizational development to assume new and more effective leadership roles to meet the needs of a growing population in a decentralized setting was reported to not have been part of the CTCIM project and the design of the CAS model. Objective 3. To what extent did CMAM stakeholders fulfill their responsibilities in support of the institutionalization process? CMAM stakeholders are donor and NGO organizations that contribute to implement the program. Stakeholders appear to be outside of the decentralization process, and are not completely aligned with the institutionalization goal. The team did not find evidence of a sustainable CMAM institutionalization written agenda. The evaluation showed that the CTCIM project was not designed to strengthen CTCIM Final Performance Evaluation - Final Report 4 ownership however. There are systemic gaps like lack of consensus on the growing role of HSAs, which prevents the institutionalization process, and stakeholders do not have formal structures and operating procedures to allow the MoH to coordinate and account for each stakeholders’ contribution. The interviewed stakeholders reported to not have an agenda that includes ownership and coordination with a common institutionalization goal. RECOMMENDATIONS The four general recommendations below are offered as a way to help spur immediate action and gain rapid momentum to sustain the achievements of the CTCIM project, help the MoH Nutrition Unit to lead and coordinate the institutionalization and implementation of the CMAM program, and contribute to strengthening Malawi’s health system: 1. Build on the strengths of Malawi’s CMAM program by assisting zonal and district teams via training and technical assistance to sustain the enabling factors identified. 2. Improve the organizational development of new CMAM program management and service delivery processes in order to ensure compliance of all stakeholders. 3. Monitor staff and recognize them for their achievements to help sustain the observed high levels of service delivery; institutionalizing and increasing the frequency of supportive supervision of CMAM staff will help achieve this and will create incentives for continued success. 4. Build the health system to effectively deliver nutrition services and treat malnutrition for the long run. Envisioning the type and scale of nutrition services that will be required in 10 years will help strengthen and sustain the health system for future populations; this strategic process should commence immediately. Stakeholders and partners might help achieve a common long￾term vision for CMAM and help mitigate and prevent issues in the near future. CTCIM Final Performance Evaluation - Final Report 5 EVALUATION PURPOSE & EVALUATION QUESTIONS EVALUATION PURPOSE There were three overarching purposes of this final performance evaluation: to measure the extent to which the CTCIM project institutionalized CMAM at national and district health offices, to evaluate the capacity of health providers to implement CMAM, and to provide lessons learned and best practices to the GoM, USAID/Malawi, and other stakeholders, on how to best scale up and institutionalize CMAM to combat acute malnutrition in Malawi. This evaluation had three main objectives: 1. To learn to what extent the model of providing technical assistance through CAS has been effective in institutionalizing CMAM within the MoH, specifically at district hospitals. 2. To understand the key factors that enabled or inhibited effective institutionalization of CMAM. 3. To understand the extent to which all stakeholders in the CMAM operational plan fulfilled their responsibilities in support of the institutionalization process The evaluation was co-funded by USAID ($165, 831.65) and the Canadian International Development Agency ($100,000 Canadian dollars) for a total of $265,831.63. The evaluation was conducted from June through September 2013, with field work taking place in June and July. This utilization-focused evaluation was designed to respond to the five evaluation questions1 below, and meet the information needs of key CMAM stakeholders: GoM, MoH, USAID, Canadian International Development Agency (CIDA), UNICEF, World Health Organization (WHO), World Bank (WB), World Food Program (WFP), Clinton Health Access Initiative (CHAI), and CWW. EVALUATION QUESTIONS 1. To what extent has CMAM built capacity at various levels? a. To what extent has CAS built the capacity of health service providers to implement CMAM? b. To what extent has CAS built the capacity of district and national level managers to manage the CMAM program? c. To what extent has CAS built the capacity of pre-service training institutions to integrate the CMAM approach into the curriculum? 2. To what extent has CMAM been incorporated into the Essential Health Package and is delivered routinely at district hospitals and primary health care service delivery points? a. What elements of CMAM have district health offices successfully been able to adopt and incorporate into their routine management and implementation systems? b. What elements of CMAM have district health offices not successfully been able to adopt and incorporate into their routine management and implementation systems? c. What have been the key factors that have enabled CMAM institutionalization in districts identified to have the capacity to independently manage and implement CMAM? d. What have been the key factors that have limited CMAM institutionalization in districts identified to have the capacity to independently manage and implement CMAM? 1 The USAID evaluation statement of work (SOW) is provided in Annex A. CTCIM Final Performance Evaluation - Final Report 6 3. To what extent has CAS strengthened the monitoring of the CMAM program? a. To what extent has the CAS built capacity in ensuring the development, training, dissemination, and use of standardized data collection, compilation, and reporting of CMAM monitoring data? b. To what extent has CAS ensured systems to ensure data quality, timeliness, and completeness of CMAM monitoring data? c. To what extent has CAS built capacity of facility, district, and national stakeholders to synthesize and use data for decision making? 4. To what extent did stakeholders of the CMAM National Nutrition Operational Plan fulfill their roles and commitments? a. What are the key factors that have enabled stakeholder commitment to the CMAM Operational Plan? b. What are the key factors that have limited stakeholder commitment to the CMAM Operational Plan? 5. What role has gender analysis played into the CMAM? a. What conclusions can be drawn from monitoring data regarding the uptake of CMAM services by gender? b. What are the factors that hindered or promoted male involvement into CMAM, and what additional inputs are needed to enhance male involvement? The evaluation team consisted of Dr. Elvira Beracochea, Team Leader; Dr. Sarah Tisch, Gender Specialist; Sarah Weber, Nutrition Specialist; Deborah Bickel, Health System Specialist, Deric Zanera, Research Assistant; and Arnold Mndalira, Logistician. CTCIM Final Performance Evaluation - Final Report 7 Box 1 USAID Investment Award Dates: a. February 2010-July 2013; Funding Amount – $1,144,019 b. May 2008 – April 30, 2009 Funding Amount--$300,000 c. August 2006-December 2006 Funding Amount--$50,000 Total Funding: $1,494, 019 PROJECT BACKGROUND THE CONTEXT Malawi has a population of approximately 15 million people. Although Malawi is a low-income country, it is taking significant steps to improve its health sector, particularly by increasing the number of health providers by 53%. These recent improvements have led to a reduction in infant and child mortality, and HIV rates. In 2005, 52% of under-five childhood mortality was due to malnutrition; the estimated prevalence of severe acute malnutrition (SAM) remains approximately 1%2. With over half the population living in poverty, heavy reliance on maize production, chronic food insecurity and poor access to malnutrition management services threaten to undermine the recent improvements in male and female child health trends. Because the incidence of malnutrition is a seasonal event from October to March, it is important to scale up delivery of nutrition services. To overcome these challenges, USAID/Malawi awarded a cooperative agreement with Concern Worldwide to implement the CTCIM activity of CMAM. CMAM, a decentralized form of treatment for moderate and severe acute malnutrition, is broadly recognized as a best-practice in the treatment of malnutrition cases without medical complications. In addition, Malawi is in the process of decentralizing government functions to improve efficiency, and place a greater responsibility at the District health level. The government has enacted some of the most progressive gender legislation in Africa, advancing efforts to make gender equality a reality. These achievements are impressive and will aid in the efforts to sustain the pace of strengthening the delivery and access to nutrition and health services, which is crucial since the Malawian population is expected to reach 25 million people by 20253. PROJECT DESCRIPTION The implementation of CMAM within Malawi has been supported by CIDA, Irish AID and others since 2002. USAID has had a long involvement in the prevention and management of acute malnutrition within Malawi and has supported CWW since 2006 (Box 1). In spite of this investment and recent progress achieved by CTCIM in treating acute malnutrition, malnutrition continues to be a problem in Malawi stemming from many factors beyond the scope of the CTCIM project, such as poverty, illiteracy, seasonal food scarcity, the country's reliance on maize, and the lack of household food diversity and food preparation knowledge. The CTCIM project was designed to address the need to institutionalize the nationwide CMAM program, which provides care to malnourished and severely malnourished children under the age of 5 through a four-prong strategy: treating severely malnourished children with medical complications in a NRU; treating severely malnourished children without medical conditions in outpatient therapy; providing a supplementary food program to underweight children; and providing outreach and community mobilization. 2 Malawi National Nutrition Policy and Strategic Plan, 2009. 3Ceteris paribus, assuming all factors remain the same. CTCIM Final Performance Evaluation - Final Report 8 The CTCIM project had four objectives: 1. Ensure that health service providers have the capacity to implement and manage CMAM; 2. Implement CMAM in at least 80% of the health facilities in every district; 3. Existence of a sustainable and standardized CMAM M&E system; 4. Evidence for support of CMAM is generated through documentation and sharing of best practices and lessons learnt and CMAM is incorporated within the pre-service training for health service providers. CTCIM’S DEVELOPMENT HYPOTHESIS This evaluation assumed that if the inputs of the stakeholders and the on-going technical assistance provided to MoH through CAS, effectively scaled up the integration of CMAM in the Essential Health Package; and institutionalized CMAM effectively, then nutrition service coverage would be sustained and increased, and nutrition-related morbidity and mortality in children would reduce. CTCIM Final Performance Evaluation - Final Report 9 Box 2. Institutionalization Metric  A measure of how well CMAM services are fully managed and supervised by District Health Offices and the Ministry of Health.  A measure of how well routine identification and treatment of male and female malnourished children through health facility services and outreach work.  A measure of how sustainable funding is through government funding channels.  A measure of how well the logistics and reporting systems are integrated within existing government systems, particularly the HMIS.  A description of the incorporation of CMAM into pre￾service training curricula. EVALUATION METHODS & LIMITATIONS This evaluation was designed to answer the five evaluation questions in as rigorous a manner as resources allowed. The corresponding methods for each question are listed in Annex III and were described in detail in the Inception Report approved by USAID/Malawi. This evaluation was guided by the CTCIM intervention logic, as captured through the development hypothesis described in the previous section. The evaluation team utilized a multi-level, mixed-methods design that combined quantitative and qualitative evaluation methodologies including a comprehensive desk review, facility, health provider and key informant interviews, and focus group discussions. The capacity of districts and facilities to deliver CMAM by health staff in NRUs and in Primary Health Centers (PHCs) was also a focus of this evaluation. Selected factors that affect the functioning of health facilities and the working conditions of health providers also affected the institutionalization of CMAM, and were evaluated through a facility survey and a health provider interview. The key informant interviews served to understand the quantitative data, and the FGDs helped to understand the perceptions of male and female beneficiaries concerning CMAM services, and issues related to child nutrition and gender. The evaluation team also presented selected findings at a workshop with the main stakeholders and facilitated the prioritization of actions for further strengthening of CMAM implementation. The evaluation was guided by an institutionalization metric, displayed in (Box 2), which was developed by the SI team. This metric helped inform the design of the data collection tools and facilitated the creation of the evaluation’s conclusions and recommendations. DATA COLLECTION Data collection was conducted by two teams; each team was comprised of two evaluators. Both teams interviewed key informants as identified by USAID/Malawi, and aimed to interview an equal number of sexes; 52% of participants interviewed were women and 48% were men. Table 1. Data by Source and Sex The evaluation team used a stratified purposive sampling method to select facilities to visit during fieldwork. Key informants were selected on an opportunistic basis as to their availability at a particular facility. As well, the team was provided with a list of CMAM stakeholders to contact for interviews. FGDs were convened by health care facility staff based on availability. An equal number of males and females were interviewed, whenever possible and as appropriate. Data collected was disaggregated by sex, which is detailed further in Table 1. Participants Female Male Total Focus groups 36 30 66 Key Informants 17 27 44 Health providers 13 12 25 Totals 66 (49%) 69 (51%) 135 (100%) CTCIM Final Performance Evaluation - Final Report 10 After pre-testing the data collection tools in a health center in Lilongwe district, each team visited the district hospital and the nearest primary health center in the five districts when possible. The team recognized that there could be potential for bias in visiting facilities located closer to the city since these facilities may have better access to resources than more isolated facilities, and perform better as a result. The evaluation team did not observe large differences in the delivery of CMAM at the facilities visited however. Since the goal of the evaluation was to assess the level of CMAM institutionalization of the CTCIM project at the facility level, the districts were selected based on their performance on Sphere standard indicators, such as death rates, cure rates and default rates4 according to the most recent CTCIM CMAM database M&E reports. The final district selection is listed in Table 2. However, it should be noted that this database did have numerous gaps due to missing or incomplete data, which complicated efforts to determine district performance levels. Further description of the development of the survey instrument tools is in the Gender Analysis Index, in Annex V. ANALYSIS Quantitative data from the facility and email surveys were analyzed calculating variable frequencies to measure the most and least common responses to survey questions. Triangulation of the various complementary data collection methods and sources strengthened the validity of findings and provided strong, evidence-based recommendations to inform future CMAM programming decisions. Given the unknown level of quality of CMAM data at the evaluation team’s disposal (a data quality assessment was not part of this evaluation), the evaluation team made every attempt to mitigate this limitation by acquiring data from a diverse array of reliable sources such as district studies and district implementation plans (DIPs). The team also explicitly noted this threat to validity when performance data was used to construct site selection criteria; all data limitations and the effects thereof are described in detail in the respective section. Gender Analysis The evaluation team conducted a gender analysis to identify whether there were any inequalities between males and females in the CMAM program, and analyzed how these gaps were addressed throughout implementation. This analysis informed the development of recommendations on how to achieve gender equality results. As noted in the 2010 Demographic Health Survey (DHS), the prevalence of stunting and underweight is higher among boys than girls5. Given the influence men have with the overall welfare of families, we examined the perceptions of men toward CMAM services and how the CAS worked with districts to improve outreach toward men. Using the quantitative and qualitative data, we examined factors such as access to assets (CMAM services); knowledge, beliefs, and perceptions; practices and participation; time and space issues; legal rights and status; and the balance of power and 4 See: http://www.spherehandbook.org/en/management-of-acute-malnutrition-and-micronutrient-deficiencies-standard-2- severe-acute-malnutrition/ 5 Malawi Demographic and Health Survey 201, p. 130-132. National Statistical Office, Zomba, Malawi and ICF Macro, Maryland, U.S. 2011. North Center South Karonga Dowa Blantyre Dedza Thyolo Mzimba North Kasungu Mangochi Salima Zomba Table 2. Selected Districts rated as per Sphere indicators from the CMAM database (2008-2012): navy districts are high performers, light blue are average and red are low performers. CTCIM Final Performance Evaluation - Final Report 11 decision making.6 The uptake of CMAM services for male and female children were analyzed during the field work through the facility survey and interviews with healthcare providers. The gender analysis helped identify factors that may have hindered or promoted male involvement with the CMAM program, and additional inputs needed to increase male involvement (Evaluation Question #5). See Annex V for the full gender analysis tables and literature review. LIMITATIONS The team acknowledges that there are inherent biases and limitations associated with this methodology. The main persistent bias in the evaluation was that it was a cross-sectional view of the results achieved by the CTCIM project. Because of this, the team was unable to appreciate the long-term involvement of CAS with all stakeholders resulting from CWW’s work since 2002. It is likely that the evaluation was not able to identify every enabling and inhibiting factor of the CTCIM project as a result. In the data collection process, there is the potential for recall bias amongst key informants; the team took steps to reduce recall bias in the protocol design phase. This included framing questions about the current situation of the CMAM program to aid accurate recall. Where possible, the team also used alternative sources to corroborate interview findings. The team also acknowledges the potential for bias due to the subjectivity of the views of respondents. The SI team purposively recruited a diverse sample of respondents and triangulated responses. Since the team was unable to avoid all bias in the data, persistent biases were taken into account during the analysis and interpretation phase and were acknowledged during analysis and dissemination. Another key limitation to the evaluation concerned establishing the validity of site visit findings since certain sites could not be visited because they were physically inaccessible due to distance and/or infrastructure challenges. The team made every reasonable effort to visit sites in 10 districts to produce a fully representative sample to understand what was to be expected of high and lower performing districts. The team has no reason to believe the excluded districts are fundamentally different from the ones selected for the evaluation. Additionally, the findings from an email survey that was sent to all CMAM partners provided a low response, and cannot be considered a valid statistical representation of the views of all CMAM partners. Of the 202 email addresses included in the CMAM partners’ database, only 21 people, or 10.4%, completed the online survey through Surveymonkey, and many respondents’ skipped questions. A final limitation to this evaluation was that the team was unable to interview several key informants, including the representative of the Clinton Health Access Initiative, the WHO representative, and the World Bank team in charge of the new nutrition activity. The team was also unable to interview the USAID Support for Service Delivery Integration (SSDI) project team or the USAID heads of the Health or HIV/AIDS Teams, and the officers in charge of the Human Resources for Maternal and Child Health (MCH) at USAID/Malawi. These informants may have added additional perspectives regarding the evaluation questions and facts that were unavailable to the evaluation team. DISSEMINATION: FINDINGS WORKSHOP The evaluation Findings Workshop was held on July 17, 2013 at the Ufulu Hotel in Lilongwe. Participants included key members from the GoM, USAID, CIDA, representatives from the 10 districts, 6 A Practical Guide for Managing and Conducting Gender Assessment in the Health Sector, Margaret E. Greene. Interagency Gender Working Group. USAID June 2013. http://www.igwg.org/Articles/genderassessmentsguide2013.aspx CTCIM Final Performance Evaluation - Final Report 12 CWW and other major stakeholders. A list of attendees is in Annex IV. The objectives of the workshop included the following: to disseminate CAS project results; to present the selected evaluation findings; to establish a consensus on a common vision for CMAM; to reach an agreement on the desired results in the light of the evaluation findings, to identify immediate next steps and clarify the roles and responsibilities of stakeholders, and to coordinate accountability activities for the GoM and stakeholders. The conclusions of the participants indicated that they were able to identify at least one immediate action they could undertake, and that they could apply the findings to daily work. CTCIM Final Performance Evaluation - Final Report 13 FINDINGS, CONCLUSIONS & RECOMMENDATIONS FINDINGS Question 1: To what extent has CMAM built capacity at the national, district and local levels? The desktop review of project documents conducted prior to the visits to the 10 districts showed that the CTCIM project achieved its objectives and developed useful and effective tools and best practices for institutionalization. (Please see a full list of documents consulted in Annex VIII). The project indicator table (Annex II) also confirms that the project met all its deliverables and built capacity at the national, district and local levels; the institutionalization metric used by the team revealed that the coordination of implementation and supervision at the district level is not standardized and streamlined, which is discussed further below. The development of an exit strategy at the start of the project helped avoid creating unsustainable structures or procedures however. The CAS model was also reported as helpful in training health care providers and facilitating the sharing of good practices among district health teams. The key informants and the stakeholders interviewed through an email survey reported that CAS was an effective model to build capacity at district and facility levels. National MoH staff reported that having advisors posted in the MoH helped align managers to better address challenges. Most of the email survey respondents (11 out of 13) reported that in their opinion CTCIM had met or exceeded expectations. Only two reported that the project only partially met their expectations. a. To what extent has CAS built the capacity of health service providers to implement CMAM? To assess the extent of the capacity built, the team looked for evidence that addressed the following criteria: 1. Whether providers demonstrates the necessary knowledge to implement CMAM 2. Whether there was at least one individual reported to deliver CMAM services 3. Whether job descriptions included CMAM 4. Whether tools and equipment were available to deliver CMAM services in facilities 5. Whether supervision takes place as reported by Health providers 6. Whether providers can use the CMAM information system and report on CMAM indicators 7. Whether documentation of standard operating procedures (SOPs) is available. The team visited 16 facilities in 10 districts; 58.3% were urban district hospitals and 41.7% were rural multiple function health facilities. There were no marked gender differences in the number of interviewed providers: 52% were female and 48% were male. Results show that out of the selected sample, 78% of health providers reported having written job descriptions, while 58% said that CMAM is included in their job description. The team understands that CMAM is being added as a function to health care provider’s job descriptions, which illustrates that the institutionalization of CMAM is in progress. Based on the quantitative data of the facility survey, we ascertained that item number 7 above, the availability of the documentation of SOPs, was not present at all visited facilities at the time of the survey. Moreover, written procedures and operations manuals were not always available, and the inclusion of CMAM in job descriptions is not uniform. The second item, the number of providers required, is still reported as insufficient but this is a systemic issue that affects the whole health system in Malawi. In any case, we found health providers performing CMAM activities in all visited facilities. CTCIM Final Performance Evaluation - Final Report 14 The person in charge of the facility was interviewed to evaluate the capacity of the facility to implement CMAM and capacity gaps were identified. For example, all the facilities had clean private consultation areas but the majority of patients were seen in open waiting areas. Voluntary Counseling and Testing for HIV services were provided by most facilities in a separate area or building in the same compound and not integrated with CMAM. The majority of the health facility staff reported a desire for more refresher training. The health provider interviews illustrated that 48% (12/25) of providers reported receiving some CMAM training, and nine providers or 36%, reported having been trained by CAS staff. This could be due to a lack of a clear understanding regarding what constitutes CAS and the reach of CAS’s support. The CMAM Facility Capacity Sumary, displayed in Table 3 below, provides an overview of how CAS strengthened the capacity of CMAM. By the end of the CTCIM project, the results of the capacity built were seen in the national coverage of CMAM, which reached 100% of the NRUs, 84 % of Out Patient Therapy (OTP) services, and 58% for Supplementary Food Programs (SFP) (CWW report 2013). CMAM services are delivered in all 28 districts (29 District Health Offices (DHOs)), and 24 of those districts currently provide CMAM services in 80% of their health facilities (CWW report 2013). CAS effectively built the capacity of the health system to provide and manage CMAM services in over 80% of facilities, thus meeting the project goal. In addition to the expanding coverage of service, the CTCIM project successfully achieved a cure rate that was above the recommended Sphere standard of 75%, kept death rates under the recommended Sphere rate of 10%, and achieved a default rate7under the recommended Sphere rate of 15%. It was evident to all key informants that CAS significantly built the capacity of health care providers to successfully implement CMAM. CMAM providers reported to have observed generally positive changes in the program at the facility level (63%) in the last two years. These positive changes included early identification of malnourished children (21%), as well as regular feedback from the community (5%), although pockets of increased rates of malnutrition (11%) were also mentioned. 7 The default rate is the rate at which children abandon the CMAM program or fail to follow-up. Indicator Number N % Has job description 18 25 72 Received Training by CAS 12 25 48 Received supervision 11 25 44 Job description includes CMAM 11 25 44 Last time received supervision months ago 9 25 38 year(s) ago 8 25 36 weeks ago 1 25 4 Immediate supervisor CMAM coordinator 7 25 28 Senior HSA 4 25 16 District Nutritionist 3 25 12 District Nursing Officer 2 25 8 SHSA and Clinical Officer 1 25 4 Challenges to deliver CMAM services Number N % Lack of CMAM resources-MUAC, Scales, 14 21 66 Lack of transportation for HSAs to get to communities 4 21 19 Stock outs of RUTF 2 21 9 Training of all health facility providers 1 21 5 Reduced number of malnourished children 1 21 5 Table 3. CMAM Facility Capacity Summary (Source: Facility Survey) Table 4. Challenges Reported by Visited Facilities (Source: Facility Survey) CTCIM Final Performance Evaluation - Final Report 15 Although CAS was cited as having helped build the capacity of many of the visited health facilities, several challenges remain including delivering health services, lacking necessary equipment, supplies and ready to use food (RUTF). Please see Table 4 for a detailed list of the most common challenges reported by the visited facilities. RUTF is an essential supply of the CMAM program. The majority of health care providers mentioned lack of storage (75%) and late delivery of CMAM supplies as core reasons for RUTF stock outs that occurred in the North a month before the evaluation took place. At the time of the health facility survey, all facilities had RUTF in stock and more abundantly in the Central and South districts visited, where malnutrition prevalence is usually higher. In addition to occasional stock outs of RUTF, health care providers lack other essential CMAM resources, such as mid-upper arm circumference (MUAC) and scales. A lack of transport to communities for the HSAs to screen and identify cases was also reported as a main challenge that limits the capacity of health providers to implement CMAM, especially in districts where health care providers have large catchment areas. The evaluation team asked healthcare providers if there were any special procedures for health care providers and facilities to prepare for the reported seasonal increase in the incidence of acute malnutrition. No one the team interviewed indicated that there was a seasonal difference within planning, and supplies were ordered following usual processes. Despite these challenges, facility personnel communicated that they took proactive measures to address these service delivery challenges. The majority of key informant demonstrated they are well-informed on the implementation of the CMAM program and have the capacity to implement CMAM. b. To what extent has CAS built the capacity of district and national level managers to manage the CMAM program? National Level Capacity. During the life of the project, the document review showed that the GoM has prioritized nutrition and elevated it to the Office of the President. Key informants reported that CAS played a significant role in providing guidance, support and coordination, and in serving as a ―technical arm‖ for the MoH in managing the CMAM program. Key informants also reported that CAS assisted with the integration of CMAM into the national health policy and strategic plans, the development of the national CMAM guidelines, and the training of national CMAM trainers. The extent of the CTCIM project’s reach at the national level is best illustrated by the fact that the GoM developed the capacity to successfully implement CMAM in most of the country. It should be noted that during the period of CMAM implementation, the stunting rate in children decreased from 53% in 2004 to 47% in 2010 (DHS survey 2010). It is outside the scope and purpose of this evaluation to understand whether any of this success, and if so how much, can be attributed to CTCIM and the work of the CAS, however it is clear from this evaluation that the CMAM program is working. CTCIM Final Performance Evaluation - Final Report 16 Table 5. Reported CAS Project Main Successes (Source: Email Survey) Nutrition policy and the creation of standards and oversight of the effective implementation of CMAM policy is the responsibility of the MoH Nutrition Unit. This is a small unit at the national level whose role and composition is in line with the decentralization process. The role of CAS advisors that supported the Nutrition Unit with the decentralization process was reported to be helpful by key informants. Stakeholders reported in the email survey to have appreciated many of the contributions made by the CTCIM project (Table 5). Stakeholders also reported a number of perceived weaknesses of the CTCIM project and CAS seen in Table 6. Table 6. Reported CAS Project Weaknesses (Source: Email Survey) What have been the weaknesses of the CAS model, if any? Options Responses N % Not institutionalized completely 7 13 53.8 Not integrated into the Essential Health Package completely 3 13 23.1 Not completely responsive to the needs of the Nutrition Unit of the MoH 3 13 23.1 Best practices and lessons learned not documented or shared 2 13 15.4 None 2 13 15.4 Not sustainable M&E system 1 13 7.7 Not built sufficient capacity 1 13 7.7 Not integrated in pre-service training 1 13 7.7 These weaknesses are also mostly aligned with the issues identified by CWW as lessons learned in the CTCIM project as per their final report; and CWW highlights a need for the institutionalization process to continue. District Level Capacity. All district level key informants reported that CAS was instrumental in building the capacity of the districts to manage the CMAM program. CAS provided the districts with general guidance and advice. CAS also helped increase coordination with stakeholders by training district CMAM focal persons as part of the District Health Management Teams (DHMT). Malawi has five zonal teams which provide technical support to their respective district teams. CMAM coordinators were found in every visited district, but the coordination function was reported to not be explicitly assigned to a member of the Zonal Teams. CAS also built the capacity of the DHMTs by implementing ―Learning Forums‖ which would facilitate the sharing of best practices and overall learning. The Learning Forums In your opinion, what have been the three main successes of CAS? Options Responses N % Ensured that health providers have capacity to implement CMAM 9 11 81.8 Ensured that CMAM is implemented in most facilities in every district 9 11 81.8 Ensured that health providers have the capacity to manage CMAM 5 11 45.5 Developed a sustainable and standardized M&E system 3 11 27.3 Conducted Learning Forums 3 11 27.3 Trained National Trainers 2 11 18.2 Development CMAM training materials 2 11 18.2 Conducted supervision 2 11 18.2 CTCIM Final Performance Evaluation - Final Report 17 were reported to empower district staff to make decisions to keep the implementation of CMAM moving forward, and also improved coordination among donors. These meetings provided a venue for implementing providers and district staff to come together and share best practices and ways to overcome challenges. CAS staff provided technical advice and on-demand training in these meetings. The CAS team reported that they helped align donors’ inputs for district distribution of critical supplies during the initial phases of the program. While there are challenges with the management and implementation of CMAM at the district level, district key informants reported that CAS was a mechanism that successfully provided them with the necessary technical support. As described earlier, the CMAM database, categorizes districts into high and low-performing areas using the Sphere standards. We did not find evidence in the facilities visited that would explain the differences in the district indicators. District informants in both the Sphere designated high and low performing districts reported to have occasional shortages of RUTF and district health care providers have transportation problems that prevent them from supervising providers in more rural facilities. All of the visited facilities had trained healthcare providers and were able to report what CMAM activities they implemented correctly. Most facilities were able to show program records, demonstrate the use of CMAM standards, and reported using the CMAM database. The evaluation team concluded that all of the facilities in the districts visited had both weaknesses and good practices to share. The KIIs with the main district health offices indicated that CAS facilitated the coordination among CMAM stakeholders working in the districts, and provided supportive supervision and follow-up, which either occurred in-person, by telephone, text message or email. KIIs also mentioned that CAS staff made themselves available to answer questions, provide advice, mentorship, and feedback. All these KIIs maintained that CMAM was successful in identifying early cases of acute malnutrition. The evaluation did not find evidence that CAS built the capacity of zonal teams beyond providing them with CMAM training, guidelines and supervision. Organizational development activities were not conducted and specific SOPs were not found. The CAS project did not build the capacity of the zonal teams during the initial years and while some emphasis has been put on training zonal trainers, it was reported to have come late in the process and follow-up trainings have been reliant on donor funding. The perceptions and views reported by several Key Informant Interviews (KIIs) at the district level are presented below: Most districts report to perceive improvements and fewer cases of malnutrition: ‖ The number of children enrolled in nutrition [Supplementary Feeding Program (SFP)] are reducing. Rare to hear a death by malnutrition now.‖ -Blantyre District There are implementation problems even in high performing districts such as Thyolo, and providers need to be empowered with effective problem-solving skills and a proactive attitude to overcome problems such as lack of volunteers: "The infrastructure for the NRU is inadequate; there is not enough outpatient therapy program (OTP) and SFP. There are not enough volunteers in the community to assist." Thyolo District c. To what extent has CAS built the capacity of pre-service training institutions to integrate the CMAM approach into the curriculum? The capacity of pre-service training institutions to incorporate CMAM into curricula is still reported to be incomplete. According to the CAS implementation plan, four key academic institutions were targeted to incorporate modules for the management of acute malnutrition by 2012. At the time of the evaluation, only one institution has reported to have a draft of these revised curricula, and none have finalized curricula. CTCIM Final Performance Evaluation - Final Report 18 KIIs were conducted at the Bunda College of Agriculture, Department of Home Economics and Human Nutrition, the Kamuzu College of Nursing, and the College of Medicine. Syllabi from the nutrition courses were also reviewed. All three institutions include training on general nutrition, identifying malnutrition, protein and micronutrient deficiencies, food supplementation (including SFP) and practical assessments with anthropometric equipment. However, key informants from all three institutions identified the limited scope and coverage of this material. The Bachelor of Science (BS) in Nursing and Midwifery at Kamuzu College of Nursing includes 15 lectures in general nutrition during the first year (of which identifying and treating acutely malnourished patients is included but very limited). Training includes a visit to the NRU for two days during the practical component, but this does not occur until student's third year of the program so the time gap may limit their recall of this information. The College of Medicine had a similar training with a week of lectures and a six-week practical. The BS in Nutrition and Food Science at Bunda College of Agriculture includes training on screening and identifying malnourished patients, information about OTPs and NRUs, as well a visit to the NRU. The training time devoted to the CMAM approach is only two hours. For a program specifically addressing nutrition and food science, this limited time devoted to training students on CMAM is a major shortfall. Although the academic institutions visited reported to have incorporated aspects of identifying and treating acutely malnourished patients; they do not consistently train students on nutrition counseling in the context of malnutrition in Malawi. While some limited attention is directed towards community mobilization, the key informants reported that the curricula do not specifically address nutritional counseling in communities. Furthermore, the KIIs said their curriculum did not cover community awareness about nutritious foods to eat during times of food insecurity and famine, or how to counsel community members on nutritious food choices during periods of food shortages. This was reported to be a major shortcoming in the curriculum of these training institutions, given the persistent malnutrition within Malawi. Question 2: To what extent has CMAM been incorporated into the Essential Health Package (EHP) and is delivered routinely at district hospitals and primary health care service delivery points? In the documents reviewed, CMAM is listed as part of the EHP, but based on health facility interviews and KIIs, CMAM is often seen as an add-on. Although delivered routinely as a separate program in hospitals and PHC facilities, CMAM is viewed as an important program that saves the lives of many children. a. What elements of CMAM have district health offices successfully been able to adopt and incorporate into their routine management and implementation systems? KIIs with staff in district health offices demonstrated in-depth knowledge of CMAM’s four main objectives and nutrition challenges in their respective districts. The KIIs reported that the CMAM database is easy to manage; these key informants also stated that they include CMAM in their DIPs. CMAM guidelines and registers were found in every district health facility. NRUs are present in all district hospitals but access is reported to be limited due to limited transportation options and the cost of transportation. The latter was reported to be a primary barrier to families accessing services in rural areas, especially during the rainy season. KIIS reported that access to the SFP program services is also limited due to high cost of transportation. CTCIM Final Performance Evaluation - Final Report 19 Box 3. Summary of Enabling Factors 1. Decentralization Act 2. Department of Nutrition and HIV/AIDs 3. Health Sector Strategic Plan (2011-2016) 4. Nutrition Policy 5. Zonal Teams technical support 6. Trained District Health Management Teams (DHMTS) 7. Facilities with SOPs and guidelines 8. Trained HSAs cadre 9. Supportive Village Chiefs and Local Leaders 10. Care groups/Positive Deviant Hearths 11. Community-based child care centers b. What elements of CMAM have district health offices not successfully been able to adopt and incorporate into their routine management and implementation systems? The evaluation team noted that the standard MoH Guidelines for CMAM were not uniformly used at the district level.8 In some health care facilities, providers reported to have developed checklists and procedures but these have not been scaled-up and are not used consistently in all facilities nationwide. Other routine implementation elements such as the ubiquitous presence of standard job descriptions and operating procedures are also not firmly in place yet in every facility. The district teams do have a supervision program and 44% of healthcare providers reported having a supervision visit this year. KIIs with district health office staff reported the following inhibiting factors to sustaining their work in the CMAM program:  There are not enough NRUs, and they are not strategically located in the areas of highest acute malnutrition  The lack of opportunities and a career path for HSAs may contribute to high attrition rates  There is unequal distribution of RUTF across districts and between facilities, and there is a lack of re-distribution procedures  A lack of SFP distribution near OTP services contributes to higher default rates as children are not able to complete the SFP, which is the last step of CMAM  Transportation was reported to be a major inhibiting factor in the procurement and delivery of supplies, and many facilities reported to experience stock outs of RUTF and SFP supplies as a result  The lack of reliable transportation and the high cost deter people from taking their children to the facilities when referred  Transportation is also problematic for some HSAs who are responsible for large catchment areas; HSAs have a hard time covering and visiting households for follow-up  The lack of transport for volunteers may lead to high attrition/inactivity of volunteers and less cases being identified at the community level  Care groups and Positive Deviant Hearths are perceived as effective ways to engage and retain community volunteers but are not available in every community. Stakeholders also reported these interventions to be effective and need to be scaled-up  Women often need to seek permission from their husbands or mothers-in-laws before taking children to health care facilities for treatment. Key informants expressed the need to train all staff in every facility on CMAM. c. What have been the key factors that have enabled CMAM institutionalization in districts identified to have the capacity to independently manage and implement CMAM? Box 3 summarizes 11 enabling factors identified in key informant and health provider interviews. Below is a description of their implications as reported by key informants: 1. Decentralization was reported to empower districts to be more responsive to the health needs of the communities they serve. Having a favorable national nutrition policy, a Cabinet level office dedicated to reducing malnutrition and addressing the needs of 8 Guidelines for Community Management of Acute Malnutrition 2012. Ministry of Health, Government of Malawi, 2013. CTCIM Final Performance Evaluation - Final Report 20 vulnerable populations such as those living with HIV/AIDS, particularly orphans, was also reported to have accelerated the implementation of CMAM. 2. Management capacity at zonal and district levels has improved but it is still in need of technical assistance to integrate CMAM as part of the EHP. 3. 66% of the health care providers interviewed reported that it was helpful to have CAS advisors’ support when they needed to solve a problem. 4. Having a nutrition policy and a focused approach to identify and treat malnourished children was reported to be essential in emergency situations. This was reported to have helped CMAM become part of the daily work of district hospitals and health centers. 5. Having trained HSAs in 81% of the facilities in the country has improved access to CMAM services. Some facilities have checklists and other SOPs to help implement CMAM. 6. 80% of the interviewed health providers reported to have a working scale and a MUAC, and 56% reported to have all of the necessary forms and registers 7. 44% of the interviewed health providers reported to have received supervision in the last year, and CMAM coordinators have supervision tools and visit facilities when transportation is available 8. All HSAs reported to rely on the help of volunteers in the communities they cover. It is hard however for HSAs to engage with volunteers as transportation problems often limit the number of visits the HSAs and volunteers can make. 9. Health care providers report they are capable of implementing CMAM, but do not feel they are up￾to-date with recent changes and could benefit from refresher training and support. This is one of the reasons why the Learning Forums were so effective and valued by health care providers and district health office staff. They were a valuable source for professional improvement. d. What have been the key factors that have limited CMAM institutionalization in districts identified to have the capacity to independently manage and implement CMAM? The desktop review of project documents showed that time and scope of work limitations constrained institutionalization. Although USAID has supported CAS since 2006, the mandate for institutionalization was not provided until 2010. This evaluation specifically looked at the time period from the mandate related to the last grant to CWW in 2010 to June 2013. This grant was to implement a short project designed to institutionalize a nationwide program for about 15 million Malawians in the context of decentralization, political changes, multiple stakeholders, and health sector funding limitations, in addition to systemic weaknesses that affect CMAM delivery. The CTCIM and other donor-funded activities were not designed to address all of these obstacles. In addition, the CAS was only designed to institutionalize four objectives of CMAM, outlined earlier in this evaluation. The organizational development required for institutionalizing CMAM at the national, zonal and district levels was not part of the CTCIM design and therefore did not take place. The integration of the CMAM database, the CTCICM project’s M&E tool, into the DHIS2 is still in progress because the project ended before DHIS2 could complete the integration process and it could be tested. Taking into consideration the limitations of the CTCIM project scope, the planned project objectives were achieved, and KIIS reported that capacity was built at the district level. Incorporating CMAM into pre-service training, which would help ensure that future providers are trained in CMAM, is an initiative that is still in progress, as it turned out to be more complex to change academic programs than had been anticipated. Although the CTCIM project was not designed to help districts with the decentralization process, it did support this, and fulfilled its objectives specified in its scope of work. The handover of Learning Forums activity and in-service training was reported to be completed, but it is undecided how these two activities will continue and which entity should have responsibility for organizing and funding them. Key district and national informants reported a lack of tangible national country ownership of the program, and of the need for improved donor coordination to strengthen the institutionalization of CTCIM Final Performance Evaluation - Final Report 21 CMAM. At the evaluation findings workshop, participants representing 10 district health offices and the Ministry of Health also emphasized the importance of more visible ownership. Question 3: To what extent has CAS strengthened the monitoring of the CMAM program? CAS created the CMAM database to be used as an M&E tool to manage the CMAM program and was used by the CAS to track performance by district. The database was transferred to the MoH Nutrition Unit in September 2012. The database was reported to be the most reliable source of information on the nutritional status of children in Malawi in several instances. CMAM coordinators that were interviewed also reported that the database was to be used by every facility in all districts. The database was provided to the evaluation team by the Unit as part of the evaluation documentation. In addition to the CMAM database, Malawi has several information systems that operate in parallel. According to one district health office informant, the country needs to start moving towards one integrated Health Management Information System (HMIS) to better track and manage data. Interviews with the HSAs also aligned with this finding. KIIs with health care providers report not being able to follow-up with individual cases after discharged from a NRU and not to know if a child actually enrolls in OTP or SFP afterwards. Additionally, the Director of the MoH HMIS communicated that DHIS2 will only meet program management information needs, and that case management information is still required at the facility level9. While the DHIS2 is being implemented and includes most of the indicators that CMAM collects, there is a need for an electronic database that allows for case management information at the facility level. This could help healthcare providers track children’s growth and monitoring, and will allow them to follow up with CMAM program defaulters and to continually track every child’s outcome. In this way, a child admitted in the NRU would be followed up after discharge and visited at home if the OTP/SFP appointment is missed. a. To what extent has the CAS built capacity in ensuring the development, training, dissemination, and use of standardized data collection, compilation, and reporting of CMAM monitoring data? KIIs reported that CAS has ensured the timely collection, compilation and reporting of CMAM monitoring data by effectively training and supervising district staff. CAS has built capacity of CMAM coordinators in every district and CMAM data are usually reported to the CMAM coordinator by the 5th of every month. Most of the reports were reported to be compiled and submitted on time. b. To what extent has CAS ensured systems to ensure data quality, timeliness, and completeness of CMAM monitoring data? CAS trained health providers and district staff on how to ensure that the data in the CMAM database is complete, accurate and submitted in a timely fashion. This statement is based on KIIs with district health providers and MoH staff. The evaluation team was unable to locate an operations manual or SOPs however that provided guidelines on how to use the database, or which provide insight on data quality standards and provide data quality checks. c. To what extent has CAS built capacity of facility, district, and national stakeholders to synthesize and use data for decision making? CAS was reported to have built satisfactory capacity to synthesize and use data to summarize and report activities. This was confirmed by KIIs, (please reference Annex VI for a full list of KIIs), and the facility and health provider questionnaire data that show that there were trained HSAs in most of the facilities surveyed, and that there were up-to-date paper OTP, RUTF and SPF registers found in every 9 See KII interview with Mr. Chris Moyo and Patrick Naphini, June 28, 2013, Ministry of Health office, Lilongwe. CTCIM Final Performance Evaluation - Final Report 22 healthcare facility. However, the zonal teams were reported to not have been made part of the system yet. The capacity of these zonal teams to synthesize data and use it to strengthen and support districts to make informed decisions was not evident. Question 4: To what extent did stakeholders of the CMAM National Nutrition Operational Plan fulfill their roles and commitments? Malnutrition has been adequately addressed and is firmly positioned at the center of the development agenda as a multi-sectoral issue in Malawi and is supported by numerous stakeholders, which include donors, project staff, MoH staff, zonal and district staff, health providers, etc. The MoH developed an operational plan for the integration and scale up of CMAM by partners and stakeholders from 2009 to 2012. The plan ensured the integration of CMAM activities into DIP and currently, all 29 health districts have included CMAM services in their 2013 DIPs and have budgeted for at least 25% of CMAM costs (indicator table through 2013). The allocation of funds was reported as a major challenge for the districts, but the inclusion of CMAM into DIPs and the annual budget was achieved. However, the operational plan was not updated during the life of the CTCIM project and the evaluation team was not able to locate quarterly or annual progress and financial reports. The evaluation team was not able to do an analysis of funding for CMAM at the district level; this is in part because funding is provided through the MoH and several donor funding streams. Additionally, it was hard to discern whether the MoH has a clear commitment from all stakeholders regarding their level of funding they predict they will contribute. However, an upcoming World Bank-funded project will invest about $30 million to improve the nutrition program and is an important opportunity to create and/or strengthen systemic structures and operating procedures. Table 7: Summary of Operational Plan Commitments Commitment Status CMAM operational in at least 80% of the facilities Achieved CMAM fully funded and costs incorporated in the MoH SWAp and DIPs Partially included in the Sector Wide Approach (SWAp) and 25% funded in the DIPs MoH Nutrition Unit and district health offices have adequate capacity to manage all aspects of CMAM Achieved at national level, but not complete at district level Management of acute malnutrition into pre-service training In progress Supplies for CMAM effectively managed in existing services Supplies are available but RUTF were reported as out of stock in some districts CMAM programs implemented effectively and efficiently Supervision is not regular and there is limited capacity to assess management effectiveness and efficiency by Zonal and District teams CMAM is linked to other health and safety net Not achieved yet (this objective was not part of the CTCIM mandate) KIIs also reported that stakeholders fulfilled a majority of their commitments in the operational plan (see Table 7), a finding that is in accordance with the views of CMAM stakeholders that were surveyed by email. However, there was not a consensus among stakeholders about the degree of institutionalization in effect, and some stakeholders reported a need to continue the institutionalization. CTCIM Final Performance Evaluation - Final Report 23 Box 4. Stakeholder Enabling Factors 1. The elevation of the issue of nutrition to the Presidential Cabinet’s agenda 2. Nutrition Policies that were in place to support CMAM and the operational plan allowed for the setting of common objectives for all stakeholders 3. CAS provided increased coordination among donors 4. The creation of the DONUT, the donor coordination group 5. Working groups and advisory boards brought more attention to health management issues and gaps and the need to increase transparency 6. A standardized M&E system through the use of the CMAM database 7. The local production of RUTF Table 8. CAS project perceived institutionalization effectiveness (Source: Email Survey) Which of these CAS functions are institutionalized and sustainable after the end of the project? Choose all that apply Options Responses N % Ensured that health providers have capacity to implement CMAM 11 11 100.0% Ensured that CMAM is implemented in most facilities in every district 8 11 72.7% Trained National Trainers 8 11 72.7% Development CMAM training materials 7 11 63.6% Developed a sustainable and standardized M&E system 6 11 54.5% Ensured that health providers have the capacity to manage CMAM 5 11 45.5% Conducted stakeholders meetings 5 11 45.5% Conducted supervision 5 11 45.5% Documented best practices 4 11 36.4% Mentored staff 4 11 36.4% Conducted Learning Forums 4 11 36.4% Provided Stationery 0 11 0.0% Created the CMAM Technical Working Group 2 11 18.2% a. What are the key factors that have enabled stakeholder commitment to the CMAM Operational Plan? A number of enabling factors were identified by key informants (see Box 4). Written policies and program guidelines as well as the meetings to coordinate work have helped stakeholders to follow the operational plan. KIIs report that the CAS helped all CMAM stakeholders with technical knowledge, information support, coordination and mentorship to scale up the use of CMAM and supplementary services to all 28 districts nationwide. The donors supporting the development of the nutrition field in Malawi created the Donors in Nutrition (DONUT), as a way to align their work, chaired on a rotating basis by one of the donor members. The DONUT participants share information about their activities and coordinate their project activities. Because this is a donor only group, the MoH is not a member and therefore does not participate in these meetings. Outside participants are able to participate on an invitation only basis. This donor coordinated mechanism is just one of several that exist for nutrition; the National Nutrition Committee and the Nutrition Development Partners meeting work in tandem with the DONUT to streamline donor activities. b. What are the key factors that have limited stakeholder commitment to the CMAM Operational Plan? Government key informants indicated three main areas where the project handover to the MoH had not been effective. First, there was a handover schedule which seemed to lack a clear plan of activities to ensure the right staff, SOPs, and processes were in place. The organizational development in terms of training and coaching required for MoH structures to assume CAS functions did not take place. It was reported that the word ―handover‖ may have been interpreted as meaning a ―transfer‖ of project activities as opposed to the full internal adoption of CAS roles, functions and responsibilities by the right staff at the appropriate level in the Malawian Health System. CTCIM Final Performance Evaluation - Final Report 24 Also, the transfer of the CMAM monitoring database took place and was given to the MoH Nutrition Unit, which is a policy and monitoring unit and is not designed to be involved with the actual maintenance and management of the database. This would include data accuracy checking, providing refresher training, and troubleshooting at the health care facility and district health office level. The CMAM operational plan is not a binding document and routine government coordination and accountability structures were not created in Malawi. As reported by key informants and confirmed by the workshop participants, stakeholders are committed to CMAM but the lack of a high-level coordinating mechanism still hinders the MoH’s ownership and ability to lead activities and hold stakeholders accountable for their commitments. There were also several areas of overlap regarding stakeholder responsibilities including the RUTF distribution, whose effectiveness criteria and processes were not clearly defined, written down or understood by all stakeholders. This was reported to have contributed to confusion on how best to distribute and re-distribute RUTF supplies. In addition to a lack of ministry-level coordination and unclear stakeholder responsibilities, the primary factors limiting stakeholder commitment are a lack of knowledge management. The MoH Nutrition Unit reported to not have an effective knowledge management strategy related to nutrition that would allow it to absorb all information and share it with all stakeholders. In addition, it was reported that the MoH Nutrition Unit did not have a mechanism to coordinate or monitor the national nutrition policy and the implementation of CMAM guidelines as implemented by various stakeholders in each district. The Unit also reported to have limited capacity to maintain the CMAM database. Malawi developed the NECS, which is a multi-stakeholder mobilization campaign. It is not clear how CMAM contributes to this campaign and vice versa however. Question 5: What role has gender analysis played into CMAM? The evaluation scope of work required that an analysis be conducted to ensure that gender issues of males and females had been identified and addressed by the CTCIM project. No documented gender analysis was available from the CTCIM project or for the predecessor grants to CWW. Thus, the dimensions of gender and malnutrition specifically in Malawi needed to be mapped out using on prior research and available data. The evaluation team used a framework for gender analysis guided by resources created through USAID specifically for the health sector, and the CIDA Framework for Assessing Gender Equality Results. 10 The 2012 and 2008 USAID gender assessments were consulted. The primary gender-related laws and policies of the government of Malawi were also reviewed.11 Secondary research and gender assessments done by other development agencies were read for insights into the factors that might enable or inhibit men and women from seeking help in understanding why a child is ill and may need medical care, as malnutrition may have many causes. These factors help identify the dimensions of how male and female caregivers view child malnutrition and why they may or may not 10 These include Tips for Conducting a Gender Analysis at the Activity or Project Level: Additional help for ADS Chapter 201, USAID 2011; A Practical Guide for Managing and Conducting Gender Assessment in the Health Sector, Margaret E. Greene. Interagency Gender Working Group and USAID June 2013. http://www.igwg.org/Articles/genderassessmentsguide2013.aspx; and the Gender Assessment Guide: A Manual for Integrating Gender into Reproductive Health and HIV programs: from Commitment to Action. Second Edition, Deborah Caro. Interagency Gender Working Group and USAID. For CIDA, the Framework for Assessing Gender Equality Results (revised 2010) and the Policy on Gender Equality (revised 2010) were consulted (http://www.acdi-cida.gc.ca/INET/IMAGES.NSF/vLUImages/Policy/$file/GE-Q&A-Culture-EN.pdf and http://www.acdi￾cida.gc.ca/INET/IMAGES.NSF/vLUImages/GenderEquality3/$file/GE-Framework-EN.pdf. 11 These include the Gender Equality Law 2013; the Malawian Constitution; and the National Gender Policy 2000,the National Gender Program . Key Informant Interview with Dr. Mary Shawa, Principal Secretary, Ministry of Gender, Children, and Social Welfare, June 26 and July 16, 2013, Lilongwe. CTCIM Final Performance Evaluation - Final Report 25 Box 5 Gender Sub-questions 1. Does the CMAM Initiative adequately respond to the specific needs of male and female participants and clients? 2. Are there any social obstacles for males and for females to achieving the objectives of CMAM that were not adequately addressed by the original project design? 3. Have the community mobilization activities hindered or promoted uptake of CMAM services by males and females? 4. What other factors have hindered or promoted male involvement in CMAM? 5. Do health facility staff members (zonal, district, community volunteers) have the appropriate set of skills to address the different needs of male and female clients? 6. Are the indicators used for monitoring and evaluation adequately capturing how the project affects males and females? 7. How have the CMAM data disaggregated by age and sex used to refer clients to other programs? 8. Have CMAM data been used to inform changes to other programs offered at the health facility? Box 6 Identification of boys and girls Do you have a routine way to identify every malnourished child? Yes 13 respondents No 0 respondents Does this routine differ for male and female children? Yes 1 respondent No 12 respondents seek help through services such as CMAM. In addition, this contextual analysis helped clarify how the CTCIM project contributed to greater focus on outreach to men at the community level. Using the analysis of secondary research and KIIs conducted in Lilongwe, gender-related questions were embedded in each of the survey tools used by the evaluation team. Table 14 in Annex V lists these specific question numbers and each question links to one of the domains explored in Annex E Table 16. The gender analysis findings are detailed in Annex V, which presents the sub gender- analysis questions developed after the field work qualitative and quantitative data had been collected. Table 14 includes the response found in documents created through the CTCIM project to these questions, and the impact of the related CTCIM response or activity. For the latter, detailed evidence and a citation is provided from the KII’s, healthcare providers’ survey, FGD synopses, facilities survey responses, and email survey responses. The specific sub-evaluation questions are answered below, highlighting this evidence. a. What conclusions can be drawn from monitoring data regarding the uptake of CMAM services by gender? The evaluation team analyzed the CMAM database on the uptake of boys and girls to see what conclusions could be drawn regarding gender and the use of CMAM services. The team then compared these data with other information sources, such as the 2010 DHS for Malawi. Since the CTCIM project CMAM database was designed to monitor an emergency response and not to follow individual cases, the only apparent indicator that is sex disaggregated is the enrollment data on boys and girls. Along with the evaluation team examination of the database, we confirmed the number of sex disaggregated indicators several times with two former CAS members who were very familiar with the construction of and use of the database and with CWW staff in Lilongwe at the time of the evaluation. Given this limitation, the evaluation team concluded that further analysis of sex disaggregation apart from enrollment in each case in this particular database is not feasible.12 The evaluation team conducted a KII with the leadership of the Ministry of Health Management Information System (HMIS) unit who indicated that the challenges with the current HMIS include: multiple reporting systems by programs; the HMIS is not meeting needs of individual programs (programs want to track more information; the HMIS does not disaggregate by sex and age (which is want districts want to track)) and the HMIS only uses totals.13 12 Review of the sex-disaggregated fields of the MOH MIS database was beyond the scope of work for this evaluation. 13 KII interview with Mr. Chris Moyo and Patrick Naphini, June 28, 2013, Ministry of Health office, Lilongwe. CTCIM Final Performance Evaluation - Final Report 26 Box 7. KII Views of Male Roles "Men should see child care or CMAM as an opportunity for better health" "Main focus should go to households because CMAM is focused at the facility level but we want to see more action at the community by involving local leaders. Has to be supported by the local leaders to be successful or CMAM will struggle to find its feet." - Kasungu District, During the visits to the 10 districts, the evaluation team asked district health office CMAM managers if they used the sex-disaggregated enrollment data in the CMAM database. At the district level, CMAM data are hand recorded in hardcopy registers for OTP, RUTF and SFP. The evaluation team was told that the information in these registers are routinely used by HSAs and their supervisors to make decisions about what they need to best meet the needs of their clients. KIIs with these individuals in the 10 districts indicated that there is no difference in the way girl and boy children are identified as malnourished or in treatment at health facilities and in hospitals (see Box 6). At the health facility level, the recording of data in the registers is on a sex-disaggregated basis and is used for planning purposes to make sure there are enough supplies and to watch trends. The health care provider survey responses confirmed that the individuals taking intake data about male and female children are not always the same people who record the information in the registers. KIIs with district Health Environmental Health Officers (EPI)14 indicated that they found the larger uptake in girls is correlated to the current population profile in Malawi15 and said the CMAM database was easier to manipulate than the HMIS, although they had heard the HMIS was being improved upon. As reported by KIIs with HSAs, use of the CMAM protocol has succeeded in providing gender-neutral services to boys and girls. There is no difference in the identification and treatment of boys and girls, nor, reportedly, of boys and girls who may be HIV+ or those who are orphans once they are taken to a health facility. When probed further as to why more girls are enrolled in CMAM than boys, particularly given the last DHS survey which found higher rates of stunting among boy than girls, typical KII answers speculated that boys grow faster than girls. The team triangulated these responses with healthcare providers to see if there were self-reported differences in the identification of malnourished boys and girls, and in the treatment of malnourished boys and girls. Please see Annex V for gender results from the facility survey and healthcare provider survey. b. What are the factors that hindered or promoted male involvement into CMAM, and what additional inputs are needed to enhance male involvement? This question was explored in the KIIs with CMAM managers and HSAs and with community members in the FGDs. The gender analysis research findings matrix depicted in Table 16 Annex V also indicates the need to actively engage men in household nutrition as a key solution in preventing child malnutrition. Secondary research pointed to intra-household food distribution and the knowledge of food nutrition, attitudes of men toward young child care; and perceptions of male and female roles toward male participation in nutrition as important to understand in finding ways to increase men’s participation in preventing malnutrition. It reported that notions of gender equality are changing within Malawi and thus there are practical opportunities for further engagement of men (see Box 7). Key informants were asked several questions concerning the engagement of men in using CMAM services. A common observation was that it is harder to get men involved with any kind of health service or activity. When asked why some men do not use CMAM services for their children, responses included that the waiting areas include both sexes, and men do not want to wait with mothers and their children. Men also communicated that the wait times were too long. Fears about speculation that there is something wrong with the family if a man brings a child to a health facility 14 See, for instance, the KIIs with EPIs in Dedza on July 2, 2013 Blantyre on July 3, 2013, and District Health Office Nutritionist in Zomba on July 5. 15 Malawi DHS 2010 p. 2. CTCIM Final Performance Evaluation - Final Report 27 rather than a woman were also mentioned. KII respondents reported that they did not see a real difference whether a CMAM volunteer was a male or female since both are respected and are part of the community. It was reported that women are preferred for this role however because they are in the community more often and seemed to care more about the work. The responses to two KII questions regarding the direct engagement of men in CMAM are quoted below: What do you think would be the best way to encourage men to use CMAM services or to work as volunteers?  ―Education and encourage male involvement. Have to change culture.‖  ―Men go for piecework employment and working in fields far away, are also far from the family homes.‖  ―Urban and rural settings are different (for engaging and retaining male volunteers).‖  ―Men are the decision makers of the household and influence what woman do in caring for family.‖  ―What would help? Give men first priority to be seen over women.‖ What parts of connecting with communities could be improved to encourage men to use CMAM when their children are sick?  ―Educating communities.‖  ―If men do take children to use CMAM services they (should) get treated first to encourage more male involvement.‖  ―Men need adequate awareness about nutrition, which they do not get in any organized way.‖  ―Empowering local leaders they can put measures in place to force men or NGOs to help with awareness/volunteer and sense of ownership would help.‖  ―At the community level the responsibility should not just be focused on women.‖  ―Don’t have Information, Education and Communication messages but we tell them to get involved.‖ From the FGDs, adult males and females did not report bias in intra-household food distribution (particularly) protein that favor boys and girls or vice versa. Both males and females do not have high levels of nutrition knowledge and there are misconceptions about breastfeeding of HIV+ mothers. Both of these factors limit the use of CMAM services by men and women. However, all men in the FGDs expressed a real interest in learning more about nutrition. CMAM service providers noted that community mobilization efforts needed to engage men to increase understanding of malnutrition prevention. This issue was discussed at length in several CAS supported Learning Forums and resulted in actionable recommendations assigning roles and responsibilities.16 The 2012 CMAM guidelines, for instance, describe loving care, play and stimulation for malnourished children to be undertaken by the mother; however the father is not mentioned.17 Ramping up CAS activities to fully take on this challenge at the level needed to show an increase in male involvement during the CTCIM project may have required additional resources and may not have been possible during the timeframe. The KII responses and the survey research supports the view that males tend not to be involved in CMAM services because they are seeking paid labor or are working and do not have the time to wait long periods at health facilities. As well, men are not usually targeted with nutrition messages and 16 See Strengthening Community Outreach CTC Learning Forum report #16, Volume 3, Issue 3. 17 See Guidelines for Community Management of Acute Malnutrition 2012. Ministry of Health, Government of Malawi, 2013. Pages 107-108. CTCIM Final Performance Evaluation - Final Report 28 information. This excludes them from having the information needed to promote healthy food consumption within the household. CONCLUSIONS The findings obtained during the evaluation illustrate that the CTCIM project achieved its four objectives. Additionally, analysis of the team’s Institutionalization Metric of the CTCIM project showed that the institutionalization process was partially completed, and that CMAM is an effective strategy that is being implemented nationwide. Below we present conclusions regarding each evaluation question: 1. To what extent has CMAM built the capacity at various levels? The primary enabling factor of the CTCIM project to effectively build capacity was that it had a clear and simple strategy and focused on four effective interventions: NRU, OTP, SFP and Community Outreach. The email partner survey helped triangulate some of these findings; although the survey only covered a small sample of respondents, it revealed that CAS created capacity and the institutionalization process was not yet complete. Management and implementation capacity of health service providers and district and national level managers Since the health system in Malawi is being decentralized, much of the long-term success of the CMAM program depends on the ownership and management capacity of the program at the district level. The capacity to lead and manage the program without assistance has not yet been achieved, but the support of CAS advisors was responsive to district needs and was appreciated by district and facility staff. It was reported that the CTCIM project did focus on building the capacity of zonal teams to take over and serve as CAS advisors, and that it did not directly assist the district level health care facilities with CMAM implementation. Effective knowledge management and technical assistance should make the work of the MoH Nutrition Unit team easier and more efficient, and ensure that the unit reaches its potential and can fulfill its role in a decentralized setting. The KIIs held in Lilongwe and at the district level indicated that while the planning capacity was effective at the national level, it was weak at district levels, and limited at zonal levels. Moreover these interviews pointed to limited CMAM management capacity due to the partial oversight and coordination of activities and resources given the different CMAM donors who do not have formal reporting and coordination structures at district levels where implementation takes place. Analytic capacity was also bolstered through the use of the CMAM database, which made it possible to keep track of progress, and support districts and facilities as needed. CMAM and the capacity of pre-service training institutions The project implemented an effective training program, however this training was not institutionalized or part of the country’s pre and in-service training systems at the time of this evaluation. A more flexible approach to creating curriculum would have been necessary but that was out of the scope of the project. A description of the incorporation of CMAM into pre-service training curricula showed that CMAM is not fully included in the curricula. The handover of the capacity building functions of the CTCIM project had a number of delays and suffered from a number of inhibiting factors that prevented a more effective integration of the CMAM database with DHIS2, in addition to the full adoption of the in-service trainings, the Learning Forums by the DoH, and the integration of CMAM into pre-service training at academic institutions, which is still in progress. The evaluation team did not find any evidence of a mandate to institutionalize the Learning Forums and consequently, these forums and in-service CMAM training seemed to have ceased when the CTCIM project ended. The new CMAM guidelines were not disseminated by the CTCIM because of the project close date, but it is expected that UNICEF and other stakeholders will support this activity. CTCIM Final Performance Evaluation - Final Report 29 While NECS campaign activities seem to be in parallel with the objectives of CMAM, they may be a drain on the already limited local resources and duplicate efforts. 2. To what extent has CMAM been incorporated in the EHP and is delivered in district hospitals and primary health care service delivery points? Although CMAM is managed and delivered in parallel to the EHP, there is a need to bolster better integration with delivery of other services. Moreover, zonal and district management capacity needs further technical assistance and training to better integrate CMAM into the EHP. There is also a need to fully incorporate CMAM into the EHP and other nutrition-related activities. Some of the district health care facilities visited have started to adapt their service delivery processes to integrate CMAM, but this is not yet a systemic process. Several important organizational practices, such as keeping checklists and SOPS, were observed, however CMAM would be more effectively incorporated into the EHP if these practices become more common practice. Transportation challenges and cost was repeatedly referenced as a hurdle to caregivers seeking help at hospitals and primary health care service delivery points, and is a fundamental challenge in incorporating CMAM’s objectives. Annual standard CMAM planning and preparation procedures for this period were not observed however. 3. To what extent has CAS strengthened the monitoring of the CMAM program? CTCIM developed the CMAM database as an effective project M&E tool; however, the tool operates in parallel with existing efforts to improve the country’s HMIS. An interview with the HMIS unit reported that the MoH has several systems in parallel that consume a lot of providers’ time, but is looking for ways to integrate them into one system. In addition, the HMIS system is also not nominal and does not allow health care providers to follow-up on each at-risk child, or to monitor children after discharge from an NRU. Effective case management requires for health care providers to screen and monitor growth, refer cases, follow up on referred cases, and document outcomes. The CMAM database system does not allow providers to identify each child in the facility catchment area by village and follow their growth until they are at least five years of age, which makes it difficult to develop effective case management processes. Analysis of the CMAM database demonstrated that it is used by CMAM coordinators and health care providers effectively; however the database included incomplete data from 2008 to 2012 regarding the performance of each district. Based on these data, districts were classified as low, mid or high performers according to Sphere standards. Our field data collection was not able to find differences in the performance of districts based on the CMAM database. The causes of varying performance may be due to the number of women and children affected by malnutrition in each district and not a reflection of the facility performance but this evaluation was not able to confirm this assumption. 4. To what extent did stakeholders of the CMAM National Nutrition Operational plan fulfill their roles and commitments? The operational plan was not updated during the two-year life of the CTCIM project and the evaluation team was unable to locate quarterly or annual progress and financial reports. As a result, this conclusion is based on the KII reports. Because a systematic approach was not used to implement the operational plan and institutionalize CMAM, and because the processes were not made obligatory, the MoH was unable to retain control of both the processes and the commitments made by various stakeholders. A lack of government-level coordination of stakeholders from the beginning has contributed to the inability of Malawi’s MoH to lead activities and hold stakeholders accountable to their roles and commitments. The DONUT was created in response to this absence of coordination and serves to ensure that donors do not replicate their efforts. As mentioned previously, the DONUT does not work CTCIM Final Performance Evaluation - Final Report 30 in isolation. Because of this, it is important to manage the overall coordination of the DONUT with the National Nutrition Committee, and the Nutrition Development partners meeting and their corresponding committees and task force, to help to better facilitate country ownership and leadership. There are ample opportunities to build on CAS best practices and lessons learned and gain momentum in implementing prevention and treatment interventions however. The advent of a new WB project with over U.S.$30M investment in nutrition, USAID presence in 15 focus districts, and the presence of Centers for Disease Control (CDC) in another 15 districts, can be leveraged to address these shortcomings. 5. What role has gender analysis played into the CMAM? There are several stakeholders and donors working with the MoH to improve health records and the access that zonal and district health offices, as well as health facilities have to utilize data for management and planning purposes, including the ability to disaggregate data by sex on several indicators. The CTCIM CMAM database was reported as a useful tool because it was easier to see trends at the district level than the current HMIS. These trends include the difference between male and female child enrollment in CMAM services. It can be concluded that this CMAM excel-database maintained by the CAS provided users at the district level with information that they could not get from the HMIS, including enrollment data of male and female children. Many commented on how this database was easier to use than the HMIS and this made it useful for planning use of human resources and in ordering supplies. At the facility level the paper OTP, RUTF, and SFP registers are used for planning by healthcare providers as well as any information from the CMAM database shared by the district health office. Further attention to involve men can be achieved by utilizing the practical recommendations made during the CTCIM project. Health care providers noted on several occasions that community mobilization efforts need to engage men to increase their understanding of nutrition and ways how to prevent malnutrition. This issue was discussed at length in several CAS supported Learning Forums and resulted in actionable recommendations assigning roles and responsibilities to different actors at the district and national level.18 In one Learning Forum dedicated to community engagement, CMAM service providers suggested low￾cost interventions such as: including men in child care and nutrition interventions; developing guidelines for male involvement; use already existing systems in the MoH to mobilize men in child care, and drilling services providers in gender mainstreaming in all services19 and implementing them is well within the current capacity of the CMAM structure. Since the close of CTCIM project, the evaluation team was unable to find which entity was directly assigned the responsibility for activities to expand male involvement at the community level. The evaluation team observed powerful, positive messages painted or posted on walls about breastfeeding, depicting a father, mother and child together in hospitals and health centers. Having more posters of the six food groups and strategies for preventing malnutrition posted in communal areas in communities is important as some influential groups (religious) and traditional beliefs still prevail, discouraging people from utilizing CMAM and other health services. 18 See Strengthening Community Outreach CTC Learning Forum report #16, Volume 3, Issue 3. 19 See Strengthening Community Outreach CTC Learning Forum report #16, Volume 3, Issue 3. Page 8. CTCIM Final Performance Evaluation - Final Report 31 RECOMMENDATIONS The four general recommendations below are offered as a way to help spur immediate action and gain rapid momentum to sustain the achievements of the CTCIM project, help the MoH Nutrition Unit to lead and coordinate the institutionalization and implementation of the CMAM program, and contribute to strengthening Malawi’s health system: 1. Build on the strengths of Malawi’s CMAM program by assisting zonal and district teams via training and technical assistance to sustain the enabling factors identified. 2. Improve the organizational development of new CMAM program management and service delivery processes in order to ensure compliance of all stakeholders. 3. Monitor staff and recognize them for their achievements to help sustain the observed high levels of service delivery; institutionalizing and increasing the frequency of supportive supervision of CMAM staff will help achieve this and will create incentives for continued success. 4. Build the health system to effectively deliver nutrition services and treat malnutrition for the long run. Envisioning the type and scale of nutrition services that will be required in 10 years will help strengthen and sustain the health system for future populations; this strategic process should commence immediately. Stakeholders and partners might help achieve a common long-term vision for CMAM and help mitigate and prevent issues in the near future. Future investments to strengthen CMAM will need to focus on addressing the following remaining challenges related to each of the evaluation questions: I: Capacity at National, District and Local Levels 1. The MoH Nutrition Unit should continue the institutionalization of the CMAM program and selected donor-funded activities that strengthen the delivery of CMAM services. A new operational plan should be developed with explicit roles and responsibilities for each stakeholder and include formal coordination and accountability structures. Technical assistance should facilitate the organizational development of the Unit and the creation of an ―ownership and coordination‖ plan for the Unit to include CMAM Advisory Services to Zonal and District Teams. 2. The development of the capacity of the zonal teams to deliver the technical advisory services that CAS delivered to districts should be completed for the institutionalization of CMAM given the context of the decentralization process in Malawi. The job descriptions and work routines of zonal teams should be revised to include CMAM so they can assist districts and facilities, expand CMAM coverage and continue to improve the quality of CMAM services. Conducting weekly transportation coordination meetings at district offices and keeping vehicle logs could help alleviate some of the transportation issues as well. 3. The leadership for organizing and conducting Learning Forums should also be taken over by zonal teams so they can support districts to sustain the achievements of the CTCIM project. Technical assistance should be provided to develop an operations manual to conduct ―Learning Forums‖ quarterly in each district and annually at national level. 4. The MoH Nutrition Unit should have a knowledge management strategy and activity plan to ensure it has access to all relevant knowledge and keeps its program’s institutional memory which is now mostly with project staff from many stakeholders and partners. Technical assistance should facilitate the development and implementation of a knowledge management strategy. 5. CMAM pre- and in-service training should be institutionalized and CMAM should be fully incorporated into pre-service training curricula. Preparation for anticipated seasonal increases in the incidence of acute malnutrition is a priority that needs to be addressed and included in the in-service training of all health care providers. Technical assistance to zonal teams should be provided so they can help their districts plan and implement CMAM in-service training and on-the-job problem solving support. In service training should include all health care facility technical staff. 2: Incorporation of CMAM into the EHP and routine healthcare delivery 1. Technical assistance to the zonal teams should be provided to develop SOPs for every CMAM activity as part of an integrated EHP delivery process. Technical assistance should also be provided CTCIM Final Performance Evaluation - Final Report 32 for districts to develop and implement plans that address systemic weaknesses and that impact supply distribution and delivery of health services. 2. Zonal and district teams should develop and implement a simple integrated supervision plan that allows them to oversee CMAM program performance at community and facility level. The supervision plan should include useful feedback to health care providers so that reinforcement of ―right‖ behaviors such as applying new CMAM guidelines, following up defaulters and vulnerable children, integrated EHP delivery, Care Group support, community mobilization and outreach to men, and proactive problem solving occurs regularly. Additionally, these teams should undergo more training to strengthen their technical skills if they are to effectively integrate CMAM into the EHP. 3. Technical assistance should continue to ensure the training, recruitment, deployment, supervision and promotion of HSAs to sustain CTCIM achievements. The Human Resources for Health work plan should include CMAM as part of the every provider’s job description and performance evaluation. 3: Monitoring of CMAM 1. Institutionalization of the CMAM information system. The CMAM Database served as an effective tool to manage the CTCIM program. In the context of DHIS2, its objectives and use will need to be reviewed and the database revised to meet case management needs as well as program management needs. Institutionalization of CMAM program management into the HMIS is a priority. SMS reporting and even online reporting will be necessary as the country grows in population and complexity. 2. The CMAM database should be revised so it is nominal so each child is individualized and followed after discharge from the NRU and their outcome noted. The case management information system would facilitate effective referral and counter-referral and avoid losing children to follow up. 4: Role and Commitments of Stakeholders in CMAM 1. All donor-funded CMAM project activities should be included in the respective district implementation plans. This will ensure synergy between districts and ensure that all activities support and contribute to the coverage of CMAM targets and delivery of CMAM services. 2. At the district level, joint ―quarterly accountability and planning‖ meetings of all implementing partners should be conducted and led by the DHO. These meetings should include reporting on project progress and joint action-oriented planning activities with all stakeholders to ensure district priority objectives are met. 3. The MoH must coordinate donor assistance in relation to CMAM. To accomplish this coordination the MoH should urge zonal teams to assist districts in reviewing project implementation plans and ensuring that stakeholder project activities align with MoH’s goals of improving CMAM service delivery and achieving CMAM performance objectives. 5: Gender20 1. If possible, the current CMAM excel database housed in the MoH Nutrition Unit should be revised so there is more sex disaggregation by indicator, at least to include cure rates, death rates and default rates so these data can be used by CMAM service providers and DHOs for resource management and community outreach purposes and by the Unit to better understand how differences in stunting rates of boys and enrollment of girls can be addressed at the policy level. 2. The new Training Manual for CMAM21 module 1, Introduction to Basic Food and Nutrition could be slightly adapted and delivered to village leaders. Session 1 covers nutrition and the six food groups, 20 The recommendations made by the evaluation team on male involvement reflect the ideas of healthcare providers, stakeholders, key informants, and FGD participants. CTCIM Final Performance Evaluation - Final Report 33 Session 2 covers malnutrition. Adding an additional module covering strategies for household food security could give leaders more tools to use with when working with their communities. This training should be made available to men, training them where men congregate, in male-only groups22 to increase knowledge and elevate the topic to part of a man’s responsibility as an individual and as a family member.23  At the community level, husbands and wives should also be trained together in nutrition and food security strategies to break the hold of lingering norms that hinder the growth and development of a married couple as a unit working together as a team for a healthy family.24  Because they directly examine children HSAs could be trained in detection of gender-based violence and refer people to the new centers, NGOs, and law enforcement. 3. Protocols used by healthcare workers should be reviewed for male inclusion. For instance, the 2012 CMAM guidelines, describe loving care, play and stimulation for malnourished children to be undertaken by the mother; however the role of the father is not mentioned.25 As mentioned earlier, Expanding CAS activities to fully take on this challenge at the level needed during the CTCIM project may have required additional resources. 21 Training Manual for Community Management of Acute Malnutrition. Revised May 2012. Ministry of Health, Government of Malawi. 22 This recommendation stems from the June 26, 2013 Key Informant Interview with Mr. Marcel Chisi, National Coordinator, Men for Gender Equality Now (MEGEN) Malawi. Mr. Chisi described the best practices used by MEGEN in motivating and working with men on changing harmful social norms. MEGEN works in 18 districts and coordinates activities of the Men Engage Network (boys and men for gender equality) which operates in over 13 African countries. MEGEN reports they are engaged with 50,000 men in Malawi. See the MEGEN website for more information on their community-driven leadership development model http://www.menengage.org/index.php?option=com_content&view=article&id=194&Itemid=97 23 The work on positive masculinities promoted by the Ministry of Gender, Children, and Social Welfare and Malawian NGOs like the MEGEN Network and the NGO Gender Network could be linked into CMAM community mobilization efforts. 24 This recommendation was suggested by the Dr. Mary Shawa, Principal Secretary, Ministry of Gender, Children, and Social Welfare, Key Informant Interview June 26 and July 16, 2013, Lilongwe. 25 See Guidelines for Community Management of Acute Malnutrition 2012. Ministry of Health, Government of Malawi, 2013. Pages 107-108. CTCIM Final Performance Evaluation - Final Report 34 ANNEXES CTCIM Final Performance Evaluation - Final Report 35 ANNEX I: EVALUATION STATEMENT OF WORK USAID/MALAWI Statement of Work: “Evaluation of Community-based Management of Acute Malnutrition (CMAM)” A. PURPOSE The purpose of the evaluation is for the Contractor to identify the extent to which the various elements of Community Therapeutic Care (CTC), now called Community-based Management of Acute Malnutrition (CMAM), the Government of Malawi (GoM) Ministry of Health (MoH) have assumed and incorporated into their routine nutrition program management. The findings, conclusions and recommendations of the evaluation will establish the effectiveness of the CMAM implementation model in Malawi, through CMAM Advisory Services (CAS), and make a determination on key elements for sustainable management, implementation and scale up of CMAM in Malawi. USAID is keenly interested to better understand more effective modalities of future support to CMAM institutionalization in Malawi. B. OBJECTIVES 1. To learn to what extent the model of providing technical assistance through CAS has been effective in institutionalizing CMAM within the MOH, specifically at district hospitals. 2. To understand the key factors that enabled or inhibited effective institutionalization of CMAM. 3. To understand the extent to which all stakeholders to the CMAM operational plan fulfilled their responsibilities in support of the process of institutionalization. C. AUDIENCE AND INTENDED USE Findings of this evaluation will provide evidence to all key CMAM stakeholders in Malawi, including USAID, MOH, Canadian International Development Agency, United Nations Children’s Fund, World Food Program and Clinton Health Access Initiative and Concern Worldwide. The evaluation findings will inform future program design and areas of investment for sustainable scale up and institutionalization of CMAM in Malawi. Select information emanating from the evaluation may also be disseminated to the communities. D. TASKS The Contractor shall conduct an evaluation and analysis of the CTCIM activity to document actual/cumulative results by performing the following tasks: 1. Evaluation Questions The Contractor must, at a minimum, address the following questions in the final evaluation report: To what extent has CMAM built the capacity at various levels? a. To what extent has CAS built the capacity of health service providers to implement CMAM? b. To what extent has CAS built the capacity of district and national level managers to manage the CMAM program? c. To what extent has CAS built the capacity of pre-service training institutions to integrate the CMAM approach into the curriculum? To what extent has CMAM been incorporated into the Essential Health Package and is delivered routinely at district hospitals and primary health care service delivery points? a. What elements of CMAM have district health offices successfully been able to adopt and incorporate into their routine management and implementation systems? CTCIM Final Performance Evaluation - Final Report 36 b. What elements of CMAM have district health offices not successfully been able to adopt and incorporate into their routine management and implementation systems? c. What have been the key factors that have enabled CMAM institutionalization in districts identified to have the capacity to independently manage and implement CMAM? d. What have been the key factors that have limited CMAM institutionalization in districts identified to have the capacity to independently manage and implement CMAM? To what extent has CAS strengthened the monitoring of the CMAM program? a. To what extent has the CAS built capacity in ensuring the development, training, dissemination, and use of standardized data collection, compilation, and reporting of CMAM monitoring data? b. To what extent has CAS ensured systems to ensure data quality, timeliness, and completeness of CMAM monitoring data? c. To what extent has CAS built capacity of facility, district, and national stakeholders to synthesize and use data for decision making? To what extent did stakeholders of the CMAM National Nutrition Operational Plan fulfill their roles and commitments? a. What are the key factors that have enabled stakeholder commitment to the CMAM Operational Plan? b. What are the key factors that have limited stakeholder commitment to the CMAM Operational Plan? What role has gender analysis played into the CMAM? a. What conclusions can be drawn from monitoring data regarding the uptake of CMAM services by gender? b. What are the factors that hindered or promoted male involvement into CMAM, and what additional inputs are needed to enhance male involvement? 2. Gender Analysis The evaluation must consider differences in the ways that women and men participate in or benefit from projects. As heads of most households, men have great influence in the overall welfare of the family; this includes household food security as well as the ability of other household members to seek health care contributing to the prevention of malnutrition at household level. In households hosting a malnourished child or pregnant and lactating mother; men can play in important role in early case identification and referral, adherence to nutrition treatment and sustaining optimal nutrition practices to void relapses. This evaluation seeks to understand the gender equality issues in the CMAM program, at all levels and how these have been addressed; and, to draw recommendations for the achievement of gender equality results. 3. The Final Evaluation report The contractor shall submit 50 hard copies and 1 electronic copy of the USAID-approved final report to USAID/Malawi. The electronic copy of the final report shall be in both PDF and MS Word format. The report must be in English, must not exceed 30 pages (excluding relevant annexes e.g. SOW, interview transcripts/notes, photos and success stories), and must include matrices and other visuals to consolidate and summarize data. Upon completion the contractor must submit flash drive comprising of all electronic products of the evaluation, including instruments and data in formats suitable for replication of the analysis, final report and the briefer. CTCIM Final Performance Evaluation - Final Report 37 The final report format will comply with the requirements set forth in the Agency’s 2011 Evaluation Policy, and must include: a. USAID branded cover page b. Executive summary i. 3-5 pages summarizing key points, including activity purpose and background, key evaluation questions, methods, findings, conclusions, and recommendations. c. Data Methods and Analysis d. Findings, Conclusions and Recommendations e. Appendices as appropriate In the final Evaluation Report the Contractor must provide a full description of methodology (or methodologies) to answer each evaluation question. Evaluation methodology shall be explained in detail and all tools used in conducting the evaluation such as questionnaires, checklists and discussion guides will be required in an Annex in the final report. E. DELIVERABLES AND REPORTS The Contractor Evaluation Team must furnish the following Deliverables and Reports: 1. Inception Report, including a detailed Evaluation Plan 2. Periodic briefings and reports 3. Oral briefing of findings to USAID/Malawi 4. Draft Report 5. Findings Workshop 6. Final report 7. Media Device, including all data 8. Final Report uploaded to the Development Experience Clearinghouse (DEC) Upon completion of the evaluation and analysis, the contractor shall submit to USAID/Malawi a media device that includes all instruments and data in formats suitable for reanalysis and submit the USAID accepted Final Report to the USAID Development Experience Clearinghouse (DEC). F. EVALUATION DESIGN AND METHODOLOGY The evaluation should utilize a mixed methods design in data collection and analysis, using primary and secondary data sources to answer each evaluation question outlined above. Quantitative data will be utilized to quantify the performance of the program and extent of CMAM institutionalization in the EHP. Qualitative data will be utilized to explain or provide insights on the contextual elements that have facilitated or hindered the program objectives. Suggested methods will, at a minimum, include 1) review of relevant program documents, 2) in-depth interviews of key informants and/or focus groups, 3) analysis of performance monitoring data. This section outlines some guidance for the study design for each of the four result areas for this evaluation. The contractor is expected to expand and improve upon this guidance as necessary. For each result area, the contractor is requested to propose the evaluation framework and assessment tools for each evaluation question, highlighting the conceptual model(s) adopted. The evaluation framework must incorporate an analysis of the intervention logic of the program; discussion of any risks and limitations that may undermine the reliability and validity of the evaluation results; and specifications of an indicator or indicators that will be used as a guide in answering each question. A detailed design and evaluation plan is required as the first deliverable. The final design and evaluation plan will be reviewed and approved by USAID before the team can begin field work. CTCIM Final Performance Evaluation - Final Report 38 Quantify the extent to which CMAM has been successfully institutionalized at district health offices The project has accumulated quantitative data on hospital admissions for malnutrition, treatment outcomes and service coverage, including baseline data prior to implementation of CMAM. The methodology for this result area may entail triangulation of routine program data to quantify the extent to which program objectives were met and the associated effects on the reduction of child malnutrition. Multiple data sources (e.g. CMAM raw data sheets, facility performance assessments and CMAM coverage data sheets) will be made available to the contractor. A checklist containing a set of components of what can be defined as institutionalization may be developed and analyzed to infer the extent to which CMAM has been successfully institutionalized at district health offices. Using the checklist, institutionalization may be quantified and expressed as ―not institutionalized, partially institutionalized, or completely institutionalized‖ by district and by site. Identify the key enabling and inhibiting attributes of successful institutionalization of CMAM at district health offices Results from routine program monitoring data indicate varying levels of performance of CMAM implementation at health facility levels. The project has periodically used program data to stratify the CMAM sites and districts ranging from ―best performing‖ and ―least performing.‖ For this result area, USAID/Malawi requests that the contractor propose an evaluation framework and tools to identify the key attributes common to sites and district health offices in each stratum, with regards to CMAM institutionalization and performance. The methodology may entail secondary data analysis and key informant interviews with CAS staff, District Health Office staff and health facility staff. CMAM performance data will likely form the basis for selecting sites within each stratum from which district and health facility staff will be selected for interview. Provide in-depth contextual information on the determinants of success or failure of institutionalization of CMAM in Malawi For this result area, the methodology may entail key informant interviews with key stakeholders including, national level nutrition program officers, District Health Management Teams and health facility staff. The contractor will also likely include program managers from the relevant development partners invested in the CMAM project. The qualitative analysis is expected to complement the quantitative analysis and provide contextual understanding on the key characteristics of sites and districts in each stratum of CMAM institutionalization or performance. Provide evidence that will inform the design of more effective programs in support of CMAM in Malawi For this result area, the contractor must identify key findings from the individual evaluation questions to formulate key conclusions and recommendations that will inform decisions on strengthening the effectiveness of CMAM in Malawi. In particular, USAID is interested in identifying actionable recommendations required of relevant stakeholders—development partners, national nutrition program, District Health Management Teams and health facilities—to strengthen the institutionalization and performance effectiveness of CMAM in Malawi. 1. Data Collection and Analysis In addition to the routine program reports and published documents listed in section III above, there are a number of primary data sources which will be made available to the contractor for secondary analysis. These data sources include: CTCIM Final Performance Evaluation - Final Report 39  CMAM monthly raw data (2008-2012)—the database contains CMAM service delivery data for all health facilities supported under the program, including indicators on admissions and treatment outcomes.  CMAM coverage data—the database contains a list of all CMAM sites in each district and the service delivery (coverage) data for each facility.  CMAM Performance data (2008-2012)—the database contains data and indicators on the performance criteria on which CAS based its classification of ―best performing‖ and ―least performing‖ CMAM sites. The contractor must summarize the evaluation methodology as part of its detailed design and evaluation plan in a matrix, as per the table below. The suggested methods and data sources have been outlined in the discussions of each result area above. However, USAID will rely on the expertise of the contractor to adjust and improve these suggestions and propose a study design that will ensure strong validity, reliability and interpretative potential of the results. Evaluation Question Indicator(s) Data Collection method(s) Data Source Sampling Comments The evaluation is expected to use existing data to the greatest extent possible augmented by primary data collected through key informant interviews, focus groups and other tools as determined by the contractor. Constraints to Data Collection and Analysis A number of factors could constrain the ability to collect or analyze data.  Language: Though English is the official language for professional communication, some of the stakeholders, in particular frontline health workers, may be more comfortable communicating their ideas in the local language, chichewa. The contractor is encouraged to include in the evaluation team individuals who are fluent in chichewa, particularly for the key informant interviews and focus group discussions.  Geography and infrastructure: The CMAM program is being implemented at over 502 health facilities in all the 28 districts of Malawi. Even with sampling, the evaluation will require considerable travel throughout the country to reach relevant health facilities and their personnel. Sample determination may be affected by the fact that some of the health facilities may not be accessible due to the combination of poor roads and heavy rains.  Data quality: there are some known deficiencies in the quality of the CMAM data. However, Concern Worldwide has an established monitoring and evaluation systems that has continuously worked to identify and rectify known data deficiencies. Sampling of Sites CMAM is being implemented at over 502 health facilities in 28 districts of Malawi. It is likely that multiple sampling frames will be required to produce representative and generalizable results for each level of interest, namely national program office, district health offices, health centers and their respective personnel who will serve as key informants. The contractor must submit a process to determine a sampling plan as part of the first deliverable, the evaluation design and evaluation plan. 2. Technical Requirements  A full description of methodology (or methodologies) to answer each evaluation question must be provided by the contractor. Evaluation methodology must be explained in detail and all tools used CTCIM Final Performance Evaluation - Final Report 40 in conducting the evaluation such as questionnaires, checklists and discussion guides are required in an Annex in the final report.  The evaluation report must include the scope of work as an annex. All modifications to the scope of work, whether in technical requirements, evaluation questions, evaluation team composition, methodology or timeline need to be agreed upon in writing by USAID.  Limitations to the evaluation must be disclosed in the report, with particular attention to the limitations associated with the evaluation methodology (selection bias, recall bias, unobservable differences between comparison groups, etc.). Sufficient information must be provided so that a reader can make an informed judgment as to the reliability, validity and generalizability of the findings.  Disclosure of conflict of interest: All evaluation team members will provide a signed statement attesting to a lack of conflict of interest, or describing an existing conflict of interest relative to the CMAM project being evaluated.  Statement of differences: If a difference arises in the interpretation of the results from the various stakeholders, the evaluation report will include a statement identifying any significant unresolved differences of opinion on the part of funders, implementers and/or members of the evaluation team. 3. Findings: Empirical Facts Collected During the Evaluation  Evaluation findings must be presented as analyzed facts, evidence and data and not based on anecdotes, hearsay or the compilation of people’s opinions. Findings must have sufficient evidence and documentation that a reader of the findings can be confident that the findings are based on actual data.  Findings must be specific, concise and supported by strong quantitative or qualitative evidence.  Sources of information need to be properly identified and listed in an annex. 4. Conclusions: Interpretations and Judgments Based on the Findings  Evaluation conclusions must be presented for each finding based on the evidence collected by the evaluation team.  Conclusions must logically follow from the gathered data and findings. Because conclusions involve interpretation of collected data, they must be explicitly justified. If and when necessary, the contractor must state his/her assumptions, judgments and value premises so that readers can better understand and assess them. 5. Recommendations: Proposed Actions for Management  Recommendations need to be supported by a specific set of findings.  Recommendations must be action-oriented, practical and specific, with defined responsibility for the action.  Contractors must take into consideration the economic and political context of the CMAM project, the strengths and weaknesses of MoH institutional capacity and the feasibility of change and innovation while framing recommendations. G. EVALUATION MANAGEMENT 1. Logistics The contractor must arrange for all logistical arrangements, including travel. The Contracting Officer’s Representative (COR) for the award will arrange for an initial introductory meeting with appropriate staff at the MOH and at the Department of Nutrition, HIV and AIDS (DNHA) prior to the initiation of work. Where necessary the COR may participate in meetings with the MOH representatives and partners. A list of relevant stakeholders and key partners will be provided to the contractor by the COR at the time of arrival but the contractor is responsible for expanding this list as appropriate and CTCIM Final Performance Evaluation - Final Report 41 arranging the meetings. The COR and other mission personnel will be available to the contractor’s team for consultations regarding technical issues, before and during the evaluation. 2. Evaluation Timeline The contractor must address each key evaluation question identified in section C.4 above in a timely and cost-effective manner. Phase 1: Desk Review—The first phase of the evaluation will begin immediately upon award of the task order and will entail thorough document review, a literature review, a review of outcome datasets, and selection/design of indicators. Phase 2: Evaluation Startup—Field work will begin in the second phase of the evaluation. Major tasks included in this phase are the in-country team planning meeting with USAID, Concern Worldwide, CIDA, the MoH, and other stakeholders; developing of a detailed the evaluation plan; construction of all data collection instruments; pilot testing instruments; training data collectors; and periodic meetings. The Inception Report (Deliverable #1) must be submitted to USAID within 20 calendar days of award. Bi-weekly updates will also be provided to USAID, including any unforeseen challenges or changes— such as changes in sites to be included in the sample (Deliverable #2). Phase 3: Data Collection, Analysis & Report Preparation—This phase of the evaluation will focus on data collection and analysis and preparation of the final report. The data collection team will complete all site visits, interviews, and focus groups, and compile any supplementary data to be analyzed. An oral briefing with the Mission will take place during this phase (Deliverable #3), as well as the creation of the Draft Report (Deliverable #4). Phase 4: Findings Workshop & Report Finalization—The final phase of this evaluation will include preparation for the Findings Workshop (Deliverable #5), including incorporating feedback from USAID & CIDA. The Final Report (Deliverable #6) will be finalized out-of-country. A Media Device with all data (Deliverable #7) will be provided to USAID and the Final Report will be uploaded to DEC (Deliverable #8). CTCIM Final Performance Evaluation - Final Report 42 ANNEX II: SUMMARY OF CTCIM PROJECT OBJECTIVES AND INDICATORS Indicator YEAR 1 2010 YEAR 2 2011 YEAR 3 2012 2013 Target Actual % Of Target Achieved Target Actual % Of Target Achieved Target Actual % Of Target Achieved Target Actual % Of Target Achieved Remarks CTCIM Objective 1. Ensure that Health Service Providers have the capacity to implement and manage CMAM 1.1 Number of relevant health staff trained in revised CTC protocols and key management skills 40 0 0 80 0 0 140 266 190 N/a N/A N/A The guidelines review was finalised in 2012 and the other NGOs were responsible for he trainings. Not part of CAS 2013 plans 1.2 Number of Zonal supervisors trained on management of acute malnutrition 15 0 0 15 0 0 15 7 46 N/A N/A N/A The guidelines review was finalised in 2012 and there was poor attendance by other Zones. Not part of CAS 2013 plans 1.3 Number of new district nutritionists trained on management of acute malnutrition 10 5 50 10 0 0 27 5 19 N/A N/A N/A No district nutritionist recruited 1.4 Number of children reached by CTC program in Malawi 38000 36316 96 43000 35283 82 40000 29914 75 40000 19500 49 Drop in the admissions due to early case identification. 2013 figure only for 1 quarter CTCIM Objective 2. CMAM is implemented in at least 80% of the Health Facilities in every district YEAR 1 2010 YEAR 2 2011 YEAR 3 2012 YEAR 2013 2.1 Number of districts that included at least 25% of CTC costs in their district health budgets/plans by project end 5 1 20 10 29 290 29 29 100 29 29 100 2.2 Number of CTC capacity assessments conducted to assist in CTC institutionalization 5 5 100 7 5 71 8 7 87 2 2 100 2.3 Number of NGO￾supported districts with phase out strategies developed 1 0 0 4 14 350 6 0 0 0 0 0 No NGO phased -out in 2010 and 2012 CTCIM Final Performance Evaluation - Final Report 43 CTCIM Objective 3. Existence of a sustainable and standardized CMAM Monitoring and Evaluation system 4.1 Number of districts submitting monthly CTC reports to MOH Nutrition Unit on time. 14 17 121 20 22 110 29 29 100 29 29 100 4.2 Number of district and health centre staff trained in data management using the skills. 172 80 47 100 91 91 50 56 112 N/A N/A N/A Not part of CAS 2013 plans CTCIM Objective 4. Evidence for support of CMAM is generated through documentation and sharing of best practices and lessons earned and CMAM is incorporated within the Pre-service Training for Health Service providers 3.1 Proposal for CMAM for long term funding is developed and submitted for funding This is a process indicator. Put milestones (stages for document development as targets) 1 1 100% n/a n/a n/a n/a n/a n/a n/a n/a n/a n/a n/a n/a 3.2 Number of success stories/best practices on CTC/CAS activities shared for global learning 4 5 125 4 11 275 4 4 100 2 2 1000 5.1 Updated nutrition curriculum incorporating CTC is developed 0 0 0 0 0 0 1 1 draft 1 1 draft Late finalization of the CMAM guidelines contributed to the delay in curriculum review 5.2 Number of key academic institutions incorporating modules for management of acute malnutrition in their curricula. 0 0 0 0 0 0 4 0 0 0 4 0 Same as above 5.3 Volunteer training manual developed & disseminated – This is a process indicator. Put milestones (stages for document development as targets) 0 0 0 1 1 100% n/a n/a n/a n/a n/a n/a Activity finalized in 2011 CTCIM Final Performance Evaluation - Final Report 44 26 The stratification sampling technique will reduce bias and ensure a sample that represents all elements of interest to the evaluation team. Evaluation Question Indicator(s) Data Collection Method(s) Data Source(s) Sampling26 Sampling Frame Comments 1) To what extent has CAS built capacity at various levels? Health service providers to implement CMAM? Level of classification; change in classification; self-reported capacity Document review; key informant interviews Facility Survey Capacity Assessment Reports; Performance data; Key informants Screening, Stratified, purposive List of Health Facilities implementing CMAM, List of CMAM documents District & national-level managers to manage the CMAM program? Self-reported capacity; CAS assessment of capacity Key informant interviews; document review Facility and Health provider surveys Capacity Assessment Reports; Performance data; Key informants Screening, Stratified, purposive List of districts and institutions that implement CMAM, List of health facilities implementing CMAM (for KIs) Pre-service training institutions to integrate the CMAM approach into the curriculum? #/% curricula integrating CMAM Document review, key informant interviews Training curricula; Capacity Assessment Reports; Key informants Screening, Stratified, purposive List of CMAM documents for desk review, List of health facilities implementing CMAM (for KIs) 2) To what extent has CMAM been incorporated into the Essential Health Package and is delivered routinely at district hospitals and primary health care service delivery points? What elements have been incorporated into routine management and implementation systems? Frequencies of CMAM features Document review, Key informant interview Facility survey Capacity Assessment Reports; Quality Reports; Annual & Quarterly Reports Screening, Stratified, purposive List of documents, List of Health Facilities implementing CMAM (for KIs) ANNEX III: EVALUATION PLANNING MATRIX MATMATRIXMATRIX CTCIM Final Performance Evaluation - Final Report 45 27 The stratification sampling technique will reduce bias and ensure a sample that represents all elements of interest to the evaluation team. Evaluation Question Indicator(s) Data Collection Method(s) Data Source(s) Sampling27 Sampling Frame Comments 2) To what extent has CMAM been incorporated into the Essential Health Package and is delivered routinely at district hospitals and primary health care service delivery points? What elements have not been incorporated into routine management and implementation systems? Frequencies of CMAM features not adopted Document review, key informant interviews Facility and health provider survey Capacity Assessment Reports; Quality Reports; Annual & Quarterly Reports Screening, Stratified, purposive List of documents, List of Health Facilities implementing CMAM (for KIs) What have been the key factors that have enabled CMAM institutionalization? # enabling features; nutrition outcome data Key informant interviews, focus groups, literature review, data pull Key informants; field literature; secondary outcome data Screening, Stratified, purposive List of documents, List of Health Facilities implementing CMAM (for KIs) Stratification will reduce bias during FGDs What have been the key factors that have limited CMAM institutionalization? # limiting features; nutrition outcome data Key informant interviews, focus groups, literature review, data pull Key informants; field literature; secondary outcome data Screening, Stratified, purposive List of documents, List of health facilities implementing CMAM (for KIs/FGDs), CTCIM Final Performance Evaluation - Final Report 46 28 The stratification sampling technique will reduce bias and ensure a sample that represents all elements of interest to the evaluation team. Evaluation Question Indicator(s) Data Collection Method(s) Data Source(s) Sampling28 Sampling Frame Comments 3) To what extent has CAS strengthened the monitoring of the CMAM program? Built capacity in development, training, dissemination, and use of standardized data collection, compilation, and reporting of CMAM monitoring data? Self-reported capacity to implement M&E system; #/% sites using an M&E system Document review; key informant interviews M&E Plan; Handover Roadmap; facility performance assessments Screening, Stratified, purposive List of documents, List of health facilities implementing CMAM (for KIs), Created systems to ensure data quality, timeliness, and completeness of CMAM monitoring data? #/% sites using an M&E system; quality of M&E systems Document review Facility performance assessments; Capacity Assessment Reports Screening, Stratified, purposive List of documents Built capacity of facility, district, and national stakeholders to synthesize and use data for decision-making? # reports, briefs, or memos created from datasets; evidence￾based decisions Document review Policy documents; facility & district reports, briefs, & memos Screening, Stratified, purposive List of documents 4) To what extent did stakeholders of the CMAM National Nutrition Operational Plan fulfill their roles and commitments? Key factors that have enabled stakeholder commitment to the Operational Plan? Identified enabling factors Document review; key informant interviews CMAM Operational Plan; key informants Screening, Stratified, purposive List of documents, List of health facilities implementing CMAM (for KIs), Key factors that have limited stakeholder commitment to the Operational Plan? Identified limiting factors Document review; key informant interviews CMAM Operational Plan; key informants Screening, Stratified, purposive List of documents, List of health facilities implementing CMAM (for KIs/FGDs), CTCIM Final Performance Evaluation - Final Report 47 29 The stratification sampling technique will reduce bias and ensure a sample that represents all elements of interest to the evaluation team. Evaluation Question Indicator(s) Data Collection Method(s) Data Source(s) Sampling29 Sampling Frame Comments 5) What role has gender analysis played in CMAM? Uptake of CMAM services by men and women? Uptake of CMAM services by men and women? Uptake of CMAM services by men and women? Uptake of CMAM services by men and women? Uptake of CMAM services by men and women? Uptake of CMAM services by men and women? Uptake of CMAM services by men and women? CTCIM Final Performance Evaluation - Final Report 48 ANNEX IV: ATTENDEES FOR THE FINDINGS WORKSHOP Institution/Title Name Mzimba N DHO Kenan Mfune Valid Nutrition Happy Botha Ministry of Agriculture Margaret Lwanda MoH- Nutrition Unit Dalitso Kango'mbe OPC Janet Guta WFP Mutinta Hambayi FAO Stacia Nordin CIDA Peter Killick CIDA Julita Manda USAID Violet Orchardson Irish Aid Adrian Fitzgerald WALA Catherine Chiphazi Principle nutritionist, Ministry of Health Sylvester Kathumba Family Health Cluster Lead - USAID/Malawi Ruth Madison Blantyre DEHO Chriswell Nkoloma DEDZA DEHO MR E .MBAWA ZOMBA DHO Reuben Chikadz MANGOCHI DHO Dr. William Peno MANGOCHI DHO K. Mamba KASUNGU Joseph Chitsime MZIMBA NORTH MR A. MWANGONDE SALIMA DHO Noel Khunga NMCM Lucy G. Gawa College of Medicine John Phuke Thylo DHO Kennedy Kanyimbo WFP Osborne Silbande Feed the Children Docile Kalunga USAID Miriam Lutz Feed the Future Martin Tembo Concern World Wide Noel Molony CHAM Massiye Nyang'wa Feed the Children Eveness Zuze CRS Stanley Mwase DFID Ruth Mwanfira LUANAR Beatrice Mtimuni Unicef Benson Kazenbe CHAI Andrews Gunda DOWA MR .MITOCHI CTCIM Final Performance Evaluation - Final Report 49 Institution/Title Name Unicef/MOH Mphatso Mapemba MoH- Nutrition Unit Tapiwa Ngulube Concern World Wide Alice Grandiwa Concern World Wide Rosealind McCulum IA Lawrence Munthali Mothers2Mothers Delia Chikuse Feed the Children Lucy Maseloo CHAM Hope Ngwira Nkhoma Synod Yoas Mvula MOAFS F.L. Kayumi Malawi Defense Force Denis Chakitaza CTCIM Final Performance Evaluation - Final Report 50 ANNEX V: GENDER ANALYSIS The evaluation required that an analysis be conducted to ensure that gender issues of males and females had been identified and addressed by the CTCIM project. The gender analysis framework was guided by resources created through the support of USAID specifically for the health sector and the CIDA Framework for Assessing Gender Equality Results. 30 The 2012 and 2008 USAID gender assessments were consulted and the primary gender-related laws and policies of the government of Malawi were reviewed.31 Secondary research was reviewed and as were gender assessments done by other development agencies for insights into the factors that might enable or inhibit men and women from seeking help in understanding why a child is ill and may need medical care, as malnutrition may have many causes. Overall gender contextual dimensions were also considered, such as: access to resources; knowledge, beliefs and perceptions; practices, roles, and participation; legal rights and status; and power. Reports concerning gender issues being addressed by the Government of Malawi were also tracked to make sure the team had an understanding of the current context. These contextual dimensions are presented in Table 16, along with further information needs, the gender-based constraints suggested by the literature, and gender-based opportunities for reaching project objectives. Table 16 was shared with the evaluation team and discussed. The contextual dimensions were then used to develop gender-specific questions for the survey research instruments. The team discussed how to best ask the questions during interviews. The instruments were then field tested and some of the gender questions adjusted to be more precisely within the potential experience of the respondent. Each instrument included gender-sensitive questions that were repetitive so that the gender-related information that each respondent would know about could be captured and then compared. Once the data were collected from the different instruments, qualitative answers were discussed and a coding system developed and applied. All the survey-based information was analyzed using frequencies. The FGD transcripts were then read, compiled, and reviewed using a simple content analysis. The responses to each question were then put into a table for each question and compared. These tables are summarized in the main evaluation report and are found at the end of this Annex. 30 These include Tips for Conducting a Gender Analysis at the Activity or Project Level: Additional help for ADS Chapter 201, USAID 2011; A Practical Guide for Managing and Conducting Gender Assessment in the Health Sector, Margaret E. Greene. Interagency Gender Working Group and USAID June 2013. http://www.igwg.org/Articles/genderassessmentsguide2013.aspx; and Gender Assessment Guide: A Manual for Integrating Gender into Reproductive Health and HIV programs: from Commitment to Action. Second Edition, Deborah Caro. Interagency Gender Working Group and USAID. . For CIDA, the Framework for Assessing Gender Equality Results (revised 2010) and the Policy on Gender Equality (revised 2010) were consulted (http://www.acdi-cida.gc.ca/INET/IMAGES.NSF/vLUImages/Policy/$file/GE-Q&A-Culture-EN.pdf and http://www.acdi-cida.gc.ca/INET/IMAGES.NSF/vLUImages/GenderEquality3/$file/GE-Framework-EN.pdf. 31 These include the Gender Equality Law 2013; the Malawian Constitution; and the National Gender Policy 2000,the National Gender Program . Key Informant Interview with Dr. Mary Shawa, Principal Secretary, Ministry of Gender, Children, and Social Welfare, June 26 and July 16, 2013, Lilongwe. CTCIM Final Performance Evaluation - Final Report 51 Drawing from the field visits, eight questions were created that examine project performance against the quantitative and qualitative data findings. These questions are: 1. Does the CMAM Initiative adequately respond to the specific needs of male and female participants and clients? 2. Are there any social obstacles for males and for females to achieving the objectives of CMAM that were not adequately addressed by the original project design? 3. Have the community mobilization activities hindered or promoted uptake of CMAM services by males and females? 4. What other factors have hindered or promoted male involvement in CMAM? 5. Do health facility staff members (zonal, district, community volunteers) have the appropriate set of skills to address the different needs of male and female clients? 6. Are the indicators used for monitoring and evaluation adequately capturing how the project affects males and females? 7. How have the CMAM data disaggregated by age and sex used to refer clients to other programs? 8. Have CMAM data been used to inform changes to other programs offered at the health facility? Table 14 lists these questions and presents how the project responded and how the project worked in relation to addressing each question. Table 14 and the summary findings of the FGD, Lilongwe based KIIs, 10 district KIIs, facility survey, healthcare provider survey and email survey were then reviewed. This review formed the basis for the conclusions and recommendations for Question 5 in the main evaluation report. Additional Analysis The primary findings of the survey research data are described in the main evaluation report. Analysis of the gender-sensitive questions was conducted to see if they revealed any additional information. We concluded that the questions were important in context and that no additional conclusions could be drawn from the findings. As well, it should be noted that the number of respondents is variable in terms of the completed answers to each question. Healthcare Providers Survey. Question 25 sought information about the responses of caregivers after they were informed by the healthcare provider that their child was malnourished. Out of 17 respondents, 10 had a positive reaction and 7 had a negative response, indicating that there is no clear majority response to this question (Table 9 below). Table 9. How do caregivers react if their children are found out to be malnourished (Question 25 Healthcare Providers Survey) 9. 10.Percent Number 11.Positive, concerned reaction, 12. 59 10 13.Negative, denial, angry reaction, 14. 41 7 15.Total 16. 100 17 17. 18. Question 26 probed further to see if there was a difference between the responses of male and female caregivers. Of the 10 respondents, 8 indicated that women have more positive reactions than men (Table 10 below). CTCIM Final Performance Evaluation - Final Report 52 Table 10. Do female caregivers react differently than male caregivers? Can you describe any differences? (Question 26 Healthcare Providers Survey) Percent Number Positive, concerned reaction, 80 8 Negative, denial, angry reaction, 20 2 Total 100 10 Facility Survey. A question was asked to see if the healthcare providers used a routine to identify every malnourished child. This was followed by a question to see if the routine varied between boys and girls. This was to see if there was a difference in the attitude that each healthcare provider approached boy and girl children (Table 11 below). Table 11. Do you have a routine way to identify every malnourished child? (Question 16 Facility Survey) Number Percent Yes 13 16 No 0 0 All respondents that answered the follow-on question but one stated that there was no difference for boy and girl children. For the one who said there was a difference no explanation was recorded on the questionnaire (Table 12 below) Table 12. Does this routine differ for male and female malnourished children? (Question 17 Facility Survey) Number Percent Yes 1 8 No 11 92 Total 12 100 Key Informant Interviews. Respondents in the 10 districts were asked the following two questions: 17. Have gender differences been considered in how CMAM is implemented in your district? a. What conclusions can be drawn from monitoring data regarding the uptake of CMAM services by sex in your district? b. Why do men use CMAM for their children? c. Why do some men not use CMAM services for their children? d. What so you think would be the best way to encourage men to use CMAM services or to work as volunteers? e. What parts of connecting with communities could be improved to encourage men to use CMAM when their children are sick? f. Are male and female CMAM volunteers regarded with the same level of respect at your district? g. Is CMAM addressing the differences in males and females eating and food distribution practices? h. In your district, do you know of domestic violence occurring if a woman does not give her husband the best food and instead uses it for her children? CTCIM Final Performance Evaluation - Final Report 53 18. Which family member decides whether a child should be enrolled in CMAM? a. Do men prefer working with a male or female CMAM volunteer? b. Do women prefer working with a male or female CMAM volunteer? c. Is there a difference in the way you see men and women behave when they bring a child to receive CMAM care? d. Based CMAM data, we notice that more girls are treated than boys. Why do you think this is? There were 14 respondents of which 9 were men and 5 were women. The KII instrument was used with Nurses, EPI officers, and Nutrition Officers if they had CMAM management or coordination responsibilities. Table 15 provides the responses of key informants interviewed during visits to the 10 districts. The responses point to how the CMAM database has helped managers and coordinators see that there are differences in the uptake of boys and girls. The responses concerning food distribution and of domestic violence point to the need for further investigation, research and observation.32 The comments concerning male involvement were used to inform the conclusions and recommendations. An email survey was sent to over 200 individuals identified as CMAM stakeholders by CWW. One question asked the respondent what they thought the primary impact of training men might be. Table 13 below indicates that most (including the ―other‖ category) thought this would have a positive impact. Table 13. What do you think the main impact of training men in nutrition would be on children's nutrition? Answer Options Response Percent Response Count No impact on reducing malnutrition of children. 0.0% 0 Would increase the amount of protein fed to children 33.3% 3 Would increase the number of meals fed to children 55.6% 5 Do not know 11.1% 1 Other (please specify) 5 answered question 9 skipped question 12 For the ―other‖ category, the following comments were made:  It would improve children’s nutritional status through implementation of preventive and treatment interventions including points 2 and 3 above  As the main bread winners they will know what type of nutritious meals to give to their children, consequently reducing malnutrition  It would improve both prevention and early seeking of care  It would help in infant and young child feeding and might help reduce malnutrition 32 See, for instance, The Link between Social Cultural Practices, GBV, HIV and AIDS: Baseline Study Report. Selphera Consulting LTD and Prime Health Consulting Services. April 2013. Government of Malawi, European Union and UN Population Fund. SADC Gender Barometer 2012. Chapters 5 GBV, Chapter 6 Health, and Chapter 7 HIV. Compiled by Emma Kaliya with Ruth Ayisi and Loveness Jambaya Nyakujarah. Gender Links, Johannesberg, South Africa. CTCIM Final Performance Evaluation - Final Report 54 Box 8. Gender Roles and Nutrition ―Gender issues exacerbate the problem. Current nutrition education programmes are attending mainly by women, but household-level decisions are usual made by men. Traditional gender roles also skew the distribution of nutritious food within the household; men are favored in both food and resource distribution, typically at the expense of women and children. The HIV and AIDS pandemic has compounded the dual burdens of malnutrition and disease.‖ Malawi National Nutrition Policy and Strategic Plan 2007-2011, p. 20; also stated in the Nutrition Capacity Assessment in Malawi, December 2009, FAO, p. 1.  More children would be admitted through the support of men The NECS plan focuses on fostering greater involvement of men to prevent child stunting and is utilizing an integrated approach among GOM ministries to achieve this goal. The email survey responses support the view that this position is accepted among CMAM stakeholders. Desk top Literature review A literature review was conducted on possible gaps between males and females to identify the possible relationships between gender roles and malnutrition, and how these might impact the implementation of the CMAM program. This literature review is not comprehensive as it was limited by time, availability of documents on the Internet, and the need to focus on the nexus of key issues related to CMAM instead of delving deeply into each subject. This literature and the GOM national agenda on addressing gender equality, GBV and preventing child stunting pointed to the following areas that might affect the quality and distribution of food made available to household members:  Domestic gender-based violence  Social norms on gender roles  Intra-household nutrition While reports and published papers addressed these topics separately, the four themes were commonly mentioned in many of the documents. This is because the source of malnutrition within families is driven by many functions, including limited access income sources; low levels of education; social norms concerning decision-making roles for males and females; and cultural practices use of traditional foods; HIV; and the availability of food, among many others. None of these features alone can explain the high stunting rates of boys and girls in Malawi (see Box 8). The GOM is making a concerted effort to effectively address these and other functions through the NECS 2011- 2016 and SUN Initiative, the National Nutrition Policy 2007- 201133; the 2008 Revised Gender Policy34; the forthcoming new National Gender Policy, and the Agriculture Sector Wide Approach 2011-201535. 33 The National Nutrition Policy and Strategic Plan, 2007-2011. Department of Nutrition, HIV and AIDs, March 2009. Office of the President and Cabinet, Republic of Malawi. 34 The Revised Gender Policy, February 2008. Ministry of Women and Child Development, Republic of Malawi, 2008. 35 The Malawi Agriculture Sector Wide Approach 2011-2015, Ministry of Agriculture and Food Security, 2011, Government of Malawi. Pp. -29-30 specifically address the links between overall female vulnerability to physical abuse, access to productive resources, and food insecurity especially among female headed-households and females living with HIV and AIDS, CTCIM Final Performance Evaluation - Final Report 55 Box 9. GBV and Stunting ―However a comparative analysis of Demographic and Health Surveys from Malawi, Kenya, Honduras, Egypt and Rwanda (Rico et al, 2011) reveals a link between maternal intimate partner violence and child malnutrition…In Malawi and Honduras, marginal associations were observed between Intimate Partner Violence, severe stunting and under-2 mortality. Researchers concluded that the relationship between intimate partner violence (intimate partner violence) and under-2-year-old mortality and stunting in Kenya, Honduras and Malawi suggests that intimate partner violence plays a role in poor child health outcomes. In addition, the findings underscore the importance of incorporating efforts to prevent violence against women into a wider range of maternal and child health programmes.‖ Assessment of the Capacity of the Health Sector to Respond to Violence against Women and Children in Malawi. Lignet Chepuka; Miriam Taegtmeyer, Esther Richards, Eleanor MacPherson; and Sally Theobold. UNICEF and the London School of Tropical Medicine, 2011. P. 16 This literature review focused on the reports that address or remark on these complex relationships and specifically informed the gender analysis in Table 16. As mentioned earlier, the analysis of this literature and CTCIM project reports guided the development of the evaluation survey research instruments and focus group discussion guides. Gender-based violence. Key informant interviews with the NGO Gender Network, the MEGEN Network and the Ministry of Gender, Children and Community Development (MOGCCD)36 pointed to a relationship between domestic violence and food distribution in the household. A recent baseline study describes in detail the different social and cultural practices by different groups in Malawi and the relationship to GBV and how some of these practices are being adjusted to modify the more harmful effects. The study identifies continued dominance of men as the household head and as the decision-maker, in both matrilineal and patrilineal groups as a contributing factor to acceptance of GBV. Female submission continues to be considered a virtue and is promulgated through family male and female members, media, teachers, and religious leaders.37 Further, a recent comprehensive study by UNICEF marks violence against women and children as a public health problem and calls for coordination across sectoral government agencies to identify and address it (see Box 9). Social norms and gender roles. The annual SADC Gender Barometer and the Malawi Gender index and the Gender Development Index both indicate that social norms and cultural practices are changing.38 However, overall male dominance continues to restrict the full participation of women in crucial arenas of decision-making in the public sphere and in the private sphere of the household, where decisions about food purchasing, preparation and distribution and nutrition are made on a daily basis. Research on barriers to participation in mother to child transmission of HIV services showed that male 36 See Key Informant Interview transcripts with Victor Maulidi, Coordinator, NGO Gender Network June 26, 2013, Lilongwe; Marcel Chisi, National Chair, Men for Gender Equality Now (MEGEN), June 27, 2013, Lilongwe; and Dr, Mary Shawa, Principal Secretary, Ministry of Gender, Children, and Community Development, June 26 and July 16, 2013, Lilongwe. 37 See The Link between Social Cultural Practices, GBV, HIV and AIDS: Baseline Study Report. Selphera Consulting LTD and Prime Health Consulting Services. April 2013. Government of Malawi, European Union and UN Population Fund, p. 43-49. This soon to be published study has an excellent, focused bibliography on social and cultural links to GBV and HIV and AIDS in Malawi; pp. 72-73. 38SADC Gender Barometer 2012. Executive Summary,. Compiled by Emma Kaliya with Ruth Ayisi and Loveness Jambaya Nyakujarah. Gender Links, Johannesberg, South Africa. Pp. 5-7. Malawi Gender Development Index 2011. Ministry of Women, Children and Community Development and the National Statistics Office, June 2012. CTCIM Final Performance Evaluation - Final Report 56 Box 10. Household Nutrition ―Dietary and child feeding practices were inappropriate in that the foods were low in energy and nutrient density. It is likely that poor dietary practices contributed much to the high prevalence of malnutrition among the under five children. Providing training to mothers and household members on appropriate dietary practices will help to improve the situation.‖ Baseline Nutrition Survey, Ntchisi ADP-Malawi. Beatrice Mtimuni, Owen Nhkoma, Mangani Katundu and Numeri Geresomo. March 2010. Swedish International Cooperation Agency and the Norweigian Ministry of Foreign Affairs. p. 41. attitudes impacts a mother’s willingness to use these services and has a negative impact on child nutrition.39 Intrahousehold nutrition decision-making. The Department of Home Economic and Human Nutrition at the Bunda College of Agriculture has ongoing research agenda to examine household nutrition throughout Malawi. In one study baseline study researchers demonstrated that in an area with low dietary diversity that chronic stunting increases a food security declines (see Box 10). Recent formative research to develop a culturally appropriate nutrition intervention in Ntchisi District found that intra-household food sharing practices vary by season and other contextual factors related to age, gender of household members, and number of children of similar age; and that inter-household food sharing practices vary by season.40 A series of articles by a research team working in Northern Malawi indicate that child nutrition occurs within a complex family environment, where older female generations have a huge influence on what children are fed and weaning patterns which are based on tradition and may not necessarily follow the recommended six Malawi food groups.41 As well, the team found that these patterns could be changed and a positive impact by linking a participatory approach to agriculture and nutrition education to multiple generations. With this approach, following a six year intervention, the team found child growth rates could be improved and sustained over time, and household nutrition and feeding behavior changed.42 This limited review of the literature helped define the domains for further investigation as presented in Table 1. This background information was then used to inform the development of the evaluation research tools. These literature and the findings of the evaluation qualitative and quantitative data indicate that efforts to treat identify and treat malnutrition through CMAM and to prevent malnutrition through the SUN initiative and the NECS strategy are well placed. Further research is needed on possible confounding issues such as GBV, social norms and cultural practices that inhibit male and female’s understanding of nutrition and preventing stunting in their children. 39 See Nyasulu,Juliet Yauka and Peter Nyasulu. ―Barriers to Uptake of Prevention of Mother to Child Transmission Services in Rural Blantyre and Balaka Districts, Malawi.‖ Journal of Rural and Tropical Public Health. April 2011, Volume 11, p.49. 40 See Formative Research to Develop a Culturally Appropriate Nutrition Intervention in Ntchisi, Malawi. PowerPoint slides. Stephen Kodish and Joel Gittlesohn, May 7, 2013. Johns Hopkins Bloomberg School of Public Health; slides 25/44, 26/44 and 27/44. 41 See Bezner Kerr, R., L. Dakishoni and L. Shumba. 2008. ―We Grandmothers Know Plenty: Breastfeeding, Complementary Feeding, and the Multifaceted Role of Grandmothers in Malawi.‖ Social Science and Medicine 66 (5): 1095–1105; p. 1103. 42 Bezner Kerr, R., Berti, P. R. and Shumba, L. 2010. ―Effects of Participatory Agriculture and Nutrition project on Child Growth in Northern Malawi.‖ Public Health Nutrition 14 (8): 1466–72; p. 2. CTCIM Final Evaluation - Draft Final Report 57 Table 14: Gender-Related Evaluation Questions CTCIM Evaluation: Gender Analysis Project Performance Evaluation Questions How the Project Responded How the Project Worked 1. Does the CMAM Initiative adequately respond to the specific needs of male and female participants and clients? CAS provided support to creating:  CMAM guidelines incorporated into routine health service delivery.  Assisted with development of national CMAM guidelines; standardized reporting forms and job aids (printed and disseminated by UNICEF).  Healthcare providers reported no difference in treatment of girl and boy children in health centers (responses to Facility Survey Questions 17 and 18 and to Healthcare Provider survey Questions 26 and 27). KII respondents from the 10 districts indicated:  The CMAM data show more girls than boys have been entered into the CMAM database to as service users, the reasons for this are unknown; except this reflects the demographic balance of Malawi (Question 17a and 18 d).  No resistance from men in seeking or accepting CMAM treatment for malnourished children had been noted (Question 17c). 2. Are there any social obstacles for males and for females to achieving the objectives of CMAM that were not adequately addressed by the original project design? CAS supported Learning Forums addressed health center access issues and volunteers retention; sharing of best practices among districts:  Strengthening Community Outreach CTC Learning Forum report #16, Volume 3, Issue 3. Recommendations on involving men, retaining volunteers and engaging village chiefs and religious leaders; with specific actions assigned to district and MOH health staff.  CMAM guidelines (2013; Section 2 focuses on community outreach including: understanding community social and  FGD summaries did not reveal social access issues for women and men to health clinics (Question 1).  FGD summaries (Question 1) and Healthcare Providers survey (Question 9) indicate transportation costs and poor road infrastructure (rainy season mud) make it difficult for remote communities to access health services, and for HSAs to travel to communities. KII respondents from the 10 districts indicated  Nutrition outreach activities are not specifically-oriented to men but rather families and women with young children CTCIM Final Evaluation - Draft Final Report 58 CTCIM Evaluation: Gender Analysis Project Performance Evaluation Questions How the Project Responded How the Project Worked cultural characteristics; conducting community dialogue, training community based service providers; case finding and referral; screening and follow-up). (Question 17e).  Incentives for volunteers might attract more male volunteers (Question 17d).  Males and females do not have difficulty speaking with women and men volunteers (Question 18 a and 18b).  Did know not of circumstances where there were differences in male and female eating and food preferences- needs additional research (Question 17g).  Did not see evidence of a link between intra-household food distribution and domestic violence. Needs further research as it is linked to poverty/income and other factors. (Question 18h). 3. Have the community mobilization activities hindered or promoted uptake of CMAM services by males and females? CAS provided support to creating:  Creation of training curriculum, training a cadre of national CMAM trainers.  Mentoring and technical backstopping support to district and national trainers to conduct CMAM trainings.  CMAM and IEC Advocacy workshop (2012) to design responsive materials.  CMAM guidelines (2013; Section 2 focuses on community outreach including: understanding community social and cultural characteristics; conducting community dialogue, training community based service providers; case finding and referral; screening and follow-up).  Learning Forums held on:  Effective Community Participation CTC Learning Forum Report #10, Volume 2, Issue 1, November 2009 KII respondents from the 10 districts indicated:  Nutrition outreach activities are not specifically-oriented to men but rather families and women with young children (Question 17e).  Incentives for volunteers might attract more male volunteers (Question 17d).  Males and females do not have difficulty speaking with women and men volunteers (Question 18 a and 18 b). CTCIM Final Evaluation - Draft Final Report 59 CTCIM Evaluation: Gender Analysis Project Performance Evaluation Questions How the Project Responded How the Project Worked  Community Participation CTC learning Forum Report #13, Volume 2, Issue 4, December 2010  Community Participation CTC Learning Forum report #10, Volume 2, Issue 1. November 2009 Volunteer issues addressed 4. What other factors have hindered or promoted male involvement in CMAM? Learning Forums held on:  Community Participation CTC Learning Forum report #10, Volume 2, Issue 1. November 2009. Need to involve men raised and retain volunteers raised and initial recommendations.  Strengthening Community Outreach CTC Learning Forum report #16, Volume 3, Issue 3. Recommendations on involving men, retaining volunteers and engaging village chiefs and religious leaders; with specific actions assigned to district and MOH health staff.  Health service provider surveys indicated that religious beliefs at times prevent male and female caregivers from using CMAM services, and HSAs inform the village headman and jointly meet with the family (Questions 22, 23, 24).  One Male Household Head FGD, two Male Caregiver FGDs and one Female Youth FGD reported religious and traditional beliefs about witchcraft dissuade addressing malnutrition. through CMAM by males and females (Question 1). KII respondents from the 10 districts indicated:  Nutrition outreach activities are not specifically-oriented to men but rather families and women with young children (Question 17e).  Men do not like to sit and wait together with women/babies at the health center for the under 5 clinic. Several suggest men with babies/children get served first. (Question 17e). 5. Do health facility staff members (zonal, district, community volunteers) have the appropriate set of skills to CAS provided support to creating:  Training a cadre of national CMAM trainers.  One Male Household Head FGD and one Female Youth FGD reported negative attitudes of heath facility staff toward CTCIM Final Evaluation - Draft Final Report 60 CTCIM Evaluation: Gender Analysis Project Performance Evaluation Questions How the Project Responded How the Project Worked address the different needs of male and female clients?  CMAM curriculum (2013)  CMAM guidelines (2013; Section 1 provides step by step details on non￾threatening nutritional counseling techniques for healthcare providers). them.  In Healthcare provider surveys administered to HSAs, several remarked on the negative attitude of nurses toward female caregivers of malnourished children. 6. Are the indicators used for monitoring and evaluation adequately capturing how the project affects males and females? CAS provided support to creating:  CMAM guidelines (2013; Section 6 provides explanation as to why sex disaggregated data are important for planning, case management; supervision and case review).  Healthcare provider surveys indicated they use CMAM reports weekly for patient/client management and planning (Question 32).  KII respondents from the 10 districts indicated the CMAM data show more girls than boys have been entered into the CMAM database to as service users. Because the CMAM protocols are applied to all children, the uptake of girls and boys shows that CMAM is reflecting the demographic balance of Malawi (Question 17a and 18 d).  Healthcare providers reported no difference in treatment of girl and boy children in health centers (responses to Facility Survey Questions 17 and 18 and to Healthcare Provider survey Questions 26 and 27). 7. How have the CMAM data disaggregated by age and sex used to refer clients to other programs? CAS provided support to creating:  CMAM data entry forms require sex disaggregation by new admission.  Learning Forum Report on Strengthening CMAM data Utilization in Districts, Volume 3, Issue 3, June 2011. The CMAM data set collects uptake information on boys and girls but does not follow each child as a case. As this is the only sex-disaggregated data collected in the CMAM excel database, it was not possible to analyze the data further as the information on other indicators is not collected on a disaggregated basis for boys and girls. CTCIM Final Evaluation - Draft Final Report 61 CTCIM Evaluation: Gender Analysis Project Performance Evaluation Questions How the Project Responded How the Project Worked 8. Have CMAM data been used to inform changes to other programs offered at the health facility? CAS provided support to creating:  CMAM guidelines (2013; Section 6 procedures for tracking transfer of patients to other treatment centers in health facility).  CTC Learning Forum Report: Integrating HIV Services with CTC, Volume 1, Issue 2, April 2007.  Nurse respondents in the Healthcare Provider surveys indicated they use CMAM reports weekly for patient/client management and planning (Question 32).  Both Nurse and EPI Officers respondents report that they use CMAM data is part of their integration activities (Question 32). The team collected photographic evidence of CMAM protocols posted next to other procedure information in Nurses stations.  Healthcare providers all responded to Question 32 in the Healthcare Provider survey that they recommend and advise that all children when identified as malnourished be tested for HIV. CTCIM Final Evaluation - Draft Final Report 62 Table 15: Key Informant Interview Summary Gender Questions Question 17. Have gender differences been considered in how CMAM is implemented in your district? Although they collect sex-disaggregated data has not looked at gender differences but knows more are boys stunted but does not know the reason why maybe boys grow faster. Useful to know which sex has greater malnutrition levels. What conclusions can be drawn from monitoring data regarding the uptake of CMAM services by sex in your district? More girls using the CMAM services, some said the same rate. Girls are being enrolled sooner in programs. Is it possible it is easier to see girls needing intervention. More malnourished girls enrolled in CMAM. Why do men use CMAM for their children? Mostly just focused on the women primarily using this service. One said perhaps the mother was busy. Men are not that involved but can’t stop their wives from coming to get help. Few fathers bring children in. Harder to get man involved in many health activities. We are supposed to emphasize increased involvement men in CMAM. Why do some men not use CMAM services for their children? Most common was that it was the role of the mother, and that men didn't want to spend the time waiting there. Also it was noted that at NRUs they don't have a separate place for men to sleep. Men don’t want to go to the health facilities providing CMAM because as there are a lot of women also waiting in the area. Some men are very uncomfortable not being in the clinic waiting with a lot of mothers, and men are not used to waiting for a long while to see the nurse or HSA. All along women have been the care providers of the sick and if a man comes there is speculation about why the man came. Not happening in many areas of health. What so you think would be the best way to encourage men to use CMAM services or to work as volunteers? Education and encourage male involvement. Have to change culture. Men go for piecework employment and working in fields far away or are making charcoal, also far from the family homes. Urban and rural settings are different. Men are the decision makers of HH and influence what woman do in caring for family. What would help? Give men first priority to be seen over women. What parts of connecting with communities could be improved to encourage men to use CMAM when their children are sick? Educating communities. If men do take children to use CMAM services they get treated first to encourage more male involvement. CTCIM Final Evaluation - Draft Final Report 63 Table 15: Key Informant Interview Summary Gender Questions Men need adequate awareness about nutrition, which they do not get in any organized way, targeted to them. Empowering local leaders they can put measures in place to force men or NGOs to help with awareness/volunteer and sense of ownership would help. Education of men might work. At the community level the responsibility should not just be focused on women. Don’t have IEC messages but we tell them to get involved. Are male and female CMAM volunteers regarded with the same level of respect at your district? Usually they were regarded the same. A few said women were preferred because they were home when people needed them, and that they cared about their work more. One said females were more trusted. No difference. To get more male volunteers need more incentives for volunteers. At one time they had 2 volunteers- a male and female per community no salary but did get lunch allowance. Male and female volunteers receive the same respect. Is CMAM addressing the differences in males and females eating and food distribution practices? Health education, trainings. Training, but respondent not sure if it had helped. One respondent said he/she wasn't sure if it had tricked down yet to the community level (addressing this issue) but "It should be priority to "critically analyze how food is being distributed". Urban setting not as true in rural girls eat with mother and boys eating with father or alone. Men make decisions about what children eat …a personality trait may override tradition. It is just unfortunate that men in community feel they should get the meats and eggs instead of the children. In your district, do you know of domestic violence occurring if a woman does not give her husband the best food and instead uses it for her children? Many people weren't sure. Some said it did happen. One said it did happen but isn't discussed, but that communities have their own way of policing it. Culture was cited as contributing since men are seen as the best. It is more likely there is domestic violence if female caregivers gives better food to a step child. Rural areas have an overall reputation of increased gender-based violence. Some men don’t want their children to have something better than they have- CTCIM Final Evaluation - Draft Final Report 64 Table 15: Key Informant Interview Summary Gender Questions -there is a link between gender-based violence and some kids with malnutrition but does not know what it is. Hasn’t heard of violence around food distribution. Question 18 Key Informant Which family member decides whether a child should be enrolled in CMAM? Many said both, some said mother some said father Father decides, unless he is away from the home. Both men and women must decide. In the NRU? Mother is supposed to get permission. Do men prefer working with a male or female CMAM volunteer? Some favor toward women since they seemed to care more about their job. One stated that only men should be trained because too high of turnover with women due to them following their husbands. No difference. Volunteers treated the same re gender Do women prefer working with a male or female CMAM volunteer? Men don't want to stay or wait. They get angry quicker. Women are more understanding and patient. They are more willing to stay. "They are happy to stay and sing songs" Men may prefer men as educators but depends on services No difference. Is there a difference in the way you see men and women behave when they bring a child to receive CMAM care? Not sure Women often have conflict with female nurses but not with CMAM volunteers. Reactions really vary mothers are ashamed and often too long. Based CMAM data, we notice that more girls are treated than boys. Why do you think this is? Their statistics show more girls getting treating and doesn’t know why. CTCIM Final Evaluation - Draft Final Report 65 Table 16: Gender Relations, Constraints, and Opportunities What are key gender relations related to each domain that affect males and females participation in CMAM? What other information about gender relations is needed? What are the gender-based constraints hindering achievement of project objectives? What are the gender-based opportunities for reaching project objectives? a. Access to Resources  % of women with access to income  Women’s access to information comes through which family member?  Do women share information with each other? How?  How do men gain information? How do they share it?  Information on whether women without their own income have access to other access to resources.  Information on the use of health services by males and females.  Information on fees and transport costs that could affect access to health services for women and men. Cost of transportation for males and females to take children to health facilities Cash to purchase healthy food (Malawi six food groups), for all family members to consume, for all 12 months of the year. Cash may be used for things other than food. b. Knowledge, beliefs, perceptions (some are norms)  Which family member decides if a child should be diagnosed by a volunteer?  Which family member decides whether a child should be enrolled in CMAM?  Information about male knowledge of nutrition and where they receive this information.  Information on school curricula focused on nutrition for school age boys and girls. Social Norms c. Practices, Roles and participation  Women are seen as responsible for babies (to what age? 2?)  What are the community practices around breastfeeding?  At what age do men focus their participation in taking care of  Research on gender roles and social norms.  Review of Third Interhousehold Survey.  Information from FGDs, KIIS, and healthcare provider surveys. Social norms CTCIM Final Evaluation - Draft Final Report 66 Table 16: Gender Relations, Constraints, and Opportunities What are key gender relations related to each domain that affect males and females participation in CMAM? What other information about gender relations is needed? What are the gender-based constraints hindering achievement of project objectives? What are the gender-based opportunities for reaching project objectives? children?  Are female and male CMAM community volunteers considered equally credible by community members?  Do men feel less welcome at health facilities than women? Does the facility have separate toilets for males and females? d. Legal rights and status Who actually makes what types of decisions in the household and under what circumstances? Is any form of consent needed for enrollment in CMAM? Has the 2013 Gender Equality Act been promulgated? Information on Personal status and family laws. Research on intrahousehold decision￾making. Status of Gender Equality Act roll-out. Role of GBV? e. Power Do women have the right to form their own opinions? Do women have freedom of movement outside the household? Can women and men who are not in the same family unit speak with each other freely? Information from focus groups about decisonmaking on child health and food distribution. Research on gender roles and social norms. Role of GBV? CTCIM Final Evaluation - Draft Final Report 67 Table 17: Gender Analysis Evaluation Questions Gender Analysis Questions in CTCIM Evaluation Research Instruments Instrument Gender-related question Questionnaire for health care provider Section A Q’s 1-3; Q’s 11, 12, 23, 24, 25, 26, 30 Facility Data Collection Q’s 13, 14, 16, 17, 18 Key Informant Interview CMAM Q’s 18, 19 Key Informant Interview Community Leader Q’s 11, 12, 14, 15 Focus Group Q’s All, as FGDs were separate for both sexes: male and female Email survey Q 21 QUESTIONNAIRE FOR HEALTH CARE PROVIDER A. Background characteristics 1. Sex: Male------1, Female---------2. 2. Age as at last birthday:_______________ 3. Marital status: Single: ------1, Married: -----2, Separated: -----3, Divorced: ------4, Widowed: -------5 11.Do families have problems with the nutrition of their children in this community? Yes------1, No--------2. 12.What months of the year do you see the most malnourished children? 23.Do you see a difference in the frequency of malnutrition between girls and boys? 24.If yes why do you think this is so? 25.How do caregivers (parents) react if their children are found out to be malnourished? 26.Do female caregivers react differently than male caregivers? Can you describe any differences? 30.Does a mother need to consult a child’s father if a child needs treatment for malnutrition? Does she need to consult anyone else in the family ? If so who ? FACILITY DATA COLLECTION 13.Are there separate toilets for male and female patients? Yes…..1; No…..2 14. If yes, are they in working order and clean? (Take photo) Yes…..1; No…..2 16. Do you have a routine way to identify every malnourished child? Yes…..1 ; No…..2; Comment 17. Does this routine differ for male and female malnourished children? 18. If yes, how? KEY INFORMANT INTERVIEW CMAM 17. Have gender differences been considered in how CMAM is implemented in your district? a. What conclusions can be drawn from monitoring data regarding the uptake of CMAM services by sex in your district? b. Why do men use CMAM for their children? c. Why do some men not use CMAM services for their children? d. What so you think would be the best way to encourage men to use CMAM services or to work as volunteers? e. What parts of connecting with communities could be improved to encourage men to use CMAM when their children are sick? CTCIM Final Evaluation - Draft Final Report 68 f. Are male and female CMAM volunteers regarded with the same level of respect at your district? g. Is CMAM addressing the differences in males and females eating and food distribution practices? h. In your district, do you know of domestic violence occurring if a woman does not give her husband the best food and instead uses it for her children? 19. Which family member decides whether a child should be enrolled in CMAM? e. Do men prefer working with a male or female CMAM volunteer? f. Do women prefer working with a male or female CMAM volunteer? g. Is there a difference in the way you see men and women behave when they bring a child to receive CMAM care? h. Based CMAM data, we notice that more girls are treated than boys. Why do you think this is? KEY INFORMANT INTERVIEW COMMUNITY LEADER 11. What are some of the reasons people take their children to see a health worker? a. Is there a difference between men and women? 12. Where do mothers take their children for HIV testing? How do they get there? 14. Which family member decides whether a child should be enrolled in CMAM? a. Do men prefer working with a male or female CMAM volunteer? b. Do women prefer working with a male or female CMAM volunteer? c. Is there a difference in the way you see men and women behave when they bring a child to receive CMAM care? d. Based CMAM data, we notice that more girls are treated than boys. Why do you think this is? 15. Do you see males and females who are HIV positive using CMAM services? Are they treated differently by health workers? EMAIL SURVEY What do you think the main impact of training men in nutrition would be on children's nutrition? Answer Options Response Percent Response Count No impact on reducing malnutrition of children. 0.0% 0 Would increase the amount of protein fed to children 33.3% 3 Would increase the number of meals fed to children 55.6% 5 Do not know 11.1% 1 Other (please specify) 5 answered question 9 skipped question 12 Table 18: Impact of Training Men on Nutrition on Children’s Nutrition CTCIM Final Evaluation - Draft Final Report 69 ANNEX VI: KEY INFORMANTS INTERVIEWED Mairead Peterson, Country Director, Concern Worldwide Megan Christensen, Health Officer, Concern Worldwide Mpumulo Jawati, Hunger & Nutrition Advisor, Irish Aid Reuben Ligowe, Save the Children Trust Mlambo, World Food Program Happy Botha, Commercial Manager, Valid Nutrition Dalitso Odalla, Finance & Admin Analyst, Valid Nutrition Dr. Mary Shawa, Principal Secretary, Ministry of Gender, Children & Social Welfare, Community Development Margaret Lwanda, Deputy Director, Nutrition, HIV/AIDS, Ministry of Agriculture Francis Mlewa, Livestock Director, Ministry of Agriculture Frieda Lkayuni, Gender Officer, Ministry of Agriculture Victor Maulidi, Deputy Cordinator, NGO Gender Network Marcel Chisi, National Chair of Men for Gender Equality Now (Megan Network) Dalitso Dembo, Chief Nutritionist, Ministry of Health Chris Moyo, Chief HMIS Officer, Ministry of Health Patrick Naphini, HMIS Officer, Ministry of Health Bensen Kazenbe, Nutrition Officer, UNICEF Emma Chinzukira, Nutritionist, World Food Program Stacia Norton, Nutrition Advisor, Food and Agriculture Organization, Alice, Concern Worldwide Catherine Mkangama, Senior Nutrition Advisor, SSDI Janet Guta, Deputy Director- Nutrition HIV and AIDS, Office of the President's Cabinet Mphatso Mapemba, CMAM Advisor, UNICEF Dr. Agnes Mwangwela, Senior Lecturer, Bunda College of Agriculture, University of Malawi Ides Chilinda, Deputy Head of Community and Mental Health Department, Kamuzu College of Nursing Brooke Mancuso, U.S. Peace Corps Volunteer. Ntcheu District Emily DeLacey U.S. Peace Corps Volunteer, Mfera Health Center, Chikhwawa District Rose Kumwenda, Executive Director, Christian Health Association of Malawi CTCIM Final Evaluation - Draft Final Report 70 ANNEX VII: ADDITIONAL ANALYSIS STAKEHOLDER PROJECTS, CAPACITY BUILDING, AND NUTRITION The United Nations World Food Program started the country program in Malawi in 2012 and will be in operation until 2016. The program aims to improve primary education outcomes, improve food insecurity to communities living in disaster prone areas, and to reduce malnutrition in vulnerable groups. WFP overseas the Supplemental Food Program which provides food supplements to children, pregnant and lactating women, and patients suffering from chronic illness and their families (HIV/AIDS and TB). WFP is a major partner in the CMAM program. The organization works to improve government capacity while providing food assistance. It also runs the Famine Early Warning System Network (FEWSNET) which assesses the rural food security in the country. The Scaling-up Nutrition (SUN) program is a global movement which was launched in 2000. It has gained momentum internationally and has received backing from large global stakeholders (IMF and World Bank). Over 30 countries are now participating in the SUN movement, including Malawi which joined in 2009. The SUN project is being implemented in 7 districts in Malawi and is a prevention program focused on community-based action to improve nutrition of children under 2 and pregnant mothers. The program focusing on the 1,000 special days (from conception to the first 2 years of life) with the goal of reducing stunting rates to under 20% through encouraging behavior change and increasing community awareness. The SUN movement has been successful in Malawi, and is currently expanding. The program hopes to create a learning forum (much like the CMAM learning forum supported through CAS) to bring together district health officers to share best practices. The Food in Agriculture Organization works closely with the Government of Malawi to combat food insecurities and improve food and nutrition throughout the country. FAO supports the government on various projects that range from improving irrigation, increasing the raising of livestock, promoting crop diversification and providing policy support in the context of food prices. The Improving Food Security Nutrition Project works to improve agriculture and nutrition at the community and household level. It provides support and encourages families to raise livestock, promotes crop diversity and the use of seasonal crop calendars and encourages planting diverse gardens using local plants. Training is conducted by district level officers who jointly train lead farmers and health volunteers to conduct weekly community awareness sessions on improved agriculture practices, nutritional components and the 6 food groups. The University of Malawi's Bunda College for Agriculture and Nutrition produces the most nutritionist in the country. The College awards a BSC in Nutrition and Food Science and incorporates the acute management of malnutrition into their curriculum. Students are taught about screening for malnutrition, the procedures for the nutritional rehabilitation centers, outpatient clinics, and the supplemental food program. They also have a practicum which includes using anthropometric equipment and visiting the NRU. Local, District, and National Capacity Building Elements CAS has built the capacity of health care providers by providing technical training for trainers at the zonal level. While funding is a challenge at the local level, trainers at the national and zonal level have the capacity to train local staff (when funding is made available through donors) to train health care providers on CMAM policies and protocols, and nutrition in general. When conducting field interviews it was evident staff that had attended trainings valued and appreciated the training. However, due to high staff turnover and lack of funding many people cited the need for trainings to improve knowledge and competence on CMAM and nutrition. "There is a real lack of knowledge among HSAs about nutrition", stated one Senior HSA at Chilmoni Health Center in Blantyre during the evaluation team's interview. CTCIM Final Evaluation - Draft Final Report 71 Box. 11 MoH Job Descriptions which include nutrition duties: Senior Disease Control & Surveillance Assistant: - In collaboration with interested sectors recognize and report treatment of malnutrition and promote community efforts to assure sufficient food intake. Senior Enrolled Community Nurse (STO): - Organizing and conducting Nutrition Programme activities. - Providing Anti-natal, Under-five, post-natal, child spacing and nutrition clinics. Public Health Nurse: - Organizing and conducting Nutrition Programme activities. She identified the need to train HSAs in nutrition. Training was one of the top recommendations HSAs made for improvement and it was obvious they were very keen to attend trainings and expand their technical knowledge. A majority of the health workers interviewed said they had job descriptions which is a component in building capacity. While over half said that CMAM was included in their job duties, we were unable to get a copy of any HSA job description. We did get a copy of the MOH job descriptions which includes some various nursing positions. Nutrition is included in the job descriptions of the senior disease control and surveillance assistant, the senior enrolled community nurse and the public health nurse (see Box 11). Adequate training on nutrition is needed to fulfill these job duties. It was evident through the information gathered during the focus group discussions and the community leader interviews that community members receive most of their information pertaining to nutrition from HSAs and nurses at the local health facilities. Refresher training is needed to effectively bridge the knowledge gap previously mentioned. Of the health care workers who have attended trainings, the majority of the training focused on the new admission criteria and the CMAM guideless. While HSA training manuals do discuss community outreach, more emphasis and training needs to be devoted toward teaching front-line health workers how to present nutrition information in simple and easy to use terms that community members can understand. In addition, there is confusion among community members about the 6 food groups and the appropriate foods to feed children. For example, one focus group discussion of fathers (5/6 were young in their early 20s) generated the following list of first foods they feed their baby after breast milk: porridge, Fanta, and diluted super deep drink. While mothers were noted as the primary caregiver responsible for nutrition in the household, fathers need to be educated and knowledgeable about appropriate foods and feeding practices too. Although community mobilization (including awareness and education) is one of the major components of CMAM it has not received a proportionate amount of attention or funding. The bulk of attention has been devoted to the treatment side For example, of the proposed activities outlined in the district plans, only a small number of activities are devoted toward community outreach. Community outreach is an area that should be scaled up to both promote better eating practices and identify cases of malnutrition. One method that has been used to increase community mobilization and identification of malnourished children has been to utilize health volunteers in the community. 53% of health providers interviewed said they supervised health volunteers. However volunteer attrition and inactivity was cited as a problem however evidence suggests when given non-finical incentives (such as trainings) volunteers remain motivated and attrition rates decrease. Trickle down trainings have proved to be expensive and the messages get diluted. Instead, district level trainings should conduct trainings for both the HSA and volunteers together (recommendation from Stacia Nortin). Also, engaging volunteers by connecting Figure 1: Topics covered in nutrition trainings CTCIM Final Evaluation - Draft Final Report 72 them to a care group, school feeding program, or Community Based Childhood Center would be a method of increasing support and structure for volunteers and may help to reduce attrition. One of the major challenges in providing adequate care for the CMAM program was availability of supplies, including transportation challenges in delivering CMAM supplies. RUTF was identified as having been out of stock in the previous month by 13 of the 20 health providers interviewed. One success that has been noted in many key informant interviews with managers at the district level was the use of Learning Forums to bring together district health managers, district nutritionists, CMAM coordinators, and other staff. The learning forums provided an opportunity to regularly discuss relevant topics, share success stories and best practices, review challenges and make unified decisions. The learning forms ranged in topics from Monitoring& Evaluating, Incorporating HIV services, and community mobilization. Under the guidance of CAS the learning forums ran from July 2006 and continued through November 2011. Many interview respondents shared their hope that the Learning Forums would continue after the phase out of CAS. The CTC Operational Plan (signed 2009) outline the inclusion of CTC/CMAM activities into the DIP of the majority of the districts implementing CMAM. "CTC activities including transport and management of RUTF, training, reporting and limited supervision are included in DIPs in the majority of districts implementing CTC. Inclusion of CMAM into DIP." CAS provided on the job training to districts for the implementation of district plans and implementing CMAM in the DIPs. For example, during a Learning Forum in February 2007, participants were trained on the steps of developing DIPs and the inclusion of CMAM. "Inclusion of CTC costs into DIPs increases district ownership of the program, and makes CTC services sustainable over the long-term. However, there are significant challenges to using district funds for CTC, including: national funding ceilings, availability of NGO funds, competing priorities at the district level, and the cost of RUTF." Learning forum #4, February 2007. Currently, all 29 health districts have included CTC services in their 2013 DIPs and have budgeted for at least 25% of CTC costs (indicator table through 2013). Although funding is a major challenge for districts, the inclusion of Figure 2: Challenges in Delivering CMAM Services at Facilities CTCIM Final Evaluation - Draft Final Report 73 CMAM into DIPs and the annual budget is a success. It is a step in districts take ownership of the program. One capacity building approach that has been critical to the success of the project is the inclusion of a CMAM focal person. As noted in the CWW CMAM evaluation report 2013, "CAS advocated for a CMAM Desk Officer at the national level and CMAM focal point persons at the district level. These position are credited for the successful, effective, and well-coordinated CMAM scale-up and implementation." However, there are many positions that remain open at the district and national levels and strong advocacy may be needed. In the CWW CAS evaluation report it states that "CAS supported the recruitment and training of National CMAM trainers." By supporting the recruitment and training for the national trainers, CAS has helped to build the capacity of the training system and has and will continue to build the knowledge of the local health workers. However, the lack of funding for trainings is a major problem. National trainers have the skill-sets and human resource capacity to train those down the ladder, but they lack the funding to mobilize these health providers. The local production of RUTF is a major success in Malawi and many countries are looking to Malawi as a model in local RUTF production (Key informant interview with Happy Botha). Prior to the production of RUTF locally, the program had to import RUTF internationally which really kept the price of CMAM high. With the creation of Valid Nutrition, RUTF could be produced at a much lower cost. While this has helped to decrease the cost, RUTF is still very expensive for the MOH. Further training should be provided to train districts on supply chain delivery in an effort to make the procurement and delivery more efficient. Also, taking out the middle man and shipping supplies straight from the distributor to the districts (rather than being stored in a warehouse) could short the time frame and reduce logistical challenges. Nutrition Related Topics Nutrition Both the WHO and UNICEF recommend exclusive breastfeeding (feeding the baby only breast milk) through age 6 months. Statistics from the 2010 DHS survey show national exclusive breast feeding rates at 71% for children 6 month and younger. However, the DHS statistics also show a 17.4% rate of stunting and a 7% rate of wasting in children under 6 months old. Research shows breastfeeding is the best nutrition for a young baby under 6 months and that breast milk has all the nutrients needed to keep the child healthy (SUN 1,000 days report). Therefore, these rates indicate either a gap in exclusive breastfeeding or the presence of chronic illness. As the exclusive breastfeeding rates and the HIV prevalence rates (GET) show, both these factors may play a role in stunting and wasting of young children. Field research and focus group discussions conducted by the evaluation team also indicated that exclusive breast feeding practices did not always occur. Knowledge of caregivers seemed to be low regarding giving children additional liquids in addition to breast milk (field research). This is a dangerous practice as it places the baby at unnecessary risk of illness, such as diarrhea disease and respiratory infections (SUN 1,000 days report). More education needs to take place at the community level to combat incorrect knowledge and unsafe practices. Breastfeeding in the context of HIV infection is also showed to be a confusing issue for mothers and fathers at the community level. There was a lack of knowledge and confusion about when HIV positive mothers should breast feed and for how long. This knowledge gap is further exasperated by the fact that the guidelines by the WHO have been revised in the last few years. It is now recommended that in developing countries (where replacement feeding is not practical) that HIV positive mothers should exclusively breastfeed for 6 months and give complimentary feeding for at least 12 months (WHO website guidelines). Children that have HIV are especially prone to malnutrition as their immune systems are weak and they are not as efficient at fighting disease. Many of the health workers at the CTCIM Final Evaluation - Draft Final Report 74 NRUs visits during the evaluation said that a disproportionate amount of children admitted to the NRU were HIV positive. Furthermore, children that have HIV are more likely to remain in the NRU for longer periods and have higher death rates (HCP surveys). Complementary Feeding The DHS data also showed that stunting rates in children increased through the period of complimentary feeding from 17.4% at 6 months to 61.3% at 23 month. This suggests that children are either not being breastfeed during this time period, not being fed enough food or not being feed appropriate and nutrition food. With the introduction of complimentary foods and liquids comes the risk of illness from food and water contamination, thus placing the children at a higher risk for illness and may contribute to malnutrition rates. Focus group discussion showed a lack of knowledge regarding appropriate complimentary foods. Some respondents said they feed their children items such as Fanta and Squash drink. A greater focus need to be on educating communities and households about the appropriate complimentary feeding practices. FAO is doing laudable work in the area of complimentary feeding by training health and agriculture extension workers to conduct village trainings and food demonstrations. This project works to improve caregivers knowledge and increase the nutrition content of complimentary feedings such as porridge by using local crops. Independent research efforts have shown favorable results (key Informant Interview with Stacia Nordin). Integration efforts of existing programs such as this should be made to target prevention and improve the capacity and reach of the CMAM program. Micronutrient deficiencies also contribute to morbidity rates. While there are many local foods that provide Vitamin A and Iron, the reliance on maize and lack of drop diversification contribute to micronutrient deficiencies. Patients and community members should be counseled and educated about local foods which contain important micronutrient such as Vitamin A and Iron and they should be encouraged to plant these types of crops. CMAM also provides Vitamin A and Iron supplements for people who do not get adequate nutrients in their diet. However stock outs were seen to be an issue, especially with Iron. This is problematic as Iron deficiencies affect more people globally than any other disease (WHO website). Iron deficient anemia (IDA) causes impaired physical and cognitive development and increases morbidity in children, and increase maternal risk of death globally (WHO). In fact anemia contributes to all maternal deaths. During the field visits we saw a number of facilities that did not have iron supplements available. The iron deficient anemia level among young children in Malawi is at 54.8% and 13% for pregnant women. These figures are troubling and efforts to both increase consumption of iron rich foods and improve the availability of iron supplements for children and women should be a priority. CTCIM Final Evaluation - Draft Final Report 75 ANNEX VIII: DOCUMENTS REVIEWED AND REFERENCES CMAM Program Documents CAS Implementation Schedule. Concern Worldwide. May 2009. CAS Performance Monitoring and Evaluation Plan. CMAM Capacity Assessment Report: DOWA District. December 2009 CMAM Capacity Assessment: Lilongwe District. CMAM Advisory Service. July 2010. CMAM Capacity Assessment Report: Scale Up to Six Health Facilities Kasungu District. CMAM Advisory Service. August 2011. CMAM Capacity Assessment Report: Scale up in Six Health Facilities Mchinji District. Mchinji District Health Office. November 2010. CMAM Health Facility Coverage. August 2012 CMAM Data Trends 2007-2012. CMAM Data Summary June-November 2012. CMAM Evaluation: Nepal Country Case Study. UNICEF. July 2012. CMAM Quality Assessment Report: Blantyre District Health Office. CMAM Advisory Service. October 2011. CMAM Quality Assessment Report: Mzimba South District Health Office. May 2011. CMAM Quality Assessment: OTP Therapeutic Programme. CMAM Advisory Service. November 2011. Community-Based Therapeutic Care Programme Capacity Assessment: Nsanje District. CMAM Advisory Services. July 2010. Cooperative Agreement No. 674-A-00-10-00011-00. Prepared by USAID: Southern Africa. Prepared for: Concern Worldwide. February 2010. Cost Effectiveness of Community-based Management of Severe Acute Malnutrition (CMAM) PowerPoint, Kate Golden, Concern Worldwide. Cost Effectiveness of Community Based Management of Acute Malnutrition in Malawi. Health Policy and Planning 2011; 1-11. Country Program Document for Malawi: 2012-2016. UNDP. CTC Coverage March 2013. CMAM Health Facility Coverage. August 2012 CTCIM Final Evaluation - Draft Final Report 76 CMAM Data Trends 2007-2012. CMAM Data Summary June-November 2012 CTC Coverage March 2013 CTC Operational Plan January 2009-December 2012: Integration of CTC into Ministry of Health Services in Malawi. Prepared by: Technical Working Group CTC Steering Committee. June 2009. CTC Lesson Learning Forum Minutes. UN Resource Centre. July 2006. CTC Learning Forum Report: Second Forum, Mangochi District. USAID and Government of Malawi. September 2006. CTC Learning Forum Report: 10th Forum, Lilongwe. USAID, Government of Malawi and Concern. November 2009. CTC Learning Forum Report: 11th Forum, Mulanje. USAID, Government of Malawi and Concern. April 2010. CTC Learning Forum Report: 12th Forum, Mzuzu. USAID, Government of Malawi and Concern. September 2010. CTC Learning Forum Report: 13th Forum, Lilongwe. USAID, Government of Malawi and Concern. December 2010. CMAM Learning Forum Report: 14th Forum, Machinga. USAID, Government of Malawi and Concern. March 2011. FANTA Training Module One: Overview CMAM. Guidelines for Community Management of Acute Malnutrition. Government of Malawi Ministry of Health. 2012. Health Surveillance Assistants Facilitator Training Manual. May 2009. Malawi CWW Cooperative Agreement 2010-2012. Malawi: CMAM Policy Environment. PowerPoint, Sylvester Kathumba. CMAM/SUN Conference. November 2011, Addis Ababa, Ethiopia. PD Extension 2009. Request for Task Order Proposals No. SOL-612-13-000005 Final Evaluation of the Institutionalization of Community-Based Therapeutic Care Activity in Malawi. USAID: Southern Africa. 2013. Scaling Up Community-Based Management of Acute Malnutrition: Implementing the CAS Program in Malawi. Concern Worldwide. Support to CMAM Institutionalization Project Handover Roadmap April-September 2012. CTCIM Final Evaluation - Draft Final Report 77 Creating an Enabling Policy Environment for Effective CMAM Implementation in Malawi. Sylvester Kathumba. Field Exchange Issue 43. July 2012. Support to CMAM Institutionalization Project Detailed Implementation Plan January-September 2012. Revised February 2012. Support to Community Therapeutic Care (CTC) Institutionalization in Malawi. CTC Advisory Service (CAS) Concern Worldwide. May 2009. Support to Community Therapeutic Care (CTC) Institutionalization in Malawi. CTC Advisory Service (CAS) Concern Worldwide. December 2008. Support to CTC Institutionalization in Malawi Program Proposal. Prepared by: Concern Worldwide. Submitted to: USAID Malawi. November 2007. Support to CTC Institutionalization Project Detailed Implementation Plan: February 2010-January 2011. CAS 2010 Annual Workplan prepared for: USAID. March 2010. Support to CTC Institutionalization Project: First Year Implementation Plan. December 2008. Support to CTC Institutionalization Schedule: May 2008 – April 2009. Revised October 2008. TORS for CTC Advisory Services Programme. Prepared by: Ministry of Health. Prepared for: Concern Worldwide. February 2006. Training Manual for CMAM Revised May 2012. Government of Malawi Ministry of Health. 2012. USAID Cooperative Agreement No. 674-A-00-08-00031-00. Prepared for Concern Worldwide. Prepared by: USAID: Southern Africa. May 2008. List of References 2008-2012 Performance Below Sphere Indicators. 2012 Global AIDS Response Progress Report: Malawi Country Report 2010 and 2011. Malawi Government. March 2012. A Life From Hunger: Tackling Child Malnutrition. Save the Children UK. February 2012. Assessment of the Capacity of the Health Sector to Respond to Violence Against Women and Children in Malawi. Lignet Chepuka Miriam Taegtmeyer, Esther Richards, Eleanor MacPherson, and Sally Theobald. 2011. Assessment of the Implementation by Malawi Government of the International Conference on Population and Development and other Relevant International and Regional Instruments. Barriers to Uptake of Prevention of Mother to Child Transmission Services in Rural Blantyre and Balaka Districts, Malawi. Juliet Yauka Nyasulu and Peter Nyasulu. Journal of Rural and Tropical Public health. April 2011. Vol. 10, p.48-52. CTCIM Final Evaluation - Draft Final Report 78 Baseline Nutrition Survey Ntchisi ADP-Malawi. Beatrice Mtimuni, Owen Nhkoma, Margsni Katunda, and Numeri Geresomo. Presented to: Swedish International Development Cooperation Agency and the Norwegian Ministry of Foreign Affairs. March 2010. Best Practices and Lessons Learning for Sustainable Community Nutrition Programming. Kinday Samba Ndure et al. Academy for Educational Development. August 1999. Decentralization in Malawi: An Overview. Prepared by Remy Prud’homme. Prepared for: The World Bank. 2010. DFID Malawi Gender Audit: Evaporated, Invisibilized or Resisted. Prepared by: Caroline Moser et al. Prepared for: DFID. October 2004. Economy Data Women Law and Business Malawi. 2012. Formative Research to Develop a Culturally-Appropriate Nutrition Intervention in Ntchisi, Malawi. WFP Debrief PowerPoint by: Stephen Kodish and Joel Gittlesohn, The Johns Hopkins Bloomberg School of Public Health. May 2013. Gender and Development Index 2011. Government of Malawi. Central Bureau of Statistics. June 2012. Gender Assessment USAID/Malawi. Prepared by: DevTech Systems. Prepared For: USAID. June 2008. Gender Equality and Female Empowerment. Scotland Malawi Partnership. 2011. Gender Equality Bill 2012. B.No.49. Parliament of Malawi. 2012. Gender-Inclusive Nutrition Activities in South Asia: Volume II: Lessons from Global Experiences. Rae Lesser Blumberg, Kara Dewhurst and Soham Sen. The World Bank. April 2013. Gender Inequalities in Rural Employment in Malawi: Policy Context. FAO. 2011. Global Regional Estimates of Violence Against Women. WHO. 2013. Increasing Agriculture’s Potential for Improved Nutrition and health in Malawi. Conference Report, September 2011. International Food Policy Research Institute. Malawi Malaria Indicator Survey 2012. National Malaria Control Programme and ICF International. 2012. Malawi Statement. Presented by: Anita Kalinde, MP. Presented to: 57th Session of the Commission on the Status of Women. March 2013. Malawi Strategic Gender Assessment. Naomi Ngwira, Garton Kamchedzera and Linda Semu. Present to: The World Bank and UNDP. June 2003. Malawi Vulnerability Assessment Committee. National Food Security Forecast. April 2013. Millennium Challenge Corporation. www.mcc.gov . National Nutrition Guidelines for Malawi. Malawi Ministry of Health, UNICEF and WHO. October 2007. CTCIM Final Evaluation - Draft Final Report 79 Nutrition Capacity Assessment in Malawi Report. FAO and Government of Malawi Department of Nutrition, HIV and AIDS. December 2009. Perfect Storm: HIV and Famine in Sub-Saharan Africa. PowerPoint by Deborah Bickel. Phenomenal Fathers: Stories from across Southern Africa. Gender Links. June 2013. http://www.genderlinks.org.za/article/celebrating-phenomenal-fathers-28-june-2013-2013-06-28. Reducing Undernutrition in Mozambique using Care Groups. Thomas P. Davis et al. Global Health: Science and Practice Journal. Vol 1. Num. 1. 2013. Republic of Malawi Multi-Sector Gender Profile. African Development Fund. October 2005. SADC Gender Protocol 2012 Barometer Malawi. Compiled by Emma Kaliya; eds. Ruth Ayisi and Loveness Jambaya Nyakujarah. 2012. SADC Protocol on Gender and Development. Signed Protocol. August 2008. Supportive Supervision/Mentoring and Monitoring for Community IYCF. UNICEF. September 2012. The Link Between Social Cultural Practices, Gender, GBV, HIV and AIDS: Baseline Study Report. Prepared by: Slaephera Consulting Ltd and Prime Health Consulting and Services. Prepared for: The Government of Malawi, EU and UNFPA. April 2013. The Meaning and Measurement of Acute Malnutrition in Emergencies. Helen Young and Susanne Jaspars. Humanitarian Practice Network. Number 56. November 2006. The Sphere Handbook: Humanitarian Charter and Minimum Standards in Humanitarian Response. The Sphere Project. 2011. UNICEF Global Nutrition Cluster – CMAM & Selective Feeding Programs. http://www.unicef.org/nutritioncluster/index_tools.html United Nations Development Assistance Framework Action Plan 2012-2016 – Malawi. UNFPA. 2012. http://profiles.unfpa.org/uploads/documents/1343387166-undaf-action-plan-17th-april-2012-with￾signatures.pdf . U.S. Agency for International Development 1300 Pennsylvania Avenue, NW Washington, DC 20523