Healthy Practices, Strong Communities Mid-term Evaluation Report September 2011 A Title II Multi-Year Assistance Program (MYAP) August 2008- July 2013 This assessment is made possible by the generous support of the American people through the United States Agency for International Development (USAID) under the terms of Cooperative Agreement No. FFP-A-00-08-00075-00. The contents are the responsibility of Mercy Corps and the Healthy Practices, Strong Communities (HPSC) Program and do not necessarily reflect the views of USAID or the United States Government 2 List of Acronyms and Abbreviations ANC Antenatal Care DIP Detailed Implementation Plan DM&E Design, Monitoring and Evaluation ENA for SMART Emergency Nutrition Assessment (Software) FANTA Food and Nutrition Technical Assistance FFP Food For Peace (USAID Office) FGD Focus Group Discussion HDDS Household Dietary Diversity Score HFPCR Horn Food Price Crisis Response Program HH Household H&N Health & Nutrition HPSC Healthy Practices Strong Communities IDP Internally displaced People IMC International Medical Corps IPTT Indicator Performance Tracking Table IR Intermediate Result IYCF Infant and Young Children Feeding Practices KII Key Informant Interview KPC Knowledge Practice and Coverage survey LC Local Council LRA Lords Resistance Army M&E Monitoring and Evaluation MAHFP Months of Adequate Household Food Provisioning MIHFP Months of Inadequate Household Food Provisioning MoH Ministry of Health MYAP Multi-Year Assistance Program OFDA Office for Foreign Disaster Assistance ORS Oral Rehydration Salts PIRS Performance Indicator Reference Sheets PMP Performance Monitoring Plan PPS Population Proportionate to Size PREP Pipeline and Resource Estimate Proposal SO Strategic Objective SOW Scope of Work UBOS Uganda Bureau of Statistics UMEMS Uganda Monitoring and Evaluation Management Services UNICEF United Nations International Children’s Education Fund USAID United States Agency for International Development VHTs Village Health Teams WASH Water, Sanitation & Hygiene WHO World Health Organization 3 Table of Contents Acknowledgement ........................................................................................................................................ 5 Executive Summary ...................................................................................................................................... 6 A. Background .............................................................................................................................................. 9 B. Methodology ......................................................................................................................................... 10 1. Evaluation team.................................................................................................................................. 10 2. Timing ................................................................................................................................................ 10 3. Data collection methods ..................................................................................................................... 11 4. Sample size and methodology ............................................................................................................ 12 5. Data analysis and dissemination ........................................................................................................ 14 6. Limitations ......................................................................................................................................... 14 C. Findings .................................................................................................................................................. 16 1. Improved Access to Food ................................................................................................................... 16 1.1 Key Activity Outputs ............................................................................................................... 16 1.2 Results ...................................................................................................................................... 17 1.2.1 Household Dietary Diversity Score ......................................................................... 17 1.2.2 Months of Adequate Food Provisioning .................................................................. 18 1.2.3 Access to Farmland, Crop Production, Value of Production and Yield .................. 19 1.2.4 Agricultural Market Access ..................................................................................... 20 1.2.5 Agricultural Technology Adoption .......................................................................... 21 1.2.6 Beneficiary Views: Most Significant Change Stories .............................................. 23 1.3 Effectiveness of Implementation ............................................................................................. 25 2. Improved Health and Nutrition ........................................................................................................... 26 2.1 Key Activity Outputs ............................................................................................................... 26 2.2 Results ...................................................................................................................................... 26 2.2.1 Anthropometric Scores and Nutritional Status ........................................................ 26 2.2.2 Prenatal Care ............................................................................................................ 31 2.2.3 Prevention of Malnutrition ....................................................................................... 32 2.2.4 Beneficiary Views: Most Significant Change Stories .............................................. 33 2.3 Effectiveness of Implementation ............................................................................................. 35 3. Increased Access to Water and Sanitation; Improved Hygiene .......................................................... 37 3.1 Key Activity Outputs ............................................................................................................... 37 3.2 Results ...................................................................................................................................... 37 3.2.1 Adoption of Healthy Hygiene Practices................................................................... 37 3.2.2 Water Sources and Use ............................................................................................ 38 3.2.3 Drinking Water Preparation ..................................................................................... 38 3.2.4 Latrine Usage ........................................................................................................... 38 3.3 Effectiveness of Implementation ............................................................................................. 39 4. Cross-cutting and Implementation Issues ........................................................................................... 40 4.1 Changing context ..................................................................................................................... 40 4.2 Coordination and collaboration ................................................................................................ 41 4.3 Funding process ....................................................................................................................... 41 4.4 Commodity delivery ................................................................................................................ 41 4.5 M&E ........................................................................................................................................ 41 4.6 Gender ...................................................................................................................................... 42 D. Recommendations .................................................................................................................................. 43 4 Annexes ...................................................................................................................................................... 47 Annex 1: MYAP Mid-Term Evaluation Scope of Work ............................................................... 47 Annex 2: Updated IPPT ................................................................................................................. 56 Annex 3: List of Villages Sampled ................................................................................................ 64 Annex 4: Focus Group Discussions and Schedule ......................................................................... 65 Annex 5: Key Informants Interviewed........................................................................................... 66 Annex 6: Household Survey Questionnaire ................................................................................... 67 Annex 7: Focus Group Discussion Guide ...................................................................................... 78 List of Tables Table 1: Overview of Data Collection Methods ......................................................................................... 11 Table 2: Population and Sample Size Distribution for Household Survey, Kitgum/Pader ......................... 13 Table 3: Improved Agricultural Technologies Adopted ............................................................................. 22 Table 4: Food Groups Consumed ............................................................................................................... 32 List of Charts Chart 1: Months of Inadequate Household Food Provisioning .................................................................. 19 Chart 2: Production According to Commodity ........................................................................................... 20 Chart 3: Price According to Commodity .................................................................................................... 20 Chart 4: Anthropometric Indices for Kitgum and Pader Districts .............................................................. 27 Chart 5: Anthropometric Indices for Kaabong District............................................................................... 28 Chart 6: Prevalence of Stunting (Children 6-59 Months) ........................................................................... 29 Chart 7: Prevalence of Underweight (Children 0-59 Months) .................................................................... 29 Chart 8: Prevalence of Wasting (Children 10-59 Months) ......................................................................... 29 Chart 9: Children with Diarrhea in Two Weeks Prior to Assessment ........................................................ 30 Chart 10: Treatments for Diarrhea Sought by Caregivers .......................................................................... 30 Chart 11: Change in IYCF Practices Utilized ............................................................................................ 32 Chart 12: Participants Demonstrating Proper Hygiene Practices ............................................................... 38 List of Figures Figure 1: Most Significant Change Stories for Women’s Gardening Groups ............................................ 23 Figure 2: Most Significant Change stories from Producer Groups ............................................................. 24 Figure 3: Most Significant Change Stories Illustrate Impacts for Tillage Beneficiaries ............................ 24 Figure 4: Most Significant Change Stories for Supplementary Feeding Groups ........................................ 33 Figure 5: Most Significant Change Stories for Mother Support Groups .................................................... 34 5 Acknowledgement We would like to extend our deepest gratitude to the individuals, households and communities of Kitgum, Pader and Kaabong Districts who gave their time and hospitality for this evaluation. Our appreciation also extends to the leaders in the three districts and ten sub-counties for supporting and collaborating with Mercy Corps for Healthy Practices, Strong Communities (HPSC) program implementation. The mid-term evaluation report is the culmination of many months of effort by many people. Special thanks and recognition go to the dedicated staff of Mercy Corps, particularly Kim Beevers for her significant contribution to this report. The leadership of Tim Sparkman and Henry Huka Duba demonstrated through their ideas, patience and enthusiasm throughout the entire evaluation process was significant. Moreover, the direction supplied by Mercy Corps’ M&E team led by Joe Dickman, James Kamukama, Lydia Gesa, Tim Joel and Andrew Tumuhameho was critical to the success of this assessment. We gratefully acknowledge all the Mercy Corps staff who came together in different ways to make the assessment a resounding success. Their efforts ranged from participation in training as data collectors, pre-testing of survey tools and data collection itself. It is a team effort that led to assessment and report success. Lastly, this assessment could not have been made possible without the generous support and direction of the United States Agency for International Development (USAID) and, in particular, the Office of Food For Peace (FFP). We recognize and kindly appreciate the generosity of the American people through USAID. Darius Radcliffe Country Director Mercy Corps Uganda 6 Executive Summary With support from the USAID office of Food for Peace (FFP), Mercy Corps Uganda is implementing the Healthy Practices, Strong Communities (HPSC) program in Acholi and Karamoja sub-regions of northern Uganda. The HPSC program is a response to the food insecurity and recovery needs resulting from many years of armed conflict in the Acholi sub￾region and prevalent violent conflict and adverse weather conditions in the neighboring Karamoja sub-region. The overall goal of HPSC is peaceful, healthy and food-secure communities for conflict and drought affected people in northern Uganda. Mercy Corps conducted an extensive mid-term assessment of the five-year HPSC program, found herein. This assessment is not an impact evaluation that examines causality, but guidance for readjustment and realignment of program activities. Accordingly, the mid-term evaluation is comprised of a mixture of qualitative inquiry and a smaller household survey. It is unlikely that the program achieved measurable change in population-based indicators at this point. Correspondingly, the mid-term evaluation aimed to examine more exploratory items and uncover why or why not approaches may be working rather than just if they are or are not working. At times, the qualitative and quantitative data are incongruous – overtly positive qualitative reviews combined with more tempered quantitative results. However, data incongruity may reveal something of the program itself: namely, the program and its activities have had a clear, high impact on beneficiary groups but seem to have struggled to influence the wider population in terms of some key behaviors and overall food security. One of the main purposes of the HPSC program is to support internally displaced persons as they transition from camps and resettle to permanent locations. At the start of the program, nearly half of the Kitgum and Pader inhabitants lived in camps. At the time of the mid-term, all camps had closed. The program did conduct activities to motivate and to support this transition – supplying seeds and tools, rehabilitating access roads, building WASH facilities closer to homes, for example. However, it is difficult to attribute resettlement or ease of resettlement to HPSC program incentives alone. Moreover, the mid-term cannot adequately examine how much HPSC influenced the broader economic base nor what challenges may have been averted due to HPSC activities. Rather, the mid-term does examine program activities within the broader context to make recommendations for improvement going forward. Results and recommendations from the evaluations will assist Mercy Corps to assess the progress of implementation and to document lessons learned. The assessment will also help Mercy Corps to formulate corrective action plans for implementation in order to maximize its impact for the remainder of the program. The program has three strategic objectives: improved access to food, improved health and nutrition for pregnant and lactating women and children under five, and increased access to water and sanitation and improved hygiene practices. Key findings under the first strategic objective, improved access to food, are as follows: • Overall food security in the region remains precarious as shown by persistently low population-wide levels of dietary diversity and adequate food provisioning. Direct program beneficiaries, however, discuss significant improvements to dietary diversity and food production as a result of their program inclusion. 7 • Smallholder farmers and vulnerable women are increasing crop production even though adoption of improved agricultural technologies is moderate. Highly fertile land and dramatically improved crop yields have dissuaded adoption of improved technologies. • Access to markets through improved infrastructure has increased. More support is needed for improved access to larger, regional and national output markets. • Trainings conducted by the program are believed to be of high quality. Follow-up trainings, however, are necessary for improved practices and understanding. • Large numbers of trainees inhibit effectiveness of Agricultural Extension Officers as well as use of demonstration plots as a training tool. • Proper use of food as an incentive is paramount to effective use of resources, time management and long-term development objectives. Key findings under the second strategic objective, improved health and nutrition, are as follows: • Population-wide anthropometric scores have not significantly improved since the baseline. Direct beneficiary most significant change stories point to positive behavior change and learning surrounding health and nutrition. • Incidences of diarrhea are notably reduced, moving from 70.0% at baseline to 38.2% at mid-term. Caregivers now also treat diarrhea more readily. Oral Rehydration Salt treatment is the most frequent treatment sought by caregivers treating diarrhea with almost a third of caregivers selecting this treatment. • Almost all women attend one ANC visit; few attend all four recommended visits. There is opportunity for the program to enhance understanding of ANC visit importance and to creatively incentivize women to attend all four visits. • Notable positive change has occurred in the number of children receiving appropriate Infant and Young Children Feed Practices. At baseline, 11.0% of children were fed according to IYCF; at mid-term, 38.0% of children were fed according to the same practices. • The program has an opportunity to improve careful and intelligent use of food and other incentives. As HPSC transitions from work within a relief to development setting, free food and other tools no longer support transition but hinder it. Program realignment toward systemic activities is now appropriate. • Training quality is high and appreciated by beneficiaries. Additional follow-up training is needed for adequate behavioral change and greater population-wide impact. • Distance to health facilities and knowledgeable health staff is a problem. Examining opportunities to support information and service extension using a system approach is appropriate. Key findings under the third strategic objective, improved access to water, sanitation and hygiene, are as follows: • There is a noted improvement in the number of program participants demonstrating proper hygiene practices. This improvement extends beyond direct program beneficiaries to the wider population as a whole. • Nearly all surveyed derive water from boreholes or covered wells. No households use unprotected water sources for drinking water. 8 • Fewer households now use VIP/pit latrines than at baseline. Access to these facilities likely decreased as a result of resettlement and will take time to build up once more. • Capital-intensive water infrastructure laid by many NGOs throughout the last decade is often non-functional. Improving water committee performance as well as access to qualified mechanics and spare equipment parts may result in fewer broken boreholes as well as more quickly and adequately repaired boreholes. HPSC began program implementation in a relief setting and served a valuable role to encourage IDPs to resettle while supporting the reemergence of productive agriculture to meet food security needs. The context has changed; currently, longer-term, systemic development programming is required. The mid-term evaluation and recommendations for future action come at an apt time. HPSC must reexamine its approach and realign its activities to effectively and intelligently impact area development. Key recommendations for future action are as follows: • Reduce the number of recipients trained but expand upon the amount of and follow-on training that each trainee receives. • Ensure incentives offered promote long-term development objectives through systemic interventions. Careful use of food and other handouts is requisite for sustained, long-term impact from the program. • The program should examine opportunities for linkages with buyers in more lucrative, higher volume markets. These opportunities may involve supporting alternative crops, livestock and poultry opportunities or other prospects. • Strengthen male inclusion in activities under all strategic objectives. Currently, women are the focus of health and nutrition activities. However, most decisions continue to be made by men and their program inclusion is a necessary precursor to wider behavior change. • Use of retail agents, be they VHTs or private operators, to expand the reach of health service and product offerings is an opportunity to better meet the health needs of communities in the remote areas in which the program works. • Improve water committee functionality as well as private sector ability to meet water and hygiene service needs. • Improve staff retention by examining opportunities for more intelligent use of program tools and selection of activities and participants to diminish workloads while contributing to better staff engagement. • Capitalize on opportunities for systemic, market-driven programming to augment sustainability of interventions wherever possible. 9 A. Background With support from USAID/FFP, Mercy Corps Uganda is implementing the HPSC program in Acholi and Karamoja sub-regions of northern Uganda. HPSC is a response to the food insecurity and recovery needs resulting from many years of armed conflict in the Acholi sub-region and prevalent violent conflict and adverse weather conditions in neighboring Kaabong district. In Acholiland, with the armed conflict over, Internally Displaced Persons (IDPs) had resettled in areas characterized by overgrown farmland and dilapidated feeder roads, dried or silted water points and little market opportunity, associated with years of neglect. In Karamoja, conflict associated with cattle raiding and dramatic shifts in weather patterns continue to beleaguer the population. Conditions in both Acholiland and Karamoja continue to exacerbate food insecurity and to encumber economic growth. Mercy Corps’ program goal is in line with the FFP objective of reducing food insecurity among vulnerable populations as well as the USAID economic growth program objective. The overall goal of HPSC is peaceful, healthy and food-secure communities for conflict and drought affected people in northern Uganda, including the following objectives: 1. Improved food production, consumption and sales among smallholder farming households. The program will promote home gardens, increase access to inputs, capital and extension services, and improve market infrastructure. 2. Improved health and nutrition among pregnant/ lactating women and children under five. The program will improve the quality of maternal and child health and nutrition care delivered by district services, health seeking behavior, child growth and development and dietary practices. 3. Sustainable safe water access and improved water, sanitation and hygiene practices. The program will work with communities towards year-round access to safe water, improved sanitation facilities and adoption of healthy hygiene practices. The program started in six sub-counties of Acholiland (Kitgum and Pader districts) in August of 2008. A year after inception, the program expanded its geographical coverage to two sub￾counties in Karamoja (Kaabong district) as well as one additional sub-county in each of Kitgum and Pader districts. Further, in October of 2011, the program again expanded its geographical coverage to include Kaabong sub-county and Kaabong town council in Kaabong district, Kacheri sub-county, Kotido sub-county and Kotido town council in Kotido district, and Morulem sub￾county in Abim District. The additional sub-counties were formerly included in the Mercy Corps Horn Food Price Crisis Response (HFPCR) program that was funded by USAID/OFDA. Mercy Corps began HPSC implementation in partnership with International Medical Corps (IMC). In August 2010, Mercy Corps assumed responsibility for IMC programming. Moreover, in July 2010, Mercy Corps concluded a year-long partnership with sub-grantee Medair who 10 worked in close coordination and collaboration with Mercy Corps to enhance the quality of services offered at health facilities and within targeted sites in Kaabong District. All program activities in Acholiland are slated to last for five years, ending in July 2013; those in Karamoja will last for three years, concluding in July 2012. In November-December 2008, Mercy Corps conducted a baseline evaluation in Kitgum and Pader districts to determine the level of knowledge and practices that were associated with key outcome and impact indicators assigned to the program. A year later, a similar assessment was conducted for the Kaabong district program extension. Both studies helped to refine indicators and to create benchmarks against which to gauge the program’s progress and to assess its impact. In addition to Mercy Corps’ commitment to learn from program implementation, it is also a FFP requirement that the five-year HPSC program be evaluated mid-implementation. Accordingly, Mercy Corps conducted an extensive mid-term assessment of the five-year program in Kitgum and Pader as well as a qualitative assessment of HPSC’s three-year component in Kaabong. Results and recommendations from the evaluations will assist Mercy Corps to assess the progress of implementation and to document lessons learned. The assessment will also help Mercy Corps to formulate corrective action plans for implementation in order to maximize its impact for the remainder of the program. A discussion of the mid-term and its findings and analysis follow. B. Methodology 1. Evaluation team This was an internally-led evaluation that included a high level of participation from Mercy Corps’ global Deputy Director for DM&E, in order to enhance objectivity and bring in added expertise. The Deputy Director for DM&E was involved in the early design stages of the evaluation, including input into the Scope of Work, sampling plan and data collection tools; assisted in training of staff and enumerators and initial oversight of field data collection; and participated in data analysis and presentation of findings. Coordination and implementation of the evaluation at the country program-level was led by the Mercy Corps Uganda DM&E Coordinator and Deputy Country Director, along with a team of four M&E staff. Local enumerators were hired in each district for assistance with data collection. Mercy Corps worked in close partnership with Grameen Foundation AppLab, which assisted in providing mobile data collection tools and analysis technologies along with associated training. 2. Timing Data collection was carried out over a five-week period in August/September 2011. Enumerator and staff trainings were held immediately prior to field data collection and data was analyzed immediately thereafter. 11 In past years this period has been the harvest season. Due to delayed rains this year, however, the harvest season did not transpire until after August/September. Accordingly, this period was marked by scarcity of food and other resources. Notably, the baseline evaluation was conducted throughout November/December. In the Acholi sub-region, the period in which the baseline evaluation occurred was the period that immediately followed the most productive of the twice yearly harvests. It is important to weigh mid-term results accordingly and particularly baseline and mid-term result comparisons. For example, households likely recall higher rates of adequate food provisioning at the time of baseline than at the time of the mid-term as a direct result of the timing of the evaluation. 3. Data collection methods The evaluation encompassed a mix of quantitative and qualitative data collection techniques in order to gauge progress against objectives and indicators as compared to targets and baseline values, as well as to ascertain lessons learned and recommendations for future implementation. Both qualitative and quantitative data was collected from Kitgum and Pader. The program has operated in the Acholi sub-region for three years; data collected here may offer a more compelling picture of program effectiveness. Only a qualitative assessment was conducted in Kaabong. The program recently began to implement activities in Kaabong; minimal program impact is expected due to this short duration. However, the program did conduct a small, annual survey that allowed for the inclusion of data for some quantitative indicators within this report. A summary of key data collection methods are shown in the table below: Table 1: Overview of Data Collection Methods Method Objective/key content Target Audience and Selected Tools Sample Size Household Surveys (HH Surveys) To assess quantitative, outcome-level changes against targets and baseline values, particularly with regard to food security, agricultural practices, access to markets, health and nutrition, water and sanitation access, hygiene knowledge and behaviors. Households in the target intervention area with children under 5 (population￾based) 207 HHs in Kitgum & Pader Focus Group Discussions (FGDs) To better understand key implementation issues and lessons learned, as well as planned and unplanned qualitative impacts (positive and negative), for various activities and beneficiary groups. Producer groups, women’s gardening groups, supplementary feeding beneficiaries, mother care groups and tillage beneficiaries, using tailored FGD Guides. 10 total FGDs in Kitgum& Pader; 12 in Kaabong Key Informant Interviews (KIIs) To gather perceptions on program implementation to date, including impact, lessons learned and future recommendations, from key beneficiaries and stakeholders. Local authorities (District Officers, Local Councils, etc.), program staff, donor representatives, and key program beneficiaries, using tailored KII Guides. 13 KIIs in Kitgum& Pader; 10 in Karamoja; 1 with UMEMS representative Document Review To understand the evolution of program activities and assess performance against targets. From Quarterly Reports, Annual Results Reports, Various 12 Baseline Assessment Reports, Mini KPC Reports, IPTT, DIP, PMP, PIRS, annual Pipeline and Resource Estimate Proposals (PREP), LAYERS monitoring results letter and monitoring data and reports Fieldwork Review Session To gather experiences not captured in survey form and key to mid-term evaluation. Enumerators 1 session in Kitgum The questionnaire used for the household survey was adapted from the same questionnaire used in the baseline survey to allow for comparison of results over time. Data collection for these surveys took place using Android-based SmartPhones that utilized the Grameen AppLab mobile survey application and back-end online data collection system, enabling ease of administration and survey uploads to an online database in real time. SmartPhone use facilitated greater ease and speed of data collection. A storytelling technique drawing on Appreciative Inquiry principles and using Most Significant Change was used in focus groups to better understand the range of individual beneficiary experiences and impacts. This involved dividing participants into pairs and having them relate stories to one another regarding the most significant changes in their lives brought about by their participation in the project. The pairs then shared their stories in plenary and discussions took place on what factors or programming elements were necessary to better maximize the positive impacts mentioned and how to reduce bad or mixed experiences. Results of these discussions helped to highlight program impacts to date and to develop recommendations for the next phase. 4. Sample size and methodology The survey sample of 207 households in Kitgum and Pader was achieved by randomly selecting 7 households in 30 randomly selected clusters, or villages (three households had to be omitted due to data quality issues). Per the advice of the USAID regional advisor, an explicit choice was made to accept this reduced sample size and margin of error as compared to the baseline survey 13 sample of approximately 900 households. This is because population-level impacts in program indicators are likely difficult to achieve in the limited timeframe and the focus of the evaluation was more on implementation processes and lessons learned. The reduced survey sample allowed for relatively greater effort to be placed on qualitative techniques in order to analyze and improve upon implementation for the next phase. The weight of the quantitative results should be considered accordingly. The clusters (villages) were selected randomly from a list using Population Proportion to Size (PPS), using population data from the most recent census. Households within clusters were selected using the segmentation technique, whereby clusters (villages) were sub-divided into three roughly equally-populated segments using a map drawn by community leaders, and one segment was chosen at random (by blindly drawing one of three numbered pieces of crumpled paper from a staff or community representative’s hand). Households within that segment were then selected at random using a sampling interval based on the total number of households in the segment divided by seven (the target number for each cluster). The first household in a segment was chosen by going to the center of the segment, spinning a pen, and walking in that direction. A random number was selected from within the segment sampling interval, and the household corresponding to that number encountered in the walking direction was interviewed first. The approximate village map developed by the authorities to determine the segments was also used to define the route taken by enumerators within the segment in order to ensure the inclusion of all relevant households within the segment sampling frame. The overall sample size of 207 households in Kitgum and Pader is based on a confidence interval of 95% (default value), estimated prevalence of key indicators of 50% (the most conservative or worst-case scenario for estimating sample size), margin of error of 10%, and design effect of 2 (default for two-stage cluster sampling). Table 2: Population and Sample Size Distribution for Household Survey, Kitgum/Pader District Sub-county Population No. of Clusters (Villages) No. of Clusters Selected Sample size (HHs) Kitgum Kitgum Matidi 12,007 26 4 27 Kitgum Lagoro 7,616 23 3 21 Kitgum Omiya Anyima 9,726 26 4 27 Kitgum Namokora 5,614 13 2 13 Kitgum Orom 20,245 62 8 56 Pader Latanya 6,775 24 3 21 Pader Wol 6,572 31 3 21 Pader Pajule 7,531 29 3 21 Total 76,086 234 30 207 For a full list of villages sampled, please see Annex 3. Qualitative data was collected using Focus Group Discussions and Key Informant Interviews. Selected FGD and KII participants were either program beneficiaries or particularly 14 knowledgeable regarding program disciplines. A list of FGDs is shown in Annex 4. A list of key informants is shown in Annex 5. Mercy Corps program staff also provided key information and documents for review throughout the assessment process. Further, initial report findings were shared with Mercy Corps HPSC staff; their feedback is reflected in this report. 5. Data analysis and dissemination For quantitative data collected via the household survey, the evaluation team analyzed the data in STATA, conducting descriptive and bi-variate comparisons with appropriate tests of statistical significance. Many of the comparisons are conducted across gender, geographic and time divides. Baseline and mid-term values across similar attributes were compared and results were reported. Anthropometric data was analyzed using ENA for SMART in combination with STATA software. Qualitative information from FGDs and KIIs was also collated and synthesized to highlight key points and facilitate identification of key outcomes and lessons learned. 6. Limitations There were both internal and external constraints experienced in the evaluation, which imposed limitations on the completeness and potential accuracy of the evaluation results and comparability with baseline data. Where possible, Mercy Corps took steps to mitigate these. The village sampling procedure was based on outdated population data. The census was taken in 2002; population movement has been very fluid in the area since then, with many IDPs returning to their villages of origin or other villages after the war. However, the census data still seemed to be the most reliable and easily accessible data source on population, as village household lists kept by local authorities were also seen to be difficult to access, outdated and generally unreliable. Perhaps most significantly in terms of comparability of data from baseline to mid-term, 47% of sampled households were still living in IDP camps at the time of the baseline in 2008, whereas by the time of the mid-term evaluation in 2011 almost all had returned to their villages or settled in new villages. This would seem to particularly influence survey results that are highly dependent on the immediate living environment, such as access to land, food consumption and access and accessibility of water sources and sanitation facilities. Indeed, access to many of these items may have been better in the camps where they were institutionally supported. Importantly, HPSC program activities aim to support IDPs during transition from camps and resettlement sites to permanent homes. Accordingly, it is expected that activity results will represent the challenges reflected in implementation at this delicate and transitional time. Yet despite this significant change and influence on results, overall changes in these variables within the project intervention area is still accurate and of interest. 15 The seasonal timing of the mid-term evaluation as compared to the baseline is also likely to have significantly affected results. The baseline survey was carried out in December 2008. As December is post-harvest and generally a period of relative food security in the region, while August can be more precarious, this timing could have adverse affects on the ability to confidently compare changes in food security from baseline to mid-term. Particularly, food security indictors such as the Household Dietary Diversity Score could be biased downwards in the mid-term evaluation, despite potential overall improvements in recent years, due to the measurement taking place in the lean season rather than post-harvest. Another limitation was due to the fact that questions on attitudes and practices can be easily biased towards responses that the respondent views as correct, rather than those based on actual behavior. For example, this could be the case for some questions on hygiene or feeding practices. Mercy Corps took steps to minimize this by asking questions in a neutral manner as well as cross-verifying where possible by checking for appropriate items in the household (such as availability of soap, hand-washing mechanisms and sanitation facilities). Moreover, during the review session with enumerators, it was uncovered that many respondents who had received Mercy Corps support did not want to acknowledge that or even appreciate the support received irrespective of the neutrality that data collectors posed. This was attributed to the perception that Mercy Corps was screening for new beneficiaries to target for upcoming program activities; some respondents were not honest about the support that they received in hopes that they would receive additional support in the future. Likewise, data on agricultural production and yield can be difficult for farmers to estimate when they have not been keeping written records, as is usually the case. Therefore, this data can also be open to biases. The mid-term study took place during the rainy season. Some institutions and organizations halt work throughout this period as roads are at times impassable for days. Accessing respondents, then, was challenging for enumerators. Further, interview time was limited for households that maximize daily dry hours in the garden. Limited interview time as well as poor accessibility due to rain could have compromised random sampling as well as the overall quality of collected information. Finally, per the Scope of Work guidance (see Annex 1), Mercy Corps conducted the mid-term evaluation internally. Mercy Corps experienced M&E staffing issues in the midst of conducting the mid-term evaluation. Amongst the five person M&E team working on the evaluation, two staff members announced resignations due to new educational and job opportunities, one staff had to take leave to care for a sick child and another was jointly responsible for other concurrent M&E responsibilities. These depleted human resources somewhat compromised the team’s ability to conduct more extensive qualitative data collection following the survey, as well as more extensive data analysis and interpretation of results. Despite these difficulties, extra effort on the part of the evaluation team enabled Mercy Corps to adequately fulfill the requirements of the Scope of Work and to be confident in the findings and recommendations presented in this report. Despite these challenges, the evaluation objectives were met and the overall report presents an accurate picture of results as well as recommendations for future action. 16 C. Findings The following section first discusses socioeconomic characteristics and then presents evaluation findings according to each HPSC strategic objective and key corresponding intermediate results. For activities conducted in Kitgum and Pader Districts, both qualitative and quantitative data is presented. Generally, for activities conducted in Kaabong District, however, only qualitative data is discussed – per the mid-term scope of work requirements. The mid-term evaluation examined all sub-counties, eight in Acholiland and two in Kaabong, benefiting from programming under HPSC during August and September 2011. According to mid-term survey data, the average household size for the geographic region covered under the program is 6.7 members. Mothers or primary caregivers range in age from 17 to 47 years with the average age falling at 29.5 years. Of these, 60.9% attained some primary education; 34% had never gone to school. Nearly all of the households surveyed rely on income from subsistence farming. Specifically, 99.5% rely on income earned from crop farming, 48.8% rely on income earned from poultry farming and 39.1% rely on income earned from animal husbandry. Significantly, at the time of evaluation, all households surveyed had returned home or resettled. Of those households surveyed for the HPSC baseline, 47% were displaced and residing in camps. 1. Improved Access to Food 1.1 Key Activity Outputs Key outputs for each intermediate result under Strategic Objective Improved Access to Food are listed below: IR 1.1 Smallholder Farmers & Vulnerable Women Have Increased Food Production • 90 women’s gardening groups were trained on vegetable crop production as well as food preservation and preparation techniques to maximize nutritional value of food intake IR 1.11 Increased Access to Agricultural Inputs, Services and Markets • 12,342 households in Kitgum and Pader as well as 1,994 households in Kaabong received vouchers for tillage service or inputs at seed and tool fairs • 160 producer groups were trained on marketing and post harvest handling • The construction of 8 bulking centers is completed or in progress • Food for Work was used to construct 116 kilometers of feeder road in Kitgum and Pader and 14 kilometers of feeder road in Kaabong IR 1.12 Environmentally Sound Improved Agricultural Practices Adopted • Tree seeds were planted in 10 tree nursery sites. Seedlings were distributed in Kitgum, Pader and Kaabong Districts. 17 1.2 Results Key results to date under this objective follow. A full listing of results can be found in the IPTT in Annex 2. 1.2.1 Household Dietary Diversity Score Access to food is defined as the ability to acquire sufficient quantity and quality of food in order to meet all household members’ nutritional requirements for productive lives. The program employs two proxy measures to best gauge household access to food: Household Dietary Diversity Score (HDDS) and Months of Adequate Household Food Provisioning (MAHFP). HDDS determines the quality of diet, measuring the number of different food groups consumed over a 24-hour period as opposed to the amount of food consumed. In order to determine the number of individual food groups consumed at the household level, respondents were asked the type of food groups they ate the previous day in line with the guidelines provided by FANTA (FANTA 2004)1 . Responses revealed that food access and diversity as measured by HDDS was an average of 3.6 out of the 12 food groups in Kitgum and Pader, per the FANTA grouping. In Kaabong, HDDS averaged 3.3. HDDS results for both geographic areas fall short of the 5.0 food groups established as the targeted amount. Moreover, at baseline, Kitgum and Pader households consumed an average of 3.8 food groups. However, the baseline HDDS of 3.8 and the mid-term HDDS of 3.6 are not statistically different (p = 0.1009, df = 1092). Moreover, differences between mid-term beneficiary HDDS and non-beneficiary HDDS are not statistically significant (p = 0.7401, df = 205). Kaabong households consumed an average of 3.0 food groups. There are several possible reasons for below targeted HDDS. First, in Kitgum and Pader Districts, the population is newly resettled. Household capacity to grow sufficient amounts of nutritious food and to raise a breadth of animals, though increasing, is not yet adequate. The HPSC program is working with women’s gardening groups to increase the variety of vegetables and fruits grown in home gardens as well as to educate these groups on the importance of feeding families a nutritious and varied diet. The program is not working with livestock or poultry, however. Second, households are not only just beginning to cultivate adequate home food sources but the availability and affordability of nutritious food at the local level is minimal. As a result of country-wide food price increases, particularly for protein-rich food sources like beans, eggs, chicken and other meats, as well as fuel price increases, households in Acholi sub-region and Karamoja could not afford to purchase food as needed. Moreover, at the time of baseline nearly 1 FANTA (2004): Measuring Household Food Insecurity workshop report, Academy for Educational Development (AED) 18 half of all households resided in IDP camps where a sizable NGO population largely met food needs. Third, the program initially focused the majority of its agricultural training on five value chain crops. As prices and demand for alternative crops changed, households and producer groups chose to substitute food crops with cash crops. For example, in 2010 cotton received a high farm gate price. Accordingly, in 2011, a large percentage of famers chose to abandon traditional food stuff crops like maize and rice and to grow cotton instead. With the large influx of cotton supply, the prices for cotton plummeted in 2011. It is possible that HDDS and MAHFP may continue to fall as households have neither adequate food stuffs nor the resources to purchase adequate food stuffs as a result of their decisions to grow cotton this year and the poor price it received. There are, however, other factors that influence HDDS and MAHFP beyond this potential correlation. 1.2.2 Months of Adequate Food Provisioning Under objective one, Mercy Corps aims to improve agricultural production, sales and consumption among the farming communities in the program area. Adequate production provides households with adequate food supply throughout the year. In order to measure the efforts made towards increased production, availability of food was assessed. This was measured through the number of months a family had enough food from their own resources; it does not consider purchased, exchanged or bartered food. When asked if they had access to adequate food supply from their own source in the last 12 months, 95.6% of households reported that there were months in which they did not have access to sufficient food. At the time of HPSC baseline, 91.4% of respondents indicated that they did not have sufficient access. Respondents from Kitgum and Pader identified 8.9 months out of the preceding 12 months as months of adequate food provisioning. This is a statistically significant reduction in perceived food access (p = 0.0253, df = 1025). Similar food access challenges seem to have affected the population with direct beneficiaries reporting 9.1 months of adequate food provisioning and indirect beneficiaries reporting 8.7 months of adequate food provisioning (p = 0.1017, df = 195). In Kaabong, respondents identified 7.8 months of the same. MAHFP results for all districts fell below the established targets of 10 and 8 months respectively. May, June, July and August were identified as months without adequate food. This is akin to the baseline as highlighted in the following table: 19 Chart 1: Months of Inadequate Household Food Provisioning Poor MAHFP results may stem from a number of factors. As discussed above, resettlement has resulted in both significantly reduced food distribution as well as only recently established household gardens and livestock supplies. This transitional time has likely resulted in food availability hardship. Further, the timing of the mid-term evaluation is also significant. Baseline measurements were taken in the period following harvest season; mid-term measurements, however, were taken in August and September – just prior to harvest. Perceptions of food availability are likely skewed accordingly. Additionally, it is important to keep in mind the level of aid dependence cultivated throughout decades of intervention efforts. Acholi and Karamojong populations are cognizant that perceptions of heightened poverty and food insecurity have often resulted in additional handouts in the past. Therefore, there is a perceived incentive to present circumstances as more dire than they truly are. 1.2.3 Access to Farmland, Crop Production, Value of Production and Yield Increased food production amongst smallholder farmers and vulnerable women is a key component of the program’s objective to improve access to food. Activities completed under this objective aim to increase production, to expand access to inputs and end markets, and to increase use of environmentally sound agricultural technologies. The program’s initial value chain assessment identified the following five crops as ideal focal crops for intervention: sesame, maize, beans, rice and groundnuts. In order to boost value chain crop production amongst beneficiaries, the program conducted training in best agronomic practices, marketing and post-harvest handling for women’s gardening groups and producer groups that chose to cultivate these crops. In 2011, beneficiaries from women’s gardening groups, producer groups and tillage beneficiaries from Kitgum and Pader produced 17,743 MT of focal agricultural commodities. The targeted amount was 3,270 MT. In Kaabong, participating beneficiaries produced 6,506 MT of focal agricultural commodities. In this district, the targeted amount was 2,057 MT. In Kitgum and Pader Districts, the change in volume of targeted agricultural commodities produced from the 20 previous fiscal year is 12,498 MT. In Kaabong, the change in volume is 6,506 MT. In total, across all districts, the dollar value of targeted agricultural commodities produced by USAID￾assisted clients in FY 2011 was $15,615,751, calculated using 2008 USD value. Actual crop yields are well above targeted yield amounts. Though farmer beneficiaries are adopting improved seed varieties and better agricultural techniques as promoted by the program, key informant interviews point to some disappointment with improved seed varieties and better agricultural practices adoption is moderate. Further, the high actual to target crop yield ratio also reflects poor targeting. Targets were initially set using demonstration plot yields, not farmer field yields. This is a program error that has since been rectified. At the time of the mid-term, the program participants under Strategic Objective 1, Improved Access to Food, included 160 producer groups; each producer group consisted of 25 – 40 members, with an average of 30 members. Further, the program also worked with 90 women’s gardening groups; each of these groups consisted of 20 members. Further, 1,698 households participated in tillage services offered by the program. Harvests in Kitgum and Pader Districts adhere to a bi-modal seasonal calendar. In the first harvest of FY 2011, program participants in Kitgum and Pader Districts produced almost twice as much maize as any other crop. In the second harvest, however, the same participants produced nearly as much maize as sesame and beans. The following charts delineate production as well as price differentials for these harvests: Chart 2: Production According to Commodity Chart 3: Price According to Commodity 1.2.4 Agricultural Market Access The program has further targeted Strategic Objective 1, Improved Access to Food, by conducting activities aimed at increasing access to markets, inputs and services by opening community access roads. All respondents indicated that they can now access a market and they generally do so with much greater ease and less time than at the start of programming. It now takes an average of 89.9 21 minutes (1 hour, 30 minutes) for Kitgum and Pader households to reach the nearest market; at the time of the baseline study, it took an average of 142.2 minutes (2 hours, 20 minutes). It is not clear why it took 142.2 minutes for inhabitants of Kitgum and Pader to reach markets when nearly half of the population was still living in camps or transitional resettlement sites. Though, some camps and sites were remotely located. For respondents from Kaabong, it takes an average of 117.4 minutes (1 hour, 57 minutes) to reach the nearest market – just under the program target. The baseline study found that the lengthy time required for farmers to reach markets hindered access to essential inputs like seeds and tools. It also surmised that reducing this time would empower farmers to become sellers in the markets as well as buyers, thus deterring middlemen and improving the profit margins for farmers. Ultimately, the baseline deduced, this would in turn boost production, increase household income and reinforce a safety net for farming households. With this in mind, the program elected to support ongoing rehabilitation and construction of farm-to-market roads. In Kitgum and Pader, program road construction has continually exceeded targeted amounts. In year 1, 25 km of roads (139% of target) were completed. In year 2, 54 km (168.8% of target) were completed. And, in year 3, 116 km (116% of target) of roads were completed. Road rehabilitation and construction in excess of targeted amounts may be attributed to both particularly high demand for access roads from communities and authorities as well as high satisfaction with the amount and quality of program road rehabilitation and construction. In Kaabong, 15 km of roads (100% of target) were completed in year 2 and 14 km (46.7% of target) were completed in year 3. In Kaabong, the program works within two sub-counties; district officials here were able to identify small segments of road for rehabilitation or construction. A late start to FY2011 programming plus heavier-than-normal rains contributed to missing the Kaabong year two target. The mid-term study also investigated the common means of transport used and time for households to reach the nearest market. At the time of the baseline, 99% of all households indicated that they walk to the market. Now, 92.7% of respondents indicate that walking is their most frequent means of travelling to the market. Moreover, 1.2% of respondents reported that a bicycle was their most frequent means of travel to a market at the time of baseline; currently, 7.3% of respondents report that a bicycle is their primary source of travel to the market. Though these changes may seem incremental, they do seem to result in time saved. However, it is difficult to assign success for these changes to the HPSC program only. 1.2.5 Agricultural Technology Adoption The program promotes a number of environmentally sound practices such as tree planting, mulching, crop rotation, and soil and water conservation as proposed precursors to good farming approaches that will improve crop production and yield. The program disseminates these sound environmental practices through extension education and demonstration farms. 22 There are eight ideal or recommended environmentally sound agricultural practices promoted by the program (see table below). At baseline, 73.0% of households measured in Kitgum and Pader utilized three or more of these technologies. At mid-term, 57.1% of farmers from Kitgum and Pader adopted at least three of the recommended technologies. This is not a statistically significant difference (p=0.6030, df=1093). At mid-term, 23.3% of farmers from Kaabong had adopted at least three of the recommended technologies. Though all beneficiaries received some training on environmentally sound agriculture practices, women’s gardening group members, producer group members and tree nursery members received more intensive training coupled with demonstration plots. Table 3: Improved Agricultural Technologies Adopted The HPSC program promotes the same sound agricultural technologies as those promoted by other development partners and associated extension agencies. These practices are associated with crop choice; hence, some are more appropriate than others and selected practice adoption may reflect this. Further, returnees have very fertile land that has been fallow for years; they do not see a strong incentive to actively improve fertility. This reduces technology adoption in Kitgum/Pader as compared to Kaabong where conditions are more or less the same since program inception. However, opportunity to improve overall adoption may lie with a smaller farmer-to￾demonstration plot ratio in which additional practice for each farmer is possible. Currently, around 126 farmers practice on each plot. Further, reexamining farmer and group motivation for attending trainings would be useful. For example, large numbers of farmers attend early trainings with the expectation that they will receive some form of in-kind compensation. When they are not compensated, famer attendance tapers. Reexamining motivation for attendance and weeding out rent seekers in advance would be fruitful. So too would injecting opportunities for motivation over the longer term – like performance awards. All of this may help to augment improved agricultural technology adoption. 2 At baseline, nearly half of respondents resided in IDP camps or transitional resettlement sites; likely, high fallowing adoption was not necessarily a conscious practice but a result of land inaccessibility. Technology Percentage of Beneficiaries Adopting (Mid-term) Percentage of Beneficiaries Adopting (Baseline) Mixed/Intercropping 85.5% 72.1% Spacing 76.8% - Crop Rotation 36.7% 68.7% Mulching 13.0% 31.2% Tree Planting 8.7% 0.0% Compost Lines 4.3% 0.0% Fallowing 2.4% 50.6%2 Zero Tillage 0.5% 0.5% 23 Figure 1: Most Significant Change Stories for Women’s Gardening Groups Members of Women Gardening Groups were asked in focus groups to relate the most significant changes in their lives due to participation in the project. The following graph represents the frequency that the themes were mentioned in stories from four focus groups, followed by a few illustrative quotes. Increased knowledge: “We were using broadcasting method as the only method of planting crops, but now when new and modern techniques have been introduced, such colonial practices are no longer being use and I promise we are going to harvest much this time round.” Christine, Lomorunyangae, Kaabong Increased income: “I used the knowledge from the training and bought onions seeds. I planted and harvested one sack and sold it, and used the money for paying school fees for my child in secondary school.” Grace, Lajulu Owiny, Pader Improved health: “I am very healthy now, even having a very smooth skin because of variety of food I got from the group.” Madalena, Orabul, Kitgum Community unity: “Before, we did not have any groups such as this whereby each and everyone depended on her own. But when Mercy Corps introduced these groups, there is a spirit of togetherness and unity amidst us.” Lucia, Lomorunyangae, Kaabong 1.2.6 Beneficiary Views: Most Significant Change Stories The mid-term evaluation is comprised of a mixture of qualitative inquiry and a smaller household survey. The rationale was two-fold: first, it is unlikely that the program achieved measurable change in indicators that are captured in a population-based survey. Second, a smaller quantitative sample size availed resources for a more thorough qualitative review. At times, the qualitative and quantitative data are incongruous – overtly positive qualitative reviews combined with more tempered quantitative results. However, data incongruity may reveal something of the program itself: namely, the program and its activities have had a clear, high impact on beneficiary groups but seem to have struggled to influence the wider population in terms of some key behaviors and overall food security. Qualitative reviews point to tangible and positive program activity impact on direct beneficiaries. The following figures highlight Most Significant Change Stories taken from focus group discussions with Women’s Gardening Groups, Producer Groups and Tillage Beneficiaries. Most program beneficiaries discussed positive life and habit changes resulting from increased volume and diversification of production, learning and application of new knowledge and technologies and health and nutrition gains due to the increased access to food and dietary diversity – all results of their program inclusion. 24 Figure 3: Most Significant Change Stories Illustrate Impacts for Tillage Beneficiaries Dietary diversity: “I am a lame woman who can’t dig for myself, but Mercy Corps has helped me to overcome my problem. Now I am able to harvest varieties of food to balance my diet which was not the case before, where I used to depend on only one type of food.” Lucia, Napusimoru, Kaabong Income and savings: “I got good yields because I had three sacks of sorghum, so I sold some part of it and bought clothes for my children and other types of food in the home.” Edward, Obolokome, Kitgum Community cooperation: “Mercy corps through this project has promoted unity and cooperation among our people, by giving money to those who were helping dig our gardens which they could do in time and now we are going to harvest much this season.” Lokuda, Napusimoru, Kaabong Figure 2: Most Significant Change stories from Producer Groups Health & nutrition: “I used to plough seeds that could take long to mature, and as a result I and my family members could suffer from hunger and as a result of starvation I lost my first born child. But when Mercy Corps provided short term crops, such has never happened in my home.” Daniel, Lokodope, Kaabong. New agricultural methods: “Mercy Corps is really very good, because this program has brought modern techniques of planting crops unlike before, when we could broadcast crops of many varieties in one garden which at the end of the day one could harvest little due to poor farming techniques. I believe this coming harvest we are going to harvest much because of the use of modern techniques.” Phillip, Lobalangit Central, Kaabong Time & energy savings: “Before Mercy corps programs my wife used to move long distances in search of vegetables, but now it does not take her all that strength to move long because we now have access to a variety of vegetables in our demo plots which one can get cheaply.” Lobalangit Central, Kaabong This graphic illustrate the relative frequency of words used in 15 individual Most Significant Change Stories, as collected from the program tillage beneficiaries during focus groups. Below is an illustrative sample of direct quotes from these stories. Food security: “I used to struggle to dig my garden alone as an individual…now I have put a variety of seeds in it which will be able to produce enough food.” Teregina, Napusimoru, Kaabong. Members of Producer Groups were asked in focus groups to relate the Most Significant Changes in their lives due to participation in the project. The graph at right shows the number of times the themes were mentioned in stories from four focus groups, followed by a few illustrative quotes. Dietary diversity: “Before I joined the group, I did not know that some vegetables existed, for example eggplant and carrot, but now through these group activities we have access to this food.” Acholi, Lobalangit Central, Kaabong 25 1.3 Effectiveness of Implementation The following section outlines successes and challenges to program activity implementation under Strategic Objective 1. These recommendations largely coincide with the LAYERS monitoring system report produced in conjunction with USAID Uganda. The section also includes recommendations for future action. Successes • Quality training: Trainings are well-implemented by people with the necessary expertise. Most FGD participants viewed trainings as effective in transferring key knowledge; farmers applied knowledge to their own fields and passed it onto others. • Effective demonstration plot use: Plots used successfully for both learning and applying new techniques. Plots also served as a food source, supplying farmers with new types of food. • Good coordination with key actors: Coordination with Local Councils is viewed as an effective aspect of implementation, especially regarding communication on activity planning and mobilization. • Good participant selection: The provision of support for tilling the gardens of vulnerable populations to help increase their food production was appreciated by recipients. Challenges and Recommendations for Improvement: • Continued training needed: All focus groups mentioned that follow-up to trainings was insufficient, leading to uncertainty and poor confidence in new technique application. Participants requested increased home visits to ensure proper implementation of the kitchen gardens. Poor follow-up also led to limitations in the ability to adopt new crops and techniques. Follow-up trainings are recommended and ongoing assistance is recommended. • Reduce training recipient number: A large number of beneficiaries were trained, yet poor adoption results ensue. Staff are overstretched and overworked and though they are able to deliver single trainings, they are unable to provide the follow-up necessary for ready adoption of new techniques and practices. To boost adoption, it is recommended first that training recipients are carefully selected for their interest and, second, that additional and ongoing support is provided to these recipients. • Connections to input and output markets needed: A majority of focus groups mentioned the desire for increased assistance in connecting their products to markets. In Kaabong, in particular, participants were not able to plant their own gardens due to a lack of seeds and low market access to buy them. • Timeliness needed: Support was late with respect to the planting season, especially regarding provision of seeds, therefore limiting harvests and overall impact. Further, though coordination with local government and other key partners did transpire, it was often delayed, reducing impact, effectiveness and overall participation. 26 2. Improved Health and Nutrition 2.1 Key Activity Outputs Key outputs for each intermediate result under Strategic Objective Improved health and nutrition for pregnant/lactating women and children under 5 are listed below: IR 2.1: Improved Quality of Maternal/Child Health and Nutrition Care Delivered by District Services • 12 health centers received program assistance • 55 maternal and child health and nutrition workers, 221 VHTs and 25 Health workers trained in MCHN care in Kitgum, Pader and Kaabong Districts IR2.2: Improved Health-seeking Behavior • Engage VHT members and lead mothers in promoting health-seeking behavior. In Kitgum and Pader Districts, 74 VHTs and 720 leader mothers were engaged to promote health-seeking behavior. In Kaabong District, 147 VHTs and 240 lead mothers were engaged for the same • Program staff attended District Health meetings monthly to advocate for improved health-seeking behavior at the sub-district level IR2.3: Optimal Dietary Practices Adopted • Community growth monitoring was conducted every month in 10 sub-counties. • Optimal infant feeding and weaning practices among caregivers of children less than 59 months of age as well as psychosocial development through health/nutrition education at nutrition feeding centers was promoted amongst pregnant and lactating women. • All mothers belonging to the women’s gardening groups were taught to grow a variety of vegetables and fruits in home gardens in order to strengthen diet diversification and to promote nutrition at the household level. 2.2 Results Key results to date under this objective as compared to targets are as follows. A full listing is in the updated IPTT in Annex 2. 2.2.1 Anthropometric Scores and Nutritional Status The following results examine anthropometric measurements taken annually and for the mid￾term evaluation from mini knowledge practice and coverage (Mini KPC) surveys for 292 children (0-59 months) participating in the HPSC program. These are compared to the anthropometric measurements taken for the baseline study that are population-based and examine 1,415 children. Anthropometric measurements are not explicitly taken for children formerly or currently enrolled in the supplementary feeding program. Rather, measurements are taken for children living in communities with which the program supports in some capacity. 27 Again, per SOW guidance, the mid-term evaluation explicitly examined fewer children than at the time of the baseline. It is not expected that the program will have achieved measureable change in population-based anthropometric indicators at this point – at least not statistically significant differences that could be captured in a population-based survey of reasonable size. Instead, anthropometric measurements when combined with qualitative data should guide a discussion surrounding lessons learned, implementation progress and readjustment to future programming. Measurements of weight, height and edema were taken for all children. Child nutritional status was determined according to weight for age, height for age and weight for height indices derived from standard deviation Z-scores from the median international reference standards (WHO 2006) as well as the prevalence of edema. Height for age and weight for height indices were only estimated for children of 6-59 months. Weight for age, on the other hand, was estimated for children 0-59 months. In Kitgum and Pader Districts, the prevalence of underweight children has fallen since the start of the program, moving from 19.9% to 12.7% (p = 0.0040, df = 1,704). Further, the prevalence of wasting in these districts has fallen from 7.7% at the beginning of the program to 4.8% at time of mid-term (p=0.2230). The prevalence of stunting, however, has increased among children 6- 59 months, moving from 30.7% at baseline to 33.2% at mid-term (p = 0.4947, df = 1,491). Again, these measurements were taken across the population for both baseline and mid-term studies. The following graph delineates these results: Chart 4: Anthropometric Indices for Kitgum and Pader Districts In Kaabong District, the prevalence of underweight children has remained constant at 24.0% since the start of the program. The prevalence of wasting, however, has fallen from 8.2% at the time of the baseline to 5.4% at the time of the mid-term. Wasting results exceed targets for Kaabong District. The prevalence of stunting, on the other hand, has grown from 34.0% at baseline to 43.0% at mid-term. The following chart more clearly delineates these results: 28 Chart 5: Anthropometric Indices for Kaabong District The effectiveness of program activities to improve beneficiary weight for age, height for age and weight for height cannot be determined using comparisons between baseline and mid-term results for anthropometric scores across the population. Examinations of change in anthropometric scores over time or relapse rates among beneficiaries who participate in the supplementary feeding program or even return rates for households with multiple children who receive care may all be better measurements of program effectiveness, for example. However, it may be worth noting that weight for age, height for age and weight for height indicators correlate to access to diverse diets and to adequate food stuffs. As HDDS and MAHFP have also decreased since the start of the program, poor anthropometric scores could speak to larger issues of access to and availability of nutritional food. The children measured at the time of the mid-term grew-up after their families left camps and transitional resettlement sites. The HPSC program is designed to act as a bridge for families during this time of transition, helping them to resettle and to begin productive livelihoods so that they can begin to adequately meet their own needs without outside assistance. Disaggregating mid-term anthropometric results according to gender reveals that the prevalence of stunting, underweight and wasting in males is greater than in females (Table X below). Typically, females exhibit higher rates of stunting, underweight and wasting than males. Statistical significance tests show that this is a random occurrence that is statistically insignificant at 5% in all cases (p = 0.4677 and df = 208 for stunting; p = 0.0959 and df = 283 for underweight; p = 0.3076 and df = 202 for wasting). 29 The following section examines the severity of stunting, underweight and wasting for children in Kitgum and Pader Districts. Prevalence of Stunting A deficit in height-for-age is referred to as stunting and a low height-for-age index reflects past under-nutrition or chronic malnutrition and is an indicator of general growth failure. Slow growth of a child results in a failure to achieve expected height as compared to a healthy, well￾nourished child of the same age. At the time of baseline, 30.8% of children measured were defined as stunted. The results of the mid-term study found a higher prevalence of stunting as delineated in the chart to the right: Chart 6: Prevalence of Stunting (Children 6-59 Months) Prevalence of Underweight Underweight, defined as weight-for-age, is a composite measure of both stunting and wasting. It identifies the condition of being underweight for a specific age and reflects both chronic and acute-under nutrition. At the time of baseline, 19.9% of children measured were defined as underweight. Currently, at the time of the mid￾term study, 12.7% of children selected from HPSC program area are defined as underweight. The chart to the right delineates mid-term results: Chart 7: Prevalence of Underweight (Children 0-59 Months) Prevalence of Wasting Wasting in children is an indicator of acute malnutrition resulting from failure to gain weight or actual weight loss and is used to reflect the overall severity of under nutrition. The anthropometric indices show that 4.8% of children participating in HPSC program area and aged 0-59 months experience wasting. At the time of the baseline study, 7.7% of the children were wasted of which 0.7% children were severely wasted. The chart to the right best delineates mid-term wasting results: Chart 8: Prevalence of Wasting (Children 0-59 Months) 30 Diarrhea Infection A total of 422 caregivers with children between 0 and 23.9 months participated in the baseline survey. In Kitgum and Pader Districts, more than two thirds (70.0%) of the caregivers reported that their children had diarrhea during the two weeks preceding the baseline survey. For the mid￾term study, only 38.2% of caregivers reported that their children had experienced diarrhea in the two weeks prior the survey (p = 0.0000, df = 793). These results significantly exceeded targets. Further, caregivers demonstrate heightened health-seeking behavior now when compared to at the time of the baseline study; currently, all respondents seek some treatment for their children for episodes of diarrhea. Of caregivers with children between 0-23 months with diarrhea in the two weeks prior to the assessment, 71.5% delivered proper care defined as using ORS or increased fluid intake. At baseline, 69.0% delivered proper care. Treatments were generally sought by caregivers in consultation with health workers. Chart 9: Children with Diarrhea in Two Weeks Prior to Assessment According to the mid-term study, caregiver respondents identified the following treatments that they sought for diarrhea: Chart 10: Treatments for Diarrhea Sought by Caregivers 31 2.2.2 Prenatal Care At the time of the baseline study, only 75.2% of the respondents reported that they had seen someone for prenatal care service and of the mothers accessing prenatal care services, 73.3% were seen by a skilled health professional. Slightly more than a quarter (26.2%) of the respondents were using unskilled health workers during their pregnancy. Unskilled health workers can be categorized as traditional health attendants and community health workers. At the time of the mid-term study, however, 99.0% of the respondents had seen someone for prenatal care services for at least one visit. Moreover, all respondents had been seen by a skilled health professional – a nurse, midwife or doctor. Overall, the number of women seeking prenatal services and, particularly, prenatal services from experienced and qualified healthcare professionals, has increased markedly over the course of the program. Accordingly, women now not only seek prenatal care in higher numbers, but they also seek this care from qualified professionals. There is also a greater shift of support amongst women towards locally available and more professional forms of prenatal support, namely nurses and midwives. Though women are overwhelmingly seeking prenatal services from qualified professionals, they are not necessarily completing the recommended four prenatal visits. At baseline, 75.0% of women completed all four recommended visits. At mid-term, 58% of women completed all four recommended visits. This is below the targeted amount of 79.0%. However, the difference between baseline and midterm results is not significantly significant (p = 0.3365, df = 765). Long distances to health facilities, poorly staffed health facilities, poor qualifications of staff in health facilities and continued use and trust of local birthing attendants may be reasons why women are not attending all four recommended visits. Women may be attending at least one prenatal visit in such high numbers in order to receive a “pregnancy card.” This card is highly valued as it is an avenue through which to receive free care, injections and medicine when pregnant. Without this card, it is difficult to receive the same care. Pregnancy cards have acted as a positive incentive; therefore, expanding upon the utility of this card and the perception of its value may be opportunities to incentivize additional visits to health facilities for pregnant women. Additionally, since Mercy Corps took over implementation of health related activities from IMC, it began to work with pregnant women only in their third trimester. Accordingly, it has been difficult for the program to adequately incentivize or affect women to attend all four recommended antenatal visits. Failure to work with pregnant women in their first two trimesters also represents a missed opportunity to impact the health of mothers and babies early on. Recently, the program began “mother care groups” to work with pregnant women to share better practices with other pregnant women. These groups are in their nascent stages; impact cannot yet be measured. 32 2.2.3 Prevention of Malnutrition Infant and Young Children Feeding Practices (IYCF) At the time of baseline, 11.0% of children between 6-23 months of age from Kitgum and Pader Districts were fed according to the three defined appropriate Infant and Young Children Feeding practices (IYCF) – continued breastfeeding, age-appropriate dietary diversity and age￾appropriate frequency of feeding. At mid-term, 38.0% of children of the same age were fed according to the three defined practices. The following chart highlights the change in IYCF practices utilized: Chart 11: Change in IYCF Practices Utilized Complementary Feeding Of the eight recommended solid complementary food groups, children measured for the mid￾term study were fed an average of only 2.6 groups regularly. This is a significant decrease from the baseline where an average of 3.4 food groups was consumed regularly by a comparable population. Though overall consumption diversity has not improved, there has been an increase in consumption of dairy products. There has also been a reduction in pulses and oil. There was an increase in the number of respondents that reported feeding their children more than one food stuff. The food groups consumed are highlighted in the following chart: Table 4: Food Groups Consumed Food Group Baseline Consumption (%) Mid-term Consumption (%) Diary 4.2 9.7 Grain 86.1 71.5 Vitamin A 73 73 Fruit 18 2.4 Egg 5 - Meat 28 21.3 Pulse 54.4 22.2 Oil 76.3 62.3 33 A number of key informants highlighted that though the program supports a range of crops – those that are part of the value chain group as well as those vegetables that are cultivated in kitchen gardens and by women’s gardening groups – the program does not support activities surrounding livestock. Accordingly, measuring complementary feeding practices that heavily weigh consumption of animal by-product, then, may not accurately reflect on the successes or failings of the program as it is. This may, however, point to a need within the program to include more activities surrounding livestock. 2.2.4 Beneficiary Views: Most Significant Change Stories The following figures highlight Most Significant Change Stories taken from focus group discussions with Supplementary Feeding Groups and Mother Support Groups. Most program participants discussed positive life changes as a result of access to food supplements and increased knowledge derived from trainings on gardening as well as improved health and hygiene practices. Again, quantitative and qualitative mid-term data purposefully offer varying, equally-valid, perspectives. The qualitative data focuses exclusively on beneficiary experiences; the quantitative data examines population-wide changes. Both offer compelling perspectives regarding HPSC programming. 34 Figure 4: Most Significant Change Stories for Supplementary Feeding Groups “Before the program started, I used to eat once a day and some days nothing, but today I can cook three times and at worst two times and now I look a changed person.” Nancy, Lokori Central, Kaabong Health: “My baby used to fall sick frequently and was under-weight compared to age. Now the baby has gained weight and is very healthy and does not fall sick frequently as before.” Agness, Lanyambira, Pader “I gave birth to my first born but he fell sick. I took him to the hospital and it was discovered that my child was malnourished. Mercy Corps helped by providing him with some food stuffs and now my baby is very healthy.” Agnes, Kakuruk, Kaabong Dietary Diversity: “Before food supply, my family used to feed on sorghum and wild leaves as vegetables, but now I swear we can balance the foods we eat.” Christine, Lokori Central, Kaabong “In my home there was nothing like balancing the diet for children and the children could always fall sick, but since I joined, balance diet has been provided and my children are doing well.” Anna, Kakuruk, Kaabong Baby weight: “When I gave birth to my first three children, they were always weighing less than 2 kgs, but when I started feeding on the food stuffs supplied I was able to give birth to a baby weighing 4.0 kgs.” Lina, Lokori Central, Kaabong Knowledge: “Before I joined the group, I used not to space my children, hence they were severely malnourished. But I now know how to space my children and they are growing well.” Catarina, Kakuruk, Kaabong The above analysis shows results from 19 individual beneficiary stories relating the Most Significant Changes brought about in their lives since participating in the program. The pie chart at left illustrates the frequency of major themes throughout the stories. Illustrative quotes are presented below. Food & nutrition: “I appreciate what Mercy Corps is doing… before Mercy Corps introduced their activities in our community our children used to die of hunger…but as Mercy Corps established this program, we have never heard that someone has died due to starvation.” Rose, Kangole North, Kaabong 35 Figure 5: Most Significant Change Stories for Mother Support Groups Members of Mother Support Groups were asked in focus groups to relate the most significant changes in their lives due to participation in the project. The following graph represents the amount of times the themes were mentioned in stories from four focus groups, followed by a few illustrative quotes. Hygiene and sanitation: “The community had no latrines, we used to defecate everywhere including even the homesteads…this caused many people to suffer diarrhea. But as this program was established in our community, hygiene has been greatly improved.” Betty, Lomorunyangae, Kaabong Disease prevention: “I used to be frequently falling sick together with my children, but when I joined the group, I learned that keeping personal hygiene is most important. I and my children are living healthy now.” Mary, Sarachom Central, Kaabong Relationships/knowledge sharing: “We gained knowledge and became trainers to train others. This encouraged the relationships among us.” Irene, Lakore, Kitgum Dietary diversity: “Knowledge from the training helped especially in how to cook varieties of food, but I ran into problems because there was no support given after the training.” Silvia, Odilang, Kitgum Harm prevention: “We believed in customs that were harmful for both children and mothers, such as mothers bathing three days after giving birth, but as we joined the group such practices have been eradicated in our community.” Christine, Lomorunyangae, Kaabong. Self-esteem: “I used to be quarrelsome, lazy…now I am changed person since I joined the group, where there is sharing of developmental ideas.” Lucy, Sarachom Central, Kaabong 2.3 Effectiveness of Implementation The following section outlines successes and challenges to program activity implementation under Strategic Objective 2. These recommendations largely coincide with the LAYERS monitoring system report produced in conjunction with USAID Uganda. The section also includes recommendations for future action. Successes • Quality and effective training: Trainings are appreciated and seem to be effective, resulting in increased knowledge of healthy behaviors amongst direct beneficiaries as well as dissemination of this knowledge to non-beneficiaries. • Improved health and nutrition: Beneficiaries are not only being trained, but they are applying and practicing what they have learned. Many of the Most Significant Change 36 stories told of how increased access to food, dietary diversity and knowledge of healthy behaviors was improving mother and baby health and nutrition. FGD participants often credited improved feeding practices brought about by program activities as a reason for improved birth-weights of newborns and less malnourishment and sickness amongst both mothers and children. • Relevant activities: The overall relevancy of the activities as well as the food provided is apt. All focus groups emphasized the critical relevancy of the food distributions and associated education, given the ongoing food insecurity in the region. • Good coordination: The program is collaborating well with district officials and offices on activities such as community mobilization and targeting and food distribution. Beneficiaries appreciated that food generally arrived on time. • VHTs are functional: VHTs supply necessary health information and services. Some FGD participants had glowing feedback on the work style and effectiveness of VHTs. “They are effective in serving the health needs of the community as they do move from house to house,” said a mother support group participant in Sarachom Central, Kaabong. Challenges and Recommendations for Improvement: • Dissatisfaction with amount of food distributed: All focus groups mentioned increasing the size of rations. “The program is good except the food is not enough for us,” said a participant in Kangole North, Kaabong. Focus groups in Kaabong in particular requested provision of additional items such as salt and sugar. Additional food aid is not recommended. More intelligent use of food aid as a proper incentive is recommended. So too is additional food aid education; corn soy blend is distributed as a treatment for malnourishment and should not be given to all children in the home, but to the child that requires treatment. • Continued training needed: All focus groups mentioned that follow-up to trainings was insufficient, leading to uncertainty and poor confidence in new technique application. Several found the frequency and depth of training, and in particular follow-up support to ensure proper application of feeding knowledge, as an area to be improved. “Health Education should be given to mothers continuously. Frequent follow up should be done to find out whether the mothers are really feeding their children the way they should,” said a mother in Lanyambira, Kitgum. Increased follow-up on trainings and group activities to guide their implementation is recommended. This would include more frequent follow-up visits to check-in on the activities of the group in order to help address any challenges that arise. Increased follow-up would also help them better learn how to apply their knowledge. “Knowledge I got from the training has helped me...however I got problems because there is no support being given after the training,” said Ayeno, a Mother Support Group member in Odilang, Kitgum. Others mentioned conducting more training of trainers in the community, so they could expand their reach. Additional trainings for more community members and greater depth of knowledge were also requested. • Distance and inability to traverse that distance are problematic: Many groups in Kaabong in particular referenced the long distance to health clinics in addition to the poor state of roads as impediments to proper care. 37 3. Increased Access to Water and Sanitation; Improved Hygiene 3.1 Key Activity Outputs Key outputs for each intermediate result under Strategic Objective Increased Access to Water & Sanitation and Improved Hygiene Practices are listed below: IR 3.1 Increased Availability of Safe Water Sources & Sanitation Facilities • 21 water points were rehabilitated or constructed in Kitgum and Pader Districts • 11 microdams were constructed in Kaabong District IR 3.2 Healthy Water, Sanitation & Hygiene Practices Adopted • 25 sites in Kitgum and Pader as well as 8 sites in Kaabong were assessed for construction and rehabilitation of institutional latrines • 15 institutional latrine sites in Kitgum and Pader as well as 3 sites in Kaabong were rehabilitated, constructed or maintained • 1,300 latrine slabs in Kitgum and Pader as well as 1,000 latrine slabs in Kaabong were produced and distributed to households 3.2 Results Key results to date under this objective as compared to targets are as follows. A full listing is in the updated IPTT in Annex 2. 3.2.1 Adoption of Healthy Hygiene Practices Hand washing with soap is an important practice for safe hygiene which can be easily observed at the individual level. Hand washing is also a critical means to achieving overall healthy hygiene practices. During the mid-term assessment, the proportion of households who reported having soap in their house was 65.2%. At the time of the baseline study, this number was only 56.9%. In Kitgum and Pader Districts, 71.1% of caregiver program participants are demonstrating proper personal hygiene behaviors like hand washing. For Kaabong, 56.2% of caregivers participating in the program are demonstrating the same. Higher than targeted results for proper food hygiene practices amongst program caregiver participants are also demonstrated across the three districts. In Kitgum and Pader, 65.3% of caregiver program participants demonstrate proper food hygiene practices; in Kaabong, this number is 53.6%. 38   Chart 12: Participants Demonstrating Proper Hygiene Practices 3.2.2. Water Sources and Use Nearly all of those surveyed (97.8%) derive their main source of drinking water from a borehole or covered well. Only 2.2% of respondents rely on open wells for their main source of drinking water. At baseline, only 79.7% of those surveyed derived household drinking water from a borehole or covered well. Moreover, at baseline 18.4% of households relied on surface water for drinking. Currently, no households report surface water nor unprotected water as sources of drinking water. 3.2.3. Drinking Water Preparation Fewer respondents reported that they treat their drinking water at the time of the mid-term than at the time of the baseline. Currently, 12.6% of respondents treat their drinking water; previously, 64.4% did so. This change could be attributed to change in perceptions of cleanliness that may have accompanied shifts in drinking water source from surface water to boreholes and covered wells as many widely consider these sources of water as clean sources. It may also be that drinking water was easier to treat at the time of baseline as treatment equipment was then supplied by camp management. At mid-term, the most common means of treating water is boiling (4.4%), then “stand and settle” (3.9%), followed by Aquatab use (1.9%). 3.2.4. Latrine Usage Latrine availability and cleanliness were investigated in order to gauge hygiene practices and latrine ownership and use. The mid-term study revealed that households use pit latrines/VIP latrines, those latrines with proper ventilation and concrete slabs, with less frequency now than at the time of baseline study. Currently, only 38.5% of those surveyed use pit latrines/VIP latrines; at baseline, 87.9% did the same. However, households now use open pit latrines, those latrines without concrete slabs, more often. Currently, 19.0% of households use open pit latrines whereas only 9.9% of households did so at the time of the baseline. Again, nearly half of all program 39 participants resided in IDP camps at the time of baseline; camps provided access to a range of services, including access to pit latrines/VIP latrines. Access to these facilities likely decreased as a result of resettlement and will take time to build up once more. 3.3 Effectiveness of Implementation The following section outlines successes and challenges to program activity implementation under Strategic Objective 3. The section also includes recommendations for future action. Successes • WASH activities are relevant: Participants identified program activities surrounding water and sanitation as relevant activities for the issues faced by their community, naming boreholes as particularly relevant to improved health for the children of the community. • WASH activities have population-wide impact: Important quantitative improvements in better WASH practice understanding and adoption point to possible program influence beyond direct beneficiaries. Population-wide WASH improvement is a notable success at this stage of programming. • Good coordination with local government: District officials have outlined extending safe water and improved sanitation as district priorities. Mercy Corps has coordinated water point and water source selection well with local government. One Community Development Officer located in the sub-county government stated, “Mercy Corps has scored highly in all important areas for development: roads, water and sanitation, agriculture.” Challenges and Recommendations for Improvement: • Improve integration of program-wide activities: The integration of MYAP program activities among sectors needs to start at the moment of registration of the targeted beneficiaries to avoid delaying program implementation. For instance, trainings of VHTs for WASH and H&N should transpire simultaneously. • Improve water committee functionality: Water committees do report water source breakages to their LC and VHTs. They also train water users on best use of water and the structures. However, the water structures are not well kept. Committee members do not highly prioritize water source management, maintenance or repair. • Improve institutional partners’ capacity to meet complementary WASH needs: Latrine cleanliness and soap and hand-washing facility availability are in question. Though it is not a program responsibility to meet these needs, meeting these needs is important to the overall Strategic Objective. Working with institutions and partners to understand their obligations as well as augmenting the transparency of these obligations would help to ensure that necessary complementary WASH components are adequately met. 40 4. Cross-cutting and Implementation Issues 4.1 Changing context In Kitgum and Pader, the context has changed dramatically since the program began in 2008. Many people have returned to living in their home villages full-time, rather than residing in camps or splitting time between the two. Increased peace and security and associated freedom of movement was most frequently cited by focus groups when discussing most significant contextual change. Additionally, groups also mentioned increased access to land for farming and gardening, increased access to quantity and/or diversity of food, decrease in prevalence of diseases as well as improved environment for child-rearing and up-bringing as further most significant contextual change in the last three years. However, though focus groups point to a range of positive changes as a result of camp closure and resettlement, respondents were also adamant that Mercy Corps activities are highly relevant. A number of challenges continue such as malnutrition, minimal availability of livestock, poor production and dilapidated infrastructure. The long recent drought was often mentioned as a major impediment to improving nutritional status of children. Participants thus see supplementary feeding activities as extremely important for staving off further malnutrition of children. Other issues mentioned as needing continued attention included road construction, agricultural seeds distribution and improvement, nutritional knowledge, WASH, tilling for the economically vulnerable, and tree planting. Karamoja has also witnessed recent improvements in living standards, according to focus group participants. The main contextual changes in Karamoja mentioned in focus groups included increased access to quantity and/or diversity of food, improved peacebuilding efforts and cooperation, increased security and freedom of movement, improved hygiene and sanitation, increased knowledge and practice regarding healthy diets, improved farming methods as well as decreased child malnutrition. Many focus group participants mentioned an improved feeding system as a primary change, referring both to supplementary feeding food distribution as well as increased production from farming and gardens, leading to improved diets. One focus group in Kakuruk village said, “There is provision of food stuffs which was not the case before. Feeding habits have been adopted such as eating three times a day, or at worst two. There are reduced numbers of malnourished children as they are taken care of.” In assessing the current relevance of Mercy Corps activities, almost all focus groups again stressed the nutritional elements of the program as critical for staving off disaster. One group in Kangole North noted, “Mercy Corps programs are still relevant because there are still children who are malnourished and suffering, hence if Mercy Corps stops their activities there will be high death rates in the communities.” Another group in Lokodope said, “All of Mercy Corps’ activities are still relevant to the problems faced by our community. So they should not leave until we stabilize.” Respondents expressed a desire to avoid reversing the nutritional and food security gains of the last few years. 41 4.2 Coordination and collaboration Focus group participants largely mentioned that the program and its various beneficiary groups were coordinating well with local village authorities such as the Local Councils and Village Health Teams. This was especially true for community mobilization for distributions, trainings, meetings and other events, as well as other types of support. At the same time, some of the implementation issues mentioned in focus groups and key informant interviews suggest that coordination could be enhanced in terms of strategic planning processes, such as planning the timing of events like seed distribution and harvest assistance (tillage activities) so that they match more appropriately with the planting season. Some participants also mentioned that strategies for advocacy regarding improved health behavior could be developed in closer coordination with Mother Support Groups and Women’s Gardening Groups. 4.3 Funding process Mercy Corps experiences particularly high staff turnover that in turn results in poorer program activity performance. One reason for high staff turnover discussed frequently in key informant interviews is the large amount of work required of staff. In FY 2011, Mercy Corps worked to improve staff morale, to institute a staff orientation to assist with ease of turnover and to boost compensation and benefits to encourage staff to stay on. A late start to FY 2011 programming also resulted in the rushed implementation of a large number of activities in a short period of time. The large flood of activities with expected completion in a short period of time further contributed to staff workloads and may have also contributed to poor staff retention. Late approval also likely hampered the efficaciousness of activity implementation. 4.4 Commodity delivery Key informants point to effective and generally timely distribution of commodities. Food distribution is frequently cited as a significantly positive activity. However, key informants and focus group participants also cite the use of food as compensation for such activities as building and maintaining roads or growing tree seedlings. Respondents recognize that without food compensation, the continuation of these activities is in doubt. Proper use of food as an incentive for program activities should be examined going forward. 4.5 M&E The monitoring and evaluation tools used to measure program activity and effectiveness are adequate. The program employs a range of tools for regular monitoring of activities. They include attendance lists, master beneficiary lists and event forms, for example. Program staff also 42 employ simple assessment and survey tools. These tools are shared for revision before final deployment. The program collects voluminous, paper-based data; generally, this data collection is at the request of the donor. The overall frequency of assessments and data collection throughout the fiscal year is laborious. Assessments include those requested by donors like the LAYERS survey, UMEMS DQA and KPCs as well as those done internally. Other reporting requirements like regular entries to the USAID performance monitoring system run by UMEMS and periodic narrative and financial reports are time-consuming. Mercy Corps staff believe data collection is excessive and tedious; moreover, staff do not believe that the data collected is used. UMEMS technical staff believes that they offer sufficient technical support to program staff and adequately assist them with the implementation of recommendations from the 2010 Data Quality Assessment. The assessment has led to improvements in the way data is captured, analyzed and controlled. In FY2011, Mercy Corps began using smart phones with the assistance of the Grameen Foundation’s AppLab. These phones are assisting Mercy Corps to monitor results and reorient activities in a looped learning process. Though still in its nascent stages, the smart phone tool is an innovative means to reduce the laboriousness of data collection and to boost staff buy-in with instant results. Going forward, this tool will help HPSC to address programming needs in real￾time and to loop learning and program effectiveness. However, the M&E system could better incorporate data sources for more nuanced and iterative program learning. Mercy Corps management has expressed a need to incorporate elements of knowledge management to improve on current M&E practice. For example, incorporating GIS in some program activities could significantly reduce workloads, particularly mapping of food distribution points and specific activity locations. 4.6 Gender Many focus group respondents appreciated the program’s gender focus in activities such as Women’s Gardening Groups, Tillage beneficiaries, and Supplementary Feeding Groups, all of which consist of either all or mostly women beneficiaries. Participants particularly commented on the knowledge gained as leading to a range of benefits, among them increased self-esteem and improved cooperation with friends and neighbors through group participation. Others mentioned increased income and purchasing power due to gardening activities. Both focus groups with producer groups in Kaabong commented that the gender balance in the program was not realistic or appropriate to norms, in terms of including more women than men in this activity. For example, the producer group in Lobalangit Central, Kaabong, noted that 30 women and only 10 men participate in their producer group. This is typical of producer groups supported by the program. Some producer group participants express frustration that women are, in their views, over-represented. 43 D. Recommendations The following section highlights recommendations for improved implementation. SO1 recommendations • Reexamine training approach. The high number of trained recipients, repeated requests for additional follow-up trainings and poor staff retention all point to a need to reexamine the way the program approaches trainings. Careful and calculated use of incentives could result in higher farmer self-selection and fewer, more committed attendees. In turn, this may free staff time to provide greater depth of training for those truly interested in adoption. • Reexamine use of incentives to promote long-term development objectives. The program’s use of FfW and distribution of free tools and tillage skews incentives and negatively affects the sustainability of program activities, perpetuating expectations of handouts and diminishing ownership of and commitment to the development advocated by the program. Careful use of food and other handouts is requisite for sustained, long￾term impact from the program, which should be redesigned to focus on systemic intervention. For example, shifting toward improving access and availability of inputs and away from free distribution of inputs and supporting the emergence of more commercial tillage providers. • Provide greater support to market access initiatives. Producers express the strong desire for the program to support them to link with buyers in order to access more lucrative, higher volume markets. The program should examine opportunities for such linkages. These opportunities may involve supporting alternative crops, livestock opportunities or other prospects that could generate such linkages. The program will need to consider such opportunities carefully and to ensure flexible programming accordingly. • Reexamine commodities and livestock support. Frequent examination of those agricultural commodities supported by the programming is important. Farming communities should be expected to change their farming preference as the region transitions to a higher level of development and as changing opportunities present themselves. Greater program flexibility will help the farming communities to best position themselves to meet output market demands. Moreover, the program should consider expanding work into the livestock sector. There is increasing demand and interest in livestock; access and availability of sector needs is key. The program is uniquely positioned to move support beyond commodities and into livestock. • Improve coordination with partners. District partners are satisfied with program work on roads. However, long-term ownership and maintenance of these roads is in question. Early and ongoing coordination of road construction and rehabilitation as well as timely handover of completed works is recommended. 44 SO2 recommendations • Reexamine training approach. Similarly to SO1, those trained under SO2 believed that follow-up to trainings was insufficient, leading to uncertainty and poor confidence in new technique application. Several found the frequency and depth of training, and in particular follow-up support to ensure proper application of feeding knowledge, as an area to be improved. Increased follow-up on trainings and group activities to guide implementation is recommended. • Strengthen male participation in health and nutrition activities. Currently, women are the focus of health and nutrition activities. However, most decisions continue to be made by men. Though the program trains women to plant alternative crops and to cook them differently while also supplementing nutrients where necessary with corn soy blend, it remains the males in the household that control these activities and food use. Accordingly, male buy-in must be encouraged. • Reexamine use of incentives to promote long-term development objectives. Participants are dissatisfied with the amount of food distributed. However, additional food aid is not recommended. More intelligent use of food aid as a proper incentive is recommended. So too is additional food aid education; corn soy blend is distributed as a treatment for malnourishment and should not be given to all children in the home, but to the child that requires treatment. • Expand VHT use. VHTs represent an opportunity to mitigate many of the barriers to adequate information, access to care, availability of life-saving products, distance to health facilities and poor staffing of health facilities. Working with VHTs to expand information and access to life￾saving products could be one way to extend health coverage while helping communities to stay connected to health facilities when needed. Further, linking VHTs with private distributors to embed product access with better information is an opportunity to better meet the health needs of communities in the remote areas in which the program works. For example, making oral rehydration salts available locally on a commercially sustainable basis will strongly impact diarrhea-related mortality and morbidity. Since many health units are far way from some communities, VHTs could provide this product as well as best information regarding its use. SO3 recommendations • Improve water committee functionality; improve private sector ability to provide service. Though a breadth of water facilities have been built throughout the last decade, a large number of them are not functioning. Though water facility problems are usually fixable, failure of water committees to collect dues and ensure proper maintenance results in broken facilities. With two years remaining in the program, it is not an effective use of resources to rehabilitate all of these broken water points only to see them break down again in the future with neither future funds nor organizational capacity to offer repairs. Rather, it is a better use of resources to support water committee functionality to fulfill 45 their duties. Further, it is important that along with committee functionality, private sector support for water points is available. This includes qualified mechanics, availability of spare parts and resources to ensure that parts can be purchased as necessary by committees. The program should work to ensure that all of these points are functional at the end of programming. Cross-cutting and general implementation recommendations • Increased follow-up on activities. The program is implementing a wide range of activities spread over a large geographic area. However, implementation of these activities may be spread thin, as the most constant and consistent feedback from beneficiaries was related to the need for increased follow-up on trainings and other activities of the various groups to ensure they are working as planned or applying knowledge learned as directed. Accordingly, focusing activities more strategically on a smaller number of more committed participants to allow for increased follow-up and enhanced impact is recommended. • Increase coordination and communication. The strategic planning process, such as planning the timing of events like seed distribution and harvest assistance, strategies for advocacy on improving health behaviors in coordination with Mother Support Groups and Women’s Gardening Groups, road rehabilitation and construction or general program staff activity, could all be improved. • Increase integration of program activities and beneficiaries. The integration of program activities among sectors needs to start at the moment of registration of the targeted beneficiaries to avoid delaying program implementation. Training needs to be better coordinated to all beneficiaries are receiving all possible training while also avoiding double counting of beneficiaries. • Ensure incentives are used to promote long-term development objectives. Ensuring sustainability of program objectives hinges on best use of food and other free incentives. Food distribution is extremely time-consuming and labor intensive. Further, handouts often go to the wrong people – resulting in wasted investment in the wrong producer groups, for example. Careful and calculated use of food and free inputs as well as the integration of systemic, market-driven programming wherever possible is essential to the success of the program. • Enhance staff retention. Program staff are overburdened with work and do not stay with the program accordingly. This results in lost knowledge and poor morale. Too much reporting, too much food distribution, too many producer group participants and so on, all contribute to poor staff retention. More intelligent use of program tools and selection of activities and participants will help to diminish workloads while contributing to better staff engagement and, accordingly, retention. 46 • Expand program reach beyond beneficiaries to larger population. The program clearly impacts direct beneficiaries as displayed through qualitative assessments. However, the program does not yet seem to significantly impact the Acholi and Karamojong populations as a whole. Sustainable, systemic programming necessitates larger, population-wide program influence. Opportunities to expand program influence beyond direct beneficiaries should be explored. • Capitalize on missed program activity opportunities. The program is entering its fourth year of implementation. With only five total years of program activity, sustainability must be an essential program focus. The program will leave Acholi sub-region and Karamoja shortly; what activities will remain in place then? Accordingly, opportunities for systemic, market-driven intervention should be considered and taken advantage of wherever possible. For example, expanded VHT use to embed information along with product availability to increase outreach while more wholly meeting community needs, represents just one of many opportunities within the program to integrate systemic change. All opportunities should be fully explored. 47 Annexes: Annex 1: MYAP Mid-Term Evaluation Scope of Work MYAP Mid-Term Evaluation - 2011 Mercy Corps Uganda Program Background With support from USAID office of FFP, Mercy Corps Uganda is implementing the Healthy Practices, Strong Communities (HPSC) program in the regions of Acholiland and Karamoja, in northern Uganda. The Program began in six sub-counties of Acholiland (Kitgum and Pader Districts) in August of 2008. A year after inception, the program expanded its geographical coverage to two additional sub-counties in Karamoja (Kaabong District) and an additional sub￾county in Kitgum and Pader Districts. Program activities in Kitgum and Pader Districts are for a five year duration and scheduled to end in July of 2013. Activities in Karamoja will last for three years and end one year earlier (July of 2012). The goal of the program is healthy, peaceful and food-secure communities in northern Uganda. The program was originally designed to respond to food insecurity and the recovery needs of returning IDPs who had been forced to live in camps to escape over 20 years of violence and conflict. However, the design was expanded to also include food insecure communities in Karamoja where prevalent conflict and adverse weather conditions have affected people who are living in similar agricultural settings. Both Regions are characterized by poor agricultural production, debilitated feeder roads, absence of basic services, dried or silted water points and little market opportunity, all of which further exacerbates food insecurity. Mercy Corps’ program goal, in line with the FFP objective of reducing food insecurity among vulnerable population, is addressing challenges to availability, access and utilization of food. The HPSC program is taking a broad and integrated approach to food security that is appropriate to the Ugandan developmental relief setting. Mercy Corps currently works with Government authorities and communities to open access roads, promote high value and nutritious foods, and promote access to safe water as well as hygienic practices. Activities are focused on development relief and support communities in transition. A baseline assessment conducted at the start of the program for Kitgum, Pader and later Kaabong will be the basis on which progress in key program indicators will be assessed. Annual KPC (Knowledge, Practices and Coverage) surveys are conducted in project sites to remain informed and influence program direction. These studies have provided critical feedback that has assisted in refining indicators and establishing benchmarks in which to gauge the program’s progress and assess impact more regularly. Along with Mercy Corps’ commitment to learning from program implementation, it is also essential that the HPSC program be evaluated at its mid-term. Starting in July of 2011, preparations will begin to conduct a comprehensive assessment of the five-year program in 48 Kitgum and Pader and the three year program in Kaabong that will examine the process of implementation and progress towards selected key outcome indicators within the beneficiary population. The mid-term will also be an opportunity to build staff capacity, with broad participation by team members. By examining intended and unintended impacts, the HPSC program will have the opportunity to learn which activities are making an impact and which are not, and to make the corrections necessary to better reach and even exceed program objectives. Results and recommendations from the evaluation will assist Mercy Corps to formulate corrective action plans for implementation to maximize its impact for the remainder of the program. Program Objectives and Results: 1. Improved access to food • Smallholder farmers and vulnerable women have increased food production • Increased access to agricultural inputs, services and markets • Adoption of environmentally sound improved agricultural practices 2. Improved health and nutrition among pregnant/ lactating women and children under five years of age • Improved quality of maternal/child health and nutrition care delivery by district services • Improved health seeking behavior • Adoption of optimal dietary practices 3. Improved access to water and sanitation and improved hygiene practices • Increased availability of safe water and sanitation facilities • Adoption of healthy water, sanitation and hygiene practices Key Program Activities The above objectives are achieved through the following key program activities: • Establishing, training and support for producer and women’s gardening groups • Establishing demonstration plots for tree nurseries and high value crops in demonstration of improved agricultural technologies and reduce the rate of environmental degradation through tree planting. • Availing agro inputs to farmers by facilitating seeds/ tool fairs and tillage vouchers. • Rehabilitating or opening farm-to-market roads through Food for Work (FfW). • Training government facility health workers, Village Health Teams (VHTs) and Mother Care Groups to enhance their nutritional knowledge and skills • Distributing supplementary food rations to moderately malnourished children and pregnant or lactating women to reduce the effect of malnutrition on vulnerable groups • Conducting nutrition education and counseling for mothers and caregivers of children under five years of age to improve child nutritional care practices 49 • Conducting regular child growth monitoring and promotion as a means to identifying children at risk of malnutrition and monitoring the impact of food supplementation and nutrition counseling/education on the at-risk group • Rehabilitating or building wells and institutional latrines to improve access to safe water and improved sanitation in the project area • Conducting training on hygiene practices to improve knowledge and skills of beneficiaries towards prevention of water and sanitation related diseases Purpose and Objectives of the Mid-Term Evaluation The purpose of the Midterm evaluation is to assess the progress of implementation and document lessons learned. Scope of the Evaluation: Evaluation objectives and key questions 1. To assess the progress of the project towards achieving objectives and meeting targets. • To what extent has the program been successful in achieving its objective-level outcomes and meeting its targets? 2. To assess the degree to which the program’s design is still relevant given the changing context and nature of the beneficiary population • Are the activities of the project appropriate in terms of addressing the original problems? Have the nature of the problems or the context in which they exists changed, and if so, have activities and strategies been altered in order to remain relevant? 3. To assess whether the program is being implemented as originally planned, identify any roadblocks and suggest strategies necessary to accomplish planned activities for the remaining years of the life of the project • Is the program being implemented as originally planned? What is working and what is not working? What are the strategies used to accomplish the planned activities? • To what extent has the program incorporated and/or has been adjusted based on findings from periodic formative research on knowledge, practices and coverage. • How appropriate are the monitoring and evaluation tools in addressing the need of the project? • How is the funding process affecting the project in terms of fund flow? • Do commodities arrive at the right and required time? • What are the internal and external constraints that may have arisen during the project implementation and how has the project responded? • What changes could be made in order to strengthen and enhance project performance? 4. To assess if there have been any unplanned consequences or impact of the project’s activities to date, whether positive or negative 50 • What changes have come about as a result of the project? What factors have contributed to these changes? 5. To assess the level to which the program has engaged key stakeholders like USAID, local government, partners and beneficiaries • What is the level of collaboration with the ministries of health, agriculture & fisheries, local government administration, other USG programs, USAID/FFP and other related stakeholders? • What is the level of local community/and or government participation in program activities of each sector? • What is the gender balance of beneficiary involvement in program activities? Scope of the Evaluation: Coverage Both qualitative and quantitative techniques will be employed in Kitgum and Pader. Owing to the relative infancy of operations, only qualitative techniques will be deployed in Kaabong. Methodology As the baseline assessment gathered quantitative information using household questionnaire, the mid-term assessment will use a similar household questionnaire, plus qualitative means of data collection through focus group discussions, key informant interviews, and review of program documents. Qualitative analysis will also be important to help account for major changes in the target population since the baseline assessment occurred. During the Baseline in 2008, the population was still moving between mother camp, transitional resettlement sites, and their own land3 . In addition, food distribution by WFP was far more prevalent. Data will be collected in the post-harvest season of August and September 2011. A household questionnaire that captures key outcome and impact indicators of food security will be utilized. This questionnaire will be adapted from that used in the baseline assessment by inclusion of previously omitted questions and leaving out questions on which similar information can be extracted from program monitoring documents or do not provide significant information to the learning objective of the midterm evaluation. As the questionnaire has already been designed to capture the most of the project indicators relevant to the mid-term assessment, only minor adjustments will be required in order to assess progress on indicators that were added after the baseline survey or those that were not well captured initially. The questionnaire will be administered to 210 households from 30 beneficiary villages of Kitgum and Pader districts. The overall sample size of 210 households is based on confidence interval of 95% (default value), estimated prevalence of key indicators of 50% (the most conservative or worst-case scenario for estimating sample size), margin of error of 10%, and design effect of 2 (default for two-stage cluster sampling). The sample of 210 households will be achieved by randomly selecting 7 households in 30 randomly selected clusters (villages) in both districts combined. The clusters (villages) will be selected using Population Proportion to Size (PPS). Households within clusters will be selected either by systematic random sampling from household lists maintained by 3 Baseline Survey Report page 32: “Although a majority of the respondents that were interviewed reported accessing farmland of their own, most accessed land either the main (mother) IDP camp or Transitional Resettlement Sites (TRS)” 51 authorities, if the lists are generally thought to be accurate and reliable. In the event that the household lists are not a reliable, selection will be made using the segmentation technique, in which case the cluster will be sub-divided into equally populated segments and one segment is chosen at random, and households within that segment are numbered and randomly selected using systematic random sampling. Data collected with questionnaires will be triangulated by additional inquiry conducted in focus group discussions (FGDs) and key informant interviews (KII). Unlike the quantitative method, this qualitative approach will be applied in all three districts of Kaabong, Kitgum and Pader. Personal observations of the evaluation team will also contribute valid results for this evaluation. For data collection in FGDs and key informant interviews, semi-structured discussion and interview guides will be developed. Participatory Impact Assessment (PIA) tools and techniques may also be deployed in FGDs to generate qualitative and quantitative data. Community representatives will be interviewed as key informants while interest groups such as Producer Groups, Women’s Gardening Groups, VHTs, and Mother Care Groups will be covered through focus groups discussions. Through this qualitative investigation, the evaluation will determine unintended impacts, lessons drawn as well as evaluation of program design, implementation activity benefits realized in relation to food security, and changing program dynamics. Further interviews will also be held with sub-country and district-level government officials, USAID representatives, and Mercy Corps implementing staff. Beneficiaries will be the source of information for both qualitative and quantitative data collection, while program implementers, government representatives and donor representatives will provide qualitative information through key informant interviews. In Karamoja (Kaabong District), the household surveys will not be conducted, as the evaluation will rely on qualitative information as described above as well as on existing documentation and monitoring information regarding the current program. Relevant documents to review include the Program Proposal, Quarterly Narrative Reports, Annual Results Reports, Baseline Assessment Reports, Mini KPC Reports, IPTT, DIP, PMP, PIRS (Original and current), annual Pipeline and Resource Estimate Proposals (PREP), and all essential monitoring and evaluation tools and databases used by HPSC. In order to maximize efficiency in data collection and analysis processes, Grameen AppsLab, a sister organization of Grameen Foundation and a leading innovator in mobile technology, will provide mobile data collection tools and train Mercy Corps staff on how to use these tools in the mid-term evaluation. Both the M&E tools and the data collected will feed into the back-end electronic M&E system, which Grameen will develop with Mercy Corps. Smartphones will be utilized during data collection and link into an electronic M&E system with a web-based dashboard that will improve the accuracy, efficiency and effectiveness of the process. Data analysis and results dissemination plan For quantitative data collected via the household survey, the evaluation team will conduct descriptive and bi-variate analyses using the appropriate tests of statistical significance. Where comparisons between groups (defined through bi-variate analyses) are made, the level of statistical significance associated with the difference between groups will be cited as a footnote to the text. Confidence intervals, p-values and other criteria used in hypothesis testing will be 52 documented as either footnote or Annex. Qualitative information will also be collated and synthesized to highlight key points and facilitate identification of key outcomes and lessons learned. Qualitative information will provide insights in understanding of quantitative results and is expected to uncover why and how program activities led to varying levels of outcomes. Initial findings from both quantitative and qualitative data collection and analysis will be presented in a half-day debrief workshop attended by district and local council leaders, donors and, program implementers. Feedback from the workshop will guide the finalizing of the draft report, which shall be shared between stakeholders such as USAID/FFP, UMEMS, and Mercy Corps to develop the final report. Evaluation Team The evaluation will be internally-led to maximize learning, but with a high level of participation from Mercy Corps’ global Deputy Director for DM&E. The Deputy Director for DM&E will bring global expertise to strengthen the evaluation design, methods, and overall execution, as well as provide a level of objectivity and critical analysis, being external to Mercy Corps Uganda. He will be the lead facilitator of the evaluation process along with the in-country evaluation team, consisting of James Kamukama (MC Uganda-DM&E Coordinator) and his cadre of M&E staff along with Tim Sparkman (MC Deputy Country Director) and Henry Huka￾Duba (MYAP Program Director). Grameen AppsLab staff will assist in providing mobile data collection and analysis technologies with associated training, and local enumerators will assist with data collection. Key Deliverables (Written) • Evaluation plan detailing methods, strategy, etc., building off the SOW • Development of appropriate data collection tools, configured in smartphones where appropriate. • Updated monitoring tools • Training content developed and delivered for data collectors and evaluation team • Data management and analysis solutions developed • Presentation of initial findings to program team • Draft evaluation report circulated for feedback • Final evaluation report according to format specified below Schedule/Timeframe - (June-September 2011) Below is a schedule of activities for the evaluation. It is the responsibility of the Deputy Director for Programs and the M&E Coordinator to ensure execution of these activities. Date Activity June 27 – July 29 • Selected reading materials shared amongst evaluation team • Desk review of documents 53 • Data entry and management solutions developed • Access to web-portal for designing surveys and managing incoming data • Design of data collection tools in conjunction with Grameen AppsLab August 1 • Lead evaluation facilitator (MC Deputy Director for DM&E) arrives in Uganda • Meet with Grameen (ApsLab) regarding mobile M&E tools and system • Review of detailed evaluation plan. August 2 • Evaluation team travel to Kitgum August 3-5 • Evaluation team and enumerator training: survey methodology, survey subject and use of smartphones • Field-test data collection tools August 6-23 • Fieldwork: Data collection (Acholiland) August 24-31 • Fieldwork: Data collection (Karamoja) September 1 -2 • Data cleaning September 3-9 • Preliminary data analysis and presentation of initial findings September 14 • Draft evaluation report completed. September 19 • Final evaluation report completed September 22 • Facilitate follow up workshop for action points from evaluation September 30 • Submission to USAID/FFP Reporting and Dissemination Requirements The final evaluation report will not exceed 40 pages not including attachments. The report shall be structured in accordance with the following guidelines4 : • Cover Page, List of Acronyms • Table of Contents which identifies page numbers for the major content areas of the report. • Executive Summary (2 to 3 pages) should be a clear and concise stand-alone document that gives readers the essential contents of the evaluation report in 2 or 3 pages, previewing the main points in order to enable readers to build a mental framework for organizing and understanding the detailed information within the report. In addition, the Executive Summary helps readers determine the key results and recommendations of the report. Thus, the Executive Summary should include: major lessons learned; maximum of two paragraphs describing the program, summary of targets and intended outcomes; 4 Taken from the Mercy Corps Baseline/Evaluation Scope of Work Template 54 areas of meaningful under or over achievement; and possibly a few lines describing the action plan developed to follow up on evaluation recommendations and how the evaluation report will be disseminated. • Methodology: sampling method including strengths and weaknesses of method used, inclusion of stakeholders and staff, rough schedule of activities, description of any statistical analysis undertaken, including justification and software package used. The discussion of any random sampling used should include details on how the random respondents were identified and invited to participate. This section should also address constraints and limitations of the evaluation process and rigor. The methodology section should also include a detailed description of data collection techniques used throughout the evaluation. • Results: Think about how best to organize this based on the evaluation questions. In some cases, it is helpful to organize the report against project objective, but in other cases it my make more sense to organize the report against evaluation questions. • Synthesis, Recommendations and Lessons Learned: This is space for the evaluation team to think about the data and results, and make concrete recommendations for current or future project improvements/changes, pull out organization lessons learned, and generally comment on data and results. Everything presented in this section must be directly linked back to the information presented in the Results section of the report. Ideally, items discussed here will not be completely new to the reader, but rather will refer to previous discussions. Recommendations that are not directly tied to Results can be included in an Evaluator Comments section for the report. • Annexes: data collection instruments in English and translation; list of stakeholder group with number and type of interactions; SOW, qualitative protocols developed and used, any data sets can be provided in electronic format, any required photos, participant profiles or other special documentation needed. For further guidance on report content, please see USAID’s Criteria to Ensure the Quality of the Evaluation Report in annex. Annex 1: Criteria to Ensure the Quality of the Evaluation Report5 • The evaluation report should represent a thoughtful, well-researched and well organized effort to objectively evaluate what worked in the project, what did not and why. • Evaluation reports shall address all evaluation questions included in the scope of work. 5USAID Evaluation Policy: http://www.usaid.gov/evaluation/ 55 • The evaluation report should include the scope of work as an annex. All modifications to the scope of work, whether in technical requirements, evaluation questions, evaluation team composition, methodology or timeline need to be agreed in writing by the technical officer. • Evaluation methodology shall be explained in detail and all tools used in conducting the evaluation such as questionnaires, checklists and discussion guides will be included in an Annex in the final report. • Evaluation findings will assess outcomes and impact on males and females. • Limitations to the evaluation shall be disclosed in the report, with particular attention to the limitations associated with the evaluation methodology (selection bias, recall bias, unobservable differences between comparator groups, etc.). • Evaluation findings should be presented as analyzed facts, evidence and data and not based on anecdotes, hearsay or the compilation of people’s opinions. Findings should be specific, concise and supported by strong quantitative or qualitative evidence. • Sources of information need to be properly identified and listed in an annex. • Recommendations need to be supported by a specific set of findings. • Recommendations should be action-oriented, practical and specific, with defined responsibility for the action. Annex 2: Updated IPPT Indicator District Unit Desired direction of change (+) or (-) Baseline Year 1 (FY 09) Year 2 (FY 10) Year 3 (FY 11) LOA Target Actual % met Target Actual % met Target Actual % met Target Actual % met Strategic Objective 1: Improved Access to Food Impact Indicator Kit/Pad Months + 9.3 10 8.9 89.0% 9.5 0.0% 0.0% 1.0 a: Months of Adequate Household Food Provisioning (MAHFP) (FFP) Kaab Months + 7 8 7.8 97.5% 8.5 0.0% 0.0% Impact Indicator Kit/Pad # + 3.8 5 3.6 72.0% 4.5 0.0% 0..0% 1.0 b: Household Dietery Diversity Score (HDDS) (FFP) Kaab # + 3 5 3.3 66.0% 4.5 0.0% 0.0% IR 1.1 Smallholder Farmers & Vulnerable Women Have Increased Food Production Monitoring Indicator Kit/Pad MT + 0 2,349 0 0.0% 2,800 5,245 187.3% 3,270 17,743 542.6% 35,819 22,988 64.2% 1.1 a:Volume of targeted agricultural commodities produced by USAID-assisted clients. (Mission) Kaab MT + 0 1,184 0 0.0% 2,057 6,506 316.3% 10,241 6,506 63.5% Monitoring Indicator Kit/Pad USD + - 869,479 0 0.0% 1,036,05 9 7,098,719 685.2% 1,209,967 12,045,450 995.5% 31,994,305 19,144,169 59.8% 1.1 b: Dollar Value of targeted agricultural commodities produced by USAID-assisted clients. (Mission) Kaab USD + - 438,080 - 0.0% 761,164 3,570,301 469.1% 5,099,744 3,570,301 70.0% Monitoring Indicator Kit/Pad % + - - - - 451 5,245 1163.0% 470 12,498 2659.1% 6,600 0 0.0% 1.1 c: Change in volume of targeted agricultural commodities produced by USAID-assisted clients(Mission) Kaab % + - - - - - - - 873 6,506 745.2% 4943 0 0.0% 57 Monitoring Indicator 1.1 d: Number of trainings and meetings held to build peace and resolve conflict among neighboring communities (Mission) Kaab # + 0 2 2 100.0% 3 4 133.3% 10 6 60.0% IR 1.11:Increased Access to Agricultural Inputs, Services and Markets Monitoring Indicator Kit/Pad # + 0 8,000 11,000 138.0% 20,000 17,891 89.5% 23,500 19,677 83.7% 92,500 48,568 52.5% 1.11a Number of vulnerable households benefiting directly from USG assistance (Mission) Kaab # + 0 200 1,529 764.5% 200 6,223 3111.5% 6,900 7,752 112.3 % Monitoring Indicator Kit/Pad # + 0 600 900 150.0% 800 556 69.5% 800 1,474 184.3% 3,800 2,930 77.1% 1.11 b: Number of participants in women gardening groups (Mission) Kaab # + 0 200 0 0.0% 200 600 300.0% 1,000 600 60.0% Monitoring Indicator Kit/Pad # + 0 63 65 103.0% 60 95 158.3% 37 0 0.0% 160 160 100.0 % 1.11 c: Number of producer organizations, water user associations, trade & business associated and CBOs receiving USG assistance (Mission) Kaab # + 0 10 0 0.0% 15 40 266.7% 70 40 57.1% Monitoring Indicator Kit/Pad % + 0 50 100 50.0% 70 100 142.9% 90% 92% 102.2% 100.0% 0 0.0% 1.11 d:Percent of assisted communities with improved community capacity as a result of project assistance (FFP) Kaab % + - 50 - 0.0% 60% 70% 116.7% 100.0% 0 0.0% Monitoring Indicator Kit/Pad # + 0 3000 2859 95.0% 9,460 7,785 82.3% 1500 1698 113.2% 15,160 12,342 81.4% 58 1.11 e: Number of recipients receiving seeds/tillage vouchers [1] (Mission) Kaab # + 0 4,088 - 0.0% 2000 1994 99.7% 8,088 1,994 24.7% Monitoring Indicator Kit/Pad # + 0 2 12 600.0% 2 12 600.0% 2 2 100.0% 22 26 118.2 % 1.11 f: Number of coordination meetings with government/research institutions(Mercy Corps) Kaab # + 0 2 5 250.0% 2 0 0.0% 12 5 41.7% Monitoring Indicator Kit/Pad Km + 0 18 25 139.0% 32 54 168.8% 100 116 116.0% 285 195 68.4% 1.11 h: Number of Kms of farm- to-market roads rehabilitated or constructed with USAID funded activities (Mission) Kaab Km + 0 15 15 100.0% 30 14 46.7% 75 29 38.7% Monitoring Indicator Kit/Pad # + 0 3 3 100.0% 5 6 120.0% 5 8 160.0% 18 0.0% 1.11 i: Number of communities that had improved infrastructure (FFP) Kaab # + 0 3 2 66.7% 3 3 100.0% 3 0.0% Monitoring Indicator Kit/Pad % + 0 50.0% 10.0% 20.0% 60.0% 30.0% 50.0% 70% 100% 142.9% 80.0% 0 0.0% 1.11 j: % of community infrastructure maintenance plans operational (Mission) Kaab % + - 50.0% 25.0% 50.0% 60% 49% 81.7% 70.0% 0 0.0% Monitoring Indicator Kit/Pad min - 143 100 89.8 111.4% 100 #DIV/ 0! 1.11 k: Average time to get to market (Mercy Corps) Kaab min - 143 120 117.4 102.2% 110 #DIV/ 0! Monitoring Indicator Kaab # + 0 40 - 0.0% 10 0 0.0% 60 0 0.0% 1.11 l: Number of Community Based Animal Health Workers trained and equipped with veterinary kits(Mercy Corps) IR 1.12 Environmentally sound improved agricultural practices adopted 59 Monitoring Indicator Kit/Pad # + 0 1860 2962 159.0% 2,000 5,791 289.6% 1500 6753 450.2% 16360 15506 94.8% 1.12 a: Number of individuals who have received USG support for short-term agricultural sector productivity training (Mission) Kaab # + 0 300 0 0.0% 500 1800 360.0% 2800 1800 64.3% Monitoring Indicator Kit/Pad % + 0 50.0% 73.0% 146.0% 50.0% 79.5% 159.0% 60.0% 57.1% 95.2% 65.0% 0.0% 1.12 b: % of beneficiaries (farmers) using project defined minimum number (3) of sustainable agriculture technologies. (FFP) Kaab % + 0 50.0% 0.0% 0.0% 60.0% 23.3% 38.8% 55.0% 0.0% Strategic Objective 2: Improved health and nutrition for pregnant/ lactating women and children under 5 Impact Indicator Kit/Pad % - 19.9% 16.9% 12.7% 133.1% 14.9% #DIV/ 0! 2.0 a:% Children 0-59 months underweight (<-2 Z score W/A (FFP) Kaab % - 24.0% 20.0% 24.0% 83.3% 18.0% #DIV/ 0! Impact Indicator Kit/Pad % - 30.7% 27.7% 33.2% 83.4% 28.0% #DIV/ 0! 2.0 b: % Children 6-59 months stunted (<-2 z score H/A (FFP) Kaab % - 34.0% 30.0% 43.0% 69.8% 30.0% #DIV/ 0! IR 2.1: Improved quality of maternal/child health and nutrition care delivered by district services Monitoring Indicator Kit/Pad # + 0 3750 3325 88.7% 5,000 9,033 180.7% 5000 6843 136.9% 27,750 19,201 69.2% 2.1 a: Number of children receiving Vit A in last 6 months (Mercy Corps) Kaab # + 0 1,250 - 0.0% 1250 4761 380.9% 5500 4761 86.6% Monitoring Indicator Kit/Pad # + 0 25 17 68.0% 40 85 212.5% 40 129 332.5% 205 157 76.6% 2.1 b: Number of health workers trained (Mercy Corps) Kaab # + 0 10 95 950.0% 10 195 1950.0% 120 290 241.7 % Monitoring Indicator Kit/Pad # + 0 11,490 17,244 150.0% 15,300 16,026 104.7% 15,300 12965 84.7% 71390 46235 64.8% 60 2.1 c: # beneficiaries receiving supplemental food rations (Mercy Corps) Kaab # + 0 8,100 4,806 59.3% 8,100 2509 31.0% 18700 7315 39.1% IR2.2: Improved health seeking behavior Monitoring Indicator Kit/Pad % + 69.0% - - - 73.0% - - 75.0% 71.5% 95.3% 80.0% 0.0% 2.2 a: % of caregivers of children 0-23 months with diarrhea last 2 weeks who delivered proper care (ORS +/or increased fluids) (Mercy Corps) Kaab % + 50.0% 55.0% - - 60.0% 95.1% 158.5% 95.0% 0.0% Monitoring Indicator Kit/Pad % + 75.0% - - - 77.0% - - 79.0% 58.0% 73.4% 75.0% 0.0% 2.2 b: % women seeking at least 4 ANC visits (Mercy Corps) Kaab % + 65.0% 68.0% - - 72.0% 64.5% 89.6% 70.0% 0.0% Monitoring Indicator Kit/Pad # + 0 24 79 329.0% 24 25 104.2% 24 24 100.0% 137 128 93.4% 2.2 c: # of community groups meeting per schedule (Mercy Corps) Kaab # + 0 20 19 95.0% 20 5 25.0% 65 24 36.9% Monitoring Indicator Kit/Pad % + 0 90.0% 73.6% 81.8% 90.0% 78.0% 86.4% 90.0% 67.0% 74.4% 90.0% 0.0% 2.2 d: % of children 0-59 in growth monitoring program gaining weight (FFP) Kaab % + 0 90.0% 84.0% 93.8% 90.0% 67.0% 74.4% 70.0% 0% 0.0% Monitoring Indicator Kit/Pad % + 28.0%* - - - 32.0% - - 34.0% 82.0% 241.2% 85.0% 0.0% 2.2 e: % children 0-6 months of age exclusively breastfed (FFP) Kaab % + 20.0%* 24.0% - - 28 30.1 107.5% 32.0% 0.0% Monitoring Indicator Kit/Pad % + - 33.0% 80.2% 243.0% 48.0% 48.9% 101.9% 55.0% 49.2% 89.5% 70.0% 0.0% 61 2.2 f: % of beneficiary children aged 6-23 months of age breastfeeding (Mercy Corps) Kaab % + 38.0% 40.0% 44.0% 110.0% 45.0% 48.6% 108.0% 55.0% 0.0% Monitoring Indicator Kit/Pad % + 11.0%* - - - 15.0% - - 17.0% 38.0% 224.7% 50.0% 0.0% 2.2 g. % of children 6-23 months of age with 3 appropriate Infant and Young Children Feeding practices (Continued breastfeeding, Age￾appropriate dietary diversity, and age-appropriate frequency of feeding) FFP Kaab % + 9.0%* 12.0% - - 15 32.2 214.7% 40.0% 0.0% IR2.3: Optimal dietary practices adopted Monitoring Indicator Kit/Pad % - 7.7%* 7.3% - - 6.9% 7.6% 90.8% 6.5% 7.4% 87.8% 6.2% #DIV/ 0! 2.3 a: % children 6-59 months in growth monitoring program who are wasted (<-2 Z scores W/H) (4) (FFP) Kaab % - 8.2%* 7.9% 8.1% 97.5% 7.5% 5.4% 138.9% 6.0% #DIV/ 0! Monitoring Indicator Kit/Pad % + 43.0% - - - 46.0% - - 49.0% 51.2% 104.5% 60.0% 0% 0.0% 2.3 b: % caregivers of children 6-23 months who reported feeding their children from at least 4 recommended food groups (Mercy Corps) Kaab % + 40.0% 43.0% - - 46.0% 36.1% 78.5% 45.0% 0% 0.0% Monitoring Indicator Kit/Pad % + - 50.0% 88.3% 176.6% 55.0% 0.0% 60.0% 38.2% 63.7% 55.0% 0% 0.0% 2.3 c: % of participants demonstrating increased nutritional knowledge (Mercy Corps) Kaab % + 40.0% 45.0% 0.0% 50.0% 17.0% 34.0% 25.0% 0% 0.0% 62 Strategic Objective 3: Increased Access to Water & Sanitation and Improved Hygiene Practices Impact Indicator Kit/Pad % - 70.0%* 61.0% 38.2% 159.7% 60.0% 3.0 a: % children 0-24 months with diarrhea in past 2 weeks (Mercy Corps) Kaab % - 76.0%* 70.0% 55.2% 126.8% 65.0% #DIV/ 0! IR 3.1 : Increased Availability of Safe Water Sources & Sanitation Facilities Impact Indicator Kit/Pad % + 72.0% 82.8% 36.0% 43.5% 70.0% 0% 0.00% 3.1 a: % households with year￾round access to safe water (Mercy Corps) Kaab % + 65.0% 72.0% 25.7% 35.7% 60.0% 0% 0.00% Impact Indicator Kit/Pad % + 7.4% 15.0% 81.2% 541.3% 70.0% 0% 0.00% 3.1 b: % population households adopting appropriate safe water behaviors (FFP) Kaab % + 7.4% 15.0% 57.1% 380.7% 60.0% 0% 0.00% Monitoring Indicator Kit/Pad % + - 20.0% - - 30.0% 50.0% 166.7% 40.0% 77.1% 192.8% 60.0% 0% 0.00% 3.1 c: % beneficiary households adopting appropriate safe water behaviors (Mercy Corps) Kaab % + 20.0% 30.0% 47.4% 158.0% 40.0% 61.4% 153.5% 65.0% 0% 0.00% Monitoring Indicator Kit/Pad # + 0 16 4 25.0% 14 12 85.7% 18 15 83.3% 53 31 58.50 % 3.1 d: # water points (Shallow & Spring) rehabilitated/ constructed by USG assistance (Missions) Kaab # + 0 4 1 25.0% 5 4 80.0% 10 5 50.00 % Monitoring Indicator Kit/Pad % + 0 50.0% 100.0% 200.0% 60.0% 57.0% 95.0% 60.0% 70.0% 116.7% 70.0% 0% 0.00% 3.1 e: % community water maintenance plans operational (Mission) Kaab % + 0 60.0% 0.0% 0.0% 60.0% 0.0% 0.0% 60.0% 0% 0.00% Impact Indicator Kit/Pad % + 19.0% 34.0% 80.4% 236.5% 80.0% 0% 0.00% 3.1 f: % people with access to hygienic latrines (Mercy Corps) Kaab % + 15.0% 25.0% 33.3% 133.2% 40.0% 0% 0.00% Impact Indicator Kit/Pad % + 42.0% 57.0% 93.3% 163.7% 80.0% 0% 0.00% 3.1 g: % people with access to using latrines (Mercy Corps) Kaab % + 32.0% 40.0% 56.6% 141.5% 70.0% 0% 0.00% 63 Monitoring Indicator Kit/Pad % + 42.0% 45.0% 13.7% 30.4% 50.0% - 55.0% 61.6% 112.0% 65.0% 0% 0.00% 3.1 h: % beneficiaries with access to using latrines (Mercy Corps) Kaab % + 32.0% 36.0% - 40.0% 58.8% 147.0% 65.0% 0% 0.00% Monitoring Indicator Kit/Pad # + 0 17 4 23.5% 25 23 92.0% 25 15 60.0% 124 42 33.90 % 3.1 i: # of institutional latrines rehabilitated/ constructed (Mercy Corps) Kaab # + 0 12 10 83.3% 16 3 18.8% 42 13 31.00 % IR 3.2: Healthy water , sanitation & hygiene practices adopted Monitoring Indicator Kit/Pad % + 6.0% 9.0% 6.1% 67.8% 11.0% 50.0% 454.5% 13.0% 71.1% 546.9% 80.0% 0% 0.00% 3.2 a: % of caregivers demontrating proper personal hygiene behaviors ( hand washing etc) (FFP) Kaab % + 6.0% 11.0% 20.6% 187.3% 13.0% 56.2% 432.3% 60.0% 0% 0.00% Monitoring Indicator Kit/Pad + 11.6% 13.6% 14.2% 104.4% 15.6% 50.0% 320.5% 17.6% 65.3% 371.0% 75.0% 0% 0% 3.2 b: % caregivers demonstrating proper food hygiene behaviors (FFP) Kaab % + 11.6% 13.6% 15.2% 111.8% 15.6% 53.6% 343.6% 65.0% 0% 0% Monitoring Indicator Kit/Pad % + 0.0% 50.0% - - 70.0% 80.0% 114.3% 80.0% 100.0% 125.0% 100.0% 0% 0% 3.2 c: % WASH committees meeting per schedule (FFP) Kaab % + 0.0% 70.0% 70.0% 100.0% 80.0% 100.0% 125.0% 100.0% 0% 0% Annex 3: List of Villages Sampled Sub-county Village HHs Selected Kitgum Matidi Bolkol central 9 Mulago B 9 Obyen East 7 Omato Dwe 11 Lagoro Laroo 13 Raokun 10 Vietnam 7 Latanya Abicinia 12 Angoko 9 Olam Icike 11 Namokora Odilang 10 Ogul 10 Omiya Anyima Abongole 8 Lanyap 8 Lwala 10 Obokolome 10 Orom Akilok south 14 Ladotonen west 7 Lagwerima 8 Lakore 11 Lalekan 11 Morolem 10 Pajule Kompetene 10 Lajul Owiny 7 Lami North 12 Wol Ayago 11 Lugila 9 Wili wili 9 65 Annex 4: Focus Group Discussions and Schedule District Subcounty Village Group type Date Kitgum Kitgum Matidi Lanyambira SFP women groups for malnourished beneficiary children under 5 10 Aug 11 Pader Latanya Abicinia SFP women groups for malnourished beneficiary children under 5 15 Aug 11 Kaabong Karenga Kakuruk SFP women groups for malnourished beneficiary children under 5 26 Aug 11 Kaabong Karenga Kangole south SFP women groups for malnourished beneficiary children under 5 29 Aug 11 Pader Latanya Abicinia SFP women groups for pregnant and lactating women beneficiaries 15 Aug 11 Kaabong Karenga Lokori central SFP women groups for pregnant and lactating women beneficiaries 26 Aug 11 Kaabong Karenga Kangole north SFP women groups for pregnant and lactating women beneficiaries 29 Aug 11 Kitgum Namokora Odilang Mother Care group members 12 Aug 11 Kitgum Orom Lakwore Mother Care group members 20 Aug 11 Kaabong Karenga Sarachom central Mother Care group members 25 Aug 11 Kaabong Kapedo Lomorunyangae Mother Care group members 1 Sep 11 Kitgum Lagoro Lakwor central Tillage service beneficiaries 11 Aug 11 Kitgum Omiya Anyima Obokolome Tillage service beneficiaries 13 Aug 11 Kaabong Karenga Naripwal Tillage beneficiaries 27 Aug 11 Kaabong Kapedo Napusimoru Tillage beneficiaries 31 Aug 11 Kitgum Pajule Lajul Owiny Producer group members 16 Aug 11 Kaabong Kapedo Lobalangit central Producer group members 27 Aug 11 Kaabong Kapedo Lakodope Producer group members 31 Aug 11 Kitgum Namokora Ogul Women Gardening Group beneficiaries 12 Aug 11 Kitgum Pajule Lajul Owiny Women gardening group members 16 Aug 11 Kaabong Karenga Sarachom central Women gardening group members 25 Aug 11 Kaabong Kapedo Lomorunyangae Women gardening group members 1 Sep 11 66 Annex 5: Key Informants Interviewed # Date Interviewer Key Informant name Position/Field District 1. 5 Sept 11 James/Andrew Okello James Okidi Senior CDO Kitgum 2. 5 Sept 11 James/Andrew Dominic Balu Probation/Welfare Officer Kitgum 3. 5 Sept 11 James/Andrew Abaru Peter DAO Kitgum 4. 6 Sept 11 James/Andrew Okumu Christopher Staff –Water and Sanitation Kitgum 5. 6 Sept 11 James/Andrew Pinyamoi David Staff-Agriculture Kitgum 6. 7 Sept 11 James/Andrew Olimocan James District Roads Inspector Kitgum 7. 7 Sept 11 James/Andrew Henry Huka Program Director Kitgum 8. 16 Sept 11 James/Andrew Elado Fred DAO Kaabong 9. 14 Sept 11 James/Andrew Ojok Raphael Tree nursery beneficiaries Kaabong 10. 14 Sept 11 James/Andrew Lokoria Phillip VHT member Kaabong 11. 16 Sept 11 James/Andrew Olila Ignatius Staff-Health & Nutrition Kaabong 12. 15 Sept 11 James/Andrew Echakan Peter CIC member Kaabong 13. 15 Sept 11 James/Andrew Olokwee CDO Kaabong 14. 16 Sept 11 James/Andrew Kibwota Godfrey Health Inspector Kaabong 15. 16 Sept 11 James/Andrew Rowland Operation Director Kaabong 16. 19 Sept 11 Jonathan Obol Peter LC I member Kitgum 17. 19 Sept 11 Jonathan Okongo Tom Tomas CIC member Kitgum 18. 19 Sept 11 Jonathan Opira Tadeo Tree nursery beneficiaries Kitgum 19. 20 Sept 11 Jonathan Owor Thomas CDO Pader 20. 20 Sept 11 Jonathan Ocitti Christopher District Roads Inspector Pader 21. 19 Sept 11 Samuel/Jonath an Okot Ajaman VHT member Kitgum 22. 16 Sept 11 James/Andrew Ojok James District Roads Inspector Kaabong 23. 14 Sept 11 James/Andrew Oluka Jairus Staff-Agriculture Kaabong 24. 23 Sept 11 Joe Stanley Lukenge M&E specialist-UMEMS Phone call 67 Annex 6: Household Survey Questionnaire MOBILE PHONE SURVEY-FINAL Household Questionnaire 1. Have you read informed consent to the respondent and the respondent agrees to be interviewed? 1. Yes 2. No 2. District: 1.Kitgum 2.Pader 3.Kaabong 3. Subcounty: 1. Namokora 2. Kitgum-Matidi 3. Orom 4. Omiya Anyima 5. Lagoro 6. Wol 7. Pajule 8. Latanya 9. Karenga 10. Kapedo 4. Parish: 1. Pugoda 2. Kalabong 3. Paibony 4. Lumule 5. Kitenyi 6. Okuti 7. Lolwa 8. Katwotwo 9. Melong 10. Palwo 11. Lakwor 12. Laber 13. Rogo 14. Ogole 15. Palenga 16. Otok 17. Awere 18. Oyuku 19. Karenga Central 20. Lobalangit 21. Lokori 22. Pire 23. Kawalakol 24. Lokial 68 25. Sangar 5. Village name: ___________________________________________ 6. Name of Main respondent: ________________________________ 7. Sex: 1.Male 2.Female 8. Age of mother/caregiver (years)____________________________ 9. Highest level of education: 1. None 2. Primary 3. Secondary 4. Tertiary 10. What do you do for a living/occupation? 1. Crop farming 2. Poultry farming 3. Livestock rearing 4. Bee keeping 5. Trader in agro products 6. Charcoal burning and/or sale 7. Stone/masonry and/or sand quarrying 8. Brick laying 9. Others(please specify) FOOD ACCESS AND AVAILABILITY 11.I would like to ask you about the various foods that you or anyone else in your household ate yesterday during the day and night. Hint: For cases where yesterday night and day were not “usual/normal” days e.g a party/feast, funeral OR if most household members mentioned above were absent, then choose another day for reference during this section. 1. I understand, let us go ahead and read me the food types. My response will be yes or no 2. I do not understand, please elaborate before we continue. [Note: Read the list of foods and listen to response. Differentiate between statements that mean no or yes]. 12.[CEREALS] Any bread, rice, biscuits, millet, sorghum, maize, wheat or posho 1.Yes 2.No 13.[ROOT & ROOT TUBERS] Any potatoes, yams, cassava or any other foods made from roots or tubers? 1.Yes 2.No 14.[VEGETABLES] Any vegetables? (e.g Bbo, Malakwang, alolot, Sukuma, Otigo Odii) 1.Yes 2.No 15.[FRUITS]Any fruits? 1.Yes 2.No 16.[MEAT POULTRY, OFFAL] Any beef, pork, lamb, goat, rabbit, game meat, chicken, duck or other birds, liver, kidney, heart or other organ meats? 1.Yes 69 2.No 17.[EGGS]Any eggs? 1.Yes 2.No 18.[FISH & SEA FOODS] Any fresh or dried fish or Small fish? 1.Yes 2.No 19.[PULSES, LEGUMES, NUTS] Any foods made from beans, peas, lentils or nuts? 1.Yes 2.No 20.[MILK & MILK PRODUCTS] Any cheese, yogurt, milk or other milk products? 1.Yes 2.No 21.[OILS/FATS] Any foods made with oil, fat, or butter? (E.g Oddi and Mooya) 1.Yes 2.No 22.[SUGAR/HONEY] Any sugars (e.g sugar cane) or honey? 1.Yes 2.No 23.[MISCELLANEOUS/OTHERS] Any other foods, such as spices/condiments, coffee or tea? 1.Yes 2.No 24.In the past 12 months, were there months in which this household did not have access to sufficient food to meet household needs? (HINT: Answer Yes/No, basing on access to food regardless of if its purchased, exchanged/barter or own produced. Answer yes household did not have to compromise number of meals, quantity of food consumed or going without food, sending away some family members and did not consume inferior foods). 1. Yes 2. No 25.If Yes, which are these months in which this household did not get sufficient food [Hint: E.g Household compromised normal number and quantity of meals, consume foodstuffs considered to be for the less well-off]. 1. July 2. June 3. May 4. April 5. March 6. February 7. January 8. December 9. November 10. October 11. September 12. August AGRICULTURAL MARKET ACCESS 26.Is there a market that you frequently go to buy or sell agriculture produce or inputs? 1.Yes 2.No 27.What are the various means that you have used to go to this market in the past 12 months 1.Foot 70 2.Bicycle 3.Motorcycle 4.Car/public transport 28.And which of them have you frequently used 1.Foot 2.Bicycle 3.Motorcycle 4.Car/public transport 29.Using the most frequent means, how long does it take you to reach the above market? [HINT: Record 1 way duration and in minutes]_________________________ AGRICULTURAL TECHNOLOGY ADOPTION 30.At the moment or during previous cropping season, did/do you practice the following on your farm [HINT: This does not include communally operated plots e.g demo plots] 1. Mulching 2. Line planting/spacing 3. Intercropping 4. Crop rotation 5. Intercropping 6. Fallowing 7. Zero tillage 8. Plant trees 9. Construct trash lines or soil bunds 10. Not Applicable PRENATAL CARE [For mothers of children 0-59 months] 31. Relationship of main respondent to youngest household member [multiple select] 1.Mother 2.Caregiver 3. Relative 4.Brother/sister 5.Grandparent 32.Did you see anyone for prenatal care while you were pregnant with (YOUNGEST CHILD’S NAME? 1.Yes 2.No 33.Who did you see for this prenatal care? 1. Doctor 2. Nurse/Midwife 3. Auxiliary midwife 4. Traditional birth attendant 5. Community Health Worker/VHT 6. Not Applicable 34.How many times did you see a health worker while you were pregnant? [Hint: Number]_____________________________ PREVENTION OF MALNUTRITION (for caregivers of children 0-59 months) 35.For caregivers of children 0-59 months, ask them to mention different ways of preventing children outlined from getting malnourished 1. Prolonged breastfeeding 71 2. Exclusive breastfeeding 3. Use clean hands when feeding them 4. Diversify diet 5. Continuous feeding whenever sick 6. More frequent feeding BREAST FEEDING PRACTICES 36. For the children 0-6 months, if any, do you currently feed them on breast milk only (Note that taking of prescribed medicine by medical doctor is okay)? 1.Yes 2.No 3.Not Applicable 37. For any children between 6-23 months, did you breastfeed them over the last 24 hours? 1.Yes 2.No 38. Now i would like to ask you about liquids or foods (YOUNGEST CHILD’S NAME) drunk/ate yesterday during the day or at night. 1. Ok. Go ahead and read the list of liquids you want to know. My response will be yes or no or Not sure 2. I don’t understand, please elaborate before we continue 39.0a: Breast milk 1.Yes 2.No 40.0b: Plain water 1.Yes 2.No 41.0c: Commercially produced infant formula 1.Yes 2.No 42.0d: Any fortified, “commercially available infant and young child food” [e.g. Cerelac] 1.Yes 2.No 43.0e: Any (other) porridge or gruel 1.Yes 2.No 44. Now i would like to ask you about (other) liquids or foods that (YOUNGEST CHILD’S NAME) may have eaten/drunk yesterday during the day or at night. I am interested in whether the child had any of them even if it was combined with other foods. 1. Ok. Go ahead and read the list of liquids you want to know. No need to ask me again if I have already answered same above, Just Tick. My response will be yes or no 2. I don’t understand, please elaborate before we continue 45.1a: Commercially produced infant formula 1.Yes 2.No 46.1b: Milk such as tinned, powdered, or fresh animal milk 1.Yes 2.No 47.1c: Cheese, yogurt, or other milk products 1.Yes 2.No 72 48.2d: Any fortified, commercially available infant and young Child food (e.g. Cerelac) 1.Yes 2.No 49.2e: Any (other) porridge or gruel 1.Yes 2.No 50.2f: Bread, rice, noodles, or other foods made from grains 1.Yes 2.No 51.2g: White potatoes, white yams, cassava, or any other foods made from roots 1.Yes 2.No 52.3h: Pumpkin, carrots, squash, or sweet potatoes that are yellow or orange inside 1.Yes 2.No 53.3i: Any dark green leafy vegetables 1.Yes 2.No 54.3j:Ripe mangoes, papayas 1.Yes 2.No 55. 3k: Foods made with red palm oil, palm nut, palm nut pulp sauce, sesame oil, Moya/shear nut, sunflower 1.Yes 2.No 56. 4l: Any other fruits or vegetables like oranges, grapefruit or pineapple 1.Yes 2.No 57. 5m: Eggs 1.Yes 2.No 58. 6n: Liver, kidney, heart or other organ meats 1.Yes 2.No 59. 6o: Any meat, such as beef, pork, lamb, goat, chicken, or duck 1.Yes 2.No 60. 6p: Fresh or dried fish or small fish 1.Yes 2.No 61. 6q: Grubs, snails, insects (white ants, grasshopper), other small protein food 1.Yes 2.No 62. 7r: Any foods made from beans, peas, lentils, or nuts? 1.Yes 2.No 63. 8s: Any oils, fats, or butter, or foods made with any of these? (e.g Oddi and Moya) 1.Yes 2.No 64. 9u: Tea or coffee? 1.Yes 73 2.No 65. 9v: Any other liquids? 1.Yes 2.No 66. 9w: Any sugar, Honey or sugary foods, such as chocolates, candy, sweets, pastries, cakes, or biscuits? 1.Yes 2.No 67. 9x: Any other solid or soft food? 1.Yes 2.No 68. How many times did (YOUNGEST CHILD’S NAME) eat solid, semi-solid, or soft foods other than liquids yesterday during the day or at night? ________________________ [Hint: Consider yesterday and recall the number of times the youngest child ate to be full. Don’t include thin soups or broth, watery gruels, small snacks or 1-2 bites s/he receives from mother or any other family member. Enter 7 if number of times is greater than 7] WATERBORNE DISEASES AND MANAGEMENT 69.Has any of the children (0-23 months) had diarrhea in the last 2 weeks? 1.Yes 2.No [Hint: Diarrhea is the passage of watery stool for 3 or more times a day with or without blood. Includes watery stools, bloody stools (consistent with dysentery)] 70.If yes, did you do anything to treat the diarrhea? 1.Yes 2.No 3.Not Applicable 71.If yes, what did you do or give to the child? 1.Fluid from ORS sachet 2.Home fluid formula (from sugar and salt) 3.Pill or Syrup: Zinc [show local packaging for Zinc to confirm with caregiver) 4.Injection 5.Intravenous (IV) injection 6.Herbal medicine 7.Not Applicable 72. Where did you seek treatment the last time any of the 0-23 months child had diarrhea 1) Health centre/hospital/clinic 2) Traditional healer 3) VHT 4) Not applicable 73. For how long was this child sick before you sought treatment?________________ [Hint: record complete days] WATER & SANITATION 74. Do you have soap in your house? 1.Yes 2.No 75. Have you used soap today or yesterday? 1.Yes 2.No 74 76. If yes, what did you use it for?[Multiple response, Do not read the options] 1. Washing Clothes 2. Washing My Body 3. Washing My Children 4. Washing Child’s Bottom 5. Washing Child’s Hands 6. Washing Hands After Defecating 7. Washing Hands After Cleaning Child 8. Washing Hands Before Feeding Child 9. Washing Hands Before Preparing Food 10. Washing Hands Before Eating 11. Not Applicable 77. What is the main source of drinking water for members in this household? 1. Surface Water (River, Stream, Dam, Lake, Pond, Canal, Irrigation Canal) 2. Unprotected Spring 3. Rain harvesting/tapping 4. Open well (In House, in Compound, outside Premises) 5. Tanker truck 6. Shop or market (Bottled water) 7. Covered Well Or Boreholes (In House, in compound, outside Premises) 8. Protected spring 9. Tap Water in house 78. For this main source, how long does it take you get the water for this household____________________________ [Hint: Consider one way & record in minutes] 79.What months during the year do you have access to water from this source? 1. January 2. February 3. March 4. April 5. May 6. June 7. July 8. August 9. September 10. October 11. November 12. December 80. What is the main source of cooking and washing water for members in this household? 1. Surface Water (River, Stream, Dam, Lake, Pond, Canal, Irrigation Canal) 2. Unprotected spring 3. Rain harvesting/tapping 4. Open well (In House, in Compound, outside Premises) 5. Tanker truck 6. Shop or market (Bottled water) 7. Covered Well Or Boreholes (In House, in compound, outside Premises) 8. Protected spring 9. Tap Water in house 81. Do you treat your water in any way to make it safer to drink? 75 1. Yes 2. No 82.If yes, what do you usually do to make the water safer to drink? 1.Boil 2.Add bleach 3.Add Aquatab 4.Use ceramic filter 5.Use bio-sand filter 6.Solar disinfection 7.Let it stand and settle 8.Not applicable 83. How do you store your drinking water? 1.Containers (bucket, Jerry can, Jerkin bottle, Drum, etc Clay pots) 2.Roof tank or cistern 3.Do not store any water specifically for drinking 4.Not Applicable 84. Type of containers if mentioned above [Hint: Ask to observe. Narrow mouthed containers are 3cm diameter or less] 1. Narrow mouthed 2. Wide mouthed 3. Both types 4. No applicable 85. Are the drinking water containers covered? [Hint: Ask to observe] 1.All are covered with hard covers 2.Some are covered with hard covers 3.All are covered with soft covers (e.g. cloth) 4.None is covered 5.Not Applicable SANITATION AND HYGIENE 86. Where do you go for defecation 1.Pit Latrine with slab 2.Ventilated Improved pit latrine 3.Bucket 4.Open pit (no slab) 5.Bush/field 6.Compositing toilet 87. If latrine/toilet, do you have access to it at least once in a normal day? 1.Yes 2.No 3.Not applicable 88. If you do not have access once a day, why do you not? 1. Toilet is far from home 2. Not always have permission to use it 3. It gets congested with users 4. Not applicable 5. Others (specify)……….. 89.Is there fecal matter present inside the facility on the slab, floor or walls? Hint: Ask to observe before selection] 1.Yes 76 2.No 3.Not Applicable 90.Do you wash your hands after defecating? 1.Yes 2.No 91.Under what other circumstances do you wash your hands? 1. after cleaning a child 2. before preparing food 3. before eating 4. before feeding a child 5. Not applicable 92. Location of hand washing facility/activity [Hint: Ask to see and note accordingly] 1.Inside/near toilet facility 2.Inside/near kitchen/cooking place 3.Elsewhere in yard 4.Outside yard 5.No specific place 6.Not able to see 93. Is there a hand washing device such as a tap, basin, bucket, sink, or tippy tap? [Hint: This item should be either in place or brought by the interviewee within one minute. If the item is not present within one minute check no, even if brought out later] 1.Yes 2.No 94. Are the hand washing water containers covered? [Hint: Ask to see] 1. All are covered with hard covers 2. Some are covered with hard covers 3. All are covered with soft covers (e.g cloth) 4. None is covered 5. Not applicable 95. Is there water in the containers? [Hint: Ask to observe. Turn on tap and/or a check container and note if water is present This item should be either in place or brought by the interviewee within one minute otherwise check no]. 1.Yes 2.No 96.Is there soap or detergent or locally used cleansing agent? [Hint: Ask to observe. Turn on tap and/or a check container and note if there is water. This item should be either in place or brought by the interviewee within one minute otherwise check no]. 1.Yes 2.No 97. Do you wash food before cooking it [Tip: Together with proper coverage and storage of food, indicate proper food hygiene behavior] 1.Yes 2.No Program performance assessment 98. Have you participated in activities or received support from Mercy Corps? [Yes/No] [Hint: Emphasize MYAP: Started in Kitgum/Pader in 2008 & Kaabong in 2009] 1.Yes 77 2.No 99. What support did you receive /what Mercy Corps activities did you participate in? [Read out] [Hint: Emphasize MYAP activities: Started in Kitgum/Pader in 2008 & Kaabong in 2009] 1. Through Mother care group or VHT 2. Health promotion/awareness participant 3. Water user committee member 4. Women gardening group member/participant 5. Member of Producer group member/participant 6. Tree nursery beneficiary 7. Seed recipient/seed fair participant 8. Supplementary Feeding Programme-Pregnant & lactating 9. Supplementary Feeding Programme-Malnourished 10. Latrine slab recipient 11. Tillage service beneficiary 12. FFW beneficiary 13. Not Applicable 100. Have you thanked the respondent 1) Yes 2) No 101. How many people are CURRENTLY in this household_______________________ 102. How many of the members are below 5 years____________________________ 103. ANTHROPOMETRY [For children 0-59 months] [Hint: Make use of child health card if available] 104. Name of child (Start with oldest)________________________[HINT: Start with oldest] 105. Relationship to the household Head: 1) Child 2) Grandchild 3) Nephew/Niece 4) Brother/Sister 5) Other relative 106. Sex of child (Male, Female) 1) Male 2) Female 107. Date of Birth (DD/MM/YYY)_________________________ 108. Age [HINT: Use age calendar. Record only completed months]___________________ 109. Edema [Press both feet gently] 1) Yes 2) No 110. Height or length in cm [HINT: record to nearest 0.1cm]________________ 111. Weight in kg [HINT: record to the nearest 0.1kg]______________________ 112. Interview location [Take GPS. Avoid satellite obstructions]______________ 78 Annex 7: Focus Group Discussion Guide6 Focus Group Discussion – Interview Guide Group type: Village: __________________________ Enumerators: __________________________ Date: __________________________ Instructions: Explain Mercy Corps’ objectives with mid-term evaluation and the purpose of this focus group in gathering information to assess project performance and recommend improvements. Pass around attendance sheet to record names. 1. How has the overall context and living situations changed within the last 2 years? Especially relating to security, food, health and agricultural production. Do you feel that MC program activities are still relevant to problems faced by the community? 2. In general, how is the quality of training and support you receive from Mercy Corps? For example, are the trainings well implemented and do the staff have the necessary expertise? Is there sufficient follow-up? How have you applied the skills learned? 3. [Activity-specific questions]? 4. Divide up into pairs, and have each pair discuss for 5-10 minutes: What is the most significant change brought about by your participation in this project? (Ask for specific things, can be HH or community level). Then have each pair share and discuss with the group. Pairs Most Significant Change Stories 5. How could MC adapt its programming to encourage more of these kinds of successful impacts? What can other stakeholders do to encourage more of these impacts? 6. In your view, what have been some of the project’s main challenges or constraints? (External and internal.) How do you feel Mercy Corps has responded to these challenges? How could they improve on these challenges in the future? Challenges How to Improve 7. How is the coordination with local leaders and government? 8. What is the gender balance of beneficiary involvement in program activities? 9. Anything else you’d like to share or comment on that is relevant to this evaluation? 6 This is the generic Focus Group Discussion Guide. Additional questions were added for FGD with each of the following groups: producer groups, women’s gardening groups, mother support groups, tillage beneficiaries, supplementary feeding groups.