Final Performance Evaluation of Northern Karamoja Growth, Health, and Governance Development Food Assistance Project Final Evaluation Report [Add Date] January 2019 This publication was produced at the request of the United States Agency for International Development. It was prepared independently by ICF Macro, Inc. Final Performance Evaluation of Northern Karamoja Growth, Health, and Governance Development Food Assistance Project Final Evaluation Report Submission Date: 01/25/2019 Submitted by: ICF Macro, Inc. 530 Gaither Rd, Suite 500 Rockville, MD 20850 Contract Number: GS-00F-189CA/7200AA18M00002 Contributors: Patricia Vondal, PhD – Team Leader Anna Afferri, MPH – Maternal, Child Health and Nutrition Specialist Agnes Nayiga Lugwana, Agriculture and Food Security Specialist Frank Emmanuel Muhereza, PhD – Conflict and Governance Specialist Ramu Bishwakarma, PhD – Lead Analyst Benita O’Colmain, PhD – Senior Survey Methods Specialist Photo Credits: Anna Afferri DISCLAIMER The author’s views expressed in this publication do not necessarily reflect the views of the United States Agency for International Development or the United States Government. Final Performance Evaluation of Northern Karamoja Growth, Health, and Governance Development Food Assistance Project – Final Evaluation Report i CONTENTS Acronyms................................................................................................................................................................................................. iv Executive Summary................................................................................................................................................................................ vi 1. Introduction ...................................................................................................................................................................................1 1.1 Project Background............................................................................................................................................................1 1.2 Theory of Change...............................................................................................................................................................2 2. Evaluation Overview....................................................................................................................................................................3 2.1 Evaluation Purpose .............................................................................................................................................................3 2.2 Design of the Performance Evaluation ..........................................................................................................................3 2.3 Evaluation Questions.........................................................................................................................................................3 3. Evaluation Sample Design, Data Collection and Analysis Methods, and Limitations ..................................................5 3.1 Sample Design......................................................................................................................................................................5 3.1.1 PBS Sample Design for Collection of Quantitative Data...........................................................................5 3.1.2 Sample Design for Collection of Qualitative Data......................................................................................6 3.2 Data Collection Methods.................................................................................................................................................6 3.2.1 Quantitative Data Collection ...........................................................................................................................6 3.2.2 Qualitative Data Collection ..............................................................................................................................7 3.2.3 Quantitative Data Analysis................................................................................................................................7 3.2.4 Qualitative Data Analysis Methods.................................................................................................................8 3.2.5 Integrating Findings from Quantitative and Qualitative Data...................................................................8 3.3 Limitations of the Evaluation Methodology..................................................................................................................9 4. Evaluation Findings .....................................................................................................................................................................10 4.1.1 SO1: Livelihoods Strengthened.....................................................................................................................................10 4.1.1 EQ1: To what extent has GHG met its defined goals, purposes, and outcomes for SO1?...........10 4.1.2 Strengths and Challenges of the Project Design, TOC, and Implementation Methods That Influence the Effectiveness of Technical Interventions in Achieving SO1 Goals .............................................17 4.1.3 EQ4. Which project outcomes are likely to be sustained? ....................................................................27 4.1.4 EQ5: What are the key lessons learned and best practices that should inform future projects in the country?.......................................................................................................................................................................29 4.2 SO2: Nutritional Status of Children Under Age Five Improved ..........................................................................30 4.2.1 EQ1: To what extent has GHG met its defined goals, purposes, and outcomes? ...........................30 4.2.2 Strengths and Challenges of the Project Design, TOC, and Implementation Methods That Influence the Effectiveness of Technical Interventions in Achieving SO2 Goals .............................................37 4.2.3 EQ4: Which project outcomes are likely to be sustained? ....................................................................47 4.2.4 EQ5: What are the key lessons learned and best practices that should inform future projects in the country?..................................................................................................................................................................50 4.3 SO3: Reduced Incidence of Conflict............................................................................................................................52 4.3.1 EQ1: To what extent has GHG met its defined goal, purposes, and outcomes for SO3?.............52 4.3.2 Strengths and Challenges of the Project Design, TOC, and Implementation Methods that Influenced the Effectiveness of Technical Interventions to Achieve SO 3 ........................................................55 4.3.3 EQ4. Which project outcomes are likely to be sustained? ....................................................................62 Final Performance Evaluation of Northern Karamoja Growth, Health, and Governance Development Food Assistance Project – Final Evaluation Report ii 4.3.4 EQ5: What are the key lessons learned and best practices that should inform future projects in the country?..................................................................................................................................................................67 5. Recommendations......................................................................................................................................................................69 5.1 SO1: Livelihoods Strengthened.....................................................................................................................................69 5.2 SO2: Improved Nutritional Status of Children Under Five ...................................................................................70 5.3 SO3: Reduced Incidence of Conflict............................................................................................................................72 Annexes Annex 1: Evaluation Statement of Work Annex 2: Uganda Joint Baseline/Endline Population-Based Survey Protocol Annex 3: GHG Final Evaluation Protocol Annex 4: Type and Number of GHG PE Respondents across Districts, Sub-Counties and Villages Annex 5: GHG Population Characteristics and Tabular Summary of PBS Indicators Annex 6: Sources of Information Annex 7: Logistic Regression Results Annex 8: Disclosure of Any Conflicts of Interest Final Performance Evaluation of Northern Karamoja Growth, Health, and Governance Development Food Assistance Project – Final Evaluation Report iii FIGURES Figure 1: Malnutrition in Children under 5 Years of Age ...........................................................................................................32 Figure 2: GHG SO2 Revised Theory of Change...........................................................................................................................38 Figure 3: Reasons Why People Do Not Use Latrines.................................................................................................................45 TABLES Table 1: Data Collection Sites by District........................................................................................................................................6 Table 2: Performance Indicators Measuring Livelihood and Food Security...........................................................................10 Table 3: Characteristics of the Selection Process by District and Village ..............................................................................13 Table 4: Number of Vulnerable Households Benefitting Directly from USG .......................................................................14 Table 5: Performance Indicators for SO2.......................................................................................................................................30 Table 6: FFP Performance Indicators for Health and Hygiene and Sanitation Practices ....................................................30 Table 7: Types of ORT Treatments.................................................................................................................................................31 Table 8: Prevalence of Malnutrition of Children under 5 Years of Age and Percentage of Children 6–23 Months of Age Receiving MAD by Household Consumption Quintiles......................................................................33 Table 9: Prevalence of Underweight Women and WDDS by Household Consumption Quintiles, 2018 ....................33 Table 10: Food Ration Distribution and Adoption of Health and Nutrition Practices.......................................................36 Table 11: GHG Support to Health Units and Community-based Staff...................................................................................39 Table 12: SO3 GHG Custom Performance Indicator.................................................................................................................53 Final Performance Evaluation of Northern Karamoja Growth, Health, and Governance Development Food Assistance Project – Final Evaluation Report iv ACRONYMS AEA agricultural extension agent AEW agriculture extension worker ALC area land committee ANC antenatal care AWOTID Abim Women Organized Together in Development BCC behavior change communication CAHW community animal health worker CHAST Child Hygiene and Sanitation Training CBO community-based organization CLTS community-led total sanitation DFAP development food assistance project DFSA development food security activity EBF exclusive breastfeeding EQ evaluation question FEWS NET Famine Early Warning Systems Network FFP Office of Food for Peace FGD focus group discussion FP family planning GHG Growth, Health, and Governance HDDS household dietary diversity score HPMA hand pump mechanics association HUMC Health Unit Management Committee IDI in-depth interview IPTT Indicator Performance Tracking Table IR intermediate result KAPDA Kaabong Peace for Development Agency KII key informant interview KOPEIN Kotido Peace Initiative LC3 Local Council Level 3 LDU Local Defense Unit LFPV Land Focal Point Volunteer LM leader mother Final Performance Evaluation of Northern Karamoja Growth, Health, and Governance Development Food Assistance Project – Final Evaluation Report v LOA life of activity M4P Making Markets Work for the Poor MAD minimum acceptable diet MC Mercy Corps MCA male change agent MCG mother care group MCHN maternal and child health and nutrition MUAC mid-upper arm circumference NGO nongovernmental organization ODF Open Defecation Free O Operation (free seeds) ORT oral rehydration therapy PBS population-based survey PLW pregnant and lactating women PM2A Preventing Malnutrition in Children Under Two Approach QET qualitative evaluation team SACCO savings and credit cooperative SGBV sexual and gender-based violence SO strategic objective TOC theory of change UPDF Uganda People’s Defense Forces USAID United States Agency for International Development USG U.S. Government VHT village health team VIP ventilated improved pit VSLA Village Savings and Loan Association WASH water, sanitation, and hygiene WDDS women’s dietary diversity score WFP World Food Programme WUC Water User Committees Final Performance Evaluation of Northern Karamoja Growth, Health, and Governance Development Food Assistance Project – Final Evaluation Report vi EXECUTIVE SUMMARY EVALUATION PURPOSE AND EVALUATION QUESTIONS The performance evaluation of the U.S. Agency for International Development (USAID) Office of Food for Peace (FFP) Growth, Health, and Governance (GHG) project is intended to present the outcomes of this project over five years of implementation and to examine the extent to which key FFP goals were met. Drawing on the household survey data from the population-based survey (PBS) baseline and endline findings and qualitative findings based on fieldwork, the report provides some of the key factors that contributed to GHG’s performance, an assessment of the sustainability of outcomes that were achieved, lessons learned, and recommendations for moving forward. A central aim of this final performance evaluation is to contribute to the knowledge base of FFP activity design. The audience for this report includes USAID FFP staff in Washington, DC, FFP activity managers in USAID/Uganda, Mercy Corps (MC) and its key implementing partners, and the Government of Uganda. PROJECT BACKGROUND The GHG development food assistance project (DFAP) was implemented by MC and its partners, Whave, Abim Women Organized Together for Development, Kaabong Peace for Development Agency, the Feinstein International Center, and World Vision, between 2012 and 2018 in the chronically food insecure and impoverished districts of Northern Karamoja, a remote rural area bordering Sudan and Kenya. The districts of Abim, Kotido, and Kaabong in Northern Karamoja are among the most underdevelopment areas in Uganda, in part because of the long history of violent, armed conflict. Safety of movement outside one’s own manyatta began improving after the forced disarmament of people in Northern Karamoja in 2010. As violent conflict declined, large areas of the region that were previously dangerous to enter became safer, allowing the population to begin reclaiming their space for economic and cultural activities. MC designed GHG to take advantage of this peace dividend and began implementing technical interventions to increase food security and the nutritional status of children under 5 years of age, as well as activities to solidify and extend the security of the region. EVALUATION QUESTIONS, DESIGN, METHODS, AND LIMITATIONS The performance evaluation for GHG used a mixed-method design based on the collection and analysis of quantitative and qualitative data. PBS endline data on important FFP impact and outcome-level indicators were collected in 42 villages and 1,260 households in the project implementation area (districts of Abim, Kotido, and Kaabong) from June 7 to July 6, 2018. Statistical findings were developed based on the comparison of PBS baseline (2013) and endline (2018) data and the significance of change from baseline in indicators measuring food security, poverty, nutrition, and security. Qualitative data were collected during fieldwork from September 23 to October 22 in the same districts to provide interpretations and explanations of the statistical findings and to deepen the understanding of factors related to key outcomes. Qualitative data were also collected to investigate additional FFP evaluation questions regarding the main factors that prohibited or promoted the achievement of those outcomes, how the project theory of change and design influenced the effectiveness of the technical interventions used to achieve the project goal and key outcomes, how the methods and process used to implement those interventions contributed to those outcomes, the sustainability of achievements to date, and best practices and lessons learned. Key informant interviews, focus group discussions, and in-depth interviews were conducted, tailored to key participants involved in each of the GHG strategic objectives, in a purposefully selected sample of six sub-counties and six villages. Specifically, interviews were conducted in two sub-counties in each of the three districts, and in a single village in each of the two sub-counties per district (see Section 3.2). Qualitative data analysis involved a content analysis of interview data to develop categories of responses, and the use of a triangulation process to identify Final Performance Evaluation of Northern Karamoja Growth, Health, and Governance Development Food Assistance Project – Final Evaluation Report vii factors related to outcomes, and to strengthen findings through a comparison of responses between sector-specific related actors at the district, sub-county, and village levels, as well as across villages. Mixed-method analytical techniques were employed to use the findings from the qualitative data to interpret the findings from the analysis of PBS data. PBS data were used to strengthen findings from the qualitative analysis for other FFP evaluation questions. Indicator Performance Tracking Table (IPTT) data and other sources were also used to strengthen findings where appropriate (see Section 3). Limitations of the evaluation methodology and findings include the following: (1) potential recall issues and respondent confusion between GHG and Apolou activities. GHG activities ended in January 2018, and MC began implementing the new development food security activity, Apolou, shortly after (Apolou continues the implementation of FFP activities in the majority of villages in Kotido and Kaabong); (2) lack of community-level data on maternal and child health and nutrition (MCHN) and water, sanitation, and hygiene (WASH) activities in two villages in Abim because of site selection restrictions with the change of implementers in this district; (3) lack of comparative data between 2013 and 2018 on participation in value chain activities; and (4) a difference in the months in which PBS data were collected (June–July 2018 for endline data and February–April 2013 for baseline data). Details on these limitations are found in Section 3.2 of this report. FINDINGS AND CONCLUSIONS This section provides a synthesis of the major findings and conclusions for each of the GHG strategic objectives (SOs). SO1: Improved Livelihoods The livelihoods of the general population of Karamojong people improved between 2013 and 2018, as demonstrated by changes in poverty indicators used to measure SO1. Per capita expenditures increased, from $0.58 to $0.92 in assisted areas. The percentage of people living on less than $1.25 per day decreased, from 93.2 percent to 87 percent; and the depth of poverty decreased, from 61.5 percent to 53.3 percent. The average household dietary score (HDDS) increased, from a score of 2.2 to a score of 3.6. This proxy indicator for strengthened livelihoods measures the ability of the population to consume a greater diversity of food items that contribute to better health and nutrition through purchases at the market or through their own production. Disaggregation of these data by gendered household shows that overall achievements in reducing poverty were driven by households with an adult female and an adult male. There was no change in the poverty status among households with adult females and no adult males. This category of households includes women in polygamous relationships in which the husband is not domiciled in the household. Analysis of qualitative data from interviews conducted during September and October 2018, combined with data from the final MC IPTT (2018), has identified a number of factors that supported the achievements in livelihood improvement of households with adult males and females, which are as follows:   The development of a commercial input value chain to increase farmer access to climate smart seeds for key staple crops. The identification of individuals who could be trained to serve as commercial input dealers, input agents, agricultural extension agents, and lead farmers, and in turn, to train farmers and promote their adoption of improved seeds and agricultural practices to increase crop production and productivity. In 2017, a year with timely and sufficient rain, these approaches produced bumper crops. The establishment of business linkages between commercial seed producers, commercial input dealers, and village input agents, and the use of guaranteed loans to encourage commercial seed producers to accept credit for purchases of stockage by input dealers Final Performance Evaluation of Northern Karamoja Growth, Health, and Governance Development Food Assistance Project – Final Evaluation Report viii               The increased crop productivity achieved by farmers using local seeds who observed their neighbors’ use of new crop and soil management practices and applied these practices in their own fields The training and use of agro-vet dealers and community animal health workers to promote farmer and kraal leader adoption of drugs, vaccines, and practices to prevent livestock disease and the use of improved livestock management practices to increase the health and productivity of livestock. This contributed to the improved quality and health of livestock that the farmers are bringing to market, which in turn stimulated more buyers and increased the sales price for livestock. The participation of livestock owners in the growing commercial livestock market The development of infrastructure to support economic activity and access to markets, and the rehabilitation and creation of new markets The expansion and strengthening of Village Savings and Loan Associations (VSLAs), increasing community access to financial services. The creation of new income sources for vulnerable women and their households by providing training to start up and manage microenterprises, and training in the value of savings from VLSA membership to expand their businesses Factors that presented challenges to the achievement of these goals include the following: Very poor start-up conditions characterized by low capacity of farmers, extreme levels of poverty, and high levels of illiteracy and underdeveloped infrastructure to support economic growth The assumption that all households are vulnerable and its influence on criteria used for selecting participants in farmers’ groups to participate in GHG agriculture-based livelihoods activities. The finding that women in adult female only households did not realize any change in their poverty status indicates that women in these circumstances may not have been sufficiently incorporated into farmers’ groups Competition to the market-based approach for adoption of modern seeds by the distribution of free seeds from the Government of Uganda Ministry of Agriculture’s initiative, Operation Wealth Creation Weak enforcement of quality control measures for commercially developed seeds Cultural resistance of Karamojong to sell livestock Outbreak of a new disease that killed many goats across the three districts and no currently available drug protocols to halt the its progression Frequent pest infestations and the spread of the army worm disease, and the ineffectiveness of biological methods to stop them Climate change effects, creating unpredictable changes in “normal” rainfall patterns during the planting season, and frequent droughts contributing to crop failure Conclusions MC’s use of the Making Markets Work for the Poor (M4P) model was the basis for the development approaches used under SO1 to bring households into commercial agricultural markets to improve their livelihoods. MC was successful in developing an effective commercial input value chain to increase the accessibility of climate smart seeds and other modern inputs. Cascaded training was used to train input agents, agricultural extension agents, and lead farmers, who then taught farmers how to use improved seeds and crop management practices, rather than GHG staff providing direct training to agropastoralists. The strength of this approach is seen in the significant increase in the percentage of farmers who used sustainable crop and livestock practices. This approach also built the capacity of key actors to provide training on their own without reliance on a donor. Final Performance Evaluation of Northern Karamoja Growth, Health, and Governance Development Food Assistance Project – Final Evaluation Report ix The success of the M4P model was diminished, however, by the inability of many households to afford the high costs of seeds, risks associated with unpredictable climate patterns, and weak enforcement of quality control measures for commercial seed producers that created distrust. The introduction of a voucher program subsidizing part of the costs and its adjustment and expansion over the duration of the project resulted in greater uptake of seeds, and in years with favorable rain patterns productivity of grain crops increased. The qualitative data collection found that farmers ceased purchasing these seeds when the voucher program ended in 2017. Farmers were still using sustainable agriculture practices, which have contributed to crop productivity, and these practices were diffused to farmers, both male and female, who did not directly participate in GHG agricultural activities. These farmers observed demonstration gardens and the practices of neighbors who were GHG participants and applied them to their own fields using local seeds. These farmers reported greater crop productivity using local seeds through adoption of these practices. The targeting strategy to improve the livelihoods of vulnerable households through involvement in GHG crop and livestock activities was implemented on the basis of baseline data showing that the majority of households in Northern Karamoja were either moderately or extremely food insecure. Selection criteria for participation was based on criteria that did not include the type of gendered household or differences in poverty level. Although this approach likely contributed to the overall improvement in livelihoods as measured by the change in values of FFP poverty indicators between 2013 and 2018, women in adult female only households did not realize any changes, indicating a need for more nuanced targeting. The interventions implemented to date for developing a commercial livestock industry have made tremendous progress. The multiple pathways employed to achieve this progress include the rehabilitating one of the primary livestock markets, creating a new market for skins and hides, increasing feeder roads and extending the road linking major markets in Kotido and Kaabong, increasing access to drugs and vaccines and skilled veterinary care, and training in disease prevention practices and other practices to increase the health and productivity of livestock. The adoption of these practices by some agropastoralists resulted in healthier livestock, which they brought to the market for live animals. The quality of the livestock attracted an increase in the numbers of interested traders and much higher prices for the livestock. The culture of cattle ownership and building and maintaining herds with no intention to sell is still important to the culture of the Karamojong people. But the growth of the commercial livestock value chain has penetrated villages. To date, only wealthier farmers with large herds who can afford to sell a portion of their livestock and who have the financial resources to purchase drugs and vaccines to improve their livestock health have entered the commercial market, and they are realizing the benefits of higher prices. Small owners have difficulty in building up their herd sizes and cannot afford to participate in the commercial markets. Support for women’s income increases through raising chickens and saving money to buy goats was important, however, the widespread disease and mortality among the goat population diminished the effectiveness of this intervention. However, smaller owners have benefitted from adopting practices to increase the health and productivity of their livestock, and sometimes they sell new livestock locally to increase their income. Demonstrated increases in the successful capture and return of stolen livestock through interventions applied under SO1, and improvements in the peace and security in Northern Karamoja, have also contributed to the growth of this industry. SO2: Improved Nutritional Status of Children Under 5 Years of Age SO2 was designed to improve the nutritional status of children under 5 years of age. Achievement of this objective was measured by FFP indicators on the prevalence of underweight, stunted, and wasted children under 5 years of age. The data on two of these indicators show an increase in the prevalence of underweight and stunted children from the 2013 baseline to the 2018 endline values. Specifically, the prevalence of underweight children under 5 years of age increased, from 21.5 percent at baseline to 27.7 percent at endline; and the prevalence of stunted children under 5 years of age increased, from Final Performance Evaluation of Northern Karamoja Growth, Health, and Governance Development Food Assistance Project – Final Evaluation Report x 32 percent at baseline to 37.6 percent at endline. At the same time, the prevalence of wasted children decreased, from 13.7 percent at baseline to 9.7 percent at endline. There has been an improvement in dietary diversity scores, which are associated with improving the nutritional status of children. The percent of children 6–23 months of age provided with a minimum acceptable diet (MAD) increased, from 2.1 percent in 2013 to 9.4 percent in 2018. Women’s dietary diversity score increased, from 2.1 in 2013 to 3.1 in 2018. Factors That Affected Achievement Levels The evaluation team’s analysis shows several factors that may be involved in prohibiting the improvement of the nutritional status of children under 5 years of age that are related to health and sanitation practices, feeding practices, and income levels. First, there is an established link between prolonged diarrhea and malnutrition. Findings for health and sanitation measures show an increase in the prevalence of diarrhea at endline, compared with baseline, and a reduction in the use of oral rehydration therapy. In 2013, the prevalence of diarrhea treatment was 89.7 percent, but it decreased over the five￾year period to 81.8 percent. The increase in the incidence of diarrhea may in part be due to the timing of PBS endline data collection during June and early July – the rainy season when one can expect the incidence of diarrhea to increase. Use of improved sanitation facilities shows no significant increase in the already low level of use. At the same time, the percentage of households with availability of soap and water at a handwashing station decreased. The use of safe drinking water is associated with decreased prevalence of diarrhea, but the percentage of households using an improved source of drinking water accessible year round has not increased from the baseline level of 37.4 percent. Access to safe water sources was found to be intermittent and this can lead to diarrheal disease. Additional factors that may be contributing to the poor nutritional status of children under 5 years of age are the Karamojong cultural feeding practices and the amount of calories consumed by children. In Northern Karamoja, cultural practices include the importance of sharing all sources of food equally among members of the household. A number of respondents said that food rations distributed by GHG meant specifically for pregnant and lactating women and children under 2 years of age were actually shared among all household members, indicating that children were not getting the full benefit of the additional caloric intake from food meant to improve their nutritional status. The level of poverty may constitute another factor prohibiting improvement in children’s nutritional status. Analysis of PBS endline data (2018) by quintiles of household consumption show lower rates of underweight and stunting in children under 5 years of age in the highest quintile compared to the lowest quintile of the population. The GHG project did create a good foundation for continued efforts to improve children’s nutritional status. Some of these achievements include the following:   The project increased access to quality health services. Several interventions contributed to this achievement, such as improving the effectiveness of health unit management committees who provide oversight of health facility procedures and in communicating village concerns to health facility staff, the training of village health teams to refer children for treatment of signs of malnutrition, training to increase the capacity of health facility staff, and constructing and equipping health facilities. The organization of pregnant and lactating women with children under age 2 years of age into mother care groups is an evidence-based model that was effective for transmitting behavioral change messages to adopt practices for improving the nutrition of their children and households, and practices to improve health and sanitation. The addition of the male change agent (MCA) model promoted behavior change in men in support of the health and nutritional status of their wives and children and aimed to reduce violent abuse of their wives. MCA groups were not established until 2016, however. Successful implementation of the MCA model may be a Final Performance Evaluation of Northern Karamoja Growth, Health, and Governance Development Food Assistance Project – Final Evaluation Report xi contributing factor to improvements in the percentage of children receiving a MAD, increases in women’s dietary diversity score, and reported decreases in domestic violence.  There was improvement of household access to safe water sources by constructing or rehabilitating boreholes, coupled with the organization of water user committees (WUCs) to ensure rapid repair of boreholes and handpumps when needed. Linkages created between district water authorities, private sector service providers, and WUCs managed through the GHG local partner Whave helped establish procedures to ensure sustained access to safe water sources. Although WUCs have not been able to solve the challenge of collecting required monthly fees from households for use in maintenance and repair of boreholes and handpumps, a foundation is in place to support the identification of solutions. Conclusions MC and its partners have worked in collaboration with health and water officials, health facility staff, local community-based organizations, village health teams, and community residents to establish the foundations required to decrease the incidence of malnutrition in children under 5 years of age. The elements of this foundation are based on a tested theory of change that involves improving access to quality health care and nutrition services for mothers and children, improving nutritional practices at the household level to increase the consumption of nutritious foods, and reducing the incidence of diarrhea in children through the adoption of improved hygiene and sanitation practices and increasing access to safe water and sanitation facilities. Analysis of PBS data on key indicators in nutrition show increases in only two of the indicators measuring the prevalence of underweight and stunted children under 5 years of age. This does not necessarily provide proof that the theory of change must be changed. GHG interventions have been successful in increasing access to quality health care and the consumption of diverse foods required for good nutrition. But the analysis of PBS baseline and endline data and qualitative data from interviews points to less success in prompting mother’s and household’s adoption of health and sanitation practices emphasized under WASH interventions, and some of these practices have actually decreased. This indicates the need to examine details of technical interventions and the methods and processes used to implement them in light of the cultural and locational context in Northern Karamoja. The analysis of quantitative and qualitative data shows that levels of household poverty and cultural feeding practices are likely sources of additional barriers to improvements in children’s nutritional status. SO3: Reduced Incidence of Conflict GHG interventions contributed to the consolidation of peace and improved security in Northern Karamoja. Important progress was achieved in reducing conflict and improving governance in the three districts. MC used a custom indicator to measure progress of this objective: percentage of respondents reporting increased movement in areas that were previously not accessible due to insecurity. The analysis of PBS data for this indicator shows a statistically significant increase, from 52.4 percent at baseline in 2013 to 87.7 percent in 2018. This finding is corroborated by qualitative findings from interviews across sub-counties and villages in the three districts. Respondents interviewed all reported that they could access fertile areas that had been inaccessible in the past. They were able to move freely to their gardens to cultivate and harvest and also to marketplaces in their areas. Livestock herders reported being able to access pastures and water wherever these could be found in project areas, although pastoralists in Kotido mentioned that they faced challenges migrating their herds into Abim, where tensions have been increasing. The Jie herders destroy crops they find in gardens in Abim. They uproot cassava and take it to their mobile kraals, claiming it grows on its own. Apart from causing food shortages, they steal animals from Abim as they return to Kotido when the dry season ends. The presence of Jie in Abim is generally abhorred. Capacity was also built to deal with violence fueled by alcohol abuse and to resolve or develop compromises in land ownership conflicts and land access cases for women. Final Performance Evaluation of Northern Karamoja Growth, Health, and Governance Development Food Assistance Project – Final Evaluation Report xii Factors That Promoted Achievement      The interventions built on the existing strengths of local communities affected by conflicts, making it possible to create and reinforce resident capacities to successfully mitigate, resolve, and prevent armed conflicts. The implementation approaches emphasized partnership with indigenous civil society, making it possible for results from capacity strengthening to be retained in the communities in which the project interventions were undertaken. The development of linkages between communities, community-based organizations, and informal and formal state structures strengthened local capability to effectively mitigate and resolve conflicts. One example is GHG’s successful encouragement of local council courts and area land committees to work closely with cultural elders and local focal point volunteers. The linkages between these structures and individuals were effective in resolving land disputes and conflicts and reduced the number of cases referred to formal courts. Factors That Presented Challenges Road ambushes and culturally organized large-scale raiding have ceased, but there is still a high risk of armed conflicts in Northern Karamoja because of continuing incidents of possession of illegal firearms, increases in criminal activities, and seasonal intensification of cattle thefts. Tensions, which occasionally flare up, have continued in communities neighboring the Turkana. In some parishes of the Sidok Sub-county in Kaabong that are close to the Kenyan border, protected kraals were reestablished due to the large number of animals stolen in cattle thefts. New sources of conflict have started occurring as a result of the increase in land areas where it is now safe to enter. Communities seeking better land for cultivation have begun moving to settle these areas. Some of these informal settlements are actual owned by individuals who have not yet reestablished cultivation. There is potential for violent conflict when owners try to evict those communities that have taken over their land. RECOMMENDATIONS This section provides a selection of some of the key recommendations under each SO. SO1 In light of the findings that adult female only households realized no significant changes in income and poverty status, future targeting strategies should be examined to ensure greater representation of women from these households in farmers’ groups and in microenterprise development activities. The targeting strategies for this group should be designed based on further investigation of the factors that underlie their poverty status relative to adult female and male households. The evaluation’s analysis of the circumstances of women who were not included as participants in farmers’ groups under GHG indicates that they may be part of the poorest segment of community populations where the project was implemented. The circumstances of these women, as well as men who were not selected for participation in farmers’ groups, should be further investigated for potential incorporation into the overall targeting strategy. The implementation of these targeting strategies should be regularly monitored to determine whether they are effective in engaging this population in strategies tailored to improve their circumstances and should be adjusted as needed. These strategies are needed to bring poorer households into commercial crop and agriculture production, to decrease poverty indicators, and to prevent the risk of creating wide disparities between the wealthiest and poorest farmers.  Given climate changes that result in unpredictable variability in seasonal rainfall patterns, more frequent droughts, and heavy flooding, livelihoods approaches in agriculture should be adjusted Final Performance Evaluation of Northern Karamoja Growth, Health, and Governance Development Food Assistance Project – Final Evaluation Report xiii by implementing strategies that better integrate household livestock and crop production to build their resilience.    Weak implementation of quality control measures developed for seed testing and sale of seeds produced by commercial national and regional suppliers that are distributed to the three districts must be addressed. This is essential to reverse trends in decreased demand and to begin building buyer trust. The availability of inputs (seed, pesticides, post-harvest handling materials) is important if farmers are to improve crop productivity. The work that was done to support the inputs value chain was relevant; however, despite some decreases in poverty levels, the higher cost of climate smart seeds must be addressed to encourage farmer adoption. The effort to create important supply-dealer linkages and business training is adequate to keep input dealers operational despite the decrease in demand for improved seeds because many dealers operate multiple businesses that can provide income streams to maintain their livelihood. In the interim, and until demand for inputs can be restored, additional support is required to equip dealers to learn how to supply the current lower demand effectively and reliably. VSLA development was key to increasing financial resources and services for village communitie and for supporting start up and growth of microenterprises. Further strengthening of VSLA financial management capacity and strategies for expansion are recommended, along with training in members’ financial literacy. We also recommend renewed attention to further strengthening of savings and credit cooperative capacity, particularly in the areas of creating sound and achievable strategic plans and effective strategies for encouraging member savings and repayment of loans on a timely basis. s SO2     A careful research design should be developed to investigate the factors that are prohibiting improvements in the nutrition status of children under 5 years of age. Our analysis of PBS findings on practices to improve nutritional status and practices to promote health and sanitation conditions, coupled with findings from the analysis of qualitative data collected from interviews, suggests some possible factors that contribute to this lack of improvement. Alternative approaches and technologies to encourage the use of sanitary practices, specifically the use of household latrines, should be researched and tested. Private sector entities involved in WASH should be engaged to provide low-cost sanitation supplies. The “Twin Leach” latrine approach1 and Arboloo2 technique are potential solutions that should be investigated for use in Northern Karamoja to overcome the fear of falling in latrine pits. Literature about ecological sanitation approaches used in Uganda is available for reference. Projects with a strong emphasis on WASH activities should carefully assess the implementation context and learn about cultural taboos and norms related to personal and community sanitation practices before identifying solutions and implementing approaches to create Open Defecation Free villages. Behavioral approaches involving men, such as the MCA model implemented under GHG, should be set up and implemented at the same time that the mother care groups are developed to maximize health and nutrition outcomes. Rigorous studies should be conducted to assess the impact of this approach on mother care group outcomes and behavioral changes in men’s behavior. The pilot mother care group plus model integrating activities from livelihoods and MCHN/WASH should be scaled up. The piloting of this model has demonstrated its 1 www.communityledtotalsanitation.org/sites/...org/files/TwinLeachPitToilets.pdf 2 Toilets that make compost: Low-cost, sanitary toilets that produce valuable compost for crops in an African context. Stockholm: EcoSanRes Programme. ISBN 978-9-197-60222-8 Final Performance Evaluation of Northern Karamoja Growth, Health, and Governance Development Food Assistance Project – Final Evaluation Report xiv effectiveness in empowering women to increase their financial autonomy, control over savings, and the adoption of practices to enhance household health and nutrition outcomes.     To ensure continuous access to safe water sources, the collection of monthly fees from households should be encouraged, along with the continued sensitization of the communities to understand the importance of preparing for the community needs for borehole maintenance and repair. Communities should be encouraged to come up with their own unique solutions to ensure all households pay their monthly fees. Communities in villages such as those found in Northern Karamoja are best placed for developing solutions and strategies that are effective within the context of their own cultures to ensure community-wide participation and regular payments from all households. SO3 Peace committee membership is based on volunteers nominated from each village. They are proud of the committee’s achievements, but it requires long hours of hard work. At this point, committee members require continuous motivation and tangible forms of support to maintain their involvement. Given the very important role they play in the reduction of thefts and improvement of the peace and safety of their communities, peace committees should be supported with regular training and public recognition for outstanding achievement. Considerations should be made for regular provision of uniforms, caps, or shoulder patches to promote enhanced recognition and to increase members’ pride, as well as the provision of a limited number of bicycles for each committee when greater speed is needed than can be afforded by tracking on foot, and technologies to promote continuous communication to supplement the reliance on word of mouth to communicate thefts, locations, strategies for capturing thefts, and coordinating with the security agencies in the districts. MC, working closely with relevant government agencies and district-based nongovernmental organizations involved in local conflict management, should support the clarification of the policy and legal status of peace committees to streamline their roles and functions in the forthcoming finalization of the development of the National Peace Policy and subsequent corresponding legislation. Traditional elders still exercise significant leverage and control over issues concerning livestock grazing rights and local conflict management and prevention. Although the authority of cultural elders may have waned in the past decades due to loss of control over armed youth (karachuna) who orchestrated virulent livestock raiding, the institution was very much alive,3 still functional and respected, and still considered relevant. Their effectiveness should be further strengthened in future programming by MC for inclusion in conflict management and resolutions strategies. Consideration should be made to develop interventions and approaches to address new sources of conflict that have started occurring as a result of the increase in land areas where it is now safe to enter. Communities seeking more land to cultivate have begun moving in to settle these areas, which have sometimes been found to be owned by individuals who have not yet reestablished cultivation. 3 On October 1, 2018, the evaluation team encountered an akiriket of the cultural elders of Kotido that was in session at their traditional shrine in the proximity of the Kotido Youth Centre in Kotido Municipality. Final Performance Evaluation of Northern Karamoja Growth, Health, and Governance Development Food Assistance Project – Final Evaluation Report 1 1. INTRODUCTION 1.1 PROJECT BACKGROUND In fiscal year 2012, the United States Agency for International Development (USAID) Office of Food for Peace (FFP) awarded the Northern Karamoja Growth, Health, and Governance (GHG) development food assistance project (DFAP) to Mercy Corp (MC) and its partners. Partners were World Vision and the Feinstein International Center, and local partners Abim Women Organized Together in Development (AWOTID), Whave, and Kaabong Peace for Development Agency (KAPDA). Implementation began in 2013 and ended in January 2018. The total funding over the life of the project was $52,404,800. The GHG goal was to improve peace and food security among chronically food insecure households in Abim, Kotido, and Kaabong Districts. Northern Karamoja is the most underdeveloped and isolated area of Uganda. Poverty levels and incidence of malnutrition are significantly higher than the rest of Uganda. Northern Karamoja is a structurally food deficit area in staple crops representing the major items of food consumption (FEWSNET 2016 ). Approximately 30 percent of household food needs must be purchased at local markets. At baseline, the Karamojong people suffered from chronic food insecurity and were heavily dependent on humanitarian and emergency food aid. At the start of GHG, 93 percent of the households were characterized as being either moderately or severely food insecure. MC’s FFP proposal (2012) provides information from its analysis of the primary causes of food insecurity, which include the breakdown of livestock husbandry as the key component of household livelihoods due to the years of armed conflict in the region, growing restrictions on herd mobility, regional conflict and insecurity, ongoing incidents of conflict within and between villages, and frequent cattle theft. Crop productivity was low due to lack of knowledge of beneficial soil and crop management practices. Farmers still planted by broadcasting seed over their fields, and they did not rotate or interplant their crops. Furthermore, they lacked access to early-maturing modern seeds tailored to the semi-arid environment. Production of staple crops is rainfall dependent. Erratic rainfall patterns and occasional serious drought frequently decrease production even further, leading to poor harvests of food crops. Northern Karamojong had insufficient income from traditional sources of livelihoods to purchase enough food required for household members on a year-round basis and had limited availability of financial services. The demand for staple food items is highest during the lean season before the next season’s crops have matured, and market prices for staple foods typically increase at this time. The health and nutritional status of children under 5 years of age was exacerbated by unhygienic practices and limited access to sources of clean water. Women’s lack of decision-making authority, low status in the household, and frequent physical abuse limited their engagement and active contributions to increase food production, household income, and the well-being of their children and household. The interaction of these factors has led to chronic food insecurity despite decades of humanitarian and emergency food aid from USAID, the World Food Programme, and other donors working in Uganda, and from the World Food Programme (WFP)4. MC designed GHG to address chronic food insecurity, the high incidence of childhood malnutrition, and recurrent conflict in the Northern Karamojan districts of Abim, Kotido, and Kaabong. GHG interventions targeted vulnerable households, and women in particular. Youth were also targeted for participation in activities across the three Strategic Objectives. At the start of the project, MC and USAID5 determined that given the high rate of extreme and moderate food insecurity all households would be considered vulnerable. 4 This summary is also based on MC’s analysis of food insecurity in Northern Karamoja, detailed in their 2012 proposal. 5 Email communication between the evaluation team leader and Mercy Corps, December 4, 2018 Final Performance Evaluation of Northern Karamoja Growth, Health, and Governance Development Food Assistance Project – Final Evaluation Report 2 1.2 THEORY OF CHANGE The GHG theory of change (TOC) assumes that by focusing on improving and strengthening livelihoods of vulnerable households and increasing household purchasing power using the Making Markets Work for the Poor (M4P) model, coupled with targeted health, nutrition, and water and sanitation interventions to increase the health and nutrition of mothers and particularly the nutritional status of children under 5 years of age, household food security and child health can be substantially improved. Furthermore, the TOC stipulates that in the context of Northern Karamoja’s history of conflict, these two strands of interventions are not sufficient to improve food security and the nutritional status of children. One of the critical preconditions for achieving the GHG goal and objectives on a sustainable basis is to consolidate and increase peace and security in the region to permit residents to fully engage in productive livelihoods activities outside the immediate area of their villages without fear. The TOC stipulates that by creating or strengthening existing capacities in Northern Karamoja in conflict prevention, mitigation, and resolution, the incidence of conflict will decrease. Other activities were implemented to improve the resilience of this population in the face of climatic or other events outside their control. The TOC also assumes that by implementing activities focused on overcoming the inequities that women and girls face in their daily lives across the three DFAP areas and activities to reduce domestic violence against women, that more significant improvements in food security, livelihoods, and maternal and child health and nutrition will be achieved. MC’s strategy is based on the assumption that following the end of major conflict and disarmament, vulnerable households are now in a much better position to gradually improve their own circumstances if they are supported by indirect means to improve their capacity. MC employed a facilitation approach for implementing this support by identifying public and private sector local actors in each district to provide services, commercial inputs, and training to vulnerable households. A key element of this approach was to build or improve the capacity of these actors to serve in selected roles through training, observation trips, and other activities. MC then relied on these individuals to cascade the training down to service and input providers at the sub-county and village levels, who in turn, provided training to household residents in targeted villages. To ensure continued access and the quality and sustainability of products and services to vulnerable groups, MC also facilitated the development of linkages between these individuals and public and private service and input providers at the sub-county and district levels. Final Performance Evaluation of Northern Karamoja Growth, Health, and Governance Development Food Assistance Project – Final Evaluation Report 3 2. EVALUATION OVERVIEW 2.1 EVALUATION PURPOSE The purpose of the final performance evaluation for the GHG DFAP is to examine outcome data, collect qualitative data to identify and understand the factors that contributed to the achievement of outcomes or performance barriers, assess the sustainability of those outcomes, and provide recommendations useful to MC for Apolou, the follow-on development food security activity (DFSA) that launched in 2018 (see Annex 1 for the Final Evaluation Statement of Work). The findings can be used to promote learning about the factors that promote success as well as factors involved in poor performance, to adjust implementation approaches and processes and targeting strategies to reach vulnerable population groups, and adjust measures to increase sustainability of outcomes. The main audiences for the report include FFP, USAID/Uganda, MC and its implementing partners, and the Government of Uganda, 2.2 DESIGN OF THE PERFORMANCE EVALUATION The performance evaluation of the GHG DFAP is based on a mixed methods design incorporating qualitative and quantitative data. The design relies on the comparison of population-based survey (PBS) baseline and endline data and the degree of change to assess progress in the achievement of key FFP outcomes, and the findings from the analysis of qualitative data from fieldwork interviews to provide context and to help interpret the statistical findings. This design provides robust evidence-based findings and conclusions and promotes a deeper understanding of those findings to answer key performance evaluation questions and to promote programmatic learning. The identification of respondents and locations in the three districts of Northern Karamoja is based on the GHG TOC and the technical strategies and facilitative approach used to reach the GHG goal and DFAP outcomes. To evaluate the effectiveness of this design, qualitative data were collected in each district to assess interventions and implementation approaches that span district, sub-county, and village levels, as well as to learn of and compare the perspectives and opinions of project stakeholders on the achievement of results under each of the three strategic objectives (SOs). 2.3 EVALUATION QUESTIONS Q1: To what extent have the projects met their defined goals, purposes, and outcomes? The evaluation team will assess and determine the following: (1) Project final data on indicators against targets set by both the partners and the key FFP indicators of depth of poverty, stunting, and undernutrition (2) Factors that promoted or inhibited the achievement of the project objectives, including, but not limited to, the effectiveness of food-for-asset and cash-for-asset interventions (3) Targeting strategies and their contribution to achieving project goals, especially with regard to gender and reaching the most vulnerable (4) The effectiveness of interventions on the poorest individuals (5) Based on evidence from sub-questions 1–5, the contribution of GHG to USAID’s efforts to reduce food insecurity among chronically food insecure households Final Performance Evaluation of Northern Karamoja Growth, Health, and Governance Development Food Assistance Project – Final Evaluation Report 4 Q2: For each technical sector, what are the strengths of and challenges in the projects’ design, including theories of change, that influence the effectiveness of the interventions? The evaluation team will assess the following: (1) The relevance of the technical interventions, including food-for asset and cash-for-asset interventions, to achieve project outcomes, and discuss those findings in relation to the projects’ theories of change (2) The plausibility of pathways and the determinants of achieving the key outcomes, highlighting any unforeseen pathways leading to unintended positive or negative consequences of the projects Q3: In each technical sector, what are the strengths of and challenges to the effectiveness of the interventions’ implementation? The evaluation team will evaluate the following: (1) The effectiveness and relevance of the technical interventions approach/methods, including food-for asset and cash-for-asset interventions, to achieve project outcomes, and discuss those findings in relation to the projects’ theories of change (2) Factors in the implementation approach and methods and the context associated with greater or lesser effectiveness in producing outputs of higher or lower quality (3) The interventions and implementation processes deemed more or less acceptable to members of the target communities Q4: Which project outcomes are likely to be sustained? The evaluation team will evaluate the following: (1) The quality and functionality of systems, processes, and institutional arrangements established or strengthened independently by the projects, as well as in collaboration with the private sector, the Government of Uganda, community organizations, and research organizations to achieve project outcomes and sustainability (2) Whether the necessary resources and capacity strengthening will exist to sustain service providers, and the likelihood that service provision will continue after the project ends (3) Communities’ perceptions on the quality, frequency, and effectiveness of the implementation processes and services provided (4) The motivation of the community and beneficiaries to demand and pay (or invest time) for the services, and communities’ perceptions of the sustainability of the services provided by the project (5) The extent to which the projects leveraged other U.S. Government and non-U.S. Government investments to achieve sustained outcomes as identified in the theories of change (6) Evidence of enhanced linkages with other service providers Q5: What are the key lessons learned and best practices that should inform future projects in the country? (1) What are the lessons learned, best practices, and approaches that should be considered in designing future food and nutrition security projects? (2) What are the lessons learned, best practices, and approaches that should be considered in implementing and implementation approaches for future food and nutrition security projects? (3) What are the risks to the effective implementation of these approaches to support project achievements? (4) Based on the lessons learned, how can potential unintended negative consequences be minimized and positive consequences be captured systematically? Final Performance Evaluation of Northern Karamoja Growth, Health, and Governance Development Food Assistance Project – Final Evaluation Report 5 3. EVALUATION SAMPLE DESIGN, DATA COLLECTION AND ANALYSIS METHODS, AND LIMITATIONS This chapter provides an overview of the methods used for the design, data collection, and data analysis of the PBS (baseline and endline) and the qualitative component of this final performance evaluation. A more thorough description of the methods are included in the PBS Protocol (Annex 2) and the Final Evaluation Protocol (Annex 3) of this report. 3.1 SAMPLE DESIGN This section provides details of the sample design for the collection of quantitative and qualitative data. The design for the collection of PBS endline data was selected to capture population-wide benefits from the GHG project. The design for collection of qualitative data was based on a non-representative, purposive sample of sites located within the PBS sample of households. 3.1.1 PBS Sample Design for Collection of Quantitative Data The quantitative results for the PE use the data collected through baseline and endline PBSs that are statistically representative of the areas where the GHG project was implemented. The household samples selected for the baseline and endline PBSs are based on a multi-stage clustered sample designed to power a test of differences between the baseline and endline estimates for the FFP child stunting indicator. The sample size is derived based on a formula to detect a difference in the prevalence of stunting from baseline to endline of 6 percentage points with 95 percent confidence and 80 percent power. Based on this calculation, the baseline PBS included 80 villages and 2,400 households, and the endline PBS included 42 villages and 1,260 households.6 These sample size calculations included a pre￾inflation factor to take into account anticipated household-level non-response as well as an adjustment factor to take into account the number of households that need to be contacted to achieve the required sample size of children under 5 years of age. The multi-stage sampling consists of three stages: (1) sampling of clusters (villages), (2) sampling of households within each village, and (3) sampling of eligible respondents in each household when needed. A detailed description of the endline PBS sampling plan is provided in the 2018 Uganda Joint Baseline/Endline PBS Protocol (see Annex 2), and a detailed description of the baseline PBS sampling plan can be found in Annex 1 of the 2013 Uganda Baseline Study Report (March, 2014).7 6 Because the input parameters were somewhat different for the two time points, the sample size requirement for the endline was updated from a baseline value of 1,557 children per project to an endline value of 988 children per program, given that this combination of sample sizes for baseline (1,577) and endline (988) ensured an overall power of 80 percent for the two time points. In principle, the sample size of required children should be identical for the baseline and endline PBSs, because the computation is made at the time of the baseline to ensure that the pair of sample sizes provides an overall power of 80 percent. However, the actual realized sample size of children from the baseline PBS (2,668 children per program) greatly exceeded the target sample size for the baseline PBS (1,557 children per program). This implied that the overall power of the statistical test would likely be considerably greater than 80 percent had the same sample size used at baseline also been used at endline, because the power of the statistical test is determined by the actual realized sample size of children at the two time points. Therefore, it was possible to reduce the sample size at endline relative to what it was at baseline and still maintain 80 percent power overall. 7 A complete description of the baseline study is provided in the Baseline Study for the Title II Development Food Assistance Projects in Uganda report, available at: https://www.usaid.gov/documents/1866/uganda-baseline-study-report-March-2014 Final Performance Evaluation of Northern Karamoja Growth, Health, and Governance Development Food Assistance Project – Final Evaluation Report 6 3.1.2 Sample Design for Collection of Qualitative Data Collection of qualitative data for this performance evaluation was based on a purposive sample of data collection sites in each district to provide contextual data and data for interpreting findings from the analysis of the PBS baseline/endline data. The findings were not intended to be generalizable to the whole population. The first criterion applied for site selection was villages located in the same sub-counties where the PBS was conducted, either close by or in the actual village where the household questionnaire was administered. The second criterion for site selection was to select villages where all three SO activities were implemented. In the district of Abim, however, qualitative data collection was limited to sites where Catholic Relief Services is not implementing its current DFSA. This limited the evaluation team’s ability to select villages in Abim in which all three SO activities were implemented, and, as a result, in the two villages that were eventually selected there were no maternal and child health and nutrition (MCHN) or water, sanitation, and hygiene (WASH) activities implemented at the village level. However, the team did conduct interviews in the district with public sector health officials and in the two sub-counties with individuals who were involved in GHG activities (see Section 3.3, Evaluation Design Limitations). Sample Size and Unit of Data Collection In each district, data were also collected at the district level and in the sub-county where each village was located to examine linkages between public and private input and service providers and trainers and village GHG participants. Accordingly, the unit of data collection and analysis comprised a single district, a sub-county within that district where the selected village was located, and the village. In each district, data were collected in two villages, in their corresponding sub-counties, and in the district. In total, data were collected in 15 locations comprising 3 districts, 6 sub-counties, and 6 villages.8 Table 1 shows the data collection sites in the three Northern Karamoja districts. Table 1: Data Collection Sites by District District Sub-county Village Abim Morulem Arema East Abim Town County Atwilo Kotido Rengen Chaicon Nakapelimoru Longelep Kaabong Lolelia Chumakod Sidok Longar 3.2 DATA COLLECTION METHODS 3.2.1 Quantitative Data Collection The baseline PBS was conducted from February 25 to April 30, 2013 (at the beginning of the typical lean season in Karamoja, which runs from March to June), and the endline PBS was conducted from June 7 to July 6, 2018 (at the end of the typical lean season). The primary objective of the baseline PBS was to assess the status of FFP and custom indicators prior to project implementation; the primary objective of the endline PBS was to assess the change in these same indicators at the end of the DFAP by undertaking statistical tests of differences between baseline and endline measures. This pre-post design allows for the measurement of change in indicators over time; however, this design does not allow statements to be made about attribution or causation relating to project impact. The population-based 8 The total number of locations was based on the number of weeks allocated by the GHG evaluation contract for fieldwork and the resources available for data collection. Final Performance Evaluation of Northern Karamoja Growth, Health, and Governance Development Food Assistance Project – Final Evaluation Report 7 survey design includes households that may or may not have received GHG project interventions so that project spillover effects to non-participants in the target area can be evaluated. The indicators are related to food security, poverty, women’s and children’s health and nutritional status (including anthropometry), WASH practices, agricultural practices, and gender (access to and decisions on credit, control over use of income, and group membership). Custom indicators were added by MC to assess women’s antenatal care, access to any kind of sanitation facility, mobility in the project area, number of crops produced, and use of private sector veterinary care. The endline PBS used the same data collection instrument as the baseline PBS.9 3.2.2 Qualitative Data Collection Fieldwork to collect data was conducted between September 22 and October 11, 2018, by a four￾person qualitative evaluation team (QET)10 and three locally hired research assistant/interpreters (see Annex 3 for a more detailed description of the QET). Data were collected in six villages (two per district) for data on each of the FFP indicators and project custom indicators designed to assess the status of the population in villages located in the PBS sample. Collection of data at the sub-county and district linked to each of the villages allowed the QET to respond to FFP evaluation questions on the effect of the GHG design, TOC, technical interventions, and implementation methods and processes on key outcomes, and the sustainability of those outcomes. The team used key informant interviews (KIIs), focus group discussions (FGDs), and in-depth interviews (IDIs) to collect data from GHG project participants in each administrative level. Annex 4 provides a list of the type and number of respondents and respondent groups interviewed at each administrative level for each SO. FGDs were conducted in villages with project participants and non-participants to examine the diffusion of knowledge and practice beyond individuals and groups targeted for direct participation in GHG activities. For SO1, FGDs were held with female agropastoralists who were directly involved as GHG participants during the project implementation period (2013–2018) and with agropastoralists who did not participate. For SO2, FGDs were conducted with women who were mother care group (MCG) members during GHG, and with women who did not participate in MCGs.11 Approximately 435 respondents participated in interviews across the three districts. 3.2.3 Quantitative Data Analysis The baseline and endline PBS data were used to assess change over time for key FFP and custom indicators (see Annex 5 for a tabular summary of these indicators and tests of differences for change over time from baseline to endline). The data were also used, as appropriate to the evaluation question, to conduct additional descriptive and bivariate analyses to help strengthen findings established from the quantitative and qualitative data. Multivariate analyses controlling for key socioeconomic and project￾specific factors as covariates were used to explore the factors that are associated with the prevalence of stunting and wasting among children under 5 years of age, and daily per capita consumption expenditures. These indicators were selected for additional analyses to help inform learning on key impact and outcome indicators. The multivariate models pool data from the baseline and endline surveys and control for a number of confounding factors in addition to project participation status to empirically explore hypothesized relationships between indicators and to determine whether project participation is associated with change over time net of other factors. 9 The PBS evaluation team supplemented the questionnaire with questions for new indicators to be included in the 2018 baseline study for the newly funded DFSAs in the Karamoja Region; however, only the 2013 baseline study modules were used to calculate the endline indicators. 10 The QET are those team members who collected qualitative data through fieldwork. They are separate from the large team of organized data collectors used for interviewing respondents in a household survey designed for the PBS. 11 The methods for selecting participants and non-participants for village FGDs are explained in Annex 3. Final Performance Evaluation of Northern Karamoja Growth, Health, and Governance Development Food Assistance Project – Final Evaluation Report 8 The designation of participating versus non-participating project households was based on household respondents’ answers to a set of questions about exposure to the project interventions. One respondent per household was asked, “Have you or someone from your household participated in GHG project activities?” Respondents who answered “Yes” were considered to be project participants. Respondents who answered “No” were considered non-participants. Respondents who reported that they or someone in their household participated in the project were asked about the type of assistance received. Project assistance could include food rations, nutrition training or meetings, agriculture-related training or meetings, or other types of assistance. About 29 percent of all households indicated that they participated in GHG project activities. Of those that said they participated in GHG project activities, 52 percent said they received food rations, 71 percent said they participated in nutrition trainings/meetings, 66 percent said they participated in agricultural trainings/meetings and 40 percent said they participated in other project activities. These responses were not validated by the project, so it was not possible to determine definitively that the respondents accurately reported the source and type of project assistance. Respondents or households were likely to be recipients of more than one type of assistance, and it was not possible to disentangle the effects of different types of project interventions. 3.2.4 Qualitative Data Analysis Methods Qualitative data analysis involved a content analysis of interview data in each district to develop categories of responses to FFP evaluation questions based on the application of the coding scheme (described in Annex 3). For each sector (livelihoods, food security, MCHN/WASH, and conflict), a triangulation process was used to compare categories of responses across respondent types in villages (e.g., lead mothers, village health team members, health clinicians), and with responses from interview participants at the sub-county and district levels. Responses for each category of village respondents were compared across all six villages spanning the districts of Abim, Kotido and Kaabong. This procedure was used to corroborate responses between different categories of respondents within and across districts to the extent possible, and to develop findings for each evaluation question. Findings from PBS baseline/endline data were then incorporated to further strengthen and validate qualitative findings where possible. For some questions, other data sources were used to corroborate qualitative findings (e.g., data from the final Indicator Performance Tracking Table [IPTT], from Famine Early Warning Systems [FEWSNET] reports, and from evaluations and assessments). For a complete list of information sources consulted, see Annex 6. 3.2.5 Integrating Findings from Quantitative and Qualitative Data The qualitative and PBS datasets were analyzed independently using techniques appropriate to that dataset. Two procedures were used for the integration of findings from each data source:   Findings from the qualitative data were used to help interpret findings from the analysis of PBS data. Findings from the PBS data were integrated with findings from the analysis of qualitative data to produce a more robust, evidence-based finding for each evaluation question. The procedures for integrating findings from qualitative and quantitative data sources are based on a triangulation process entailing comparisons that examine the extent to which these separately developed findings corroborate each other. It is important to note that there are limits to triangulating findings from every data source. The findings from each dataset must provide a source of evidence that closely relates to the evaluation question addressed by the qualitative data. If findings from the two major sources of data addressing the same topic do not corroborate or validate each other, findings from other data sources can still be used to (1) help resolve those differences, and (2) help interpret or develop a more comprehensive overall finding with greater explanatory power. Final Performance Evaluation of Northern Karamoja Growth, Health, and Governance Development Food Assistance Project – Final Evaluation Report 9 3.3 LIMITATIONS OF THE EVALUATION METHODOLOGY Time differences in PBS data collection between baseline and endline. Although the baseline and endline PBS data were collected during the typical lean season, which runs from March to June12 in the Karamoja region, the timing was not exactly the same. The baseline PBS data were collected at the beginning of the lean season, which includes the dryer months of March and April. The endline PBS data were collected toward the end of the lean season during June and the first week of July, when rains are heavier and diarrhea rates are expected to be higher. During these months the rainfall was heavy, and in many areas flooding was a problem. Indicators that are sensitive to seasonality, such as the household dietary diversity score (HDDS), the women’s dietary diversity score (WDDS), and the children’s minimum acceptable diet (MAD), may have lower values at endline compared to baseline because of food becoming scarcer as the lean season progressed. Recall issues. The QET faced potential recall issues and respondent confusion between GHG and Apolou activities. GHG activities ended in January 2018, and MC began implementing the new DFSA, Apolou, shortly after that in the majority of villages in Kotido and Kaabong located in the GHG implementation areas. The QET addressed this potential problem by telling respondents that our questions concerned their experience with MC and the GHG project during the years 2012 through 2017, and by helping them recall that time period by referencing key events that occurred during those years. Examples include reference to the 2016 Presidential election, the year that vouchers were first distributed to help participants recall events before and after, and the time period when food rations were distributed to pregnant and lactating women (PLW). Not only did this work, but the team also found that many village participants had no problem recalling details of activities in each year. Lack of village-level MCHN/WASH data from the District of Abim. Changes in implementing partner DFSA coverage areas limited the QET’s ability to select villages that had implemented all three SOs in each district. In the district of Abim, data on village-level MCHN and WASH GHG activities could not be collected in the two villages selected for fieldwork. FFP DFSA activities in Abim are now covered by CRS which limited the locations within areas of PBS endline data collection where qualitative fieldwork could be conducted to avoid interference with CRS activity implementation and confusion between current CRS and previous MC activities conducted under GHG. This in turn limited the number of possible village sites for qualitative data collection. In the final selection, the team had to select villages where only livelihoods, conflict, and governance activities were implemented. However, the MCHN/WASH subject matter expert did conduct several interviews with public sector health officials at the district and sub-county levels. Limited data on value chain activities. Data for the FFP indicator on participation in value chain activities at endline were collected from only those farmers with the intention to sell their crops and livestock for income (about 36 percent of all farmers) whereas the baseline survey collected data on participation in value chain activities for all farmers, so the evaluation of livelihoods strengthened could not evaluate change over time in the use of value chain activities for all farmers. One of the elements of value chain activities included in the household survey is measuring the change in purchase of inputs for crop production, including purchase of improved inputs (seeds) and inputs for livestock production and health, including increasing access and use of veterinary services. These are important activities in the GHG livelihood strategy that are not able to be assessed with the data from these indicators. The analysis of interview data with both male and female agropastoralists across the three districts showed a decrease in the use of improved seed, but the QET was not able to support this finding with PBS data. 12 FEWS NET Seasonal Calendar for a Typical Year. Available at: http://fews.net/east-africa/uganda/food-security￾outlook/february-2018 Final Performance Evaluation of Northern Karamoja Growth, Health, and Governance Development Food Assistance Project – Final Evaluation Report 10 4. EVALUATION FINDINGS This section of the report is organized by the three GHG SOs, and the findings and conclusions are structured based on the evaluation questions contained in the FFP Uganda Performance Evaluation Scope of Work (see Section 2.3). Evaluation question 1 (EQ1) addresses the extent to which GHG met its defined goals and outcomes. Subsequent evaluation questions ask about the effects of different aspects of the design of the project on project outcomes, specifically: (1) the effect of project targeting strategies, (2) the influence of the project design and TOC on the effectiveness of technical interventions, and (3) the effect of the methods and processes used to implement the technical interventions. We combined findings and conclusions for EQ2 and EQ3 to clarify findings on technical interventions, improve the narrative, and reduce redundancies, given that these two questions about interventions are so closely related. EQ4 asks about the sustainability of GHG outcomes, and EQ5 reviews lessons learned and best practices. Findings and conclusions for each of these questions are presented for each SO. Section 5 of the report presents the QET’s recommendations based on the findings and conclusions from each SO (EQ5). The recommendations are presented for their potential application in Apolou, the current DFSA being implemented by MC, and for future programming. 4.1.1 SO1: LIVELIHOODS STRENGTHENED 4.1.1 EQ1: To what extent has GHG met its defined goals, purposes, and outcomes for SO1? The key measures of strengthened livelihoods and improved food security are increases in income, using poverty measures as a proxy, and increases in the HDDS, which demonstrates an ability to produce or purchase a diversity of nutritious foods. The data for these indicators are shown in Table 2. Table 2: Performance Indicators Measuring Livelihood and Food Security Indicator Baseline 2013 Endline 2018 Significance of Change Food Security Indicators Average household dietary diversity score 2.2 3.6 *** Poverty Indicators Per capita expenditures as a proxy for income of U.S. Government-assisted areas $0.58 $0.92 *** Male and female adults $0.58 $0.91 *** Adult female, no adult male $0.63 $0.86 ns Percentage of people living on less than $1.25 per day 93.2 87.0 ** Male and female adults 93.3 87.3 ** Adult female, no adult male 92.6 88.5 ns Mean depth of poverty 61.5 53.3 ** Male and female adults 61.9 53.2 *** Adult female, no adult male 57.1 55.5 ns Agricultural Indicators Percentage of farmers who used financial services in the past 12 months1 31.6 31.1 ns Male 33.7 29.4 ns Female 29.9 32.4 ns Final Performance Evaluation of Northern Karamoja Growth, Health, and Governance Development Food Assistance Project – Final Evaluation Report 11 Indicator Baseline 2013 Endline 2018 Significance of Change Percentage of farmers who used at least 3 sustainable agricultural practices in the past 12 months 16.7 37.2 *** Male 20.8 44.6 *** Female 13.3 31.4 *** Percentage of farmers who used at least two sustainable crop practices in the past 12 months2 19.2 34.8 *** Percentage of farmers who used at least two sustainable livestock practices in the past 12 months3 10.5 17.8 *** Percentage of farmers who used at least one sustainable NRM practices in the past 12 months4 16.4 10.4 * Percentage of farmers who used improved storage practices in the past 12 months5 48.7 51.0 ns Male 56.3 50.9 ns Female 42.4 51.1 ns ns=non-signifiant, * p<0.05, ** p<0.01, *** p<0.001 1 Financial services include savings, credit, and agricultural insurance. 2 Sustainable agricultural practices for crops include soil preparation by ox plow or tractor, planting seeds in rows, crop rotation, use of fertilizer, and intercropping. This sub-indicator is based on all farmers, not just those that reported raising crops. 3 Sustainable livestock practices include use of animal shelters, vaccination, deworming, homemade animal feeds made of locally available products, use the services of community animal health workers, and purchased drugs/medicines to give to animals. This sub-indicator is based on all farmers, not just those that reported raising goats or cattle. 4 Sustainable NRM practices include agroforestry or cultivation of fruit trees, management of natural regeneration, soil conservation on hillsides, and construction of water catchments. 5 Improved storage practices include cereal banks, silos, and granaries. Northern Karamoja can still be characterized by its high level of poverty. One of GHG’s most important achievements is reflected in the improvement shown in all three poverty measures shown in Table 2. The data show that there was also an improvement in the HDDS (from 2.2 to 3.6) among households in Northern Karamoja. Dietary diversity increases serve as another (proxy) indicator for improved livelihoods because these increases imply that they were attained by purchase of diverse food sources or through production by households. The disaggregation of poverty indicator data by gendered households shows that these improvements in livelihoods are mostly driven by adult male and female households. Adult female-only households did not see a change in their status. This is discussed in the section on targeting strategies. MC used two major approaches to strengthen livelihoods in Karamoja: (1) improving the production and productivity of crops and livestock among Northern Karamoja’s predominantly agropastoralist population and (2) the development of non-agricultural livelihoods through training and support of women and youth to start and grow mini and small microenterprises, and improving the business environment for small business growth. The sections that follow describe some of the key factors that promoted the improvement in livelihoods demonstrated by findings on key FFP indicators. Factors that Promoted Strengthened Livelihoods in Karamoja Findings from our analysis of all data sources, coupled with team discussions, leads us to conclude that the following factors were involved in the achievements to date in strengthening livelihoods in Northern Karamoja:  The development of a commercial input value chain to establish a regular supply of improved seeds and other modern inputs through building the capacity of, and establishing business Final Performance Evaluation of Northern Karamoja Growth, Health, and Governance Development Food Assistance Project – Final Evaluation Report 12 relationships between, commercial seed suppliers, input dealers at the district and sub-county levels, and input agents serving villages selection. Several avenues to promote farmer adoption of improved seeds and other inputs to boost crop productivity were developed: a cascaded training approach involving the training of input dealers and agents to inform farmers how to use these inputs; the establishment of a cadre of agricultural extensions to provide direct training to farmers in sustainable agricultural practices and the use of modern inputs; and the selection of lead farmers to establish demonstration gardens.        The training and use of agro-vet dealers and community animal health workers (CAHWs) to promote farmer and kraal leader adoption improved livestock management practices to increase the health and productivity of livestock13 and their use of vaccines and drugs for the prevention and treatment of livestock diseases, which was a large barrier to the growth of a livestock industry The improved quality and health of livestock that farmers are bringing to the market has contributed to the growing livestock trade characterized by more buyers coming into Karamoja from different locations (Kampala, Mbale, Soroti, Gulu, South Sudan, Kenya), 14 resulting in opportunities to realize better incomes from sale of livestock. Interviews with kraal leaders and farmers confirmed the higher prices paid for livestock. 15 The development of a new livestock market and improvement of a livestock market for skins and hides resulted in opportunities to realize better incomes for wealthier households with large herds, meat traders and members of the butchers’ associations. 16 In addition to linkages to buyers of hides and skins in Lira, Soroti, and Mbale, the members of the meat handlers’ and butchers’ associations learned that horns can be sold, and they now sell to buyers from as far as Jinja and Kampala. Increased market access by improving feeder roads and the extension of an improved road linking markets in Kotido and Kaabong. 17 The organization and capacity building of traders into associations, which has enabled them to operate business, identify new markets, and advocate local governments and town councils to resolve factors that hinder business. 18 The provision of entrepreneurial skills development by training women and youth in business start-up, management, and expansion. Expanding financial services for rural communities by strengthening and creating additional village savings and loan associations (VSLAs), and training members in the value of savings. These activities provided vulnerable households with the possibility of starting up and expanding mini and micro businesses. 13 The evaluation team was informed by the district veterinary officer of Kaabong that GHG contributed to improvement in the health of livestock through regular de-worming, vaccination, and spraying. 14 Stites, E., Howe, K., Redda, T. and Akabwai, D. (2014). “A Better Balance:” Revitalized Pastoral Livelihoods in Karamoja, Uganda. Feinstein International Center, Tufts University. 15 KII with kraal leader, Chumakod Village, Kaabong, October 6, 2018; KII with kraal leader, Longar Village, Kaabong, October 7, 2018; KII with district veterinary officer, Kaabong, October 5, 2018; KII with a CAHW, Aremo East, Abim District, September 27, 2018 16 KII with chairperson, Kotido Butchers’ and Meat Handlers’ Association, Kotido District, October 1, 2018; KII with commercial officer, Kotido, October 1, 2018 17 KII with chairperson, Kaabong business forum, Kaabong, October 10, 2018; KII with chairperson, Kotido business forum, Kotido, October 2, 2018 18 KII with business forum chairpersons, Kaabong and Kotido; KII with a commodity food trader and agro dealer, Kotido; KII with commodity trader and input dealer, Kaabong Final Performance Evaluation of Northern Karamoja Growth, Health, and Governance Development Food Assistance Project – Final Evaluation Report 13 Targeting Strategies and Their contributions to Achieving Project Goals with Regard to Gender and Reaching the Most Vulnerable The target population for GHG, described in MC’s approved DFAP proposal, was economically vulnerable households. MC determined with USAID at the start of GHG that given the rate of moderate and severe food insecurity, all households in the region should be considered vulnerable. This provided MC with flexibility to target households on an intervention by intervention basis who should be considered to make the most impact.19 Within the population of economically vulnerable households, the most vulnerable targeted for interventions were women, especially women with small children, and youth. GHG used organized farmer groups, and in some cases, farmers groups were organized, to involve male and female farmers in crop and livestock activities. Farmer groups received training on the use of improved seeds, crop and soil management practices, livestock management, maintenance of livestock health, and prevention of livestock diseases. The overall strategy MC used for targeting men and women for involvement in GHG activities was to engage local leaders in discussing selection criteria and mobilizing participants based on those criteria. The QET routinely asked questions during FGDs and KIIs in each village about how people were selected for participation in farmer groups. Table 3 shows the range of local leaders involved, the selection criteria that were used, and the source of information by village. Table 3: Characteristics of the Selection Process by District and Village District Village and Source of Information Individuals MC Involved in Discussing Criteria and Mobilizing Participants Description of Selection Criteria Provided by Source of Information Abim Arema East FGDs with male farmers and female farmers Agricultural extension agents Membership in a group, have land to cultivate, active in agriculture Atwilo KII with lead farmer Agricultural extension agents Same criteria reported Lead Farmer reported that MC provided the selection criteria Kotido Chaicon FGD with male farmers KII with local council director Local council director and leader of village health team Strong enough to farm, have land to cultivate, willing to share knowledge According to local council director, MC provided the selection criteria Longelep FGD with female farmers Agricultural extension agents Not in any group, have land to cultivate, hard-working, willing to engage in a GHG agricultural activities Kaabong Longar FGDs with male farmers and female farmers Female farmers Village chair Hard working, membership in a group, strong enough to farm, able to learn and share knowledge with others Table 3 indicates that MC developed the criteria for participant selection, at least in Abim and Kaabong, which accounts for the similarity in criteria across the three districts. In Kaabong, this information was provided from separate FGDs with male and female farmers in one of the two villages, and they were not likely to know who developed those criteria. One of the main differences in selection criteria was that in Kaabong and Abim, participants had to be a member of an existing agricultural group; in Kotido, the criteria for selection specified participants that are not members of a group. Another important 19 Email communication with the MC Monitoring and Evaluation Advisor in Kamapala, December 4, 2018 Final Performance Evaluation of Northern Karamoja Growth, Health, and Governance Development Food Assistance Project – Final Evaluation Report 14 criteria mentioned in Abim and Kotido was land ownership. Although this was not mentioned by farmers interviewed in Kaabong (Longar Village), we assume this was part of the selection criteria applied to villages in this district.20 The information collected during village interviews was confirmed by a KII conducted with the MC Livelihoods Officer in Kaabong.21 The QET was informed that in all villages MC staff worked with local leaders to identify and mobilize participants for involvement in crop and livestock activities. These leaders were given the liberty to identity others to be part of the group based on the selection criteria, emphasizing that individuals had to be an active farmer and be willing to participate in GHG activities. The data collected by the project on the number of vulnerable households benefitting directly from U.S. Government (USG) activities from 2012 until 2018 demonstrates that MC’s targeting strategy was based on the consideration that all rural households in the three districts were vulnerable (Table 4). As required by this FFP indicator, data were disaggregated by gendered household. Households with adult females only includes women in a polygamous relationship in which the husband is not domiciled in the household and widows. Table 4: Number of Vulnerable Households Benefitting Directly from USG Gendered Household LOA Target LOA Achievement Adult female, no adult male 17,311 20,051 Adult male, no adult female 4,163 6,879 Female and male adults 75,129 77,302 LOA=life of activity Note: The FFP definition of vulnerable households are families that are likely to suffer death or severe consequences in the face of droughts, delayed rains, hunger, poor health habits and hygiene. This includes families with pregnant mothers, children under age two, lactating women; and [whose households are] located far from trading centers and government services, are neglected and have no food, income, productive assets and any other means of survival.” Source: MC IPTT, final, 2018 The targeting based on gender was an important ongoing consideration in MC interventions across all SOs. MC’s proposal stated that all GHG activities were to be designed and implemented to ensure maximum empowerment of female and male participants of all ages. MC’s cross-cutting team made specific efforts to ensure women’s continuing participation in GHG agricultural and entrepreneurial interventions, and service providers were given gender sensitivity training to help support and encourage women’s involvement.22 For example, gender training was integrated in training provided to agricultural extension agents (AEAs) and community animal health workers (CAHWs) providing crop farming and livestock services, and to trainers selected for capacity-building activities in business forums and for microenterprise development.23 An AEA interviewed in the village of Aremo East in Abim said that the gender training he received helped him increase women’s participation. Several respondents interviewed noted that there were a greater number of women involved in farmers’ groups than men. One of those respondents, a male AEA interviewed in the village of Chumakod in 20 Ibid. Land ownership was one of the criteria used to enable farmers to try new agricultural practices and plant improved seeds. The QET further assumes that the information provided during FGDs with male and female farmers on how farmers were selected for participation in farmers’ groups was not necessarily inclusive of all the criteria that was actually applied. 21 Interview conducted with Livelihoods Officer for MC’s Apolou DFSA (and former staff member for GHG) in Kaabong on October 10, 2018. 22 The GHG cross-cutting gender team developed a framework and theory of change for increasing the involvement of women and youth with specific activities that were developed and implemented under each SO. 23 Mercy Corps Uganda, GHG Annual Results Report for FY 2016, pp. 4 Final Performance Evaluation of Northern Karamoja Growth, Health, and Governance Development Food Assistance Project – Final Evaluation Report 15 Kaabong, explained it by saying, “men have a mindset that crop farming is for women.”24 In actuality, MC did target women’s involvement in farmer’s groups, given the lead role that women play in crop production. A total of 21,769 individuals received GHG-supported short-term agricultural sector productivity or food security training, of which 14,082 (74 percent) were women.25 This total not only includes women who were participants in farmers’ groups, but women who were also members of VSLA groups (the majority of whom are women) and members of MCGs (as discussed in Section 4.2). Male and female youth are included in this indicator as well, but disaggregations for this indicator did not include youth. In each of the sampled villages in Abim, Kotido, and Kaabong, FGDs were conducted with female agropastoralists that participated in GHG agricultural activities through farmers’ groups and with a group of female agropastoralists who did not participate in farmers’ groups, to contrast their situations and the effects of actively participating in project activities. The findings from the data analysis from these two groups across the six locations indicate that participation in a farmers’ group was important for improvements in household livelihoods. The “non-participants” interviewed in each location included a mix of women in different circumstances: widows, married women living in households with an adult male (their husbands), and married women in polygamous relationships who lived in households without an adult male. Despite these circumstances, these women shared common characteristics. They were generally not members of any group (productive or saving) and did not receive training in improved agricultural or livestock practices. Some said that they learned improved technologies and practices for crop production by observing their neighbors. But as a group, they were not using improved seeds. They either used seeds they saved from the previous harvest or purchased local seed as needed in the local markets. These women reported that they collected wild fruit, provided casual labor, burned charcoal, and collected firewood for sale as sources of income, particularly to buy food during lean seasons or for unforeseen emergencies. They owned a few chickens, ducks, one or two goats, and zero to two cows.26 None of them were certain why they were not selected to participate in farmers’ groups for GHG activities, although a few reported being told that the groups already had the maximum amount of members and that they would be selected to join one “at a later time.” Further investigation is required to determine whether it is possible that female farmers who were not in farmers’ groups make up a component of the poorest population. Training and support to increase sources of income outside of agricultural production was another focus of the GHG livelihood strategy. Women were trained in business skills and how to begin or expand microenterprises through several different interventions. For example, 140 female small business owners were selected for training to play important roles in the stocking agent/distribution networks developed for the commercial input value chain, enabling them to expand their businesses. The 73 women involved in MC’s pilot MCG plus (MCG+) intervention27 benefitted from the economic empowerment grants to initiate income-generating activities, and those interviewed reported being able to expand their businesses slowly by saving money through their VSLA membership and using a portion of those savings for investment in their businesses. GHG provided in-kind grants that enabled youth groups and women’s groups to open bakeries, hair salons, tent rentals, grinding mills, honey processing, and livestock trading businesses so they could continue to finance their social outreach and peace programming. KIIs 24 FGD with VSLA group (received agricultural training) in Chaicaon; KIIs with a lead farmer in Atwilo Abim, an AEA in Aremo East, Abim, and an AEA in the village of Chumakod in Kaabong. 25 GHG LOA IPTT, 2018 26 FGDs with all non-beneficiaries in Aremo East and Atwilo in Abim District, Chaicaon and Longelop in Kotido District, and Chumakod and Longar in Kaabong District. 27 MCG+ was a MC initiative that combines specific interventions from SO 1 and SO 2. This pilot model is the same as the model used for MCG, but differs by adding income-generating activities for MCG+ participants, such as enrolling them in VSLAs, training them in the value of savings and investment, providing them with economic empowerment grants to initiate income generating activities and with training and skills development for business start-up and expansion. Final Performance Evaluation of Northern Karamoja Growth, Health, and Governance Development Food Assistance Project – Final Evaluation Report 16 conducted with women entrepreneurs in each district indicate that the money earned through their new businesses enabled them to cover unexpected expenses without resorting to low-paying activities, such as collecting and selling firewood or charcoal or by engaging in casual labor. They were able to begin improving the livelihoods of their households in small ways by buying food when necessary and paying for school fees and clothes to keep their children in school. Youth were also an important focus of GHG activities. Youth received training in agricultural productivity and use of modern methods and were the recipients of vouchers to purchase agricultural inputs. GHG also involved youth in vegetable production projects, teaching them new skills for income generation. Youth were selected for involvement in the Karamoja livestock mapping, which involved identifying livestock grazing areas, livestock-related resources, migratory routes, and hotspots of conflict. Youth were also included in training to provide them with skills to serve as trainers in business forums on business planning.28 GHG built the capacity of two cohorts of CAHWs to provide services to farmers. The second round identified youth (at least two per sub-county) and encouraged the communities to select and send a male and female youth.29 Effectiveness of Targeting Interventions to the Poorest Initially, MC conducted a participatory assessment to identify the poorest households in GHG target villages. The plan was to repeat the assessment annually to determine the extent to which GHG project interventions improved the status of the poorest. However, after the joint decision made by FFP and MC to treat all households as food insecure, GHG did not focus on identifying and targeting the poorest households and the participatory assessment was not repeated. Assuming that all households were food insecure allowed MC the flexibility of targeting specific sub-populations within villages based on the type of project intervention. However, MC maintained their focus on engaging women, and especially women with young children, and youth in project activities. By treating all households as food insecure we do not know with any certainty the characteristics of the poorest households within target villages and whether specific targeting per project activity was effective in improving the circumstances of the poorest30. We can infer that women living in households without an adult male may be among the poorest, as indicated by the lack of change in their circumstances of poverty. The lack of another adult cash earner and source of labor living in the household may be one of the factors contributing to their status of poverty. Another indicator of the impoverishment of these women comes from 2018 PBS endline data on women’s underweight and household consumption quintile. The data show that 49.7 percent of women living in households with no other cash earner are underweight, compared to 24.8 percent of underweight women in the overall population in Northern Karamoja (see Section 4.2.1). These households include widows and women in polygamous relationships, which have been noted to have higher levels of household poverty because the husband’s resources are divided among multiple households.31 Widows are also disadvantaged, but for different reasons. By cultural tradition, new widows are encouraged to marry their late husband’s brother who would then be in charge of the deceased’s assets (land and livestock); but if widows do not choose to marry the brother, they are nonetheless obligated to transfer their late husbands’ assets to his brother. Although women in general were specifically targeted for project interventions, findings from the analysis of PBS data FFP poverty indicators and nutrition indicators by gendered households leads us to conclude that women living in households alone without an adult male were not specifically targeted for participation in farmers’ groups; or, if targeted, other barriers limited their potential to effectively 28 Mercy Corps Uganda, GHG Quarterly Report for January to March 2016 29 KII with Livelihoods Officer, MC Kaabong, October 8, 2018 30 Because GHG did not repeat assessments to identify the poorest households for targeting project activities, the evaluation team could not determine project effects on the poorest and changes in their circumstances. 31 Cited in the Food Security Desk Review for Karamoja, Uganda, FANTA, January 2017 Final Performance Evaluation of Northern Karamoja Growth, Health, and Governance Development Food Assistance Project – Final Evaluation Report 17 benefit from GHG interventions. The criteria on who to involve in farmers’ groups for participation in crop production and livestock interventions was not guided by household poverty status. As a result, it was possible that the poorest, such as women from households with no adult male, may have unintentionally been underrepresented in these groups. Although the source of qualitative data from FGDs with women farmers who were not GHG participants in farmers’ groups is limited to six villages, analysis of interview data show that some of these women do live in households with their husbands, indicating that the poorest are not only those adult women who are living in households without an adult male. A targeted assessment would be required to understand all the characteristics that comprise the poorest individuals and the effects of the selection criteria used for inviting farmer participation in GHG agricultural activities on the poorest and on those individuals who were not selected for participation. This assessment should include male farmers who were not selected for participation in farmers groups.32 4.1.2 Strengths and Challenges of the Project Design, TOC, and Implementation Methods That Influence the Effectiveness of Technical Interventions in Achieving SO1 Goals The GHG project design is based on a market-based approach to help vulnerable agropastoralist households increase their incomes and improve their own situations on a sustainable basis using the pro-poor market model (M4P) linking vulnerable agro-pastoralist households to (1) inputs and services, (2) markets, and (3) financial products and services. With improved access, vulnerable households were expected to improve their livelihoods. A second component of the project design to strengthen livelihoods focused on building the capacity of vulnerable target groups (women and youth) to establish and gradually expand MSMEs, and to improve the business environment to support MSME growth. Crop Production The major intervention for increasing livelihoods of farming households was the promotion of their involvement in commercial agriculture using a pro-poor market-based model (M4P). The aim of these interventions was to increase their food security and livelihoods by increasing the productivity of their crops. GHG promoted farmer uptake of improved, climate smart seeds for staple food crops in their area. Central to this approach was the development of a robust input value chain, interventions to create farmer demand for improved seeds, and the provision of training to farmer groups and the correct use of the improved seeds, and crop and soil management practices to enhance productivity. The input supply chain was developed to ensure a consistent supply of improved seeds and their purchase by linking market actors at the district, sub-county, and village levels with farmers. GHG linked agro-input dealers to commercial seed suppliers and input dealers and agents, built the capacity of input dealers and agents to effectively participate in the market, and provided loan guarantees to commercial suppliers to gain their willingness to accept purchases on credit to enable adequate stocking. These actions created a larger cohort of providers of agricultural inputs.33 To improve coverage of and access to agricultural inputs at the grassroots level, the dealers supported by MC identified local traders to function as village input agents. This was a major undertaking and was very effective. To create demand for improved seeds, interventions included the establishment of farmers’ groups for the locus of activities, the establishment of agricultural extension agents to provide training and guidance to farmer groups on the correct use of these seeds and practices, and the selection and use of lead farmers to establish demonstration gardens to show case how production and productivity can be increased. 32 The QET did not have sufficient fieldwork time or resources to conduct FGDs with male non-participants as well as with female non-participants. Given time and resource parameters, the team chose to interview female non-participants since women were a target group for GHG activities across the three SOs. 33 KII with District Commercial Officer, Abim District, September 26, 2018 Final Performance Evaluation of Northern Karamoja Growth, Health, and Governance Development Food Assistance Project – Final Evaluation Report 18 Two major approaches characterize the implementation approaches and methods MC used to promote improvements in livelihoods. The first approach was the use of a facilitation approach using cascaded training for the implementation of technical interventions meant to improve the livelihoods of rural households. The second approach was to create linkages between actors at the district, sub-county, and village levels to support livelihoods improvements. Both approaches were designed to help move the population of Northern Karamoja away from a dependency relationship with the government, donors, and nongovernmental organizations (NGOs) to provide their food security toward improving their own food security and livelihoods. These approaches were also used to help sustain outcomes after FFP interventions by building local capacity in Northern Karamoja to improve the economy on their own and building the capacity to improve and stabilize their own livelihoods. Instead of directly providing rural households with training for the improvement of their livelihoods, MC selected and trained input and service providers, primarily at the district and sub-county levels, who in turn were expected to teach, advise, and encourage village residents to adopt new approaches. For example, MC recruited and trained AEAs to train farmers on improved agronomic practices and post￾harvest handling and storage methods; and selected and trained high-performing farmers to serve as lead farmers to prepare demonstration gardens. Commercial seed dealers and input agents were trained on the appropriate use of improved seeds, and they in turn provided this information to farmers to promote uptake. Several input dealers reported that they set up demonstration gardens near their businesses to promote the sale of improved varieties of crop seeds.34 AEAs were the main providers of training to farmers. They were recruited from outside Karamoja because of the difficulties in finding locally available people with the required qualifications. The challenge with this approach is that these AEAs had 12-month contracts and sometimes left while the project was still in process. This left a gap in qualified individuals to train farmers in improved practices.35 The turnover of AEAs in some areas prompted MC to recruit less-qualified but high-performing farmers to take on the role. This worked well, but district agriculture officials were against referring to these farmers as extension agents because they did not have the right qualifications. They were concerned that locally recruited individuals might offer farmers erroneous technical advice and therefore would require regular supervision.36 Low uptake of improved seeds, however, showed that training, advice, and demonstrations in conjunction with making seeds and other technologies and service more accessible through the development of the input value chain were not sufficient to promote farmer demand for improved seeds. Assessments showed that women, who were the principal buyers of seed, given their role as cultivators, could not afford the higher costs of improved seeds and could not afford to fail by taking the risk of trying the seeds if they did not increase crop productivity. By their own admission, MC underestimated the depth of poverty in Northern Karamoja. The development of a voucher program that partially subsidized the cost of improved seed was introduced to decrease the risk of testing the seeds with the hope of subsequent adoption. The seeds were distributed by AEAs to members of farmers’ groups, who were trained on improved agronomic practices. Vouchers were heavily targeted to women with young children in MCGs, who were identified as the most vulnerable.37 The intention was to create a sliding voucher scale, with farmers taking on a higher cost each year. Initially, many of these vouchers were not redeemed. Reportedly, the cost of the seeds was still too high. The model was adjusted several times to an increasing scale, with farmers 34 KIIs with input dealers in Abim Town and in Kaabong Town 35 KII, Livelihoods Officer, Mercy Corps Uganda, Kaabong Office 36 KII with the District Production and Marketing Officer, Kotido District 37 Mercy Corps Uganda, GHG Quarterly Report for January to March 2017, pp10 Final Performance Evaluation of Northern Karamoja Growth, Health, and Governance Development Food Assistance Project – Final Evaluation Report 19 contributing less.38 The result was the redemption of more vouchers. Farmers accessed 3 metric tons of seed in 2015, 26 metric tons of seed in 2016, and 589 metric tons of seed in 2017.39 Qualifications for receiving vouchers were expanded. A review of a memorandum of understanding between an input dealer in Kaabong and MC showed that male change agents (MCAs), youth groups, peace ambassadors, land committee members, and women’s groups also qualified to purchase seeds at subsidized prices if they presented vouchers. Vouchers were also distributed to VSLA members and to VLSAs associated with savings and credit cooperatives (SACCOs) supported by the project. This approach of voucher distribution enabled a larger number of households to be reached, and therefore more farmers had the chance to try improved seed. GHG reached approximately 30 percent of the households in Northern Karamoja with voucher assistance (24,000 of 80,000 households).40 The addition of the voucher program to activities that increased access and availability of modern inputs, coupled with farmer training, was effective as measured by increased uptake of improved seeds and other technologies and practices. MC IPTT data for 2017 show that 33,097 farmers (50.6 percent men and 49.3 percent women)41 were implementing the new technologies and practices. Female farmers who did not participate in farmers’ groups reported that observation of demonstration gardens helped them learn improved agronomic practices.42 They also reported that they observed what other farmers did then copied it using their own seeds. Some participants in the FGD with male farmers in Kaabong said that they observed their neighbors’ application of new agronomic practices and applied them to their gardens planted with local seeds. They reported increased productivity from local seeds as a result of trying some of the practices, such as crop rotation, line planting, and intercropping. When rainfall was adequate, the implementation of taught practices and use of improved seeds led to improved yields, with farmers having more to eat and some to sell.43 But this was only reported for 2017, in which there was a bumper crop that shortened the length of the lean season for some farmers. The unpredictability of onset of rains, coupled with either too much or insufficient rainfall and incidences of drought leading to crop ruination, have also contributed to making farmers become more risk averse to invest money for the purchase of improved seeds. Farmers cannot afford crop failure. In 2015 and 2016 prolonged dry seasons, drought, and inadequate rain greatly reduced harvests. 44 Data collected from interviews and FGDs indicated despite the application of improved technologies and practices, farmers are highly likely to suffer hunger in 2019 due to the severity of the dry season in 2018.45 The early season plantings were washed away by heavy rain and flooding, and the second season plantings did not do well because the rains stopped too early.46 38 In 2016, the voucher value was 50 percent. In 2017, the voucher value for MCGs was 80 percent for cereals, 77 percent for legumes, and 71 percent for vegetables. For other target groups added to project activities, the voucher value was 57 percent.. 39 Mercy Corps Uganda. GHG Annual Results Report for FY 2017, pp 6 40 Mercy Corps (2017): Annual Results Report. Fiscal Year 2017. Northern Karamoja Growth, Health and Governance (GHG) Program, Submitted November 6, 2017. Resubmitted December 7, 2017 41 MC Final IPTT, 2018 42 FGD with female non-participant farmers in Chaicaon and Longelep 43 FGDs with male and female participant farmers in Aremo East and Atwilo in Abim, in Chaicaon and Longelep in Kotido, and in Chumakod and Longar in Kaabong 44 FEWSNET Uganda reports, 2015, 2016. 45 FGDs with male and female agropastoralists in all visited villages in Abim, Kotido and Kaabong districts; FGDs with non￾participant farmers in all visited villages in Abim, Kotido, and Kaabong districts; KII with the District Production and Marketing Officer Kaabong 46 These reports are verified by the Famine Early Warning Systems Network Forecast and Food Security Outlook for September 2018, which mentions significantly below average harvests in Karamoja, following an estimated 60 to 80 percent crop loss. Final Performance Evaluation of Northern Karamoja Growth, Health, and Governance Development Food Assistance Project – Final Evaluation Report 20 A FEWSNET Update Message for Uganda published in November 2018 with projections through January 2019 confirms these fears. It reported an unusually harsh dry season, amplified by land surface temperatures 3–7 degrees Celsius above average and a projected crop production of second season crops at 10–15 percent below average, which would likely be greater in rainfall-deficit areas. The report notes that poor households with limited income based on firewood and charcoal production are relying on market purchases for food and are struggling to meet minimum food needs. Overall, in Karamoja an estimated 50 percent of the population are stressed (IPC Phase 2), and an estimated 10 percent of the population, which includes Kotido and Kaabong, are in a food security crisis situation (IPC Phase 3).47 The FEWSNET Climate Risk Assessment for Food Security (2017) notes the strong climate variability and change affecting Karamoja, which is characterized by reoccurring droughts, flash floods, and extended dry seasons. The FEWSNET analysis of climate trends since 1960 shows the expansion of warmer regions in Uganda that have extended into Karamoja, an increase in the average temperature, and in the average number of days of extreme heat (28 percent increase between 1960 and 2003), a decreased reliability in the onset of the rainy season, a reduction in the total amount of rainfall by 20 percent, and a shorter rainy season. Projections going forward are the continuation of reduced rainfall and pronounced variability from year to year and within years.48 An increased incidence of agricultural pests and weeds are associated with longer dry seasons and extended periods of extreme heat.49 Lead farmers interviewed in Kotido and Abim noted the constant outbreak of crop pests and the advent of army worm infestation, both representing a huge challenge to crop farmers.50 Although GHG promoted the use of biological methods for pest control, farmers were advised to use chemical pest control to manage the spread of army worms.51 Interviews with agropastoralists in Abim, Kotido, and Kaabong revealed a growing distrust with improved seeds from the commercial input market. Farmers reported purchasing seeds that did not germinate. Seeds packets purchased to plant staple crops sometime revealed errors in packaging when a totally unexpected crop came up. Other farmers reported that they could not plant in time to benefit from the onset of the rainy season because of late delivery of seeds. When problems were reported calling into question the quality of seeds, MC worked closely with the district and sub-county agricultural officers to identify the issues and referred them to the commercial seed producer who then provided compensation to the farmers. Quality control regulations for commercial seed production were developed by the Government of Uganda working with donors, but reportedly there is weak oversight and enforcement. Farmers reported they do not want to pay high prices for seeds that cannot be guaranteed to germinate or for seeds they do not want due to packaging errors. These factors, combined with the end of the voucher system at the end of 2017 contributed to a significant decrease in purchases of seeds.52 Commercial dealers and input agents interviewed reported that they stocked much less seed in 2018 due to the greatly reduced demand. Some agents did not stock these seeds at all. Those farmers who wanted improved seed could not get them from agents and resorted to the local markets to purchase local seeds or used home saved stocks. 47 Famine Early Warning Systems Network Uganda Update Message for November 2018-January 2018, November 2018 48 FEWSNET Climate Risk Assessment for Food Security, Uganda, p. 3, 2017 49 Ibid. 50 KII with lead farmer in Atremo East, Abim district; KII with lead farmer Atwilo, Abim district; KII with agro dealer in Kotido Town, Kotido district; KII with the District Veterinary Officer, Kaabong district 51 KII with MC Livelihoods Officer, Kaabong District. 52 Decreases in purchase were widely mentioned during interviews with commercial dealers and input agents in Abim, Kotido and Kaabong, and this was corroborated by FGDs with farmers across the six village sites, many of whom reported that they have stopped purchasing improved seeds. We lack specific PBS data that are representative of the population interviewed that could demonstrate the take off in the use of improved inputs and then a decrease in use in 2018. Final Performance Evaluation of Northern Karamoja Growth, Health, and Governance Development Food Assistance Project – Final Evaluation Report 21 Livestock Production A market-based approach was also used to strengthen livelihoods through developing a commercially oriented livestock industry to stimulate greater sales of livestock and livestock products. To support the growth of the industry, GHG focused on activities to improve the health and productivity of livestock in Northern Karamoja by increasing the access and availability of veterinary services, vaccines, and drugs to meet the serious challenges of livestock disease, and to provide training in improved livestock management practices. The livestock population was greatly reduced during the years of serious armed conflict, wholesale theft of cattle through large, organized cattle raids, poor nutrition, and disease. Livestock kept close together in kraals for protection during the years of greatest armed conflict and insecurity were more susceptible disease. As security in the region began increasing, more freedom of movement in areas previously not entered because of safety risks became possible (see Section 4.1.3 on conflict prevention, management, and resolution). Livestock owners stopped confining their cattle in kraals and began herding them again on a seasonal basis to pastures adequate for livestock feeding and nearby water sources. The incidence of livestock disease was a serious problem, however, and there was little access to trained veterinarians or drugs for treating animals. Measures to prevent diseases were not widely known or used. To rebuild the population of healthy livestock, GHG technical interventions focused on increasing livestock owner access to services for the prevention and treatment of livestock illnesses through improved herd management practices and use of drugs and vaccines. GHG strengthened the capacity of 129 CAHWs by working with them to demonstrate animal health and preventative practices, and by linking them to animal vaccine and drug sellers for treatment of common livestock diseases. To improve access to veterinary drugs, the project linked agro-vet shops to national pharmaceutical companies for drugs and embedded services. From January to March 2017, 98 GHG-trained CAHWs provided services to 12,509 farmers (14 percent female), treating 37,243 animals. In the same period, 6 GHG-trained agro-vet shops provided services to 3,118 farmers and CAHWs.53 Farmers showed an improvement in the practice of animal management and preventive herd health practices, such as spraying and de-worming, and an increased willingness to access CAHWs to have their animals vaccinated and their diseased animals treated.54 In 2017, 19,605 farmers and others applied improved technologies or management practices promoted by GHG.55 Farmers and kraal leaders were also purchasing drugs and administering practices such as spraying, de-worming and treatment of familiar diseases without calling the CAHWs.56 We do not have quantitative data on improvements in the condition of livestock, but interviews with kraal leaders and other individuals associated with the industry reported that buyers have noted the healthier conditions of livestock brought to markets for sale. There was an increase in buyers coming to purchase livestock from a wider range of locations, and as a result, the sale prices offered for livestock have increased.57 The increase in buyers and the volume of trade in live animals was also reported in the Food Security Desk Review for Northern Karamoja.58 53 MC Uganda, GHG Quarterly Report for January to March 2017, p. 6 54 KIIs with CAHWs in Aremo East, Abim district; KII with Owner Genesis Agro-vet Shop, Abim District; KII with manager JICAHWA Agro-vet shop, Kotido District; KII with District Production and Marketing Officer, Kaabong District 55 Mercy Corps Final IPTT, 2018 56 KIIs with CAHWs in Chumakod, Kaabong District; KIIs with kraal leaders in Atwilo, Abim District; Chumakod and Longar, Kaabong district 57 KIIs with kraal leaders in Longar and Chumakod in Kaabong district; KIIs with chairperson Kaabong business forum; KII with chairperson Kotido Butcher and Meat Handlers’ Association 58 Food Security Desk Review, FANTA, p. 27, January 2017 Final Performance Evaluation of Northern Karamoja Growth, Health, and Governance Development Food Assistance Project – Final Evaluation Report 22 With growing use of interventions that increased access to trained livestock care services, and the application of vaccines and drugs, the livestock population began slowly increasing, with a 49 percent increase in total growth in the livestock population from 2008 to 2017. 59 The Food Security Desk Review cited a reference confirming that herders have been rebuilding their livestock herds based on 2016 reports of an active local demand for breeding and young stock.60 The other major GHG intervention to increase livelihoods from livestock involved the development of a commercial livestock value chain by increasing access to and availability of commercial livestock markets.61 Investments included the construction of 36 kilometers of road62 connecting Kacheri to Lobalangit, the renovation of the Kotido town meat slaughter market, the creation of a modern market for sale of livestock skins and hides63, and the construction of the Kaabong Komuria livestock market. The circuit road connecting Kacheri to Lobalangit shortened travel time from Karenga to Kacheri and improved the transportation of goods.64 It eased tracking of stolen animals and movement of herders from kraals to centers to purchase animal drugs.65 The developments at Komuria formalized the livestock market in a number of ways. Livestock negotiations and sales occurred with the guarantee of animal security, the town council revenue base was increased through the tax on livestock sales, and a formal area was created for conducting livestock transactions.66 Purchasers came from a variety of locations, including Kenya, South Sudan, Kampala, and other districts of Uganda.67 According to an interview with the Veterinary Officer in the District of Kaabong, the number of livestock buyers coming to the market is increasing.68 Despite important achievements, serious challenges remain to steadily increase household income streams through the development of a modern, commercial livestock industry and supporting vulnerable women by encouraging goat production. Disease outbreaks are still persistent. The prevalence of disease among cattle was linked to the following:    Karamojong practices involving seasonal migration with their livestock during the dry season to access water sources and better pastures; movement of animals to seasonal locations brings them into contact with tsetse and other disease-infested areas Irregular vaccination programs Improper administration of treatments by livestock owners 59 Mercy Corps Uganda, GHG Annual Results Report 2017, p. 5 60 Food Security Desk Review, FANTA, p. 27, January 2017 61 KII with manager of a livestock skin and hide processing business located next to the slaughter house in the renovated Kotido town meat slaughter market, noted that livestock owners are learning to use smaller brands to mark their cattle so there is more hide suitable for processing. They receive more money than owners who employ large scale branding across the hides of their cattle. 62 Mercy Corps Final IPTT, 2018 63 KII with manager of a livestock skin and hide processing business located next to the slaughter house in the renovated Kotido town meat slaughter market, noted that livestock owners are learning to use smaller brands to mark their cattle so there is more hide suitable for processing. They receive more money than owners who employ large scale branding across the hides of their cattle. 64 KII with chairperson Kaabong Business Forum, Kaabong District 65KII with kraal leader Chumakod, Kaabong District 66 KII with chairperson of the Kaabong Business Forum, October 10, 2018 67 Stites, E., Howe, K., Redda, T., and Akabwai, D. (2014). “A Better Balance:” Revitalized Pastoral Livelihoods in Karamoja, Uganda. Feinstein International Center, Tufts University. 68 KII with District Veterinary Officer, Kaabong District, October 5, 2018 Final Performance Evaluation of Northern Karamoja Growth, Health, and Governance Development Food Assistance Project – Final Evaluation Report 23 Interviews with kraal leaders, CAHWs, and farmers indicate that there is an epidemic of a disease affecting goats throughout Northern Karamoja, causing high rates of mortality. At present, there is no known treatment for the disease. This high mortality in goats is a setback for women who participated in GHG livestock interventions to gradually improve their incomes by converting their savings from keeping chickens into goats.69 Despite achievements in increasing livestock owners’ access to markets that can provide higher prices, interviews with the Kaabong Business Forum Chair and kraal leaders in Chumakod and Longar revealed that cattle transactions are still conducted in many impromptu locations, primarily by small holders, where farmers are not likely to obtain the best possible price. The reported use of out of market transactions may lead to a farmer being taken advantage of by traders.70 Although more people are involved in the sale of livestock, and better prices are offered for healthy animals, interviews with CAHWs, male and female farmers and kraal leaders show that high prices do not prompt the sale of livestock. The importance of cattle ownership is a key element that defines Northern Karamojan culture. They are slaughtered to provide meat in the community for the purpose of celebrating important individual milestones and important community events. Livestock farmers will sell whether prices are high or low to solve a pressing household need,71 such as school fees, medical bills, food in periods of severe hunger, and treatment for sick animals. Small holders in particular are also careful not to use all the money earned to solve the need, but to spare some to buy a calf to replace what was sold.72 This practice may cause farmers to reap the least gains from the growing livestock markets. To date, it is the wealthiest farmers with large herds who can afford to sell some of their stock in the commercial markets who are benefiting from the developments of a commercial livestock market. Conclusions MC’s strategy based on the M4P model was the basis for the development approaches used under SO1 to bring households into commercial agricultural markets to improve their livelihoods. MC was successful in developing an effective commercial input value chain to increase the accessibility of climate smart seeds and other modern inputs. Cascaded training was used to train input agents, AEAs, and lead farmers, who then taught farmers how to use improved seeds and crop management practices, rather than GHG staff providing direct training to agropastoralists. The strength of this approach is seen in the significant increase in the percentage of farmers who used at least three sustainable agriculture (crop, livestock, or natural resource management) practices. This approach also built the capacity of key actors to provide training on their own without reliance on a donor. However, the success of the M4P model was diminished by the inability of many households to afford the high costs of seeds and take on the risks associated with unpredictable climate patterns, and by the weak enforcement of quality control measures for commercial seed producers that created distrust. The introduction of a voucher program subsidizing part of the costs and its adjustment and expansion over the duration of the project resulted in greater uptake of seeds, and in years with favorable rain patterns, productivity of grain crops increased. The FEWSNET risk assessment for food security in Northern Karamoja (2017) shows the great risk of relying on a crop productivity strategy for improving livelihoods, especially in Kotido and Kaabong, where populations are in a crisis phase (Uganda Update Message, November 2018). On this basis, we conclude that an agriculture-based livelihood improvement strategy should better integrate 69 FGD with Atobukuthe Akoro women’s group, Chaicaon, Kotido District; FGD four members of MCG+ in Longar, Kaabong District; and FGDs with female beneficiaries and non-project beneficiaries in Chumakod, Lolelia Sub-county in Kaabong District 70 Stites, E., Howe, K., Redda, T., and Akabwai, D. (2014). “A Better Balance:” Revitalized Pastoral Livelihoods in Karamoja, Uganda. Feinstein International Center, Tufts University. 71 KII with kraal leader Longar, Kaabong District, October 10, 2018 72 FGD with male farmers in Aremo East, Abim District. Final Performance Evaluation of Northern Karamoja Growth, Health, and Governance Development Food Assistance Project – Final Evaluation Report 24 crop and livestock productivity interventions focused on household needs to achieve greater food security and to increase their resilience. One of the findings of qualitative data collection was that farmers ceased purchasing these seeds when the voucher program ended in 2017. Interestingly, interviews with farmers across the three districts found that they are still using sustainable agriculture practices, which have contributed to crop productivity, and these practices were diffused to farmers, both male and female, who did not directly participate in GHG agricultural activities by observation of demonstration gardens and their neighbors and applied to local seeds. This qualitative finding helps explain the PBS finding of an increase in the percentage of households using sustainable agricultural practices over the past 12 months, from 16.7 percent at baseline in 2013 to 37.2 percent in 2018. The interventions implemented to date for developing a commercial livestock industry have made progress. The multiple pathways employed to achieve this progress include rehabilitating one of the primary livestock markets, creating a new market for skins and hides, increasing feeder roads and extending the road linking major markets in Kotido and Kaabong, increasing access to drugs and vaccines and skilled veterinary care, and providing training in disease prevention practices and other practices to increase the health and productivity of livestock. The adoption of these practices by some agropastoralists resulted in the healthier livestock they brought for sale at the market for live animals peaked the notice of traders and buyers. The quality of these livestock attracted an increase in the numbers of traders and much higher prices for the livestock. The culture of cattle ownership and building and maintaining herds with no intention to sell remains important to the culture of Karamojong people. But the growth of the commercial livestock value chain has penetrated villages. To date, only wealthier farmers with large herds who can afford to sell a portion of their livestock have entered the commercial market and they are realizing the benefits of higher prices. Small owners have difficulty in building up their herd sizes and cannot afford to participate in the commercial markets. However, some smaller owners have benefitted from adopting practices to increase the health and productivity of their livestock, and the production of new livestock are sometimes sold locally to increase their income. Demonstrated increases in the successful capture and return of stolen livestock through interventions applied under SO1, and improvements in the peace and security in Northern Karamoja, have also contributed to the growth of this industry. The targeting strategy to improve the livelihoods of vulnerable households through involvement in GHG crop and livestock activities was implemented on the basis of baseline data showing that the majority of households in Northern Karamoja were either moderately or extremely insecure. Selection criteria for participation was based on criteria that did not include the type of gendered household or differences in poverty level. Although this approach likely contributed to the overall improvement in livelihoods as measured by the change in values of FFP poverty indicators between 2013 and 2018, women in adult female-only households realized no changes in their poverty level, indicating a need for more nuanced targeting. Targeting criteria should also be sure to capture the small livestock owners who are unable to participate in commercial market, who are otherwise being left behind. Promotion of Microenterprise Development and an Improved Business Environment The major technical interventions to promote additional sources of income through entrepreneurship were providing business development training and entrepreneurship skills to vulnerable populations, specifically women and youth, increasing the number of VSLAs and their viability, strengthening the capacity of SACCOs to provide financial services to expand member businesses, providing business development services to micro and small enterprise (MSME) owners through creation of business forums, and building advocacy skills to improve the environment for business operations in each district. The project supported 633 micro-businesses to receive business development services. Of those, 41 percent were businesses run by women, and 59 percent were businesses run by men.73 The project 73 Mercy Corps Final IPTT 2018 Final Performance Evaluation of Northern Karamoja Growth, Health, and Governance Development Food Assistance Project – Final Evaluation Report 25 also supported 140 female-owned businesses to enter the stocking agent/distribution networks that it supported. Some of the business development services were provided through the activities of business forums. GHG worked with three trader associations in Abim, Kotido, and Kaabong to establish a district-level business forum in each district. The purpose of the forums was to bring business people together so they have a unified voice to advocate for improved business environment in their respective district. Capacity development training was provided as well as some operational costs, enabling them to register and become formal entities, run association offices, hold annual meetings, conduct members’ business development, and organize exchange visits. In 2016, business forums had 514 members, of whom 47 percent were women. The business forums collaborated with the local government officials, specifically the District Commercial Officers and town council officials.74 Some of the achievements of their collaboration include the following: success in harmonizing cereal and grain prices among produce dealers (Kaabong and Kotido), creating improvements in the Kaabong markets so that market vendors were able to return to using a market they had abandoned, and lowering taxes levied to produce traders (Kotido and Kaabong). These achievements were made by business forum executives presenting the issues to the town council and local government leadership using their skills to reach agreement; and show the potential of business forums to support private sector expansion. Proof in the improvement in the business environment was recounted by a district commercial officer,75 who said that the evidence is shown by the increase in new product and services, more traders coming into the district to purchase and sell different commodities, including crops,76 the increase in different kinds and sizes of business, ranging from the micro-level street vendors selling food, clothing, and household goods to new small shops selling higher end cultural items that attract foreign residents and short-term consultants, small metal worker industries, stores selling electronic equipment, stationery, computer ink and paper, and the establishment of a large grocery store in town. Several issues impeded the growth of business forums and limited their ability to support small business development beyond the immediate district town areas and their contributions to private sector growth. One was the lack of ownership by business forum members. This is evident in their reluctance to pay the mandatory subscription fee critical to running and expanding the forums, as reported by forum managers in each district. According to district commercial officers, the entitlement attitude of members is also a hindrance to performance because forum members expect payments for attending trainings arranged for their own benefit.77 Members are relatively passive and do not actively contribute to program development and encouragement of new entrepreneurs. Among the targeted groups supported to initiate microbusinesses were 73 members from MCG+. The MCG+ model was piloted to test an approach that used interventions integrating livelihood improvement and MCHN activities to increase incomes and the health and nutrition status of PLW, one of the key vulnerable groups targeted under GHG. The 73 MCG+ members received a small grant, business development skills, and were helped to establish VSLAs with encouragement to “save with a purpose” to invest in the expansion of their enterprises. The women also received vouchers to purchase improved vegetable seeds for planting in permagardens next to their houses. Their use was promoted to encourage dietary diversity. Using the small grants, the women set up businesses for sale 74 KII with the Business Forum Chair of Kaabong, KII with two members of the Abim Business Forum Executive Committee, KII with District Commercial Officers for Abim 75 KII with the District Commercial Officer, Kotido District, October 1, 2018 76 KII with input dealer and food commodity trader, Kotido District, October 1, 2018 77 KII with business forum chair persons Kotido and Kaabong, KII with two members of the executive committee of the Abim business forum, KII with the Abim District Commercial Officer, KII with the Kotido District Commercial Officer, KII with agro-input dealer in Abim, KII with commodity and agro-input trader in Kaabong Final Performance Evaluation of Northern Karamoja Growth, Health, and Governance Development Food Assistance Project – Final Evaluation Report 26 of food items (sim sim, beans, vegetables, maize, silver fish), household commodities (soap, cooking oil, salt), purchase of small ruminants to slaughter and sell meat, and brewing and purchase of utensils for use in the brewing business.78 Women involved in the MCG+ pilot described how they were able to undergo a complete shift in the way they earned incomes. Previously, during the lean period, they coped by engaging in the charcoal trade, gathering wild fruits, providing casual labor, and collecting firewood, all of which provide very little money. After receiving small grants, and training in business, they began saving earned income from their microenterprises. They reported receiving enough income so that they no longer needed to engage in low-paying activities to support their household needs. They stressed that the profit generated from their businesses had enabled them to make food purchases, buy different foods to diversity their diets, meet household expenses (including for school requirements and health services) with more ease, and save on a weekly basis. They also reported having no problem repaying their VSLA loans. In one instance a woman used the same income to pay for the construction of her own latrine.79 GHG increased the accessibility of financial services in rural communities, and this was a key component enabling the start-up and development of microenterprises and necessary for their sustainability. GHG expanded the number of VLSAs and provided additional support to VSLA membership with training in financial management and agricultural practices. The project also supported other groups (MCG+, butchers’ and meat handlers’ association in Kotido, and others) to form and strengthen their own VSLAs. Members interviewed reported that capacity development support helped them develop strong procedures and rules for operating their VSLA, taught them the importance of saving, and for the first time, having a source of credit to set up small businesses, purchase food, and meet other household expenses.80 Women borrowed to start small businesses, and reportedly men commonly borrowed to purchase animals and animal drugs. No group mentioned challenges associated with loan defaults, indicating the quality of capacity development that GHG provided to VSLA members. Another component of the strategy to increase access to financial services was the provision of support to SACCOs. GHG supported SACCOs by strengthening their capacity to provide financial products and services through linkages for training, provision of incentives to encourage joiners, and provision of office equipment, safe boxes, and motorbikes. This intervention was found to be less successful than project interventions to increase and develop VSLAs. SACCO managers reported that membership increased but with more men than women registering as well as borrowing. SACCO managers interviewed reported poor savings and loan repayment rates and difficulties with debt collection. They reported that these problems impede their ability to develop programs, recruit more staff, and monitor membership that would support the growth of stronger SACCOs.81 The SACCOs provide the same products for all kinds of borrowers and have not developed differentiated financial products appropriate to different scales of business because of their limited resource base and staff. They are able to serve low-income customers but not the poorest because of the associated risk.82 Conclusions The livelihoods development component focused on micro and small enterprise development and improving the business climate for private sector growth, which is crucial for continuing the overall economic development of Northern Karamoja and ending its isolation, given the relative lack of 78 FGDs with MCG+ members in Longar, Sidok Sub-county Kaabong, FGDs with MCG+ members in Chaicaon, Regen Sub￾county in Kotido District 79 FGD with MCG+ in Sidok Sub-county in Kaabong District 80 KII with chairperson of Atobukuthe Akoro women’s group in Chaicaon in Kotido district, FGD with male agropastoralists in Longelep 81 KII with the manager Kitogongo SACCO, Kotido District and Ethur Rural Development SACCO in Abim District 82 KII with the manager Kitogongo SACCO, Kotido District Final Performance Evaluation of Northern Karamoja Growth, Health, and Governance Development Food Assistance Project – Final Evaluation Report 27 Government of Uganda investment in this region. GHG also addressed the limited availability of financial services in rural areas with their strategy to increase and strengthen VSLAs. The development of non￾agriculture based livelihood opportunities supported by the expansion of financial services is very important to scale up given the risks associated with climate change for entry into commercialized agriculture, the limited opportunities for youth to earn income, and in general, to address the circumstances of poverty in the communities of Northern Karamoja. 4.1.3 EQ4. Which project outcomes are likely to be sustained? Agricultural Service Providers and Continuity of Service Provision MC created and paid for the positions of agriculture extension workers (AEWs) in GHG. This made their service a short-term measure. The AEWs who have remained in the community said that farmers continue to approach them for advice, which they provide. The ability of AEWs to provide services beyond their immediate neighborhoods is hindered by a lack of resources. Willingness and the ability of farmers to pay for extension services is low. One AEW in Aremo East said that distant groups that sought his services become non-responsive after he proposed a fee of Ush 10,000 (US$2.6). The district production and marketing ccoordinators in the three districts have a good awareness of GHG interventions. MC participated in production sector working meetings hosted by the districts and submitted reports. The capacity of government offices to fill the AEW capacity gap is restricted by inadequate staff numbers in the agricultural and veterinary departments. A recruitment process is underway, but it is slow and constrained by resource limitations. The CAHWs, based in the communities, were established before the GHG project began by other donors and international organizations. Their longer period of existence compared to AEAs makes their services better grounded and more likely to be sustained. They are well linked to the district veterinary office and are recognized as an important resource for the understaffed veterinary departments for disease surveillance, mass vaccinations, and spraying or treatment. This linkage and being easily identifiable gives them access to refresher training and other support (e.g., veterinary kits) from other development agencies. However, earnings are still low, and government support is poor. One CAHW in Kotido said that he had vaccinated more than 800 animals on behalf of the district without receiving even a bottle of water to drink. The CAHWs complement this source of income with other activities such as brick making,83 cash for work,84 and farming. This has the potential to affect their availability to offer animal health services. Inputs Service Delivery for Increasing Crop and Livestock Productivity The evaluation findings showed that national and regional suppliers have established relationships with input dealers and agro-vets. Input dealers and agro-vets are confident in placing orders from sources established during the project and have gone further to identify other sources of veterinary supplies and seed in Soroti. Interviews with input dealers, agro-vets, and national seed companies show that credit arrangements established at the beginning of the project have not continued. With diminished farmer demand for improved seeds, suppliers cannot be certain that loans made to district and village agents for stocking seeds will be repaid. Commercial seed producers and other input suppliers now require up￾front cash down-payments. Seed companies consulted said that the business behaviors (in general, not limited to Karamoja) make supply on credit risky in the absence of a credit guarantee. One agro-vet in Kotido who had independently purchased Ush 24,000,000 (US$6,469) worth of de-worming drugs on credit in 2017, had not repaid the full amount by the time of the evaluation. 83 KII with CAWH, Atwilo, Abim District, September 29, 2018 84 KII with CAHW, Logelep, Kotido District, October 4, 2018 Final Performance Evaluation of Northern Karamoja Growth, Health, and Governance Development Food Assistance Project – Final Evaluation Report 28 Without credit arrangements between input dealers and input suppliers, input dealers were less able to provide timely, reliable delivery of services and inputs for agents and farmers. Under these circumstances input dealers procure smaller volumes that they can afford to make down payments for, sell, and restock, which increased chances of stock-out gaps. Interviews show that input dealers could borrow from financial institutions such as SACCOs and commercial banks to stock their shops adequately if they wanted.85 Some dealers borrowed the money necessary to make the required 10 percent of order value to the input suppliers during GHG from the same sources.86 The same arrangements can be drawn on if needed, and if dealers are working with a much lower demand. Other input sellers are still averse to bank loans for business because of bad experience and prefer to stock a little, sell, and restock. Other input dealers have the advantage that other development agencies channel seed distributions through their business.87 All input dealers interviewed operate multiple business, such as inputs, produce (grain, legumes, and cereals), and construction or commodity retail (soap, oil, etc.). This implies that the businesses will continue to operate, albeit that they will need to ensure that services are timely and reliable. Financial Services, Business Development, and Business Advocacy Interviews with the managers show that VSLAs are operating without external guidance and relying heavily on the principles taught during GHG. All the VSLAs visited had well-established procedures of operation that were strictly followed. This is makes them likely to continue as long as members contribute savings and repay their loans. VSLAs still require guidance, however, and can be misled by erroneous advice from authoritative persons. For instance, when asked why the VSLA had not increased its members, one VSLA manager responded that MC said that 30 members was a good number. She added, “We are happy to increase the number if you advise us to do so.” The continuity of some SACCOs to provide financial services to small businesses and poor people is limited by a number of factors that inhibit the realization of their full potential, including the challenge of limited savings deposited by members, poor repayment of loans and loan defaults, poor debt collection, inability to develop differentiated products for different clientele because of limited resources due to poor savings and loan repayment practices, low human resource capacity to develop the products, and difficulty in attracting new joiners. The SACCOs are able to borrow from the Microfinance Support Center but must meet standard requirements laid out in policies, procedures, and regulations for SACCOs. Without more capacity building to develop realistic and achievable strategic plans and training in behavior change communication (BCC) messaging to highlight the benefits of membership and to encourage loan repayment, the SACCOs supported to date may not be able to sustain themselves. The business forums are well linked to district local governments and town councils that have offered office space (Kaabong and Abim). The business forums have also entered into partnership with other agencies such as with the United Nations Development Programme on cross-border trade to broker trade relationships with the Turkana as a means of peace building.88 The ability of business forums to continue delivering business development services and advocacy to improve business environments is uncertain because they have a number of challenges, such as limited membership payment and involvement as discussed in Section 4.1.2. It will be difficult for the forums to continue operating without additional capacity building and willingness of members to pay monthly dues. Interviews with forum leaders show that the relevant local 85 Interviews with business forum chairpersons and executives in Abim, Kotido, and Kaabong indicated the loans are easily available from SACCOs, VSLA, and commercial banks (specifically Centenary Bank) as long as records are in order. 86 An input dealer in Abim Town said that she made the 10 percent down payment on inputs orders during GHG by borrowing from a commercial bank (Centenary bank) and a SACCO. 87 KII with agricultural input dealer and commodity food dealer in Kaabong Town 88 KII with chairperson Kaabong Business Forum Final Performance Evaluation of Northern Karamoja Growth, Health, and Governance Development Food Assistance Project – Final Evaluation Report 29 government departments are not likely to provide this support because they also expect allowances. Over the past five years, the business forums mainly operated using financial support from the GHG and the USAID-funded SPEED project implemented by the International Rescue Committee, both of which have since ended. GHG provided all three business forums with equipment to operate business centers as income-generating sources at the start of 2018, but these activities have not drawn in sufficient numbers of customers to support the forums. All three business forums had not been able to operationalize the centers to their full potential by the time fieldwork for the evaluation was conducted. 4.1.4 EQ5: What are the key lessons learned and best practices that should inform future projects in the country? Targeting approaches that do not clearly define the criteria for selection of participants for project activities without taking into account the variation in poverty within communities and differences in circumstances between gendered types of households are likely to exclude women from adult female only-households and households with fewer resources from accessing services that support strengthened livelihoods and resilience. The business forum approach helped improve member businesses and contributed to changes in local regulations that inhibited private sector growth. However, implementation approaches must be designed to deal with entitlement attitude of members found in each district who do not pay their monthly membership dues. This factor limits the potential of business forums to become an effective force for substantial increases in private sector development. Other means of income generation that are feasible in the Karamoja environment should be explored to support the growth of business forums coupled with further capacity building activities. The GHG interventions that supported start-up of micro-businesses, business development, and increased access to financial services through the development and strengthening of VSLAs can lead to a shift away from low-paying unskilled labor activities and reliance on natural resource exploitation. This includes the pilot activity for the pilot MGC+, combining interventions from livelihoods development and MCHN/WASH and leading to a more dramatic shift in income-earning power. With their profits, these mothers were able to make food purchases, meet household expenses with more ease, and buy different foods to diversify their diets and save on a weekly basis and expand their businesses. These approaches should be expanded to include more women and greater numbers of youth, while at the same time steering them into different product areas following market assessments to identify areas of unmet needs. The M4P strategy for bringing farmers into commercial agricultural markets and increasing livelihoods through the promotion of climate smart seeds to improve crop productivity is risky, given the climate change issues that affect Northern Karamoja and the high price of seeds. However, interventions to adopt sustainable agricultural practices have been shown to increase productivity are important to continue to address food consumption needs. Cultural factors that inhibit the further development of a livestock industry that could truly strengthen livelihoods must be thoroughly understood and respected before making further investments in a livestock value chain. Cattle are an important cultural symbol in Karamojong society. Cattle ownership and the size of cattle herds are important markers of prestige, but the importance of cattle goes beyond the size of one’s herd. Karamojong people do not want to sell their cattle except to resolve urgent household needs. At the same time, the commercialization of the livestock industry, including interventions to increase livestock health, are important contributors to livelihoods. Despite the general unwillingness of the Karamojong to sell their livestock for their commercial value, the commercial livestock model has begun penetrating the community level by those who recognize the various opportunities (live cattle, milk, hides and skins, beef, etc.) and benefit from participation in the market are mainly wealthy farmers with large herds and livestock traders. Without interventions focused on Final Performance Evaluation of Northern Karamoja Growth, Health, and Governance Development Food Assistance Project – Final Evaluation Report 30 assisting small holders with few resources for investment in drugs and purchase of animals to increase herd size, the commercialization strategy risks leaving these farmers behind. 4.2 SO2: NUTRITIONAL STATUS OF CHILDREN UNDER AGE FIVE IMPROVED 4.2.1 EQ1: To what extent has GHG met its defined goals, purposes, and outcomes? SO2 was designed to improve the nutritional status of children under 5 years of age. Achievement of this objective was measured by FFP indicators on the prevalence of underweight, stunted, and wasted children under 5 years of age. The change in indicator values from the 2013 baseline to 2018 endline values and the degree of significance of change are shown in Table 5. Table 5: Performance Indicators for SO2 Indicator Baseline Endline Significance Level Household dietary diversity score 2.2 3.6 *** Prevalence of underweight women of reproductive age 20.9% 24.8% ns Women’s dietary diversity score 2.1 3.1 *** Prevalence of underweight children under 5 years of age 21.5% 27.7% * Prevalence of stunted children under 5 years of age 32.0% 37.6% † Percent of wasted children under 5 years of age 13.7% 9.7% * Prevalence of children 6–23 months of age receiving a minimum acceptable diet 2.1% 9.4% *** Prevalence of exclusive breastfeeding of children under six months of age 60.8% 71.7% ns ns=non-significant, † p<0.1, * p<0.05, ** p<0.01, *** p<0.001 Overall, the data show that the nutritional status of children under 5 years of age has not improved. With the exception of the prevalence of wasting, which shows a decrease from 13.7 percent to 9.7 percent, the prevalence of underweight and stunted children shows an increase from baseline. Some of the practices associated with improving the nutritional status of children, specifically, providing children 6–23 months of age with a MAD has increased, as has the WDDS for women of reproductive age and the HDDS. Based on these data, we can suggest that without these increases in dietary diversity, children’s nutritional status may have been even lower. The prevalence of exclusive breastfeeding (EBF) of children under 6 months of age has increased, from 61 percent in 2013 to 71 percent in 2018, but it remains under the minimum of 80 percent as required by international standards. Several factors related to children’s health status, health and sanitation practices, feeding practices, and level of income together may be barriers to the improvement of the nutritional status of children under five. First, there is an established link between prolonged diarrhea and malnutrition (Table 6). Table 6: FFP Performance Indicators for Health and Hygiene and Sanitation Practices Indicator Baseline Endline Significance Level Prevalence of children under 5 years of age with diarrhea reported in the past two weeks 21.3% 35.4% *** Prevalence of children under 5 years of age with diarrhea treated with oral rehydration therapy 89.7%a 81.8% * Percentage of households using an improved source of drinking water 37.4% 37.9% ns Percentage of households using improved sanitation facilities 12.7% 13.7% ns Final Performance Evaluation of Northern Karamoja Growth, Health, and Governance Development Food Assistance Project – Final Evaluation Report 31 Indicator Baseline Endline Significance Level Percentage of households with access to sanitation facilities— not necessarily improved 40.5% 49.0% ns Percentage of households with soap and water at a handwashing station commonly used by household members 11.0% 2.6% *** a The baseline indicator result was recalculated to exclude the use of “increased fluids” which is considered a treatment for diarrhea but not an ORT treatment. ns=non-significant, † p<0.1, * p<0.05, ** p<0.01, *** p<0.001 As shown in Table 6, PBS findings for hygiene and sanitation measures show an increase in the prevalence of diarrhea reported over the past two weeks, from 21. 3 percent to 35.4 percent, and a reduction in the use of oral rehydration therapy (ORT), from 89.7 percent to 81.8 percent. Children who experienced diarrhea in the past two weeks are 70 percent more likely to be stunted regardless of their households’ project participation status and socio-economic conditions (see Table A7.1, Annex 7). It is important to note that the PBS endline data were collected in June and July during the rainy season when diarrhea rates are expected to be higher. There were heavy rains and serious flooding during this time period. 89 Some of this increase may be related to this incident, which shows the difficulty of using this indicator to measure progress toward decreasing the prevalence of diarrhea. At the same time, we also looked at change in the use of three types of treatments. As shown in Table 7, the use of sachets for treatment of diarrhea did not change, and there was an increase in the use of reconstituted oral rehydration salts (ORS) fluids and government-recommended homemade fluids from baseline to endline. These increases are masked when looking at the overall indicator, which decreased at endline. Note that more than one method may be used per child, so we are seeing an increase in the types of ORT treatments being used for the child, but not an increase in the number of children receiving any type of ORT treatment at all. This may indicate the results of support provided by GHG, by stocking of additional types of ORT treatments available at health facilities or outpatient clinics. The increase in the use of government-recommended homemade fluids from 13.4 percent to 38.6 percent may reflect the effects of training provided to mothers in MCGs. Table 7: Types of ORT Treatments ORT Treatments Baseline 2013 Endline 2018 Significance ORT sachets such as Zinkid or RESTORE 74.6 % 78.6% Reconstituted ORS liquids provided by government health facilities 30.6 % 62.2% *** Government-recommended homemade fluid 13.4% 38.6% *** ns=non-significant, † p<0.1, * p<0.05, ** p<0.01, *** p<0.001 Data on FFP measures for access to safe sources of water and sanitation and hygiene practices from 2013 to 2018 indicate that some of the increase in the prevalence of diarrhea at endline may be linked to these factors. Use of improved sanitation facilities showed no increase in the already low level of use. At the same time, the percentage of households with availability of soap (or ash) and water at a handwashing station decreased, from 11 percent in 2013 to 2.6 percent in 2018 at endline. However, regression results (models 5 and 6, Table A7.1, Annex 7) show that households with access to soap and water at a handwashing station are 40 percent less likely to have stunted children regardless of their socio-economic conditions. The use of safe drinking water is a factor associated with the prevalence of diarrhea, but the percentage of households using an improved source of drinking water has not increased from the baseline level of 37.4 percent. Children from households with an improved drinking water source are 30 percent less likely to be wasted regardless of project participation and socio￾economic status. (models 5-7, Table A7.2, Annex 7).The use of sanitation and other practices is 89 Email communication between PBS field enumerators and ICF reporting on the difficulty of reaching villages because of heavy flooding. Final Performance Evaluation of Northern Karamoja Growth, Health, and Governance Development Food Assistance Project – Final Evaluation Report 32 discussed in detail under EQ2, which presents findings on the effectiveness of technical interventions and the methods and processes used to implement interventions to achieve project outcomes. A second set of factors that may be contributing to the poor nutritional status of children under 5 years of age are the cultural feeding practices and the number of calories consumed by children. In Northern Karamoja, cultural values include the importance of sharing all sources of food equally among members of the household.90 A number of respondents interviewed in Kotido and Kaabong91 reported that food rations distributed by GHG meant specifically for PLW and children under 2 years of age were actually shared among all household members, indicating that children were not getting the full benefit of the additional caloric intake. We do not have data on calorie consumption of children under 2 years of age to compare between baseline and endline to examine possible links of children’s malnutrition data with feeding practices. The level of poverty in some households may be another factor affecting children’s nutritional status Figure 1 shows the data on malnutrition indicators in 2018 for all children under 5 years of age and provides a table on the prevalence of malnutrition (stunting and underweight) in children under 5 years of age and the percentage of children under 2 years of age receiving a MAD by household consumption quintiles in 2018. The data show that the prevalence of underweight children in the poorest quintile is 32.7 percent, compared to 19.8 percent of children in the richest quintile. The prevalence of stunted children in households in the poorest quintile is 37.8 percent, compared to 28.6 percent of children in households in the richest quintile. This indicates that the level of poverty affects nutritional outcomes. We cannot assume that children in the higher consumption quintiles automatically have higher MAD values. About 3.5 percent of children between 6–23 months of age consume a MAD in the bottom third quintiles (quintiles 3, 4, and 5), 7 percent of children in the poorest quintile consume a MAD. There may be several factors that affect children’s MAD values other than the income. Note that the percentage of children consuming a MAD in the poorest quintile, at 7.1 percent, is still triple the total value of all children consuming a MAD at baseline in 2013 (2.1 percent). Figure 1: Malnutrition in Children under 5 Years of Age *** = p<0.001. ** = p<.01, * = p<.05, + = p<.10 90 This practice was described by evaluation team member Dr. Frank Muhereza, who has conducted research on traditional political systems and cultural practices in Northern Karamoja for 30 years. The practice of sharing food is also noted in the Food Security Desk Review, FANTA, p. 79, 2017. 91 This was brought up several times during FGDs with MCGs and KIIs with village health team members, and in one case, by the sub-county health assistant. 21.5% 32.0% 13.7% 27.7% 37.6% 9.7% 0% 20% 40% 60% 80% 100% Prevalence of underweight children under 5 years of age Prevalence of stunted children under 5 years of age Prevalence of wasted children under 5 years of age 2013 Baseline 2018 Endline * * † Final Performance Evaluation of Northern Karamoja Growth, Health, and Governance Development Food Assistance Project – Final Evaluation Report 33 Table 8: Prevalence of Malnutrition of Children under 5 Years of Age and Percentage of Children 6–23 Months of Age Receiving MAD by Household Consumption Quintiles Prevalence Underweight Children Percent (n) Sign.1 Prevalence Stunted Children Percent (n) Sign.1 Percentage Receiving MAD Percent (n) Sign.1 Quintile 1 (ref), Poorest 32.76 (215) 37.80 (215) 7.10 (66) Quintile 2 36.19 (212) 45.39 (212) 7.99 (55) Quintile 3 24.92 (213) 40.88 (212) 3.58 (56) Quintile 4 24.89 (213) 34.72 (209) 12.40 (67) Quintile 5 (Richest) 19.81 (216) * 28.66 (213) + 15.40 (62) All Children 27.7 (1,069) 37.6 (1,061) 9.4 (246) 1 Significant results (*** = p<0.001. ** = p<.01, * = p<.05, + = p<.10) are in relation to first quintile The prevalence of underweight women when examined by household consumption quintiles (see Table 9) shows that the prevalence of underweight is higher in women in the poorest quintile (36.1 percent), compared to women in the richest quintile (19.8 percent). Women in the poorest quintile show a lower WDDS score, compared to women in the richest quintiles. Of note are the data comparing the prevalence of underweight women and WDDS for women in adult female-only households with no education and no other cash earner in the household with women in the richest household consumption quintiles. The prevalence of underweight women in adult female-only households is higher, but they have a dietary diversity score, which is similar to those women in quintile 4. Again, the WDDS can be influenced by various factors, including cultural factors, at the community and household-level, and this needs further analysis to understand why the average value for the fourth quintile is similar (at least in an absolute terms) to adult female-only households with no education and no other cash earner in households. Table 9: Prevalence of Underweight Women and WDDS by Household Consumption Quintiles, 2018 Prevalence of Underweight Women Percent (n) Sign.1 WDDS Percent (n) Sign.1 Quintile 1 (Ref.), Poorest 35.10 (166) 2.76 (214) Quintile 2 25.36 (166) * 2.73 (214) Quintile 3 26.63 (166) + 3.17 (214) * Quintile 4 24.00 (166) + 2.87 (214) ** Quintile 5 Richest 13.55 (165) ** 3.80 (214) *** All women 24.76 (829) 3.07 (1,070) Women in adult female-only households with no education and no other cash earner in household 41.9 (57) 2.83 (84) 1 Significant results (*** = p<0.001. ** = p<.01, * = p<.05, + = p<.10) are in relation to first quintile Final Performance Evaluation of Northern Karamoja Growth, Health, and Governance Development Food Assistance Project – Final Evaluation Report 34 Factors Involved in the Decrease in Prevalence of Wasting in Children under 5 Years of Age The prevalence of wasted children under 5 years of age decreased, from 13.7 percent in 2013 to 9.7 percent in 2018. Qualitative data analysis shows that several factors may be contributing to this decrease. Women in MCGs participating in FGDs held in Kotido and Kaabong noted the general improvement in the nutritional status of children during the GHG project. It is likely that these women were referring to acute malnutrition or wasting.92 The signs of moderate and severe wasting are easily recognizable by mothers, and they were referring to a remission of those signs when they stated that the nutritional status of their children had improved. It is important to stress that WFP and its partners, such as Community Action for Health, operated in the same implementation areas as GHG with acute malnutrition recuperation programs. Reasons for the general decrease in the prevalence of wasting can be found primarily due to early referral and treatment of acutely malnourished children based on mid￾upper arm circumference (MUAC) measurements taken by trained village health team (VHT) members and leader mothers (LMs). This helps to explain why the mothers interviewed reported that their children’s nutritional condition improved. Other contributing factors include BCC messages aimed at MCGs to increase knowledge of the importance of food diversity and to introduce kitchen gardens for vegetable production, increased understanding of the importance of EBF from birth, and in general, increased access to quality health care.93 There is one more factor to consider in the decrease in the prevalence of stunting. Considering that the PBS endline was carried out in June/July 2018 during the end of the lean season before harvests, the anthropometric measurements for wasting should have revealed many more cases of wasting. The fact that wasting prevalence decreased may also be attributable in part to the previous harvest season (2017), which was widely reported as very prolific94. Some farmers participating in FGDs (see Section 4.1.2 under Crop Production) in each of the three districts reported experiencing a shorter lean season because they were able to continue consuming the food from that harvest until mid-2018. Targeting Strategies and Their Contribution to Achieving Project Goals, Especially with Regard to Gender and Reaching the Most Vulnerable MC targeted PLW and children under 2 years of age, considered under the GHG project to be the most vulnerable population group, following the Preventing Under Two Approach (PM2A) in which the first 1,000 days of life of a child (from conception until 24 months of life) are targeted. This strategy was, in general terms, the most appropriate to contribute to malnutrition prevention and to address conditions in the most vulnerable moments of life. As entry criteria, GHG selected PLW and children under 2 years of age in the targeted locations with an active antenatal card from the health facility and an updated child health card, and exit criteria was set at 24 months of age of the child. World Vision used Last Mile Mobile Solution95 to track the beneficiaries. PLW were permitted to enroll once during the life of the project which served as a deterrent for women to become pregnant more often to receive food; however, that also meant that women who became pregnant 18–24 months after the first pregnancy when GHG was still ongoing were excluded and did not received any food rations. Although this enrollment strategy maximizes coverage, it challenges the project administrators to start up rapidly with BCC materials and health services from the beginning of the project. It also enrolls older children who can participate in the project for few months prior to discharge, which is not enough time 92 Characteristics that MCG FGD participants mentioned include “very thin children,” “skeletons,” “color of the hair changes to blond,” “legs inflated” 93 These technical interventions are discussed in detail under EQ2 (influence on project design and TOC on the effectiveness of technical interventions and under EQ3 (the effect of implementation methods and processes on the outcomes of technical interventions). 94 See also FEWSNET Uganda, 2017 confirming the size of the harvest that year. 95 https://www.wvi.org/disaster-management/last-mile-mobile-solution-lmms Final Performance Evaluation of Northern Karamoja Growth, Health, and Governance Development Food Assistance Project – Final Evaluation Report 35 to improve their nutritional status or effect behavior change. The evaluation found that 20 percent of this initial cluster of children assisted with food rations were graduated after 18 months and did not receive the full package of interventions for the PM2A approach. In October 2016 and three months from closure of the food rations distribution, the project was still enrolling 504 new children under 2 years of age. GHG closed food distribution with approximately 3,290 PLW and 7,100 children under 2 years of age still in the database96 who had received food for 12 or less months and who represent approximately 17 percent of the total participants. For those beneficiaries, the level of “absorption” of the BCC messages delivered by the MCGs is potentially higher than the first groups enrolled in 2013 because they were exposed to the messages throughout GHG implementation, but the caloric intake from the food rations, diluted with the sharing practices, impacted less on their nutritional status. The project phased out the food rations 12 months before the end of all activities. Although there are not strict policies to support any one strategy for phase-out or phase-over of the PM2A, with this number of beneficiaries GHG would have stopped the enrollment of pregnant women 12 months prior to December 2016 and continued for at least 6 months until June 2017 and exited the children enrolled lately in 2016 at 18 months of age. Food rations were conditioned on joining a MCG and receiving a package of preventative and curative health services delivered by the health unit. GHG chose the MCG model to spread BCC messages, and mothers became MCG participants after the enrollment in the food rations. They were pleased with the food ration composition and attributed the well-being of their children to it. LMs were chosen by community members for their motivational skills and their engagement at the community level. They were members of the community and also food beneficiaries. MCG members, VHTs, and sub-county health assistants reported that the food rations, including those designated exclusively for the PLW and children under 2 years of age, were shared in the household, with extended family, and, in few instances, also with neighbors. MCG members, MCG+ participants, VHTs, directors of the health facility in charge, and MCAs who were interviewed97 were all consistently in agreement that the food distributions had improved the nutritional status of children during the implementation of GHG. Children from households that received “food rations and nutrition training” are approximately 20 percent less likely to be stunted at endline (see Table A7.1, Annex 7). GHG conducted monthly food distributions in 36 food distribution points across sub-counties and parishes in the three districts of operation. The project exploited the opportunity of the beneficiaries coming from their villages to collect their monthly ration to provide health services using an outreach model. Respondents, including MCG promoters, health unit staff, and health district authorities, also reported an increase in use of health care services, mainly for antenatal care (ANC) visits, child immunization, and institutional deliveries. This increase in use may have been prompted by several factors such as the just mentioned conditional provision of the food commodities and the combined outreach activities provided by the health facility staff, coupled with the increasing quality of health services delivery (discussed under EQ3). Community screening of cases of acute malnutrition conducted by VHTs and LMs trained to take MUAC measurements also prompted an increase in use of health care services. Participants had clearly become more accustomed to and more comfortable using health unit services because of the stimulus provided by the food distributions.98 Table 10 provides data on the distribution of food rations and the number of children under 5 years of age reached by nutrition interventions. 96 GHG Food distributions sheet, December 31, 2016 97 Qualitative analysis of the data from these respondents in Kotido and Kaabong corroborate this statement. However, the findings from PBS data on malnutrition indicators does not confirm that these statements are true. 98 The evaluation considered including the use of food as incentive to seek health care services as good practice. However, this is not an efficient practice because of the very high costs of moving food commodities. Final Performance Evaluation of Northern Karamoja Growth, Health, and Governance Development Food Assistance Project – Final Evaluation Report 36 Table 10: Food Ration Distribution and Adoption of Health and Nutrition Practices Indicator LOA Target LOA Achievement Percentage of Target Achieved Number of households adopting three or more improved gardening or poultry production practices promoted by the project 4,165 3,584 86% Number of children 6–23 months of age receiving PM2A rations 46,616 40,441 86.7% Number of PLW receiving rations 21,922 18,222 83% Number of individuals receiving a protection ration, disaggregated by sex 243,271 201,900 83% Number of children under 5 years of age (0–59 months) reached by nutrition-specific interventions through USG-supported nutrition programs 93,470 76,968 82.3% LOA=life of activity GHG reported that food ration distributions had an impact on 18,222 PLW, 40,441 children under 2 years of age, and 201,900 participant households. This is slightly less than the targets projected for 2018, but the project has achieved impact on behavioral change regarding the knowledge and use of diversified food use.99 The application of this knowledge is shown by PBS data on the increase in the percentage of children under 2 years of age consuming a MAD from 2013 to 2018. In additional to the health and nutrition messages, the concept of permagardens to promote diet diversity and production of nutrient-rich crops was introduced to LMs, who in turn introduced them to members of MCGs. Cooking demonstrations were provided to LMs as a complementary activity to gather mothers together and teach them how to use local food produced in their own kitchen gardens and fields. GHG did not include specific numbers of cooking demonstrations in quarterly or annual reports. However, in 2016, the project reported that approximately 2,800 households had created permagardens to have year-round access to vegetables.67 Vegetable, legumes, and cereals seeds were distributed to MCGs through e-vouchers that were used to promote uptake of improved seeds by subsidizing part of the costs. Vouchers for seeds were also distributed as part of the intervention package for the MCG+ pilot activity that integrates livelihood improvement interventions from SO1 with activities from SO2. Interviews held with MCGs indicated that the operationalization of the kitchen gardens component decreased from what was reported during GHG by 2018 but we do not have sufficient interview data from villages across districts that would support a more robust qualitative finding that diverges from the from MC’s IPTT data. Nonetheless, training provided on how to grow vegetables is still being used by care group members who are able to access individual loans or share￾outs from the VSLA to purchase vegetable seeds (i.e., MCG+). Input dealers interviewed in the three districts reported that while farmers are no longer purchasing improved seeds for grains, they are buying vegetable seeds (see Section 4.2.1 under Crop Production) which are cheaper than improved grain seeds. Poultry production, one of the GHG interventions to improve women’s livelihoods, was barely mentioned during the FGDs with MCGs and LMs. The QET team did not collect sufficient data about these activities or on the use of chicken eggs and meat for household consumption to draw any conclusions about the contributions of poultry production to nutritional status. Conclusions Quantitative evidence from the comparison of PBS baseline and endline data show a lack of improvement in the nutritional status of children under 5 years of age with respect to the prevalence of 99 We could not find any information on how the annual targets for food rations were calculated or if the annual expected number of births was taken into consideration in the MC ARRs or in other documents. Final Performance Evaluation of Northern Karamoja Growth, Health, and Governance Development Food Assistance Project – Final Evaluation Report 37 underweight and stunted children. The QET maintains that targeting PLW and children under 2 years of age for interventions based on the PM2A approach to decrease the prevalence of malnutrition was an appropriate strategy. Findings from the analysis of the qualitative and quantitative data indicate that several factors may be undercutting the effectiveness of this strategy, prohibiting improvement in nutritional status. These include hygienic and sanitation practices, cultural feeding practices, and the degree of poverty measured by household consumption quintiles. The PBS findings contrast with statements collected during fieldwork, indicating a general “improvement of the nutritional status of children” during GHG reported by MCG members, LMs, and health facility staff. Our analysis shows that women in MCGs and LMs are probably referring to acute malnutrition or wasting. The signs of moderate and severe wasting are easily recognizable by mothers who referred to a remission of these signs of wasting when they reported the nutritional status of their children improved under GHG. Early referrals of acute malnutrition cases by VHTs and LMs for treatment may be a key factor in the decrease of prevalence of wasted children under 5 years of age. Contributing factors for the general decrease in the prevalence of wasting include the practice of breastfeeding promoted by MCG BCC messages, the increased knowledge about food diversity that GHG introduced, mainly through use of a kitchen garden model and cooking demonstrations, and in general, increased access to nutrition and health services and reported improvements in the quality of health care. One of the biggest benefits from the conditional food rations for the SO2 target group, PLW and children under 2 years of age, was the increased use of health services by participants promoted by the project. GHG reported the food ration distributions reached 18,222 PLW, 40,441 children under 2 years of age, and 201,900 participant households. This is slightly less than the targets, but the project has achieved impact on behavioral change regarding the knowledge and use of diversified food use for child growth and overall household well-being. Dietary diversity plays an essential role in nutritional outcomes. However, given the persistency in elevated levels of undernutrition, the data show that the increase in household dietary diversity and MAD was insufficient to improve the nutritional status of children under 5 years of age. Solving this problem will require research on the critical barriers in household feeding practices, income, and sanitation practices. 4.2.2 Strengths and Challenges of the Project Design, TOC, and Implementation Methods That Influence the Effectiveness of Technical Interventions in Achieving SO2 Goals GHG designed three intermediate results (IRs) to support the improvement of nutritional status of children under 5 years of age. Those results aimed to improve access to quality health care and nutrition for mothers and children, improve health and nutritional practices at the household level, and reduce diarrhea incidence in children through increasing access to safe water and sanitation facilities. The GHG SO2 TOC was revised shortly after the project’s inception. The SO changed its focus from children under 2 years of age to children under 5 years of age. IR 2.2 transformed from “Household maternal and child health and nutrition practices improved” to “Increased consumption of nutritious foods for households,” and IR 2.3 changed from “Sustainable access and appropriate use of safe water and sanitation facilities improved” to “Reduced incidences of diarrheal diseases among children.” As described in GHG annual reports, the maternal and child health strategy also aimed to address the “Three Delays.” This model proposes that pregnancy-related issues are overwhelmingly due to delays in (1) deciding to seek appropriate medical care, (2) reaching an appropriate facility, and (3) receiving adequate care when a facility is reached. The remainder of this section of the report discusses the effectiveness of technical interventions for each of the three IRs that were designed based on the revised TOC and the strengths and challenges of implementation methods that influence the effectiveness of interventions in improving children’s nutritional status. Final Performance Evaluation of Northern Karamoja Growth, Health, and Governance Development Food Assistance Project – Final Evaluation Report 38 Figure 2: GHG SO2 Revised Theory of Change 4.2.2.1 Access to Quality Maternal and Child Health and Nutrition Services Improved IR 2.1 MC has one indicator to measure increases in access to health care, the percentage of births receiving at least four ANC visits during pregnancy. The ANC rate reached 77 percent in the GHG targeted areas at endline, compared to 49.2 percent at baseline. Our analysis of interviews from the collection of qualitative data at the district, sub-district, and village levels combined with quantitative data from the MC IPTT at the output level does provide a greater range of evidence to indicate that GHG has been successful in increasing community access to quality maternal and child health and nutrition services. Health Unit Management Committees and Village Health Teams.100 The central components of the GHG design to increase access to quality health care focused on improving facility governance through functioning Health Unit Management Committees (HUMCs), training and supporting voluntary village health team (VHT) members, constructing and equipping health centers, and temporarily placing medical students to transfer knowledge between the students and the local staff. The HUMCs serve as a community-based accountability mechanism with the power to oversee facility management and represent community concerns. An initial training of trainers provided by the Ministry of Health and supported by GHG was cascaded to HUMCs and sub-county health assistants in the three targeted districts. This training has increased their knowledge of roles and responsibilities. The reconstitution of HUMCs in all health facilities and the subsequent training of the committees ensured transparency in financial management and general administration of the health unit resources. These results were reported from FGDs with HUMCs and VHTs,101 and from KIIs with health facility in￾charges across the three districts of Abim, Kotido, and Kaabong, At the same time, training was also provided to 505 government health staff in planning and management of health facilities, exceeding its 100 Many of the data for the IR 2.1 at health facility level and with local health authorities reflect the information collected in the three districts of Abim, Kotido, and Kaabong. Community-based interviews were not conducted in Abim district as noted in the evaluation limitations. 101 Chachaion, Longelep, Chumakod Final Performance Evaluation of Northern Karamoja Growth, Health, and Governance Development Food Assistance Project – Final Evaluation Report 39 target by 50 percent. The twinning of these interventions contributed to better management and financial accountability of health facilities, including drugs management. The use of community dialogues and exchange visits was an effective means to increase the quality of health care as evidenced by reports of sub-county health assistants102 and HUMCsthat health staff attitude and behavior toward clients have become better and waiting times have been reduced. Clients have the possibility to “speak-up” using Citizen Report Cards that GHG implemented to increase transparency and the quality of health care provision at health facilities. During the implementation of the project, HUMCs were able to engage communities through the VHTs and link with sub-county health representatives. GHG reported that more than 6,745 people participated in community dialogues in 2016–2017 to increase the exchanges between local population and the health unit service providers. GHG organized 50 exchange visits in 2015 with HUMCs from other sub-counties to strengthen knowledge sharing and create a role model for other HUMCs. GHG build the capacity of VHTs through multiple rounds of training and refresher trainings delivered in collaboration with the health district officer and sub-county health assistants and regular support. GHG training included lessons (or modules) in family planning, infant and young child feeding, nutrition and care-seeking behavior through pregnancy and early infancy, acute malnutrition training on MUAC, and hygiene promotion. VHT members were also trained on the MCG model and served as MCG promoters. VHT members were trained to mentor and “cascade” their training on each lesson to LMs. The LMs disseminated the lessons to their neighbor women in MCGs. This training and regular support provided to VHT members through monthly review and mentoring meetings was crucial in strengthening the bond between the VHTs and the health facilities. The sensitization provided by VHTs contributed to the reduction of home deliveries and increase in community knowledge of the importance of hygiene practices. GHG’s preventative feeding component, discussed under the section on SO 2 targeting strategies, also contributed to the increase in access to health and nutrition services. This project component brought together food recipient beneficiaries (children under 2 years of age and PLWs) on a monthly basis to pick up their monthly ration. GHG used the opportunity to provide health services using an outreach model. Health facility staff coordinated their outreach activities with the day of food distribution. Overall, GHG provided training in child health and nutrition to 38,935 people (92 percent of the target), as shown in Table 11, with messages including the importance of going to health facilities for ANC visits, immunizations, and institutional births. Table 11: GHG Support to Health Units and Community-based Staff Indicator LOA Target LOA Achievement Percentage of Target Achieved Number of government health staff trained in planning and management of health facilities 250 505 202% Number of people trained in child health and nutrition through USG-supported programs 42,238 38,935 92% Number of VHTs consistently attending monthly review and mentoring meetings supported by the USG 2,478 2,243 90.5% Note: GHG support provided in Abim and Kotido (Abim Town, Rengen, Nakapelomoru) Morulem, Arembwola-Abim, Loleila, Sidok 102 In Abim and Kotido (Abim Town, Rengen, Nakapelomoru) Morulem, Arembwola-Abim, Loleila, Sidok Final Performance Evaluation of Northern Karamoja Growth, Health, and Governance Development Food Assistance Project – Final Evaluation Report 40 Investment in Health Facilities. Another component of interventions to improve access to quality health care and nutrition services was investment in the construction of new facilities and maternity wards and the provision of equipment. The locational placement of these facilities with respect to distance from communities was a factor in supporting increased access to health care. Specifically, GHG constructed 14 health facilities comprising 7 maternity wards, 3 outpatient departments, and equipped them to support basic MCHN services.103 In addition, one housing facility for staff and three housing facilities for medical interns were built. A summer program involving the placement of medical students’ in local health facilities was intended to temporarily bridge chronic and Abim staffing gaps, foster learning, and increase capacity building. Conclusions. Findings from the analysis of interview data from respondents at each administrative level coupled with quantitative data on specific interventions from MC’s final IPTT (2018) provide evidence of the effectiveness of the GHG strategy to date on increasing access to quality health care and nutrition services. As reported under Section 4.2.2.1, MC has one indicator to measure increases in access to health care, the percentage of births receiving at least four ANC visits during pregnancy, and this increased from 49.2 percent at baseline (2013) to 77 percent at endline (2018). No specific indicators were built in the GHG project monitoring system about immunization coverage and institutional deliveries, but both activities were reported as being increased by care group members, VHTs, and health facility staff during interviews. It has been reported that children’s health improved mainly due to the project training provided to health facility staff and VHTs (prompt referrals to the health facility due to regular MUAC screening at the community level and according to illness’s severity) and availability of drugs to treat the most recurrent child conditions such as fever and following the syndromic approach. Examples of GHG health infrastructures, maternity ward and student lodge, Kaabong 4.2.2.2 Consumption of Nutritious Foods for Households Increased IR 2.2 Care group model. GHG adopted the MCG approach model to promote behavior change messages to PLW through LMs. Over the life of GHG, 231 MCGs (approximately 2,480 LMs and more than 20,000 neighborhood women) and 830 MCA groups were created and supported.104 The number of MCGs sustained was 38 percent less than the 371 targeted at the inception, but this model has demonstrated its relevance for the diffusion and uptake of BCC messages at the community and household levels. MC and World Vision both created MCGs in their respective sub-counties following the general design and implementation models recommended, with the main difference that World Vision financially incentivized the LMs to mentor the neighbor women. In 2015, MC took over all of the World Vision MCGs and re-addressed this strategy, ceasing payment for the LMs. Some attrition was reported from LMs engaged by World Vision in its implementation areas as it withdrew, and just after MC began taking over MCG activities in these locations. The attrition did not last long, and these LMs agreed to comply with the MC strategy. LMs were trained by VHT heath promoters on modules that included EBF, complementary feeding, family planning, and hygiene. Some local health authorities were critical of MC during interviews for not fully involving them in in the creation of community-based organizations (CBOs) and particularly the MCGs. Interviews with GHG managers reported that the 103 In Abim, Kotido, and Kaabong. Numbers are from MC final IPTT. 104 MC final IPTT, 2018 Final Performance Evaluation of Northern Karamoja Growth, Health, and Governance Development Food Assistance Project – Final Evaluation Report 41 project underestimated the importance of involving local health authorities. They explained this oversight due to the fast pace of implementation start-up. In addition, their request for monetary incentives to participate in GHG activities made it difficult to engage them regularly, and some refused to engage at all. Financial incentives to engage in GHG activities was a practice MC disallowed consistently (and as described in the case of LMs previously engaged under World Vision) across all SOs as part of its strategy to encourage Northern Karamojong to take on responsibility for improving conditions in their own areas and by increasing their capacity to do so. However, MC began providing some incentives for local official participation mid-way during the project and this issue ceased to be a problem. In 2016, GHG piloted the MCG plus (MCG+J) model and created 73 groups during fiscal year 2016. This built on the progress of the care groups by looping in LMs and selected neighbor women for more intensive training in life skills and asset building through savings and small business development combining activities from SO1 and SO2. Findings under this activity are described under SO 1, Section 4.1, but discussions held with MCG+ participants during the FGDs105 indicated very positive effects in household well-being.106 MCGs. Women from MCGs participating in FGDs in Kotido and Kaabong were asked about their recall of GHG BCC health messaging. The most frequently reported messages were timely immunization, recognition of signs of child illness, care-seeking behavior, healthy timing and spacing of pregnancy tied to EBF, and household hygiene practices. To compare health and nutrition outcomes between MCG members and women who were not participants under GHG, the evaluation conducted FGDs with non￾participants in one village in Kotido and in one village in Kaabong.107 Despite the fact that they were not able to explain why they were not selected for enrollment to receive food rations or to join the MCG group in their village, the non-participants were aware of the BCC messages shared by the LMs in the manyattas and were able to describe the most important practices for health, such as proper food preparation, hygiene, family planning, and seeking VHT advice when their children are sick. However, the analysis of data from MCG focus groups shows that that MCG participants do not practice all the BCC messages with the same intensity as the knowledge they gained. For instance, hygiene (mainly handwashing and use of latrines) and family planning practices were less reported as adopted, compared with EBF, ANC visits, and institutional deliveries. Details of the family planning component are described below, and an expanded overview of the WASH component to decrease the incidence of childhood diarrhea is described under Section 4.2.3.3. Men as agents for behavior transformation. In 2015, MC introduced the MCA model after conducting extensive research on the application of this model in other countries. The MCA model was designed to address gender inequality, a key factor in children’s malnutrition status. The model was adapted to Karamoja settings and was designed to complement the existing MCG platform to reach men108 through a peer-to-peer approach. It was implemented in World Vision project areas by MC after the latter took over the implementation of the care groups. MC created MCAs by organizing groups of men in each community who volunteered to participate, and by training them in BCC activities to promote health and nutrition messages. MCAs were used to increase men’s participation and support for women’s and children’s health, as well as for reducing domestic violence and empowering women. The creation of MCAs was not planned from the beginning of the project and did not get started until 105 MCG+ FGDs in Sidok, Longar, and Chachaion (Kotido and Kaabong) 106 A similar conclusion was made by Advanced Marketing Systems in the Final Evaluation of the Growth, Health and Governance Project report, 2017. 107 Longelep and Longar Villages 108 GHG, Gender and BCC, Final Report, 2017 Final Performance Evaluation of Northern Karamoja Growth, Health, and Governance Development Food Assistance Project – Final Evaluation Report 42 midway during the GHG implementation period. Despite the reduced time for implementation, the practice of using men to inform and sensitize peers produced great results109 and should be replicated in activities that have a focus on health and nutrition behavior change. While no indicators were established to track changes or impact of MCAs, GHG monitored the progress of activities and their results, and reported success stories on increases in gender empowerment achieved through adoption of the MCA model. Some of the activities, for example, involved inviting men to trainings on different health and nutrition and hygiene topics provided to women enrolled in MCGs, and the wives of MCAs assisted and role played in some gender related lessons. The MCAs and other groups interviewed articulated a growing understanding that women need to reduce their physical workload during pregnancy and eat a variety of foods, and that men should take care of the children at the same level that women do. These findings mirrored those of the 2017 GHG Gender and BCC impact assessment report. The changes in male involvement brought about by BCC messaging and the involvement of MCAs were described during interviews as shared decision-making, bathing the children, fetching water during their wives’ pregnancy, preparing and cooking food when their wives were not at home, and accompanying women and children to the health facility. In addition, drinking habits and domestic violence decreased. At baseline, GHG set a target of 85 percent of married women reporting making health-related decisions on own or jointly.110 Although this indicator was eliminated and not measured at endline, the qualitative data collected from the MCHN participants, mainly MCGs, MCG+, MCAs, and LMs, reported that there is much more autonomy on health decision-making compared to before the intervention. Sexual and reproductive health—family planning. During interviews with health facility staff, they reported they were equipped with FP supplies, with oral pills, injections (Depo Provera®), and, in a few instances, with implants. An increased joint decision between husband and wife to discuss family size and which FP method to use was recounted during the FGDs with MCHN participants and may be indicative of a growing awareness and the start of a shift in social norms around the importance of joint decision-making on family planning. These findings were in line with those of the 2017 Gender and BCC impact assessment. Despite the availability of a range of contraceptives and reported increases in joint decision-making about family size and FP methods, the PBS endline data show 80.6 per cent of the women between 15 and 49 years of age and are married or in a sexual relationship reported not using any contraceptive, meaning that the estimated contraceptive prevalence rate (CPR) is 19.4 for the GHG project area. Moon Beads Users, Kotido 4.2.2.3 Incidences of Diarrheal Diseases Among Children Reduced IR 2.3 GHG placed great emphasis on the WASH activities as a major component of the technical approach to reduce the incidence of diarrheal diseases among children contributing to improvements in the nutritional status of children under 5 years of age. The key components included increasing access to safe drinking water, the creation of linkages with private water service providers, the construction of 109 FGDs with MCGs, LMs, MCAs in Kotido and Kaabong. 110 This indicator was eliminated by FFP in January 2018 and was not calculated at endline. However, a similar indicator referring to mothers with children under 2 years of age was calculated during baseline for the Apolou Program. Approximately 38.8 percent of women in a union with children under 2 years of age make maternal health and nutrition decisions alone or jointly with a spouse or partner. Final Performance Evaluation of Northern Karamoja Growth, Health, and Governance Development Food Assistance Project – Final Evaluation Report 43 latrines and promotion of Open Defecation Free (ODF) communities. These activities were coupled with hygiene messages spread to the communities by Water User Committees (WUCs), VHTs, MCGs, and MCAs. At an institutional level, GHG constructed ventilated improved pit (VIP) latrines and handwashing stations in selected health facilities and primary schools and created School Hygiene Clubs. At the community level, households were trained to construct pit latrines and handwashing stations called tippy-taps, introduced as part of the community-led total sanitation (CLTS) approach. Increasing access to safe water. GHG’s strategy to ensure safe drinking water to targeted households was to construct or renovate existing boreholes. The mapping of the communities in need of a safe water source was a joint effort with the water district authorities and implemented by GHG’s partners, World Vision and Whave. These partners established access to safe water to 42.2 percent of the households targeted in the intervention area. Whave and World Vision rehabilitated or constructed 456 (45 new and 411 rehabilitated) boreholes during the five years of implementation, exceeding the life of activity target of 290. But there is a long way to go. PBS endline results show that 37.9 percent of the households in Northern Karamoja have used an improved source of water, compared to 37.4 percent at baseline. A total of 9 percent of households are using a non-improved source of drinking water, and 5.2 percent of these are still using surface water, such as rivers, ponds, lakes. A total of 9.6 percent of households reported boiling water as treatment prior to drinking, but 88.2 percent still drink water without any treatment. Among those surveyed, 32.2 percent stored water in containers with lids, and 51 percent are still storing water in open buckets or jerry-cans without lids. From further investigation of the endline data, 50.5 percent of the respondents admitted that water was generally available from the source year-round but 37.7 percent cited that water was not available for a day or two during the two weeks prior to the survey. This could be linked to a recent breakage of the “improved source of drinking water111” which requires that the water from these sources be available in in the last two weeks prior the PBS. When we consider only the condition of using an improved source of water without the time factor, the actual indicator estimates are that 90.2 percent of the households had access to an improved source of water and this gives credit to GHG’s efforts to provide the targeted communities with a safe water point. However, it has been found that a water supply that is intermittent, as is the case in Northern Karamoja, can still lead to diarrheal disease.112 One of the main challenge for communities with boreholes constructed by Whave and World Vision is to ensure prompt repair of the borehole in case of breakdown. To fill the demand for repairs to ensure continuous access to safe water, GHG established strong links between communities through WUCs and hand pump mechanics associations (HPMAs), who are organized private sector providers capable of providing repair services in each of the three districts. However, FGDs with WUCs revealed that there can be a delay between the time that WUC’s request repairs and the time repairs are actually made before boreholes are repaired, during which people will turn to unsafe sources of drinking water. GHG created and trained 358 community-based WUCs composed of gender-balanced volunteer members nominated by their own community to be in in charge of the management and maintenance of boreholes and also to promote hygiene messages. GHG partners World Vision and Whave113, in 111 Improved water sources, as defined by the FFP indicator, are those water sources that are safe to drink. 112 The fecal-oral pathway from animal feces is another important pathway that increases the risk of diarrhea in children (FFP Comment January 8, 2019, Report Review and Comment Matrix.). This pathway was not examined as a contributor to diarrhea and to malnutrition during fieldwork, However, the QET HCN/WASH specialist noted that in conducting FGDs with MCGs in two villages (one in Kotido, one in Kaabong), that there were many chickens running around the compound and chicken feces were all around the areas where children played. As poultry are promoted as an economic activity to improve women’s livelihoods, these effects should be examined. Another fecal-oral pathway from animal feces may be from vermin. A local health assistant interviewed in Kaabong noted that people live in close contact with vermin in their manyattas and households. 113 Whave Kaabong Final Performance Evaluation of Northern Karamoja Growth, Health, and Governance Development Food Assistance Project – Final Evaluation Report 44 collaboration with district water representatives, provided the training based on Ministry of Water and Environment policies to promote safe water access, sanitation, and hygiene. One of main tasks of the WUCs is the collection of monthly fees from each household to create savings for use when needed to repair borehole breakage and the fencing around boreholes. Although WUCs have been effective in providing oversight of boreholes and promoting hygiene messages, those WUC members participating in FGDs indicated the challenge they have in collecting fees from households. Strengthening the link between private sector providers and WUCs. In the three districts of operation, GHG and its implementing partners worked to train, support, and equip the HPMAs and involved the participation of both HPMAs and WUCs together in sectorial meetings with district water authorities. Whave served as the direct link between the WUCs and the HPMAs in case of borehole breakdowns and was responsible for mobilizing the mechanics for repairing broken boreholes. Findings from FGDs with WUCs confirmed they understand how to contact the mechanics in case of borehole breakdown. Water district authorities cited the collaboration and support received by GHG and its adherence to national water strategies and partnerships in supervisions and community WASH activities. A few HPMA members described difficulties in spare parts provision and infrequent support from the water district representatives. Access to improved latrines and handwashing. The project design for improving sanitation and hygienic practices was based on the CLTS approach to end open defecation and strengthen communities in the construction of handwashing stations and pit latrines at the household level. The CLTS approach promoted the use of local materials to build those installations and other food protection techniques such as dish racks. During the life of GHG, this approach gained momentum, and a considerable number of pit latrines were constructed. CLTS does not specify technical standards for toilets but following a set of criteria established by the District Water Officers, building a household latrine was a prerequisite to obtain a borehole, and a critical element to achieve the ODF status. By the end of the project, IPTT data show that 37 communities were verified by district health officers as ODF, representing 132 percent of the project target (28) met. Interviews with GHG participants reported that the use and the maintenance of the latrines is an important issue. Traditional and cultural beliefs were recounted as one of the main obstacle for the non-use of the latrines (see Figure 3). For example, in Karamojong households, the latrines cannot be shared with in-laws, and pregnant women fear that the baby could be delivered and fall in the pit. Latrines broke down frequently. Flooding was one of the factors causing latrine collapse. The local material used to build footrests was reported as “weak” and systematically attacked and destroyed by termites. The frequent breakdown of household toilets probably led to decisions not to rebuild them. During fieldwork the team learned of a local regulation in Karamoja that prohibits cutting trees, so the availability of strong logs for construction which may have solved problems of latrine breakdown, was very limited. This regulation was not foreseen by GHG. Final Performance Evaluation of Northern Karamoja Growth, Health, and Governance Development Food Assistance Project – Final Evaluation Report 45 Figure 3: Reasons Why People Do Not Use Latrines Toilet is dirty or smelly Open defecation is considered more pleasant, in the open air Open defecation is embedded as a routine and habit or social norm Fear of collapse or danger to children Not enough privacy Social taboos (e.g., father-in-law and daughter-in-law should not use the same one) No toilet was available when away from the dwelling (e.g. farming in household gardens far from the manyatta, herding livestock) The GHG’s target was to reach 22.1 percent of households using an improved sanitation facility at the end of project implementation. PBS findings from the comparison of PBS baseline and endline data show that only 13.7 percent of the households surveyed are using an improved latrine (11.5 percent of those used a pit latrine with slab) in GHG intervention areas, compared to the baseline value of 12.7 percent (see Table 6). It is important to highlight that the definition of this FFP indicator mentions “improved facility,”114 and this implies, among others, a pit latrine with a slab. Nearly 65 percent of households reported using non-improved sanitation facilities, mostly latrines without slabs or open defecation. These types of latrines were not implemented by the CLTS approach in the GHG intervention areas. This means that the latrines built under CLTS guidance during GHG implementation can be defined as “unimproved” and do not fit under the definition of the FFP indicator. The project reported that more than 1.412 pit latrines and 497 tippy taps were built during the five years of implementation. GHG built 20 institutional VIP latrines in schools and health facilities in the three implementation districts. In all these schools, interventions included the creation of the School Health Clubs115 to create awareness among pupils on the use of latrines and trained teachers and patrons on the Child Hygiene and Sanitation Training (CHAST) approach. Handwashing. GHG established a life of activity (LOA) target of at least 30 percent of the targeted households using soap or ashes at the handwashing station. Results from the PBS endline data show that only 2.6 percent of the households in GHG intervention areas were practicing handwashing with soap or ashes. To corroborate this result, additional analysis was done, and disaggregation of the handwashing data collected showed that a handwashing station was observed in 31.4 percent of the households surveyed, and water was available in 11.3 percent of those. A total of 90.2 percent of these installations lacked any kind of cleansing agent such as soap or ashes. Findings from the qualitative data collection strongly support these data. During FGDs with MCGs in Kotido and Kaabong, participants expressed knowledge of the critical moments for washing hands and were able to mention soap or use of ashes if soap was unavailable; however, they cited that the jerry cans used to build the tippy taps were spoiled or that filling them was not a priority after water collection. Children often used jerry cans as toys which contributed to their breakage. The evaluation concludes that handwashing was not as well adopted as other hygiene practices supported by GHG, such as waste management (burning), the use of dish racks, and personal hygiene practices as regular bathing. During fieldwork for this evaluation, the 114 FFP Indicators Handbook Part I: Indicators for Baseline and Final Evaluation Surveys April 2015 115 School Health Club FGD held in Longar, Kaabong Final Performance Evaluation of Northern Karamoja Growth, Health, and Governance Development Food Assistance Project – Final Evaluation Report 46 QET rarely observed tippy taps functioning and installed in the villages visited, and there were no signs of other alternative mechanisms116 to replace the lack of handwashing stations. Latrine use increased during the project, as reported by GHG annual results data, and a number of villages achieved ODF status as a result of efforts to promote building of latrines and in line with the CLTS approach. These findings are not supported by PBS data on access. But promoting the use of pit latrines and handwashing stations has not been successful in reducing the incidence of diarrhea. Following endline results, the percentage of children under 5 years of age who had diarrhea in the two weeks prior to the household survey almost doubled, compared to GHG project baseline (35.4 at endline, compared to 21.3 percent at baseline). We have already noted that some of this increase is likely due to conditions of heavy rain and flooding during the June–July 2018 period when the endline PBS was conducted which can lead to higher rates of diarrheal reoccurrences. These findings contrast with the findings from FGDs with MCGs, MCAs, and WUCs that diarrheal recurrences reduced during the implementation of GHG. Although this finding expresses a viewpoint of the GHG participants, it could be considered valid if respondents were referring to the extremely poor hygienic conditions of the manyattas before the execution of the project. Conclusions Data on nutrition indicators on the prevalence of underweight and stunted children under age 5 show no improvement. Our analysis of quantitative and qualitative data sources indicates some of the likely factors that have prohibited advances: inadequate adoption of hygiene and sanitation practices, cultural feeding practices and level of household income, and these findings provide a starting point for a rigorous study to understand the interplay of these factors and to identify other factors this evaluation may not have captured. However, we conclude that interventions based on MC’s theory of change provide the basis for improvements in these important measures. GHG’s strategy to increase access of quality health care and nutrition services has been successful in achieving important advances. The project made considerable efforts to increase the delivery of quality health care services and address the “three delays model.” Women participating in MCGs received mentoring from LMs and VHTs and are now aware of seeking care in time, and there has been an increase in the use of health care facilities and maternal units measured by the increased percentage of ANC visits. As reported by FGDs with MCG participants and interviews with health facility staff, there has been an increase in institutional deliveries. Adequate care is provided by health providers, both VHTs and health facility staff, who received technical training, who promptly refer the most serious cases to the upper level of care. Improvement of some feeder road conditions and the new infrastructure constructed by GHG addressed the second delay by enabling women to reach health facilities faster. Finally, accountability of the health units was enhanced by building the capacity of HUMCs to provide management oversight of health facilities, and with input from the community through VHTs and Report Cards, the HUMCs have enhanced the quality of the consumer experience in health facilities, contributing to increases in the quality of care. GHG interventions to build the capacity of health facility staff and HUMCs, increase the placements of health professionals, and link the health facilities to the target communities to improve accountability and quality of care, were crucial components of the overall strategy. A better link with the health unit resulted in the VHTs being provisioned on time with drugs to treat children at the community level. VHTs and health facility staff interviewed reported a better partnership with the health facility staff in term of drugs provision and regular meetings. GHG training and follow-up with VHTs and health care providers has facilitated the delivery of quality health and nutritional services. Access to quality health care will continue in the communities where the engagement of VHTs and HUMCs was high, the 116 Handwashing stations include different kind of devices or tools. A bucket or a jerry-can with water and soap can be considered as a handwashing stations. Final Performance Evaluation of Northern Karamoja Growth, Health, and Governance Development Food Assistance Project – Final Evaluation Report 47 distance to the health units was relatively moderate, medical students contributed with their expertise to the capacity building of the health unit staff, and the staff was trained to better attend the patients. Care groups are an evidence-based approach to change behavior in health, nutrition, and hygiene practices, and GHG chose this as the main focus to improve health and nutrition practices at the household level. The implementation of MCAs, implemented mid-way through the project, showed that engaging men was critical for changing social norms and attitudes of men. GHG realized that MCGs have less impact without bringing community men into the program and engaging their support for women’s and children’s health and nutrition. WASH interventions identified by GHG were technically appropriate given the strong evidence-based correlation between stunting and poor sanitation practices. However, and as discussed under Section 4.2.1, poor adoption of improved sanitation practices was one of the likely factors that prohibited improvements in the nutritional status of children under 5 years of age. Borehole construction and rehabilitation was a significant effort and a relevant strategy to provide safe water access to the targeted communities. The linkages created by GHG between Whave, the private sector, and WUCs to maintain these water sources were a considerable achievement that will help promote the sustainability of safe water sources. WUCs displayed proactivity in sensitizing the communities on hygiene, latrine construction, and, above all, the maintenance of boreholes for access to safe water. But the challenges of collecting monthly fees from some households to repair broken boreholes remain. Continued sensitization of communities about the importance of having a functioning safe water source available in the proximity of the village will help address this challenge. 4.2.3 EQ4: Which project outcomes are likely to be sustained? Although there are no significant changes in the prevalence of malnutrition in women of reproductive age and in children under 5 years of age, with the exception of wasting, interventions implemented at the community level and in health units and their oversight by HUMCs, provide a solid base to leverage the implementation of the Apolou DFSA. In this section we discuss selected interventions that indicate greater and lesser potential of sustainability. Community-based Health Supervision Strengthened and Capacity of Health Centers and Managers Improved GHG support improved resource management and supervisory capacity at health centers, resulting in sustainable health care service delivery. Data from project performance monitoring as well as the endline data, show that the HUMCs have the potential to serve as powerful structures for health accountability, successfully advocating for facility and staffing with the health district and supervising the VHTs on their provision of primary health care services. This institutional arrangement has worked well, and they were still working during the evaluation fieldwork period. The engagement of VHTs with communities has clearly increased the visibility of the health units by improving access to and demand for health care services. These linkages are likely to be sustained in the future. People from the communities are pleased to have free health care services in the proximity of their households and health providers available to provide treatment. They understand the importance of preventative services, and are likely to continue coming in for these services. GHG worked in collaboration with public health institutions, with the Ministry of Health policies, and in coordination with other non-USG projects (such as WFP and Community Action for Health), guaranteeing ownership and continuity of these actions promoted by the project. Community-based Health Outreach Strengthened Because the new MC USAID-funded activity Apolou is targeting MCGs, MCG+, and MCAs, it is difficult to understand how long those CBOs could have continued without any further support. However, these approaches demonstrated the effectiveness of using care groups as a platform for social and Final Performance Evaluation of Northern Karamoja Growth, Health, and Governance Development Food Assistance Project – Final Evaluation Report 48 behavior change, transferring life and business skills through the MCG+ model, and the success of using the MCA approach to engage men more as caregivers and partners. These models are sustainable and effective as reported by many enthusiastic participants and by the project’s achievements, but the QET would stress how fragile these groups are and the need for follow-up for these organizations to continue on their own. One of the weaknesses of the project reported by former GHG managers was the lack of involvement of local authorities in the creation of those care groups. Given the need for regular follow-up, meetings, and refresher trainings to ensure the continuity of the MCGs and MCAs, the QET believes that it is imperative that local authorities are involved at the outset to strengthen their ownership and involvement. Access to Safe Drinking Water in Target Communities and Schools Improved/Government and Private Sector Capacity to Ensure Sustainable WASH Interventions Improved Drilling and repairing boreholes was a good strategy to improve access to drinking water, but because of the high expense involved, the drilling component is neither sustainable by the local authorities nor by communities without the involvement of a national or international organization. The sustainability aspects of the access to safe water is the capacity of a community to maintain and repair its borehole, and GHG has strengthened the capabilities of the WUCs to monitor, supervise, and problem-solve this piece of the equation. The institutional arrangements created or facilitated by GHG with the local water authorities were reported as very good, and the collaboration from the beginning of the project with those authorities lays a good foundation for the sustainability of this initiative. However, barriers were identified that contribute to water users’ willingness to pay. First, institutional barriers indicated the low performance of the WUCs and their influence on the regular collection of user fees from all hand pump users. Second, operational barriers frequently cause hand pumps to remain unrepaired for an extended period, discouraging users from paying, because the source is considered unreliable. This created a cycle of low willingness to pay among some (but not all) community residents, insufficient funds for repairs and investment in maintenance leading to frequent pump breakdowns and long downtimes. This undermined the livelihood of hand pump mechanics, who went months without receiving payment. People’s perceptions of the water sources rehabilitated or constructed by World Vision and Whave were positive, and the communities are pleased to have a water source accessible and close to the village. However, as previously mentioned, their willingness to contribute to repairs or to donate regular fees for maintenance is a problem and a real threat to the sustainability of this activity. Households’ monthly quota contribution is limited and mainly unsolved due to poverty. Many community members stated that they cannot pay because of competing demands for money in the household. An added problem is that water is seen as a common good and the universal water access argument requires approaches that are sustainable. People argue over why they should pay for something that is considered free117. Whave has improved approaches at community level that focused on non-functional water sources by completing quality repairs through local service providers, such as hand pump mechanics, and by ensuring that WUCs have the resources to prioritize timely maintenance and repairs. Although this is a viable solution because it involves local resources, it does little to change the behaviors and attitude of the households to contribute to those repairs. 117 These findings from interviews with WUCs in Kotido and Kaabong are in contrast to MC reports of project activities and arrangements implemented by GHG to sustain water contracts. MC reports that contract renewals were based on the willingness of the community and their committees. MC further states that Whave sensitized communities and created community awareness of the importance of their role. WUCs reportedly informed Whave after the money was collected. Final Performance Evaluation of Northern Karamoja Growth, Health, and Governance Development Food Assistance Project – Final Evaluation Report 49 Communities’ Sanitation Access and Hygiene Practices Improved Despite evidence of weak outcomes in nutritional status of children and in the incidence of diarrhea, using the CTLS model is still a valid approach for reducing childhood illnesses. This approach has a strong potential to be sustainable because the mechanisms to implement and supervise the model are engrained in the local water authorities’ agenda and are followed by any agency implementing WASH in Northern Karamoja. CLTS is an approach that needs further adaptation and should be revisited, depending on the context to be successful and sustainable.118 Despite the number of latrines built under GHG, the fact that many of them are not used anymore is a sign that more efforts should be made to understand the social norms impeding the adoption of sanitation and hygiene behaviors. Community perceptions are the foundation for the sustainability of the use of latrines and handwashing stations and must be assessed before any intervention is put in place. Our data indicate that CLTS activities and BCC messaging created an understanding in communities that having and using their own household toilet is better for children’s health than OD, but understanding is not enough to push them maintain, rebuild as necessary, and use latrines. In addition, better design alternatives need to be developed to support the communities on how to build latrines within the limitation of the government policy and local material. Long-term success in the use of the latrine depends on their quality and on the technical support provided for the construction: poor construction and materials are significant factors in the decision to abandon latrines, and higher-quality toilets are more likely to last and be maintained. Motivational factors for latrine adoption and sustaining the ODF status include positive social pressure like prestige, convenience, privacy, and health benefits. The sustainability of handwashing behavior depends on multiple variables, including water availability, distance from the water source, how water is transported and stored, and who brings the water (often women and children). Ash is used when soap is unaffordable, so the lack of soap is not a factor. Conclusions The sustainability analysis of the MCHN interventions indicates that only a portion of the activities implemented could be maintained and have an impact on the improvement of the nutritional status for children under 5 years of age in the future. An example of a non-sustainable activity was the distribution of food commodities. Nevertheless, GHG has laid the foundations that can potentially be leveraged and, coupled with more focused interventions that reflect the local context, could lead to a decrease of the prevalence of stunting. GHG was able to make changes in a short period of time with the creation of MCAs and MCG+. Communities perceived these approaches as highly beneficial, and members from MCAs and MCGs are motivated to continue to be involved as mentors beyond the end of the project. The strengthening of the management of the health facilities through the HUMCs and the focus on health resources based at community level (the VHTs) have motivated clients to access the health centers and be confident in the care provided. As long as these institutional arrangements are maintained, the quality health services provision should continue. The challenges of achieving water and sanitation sustainability are multiple and complex. Habits are hard to break, and sustainability of behavior change remains a challenge. Access to safe drinking water in target communities, coupled with government and private sector capacity to ensure viable WASH interventions, could be sustained in the next years if the WUCs can maintain water sources and quickly arrange repairs when needed; and if the water local authorities can be more proactive in supporting the HPMAs. The presence of local players such as Whave and the HPMAs are a plus to achieve safe water sustainability. It is important to consider that local organizations 118 Sustainability of community-led total sanitation outcomes: Evidence from Ethiopia and Ghana. Int J Hyg Environ Health. 2017 May; 220(3): 551–557. Final Performance Evaluation of Northern Karamoja Growth, Health, and Governance Development Food Assistance Project – Final Evaluation Report 50 are often financially limited, and this could potentially compromise the continuation of the water services provision after the funds end or a project terminates. CLTS can be sustained in the presence of training provided to local actors, with the acceptance that this model is not appropriate in all settings without adaptation. The community should choose whether to adopt it, and it should be combined with efforts to address barriers that households face and to build higher-quality latrines. CLTS should not be a standalone strategy for addressing sanitation and hygiene and will result in low durability latrines and handwashing practices if a proper analysis of the context does not precede the implementation. The evaluation findings pertaining to the status of children’s nutritional status and the effectiveness of GHG MCHN and WASH interventions confirm many of the same findings documented in the GHG final evaluation report (Advanced Marketing Systems, 2017). Knowledge can be transferred on health, nutrition, and hygiene, but there is no guarantee that practices will be adopted or that the intended impact will be attained. There is a need to have a deeper understanding of the household dynamics in food use and causes of undernutrition to better understand why nutrition indicators are not improving. 4.2.4 EQ5: What are the key lessons learned and best practices that should inform future projects in the country? The reinforcement of the capacity of the HUMCs to better manage the health unit and the improved link between the health facility staff and the VHTs created the conditions to increase the health facility access and use. Along with the community dialogue approach, capacity building has helped health providers to better attend the patients accessing the health unit such as reducing the burden of long wait times. The evaluation considers these two interventions as good practices of capacity building and accountability. GHG constructed and equipped new health facilities, which has improved the quality of and demand for maternal and child health services. These improvements, along with BCC, resulted in higher numbers of women visiting ANC services during pregnancy, delivering babies in a health unit, and following up on children’s growth. VHTs took on numerous roles within GHG as promoters of LMs, MCG supervisors, and as trainers for permargardening). They also serve as an entry point in the Ugandan health system by conducting community nutritional screenings and providing referrals of children for treatment of acute malnutrition, checking on the health status of household members, treating children’s common illnesses, and providing referrals to clinics), as well as attending regular coordination meetings with health facility in-charge, MCGs, HUMCs, and with other NGOs. It should be acknowledged that the number of obligations as a VHT member competes with their need to attend to farm and livestock chores, and in the case of female VHT members, from taking care of household chores and children. The use of food rations for mothers enrolled in care groups encouraged women to use health care services such ANC and to obtain immunization for their children. GHG could have achieved the same results without the use of distributed food and could have had even larger and more sustainable outcomes considering the use of imported commodities was limited at the start-up phase only to encourage enrollment. Because food is widely shared within the family as is customary in households in Karamoja, distributed food in GHG had limited impact on the reduction of child malnutrition and stunting, and this activity required substantial logistical and financial resources. The BCC approach of using MCGs to improve a household’s health and nutrition knowledge and behavior change is a strategy from Title II programs that GHG integrated into its design. GHG undertook annual assessments and analysis to monitor the implementation of the activities and readdress its strategies to better serve the project participants. These assessments on analysis of barriers, gender, and functionality of MCGs resulted in the development of new approaches, such as Final Performance Evaluation of Northern Karamoja Growth, Health, and Governance Development Food Assistance Project – Final Evaluation Report 51 MCG+. However, implementation of the MCG model coupled with BCC messaging has not resulted in progress in the adoption or consistent practice of hand-washing. The involvement of the local authorities in the creation of these CBOs is necessary to increase the ownership and use of these organizations to provide communities with health, nutrition, and hygiene messages. There is a constant debate about the financial incentives given to community-based volunteers; however, the evaluation would stress that the practice of delivering a monthly salary to the LMs affects sustainability after the end of the project and obliged other stakeholders to align with a monetary demand if they are using the same volunteer. Although it is recognized that volunteers and above all women have multiple tasks to comply with (farming, household management, children care, etc.) the care group model relies primarily on intangible incentives to motivate participants, prevent volunteer burnout, and ensure sustainability of the interventions.119 GHG developed a complementary BCC programming intervention to the MCG approach through an adaptation of the MCA model, designed to address gender inequality as a main cause of malnutrition. Although the MCA model was implemented mid-way through the GHG project, it was successful in increasing awareness of the importance of maternal and child health and nutrition. Participants in MCA groups, according to interviews with MCAs and MCGs in Kotido and Kaabong, also played a very important role in the reduction of domestic and gender-based violence that was reported by many interview participants during this evaluation.120 The creation of MCG+ has improved household income and the welfare of children as reported by participants that were interviewed. The evaluation estimates that use of MCG+ and the MCA approaches are two of the best practices under GHG that should be continued in contexts like Karamoja. The level of impact and change from these approaches should be captured through systematic monitoring or assessments, and the use of project developed custom indicators. Under the WASH intervention, GHG created linkages between the WUCs and local private water service providers, such as Whave and the district-based HPMAs, and have followed the CLTS model to improve sanitation and hygiene in poor-resource settings. The introduction of a national policy into the GHG design was aimed at guaranteeing the sustainability and ownership of this model after the project’s end. Adaptation to national strategies and plans is considered a best practice, rather than introducing locally non-tested innovations. However, findings shown that the partnership between local water authorities and private water service providers needs to be reinforced because the government’s mandate, resources, and influence are limited, and a better analysis of government capabilities and the budget allocated to WASH activities should be undertaken. GHG implementing partner Whave created a task force at the district level for coordination of district and local government activities. This task force led to improved collaboration and coordination with other stakeholders involved in WASH and CLTS activities and avoided overlapping. Unfortunately, having a taskforce of WASH actors spreading hygiene promotion messages does not guarantee the adoption of hygiene practices, and follow-up is needed to ensure uptake of behavior change. However, 119 The care group manual, 2010 www.coregroup.org 120 Interview participants described the reduction of domestic and gender-based violence across the three SOs and across the three districts. Interviews with peace committee members revealed that members actively engage themselves in reducing domestic violence, and members like to see themselves as role models for how one should act. Male members talked about their role in confronting those men who beat their wives when they hear about beatings from other community residents. Some MCAs are also peace committee members. An interview with the director of the district security committee in Kotido described their committee’s involvement in reducing domestic violence also. This is also discussed under 4.3, SO 3, Reduced Incidents of Conflict, in section 4.3.1 under the sub-heading Targeting Gender and Youth. Final Performance Evaluation of Northern Karamoja Growth, Health, and Governance Development Food Assistance Project – Final Evaluation Report 52 having a local organization that serves as an intermediary between the project and the communities has at least facilitated communications and problem-solving in the case of borehole breakages. Household capability to pay monthly fees so that cash is ready and available to disburse for maintenance and repair of the boreholes must be addressed. This activity is difficult to implement for some households that are more likely to contribute when a problem in the borehole is present, rather than contributing money for water services to save for future potential issues. The WUCs interviewed have become frustrated about their ability to collect funds without much recourse to seek payment from non-payers. Despite the implementation of the CLTS model, inadequate attention to socio-cultural norms, traditional beliefs, and taboos about indoor defecation, and the use of inadequate local materials to build pit latrines represent a risk in the implementation of such an approach and affected the adoption and maintenance of sanitation practices. The limited use of handwashing stations, for example, was primarily related to the water source proximity, the availability of water containers, and the cost of the soap, and despite the fact that the CLTS model indicates the alternative use of ashes is acceptable. Limited use of pit latrines was more associated with the use of poor local materials and cultural customs. Overcoming social norms and traditional beliefs is possible with continued sensitization to split from “shame to status,” but the use of the local materials is more difficult to overcome due to the environmental regulations and requirements. GHG created school hygiene clubs to disseminate WASH messages using a child-to-child strategy empowered around issues of sanitation and hygiene. Interview data indicate that early education and engagement in the importance of sanitation and hygiene may yield future progress in this cohort’s adoption of practices that will benefit the health and nutrition of the next generation. Indicators should be built to monitor or evaluate this activity in addition to the output indicator for the number of VIP latrines constructed in schools. 4.3 SO3: REDUCED INCIDENCE OF CONFLICT 4.3.1 EQ1: To what extent has GHG met its defined goal, purposes, and outcomes for SO3? Northern Karamoja has a long history of violence related to the culture of cattle ownership and competition over resources. Violence increased to deadly levels from armed conflict during the accumulation of arms following the cessation of war and retreat of the War Lords. Large scale theft of cattle made possible from the use of arms, and the use of arms by every household for protection and security in community manyatas made Northern Karamoja a dangerous place to go to. That impression remains today. Forced Government disarmament of the population began in 2006 with victory claimed by 2010. However, there was a remaining uneasiness of moving beyond immediate communities due to continued anxiety over security in areas where risks of danger were very high. The combination of these factors led to the greater isolation of the region of Karamoja as a whole, but in particular in Northern Karamoja, less participation in development other than years of direct humanitarian assistance, and the decapitalization of households leaving communities impoverished at rates higher than any other region of Uganda. GHG built on efforts that began in 2010 to reduce the incidence of conflict and to secure the peace and security of the region by designing a strategic plan for increasing regional and local level capacity to prevent, mitigate and resolve conflict working closely with government officials at all strata and local communities. By the end of the project, qualitative and quantitative evidence demonstrate that the GHG project contributed to the further consolidation of peace and security in Northern Karamoja. Significant progress was achieved in reducing conflict and improving governance in the three districts. MC used a custom indicator to measure progress of this objective: percentage of respondents reporting increased movement in areas that were previously not accessible due to insecurity. The data presented in Table 12 show a significant increase, from 52.4 percent at baseline in 2013 to 87.7 percent in 2018. Final Performance Evaluation of Northern Karamoja Growth, Health, and Governance Development Food Assistance Project – Final Evaluation Report 53 Table 12: SO3 GHG Custom Performance Indicator Indicator Baseline 2013 Endline 2018 Significance of Change Percentage of respondents reporting increased movement in areas that were previously not accessible due to insecurity 52.4% 87.8% *** *** p<0.001 Data from the fieldwork corroborate this finding. Not only did respondents report increased movement in areas that were previously insecure, but across sub-counties and villages, respondents interviewed all reported that they could access fertile areas that were inaccessible in the past. They were able to move freely to their gardens to cultivate and harvest and also to marketplaces in their areas. Livestock herders reported being able to access pastures and water wherever these could be found in project areas, although pastoralists in Kotido encountered challenges migrating with their herds into Abim, where tensions have been increasing. When the GHG project began in 2012, the government had already declared disarmament a success, which had been announced in July 2010. Between 2012 and 2017, no large-scale, culturally sanctioned and organized livestock raiding or road ambushes were reported anywhere in the three districts. Although in 2017 there were concerns regarding intensification of conflicts associated with cattle thefts in communities along the border with Turkana, and in both Kaabong and Kotido Districts, 121 interviews conducted in September and October 2018 with village representatives from sub-county peace committees, local government officials, security agencies, and CBOs involved in reducing conflict across the three districts all reported that the security situation in the region was much improved. Regular dialogue meetings convened between Kotido and Abim districts by district and sub-county local governments and community structures whose capacities have been strengthened by GHG had in the past helped deter violence, and as tensions eased, the ability of local population to move to areas previously rendered inaccessible by insecurity increased. Analysis of interview data shows that a number of factors are involved in the improved security achieved in Northern Karamoja. For example, in early 2018, cross-district cattle thefts between Abim, Kotido, and Kaabong started occurring on a small scale. By September 2018, thefts had reportedly escalated to between 50 and 100 heads of cattle stolen between the Turkana, and the Jie and Dodoth,122 but these conflicts were quickly addressed, which prevented the possibility of revenge actions. The support provided by GHG led to improved negotiating skills in both the formal and community structures. Both were able to act swiftly once a theft was reported. Often perpetrators were quickly identified and arrested, and the stolen animals recovered. Supporting this capacity to act quickly in catching and restoring stolen livestock was the improved efficiency in reporting of incidents of armed conflicts by community structures. Local community structures were able to provide early warnings of conflict, and early response interventions, which helped avert escalation of animal theft incidents. It also made quick recovery possible; there was increased flow of information from communities, peace committees, and the security agencies, making it possible to share information of animal theft, and there was quick recovery of stolen animals after theft was promptly reported to the Uganda People’s Defense Forces (UPDF) and Local Defense Units (LDUs). Youth involvement on peace committee activities supported 121 See MC Uganda, Annual Results Report for FY2017, p. 21 122 KII, LC1 chairperson, Longelep, Potongor Parish, Nakapelimoru Sub-county, Kotido District, October 3, 2018 Final Performance Evaluation of Northern Karamoja Growth, Health, and Governance Development Food Assistance Project – Final Evaluation Report 54 under GHG made it possible for youth in the wider community to cease considering the UPDF as their enemy.123 This led to a significant improvement in trust and confidence between the UPDF and youth.124 Reduced conflicts and improved peace and security created a conducive environment not only for targeted project beneficiaries but also the wider communities in the three districts to fully engage in productive livelihood activities. MC monitoring data show that close to 88 percent of GHG-supported community structures were able to successfully mitigate or resolve at least two community conflicts over the LOA.125 In Abim, a large population had resettled to fertile areas in Kobulin, Nyarkidi, and Apeitolim. In Kotido, the population had resettled in the fertile areas of Kacheri and “old” Lolelia sub-county. In Kaabong, new settlements had emerged in many areas, such as Kapedo, “new” Lolelia, Lomaler, Lopua, Moruitit, and Sangar, and improved and diversified food production led to better nutrition among GHG-targeted households, as was demonstrated by respondents interviewed by the evaluation team in Lolelia in Kaabong. Factors That Inhibited Achievement of SO3 Significant progress was achieved in reducing conflict and improving governance in the three districts, but issues remain to be addressed with respect to cattle thefts, gender, and land disputes. The majority of the population has realized the dangers of holding onto guns, but a few people still possessed illegal firearms.126 Cattle thefts continued to occur and were orchestrated by habitual criminals. In Abim, security for animals was poor, and thefts were not always promptly and accurately reported, making tracking and recovery difficult. Abim communities seldom participated in the tracking of stolen animals.127 In Kaabong, protected kraals had been re-established in the parishes of Sidok close to the Kenyan border, where cattle thefts associated with the presence of Turkana are rife.128 Livestock markets had become a source of some of the difficulties, with the occurrence of cattle thefts. Those who stole animals took advantage of busy market days to sell them to unsuspecting traders from outside Karamoja.129 Even when the numbers of local women on peace committees increased, the involvement of women continued to be undermined by negative cultural practices, such as widow inheritance, defilement, early and arranged marriages, and courtship rape. A national policy and legal framework does not yet exist to deal with these negative cultural practices in the country. The proposed marriage bill (2017) is still the subject of intense debates in parliament because it seeks to recognize the rights of women cohabiting with their partners.130 Women who secured their land rights were often subjected to battery for affirming their land rights when they tried to sell the land they secured in the homes of their biological parents.131 Despite the success of the work of AWOTID and Land Focal Point Volunteers (LFPVs) in resolving land disputes and conflicts, they put their lives at risk, especially where mediation efforts failed. 123 KII, commander, 405 Brigade Nakapelimoru, Kotido, October 1, 2018 124 Ibid 125 MC Final IPTT 2018 126 KII, Local Council Level 3 chairperson, Sidok Sub-county, Kaabong District, October 10, 2018 127 FGD with the Commanding Officer, Administration Officer, and Intelligence Officer, Oscar UPDF/LDU Battalion at Abuk, Abim District, September 29, 2018 128 KII, Local Council Level 3 Chairperson, Sidok Dub-county, Kaabong District, October 10, 2018 129 KII, Commander, 405 Brigade Nakapelimoru, Kotido, October 1, 2018 130 FGD with members of Kotido Elders Council, Kotido District, October 2, 2018 131 KII, Program Manager, AWOTID, Abim, September 27, 2018 Final Performance Evaluation of Northern Karamoja Growth, Health, and Governance Development Food Assistance Project – Final Evaluation Report 55 If violent conflicts broke out, LFPVs could be targeted by the party to a land conflict who felt let down by the outcome of the mediation efforts.132 When GHG ended, the frequency of peace dialogues and inter-community meetings reduced, and, in some cases, ceased to occur. This affected the progress that had been achieved through some of the peace initiatives that were supported through community structures.133 Targeting Strategies: Gender and Youth To empower women under SO3, a focus on increasing awareness of gender issues and the participation of women was integrated in all interventions under SO3. Vulnerable populations, especially women and youth, were directly targeted to participate in community and local government structures for conflict management training. Although the restructuring of peace committees was partly intended to bring the rural-based population on board, the effect of this targeting enhanced representation of women and youth in informal and formal governance structures. GHG targeted the inclusion of women in all structures for conflict management, including peace committees, area land committees (ALCs), the District Land Boards, and the Local Council 3 (LC3) Court Committee. The number of women local women participating in substantial roles in peacebuilding processes increased, from 110 at baseline in 2013 to 364 by the end of GHG, surpassing the established target by 130 percent. 134 FGDs with peace committee members, both male and female, reported the gradual acceptance of women on peace committees. Women on peace committees in Rengen Sub-county reported no longer feeling intimidated by working among men. Women used their positions on peace committees to dissuade their sons and husbands from indulging in cattle thefts and to confront them when they suspect involvement in stealing cattle.135 The inclusion of women in committees and the focus on increasing awareness of gender issues has begun contributing to changes women’s rights and how women are treated. For example, in Abim, there was equal gender representation in LFPVs for mediating land conflicts at the parish level, which made it possible to address women’s land rights (See Section 4.3.2.3 for a more detailed discussion of gender issues). LC3 council courts reprimand men who beat their wives.136 LC3 council court reprimands, together with the actions of MCA participants and peace community members, have resulted in a reduction in cases of domestic violence and sexual and gender-based violence (SGBV). According to the members of the Kotido Sub-county peace committee, forced marriages had decreased.137 4.3.2 Strengths and Challenges of the Project Design, TOC, and Implementation Methods that Influenced the Effectiveness of Technical Interventions to Achieve SO 3 Three major technical interventions were designed to reduce conflict and consolidate and increase peace and security. These interventions are represented under IRs which involved the following: (1) strengthening local conflict management capacity, (2) increasing cooperation between formal and informal government structures, and (3) enhancing constructive male and female and youth engagement in peace and development initiatives. SO3 interventions were based on an analysis of the context, and, whenever necessary, implementation of interventions was adjusted to respond to not only the changing 132 KII, Project Manager, AWOTID, Abim, September 27, 2018 133 KII, 3rd Division Intelligence Officer, Moroto, interviewed in Kotido, October 4, 2018 134 MC Final IPTT 2018 135 FGD with members of Rengen Sub-county Peace Committee, Kotido District, October 2, 2018 136 KII, LC3 chairperson, Rengen Sub-county, Kotido District, October 2, 2018 137 FGD, Nakapelimoru Sub-county Peace Committee, Kotido, October 3, 2018 Final Performance Evaluation of Northern Karamoja Growth, Health, and Governance Development Food Assistance Project – Final Evaluation Report 56 conflict context but also to community needs and demands. The GHG project’s adaptive management approach was tailored to the interventions for conflict reduction in Northern Karamoja. GHG emphasized collaboration, learning, and adaptation while undertaking interventions to achieve the project goal and objectives. GHG’s open dialogue management style, which allowed direct feedback on activities implemented to be elicited, increased opportunities for reflection and learning. The latter ensured that GHG strategies remained relevant to the overarching goal of the GHG project while remaining responsive to the changing conflict dynamics in Karamoja.138 4.3.2.1 Local Conflict Management Capacity Strengthened—IR 3.1 The design for IR 3.1 was built on the recognition that although forceful disarmament removed most of the illegal firearms in civilian hands and ended large-scale, culturally organized livestock raiding and road ambushes, it neither eradicated cattle thefts and other forms of crimes by habitual criminals, nor prevented the emergence of new conflicts, which were escalated by inherent weaknesses in the formal and informal structures and institutions for conflict management. Traditional authorities were not the only relevant community structure. Capacity strengthening of community structures also targeted peace committees, ALCs, LFPVs, and women’s peace groups. In addition to training, community structures also needed an avenue for generating income so they could be more effective and become sustainable. Under GHG, however, funding grants for supporting the establishment of income-generating activities for peace structures under GHG’s Strategic Investment Fund were extended only to women’s peace groups and not to peace committees (Mercy Corps Uganda 2017, 22). The plausibility of this pathway was also undermined by the support to the government Bolus tagging system, which, by its failure, did not augment local conflict management capacity strengthening. It was neither embraced by local communities, nor did it reduce cattle thefts. It did not add value to achieving the outcome at the SO level. Although plausible, the pathway was encumbered by challenges that affected outcomes. MC’s use of a project-wide facilitative approach entailed relying on local actors to implement project interventions necessitated strengthening the capacities of local community structures. Local community structures that were targeted under SO3 included peace committees, LFPVs, and cultural elders. Undertaking interventions primarily through local formal and informal authority structures bolstered the capacity of local systems for conflict mitigation, resolution, and prevention. The training provided to strengthen the capacities of local community structures encompassed a number of areas, including conflict mediation and resolution skills, consensus-building techniques, conflict monitoring and reporting, conflict referral pathways, conflict early warning and response, advocacy, reconciliation, and mediation, as well as linking formal and informal governance structures. The GHG strategy was to build on existing strengths in local communities affected by conflicts to create and reinforce their capacities to mitigate, resolve, and prevent armed conflicts (Mercy Corps Uganda, 2012, p. 2-4). This approach of targeting through local structures worked well. Peace committees were initially conceived to facilitate stolen animal recovery, but they became extremely important institutions for handling all the different forms of conflicts in the villages. They also functioned as monitors for early conflict warning, response, and mitigation, and were the first point of call for incidents of domestic violence. Domestic violence cases reduced because of the severe sanctions that were implemented. Peace committees are also involved in community policing.139 The increased effectiveness of peace committees in facilitating quick recovery of stolen animals in Kaabong and Kotido was confirmed by the UPDF unit at Abuk in Abim District140 and corroborated with the Brigade Commander of the 405 138 Advanced Marketing Systems, Final Evaluation of the Growth, Health and Governance Project, p. 16-17, 2017. 139 FGD with members of Rengen Sub-county Peace Committee, Kotido District, October 2, 2018 140 FGD with the Commanding Officer, Administration Officer, and Intelligence Officer, Oscar UPDF/LDU Battalion at Abuk, Abim District, September 29, 2018 Final Performance Evaluation of Northern Karamoja Growth, Health, and Governance Development Food Assistance Project – Final Evaluation Report 57 Brigade at Nakapelimoru.141 Quick recovery of stolen animals had reduced mob justice, which was noted in the 2016 GHG Annual Results Report (Mercy Corps Uganda, 2016, 20). As a result of the effective capacity strengthening received by community structures, the population preferred local community structures to handle conflicts rather than the police and courts of law (Mercy Corps Uganda, 2017, 22). LFPVs created by AWOTID with support from GHG provided a platform through which land conflicts were effectively engaged in Abim. AWOTID trained a total of 70 (35 female and 35 male) parish￾resident LFPVs, who handled land conflict mitigation, resolution, and management. In each parish, two LFPVs were selected by the community (one female and one male). As a result of the community structures developed for land conflict management, large-scale mob violence associated with land conflicts between communities diminished. Although there were still land conflicts in Abim, within communities and between communities, the scale and intensity of violence was significantly reduced.142 The support provided to community structures led to improved reporting of conflicts, which made it easier to quickly address and mitigate conflicts and prevented escalation. GHG supported training for peace committees but could have done more by supporting income-generating activities for the peace committees under the Strategic Investment Fund. GHG used this fund to support key investments to improve the activities of women’s peace groups (such as the Kawalakol and Karenga women’s peace groups) that were involved in community-based conflict mitigation activities (Mercy Corps Uganda, 2017, 22). 4.3.2.2 Increased Cooperation Between Formal and Informal Government Structures—IR 3.3 The GHG design for I.R. 3.3 was built on the recognition that resolving and preventing conflicts in Northern Karamoja could not be achieved if all relevant stakeholders, state (formal) and non-state (informal), were not on board, complementing each other. The project correctly targeted not only the strengthening of the abilities and capacities of formal and informal conflict management structures and institutions to share formation on conflicts, but also their capacity to work together to resolve and prevent conflicts.143 As a result of the improved collaboration between the formal and informal structures, it was easier for the UPDF and its auxiliary forces to consolidate the peace security in Northern Karamoja as evidenced by the fact that no large scale cattle raids or road ambushes had been reported in Karamoja since 2013. People now feel free to move everywhere in Karamoja.144 The improved collaboration between the UPDF and law enforcement agencies enabled security agencies to address the still high rates of crime. This was confirmed by the UPDF Brigade Commander of the Nakapelimoru-based 405 Brigade.145 GHG’s approach to conflict management involved enhancing social cohesion by building conflict management capacities and platforms within and between communities and by strengthening the capacities of and cooperation between actors and institutions with responsibilities for conflict management in the formal and informal sectors. GHG’s approach to this involved working with formal courts to allow cultural elders and LFPVs to mediate land conflicts, allowing testimonies of elders to be introduced as evidence in Grade I Magistrates’ Courts. Grade I Magistrates occasionally issued 141 KII, Commander, 405 Brigade Nakapelimoru, Kotido, October 1, 2018 142 KII, Project Manager AWOTID, Abim, September 27, 2018 143 This statement is based on the recognized importance and effectiveness of involving both traditional community or tribal based structures and state-based structures in peace and security initiatives in countries where traditional structures (such as those in Africa) historically involved in conflict prevention, management and resolution are respected and considered to be vital by rural populations in particular. The primary point being made is that MC added skills development and introduced new practices specifically for helping traditional and state-based governance structures to work together collaboratively for increasing peace and security in Northern Karamoja. 144 KII, Commander, 405 Brigade Nakapelimoru, Kotido, October 1, 2018 145 Ibid. Final Performance Evaluation of Northern Karamoja Growth, Health, and Governance Development Food Assistance Project – Final Evaluation Report 58 administrative orders appointing cultural elders and LFPVs as mediators in civil suits involving land conflicts. The GHG approach also encouraged local council courts and ALCs to work closely with cultural elders and LFPVs in resolving land disputes and conflicts to reduce the number of cases referred to formal courts. The level of confidence and trust in the cultural elders in resolving land conflicts increased. As a result, the number of land conflict cases mediated by cultural elders and LFPVs increased.146 In Kotido and Kaabong, the UPDF collaborated with sub-county leaders and local NGOs whenever theft of cattle was reported, as a procedure.147 Several beneficial consensus-building forums were conducted within148 and between district local governments in Karamoja,149 with government agencies,150 and with neighboring districts,151 and also with cross-border communities, especially the Turkana.152 This strategy of increasing community capacity to engage with government and increasing government capacity to identify and respond to community needs was successful in building trust and confidence among parties to conflict by creating a conducive environment for greater cooperation in mitigating, resolving, and preventing conflicts. In 2017, although activities involving the collaboration between informal structures and formal structures continued, either no support was provided for these activities (Mercy Corps Uganda 2017, 20-3), or the activities were pursued through youth-led CBOs (Mercy Corps Uganda 2016, 21). This affected the outcome in Kotido, where the most active of these structures, the peace committees at the sub-county levels, were not directly supported under GHG. 4.3.2.3 Enhanced Constructive Male and Female and Youth Engagement in Peace and Development Initiatives—IR 3.3 The third SO3 technical intervention, “Enhanced constructive male and female youth engagement in peace and development initiatives,” reveals a major principle in conflict management, which underscores the importance of involving those who are the intended “beneficiaries” of the planned interventions— women and youth. This intervention was extremely relevant to respond to issues of promoting sustainable peace, which works best for those who need it most because they are most affected by insecurity or are at most at risk of being recruited into armed conflicts. This intervention enhanced the downward accountability of service providers by empowering participants to demand services and to hold servicer provides accountable in a transparent manner. Women’s Participation in Peace Building GHG project interventions to enhance women’s participation in peace-building activities entailed several approaches. First, GHG adopted a gender-sensitive approach in undertaking all interventions under SO3. All GHG interventions were subjected to a gender analysis (Advanced Marketing Solutions 2017, 8). Interventions under SO3 were to be guided by gender analysis and justified on the basis of their impact on different categories of men and women, with the generation of gender disaggregated data (Mercy Corps Uganda, 2012, 4). Second, GHG targeted cultural norms, which define gender roles in ways that create inequities between men and women. Those targeted included widow inheritance, forced marriage, early marriage, and polygamy, among others. These norms were targeted because they 146 Ibid. 147 This was revealed to the QET by 3rd Division Intelligence Officer, Moroto, during an interview discussion at KOPEIN offices in Kotido, October 4, 2018 148 See Mercy Corps Uganda, GHG Monthly Activity Report, January 2017. 149 See communication from District Chairperson, Abim, of 13 March 2017, on the subject: ‘Request for facilitation for a dialogue meeting,’ addressed to the Coordinator, AWOTID Uganda, Abim District, Ref. CR/ABM/COU/224. 150 See minutes of intra-land dialogue meeting between the National Forestry Authority and communities of Angwee, Yenglemi, Entebbe, and Kalakala North wards, held on Saturday, 18 February 2017, at Akur Central Forest Reserve, Abim Town Council. 151 See Mercy Corps Uganda, GHG Monthly Activity Report, October 2016. 152 See Mercy Corps Uganda, GHG Monthly Activity Report, January 2017. Final Performance Evaluation of Northern Karamoja Growth, Health, and Governance Development Food Assistance Project – Final Evaluation Report 59 underlie the widespread occurrence of gender-based violence, the effects of which have shown to increase conditions of food insecurity. They were targeted through gender trainings for men and women, the inclusion of men in GHG activities and training designed initially for women participating in MCGs, interventions from MCAs, as well as through dramatic programs and songs that raised awareness of the negative effects of these practices in communities (Mercy Corps Uganda, 2012, 8). The third approach was to sensitize women and heighten their awareness of events and circumstances that made them vulnerable to SGBV. Women are subjected to rape and sexual assault in the event of unsuccessful cattle raids or theft. Gender-based violence, including rape, is widespread, and women are especially vulnerable to rape because of their daily household responsibilities for collecting firewood and fetching water outside of the manyatta, because women are responsible for performing daily tasks on their own outside of their manyattas, and especially in places afflicted by insecurity associated with livestock raids and thefts before disarmament was accomplished (Mercy Corps Uganda, 2012, 8) and gains in regional security were achieved through support from GHG interventions. Fourth, GHG targeted strengthening women’s rights to land. Although women are responsible for providing food for their families, they have limited access to food in terms of quality, diversity, and amount. In addition to feeding their families, they are also responsible for caring for children, cooking, and cleaning. Cattle, which are an important source of food for households, are the property of the husband. Women are allowed ownership of chickens, granaries, and sometimes household items. Women are often viewed as minors their entire lives. As such, men make the major decisions, which often exposes women to abuse and exploitation (Mercy Corps Uganda, 2012, 8). The fifth approach entailed establishing, building, and strengthening the capacities of women’s peace groups as a strategy to bring women on board in leadership and decision-making structures as well as to support their effective involvement and engagement through appropriate training in leadership and income generation (Mercy Corps Uganda, 2017, 22). The approaches adopted by GHG led to increased involvement of women in peace-building activities. By the end of the project, more women were involved in peace committees. In Kotido, effective participation of women and youth in conflict mitigation and resolution activities had increased because of women’s involvement in sub-county peace committees. In the Rengen Sub-county peace committee executive committee, 4 of the 20 members are women, 153 and there is one woman in the Nakapelimoru sub-county executive committee. 154 In the traditional authority structures, such as the Akiriket among the mainstream Karamojong tribes and the Othem Abiro among the Ethur of Abim, the involvement of women is prohibited. The informal structures that the GHG project supported allowed women into their membership. For example, the Kotido Elders Council had a membership of 65, of whom 40 were women, and 25 were men.155 More women were involved in peace committees at the parish level than at the sub-county level.156 MC recognized the difficulty of achieving gender parity on peace committees, and even when there were women on the peace committees, they did not play a substantive role (Mercy Corps Uganda, 2016, 21). Even when women did not contribute substantively to the peace committees, in the villages the majority who turned up whenever security meetings were called were women. Peace committees tasked women to control their sons and husbands to desist from stealing cattle. Women reported to the peace committees and Local Council 1 immediately if they detected suspicious actions of their sons and husbands likely to lead to theft, and they were doing so without fearing any form of repercussions. 153 KII, LC3 Chairperson, Rengen Sub-county, Kotido District, October 2, 2018 154 FGD, Nakapelimoru Sub-county Peace Committee, Kotido, October 3, 2018 155 FGD with members of Kotido Elders Council, Kotido, October 2, 2018 156 KII, Chairperson, Dodoth Elders Council, Kaabong, October 9, 2018 Final Performance Evaluation of Northern Karamoja Growth, Health, and Governance Development Food Assistance Project – Final Evaluation Report 60 Whenever cattle theft took place, women were always told to be on the lookout for strange animals. Women also function as early warning conflict monitors in the villages.157 Constructive Youth Engagement in Peace and Development The strategy to constructively engage youth in GHG was designed to address growing problems of widespread youth unemployment. These include violence against women, alcohol abuse, and theft. The strategy was pursued by strengthening youth group capacity to participate in peace and development initiatives and increasing youth participation in government-sponsored conflict and development activities, thus giving them a stake in creating peace and security in their communities The GHG project adopted a youth-sensitive approach across all interventions under SO3 to ensure that youth issues were articulated and integrated (Advanced Marketing Solutions, 2017, 8). In Kotido, the number of youth-led CBOs supported by MC increased from two to six, including the following: Jie Community Development Initiative (JIECODI), Jie Disarmed Youth Group (JIEDAY), Karamoja Action Research Team (KART), Lotanyat Youth Group (LOTYOG), Pastoralism and Development Organization (PADO), and, Napumpum Cattle Traders Youth Association (NATYA) (Mercy Corps Uganda, 2016, 21; Mercy Corps Uganda, 2015, 4; Mercy Corps Uganda, 2014, 20). The GHG approach for supporting trainings and other forms of initiatives provided to these youth-led CBOs significantly strengthened the capacity for youth engagement not only in conflict mediation and resolution but also in open forum engagement and consensus-building (Mercy Corps Uganda, 2017, 21). Each of these youth￾led CBOs also mentored several smaller individual youth groups, thus spreading the capacities further. The support to youth-led CBOs was scaled down after the project ended. Youth were also included in sub-county peace committees with older adult men and women. Some of the men participating in FGDs in one of the sub-county peace committees mentioned that were serving as mentors and role models for young men. Young men were engaged in the collaborative approach between peace committees and the UPDF to track down and capture cattle thieves. One of the positive outcomes of youth involvement in peace committees was that youth in the wider community began to cease considering the UPDF as their enemy.158 4.3.2.4 Addressing Alcohol Abuse Addressing alcohol abuse was not part of the original project design but was taken on board after studies revealed that it was an underlying cause of the rising incidences of domestic violence. In 2012, MC ranked SGBV as the third most critical security concern in Uganda. The strategy to address it was to target negative Karamojong customs and practices, as well as informal traditional institutions that were renown for biases and stereotypes. The solution that was proposed was prohibiting these practices using legislation (Mercy Corps Uganda, 2013b, 13). The districts also viewed alcohol abuse as an area in which urgent attention was needed. The districts were grappling with the consumption of the portent but portable sachet waragi drinks, nicknamed “easy￾load” in Kotido,159 as well as other crude forms that were transported in truckloads of 20 liter plastic jerrycans from as far as Jinja district. Alcohol was not only a cause of poverty and a driver for cattle thefts and armed conflicts, but also a cause of domestic violence. During a district council seating in Kotido in November 2016, it was revealed that Jie boys sell crude waragi to Turkana as bait to steal 157 KII, LC3 Chairperson, Rengen Sub-county, Kotido District, October 2, 2018 158 KII, Commander, 405 Brigade Nakapelimoru, Kotido, October 1, 2018 159 See Min. 33/COU/05/2016, in Minutes of the 1st Council Meeting of the 10th Council of Kotido District Local Government and swearing in ceremony held on Wednesday 25th May 2016 in the Medical Hall (at the District Headquarters) from 9:00 a.m. to 2:55 p.m., pp. 12. Waragi is local potent gin ‘manufactured’ using ethanol as a key ingredient. Final Performance Evaluation of Northern Karamoja Growth, Health, and Governance Development Food Assistance Project – Final Evaluation Report 61 their animals after they get drunk.160 In Kaabong, the main concern raised in the district council was that waragi was killing many people in the communities, including leaders, hence the need to identify the sources of the waragi and stop its entry into the district.161 GHG supported activities aimed at reducing alcohol consumption not only to leverage increased gender equality outcomes but also to address a critical cause of poverty. In 2016, when MC began to implement interventions to address alcohol consumption, including supporting public campaigns, the impact of GHG’s other activities targeting domestic violence began to yield positive dividends (Mercy Corps Uganda, 2016, 22). Analysis revealed that GHG support for alcohol control activities was an unplanned and unintended pathway for addressing domestic violence, and this campaign entailed interventions that MC had not originally intended. MC supported Karamoja District and sub-county leaders and councilors to visit Gulu to learn about their experience in addressing alcohol abuse and how they went about formulating an ordinance on alcohol consumption. On return, and with support from MC, Abim and Kotido Districts formulated ordinances against alcohol consumption, which were at different stages of approval by the attorney general’s office in Kampala. The process of formulating the ordinances also entailed sensitization and awareness of dangers associated with alcohol consumption, such as poverty, domestic violence, and reduced productivity. As a result of these activities, alcohol was no longer freely trafficked into Karamoja. When impounded, both transporters and alcohol owners are fined heavily, and confiscated alcohol is destroyed.162 To curb alcohol abuse, some sub-counties banned its importation from outside Karamoja (Advanced Marketing Systems 2017, 48). A number of villages took further steps and formulated by-laws to restrict the importation and consumption of alcohol. Any import of alcohol is closely monitored. Several villages formed anti-alcohol committees (Mercy Corps Uganda, 2016, 22). In Morulem Sub-county, cases of drunken male community members beating their wives and children to death, health cases associated with alcohol abuse, and cases of men drinking to their death were reduced.163 Efforts to reduce locally produced alcohol using sorghum are being addressed by local councils. After learning how to formulate by-laws to control alcohol trade and consumption, community structures also transferred the skills they had acquired with support from GHG to control the wasteful handling and consumption of locally produced sorghum by households in the villages. In Rengen, local councils formulated and passed a by-law to discipline whoever was found selling sorghum to prevent food stock running out sooner than expected.164 The implementation of this by-law started in 2017. Those found selling sorghum were subjected to public flogging as punishment and sorghum they were trying to sell confiscated.165 The Local Council Level 1 chairperson told the evaluation team: “We passed a by-law in the village stipulating that anyone found selling his or her sorghum would be subjected to canes in the presence of all people in the village. It has not helped much, because people still sell their sorghum. We caught two people selling their sorghum recently. We confiscated the sorghum and put it in the village store. We have cautioned buyers to stay away from the village. If we find any, we usually chase them from the village. They normally go away once we tell them, but people continue selling their sorghum. They carry it to the town in small quantities.” 166 160 See Minute No. 20/COU/09/2016 in Minutes of the 4th Meeting of the 10th Council of Kotido District Local Government held on Friday 28th November 2016 at the Youth Centre (Former Court Hall), pp. 20. 161 See Min. 06/05/2016 – District Chairperson address District Council, in Minutes of the 1st Council Meeting of the 1st Session of the 3rd Council of Kaabong District held on 23rd May 2016 at the District Council Hall, pp. 9. 162 KII, Sub-county Chief, Morulem, Abim District, September 26, 2018 163 Ibid. 164 KII, LC1 Chairperson, Rengen, Kotido, October 2, 2018 165 Ibid. 166 Ibid. Final Performance Evaluation of Northern Karamoja Growth, Health, and Governance Development Food Assistance Project – Final Evaluation Report 62 Overall Factors in the Implementation Approaches, Methods, and Context in Influencing Results Achieved The following factors in the implementation approaches, methods, and context influenced the results that were achieved:     The relevance of the GHG project in addressing the needs in the communities in the target districts was enhanced by intervention approaches that emphasized partnership with indigenous civil society that make it possible for results from capacity strengthening to be retained in the communities where the project interventions were undertaken. GHG built its interventions on the existing strengths of local communities affected by conflicts, which made it possible to create and reinforce resident capacities to mitigate, resolve, and prevent armed conflicts. The project was not building from the scratch. It was starting from what existed. The flexibility in the approaches used by MC in undertaking interventions under SO3 enabled GHG and its implementing partners to adjust implementation activities to accommodate the continually changing context in the districts in Northern Karamoja. The effect was solutions that worked for both the district local governments and GHG. GHG was also willing to listen to feedback and learn lessons as project implementation progressed. The districts and communities where the implementation took place were willing to provide feedback. 4.3.2.5 New Challenges The main challenge with GHG’s general approach was its inability to respond programmatically to all the emerging issues in the conflict context of Karamoja. Beginning in 2012, GHG’s focus in Kotido and Kaabong was animal theft. By 2015, land conflicts within families, clans, and communities, and between communities, sub-counties, and districts had become a critical constraint in the communities. MC recognized that the conflict context had changed as land conflicts and criminal activities (petty crime, domestic violence, rape and murders) had become the primary conflict issues (Mercy Corps Uganda, 2016, 20). In the same way, although the focus of GHG in Abim was land conflicts, animal thefts, which peak during the dry season, continued to be a critical constraint to livelihoods in Abim.167 The primary focus of GHG in Abim continued to focus exclusively on land through the life of the project. In Abim, community structures for responding to land conflicts were not adequate to address cattle thefts168 in the way that peace committees in Kaabong and Kotido were ingeniously handling other forms of conflicts in addition to cattle thefts. 4.3.3 EQ4. Which project outcomes are likely to be sustained? Local Capacity to Manage and Resolve Conflicts: Sustainability of Peace Committee Structures Peace committees play an essential role in conflict early warning and response; but there are legal, financial, and technological issues that should be addressed to sustain these community-based structures. In terms of legal sustainability, peace committees have no clear legal or policy framework. Uganda signed the Inter-Governmental Authority on Development Conflict Early Warning protocol in 2002. This protocol requires member countries to establish peace structures at all levels, a responsibility assigned to the Conflict Early Warning and Response Unit in the Ministry of Internal Affairs. The Unit spearheaded establishment of peace committees in Karamoja and neighboring regions. However, a National Peace Policy is not yet in place to domesticate this aspect of the Conflict Early Warning 167 FGD with the Commanding Officer, Administration Officer, and Intelligence Officer, Oscar UPDF/LDU Battalion at Abuk, Abim District, September 29, 2018 168 KII, Commander, 405 Brigade Nakapelimoru, Kotido, October 1, 2018 Final Performance Evaluation of Northern Karamoja Growth, Health, and Governance Development Food Assistance Project – Final Evaluation Report 63 protocol. A draft has been before cabinet awaiting approval since 2017. The draft must be approved before the National Peace Policy can be tabled to parliament for discussion and promulgation. Without the approval of the national policy on peace building and conflict prevention, peace committees do not have a clear legal framework. Without a policy or legal framework, these peace committees remain ad hoc structures that can be done away with by an administrative decision. Peace committees were constituted differently and perform different functions in the three districts. This implies that not much attention was given to the organizational and institutional sustainability of peace committees. The peace committees were self-regulated; none had formulated constitutions, rules and regulations, or by-laws for guiding the conduct of committee members. It terms of financial sustainability, peace committees found it difficult to leverage consistent public funding without a clear policy and legal framework. Neither the ministries nor district local government have budgets or provide funds for peace committees. When funds have been budgeted and approved for peace-building activities, especially under the Office of the Prime Minister, these funds can be spent in any way, on any activities, other than peace committees. Funding from districts was irregular, limited, and incidental, when available. Funding for peace committees was provided by NGOs, but it was limited to the life-span of their projects. Security Committees are funded through funds that go to the Office of the Regional District Council from the Office of the President. There is no similar funding arrangement for peace committees. Membership of the peace committees is voluntary. Members devote a considerable amount of time executing committee functions on behalf of their villages and sub-county. The only benefit associated with membership were the animals collected as a fine from those arrested during cattle theft under the Moruitit resolutions. There was no support provided to help members develop income-generating activities or any other form of support extended which would enable them to obtain some income to support their households needs. Such support would enable peace committee members to leverage financial support through groups with which they are affiliated, as a form of recognition for their work. There was no activity to help peace committees establish VSLAs, or preferential access to micro-credit or other forms support provided under GHG, such as seed vouchers. Peace committee members were not given any support for using the animals with which they are rewarded after recovery of stolen animals as a form of capital, to use as contributions to a savings scheme or to invest in income-generating activities. Neither in Kaabong or Kotido was there a discussion about supporting members of peace committees to make monthly financial and other in-kind contributions in support of the activities of the committees, as a form of internal fundraising, such as a form of voluntary membership subscription contribution. After GHG ended, none of the peace committees had thought about sourcing for alternative support from elsewhere. Regarding the technological sustainability of peace committees, the 2017 GHG Annual Results Report showed that GHG provided mobile phones and radios to peace committee members to improve communication and access to information to bolster their ability to act as an early warning system and quickly respond to conflicts (Mercy Corps Uganda, 2017, 22). MC also donated mobile phones and radios to sub-county leaders at the end of the GHG, but there were not enough for all of them. Continuity of Service Provision After End of the GHG Project Some of the local structures supported by GHG continue to actively provide some forms of service provision. For example, LFPVs in Abim have continued mediating land conflicts since AWOTID’s GHG sub-contract ended.169 Peace committees also continued performing their functions after GHG ended. Peace committees have continued intervening in cases of domestic violence in their respective communities. 169 KII, Project Manager AWOTID, Abim, September 27, 2018 Final Performance Evaluation of Northern Karamoja Growth, Health, and Governance Development Food Assistance Project – Final Evaluation Report 64 In Kaabong and Kotido, one year after GHG ended, peace committees were initiating meetings for negotiating access to pastures in neighboring districts.170 Peace committees created a platform and forums for ongoing dialogue and consensus building. However, interview respondents across the three districts reported that the frequency of peace dialogues and inter-community meetings was reduced after the end of the project, yet continuous dialogue formed the very foundation that informed the pathway preferred by GHG under SO3. In some cases, no meetings have been held since GHG ended. Peace building and mind-set change are ongoing processes that require continuous engagements to succeed. The end of GHG affected the progress that had been achieved with some of the peace initiatives that were supported through community structures. Continuous dialogues are necessary to build confidence between communities, which increases trust. Communities left without continuous engagement could easily relapse into violence after the communication channels break down.171 Although the support provided by GHG helped consolidate the peace that had been achieved in Northern Karamoja, there are still challenges. Insecurity increases during the dry season, especially in communities in Kaabong and Kotido that share pastures and water with the Turkana of Kenya. The evidence that these places were still affected by insecurity associated with cattle thefts was the continued existence of protected kraals. Protected kraals were dismantled throughout Karamoja after the government declared the success of forceful disarmament, but have been reintroduced in Sidok and Loyoro Sub-counties. The most affected parishes in in Sidok Sub-county included Locherep, Kasimeri, and Longaro. In these parishes, LDUs are embedded in kraals both day and night for the protection of herds from cattle thieves.172 In Abim, the evaluation team was told by Othema Abiro cultural elders, ALCs, and AWOTID LFPVs that although they continued working after GHG ended in 2017, they were no longer as actively involved in conflict mitigation, resolution, and management as they were during the project. AWOTID used to facilitate their attendance in dialogue meetings and to mediate land conflicts. Whatever services they have been able to continue providing in 2018 was primarily from deployment of their own resources. Transportation is an issue. Cultural elders, ALCs, and LFPVs said they lack transport and wished they could get bicycles. In Kaabong, peace committees interviewed also continued working after support from GHG and KAPDA ended. They, too, mentioned the problem of lack of transport and wished they could get bicycles for use during tracking of stolen animals. GHG gave out mobile phones as an exit strategy. The people who received phones said that they lack money for airtime to keep communicating with each other. Each sub-county received one mobile phone. In Kotido, peace committees continue to provide their services. They receive continued capacity building support from Kotido Peace Initiative (KOPEIN), a local NGO long involved in conflict management work in the district.173 KOPEIN was not a local GHG implementing partner; however they continue working with peace committees and cultural elders in conflict management and they fill a gap in support following the termination of the Pastoralists Poverty Frontiers contract. In August 2017, at the end of the GHG project, MC donated equipment to conflict stakeholders in the project area. In Kotido, MCAs were given radios. The sub-county peace committees were given ordinary Nokia mobile phones (not smart phones). The Rengen LC3 chairperson told the evaluation 170 KII, LC3 Chairperson, Rengen Sub-county, Kotido District, October 2, 2018 171 KII, 3rd Division Intelligence Officer, Moroto, interviewed in Kotido, Oct. 4, 2018 172 KII, LC3 Chairperson, Sidok Sub-county, Kaabong District, October 10, 2018 173 KII with Director of KOPEIN. Final Performance Evaluation of Northern Karamoja Growth, Health, and Governance Development Food Assistance Project – Final Evaluation Report 65 team: “We were told the phones were for communicating security issues and the radio was for listening to community issues mentioned on radio programs. We donated the phone to the LDU commander and the radio to the GISO.” 174 Officials in Rengen said the donation was a little too late, and the equipment donated was not enough to have a meaningful impact.175 Peace committees interviewed wished they were supported with more tangible equipment for their work, such as bicycles for peace committee members and at least a motorcycle for each sub-county peace committee chairperson.176 The targeted participants were never adequately prepared for the phase-out and exit of GHG, even when MC claimed that the GHG was designed to have “an early focus on exit,” which enabled the implementing partners’ exit to be “baked in” to ensure that “intervention benefits remain in the target community well after the intervention is completed” (Mercy Corps Uganda, 2013b, 18). The respondents in the communities felt that GHG implementing partners were never properly facilitated to smooth their exit from supporting GHG interventions in the communities.177 Community Acceptability and Perceptions of GHG Interventions and Implementation Processes and Their Frequency The evaluation team interviewed respondents at district, sub-county, and community levels. There were different perceptions about the quality, frequency, and effectiveness of the implementation processes and services provided. The interventions and implementation processes that were considered more acceptable to project participants included peace committees in Kotido and Kaabong as evidenced by involvement of other people in communities Members of the peace committees interviewed indicated that when theft of animals takes place in a community, everybody gets involved in passing on information about the theft to the next person, by either word of mouth, or telephone for those who have them, which gets everybody involved in recovery of the stolen animals. It is usually expected that everyone in the community, including the elders, kraal leaders, the youth (both male and female), the women, and children, be vigilant and observant and to monitor movements of people and animals, and to detect and report suspicious acts that may indicate a possible involvement in animal-theft related crimes.178 Community preference for non-formal structures in the management of conflicts is partly explained by the “win-win” formula used in which compromises are sought, rather than the formal structures where “winners take all.” Blood compensation was still practiced, especially in capital offenses involving murder. The Jie paid several cows as compensation for a pregnant Bokora woman from Lokopo who was killed by the Jie while coming from Kanagar. Even when the case was reported to the police, the family of the culprits surrendered cows for blood compensation to the UPDF for delivery to the relatives of the deceased as a way of preventing the start of revenge killings.179 The work of the peace committees in Kaabong and Kotido were guided by the spirit of what in Ngakarimojong is called Tochak Lotem, meaning “let us come together and solve this issue.” When an incident occurs in the village, an alarm is made at around 4 p.m., and the person who sees the fighting mobilizes others, so they can come and find out what exactly happened. Tochak Lotem is also the code 174 KII, LC3 Chairperson, Rengen Sub-county, Kotido District, October 2, 2018 175 Ibid 176 FGD, Rengen Sub-county Peace Committee, Kotido, October 2, 2018 177 FGD, Rengen Sub-county Peace Committee, Kotido, October 3, 2018 178 FGD with the Commanding Officer, Administration Officer, and Intelligence Officer, Oscar UPDF/LDU Battalion at Abuk, Abim District, September 29, 2018 179 See Min. 3.0/04/2018 ‘Remarks from sub-county Chairperson’, in Minutes of the Lokopo Sub-county Council Meeting held on 26th April 2018 at the Community Hall, Aramam, pp. 7. Final Performance Evaluation of Northern Karamoja Growth, Health, and Governance Development Food Assistance Project – Final Evaluation Report 66 word for convening a public gathering that constitutes a public court for emergency handling of domestic violence. Through Tochak Lotem, information is quickly passed from one person to another, and people quickly gather at the Etem (gathering place). Members of the peace committees are supposed to be exemplary by not indulging in domestic violence. They are role models. In Kotido, some of the peace committee members were also MCAs in their communities. They function like a community police because they are the first point of reference for law and order enforcement. In addition to cattle thefts, peace committees handled domestic violence, land conflicts, and any other conflicts in the communities.180 When a borehole gets spoiled, the peace committees take reports to the sub-county and district officials.181 Communities in much of Kaabong District welcomed the establishment of mobile peace committees that maintained law and order in the dry season grazing areas.182 Community and Participant Motivation to Demand Services Members of communities were motivated to join peace committees to ensure that there was peace in their communities. They all knew before they joined that there was no payment. Individual motivations varied. Some wanted to create a conducive environment for their children to remain in school and they wanted to be able to reach their gardens without fear. Others wanted to be able to continue using the road without fear of an ambush.183 Members of the community find it convenient for their land conflict cases to be handled through mediation by AWOTID and the LFPVs rather than proceed with lengthy and expensive litigation processes through the magistrate’s courts.184 However, although trainings of community structures were appreciated by the young people, who were also specifically targeted, the trainings were few and far apart. Communities felt that GHG did a very good job but ended too soon and too abruptly. Leveraging of Other USG and Non-USG Investments to Achieve Sustained Outcomes The evaluation team did not find evidence that the GHG project leveraged other non-USG investments to achieve sustained outcomes as identified in the TOC Evidence of Enhanced Linkages with Other Service Providers There was evidence that some of the linkages established under GHG with participation from project￾supported local structures have since been enhanced with both state and non-state structures.. These enhanced linkages also serve to enhance the capacity and the likelihood of sustainability of those local structures. A number of these linkages were considered by individuals interviewed across the three districts to be successful. In Abim, the Grade I magistrate sometimes appointed LFPVs created by AWOTID as land conflict mediators. Those LFPVs who performed well have continued working with the Grade 1 magistrate’s court as mediators in land cases. In Kaabong and Kotido, peace committees worked closely with the UPDF and its auxiliary forces in the recovery of stolen animals during and after the GHG project closed.185 Cultural elders in Kotido186, Kaabong, and Abim were often supported by 180 KII, LC3 Chairperson, Rengen Sub-county, Kotido District, October 2, 2018 181 FGD, Nakapelimoru Sub-county Peace Committee, Kotido, October 3, 2018 182 KII, LC3 Chairperson, Sidok Sub-county, Kaabong District, October 10, 2018 183 FGD, Nakapelimoru Sub-county Peace Committee, Kotido, October 3, 2018 184 KII, Project Manager AWOTID, Abim, September 27, 2018 185 FGD with the Commanding Officer, Administration Officer, and Intelligence Officer, Oscar UPDF/LDU Battalion at Abuk, Abim District, September 29, 2018 186 Ibid Final Performance Evaluation of Northern Karamoja Growth, Health, and Governance Development Food Assistance Project – Final Evaluation Report 67 the sub-county local governments and worked closely with the community development office as their first point of reference.187 Testimonies by cultural elders were permissible in courts.188 Linkages were established not only with state structures but also with non-state structures. In addition to the GHG support that cultural elders in Kotido, Kaabong, and Abim received, they were also supported by other national and international NGOs. The most frequently mentioned and extremely beneficial linkages mentioned during interviews and FGDs with cultural elders were with the following NGOs: ADRA, which supported not only training, but often provided financial and logistical support, including transport for the work of cultural elders in Kotido189, Abim, and Kaabong; KOPEIN and the Warrior Squad190in Kotido, and DADO in Kaabong, not only provided training to cultural elders, but also occasionally provided logistical support for their involvement in conflict management activities. The Rengen Sub-county peace committee have, on their own, been communicating with members of Turkana peace committees in Loima and Lokiriama in Kenya.191 4.3.4 EQ5: What are the key lessons learned and best practices that should inform future projects in the country? Key Lessons Learned      Forceful disarmament removed most of the illegal firearms in civilian hands and ended large￾scale, culturally organized livestock raiding and road ambushes, but it neither eradicated cattle thefts and other forms of crimes, nor prevented the emergence of new forms of conflicts such as land disputes between and within communities, which were escalated by inherent weaknesses in the formal and informal structures and institutions for conflict management. Involving young men in peace committees and increasing their interactions with the government in concrete activities is an effective method for reducing youth crime. The more youth the project brings into peace committees, the less youth there are in the communities who are at risk of joining armed violence. Using peace committees for conflict management is an entry point for a new institutional dispensation for harnessing the productivity of youth and also improving their lives. As the project brings more youth into peace committees, more disciplined young adults are available for productive economic engagement. Members of peace committees strive to be role models, which brings the best out of individuals who self-sanction their public dispositions. For conflict management to succeed, it has to work for those either most affected by conflicts (women), or most at risk at being conscripted into the conflicts (youth), and for that to happen, both women and youth have to be directly involved. A deliberate effort has to be undertaken on a continual basis to target their involvement and participation. Working through local CBOs and local community structures is an effective way to build inclusive governance in Karamoja because they are closest to the intended participants, and they can be easily held downwardly accountable, so credibility can be ensured. They are readily accepted, and any support that contributes to strengthened capacity is retained in the targeted community. Continuous dialogues are necessary to build confidence between communities because they increase trust between parties to conflicts. Communities left without continuous engagement easily relapse into violence after channels of communication breakdown. 187 KII, Sub-county Chief, Nakapelimoru, Kotido District, October 3, 2018 188 FGD with members of Kotido Elders Council, Kotido, October 2, 2018 189 FGD with members of Kotido Elders Council, Kotido, October 2, 2018 190 Ibid 191 FGD, Rengen Sub-county Peace Committee, Kotido, October 2, 2018 Final Performance Evaluation of Northern Karamoja Growth, Health, and Governance Development Food Assistance Project – Final Evaluation Report 68 Best Practices for Future Project Design and Implementation Approaches One best practice observed from GHG implementation under SO3 was the emergence of peace committees that were initially intended for addressing conflicts associated with cattle thefts but had become extremely relevant for not only addressing other forms of conflicts in the communities, such as domestic violence, land conflicts, and responding to criminal activities, which had transformed peace committees into a form of community policing, but also as an avenue for creating bonding, bridging, and linking social capital. Peace committees had become the first point of call whenever any acts of lawlessness became apparent. The members of peace committees had become role models, and with targeted support, they could become more effective. Risks to the Effective Project Implementation Approaches Road ambushes and culturally organized large-scale raiding have ceased, but there is still a high risk to reversion to armed conflicts in Northern Karamoja because of continuing incidents of possession of illegal firearms, increasing criminal activities, and seasonal intensification of cattle thefts. Tensions, which occasionally flare up, have continued in communities neighboring the Turkana. In some parishes of the Sidok Sub-county of Kaabong that are close to the Kenyan border, protected kraals were reestablished due to incessant cattle thefts where numbers of raided animals are becoming larger. Minimizing Potentially Unintended Negative Consequences Although flexible programming ensures that project implementation is capable of adjustment to minimize the adverse negative effects from unintended consequences, actors in strengthened community structures whose work puts their lives at risk as a result of retribution from parties affected by their decision-making and interventions need to be supported to establish working relationships with relevant security agencies for back-up. Final Performance Evaluation of Northern Karamoja Growth, Health, and Governance Development Food Assistance Project – Final Evaluation Report 69 5. RECOMMENDATIONS 5.1 SO1: LIVELIHOODS STRENGTHENED       In light of the findings that adult female-only households realized no significant changes in income and poverty status, future targeting strategies should be examined to ensure greater representation of women from these households in farmers’ groups and in microenterprise development activities. The targeting strategies for this group should be designed based on further investigation of the factors that underlie their poverty status relative to households with adult females and adult males. The evaluation’s analysis of the circumstances of women who were not included as participants in farmers’ groups under GHG indicate that they may be part of the poorest segment of community populations where the project was implemented. The circumstances of these women, as well as men who were not selected for participation in farmers’ groups, should be further investigated for potential incorporation into the overall targeting strategy. The implementation of these targeting strategies should be regularly monitored to determine whether they are effective in engaging the poorest segments of the community in strategies tailored to improve their circumstances and adjusted as needed. These actions are necessary for households with limited resources to benefit from the commercialization of crop and livestock production. Such measures will support further declines in FFP poverty indicators and prevent a growing disparity between wealthier and poorer farmers in Northern Karamoja communities. Given climate changes characterized by unpredictable variability in seasonal rainfall patterns, more frequent droughts, and heavy flooding, livelihoods approaches based on agriculture should be adjusted with strategies that include non-farm income generation, and that better integrate activities to improve livestock and crop production at a household level to build their resilience. Weak implementation of quality control measures developed for seed testing and the sale of seeds produced by commercial national and regional suppliers that are distributed to the three districts must be addressed. This is essential to reverse trends in decreased demand and to begin building buyer trust. Affordability of improved seeds is a major problem that has to be addressed on a more economically sustainable basis based on the foundation of experiences in the use of vouchers to partially subsidize their high cost. The availability of inputs (seed, pesticides, post-harvest handling materials) is important if farmers are to improve crop productivity. The work that was done to support the inputs value chain was relevant. The effort to create important supply–dealer linkages and business training is adequate to keep them operational, while demand for inputs is low because many dealers operate multiple businesses that can provide income streams to maintain their livelihood. In the interim, and until demand for inputs can be restored, additional support is required to equip them to learn how to supply the current lower demand effectively and reliably. Agricultural extension workers who were identified among well-performing farmers and live among the communities are a sustainable approach to continue these services for farmers. Their service is retained in the community after the project. However, Ministry of Agriculture Fisheries and Animal Industry regulations are clear on the minimum qualifications required of an extension worker. Therefore, we recommend that these workers be given a different title, and to increase their effectiveness over time, they should work under the supervision of qualified personnel at the sub-county agriculture office or recruiting body. Farmer training on improved crop, soil, and water management technologies should be continued. These practices have shown to increase the productivity of local seeds for those who cannot afford or who do not want to risk, use of improved seeds. Special emphasis should be Final Performance Evaluation of Northern Karamoja Growth, Health, and Governance Development Food Assistance Project – Final Evaluation Report 70 placed on practices and affordable technologies that save labor to increase the participation of adult female-only households that are likely to have labor constraints.        District and sub-county agriculture departments are financially constrained and are not able to train and provide salaries for CAHWs, despite their important contributions to increasing the health and quality of livestock. To sustain and grow the numbers of CAHWs, we support the implementation of recommendations included in the Feinstein International Center study on “Revitalized Pastoral Livelihoods in Karamoja.” It recommends expanding nonprofit support of CAHWs, including facilitating travel to kraals, regular refresher trainings, and increasing their numbers. In light of the increase in livestock diseases, the sensitization of farmers and kraal leaders on livestock management and animal preventative practices should be continued. It will be difficult to stop untrained persons from attempting to cure the diseases of animals under their care, but the correct application of treatment protocols and follow-through measures must be emphasized to prevent large-scale disease outbreak. VSLA development was key to increasing financial resources and services for village communities and in supporting start-up and development of microenterprises. Further strengthening of VSLA financial management capacity and strategies to expand that capacity are recommended, along with training in members’ financial literacy. We also recommend renewed attention to further strengthening of SACCO capacity, particularly in the areas of creating their own sound and achievable strategic plans and effective strategies for encouraging member savings and repayment of loans on a timely basis. Business forums can make significant contributions to improving the business environment and facilitating and organizing business development services suited to the needs of different kinds of businesses. Successes to date in working with district officials to eliminate some of the regulations that stifle small business growth show their potential. Business forums are unlikely to survive without additional capacity building. Technical support should be directed at growing management and leadership capacity, developing service delivery strategies, and strengthening ownership of members to not only encourage payment of dues but also their active participation in forum activities and developing agendas. Efforts should focus on businesses in town council areas and expand to the entire district gradually as the forum management capacity develops. 5.2 SO2: IMPROVED NUTRITIONAL STATUS OF CHILDREN UNDER FIVE A careful research design should be developed to investigate the factors that are prohibiting improvements in the nutrition status of children under 5 years of age. Our analysis of PBS findings on practices to improve nutritional status and practices to promote health and sanitation conditions, coupled with findings from the analysis of qualitative data collected from interviews, point to poor adoption of hygiene and sanitation practices, cultural feeding practices, and level of income as some possible factors that contribute to this lack of improvement. When food security programs, such as GHG, decide to use food commodities as one of the activities to increase caloric intake in women and children, we recommend using start-up registration criteria that initially targets only PLW with children up to 6 months of age to allow for enrollment of a smaller target group and a gradual build-up of participant numbers. This approach is recommended because it helps simplify the start-up phase, allows the program more time to develop BCC materials, and provides an opportunity for all participants to receive program services for as long as possible. Partners should agree to follow a common model for implementing MCGs. This model should include non-financial incentives for community-based volunteers (i.e., LMs) to avoid dependency and increase ownership. In contexts such as those found in Northern Karamoja, where Final Performance Evaluation of Northern Karamoja Growth, Health, and Governance Development Food Assistance Project – Final Evaluation Report 71 dependency is high following decades of humanitarian aid, activities should be employed to stimulate volunteerism. Techniques to motivate community volunteers that include non-monetary incentives should be explored. Examples include material incentives provided on an annual basis as a tangible “thank you,” supportive supervision, formal and informal recognition for community achievements, a shared sense of purpose, and personal development.      Behavioral approaches involving men, such as the MCA model implemented under GHG, should be set up and implemented at the same time that the MCGs are developed to maximize health and nutrition outcomes. Rigorous studies should be conducted to assess the impact of this approach on MCG outcomes and behavioral changes in men’s behavior. Their contribution to the reduction of domestic abuse and GBV should be assessed, but as part of an overall assessment of how activities across the three SOs have contributed to this reduction. This includes women’s empowerment activities and training in financial management and business start-up and expansion and gender sensitization training provided to men such as those involved as AEAs under SO 1, the inclusion of men in MCG training activities under SO 2, and several activities implemented under SO 3 including interventions to reduce alcoholism, gender sensitization training for youth, the involvement of youth in peace committees and in youth￾based peace committees, the involvement of women in land use dispute resolution committees in Abim, and the inclusion of women in peace committees and security committees at all administrative levels. The assessment should include the unplanned involvement of sub-county peace committee members in monitoring and addressing domestic abuse in their villages, and male members who have elected to serve as role models for appropriate behavior for young men. These roles were taken up by peace committee members on their own initiative. The pilot MCG+ model integrating activities from livelihoods and MCHN/WASH should be scaled up. The piloting of this model has demonstrated its effectiveness in empowering women to increase their financial autonomy, control over savings, and the adoption of practices to enhance household health and nutrition outcomes. To ensure continuous access to safe water sources, the collection of monthly fees from households should be encouraged, with the continued sensitization of the communities to understand the importance of preparing for maintenance and repair of community boreholes. Communities should be encouraged to come up with their own unique solutions to ensure that all households pay their monthly fees. Communities in villages such as those found in Northern Karamoja are best placed for developing solutions and strategies that are effective in the context of their own culture to ensure community wide participation and regular payments from all households. VSLAs could be used to deposit community savings for water monthly fees, for subsequent deposit in SACCOs, which would improve the security of these fees and at the same time generate profits. This approach could also be used to promote membership increases in VSLAs and SACCOs. Private sector entities involved in WASH should be engaged to provide low-cost sanitation supplies. For example, some latrine slabs are designed to be manufactured using both a prefabricated mold and in-situ construction. The defecation slab is called a sanplat (short for sanitary platform) and provides a good method for upgrading a simple latrine, improving latrine appearance and hygiene practices. Alternatively, WUCs could be trained to build cement slabs from molds using the same monthly fees collected from households for borehole maintenance. Alternative approaches and technologies to encourage use of sanitary practices, specifically the use of household latrines, should be researched and tested. Private sector entities involved in WASH should be engaged to provide low-cost sanitation supplies. The “Twin Leach” latrine Final Performance Evaluation of Northern Karamoja Growth, Health, and Governance Development Food Assistance Project – Final Evaluation Report 72 approach192 and Arboloo193 technique are potential solutions that should be investigated for use in Northern Karamoja to overcome the fear of falling in latrine pits. Literature about ecological sanitation approaches used in Uganda is available to draw on. Programs with a strong emphasis on WASH activities should carefully assess the implementation context and learn about cultural taboos and norms related to personal and community sanitation practices before identifying solutions and implementing approaches to create ODF villages.      MC, working closely with local governments and the district-based NGOs involved in local conflict management, should support the strengthening of the organizational, financial, and technological sustainability of the peace committees and other local community structures involved in conflict management, such as the cultural elders, to increase their motivation to consolidate the peace dividends in Northern Karamoja. Among others, local community conflict management structures such as peace committees and cultural elders need to be supported with regular training and public recognition in moments when they have achieved stand-out performances. Specifically, for peace committees, considerations should be made to regularly provide them with uniforms, means of transport, avenues of continuous communication, and a modest form of compensation for their work. MC, working closely with relevant government agencies and district-based NGOs involved in local conflict management, should support clarification of the policy and legal status of peace committees to streamline their roles and functions in the forthcoming draft of the national peace-building and conflict management frameworks. Household water treatment and safe storage is an important public health intervention to improve the quality of drinking water and reduce diarrheal disease and should be explored. This include sensitization about household water treatment, such as boiling and the use of clean water containers with lids. To sustain their existence without external grants, local organizations should have a clear fundraising strategy, a funds pipeline that address gaps between grants, and an internal assets analysis to determine income-generating possibilities. 5.3 SO3: REDUCED INCIDENCE OF CONFLICT 192 www.communityledtotalsanitation.org/sites/...org/files/TwinLeachPitToilets.pdf 193 Toilets that make compost: Low-cost, sanitary toilets that produce valuable compost for crops in an African context. Stockholm: EcoSanRes Programme. ISBN 978-9-197-60222-8 ANNEX 1: EVALUATION STATEMENT OF WORK Statement of Work Population-Based Final Evaluations of Food for Peace Development Food Assistance Projects GHG and RWANU in Uganda INTRODUCTION The final evaluation of the 2012 Uganda Title II Food for Peace (FFP) projects is the second and final phase of a pre-post evaluation strategy. The baseline study, conducted from mid-February through April 2013, employed a mixed-method approach, and was designed to provide information on all four aspects of food security – availability, access, utilization and stability. The study investigated household food access, sanitation and hygiene, agriculture, household expenditures and assets, dietary diversity, and anthropometry among women and children. As with the baseline study, the Uganda final evaluation will also use a mixed method approach but will also utilize and integrate secondary data and project final monitoring data. Methods will be chosen in order to generate the highest quality and the most credible and robust evidence possible to answer evaluation questions. BACKGROUND In fiscal year (FY) 2012, USAID/Food for Peace (FFP) awarded Mercy Corps (MC) and ACDI/VOCA five-year Title II development food assistance projects (DFAP) in Uganda with an overall investment of over $100 million. The MC Northern Karamoja Growth, Health and Governance Program (GHG) aimed to improve food security of households in Kaabong, Kotido, and Abim, while the ACDI Resilience through Wealth, Agriculture and Nutrition (RWANU) sought to reduce the food insecurity of households in Napak, Moroto, Nakapiripirit and Amudat regions. GHG offered a range of economic, health, and governance initiatives to solidify the gains from increased security and establish a foundation for broader self-sufficiency. Targeted food aid for pregnant and lactating women and children under the age of two was used to transition vulnerable populations from decades of food aid by filling nutrition deficits in highly food insecure households. GHG expanded in its penultimate year to conduct a livestock sector pilot program to identify opportunities for livestock production and marketing in Karamoja’s post-conflict development. The project had three strategic objectives (SO):    SO1: Pro-Poor Market Development (Livelihoods strengthened) – focused on building local capacity to provide vital products and services on a commercially sustainable basis to vulnerable households; SO2: Nutritional Status among Children under Five Improved – focused on improving local public and private health care, promoting improved household food consumption, and improving water infrastructure and sanitation and hygiene behaviors; SO3: Reduced Incidences of Conflict – focused on helping local conflict mitigation structures adapt to the current conflict dynamic, while supporting traditional authority structures and male and female youth to play more constructive roles in improving security. In the implementation of GHG, MC collaborated with numerous non-governmental and academic partners: World Vision (WV), Whave, Kaabong Peace and Development Agency (KAPDA), Abim Women Organized Together In Development (AWOTID), and Tufts University’s Feinstein International Center (FIC). MC implemented all economic programming and was responsible for ensuring that a gender sensitive approach underlay all activities. WV led supplementary feeding activities, and in collaboration with MC and WHAVE, conducted community-level public health initiatives, behavior change, water, sanitation, and hygiene programming. KAPDA and AWOTID concentrated on conflict management and governance activities, working primarily through local formal and informal authority structures to bolster local systems for conflict reduction and government service improvement. Lastly, Tufts University’s FIC conducted annual impact evaluations using its community-focused Participatory Impact Assessment methodology to create a strong understanding of the impact of the consortium’s work, and to inform implementation. The GHG consortium used a facilitative approach to implementation, pushing local actors to take the lead in sustainably providing products. ACDI/VOCA implemented RWANU in consortium with Welthungerhilfe (WHH) and Concern Worldwide (Concern). ACDI/VOCA was responsible for overall program management, as well as led activities related to crop production, alternative livelihoods, increasing resiliency through the promotion of group savings, improving market linkages, and managing food distribution. WHH carried out the technical training and input provisions related to livestock, while Concern implemented the nutrition, health, hygiene and health outreach component. ACDI/VOCA and partners collaborated to ensure integration across activities and promotion of gender equity and women’s empowerment, environmental stewardship, and conflict mitigation. Two SOs guided implementation:   SO1: Improved access to food for men and women: Interventions were tailored to the three different livelihood zones, and activities sought to reduce the risks associated with rain-fed agriculture; identify and promote low-risk, higher-return commodities; and increase livestock ownership while drawing households into market systems. RWANU also focused on building the capacity of farmer groups and small producer groups through training in technical skills, savings mobilization, basic business skills and marketing. SO2: Reduced malnutrition in pregnant and lactating women and children under five: Activities focused on preventing malnutrition during the first 1,000 days of life through a package of curative and preventative health care, behavior change activities, and improved consumption of micro- and macronutrients. RWANU implemented activities around proper infant and young child feeding practices and the nutritional needs of pregnant and lactating women and children under two, while promoting equitable intra-household food distribution. RWANU implemented these activities through the Mother Care Group approach, male change agents, livelihood trainings, opinion leaders and social behavior change activities. EVALUATION PURPOSE & QUESTIONS The overarching purpose of the final evaluation is to measure the development outcomes of the GHG and RWANU projects. In the table below, FFP has identified the evaluation questions and provided the fundamental elements that should shape the Evaluation Team’s (ET) research. It is anticipated that the ET will address these, but it is not limited to working solely within this guidance. Q1: To what extent have the projects met their defined goals, purposes and outcomes? The ET will evaluate the contribution of GHG and RWANU to USAID’s efforts to reduce food insecurity among chronically food insecure households. 11 The ET will support its determination using both quantitative and qualitative methodologies when discussing the following: (1) project final on indicators against targets set by both the partners and the key FFP indicators* of Depth of Poverty, Stunting, and Undernutrition; (2) factors that promoted or inhibited the achievement of the project objectives, including, but not limited to the effectiveness of food-for-asset and/or cash-for-asset interventions; (3) plausibility of pathways and the determinants of achieving the key outcomes; (4) targeting strategies and their contribution to achieving project goals (especially with regard to gender and reaching the most vulnerable); (5) the appropriateness and effectiveness of interventions on the poorest individuals. *FFP’s established targets are: a minimum of 2 to 2.5 percentage point annual reduction of prevalence of stunting, a minimum of 3 to 4 percentage point annual reduction of prevalence of underweight, and a minimum of 4 percentage point annual reduction of depth-of-poverty. 1 2012 Food for Peace Country Specific Information: Uganda http://pdf.usaid.gov/pdf_docs/pdacu288.pdf Q2: Based on the evidence, which project outcomes are likely to be sustained? The ET will evaluate the functionality and final of systems and processes established independently by the projects, as well as in collaboration with the private sector, Government of Uganda, community organizations, and research organizations to achieve project outcomes and sustainability. It will support its determination using both quantitative and qualitative methodologies that explore the following: (1) the quality of the processes, systems, and institutional arrangements developed and/or strengthened to sustain the necessary and critical services; (2) communities’ perceptions on the quality, frequency, effectiveness, and sustainability of the services provided by the project; (3) the likelihood that service providers will continue providing services after the project ends; (4) the motivation of the community and beneficiaries to demand and pay (or invest time) for the services; (5) whether the necessary resources and capacity strengthening will exist to sustain service providers; (6) the extent to which the projects leveraged other USG and non-USG investments to achieve sustained outcomes as identified in the theories of change; (7) evidence of enhanced linkages with other service providers. Q3: In each technical sector, what are the strengths of and challenges to the efficiency and effectiveness of the interventions’ implementation and their acceptance in the target communities? The ET will evaluate the effectiveness and relevance of the technical interventions, including food-for asset and/or cash-for-asset interventions, to achieve project outcomes, and discuss those findings in relation to the projects’ theories of change. It will support its determination using both quantitative and qualitative methods when discussing the following: (1) factors in the implementation and context associated with greater or lesser efficiency and effectiveness in producing Outputs of higher or lower quality; (2) the interventions and implementation processes deemed more/less acceptable to members of the target communities. Q4: What key lessons learned and best practices should inform future projects in the country? During the course of its research, the ET should identify best practices, strengths, and challenges in the projects’ designs (including theories of change) and risks to their effective implementation, supporting project achievements, as well as approaches that should be considered in designing future food and nutrition security projects. The ET will support its determination using both quantitative and qualitative methodologies when discussing the following: (1) the unintended positive and/or negative consequences of the projects, and (2) ways to minimize potential unintended negative consequences and systematically capture positive consequences. AUDIENCE & INTENDED USES The primary audience of the evaluation reports will be MC and ACDI (and their sub partners), and USAID (FFP/Washington, USAID/Uganda). The reports will also be shared with the Government of Uganda. Findings from the final evaluation will be used to determine the final of the two DFAPs; influence the implementation of the current development food security activities (DFSA), as appropriate; as well as inform and shape future USAID projects. USAID will make extensive use of findings from the evaluations to make different presentations and bulletins as part of a wider dissemination of best practices and lessons learned. The evaluation recommendations may be used by FFP to refine proposal guidelines and project policy. FINAL EVALUATION METHODOLOGY The final evaluation will use a mixed-methods approach. The ET will begin with a desk review of project documents, validate its understanding of the projects via consultations with USAID and implementing partners, conduct a population-based household survey using all implementation villages as the sampling frame, and conduct qualitative research in villages selected via non-probability sampling method. It is preferred that, if possible, the firm conducts quantitative and qualitative components sequentially to allow the quantitative data to inform the qualitative research. a) Desk Review The evaluation team should review the following documents to contextualize and refine the evaluation questions, as well as to gain an in-depth understanding about the project design, implementation, and the food security situation in the area. Partner annual monitoring data should be reviewed when preparing for qualitative research, considered in relation to the evaluation findings, and incorporated into the report in support of/contrast to select evaluation findings. While FFP recommends the below documents for pre-evaluation learning, the literature review should not be limited to the following:         Project proposals Pipeline Resource Estimate Proposals (PREPs) Annual results reports (ARR), including Indicator Final Tracking Tables (IPTT) for final against targets Midterm review reports and corresponding action plans developed by the two projects Baseline Study for the Title II Development Food Assistance Projects in Uganda, 2013 Uganda Demographic Health Survey 2016 Partner formative research and barrier analyses so as to better understand the context and if/how the studies influenced programming Monitoring data and reports b) Consultations As a supplement to the desk review, consultations with FFP and partner staff in Washington, DC and Uganda will allow the ET to corroborate its understanding of the design, approaches and interventions employed by each DFAP and acquired through the desk review. It is recommended that the ET engage with the staff at each organization prior to beginning fieldwork. Equally important to engaging pre-data collection is to reconnect post-data collection to “ground truth” findings with FFP/Uganda and the partner staff. In the case of major disagreements, the program staff should provide evidence in support of the argument, and pending time constraints, the ET may revisit the field. c) Quantitative Endline Survey The quantitative endline household survey for the 2012 DFAPs will be integrated with the baseline survey for the 2017 DFSAs. FFP made this decision to increase survey efficiency and reduce the data collection burden. The 2018 PBS (the term used to denote both the combined 2018 endline and baseline survey) will collect data on the same population-level impact and outcome indicators (presented below) that were collected during the 2013 baseline survey, as well as, resilience indicators2 and a few additional indicators identified by the new DFSA partners (following the 2018 baseline workshop). DFAP baseline data were collected between late February and the end of April, and data collection for the final evaluation must occur during the same time period. The 2018 PBS should use the same data collection instrument for the endline indicators, level of statistical precision (95 percent confidence intervals), and statistical power (80 percent) as the baseline study3. The 2018 PBS design does not need to be identical to the baseline; if the projects reduced their target areas, for example, the sampling frame of households used for the baseline may need to be adjusted.        Prevalence of underweight children under five years of age Prevalence of Poverty: Percent of people living on less than $1.25/day Mean Depth of Poverty Per capita expenditures (as a proxy for income) of USG-targeted beneficiaries Prevalence of stunted children under five years of age Prevalence of underweight women (of reproductive age) Percentage of farmers who used at least [a project-defined minimum number of] sustainable agriculture (crop/livestock and/or NRM) practices and/or technologies in the past 12 months 2 Resilience indicators will be calculated, and analysis conducted, for the baseline study only. 3 Baseline Study for the Title II Development Food Assistance Projects in Uganda https://www.usaid.gov/sites/default/files/documents/1866/Uganda%20Baseline%20Study%20Report%2C% 20March%202014.pdf                 Percentage of farmers who used improved storage practices in the past 12 months Percentage of farmers who used financial services (savings, agricultural credit, and/or agricultural insurance) in the past 12 months Percentage of farmers who practiced the value chain activities promoted by the project in the past 12 months Average Household Dietary Diversity Score (HDDS) Prevalence of children 6-23 months receiving a minimum acceptable diet (MAD) Women's Dietary Diversity (Score): Mean number of food groups consumed by women of reproductive age (WDDS) Prevalence of exclusive breastfeeding of children under 6 months of age Percentage of children under age five who had diarrhea in the prior two weeks Percent of children under five years old with diarrhea treated with Oral Rehydration Therapy (ORT) Percent of households using an improved drinking water source Percent of households using an improved sanitation facility Percent of households with soap and water at a handwashing station commonly used by family members 6 Percentage of births receiving at least four antenatal care (ANC) visits during pregnancy Percentage of livestock farmers accessing government or private sector veterinary services (diseases diagnosis and drugs) and livestock vaccination Percentage of respondents who know at least 3 of the 5 critical moments for hand washing 3 submodules of the Women's Empowerment in Agriculture Index (WEAI)* * The 2013 baseline study collected and reported on the WEAI, but the questions asked in the WEAI are largely not relevant to FFP target populations. WEAI is designed for communities/populations that are at slightly higher wealth quintiles, where families can report on having some assets. The majority of Karamajong are extremely poor and do not have any of these assets, and therefore, most of the questions asked in WEAI and the score itself is not a valid measure of gender empowerment for FFP target households. Note: Based on USAID’s strategic interests, a few additional questions may be incorporated into the household questionnaire (and any corresponding indicators added). All quantitative data is owned by USAID and made available to the public barring rare exceptions. d) Qualitative Research Qualitative methods will be used to collect information to answer evaluation questions and to support the interpretation of the quantitative data. The ET will design the overall qualitative study approach and should consider a variety of primary data collection methods, such as semi-structured in-depth interviews, group discussions, key informant interviews, direct observations, and case studies (the ET may choose to use the most significant change methodology to identify a selective set of case studies). These methods - to the maximum extent possible - will ensure that if a different, well-qualified evaluator were to undertake the same evaluation, he or she would arrive at the same or similar findings and conclusions. The ET should decide on specific methods before traveling to Uganda, and present those to FFP along with the number of interviews, FGDs, etc., per project, in the inception report. Following discussion and agreement, the ET will finalize the methods during the team meeting in￾country. The evaluation team leader and members will be responsible for interviewing the direct, indirect and non-beneficiaries in their households and communities, as well as look for evidence of ongoing learning and activities (such as home gardens, etc.). The ET will also be responsible for interviewing relevant stakeholders for the evaluation and analyzing the qualitative data. Should the ET decide to hire additional researchers to complement the data collection effort, they cannot replace the evaluation team members’ role of collecting primary data using qualitative methods. The ET will contribute to the interpretation of the quantitative results using qualitative findings. In addition to the factors specifically identified earlier as essential to responding to the evaluation questions, during its research during the qualitative study the ET should also consider the efficacy of the following cross-cutting interests: project management; final monitoring; strategies to improve gender equality at the participant and project management levels; environmental considerations; and conflict sensitivity. Lastly, it is expected that the evaluation will speak to lessons learned and best practices. The ET may find it useful to apply non-probability sampling methods to select a sub set of enumeration areas from the quantitative survey. In selecting interview sites, the evaluation team should strategically select large-enough-yet-manageable interview sites that generally represent the target area. As with the quantitative household survey, qualitative sampling should include both individuals who directly participated in the DFAP (beneficiaries) and those not specifically targeted with any intervention (indirect/non-beneficiaries). (The latter should be included to allow learning on spillover, triangulate the information provided by the direct beneficiaries, and to understand their perspectives on the achievements or limitations of the interventions offered by RWANU and GHG.) In addition, the qualitative team should interview USAID personnel, project staff, knowledgeable people from the community, local government staff, community leaders, host Government officials, and other agencies and individuals as appropriate. e) Data Analysis and Interpretation The evaluation team will statistically compare the endline data for each of the two strata with that of the baseline for that stratum, and also for the overall country level, in order to detect changes (if any) for all key indicators. The ET will conduct descriptive and inferential analyses to describe the results, as well as various econometric analyses to identify the determinants of key outcomes and the magnitude and direction of changes. In advance of fieldwork, the evaluation team should develop a data analysis plan and submit to the COR for approval. When analyzing the data, however, the evaluation team should not limit itself to the data analysis plan; rather, the evaluation team should keep an open and curious mind to look for correlations between variables. In presenting the analysis, the evaluation team should be cognizant about the readers’ familiarity with the statistical presentation. FFP suggests avoiding jargons, but rather describe the statistical terms in a common language. Interpreting the results is as critical as the analysis. Oftentimes, it can be difficult for a reader to fully understand the key points and utility of the findings conveyed in a report. The analysis and interpretation should be presented in a “story telling format” so that the readers can see a human face as they read the report. While it is important for the reader to understand whether level of stunting is reduced in the area, it is equally important to understand the pathway; for example, how learning derived from project participation influenced people’s practices, which in turn resulted in positive changes in food security outcomes at the household and/or community level. Similarly, it is equally important for the readers to know some of the challenges participants faced that might have prevented them from reaping the full benefits of the projects. REPORT The ET will produce two reports in English, not to exceed 50-pages, for each DFAP. The draft reports will be shared within FFP, USAID/Uganda, and the partners for review and comment over a two-week period. The final report should include a table of contents, table of figures (as appropriate), acronyms, executive summary, introduction, purpose of the evaluation, research design and methodology, limitations, findings, conclusions, lessons learned, and recommendations. All evaluation questions should be answered, and the evaluation methodology should be explained in detail. To ensure a high-quality deliverable, the reports should reflect a thoughtful, well-researched and well￾organized effort to objectively evaluate what worked in the project, what did not, and why. Where noteworthy, the discussion should highlight and discuss the outcomes and impacts on males versus females. The report must integrate the quantitative analysis from the PBS and include a statistical comparison between baseline and endline results. Learning from the qualitative research will help to contextualize and interpret the quantitative data. The ET should also draw from partners’ annual monitoring data, where appropriate, to substantiate findings. For example, if a qualitative finding is that beneficiaries report increased yields, the ET should compare/contrast that finding against data collected by the partners. Findings should be specific, concise, and supported by strong quantitative and/or qualitative evidence, and presented as analyzed facts/evidence/data, and not be based on anecdotes, hearsay or a compilation of people’s opinions. It should include analytical methods to include appropriate tests of differences; econometric analysis to evaluate the theories of change and to explore the causal relation between the outcome and activities/variables based on the theoretical models; it is expected that the contractor will interpret the analytical findings. The report should disclose limitations to the evaluation, with particular attention to the limitations associated with the evaluation methodology, e.g. selection bias, recall bias, unobservable differences between comparator groups, etc. Recommendations should be supported by a specific set of findings, and be action-oriented, practical, and specific. It is expected that the final reports will address and incorporate feedback, as appropriate, from the reviewers. Should the ET disagree with any of the comments, it should raise this with the COR immediately for discussion. EVALUATION TEAM The Evaluation Team Leaders will be responsible for designing and managing the evaluations and overseeing the work of the evaluation team members; coordinating with the implementing partners, USAID Mission and other stakeholders; coordinating with the endline PBS team; analyzing the findings and ensuring the quality of the report. As this is a mixed-method final evaluation, in addition to the evaluation team, the endline survey will require extensive participation of the following personnel: Senior Survey Method Specialist, Data Analyst, Survey Coordinator, Data Management Specialist, Anthropometry Specialist, Country Operations Manager, and Survey Monitors. The PBS data collection team should be hired locally, if possible. The evaluation team will require in-country participation from the Evaluation Team Leader and up to three subject matter specialists. As FFP projects are multi￾sectoral, the evaluation team must possess expertise and field experience with food security and rural integrated programming and demonstrate an in-depth knowledge of the following technical sectors and cross-cutting areas: agriculture and off farm livelihoods, nutrition; water, sanitation, and hygiene (WASH); gender, youth, resilience, and disaster risk management. The subject matter specialists must also possess experience and knowledge about the specific processes used by the projects (e.g., Care Groups, Farmer Field Schools, etc.) FIELD LOGISTICS The ET is responsible to arrange and pay for all logistics including anthropometric equipment, and transportation. It is strongly recommended that the ET consults the Mission on identifying a reputable car-rental firm. The Mission and partners also should be consulted on identifying interpretation services. The ET should request assistance from the partners on making introductions, as necessary, to local ministry representatives and community leaders. DELIVERABLES The ET shall produce the following deliverables during the evaluation and submit to the Contracting Officer’s Representative (COR) for review and approval. All draft documents should be submitted in Microsoft Word or Microsoft Excel, or in the rare occasion both PDF and Word/Excel. The COR must approve all deliverables. Deliverable Timeline Work Plan includes a brief synthesis and timeline for the Uganda final evaluation, with the timeline including major activities throughout the study, including dates by which field guides and training materials will be completed. Only one work plan detailing both baseline study and final evaluation activities is required TBD PBS Enumerator Guide, Supervisor Manual, and Anthropometry Guide* provide revised detailed instructions on supervisor, enumerator and anthropometry trainings. Note that the PBS should use the supervisor, enumerator and anthropometry training guides developed for the baseline. Minor adjustments will be needed to accommodate the new indicators. Only one set of guides that serves both the baseline and endline surveys is required TBD PBS Data Treatment and Analysis Plan • details how the data will be cleaned, weighted, and analyzed and must include: programming specifications and editing rules for cleaning data, data dictionary codebook, SPSS syntax and output for all analyses and variable transformations into indicators; and • includes a descriptive, inferential, and econometric analyses plan. TBD Only one DTAP that serves both the baseline study and final evaluation is required, but it must clearly differentiate between the different analytical approaches used for each. PE Inception Report (~20 pages) • briefly synthesizes the literature review; • describes the qualitative evaluation methodology (including evaluation questions contextualized based on the literature review, sample site selection strategy and number of sites to be selected, number of interviews/discussions per project, types of interviewees) • introduces the evaluation team members and their roles; and • details how the qualitative information will be analyzed and integrated with quantitative. TBD Final Evaluation Protocol (a combined protocol that discusses both baseline/endline surveys and qualitative component) (~30 pages) • describes the evaluation questions to be answered; • present specific data collection methods by evaluation question; • identifies indicators to be collected; • introduces the local partner (data collection firm); • discusses the quantitative and qualitative analysis methods and plan; • presents PBS sample size, design and plan, survey design, questionnaire design, site selection plan for qualitative research; and • presents the fieldwork plan (including trainings and field support/supervision, data management, quality control, recording, analysis and reporting). TBD Pertinent Permissions and approvals demonstrate official approval from all relevant institutional review boards and from host country institutions to collect data, conduct the evaluation, and release data and reports, as required, as well as a statement affirming adherence to all requirements specified in USAID’s Scientific Research Policy. TBD PBS Quantitative Survey and Qualitative Instruments • include both English and Ngakarimojong versions of the household survey (note: the instrument must be back-translated to English via a second translator to ensure accurate translation. Following the pilot of the survey, any modifications based on field experience will again require translation and back translation to ensure accuracy). TBD [preliminary estimates submitted • describe site selection methodology and factors used to select sample communities; • list sample communities; • discusses groups to be interviewed; and • explain criteria used to select respondents. in advance to support qualitative research] Draft Evaluation Reports (one for RWANU and one for GHG) • will be no more than 50 pages per DFAP (excluding annexes); • include as an annex tables for FFP indicator estimates, including comparisons between baseline/endline values October 2018 In-country briefings to USAID/Uganda and FFP stakeholders in Uganda Two 60-minute presentations of the major findings of the baseline study to provide an opportunity for immediate stakeholder feedback that can be considered for the revision (as appropriate and without compromising the validity or independence of the evaluation): o One presentation to USAID/Uganda; o One presentation to FFP stakeholders in Uganda, including the DFSA partners, donors, and Government of Uganda (invited by USAID/Uganda and the partners), via a 60-minute PowerPoint presentation November 2018 Briefing to FFP/Washington comprises a 60-minute PowerPoint presentation on the evaluation, major findings, lessons learned, and recommendations November 2018 Final Evaluation Reports • include items identified in the draft report as well as a three- to five-page executive summary of the purpose, background of the project, methods, findings, conclusions and recommendations, and the following annexes: the scope of work, tools used in conducting the evaluation (questionnaires, checklists, and discussion guides), and any substantially dissenting views by any Team member, USAID or the PVOs on any of the findings or recommendations; and • must be 508 compliant and uploaded to the Development Clearinghouse following COR approval. December 2018 Data (to be submitted at the time of the final report*) • include a separate electronic file of all quantitative data in an easily readable format that is organized and fully documented so as to facilitate use by those not fully familiar with the project or the evaluation; • provides cleaned data, sampling weights at each stage, final sampling weights, and all derived indicators; • includes a second final data set in CSV format that has been anonymized to protect individual confidentiality for use as a public data file in the USAID Open Data; and • include a separate electronic file of all transcribed (as applicable) qualitative data in machine-readable format; and • include a separate file detailing GPS coordinates of households that participated in the PBS. *FFP may request data sets earlier for internal use only December 2018 ANNEX 2: UGANDA JOINT BASELINE/ENDLINE POPULATION￾BASED SURVEY PROTOCOL June 28, 2018 This publication was produced for review by the U.S. Agency for International Development. It was prepared by ICF Macro, Inc. Uganda Joint Baseline/Endline Population-Based Survey Protocol - FINAL Office of Food for Peace (FFP) Contract #: GS-00F-189CA/7200AA18M00002 Uganda Joint Baseline/Endline Population-Based Survey Protocol – USAID Office of Food for Peace i Abbreviations BL Baseline CAPI Computer-assisted personal interview CRS Catholic Relief Services DFAP Development food assistance program DHS Demographic and Health Survey DFSA Development food security activity EL Endline FANTA Food and Nutrition Technical Assistance Project III FFP Office of Food for Peace FIES Food insecurity experience scale GHT Gendered household type HDDS Household dietary diversity score IFSS Internet file streaming system IP Implementing partner IPA Innovations for Poverty Action MCHN Maternal and child health and nutrition MC Mercy Corps PBS Population-based survey PPI Poverty probability index PPS Probability proportional to size USAID U.S. Agency for International Development WASH Water, sanitation, and hygiene Uganda Joint Baseline/Endline Population-Based Survey Protocol – USAID Office of Food for Peace ii Table of Contents 1. BACKGROUND AND PURPOSE ...................................................................................................................... 1 2. DESIGN 2 2.1 Indicators to be Measured...........................................................................................................................2 2.2 Sampling Plan...................................................................................................................................................2 2.2.1 Sampling Frame....................................................................................................................................... 2 2.2.2 Sample Size .............................................................................................................................................. 3 2.2.3 Sample Selection..................................................................................................................................... 5 2.3 Questionnaire................................................................................................................................................... 7 3. FIELD PROCEDURES .......................................................................................................................................... 7 3.1 Data Collection Mode................................................................................................................................... 7 3.2 Field Manuals.................................................................................................................................................... 7 3.3 Training...............................................................................................................................................................8 3.4 Data Collection..............................................................................................................................................10 3.5 Quality Control..............................................................................................................................................10 4. DATA PROCESSING AND ANALYSIS ............................................................................................................11 4.1 Data Transmissions.......................................................................................................................................11 4.2 Data Analysis ...................................................................................................................................................11 4.3 Dissemination of Findings .........................................................................................................................12 5. ETHICAL CONSIDERATIONS .........................................................................................................................12 5.1 Ethics approval...............................................................................................................................................12 5.2 Verbal Informed Consent..........................................................................................................................12 6. TIMELINE ............................................................................................................................................................13 Annex 1 – Joint BL/EL PBS Indicators Annex 2 – Training Agendas Uganda Joint Baseline/Endline Population-Based Survey Protocol – USAID Office of Food for Peace 1 1. BACKGROUND AND PURPOSE USAID’s Office of Food for Peace (FFP) is the U.S. Government leader in international food assistance. In 2018, FFP awarded ICF International a contract to conduct a joint baseline (BL)/endline (EL) population-based household survey (PBS) in the Karamoja region of Uganda. The BL survey is for two newly funded development food security activities (DFSAs), beginning in fiscal year (FY) 2017 which will be implemented by Catholic Relief Services (CRS) and Mercy Corps (MC) in seven districts of Karamoja. The EL survey is for two development food assistance programs (DFAPs) that were implemented by ACDI VOCA and MC in the same seven districts of Karamoja. These programs started in FY 2012 and ended in FY 2017. The implementation areas for the new DFSAs and prior DFAPs overlap to a large extent. For this reason, ICF will administer a joint BL/EL PBS using a common questionnaire in the overlap and non￾overlap areas encompassed by the prior DFAPs and current DFSAs. The common questionnaire will be driven by the indicators required for the baseline PBS for the new DFSAs, many of which (but not all) overlap with those required for the DFAPs. The purpose of the BL PBS for the DFSAs is to assess the current status of key indicators, to serve as a point of comparison with indicators collected at EL and to have a better understanding of the prevailing conditions and perceptions of the populations in the DFSA implementation areas. The study results will also be used to further refine program targeting and, where possible, to understand the relationship between variables to inform program design. The results of the EL PBS for the prior DFAPs will be used to evaluate change over time in the indicators that were collected at BL as part of a performance evaluation of these DFAPs. The fieldwork for the joint BL/EL PBS will be conducted in June-July 2018. ICF has subcontracted the International Research Consortium of Uganda (IRC), a local data collection firm, to support the field implementation of the joint BL/EL PBS. ICF will work closely with the survey subcontractor in the implementation of the PBS. All deliverables will be approved by ICF and the USAID COR. Implementing partners and their roles From 2012-2017, two implementing partners (IPs) conducted the prior DFAPs in the Karamoja region: (1) ACDI/VOCA and its partners implemented the Resiliency through Wealth, Agriculture and Nutrition in Karamoja (RWANU) Program in Amudat, Moroto, Napak and Nakapiripirit districts. (2) MC and its partners implemented the Growth Health and Governance Program (GHG) Program in Abim, Kotido and Kaabong districts. In 2017, FFP awarded two new DFSAs in the Karamoja region: (1) MC and its partners will implement the Apolou DFSA in Kaabong, Kotido, Moroto, and Amudat districts. (2) CRS and its partners will implement the Nuyok DFSA in Abim, Nakapiripirit and Napak districts. Uganda Joint Baseline/Endline Population-Based Survey Protocol – USAID Office of Food for Peace 2 2. DESIGN The BL component of the joint BL/EL PBS serves as the first phase of a pre-post survey cycle for the new DFSA awards, and the EL component of the joint BL/EL PBS serves as the second phase of a pre￾post survey cycle for the DFAP awards. The BL survey for the DFAPs was conducted from late February to end of April of 2013. The pre-post design (using the 2013 BL PBS and the 2018 EL component of the joint BL/EL PBS) allows for the determination of statistically significant change in indicators; however, it does not allow statements to be made about attribution or causation relating to program impact. 2.1 Indicators to be Measured The PBS will collect data to measure key FFP impact and outcome indicators, resilience indicators and custom indicators developed by the IPs. The FFP indicators are related to food security: poverty; water, sanitation, and hygiene practices; agricultural practices; women’s and children’s health and nutritional status (including anthropometry); and gender. A definition and full description of each of the key FFP indicators is available in the FFP Indicator Handbook (April, 2015). 1 FFP resilience indicators measure household well-being, exposure to shocks, resilience capacities (absorptive, adaptive, and transformative), and responses. Definitions for resilience and custom indicators are provided in the Uganda PBS Data Treatment and Analysis Plan (DTAP). The full list of FFP, resilience and custom indicators is provided in Annex 1. 2.2 Sampling Plan 2.2.1 Sampling Frame The target population for the joint BL/EL PBS consists of two components: 1) all households in the areas where the prior DFAPs were implemented and 2) all households in the areas where the new DFSAs will be implemented. These target populations overlap since the new DFSAs will be implemented in most of the same areas where the prior DFAPs were implemented. The sampling frames for the BL and EL PBSs were constructed taking into account these overlapping geographies. ICF used the list of target areas provided by the IPs and the most recent census data for constructing the sampling frame. The most recent Ugandan Census was conducted in 2014 by the Uganda Bureau of Statistics. The census administrative geographic levels are as follows:       District County Sub-County Parish Village Enumeration Area (EA) Table 2 provides 2014 census estimates of the number of EAs and households included in the BL and EL sampling frame for each program. 1 Food and Nutrition Technical Assistance III Project (FANTA III). 2015. FFP Indicators Handbook Part I: Indicators for Baseline and Final Evaluation Surveys. April 2015. Washington, DC. A newer version of the FFP Indicators Handbook is pending release in 2018. Uganda Joint Baseline/Endline Population-Based Survey Protocol – USAID Office of Food for Peace 3 Table 2. EAs and Households Included in the Baseline and Endline Sampling Frames Number of EAs Number of Households Rwanu Endline (ACDI/VOCA) 652 49,540 GHG Endline (MC) 740 48,903 Apolou Baseline (MC) 974 68,263 Nuyok Baseline (CRS) 831 54,741 Note: The 2012 Rwanu DFAP included the southern districts of Amudat, Moroto, Nakapiripirit and Napak, while the 2012 GHG DFAP included the northern districts of Abim, Kaabong and Kotido. The districts were divided differently for the new DFSAs. The Nuyok DFSA included the districts of Abim, Nakapiripirit and Napak, while the Apolou DFSA included the districts of Amudat, Kaabong, Kotido and Moroto. 2.2.2 Sample Size The sample size for the joint BL/EL PBS was derived by: 1) calculating the sample size needed for the BL survey for the DFSAs (assuming a statistical test of differences will be implemented at EL five years hence), 2) calculating the sample size needed for the EL survey for the prior DFAPs (assuming a statistical test of differences will be implemented at EL relative to the BL five years earlier), and 3) deriving a joint sample size based on these sample sizes taking into account the overlap between the DFSA and prior DFAP implementation areas. 1) BL Sample Size for the DFSAs The sample size calculation for the BL project areas for the 2017 DFSAs is based on adequately powering a statistical test of differences in the prevalence of stunting because stunting is a key measure of food insecurity. The following criteria were used for deriving the sample size:       Design effect of 2.0 Confidence level of 95 percent Power level of 80 percent Expected reduction in stunting over the life of the project of 8.0 percentage points Use of the Stukel/Deitchler inflation and deflation factors (see Addendum to the Feed the Future Population-Based Survey Sampling Guide2) to determine the number of households needed for the required sample of children under five years of age 5 percent inflation of the household sample size to adjust for estimated household nonresponse The formula used for deriving the sample size is based on a statistical test of the difference of proportions (or prevalence) for an indicator (e.g., from BL to EL), as described in Appendix A of the Feed the Future Population-Based Survey Sampling Guide. Table 3 provides the target sample size derived using estimates from the 2011 Uganda Demographic and Health Survey (DHS) for the two input parameters to the sample size calculation: 1) prevalence of stunting in rural households, and 2) number of children per household. The sample size calculation yields a total of 82 clusters (with 30 households per cluster) and 2,460 households (41 clusters and 1,230 households for each of the two DFSAs). 2 Stukel, Diana, Feed the Future Population-Based Survey Sampling Guide (2018), Washington, DC, FHI 360/ FANTA. Available at: https://www.fantaproject.org/monitoring-and-evaluation/sampling Uganda Joint Baseline/Endline Population-Based Survey Protocol – USAID Office of Food for Peace 4 Table 3. BL PBS Sample Size for each DFSA and Overall Indicator Prevalence of Stunting (P1)* Number of Children per Household* Number of Children Needed Household Sample Size Needed** Households Needed with 5 Percent Nonresponse Adjustment Clusters per DFSA Overall Sample Size of Households Needed Prevalence of stunting 0.36 0.98 829 1,157 1,230 41 2,460 *Source: 2011 DHS estimates for rural households (where estimated household size is given as 5.05 and estimated proportion of population under five years of age is given as 0.192) **Includes Stukel/Deitchler inflation and deflation factor adjustments Assumptions for all calculations: one-sided test, alpha=0.05, beta=0.80, households per cluster=30 2) EL Sample Size for the DFAPs The sample size for the EL survey was calculated based on the number of children needed to detect a 6 percent reduction in stunting3 over the life of the program cycle (between BL and EL) in the prior DFAP implementation areas. The BL sample size for the prior DFAPs was 2,400 households in each of the two program areas4 and valid height and weight measurements were obtained for a total of 5,335 children under five years of age across both programs. The prevalence of stunting at baseline was 34.5 percent and the design effect for the prevalence of stunting was 4.0.5 Using these parameters, the number of children needed at endline is 1,975 across both programs. Inflating to the household level yields a sample size of 2,440 households across both programs (assuming an average household size of 6.3 persons, 19.4 percent of the population is comprised of children under five years of age6 and a 5 percent nonresponse rate). 3) Sample Size for the Joint BL/EL PBS The joint BL/EL sample size was derived by first identifying the number of households in each of 3 areas: 1) the overlapping area between the prior DFAPs and the new DFSAs, 2) those in the prior DFAP area only (old), and 3) those in the DFSA area only (new). Then, the sample size was proportionately allocated among the districts in these three groups relevant to the particular survey in question (BL versus EL), and based on the proportion of households in each group. To elaborate, this allocation was done separately for the prior DFAPs yielding an EL sample size of 1,220 by allocating to “old” and “overlap” EAs, and for the new DFSAs yielding a BL sample size of 1,230 by allocating to “new” and “overlap” EAs. As a result, there were two sample sizes for the “overlap” EAs that had to be simultaneously satisfied – one in relation to the prior DFAPs and one in relation to the new DFSAs. The overall joint BL/EL sample size requirement was then calculated based on: 1) the sample size requirement for the prior DFAPs only; 2) the sample size requirement for the new DFSAs only; and 3) 3 A 6 percentage point reduction in stunting was used here (rather than the 8 percentage point reduction referred to in the paragraph on BL Sample Size for DFSAs) to be consistent with the percentage point drop posited at BL for the DFAPs. 4 A detailed description of the sampling for the baseline study for the prior DFAPs can be found in Appendix 2 of the “Baseline Study of Title II Development Food Assistance Programs in Uganda” Report, March 2014. Available at: https://www.usaid.gov/opengov/developer/datasets/UgandaBaselineReport-March2014.pdf 5 These parameters were obtained from the 2013 FFP baseline survey in Uganda. 6 Ibid. Uganda Joint Baseline/Endline Population-Based Survey Protocol – USAID Office of Food for Peace 5 the maximum of the sample size requirement for the overlapping prior DFAP and DFSA implementation areas (see Table 4). This resulted in a total of 56 clusters and 1,680 households per BL DFSA. Table 4. Sampled Households and Clusters by BL DFSA Implementation Area Project Sample Size Requirement (Prior DFAP/EL) Sample Size Requirement (New DFSA/BL) Joint BL/EL Sample Size Requirement Number of Clusters CRS DFSA 1,220 1,230 1,680 56 Overlap 939 916 990 33 Old 281 0 330 11 New 0 314 360 12 MC DFSA 1,220 1,230 1,680 56 Overlap 1,220 820 1,230 41 Old 0 0 0 0 New 0 410 450 15 TOTAL 2,440 2,460 3,360 112 2.2.3 Sample Selection The sample for each project was selected using stratified multi-stage cluster sampling with three stages of sampling: (1) selection of clusters (EAs), (2) selection of households, and (3) selection of individuals. First stage sampling of clusters Within each project, stratification is done at the district level and the number of EAs to be sampled are proportionately allocated at the district level based on the distribution of households across all districts. EAs are then selected from the sampling frame for each project using probability proportional to size sampling (PPS). The total number of EAs sampled for each project for the joint BL/EL PBS is based on the joint sample size requirement for each project as a proportion of the overall joint sample size requirement (Table 5).7 Table 5. EAs and Households Included in the BL and EL Samples Number of EAs Number of Households Rwanu Endline (ACDI/VOCA) 43 1,290 GHG Endline (MC) 42 1,260 Apolou Baseline (MC) 56 1,680 Nuyok Baseline (CRS) 45 1,350 7 The first stage of sampling included two phases due to a recalculation of the sample size after the listing exercise was completed. The original calculation of the sample size did not take into account the number of children with valid height and weight measurements obtained at baseline and was therefore substantially larger than the sample size after adjustment for these children. The first phase of sampling included selection of 95 EAs per project for a total sample size of 2,850 households – this was based on the original sample size calculations. The listing exercise took place in these 95 EAs for each project. The adjusted sample size required 56 EAs per project and these 56 EAs were allocated as described in step 3 above and randomly selected from the previously selected EAs. Uganda Joint Baseline/Endline Population-Based Survey Protocol – USAID Office of Food for Peace 6 Second stage sampling of households: At the second stage of sampling, 30 households are randomly selected per cluster using systematic sampling. Before the selection of households can take place, a listing exercise is conducted to identify and count each household in the cluster. GPS coordinates are taken for each cluster and the name of the head of household is recorded for each household. For the purposes of the household survey a household is defined as follows: A person or group of people who live together and share meals (“eating from the same pot”). This is not the same as a family. A family includes people who are related, but a household includes any people who live together, whether or not they are related. For example, three unrelated men who live and cook meals together would not be considered one family, but they would be considered one household. For men with more than one wife (polygamous situations), households will be treated in accordance with the below definition: If the wives live in the same homestead (dwelling structures and adjoining land occupied by family members) and also share the same eating arrangements, they will be treated as the same household. But if the wives live independently and do not share the same eating arrangements they will be treated as separate households. Third stage selection of individuals within sampled households: The household roster will be completed at the beginning of the interview, thus identifying all members of the selected household. The selection of individuals within households will be dependent on which questionnaire module (See Section 2.3 below) the individuals are eligible for. The protocol for the selection of individuals within households (and their potential proxy respondents) will be as follows:      For the modules requiring data about the household (C, CC, F, H), no individuals are sampled since the household is the sampling unit. The head of household or any responsible adult will be interviewed on behalf of the household. For the children’s module (D), data and anthropometry measures will be collected for all eligible children. The mother or caregiver of the selected children under five years of age will be interviewed as a proxy respondent. For the woman’s module (E), all woman between the ages of 15-49 will be selected. No proxy respondents are allowed. For women’s anthropometry, only non-pregnant women will be measured. For the agricultural module (G), all farmers within the household who have ownership or decision-making power over all plots of land and/or livestock that are part of the “farm” will be interviewed. If a farmer has migrated for an extended period to work outside of the household, the spouse and/or another responsible adult farmer that can answer the agricultural questions can be interviewed as a proxy respondent. For the gender modules (J and K), all cash earners that are married or in a union and all parents of children under two years of age that are married or in a union will be interviewed. No proxy respondents are allowed. Uganda Joint Baseline/Endline Population-Based Survey Protocol – USAID Office of Food for Peace 7 2.3 Questionnaire The questionnaires for the PBS were developed based on the core FFP and resilience indicators. All questionnaire modules follow FFP and Feed the Future guidelines, as described in the FFP Indicators Handbook (April 2015) and the Feed the Future Indicator Handbook (September 2016).8 The ICF team revised the questionnaire from the 2013 BL survey in Uganda, and gathered information before, during, and after the BL planning workshop (January 2018) to tailor the questionnaires to the context of Uganda. The resilience module was contextualized by Tango International based on prior work in Uganda. The questionnaire consists of separate modules covering the following topics:              Module A: Household identification and informed consent Module B: Household roster Module C: Household food security Module CC: Mobility, Local Government Responsiveness and Poverty Probability Index (PPI) Module D: Children’s nutrition and health Module E: Women’s nutrition and health Module F: Water, sanitation, and hygiene Module G: Agriculture Module H: Poverty Module J: Gender – Cash Module K: Gender – Maternal and Child Health and Nutrition (MCHN) Module L. Gender – Household Decision-Making, Access To Credit And Group Participation Module R: Resilience The questionnaire will be translated into three local languages (Karamojong, Pokot, and Lethur). The total time for completing the survey in each household is expected to be approximately two to three hours, depending on the size of the household. 3. FIELD PROCEDURES 3.1 Data Collection Mode The data for the joint BL/EL PBS will be collected with tablets using Computer-Assisted Personal Interviewing (CAPI). Tablets will be loaded with a CSPro data entry application developed at ICF for FFP surveys and tailored to fit the Uganda questionnaire. All data will be entered directly into the tablets and edited while interviewing in the field. 3.2 Field Manuals Prior to the start of training and fieldwork, the ICF team will develop training manuals based on those developed for prior BL surveys and using FFP, Feed the Future, and DHS guidelines. The manuals will be used for household survey training and fielding purposes and will provide guidance to field staff on the survey protocol and procedures. The ICF team will customize the field manuals for Uganda to align with 8 Available at https://feedthefuture.gov/sites/default/files/resource/files/Feed_the_Future_Indicator_Handbook_Sept2016.pdf Uganda Joint Baseline/Endline Population-Based Survey Protocol – USAID Office of Food for Peace 8 the final questionnaire and Uganda-specific field protocols. The supervisors’ manual will describe the study design and objectives, supervisors’ roles and responsibilities, rules and regulations, ethics, fieldwork preparations, and quality control requirements and procedures. The interviewers’ manual will include guidelines for implementation of the survey and fieldwork procedures, including interviewing techniques and procedures for completing the questionnaires. This latter manual will also include detailed explanations and instructions for each question. The anthropometry training manual will include detailed instructions for all anthropometry specialists on proper procedures for taking accurate anthropometry measures (height/length and weight) along with procedures to conduct anthropometry standardization testing. 3.3 Training Using the manuals described above, the ICF team will work together with IRC, the local data collection sub-contractor, to conduct in-depth trainings for supervisors, interviewers, and anthropometry specialists. Prior to the start of training, the field team (ICF survey coordinator, local survey monitors, IRC’s country operations manager, and the lead anthropometry specialist) will develop a detailed training curriculum and timeline for supervisors and interviewers’ trainings, and the anthropometry training and standardization testing, including local sites where the anthropometry standardization testing activities will take place. These training curriculums are provided in Annex 2. The organization and flow of the training will be adapted to fit the situation and logistics in Uganda.9 The training curriculum and timeline and all training manuals will be submitted to FFP for approval prior to the start of trainings. Interviewer training will involve review of the questionnaire, module by module, along with practical sessions on handling and entering data into the tablets using the CAPI template and transferring data from interviewers’ tablets to supervisors’ tablets. Interviewers will participate in role playing and mock interviews and the questionnaires will be further checked for content, consistency and flow, as well as validity and reliability. Revisions to the questionnaire will be made at the end of the training as needed. Supervisor training will cover the topics of supervisors’ roles and responsibilities; rules, behaviors, and ethics; household and respondent selection; use of the field control sheet, maps, and GPS; and data collection. It will include a detailed review of the CAPI survey procedures for receiving and transmitting completed interviews. The anthropometry training will include instruction on taking accurate measurements, types of possible measurement errors, and reading and recording measurements followed by some practical sessions. Anthropometry training will also include a training session for all interviewers as anthropometry assistants, which require them to hold children two to five years of age to ensure that their feet and knees are in the correct position for standing measurement, and to hold children younger than two years of age to ensure that their heads are correctly positioned for recumbent length measurement. Anthropometry standardization testing will be conducted for all anthropometrists after the anthropometry training is completed. This involves objectively testing anthropometrists’ accuracy (ability to obtain anthropometric weights and measurements as close to the true weight and measurement as 9 Note that the questionnaire was not pretested (to identify problematic questions or contextual changes) because the pretesting was undertaken when the questionnaire was developed at baseline. Uganda Joint Baseline/Endline Population-Based Survey Protocol – USAID Office of Food for Peace 9 possible with minimal variation in comparison to the trainer’s weights and measurements), and precision (the ability for an anthropometrist to repeat his/her own weights and measurements with minimal variation). Upon completion of the trainings, all survey staff will participate in a pilot study in pre-selected non￾sampled villages near the project areas. The pilot test will provide the survey team practice on:          Locating of selected villages and selected households by supervisors GPS data collection at the household level CAPI data entry and respondent selection routines by interviewers CAPI data editing, survey management by supervisor CAPI data transmission to control room by interviewers Appropriate interviewing behavior Team dynamics Distribution of work assignments and coordination by supervisors Completion of field control sheets by supervisors Each interviewer will complete at least two full-interviews on paper and with the tablet during the pilot test. Supervisors will observe the interviewers in their teams during the pilot test and take notes on their performance. IRC’s survey management team, the ICF survey coordinator and the local survey monitors, the ICF anthropometry trainer will also participate in the pilot test. Together with the supervisors, they will debrief the team members the day after the pilot test is completed. They will provide feedback and clarify/troubleshoot any issues encountered during the pilot study. Based on the discussion at the debrief session, ICF will make final modifications to field procedures and manuals, if required. Table 6 summarizes the sequence of field preparation activities. The ICF survey coordinator will oversee all activities. Table 6: Field Preparation Activities Duration Activities Participants 15 days Listing exercise Listers 15 days Interviewer training Interviewers, supervisors 5 days Anthropometry Training Anthropometry specialists, interviewers, supervisors 5 days Anthropometry standardization testing Anthropometry specialists 3 days Supervisor/field procedure training Supervisors, field coordinators 4 days Pilot test and debrief IRC survey management team, ICF survey coordinator, local survey monitor, supervisors, interviewers, and anthropometry specialists Prior to the start of data collection, the field team will ensure that all required permissions and ethical review approvals have been obtained. They will develop a detailed field movement plan that will describe the location and timing for each field team throughout the data collection period. The field movement plan will be submitted to FFP for approval prior to the start of data collection activities. Uganda Joint Baseline/Endline Population-Based Survey Protocol – USAID Office of Food for Peace 10 3.4 Data Collection Data collection will start immediately after the pilot study. To collect data from the sampled 3,360 households for the joint BL/EL, there will be 18 teams, each consisting of six field team members (one supervisor, four interviewers, and an anthropometry specialist). Accordingly, IRC will hire a total of 72 interviewers, 18 anthropometry specialists, and 18 supervisors. In addition, IRC will engage five field coordinators and two IT Specialists, making the total number of field personnel for the survey to be 115. Given 112 clusters to cover and 18 teams to undertake the work, each team will, on average, collect data from 7 clusters. Estimating each interviewer can complete two household interviews per day, a total of roughly 35 days will be required to complete the data collection from the 3,360 households, including time for travel. 3.5 Quality Control Working in close partnership with IRC, the ICF team will ensure high-quality PBS data through a strong focus on training field staff and monitoring data collection. The ICF team will be using CAPI data collection, which allows for real-time editing of data, frequent uploading of collected data, continuous data quality review, and correction of field staff behavior as data collection proceeds. ICF requires that the field teams upload collected data from completed clusters at minimum on a weekly basis. During critical periods, including training, anthropometry standardization testing, piloting, and at the beginning of fieldwork, the ICF survey coordinator will be in-country to coordinate and oversee these activities. When the ICF survey coordinator leaves the country, the local survey monitors will oversee fieldwork activities and closely update the ICF survey coordinator on fieldwork progress or any issues encountered during data collection. Table 7 provides survey procedures and safeguards for field supervision. Table 7: Procedures and Safeguards for Fieldwork Oversight Goal Procedure or Safeguard Proper fieldwork oversight  Maximum ratio of one supervisor for every four interviewers and one anthropometry specialist. The subcontractor (IRC) will provide one field coordinator to oversee every four-or-five survey teams. Proper selection of households and respondent  Adherence to household and respondent selection methods per ICF protocol Assurance of questionnaire accuracy  Complete review of data immediately after the interview is conducted  In the event of errors or omissions, required corrections will be made before the interviewer proceeds to the next household Prevention of fraud in interviewing  Spot-checks with households on the day of the interview to ensure honesty on the part of the interviewer. Proper spot-checks involve verifying demographic information of the household respondents and other information to make sure that interviewers are recording data that is accurate and truthful. Fifteen percent of the completed interviews should be randomly selected for spot-checks.  In the event of fabrication or falsification of data collected, the interviewer will be fired from the project immediately Completion of interviews  If the entire interview is not completed on the first visit, interviewers will make up to a total of three visits to the household to complete the Uganda Joint Baseline/Endline Population-Based Survey Protocol – USAID Office of Food for Peace 11 Goal Procedure or Safeguard interview. The interviewer will plan one or two follow-up visits with the respondents to successfully complete the interview.  The supervisor will ensure that each household survey is appropriately completed. All interview items should be 100 percent complete. 4. DATA PROCESSING AND ANALYSIS 4.1 Data Transmissions For transmission of data from the field, IRC will use Internet File Streaming System (IFSS), a cloud-based electronic file delivery web service. The primary objective of the service is to deliver files from one user to another in a way that is fast and secure. The ICF CSPro programmer assigned to the project will work in-country to set up and test the cloud-based data transmission system, and to provide technical support during the first week of data collection to ensure that tablets and the IFSS transmission system are operating smoothly. IRC will upload data to the IFSS on a weekly basis. Data transmissions will be on a weekly basis and will begin after interviewers have completed all interviews in their first assigned cluster. For the final dataset, the CSPro programmer will develop a program to run quality control checks and convert the raw data exported from the CSPro application into the data format needed for analysis using Stata, SPSS or SAS. 4.2 Data Analysis ICF will generate estimates for all FFP and project-specific indicators, along with additional analyses to explore relationships and plausible determinants for key outcome indicators and a select number of resilience indicators. For indicators that were collected at BL in the old DFSA project areas, a statistical comparison of BL (2013) and EL (2018) estimates will be conducted to determine population-level change over time. All descriptive, bivariate, and multivariate analyses to be conducted will be discussed with FFP and clearly defined in the Data Treatment and Analysis Plan (DTAP) while recognizing that after the analysis begins, there may be other interesting analyses to pursue. Tango International will generate the resilience indicators and conduct the detailed resilience analyses. Innovations for Poverty Action (IPA) will lead the analysis for the PPI. The DTAP will be prepared following completion of the PBS data collection protocol and will be submitted to FFP for approval prior to the start of data analysis. Final data files and documentation will be delivered to FFP following the completion of the data analysis and vetting of the PBS results with all stakeholders. All personal identifying information will be removed from the datasets prior to delivery to FFP in order to protect the confidentiality of survey respondents. The final data files will include:      Sampling frames for each DFSA Raw datasets generated from the CSPro data entry application Edit rules and programming specifications for data cleaning Data dictionary/code book for each final dataset Syntax for all analyses and variable transformations Uganda Joint Baseline/Endline Population-Based Survey Protocol – USAID Office of Food for Peace 12  Final analytic datasets, including sampling weights and all derived indicators, in STATA format and comparable datasets in CSV format that have been anonymized to protect individual confidentiality, for use as a public data files in the USAID Open Data warehouse. 4.3 Dissemination of Findings All of the data is collected at the regional or sub-regional level, and analysis and dissemination of findings will focus on populations rather than on individuals. ICF will prepare three reports—one Baseline Study Report (for newly awarded DFSA programs) and two Endline Evaluation Reports for the two expired DFSA programs. The reports will be reviewed and finalized after USAID approval. Upon approval, the reports and the datasets will be uploaded to the USAID website and made available to the public. Additionally, USAID/FFP will host one to two day in-country briefing sessions with USAID/Uganda, relevant Uganda government agencies and the IP organizations to present the findings. A data utilization workshop will also be conducted with the BL DFSA IPs to further familiarize them with the baseline study results and how these results can be used to support their program planning and target setting. 5. ETHICAL CONSIDERATIONS 5.1 Ethics approval Ethical approval will be obtained from the ICF Institutional Review Board and the Mildmay Uganda Ethics Review Committee and thereafter the protocol will be registered with the Uganda National Council for Science and Technology. Permission to access the study communities will be obtained after obtaining approval from the offices of the Chief Administrative Officers for the targeted districts as well as the USAID Food for Peace Office in Uganda. 5.2 Verbal Informed Consent Verbal informed consent will be obtained before each interview after explaining to each respondent the objectives and purposes of the study and other information safeguards. Guidelines for requesting verbal consent are included in Module A of the household survey instrument. The interviewer will read to each eligible respondent a statement of informed consent that clearly outlines the subject’s rights. Participation in the interviews is completely voluntary. If there is any question that the respondent does not want to answer, the respondent can choose to skip to the next question. Additionally, respondents can choose to terminate the interviews at any time during the interview. The interviewer is required to certify that (a) each eligible respondent has given his/her informed consent before being personally interviewed, (b) the adult caregiver in the household has given his/her consent before any child under 5 years of age is measured, and (c) adult guardians of eligible respondents of minors under 18 years of age have given consent before they are interviewed. The adult guardians normally will be the parents or other close family member living in the household (e.g. grandparents or aunts/uncles). All children eight years of age and above will be required to assent to participate in the study and this will be done after obtaining the parent’s/guardian’s consent. The child’s assent or dissent will take precedence over the parent’s or guardian’s consent. Risks and benefits: The study presents no direct risks or benefits to the participants as they will not undergo any invasive procedures and they will not be directly compensated for their participation. The study carries a relatively low burden. There are different respondents for each module and therefore no Uganda Joint Baseline/Endline Population-Based Survey Protocol – USAID Office of Food for Peace 13 individual respondent will be interviewed for more than two hours. In addition, respondents will not incur transportation or other costs as a result of participating in the survey. 6. TIMELINE Table 8 provides the timeline for critical activities for the joint BL/EL PBS. Table 8. Uganda Joint BL/EL PBS Critical Activities Timeline Activity Date Research protocol submitted to the ethics committee Jan 31, 2018 Listing training and listing exercise Jan/Feb, 2018 Questionnaire Finalized for Main Training May 7, 2018 Refresher training May 9-10, 2018 Main Training (interviewer and supervisor) May 11-31, 2018 Anthropometry training and standardization May 16-31, 2018 Field pilot practice June 1-4, 2018 Pilot debriefing and procedural adjustments if needed June 5-6, 2018 Household survey fieldwork starts June 7, 2018 Household survey fieldwork ends July 6, 2018 Final clean dataset delivered to ICF July 15, 2018 Note that the first two activities were completed under the prior EVELYN contract. ANNEX 1 – JOINT BL/EL PBS INDICATORS Uganda Joint Baseline/Endline Population-Based Survey Protocol – USAID Office of Food for Peace 14 Indicator Disaggregation Level 2018 BL 2018 EL FOOD SECURITY 1. Average Household Dietary Diversity Score (HDDS) None   2. Prevalence of moderate and severe food insecurity in the population, based on the Food Insecurity Experience Scale (FIES) [30 day recall] GHT   3. Prevalence of moderate and severe food insecurity in the population, based on the Food Insecurity Experience Scale (FIES) [12 month recall] GHT  POVERTY 4. Per capita expenditures (as a proxy for income) of USG-assisted areas GHT   5. Prevalence of Poverty: Percent of people living on less than $1.25/day 2005 PPP (EL) or $1.90/day 2011 PPP (BL) GHT   6. Depth of Poverty: Mean percent shortfall relative to the $1.25/day (EL) or $1.90/day (BL) poverty line GHT   7. Depth of Poverty of the Poor: Mean percent shortfall of the poor relative to the $1.90/day 2011 PPP poverty line GHT  WATER, SANITATION, AND HYGIENE 8. Percentage of households using an improved drinking water source None  9. Percentage of households using basic drinking water services None  10. Percent of households in target areas practicing correct use of recommended household water treatment technologies None  11. Percent of households that can obtain drinking water in less than 30 minutes (round trip) None  12. Percentage of households using an improved sanitation facility None  13. Percentage of households with access to a basic sanitation service GHT  14. Percent of households in target areas practicing open defecation None  15. Percent of households with soap and water at a handwashing station commonly used by family members None   AGRICULTURE 16. Percentage of farmers who used financial services (savings, agricultural credit, and/or agricultural insurance in the past 12 months Sex   17. Percentage of farmers who practiced the value chain activities promoted by the project in the past 12 months Sex   18. Percentage of farmers who used at least [a project-defined minimum] sustainable agriculture (crop, livestock and natural resource management) practices and/or technologies in the past 12 months Sex, type of practice  19. Proportion of producers who have applied targeted improved management practices or technologies* Sex, type of practice, type of commodity  20. Percentage of farmers who used improved storage practices in the past 12 months Sex   WOMEN’S HEALTH AND NUTRITION 21. Prevalence of underweight (BMI < 18.5) women of reproductive age None   22. Prevalence of women of reproductive age consuming a diet of minimum diversity None  23. Percentage of women of reproductive age who are currently using, or whose sexual partner is currently using, at least one contraceptive method, regardless of the method used None  24. Percent of births receiving at least four antenatal care (ANC) visits during pregnancy None   25. Prevalence of women of reproductive age who consume targeted nutrient-rich commodities Sex, type of commodity  CHILDREN’S HEALTH AND NUTRITION 26. Prevalence of healthy weight (WHZ ≤ 2 and ≥ -2) among children under five (0-59 months) Sex  ANNEX 1 – JOINT BL/EL PBS INDICATORS Uganda Joint Baseline/Endline Population-Based Survey Protocol – USAID Office of Food for Peace 15 Indicator Disaggregation Level 2018 BL 2018 EL 27. Prevalence of underweight children (WAZ < -2) children under five (0-59 months) Sex  28. Prevalence of stunted children (HAZ < -2) children under five (0-59 months) Sex   29. Prevalence of wasted children (WHZ < -2) children under five (0-59 months) Sex   30. Percentage of children under age five who had diarrhea in the past two weeks Sex   31. Percentage of children under five years old with diarrhea treated with oral rehydration therapy Sex   32. Prevalence of exclusive breastfeeding of children under six months of age Sex   33. Prevalence of children 6-23 months receiving a minimum acceptable diet Sex   34. Prevalence of children 6- 23 months who consume targeted nutrient-rich commodities Sex, type of commodity  GENDER 35. Percentage of men and women in union who earned cash in the past 12 months Sex  36. Percentage of women in union and earning cash who report participation in decisions about the use of self-earned cash None  37. Percentage of women in union and earning cash who report participation in decisions about the use of spouse/partner’s self-earned cash None  38. Percentage of men in union and earning cash who report spouse/partner participation in decisions about the use of self-earned cash None  39. Percentage of men and women in union with children under two who have knowledge of maternal and child health and nutrition (MCHN) practices Sex  40. Percentage of men/women in union with children under two who make maternal health and nutrition decisions alone Sex  41. Percentage of men/women in union with children under two who make maternal health and nutrition decisions jointly with spouse/partner Sex  42. Percentage of men/women in union with children under two who make child health and nutrition decisions alone Sex  43. Percentage of men/women in union with children under two who make child health and nutrition decisions jointly with spouse/partner Sex  RESILIENCE 44. Shock exposure index None  45. Cumulative impact of shock exposure index None  46. Absorptive capacity index None  47. Adaptive capacity index None  48. Transformative capacity index None  49. Ability to recover from shocks and stresses index None  50. Proportion of households participating in group-based savings, micro-finance or lending programs None  51. Index of Social Capital at the household level None  CUSTOM INDICATORS 52. Percentage of respondents reporting increased movement in areas that were previously not accessible due to insecurity None  53. Percentage of households with access to a sanitation facility – not necessarily improved None  54. Average number of crops produced per farmer in the past 12 months None  55. Percentage of farmers adopting farmer managed natural regeneration practices in the past 12 months None  56. Percentage of livestock owners accessing government or private sector vet care in the past 12 months None  ANNEX 1 – JOINT BL/EL PBS INDICATORS Uganda Joint Baseline/Endline Population-Based Survey Protocol – USAID Office of Food for Peace 16 Indicator Disaggregation Level 2018 BL 2018 EL 57. Average rating of government's ability to be responsive to citizens' needs (including transparency, inclusivity, effectiveness) as measured on scorecard Sex  58. Percent of target population who can state at least one health benefit of waiting at least two years after last live birth before attempting the next pregnancy Sex, Age  BL = DFSA Baseline Study EL = DFAP Endline Evaluation GHT= Gendered Household Type, FIES = Food Insecurity Experience Scale * Pending confirmation of feasibility with existing data ANNEX 2 - TRAINING AGENDAS Uganda Joint Baseline/Endline Population-Based Survey Protocol – USAID Office of Food for Peace 17 2018 Food for Peace Population-Based Survey in Uganda Training Agenda for Interviewer and Supervisor (Main) Training Training Venue:   May 11-19: Makerere University Senate Building Level 4, Senate Conference Hall May 21-31: Makerere University Food Science and Technology Building, Food Science Conference Hall Date Time Discussion Topic Note (Day 1) May 11 08:00-09:00 Registration  Introduction & Overview  Household identification & Consent  Household roster 09:00-10:30 Introduction of the trainees and resource persons Background of FFP Population-Based Survey Detailed explanation of the objectives of the survey  Personal qualities and performance standards in interviewing  Role of pretest interviewers  Dos and don’ts of interviewing 10:30-10:45 Tea-break 10:45-12:30 Discussion on field procedures Discussion on survey methodology including sample design Familiarization with questionnaire (Module-by-Module explanation of questionnaires) 12:30-1:30 Lunch-break 1:30-3:30 Discussion on Module-A (Identification and Consent)) 3:30-3:45 Tea-break 3:45-5:30 Discussion on Module- B (Household roster) (Day 2) May 12 8:30:00-10:30 Review of previous day’s sessions Discussion on Module- B (Household roster)  Household Roster 10:30-10:45 Tea-break 10:45-12:30 Discussion on Module- B (Household roster) 12:30-1:30 Lunch-break 1:30-3:30 Practice and role play: Module A and B 3:30-3:45 Tea-break 3:45-5:30 Practice and role play: Module A, B Debrief (Day 3) May 14 8:30:00-10:30 Review of previous day’s sessions Discussion Module C  Food Access & Food Security  Mobility, Local Government Responsiveness, and Poverty Probability Index 10:30-10:45 Tea-break 10:45-12:30 Discussion on Module C, CC 12:30-1:30 Lunch-break 1:30-3:30 Discussion on Module F 3:30-3:45 Tea-break ANNEX 2 - TRAINING AGENDAS Uganda Joint Baseline/Endline Population-Based Survey Protocol – USAID Office of Food for Peace 18 Date Time Discussion Topic Note 3:45-5:30 Practice and role play: Module C, CC, F Debrief  Water, Sanitation and Hygiene (Day 4) May 15 8:30:00-10:30 Review of previous day's sessions Discussion of Module D1, D2, E  Children’s Nutritional Status and Feeding Practices  Children’s Diarrhea and Oral Rehydration Therapy  Women’s Nutrition, Breastfeeding, and Antenatal Care 10:30-10:45 Tea-break 10:45-12:30 Discussion of D1, D2, E 12:30-1:30 Lunch 1:30-3:30 Practice and role play: Module D1, D2, E 3:30-3:45 Tea-break 3:45-5:30 Practice and role play: Module D1, D2, E Debrief (Day 5) May 16 8:30:00-10:30 Review of previous day's sessions Discussion of Module G  Agriculture 10:30-10:45 Tea Break 10:45-12:30 Discussion of Module G 12:30-1:30 Lunch 1:30-3:30 Practice and role play: Module G 3:30-3:45 Tea Break 3:45-5:30 Practice and role play: Module G Debrief (Day 6) May 17 8:30:00-10:30 Review of previous day’s session Discussion of Module J, K, L  Gender-Cash  Gender-Maternal Child Health and Nutrition (MCHN)  Gender￾Household Decision-Making, Access to Credit and Group Participation Request participation of Agriculture Specialist from IPs 10:30-10:45 Tea Break 10:45-12:30 Discussion of Module J, K, L 12:30-1:30 Lunch 1:30-3:30 Explanations on Module G by experts from IPs 3:30-3:45 Tea Break 3:45-5:30 Explanations on Module G by experts from IPs (Day 7) May 18 8:30:00-10:30 Review of previous day’s sessions Practice and role play: Module J, K, L  Gender-Cash  Gender-Maternal Child Health and Nutrition (MCHN)  Gender￾Household Decision-Making, Access to Credit and Group Participation  Poverty Measurement 10:30-10:45 Tea Break 10:45-12:30 Practice and role play: Module J, K, L Debrief 12:30-1:30 Lunch 1:30-3:30 Discussion of Module H 3:30-3:45 Tea Break 3:45-5:30 Discussion of Module H (Day 8) 8:30:00-10:30 Review of previous day’s sessions ANNEX 2 - TRAINING AGENDAS Uganda Joint Baseline/Endline Population-Based Survey Protocol – USAID Office of Food for Peace 19 Date Time Discussion Topic Note May 19 Discussion of Module H  Poverty 10:30-10:45 Tea Break Measurement 10:45-12:30 Practice and role play of Module H 12:30-1:30 Lunch 1:30-3:30 Practice and Role Play: Module H Debrief 3:30-3:45 Tea Break 3:45-5:30 Practice and Role Play: Module H Debrief (Day 9) May 21 8:30:00-10:30 Anthropometry Assistant Training  Training on Anthropometry Assistant THIS IS ONE-DAY SESSION FOR ALL INTERVIEWERS WHO WILL SERVE AS ANTHROPOMETRY ASSISTANTS. PLEASE REFER TO ANTHROPOMETRY TRAINING AGENDA FOR COMPLETE DETAILS ON TRAINING OF ANTHRPOMETRY SPECLAISTS. 10:30-10:45 Tea Break 10:45-12:30 Anthropometry Assistant Training 12:30-1:30 Lunch 1:30-3:30 Anthropometry Assistant Training 3:30-3:45 Tea Break 3:45-5:30 Anthropometry Assistant Training (Day 10) May 22 8:30:00-10:30 Review of previous day’s sessions Discussion: Module R  Resilience 10:30-10:45 Tea Break 10:45-12:30 Discussion: Module R 12:30-1:30 Lunch 1:30-3:30 Practice and role play: Module R 3:30-3:45 Tea Break 3:45-5:30 Practice and role play: Module R Debrief (Day 11) May 23 8:30:00-10:30 Introduction to CAPI Tablet basics  CAPI Training 10:30-10:45 Tea Break 10:45-12:30 Distribution of Tablets Data entry exercise 12:30-1:30 Lunch 1:30-3:30 CAPI menu system Assigning households 3:30-3:45 Tea break 3:45-5:30 Household data entry Demonstration and practice Enter data from paper questionnaire (Day 12) 8:30:00-10:30 Review of previous day’s sessions ANNEX 2 - TRAINING AGENDAS Uganda Joint Baseline/Endline Population-Based Survey Protocol – USAID Office of Food for Peace 20 Date Time Discussion Topic Note May 24 Household characteristics data entry Continue entering data  CAPI Training 10:30-10:45 Tea Break 10:45-12:30 Listing eligible members for anthropometry Continue entering data 12:30-1:30 Lunch 1:30-3:30 Continue entering data 3:30-3:45 Tea Break 3:45-5:30 Practice transferring data to Supervisor’s tablet (Day 13) May 25 8:30:00-10:30 Review of previous day’s sessions Listing cases for outstanding modules Continue entering data  CAPI Training 10:30-10:45 Tea Break 10:45-12:30 Downloading updated programs 12:30-1:30 Lunch 1:30-3:30 Data review by Supervisor’s tablet. Check and modify data as needed 3:30-3:45 Tea Break 3:45-5:30 Discussion on error after reviewing data. Discussion on CAPI part done so far. (Q/A session). Transfer data to Supervisor’s tablet, Supervisor uploads data (Day 14) May 26 8:30:00-10:30 Review of previous day’s sessions Entering anthropometry data Mock interviews with CAPI  CAPI Training 10:30-10:45 Tea Break 10:45-12:30 Mock interviews with CAPI 12:30-1:30 Lunch 1:30-3:30 Mock interviews with CAPI Transferring data to supervisor’s tablets 3:30-3:45 Tea Break 3:45-5:30 Discussion on error after reviewing data. Discussion on CAPI part done so far. (Q/A session). (Day 15) May 28 8:30:00-10:30 Closing clusters Practice finalizing work in a cluster  CAPI Training 10:30-10:45 Tea Break 10:45-12:30 Mock Interview with CAPI 12:30-1:30 Lunch 1:30-3:30 Mock interviews with CAPI Transferring data to supervisor’s tablets Fixing duplicate households 3:30-3:45 Tea Break 3:45-5:30 Discussion on error after reviewing data. Discussion on CAPI part done so far. (Q/A session). (Day 16) May 29 8:30:00-10:30 Review of previous day’s sessions Mock interviews with CAPI Supervisor Training 10:30-10:45 Tea Break 10:45-12:30 Mock interviews with CAPI ANNEX 2 - TRAINING AGENDAS Uganda Joint Baseline/Endline Population-Based Survey Protocol – USAID Office of Food for Peace 21 Date Time Discussion Topic Note 12:30-1:30 Lunch 1:30-3:30 Discussion on error after reviewing data. Discussion on CAPI part done so far. (Q/A session). 3:30-3:45 Tea Break 3:45-5:30 Discussion on error after reviewing data. Discussion on CAPI part done so far. (Q/A session). (Day 17) May 30 8:30:00-10:30 Review of previous day’s sessions Mock interviews with CAPI in languages Supervisor Training 10:30-10:45 Tea Break 10:45-12:30 Mock interviews with CAPI in languages 12:30-1:30 Lunch 1:30-3:30 Discussion on error after reviewing data. Discussion on CAPI part done so far. (Q/A session). 3:30-3:45 Tea Break 3:45-5:30 Discussion on error after reviewing data. Discussion on CAPI part done so far. (Q/A session). (Day 18) May 31 8:30:00-10:30 Review of previous day’s sessions Mock interviews with CAPI in languages Supervisor Training 10:30-10:45 Tea Break 10:45-12:30 Mock interviews with CAPI in languages 12:30-1:30 Lunch 1:30-3:30 Discussion on error after reviewing data. Discussion on CAPI part done so far. (Q/A session). 3:30-3:45 Tea Break 3:45-5:30 Discussion on planning of pilot test (Day 19- Day 22) June 1 to June 4 Pilot Test Day 23 June 5 Pilot Debriefing and Deployment plan/logistics ANNEX 2 - TRAINING AGENDAS Uganda Joint Baseline/Endline Population-Based Survey Protocol – USAID Office of Food for Peace 22 Anthropometry training and standardization testing schedule May 2018 Sunday Monday Tuesday Wednesday Thursday Friday Saturday FIRST WEEK 13 MAY 14 MAY 15 MAY 16 MAY 17 MAY 18 MAY 19 MAY 9am to 1 pm Meeting to define schedule and logistics Equipment checkup including standardization. Take the equipment to the main training center and prepare the anthropometrist’s kit. At main training center: Introduction to Anthropometry At main training center: Introduction to standing height and weight of children 2-5 yrs. old, Recumbent length and weight of children <2 yrs. old At facility for children: Hands on practice, every procedure with special focus on the 2-5 yrs. age category At facility for children: Hands on practice, every procedure with special focus on the < 2yrs age category 2pm to 4pm At main training center: Height and weight of adults (mothers).Training and practice on each other To be used as needed To be used as needed To be used as needed SECOND WEEK 20 MAY 21 MAY 22 MAY 23 MAY 24 MAY 25 MAY 26 MAY 9am to 1pm At main training center: Introduction to anthropometry and practice (ASSISTANT) At main training center. Introduction to standardization testing of mothers. At facility for children: Standing height and weight standardization test of children 2-5 yrs. old At facility for children: Length standardization test of children <2 yrs. old At facility for children: Re￾standardization test At facility for children: Hands on practice, every procedure. 2pm to 4pm At facility for children: Standing height and weight of children 2-5 yrs. old, training and practice (ASSISTANT) To be used as needed To be used as needed To be used as needed At facility for children: Hands on practice, every procedure. At facility for children: Recumbent length and weight training of children <2 yrs. old (ASSISTANT) THIRD WEEK 27 MAY 28 MAY 29 MAY 30 MAY 9am to 1pm Introducing the quality control sheet and Z￾score tables to team supervisors Continued practice Anthropometrists familiarizing with their teams in preparation for pilot 2pm to 4pm To be used as needed To be used as needed ANNEX 2 - TRAINING AGENDAS Uganda Joint Baseline/Endline Population-Based Survey Protocol – USAID Office of Food for Peace 23 NOTES: 1. Interactive Training Training will be interactive and participatory with practice, testing and discussions. 2. Flexibility The above schedule WILL change according to the needs as the training progresses. 3. Introduction to Anthropometry The Introduction to Anthropometry session will include subjects of the measurements taken, the importance of taking accurate measurements, types of measurement errors, reading and recording measurements, reading and recording systems. Also, definitions of measurements, derived anthropometry indices (i.e., stunting, wasting, underweight) will be presented (note—this covers some of the basic science of anthropometry so that the trainees will have a greater understanding of anthropometry, its use in population surveys with the intention of a greater ‘ownership’ of the anthropometry component of the survey). 4. Other subjects to be covered and trained (to be inserted in the above schedule:      Bilateral pedal edema training Age assessment training Anthropometry questionnaire training, completion Team supervisor training that includes the use of growth charts to evaluate acceptable measurements, taking replicate measurements occasionally, use of the Anthropometry Supervisor Checklist and other quality control tasks Understanding weighing and measuring instruments ANNEX 3: GHG FINAL EVALUATION PROTOCOL D Final Performance Evaluation of the Northern Karamoja Growth, Health and Governance (GHG) Development Food Assistance Project Evaluation Protocol - FINAL Office of Food for Peace Contract #: GS-00F-189CA/7200AA18M00002 October 17, 2018 This publication was produced for review by the U.S. Agency for International Development. It was prepared by ICF Macro, Inc. Final Performance Evaluation of the Northern Karamoja Growth, Health and Governance (GHG) Development Food Assistance Project – Final Evaluation Protocol i ACRONYMS AWOTID Abim Women Organized Together in Development DFAP development food assistance project DFSA development food security activity FFP Office of Food for Peace FGD focus group discussion FIC Feinstein International Center FY fiscal year GHG Growth, Health, and Governance IDI in-depth interview IPTT indicator performance tracking table KAPDA Kaabong Peace and Development Agency KII key informant interview LOA Life of Activity MC Mercy Corps PBS population-based survey PLW pregnant and lactating women RWANU Resiliency through Growth, Agriculture, and Nutrition SACCO Savings and Credit Cooperative SME subject matter expert SO strategic objective TOC theory of change VSLA Village Savings and Loan Association USAID United States Agency for International Development WASH water, sanitation, and hygiene Final Performance Evaluation of the Northern Karamoja Growth, Health and Governance (GHG) Development Food Assistance Project – Final Evaluation Protocol ii TABLE OF CONTENTS 1. Background and Purpose of the Evaluation ....................................................................................................... 1 2. Research Questions and Sub-Questions ............................................................................................................ 1 3. Methodology .............................................................................................................................................................. 4 3.1. Evaluation Design ............................................................................................................................................ 4 3.1.1. Overview of Design .......................................................................................................................... 4 3.1.2. Overview of Data Sources .............................................................................................................. 4 3.2. Qualitative Data Collection Methods ......................................................................................................... 5 3.2.1. Types of Respondents ...................................................................................................................... 6 3.2.2. Site Selection and Locations and Respondent Selection .......................................................... 6 3.2.3. Field Processes ................................................................................................................................... 9 3.2.4. Data Management Processes .......................................................................................................... 9 3.2.5. Informed Consent Procedures and Ethical Considerations .................................................. 11 3.2.6. Training Interpreters, Including on Human Subjects Protocols............................................ 11 3.3. Data Analysis .................................................................................................................................................. 12 3.3.1. Overview of Data Analysis Plan ................................................................................................... 12 3.3.2. Qualitative Data ............................................................................................................................... 13 3.3.3. Procedures for Analyzing PBS Data ............................................................................................ 15 3.3.4. Integration of PBS Data .................................................................................................................. 15 3.3.5. Integration of LOA Performance Data from IPTT and Information from Document Review ............................................................................................................................................................. 16 3.3.6. Preparation of Comprehensive, Data-based Findings ............................................................. 16 3.3.7. Proposed Report Outline .............................................................................................................. 16 Annex A. GHG Evaluation Design Matrix ................................................................................................................. 19 Annex B. Informed Consent ......................................................................................................................................... 39 Annex C. Draft Final Travel Itinerary ......................................................................................................................... 41 Annex D. Summary of Population-based Survey Methodology ............................................................................ 42 Annex E. PBS Endline Indicators .................................................................................................................................. 44 Annex F. Interview and Focus Guides ........................................................................................................................ 45 Final Performance Evaluation of the Northern Karamoja Growth, Health and Governance (GHG) Development Food Assistance Project – Final Evaluation Protocol 1 1. BACKGROUND AND PURPOSE OF THE EVALUATION The Uganda Growth, Health, and Governance (GHG) United States Agency for International Development (USAID) Office of Food for Peace (FFP) Title II development food assistance project (DFAP) in Northern Karamoja was awarded to Mercy Corps (MC) and its partners in 2012. Implementation began in fiscal year (FY) 2013. GHG was originally planned to end in FY 2017, but it was extended until January 2018. The final performance evaluation of GHG is the second and final phase of a pre-post evaluation strategy. The purpose of the final performance evaluation for the GHG DFAP is to examine outcome data, identify and understand the factors that contributed to the achievement of outcomes or performance barriers, assess the sustainability of those outcomes, and provide recommendations for MC’s follow-on development food security activity (DFSA) that launched in 2018. This protocol report provides greater detail on the evaluation design, sources of data for the evaluation, methodologies for data collection, and analytical procedures to produce findings incorporating qualitative and quantitative data. 2. RESEARCH QUESTIONS AND SUB-QUESTIONS This section provides the GHG final performance evaluation questions from the evaluation scope of work with initial ideas for “contextualizing” individual sub-questions based on the team’s literature review to date. Annex A provides an Evaluation Matrix detailing the specific data sources that will be used to answer each question and the analytical procedures that will be employed to answer those questions. Q1: To what extent have the projects met their defined goals, purposes, and outcomes? The evaluation team will assess and determine the following: (1.1) Project performance on final indicator values against targets (set by both the partners and FFP on the key indicators of depth of poverty, stunting, and undernutrition) (1.2) Factors that promoted or inhibited the achievement of the project objectives, including but not limited to the effectiveness of food-for-asset and cash-for-asset interventions How has the discontinuation of food rations provided by GHG in FY 2017 affected the health and nutrition status of pregnant and lactating women (PLW) and children under two? Or is it too early to assess? Is there any preliminary evidence of the practical effect from interviews with women who were pregnant or lactating during the GHG DFAP? What factors have inhibited or promoted real progress toward the achievement of targets for reducing the high rate of childhood malnutrition? The 2017 final evaluation suggested that despite two droughts during project implementation, performance would have been worse without GHG activities. Which behavior change initiatives and activities promoted by GHG have had the greatest impact on affecting change in nutritional status of PLW and children under two? Final Performance Evaluation of the Northern Karamoja Growth, Health and Governance (GHG) Development Food Assistance Project – Final Evaluation Protocol 2 How did project-supported infrastructure development affect outcomes (e.g., new roads linking villages to markets, new [additional] health centers, livestock slaughterhouse meeting international safety and health requirements)? (1.3) Targeting strategies and their contribution to achieving project goals, especially with regard to gender and reaching the most vulnerable How effective was targeting women and youth for involvement in conflict prevention and resolution to achievements made in conflict resolution? (1.4) The effectiveness of interventions on the poorest individuals How effective was targeting of livelihood activities for vulnerable mothers and households in the agriculture￾based livelihood zone in reducing their overall poverty status? (1.5) Based on evidence from sub-questions 1–5, what has been the contribution of GHG to USAID’s efforts to reduce food insecurity among chronically food insecure households? Q2: For each technical sector, what are the strengths of and challenges in the project’s design, including theories of change (TOCs) that influence the effectiveness of the interventions? The evaluation team will assess the following: (2.1) The relevance of the technical interventions, including food-for asset and cash-for-asset interventions, to achieve project outcomes, and discuss those findings in relation to the projects’ TOCs How effective have the revisions (additions) to the Making Markets Work for the Poor approach been in increasing livelihoods among the poorest farmers? How has the discontinuation of food rations provided by GHG in FY 2017 affected the health and nutrition status of PLW and children under two? Or is it too early to assess how this has affected PLW and children under two? Is there any preliminary evidence of the practical effect from interviews with PLW? Since the Government of Uganda stopped registering pregnant women, has this affected the health and nutrition status of PLW and children under two? Which specific water, sanitation, and hygiene (WASH) interventions and approaches can be linked to decreases (or show great potential) that were achieved in reducing malnutrition among children under five? (2.2) The plausibility of pathways and the determinants of achieving the key outcomes, highlighting any unforeseen pathways leading to unintended positive and negative consequences of the projects Q3: In each technical sector, what are the strengths of and challenges of the effectiveness of the interventions’ implementation? The evaluation team will evaluate the following: (3.1) The effectiveness and relevance of the technical interventions, including food-for asset and cash-for￾asset interventions, to achieve project outputs of greater or lesser quality, and discuss those findings in relation to the projects’ TOCs Strategic objectives (SO) 1, 2, and 3: How effective has the creation of gender-based groups (women’s savings groups, mother care groups, male change agents, and women’s peace groups) been in promoting gender equity and equality, and what has been the effect of these groups on incidence of sexual and gender-based violence? How effective was blanket feeding in promoting behavior changes in PLW? Has the type of training provided to young men and their participation in youth peace groups contributed to reduction in violence against girls and women? What evidence is there to support this? Final Performance Evaluation of the Northern Karamoja Growth, Health and Governance (GHG) Development Food Assistance Project – Final Evaluation Protocol 3 (3.2) Factors in the implementation approach and methods, and the context associated with greater or lesser effectiveness in producing outputs of higher or lower quality How effective has the use of smart subsidies and training been in promoting greater adoption of improved seeds and climate-smart seeds among the poorest farmers in each district? How effective have these methods been in promoting greater adoption by women-headed household in each district? How has the discontinuation of food rations impacted women’s participation in mother care groups and mother care groups plus, and health facility access for pregnant women and children under five? (3.3) The interventions and implementation processes deemed more or less acceptable to members of the target communities Q4: Which project outcomes are likely to be sustained? The evaluation team will evaluate the following: (4.1) The quality and functionality of systems, processes, and institutional arrangements established and strengthened independently by the projects, as well as in collaboration with the private sector, the Government of Uganda, community organizations, and research organizations, to achieve project outcomes and sustainability What has been done to build and sustain Karamoja’s nascent livestock value chain based on the work done with farmers and herders under the FY 2017 GHG’s Livestock Systems Pilot in the sub-counties of Kacheri and Rengen in Kotido and in the sub-counties of Lolelia and Kalapata in Kaabong? Will the organization of public-private sector actors be able to continue functioning effectively to maintain existing boreholes and build new ones to increase access to clean water and reduce the incidence of diarrhea among children under five? (4.2) Whether the necessary resources and capacity strengthening will exist to sustain service providers, and the likelihood that service provision will continue after the project ends What is the likelihood that the establishment of agricultural extension workers at the sub-district level paid by GHG will be incorporated as employees in district-level government? What resources are available to continue their positions and provide ongoing capacity training so that these agents can provide effective services to farmers? Are community animal health workers still able to meet the demand of livestock owners for services on a sustainable basis? What resources are available to provide ongoing capacity training so that community animal health workers can provide effective services? To what extent will communities that have begun paying for water and sanitation services continue? What has been done to promote community payment for services in other locations? With the disbanding of government-provided village health workers, what resources are there to continue providing services at the village level? Will the use of GHG’s strategic funds for in-kind grants be sufficient to cement the achievements of key local entrepreneurs and businesses provided with capacity development training under the GHG business development program? (4.3) Communities’ perceptions on the quality, frequency, and effectiveness of the implementation processes and services provided What are the perceptions of communities about the quality and longevity of community-built toilets and the improved toilets constructed in schools and at the slaughterhouses? Final Performance Evaluation of the Northern Karamoja Growth, Health and Governance (GHG) Development Food Assistance Project – Final Evaluation Protocol 4 (4.4) What is the motivation of the community and beneficiaries to demand and pay (or invest time) for the services, and communities’ perceptions on sustainability of the services provided by the project? According to final evaluation, GHG has the only program in which villagers actually pay for private sector water and sanitation services. What factors contribute to their willingness to pay for these services? (4.5) The extent to which the projects leveraged other U.S. Government and non-U.S. Government investments to achieve sustained outcomes as identified in the TOCs (4.6) Evidence of enhanced linkages with other service providers Q5: What are the key lessons learned and best practices that should inform future projects in the country? (5.1) What are the lessons, best practices, and approaches that should be considered in designing future food and nutrition security projects? (5.2) What are the lessons, best practices, and approaches that should be considered in implementing and implementation approaches for future food and nutrition security projects? (5.3) What are the risks to the effective implementation of these approaches to support project achievements? (5.4) Based on the lessons learned, how can potential unintended negative consequences be minimized and positive consequences be captured systematically? 3. METHODOLOGY 3.1. Evaluation Design 3.1.1. Overview of Design The overall design for this final evaluation of the GHG DFAP is based on the collection of both qualitative and quantitative data. The performance evaluation team will use a mixed methods approach that involves analytical procedures to develop independent findings from the primary qualitative and quantitative data sources. The findings from the analysis of both sources will be integrated to develop a comprehensive, evidence-based finding for each evaluation question. 3.1.2. Overview of Data Sources The data sources used for this evaluation include primary qualitative data collected during fieldwork, primary quantitative data from the comparison of population-based survey (PBS) baseline (2013) and endline (2018) data, and the GHG final Life of Activity (LOA) Indicator Performance Tracking Table (IPTT), and secondary data from a desk review of various project documents. Qualitative Data from Fieldwork Qualitative data will be collected from September 26 through October 10, 2018 during fieldwork in the districts of Abim, Kotido, and Kaabong in a purposively selected sample of sub-counties and villages in each district. Details of data collection methods are described in section 3.2. The GHG evaluation team will conduct focus group discussions (FGDs), key informant interviews (KIIs), in-depth interviews (IDIs), and small group interviews involving the participation of stakeholders, GHG beneficiaries, and non￾beneficiaries across these three administrative levels. Final Performance Evaluation of the Northern Karamoja Growth, Health and Governance (GHG) Development Food Assistance Project – Final Evaluation Protocol 5 Quantitative Data from PBS The baseline household PBS for the two DFAPs (GHG and RWANU) was conducted from late February to end of April 2013 and included 2,400 households from 80 villages across the three districts in the GHG project area (with 2,400 additional households included from the RWANU project area). Endline PBS data were collected in June/July 2018 from 1,260 households in 42 villages in the GHG project area. The endline data were collected as part of a larger endline survey for the two DFAPs, combined with a baseline for the two new DFSAs (Nuyok and Apolou). The PBS questionnaires were based on the core FFP and resilience indicators and consisted of 13 separate modules covering food security, poverty, WASH, agriculture and women’s and children’s health and nutrition. The pre-post design (using the 2013 baseline PBS and the 2018 endline component of the joint baseline/endline PBS) allows for the determination of statistically significant change in indicators; however, it does not allow statements to be made about attribution or causation relating to program impact. Additional methodological details of the PBS are provided in Appendix D. The indicators used in the PBS endline are presented in Appendix E. Quantitative Data from the Project IPTT MC’s monitoring and evaluation team collected data for annual monitoring of GHG project-level indicators related to direct participants. For each indicator, base values were established, as well as targets for annual performance and for the LOA. The performance evaluation team will use the final IPTT data, which include the final LOA performance data, the LOA target, and a calculation of the extent to which LOA targets were met. Although information from the IPTT is not the primary source of quantitative data for this performance evaluation, these data will be used to address specific evaluation questions about project outputs, the GHG theory of change (TOC), and pathways implemented to reach project objectives and FFP goals. Secondary Data from the Desk Review The team read a range of project-specific documents (the approved grant proposal, annual results reports, quarterly reports), as well as evaluation reports, barrier analysis reports, sector assessments conducted by Feinstein International Center (FIC), and other documents providing contextual data. Project documents were used to identify (1) the strategies, interventions, and implementation approaches used to achieve outcomes for each SO; and (2) the project participants, beneficiaries, and other stakeholders involved in each SO. These documents were also used to review the narrative of progress and challenges during 2013 to 2018 and to identify and understand changes that were made during LOA. 3.2. Qualitative Data Collection Methods The evaluation team will use FGDs, KIIs, and IDIs, and small group interviews with a range of beneficiary groups, farmers, and pregnant and lactating women who were not beneficiaries (under SO 1 and SO 2), and other GHG stakeholders at the village, sub-county, and district levels. IDIs will be used with individuals who may not want to risk speaking up in a group setting. The purpose of collecting qualitative data in each of these administrative levels is to assess interventions and implementation approaches that span the village, the sub-county, and the district it is located in, as well as to learn of and compare the perspectives and opinions of project stakeholders on the achievement of results under each of the three SOs, as described in Section 3.2.1. Final Performance Evaluation of the Northern Karamoja Growth, Health and Governance (GHG) Development Food Assistance Project – Final Evaluation Protocol 6 3.2.1. Types of Respondents Table 1. Types of Respondents at the Village, Sub-county, and District Levels Location Type of Respondent* Village Targeted beneficiary groups Members of community-based organizations (Water User Associations, Mother Care Groups, Women’s Peace Groups) and Peace Committees were mainly targeted at the sub￾county, parish, and district levels. However, informal women’s peace groups were targeted in Kaabong. Village leaders involved in promoting the adoption of inputs, practices, behavior change (lead farmers, lead mothers, male change agents, traditional village leaders [i.e., elders]) Local agricultural input suppliers, agro-input dealers, agro-vets, etc.; few at the village level Local traders Leaders of village savings and loan associations Sub-county Private sector providers (agro-input dealers, veterinary medicines) Service providers Traders Financial service providers (savings and credit cooperatives) Business forums, agro-input dealers, agro-vets, etc.; Agricultural extension workers, community animal health workers Peace committees Government officials from sub-county-level offices (Health, Sanitation, Water, Agriculture, etc.) Local councils District Commodity traders Commercial input suppliers Financial service providers (commercial banks) Business forums Health facilities Government officials from district-level offices (Health, Sanitation, Water, Agriculture, etc.) Chambers of Commerce Individuals and offices associated with the criminal justice system, peace and conflict resolution, land administration, and security *The exact composition of respondents who will be interviewed in each location will differ slightly per SO. 3.2.2. Site Selection and Locations and Respondent Selection Number of Sites Included by Type Data will be collected in a total of 15 sites comprising 3 districts, 6 sub-counties, and 6 villages. The unit of data collection comprises a district, one sub-county, and one village located in that sub-county. This data collection unit will also serve as the unit of analysis (see Section 3.3.1., Overview of Data Analysis Plan). For each of the three implementation districts, data will be collected at the district level, in two sub-counties, and in two villages. In each district, data will be collected in a total of five sites. Table 2 provides a summary of this information. Final Performance Evaluation of the Northern Karamoja Growth, Health and Governance (GHG) Development Food Assistance Project – Final Evaluation Protocol 7 Table 2. Location and Number of Data Collection Sites Location Number Number of Interview Sites per Location District 3 1 Sub-counties 6 2 Villages 6 2 Total number of data collection sites and interview sites per district 15 5 Site Locations, by District Table 3 shows the results of criteria used to draw a purposefully selected sample of data collection sites in Abim, Kotido, and Kaabong. Table 3. Data Collection Sites by District District Sub-county Parish Village Abim Morulem Arema Arema East Abim Town County Wiawer Atwilo Kotido Rengen Napongo Chaicon Nakapelimoru Potongor Longelep Kaabang Lolelia Kaimesel Chumakod Sidok Longaro Longar NOTE: These sites are being “reality-checked” for ease of access and, distances; and they may change based on the outcome of this check. Total Number of Respondents, by Site Individuals will participate in either KIIs, IDIs, small group interviews of two to three people, or FGDs limited to six to eight people. Table 4 shows an estimate of the minimum number of respondents that will participate in interviews at each site. Table 4. Maximum Number of Respondents Participating in Interviews by Type of Location and Across all Data Collection Sites Locations Maximum Number of Respondents by Type of Location Covering 3 SOs* Total Number of Respondents Across All Data Collection Sites by Type of Location Village Sites (6) 80 480 Sub-County Sites (6) 40 240 District Sites (3) 30 90 Total number of respondents across all data collection sites 810 *Not all village-level data collection sites were covered under all three GHG SOs. Respondent Identification and Recruitment Process Respondent Identification. The types of respondents we have identified for interviews at the district, sub-county, and village levels are included in Table 1 under Section 3.2.1. Recruiting Process. The evaluation team will rely on three research assistants/interpreters hired by International Research Consortium to serves as recruiters and to schedule interview times prior to the Final Performance Evaluation of the Northern Karamoja Growth, Health and Governance (GHG) Development Food Assistance Project – Final Evaluation Protocol 8 arrival of the evaluation team. Each recruiter will cover one of the three districts to begin the recruitment and scheduling process at the district, sub-county, and village levels. We have asked MC to assign a staff member in each district who can accompany the recruiter to facilitate introductions, starting at the district capital with the highest-level district official. The recruiting process for the three levels is described as follows.    District Process. The recruiter will be provided with a script to describe the purpose of the evaluation and the schedule and dates of data collection for that district, and information of the characteristics and types of people, offices, and organizations with which the evaluation team would like to hold interviews and FGDs. Drawing on the field knowledge of the MC staff to identify those individuals and locations, the recruiter will proceed to meet with each individual. The recruiter will use the same script to describe the purpose of the evaluation and why the evaluation team would like to interview him or her, and to request his or her participation. Finally, the recruiter will review the calendar of interviews for the district to show the date the evaluation team will arrive and to schedule (to the extent possible) a specific date and time for the interview. The individual will also be asked for contact information for the evaluation team to confirm the interview prior to the interview date, and the individual will be provided with the recruiter’s contact information. Sub-county Process. The process described for the district level will be repeated in each of the two sub-counties. Village Process. In the villages, the recruiters will use a pre-recruitment: process. In this process, the MC staff member will be asked to accompany the recruiters to each village and introduce them to the village elders. The recruiters will use a modified version of the script to describe the purpose of the evaluation and why the team would like to interview individuals in the village, explain the permissions obtained to date to interview from individuals in the district and sub-county, and review the dates scheduled for data collection in that village. With the assistance of the MC staff members and elders to identify village leaders related to each SO (e.g., lead mothers, lead farmers), the recruiters will make brief visits with each leader to ask for his or her help in recruiting individuals who participated in GHG activities, especially targeted populations, on the specific date scheduled for data collection in that village. The recruiters will provide a brief description of the profile of participants sought for each FGD, KII, and IDI, and the leaders whether they are willing to recruit individuals for interviews a day or two before the evaluation team is scheduled to enter the village. They will also be asked to recruit individuals who were not members of target populations or did not participate directly in GHG activities. The recruiters will ask for contact information from each leader willing to engage in this process so that he or she can be contacted the day before interviews are scheduled to confirm participation of identified village residents. The recruiters will give each leader their own contact information. Final Performance Evaluation of the Northern Karamoja Growth, Health and Governance (GHG) Development Food Assistance Project – Final Evaluation Protocol 9 3.2.3. Field Processes The evaluation team will collect data in the locations selected for the purposive sample of sub-counties and villages for each of the three GHG implementation districts. The fieldwork will be based on conducting interviews in each data collection unit (i.e., district, sub-county, and village). In each district, the team will collect data from five sites (refer to Table 2 in Section 3.2.2). The team will follow the fieldwork protocol of beginning interviews in each district with a brief meeting with the highest-level officials available to introduce the team members, review the purpose of the meeting, the locations within the district where interviews will be conducted, and the approximate length of time the team will spend in the district. From there, each subject matter expert (SME) will begin collecting data in the district according to the pre-arranged schedule of interviews. When the team has completed the scheduled interviews in the district, they will move to the first sub-county for data collection, and then to the village. The team will then travel to the second sub-county in the district to conduct interviews, and lastly to the village in that sub-county. The team will follow the same fieldwork process in the next two districts. Following the completion of data collection in each district, team members will review their interview notes and participate in a discussion of key observations and issues in each of the three SOs across the two data collection units. When fieldwork has been completed in the third and final district, the team leader will facilitate a day-long discussion of preliminary (pre-analysis) findings and key observations across the three districts per SO. The team will then begin preparing materials for the Mission out￾briefing presentation before traveling back to Kampala. During the period of data collection, the team leader will contact the ICF technical director on a weekly basis (or more frequently when necessary) to report on progress to date, report any problems and discuss possible solutions, and discuss any other important issues raised by FFP, the ICF technical director, or the team leader. Annex C contains the complete fieldwork schedule following the Mission in-briefing and the one-day orientation meeting that will be held with the full team when the team leader and SMEs arrive in the first district. The full team comprises the team leader, three SMEs, and three interpreters. 3.2.4. Data Management Processes Reviews and Discussions of Interview Notes to Ensure Learning throughout the Fieldwork Period During fieldwork evenings, team members will review the day’s field notes with their assigned interpreter to ensure that the notes are correct and do not miss any important responses from the interview participants. Each SME, or their note-taker, will begin typing up notes from each interview into electronic files. Each SME will use the pre-coded interview guides for each data collection instrument so that field notes will be typed up in the space following each question in the instrument. Undertaking this step will greatly help the data analysis process once the fieldwork is completed. All field notes will be taken in English, so no translations are necessary. ICF does not anticipate that the SMEs will be able to type their notes from every interview conducted that day. This will be an ongoing challenge. Team members will be asked to share, on an informal basis, any observations or impressions that struck them at the end of every day during or after the evening meal or during travel between locations. This informal sharing of observations will also provide the opportunity for anyone to raise any problems Final Performance Evaluation of the Northern Karamoja Growth, Health and Governance (GHG) Development Food Assistance Project – Final Evaluation Protocol 10 encountered or issues that need to be addressed to improve the fieldwork and the effectiveness of the data collection process. The team leader will be able to review progress and identify any emerging issues during these discussions. More structured team discussions will be held following the completion of interviews in each district to identify and discuss emerging issues or themes, distinct differences between sub-counties and villages in that district, and key observations. The team leader will take notes during the discussion. Following the completion of interviews in the second district, the team leader will check to see whether any of the issues or observations can build on what the team learned following completion of the first district, as well as ensure that new or different issues for accumulated learning are identified and discussed. These notes will serve as the basis for the day-long team discussion scheduled following the completion of interviews in the third district. The team will identify important themes, issues, key observations, and potential findings for each SO (and that cross over each SO). The results of these discussions will be used to prepare materials for the Mission out-briefing in Kampala. Quality Control Measures The team leader will engage in a number of practices to ensure quality control of fieldwork procedures and protocols, data collection, and the quality of fieldwork notes that will be used for data analysis. These practices are described as follows.    - Proactively identifying and solving issues emerging from daily discussions that could become problems and taking active steps to address and find solutions for current problems. Engaging the ICF technical director on a weekly basis, or more frequently as necessary, for finding acceptable solutions to problems that may entail discussions with ICF management or FFP. During briefings on progress (or lack thereof), as described in Section 3.2.3, Field Processes, the team leader will also report on progress to date and discuss issues identified by FFP, the director, or the team leader. Ensuring the field schedule is followed and that each SME is covering the interviews schedule for each day. The team leader will adjust the field schedule and the timing of interviews arranged in advanced if required, and immediately notify respondents and representatives of respondent groups in advance of scheduled interviews to adjust arrangement. When it is necessary to maintain the field schedule, the team leader will hold discussions with SMEs to discuss time savings measures that could be employed. For example, the team leader could work with the SME to conduct triage with interview questions to preserve the most important questions and to shorten the length of the interview; hold discussions with the SMEs and their assigned interpreter to identify and change any practices they use on a routine basis that lengthen the overall time of the interview with the advice of other team members; or task the field assistants with confirming interview dates, times, and locations for interviews to ensure that all scheduled interviews for the day can be conducted as planned. Other factors, such as weather conditions, road conditions, automobile problems, and illnesses, are beyond the team’s control. Serious issue may require larger adjustments to the interview schedule and will be discussed with the ICF contract manager and the Mission FFP point of contact if necessary to develop possible solutions. Final Performance Evaluation of the Northern Karamoja Growth, Health and Governance (GHG) Development Food Assistance Project – Final Evaluation Protocol 11   Once a week, the team leader will ask to review an interview for each SO that has been typed during the course of fieldwork. The team leader will review the completed interview notes for quality, clarity, and thoroughness, and, as necessary, hold a discussion with the SME and interpreter about any improvements that can be made to add value and to ensure the quality of the information to support the data analysis phase. During structured team discussions following the end of data collection in each district, the team leader will discuss the importance of, and questions about, the evidence for preliminary findings or emerging themes. This will ensure that SMEs will be prepared to provide some evidence for any key preliminary findings to the Mission during the out-briefing and that the preliminary findings will be available to use for data analysis. 3.2.5. Informed Consent Procedures and Ethical Considerations The team leader and the SME covering SO 2 have recently taken training and were tested on the protection of human subjects, ethical considerations, and informed consent procedures. Both individuals have certifications from this training course. With assistance from the certified SME, the team leader will be responsible for ensuring that procedures for protecting human subjects, for ethical behavior, and for obtaining informed consent are consistently followed during each interview. ICF prepared a statement approved by its Institutional Review Board that will be read by the SME or the interpreter at the beginning of each interview (see Annex B). The statement informs the respondents that their names will not be written down to ensure that, as individuals, they will never be associated with the interview in which they are about to participate. It describes what the team will do to ensure the confidentiality of the discussion (disposing of notes taken during the interview after the evaluation is completed or erasing the tape recording of the interview). As part of the ethical considerations the team will implement, the statement also informs the respondents that if any question or topic they are asked to discuss makes them uncomfortable in any way, they will not be forced to say anything about that topic. Moreover, if anyone becomes uncomfortable during the interview period, they may leave without being questioned. There will be no consequences for individuals electing not to participate or who choose to leave during the meeting. If the respondents (or those who remain) consent to participating in the interview, they will be asked to sign the consent form or to give their consent verbally. The team leader will remind the team members and interpreters on a regular basis, and review as necessary, the procedures for safeguarding and protecting the respondents who will participate in interviews and the procedures for obtaining informed consent. 3.2.6. Training Interpreters, Including on Human Subjects Protocols Each of the three interpreters will be assigned to one of the SMEs on the team to cover the sector for which the expert is responsible. The approved topic guides for that sector will be transmitted in advance of the evaluation team’s arrival so the interpreters can begin familiarizing themselves with the topic guides for the expert they are assigned to and begin translating each guide into the local language. Each translator will also receive the field calendar; a list of the types of individuals and groups that will be interviewed at the district, sub-county, and village levels; the form the team will use for informed consent procedures; and an introduction to be read at the beginning of each interview related to the protection of human subjects. These forms are included as Annex C. Final Performance Evaluation of the Northern Karamoja Growth, Health and Governance (GHG) Development Food Assistance Project – Final Evaluation Protocol 12 A day-long training and orientation session will be held on the first day of the evaluation team’s arrival in Northern Karamoja. Training and orientation topics will include the following:      Review and training on the use of each topic guide for which the interpreter is responsible. One-on-one discussions will be held between each interpreter and the SME to review each topic guide. The purpose is to clarify both the meaning and the intent of interview questions for each individual or group interviewed at the district, sub-county, and village levels; train the interpreter when to probe interview participants for specific information; and review procedures the interpreter should use to ensure that each participant selected for FGDs has ample opportunity to speak up. This is the time for interpreters to discuss any questions they have about the content of the questions. Review of the documents sent to the interpreters in advance, and their relationship to human subject protocols, ethics, and informed consent procedures Discussion on the roles of the interpreter, the SME, and the note-taker during scheduled interviews, and the procedures that will be used during the interview Protocols for beginning fieldwork at the district, sub-county, and village levels Review of fieldwork calendar 3.3. Data Analysis 3.3.1. Overview of Data Analysis Plan ICF will use mixed method analytical procedures to develop findings for each evaluation question. This involves integrating findings from the analysis of data from each data source, both qualitative and quantitative. The primary sources of data for the GHG performance evaluation are the set of responses from the qualitative component collected during the fieldwork phase and the PBS baseline and endline data. The two data sets will be analyzed independently using techniques appropriate to that data set. Findings based on the analysis of each data source will also be developed independently. The findings from the qualitative data are based on the analysis of interview response data for each evaluation question. Findings from the quantitative data are based on calculations of the statistical significance of changes in indicators from baseline to endline. Two procedures will be used for the integration of findings from each data source:   Findings from the PBS data are integrated with findings from the analysis of qualitative data to produce a more robust, evidence-based finding for each evaluation question. Findings from the qualitative data will be used to provide contextual information and to help interpret findings from the analysis of PBS data. Analysis of performance data from the final GHG IPTT will be conducted to develop findings for specific evaluation questions related to project outputs, the project TOC, and pathways used to achieve goals. As appropriate to the evaluation question, data from the IPTT will be integrated with qualitative findings as an additional source of evidence. The procedures for integrating findings from qualitative and quantitative data sources are based on a triangulation process entailing comparisons that examine the extent to which these separately developed findings corroborate each other. It is important to note that there are limits to triangulating Final Performance Evaluation of the Northern Karamoja Growth, Health and Governance (GHG) Development Food Assistance Project – Final Evaluation Protocol 13 findings from every data source. The findings from each data set must provide a source of evidence that closely relates to the evaluation question addressed by the qualitative data. If findings from the two major sources of data addressing the same topic do not corroborate or validate each other, findings from other data sources can still be used to (1) help resolve those differences, and (2) help interpret or to flesh out a more comprehensive overall finding with greater explanatory power. 3.3.2. Qualitative Data Coding Qualitative analysis will be supported by pre-coded data collection instruments developed for every interview. Codes have been developed to identify the SO for the focus of evaluation questions, location, administrative level, type of respondent, gender, and each evaluation question addressed during a given interview. A master set of data collection instruments covering each of the three SOs will be coded as appropriate for each team member. During fieldwork, field notes for interviews will be typed directly into these pre-coded templates. Table 5 provides information on the category and range of codes that will be applied to each data collection instrument. Table 5. Category and Type of Codes Category Codes by Type Sectors SO 1 (Livelihoods), SO 2 (MCHN/WASH), SO 3 (Conflict) Location Name of the implementation district Administrative Level Where the Interview Is Conducted D (District), C (Sub-County), and V (Village) Type of Respondent TB: targeted beneficiary group (e.g., youth, pregnant or lactating women, adult women from poor households) NB: non-beneficiary group (e.g., farmers not provided with vouchers for seed purchase, women not included in mother care groups) CBO: community-based organization (e.g., village peace committees, mother care groups, water user associations, male change agent group) VL: village leaders (e.g., elders, lead farmer, lead mother, traditional birth attendant) PS: private sector representative from a business or organized group (e.g., commodity buyer, input supplier, service providers, agricultural extension agent, chamber of commerce) PS: public sector representative or group (e.g., government officials, staff from a government funded health facility, police, judge) Gender M, F Evaluation Question Number from the GHG Performance Evaluation Scope of Work Uses the number of the actual question (e.g., 1.1, 1.2, 1.3) Procedures for Analyzing Qualitative Data The evaluation team will use standard qualitative techniques for data analysis to develop findings from responses to questions from the fieldwork. The procedures for analyzing the qualitative data are outlined in detail in the Evaluation Design Matrix found in Annex A. The following steps provide a summary of the steps taken to analyze the data. Final Performance Evaluation of the Northern Karamoja Growth, Health and Governance (GHG) Development Food Assistance Project – Final Evaluation Protocol 14 1. Analysis will be conducted based on locations across each district. District-specific analysis will not be conducted (at least not in the first stage). Using the data codes, SMEs will organize field notes by district, sub-counties, and villages across each district, and within those location types, by type of respondent (using prearranged titles) or respondent group, and by gender. 2. For each evaluation question, the SMEs will do the following: a. Identify the categories of responses per respondent or respondent group interviewed to collect data for that question across all three districts. b. Repeat the process across all six sub-counties. c. Repeat the process across all six villages. 3. In each category of response, each SME will identify distinct themes in those categories and note which themes are common across categories as well as any outlier themes at the district, sub-county, and village levels for each evaluation question. 4. The results of comparative thematic analysis will be used as the basis for developing findings for each question. The SMEs will conduct a process of triangulating findings using different types of comparative analysis as appropriate to each evaluation question. Comparative analysis methods will be used as follows: (1) for questions used only at the village level, for questions used only at the sub-county level, and for questions used only at the district level; (2) across categories of respondents at each administrative level; and (3) for questions that are the same or very similar for district, sub-county, and village interviews. For each type of comparative analysis, the SMEs will determine whether themes can be corroborated for each type of comparative analysis conducted per question. For themes that cannot be corroborated, each SME will try to resolve differences by re-examining interview data or by using other data sources (e.g., documents reviewed from the desk review). The SMEs will clearly identify those themes that cannot be corroborated. Team Organization for Qualitative Analysis: Roles and Responsibilities The roles and responsibilities of the team leader and SMEs are shown in Table 6. Table 6. Data Analysis Responsibilities by Team Member Team Member Data Analysis Responsibilities Team Leader Provides:  An organized schedule for data analysis and development of findings and conclusions with interim and final dates for completion  A copy of the evaluation design matrix to serve as a guide for analysis, which includes, for each evaluation question, the data sources and specific kinds of data from those sources that will be used in the analysis, and the analytical procedures  A review and discussion of the evaluation design matrix  A review of the coding system and its application to each data collection instrument  Technical guidance on organizing interview responses using the coding system  A copy of each data collection instrument per SO, fully coded  A review and guidance on mixed method analytical procedures to produce evidence￾based findings  Requirements and technical guidance for writing up findings  Technical review of findings for evaluation questions from each SME on a timely basis  Guidance on the development of conclusions that are directly linked to evidence-based findings Final Performance Evaluation of the Northern Karamoja Growth, Health and Governance (GHG) Development Food Assistance Project – Final Evaluation Protocol 15 Team Member Data Analysis Responsibilities  Assistance to SMEs when requested Responsible for the following:    Ensuring that the analysis and findings cover all aspects of each evaluation question Ensuring the technical quality of findings and conclusions for each evaluation question and that conclusions are directly linked to evidence-based findings Ensuring that analysis and the development of findings and conclusions remain on schedule SMEs for each SO Responsible for the following:  Organizing interview response data from notes taken in the field based on the coding system  Analyzing the data using standard qualitative data analysis methods to produce evidence￾based findings for each evaluation question that cover all aspects of that question  Using mixed method analytical procedures to integrate findings from the analysis of PBS baseline/endline data for core FFP indicators and custom indicators related to the SO, integrating findings from indicator performance data comparing base values and annual targets to LOA performance targets from the final IPTT, and integrating findings from document reviews as appropriate  Preparing a comprehensive overall finding citing evidence from sources that covers all aspects of the evaluation question  Revising findings based on the team leader’s technical review  Developing conclusions and recommendations based on each major finding  Revising conclusions and recommendations based on discussions with the team leader and the results of the team leader’s technical review 3.3.3. Procedures for Analyzing PBS Data Annex D, Summary of the PBS Methodology, contains an overview of the procedures used for the analysis of PBS data. 3.3.4. Integration of PBS Data Details on how the SMEs will integrate qualitative findings with findings from the analysis of PBS data are found in Annex A, the Evaluation Design Matrix. This section summarizes the basic approach. Each SME will use findings from the quantitative analysis of PBS data comparing baseline and endline data, and the significance of change, as appropriate to each evaluation question. PBS data are based on and can be used for findings from the qualitative analysis of village-level data. The analyst will determine whether those findings from the quantitative analysis provide an additional source of evidence for the findings established by the qualitative analysis and whether findings from the PBS data corroborate the qualitative finding for that question. This process can be followed for PBS findings disaggregated that match disaggregations used for village findings from the qualitative data as appropriate to the question. The team leader may request ICF PBS data analysts to use specific cross-tabs for analysis that will help strengthen findings established to date from the qualitative and quantitative data analysis, or to explain findings that cannot be corroborated. Findings from the qualitative data will also be used, as appropriate to the evaluation question, to interpret findings from the PBS data analysis. Final Performance Evaluation of the Northern Karamoja Growth, Health and Governance (GHG) Development Food Assistance Project – Final Evaluation Protocol 16 3.3.5. Integration of LOA Performance Data from IPTT and Information from Document Review The same processes will be used with the results from the final IPTT with the beneficiary-based indicators that are appropriate to evaluation questions related to the quality and acceptability of project outputs (question 3.2), adoption of behavioral changes, new practices (e.g., livestock management, soil management, child feeding practices) and new technologies (e.g., agricultural inputs, seeds for newly introduced crops, medicines or feed for livestock). Data from indicators measuring project outputs and sub-intermediate results will also be sued to examine pathways to achieve project and FFP goals and to assess the GHG TOC. Findings from secondary data based on document reviews will be used to help interpret findings from the PBS data as appropriate and, in some cases, selected materials maybe used to provide contextual information. Information from document reviews will also be used as appropriate to help resolve findings that do not corroborate each other. 3.3.6. Preparation of Comprehensive, Data-based Findings The analysts for each SO will prepare a comprehensive finding for each evaluation question based on the integration of findings from each data source, citing the evidence used for the finding. 3.3.7. Proposed Report Outline This section provides the proposed outline for the Final Performance Evaluation Report. Table of Contents Acronyms EXECUTIVE SUMMARY A. Summary of Evaluation Design and Methodology a. GHG Performance Evaluation Qualitative Design and Methods b. PBS Quantitative Design and Methods B. Findings by Strategic Objective a. SO 1 b. SO 2 c. SO 3 C. Conclusions by Strategic Objective a. SO 1 b. SO 2 c. SO 3 D. Principal Recommendations I. Introduction A. Identifies the DFAP that was evaluated and provides a summary description of the project: goal, duration, funding, implementing partners, major FFP components covered in the project as organized by SO (e.g., livelihood development, improved nutrition, etc.), involvement with the Government of Uganda Final Performance Evaluation of the Northern Karamoja Growth, Health and Governance (GHG) Development Food Assistance Project – Final Evaluation Protocol 17 B. Provides a summary description of the PBS, design for baseline and endline comparison, data collection instrument based on core FFP indicators and modules C. Purpose of the performance evaluation and the PBS (inform FFP learning and programming, use for design of future DFSAs, etc.) II. Evaluation Design and Methodology1 A. Design for the GHG Performance Evaluation a. Mixed Method Design i. Types and Sources of Data b. Purposive Sampling Strategy Sample Size and Unit of Analysis c. Data Location Sites d. Respondent Types e. Data Collection Methods B. Design for the baseline and endline PBS a. Sampling Methods and Sample Size b. Data Location Sites c. Household Questionnaire Design d. Data Collection Procedures C. Data Analysis Methodology a. Overall Approach (draws on mixed method analytical techniques from different types and sources of data) b. Qualitative Analysis Methods c. Quantitative Analysis Methods for PBS Data d. Integrating Qualitative and Quantitative Findings e. Integrating Data from Other Sources i. Mercy Corps Final IPTT ii. Secondary Data Sources D. Limitations of the Evaluation Design and Methodology III. Findings and Conclusions by Strategic Objective2 (the heart of the report) A. SO 1 Findings B. SO 1 Conclusions C. SO 2 Findings D. SO 2 Conclusions E. SO 3 Findings F. SO 3 Conclusions IV. Cross-Cutting Findings and Conclusions A. Gender B. Sustainability C. GHG DFAP Management D. Theory of Change and Pathways 1 This section can be a summary write-up of the design and methodology from the protocol report, updated with any changes. 2 Findings should cover all aspects of each evaluation question from the scope of work, plus any unanticipated findings not specifically related to the evaluation questions. Findings must cite the evidence. Final Performance Evaluation of the Northern Karamoja Growth, Health and Governance (GHG) Development Food Assistance Project – Final Evaluation Protocol 18 V. Lessons Learned and Best Practices by Strategic Objective and Cross-Cutting Topics A. SO 1 B. SO 2 C. SO 3 D. Gender E. Sustainability VI. Major Recommendations A. SO 1 B. SO 2 C. SO 3 D. Overall Project Recommendations ANNEXES A. Illustrations B. Tables C. Scope of Work D. Evaluation Questions E. Data Location Sites F. Respondent Types Interviewed by Location G. PBS Indicators H. Field Schedule for GHG Performance Evaluation I. Qualitative Data Collection Instruments J. PBS Household Questionnaire K. References Final Performance Evaluation of the Northern Karamoja Growth, Health and Governance (GHG) Development Food Assistance Project—Final Evaluation Protocol 19 ANNEX A. GHG EVALUATION DESIGN MATRIX Evaluation Questions and Sub-Questions Data Sources: Secondary Data from Document Reviews3 Data Sources: Qualitative Interview Data Data Sources: Quantitative Data from PBS Baseline and Endline Survey, IPTT Summary of Analytical Procedures to Produce Findings Q 1. To what extent has GHG met its defined goals, purposes, and outcomes? Assess the following: Q 1.1. Project final performance on value of indicators against targets set by partners and the key FFP indicators of depth of poverty, stunting, and undernutrition No fieldwork data will be used for this question PBS baseline 2012 data PBS endline 2018 data FY 2017 IPTT Step 1: Examine the following: (1) ICF data tables that combine estimated endline indicators values (2018) with baseline values (2012) for core FFP and project custom indicators, and significance of change (2) the FY 2017 IPTT, which compares the value of annual monitoring project indicators at baseline (2013) with 2017 LOA achievements, and provides the percentage of the target achieved Step 2: Prepare narrative findings based on Step 1 for core FFP and custom indicators, and IP project indicators. Step 3: Prepare conclusions based on those findings on the extent to which goal, purpose, and outcomes were met. Q 1.2. Targeting strategies and their contribution to achieving project goals, especially with regard to gender and reaching the most vulnerable MC grant narrative (2012) describing target groups targeting strategies, including interventions based Data from group interviews with targeted groups and individuals per SO Data from interviews with “key players” Findings from question 1.1 from PBS and IPTT data, with disaggregations based on gender and type of household Step 1 Describe GHG targeting strategies (e.g., M4P; market- based approaches; fostering linkages between beneficiaries, private sector, and government agencies and officials; vouchers; capacity development; food aid distribution) specific each SO to reach objectives and goals and target groups for those strategies, under each SO, 3 Documents listed in this column may not include all types of documents that each evaluation team member will use. Final Performance Evaluation of the Northern Karamoja Growth, Health and Governance (GHG) Development Food Assistance Project—Final Evaluation Protocol 20 Evaluation Questions and Sub-Questions Data Sources: Secondary Data from Document Reviews3 Data Sources: Qualitative Interview Data Data Sources: Quantitative Data from PBS Baseline and Endline Survey, IPTT Summary of Analytical Procedures to Produce Findings on targeting strategy for each SO ARRs for any documentation of changes in target groups, targeting strategies, and discussions in ARRs with respect to targeting strategy details as they have evolved since 2013 GHG Final Evaluation Report 2017 FIC, other assessments 2017 Performance Evaluation involved in each targeting strategy per SO Data from interviews with IP partners for each SO Example of targeting strategy and key players for SO 1 to illustrate the following:  Key players involved in market￾based approach using M4P model at the village, sub￾district, and district levels, including Smart Subsidy “program” to involve the most vulnerable groups  Key players in strategy to provide better services to female farmers, make services, loans, inputs more accessible: Retail especially related to gender and the most vulnerable. Step 2: Per SO, analyze qualitative data related to targeting strategies used to reach SOs and goals, and their effectiveness from perspective of targeted beneficiary groups (especially gender-based targeted and targeting of most vulnerable) to produce findings. Step 3: Per SO, analyze data related to targeting strategies from the perspective of key players involved to produce findings about effectiveness of those strategies to produce findings. Step 4: Per SO, triangulate findings between targeted beneficiary groups for each strategy and key players involved in those strategies to determine degree of convergence or divergence. Reexamine data to try to reconcile instances of divergence (continue this process under step 6 using quantitative findings). Prepare an overall finding to answer EQ 1.2 per targeting strategy. Step 5: Examine findings produced for EQ 1.1 from the analysis of PBS baseline/endline beneficiary data and significance of change on key FFP indicators disaggregated by sex and by type of household as appropriate to the indicators (e.g., food security, poverty, nutrition, stunting indicators). Final Performance Evaluation of the Northern Karamoja Growth, Health and Governance (GHG) Development Food Assistance Project—Final Evaluation Protocol 21 Evaluation Questions and Sub-Questions Data Sources: Secondary Data from Document Reviews3 Data Sources: Qualitative Interview Data Data Sources: Quantitative Data from PBS Baseline and Endline Survey, IPTT Summary of Analytical Procedures to Produce Findings Input suppliers, SACCOs (received female friendly training to improve services to women to encourage use of improved inputs, etc.) at sub-district level Step 6: Per SO, use IPPT findings produced under EQ 1.1 for selected indicators specific to each targeting strategy and the target beneficiary groups (e.g., adoption of inputs, use of loans, changing practices and behaviors); and secondarily, indicators associated with key players providing specific services, advice, and goods related to those strategies. Step 7: Mixed method analysis: For major strategies used for each SO described in Step 1, compare findings. Triangulate findings from Steps 3, 4, and 5. Do quantitative findings corroborate qualitative findings? Can divergent findings from qualitative data be resolved with quantitative data? Can qualitative data findings be used to explain quantitative findings from PBS data showing significant statistical declines or increases? For convergent findings that need further investigation, use findings from the literature review of secondary data sources (e.g., 2017 Final Evaluation Report, FIC documents, ARRs, FEWSNET) to try to reconcile findings that converge. Step 8: Per each SO, drawing on the results of Stage 6 analysis, prepare and explain an overall finding for each target strategy that links target strategies to intended beneficiary groups. Include divergent findings that cannot be resolved. Q 1.3. The effectiveness of interventions on the poorest individuals Same data sources, focusing on the poorest beneficiary Same as above, but using a subset of interview data with PBS 2018 Endline Report Use the same procedures as described under EQ 1.2 with following variations: Final Performance Evaluation of the Northern Karamoja Growth, Health and Governance (GHG) Development Food Assistance Project—Final Evaluation Protocol 22 Evaluation Questions and Sub-Questions Data Sources: Secondary Data from Document Reviews3 Data Sources: Qualitative Interview Data Data Sources: Quantitative Data from PBS Baseline and Endline Survey, IPTT Summary of Analytical Procedures to Produce Findings Same overall approach as EQ 1.2, but focused only on representatives from poorest individuals groups as defined by each SO poorest groups and individuals per SO FY 2017 IPPT Step 1: For each SO, use a subset of the qualitative analysis and findings related to the poorest beneficiary groups from Steps 2 (analysis of beneficiary interview responses) and 3 under EQ 1.2 to develop an overall finding covering each strategy and its interventions from the qualitative data. Include convergent findings that cannot be reconciled from interview data. Step 2: For each SO, use a subset of those findings from Step 4 under EQ 1.2 examining significance of change between 2013 and 2018 on indicators measuring poverty, disaggregated by male/female and type of households. Request bivariate analysis of PBS data comparing disaggregated findings on poverty indicators with findings measuring food security, nutrition, stunting (disaggregated by gender or type of household) to understand the relationships between poverty and status and well-being of poorest groups (whether based on gender or type of household). Step 3: Per SO, use findings from indicators related to specific strategies and interventions (adoption, behavior change, practices) disaggregated by gender, type of households that are identified as the poorest to identify the involvement of the poorest. Step 4: Request bivariate analysis of PBS data for those indicators on practice and use of key Final Performance Evaluation of the Northern Karamoja Growth, Health and Governance (GHG) Development Food Assistance Project—Final Evaluation Protocol 23 Evaluation Questions and Sub-Questions Data Sources: Secondary Data from Document Reviews3 Data Sources: Qualitative Interview Data Data Sources: Quantitative Data from PBS Baseline and Endline Survey, IPTT Summary of Analytical Procedures to Produce Findings interventions per SO with food security, poverty, nutrition, and stunting indicators by gender and type of household. Step 5. Use the procedures described under Step 6 for EQ 1.2 to develop a set of overall findings on the effect of strategies and interventions on the poorest individuals and groups per SO. Include divergent findings that cannot be resolved. Q 1.4. Based on evidence and the findings from Sub-questions 1–3, what has been the contribution of GHG to the United States Agency for International Development’s efforts to reduce food insecurity among chronically food insecure households? Q 2. For each technical sector, what are the strengths of and challenges in the project’s design, including theories of change, that influence the effectiveness of the interventions? Assess the following: 2.1. The relevance of the technical interventions, including food-for asset or cash-for-asset interventions, to achieve project outcomes, and discuss those findings in relation to the projects’ theories of change Information from MC grant narrative (2012) describing TOC, project design and proposed technical interventions per SO quarterly reports Information from ARRs related to project design Examples of design elements and relationship to interventions per SO: SO 1: M4P links farmers/pastoralists with private sector actors and markets POP: Power of the Purse Findings from comparison of PBS baseline and endline data and FY 2017 data on percentage of which LOA targets were met for annual monitoring indicators (from question 1.1 above) SMEs will apply their own expertise to assess the relevance and effectiveness of key technical interventions used to achieve project outcomes based on the project design as described in the MC grant proposal and in any changes made in interventions documented in ARRs; and then conduct the following analytical steps by using evidence to strengthen (or bring into question) their own assessment. Final Performance Evaluation of the Northern Karamoja Growth, Health and Governance (GHG) Development Food Assistance Project—Final Evaluation Protocol 24 Evaluation Questions and Sub-Questions Data Sources: Secondary Data from Document Reviews3 Data Sources: Qualitative Interview Data Data Sources: Quantitative Data from PBS Baseline and Endline Survey, IPTT Summary of Analytical Procedures to Produce Findings elements governing interventions for each SO, including changes that have been made, related findings from the 2017 Final Evaluation, and other relevant sector assessments SO 2: 1,000-day approach Data from group interviews and KIIs on design elements related to specific interventions under each SO; participants will vary by SO. - composition of participants will vary by SO (e.g., beneficiaries at village level, government officials and private sector actors) Interviews with representatives from each IP in the MC consortium Step 1: Examine findings from the analysis of PBS and IPPT data on indicators that provide measures of the project technical interventions from output and intermediate result data in relationship to project outcome level findings (actual LOA performance versus targets). Step 2: Apply findings from the qualitative analysis of field interviews to obtain perspectives of effectiveness of interventions from beneficiaries, government officials, and private sector actors engaged in interventions related to project outcomes for each SO. Triangulate the findings of effectiveness from each group’s perspective to determine the extent to which they corroborate each other. Reexamine the qualitative analysis from each group to try to resolve findings that cannot be corroborated. Document any unintended consequences identified through this analysis. Step 3: Compare the findings from Steps 1 and 2 with statements contained in ARRs and the findings and conclusions documented in the GHG Final Evaluation Report (2017) and any other sector assessments and evaluations relevant to technical interventions implemented for each SO. Try to resolve uncorroborated findings from Step 2. Determine the extent to which findings from all sources corroborate each other. For findings that do not corroborate each other, reexamine the analysis from the qualitative data or the findings and conclusions from the GHG Final Evaluation Final Performance Evaluation of the Northern Karamoja Growth, Health and Governance (GHG) Development Food Assistance Project—Final Evaluation Protocol 25 Evaluation Questions and Sub-Questions Data Sources: Secondary Data from Document Reviews3 Data Sources: Qualitative Interview Data Data Sources: Quantitative Data from PBS Baseline and Endline Survey, IPTT Summary of Analytical Procedures to Produce Findings Report to try to resolve differences. Document uncorroborated findings that could not be resolved. Step 4: For each SO, apply the results of the analysis from Steps 1–3 to develop a set of findings that include the strengths and weaknesses of the project design on the effectiveness of each major technical interventions to achieve program outcomes. Include findings that cannot be corroborated, as well as any unintended consequences identified during these analytical procedures. Step 7: Use the findings from the steps above to address how they relate to the overall project TOC. Do these findings support the TOC? Do the findings support each pathway linking lower to higher results to a greater or lesser extent? Describe the results of this assessment. 2.2. The plausibility of pathways and the determinants of achieving the key outcomes, highlighting any unforeseen pathways leading to unintended positive or negative consequences of the projects MC approved grant proposal (2012), GHG 2017 Final Evaluation Report, barrier analysis reports, FIC sector assessments Same as listed under EQ 2.1 PBS data: Comparison of baseline data (2012) with endline (2018) data and significance of change for indicators used to measure key outcomes Step 1: For each SO, use findings from the document reviews to identify and document the pathways and key determinants of achieving key outcomes. Step 2: Compare these pathways to the technical interventions and targeting strategies used to achieve key outcomes under each SO identified under the analytical procedures for addressing EQ 1.2 and EQ 2.1. Do those interventions and targeting strategies reflect the pathways articulated to achieve key outcomes? Final Performance Evaluation of the Northern Karamoja Growth, Health and Governance (GHG) Development Food Assistance Project—Final Evaluation Protocol 26 Evaluation Questions and Sub-Questions Data Sources: Secondary Data from Document Reviews3 Data Sources: Qualitative Interview Data Data Sources: Quantitative Data from PBS Baseline and Endline Survey, IPTT Summary of Analytical Procedures to Produce Findings Step 3: Identify the key determinants of achieving key program outcomes for each SO articulated in the MC grant proposal (2012) and from barrier analysis documents. Assess the extent to which they were incorporated in the pathways and technical interventions used to achieve key outcomes. Step 4: Prepare findings for each SO on the plausibility of the pathways used to achieve key outcomes by comparing the results from Steps 1–3 above with the findings from Steps 1, 6, and 7 under E.Q. 2.1 and the findings from Step 8 under EQ 1.2. Step 5: Identify and document any additional key determinants to achieving each outcome from this analysis, and additional or changed pathways incorporated during the LOA per SO, and any unintended consequences, both negative and positive. Q3: In each technical sector, what are the strengths of and challenges to the effectiveness of the interventions’ implementation? 3.1. The effectiveness and relevance of the technical interventions, including food-for asset and cash-for￾asset interventions, to achieve project outputs of greater or lesser quality, and discuss those findings in MC approved grant proposal 2012 ARRs from 2013– 2017 and selected quarterly reports Sector assessments and evaluations Data from KIIs and FGDs with individuals and groups at the community, sub￾county, and district levels associated in any way with project outputs under each SO Final IPTT comparing baseline to endline values versus LOA targets for indicators measuring project outputs Step 1: For each SO, use findings from document reviews to identify and document the technical interventions used to achieve project outputs. Step 2: Analyze qualitative findings on the effectiveness and relevance of technical interventions for key project outputs per SO, and the quality of those outputs from interviews with stakeholders at each administrative level (e.g., Final Performance Evaluation of the Northern Karamoja Growth, Health and Governance (GHG) Development Food Assistance Project—Final Evaluation Protocol 27 Evaluation Questions and Sub-Questions Data Sources: Secondary Data from Document Reviews3 Data Sources: Qualitative Interview Data Data Sources: Quantitative Data from PBS Baseline and Endline Survey, IPTT Summary of Analytical Procedures to Produce Findings relation to the projects’ theories of change relevant to project outputs associated with key outcomes GHG Final Evaluation Report 2017 community beneficiaries, and from community￾based organizations, government offices, private sector businesses) that are involved in any role or function associated with technical approaches. Triangulate responses from stakeholders on the effectiveness and relevance of the technical approach in terms of achieving those outputs to compare perspectives and determine the extent to which opinions can be corroborated. Use procedures (described under EQs 1 and 2) to try to resolve or explain uncorroborated points of view. Note: This analysis will also include stakeholder opinions about implementation processes and approaches used for technical interventions to develop findings relevant to EQ 3.2. Step 3: Assess effectiveness and relevance of the technical interventions from the analysis of quantitative data in the final IPTT. For each SO, document the extent to which indicators measuring project outputs have met their LOP targets as a measure of effectiveness, and indirectly, of relevance. For those project outputs that have not met their LOP targets, one possible reason for underachievement is that the interventions used may not be as relevant as hypothesized to achieve that output. It should be noted, however, that the implementation approach or process used in the technical intervention may have affected the degree of project output achievement, as may contextual Final Performance Evaluation of the Northern Karamoja Growth, Health and Governance (GHG) Development Food Assistance Project—Final Evaluation Protocol 28 Evaluation Questions and Sub-Questions Data Sources: Secondary Data from Document Reviews3 Data Sources: Qualitative Interview Data Data Sources: Quantitative Data from PBS Baseline and Endline Survey, IPTT Summary of Analytical Procedures to Produce Findings factors. Use the findings from EQ 3.2 as another measure of the effectiveness Step 4: For each SO, document results of literature review discussing effectiveness of technical interventions to produce project outputs and the quality of those outputs. Step 5: Triangulate the results from Steps 2–4 to determine the extent to which findings from each data source (documents, field interviews, IPTT data) on the effectiveness and relevance of project outputs corroborate each other. Use procedures to try to resolve findings from the three sources that cannot be corroborated. Include an examination of the findings that assess how the implementation method or approach used in each intervention, and the context in which the interventions are being implemented, may have affected the effectiveness of technical interventions. This set of findings should be reexamined to help resolve uncorroborated findings on the effectiveness of technical approaches. Step 6: Assess the quality of project outputs based on SME’s judgement incorporating the results of the analysis of qualitative data referring to quality of those outputs and the quality of the implementation approach and methods used to produce those outputs. Final Performance Evaluation of the Northern Karamoja Growth, Health and Governance (GHG) Development Food Assistance Project—Final Evaluation Protocol 29 Evaluation Questions and Sub-Questions Data Sources: Secondary Data from Document Reviews3 Data Sources: Qualitative Interview Data Data Sources: Quantitative Data from PBS Baseline and Endline Survey, IPTT Summary of Analytical Procedures to Produce Findings Step 7: Develop a set of findings per SO for each project output based on the analytical results from Steps 5 and 6 above. Step 8: Discuss the findings on the relevance and effectiveness of project outputs from Step 5 in light of the TOC. 3.2. Factors in the implementation approach and methods, and the context associated with greater or lesser effectiveness in producing outputs of higher or lower quality Same as above, focusing on the approaches and methods taken to implement the interventions meant to produce outputs and contextual information that can be discerned from documents. FEWS NET documents and assessments and assessments of conflict and security should be reviewed for additional contextual information. Data from KIIs and FGDs with individuals and groups at the community, sub￾county, and district levels associated in any way with project outputs under each SO Step 1: Use findings from the document review to identify and document the implementation approaches and methods related to each technical intervention documented under Step 1 for E.Q. 3.1 Step 2: Document the context during the time period when project outputs were being implemented (political, security, hungry season, or following harvest, weather events, conflicts, etc.) Step 3: Use the subset of findings from EQ 3.1 Step 6 on the quality of each project output per SO and align them with the implementation approach and methods used to implement the technical approach for producing those same outputs. Step 3: Use the subset of findings from Step 2 on the implementation context and from Step 3 associating findings on quality of project outputs with the implementation methods and approach used to produce that output; compare them with the findings from EQ 3.3 on the acceptability of interventions and implementation processes used for those project outputs. Final Performance Evaluation of the Northern Karamoja Growth, Health and Governance (GHG) Development Food Assistance Project—Final Evaluation Protocol 30 Evaluation Questions and Sub-Questions Data Sources: Secondary Data from Document Reviews3 Data Sources: Qualitative Interview Data Data Sources: Quantitative Data from PBS Baseline and Endline Survey, IPTT Summary of Analytical Procedures to Produce Findings Step 4: Use the results of this comparison to identify factors that can plausibly be associated with producing project outputs of greater and lesser quality for each SO. Q. 3.3. The interventions and implementation processes deemed more or less acceptable to members of the target communities Data from FGDs and KIIs conducted with groups and individuals interviewed at the community level This question will be answered by using findings from EQ 4.3. Acceptability of interventions and implementation processes will be incorporated as part of the assessment to respond to EQ 4.3. Q4: Which project outcomes are likely to be sustained? 4.1. The quality and functionality of systems, processes, and institutional arrangements established and strengthened independently by the projects, as well as in collaboration with the private sector, Government of Uganda, community organizations, and research organizations to achieve project outcomes and sustainability Literature review of all sources describing these systems and institutional arrangements describing each “system” established or strengthened involved in a specific strategy and its interventions to create a result. The literature review must include a review of relevant documents that discuss these systems as part of the GHG exit strategy to ensure sustainability Per SO, interviews with one or two representatives from each part of the “system” based on their responsibilities, including coordination, leadership, institutional agreements, management, production, oversight, quality control, etc. This will include representatives from organized community groups playing a direct role in a specific system to achieve certain PBS data comparing significance of change between baseline (2012) and endline (2018) indicator values Data from 2017 IPTT The assessment of the quality and functionality of systems, processes, and institutional arrangements will be based on (1) standards (either formal or informal) established by GHG as measures of quality and functionality; and (2) the expertise of each SME. Step 1: For each SO: (1) document the systems, processes, and institutional arrangements established and strengthened to achieve certain results by the project alone, as well as with other stakeholders involved in a collaborative process, including collaborating partners from private sector, Government of Uganda, etc.; and (2) document measures of quality and functionality established by GHG or in coordination with government and private sector partners. Step 2: For each SO per system, analyze interview data from each group of involved actors and players for each part of the system. Develop Final Performance Evaluation of the Northern Karamoja Growth, Health and Governance (GHG) Development Food Assistance Project—Final Evaluation Protocol 31 Evaluation Questions and Sub-Questions Data Sources: Secondary Data from Document Reviews3 Data Sources: Qualitative Interview Data Data Sources: Quantitative Data from PBS Baseline and Endline Survey, IPTT Summary of Analytical Procedures to Produce Findings of beneficiary outcomes. Sources include: exit strategy, quarterly reports and ARRs, final evaluation (2017) specific assessments related to each SO (e.g., FIC assessment of the development and strengthening of the livestock market). Relevant Government of Uganda documents, etc. Literature review explaining Government of Uganda interventions, private sector contributions, and research assessing these systems and arrangements defined results under each SO. Systems, institutional arrangements, processes involved in a defined result cover actors that play a role in making that system work across a village, sub-county, and district. Accordingly, for each system per SO, interviews will be conducted at the village, sub-county, and district levels with key actors who are involved. Interviews will focus on how the system and its arrangements and processes work to produce certain results, their quality and functionality, from the perspective of involved actors at each administrative level. Interviews will also include findings based on interview responses from each “group” from each administrative level on their perspectives of quality, functionality, and sustainability. Include issues brought up by each group that represent problems with functionality and quality, as well as those identified as being especially effective. Step 3: For each SO per system: Triangulate findings from each group (Step 2) to determine degree of corroboration and divergence of perspectives and opinions. Develop an overall finding for each system per SO. Step 4: For each SO per system, identify which FFP indicators and which project indicators are relevant to examining results on specific beneficiary populations related to a specific system, as well as indicators that directly or indirectly measure functionality, quality, and effectiveness of that system and its processes and institutional arrangements. Step 5: Drawing on results of Step 4, for indicators pertinent to each system, use findings from EQ 1.1 on performance and extent to which goals, objectives, and outcomes are met. Step 6: For each SO per system, triangulate findings from the analysis of the qualitative data (Steps 2 and 3) with quantitative findings from Step 5. Final Performance Evaluation of the Northern Karamoja Growth, Health and Governance (GHG) Development Food Assistance Project—Final Evaluation Protocol 32 Evaluation Questions and Sub-Questions Data Sources: Secondary Data from Document Reviews3 Data Sources: Qualitative Interview Data Data Sources: Quantitative Data from PBS Baseline and Endline Survey, IPTT Summary of Analytical Procedures to Produce Findings responses on specific factors and challenges that affect quality, functionality, and sustainability. Step 7: Use similar procedures described under EQ 1.2, Step 7 to prepare a set of overall findings for each system per SO. 4.2. Whether the necessary resources and capacity strengthening will exist to sustain service providers, and the likelihood that service provision will continue after the project ends ARRs and budget allocation data GHG Final Evaluation Report 2017 Relevant sector assessments, evaluations, reports Interview data from fieldwork at district, sub-county, and community levels with services providers and with representatives of any institutions, organizations, groups, government offices, business, and CBOs involved in either coordination, regulation, oversight, quality control, delivery, maintenance of those services Interviews with Mercy Corps and relevant IPs Step 1: Identify all services and service providers under each SO at district, sub-county, and community levels and all offices, institutions, organizations, groups involved in the organization, delivery, maintenance of those services, including any CBOs and end users of those services. Step 2: For each service and service provider, identify the resources and their sources arranged to sustain that service. Step 3: For each service and service provider, and any organization, government office, institution, CBO, and business involved in the organization, delivery, oversight/regulation, quality control, maintenance of those services, identify type of capacity development provided meant to (purposefully or not) sustain that service and service provider. Step 4: Validate outcome of Steps 1–3 with interview data related to each service and service provider. Step 5: Triangulate responses related to sustaining the service from the end user (beneficiaries), the service provider, representatives from other Final Performance Evaluation of the Northern Karamoja Growth, Health and Governance (GHG) Development Food Assistance Project—Final Evaluation Protocol 33 Evaluation Questions and Sub-Questions Data Sources: Secondary Data from Document Reviews3 Data Sources: Qualitative Interview Data Data Sources: Quantitative Data from PBS Baseline and Endline Survey, IPTT Summary of Analytical Procedures to Produce Findings businesses, organizations, offices, etc., involved in that service at the district, sub-county, and community levels. Step 6: Compare responses from the step above with findings developed for EQ 4.1 on the quality and functionality of systems, processes, and institutional arrangements that are specifically related to those services and service providers to determine the extent to which some of those findings corroborate (or do not) the findings from interview data. Step 5: Go back to the data and analysis conducted in earlier steps under EQs 4.1 and 4.2 to try to resolve findings that cannot be corroborated. Step 6: Cross-check findings from Step 5 with findings from document reviews and from interviews with IPs as an additional means of corroborating or resolving findings that cannot be corroborated. This step may also result in a set of findings that cannot be corroborated. Document remaining findings that cannot be resolved. Step 7: SMEs for each SO apply their own assessment based on the results of the steps above to develop an overall finding for each of those services with respect to the likelihood those services will be continued. 4.3. Communities’ perceptions on the quality, Interview data from KIIs and FGDs Step 1: Use results from Step 1 under EQ 4.2 identifying each service provided per SO. Use Final Performance Evaluation of the Northern Karamoja Growth, Health and Governance (GHG) Development Food Assistance Project—Final Evaluation Protocol 34 Evaluation Questions and Sub-Questions Data Sources: Secondary Data from Document Reviews3 Data Sources: Qualitative Interview Data Data Sources: Quantitative Data from PBS Baseline and Endline Survey, IPTT Summary of Analytical Procedures to Produce Findings frequency, effectiveness of the implementation processes and services provided conducted at the community level on perceptions related to implementation processes and services provided, and to EQ 4.4 on motivation to demand and pay or invest time for services and perceptions on sustainability of those services results from EQ 3.1 documenting implementation processes for producing project outputs in each SO, and document reviews to identify implementation approaches used for organizing and developing systems, institutional arrangements, etc., identified under EQ 4.1 used to produce results at the intermediate result level per SO. Step 2: Analyze responses from FGDs with agropastoralists, members of mother care groups (MCGs), water user committees, lead mothers and other village residents interviewed about their perceptions on the effectiveness of implementation approaches. Step 3: Same as above, but with focus on perceptions of the quality of services provided. Step 4: For each SO, develop sets of separate findings from the analytical results from Step 2 and Step 3. 4.4. What is the motivation of the community and beneficiaries to demand and pay (or invest time) for the services, and communities’ perceptions on sustainability of the services provided by the project? ARRs for documentation of payment and demand for services by communities and beneficiaries Sector assessment and evaluation reports covering project services and Subset of interview data from EQ 4.3 focusing on beneficiaries of services and any individuals and groups involved in any way with those services (e.g., water user associations, health unit management Step 1: Use findings from EQ 4.3 on services and perceptions of sustainability of services for each SO. Integrate, as appropriate, documentation from ARRs, sector assessments, and the GHG 2017 Final Evaluation related to demand or services and willingness to pay or invest time in for those services. Step 2: Develop a separate set of findings for each SO that have plausible explanatory power to explain motivation among individuals, groups of people, and types of households. Final Performance Evaluation of the Northern Karamoja Growth, Health and Governance (GHG) Development Food Assistance Project—Final Evaluation Protocol 35 Evaluation Questions and Sub-Questions Data Sources: Secondary Data from Document Reviews3 Data Sources: Qualitative Interview Data Data Sources: Quantitative Data from PBS Baseline and Endline Survey, IPTT Summary of Analytical Procedures to Produce Findings sustainability of services GHG Final Evaluation Report 2017 committee members, lead mothers, community animal health workers, lead farmers, etc.) for services provided under each SO Step 3: Use findings from E.Q. 4.3 related to community perceptions of the quality, frequency and effectiveness of services per SO, and findings from Step 2, to prepare findings on community perceptions of the sustainability of those services (per SO). 4.5. The extent to which the projects leveraged other USG and non-USG investments to achieve sustained outcomes as identified in the theories of change ARRs and quarterly reports to identify other USG and non￾USG players and stakeholders associated with project outcomes under each SO; discussions about the role each entity played to achieve sustained outcomes Other GHG documentation that describes how they leveraged investments from other USD and non￾USG entities Project budget data and any other relevant documents that showing line Interviews with IPs to confirm identified USG and non-USG investments leveraged for SOs and high-level intermediate results related to the TOC listed under Step 1 of analytical procedures, and to discuss the role they played to achieve sustained outcomes Interview data from KIIs and FGDs with government officials and commercial businesses at the sub￾county and district levels that play a role in achieving and sustaining those outcomes Step 1: Use the findings from the literature review to identify other USG and non-USG investments involved in project outcomes and their sustainability and the roles they played. Step 2: Per each outcome per SO, prepare a table that includes other USG and non-USG investors identified in Step 1, the roles they played to achieve outcomes identified in the TOC, and any specific activities that contributed to the sustainability of those outcomes described in documents reviewed for this question, noting sources of information. Step 3: Provide evidence of verification of role and contribution to the sustainability of those outcomes described in GHG documents and by interviews with IPs, with line item budget data, the findings from the GHG Final Evaluation Report 2017, and with the findings from the analysis of interview data from government officials and commercial businesses. Include this information in the table created in Step 2. Final Performance Evaluation of the Northern Karamoja Growth, Health and Governance (GHG) Development Food Assistance Project—Final Evaluation Protocol 36 Evaluation Questions and Sub-Questions Data Sources: Secondary Data from Document Reviews3 Data Sources: Qualitative Interview Data Data Sources: Quantitative Data from PBS Baseline and Endline Survey, IPTT Summary of Analytical Procedures to Produce Findings item contributions from other USG partners and non￾USG partners GHG Final Evaluation Report 2017 Note: The GHG evaluation team will not have time to interview representatives from other USG investors or with representatives from non-USG investors. Step 4: SMEs will review and assess the results recorded in the table developed under Steps 2 and 3 to prepare a set of findings on the extent to which GHG leveraged investments from other USG and non-USG entities for each outcome identified in the TOC and to achieve the sustainability of that outcome. 4.6. Evidence of enhanced linkages with other service providers ARRs and selected quarterly reports Exit and sustainability strategies Sector assessments and evaluations related to service providers associated with each SO GHG Final Evaluation Report 2017 Interview data from KIIs and FGDs with service providers associated with each SO on the capacity development they received, the effectiveness and benefits of linkages they have, and any changes in how the linkages are organized and maintained with other service providers organized by GHG to achieve specific outcomes at the community and beneficiary levels (e.g., linkages between input suppliers, linkages between health care Final IPTT data on indicators related to service providers, capacity development of service providers (Note: This may not be possible unless data are disaggregated sufficiently to use any quantitative data for evidence.) Step 1: Identify and document findings from the literature describing linkages between service providers used to achieve specific outcomes per SO at the community and beneficiary levels and the arrangements between service providers developed by GHG. Step 2: Add information from the literature review on any actions taken during the LOP to enhance those linkages as a means of improving project outcomes. Supplement with evidence from any relevant indicator data from the final IPTT. Step 3: Conduct qualitative analysis of interview data from each service provider involved in some kind of linkage with other service providers to achieve a specific outcome under each SO on their opinions and perceptions on the effectiveness and sustainability of those linkages as well from any actions taken to enhance those linkages implemented by the project or by service providers involved in that linkage. Final Performance Evaluation of the Northern Karamoja Growth, Health and Governance (GHG) Development Food Assistance Project—Final Evaluation Protocol 37 Evaluation Questions and Sub-Questions Data Sources: Secondary Data from Document Reviews3 Data Sources: Qualitative Interview Data Data Sources: Quantitative Data from PBS Baseline and Endline Survey, IPTT Summary of Analytical Procedures to Produce Findings providers at the village, sub-county, and district levels) Step 4: Triangulate the findings of the analysis per SO from each service provider involved in the link to compare perspectives and determine the extent to which findings corroborate each other. Try to resolve findings that do not corroborate each other by reexamining data from the literature review and from interviews with each service provider. Step 5: Prepare a set of findings for each SO related to linkages of service providers on the extent to which those linkages have been enhanced, drawing on evidence from Step 4. Q5: What are the key lessons learned and best practices that should inform future projects in the country? 5.1. What are the lessons, best practices, and approaches that should be considered in designing future food and nutrition security project? Lessons learned, best practices, etc., related to GHG design features identified by respondents at the district, sub-county, and village levels Responses to this question will draw from the analysis and findings from EQs 1–4, and in particular, EQs 1 and 2, and will include those lessons learned and best practices identified by field work respondents. This process will be conducted for each SO. 5.2. What are the lessons, best practices, and approaches that should be considered in implementing and implementation approaches for future food and nutrition security projects? Lessons learned, best practices, etc., for implementation approaches identified by respondents at the district, sub-county, and village levels Responses to this question will draw from the analysis and findings from EQs 1–4, but in particular, EQ 3 and EQ 4, sub-questions 1–4, and will include those lessons learned and best practices identified by fieldwork respondents. This process will be conducted for each SO. 5.3. What are the risks to the effective implementation of these SME knowledge of risks combined with literature review Risks identified through the analysis of data from selected Responses to this question will draw from EQ 3, EQ 4, sub-questions 1–4, and from the response to EQ 5.2 based on risks identified from secondary Final Performance Evaluation of the Northern Karamoja Growth, Health and Governance (GHG) Development Food Assistance Project—Final Evaluation Protocol 38 Evaluation Questions and Sub-Questions Data Sources: Secondary Data from Document Reviews3 Data Sources: Qualitative Interview Data Data Sources: Quantitative Data from PBS Baseline and Endline Survey, IPTT Summary of Analytical Procedures to Produce Findings approaches to support project achievements? Include risk assessments from FIC, FEWSNET, Crisis, other sources as found KIIs with representatives from each group (village formed group, specific government officials from related offices and agencies, and private sector actors) and qualitative data. This process will be conducted for each SO. 5.4. Based on the lessons learned, how can potential unintended negative consequences be minimized and positive consequences be captured systematically? Unintended consequences, positive and negative, identified in ARRs, selected assessments and evaluations, and the GHG Final Evaluation 2017 Unintended consequences will be identified by respondents at the district, sub-county, and village levels and from interviews conducted with IPs Responses to this question will draw from the responses to EQs 5.1 through 5.3 and from consequences identified by fieldwork respondents and IPs, and those identified from secondary data sources. Key: ARR=annual results report; CBO=community-based organization; EQ=evaluation question; FEWS NET=Famine Early Warning Systems Network; FFP=Food for Peace; FGD=focus group discussion; FIC=Feinstein International Center; GHG=Growth, Health, and Governance; IP=implementing partner; IPTT=Indicator Performance Tracking Table; KII=key informant interview; LOA=life of activity; M4P= Making Markets Work for the Poor; MC=Mercy Corps; PBS=population-based survey; SACCO=;Savings and Credit Cooperative; SME=subject matter expert; SO=strategic objective; TOC=theory of change; USG=U.S. Government Final Performance Evaluation of the Northern Karamoja Growth, Health and Governance (GHG) Development Food Assistance Project—Final Evaluation Protocol 39 ANNEX B. INFORMED CONSENT Consent Forms A one-page consent form will be used for each interview. Consent can be verbal or by signature. The note taker will record the number of participants that verbally gave their consent to participate in the interview or the number who provided consent by signature. The consent form that will be used for village FGDs is presented below. UGANDA PERFORMANCE EVALUATION – GHG CONSENT FORM FOR RESPONDANTS PARTICIPATING IN FOCUS GROUP INTERVIEWS: VILLAGES CONSENT TO BE INTERVIEWED Date District Name of ICF Interviewer I have listened to the ICF Interviewer describe the purpose of this discussion and how confidentiality will be protected. I consent to be interviewed. Number of persons that verbally agreed to be interviewed: Consent Procedures and Consent Statement The consent procedures will be followed at the beginning of each focus group discussion, small group interview, key informant interview, and in-depth interview. The interpreter or SME will read the script for the consent statement out loud, ask the respondent/s if they consent to be interviewed, and allow time for those respondents who choose not to stay to leave without questions. The interpreter or SME will record the date, location and a count of those respondents who consent to be interviewed. Welcome and Introduction The interpreter welcomes participants or the individual and thanks them for agreeing to participate. The interpreter begins by introducing her/himself, the SME, and note taker (if different from the SME). The SME and note taker will greet the people assembled (or the individual who will be interviewed). The interpreter does not ask for the names of any of the people assembled. Final Performance Evaluation of the Northern Karamoja Growth, Health and Governance (GHG) Development Food Assistance Project—Final Evaluation Protocol 40 He/she states the purpose of the overall evaluation and the purpose of this discussion/or interview. Script: Before we begin this discussion, I want to explain a few things about your participation. First: We will be asking many questions. It is important for you to know right now that we will not try to persuade you to answer any questions that you don’t want to. You have the right not to answer any question. We will not ask you to explain why you don’t want to answer the question. You are also free to leave the discussion at any time without explanation. Second: We will be recording this discussion and taking notes so that we will not forget anything important said here today. But we will not record your names. No one outside of this group of people/room will know what you said. The notes we take are the property of X (name of SME) and will not be given to any other person. The recording of this interview will be erased after we listen to the discussion. Do you have any questions? Please tell me if you are still willing to participate in this discussion. Anyone who does not want to participate after hearing what I just said is free to leave now. End of Script Final Performance Evaluation of the Northern Karamoja Growth, Health and Governance (GHG) Development Food Assistance Project—Final Evaluation Protocol 41 ANNEX C. DRAFT FINAL TRAVEL ITINERARY Sunday Monday Tuesday Wednesday Thursday Friday Saturday 9/23 9/24 9/25 9/26 9/27 9/28 9/29 Pat Vondal and Anna Afferri arrive in Kampala Sunday evening Team meeting Interviews with Mercy Corps staff in Kampala as available Meet with Logistics Coordinator MAF 7 a.m. flight to District Kotido Day-long team meeting with interpreters in Kotido Overnight in Kotido Depart for Abim— begin interviews in District 1 Drive to sub-county Morulem to conduct interviews Overnight in Abim Return to Morulem to complete interviews as necessary Drive to Aremo East Village - begin interviews Overnight in Abim Return to Village of Aremo East to complete interviews Overnight in Abim Depart for sub-county Abim Town County to conduct interviews Drive to village of Atwilo—begin interviews Overnight in Abim 9/30 10/1 10/2 10/3 10/4 10/5 10/6 Return to village x to complete interviews Overnight in Kotido Begin interviews in Kotido— District 2 Drive to sub-county Rengen—begin interviews Overnight in Kotido Return to Rengen to complete interviews Drivel to Chaicaon Village—begin interviews Overnight in Kotido Return to Chaicon to complete interviews Drive to sub-county Nakapelimoru—begin interviews Overnight in Kotido Return to Nakapelimoru to complete interviews Drive to Longelep Village—begin interviews Overnight in Kotido Return to Longelep to complete interviews Depart for district of Kaabong for overnight Begin interviews in Kaabong— District 3 Drive to sub-county Loleliabegin interviews Overnight in Kaabong 10/7 10/8 10/9 10/10 10/11 10/12 10/13 Return to Lolelia complete interviews Travel to Chumakod Village to begin interviews Return to Chumakod to complete interviews Overnight in Kaabong Travel to sub-county Sidok to conduct interviews Travel to Village Longar to begin interviews Return to Longar to complete village interviews Overnight in Kaabong Day-long discussion and review of what the team learned across districts per SO— prepare out-briefing materials Fly to Kampala Mission out-briefing Team departs Note: This schedule is subject to change based on any new information regarding dates, etc. Final Performance Evaluation of the Northern Karamoja Growth, Health and Governance (GHG) Development Food Assistance Project—Final Evaluation Protocol 42 ANNEX D. SUMMARY OF POPULATION-BASED SURVEY METHODOLOGY The questionnaires for the population-based survey were developed based on the core Food for Peace (FFP) and resilience indicators. All questionnaire modules followed FFP and Feed the Future guidelines, as described in the FFP Indicators Handbook (April 2015) and the Feed the Future Indicator Handbook (September 2016).4 The ICF team revised the questionnaire from the 2013 baseline survey in Uganda, and gathered information before, during, and after the baseline planning workshop (January 2018) to tailor the questionnaires to the context of Uganda. The resilience module was contextualized by Tango International based on prior work in Uganda. The questionnaire consists of separate modules covering the following topics:              Module A: Household identification and informed consent Module B: Household roster Module C: Household food security Module CC: Mobility, Local Government Responsiveness and Poverty Probability Index (PPI) Module D: Children’s nutrition and health Module E: Women’s nutrition and health Module F: Water, sanitation, and hygiene Module G: Agriculture Module H: Poverty Module J: Gender—Cash Module K: Gender—Maternal and Child Health and Nutrition (MCHN) Module L: Gender—Household Decision-Making, Access to Credit and Group Participation Module R: Resilience The questionnaire was translated into three local languages (Karamojong, Pokot, and Lethur). The total time for completing the survey in each household was approximately two to three hours, depending on the size of the household. Data were collected with tablets using computer-assisted personal interviewing (CAPI). Tablets were loaded with a CSPro data entry application developed at ICF for FFP surveys and tailored to fit the Uganda questionnaire. All data were entered directly into the tablets and edited while interviewing in the field. Data from the sampled 3,360 households for the joint baseline/endline were collected over approximately 35 days by 18 teams, each consisting of six field team members (one supervisor, four interviewers, and an anthropometry specialist), coordinated by five field coordinators and supported by two information technology specialists, for 115 the total field personnel for the survey. Quality was ensured through a strong focus on training field staff and monitoring data collection. Monitoring was made possible through the use of CAPI data collection, which allows for real-time editing of data, frequent uploading of collected data, continuous data quality review, and correction of field staff 4 Available at https://feedthefuture.gov/sites/default/files/resource/files/Feed_the_Future_Indicator_Handbook_Sept2016.pdf Final Performance Evaluation of the Northern Karamoja Growth, Health and Governance (GHG) Development Food Assistance Project—Final Evaluation Protocol 43 behavior as data collection proceeds. Field teams were expected to upload collected data from completed clusters at minimum on a weekly basis. For the end-line data analysis, ICF generated estimates for all FFP and custom indicators that were collected at baseline, along with additional analyses to explore relationships and plausible determinants for key outcome indicators. A statistical comparison of baseline (2013) and endline (2018) estimates was conducted to determine population-level change over time. Additional population-based data analyses are currently ongoing (September 2018). Final Performance Evaluation of the Northern Karamoja Growth, Health and Governance (GHG) Development Food Assistance Project—Final Evaluation Protocol 44 ANNEX E. PBS ENDLINE INDICATORS Indicator Disaggregation Level FOOD SECURITY 1. Average Household Dietary Diversity Score None POVERTY 2. Per capita expenditures (as a proxy for income) of USG-assisted areas GHT 3. Prevalence of Poverty: Percent of people living on less than $1.25/day 2005 PPP GHT 4. Depth of Poverty: Mean percent shortfall relative to the $1.25/day poverty line GHT WATER, SANITATION, AND HYGIENE 5. Percentage of households using an improved drinking water source None 6. Percentage of households using an improved sanitation facility 7. Percent of households with soap and water at a handwashing station commonly used by family members None AGRICULTURE 8. Percentage of farmers who used financial services (savings, agricultural credit, and/or agricultural insurance) in the past 12 months Sex 9. Percentage of farmers who practiced the value chain activities promoted by the project in the past 12 months Sex 10. Percentage of farmers who used at least [a project-defined minimum] sustainable agriculture (crop, livestock and natural resource management) practices and/or technologies in the past 12 months Sex, type of practice 11. Percentage of farmers who used improved storage practices in the past 12 months Sex WOMEN’S HEALTH AND NUTRITION 12. Prevalence of underweight (BMI < 18.5) women of reproductive age None 13. Percent of births receiving at least four antenatal care (ANC) visits during pregnancy None CHILDREN’S HEALTH AND NUTRITION 14. Prevalence of underweight children (WHZ < -2) children under five (0-59 months) Sex 15. Prevalence of stunted children (HAZ < -2) children under five (0-59 months) Sex 16. Prevalence of wasted children (WHZ < -2) children under five (0-59 months) 17. Percentage of children under age five who had diarrhea in the past two weeks Sex 18. Percentage of children under five years old with diarrhea treated with oral rehydration therapy Sex 19. Prevalence of exclusive breastfeeding of children under six months of age Sex 20. Prevalence of children 6-23 months receiving a minimum acceptable diet Sex CUSTOM INDICATORS 21. Percentage of respondents reporting increased movement in areas that were previously not accessible due to insecurity None 22. Percentage of households with access to a sanitation facility – not necessarily improved None 23. Average number of crops produced per farmer in the past 12 months None 24. Percentage of farmers adopting farmer managed natural regeneration practices in the past 12 months None 25. Percentage of livestock owners accessing government or private sector vet care in the past 12 months None BL=baseline, BMI=body mass index, EL=endline, GHT= gendered household type, HAZ=height-for-age z-score, PPP= purchasing power parity, USG=U.S. Government, WHZ= weight-for-height z-score * Pending confirmation of feasibility with existing data Final Performance Evaluation of the Northern Karamoja Growth, Health and Governance (GHG) Development Food Assistance Project—Final Evaluation Protocol 45 ANNEX F. INTERVIEW AND FOCUS GUIDES INTERVIEW GUIDES FOR SO 1 FOCUS GROUP DISCUSSION: MICRO AND SMALL BUSINESS OWNERS at VILLAGE LEVEL (OFF-FARM LIVLIEHOODS) INTERVIEW DATA Facilitator Recorder Interpreter Date of Interview District Sub-County Parish Village PARTICIPANT INFORMATION Number and Type/Gender of Participants: Please check M adult – married M adult - widowed F adult – married F adult - widowed M youth F youth Number of Participants Interviewed Interview Tape Recorded (Please check) Yes No Number of Participants Consenting to the Interview Verbally Number of Participants Consenting to the Interview with Signature 1. What is your business? 2. How long have you operated the business? 3. What prompted you to select this business? 4. What were your sources of income before this business? (or your other sources of income now) 5. How did each of the following activities you participated in help you to set up and continue running your business? o o o o o Business development training Training in product handling and presentation Mentoring Member of a VSLA Member of a SACCO Final Performance Evaluation of the Northern Karamoja Growth, Health and Governance (GHG) Development Food Assistance Project—Final Evaluation Protocol 46 o o o o o o o o Member of a business forum 6. Did/do you take out loans for your business (or at start-up)? Where did you apply for your loan? 7. Did/do you face any challenges in: Applying for a loan? Please explain Repaying the loan? Please explain 8. Do you have a savings account? 9. Where do you buy (or collect) the products you use for your business (or the items you use to make the product you sell)? 10. What kinds of people buy your products? Who do you sell to? 11. Are the poorer members of your community or this area able to afford your products/services and products? 12. How do your customers know about your products and services? People see you at the market Word-of-mouth Radio Other-mouth, radio, see you at 13. Has the business generated profits since it was established? 14. Are you able to continue to running your business in this community successfully on your own? (Please give reasons for your answer). 15. How do you intend to keep your business going? 16. What are the main challenges to the running of your business? 17. How has having this business changed your life and that of your family? 18. How has this business benefitted your community? Final Performance Evaluation of the Northern Karamoja Growth, Health and Governance (GHG) Development Food Assistance Project—Final Evaluation Protocol 47 District Level Focus Group Discussion – Members of a Business Forum INTERVIEW DATA Facilitator Recorder Interpreter Date of Interview District Sub-County Parish Village PARTICIPANT INFORMATION Number and Type and Gender of Participants: Please check M from HHs with adult M&F F from HH with adult M&F F from HH with adult F only M Youth F Youth Total Number of Participants Interviewed Interview Tape Recorded (Please check) Yes No Number of Participants Consenting to the Interview Verbally Number of Participants Consenting to the Interview with Signature 1. When was this Business Forum formed? 2. Who are its members and what kind of businesses are represented? 3. What markets do you sell your products in? 4. Number of men and women are on the forum? 5. What activities does the business forum do? 6. How often do you meet and what prompts the meeting? 7. What are the benefits of being a member? 8. Which of the program activities have been most important for helping to set up and run for members of this forum and other for small businesses in your district? a. Support to form and build the capacity of business forums b. Matching grant to hold meetings c. business development services d. Linkages between national and regional buyers and local producers and traders e. Market information f. Rehabilitation of feeder roads g. Rehabilitation of markets 9. How have women, youth and men benefitted from the activities of business forum? 10. How easy is it to secure a loan for your businesses? 11. How reasonable are the repayment terms? 12. Are women treated the same as men when applying for loans? Final Performance Evaluation of the Northern Karamoja Growth, Health and Governance (GHG) Development Food Assistance Project—Final Evaluation Protocol 48 13. What is the relationship between this business forum and the Chamber of Commerce? How do you benefit from the relationship? 14. Have any new relationships between national and regional buyers and yourselves formed as a result of forum activities. a. If yes, give examples of the new relationships. 15. Are there instances where buyers from elsewhere have trained you to improve production, handling, and profitability as a result of relationships built on the forum? a. If yes, please give examples of such instances where buyers trained to improve production, handling, and profitability. 16. Is the local government involved in functioning of the forum? 17. How are connections with the local government helping to meet the objectives of the forum? (4.6) 18. What are the major challenges individuals’ business owners’ face? Has this forum helped to overcome all of them? What else is needed? 19. What measures can be taken to keep the forum effective for business growth, building linkages and markets? (5.2) 20. What lessons have the members of this forum learned that you are applying today? (5.1) Final Performance Evaluation of the Northern Karamoja Growth, Health and Governance (GHG) Development Food Assistance Project—Final Evaluation Protocol 49 District Level Focus Group Discussion – Chamber of Commerce INTERVIEW DATA Facilitator Recorder Interpreter Date of Interview District Sub-County Parish Village PARTICIPANT INFORMATION Number and Type and Gender of Participants: Please check (add type) M: F: Youth: Male Youth: Female Total Number of Participants Interviewed Interview Tape Recorded (Please check) Yes No Number of Participants Consenting to the Interview Verbally Number of Participants Consenting to the Interview with Signature 1. When was the Chamber of Commerce constituted? 2. Who are its members? Please describe. 3. How many members are men? Women? 4. What is the importance of the Chamber of Commerce? 5. How important was the support you received to set up and run this Chamber? E.g. a. Support to set up the Chamber b. Technical Assistance for operational guidelines to run the Chamber, leadership training, building membership, rules and regulations governing the Chamber’s activities, capacity building in advocacy skills, other. c. Matching Grants to convene meetings d. Other Final Performance Evaluation of the Northern Karamoja Growth, Health and Governance (GHG) Development Food Assistance Project—Final Evaluation Protocol 50 6. Do you think the right people/ organizations are represented or are there important actors missing? Please explain 7. How often does the Chamber meet? 8. How are the costs of your meetings met? a. What is members’ contribution to cost of meetings? b. What is the project’s contribution to cost of meetings? 9. What is the relationship between the Chamber and local business forums? a. What is the benefit of that relationship for this Chamber, for members of business forums? 10. Has the Chamber identified needs in regard to developing the local business environment?1.4 a. If yes which are these needs? b. How were the needs identified? Who was involved? 11. What actions (advocacy efforts) were taken concerning the identified needs and priorities? 12. What were the outcomes of the Chamber’s advocacy efforts on the needs identified in question 9? 13. What can members do differently to improve the effectiveness of the Chamber? (5.1) 14. What lessons has the Chamber learned on setting up and running an effective Chamber of Commerce? (5.2) On how to support Business Forums so their members will succeed? (5.2) Final Performance Evaluation of the Northern Karamoja Growth, Health and Governance (GHG) Development Food Assistance Project—Final Evaluation Protocol 51 KII with Manager or Leader of a Village Savings and Loan Association INTERVIEW DATA Facilitator Recorder Interpreter Date of Interview District Sub-County Parish Village PARTICIPANT INFORMATION Number and Type and Gender of Participants: Please check (add type) M: F: Total Number of Participants Interviewed Interview Tape Recorded (Please check) Yes No Number of Participants Consenting to the Interview Verbally Number of Participants Consenting to the Interview with Signature 1. When was this VSLA established? 2. What are the activities of this VSLA besides savings and lending? 3. Which kind of training and support have you received that were most useful for setting up (if relevant) and helping you to operate the VSLA effectively? a. How has this training/support been useful? b. What training or support has been less useful? 4. Which kind of training and support have you received that were most useful for helping you improve services to members of the VSLA? a. How has this training/support been useful? b. What training or support has been less useful? 5. Has the VSLA been able to retain its members? a. Why or why not? Please explain. 6. Has this VSLA been able to attract new members? Final Performance Evaluation of the Northern Karamoja Growth, Health and Governance (GHG) Development Food Assistance Project—Final Evaluation Protocol 52 a. Why or why not? Please explain. 7. Does this VSLA usually have sufficient savings to support member demand for loans? 8. Are members always able to repay VSLA loans within the specified time frame? 9. Do you provide any services to non-members? a. If yes, what services do you provide? b. How is this VSLA able to provide services to non-members? 10. Have you observed changes in the lives of your members as a result of the VSLA activities? a. If yes, what changes have you observed? (describe for men & women) 11. What are most frequent reasons for members come to this VSLA? (probe for men and women) a. Withdraw of savings b. Borrowing 12. What are the most frequent reasons that members want to borrow money? 13. Have you observed changes in reasons for withdraw of savings? a. If yes, what are those changes? b. Do you know why these changes have occurred? 14. Have you observed changes in reasons for borrowing? a. If yes, what are those changes? b. Do you know why these changes have occurred? Final Performance Evaluation of the Northern Karamoja Growth, Health and Governance (GHG) Development Food Assistance Project—Final Evaluation Protocol 53 KII with District Manager of a MFI or SACCO INTERVIEW DATA Facilitator Recorder Interpreter Date of Interview District Sub-County Parish Village PARTICIPANT INFORMATION Number and Type and Gender of Participants: Please check (add type) M: F: Total Number of Participants Interviewed Interview Tape Recorded (Please check) Yes No Number of Participants Consenting to the Interview Verbally Number of Participants Consenting to the Interview with Signature 1. When was this SACCO established? 2. What are the activities of the SACCO besides saving and credit? 3. How do you provide services to the poorest communities in the district? 4. What services do you provide for men? For women? 5. Which of the following areas of support or training this SACCO received to date have been most useful (and explain why)? a. Loan guarantees b. Tools, equipment c. Training in credit management d. Training in financial management e. Training in governance f. Training to increase number of female members g. Security h. Other 6. Which of those areas of training or support have been most useful for helping you extend services to: a. The poorest communities in the district b. Women c. Youth 7. How is this SACCO sustained other than through interest from member loans? 8. Have you observed changes in the numbers of men? Of women? Please describe these changes: a. In Joining the SACCO b. In Borrowing from the SACCO? 9. What are members’ reasons for borrowing? Final Performance Evaluation of the Northern Karamoja Growth, Health and Governance (GHG) Development Food Assistance Project—Final Evaluation Protocol 54 a. Are the different for women members come to borrow money compared to men? b. If yes, please describe these changes 10. Have you noticed any change in members’ reasons for borrowing? a. If yes, describe the changes b. Are those changes in reasons different for women members compared to men? c. If yes, please describe those difference? 11. How have the activities of the SACCO helped to promote: a. Farmer crop production b. Livestock production activities c. Small businesses in rural communities 12. Is the SACCO linked to commercial banks? 13. If yes, how and to which bank? 14. How does the linkage to a commercial bank help to improve services to your members? Final Performance Evaluation of the Northern Karamoja Growth, Health and Governance (GHG) Development Food Assistance Project—Final Evaluation Protocol 55 KII with District Manager of a Commercial Bank INTERVIEW DATA Facilitator Recorder Interpreter Date of Interview District Sub-County Parish Village PARTICIPANT INFORMATION Number and Type and Gender of Participants: Please check (add type) M: F: Total Number of Participants Interviewed Interview Tape Recorded (Please check) Yes No Number of Participants Consenting to the Interview Verbally Number of Participants Consenting to the Interview with Signature 1. When was this bank established in this district? 2. Which of the following areas of support or training this bank received to date have been most useful (and explain why)? a. Loan guarantee b. Capacity building to improve operations c. Other 3. Which has been most useful for helping you extend services to the poorest communities in the district? 4. Have you observed any changes in the business environment in the district (number and variety of small businesses emerging, women involved in set up of businesses)? a. If yes, explain your answer b. If no, why? 5. If yes, has this translated into a critical mass of small business that is able to transact with a commercial bank? Of women? a. Yes (how?) b. No Why? 6. Have requests for loans increased or decreased overall in the last six months? 7. Is there an increase in the number of women owned businesses applying for loans? 8. What percentage of loan requests are from women? 9. What is the nature of the relationship you have with SACCOs? 10. How has this benefitted: a. This bank b. The SACCOs? 11. What more should be done to encourage the emergence of Northern Karamojong businesses? Final Performance Evaluation of the Northern Karamoja Growth, Health and Governance (GHG) Development Food Assistance Project—Final Evaluation Protocol 56 KII with Officer Responsible for Business Programming at Voice of Karamoja Kotido District INTERVIEW DATA Facilitator Recorder Interpreter Date of Interview District: Sub-County Parish Village PARTICIPANT INFORMATION Number and Type and Gender of Participants: Please check (add type) M: F: Total Number of Participants Interviewed Interview Tape Recorded (Please check) Yes No Number of Participants Consenting to the Interview Verbally Number of Participants Consenting to the Interview with Signature 1. How many years have you broadcast business-oriented programs in this district? 2. Please describe the type of business-oriented programs you have for: a. For farmers and herders b. Traders c. Others (please specify) 3. What kind of information is shared through these programs? 4. Which kinds of support or training you have received that have been most helpful for: a. Developing skills b. Developing networks c. Conducting research to produce and deliver business programing to farmers and traders in Karamoja? 5. What response have you received from your audiences? 6. How do you obtain information on your audience’s opinion about the business programs? 7. What do you do with audience response and their opinions? Please give some examples? 8. Have your listeners increased since the initial years when you began programming? a. If yes, how do you think this happened? How do you know? b. What is the reason listeners have increased? 9. Has there been any change in the participation of your listeners? a. If yes, describe the change b. To what do you attribute the change 10. What is the difference between participation of men and women? 11. What is the difference between participation of the more rural and urban listeners? 12. Can you explain the contribution of your business programs to your listening audiences? How have your programs benefitted them (farmers/herders/traders)? Final Performance Evaluation of the Northern Karamoja Growth, Health and Governance (GHG) Development Food Assistance Project—Final Evaluation Protocol 57 a. How do you know your programs have benefitted your listeners? 13. Have you received sponsors for your business programs? Please describe. 14. What challenges have you faced: a. While developing business-oriented programing for farmers, herders and traders and building your listening audiences? b. To maintain these programs? 15. What lessons have you learned: a. That should be considered when initiating business programing? b. That you have applied to improving your programming? 16. Are other radio stations in Karamoja starting to provide business programs like yours? Final Performance Evaluation of the Northern Karamoja Growth, Health and Governance (GHG) Development Food Assistance Project—Final Evaluation Protocol 58 KII with COMMODITY TRADERS – DISTRICT LEVEL INTERVIEW DATA Facilitator Recorder Interpreter Date of Interview District Sub-County Parish Village PARTICIPANT INFORMATION Type/Gender of Participants: Please check M: F: Number of Participants Interviewed Interview Tape Recorded (Please check) Yes No Number of Participants Consenting to the Interview Verbally Number of Participants Consenting to the Interview that Provided their Signature 1. What commodities do you trade? 2. What is the current market for those commodities? a. Has it changed over the past three years or so? Please explain 3. How long have you been operating in this this district? 4. Is the number of commodity traders in the district increasing or reducing? (please give a reason for your answer) 5. Which of the following activities were you a beneficiary of? a. Supporting business to business linkage b. Networking meetings c. Grants to purchase & store produce for subsequent resale at reasonable prices in the lean season d. Training e. Mentoring f. Sharing rental cost of storage space g. Providing storage materials 6. How have these impacted (positively or negatively your business? 7. Which activities have been most useful for helping you improve your business? a. How have the interventions been useful? 8. Is there a change in the variety and volume of produce that your business has handled in the last two seasons? Describe the change and the reason for it. 9. How do you obtain the produce that you trade in this area? (Do you operate use farmer groups, agents or other strategy)? Final Performance Evaluation of the Northern Karamoja Growth, Health and Governance (GHG) Development Food Assistance Project—Final Evaluation Protocol 59 10. The program’s intention was that commodity traders buy early at low prices and store the commodities for later sale using an improved business model (high volume/low margin). This was to avoid very high prices during the lean season which contributing to high levels of food insecurity and under nutrition among children. In your view did this strategy work? (Please give a reason for your answer?) 11. How have your prices compared to other competitors the last two lean seasons? 12. Were you able to break even? 13. Where do you sell produce? Do you add value? 14. Has the construction/ rehabilitation of feeder roads affected your ability to buy or sell produce? 15. What other things / services/ conditions (not in place now) are needed to help your and other businesses operate successfully in this district? 16. Have you tried to communicate these issues or needs to the local government authorities? a. If yes what avenue did you use? b. What was the response? 17. What have you learned over the past three years that you are applying now in your sales strategy? Final Performance Evaluation of the Northern Karamoja Growth, Health and Governance (GHG) Development Food Assistance Project—Final Evaluation Protocol 60 GUIDE for KII WITH AGRICULTURE EXTENSION AGENTS INTERVIEW DATA Facilitator Recorder Interpreter Date of Interview District Sub-County Parish Village PARTICIPANT INFORMATION Type/Gender of Participant/s: Please check M: F: Number of Participants Interviewed Interview Tape Recorded (Please check) Yes No Number of Participants Consenting to the Interview Verbally Number of Participants Consenting to Interview with Signature 1. Which of the taught practices (listed below) have been best adopted by farmers and why? o o o o o o o o o Use of recommended inputs (seed, fertilizer, Adoption of new recommended crops Soil conservation (crop rotation etc.) Post-Harvest Handling and storage 2. Which ones were least adopted and why? 3. Did you observe any difference in practices adopted by male, female and youth? If yes, what are the differences and what is the reason for it? 4. How did you work with the lead farmers? 5. How effective were the demonstration gardens and lead farmers for strengthening trained concepts on crop production and improving adoption by program beneficiaries and non￾beneficiaries?. 3.2 6. Did the presence of input agents help to increase adoption of improved technologies? How did it help to increase adoption? How did you work with the input agents? 7. What was the role of the smart subsidies in increasing adoption of recommended practices (in particular use of seed and post-harvest & storage materials? 3.2 8. In your view did the program reach the most vulnerable farmers (widows, women in polygamous marriages, single mothers, and youth)? If yes, which approaches were most helpful for reaching them? 9. What changes (positive or negative) have you observed in the lives of farmers as a result of the above activities? Are the changes in the lives of male farmers different than of female farmers? 1.4 Final Performance Evaluation of the Northern Karamoja Growth, Health and Governance (GHG) Development Food Assistance Project—Final Evaluation Protocol 61 10. What changes (positive or negative) have you observed in the lives of farmers as a result of the other activities such as linkage to VSLAs and SACCOs, repair of feeder roads? 1.4 o o o o o o o o o o VSLAs SACCOs Repair of Feeder Roads Other Are the changes in the lives of male farmers (related to either VSLAs, SACCOs, repair of feeder roads) different than in the lives of female farmers? Are the changes in the lives of male farmers different than of female farmers? 11. How has the integration of gender messages in services improved women empowerment? Do more female farmers come to you for advice? Do you reach out to female farmers? If yes, with what result? 12. How did you work with the sub county agricultural officers and district agricultural office? What, if any, benefits were there from coordinating with the sub-county and district agricultural office? What were the challenges in coordinating with the sub-county and district agricultural offices? 13. Will you continue to provide the extension services in this community after the program? (Please give reasons for your answer). 4.2 14. If yes, what resources are available to enable you to keep providing advisory services to farmers? 15. What lessons have you learned that you are applying in your role as an agricultural extension agent now? 16. Do you find that farmers more willing to pay for your services outside of the project’s support? (4.4) Final Performance Evaluation of the Northern Karamoja Growth, Health and Governance (GHG) Development Food Assistance Project—Final Evaluation Protocol 62 KII GUIDE FOR LEAD FARMERS INTERVIEW DATA Facilitator Recorder Interpreter Date of Interview District Sub-County Parish Village PARTICIPANT INFORMATION Gender Male Female Number of Participants Interviewed 1 Interview Tape Recorded (Please check) Yes No Participant Consenting to the Interview Verbally Or Consent Form Signed 1. When did you start working as a lead farmer in your community?  How were you selected? 2. What services do you provide to farmers? 3. How do you provide services? 4. How effective was your work as a lead farmer in promoting farmers to adopt: a. New crop and soil management practices? b. Adopting the use of improved seeds and fertilizers? c. Willingness to pay for improved seeds and fertilizer 5. What are the main constraints farmers face in adopting new crop and soil management practices? 6. What are the main constraints farmers face in using improved seeds and fertilizers? 7. Are there any security problems that still affect household ability to plant and manage crop production, especially for women? a. If yes, what is the village doing to resolve security problems. 8. In addition to your work as a lead farmer working on a demonstration plot, which other approaches were most useful for: (probe for male change agents, mother care groups, savings groups, AEA training and guidance, capacity building of Input agents, smart subsidies, and gender messages e.g. “couples who make farming decisions and share farm work get more yields)?      Learning about crop and soil management practices? Adopting the use of improved seeds, fertilizers, etc.? Increasing the willingness of farmers to pay for improved seeds and fertilizer? Encouraging carrying out of taught practices Behavior change in how men and women work together in households? 9. Which ones were least useful and why? Final Performance Evaluation of the Northern Karamoja Growth, Health and Governance (GHG) Development Food Assistance Project—Final Evaluation Protocol 63 10. Are there differences in what practices were adopted between women (including widows), youth and men? 11. If, there are, what is the reason for these differences? 12. Which is the most significant change that you have observed in your village? 13. Do you have any success stories in how men and women in households are working better together? 14. For farmers who adopt new practices, do you know if any of them: a. Have increased the productivity of the crops they plant? b. Led to farmers planting new kinds of crops? 15. How do you work with the sub county agricultural agent? a. How beneficial is your working relationship with the agricultural agent? 16. Which aspects of your work as a lead farmer will you continue to do without assistance? 17. Which ones will be very difficult for you to continue and why? 18. Will you continue to work as a lead farmer? Please explain. 19. What could be done differently, or in addition to your work as a lead farmer, to encourage farmers to adopt new agricultural practices (soil and crop management) and improved seeds and fertilizers? Final Performance Evaluation of the Northern Karamoja Growth, Health and Governance (GHG) Development Food Assistance Project—Final Evaluation Protocol 64 KII GUIDE FOR KRALL LEADERS INTERVIEW DATA Facilitator Recorder Interpreter Date of Interview District Sub-County Parish Village PARTICIPANT INFORMATION Gender Male Female Number of Participants Interviewed 1 Interview Tape Recorded (Please check) Yes No Participant Consenting to the Interview Verbally Or Consent Form Signed 1. When did you begin as a krall leader? a. How were you selected? b. What are your specific roles and responsibilities? 2. How is the current market for livestock? a. Is it favorable for livestock owners? b. Has it changed over time (negative or positive) c. What accounts for the change in the livestock market? 3. How do you get news on livestock markets? 4. Who do you work with to collect this news? 5. How do you spread this news to livestock owners? 6. How did the training on collecting market news, exchange visit and the monthly airtime help you to do this work? 7. How have livestock owners responded to this news over time? 8. Is there a difference between how men, women and youth acted upon this news? 9. What has having this news changed for livestock farmers? 10. Has it affected any of these?: (please explain your answer) a. Number of animals being sold b. Health of animals being sold c. Prices at which animals are brought from farmers? 11. Has the building, fencing, cleaning of the Kotido slaughter house/ Kaabong livestock markets changed the buying and selling of animals in any way? (please explain your answer) 12. Has the change continued? 13. What are the main challenges to the doing your work? 14. How big a problem is livestock theft? a. Has it increased or decreased? Why/How Final Performance Evaluation of the Northern Karamoja Growth, Health and Governance (GHG) Development Food Assistance Project—Final Evaluation Protocol 65 15. Are there any issues or problems in obtaining water, pasturage and other resource requirements for livestock for the owners who maintain their livestock in this kraal? Please explain. a. Have any of these been addressed, and with what degree of success? 16. What livestock management practices have livestock owners adopted to: a. Prevent/cure disease b. Increase and maintain the health of their stock? 17. Have you noticed any change in the health (size) of animals that farmers are taking to the market? a. What is the reason for this change? 18. Which of all the approaches used were most useful for: (male change agents, savings groups, livestock health campaigns, feeding campaigns, capacity building of CAHWs and drug shops, smart subsidies, and gender messages e.g. “couples who make farming decisions and share farm work get more yields)? a. Learning about livestock management? b. Encouraging adoption of livestock management practices to prevent or cure disease, to increase the health and productivity of livestock? c. Changing how men and women work together in households? 19. Are there differences in what was adopted between the women (including widows), youth and men? 20. If, there are, what are these differences? a. What accounts for these differences? 21. What should be done differently to improve livestock health and marketing or in addition to what is being done currently? a. Have there been any changes over the past several years? 22. How did the set-up of businesses such as processing, hair salons, and mosquito net businesses changed lives for women, men and youth in the community? (We need to find out which IGAs were supported in the particular village?) Final Performance Evaluation of the Northern Karamoja Growth, Health and Governance (GHG) Development Food Assistance Project—Final Evaluation Protocol 66 KII GUIDE FOR TILLAGE / TRANSPORT SERVICE PROVIDERS INTERVIEW DATA Facilitator Recorder Interpreter Date of Interview District Sub-County Parish Village PARTICIPANT INFORMATION Number and Type and Gender of Participants: Please check M: Adult M: Youth Total Number of Participants Interviewed Interview Tape Recorded (Please check) Yes No Number of Participants Consenting to the Interview Verbally Number of Participants Consenting to the Interview with Signature 1. When did you start your tillage/ transport service? 2. What motivated you to start this business? 3. How did you obtain your plow or tractor? 4. How many customers do you have in this community? 5. Who are your most frequent customers? (probe for size of farms and differences between men and women) 6. Do you have enough customers to continue providing services? 7. What do you charge for your service? a. Must customers pay before you provide the service, right after the service, or when they have harvested their crop? 8. Are the poorest able to afford your service? 9. Where do buy spares? 10. How do you do handle maintenance and repairs? 11. Can you get these services whenever you need them? 12. Is the business profitable? 13. Do you intend to continue providing this service? 14. What are the main challenges to the operation of your work? 15. What can be done to improve the future of tillage/ transport services in our community/ district? Final Performance Evaluation of the Northern Karamoja Growth, Health and Governance (GHG) Development Food Assistance Project—Final Evaluation Protocol 67 FOCUS GROUP DISCUSSION GUIDE FOR VILLAGE LIVESTOCK KEEPERS Note: This guide can be used in agro-pastoralist and pastoralist villages INTERVIEW DATA Facilitator Recorder Interpreter Date of Interview District Sub-County Parish Village PARTICIPANT INFORMATION Number and Type and Gender of Participants: Please check M: HH with Male and Female Adults FL: HH with Male and Female Adults F: HH with Female Adults Only Youth: Male Youth: Female Total Number of Participants Interviewed Interview Tape Recorded (Please check) Yes No Number of Participants Consenting to the Interview Verbally Number of Participants Consenting to the Interview with Signature 1. What kind of animals do you keep? 2. NOTE: THIS QUESTION is ONLY for owners of herding animals: cattle/goats, etc. Do you herd your animals each day for pasture and water; or do you keep them at home and give them feed and water? o o o o o o o o o o If yes…keep them at home: Why do you keep them at home? If yes…herd them each day: Are you able to find good pasture and sources of water for your livestock year around? What challenges do you face in herding your animals? 3. Have you observed a change in the health of your animals in the past three years? If yes, please describe how their health has changed: (milk production, ability to reproduce, condition of coat, susceptibility to disease) To what do you attribute the observed change? 4. Have there been any increases or decreases in the death of your animals? What are the causes (in either increase or decrease of livestock death? 5. Have you increased the size of your herd in the past two years? Please explain your answer. 6. Which of the following activities did you participate in? Training in livestock management practices Training in application of livestock medicines from drug shops Participating in preventative herd health programs Participating in livestock feeding programs Final Performance Evaluation of the Northern Karamoja Growth, Health and Governance (GHG) Development Food Assistance Project—Final Evaluation Protocol 68 o o o o o o o o o o o o o o Periodic spraying and deworming of your animals 7. Which have been most useful for improving the health and productivity of your animals? (please give a reason for your answer) 8. Which activities have not been very useful? (please give a reason for your answer) 9. How important has the Community Animal Health Worker been for helping you to improve the health of your animals? 10. Do you pay for the services of the CAHW? 11. Is there a drug shop for livestock in your community? 12. Did you receive a subsidy for the purchase of drugs and services for spraying and deworming your animals? If yes: Did you continue to spray, deworm, and treat your animals even without the subsidy? If yes: How do you pay for the services? 13. Are you a member of a SACCO or VSLA? If yes, ask the following: What services do you use: savings, loans, or both? Have you been able to get loans in the amount you need when you apply? Have you been able to repay the loans you have taken out when they are due? 14. Do you give supplementary feed to your animals? (Napier grass, silage, hay, mineral block)? If yes, what is the reason you give your animals supplementary feed? Where do you get it from? 15. Which animal products do you sell? 16. Where do you sell (milk, animals etc.)? 17. Do you have a Kraal leader? What is the role of the Kraal Leader? 18. How do you get market information? o Do you use the market information for making decisions? Please explain how. 19. How have the following infrastructure developments made any difference to your livestock business? Give a reason for your answer Kaabong Komuria livestock market Kotido Town slaughter house Feeder road construction 20. Do you face any of the following problems as a livestock owner? Cattle theft Fights over pasture and water resources with other villages? Unsafety of areas where you herd your animals? 21. Have there been any improvements in solving these problems? Please explain. 22. What difference have you observed in your life, the life of your family members as a result of support you have received for improving the health and productivity of your animals, livestock markets, roads (etc.) (discuss differences for men, women, youth and children) Final Performance Evaluation of the Northern Karamoja Growth, Health and Governance (GHG) Development Food Assistance Project—Final Evaluation Protocol 69 FOCUS GROUP DISCUSSION GUIDE FOR VILLAGE POULTRY KEEPERS Note: This guide can be used in agricultural, agro-pastoralist and pastoralist villages INTERVIEW DATA Facilitator Recorder Interpreter Date of Interview District Sub-County Parish Village PARTICIPANT INFORMATION Number and Type and Gender of Participants: Please check F: HH with Male and Female Adults F: HH with Female Adults Only Youth: Female Total Number of Participants Interviewed Interview Tape Recorded (Please check) Yes No Number of Participants Consenting to the Interview Verbally Number of Participants Consenting to the Interview with Signature 1. What kind of chickens do you keep? 2. What is the size of your flock (both mature hens, roosters, and chicks) 3. When did you start keeping poultry? 4. What motivated you to start keeping poultry? 5. What is the importance of the poultry to your household? o o o o o o o o o Are they for home consumption or sale? What members of the household are given eggs to eat? (if any) 6. What kind of services or support have you received for your poultry? Poultry vaccination Poultry husbandry techniques Other (please specify) 7. How helpful are vaccines for improving the health and condition of your poultry: Please explain your answer. 8. Which specific poultry husbandry techniques have been the most useful for the health and condition of your poultry? For curing them? Please explain your answer. 9. QUESTIONS 8, 9 and 10 for FARMERS with EXOTIC BREEDS ONLY Where do you buy: The chicks Feed Equipment Other 10. Are they always available from this/these sources when you need them? 11. How are you able to afford the chicks, equipment and feed? Final Performance Evaluation of the Northern Karamoja Growth, Health and Governance (GHG) Development Food Assistance Project—Final Evaluation Protocol 70 12. Are you a member of a VSLA or SACCO? 13. If yes, how does your membership to the SACCO or VSLA help you in any way to purchase the chicks and the necessary inputs and materials? o o o o o o o o o Have you been able to repay the loans you have taken out by the date payment is due? Please explain. 14. Were you a beneficiary of the poultry vaccination program? 15. Did you continue to vaccinate your chickens even without the subsidy? Against which diseases do you vaccinate your poultry? When do you vaccinate? Who provided the vaccination services? Are these services always available when you need them? 16. What poultry products do you sell? Where do you sell them? 17. Have you earned profits from your poultry business? 18. What are the challenges you’ve faced in keeping poultry? Are you able to resolve any of these challenges? Will you continue keeping chickens? 19. QUESTION ONLY FOR FARMERS WITH EXOTIC BREEDS: Will you keep rearing this kind of chicken? Please give a reason for your answer? 20. What difference has keeping poultry made to your household and members of your household? Final Performance Evaluation of the Northern Karamoja Growth, Health and Governance (GHG) Development Food Assistance Project—Final Evaluation Protocol 71 INTERVIEW GUIDES FOR SO 2 GUIDE for KEY INFORMANT INTERVIEW with GHG IP WHALE MANAGERS in Kaabong, Kotido and Kampala INTERVIEW DATA Facilitator Recorder Interpreter Date of Interview District: Sub-County Parish Village PARTICIPANT INFORMATION Number and Type and Gender of Participants Interviewed Male: Female: Total Number of Participants Interview Tape Recorded (Please check) Yes No Number of Participants Consenting to the Interview 1. Please tell me about your role at the GHG program. What is it under the new FFP contract? 2. How was the partnership with Mercy Corps as the Prime of GHG? Access to Clean Water and Linkage to Disease 3. Which specific safe water access interventions can be linked to decreases in water-borne diseases among children under five? 4. Since the hand-pump were build/rehabilitated, what changes have there been in the communities served by WHAVE/ WV? a. Which challenges did you have encountered in order to handle with the GHG partnership requirements? 5. How the support you have provided has help to stop water-borne diseases spreading? 6. Taking about the boreholes and hand pumps. Were these interventions sufficient to increase safe water access to the beneficiaries? a. Comments on the water investments: challenges in procurement of spare parts, challenges in the maintenance of the water installations, water quality control, hand pump protection from animals, hygiene of the water pump, coordination with water committees, etc. Final Performance Evaluation of the Northern Karamoja Growth, Health and Governance (GHG) Development Food Assistance Project—Final Evaluation Protocol 72 Community-Led Total Sanitation (CLTS) 7. Were the CLTS interventions sufficient to increase community latrines construction, maintenance and utilization? a. Please comment on the following CLTS and OFD issues:       challenges in procurement construction material, challenges in the maintenance of the latrines, showers and hand washing station (tippy-tap), procurement of soap, Open Free Defecation award, and challenges of maintaining OFD status Effectiveness of social and behavioral communication approaches to change community practices Effectiveness of the organization of the community-based committees, etc. 8. WHAVE supported the task-force on CLTS between government, community-based organizations and other NGOs. How effective was this strategy in order to implement the CLTS approach? a. Do you think all parties will continue to work on the task force? b. Is it sustainable? Strategies to increase access and availability of clean water 9. What water interventions promoted by the program were acceptable by the beneficiaries? 10. How effective are water user associations? 11. Can you contrast the effectiveness of gender balanced water user associations and women only water user associations? 12. What are the perceptions of the communities about the quality of boreholes and hand pump promoted by WHAVE/WV? 13. According to final evaluation, GHG has the only program in which villagers actually pay for private sector water and sanitation services. What factors contribute to their disposition and motivation to pay for these services? a. Are the communities in which villagers are paying for these services continuing to pay to your knowledge? b. What have you learned from this success that you are applying under the current grant? 14. GHG have invested significant efforts in the creation of linkages with private water service providers and water user associations. How effective has this been for ensuring continual provision of clean water services? What have been the challenges to this approach? a. Do you think this approach is sustainable in the communities where you have already supported this approach? Why or why not? b. Are there some districts or parishes where this model has not worked very well? Why? 15. To what extent will your organization be able to continue service provision in the communities that were targeted by GHG? 16. What have you learned from implementing this approach in communities under the GHG project that you are applying under the current grant? 17. Can you give us a success story you have recorded since the hand pump were constructed/rehabilitated? Final Performance Evaluation of the Northern Karamoja Growth, Health and Governance (GHG) Development Food Assistance Project—Final Evaluation Protocol 73 GUIDE for KEY INFORMANT or SMALL GROUP INTERVIEW with the ABIM HAND PUMP MECHANICS ASSOCIATION INTERVIEW DATA Facilitator Recorder Interpreter Date of Interview District Sub-County Parish Village PARTICIPANT INFORMATION Number and Type of Participants Interviewed M: F: Total Number of Participants: Interview Tape Recorded (Please check) Yes No Number of Participants Consenting to the Interview Verbally Number of Participants that Signed a Consent Form 1. How long have you been providing hand pump services? 2. Did you receive any specific training/support from the GHG project? (Probe for business and organizational training and assistance on infrastructure construction). 3. Since the hand-pumps were build/rehabilitated, what changes have there been in the communities served by HPMA i. Which challenges have you encountered as a member of the partnership with sub￾district health assistants, the district water office, and community water association? ii. How effective is the coordination with sub-district health assistants and the district water office and community water associations in providing rehabilitated or new hand pumps and bore holes? In ensuring necessary maintenance of boreholes and hand pumps? 4. Have the services you provided to communities to date helped to stop water-borne diseases spreading any of those communities? Please explain. 5. Were the following activities sufficient to increase safe water access to the beneficiaries in the communities served to date: (read list) a. Addressing challenges in procurement of spare parts b. Addressing challenges in the maintenance of the water installations c. Water quality control d. Hand pump protection from animals e. Hygiene of the water pump f. Coordination with water committees, etc. 6. If these activities were not sufficient to increase safe water access in the communities served to date, what else must be done? Who should be involved? Final Performance Evaluation of the Northern Karamoja Growth, Health and Governance (GHG) Development Food Assistance Project—Final Evaluation Protocol 74 7. How acceptable are the new/rehabilitated hand pumps and bore hole to the communities you serviced? 8. Can you make any comments on how communities feel about the quality of boreholes and hand pump rehabilitated by HPMA? 9. According to final evaluation, GHG has the only program in which villagers actually pay for private sector water and sanitation services. What factors contribute to their disposition and motivation to pay for these services? a. Why? 10. To what extent will you be able to continue your role in ensuring safe water service in the communities you serve? a. What is needed for you to be able to continue in this role? Final Performance Evaluation of the Northern Karamoja Growth, Health and Governance (GHG) Development Food Assistance Project—Final Evaluation Protocol 75 GUIDE for KEY INFORMANT INTERVIEW with MAKERERE UNIVERSITY in Kampala INTERVIEW DATA Facilitator Recorder Interpreter Date of Interview District Sub-County Parish Village PARTICIPANT INFORMATION Number and Type of Participants Interviewed Positions of Participants represented at the interview: M: F: Total Number of Participants: Interview Tape Recorded (Please check) Yes No Number of Participants Consenting to the Interview Verbally Number of Participants that Signed a Consent Form 1. How effective was the partnership agreement signed with the health district in order to deploy medical student in the health facilities target by GHG program? b. What has worked well? c. What is still needed? d. How GHG have facilitate this agreement? 2. How has the placement of medical students impacted the performance of the district health facilities in Northern Karamoja in term of service provision? Please explain. 3. Have any of the students increased their medical knowledge and skills during their internships with district health facilities? Please explain. a. How have these students increased their medical knowledge and skills during the internship? (Probe for the transfer of knowledge and skills to local staff and interns; assistance in bridging temporary staffing gaps; encouraging students to consider relocating to Karamoja to work after graduation and; establishing a long-term partnership between Ugandan medical schools and district health authorities in Karamoja) 4. In your opinion, will the district health office will continue to support the deployment of medical student now GHG has ended? a. How will they continue to support this program? 5. Which challenges did you foresee in the deployment, retention and motivation of medical students in remote areas of Karamoja? a. What did you and other involved parties do to meet these challenges? Were they successful in improving selection, deployment, and the retention and motivation of medical students? Final Performance Evaluation of the Northern Karamoja Growth, Health and Governance (GHG) Development Food Assistance Project—Final Evaluation Protocol 76 6. What actions will be taken (or have been taken so far) to maintain the motivation of the students deployed in such remote areas? Final Performance Evaluation of the Northern Karamoja Growth, Health and Governance (GHG) Development Food Assistance Project—Final Evaluation Protocol 77 GUIDE for FOCUS GROUP INTERVIEW with WATER USER COMMITTEES MEMBERS in VILLAGES INTERVIEW DATA Facilitator Recorder Interpreter Date of Interview District: Sub-County Parish Village PARTICIPANT INFORMATION Number, Type and Gender of Participants Interviewed All Women Water User Committees F: Or Gender Mixed Water User Committees M: F: Total Number of Participants Interview Tape Recorded (Please check) Yes No Number of Participants Consenting to the Interview Verbally Number of Participants Consenting to the Interview with Signature 1. When this Water User Committee created? How many years ago? 2. How were you been selected to join the committee? 3. What activities is the committee responsible for? 4. Has your committee received any support (Probe for materials, training, supervision, assets from other NGOs, etc.)? a. What was the most useful kind of support? b. Where is more support needed 5. What have been the main benefits of participating in this committee? a. What would you like to change about this WASH committee? Partnership Questions 1. How is this committee partnership with the sub-county and district Water officers and assistants and bore hole/hand pump providers? a. What kind of partnership do you have with Village Health Workers? With health facility staff? With lead mothers for Mother Care Groups? 2. What are the benefits to your community from this partnership with these different groups and individuals? 3. Which specific interventions can be linked to decrease in water-borne diseases among children under five? Final Performance Evaluation of the Northern Karamoja Growth, Health and Governance (GHG) Development Food Assistance Project—Final Evaluation Protocol 78 4. How the support you have provided to you community has help to stop water-borne diseases spreading? 5. Since this WASH committee was formed, what changes have there been in your community about water, sanitation and hygiene? a. What were the main reasons for those changes? 6. How the communities have reached to the concept of Open Free Defecation versus Open Defecation? 7. Since the hand-pump were build/rehabilitated, what changes have there been in the communities? 8. Which challenges did you have encountered in order to handle with the WASH needs of your village? 9. Taking about the boreholes and hand pumps. Were the safe water interventions sufficient to increase safe water access to the beneficiaries? a. Comments on the water investments: challenges in procurement of spare parts, challenges in the maintenance of the water installations, water quality control, hand pump protection from animals, hygiene of the water pump, coordination with water committees, etc. 10. Taking about the Community-led Total Sanitation. Were these interventions sufficient to increase community latrines construction, maintenance and utilization? a. Comments on the CLTS and OFD: challenges in procurement construction material, challenges in the maintenance of the latrines, showers and hand washing station (tippy￾tap), procurement of soap, Open Free Defecation award, etc. 11. What WASH interventions promoted by GHG were more acceptable by the beneficiaries? 12. What are the perceptions of the communities about the quality of latrines promoted by CLTS approach? 13. Overall, to what extent do you feel that the support you received from GHG has prepared you to carry on with the committee activities? 14. According to final evaluation, GHG has the only program in which villagers actually pay for private sector water and sanitation services. What factors contribute to their disposition and motivation to pay for these services? Why? 15. To what extent will you be able to continue the activities of the WASH committees in your community? 16. GHG have invested significant efforts in the creation of linkages with water private service providers. (Probe for WHAVE and Abim HPMA). To what extent will your committee will be able to continue to cooperate with them? 17. Can you give us a success story you have recorded as WASH committee? Final Performance Evaluation of the Northern Karamoja Growth, Health and Governance (GHG) Development Food Assistance Project—Final Evaluation Protocol 79 GUIDE for FOCUS GROUP INTERVIEW with SCHOOL HYGIENE CLUBS MEMBERS in SCHOOLS INTERVIEW DATA Facilitator Recorder Interpreter Date of Interview District Sub-County Parish Village PARTICIPANT INFORMATION Number, Type, and Gender of Participants Interviewed M: F: Total Number of Participants Interview Tape Recorded (Please check) Yes No Number of Participants Consenting to the Interview Verbally Number of Participants Consenting to the Interview with Signatures 1. When was this Health Club created? How many years ago? 2. Why did you join the club? 3. What was necessary for you to do to join the club? Can anyone join? 4. What are the activities of your club? 5. Did members of the club receive training? (probe for the type of training provided) 6. Who provided the training? (Probe for GHG staff, patrons, school teachers, WASH committees, others) 7. What was the quality of the training members received? 8. How useful was the training to members to help you execute the club’s responsibilities and activities? Please explain. 9. What was the most useful support your club have received? (Probe: CHAST training, VIP latrines, handwashing stations, soap provision, etc.) 10. What have been the main benefits to you as individual members from participating in the club? a. Is there anything you would like to change about this club? 11. Which specific intervention implemented by your club are the most important to reduce diarrhea (running belly)? (Probe for handwashing with soap, latrines construction, safe water, protection of food from insects, body cleaning, etc.) 12. Since this club was created, what changes have there been in your school and household about safe water consumption, latrines use and hygiene? (Probe for increased school attendance, girls’ schools attendance, household behavior change on WASH, reduced health facility visits, etc.) a. What were the main reasons for those changes? Final Performance Evaluation of the Northern Karamoja Growth, Health and Governance (GHG) Development Food Assistance Project—Final Evaluation Protocol 80 13. What have been the main benefits of having latrines and handwashing stations or a hand￾pump in your school? a. And the main shortcoming? 14. Can you describe at least 3 critical moments when you should wash your hands? 15. What have been the biggest challenges at your school and home to adopt hygiene practices? 16. What is your motivation to continue to use latrines, to wash your hands and to drink safe water? 17. Can you give us a short story about what you have witnessed in your school since the latrines/handwashing/hand pump were constructed/rehabilitated in your school? Final Performance Evaluation of the Northern Karamoja Growth, Health and Governance (GHG) Development Food Assistance Project—Final Evaluation Protocol 81 GUIDE for FOCUS GROUP DISCUSSION with MALE CHANGE AGENTS in VILLAGES INTERVIEW DATA Facilitator Recorder Interpreter Date of Interview District Sub-County Parish Village PARTICIPANT INFORMATION Number, Gender and Type of Participants Interviewed M: Total Number of Participants Interview Tape Recorded (Please check) Yes No Number of Participants Consenting to the Interview Verbally Number of Participants Consenting to the Interview with Signature 1. When was the Male Change Agent Group in your community created? How many years ago? 2. How many men are in your group? - - - - Has there been a lot of turnover in membership? 3. How were you selected? 4. Please explain the type of training you received to act as a Male Change Agent. A. Did the training equip you sufficiently to fill this role in the community? B.What was the quality of the training you received? Could it be improved in any way? C.Are there any other areas of training your group thinks are necessary? 5. What sensitization topics/community dialogues does your group focus on? (Probe for): a. Women and children health - ANC/Institutional deliveries (accompany the women to the HF) b. Shared workload c. Inclusion of women on decision-making d. Helping women in household chores and with children child care e. Wife beating f. Family Planning g. Farming h. Gender equality i. WASH (latrines and Tippy-Tap construction; boreholes maintenance fee sensitization ) j. Peer-to-peer training and mentorship Final Performance Evaluation of the Northern Karamoja Growth, Health and Governance (GHG) Development Food Assistance Project—Final Evaluation Protocol 82 6. What is the attitude of this community toward your Male Change Agent group and the work you do? Are you accepted? Has this changed over time? - - - - - - - Do you believe that your group will have a permanent role in the community? 7. Do you work with any committees or groups in the community? (Probe for Mother Care Groups, Lead Mothers, Water User Associations, Village Peace Groups, Youth Groups, etc.) A. Please explain your role when you work with these committees. What do you hope to achieve by your involvement? B. How effective are you in this role? Please provide some examples. 8. Do you work with traditional leaders in the community? (probe for elders, village leaders, traditional birth attendants, traditional healers) 9. Have any of these village leaders changed their attitudes about greater equality between men and women, family planning, nutrition, hygiene, or health? Please explain. a. If yes (to any of the above), how has their actual behavior changed? b. What are some of the remaining challenges in working with traditional leaders in the community? 10. Have men have changed their attitudes toward family planning, family nutrition, hygiene and health since the creation of your group? (2.1) If yes (to any of the above), how has their actual behavior changed? What effect has their changed attitudes and behavior had on their wives and children? On themselves? What are some of the remaining challenges in working with men? Please explain. 11. How has the participation of men in the MCA activities helped relations between men and women in your village? Please explain. 12. How has your work with the MCA Groups empowered women? Please give examples. 13. What changes have occurred in men, women and children’s lives as a result of the MCAs? 14. Do you believe behavior changes adopted by men toward women and girls will be permanent? Please explain your answer. 15. What is your motivation to continue to be part of the MCAs in your community? Final Performance Evaluation of the Northern Karamoja Growth, Health and Governance (GHG) Development Food Assistance Project—Final Evaluation Protocol 83 GUIDE for KEY INFORMANT or SMALL GROUP INTERVIEW with MEMERS of HEALTH UNIT MANAGEMENT COMMITTEES (HUMC) INTERVIEW DATA Facilitator Recorder Interpreter Date of Interview District Sub-County Parish Village PARTICIPANT INFORMATION Number and Type of Participants Interviewed Number of Men Number of Women Total Number of Participants Interview Tape Recorded (Please check) Yes No Number of Participants Consenting to the Interview Verbally Or Number of Participants that Signed Consent Form 1. How long ago was this committee created? 2. How has the training you received helped your work? 3. How has GHG engaged with the HUMCs/Health Facility staff? 4. Has your committee been able to work with health facilities staff to improve: a. Financial and managerial practice, b. Accountability c. Quality health service delivery? 5. What contributions has the health facility made to the quality of care for children and mothers in your area? a. How are any of these contributions related to the work of the HUMC/health facility? 6. How were the following interventions relevant to support the improvement of the quality of health service delivery? a. Citizen Report Cards b. Participatory planning and budgeting c. Health Management Information System d. Monitoring, supervision and reporting e. Community dialogues training f. Other 7. What changes, positive or negative, have occurred in women and children’s lives as a result of HUMC activities? 8. How effective has the creation of gender-based groups (mother care groups, MCAs, etc.) been in promoting health facility access/demand? Final Performance Evaluation of the Northern Karamoja Growth, Health and Governance (GHG) Development Food Assistance Project—Final Evaluation Protocol 84 9. How effective was the support GHG have given to the health facility (medical equipment, HMIS training, infrastructure construction, etc.) in promoting healthcare provision? 10. Which factors enabled your committee to better manage the health facility? 11. How effective was your role in supporting/collaborating with the VHTs? a. Can you identify better ways to strength this collaboration? 12. With the disbanding of village health teams and their substitution with CHEWs, how have you foreseen the demand for healthcare by the communities? 13. What key activities are in place to ensure the health facility will continue to provide health services to the communities? 14. What would you like to see done differently in future projects with regard to the training, health facility management support and capacity building you have received from GHG? Final Performance Evaluation of the Northern Karamoja Growth, Health and Governance (GHG) Development Food Assistance Project—Final Evaluation Protocol 85 GUIDE for KEY INFORMANT or SMALL GROUP INTERVIEW with HEALTH FACILITY PROVIDERS INTERVIEW DATA Facilitator Recorder Interpreter Date of Interview District Sub-County Parish Village PARTICIPANT INFORMATION Number and Type of Participants Interviewed Number of Men Number of Women Total Number of Participants Interview Tape Recorded (Please check) Yes No Number of Participants Consenting to the Interview Verbally Or Number of Participants that Signed Consent Form 1. How has GHG engaged with staff in this health facility? 2. How effective was the support GHG has given to the health facility (medical equipment, HMIS training, infrastructure construction, etc.) in promoting healthcare provision? 3. How were the following GHG interventions relevant to support the health service delivery? a. Training of the health providers b. Deployment of medical students c. Collaboration with the HUMC and VHT d. Support of the Health Management Information System e. Community Dialogues 4. How has the training you received helped your work? 5. What contributions has the health facility made to the quality of care for children and mothers in your area? a. Are any of these contributions are related to the work of the HUMC? 6. How you been able to work with the HUMC to improve and maintain quality health service provision? a. How will you maintain the collaboration with the HUMC in the future? 7. How effective was your role in supporting/collaborating with the VHTs? a. Can you identify better ways to strength this collaboration? 8. What changes, positive or negative, have occurred in women and children’s lives as a result of HUMC/Health Facility activities? 9. How effective has the creation of gender-based groups (mother care groups, MCAs, etc.) been in promoting health facility access/demand? Final Performance Evaluation of the Northern Karamoja Growth, Health and Governance (GHG) Development Food Assistance Project—Final Evaluation Protocol 86 10. With the disbanding of village health teams and their substitution with CHEWs, how you foreseen the demand for healthcare by the communities? 11. What would you like to see done differently in future projects with regard to the training, health equipment and in general, the support you have received from GHG? Final Performance Evaluation of the Northern Karamoja Growth, Health and Governance (GHG) Development Food Assistance Project—Final Evaluation Protocol 87 GUIDE for KEY INFORMANT with REPRESENTATIVE and a VILLAGE HEALTH TEAM or a HEALTH TEAM EXTENSION WORKER INTERVIEW DATA Facilitator Recorder Interpreter Date of Interview District Sub-County Parish Village PARTICIPANT INFORMATION Number and Type of Participants Interviewed Village Health Team Member: - specify male or female Community Health Extension Worker - specify male or female Interview Tape Recorded (Please check) Yes No Number of Participants Consenting to the Interview Verbally Or Number of Participants that Signed Consent Form 1. What has been the most important contribution your VHT has made (or as a CHEW you have made) toward health and nutrition in your community? 2. Which behaviour change initiatives and activities of your work had impacted the change in nutritional status of pregnant and lactating women and children under the age of two? 3. How effective was targeting PLW and children U2 with food rations and behavior change messages to reduce children malnutrition? 4. What changes, positive or negative, have occurred in women and children’s lives as result of the activities of the VHT/CHEWS? 5. What the VHT/CHEW can do better or still needs to be done? 6. What activities were you involved in GHG implementation? (Probe for: IMAM and IYCF training, Leader Mothers training, support and supervision; referrals to health facility for births, ANC, malnutrition, diarrhea; monthly meeting reviews, MUAC monthly screening, family planning, community dialogues, etc.) a. What do you think about the training received by GHG? b. Was this training enough to your own development and to make any difference in your work? How? 7. How important was the presence of male change agents to influence behaviour changes in health, nutrition and hygiene? 8. How you coordinate with the health facility or HUMC? Tell me about any improvement or challenge you have had since. 9. What have been the biggest challenges for the women and households to adopt improved the notions in nutrition, health, and hygiene you have learned? (Probe for traditional healers, traditional birth attendants, 3-delays model, out-of-pocket) Final Performance Evaluation of the Northern Karamoja Growth, Health and Governance (GHG) Development Food Assistance Project—Final Evaluation Protocol 88 10. Which support did you received from GHG? (Probe for: transport, non-monetary incentives, agro tools/mini-grants and stipend)? 11. With the disbanding of village health teams, what resources are there to continue providing health services in the village? 12. Did you see any improvement in your village in the nutrition status of children in the last years? a. If YES, how do you think any improvements in nutrition, health, and hygiene practices will be maintained now that the program has ended (Abim)? 13. How will you continue to provide health support to your community now the program has ended (Abim)? Final Performance Evaluation of the Northern Karamoja Growth, Health and Governance (GHG) Development Food Assistance Project—Final Evaluation Protocol 89 GUIDE for GROUP INTERVIEW with GRADUATED MCHN and MCG BENEFICIARIES INTERVIEW DATA Facilitator Recorder Interpreter Date of Interview District Sub-County Parish Village PARTICIPANT INFORMATION Number and Type of Participants Interviewed All female Graduated MCHN Beneficiaries: Or Graduated Mother Care Group Members: Total Number of Participants Interview Tape Recorded (Please check) Yes No Number of Participants Consenting to the Interview Verbally Number of Participants Consenting to the Interview with Signature 1. Before you graduated, what kind of activities did you participate in? Probe the following types of activities: SO 2 SO 1 - SO 3 Growth Monitoring (MUAC screening) and Promotion services Training on health-seeking behavior for mother and children Institutional delivery Training on use of contraceptives Training on key family nutrition behavior Cooking demonstrations Food Ration Distribution Permagardens Keeping Poultry Participation and training in Mother Care Groups (MCG) Participation in MCG+ Safe water utilization Community-latrines construction /handwashing (CLTS) Health facility access/utilization Participation and training in Water User Associations Training on farming as a business Training on production of nutrient￾rich crops Training on crop preservation and food processing Participation in Farmer field schools Participation in VSLA Member of the SACCO Entrepreneurship and business development training for your business Membership in business forums Feeder road rehabilitation Crop storage facilities Village Peace Committees …….. …… Final Performance Evaluation of the Northern Karamoja Growth, Health and Governance (GHG) Development Food Assistance Project—Final Evaluation Protocol 90 2. How were people selected to participate in these activities? 3. Has participation in any of these activities or committees changed your lives and the lives of your children? Please explain. 4. (Probe for targeting) Are there people in your village who should have participated in the project but did not? 5. If YES, why did they not participate? 6. How effective was targeting pregnant and lactating women and children under age five with food rations to reduce children malnutrition? 7. How effective were the following activities for preventing and reducing malnutrition? Why? a. The creation of MCGs and its training sessions b. Preventative blanket feeding program c. The access to safe water, community latrines and handwashing stations d. Handwashing at critical times e. Permagardens f. Poultry/husbandry g. Access to health facility (delivery of health services) 8. Which activities have been least useful? Why 9. What changes, positive or negative, have occurred in your lives and the lives of your children by participating in MCG, receiving food assistance, and other activities promoted by GHG? 10. How effective were the services and training you received in your mother care group for treating members’ children who were malnourished? (Probe MUAC monthly screening) 11. Have you noticed any reduction in child malnutrition cases in your village? a. If YES, what is, in your opinion, the reason why there are less malnourished children in your community? 12. Are you continuing to apply what you learned in your mother care group? a. What can you continue easily? b. What will be harder? Final Performance Evaluation of the Northern Karamoja Growth, Health and Governance (GHG) Development Food Assistance Project—Final Evaluation Protocol 91 GUIDE for GROUP INTERVIEW with MOTHER CARE GROUP MEMBERS / MCG+ GROUP MEMBERS INTERVIEW DATA Facilitator Recorder Interpreter Date of Interview District Sub-County Parish Village PARTICIPANT INFORMATION Number and Type of Participants Interviewed All female Mother Care Group: Or MCG+: Total Number of Participants Interview Tape Recorded (Please check) Yes No Number of Participants Consenting to the Interview Verbally Number of Participants Consenting to the Interview with Signature 1. Has your participation in the MCGs resulted in any changes in your health and the health of your children? Please describe. 2. What changes, positive or negative, have occurred in your lives and the lives of other women and children’s that are MCG members? 3. Has the discontinuation of food rations impacted women’s participation in MCG and health facility access for pregnant women and children? How? 4. What is your motivation to continue to participate in your MCG? USE OF HEALTH FACILITIES 5. How easy is it to get to the nearest health facility? 6. How often do you visit the nurses there?     What was/is the main reason for your visit? Have you changed the frequency of your visits since you are a MCG member? Why? How often do you bring your children? What is the main reason? 7. Has the health facility improved in the last several years? Please explain.  If YES, what was the main factor for this improvement? Final Performance Evaluation of the Northern Karamoja Growth, Health and Governance (GHG) Development Food Assistance Project—Final Evaluation Protocol 92 SOURCES AND USE OF OWN GROWN HOUSEHOLD FOOD 8. Do you have a vegetable garden or keep poultry?       - -           If YES (to either one), which are the benefits you can list to have a vegetable garden and/or to keeping poultry on your own? (Probe for increase food diversity at HH level, increased profit by selling surplus (i.e. eggs or milk) If NOT, why haven’t you planted a vegetable garden and/or started keeping poultry? 9. Have you attend any cooking demonstrations? If YES, what are the benefits from attending cooking demonstrations that you use. Are there any cooking methods demonstrated you haven’t used?Why not? FOOD DISTRIBUTION TOPICS 10. Have you (and do you still) receive food rations? If YES, were the food rations sufficient for you and your family? Tell me more about how you felt about the rations. Comments on the ration: (probe for timeliness of delivery; quantity; quality Opinion of the food rations (probe for acceptability to the mother and family as a food, tastiness, ease of preparation) 11. What has been the influence of the food ration in attracting women to the program? 12. Has the discontinuation of food rations affected the health and nutrition status of pregnant women and children under five? WASH TOPICS 13. Does your household have a latrine? If yes, did anyone teach you how to build, use and keep clean your latrine? If YES, who? If YES, how? 14. What do you and other people in this village think about the quality of toilets that are being promoted? 15. Since latrines and the hand-pump were built (or rehabilitated/repaired), what changes have you seen in your own family family’s sanitation and hygiene behaviors? If yes, which behaviors have they changed? What are the main reasons they have made those changes? 16. Who in your family most often use the latrines? Why? 17. a. From having handwashing stations? What have been the main benefits of having latrines for own family? And the main shortcoming? b. From having a hand-pump? 18. Can you tell me at least 3 critical moments when you should wash your hands? Do you wash your hands at those times? Why or why not? If yes, do you wash your hands at those times every day? If yes, has this had any effect on the health of your children? Please explain? 19. Do you contribute or pay in order to maintain or repair the village hand pump? If YES, what is your motivation to continue to pay for private water services? Why? If NO, why don’t you? Final Performance Evaluation of the Northern Karamoja Growth, Health and Governance (GHG) Development Food Assistance Project—Final Evaluation Protocol 93 20. Since a borehole and hand-pump were installed/repaired in your village, has it had any effect on disease and sickness of you or your children that happened when the village water was not safe for use? 21. What is the difference between Open Defecation versus Open Defecation Free?          What efforts have been made in this village to stop people from defecating outside? Who has been involved in this effort? Have they been effective? Please explain. 22. Can you give us a success story about your village or family since the latrines/handwashing/hand pump were constructed/rehabilitated? GENDER ISSUES 23. Who in your household decides which foods are purchased and who decides which foods are eaten at each meal? 24. What do you know about the Male Change Agent activities? 25. Has (or is) a Male Change Agent talked with your husband?  If yes, has your husband changed his attitudes about family nutrition, hygiene and health, or use of contraceptives since talking with the Male Change Agent? Do you have a greater role in making decisions for your household now? Please explain. Does your husband treat you respectfully? Have there been any other changes between you and your husbands? 26. Have other member of the community with great influence changed their attitudes towards women’s role in the household, use of contraceptives, family nutrition, hygiene and health? (Probe for elders, village leaders, traditional birth attendants, traditional healers) If yes, what effect has this change had on your husbands? You? Your community? 27. Have members of this group received encouragement or training to making household decisions alone or with your husband? To take a greater role in improving conditions in your village? If yes, please describe. Who provided this encouragement/training? How have you changed because of this encouragement/training? Has it affected your daughters’ behaviour? Please describe. 28. How have changes in men’s attitudes and behaviours affected your life and the lives of your children? 29. Do you think these changes in attitude and behaviour will be permanent? Please explain your answer. Final Performance Evaluation of the Northern Karamoja Growth, Health and Governance (GHG) Development Food Assistance Project—Final Evaluation Protocol 94 KII with LEAD MOTHERS INTERVIEW DATA Facilitator Recorder Interpreter Date of Interview District Sub-County Parish Village PARTICIPANT INFORMATION Number and Type and Gender of Participants Interviewed Lead Mother: 1 Total Number of Participants: 1 Interview Tape Recorded (Please check) Yes No Number of Participants Consenting to the Interview Verbally Number of Participants Consenting to the Interview with Signature 1. When was your Mother Care Group formed? 2. How many women are in your group? 3. What are your responsibilities as the Lead Mother? 4. What kind (if any) training did you receive to be a Lead Mother? a. Did it help you perform your responsibilities? b. Were there areas where you believe you needed additional training or guidance? 5. What activities does your MCG do? (Probe for): a. ANC/Institutional deliveries b. Children Nutrition c. MUAC screening d. Immunization e. Growth monitoring f. Family Planning g. Cooking demonstration h. Gender equity i. MCG+3 6. As the Lead Mother, how do you think the activities your MCG is involved in benefit the members? a. How has it helped the women in your village? 7. How effective was targeting pregnant and lactating mother and children under 5 with food rations and behaviour change messages to reduce children malnutrition? Final Performance Evaluation of the Northern Karamoja Growth, Health and Governance (GHG) Development Food Assistance Project—Final Evaluation Protocol 95 8. How has the care group helped the most vulnerable mothers in your community? Give an example. 9. What has been the influence of the food ration in attracting women to the program? 10. Has discontinuation of food rations affected the health and nutrition status of pregnant women and children under age 5? 11. What changes, positive or negative, have occurred in women and children’s lives as a result of this mother’s care group? 12. How does your work with the Care Group empower women in this community? Please give examples. 13. Do you work with male change agents in this community? Please explain. 14. If yes, has the combined work of this group and the Male Change Agent working with husbands led to men taking a greater role in the health and nutrition of their infants and children? Please explain. 15. How effective was the blanket feeding in promoting behaviour changes in pregnant and lactating mothers and their children age five and under? 16. Which challenges have you faced as the lead mother for this care group a. Were you able to address these challenges? Please explain. 17. How the behaviour changes will continue now the program has ended? 18. What is your motivation to continue to lead your MCG? 19. What have you learned as lead mother that should be applied to helping mother care groups to be more successful? Are they (whatever the lead mother mentions) being applied now to the mother care group you lead? Mother Care Group Plus (Focus Group Discussion) 1. What is the difference between MCG and MCG Plus? 2. How were you chosen to be MCG+ members? 3. How has your participation in the MCG+ improved your household income, welfare and the health/nutrition status of your children? 4. What benefits do you have compared to the MCG? 5. How have you been linked to VSLA or SACCO? 6. How has your participation in the MCG+ have changed the relationship with your husband? 7. Explain how did/are you using the savings coming from the MG+? 8. What changes, positive or negative, have occurred in women and children’s lives as a result of GHG? 9. How has the discontinuation of food rations impacted women’s participation in MCG+? Non-MCHN Beneficiaries (Focus Group Discussion) 1. What do you know about GHG program? 2. How much did you understand about why and how you were not selected to participate in the program? 3. Did you attend/listen at any health/nutrition and hygiene trainings? a. If Yes, What was the message you consider the most important? 4. In your opinion and knowledge of the program, how has GHG helped the women, the men and children in your community? Final Performance Evaluation of the Northern Karamoja Growth, Health and Governance (GHG) Development Food Assistance Project—Final Evaluation Protocol 96 5. How do you feel about the food rations delivered to PL women and children U2? a. What about other children over 2yrs? b. What about non-PLW? 6. What changes, positive or negative, have occurred in women and children’s lives as a result of GHG? (2.2) 7. What can you say about the health and nutrition situation in your community? Key Informant Interviews with Local Health and WASH Authorities at District and Sub￾district Level This topical outline is used to guide the key informant interviews with representatives from the Ministry of Health, Water and Environment who participated in the GHG Program to be held during the field visits in the GHG targeted districts. The groups and intermediaries covered by this topical outline include: District Health Officer5 District Water Officer Sub-district health and WASH assistants Health Unit Management Committees (HUMCs) Health Facility staff Village Health Teams (VHTs) / Community Health Extension Workers (CHEWs) DISTRICT HEALTH OFFICER (Key Informant Interview)/ Sub-district Health Assistant 1. How has your office been engaged by the program? 2. Which behaviour change initiatives and activities promoted by GHG have had the greatest impact on affecting change in nutritional status of pregnant and lactating women (PLW) and children under 2? (1.2) 3. Do you think the GHG MCHN and WASH activities targeted the most vulnerable? (1.3) a. If NOT, why? 4. How effective was targeting PLW and children U2 with food rations and behaviour change messages to reduce children malnutrition? (1.3) 5. How GHG activities were in line with the national strategy to reduce children malnutrition? (2.1) 6. How has the discontinuation of food rations provided by GHG affected the health and nutrition status of PLW and children under two? (2.1) 7. What has been the influence of the food ration in attracting women to the program? (2.2) 8. What changes, positive or negative, have occurred in women and children’s lives as a result of GHG? (2.2) 9. How effective was blanket feeding in promoting behaviour changes in PLW? (3.1) 10. What health and nutrition interventions such as food rations, care groups creation and training, permagarden creation and support were more or less acceptable by the beneficiaries? (3.3) 5 The questions in bold will be addressed exclusively to the District Health Officers Final Performance Evaluation of the Northern Karamoja Growth, Health and Governance (GHG) Development Food Assistance Project—Final Evaluation Protocol 97 11. What plans do you have for continuing to work with beneficiaries of GHG now the program has ended? (4.1) 12. With the disbanding of government-provided village health teams, what resources are there to continue providing services at the village level? (4.2) 13. Are there other organizations (local, international, private, etc.) working in the same communities as GHG implementing child health/nutrition projects? (4.5) a. If YES, how your office has enhanced linkages between them and GHG? (4.6) 14. GHG backed-up the mobilisation of medical students to support and strengthen the health service provision at community-level. How the district continue to support the deployment of those medical students now the program has ended? (4.6) 15. How effective this deployment have been in increase health service provision and health staff capacity building? (3.2) 16. GHG have invested in the creation of linkages with private health service providers. To what extent will your office and the private services providers (private clinics, drug shops, pharmacies and Mango for grants) will be able to continue collaboration in the communities targeted by GHG? (4.6) 17. What are the lessons learned and best practices that should be considered in implementing MCHN and WASH programs? (5.2) 18. What would you like to see done differently in future projects with regard to how your office has been engaged? (5.4) 19. Are there any additional activities you could recommend for similar programs? (5.1) DISTRICT WATER OFFICER / Sub-district WASH Assistants (Key Informant Interview) (Same as the above questions, except the bold ones) 1. Which specific WASH interventions and approaches can be linked to decreases (or show great potential) that were achieved in reducing malnutrition among children under five? (2.1) 2. What changes, positive or negative, have occurred in women and children’s lives as a result of WASH activities such as CLTS, CHAST training, open defecation free awareness, VIP latrines in schools, boreholes drilling and reparation? (2.2) 3. How effective was the coordination with the organizations involved in the WASH component such as WHAVE, village WASH Committees, School Hygiene Clubs, Abim HPMA? (3.1) 4. What WASH interventions such as CLTS, CHAST, institutional latrines, boreholes drilling and reparation were more or less acceptable by the beneficiaries? (3.3) 5. What are the perceptions of the communities about the quality of toilets promoted by GHG? (4.3) 6. GHG was the only program in which villagers actually paid for private sector water and sanitation services. What factors contribute to their willingness to pay for these services? (4.4) 7. GHG have invested significant efforts in the creation of linkages with WASH private service providers. To what extent will your office and the private services providers such as Abim HPMA, KLR ltd., ICON ltd and WHAVE will be able to continue service provision in the communities targeted by GHG? (4.6) 8. Do HUMCs arrange community dialogue? Who joins in? Health facility managers, MCG members? VHTs. CHEWS? Sub-district health officers? Do they lead the dialogue? Do staff from HUMC, etc. Final Performance Evaluation of the Northern Karamoja Growth, Health and Governance (GHG) Development Food Assistance Project—Final Evaluation Protocol 98 attend these community dialogues? Do they take it seriously? Can they give examples of topics discussed in community dialogue that have resulted in changes? Or that are disregarded by health facility providers, etc.? Final Performance Evaluation of the Northern Karamoja Growth, Health and Governance (GHG) Development Food Assistance Project—Final Evaluation Protocol 99 INTERVIEW GUIDES FOR SO 3 Please Note: Each of these guides are for group interviews. The evaluation team reserves the right to use some of these guides for either small group interviews of two to three people or for use in conducting key informant interviews. GUIDE for GROUP INTERVIEWS with GHG IMPLEMENTING PARTNERS INTERVIEW DATA Facilitator: Recorder: Interpreter: Date of Interview: District: Sub-County: Parish: Village: PARTICIPANT INFORMATION Number, Type and Gender of Participants Interviewed All Youth Committee Female:____________________ Male:___________________ Mixed Gender / Age Committee Female:__________________ Male:____________________ All Women Committee Women:____________________ __ Total Number of Participants: _____ Interview Tape Recorded (Please check) Yes: ___________ No: ____________ Number of Participants Consenting to the Interview Verbally: Number of Participants Consenting to the Interview with Signatures: Q1 What do you think about the current level of (armed) conflict and violence is your areas of operation - have they increased or reduced since you started working to reduce armed conflicts?  Has the security situation changed? Please explain. Q2 What kinds of conflicts did you help facilitate to prevent or mitigate between 2013 and 2017? Q3 What kind of support did you receive between 2013 and 2017 to strengthen your capacity to prevent, mitigate and resolve conflicts? 1. How did this support help you perform your roles better? Q4 How did interventions you facilitated lead to a reduction in incidences of different forms of armed conflicts [EQ1.2], including sexual and gender-based violence (SGBV): 1. Faced by Karamojong women and; 2. An improvement in food security in targeted communities? Final Performance Evaluation of the Northern Karamoja Growth, Health and Governance (GHG) Development Food Assistance Project—Final Evaluation Protocol 100 Q5 Which formal and non-formal structures/organizations did you work/collaborate with at the village, sub-county and district levels in undertaking conflict prevention, mitigation and resolution between 2013 and 2017?            How were these structures and organizations organized? What arrangements did you facilitate? What role did each play? How well did the collaboration / or arrangements between these structures and organization work in terms of effectiveness? What should have been done differently to ensure more effectiveness of the collaboration? Or, what else should be done to ensure the effectiveness of this collaboration? Q6 What factors may have promoted or inhibited the collaboration with these organizations’ ability to effectively: Prevent, Mitigate, and Resolve the different types of local conflicts between 2013 and 2017? Please explain your answer. Q7 How did you involve women in: 1. Conflict management training and in, 2. Actual conflict management between 2013 and 2017? How effective was their involvement in reducing armed conflicts? Q8 How did you involve youth who are most at risk to engage in violence in your conflict management activities between 2013 and 2017? 1. How effective was your approach in reducing youth violence, and in 2. Reducing armed conflicts? Q9 What kinds of dialogues and consensus building activities did you facilitate during: 1. Conflict prevention, mitigation and; 2. Conflict resolution between 2013 and 2017? 3. Who was involved? 4. How effective were these activities? [ Q10 Which of the activities you implemented between 2013 and 2017 have continued smoothly to-date, and why? Which activities have encountered setbacks, and why? What can be done/or have you done to address these setbacks? Q11 Which approaches and implementation strategies that were used by your organization between 2013 and 2017 can ensure sustained reduction in incidences of armed conflicts? Please explain. Q12 What do you think was your most significant contribution to conflict prevention, mitigation and resolution between 2013 and 2018? Q13 What risks were associated the approaches and implementation strategies used by your organization between 2013 and 2018? If the consequences from any of these risks occurred, how did you address them? Q14 What lessons and best practices has your organization identified that you are applying now to reduce armed conflict? Q15 Have there been any unintended negative or positive consequences from the conflict management activities you facilitated? Please explain. Q 16 How have you ((or did you) applied the lessons you have learned about these unintended negative and positive consequences between 2013 and 2018? Please explain. Final Performance Evaluation of the Northern Karamoja Growth, Health and Governance (GHG) Development Food Assistance Project—Final Evaluation Protocol 101 GUIDE for GROUP INTERVIEWS with DISTRICT SECURITY COMMITTEES INTERVIEW DATA Facilitator: Recorder: Interpreter: Date of Interview: District: Sub-County: Parish: Village: PARTICIPANT INFORMATION Number, Type and Gender of Participants Interviewed Female:____________________ Male:___________________ Total Number of Participants: ___________ Interview Tape Recorded (Please check) Yes: ___________ No: ____________ Number of Participants Consenting to the Interview Verbally: Number of Participants Consenting to the Interview with Signatures: Q1 What do you think about the current level of (armed) conflict and violence in this district?           Have these incidents increased or reduced since your committee started work to reduce armed conflicts? Has the security situation changed over the past three years? Please explain. Q2 What role does the district security committee play in conflict prevention, mitigation and resolution? What kinds of conflicts were prevented, mitigated, or resolved through your committee’s involvement between 2013 and 2018? Q3 What kind of support did you receive during this time period to strengthen your capacity to prevent, mitigate and resolve conflicts? Did this support help you perform your roles better, or not. Please explain. Q4 How have interventions this committee participated in between 2013 and 2018 lead to a reduction in incidences of different forms of armed conflict,, including sexual and gender￾based violence (SGBV): Faced by Karamojong women, and Led to an improvement in food security in targeted communities? Q5 Which formal and non-formal structures/organizations did your committee work/collaborate with in the district, sub-county and village level in undertaking conflict prevention, mitigation and resolution between 2013 and 2017? How well did the collaboration work? What could/should have been done differently to ensure more effectiveness of the collaboration? Have there been any changes made in the way the collaboration is organized to function? Please explain (if yes). Do you think this collaborative approach is an effective means to reduce conflict in this area? Why or why not? Final Performance Evaluation of the Northern Karamoja Growth, Health and Governance (GHG) Development Food Assistance Project—Final Evaluation Protocol 102 What factors may have promoted or inhibited your committee’s ability to effectively prevent, mitigate, and resolve the different types of local conflicts within this collaborative? Please explain. Q6 How did you involve women in your conflict management activities between 2013 and 2017? How effective was involving women in reducing armed conflicts? Q7 How did you involve youth who are most at risk to engage in violence in your conflict management activities between 2013 and 2017? How effective was it in reducing armed conflicts Q8 What kinds of dialogues and consensus building activities did you undertaken during conflict prevention, mitigation and resolution between 2013 and 2018? Who was involved? How effective were these activities? Q9 Which of the specific activities you participated in as part of the collaborative during this time frame have continued smoothly to date and are still implemented?    Which activities have encountered setbacks, and why? What was done to address those setbacks? Or are those specific activities no longer used? Q10 What actions have been taken to date to ensure the collaboration for conflict prevention, mitigation, and resolution activities are long-lasting? Do you believe the actions taken to date will be effective to continue these efforts? Q10 What plans do you have to ensure activities you were involved in between 2013 and 2018 aimed at conflict prevention, mitigation and resolution continue? Q11 What do you think was your most significant contribution to conflict prevention, mitigation and resolution between 2013 and 2018? Final Performance Evaluation of the Northern Karamoja Growth, Health and Governance (GHG) Development Food Assistance Project—Final Evaluation Protocol 103 GUIDE for GROUP INTERVIEWS with PEACE COMMITTEES INTERVIEW DATA Facilitator: Recorder: Interpreter: Date of Interview: District: Sub-County: Parish: Village: PARTICIPANT INFORMATION Number, Type and Gender of Participants Interviewed All Youth Committee Female:____________________ Male:___________________ Mixed Gender / Age Committee Female:__________________ Male:____________________ All Women Committee Women:_______________________ Total Number of Participants: _______ Interview Tape Recorded (Please check) Yes: ___________ No: ____________ Number of Participants Consenting to the Interview Verbally: Number of Participants Consenting to the Interview with Signatures Q1 What do you think about the current level of (armed) conflict and violence in this district?        Have these incidents increased or reduced since your committee started work to reduce armed conflicts? Has the security situation changed over the past three years? Please explain. Q1 How many years ago was this peace committee formed? Has the membership stayed the same? Are any members involved in other village groups? (probe for participation in mother care groups, water user associations) Q2 What roles does the peace committee play within the community? Within district efforts to promote peace and reduce conflict? Q1 What are the most common kinds of conflicts over livestock and land in the district/sub￾county that affected your community over the last five years? What effect did they have on this community? Do any of these conflicts continue to be a problem for your community? Q2 What kind of support did you receive between 2013 and 2018 to build your capacity to prevent, mitigate and resolve conflicts? Did this support help your committee to be effective? Please explain Q3 How did interventions your committee participated in lead to a reduction in incidences of different forms of armed conflicts, including sexual and gender-based violence (SGBV) 1. Faced by Karamojong women [EQ1.4] and 2. Led to an improvement in food security in your committee? Final Performance Evaluation of the Northern Karamoja Growth, Health and Governance (GHG) Development Food Assistance Project—Final Evaluation Protocol 104 Q4 Which formal and non-formal structures/organizations did you work/collaborate with in undertaking conflict prevention, mitigation and resolution?  How well did the collaboration work together?  What should have been done differently to ensure more effectiveness of the collaboration?  Is this collaboration with other committees and offices and government officials an effective way to manage and resolve conflicts that should be continued? Please explain. Q5 What factors may have promoted or inhibited your committee’s ability to effectively participate in prevent, mitigate, and resolve the different types of local conflicts? How and why? Q6 How have women on this committee participate in conflict management activities in your community or in the area? Women in your conflict management activities?  How effective has women’s participation been in reducing armed conflicts  For women: what challenges did you face in your community in playing a role in this peace committee? And where did these challenges come from?  What success (if any) have you had in addressing these challenges? Do members of this community still oppose or hold negative attitudes about the involvement of women in the peace committee? Q7 How have the youth in this community who are most at risk to engage in violence in been involved in conflict management and resolution activities?  How effective has the involvement of youth at risk been in reducing conflict in this community and in armed conflicts? Q8 What kinds of dialogues and consensus building activities did you participate in during conflict prevention, mitigation and resolution?  Who was involved?  How effective were these activities? Q9 Which of the activities that you implemented before 2017 have continued smoothly to￾date, and why? Which activities have encountered setbacks, and why? What can be done to address these setbacks? Q10 What actions have you taken to ensure this committee continue to be involved in over the past five years aimed at conflict prevention, mitigation and resolution continue?  Do you think the peace committee should continue its role on a permanent basis? Why? Q11 What do you think this committees most significant contributions have been in conflict prevention, mitigation and resolution between 2013 and 2018? Please explain. Q12 Has this committee become more responsive to the needs of the community related to conflict issues?  If yes, explain how the committee has been able to be more responsive? Did the committee do anything different to become more responsive?  How does the community view the role of this committee and its effectiveness in helping to prevent or resolve conflict? Are you accepted? Q13 What risks were associated the approaches your peace committee used over the past five years to manage conflict?  Did any of these risks actually occur over the past five years? Please explain.  Were you able to anticipate any of these risks and take measures to address them?  How did the committee address those risks that actually did occur?  Do you still face the same risks today? Final Performance Evaluation of the Northern Karamoja Growth, Health and Governance (GHG) Development Food Assistance Project—Final Evaluation Protocol 105 Q14 What has this peace committee learned over the past years about methods to prevent, mitigate or resolve armed conflict that you apply today to become more effective in your role? Q15 Were there any consequences of the committee’s activities that you didn’t anticipate? Please explain?  Were there any unintended positive consequences for this committee or your community that you apply today?  For those unforeseen negative consequences that occurred because of the committee’s activities, have you done anything to change those activities? Final Performance Evaluation of the Northern Karamoja Growth, Health and Governance (GHG) Development Food Assistance Project—Final Evaluation Protocol 106 GUIDE for GROUP INTERVIEWS with DISTRICT LOCAL GOVERNMENT AUTHORITIES INTERVIEW DATA Facilitator: Recorder: Interpreter: Date of Interview: District: Sub-County: Parish: Village: PARTICIPANT INFORMATION Number, Type and Gender of Participants Interviewed Female:____________________ Male:___________________ Total Number of Participants: __________ Interview Tape Recorded (Please check) Yes: ___________ No: ____________ Number of Participants Consenting to the Interview Verbally: Number of Participants Consenting to the Interview with Signatures: Q1 What do you think about the current level of (armed) conflict and violence in this district?  Have these incidents increased or reduced since your committee started work to reduce armed conflicts? Has the security situation changed over the past three years? Please explain. Q2 What are the most common kinds of conflicts over livestock and land in the district/sub￾county that affected the communities over the last five years? Q3 What kind of support did you receive between 2013 and 2018 to strengthen your capacity to prevent, mitigate and resolve conflicts? How did this support help you perform your roles better Q4 How did your interventions lead to a reduction in incidences of different forms of armed conflicts, including sexual and gender-based violence (SGBV) faced by Karamojong women and improvement in food security in targeted communities you serve? And why? Q5 What factors may have promoted or inhibited your ability to effectively prevent, mitigate, and resolve the different types of local conflicts? How and why? Q6 Which formal and non-formal structures/organizations did you work/collaborate with in the district, sub-country and village levels in undertaking conflict prevention, mitigation and resolution?  How well did the collaboration work? Please explain.  What should have been done differently (or could be done now) to increase the effectiveness of the collaboration?  Do you think a collaboration like this is the most effective way to manage and resolve conflict in this district? Q7 How did you involve women in your conflict management activities? How effective was it in reducing armed conflicts Final Performance Evaluation of the Northern Karamoja Growth, Health and Governance (GHG) Development Food Assistance Project—Final Evaluation Protocol 107 Q8 How did you involve youth who are most at risk to engage in violence in your conflict management activities?  How effective were those youth in contributing to the reduction of armed conflicts? Q9 What kinds of dialogues and consensus building activities did you participate in during conflict prevention, mitigation and resolution?  Who was involved?  How effective were these activities? Q10 Which of the activities that you implemented before 2018 have continued smoothly to-date, and why that you continue to implement?  Which activities have encountered setbacks, and why?  What did you do to address these setbacks? Q11 What actions have you taken or contributed to ensure activities aimed at conflict prevention, mitigation and resolution that you were involved in before 2018 continue? Q12 What do you think was your most significant contribution to conflict prevention, mitigation and resolution over the past five years? Q13 In what ways have you become more responsive to the needs of communities affected by conflicts? Please explain. Q14 What risks were associated the approaches and implementation strategies used by your organization between 2013 and 2018?  Were you able to anticipate any of them that actually occurred?  Were you able to address these effectively? Please explain. Q15 What lessons can be drawn and best practices inferred from the activities you have been undertaking to reduce armed conflict that you are continuing to practice today? Q16 Were there any unintended positive or negative consequences from the activities you participated in to reduce armed conflict that you did not anticipate?  How did you address these consequences that occurred, both positive and negative? Final Performance Evaluation of the Northern Karamoja Growth, Health and Governance (GHG) Development Food Assistance Project—Final Evaluation Protocol 108 GUIDE for GROUP INTERVIEWS with MEMBERS OF LAND CONFLICT MANAGEMENT STRUCTURES INTERVIEW DATA Facilitator: Recorder: Interpreter: Date of Interview: District: Sub-County: Parish: Village: PARTICIPANT INFORMATION Number, Type and Gender of Participants Interviewed Male: Female: Total Number of Participants: Interview Tape Recorded (Please check) Yes: ___________ No: ____________ Number of Participants Consenting to the Interview Verbally: Number of Participants Consenting to the Interview with Signatures: Q1 What do you think about the current level of (armed) conflict and violence in this district?      Have these incidents increased or reduced since your office started work to reduce armed conflicts? Has the security situation changed over the past three years? Please explain. Q2 What kinds of land conflicts were you involved in mediating and resolving between 2013 and 2018? Q3 What kind of support did you receive to strengthen your capacity to mediate and resolve land conflicts in your areas of jurisdiction between 2013 and 2018? How did this support help you perform your roles better Q4 How did your interventions contribute to a reduction in incidences of armed conflicts associated with land over the past five years and why? Q5 How did your interventions in the management of land conflicts lead to a reduction in incidences of sexual and gender-based violence (SGBV) faced by Karamojong women between 2013 and 2018; and why? Q6 Has your office’s work led to women taking actual ownership of land they rightfully inherited? Please explain what your office’s role was. Is there a trend now in women taking ownership over the land they inherited? What obstacles remain to be addressed? Q7 How did your interventions in the management of land conflicts lead to an improvement in food security in the communities you serve over the past five years? And why? Q8 What factors may have promoted or inhibited your ability to effectively prevent, mitigate, and resolve the different kinds of land conflicts between 2013 and 2018? How and why? Q9 How did you involve women in your land conflict management activities over the past five years? How effective was the involvement of women in reducing land conflicts Final Performance Evaluation of the Northern Karamoja Growth, Health and Governance (GHG) Development Food Assistance Project—Final Evaluation Protocol 109 Q10 How did you involve youth who are most at risk to engage in violence in your land conflict management activities between 2013 and 2017?        How effective was it in reducing land conflicts Q11 What kinds of dialogues and consensus building activities did you undertaken during land conflict management? Who was involved? How effective were these activities? Q12 Which of the land conflict management activities that you implemented between 2013 and 2018 have continued smoothly to-date that you continue practicing and why? Which activities have encountered setbacks, and why? What can be done to address these setbacks? Q13 What plans do you have to ensure your land conflicts management activities continue unabatedly? Q14 In what ways can Land Focal Point Volunteers (LFPV) continue to be relevant and useful in the management of land conflicts? Q15 What do you think was your most significant contribution to land conflict management between 2013 and 2018? Q16 What risks were associated the approaches and implementation strategies used by your organization for managing land conflicts? Were you able to anticipate any of them? How did you address them? Are they still risks? Q17 What lessons can be drawn and best practices inferred from the activities you implemented over the past five years that you are applying now? Q18 Were there any positive consequences from your activities that you didn’t intend to occur? What were these? Have you taken actions to ensure these positive consequences continue? Were there any negative consequences that occurred you didn’t anticipate? What were these and how have you addressed them? Final Performance Evaluation of the Northern Karamoja Growth, Health and Governance (GHG) Development Food Assistance Project—Final Evaluation Protocol 110 GUIDE for GROUP INTERVIEWS or KEY INFORMANT INTERVIEW with TRADITIONAL LEADERS INTERVIEW DATA Facilitator: Recorder: Interpreter: Date of Interview: District: Sub-County: Parish: Village: PARTICIPANT INFORMATION Number, Type and Gender of Participants Interviewed Type of Traditional Leader: Female:____________________ Male:___________________ Total Number of Participants: ___________ Interview Tape Recorded (Please check) Yes: ___________ No: ____________ Number of Participants Consenting to the Interview Verbally: Number of Participants Consenting to the Interview with Signatures: Q1 What do you think about the current level of (armed) conflict and violence in this district?        Have these incidents increased or reduced since your office started work to reduce armed conflicts? Has the security situation changed over the past three years? Please explain. Q2 What roles have you traditionally played in preventing, mitigating, and resolving conflicts? Has your role changed? Please explain. Q3 What kinds of conflicts have you been involved in preventing, mitigating and resolving over the last five years? Q4 What were some of the most difficult conflicts you were involved in preventing, mitigating and resolving between 2013 and 2018? What are some of the most outstanding outcomes of from your involvement? Q5 What kind of (or did you receive) support did you receive to strengthen your capacity to prevent, mitigate and resolve conflicts over the past five years? 1. How effective was that support? Please explain. Q6 What factors promoted or inhibited your ability to effectively participate in either preventing, mitigating, or resolving local conflicts between 2013 and 2018? Q7 Which formal and non-formal structures/organizations did you (OR DO YOU STILL) work/collaborate with in undertaking conflict prevention, mitigation and resolution over the last five years? How well did the collaboration work? What should have been done differently to ensure more effectiveness of the collaboration? Do you think that collaborating with different committees and government officials is an effective way to manage and resolve conflict? Final Performance Evaluation of the Northern Karamoja Growth, Health and Governance (GHG) Development Food Assistance Project—Final Evaluation Protocol 111 Q8 How can your collaboration with government officials and offices strengthened to become more effective in preventing, mitigating and resolving conflicts? Q9 Were women involved in the conflict management activities you participated in between 2013 and 2018? If yes, how were they involved? What role did they play? How effective was the involvement of women in reducing armed conflicts? Please explain Q10 Were youth who are most at risk to engage in violence involved in the conflict management activities you participated in? If yes, how were they involved? What role did they play? How effective the participation of youth in reducing armed conflicts or other types of conflict that occur in this communities? Please explain. Q11 What kinds of dialogues and consensus building activities did you participate in during conflict prevention, mitigation and resolution? Who was involved? How effective were these activities? Q12 How did these interventions lead to a reduction in incidences of different forms of armed conflicts, including sexual and gender-based violence (SGBV) faced by Karamojong women and improvement in food security in your community? And why? Q13 How did you (or do you) encourage and bring on board youth and women in conflict management and resolution activities with you over the past five years?  What was the outcome? How can it be improved? Q14 What community demands did you respond to as traditional leaders between over the past five years? What were the outcomes? How can it be improved? Q15 Which of the activities you have been implementing have continued smoothly, and why? Which activities have encountered setbacks, and why? What can be done to address these setbacks? Q16 Do you wish to continue your involvement in conflict management and reduction? Please explain.  Does the district have any plans for supporting the continuation of your role in the collaboration to manage and resolve conflicts going forward? If yes, what plans? Final Performance Evaluation of the Northern Karamoja Growth, Health and Governance (GHG) Development Food Assistance Project—Final Evaluation Protocol 112 GUIDE for GROUP INTERVIEWS with CIVIL SOCIETY ORGANIZATIONS INTERVIEW DATA Facilitator: Recorder: Interpreter: Date of Interview: District: Sub-County: Parish: Village: PARTICIPANT INFORMATION Number, Type and Gender of Participants Interviewed Male: Female: Total Number of Participants: __________ Interview Tape Recorded (Please check) Yes: ___________ No: ____________ Number of Participants Consenting to the Interview Verbally: Number of Participants Consenting to the Interview with Signatures : Q1 What kinds of conflicts has your organization been involved in preventing, mitigating and resolving over the last five years?  What was the role this organization played in either preventing, mitigating or resolving conflict? Q2 What were some of the most difficult conflicts you were involved in preventing, mitigating and resolving between 2013 and 2017? What are some of the most outstanding outcomes of your involvement? Q3 What kind of support did you receive to strengthen your capacity to prevent, mitigate and resolve conflicts between 2013 and 2017?  Was this support effective to help your organization play its roles?  How could it be improved? Q4 What factors promoted or inhibited your organization’s ability to effectively prevent, mitigate, and resolve local conflicts? Q5 Which formal and non-formal structures/organizations did you work/collaborate with in undertaking conflict prevention, mitigation and resolution?  How well did the collaboration work?  What should have been done differently to ensure more effectiveness of the collaboration? Q6 How can your collaboration with formal government structures be made even more beneficial in mitigating and resolving conflicts? Q7 Did you involve women in your organization in its conflict management activities?  If yes, what roles did they play? The same as men’s?  How effective was their involvement in reducing armed conflicts Q8 Did you involve youth who are most at risk to engage in violence in your organization’s conflict management activities?  If yes, how were they involved? What role did they play?  How effective the involvement of youth in reducing armed conflicts Final Performance Evaluation of the Northern Karamoja Growth, Health and Governance (GHG) Development Food Assistance Project—Final Evaluation Protocol 113 Q9 What kinds of dialogues and consensus building activities did you undertaken during conflict prevention, mitigation and resolution? Who was involved? How effective were these activities? Q10 How did the interventions your organization was involved in lead to a reduction in incidences of different forms of armed conflicts including sexual and gender-based violence (SGBV)  Faced by Karamojong women;  And to any improvement in food security in targeted communities? Q11 How did you or your organization bring on board youth and women in your activities between 2013 and 2017?  What was the outcome? How can it be improved?  If you did not bring on board youth and women, what was the reason why? Q12 What community demands did your organization respond to?  What were the outcomes  How could your organization’s responses improved? Q13 Which of the activities you have been implementing have continued smoothly, and why? Which activities have encountered setbacks, and why? What can be done to address these setbacks? Q14 Does the district have any plans for supporting the continuation of your work going forward? If yes, what plans? Final Performance Evaluation of the Northern Karamoja Growth, Health and Governance (GHG) Development Food Assistance Project—Final Evaluation Protocol 114 GUIDE for GROUP INTERVIEWS with MAGISTRATES’ COURTS INTERVIEW DATA Facilitator: Recorder: Interpreter: Date of Interview: District: Sub-County: Parish: Village: PARTICIPANT INFORMATION Number, Type and Gender of Participants Interviewed Female:____________________ Male:___________________ Total Number of Participants: __________ Interview Tape Recorded (Please check) Yes: ___________ No: ____________ Number of Participants Consenting to the Interview Verbally: Number of Participants Consenting to the Interview with Signatures: Q1 What do you think about the current level of (armed) conflict and violence in this district?  Have these incidents increased or reduced since your office started work to reduce armed conflicts? Has the security situation changed over the past three years? Please explain Q2 How many cases involving livestock theft were handled by this court (both criminal and civil) between 2013 and 2017? Q3 How many cases involving land conflicts were handled by this court (both criminal and civil) between 2013 and 2018? Q4 How many cases involving livestock theft were referred back or withdrawn for mediation under the traditional system? Q5 How many such cases involving land conflicts were referred back or withdrawn for mediation under the traditional system? Q6 Have cases of involving land conflicts or livestock thefts increased or reduced compared to the period before 2013; and why? Q7 What explains local community preference for Sub-County structures (whether formal or informal) in the handling of cases related to livestock thefts and land conflict, instead of the formal courts of law? Q8 What kind of support did you receive between 2013 and 2018 to strengthen your capacity to handle cases related to livestock thefts and land conflicts? How did this support help you perform your roles better? Q9 How did the handling of cases related to livestock thefts and land conflicts lead to a reduction in incidences of different forms of armed conflicts, including sexual and gender-based violence (SGBV)  Faced by Karamojong women and  Led to improvement in food security in targeted communities in this district and why? Q10 What factors may have promoted or inhibited the ability of formal courts to effectively handle cases related to livestock thefts and land conflict? How and why? Final Performance Evaluation of the Northern Karamoja Growth, Health and Governance (GHG) Development Food Assistance Project—Final Evaluation Protocol 115 Q11 Which formal and non-formal structures/organizations did formal court work/collaborate with in the handling of cases related to livestock thefts and land conflicts?  How well did the collaboration work?  What should have been done differently to increase the effectiveness of this collaboration? Q12 What kinds of dialogues and consensus building activities did you undertake in the handling of cases related to livestock thefts and land conflicts?  What was the approach taken?  How effective were these activities? Final Performance Evaluation of the Northern Karamoja Growth, Health and Governance (GHG) Development Food Assistance Project—Final Evaluation Protocol 116 GUIDE for GROUP INTERVIEWS with LOCAL COMMITTEES INTERVIEW DATA Facilitator: Recorder: Interpreter: Date of Interview: District: Sub-County: Parish: Village: PARTICIPANT INFORMATION Number, Type and Gender of Participants Interviewed Type/Name of Committee: Males: Females: Total Number of Participants: __________ Interview Tape Recorded (Please check) Yes: ___________ No: ____________ Number of Participants Consenting to the Interview Verbally: Number of Participants Consenting to the Interview with Signatures: Q1 Would you consider the security situation in your community to be better now (2018) compared to five years ago (2013)? Why? Q2 What kinds of conflicts have been most prevalent in your community over the last five years? Q3 What kind of interventions were undertaken in your community between 2013 and 2018 to address conflicts? How did these interventions contribute to the reduction of incidences of armed conflicts including sexual and gender-based violence (SGBV) faced by Karamojong women Q4 In what ways did the interventions that were undertaken in your community for reducing armed conflicts lead to an improvement in food security? And why? Q5 Would you consider the food security situation in your community now (2018) to be better than it was five years ago (2013)? Why? Q6 Which formal and non-formal structures/organizations were involved in conflict management in your community? Which of these were most beneficial? Q7 How were women in your community involved in conflict management activities? How effectively did this contribute to the reduction of armed conflicts Q8 How were the youth in your community who are most at risk to engage in violence involved in conflict management activities? How effectively did this contribute to the reduction of armed conflicts Q9 What kinds of dialogues and consensus building activities were undertaken in your community as part of conflict management interventions? Who was involved? How effective were these activities? Q10 Which of the activities that were implemented in management of conflicts in your community over the past five years had continued smoothly to date and are still continued, and why?  Which activities have encountered setbacks, and why?  What have you done to address these setbacks? Final Performance Evaluation of the Northern Karamoja Growth, Health and Governance (GHG) Development Food Assistance Project—Final Evaluation Protocol 117 Q11 NOTE: Ask either Q 11 or Q12 depending on which organization was involved in your community, or, ask both questions if both AWOTID and KAPDA were involved. What contribution did AWOTID make in conflict management in your community over the past five years? Q12 What contribution did KAPDA make in conflict management in your community between 2013 and 2018? Q13 What contribution did the following committees and individuals make in conflict management in your community over the past five years?  District security committee  Village Peace Committee  Land Focal Point Volunteers  Traditional Leaders  Others ANNEX 4: TYPE AND NUMBER OF GHG PERFORMANCE EVALUATION RESPONDENTS ACROSS DISTRICTS, SUB-COUNTIES, AND VILLAGES Location Type of Respondent1 Totals 3 Districts Abim, Kotido, Kaabong Director and a Deputy Director Office of Health 3 Director, Office of Water and Environment 3 Director, Commercial Office 3 Director, Office of Agriculture 3 Director, Production and Marketing 2 Veterinary Officers and one Agro-Veterinary Officer 3 Chair, Hand Pump Mechanics Association 2 Directors and members of Business Forum 4 Manager, Voice of Kotido Radio & Director of Programming on Business Topics 2 Chair, Defense Committee 1 Secretary, District Disaster Management Committee 1 Lawyer, Uganda Legal Aid Clinic 1 Chair, Commodity Association 1 Managers from Commercial Seed Companies 4 Commodity food traders 2 Agro-Input Dealers (one of them is also a commercial crop buyer) 3 District Army Commander, Brigadier Commander, Intelligence Officer, and other staff from Uganda People’s Defense Force 8 Livestock Market Director 1 Livestock Abattoir 1 Manager, Livestock Skins and Hides 1 Chair, Butcher and Meat Association (and trader) 1 CAHW Association Members 5 Agro-vet shop owner (and chair of the CAWH Association) 1 Krall Leaders 2 Whave staff (GHG IP) 4 Managers, SACCO 2 KAPDA (GHG IP) 1 Chair, KOPEIN (local Peace Organization) 1 Secretary, District Land Board (and member of District Risk Management Committee) 1 Representatives (Chair, Secretary) and member from District Elders Councils (chair, secretary, other members) 8 Mercy Corp Staff (previously under GHG, now under Apolou) 5 Total District Respondents 80 6 Sub-counties: 2 per district In Abim: Sidok Abim Town County In Kotido: Morulem Rengen In Kaabong: Nakepelimoru Lolileo Sub-County Local Council Chair - L2, one also serves as the Chair of the Sub-County Peace Committee and one also serves as the chair for the District Disaster Preparedness Committee 5 Sub-County Chiefs of Local Councils, one also serves as the Senior Assistant Secretary of the Local Council 5 SC Health Officer 2 SC Health Assistant 2 SC Water Officer and one Deputy Water Officer 2 Peace Committee members (combines members of parish and sub-county peace committees 50 Youth Peace Group members 2 School Hygiene Club members 9 Chair, Land Management Committee 1 Agriculture Extension Agents 5 1 A number of respondents at the district and sub-county levels served in multiple roles relevant to GHG. In these cases, individuals were interviewed on topics related to each role. Location Type of Respondent1 Totals Health Unit Management Committee members 15 Head Clinician for Health Facility 2 Animal Husbandry Officer (also serves as Agriculture Officer and as an agro-vet supplier) 1 Ag Extension Officer (also serves as Chair of the Disaster Risk Management Committee and as an agro-input agent for his village) and one Agricultural Extension Officer (also serves as the Livestock Extension Officer) 2 Agro-Vet Store manager 1 Director and one member of the Area Land Management Committee 2 Land Point Volunteer for Areal Land Management Committee 1 Director of AWOTID (GHG IP) 1 Krall leader 2 Agrovet Supplier 1 Total Sub-County Respondents 109 6 Villages: 2 per district Abim2 Aremo East Longelep Kotido Longar Atwilo Kaabong Chaicon Chumakuku Chair, Village Council, Vice Chair Local CouncilL3 5 Male Change Agent group members 12 Lead Mothers 21 Mother Care Group members 19 Non-participants in Mother Care Groups 16 Mother Care Group Plus members (includes at least 1 VSLA member) 12 Lead Farmers 4 Agriculture Input Agents (one also serves as the sub-county Agro Extension Agent) 4 Agro-vet Agent 1 Community Animal Health Workers 7 Male Agro-pastoralists 30 Female Agro-pastoralists – beneficiaries (some were VSLA members) 39 Female Agro-pastoralists – non-beneficiaries 35 Village Health Team members 4 Chair, members of Village Savings and Loan Associations 2 Women’s Economic Empowerment Committee members (some of these committee members were also VSLA members) 13 Water User Committee Members (in some villages referred to as WASH Committees) 20 Krall Leaders 2 Total Village Respondents 246 Kampala: Director of Mercy Corps programs in Uganda, and Senior Gender Advisor for Apolou (formerly serving under the GHG DFAP) 2 Total Number of Respondents Interviewed Across all Locations 435 2 In Abim, there were no health activities implemented in the villages of Aremo East and Longelep because of site selection parameters. ANNEX 5: GHG POPULATION CHARACTERISTICS AND TABULAR SUMMARY OF PBS INDICATORS Lower Upper FOOD SECURITY INDICATORS Average Household Dietary Diversity Score (HDDS) 3.6 3.4 3.9 944 41,560 1.9 0.13 2.1 Prevalence of moderate and severe food insecurity in the population, based on the Food Insecurity Experience Scale (FIES) [30 day recall] 89.1 86.4 91.7 1,148 50,116 25.4 1.35 1.8 Male and female adults 89.1 86.5 91.7 887 38,869 24.9 1.34 1.6 Adult female, no adult male 91.3 87.8 94.8 202 8,724 22.7 1.79 1.1 Adult male, no adult female 80.2 70.0 90.3 56 2,402 37.6 5.17 1.0 Child, no adults NA NA NA 3 122 NA NA NA POVERTY INDICATORS Per capita expenditures (as a proxy for income) of USG-assisted areas $.92 $.74 $1.11 6,640 291,088 $1.70 0.09 1.8 Male and female adults $.91 $.74 $1.09 5,625 246,964 $1.61 0.09 1.6 Adult female, no adult male $.86 $.62 $1.10 860 37,236 $1.50 0.12 1.1 Adult male, no adult female $1.80 $.80 $2.81 137 6,220 $5.05 0.50 0.8 Child, no adults NA NA NA 18 668 NA NA NA Prevalence of poverty: Percent of people living on less than $1.25/day 87.0 82.8 91.2 6,640 291,088 33.6 2.06 2.1 Male and female adults 87.3 83.1 91.5 5,625 246,964 31.8 2.07 2.0 Adult female, no adult male 88.5 82.3 94.6 860 37,236 37.2 3.02 1.2 Adult male, no adult female 65.2 42.5 87.9 137 6,220 73.6 11.20 1.2 Child, no adults NA NA NA 18 668 NA NA NA Mean depth of poverty 53.3 49.0 57.6 6,640 291,088 28.0 2.13 2.6 Male and female adults 53.2 49.0 57.3 5,625 246,964 26.5 2.05 2.3 Adult female, no adult male 55.5 48.5 62.5 860 37,236 32.1 3.48 1.5 Adult male, no adult female 41.1 29.0 53.1 137 6,220 54.1 5.96 0.9 Child, no adults NA NA NA 18 668 NA NA NA WASH INDICATORS Percentage of households using an improved source of drinking water 37.9 32.8 43.1 1,178 51,441 48.5 2.55 1.8 Percentage of households using improved sanitation facilities 13.7 8.9 18.6 1,178 51,441 34.4 2.39 2.4 Percentage of households with soap and water at a handwashing station commonly used by family members 2.6 0.3 4.9 1,178 51,441 16.0 1.13 2.4 AGRICULTURAL INDICATORS Percentage of farmers who used financial services in the past 12 months 31.1 26.4 35.7 1,506 68,704 46.3 2.29 1.9 Male 29.4 24.5 34.3 666 30,206 45.7 2.43 1.4 Female 32.4 26.8 38.0 840 38,498 46.7 2.78 1.7 Percentage of farmers who practiced value chain activities promoted by the project in the past 12 months 21.0 16.6 25.4 1,506 68,704 40.8 2.18 2.1 Male 24.8 19.1 30.4 666 30,206 43.3 2.78 1.7 Female 18.1 13.9 22.3 840 38,498 38.4 2.09 1.6 Percentage of farmers who used at least three sustainable agricultural practices in the past 12 months 37.2 32.8 41.7 1,506 68,704 48.4 2.20 1.8 Male 44.6 39.3 49.8 666 30,206 49.8 2.60 1.3 Female 31.4 26.6 36.3 840 38,498 46.3 2.39 1.5 Percentage of farmers who used at least two sustainable agricultural (crop) practices (past 12 months) 34.8 30.1 39.4 1,506 68,704 47.8 2.31 1.9 Percentage of farmers who used at least two sustainable agricultural (livestock) practices (past 12 months) 17.8 14.5 21.0 1,506 68,704 31.9 1.61 1.6 Percentage of farmers who used at least one sustainable agricultural (NRM) practice (past 12 months) 10.4 7.1 13.8 1,506 68,704 30.6 1.66 2.1 Percentage of farmers who used improved storage practices in the past 12 months 51.0 42.7 59.4 1,506 68,704 50.0 4.12 3.2 Male 50.9 42.5 59.3 666 30,206 50.1 4.16 2.1 Female 51.1 42.3 59.9 840 38,498 49.9 4.34 2.5 WOMEN'S HEALTH AND NUTRITION INDICATORS Table A5.1. FFP Uganda Endline Indicators - GHG Project Area Indicators, 95% Confidence Intervals and Base Population [Uganda, 2018] Indicator Value Confidence Interval Number of Records Weighted Population Standard Deviation Standard Error DEFT Lower Upper Table A5.1. FFP Uganda Endline Indicators - GHG Project Area Indicators, 95% Confidence Intervals and Base Population [Uganda, 2018] Indicator Value Confidence Interval Number of Records Weighted Population Standard Deviation Standard Error DEFT Prevalence of underweight women 24.8 20.5 29.0 829 39,412 43.2 2.08 1.4 Women’s Dietary Diversity Score (WDDS) 3.1 2.9 3.3 1,070 57,901 1.5 0.11 2.3 CHILDREN'S HEALTH AND NUTRITION INDICATORS Prevalence of underweight children under 5 years of age (Total) 27.7 23.6 31.8 1,069 47,472 44.8 2.01 1.5 Male 31.2 26.1 36.3 525 23,801 45.8 2.52 1.3 Female 24.2 20.2 28.1 544 23,670 43.3 1.96 1.1 Prevalence of stunted children under 5 years of age (Total) 37.6 33.3 41.8 1,061 47,092 48.4 2.09 1.4 Male 40.5 35.0 46.0 521 23,579 48.6 2.72 1.3 Female 34.6 29.8 39.3 540 23,513 48.1 2.35 1.1 Prevalence of wasted children under 5 years of age (Total) 9.7 7.6 11.8 1,057 46,857 29.7 1.04 1.1 Male 12.2 8.7 15.7 521 23,561 32.4 1.73 1.2 Female 7.3 4.8 9.8 536 23,296 26.2 1.24 1.1 Percentage of children under age 5 with diarrhea in the last two weeks (Total) 35.4 31.2 39.7 1,156 51,264 47.9 2.10 1.5 Male 33.4 28.0 38.9 559 25,311 46.7 2.69 1.4 Female 37.4 32.2 42.5 597 25,953 48.9 2.54 1.3 Percentage of children under age 5 with diarrhea treated with ORT (Total) 81.8 75.7 87.9 416 18,163 38.6 3.02 1.6 Male 79.6 71.7 87.5 191 8,460 40.0 3.90 1.3 Female 83.7 76.4 91.1 225 9,703 37.6 3.63 1.4 Prevalence of exclusive breast-feeding of children under six months of age 71.7 61.9 81.5 113 5,162 45.3 4.85 1.1 Male 71.0 58.1 83.9 56 2,752 43.0 6.37 1.1 Female 72.5 60.7 84.3 57 2,410 46.1 5.84 1.0 Prevalence of children 6-23 months of age receiving a minimum acceptable diet (MAD) 9.4 5.7 13.2 306 13,232 29.3 1.85 1.1 Male 9.2 3.7 14.6 156 6,791 28.9 2.69 1.2 Female 9.7 4.7 14.8 150 6,441 30.3 2.50 1.0 CUSTOM INDICATORS Percentage of respondents reporting increased movement in areas that were previously not accessible due to insecurity 87.8 83.7 91.8 1,161 50,704 32.8 2.00 2.1 Percentage of households with access to a sanitation facility – not necessarily improved 49.0 38.8 59.2 1,171 51,105 50.0 5.04 3.4 Average number of crops produced per farmer in the past 12 months 2.5 2.4 2.7 1,495 68,193 1.4 0.07 1.8 Percentage of farmers adopting farmer managed natural regeneration practices in the past 12 months 26.8 23.9 29.7 1,506 68,704 44.3 1.42 1.2 Percentage of livestock owners accessing government or private sector vet care in the past 12 months 15.7 10.4 20.9 524 24,172 36.4 2.59 1.6 Percentage of mothers of children (0-23 months) attending 4 or more ANC visits with youngest child 77.1 71.8 82.5 380 20,099 42.1 2.64 1.2 WEAI COMPONENTS Female- Access to and decision on credit 30.0 24.5 35.6 1,027 44,718 45.9 2.76 1.9 Male- Access to and decision on credit 27.2 21.3 33.1 626 27,057 44.5 2.91 1.6 Female- Control over and use of income 75.6 71.1 80.2 1,027 44,718 43.0 2.25 1.7 Male- Control over and use of income 75.5 69.8 81.3 626 27,057 43.0 2.85 1.7 Female- Group membership 46.0 41.1 50.8 935 40,773 49.9 2.39 1.5 Male- Group members 47.2 42.6 51.8 577 25,109 50.0 2.29 1.1 NA : Not available 2013 Baseline 2018 Endline Raw Difference (Endline - Baseline) Significance Level1 FOOD SECURITY INDICATORS Average Household Dietary Diversity Score (HDDS) 2.2 3.6 1.4 *** 2,078 944 POVERTY INDICATORS Per capita expenditures (as a proxy for income) of USG-assisted areas $0.58 $0.92 $0.34 *** 15,127 6,640 Male and female adults $0.58 $0.91 $0.33 *** 13,613 5,625 Adult female, no adult male $0.63 $0.86 $0.23 ns 1,384 860 Adult male, no adult female NA NA NA NA 130 137 Child, no adults NA NA NA NA 0 18 Prevalence of poverty: Percent of people living on less than $1.25/day2 93.2 87.0 -6.2 ** 15,127 6,640 Male and female adults 93.3 87.3 -6.0 * 13,613 5,625 Adult female, no adult male 92.6 88.5 -4.1 ns 1,384 860 Adult male, no adult female NA NA NA NA 130 137 Child, no adults NA NA NA NA 0 18 Mean depth of poverty (expressed as percent of poverty line) 61.5 53.3 -8.2 ** 15,127 6,640 Male and female adults 61.9 53.2 -8.7 ** 13,613 5,625 Adult female, no adult male 57.1 55.5 -1.6 ns 1,384 860 Adult male, no adult female NA NA NA NA 130 137 Child, no adults NA NA NA NA 0 18 WASH INDICATORS Percentage of households using an improved source of drinking water 37.4 37.9 0.5 ns 2,399 1,178 Percentage of households using improved sanitation facilities 12.7 13.7 1.0 ns 2,399 1,178 Percentage of households with soap and water at a handwashing station commonly used by family members 11.0 2.6 -8.4 ** 2,323 1,178 AGRICULTURAL INDICATORS Percentage of farmers who used financial services in the past 12 months 31.6 31.1 -0.5 ns 2,721 1,506 Male 33.7 29.4 -4.3 ns 1,245 666 Female 29.9 32.4 2.5 ns 1,476 840 Percentage of farmers who used at least three sustainable agricultural practices in the past 12 months 16.7 37.2 20.5 *** 2,754 1,506 Male 20.8 44.6 23.8 *** 1,256 666 Female 13.3 31.4 18.1 *** 1,498 840 Percentage of farmers who used at least two sustainable agricultural (crop) practices (past 12 months) 19.2 34.8 15.6 *** 2,752 1,506 Percentage of farmers who used at least two sustainable agricultural (livestock) practices (past 12 months) 10.5 17.8 7.3 *** 2,752 1,506 Percentage of farmers who used at least one sustainable agricultural (NRM) practice (past 12 months) 16.4 10.4 -6.0 * 2,752 1,506 Percentage of farmers who used improved storage practices in the past 12 months 48.7 51.0 2.3 ns 2,719 1,506 Male 56.3 50.9 -5.4 ns 1,167 666 Female 42.4 51.1 8.7 † 1,352 840 WOMEN'S HEALTH AND NUTRITION INDICATORS Prevalence of underweight women 20.9 24.8 3.9 ns 1,776 829 Women’s Dietary Diversity Score (WDDS) 2.1 3.1 1.0 *** 2,233 1,070 CHILDREN'S HEALTH AND NUTRITION INDICATORS Prevalence of underweight children under 5 years of age (Total) 21.5 27.7 6.2 * 2,747 1,069 Male 23.8 31.2 7.4 † 1,323 525 Female 19.2 24.2 5.0 * 1,424 544 Prevalence of stunted children under 5 years of age (Total) 32.0 37.6 5.6 † 2,747 1,061 Table A5.2. Comparison of Baseline and Endline Indicators - GHG Project Area [Uganda 2013, 2018] Number of Observations Baseline Endline 2013 Baseline 2018 Endline Raw Difference (Endline - Baseline) Significance Level1 Table A5.2. Comparison of Baseline and Endline Indicators - GHG Project Area [Uganda 2013, 2018] Number of Observations Baseline Endline Male 33.7 40.5 6.8 ns 1,323 521 Female 30.4 34.6 4.2 ns 1,424 540 Prevalence of wasted children under 5 years of age (Total) 13.7 9.7 -4.0 * 2,747 1,057 Male 14.3 12.2 -2.1 ns 1,323 521 Female 13.1 7.3 -5.8 *** 1,424 536 Percentage of children under age 5 with diarrhea in the last two weeks (Total) 21.3 35.4 14.1 *** 2,886 1,156 Male 22.6 33.4 10.8 *** 1,398 559 Female 20.1 37.4 17.3 *** 1,488 597 Percentage of children under age 5 with diarrhea treated with ORT (Total)3 89.7 81.8 -7.9 * 582 416 Male 89.6 79.6 -10.0 * 289 191 Female 89.9 83.7 -6.2 ns 293 225 Prevalence of exclusive breast-feeding of children under six months of age 60.8 71.7 10.9 ns 311 113 Male 61.4 71.0 9.6 ns 159 56 Female 60.1 72.5 12.4 ns 152 57 Prevalence of children 6-23 months of age receiving a minimum acceptable diet (MAD) 2.1 9.4 7.3 *** 803 306 Male 2.3 9.2 6.9 * 407 156 Female 1.9 9.7 7.8 ** 396 150 CUSTOM INDICATORS Percentage of respondents reporting increased movement in areas that were previously not accessible due to insecurity 52.4 87.8 35.4 *** 2,399 1,161 Percentage of households with access to a sanitation facility – not necessarily improved 40.5 49.0 8.5 ns 2,399 1,171 Average number of crops produced per farmer in the past 12 months 2.6 2.5 -0.1 ns 2,739 1,495 Percentage of farmers adopting farmer managed natural regeneration practices in the past 12 months 16.4 26.8 10.4 *** 2,721 1,506 Percentage of mothers of children (0-23 months) attending 4 or more ANC visits with youngest child 49.2 77.1 27.9 *** 944 380 WEAI COMPONENTS Female- Access to and decision on credit 22.6 30.0 7.4 * 2,309 1,027 Male- Access to and decision on credit 21.8 27.2 5.4 ns 1,829 626 Female- Control over and use of income 61.7 75.6 13.9 *** 2,332 1,027 Male- Control over and use of income 60.1 75.5 15.4 *** 1,971 626 Female- Group membership 65.0 46.0 -19.0 *** 1,736 935 Male- Group members 56.0 47.2 -8.8 * 1,380 577 1 ns = not significant, † p<0.1,* p<0.05, ** p<0.01, *** p<0.001 2 Expressed in constant 2010 USD 3 The baseline indicator result was recalculated to exclude the use of “increased fluids” which is considered a treatment for diarrhea but not an ORT treatment. NA : Not available Note: Comparisons between baseline and endline for value chain activties and farmers accessing vet care services are not provided due to methodological differences in the data collected at baseline and endline. 2013 Baseline 2018 Endline Total population 559,850 291,088 Male 269,269 135,530 Female 290,581 155,558 Total households 87,812 51,441 Male and Female Adults 77,495 39,744 Adult Female no Adult Male 9,660 8,813 Adult Male no Adult Female 658 2,618 Child No Adults 0 266 Women of reproductive age (15-49 years) 118,040 57,901 Children ages 0-59 months 111,334 51,264 Males ages 0-59 months 55,689 25,311 Females ages 0-59 months 55,645 25,953 Children ages 6-23 months 34,540 13,232 Males ages 6-23 months 18,039 6,791 Females ages 6-23 months 16,501 6,441 Children ages 0-5 months 12,645 5,162 Males ages 0-5 months 6,712 2,752 Females ages 0-5 months 5,933 2,410 Table A5.3. Population and Household Characteristics in the GHG Project Area [Uganda, 2013, 2018] Source: FFP Baseline (2013) and Endline (2018) PBS in Uganda ANNEX 6: SOURCES OF INFORMATION Documents Reviewed Advanced Marketing Systems, Final Evaluation of the Northern Karamoja Growth, Health and Governance Project, January 9, 2017. Baseline Study for the Title II Development Food Assistance programs in Uganda, ICF International, 2014. Famine Early Warning Systems Network (FEWS NET) Forecast and Food Security Outlook, Uganda, July 2018. Famine Early Warning Systems Network (FEWS NET) Seasonal Monitor for East Africa, July 6, 2018. Famine Early Warning Systems Network (FEWS NET) Seasonal Monitor for East Africa, Floods in Sudan as Ethiopia, Southern Sudan and Uganda Experience Seasonal Water Deficits, August 17, 2018. Famine Early Warning Systems Network (FEWS NET), East Africa, Uganda Seasonal Update: Second Season Bimodal Harvests Begin While Karamoja Endures Harsh Dry Season. November 2018. FANTA, Food Security Desk Review for Karamoja, Uganda, January 2017. FANTA, Sustaining Development: A Synthesis of Results from a Four Country Study of Sustainability and Exit Strategies Among Development Food Assistance Projects, December 2016. Feinstein International Center, A Better Balance: Revitalized Pastoral Livelihoods in Northern Uganda, July 2016. Feinstein International Center, Youth on the Margins in Search of Decent Work in Northern Uganda, Report 14, December 2016. Feinstein International Center, Trading, Power, and Politics Youth in a Northern Uganda Marketplace. Report 15, December 2016. Feinstein International Center, Five Years On: Livelihood Advances, Innovations, and Continuing Challenges in Karamoja, Uganda. 2017. Food Security and Nutrition Assessment in Karamoja Sub-Region, Abim District Report, IBFAM Uganda, Department of International Development, July 2017 Food Security and Nutrition Assessment in Karamoja Sub-Region, Kaabong District Report, IBFAM Uganda, Department of International Development, July 2017 Food Security and Nutrition Assessment in Karamoja Sub-Region, Kotido District Report, IBFAM Uganda, Department of International Development, July 2017 FY 2017 Annual Review Report for the Northern Karamoja Growth, Health and Governance Development Food Assistance Project, Mercy Corps, 2017. FY 2016 Annual Review Report for the Northern Karamoja Growth, Health and Governance Development Food Assistance Project, Mercy Corps, 2017. FY 2015 Annual Review Report for the Northern Karamoja Growth, Health and Governance Development Food Assistance Project, Mercy Corps, 2016. FY 2014 Annual Review Report for the Northern Karamoja Growth, Health and Governance Development Food Assistance Project, Mercy Corps, 2015 FY 2013 Annual Review Report for the Northern Karamoja Growth, Health and Governance Development Food Assistance Project, Mercy Corps, 2014. FY 2013- FY 2017 Quarterly Reports for the Northern Karamoja Growth, Health and Governance Development Food Assistance Project, Mercy Corps. GHG, Gender and Behavior Change Impact Assessment Report, Mercy Corps, February 2017. GHG, Karamoja Stress Resilience Assessment, August, 2016. GHG, Livelihood Dynamics in Northern Karamoja: A Participatory Baseline Study for GHG, Mercy Corps, May 2013. GHG, Maintaining Momentum for Dynamic Change: Lessons Learned from Gender Integration into the GHG Program. Mercy Corps Power Point Presentation. Date Unknown. GHG, Navigating Complexity: Adaptive Management at the Northern Karamoja GHG Project, Mercy Corps and Engineers Without Borders, Date Unknown. GHG, Protocol on Formation of Promoters and Mother Care Groups, date not included. GHG, Rapid Assessment Health Facility Report, Mercy Corps, October 2013. GHG, Strengthening Health Facility Capacity in Maternal and Child Health Service Delivery, Mercy Corps, Date Unknown GHG, Women’s Groups: Results of Mapping Exercise. Mercy Corps. August 2013. Husseini et a.l Thresholds of Socioeconomic and Environmental Conditions Necessary to Escape From Childhood Malnutrition: a Natural Experiment in Rural Gambia, BMC, 2018. Mercy Corps Indicator Performance Tracking Table (IPTT) Revised. 2018 Mercy Corps SUSTAIN Narrative Proposal, June 14, 2012. Pickering et al., Effect of a Community Led Sanitation Intervention on Childhood Diarrhea on Child Growth in Rural Mali: A Cluster Randomized Controlled Trial, Lancet Global Health, Vol. 3, e. pp. 701- 711, 2015. TOPS Case Study, Seeds E-Voucher: An Approach to Inclusive Agri-Market Development Case Study for GHG, October 2016. USAID Climate Risk Profile: Climate Risk Screening for Food Security – Karamoja Region, Uganda, January 2017 USAID/FFP, 2016-2025 Food Assistance and Food Security Strategy, October 2016. World Food Programme, The Impacts of Climate Change on Food Security and Livelihoods, C-ADAPT Paper, Government of Sweden, 2016 World Vision, Barrier Analysis, GHG Project, May 2013. World Vision, Northern Karamoja Village Health Team Functionality Assessment, 2013. ANNEX 7: LOGISTIC REGRESSION RESULTS This annex provides the results of multivariate logistic regression models conducted for three outcomes: (1) child stunting, (2) child wasting, and (3) daily per capita expenditures These models were developed to explore factors that might help to explain these outcomes. A descriptive summary of the statistically significant findings for each model is provided below followed by the regression results for child stunting (Table A7.1), child wasting (Table A7.2) and daily per capita expenditures (Table A7.3). Child Stunting  The regression model indicates a higher prevalence of child stunting at endline (EL) compared to baseline (BL) [Model 1]. This re-confirms the increase in stunting rates from BL-EL reported in the draft report.                When households’ project participation status is controlled for (Model 2), the difference in stunting between the BL and EL is no longer significant. This would suggest that child stunting likely did not worsen from BL to EL for households that participated in project activities. Among the different types of project activities, children in households that received “food rations and nutrition training”, have, on average, a lower prevalence of stunting at the endline (Models 3-6). Child stunting is more prevalent among children between the ages of 12-23 months and 24-35 months compared to children below 6 months regardless of their households’ project participation and socio￾economic status. (Model 4) Among households with similar socio-economic conditions and project participation status, female children on average are less likely to be stunted than male children (Models 5 & 6). Households with at least one secondary or higher-level educated member, on average, tend to have a lower prevalence of child stunting compared to households with members having no formal education regardless of households’ socio-economic conditions and project participation status (Model 5 & 6). Households that have access to an improved handwashing station, and have access to agricultural financial services tend to have lower stunting rates regardless of other socio-economic conditions. (Models 5 & 6). Children who experienced diarrhea in the past two weeks, on average, tend to have higher stunting rates regardless of their households’ project participation status and socio-economic conditions (Model 7). The prevalence of stunting is lower in the Kotido and Abim Districts compared to the Kaabong District regardless of other socio-economic conditions and households’ project participation status (Model 7). Child Wasting The regression model indicates a lower prevalence of wasting at endline compared to baseline, consistent with bivariate results. Also, child wasting at EL is lower compared to BL regardless of households’ project participation status and other socio-economic conditions (Models 1-7). Children in households that received “food rations and nutrition training” tend to have a lower prevalence of wasting compared to children in households that did not, net of other socio-economic background and project participation types (Models 5-7). Children in households that received “food rations only” tend to have a higher prevalence of wasting compared to children in households that did not, net of their participation in other interventions and other s ocio-economic background (Model 3-7). Overall, households’ participation in any type of project activity contributes to about a 3 percentage point decline in the predicted probability of wasting. Wasting is more prevalent among children between the age of 6-11, 12-23, and 24-35 months compared to children below 6 months regardless of households’ project participation and socio-economic status (Models 5-7). Among the households that share similar socio-economic and project participation status, female children, on average, are less likely to be wasted than male children (Models 5-7). Children in households at the bottom 2nd and 3rd consumption quintiles have higher rates of wasting than the children in the poorest quintile, regardless of households’ project participation status and other socio￾economic conditions (Models 5-7).             Wasting rates are higher in households with more family members compared to households with fewer family members regardless of households’ project participation status and other socio-economic conditions (Models 5-7). Households with at least one member with a primary-level education, on average, have a lower prevalence of child wasting compared to households with members having no formal education regardless of their socio-economic conditions and project participation status (Models 5-7). Households with an improved drinking water source tend to have a lower prevalence of child wasting regardless of project participation and socio-economic status (Models 5-7). The prevalence of wasting is lower in the Kotido and Abim Districts compared to the Kaabong District regardless of households’ socio-economic and project participation status (Model 7). Per Capita Consumption Expenditures: Bivariate results indicate an increase in daily per capita expenditures at endline compared to baseline. Regression models also indicated that per capita expenditures improved over the project period regardless of households’ project participation and other socio-economic status (Models 1-7). Households’ participation in different types of project interventions contributed to a 3 percentage point increase (0.34 to 0.37) in predicted probability of per capita expenditures (Models 1-3). Households that participated in “agriculture training and other activities” tend to have lower per capita expenditures compared to those who did not participate in this activity, regardless of their participation in other activities and socio-economic status (Models 3-5). Households with more family members tend to have lower per capita expenditures than those with fewer family members regardless of households’ project participation status and other socio-economic conditions (Models 4-5). Households with at least one primary-level or secondary or higher-level educated member, on average, tend to have higher per capita expenditures compared to households with members having no formal education regardless of their other socio-economic conditions and project participation status (Models 5- 7). Households practicing sustainable agriculture are likely to have higher per capita expenditures compared to households that don’t, regardless of their project participation status and other socio-economic conditions (Models 4-5). Households with access to agricultural financial services and improved storage practices are likely to have higher per capita expenditures, regardless of project participation and socio-economic status (Model 4). The difference however is diluted when geographic conditions of different districts within GHG area are controlled (or considered same as Kaabong district) (Model 5). Households in Kotido and Abim are likely to have higher per capita expenditure compared to households in Kaabong, regardless of project participation and other socio-economic status (Model 5). Table A7.1: Results of logistic regression of the prevalence of moderate-to-severe stunting, MC-GHG [Uganda 2013, 2018] Variables Model 1 Model 2 Model 3 Model 4 Model 5 Model 6 Model 7 odds ratio SE odds ratio SE odds ratio SE odds ratio SE odds ratio SE odds ratio SE odds ratio SE Year (2018) (2013= reference year)) 1.28+ (0.16) 1.17 (0.17) 1.17 (0.17) 1.17 (0.17) 1.19 (0.18) 1.14 (0.16) 0.82 (0.15) Project participation variables HHs regularly participated in GHG activities 1.01 (0.29) 0.92 (0.26) 0.98 (0.29) 0.80 (0.23) 0.77 (0.23) 1.46 (1.29) HHs received food ration 1.85 (0.76) 1.93 (0.78) 1.91 (0.81) 1.94 (0.83) 2.02 (0.88) 1.27 (1.60) HHs regularly received nutrition training 1.16 (0.53) 1.46 (0.76) 1.32 (0.70) 1.33 (0.70) 1.41 (0.77) 11.80 (22.06) HHs participated in any other activities 0.99 (0.19) 1.36 (0.69) 1.50 (0.78) 1.59 (0.81) 1.61 (0.85) 0.38 (0.44) HHs received food ration and nutrition training 0.44 (0.23) 0.22* (0.14) 0.24* (0.17) 0.27+ (0.19) 0.22* (0.16) 0.05 (0.11) HHs received agriculture related training 0.58 (0.26) 0.62 (0.29) 0.58 (0.28) 0.45 (0.26) 0.48 (0.29) 2.03 (2.76) HHs received agriculture and nutrition training 2.36 (1.45) 1.75 (1.20) 2.04 (1.39) 2.69 (1.91) 2.44 (1.80) 0.05 (0.12) HHs received food ration, agriculture and nutrition training 2.86 (2.01) 2.50 (1.77) 1.96 (1.44) 2.18 (1.64) 42.25* (72.98) HHs received agriculture training and other activities 0.76 (0.46) 0.64 (0.39) 0.78 (0.44) 0.76 (0.43) 5.79 (7.50) HHs received food ration, nutrition, agriculture training and other activities 0.68 (0.39) 0.72 (0.43) 0.61 (0.44) 0.64 (0.47) 0.11+ (0.13) Child age (<6 month= reference age) 6 to 11 month 1.20 (0.24) 1.09 (0.26) 1.12 (0.27) 0.09** (0.08) 12 to 23 month 1.87*** (0.31) 2.01*** (0.37) 2.05*** (0.37) 0.16* (0.14) 24 to 35 month 2.17*** (0.30) 2.07*** (0.36) 2.12*** (0.36) 36 to 47 month 1.43* (0.24) 1.39 (0.29) 1.38 (0.29) 48 to 59 month 1.18 (0.21) 1.10 (0.23) 1.10 (0.23) Child's gender (ref: male) 0.83 (0.12) 0.72** (0.08) 0.71** (0.08) 0.74 (0.15) Consumption expenditure quintiles (Poorest quintile= reference group) Quintile 2 1.33* (0.18) 1.32* (0.17) 0.81 (0.23) Quintile 3 1.22 (0.18) 1.24 (0.19) 1.50 (0.42) Quintile 4 1.02 (0.17) 1.05 (0.17) 0.75 (0.20) Quintile 5 (Richest) 0.86 (0.14) 0.90 (0.14) 0.63 (0.19) Household size 0.98 (0.02) 0.99 (0.02) 0.99 (0.04) Gendered household type (both male and female adults= reference group) Adult female only households 0.95 (0.13) 0.99 (0.15) 0.63 (0.21) Adult male only households 0.79 (0.86) 0.77 (0.83) Household head's education (Ref: HHs with no educated HH members) Primary level education 0.86 (0.14) 0.88 (0.13) 1.17 (0.29) Secondary or higher education 0.70** (0.09) 0.71* (0.11) 0.93 (0.28) WASH Households with soap and water at handwashing station 0.61* (0.14) 0.63* (0.14) 0.97 (0.25) Variables Model 1 Model 2 Model 3 Model 4 Model 5 Model 6 Model 7 odds ratio SE odds ratio SE odds ratio SE odds ratio SE odds ratio SE odds ratio SE odds ratio SE Households using improved water sources 0.97 (0.11) 1.00 (0.12) 0.91 (0.17) Agriculture variables Households using minimum financial services 0.80+ (0.10) 0.82+ (0.09) 0.65+ (0.15) Households using minimum value chain practices 1.11 (0.13) 1.09 (0.13) 0.90 (0.17) Households using minimum sustainable agriculture practices 1.15 (0.14) 1.22 (0.15) 1.19 (0.26) Households using improved storage practices 0.82+ (0.09) 0.86 (0.09) 1.07 (0.19) Household consuming >=4 food groups 1.05 (0.12) 1.03 (0.12) 0.88 (0.21) Geographic districts (Kaabong= reference district) Kotido 0.74** (0.08) 0.57** (0.11) Abim 0.70+ (0.15) 0.58+ (0.17) CHN Variables for children 6-23 months % of children 0-59 months (1,825 days) of age who had diarrhea in the prior two 1.71** (0.34) children 6-23 months receiving a minimum acceptable diet 1.10 (0.46) Constant 0.47*** (0.04) 0.47*** (0.04) 0.47*** (0.04) 0.41** (0.12) 0.72 (0.20) 0.79 (0.23) 11.71* (11.57) Observations (<5years old children in BL and EL data) 3,808 3,808 3,808 3,795 2,954 2,954 840 F-statistic 3.88 2.78 2.80 5.52 7.30 7.03 1.92 Prob>F 0.05 0.01 0.00 0.00 0.00 0.00 0.01 Standard error in parenthesis *** p<0.001, ** p<0.01, * p<0.05, + p<0.10 Table A7.2: Results of logistic regression of the prevalence of moderate-to-severe wasting, MC-GHG [Uganda 2013, 2018] Variables Model 1 Model 2 Model 3 Model 4 Model 5 Model 6 Model 7 odds ratio SE odds ratio SE odds ratio SE odds ratio SE odds ratio SE odds ratio SE odds ratio SE Year (2018) (2013= reference year)) 0.68* (0.11) 0.65* (0.12) 0.65* (0.12) 0.66* (0.13) 0.67* (0.13) 0.60** (0.12) 0.57** (0.12) Project participation variables HHs regularly participated in GHG activities 0.45 (0.39) 0.43 (0.31) 0.44 (0.32) 0.23 (0.28) 0.19 (0.25) 0.20 (0.27) HHs received food ration 3.67 (3.07) 3.79+ (2.83) 3.77+ (2.72) 10.69** (9.65) 12.96** (12.52) 12.55** (12.01) HHs regularly received nutrition training 1.47 (1.36) 1.33 (1.25) 1.34 (1.25) 2.67 (3.55) 3.03 (4.14) 2.69 (3.62) HHs participated in any other activities 1.96 (0.81) 2.17 (1.46) 2.26 (1.52) 2.38 (1.87) 2.57 (2.01) 2.65 (2.03) HHs received food ration and nutrition training 0.22+ (0.19) 0.31 (0.31) 0.31 (0.30) 0.10* (0.10) 0.06* (0.07) 0.07* (0.07) HHs received agriculture related training 1.81 (1.70) 1.83 (1.93) 1.93 (2.02) 1.17 (1.11) 1.52 (1.55) 1.41 (1.45) HHs received agriculture and nutrition training 0.70 (0.76) 0.71 (0.74) 0.70 (0.74) 1.10 (1.16) 0.90 (1.03) 1.00 (1.15) HHs received agriculture training and other activities 1.16 (1.16) 1.02 (1.00) 0.80 (0.98) 0.71 (0.91) 0.72 (0.90) HHs received food ration, nutrition, agriculture training and other activities 0.56 (0.40) 0.57 (0.42) 0.73 (0.55) 0.85 (0.75) 0.81 (0.70) Child age (<6 month= reference age) 6 to 11 month 1.50 (0.40) 2.70** (0.94) 3.00** (1.09) 2.81** (1.03) 12 to 23 month 1.82* (0.49) 3.39*** (1.16) 3.66*** (1.26) 3.44*** (1.21) 24 to 35 month 1.25 (0.32) 1.74+ (0.56) 1.86+ (0.62) 1.79+ (0.60) 36 to 47 month 0.91 (0.25) 1.50 (0.50) 1.49 (0.51) 1.47 (0.51) 48 to 59 month 0.70 (0.21) 1.04 (0.36) 1.04 (0.36) 1.04 (0.36) Sex (ref: male) 0.78 (0.13) 0.74* (0.11) 0.72* (0.11) 0.72* (0.11) Consumption expenditure quintiles (Poorest quintile= reference group) Quintile 2 1.62* (0.33) 1.63* (0.32) 1.61* (0.31) Quintile 3 1.86** (0.44) 1.93** (0.44) 1.93** (0.44) Quintile 4 1.15 (0.26) 1.24 (0.27) 1.22 (0.26) Quintile 5 (Richest) 0.83 (0.23) 0.92 (0.25) 0.92 (0.25) Household size 1.06* (0.03) 1.09*** (0.03) 1.09** (0.03) Gendered household type (both male and female adults= reference group) Adult female only households 0.79 (0.22) 0.90 (0.26) 0.90 (0.26) Adult male only households 1.12 (0.77) 1.12 (0.64) 1.10 (0.63) Variables Model 1 Model 2 Model 3 Model 4 Model 5 Model 6 Model 7 odds ratio SE odds ratio SE odds ratio SE odds ratio SE odds ratio SE odds ratio SE odds ratio SE Household head's education (Ref: HHs with no educated HH members) Primary level education 0.62+ (0.15) 0.63* (0.14) 0.61* (0.14) Secondary or higher education 1.04 (0.36) 1.10 (0.36) 1.09 (0.34) WASH Households with soap and water at handwashing station 0.71 (0.21) 0.79 (0.23) 0.79 (0.23) Households using improved water sources 0.67** (0.10) 0.73* (0.11) 0.73* (0.11) Agriculture variables Households using minimum financial services 1.10 (0.16) 1.19 (0.17) 1.19 (0.17) Households using minimum value chain practices 0.96 (0.13) 0.92 (0.12) 0.90 (0.12) Households using minimum sustainable agriculture practices 0.98 (0.18) 1.17 (0.23) 1.18 (0.23) Households using improved storage practices 0.95 (0.13) 1.09 (0.15) 1.10 (0.15) Household consuming >=4 food groups 0.96 (0.18) 0.91 (0.16) 0.90 (0.16) Geographic districts (Kaabong= reference district) Kotido 0.37*** (0.07) 0.36*** (0.07) Abim 0.36*** (0.08) 0.37*** (0.08) CHN Variables for children 6-23 months % of children 0-59 months (1,825 days) of age who had diarrhea in the prior two 1.36 (0.27) Constant 0.16*** (0.02) 0.16*** (0.02) 0.16*** (0.02) 0.19*** (0.08) 0.09*** (0.04) 0.11*** (0.05) 0.11*** (0.05) Observations (<5years old children in BL and EL data) 3,804 3,804 3,804 3,791 2,951 2,951 2,943 F-statistic 5.84 1.78 2.41 3.63 3.23 3.86 4.02 Prob>F 0.02 0.09 0.01 0.00 0.00 0.00 0.00 Standard error in parenthesis *** p<0.001, ** p<0.01, * p<0.05, + p<0.10 Table A7.3: Results of OLS regression of the per capita consumption expenditures, MC-GHG [Uganda 2013, 2018] Variables Model 1 Model 2 Model 3 Model 4 Model 5 Coeff SE Coeff SE Coeff SE Coeff SE Coeff SE Year (2018) (2013= reference year)) 0.34*** (0.10) 0.37*** (0.10) 0.37*** (0.10) 0.21* (0.10) 0.29** (0.09) Project participation variables HHs regularly participated in GHG activities -0.02 (0.36) -0.25 (0.34) -0.22 (0.36) -0.22 (0.35) HHs received food ration 0.45 (0.69) 0.50 (0.67) 0.52 (0.74) 0.53 (0.72) HHs regularly received nutrition training -0.33 (0.40) -0.33 (0.41) -0.27 (0.45) -0.30 (0.46) HHs participated in any other activities -0.01 (0.18) 0.76 (0.52) 0.78 (0.50) 0.81 (0.50) HHs received food ration and nutrition training -0.47 (0.71) -0.40 (0.70) -0.45 (0.78) -0.40 (0.79) HHs received agriculture related training 1.23 (1.02) 1.44 (1.03) 1.66 (1.13) 1.63 (1.12) HHs received agriculture and nutrition training -1.20 (1.01) -1.10 (1.02) -1.36 (1.11) -1.30 (1.10) HHs received food ration, agriculture and nutrition training 0.06 (0.19) 0.06 (0.19) 0.02 (0.17) HHs received agriculture training and other activities -0.95+ (0.51) -0.96+ (0.49) -1.00* (0.50) HHs received food ration, nutrition, agriculture training and other activities -0.23 (0.22) -0.12 (0.23) -0.13 (0.23) Household head's education (Ref: HHs with no educated HH members) Primary level education 0.14* (0.06) 0.11+ (0.07) Secondary or higher education 0.49*** (0.10) 0.41** (0.12) Household size -0.04*** (0.01) -0.04** (0.01) Gendered household type (both male and female adults= reference group) Adult female only households 0.09 (0.07) 0.03 (0.07) Adult male only households 0.25 (0.21) 0.38 (0.28) Child only households -0.65*** (0.09) -0.77*** (0.12) Agriculture variables Households using minimum financial services 0.12+ (0.06) 0.11 (0.08) Households using minimum value chain practices 0.04 (0.06) 0.06 (0.08) Households using minimum sustainable agriculture practices 0.21** (0.07) 0.15* (0.07) Households using improved storage practices 0.11+ (0.07) 0.09 (0.07) Geographic districts (Kaabong= reference district) Kotido 0.16+ (0.09) Abim 0.17+ (0.09) Constant 0.58*** (0.04) 0.58*** (0.04) 0.58*** (0.04) 0.63*** (0.08) 0.52*** (0.09) Observations (# of HHs in pooled dataset) 3,565 3,565 3,565 2,752 2,374 R-Squared 0.02 0.04 0.05 0.10 0.10 Standard error in parenthesis *** p<0.001, ** p<0.01, * p<0.05, + p<0.10 ANNEX 8: DISCLOSURE OF ANY CONFLICTS OF INTEREST [The Evaluation Policy requires that evaluation reports include a signed statement by each evaluation team member regarding any conflicts of interest. A suggested format is provided below.] Name Anna Afferri Title Subject Matter Expert – Maternal and Child Health, Nutrition and WASH Organization ICF Evaluation Position? Team Leader x Team member Evaluation Award Number (contract or other instrument) 18MSCA0074 USAID Project(s) Evaluated (Include project name(s), implementer name(s) and award number(s), if applicable) Growth, Health and Governance Development Food Assistant Project. Prime implementer is Mercy Corps. Other partners included Feinstein International Center, World Vision, Whave, Abim Women Organized Together for Development and Kaabong Peace for Development Agency. I have real or potential conflicts of interest to disclose. Yes X No If yes answered above, I disclose the following facts: Real or potential conflicts of interest may include, but are not limited to: 1. Close family member who is an employee of the USAID operating unit managing the project(s) being evaluated or the implementing organization(s) whose project(s) are being evaluated. 2. Financial interest that is direct, or is significant though indirect, in the implementing organization(s) whose projects are being evaluated or in the outcome of the evaluation. 3. Current or previous direct or significant though indirect experience with the project(s) being evaluated, including involvement in the project design or previous iterations of the project. 4. Current or previous work experience or seeking employment with the USAID operating unit managing the evaluation or the implementing organization(s) whose project(s) are being evaluated. 5. Current or previous work experience with an organization that may be seen as an industry competitor with the implementing organization(s) whose project(s) are being evaluated. 6. Preconceived ideas toward individuals, groups, organizations, or objectives of the particular projects and organizations being evaluated that could bias the evaluation. I certify (1) that I have completed this disclosure form fully and to the best of my ability and (2) that I will update this disclosure form promptly if relevant circumstances change. If I gain access to proprietary information of other companies, then I agree to protect their information from unauthorized use or disclosure for as long as it remains proprietary and refrain from using the information for any purpose other than that for which it was furnished. Signature Anna Afferri Date 18th November 2018 Name Dr. Frank Muherez Title Subject Matter Expert in Conflict Analysis Organization ICF Evaluation Position? Team member Evaluation Award Number (contract or other instrument) 18MSCA0074 USAID Project(s) Evaluated (Include project name(s), implementer name(s) and award number(s), if applicable) Growth, Health and Governance Development Food Assistant Project. Prime implementer is Mercy Corps. Other partners included Feinstein International Center, World Vision, Whave, Abim Women Organized Together for Development and Kaabong Peace for Development Agency. I have real or potential conflicts of interest to disclose. No If yes answered above, I disclose the following facts: Real or potential conflicts of interest may include, but are not limited to: 1. Close family member who is an employee of the USAID operating unit managing the project(s) being evaluated or the implementing organization(s) whose project(s) are being evaluated. 2. Financial interest that is direct, or is significant though indirect, in the implementing organization(s) whose projects are being evaluated or in the outcome of the evaluation. 3. Current or previous direct or significant though indirect experience with the project(s) being evaluated, including involvement in the project design or previous iterations of the project. 4. Current or previous work experience or seeking employment with the USAID operating unit managing the evaluation or the implementing organization(s) whose project(s) are being evaluated. 5. Current or previous work experience with an organization that may be seen as an industry competitor with the implementing organization(s) whose project(s) are being evaluated. 6. Preconceived ideas toward individuals, groups, organizations, or objectives of the particular projects and organizations being evaluated that could bias the evaluation. I certify (1) that I have completed this disclosure form fully and to the best of my ability and (2) that I will update this disclosure form promptly if relevant circumstances change. If I gain access to proprietary information of other companies, then I agree to protect their information from unauthorized use or disclosure for as long as it remains proprietary and refrain from using the information for any purpose other than that for which it was furnished. Signature Date November 19, 2018 Name Patricia Vondal, PhD Title Senior Evaluation Specialist Organization ICF Evaluation Position? X Team Leader Team member Evaluation Award Number (contract or other instrument) 18MSCA0074 USAID Project(s) Evaluated (Include project name(s), implementer name(s) and award number(s), if applicable) Growth, Health and Governance Development Food Assistant Project. Prime implementer is Mercy Corps. Other partners included Feinstein International Center, World Vision, Whave, Abim Women Organized Together for Development and Kaabong Peace for Development Agency. I have real or potential conflicts of interest to disclose. Yes X No If yes answered above, I disclose the following facts: Real or potential conflicts of interest may include, but are not limited to: 7. Close family member who is an employee of the USAID operating unit managing the project(s) being evaluated or the implementing organization(s) whose project(s) are being evaluated. 8. Financial interest that is direct, or is significant though indirect, in the implementing organization(s) whose projects are being evaluated or in the outcome of the evaluation. 9. Current or previous direct or significant though indirect experience with the project(s) being evaluated, including involvement in the project design or previous iterations of the project. 10. Current or previous work experience or seeking employment with the USAID operating unit managing the evaluation or the implementing organization(s) whose project(s) are being evaluated. 11. Current or previous work experience with an organization that may be seen as an industry competitor with the implementing organization(s) whose project(s) are being evaluated. 12. Preconceived ideas toward individuals, groups, organizations, or objectives of the particular projects and organizations being evaluated that could bias the evaluation. I certify (1) that I have completed this disclosure form fully and to the best of my ability and (2) that I will update this disclosure form promptly if relevant circumstances change. If I gain access to proprietary information of other companies, then I agree to protect their information from unauthorized use or disclosure for as long as it remains proprietary and refrain from using the information for any purpose other than that for which it was furnished. Signature Date November 19, 2018 U.S. Agency for International Development 1300 Pennsylvania Avenue, NW Washington, DC 20523