August 2019 This publication was produced at the request of the United States Agency for International Development. It was prepared by Rachel Jean-Baptiste Salomonsen, PhD, MPH, Team Leader, Oxford Epidemiology Services LLC and Consultant to USAID/SITES, Jarvice Sekajja, PhD, USAID/SITES, and directed by Joseph Mwangi, USAID/Uganda SI and Edgar Agaba, USAID/Uganda SI. Lessons from PMI team were written by Kassahun Belay, MD, MPH, Joel Kisubi, and Mame Niang, MD, MPH. The GoU NMCD perspective was written by Damian Rutazaana and extensively reviewed by Gloria Sebikaari, USAID/Uganda COR for MAPD, and the Sam Gudoi and Emily Goodwin from the Malaria Action Program for Districts (MAPD). Malaria Action Program for Districts (MAPD) Midterm Learning Review Masaka Sunrise by Rachel Jean-Baptiste Salomonsen Malaria Action Program for Districts (MAPD) Midterm Learning Review August 2019 USAID Contract No. Contract No. AID-617-C-17-00017 MAPD Learning Review iii This document is available in printed or online versions. Online documents are made available through the Development Experience Clearinghouse (http://dec.usaid.gov). This document was submitted by SITES to the United States Agency for International Development under USAID Contract No. AID-617-C-17-00017 MAPD Learning Review v ACKNOWLEDGEMENTS This evaluation would not have been possible without the support, cooperation, and sharing of information and experiences, perceptions, and viewpoints of different stakeholders, providing vital material for this report’s findings and conclusions. The Learning Team wishes to acknowledge a debt of gratitude to all those, particularly national stakeholders, District Health Management Teams, health facilities staff, and many others who gave generously of their time, and shared their thoughts, at times extensively and with great depth. Special thanks are due to the leadership and staff MAPD for their continuous friendly support, flexibility and practical help. The SITEs behind the scenes staff who supported the work, including Kenneth Kasule who made took care of all logistics and supported with DHIS 2 analysis, Andrew Kunihira, Immaculate Baseka, Albert Daniel Ongia, and Magaya Mungo Abaisali who supported with data collection, as well as all of the drivers, Charles Muhereza, Bruce Asaba, Aggrey Magumba, who took us safely to all site visits. Without all of you putting in 100% effort, this would not have been possible. We offer special thanks to SITES administrative team, starting with Dr Daraus Bukenya, Barbara Amuron, Prossy Mpalanyi, Alex Kagenda and others who made it possible for us to get this work done. Last but not least, we would like to thank Joseph Mwangi for directing this Learning Review, and Edgar Agaba, of USAID/Uganda SI team for his support in coordinating this activity. MAPD Learning Review vii CONTENTS ACRONYMS........................................................................................................................................1 EXECUTIVE SUMMARY...................................................................................................................3 I. Project Background .....................................................................................................................9 II. Learning Purpose and Questions............................................................................................10 LEARNING REVIEW PURPOSE ............................................................................................................10 LEARNING REVIEW QUESTIONS.......................................................................................................10 III. Learning Design, Methods & Limitations..............................................................................11 IV. Findings.........................................................................................................................................15 LEARNING QUESTION 1: DOES PERFORMANCE (HMIS AND MIS) DATA SUGGESTS A DISCERNABLE ASSOCIATION BETWEEN MAPD INVESTMENTS AND INCREASED AVAILABILITY, AND UPDATE OF QUALITY MALARIA PREVENTION, DIAGNOSIS AND CASE MANAGEMENT SERVICES?......................................................................................................15 LEARNING QUESTION 2: TO WHAT EXTENT DO INSTITUTIONAL, STRUCTURAL, MANAGEMENT AND LEADERSHIP FACTORS (AND OTHERS IDENTIFIED DURING THE REVIEW) INFLUENCE IMPROVEMENTS IN SERVICE DELIVERY IN THE MAPD DISTRICTS?................................................................................................................................................23 LEARNING QUESTION 3: IS THE GEOGRAPHICAL COVERAGE AND SCALE OF MAPD ADEQUATE TO SIGNIFICANTLY INFLUENCE ACHIEVEMENT OF THE GOAL TO GET UGANDA TO PRE-ELIMINATION STAGE BY 2020?.....................................................................30 LEARNING QUESTION 4: WHAT IS THE EVIDENCE FOR IMPROVED FULFILLMENT (OR LACK OF IMPROVEMENT) OF THE ROLE OF GOU INSTITUTIONS (NMCD AND DHMT) IN QUALITY IMPROVEMENT OF MALARIA PREVENTION AND TREATMENT SERVICES? .......................................................................................................................................................................32 LEARNING QUESTION 5: HOW DO YOU COMPARE APPROACHES BY MAPD TO THOSE OF RHITES?................................................................................................................................33 V. Conclusions and Learnings.......................................................................................................35 CONCLUSIONS.........................................................................................................................................35 SUMMARY LEARNINGS.........................................................................................................................35 Annexes ..............................................................................................................................................42 1 USAID UGANDA MAPD MIDTERM LEARNING REVIEW ACRONYMS CAO Chief Administrative Officer CDC Centers for Disease Control and Prevention CDCS Country Development Corporation Strategies CDFU Communication for Development Foundation Uganda COR Contracting Officer’s Representative DHMT District Health Management Team DHIS District Health Information System DHO District Health Officer DO Development Objective FGD Focus group discussion FY Fiscal Year GoU Government of Uganda HMIS Health Information Management System iCCM Integrated community case management IDI Infectious Diseases Institute IP Implementing Partner IPTp Intermittent preventive treatment in pregnancy IR Intermediate Result JMS Joint Medical Stores KII Key informant interview LC V Local Council 5 MAPD Malaria Action Program for Districts M&E Monitoring and Evaluation MOES Ministry of Education and Sports MoH Ministry of Health MIS Malaria Indicator Survey NGO Non-governmental organization NMCD National Malaria Control Division PI Performance improvement PMI President’s Malaria Initiative PMP Performance management plan PNFP Private not for profit SOW Scope of Work USAID UGANDA MAPD MIDTERM LEARNING REVIEW 2 SI Strategic Information SP Sulfadoxine and Pyrimethamine (Fansidar) SSS Social and Scientific Systems TWG Technical Working Group UMRSP Uganda Malaria Reduction Strategic Plan UNICEF United Nations Children’s Fund USAID United States Aid for International Development USG United States Government VHT Village Health Team WHO World Health Organization MAPD Learning Review 3 EXECUTIVE SUMMARY LEARNING REVIEW PURPOSE AND LEARNING QUESTIONS The purpose of this Learning Review of the Malaria Action Program for Districts (MAPD) was to learn what could be done differently to increase the chances of achieving MAPD’s anticipated results, specifically to improve the health status of the Ugandan population through reducing malaria related morbidity and mortality that negatively affects child development and resilience of Ugandan households. PROJECT BACKGROUND MAPD is a $41 million five-year contract funded by the President’s Malaria Initiative (PMI), through the United States Agency for International Development (USAID). Implementation is led by the Malaria Consortium, with Jhpiego, Banyan Global, Deloitte, Communication for Development Foundation Uganda (CDFU), Infectious Diseases Institute (IDI), and Infectious Diseases Research Collaboration (IDRC) as sub-partners. This project is implemented in 43 original (now 52) districts across 5 regions: Arua – West Nile, Hoima – Bunyoro, Fort Portal – Rwenzori, Kampala – Central 2, and Masaka – Central 1. The goal of MAPD is to improve the health status of the Ugandan population through reducing malaria related morbidity and mortality. LEARNING REVIEW QUESTIONS, DESIGN, METHODS, AND LIMITATIONS Using a cross-sectional design, the following five questions were investigated: 1. Does performance (HMIS and MIS) data suggest a discernible association between MAPD investments and increased availability, and uptake of quality malaria prevention, diagnosis and case management services? 2. To what extent do institutional, structural, management and leadership factors (and others identified during the review) influence improvements in service delivery in the MAPD districts 3. Is the geographical coverage and scale of MAPD adequate to significantly influence achievement of the goal to get Uganda to pre-elimination stage by 2020? 4. What is the evidence for improved fulfilment (or lack of improvement) of the role of GOU institutions (NMCD and DHMT) in quality improvement of malaria prevention and treatment services? 5. How do you compare approaches by MAPD to those of RHITES? The Learning Team was comprised of PMI, USAID SI Unit, NMCD, MAPD, SITES and external consultants from Oxford Epi. The Learning Review employed several quantitative and qualitative methods, including document reviews, key informant interviews, group interviews, focus group discussions, and client exit interviews that targeted national and district level stakeholders from six selected districts. Limitations to the design of this learning review were that only 4 of 52 MAPD districts and 1 RHITES-SW district were visited by the team. In addition, district selection was purposive, and did not include West Nile, a region covered by MAPD and with the highest prevalence of malaria in the country. However, this was unavoidable due to time and resource constraints. USAID UGANDA MAPD MIDTERM LEARNING REVIEW 4 FINDINGS, LEARNINGS, AND RECOMMENDATIONS Findings Question 1: MAPD Performance Data MAPD reported strong improvements on certain indicators. When their data is reviewed against baseline, we see impressive improvements in the proportion of pregnant women who receive three or more doses of IPTp, from 0% to 47%. Improvements were also seen in the proportion of presumptive malaria cases (fevers) tested using RDT or microscopy prior to providing malaria treatment, from 69% in 2016, to 94% as of March 2019; and in the proportion of patients who tested negative for malaria but still received treatment, from 27% in 2016, to 9% by March 2019. These achievements also vary greatly when analyzed at district and health facility levels, highlighting room for improvement. Question 2: Factors that influence improvements Institutional, structural, management and leadership factors affecting MAPD’s ability to deliver as identified during the learning review comprised of relationships, communication and understanding, formal organizational leadership and collaborative approaches to minimizing the burden of malaria in districts. MAPD was recognized for its strong support to the NMCD in improving the policy environment for malaria in pregnancy (MIP). The team found that relationships between PMI and NMCD, and between PMI and MAPD were strong, but relationships were strained between MAPD and NMCD, and multiple misunderstandings existed between MAPD and some districts. The team noted that all of these relationships needed to be clear, strong, and functional in order to maximally influence improvements in reducing malaria burden. MAPD is perceived to be Implementing Partner (IP)-led, and not government-led or district-led, signaling weaknesses in country ownership. In terms of project management, many activities, like trainings, were one-off, slow to start; DHMTs told learning participants that the recommended onsite mentorship approach was not intensively carried out. There was no evidence of intense on-site mentorship crucial for fostering ongoing professional development, maintaining competency, encouraging professional expertise and promoting leadership for sustainable high-quality care outcomes. Mentorship interventions were carried out as one-off activities, which does not translate into the desired learning outcomes that could effectively build the health workers’ capacity on site. Activities along the data continuum including data cleaning and data use, were not maintained on a regular basis. The project has had difficulty aligning with district planning, both in terms of fiscal year alignment, as well as understanding. Finally, despite the varying malaria burden, and performance in the 52 districts, the Learning Team did not observe consistent effort to use data to adapt context specific and targeted interventions. It should be noted that at the time when the project was designed, PMI perceived the malaria epidemic as holoendemic, and it is likely that stratification of districts would not have been as appropriate. Learning participants all took note of the need to potentially adjust malaria programming and interventions in light of the changing malaria distribution within the country. Question 3: Adequacy of Geographic coverage and scale of MAPD Based on the most recent data from the MIS 2019 (not yet published), the Learning Review team found that MAPD is operating largely in areas of lower malaria prevalence, except for West Nile region, which has the highest malaria prevalence. MAPD interventions cover 54% of persons living with the malaria parasite (PWMP), yet receives 3.4 times as much funding as integrated projects that MAPD Learning Review 5 focus on the remaining 40% of PWMP. The team found that there is need to reassess the geographic distribution of MAPD, or at least PMI funding, to ensure that PMI is positioned to maximally support Uganda in decreasing its malaria burden in the most cost-effective and efficient manner. Question 4: Improved fulfilment of the Role of GOU institutions At all levels, the Government of Uganda is paying increased attention to malaria. The team found the GOU leadership to be very engaged in malaria elimination. There is a working group for malaria within the President’s Office, in Parliament, and the Minister and PS are engaged. The NMCD leadership is dynamic, and the Global Fund (GF), managed by the Office of the Permanent Secretary (PS), has been working with PMI and NMCD to ensure that malaria commodities are available. The new GoU national strategy for malaria is being drafted, and with support from WHO and other donors, will speak to the evolved epidemiologic picture of malaria within Uganda which requires both a repositioning of malaria more within a public health framework, as well as stratification of malaria interventions based on district-specific (and sub-district) needs. Nevertheless, districts are severely underfunded, and as such, often cannot fulfil their own roles in minimizing the burden of malaria. Furthermore, part of the work to be done by MAPD is district capacity building for malaria control in their district, and outside of technical capacity in case management, this has not been done adequately. However, the team understands that MAPD recently conducted a leadership assessment of all districts. They have identified management capacity areas to strengthen and will be integrating these in their workplan for the next fiscal year (PY 4). Question 5: Comparison of Approaches of MAPD vs. RHITES Because of time, comparison of MAPD project to RHITES projects was limited to RHTES Southwest, which was implementing interventions in the region for the last five years. RHITES is primarily a PEPFAR-funded activity with intensive data driven programming and planning. PMI’s funding to RHITES Southwest is more than three times smaller than to MAPD, yet malaria programming has benefited greatly from the data-driven programming approach. MAPD was completing its 3rd year of implementing malaria only interventions, and PMI does not have the same data-driven focus. The team found that many of the interventions were the same or very similar. Both used multiple interventions, including training, mentorship and supervision, site visits, clinical and mortality audits, and data reviews. However, compared to MAPD, these same activities in districts supported by RHITES Southwest were done more regularly and intense, engaging districts, and enabling them to lead the processes. This may have contributed to RHITES SW demonstrating the expected reduction in the incidence of malaria in pregnancy. RHITES Southwest was able to integrate its work in malaria with other supported project work, particularly HIV supported by PEPFAR, thus gaining some advantage. The comparison here is observational. Time and learning design limitations prevented a deeper comparison that would have controlled for key confounders, including socio-economic status of the populations served and malaria transmission rate. Learning Each of the institutions implementing policy, strategy, program and project management and technical interventions (PMI, MAPD and NMCD) took notes and shared respective “learning” every day of the review. The following are extracts from these daily readouts. USAID UGANDA MAPD MIDTERM LEARNING REVIEW 6 PMI • Strengthen community engagement –PMI needs to consider maintaining the good achievements at facility level but go beyond the facility and take the message of malaria prevention and treatment more rigorously at community, household and individual levels. • Strengthen Social Behavior Change (SBC) –PMI needs to consider revisiting the SBC approach to focus on issues of net use, IPTp use, early self-reporting when febrile. • Strengthen quality reporting at all levels in support of PMI's focus on data use for decision making, with a focus on data use at the facility and community levels. • Better define, streamline, and strengthen capacity building at all levels through joint planning and implementation with GoU stakeholders, as well as supportive supervision from the central level to districts, from districts to health facilities, and from health facilities to communities, households and Individuals. • Consider intervention prevalence-based zooming i.e. a stratified differentiated intervention package depending on prevalence as reported by the current Malaria Indicator Survey (MIS). Emerging low prevalence areas in the central region would ideally have different interventions as compared to high prevalence areas such as West Nile. • Respond to the question of an increase in mobile populations such as refugees and migrant workers traveling from high prevalence to low prevalence areas who are a potential reservoir that can drain the impact currently created by PMI interventions or even contribute to an epidemic or upsurge. • Anecdotal evidence from the review indicates that nets are lasting less than three years with most Focus Group Discussions (FGDs) reporting 1-2 years of usable life. PMI is currently running a net durability study and this study will advise us how long a net last in Uganda. At the time of this Learning Review, the data was not yet available. • Rethink PMI's work with VHTs. Interviews with FGDs revealed varying amounts of allowances, particularly transport and lunch allowances, provided by different actors from USD 8-15 per sitting or per month or per quarter. MAPD • District Capacity Building: Some gaps in understanding, ownership, use and follow-up • Consistency of interventions is required for effectiveness and acceptance of MAPD • MIP cases high though IPTP3 high and LLIN ANC high, and reported LLIN use of pregnant women high • Improvement is needed in terms of community data and use at HF, District and Central level • Need to strengthen data collection and use from HFs using district leadership and follow up and data use itself, to ensure improvements NMCD • NMCD should scale up its role for supporting policy review and development within the context of changing malaria landscape. This will enable partners to have access to frameworks, MAPD Learning Review 7 policies and guidelines that address current trends in malaria prevention and treatment and quality service delivery (KII, National Level Stakeholders, Kampala). • In addition, there is need for NMCD to strengthen the Village Health quality improvement standards and facilitate iCCM for improved malaria prevention, diagnosis and treatment. NMCD should ensure that VHT commodities supply chain is improved, challenges of reporting among VHTs addressed to reduce on misinterpretation and double counting. (KII, NMCD, Global Fund, Kampala). • Private sector malaria reduction is still very weak and will require innovation and intensified action considering the majority of patients first seek care in private sector. Challenges persists in reporting, adherence to test and treat, affordability of quality RDTs, LLINs and IRS commodities. NMCD and partners have to scale up malaria reduction efforts in the private sector. • Scale up of IPTp3 did not translate in reduction of malaria in pregnancy. Other efforts must simultaneously be scaled up to reach the vulnerable group and review continuously the effectiveness of IPTp NMCD should develop tailored strategies for mobile populations case in point was Mityana where the malaria hotspots had many immigrants that were not using mosquito nets and uptake of other malaria interventions. • Encourage partners and districts to leverage from other partners and interventions through integration to maximize efforts as was witnessed with Isingiro District and RHITES SW • The districts did not have targets and were also not aware of UMRSP goals. NMCD to support MAPD/partners to develop district specific goals and targets in line with the Uganda malaria reduction strategic plan. • Challenges persist in net use and malaria prevention practices. NMCD to review of BCC implementation by partners as it is not translating to practice and ownership of malaria control by communities. • NMCD to provide guidance to partners on stratification and focus on malaria hotspots to have maximum benefit from investment in malaria. There were interventions that were spreading too thin and having minimal effect. • A number of activities were implemented without good monitoring and evaluation such as school net distribution, school drama clubs and as such could not demonstrate results. NMCD to endure that activities implemented by partners have good M& E systems to demonstrate impact. • Close collaboration and implementation by NMCD, districts and MAPD is very crucial to achieve the project and UMRSP goals. ADDITIONAL RECOMMENDATIONS • Engage the NMCD in regular, intense review of MAPD results/data at national level (for all 52 districts), ask for guidance, and ensure joint understanding. Use of the MAPD conceptual framework may help. • Facilitate NMCD in regular, data-drive dialogue with DHMT and district politicians around lowering malaria burden USAID UGANDA MAPD MIDTERM LEARNING REVIEW 8 • USAID should consider conducting a more formal comparison of cost-effectiveness of integrated malaria programming vs. stand-alone bivariate malaria programming. In such an analysis, it will be important to capture not just interventions, but how they were implemented, and how they were received in order to tease out contributive factors. MAPD Learning Review 9 I. PROJECT BACKGROUND The Malaria Action Program for Districts (MAPD) project is a 5-year contract (2016-2021) funded by the President’s Malaria Initiative (PMI) through the United States Agency for International Development (USAID) and UKAid through the Department for International Development (DFID). The project is implemented by the Malaria Consortium, in partnership with Jhpiego, Banyan Global, Communication for Development Foundation Uganda (CDFU), Deloitte Uganda, Infectious Diseases Institute (IDI) and Infectious Diseases Research Collaboration (IDRC). MAPD aims to improve the health status of the Ugandan population by reducing childhood and maternal morbidity and mortality due to malaria with special focus on women and children. MAPD supports the National Malaria Control Division’s (NMCD) Uganda Malaria Reduction Strategic Plan (UMRSP) 2014-2020. Over the five-year period, MAPD is expected to produce three key results: Result 1: Effective malaria prevention programs implemented in support of the National Malaria Control Strategy. This result focuses specifically on at least 85% of pregnant women receiving three or more doses of IPTp, and 85% of pregnant women and children under 5 years sleeping under an insecticide-treated net (ITN). Result 2: Effective malaria diagnosis and treatment activities implemented in support of the National Malaria Strategy. This result specifically focuses on working with Village Health Teams (VHTs) to implement iCCM in the highly endemic districts in the Central region; improving diagnostic capacity, and improving health provider capacity for the management of simple and severe malaria. Result 3: Capacity of NMCD and DHMTs to manage and sustain efficient malaria activities in focus districts built. This result focuses specifically on strengthening capacity of DHMTs to set and meet capacity improvement targets, manage stock efficiently, and ensure health workers are trained in the diagnosis and treatment of simple and severe malaria. It also focused on demonstrating value for money, meaning that cost-efficiency for delivering malaria services can be demonstrated. To achieve these results, MAPD employs a health systems-thinking approach towards implementing results-oriented, field-tested strategies that build on predecessor Stop Malaria Project, also led by JHU and Malaria Consortium as a technical partner. MAPD aims to improve the health status of the Ugandan population by reducing childhood and maternal morbidity and mortality due to malaria with special focus on women and children. MAPD supports the NMCD’s Uganda Malaria Reduction Strategic Plan (UMRSP) 2014-2020. The project covers 52 districts in central, mid-western and west Nile regions of Uganda, reaching an estimated population of 13 million Ugandans, or 31% of the Ugandan population. At the time of the Learning Review, MAPD was completing its third year of implementation, and data from the recent Malaria Indicator Survey (MIS) 2018/2019 released a month before the Review indicated changing scenario of malaria prevalence and prevention practices in Uganda. USAID therefore found it necessary to assess the effectiveness of MAPD’s current approach to critically reduce the malaria burden and render households more resilient. This review intended to facilitate learning while identifying major bottlenecks and challenges that may require adaptation in the design, interventions, or management including geography, delivery method or scale of implementation. USAID UGANDA MAPD MIDTERM LEARNING REVIEW 10 II. LEARNING PURPOSE AND QUESTIONS LEARNING REVIEW PURPOSE The purpose of this learning review is for USAID to learn what can be done differently to increase the chances of achieving anticipated results. The review will assess the extent to which activities of MAPD are on track, and at the same time provide an opportunity for USAID staff to learn how the quality of malaria services are improving (or not) at different project sites. The findings of this learning review are expected to inform various ways that the PMI team can adjust and improve MAPD’s current approach. Specifically, this assessment will allow PMI to gather evidence that can inform strategies and direction of the project workplan in years 4 and 5, and of PMI’s support to the GOU’s malaria control strategy post-MAPD. LEARNING REVIEW QUESTIONS 1. Does performance (HMIS and MIS) data suggest a discernible association between MAPD investments and increased availability, and uptake of quality malaria prevention, diagnosis and case management services? 2. To what extent do institutional, structural, management and leadership factors (and others identified during the review) influence improvements in service delivery in the MAPD districts? 3. Is the geographical coverage and scale of MAPD adequate to significantly influence achievement of the goal to get Uganda to pre-elimination stage by 2020? 4. What is the evidence for improved fulfillment (or lack of improvement) of the role of GOU institutions (NMCD and DHMT) in quality improvement of malaria prevention and treatment services? 5. How do you compare approaches by MAPD to those of RHITES? 11 USAID UGANDA MAPD MIDTERM LEARNING REVIEW III. LEARNING DESIGN, METHODS & LIMITATIONS Design of the Learning Review: The Learning Team used a cross-sectional design, with purposive sampling of key informants, districts and sites, and mixed methods for data collection. Their approach was inclusive, comprehensive, iterative and flexible, allowing learning from previous findings to influence modification to questions asked of subsequent stakeholders. Of note, interviews of all stakeholders were completed before interviews with MAPD project implementing team. Summary of Stakeholder Engagement: Data was collected from all stakeholders identified by PMI, including NMCD, the Global Fund to fight AIDS, Tuberculosis and Malaria (Global Fund, or GF), Joint Medical Stores (JMS), World Health Organization (WHO), DFID/UKAid, MAPD staff II staff, District Health Management Teams (DHMT), Chief Administrative Officer (CAO), Local Council 5 (LC5) and others from six districts (4 MAPD, 2 RHITES), health facility staff, Village Health Workers and beneficiaries from 10 health facilities. Sampling strategy: Convenience sampling was used to identify stakeholders for KIIs and FGDs, stratified by stakeholder type. Data Collection Methods and Rational Aligned: The team used multiple data collection methods to address each review question. Summarized in Table 1 below, these include document reviews, secondary DHIS II and MAPD performance data analysis, focus group discussions (FGDs), key informant interviews (KIIs), exit interviews and site visits with stakeholders from NMCD and DHMTs, MAPD, and National Stakeholders. Where data from one set of discussions seem to indicate a finding, team members used subsequent data collection encounters to further validate, invalidate, or broaden/deepen understanding of the finding by discovering new perspectives. All tools used in this learning review can be found in Annex 2. Learning At the end of each day, a discussion among the learning participants was held to flesh out what was learned and its potential impact. Specifically, PMI, USAID SI, and NMCD were each asked the following set of questions: • What did you hear? • What did you learn? For PMI: • What are the implications for the upcoming MAPD workplan? • What are the implications for PMI planning? • Are there implications for MAPD SOW? For USAID SI: • How can you better support the program with more granular data to better guide program performance management? • What additional data analysis is required? Joint Analysis of Findings Daily sessions- report back, learning, and implications Consensus on Guiding Questions Joint Data Collection -National, District and site levels Figure 1: Learning Review Approach USAID UGANDA MAPD MIDTERM LEARNING REVIEW 12 For NMCD: • What are the implications for policy and national coordination? Table 1: Data Collection Methods Employed for Each Learning Question Data Collection Method Evaluation Questions 1: 2: 3: 4: 5: Document Review X X X X X Secondary Analysis of DHIS 2 data X X X National Stakeholder Interviews X X X X Group Interview of District Leadership X X X X X KII Health Facility Staff X X X X X FGD VHTs X X X X X FGD Beneficiaries X X Exit Interviews Beneficiaries X X Group Interview MAPD Staff X X X X X Daily Learning Discussions post data collection X X X X X Ethical considerations and assurances: The Learning Team implemented a policy of obtaining verbal informed consent for all group and individual key informant interviews, exit interviews, and focus group discussions (see Annex 3 for an example), and all interviews and discussions were done on a voluntary basis. Participants were given the option to opt-out of any questions or the whole interview if at any time they believed a response would contain sensitive information. The information provided as part of these interviews and discussions are not be linked to any specific person, only to general identifying information such as stakeholder type (National, NMCD, DHMT, etc.) in this report and in the presentations made. Only members of the Learning Team have access to the transcripts and raw data. This report is a synthesis of the Team’s analysis drawn from interviews from numerous respondents. Any included quotes to highlight particular issues do not include names. We did not collect data from minors, nor did we collect personally identifiable information directly from patient records or anywhere else, though health facility registers were briefly reviewed at times. Deviations and adjustments: The team was not able to visit Mbarara District Health Management Team, and instead of the team traveling to Isingiro district, members of the Isingiro DHMT, including the District Health Officer, met the team in Mbarara for the interview because they had other engagements in Mbarara that same day. Only one health facility was visited in Mbarara and Isingiro each, instead of two as was planned. These changes were due to a better understanding of the workload, and the need to remain on schedule for the remaining activities. Procedures used to ensure that the data were of highest achievable quality: The team developed the tools together, and almost always collected the qualitative data in pairs. Daily learning meetings provided MAPD Learning Review 13 opportunities where they could share what they learned and obtain any additional guidance to make their data collection activities more reliable. Data Analysis Plan Analysis of Question 1: Does performance (HMIS and MIS) data suggest a discernible association between MAPD investments and increased availability, and uptake of quality malaria prevention, diagnosis and case management services? Data reviewed from MAPD approved PMP, as well as the DHIS 2 were used to document frequencies before and after MAPD implementation for IPTp2, IPTp 3, LLIN, RDT for presumptive malaria cases, ACTs for RDT negative cases from May 2014 (or earliest available data from that date) to May 2019. We analyzed this both by comparing bar charts as well as trend lines. This was triangulated with data from KIIs with the NMCD and other national stakeholders, as well as with district leadership. We also reviewed additional evidence from focus group discussions, and exit interviews. Analysis of Question 2: To what extent do institutional, structural, management and leadership factors (and others identified during the review) influence improvements in service delivery in the MAPD districts? After triangulating data from various data sources, including KIIs and group interviews with national and district stakeholders, as well as from participating in daily learning discussions, we used a relational model to qualitatively explain institutional relationships between PMI, NMCD, Districts and MAPD, and the impact of their strength or fragility with regards to their ability to influence improvements in service delivery in the districts. Using this lens, observed strengths and learnings were documented for each of the areas separately: institutional, structural, management and leadership. Finally, we summarize the influence they have on improvements of service delivery in project districts, using the districts we visited as examples. Analysis of Question 3: Is the geographical coverage and scale of MAPD adequate to significantly influence achievement of the goal to get Uganda to pre-elimination stage by 2020? In order to answer this question, the team conducted a visual comparison of current geographic scale of MAPD districts with Uganda MIS 2019 data on malaria prevalence to identify similarities and deviations. These differences were further discussed in KIIs with GoU and national stakeholders, and further triangulated with field visits where we validated our understanding. Our analysis was further clarified by generating maps with color-coded incidence of malaria against MAPD Districts to visually demonstrate the ‘goodness of fit’ of MAPD districts against current realities of the malaria burden in Uganda. We used national census data to estimate the number of people with malaria parasites per district. This helped us to map the burden of disease. Analysis of Question 4: What is the evidence for improved fulfillment (or lack of improvement) of the role of GOU institutions (NMCD and DHMT) in quality improvement of malaria prevention and treatment services? KIIs with GOU, national stakeholders and district leadership provided the first level evidence for the role of GOU institutions in quality improvement of malaria interventions. Qualitative review of data use (who uses the data, what data are they using, when does it happen, how does it happen, and for what reason is data used)- from districts and health facilities provided further triangulation of the findings. Analysis of Question 5: How do you compare approaches by MAPD to those of RHITES? Qualitative summary highlights of key differences in approaches of integrated RHITES projects vs malaria￾only MAPD project were teased out from KIIs. The in-depth analysis focuses on RHITES Southwest since that is the only RHITES project we visited and interviewed staff. Data collected from RHITES SW project, district, and health facility staff were further triangulated with data from the DHIS 2 that allowed for visual USAID UGANDA MAPD MIDTERM LEARNING REVIEW 14 comparison of RHITES SW and MAPD for indicators for malaria in pregnancy (MIP), including IPTp 3 and LLINs. Limitations of the design and analytic methods: The team notes that the learning review data was collected in three of the five MAPD regions, and 4 districts out of 52 districts where MAPD operates. One of the regions not included for data collection was West Nile, which has the highest malaria incidence and prevalence in the country at the moment. Further, districts were sampled based on travel convenience for USAID staff. These limitations may affect the generalizability of the findings. We attempted to minimize them by using our group interview with MAPD staff to validate some of the key findings, and this helped the team to gain a deeper understanding of what was learned in the field and how it applies to the rest of the project. With only one week allocated to data collection in the field, however, a limited budget, and in consideration of the schedules of the entire Learning Team members, especially PMI, the limitations were noted by USAID, but it was deemed more important to conduct the Learning Review in these limited regions, and with those specific handpicked districts to afford the team this learning opportunity. MAPD Learning Review 15 IV. FINDINGS In total, data was collected from 8 national level stakeholders, and 56 District Local Government officials. In addition, the team conducted exit interviews with a total of 33 care takers of children under five who had brought their children suspected of malaria to receive care at the health facility that day. We conducted focus group discussions with 62 VHT members (16 Male, 46 Female) and 86 mothers of children under five. The districts and types of stakeholders interviewed are listed in Table 2 below, and the list of persons met is included in Annex 3. Data was collected from six districts, including two RHITES SW districts. Informed consent was obtained for all participants. The team also reviewed 32 documents (Annex 4). Table 2: Type of Stakeholders Interviewed for MAPD Learning Review Region District MAPD RHITES SW Stakeholders Health Facilities Central 1Masaka  DHMT, CAO, LC5 Masaka RRH Kamulegu HC III Lyatonde  DHMT, CAO, LC5 Lyatonde Hospital Kaliiro HC III Central 2Mityana  DHMT, CAO, LC5 Magala HC III Naama HC III Kampala  NMCD, GF, DFID, WHO, JMS, MALARIA CONSORTIUM, MAPD --- Tooro/ RwenzoriKabarole  DHMT, CAO, LC5 Fortportal RRH Rutete HC III Western Mbarara  Mbarara RRH Isingiro  DHMT, RHITES SW Rwekubo HC IV Nyarubugo HC III LEARNING QUESTION 1: DOES PERFORMANCE (HMIS AND MIS) DATA SUGGESTS A DISCERNABLE ASSOCIATION BETWEEN MAPD INVESTMENTS AND INCREASED AVAILABILITY, AND UPDATE OF QUALITY MALARIA PREVENTION, DIAGNOSIS AND CASE MANAGEMENT SERVICES? MAPD provided indispensable support to the NMCD to revise many guidelines, including for the prevention of Malaria in Pregnancy (MIP) to include IPTp 3. After dissemination of the changes to MIP guidelines, a significant increase in IPTp3 uptake was observed, from 0% to 47% of pregnant women as of March 2019. MAPD also helped the country maintain distribution of LLINs to nearly three quarters of pregnant women. USAID UGANDA MAPD MIDTERM LEARNING REVIEW 16 However, these interventions did not result in a substantial drop in MIP cases in the OPD or IPD as would be expected. With regards to malaria case management (test and treat), significant improvements were seen in testing suspected malaria cases, and significant decreases in providing ACTs to people who test negative for malaria. With the exception of the MIP policy change which the NMCD has squarely attributed to MAPD, the involvement of other players in this area, including Results-based financing projects that pay for improved performance of the same indicators in some districts makes it difficult to ascertain the extent to which these results are only due to MAPD intervention. Further data collection of RBF was beyond the scope of this learning review, though the team noted the need to better understand the value-add of MAPD in a district that includes RBF for the same indicators. Learnings for Result 1: MAPD had three main expected results. First, it was expected to effectively prevent malaria in pregnancy by ensuring that at least 85% of pregnant women received three or more doses of IPTp3 and sleep under an LLIN. When the project started in August 2016, the NMCD had not yet adapted the 2012 WHO recommended revisions of the MIP policy that suggested 3 or more doses of IPTp, in addition to LLIN use as measures to prevent malaria among pregnant women. Working with the NMCD, MAPD developed an addendum to the existing policy, disseminated it to districts and health facilities, developed mentorship tools, and supported the NMCD and the DHI in adapting existing tools to enable the collection of IPTp3 as an indicator. New HMIS tools reflective of these changes are due to be printed in October 2019. The first IPTp dose is now given to pregnant women from 13 weeks gestation onward, and at every subsequent visit. MAPD supply DOTS materials – cups, safe water (water filters) in MAPD districts to ensure every pregnant woman receives at least three doses of IPTp. The indicator is being monitored, and results indicate significant improvement in IPTp3 use, going from 0% in 2016, to 47% in 2019. While not yet at 85%, the results at this midterm review are encouraging and MAPD appears to be on track for reaching this target by the end of its term. With regards to LLIN, MAPD supported the NMCD to develop and disseminate the National LLIN Routine Distribution Strategy, and supported national level coordination mechanisms and mapping for different donor supported LLINs for national LLINs delivery and support. They also worked collaboratively with the MOH and the Ministry of Education and Sports (MOES) to revise guidelines for school-based distribution, and distributed them to all schools in 26 districts, thus paving the way for net distribution in schools. They supported routine LLIN distribution in ANC and in EPI clinics. They have identified and trained technical resource persons within the districts to serve as supervisors and mentors for MIP, including net distribution to pregnant mothers. Nevertheless, the results have not dramatically improved since the start of the project. As of October, to March 2019, LLIN distribution among pregnant women was at 74.0%, up slightly from 72.7% Oct to Mar 2016. The net distribution has also been irregular and not supported to last mile in a number of districts. These results are summarized in Figure 2 below. MAPD Learning Review 17 Use of IPTp is also being incorporated in ANC guidelines, and efforts on quantification based on the new guideline continues. One district management team expressed the view that while the policy for IPTp changed, the availability of SP for health facilities did not. SP is an expensive commodity, and at the moment is charged to the health facility budget which did not increase with this new policy. This implies that health facilities may not be able to meet the new demand created by this policy change. While this is true, MAPD is currently working with NMCD and NMS to improve this process. They have helped the country with forecasting, quantifications, and costing for SP in order to meet the new minimum of 3 doses per pregnant women. As of the date of the interview in July 2019, they had obtained a letter from the Permanent Secretary of the MoH that ensures that SP is made part of essential commodity, thus part of the kit that NMS is supposed to provide to facilities free of charge. However, as of now, facilities still need to pay for SP. PMI attempted to address the initial gap by buying SP to be distributed to PNFP and public institutions. However, this is not enough to meet demand, and some districts are redistributing SP from low to high demand facilities. MAPD is also supporting sites to ensure that MIP is implemented according to guidelines. Nevertheless, despite high net distribution to pregnant women, and increases in IPTp3, the incidence of malaria in pregnancy has only decreased slightly in the OPD since MAPD started. This is also in the environment of Universal LLIN distribution conducted in 2017/18 to all Ugandans. Severe malaria still occurs at a similar rate among pregnant women. Incidence of malaria among pregnant women is displayed in Figure 3. 0.0% 10.0% 20.0% 30.0% 40.0% 50.0% 60.0% 70.0% 80.0% %ANC Received LLINs % ANC received IPTp2 %ANC Received IPTp3 Oct-Mar 2019 Oct-Mar 2016 Figure 2: Uptake of Interventions to Prevent Malaria in Pregnancy in MAPD Districts USAID UGANDA MAPD MIDTERM LEARNING REVIEW 18 Discussions with MAPD revealed that the project had implemented some interventions to influence net use, but did not collect sufficient data on their effectiveness. There are dipstick analyses done, but it was never clear how their data was used to inform adaptations to interventions for net use. MAPD conducted mass communication activities in 2018, during which they conducted region-specific SBCC analyses so that messages could be tailored to debunk specific local beliefs. Mass media used included radio and TV. They have also designed interpersonal communication messages and activities. In particular, they now have 2,000 key influencers in 3 regions – Rwenzori, Masaka, and Hoima regions. They have also rolled out 757 Village Health Clubs in 2 districts- Kabarole and Bunyangabu. In these clubs, the village becomes the club, all residents are members, and meetings are held and facilitated by the VHTs. This is a good innovation as VHTs work voluntarily and their motivation and support is a national challenge. However, impact of the intervention on program objectives and goals is not known. MAPD have developed some M&E materials, but did not share any data to facilitate an assessment of their impact. However, preliminary results from the recently completed Malaria Indicator Survey shows decreased use of LLINs overall, thus indicating that current Social and Behavioral Change (SBC) approaches are not yielding results and messaging and media delivery are not resulting in a sense of personal responsibility for malaria prevention. MAPD have started to distribute LLINs in schools, and also started school clubs in several regions, including in the districts we visited. However, DHMTs were unsure as to how functional these school clubs were. 0.00% 5.00% 10.00% 15.00% 20.00% 25.00% Apr-Sept 2014 Oct to Mar 2015 Apr to Sept 2015 Oct to Mar 2016 Apr to Sept 2016 Oct to Mar 2017 Apr to Sept 2017 Oct to Mar 2018 Apr to Sept 2018 Oct to Mar 2019 MIP-OPD MIP-IPD CFR “We joined them as a team to conduct training of teachers so that they harmonize and teach students to take malaria messages home- what to do, what to tell their parents in terms of preventing malaria and treating it. We did not participate in the follow up. Schools were supposed to have malaria corners, see how kids become conversant with messages they are given, formation of school health clubs. Not sure what has happened since the training.” District Health Management Team Figure 3: Incidence of Malaria in Pregnancy in OPD, IPD, and Case Fatality Rate MAPD Learning Review 19 Learnings for Result 2: MAPD’s second expected result is to support districts to implement effective malaria diagnosis and treatment activities in line with the National Malaria Strategy. This result specifically focuses on working with Village Health Teams to implement iCCM in the highly endemic districts in the Central region; improving diagnostic capacity, and improving health provider capacity for the management of simple and severe malaria. MAPD supported the NMCD to review and disseminate its malaria case management policy, and in 2018, they supported the new guidelines for Integrated Management of Malaria (IMM) in line with the new policy. This new guideline is waiting for final approval by MOH. MAPD also supported the development and final sign off of Uganda’s first National QI standards for Malaria, having just received final MOH approval early 2019. Working with the NMCD, WHO, and the Uganda National Health Laboratory Services (UNHLS), MAPD supported the development and finalization of the laboratory quality assurance manual and parasite-based diagnostic guidelines. Together with national trainers, these new guidelines are used to train district staff. Advanced training such as the WHO Microscopy Certificate Course, offered at levels 1 to 5, was also provided to a select number who went on to receive certificates by WHO. With support from MAPD, Uganda now has 36 WHO qualified macroscopics, including 7 at Level 1, more than most other countries in Africa. External Quality Assurance/Quality Control scheme for malaria microscopy was expanded, leading to very low discordancy rate (2.8%). Most relevant personnel have been trained in malaria testing either on microscopy (lab personnel) or with RDTs (other health personnel). As a result of these interventions, testing of presumptive malaria cases (either RDT or microscopy) has increased from 69% at the start of MAPD in 2016 to 94% as of March 2019. With regards to case management, MAPD built capacity of the district and health facility staff through didactic trainings using the cascade training model. These trainings were further supplemented with onsite capacity building through district level trainers. Specifically, technical focal persons from the district got to health facilities and provided technical assistance. MAPD integrates some quality improvement activities, including clinical case reviews and malaria mortality reviews. As a result, significant improvements have also been noted in this area. At the beginning of MAPD project, 27% of patients who tested negative for malaria were still provided antimalarial. As of March 2019, this dropped to 9%. These results are summarized in Figure 4 below. However, it should be noted that there are other players focusing on improving the same indicators. For example, Results-based financing projects in Kabarole District and elsewhere also focus on improving IPTp3 and malaria test and treat. USAID UGANDA MAPD MIDTERM LEARNING REVIEW 20 Figure 4: Malaria Test and Treat Before and After MAPD Interventions, all MAPD Districts Learning: Community engagement is weak. MAPD may benefit from a shift in focus from clinical to community, while maintaining clinical gains. The Learning Team did not see much evidence of MAPD working with Village Health Teams to implement iCCM in any of the districts visited. MAPD management agree that there have been significant setbacks to implementing iCCM, some of which were beyond their control. Originally, MAPD was to implement iCCM in 2 districts, but after discussions with PMI and subsequent changes in district boundaries, they are now responsible for implementing iCCM in 12 districts. There is a national iCCM strategy, but it promotes a blanket strategy in which iCCM is implemented in the whole district, as opposed to the donut strategy, in which iCCM is implemented only in parts of the district where the population is far from a health facility to help improve accessibility. These discussions are ongoing with the MOH. Another key barrier has been availability of commodities for iCCM. While PMI can ensure supply of malaria commodities (RDTs, ACTs, LLINs), the guidelines for iCCM includes commodities for Pneumonia, as well as diarrhea, and often these are out of stock. The current guideline does not allow for 69.0% 48.0% 95.0% 27.0% 94.0% 46.0% 100.0% 9.0% 0.0% 20.0% 40.0% 60.0% 80.0% 100.0% 120.0% % Suspected Malaria Tested % Tested Positive for Malaria % Malaria Positives Treated % Malaria Negatives Treated Oct-Mar 2016 Oct-Mar 2019 MAPD Learning Review 21 iCCM to focus only on one health condition. However, MAPD is in discussions with the MOH to move forward guidelines for malaria-only community case management. NMCD has agreed to push this forward because malaria is the biggest burden of morbidity and mortality among children under five, the population served by iCCM. At the time of the Learning Review, MAPD had just obtained verbal agreement from NMCD to go ahead and focus on CCM (community case management for malaria for U5). MAPD is still waiting for official, written permission from NMCD and the PS. Yet even once that is obtained, there will still be an issue with accessing sufficient number of gloves for VHTs. While viewed as a commodity, because it is not specific to malaria, PMI cannot provide it. MAPD has participated in annual procurement planning meetings with NMS with the intention of seeing if health facilities could accommodate access to non-malaria commodities for iCCM, including gloves. While some can accommodate, it is still not sufficient. MAPD has supported a policy change that make NMS provide commodities to the ‘catchment area’, thus including VHTs and Health Facilities, as opposed to VHTs separate from Health Facilities. This has also harmonized the formulation of what the catchment area receives to include iCCM commodities. It is hoped that this change will facilitate redistribution of commodities as needed between VHTs and Health facilities, but this policy is not yet implemented. However, it still may not resolve the issue of gloves to VHTs. Since mid- PY2 to date, MAPD has been implementing iCCM in 7 of the 12 districts; six of the seven districts are in Hoima and Rwenzori regions. These were 7 districts where Malaria Consortium had been implementing iCCM with support from another donor, and MAPD took over these districts when that support ended. MAPD is now faced with a third major challenge, namely data and reporting. iCCM is implemented by Village Health Workers (VHTs). VHTs are expected to report quarterly to districts, yet at the time of the learning review, only 30% VHTs had reported. Furthermore, these reports are not fully integrated within the health facility report, are treated separately within the DHIS 2, and no one uses the data. From site visits, the team learned of other challenges with data from VHTs including poor handwriting; outright lies (if the child is 6 years old, they can still say s/he is under 5 years old), and other misinterpretation of the indicators, as well as double counting /double reporting. There were reports from some health facility staff that some VHTs fake reports of home visits, highlighting the need for improved supportive supervision if VHTs are to actualize their potential in making a strong contribution to decreasing the malaria burden in their communities. In one of the MAPD districts visited, iCCM is supported by other partners like TASO, UNICEF, and PACE with a focus on data reporting and integrating VHTs. Learning for Result 3: MAPD’s third expected result was to build capacity of the NMCD and DHMTs to manage and sustain efficient malaria activities in focus districts. This result focuses specifically on strengthening capacity of DHMTs to set and meet capacity improvement targets, manage stock efficiently, and ensure health workers are trained in the diagnosis and treatment of simple and severe malaria. It also focused on demonstrating value for money, meaning that cost-efficiency for delivering malaria services can be demonstrated. MAPD was designed as a case management improvement project. But now the [National] Malaria Program is more than that. There is now a very strong focus on SBCC to get individuals to participate in getting themselves malaria free. We must strengthen the personal response to malaria NMCD USAID UGANDA MAPD MIDTERM LEARNING REVIEW 22 All districts visited say that MAPD has trained district staff in malaria testing and case management, and have worked with these trained staff to cascade the training to lower level health facilities within the district. MAPD has also supported them in supervising malaria activities. However, among the three result areas, this is where MAPD’s current performance was perceived to be weakest by the NMCD and by the DHMTs. The NMCD’s view is that MAPD has focused largely on building technical capacity of districts – knowing how to diagnose and treat malaria, but not in the actual management of the malaria program. MAPD management did not have the same understanding as the district leadership, or possibly even NMCD, that this was the expectation. One District Health Officer put it succinctly: Other DHOs in the districts we visited echoed similar views. There is a general sense that the MAPD project has not sufficiently built capacity of the district and NMCD to implement the malaria program to contain, and possibly eliminate malaria. Yet this aspect of the project is critical. NMCD is currently undergoing a paradigm shift, moving from a program that waited for people to fall sick, come and get treated, to one that stratifies its activities and moves the country towards pre-elimination. Different technical and managerial skills are needed. The NMCD will need to be able to provide districts with the technical direction they require through policies, guidelines, indicators, etc., and enable districts to assume their role in implementing the guidelines, manage supply chain, set and monitor goals, manage human resources, finances, etc., in accordance with their own malaria epidemiology. All of the districts visited expressed concern over the lack of regularity with which MAPD provided support in the area of data management, particularly at health facilities – the point of collection, but also with District Biostatisticians and HMIS staff. Weak data processes across the continuum (collection, cleaning, analysis, and use), as well as lack of data from key constituents (i.e. VHT, private sector, etc.), unless improved, will hinder efforts to assess needs and make smart decision to decrease the burden of malaria. Stakeholders at both NMCD and district levels expressed concerns that current approaches of MAPD will not lead towards original intention of district capacity building for malaria prevention and control. “MAPD was envisioned as a district capacity building project, but it has limited this to service delivery. They need to position them to steward a program – leadership, a full understanding of malaria programming and the skills needed to run it, including leadership and operations management.” NMCD “In the beginning [of the MAPD project], we were guided and came up with district plans – all districts prepared plans to be supported through MAPD. I don’t know where they ended; we were never supported, much as we sat together for 4 days to create our plans. We were never directly supported by MAPD [to implement them].” District Health Officer (DHO) “The expectation of the NMCD is that district leadership is empowered to prioritize, lead and owns malaria reduction interventions as evidenced by decisions/innovations based on regular data use.” NMCD MAPD Learning Review 23 MAPD agrees that to date there has been minimal focus on managerial capacity building of DHMTs, and non-systematic engagement of higher-level leadership, like the CAO and LC 5. However, MAPD recently conducted a management assessment of each district. Findings from the DHMT assessment revealed that most districts are weak or very weak in the area of leadership, and all are grossly underfunded, receiving approximately 1,500,000 Uganda Shillings or approximately $405 USD per month to implement malaria activities in the district. This barely covers operational costs. During year 4, MAPD will be developing a framework on how to address leadership and managerial aspects of capacity building of DHMTs based on the results of the management assessment. LEARNING QUESTION 2: TO WHAT EXTENT DO INSTITUTIONAL, STRUCTURAL, MANAGEMENT AND LEADERSHIP FACTORS (AND OTHERS IDENTIFIED DURING THE REVIEW) INFLUENCE IMPROVEMENTS IN SERVICE DELIVERY IN THE MAPD DISTRICTS? Institutional, structural, management and leadership factors affecting service delivery identified during the learning review comprised of coordination and communication mechanisms, formal organizational leadership and collaborative approaches to service delivery. A number of KIIs mentioned MAPD’s contribution to improved management of access to malaria commodities and to ensuring minimal stock outs for malaria commodities. (KII NMCD, HF In-charge, FGDs, Districts). MAPD was also acknowledged by NMCD for their contribution in MIP, and LLIN coverage (KII, NMCD, Kampala, KII- District level and Health Facilities in-charges). However, recent data from MIS 2019 provides evidence that the use of mosquito nets is declining. MAPD’s prevention approach and social behavioral component need strengthening and an integrated public health approach to malaria prevention, diagnosis and treatment as opposed to the non-integrated clinical approach explored. National stakeholders also expressed the view that MAPD should design integrated social and cultural context specific SBC interventions, carry out net use surveys to establish the factors contributing to the down ward trend in net use and focus on stratification and layering of malaria interventions for effective targeting, prevention and treatment of malaria in Uganda. MAPD worked with MOH to update and disseminate the IPTp3 policy at health facility level; participated in developing the interim addendum to the HMIS 009 and 005; incorporating ANC guidelines into the MIS and protocol on quantification of SP. Working with MOH, MAPD supported the National LLINs strategy development and its collaboration with JMS has contributed to minimal stock out of malaria commodities at health facility levels (KII, National Level Stakeholders, Kampala). Improved coordination, planning, collaboration and networking with the different malaria actors facilitated timely access to commodities and services (KII, National Level Stakeholders, Kampala). Efforts to build human resources capacity at health facility to diagnose and treat malaria were recognized by the NMCD as well as by all districts visited. Learning for the influence of institutional and structural factors: The participants expressed the need for MAPD to focus capacity building interventions for DHMTs to be able to lead the planning, monitoring and implementation of malaria programs in their districts, including “…capacity building has mostly focused on technical capacity building, the capacity to diagnose and treat malaria, quality assurance (by positive control) ensured and integrated management of malaria.” NMCD USAID UGANDA MAPD MIDTERM LEARNING REVIEW 24 engaging private sector. Interviews with WHO revealed that at least 60% of outpatient consultations are done in the private sector. The Learning Review team found similar results from exit interviews, since the health facility was the first place where care was sought only for 35% of patients seen that day. Others, nearly two-thirds, had previously gone to a private clinic or a drug shop, and a small minority had previously been seen by a VHT (Figure 5). The private sector’s major role as a primary care provider makes it imperative that at least certain aspects of their work, especially guidelines and reporting, follow the national standard. In Lyantonde, the Learning Team found a great example of how a district leadership is working with the private sector. They have a very high level of private clinics reporting (14 out of 17) regularly on key indicators. One strategy they have used to successfully work with the private sector was to not renew private clinic registrations unless they report. However, this was not accomplished with support from MAPD, whose work with the private sector to date was found to be insufficient in creating lasting change at the any level. Figure 5: First Place to Seek Health Care for Current Fever Learning from Client Exit Interviews Group interviews with district officials and with MAPD staff provide evidence that to date, MAPD has done very little to support districts in ways that truly enhance their capacity to lead. MAPD need to support DHMTs to develop the capacity to integrate key functions from the private sector, in particular, reporting, and to ensure that they follow national policies and guidelines for malaria test and treat. MAPD staff assured the Learning Team that interventions to improve this area will begin in PY 4 based on results of a recent leadership assessment of all DHMTs in MAPD districts. “District capacity building was limited to service delivery capacity improvement. Areas of governance, leadership and management, which are critical for successful implementation of projects are not considered.” NMCD MAPD Learning Review 25 In interviews with multiple district officials, the Learning Team learned that there is need to strengthen the capacity of the district leadership and health facilities in quality data collection, analysis, timely reporting and data use to inform malaria programming and service delivery. There are also identified gaps in terms of quality of data collected and timely reporting for community data and use at HF, District and Central level and this calls for continuous Quality of data improvement and data quality checks support. District capacity also should be supported to continuously make use of data for decisions about malaria prevention and treatment. (Observation, Participants, Learning review). Participants mentioned the challenge of misalignment between USAID MAPD and the district planning calendars, with the USAID MAPD planning period taking place in August/ September, while the district planning is scheduled around June / July period. (MAPD Headquarter staff, Kampala) MAPD Program has contributed to increased access to and availability of malaria in pregnancy and malaria diagnosis services in MAPD Districts, implemented the school net distribution activities in selected schools (KII, National Level Stakeholders, Kampala). The school net distribution activity was a one–off activity, possibly due to PMI funding. However, such activity requires consistency and scaling up if it is to have the desired effect of reducing malaria incidence in schools. The Learning Team learned that there are multiple committees on malaria, but at the district level they are not very well coordinated and do not meet regularly. Malaria Committees could play a significant role in coordination of activities at the district, including mobilization of resources and facilitating working relationship between the multiple stakeholders (public and private) in the districts, thus enabling the district leadership to plan, budget and disseminate information on malaria in a coordinated manner. MAPD needs to ensure that the Malaria committees in districts meet regularly and that holistic programming for malaria is supported. (KII, GF, Kampala). Through multiple interviews followed by the Learning Team’s internal reflections that took place during daily learning review meetings, the need to refocus SBC efforts became increasingly evident. In particular, participants expressed that now the need is to promote personal responsibility in preventing malaria. In interviews with WHO, the suggestion was made that SBC for malaria should follow a similar pattern as that for HIV. In particular, there was reference made to creating a socially recognizable signal for malaria prevention, similar to the ‘bell’ heard on the radio prior to or as part of a behavior change message to prevent HIV. Other stakeholders at national and district levels echoed similar views. KEY LEARNING POINT Malaria has reinvented itself from being health sector issue to being a public health issue NMCD, GF, PMI “Social behavior change interventions need to be strengthened to also effectively support individual responsibility to malaria… that is how we shall be able to reduce malaria incidences in Uganda.” NMCD USAID UGANDA MAPD MIDTERM LEARNING REVIEW 26 MAPD needs to strengthen the community component of malaria prevention and treatment, scaling up innovative SBC approaches and interventions, as well as actively engaging the community in taking personal responsibility for malaria prevention, diagnosis and treatment. SBC will need to be designed specific to social and cultural contexts, and will need to integrate a public health approach as opposed to clinical approach to malaria prevention, treatment and diagnosis. In addition, the iCCM component under MAPD needs strengthening in terms of capacity building, harmonization of interventions as well reporting on activities. There is also need for dialogue and consensus around sustainable incentives and renumeration of VHTs. (KII, NMCD, Global Fund, Kampala). Learning for Management and Leadership: MAPD is managed by an Implementing Partner with technical expertise in malaria. Most of the technical staff were hired and have contributed to the attainment of the project objectives to date. However, the team discovered key misunderstandings between MAPD and the NMCD, and MAPD and district teams that has negatively impacted the management of the project. Further, the Learning Team observed that some interventions, like iCCM, private sector engagement, and others, were not started, or slow to start. Other interventions, like those in the area of capacity strengthening along the continuum of data has enjoyed limited momentum in implementation which is likely to have a detrimental effect on attainment of intended project objectives. MAPD management told the Learning Team that whereas Year 1 was used for setting up the project, Year 2 was used to implement the project and there was no clear explanation of what is going on in year 3, though activities are well described in their approved workplan for Year 3, and appear to be reinforcing capacity building efforts from Year 2. Funding difficulties due to a number of factors, including the Government shutdown, may have also influenced the amount of work that MAPD could do in PY 3. The Learning Team was told by three of four DHMTs visited that many of the approaches to implement interventions for MAPD lacked intensity, consistency. These include one-off trainings, irregular data use support, among others. In addition, the project overall has experienced an imbalance in focus between clinical and public health approaches to decreasing malaria burden in Uganda, though this may be related to perceived PMI priorities of improving clinical care in the beginning of the project. MAPD has not yet stratified its implementation approach despite the differences in malaria burden within the different districts. Though some data is being collected, the learning participants did not see evidence of MAPD using data consistently to gain insights that drive or adjust project implementation. Discussions with the NMCD, and further internal reflection of the Learning Team underscored the need to re-align the MAPD project to the current realities of malaria in Uganda. The NMCD, WHO, GF, and DFID all expressed the need for stratification and geographic and context specific programming to address the challenges of malaria by region, district, and even sub-districts. Some of the districts visited, including Masaka, were already thinking along these lines. “The structure of MAPD is beautiful but execution has had its own challenges… “ NMCD There is joint implementation in all we do using district technical resource persons. We never do any activity without the district and/or NMCD, but the part to improve is the joint planning. MAPD Staff MAPD Learning Review 27 MAPD developed and signed MOUs with 42 districts in 5 regions indicating the nature of assistance, roles and responsibilities of the Districts and MAPD in malaria prevention, diagnosis and treatment. Whereas MAPD and respective Districts’ roles were clearly stipulated in the MOU, DHMTs and National Stakeholders told the Learning Team that MAPD did not fulfill its obligation of providing the MAPD project approved activities and the schedule for implementation to guide the District operations, and did not communicate to the districts their respective output targets. As plans changed, several participants mentioned that MAPD did not effectively engage with the districts about the changes in the agreed plan and funding and did not entirely harness the existing systems to provide the necessary support for strengthening their capacity in a sustainable manner (KII, DHMTs in all four districts). Thus, while MAPD said they had implemented joint planning in most districts, all district teams interviewed said there were no joint planning, or overall joint implementation of MAPD activities in their districts, except for supportive supervision. The Learning Team also found evidence of MAPD providing some support in improving commodity planning and management, though this is relatively new (started mid-Year 3). MAPD also raised concerns about the challenges to joint planning given the misalignment of fiscal years—MAPD follows the USG fiscal year of October 1 to September 30th, while the districts follow the GoU fiscal year of July 1 to June 30th . Providing clear support and ensuring strong collaboration at the organizational level was an important element noted by many learning review participants at national, district and organizational levels. Many of the learning participants emphasized that all stakeholders share the responsibility of preventing and treating malaria, and ensuring effective coordination of activities is key as a means of attaining the pre-elimination stage (KII MOH Malaria Control Division; National level stakeholders, District leaders). However, as pointed out by a National Stakeholder, MOH bureaucracy and lack of communication or understanding from MAPD to NMCD created delays. MAPD’s inability to share its detailed budget with the NMCD or DHMTs made them appear as if they were not being transparent about funding, and their way of working to respond to their contract at times made them look as if they wanted to lead, instead of support, district activities. National stakeholders articulated the value and importance of role delineation and communication between NMCD and MAPD, and in particular, that this was lacking. Interviews with all national stakeholders raised this issue, and MAPD staff referred to ‘explosive’ discussions with the NMCD. The diagram below illustrates the importance of communication and coordination relationships, clarity in role delineation between MOH and MAPD and the likely effects of having a consistent focus for applying different approaches to malaria prevention, diagnosis and treatment. Effective formalized communication processes are critically important for facilitating collaboration to ensure regular opportunities between the different stakeholders to stay connected. Figure 6 illustrates the key relationships. The [MOH] bureaucracy drives me crazy and the inactivity and inability that happens when you get caught up in meetings and bureaucracy. And you’re unable to act because you are too busy exchanging emails on whose responsibility it should be to lead an activity rather than getting out there and actually implementing and making things happen” National Stakeholder “… If we are going to jointly address the issues in malaria prevention, diagnosis and treatment, we need to talk to each other in a mature and beneficial way.” National Malaria Program Stakeholder USAID UGANDA MAPD MIDTERM LEARNING REVIEW 28 The team noted that there are at least four key players in reducing the malaria burden: NMCD, districts, PMI, and MAPD. The strength and harmony of the relationships between these players can ensure or impede successful implementation of MAPD, and ultimately the reduction of the burden of malaria in Uganda. The team found that the relationship between PMI and NMCD was solid, with NMCD expressing appreciation for PMI, and other national stakeholders, including the GF, DFID, and WHO expressing appreciation for the role that PMI plays in helping to coordinate malaria activities with them in support of the NMCD. However, at the time of this Learning Review, the team also learned that there are significant misunderstandings between MAPD and some districts, and the relationship between MAPD management and NMCD leadership is severely strained. Participants spoke about the need to develop more formal working relationships and the increased need to engage the DHMTs as well as the NMCD in implementing activities supported by MAPD. On the other hand, MAPD says they have been engaging NMCDs and DHMTs by ensuring their attendance at District Workplan meetings and integrating them in the agenda. And though this can be demonstrated by reviewing the attendance sheet, National and district stakeholders expressed a feeling of being more of a witness. There was lack of clarity on roles of the different stakeholders and players namely NMCD, the districts and MAPD. Being clear about the mandate of each player and ensuring that they are well understood by all parties is important for malaria prevention and treatment in Uganda. Misinterpretations about each other’s mandates seemed to be detrimental to collaboration. Several national stakeholders, including NMCD, DFID, GF, JMS, and others, shared the view that MAPD project is being implemented in the project mode rather than supporting the existing structures of DHMT. They expressed that MAPD has not supported DHMTs and health facilities in a sufficient, regular manner, or in a manner that enables DHMTs to take responsibility for ensuring results of the malaria program. NMCD and some DHMTs had a very minimal view of what the project is actually Figure 6: Important Relationships to Deliver Results Reduce Malaria Burden PMI MAPD DISTRICTS NMCD NMCD is not invited to participate in planning with MAPD. Instead, MOH-NMCD is witnessing more launches of district malaria work plans yet as national stakeholders we are not invited.” National Stakeholder “[Is] interaction with DHMTs, health facilities and community as well as use of data a priority in MAPD districts?” NMCD MAPD Learning Review 29 supposed to accomplish, as reflected in some of the questions they asked the Learning Team. This contributes to minimal ownership of project interventions by NMCD and the district leadership. Following discussions with the Learning Team, it was agreed that PMI need to rethink its packaging of malaria interventions. They need to focus on zone-based targeting for low, medium and high incidence zones, and also explore any change in incidence and prevalence of malaria in children above 5 years. Ecological zoning could be used to inform malaria programming, scale up support for entomological surveillance / monitoring and make use of meteorological data to program for malaria interventions. There is need to promote context specific behavioral change approaches and to continuously make use of data to inform malaria programming. PMI cannot keep on addressing malaria prevention and treatment using the traditional approaches when the malaria programming and service delivery landscape is evolving (Observation, Participants, Learning review). For instance, Malaria incidence in Lyantonde district (which is a MAPD district) stands at 7%, which is pre- elimination stage. Interventions required in Lyantonde now are different than those required in West Nile districts, where prevalence is 50% or higher. MAPD, together with PMI, could identify areas with hot spots or provide guidance on how to identify hot spots, stratification and targeted programming. There is need to improve on drug shops and private sector reporting and to formally engage the private sector. MAPD staff expressed agreement with this. Technical and political leadership in some districts participated in the meeting with the Learning Team, signaling their interest in malaria prevention. In some districts, such as Lyantonde and Kabarole, it was observed that political will towards malaria prevention exists, and the political leaders support programming for malaria prevention and service delivery. In these districts, there was evidence of team work and coordination among DHMT members, district support for malaria interventions and decision making, data for commodity distribution was evident, and the districts have not experienced stock outs in the most recent period (Observation, Participants, Learning review). However, the Learning Team realized that MAPD is not maximizing their engagement with political leadership for malaria control. From the interactions with the National stakeholders and the Districts, we learned that there has been limited political engagement in malaria prevention at district level. The participants expressed the need for increased political engagement in malaria prevention using the existing district coordination mechanisms. It was also learned that the districts are not actively engaged in programming for malaria. National Level stakeholders expressed the concern that districts need to pick up malaria programming and adapt to revolving malaria prevention, diagnosis and treatment trends to be able to effectively reduce malaria incidences in their districts. (KII, NMCD, GF, Kampala). USAID UGANDA MAPD MIDTERM LEARNING REVIEW 30 LEARNING QUESTION 3: IS THE GEOGRAPHICAL COVERAGE AND SCALE OF MAPD ADEQUATE TO SIGNIFICANTLY INFLUENCE ACHIEVEMENT OF THE GOAL TO GET UGANDA TO PRE-ELIMINATION STAGE BY 2020? MAPD covers 52 districts, most of which are in the buffer zone, with the exception of 11 districts in the West Nile Region. In Figure 7, MAPD districts are outlined in red. In 2016 USAID made a strategic decision to geographically separate activities of MAPD, a project with a singular focus on malaria, from those of several RHITES projects that would integrate malaria activities with other health systems strengthening and disease activities funded by other streams, including PEPFAR. The justification behind this decision was that RHITES would leverage PEPFAR and other funds to implement malaria prevention and case management interventions. Results of the 2019 Malaria Indicator Survey (not yet published) confirms that Uganda has three distinct malaria zones: a high malaria belt with prevalence ranging between a high of 50% in West Nile to a low of 23% in Lango. In addition, this zone includes Karamoja (42%) and Busoga (39%) sub regions. The intermediate Buffer zone has malaria prevalence ranging between 20% in Teso sub region and 10% in Bugisu sub region, and includes Bunyoro at 15% and Northern Buganda at 14%. The lower prevalence zone consists of Kigezi, Kampala, South Buganda, Ankole, Bukedi, and Tooro sub regions. Prevalence in this zone is as low as <1% in Kigezi and as high as 7% in Tooro. More than 55% of malaria cases reported during the period April 2018 and March 2019 were in in the high malaria belt population (source: DHIS2). The overall malaria prevalence in Ugandan population is 17%, 33% in Refugee settlements and 7% in districts where Indoor Residual Spraying has been carried out. Currently, MAPD’s coverage represents 54% of the people with malaria parasite (PWMP); while the RHITES integrated programs cover 71 districts representing 40 % of PLMP. 8 districts are not covered by any USG malaria intervention. Figure 9 shows the incidence of malaria in the 52 MAPD districts compared to RHITES, and non-supported districts. From here it is evident that MAPD covers 1.7 times as many incident malaria cases compared to RHITES sites. Figure 8 Malaria Prevalence by Region (UMIS 2018-19) Figure 7: Uganda Malaria Incidence April 2018 to March 2019 MAPD Learning Review 31 Figure 9: Malaria Incidence Rate in MAPD, Non-USG, and RHITES Districts Source and Year: DHSI2 (April 2018 - March 2019) As the project with the USG’s largest investment for malaria, MAPD does cover only slightly more than half of the malaria burden, yet receives 3.4 times as much funding for malaria compared to RHITES projects that cover 40% of the PWMP, including many high burden districts. Viewed from this perspective, PMI investment do not appear to align with the current malaria burden. In line with the changing malaria epidemic in Uganda, it became clear to the Learning Team that PMI and NMCD should explore targeting high malaria burden districts with high impact interventions, and vary the scale and degree of intensity with the need. This may imply allowing some degree of overlap between MAPD and RHITES, or finding other ways of intensifying (and stratifying) malaria interventions within integrated projects. Interviews with national stakeholders including NMCD, WHO, the GF, and DFID revealed that the next malaria strategy for Uganda will include a stratified approach. WHO is currently in the process of supporting the NMCD to adapt the Malaria Elimination Framework. They are conducting a surveillance assessment that will help all stakeholders understand what is currently happening in the country, up to sub-county level, and help the NMCD to develop a general and an entomological surveillance framework with the view of moving Uganda towards malaria pre-elimination (KII, WHO). If Uganda is to push towards pre-elimination of malaria as per Uganda Malaria Reduction Strategic Plan UMRSP 2014 – 2020 strategy, investment must follow the burden of disease. The country needs to move ahead with stratification by prevalence as proposed by the NMCD. In addition, it is time for micro-targeting differentiated interventions going to different strata, districts, sub-counties and ecological zones of entomological significance. The Learning Review team heard that wetlands, crater lakes neighborhoods, settlements near forests, refugee camps and settlements, and institutions such as referral hospitals and high-volume boarding schools all need entomological surveillance. In addition, surveillance using spatial analysis of meteorological data will provide early warning to weather-prone geographic zones, and could help with pre-planning for malaria seasons. 181 65 105 0 20 40 60 80 100 120 140 160 180 200 MAPD None USG Malaria Support RHITES Malaria cases per 1000 Support PLMP by MAPD vs RHITES and None USG Support USAID UGANDA MAPD MIDTERM LEARNING REVIEW 32 LEARNING QUESTION 4: WHAT IS THE EVIDENCE FOR IMPROVED FULFILLMENT (OR LACK OF IMPROVEMENT) OF THE ROLE OF GOU INSTITUTIONS (NMCD AND DHMT) IN QUALITY IMPROVEMENT OF MALARIA PREVENTION AND TREATMENT SERVICES? At the level of the NMCD: Learning NMCD supported development, approval processes and dissemination of protocols, strategies, and national standards for quality improvement. NMCD also facilitated the updating and dissemination of IPTp3 policy; ANC guidelines and protocol on quantification of SP. polices ad guidelines for school net distribution, malaria commodities, iCCM, Social behavioral change and revised the national malaria strategy (KII, NMCD, Kampala). At the National level, NMCD facilitates Thematic- Technical Working Groups and Roll Back Malaria partnership meetings with malaria implementing partners. This has enabled NMCD to get feedback from partners and support programming. Other coordination mechanisms for malaria prevention and treatment include the Surveillance Monitoring and Evaluation Operation Research TWG, the iCCM TWG, Program Management TWG, Behavioral Change, Communication and advocacy TWG, Integrated Vector Control TWG and Roll Back Malaria (RBM) Partnership meetings (KII, NMCDP, Global Fund, Kampala). At the level of Districts: Learning Data quality and use in programming and service delivery needs improvement Making use of data for programming needs to be strengthened, as data use is not maximized across the data continuum. It was learned that VHT data is submitted directly to district without utilization at facility level. Cleaning and review of data is inconsistent in some districts, and there is minimal capacity at health facilities and districts for data analysis and getting meaning out of the data. Therefore, use of data at all levels is still insufficient. (KII, NMCD, Global Fund, Kampala). National level support to the districts in terms of mentoring and support supervision visits, quarterly data review meetings, mortality reviews and clinical audits needs to be scaled up at district and health facility levels. (KII, NMCD, Kampala). Staff turnover at health facilities affects quality improvement efforts During the interactions with the different participants, the concern for continuous staff turnover for staff at district level, whose capacity they have built was raised. Due to a number of structural, social, administrative and leadership factors, staff who have been trained on various aspects of malaria diagnosis and management have left for other institutions /organizations creating fresh capacity building gaps (KII, Health Facility in charge, Kabarole District) “Continuous’ aspect of QI not adhered to- monitoring not regular It was observed during the learning review visits that there was no evidence of on-going and proactive engagement to improve care processes for malaria diagnosis, treatment and prevention using quality improvement approaches at health facilities visited. The quality improvement interventions carried out in some health facilities were one-off activities (Participants, Learning Review). MAPD Learning Review 33 LEARNING QUESTION 5: HOW DO YOU COMPARE APPROACHES BY MAPD TO THOSE OF RHITES? From the interactions with the stakeholders at national, district and health facility levels, MAPD approaches to malaria service delivery appear to be related to the RHITES approaches in certain instances but majorly they are different in the levels of intensity and engagement with the national and district stakeholders, DHMTs and at Health facility level. The work processes in MAPD tend to be less cognizant of the need for active engagement of the National level managers in planning, collaboration and coordination processes; while in RHITES SW, there was a lot more group work, active engagement and acknowledgement of the National and district level roles (KII, District leaders and stakeholders- Mbarara and Isingiro districts) MAPD Approaches MAPD is a malaria only program, that focuses on reducing malaria-related childhood and maternal morbidity and mortality in Uganda. At national level, MAPD worked with MOH to update and disseminate the IPTp3 policy at health facility level; participated in developing the interim addendum to the HMIS 009 and 005; incorporating ANC guidelines into the MIS and protocol on quantification of SP, and supported the development of other guidelines. MAPD has built the capacity of health care and laboratory service providers in selected districts of operation (KII, NMCD, Kampala). From the discussions held with the different national and district level stakeholders, the MAPD approaches to malaria were considered to be multi-pronged. MAPD participated and worked with NMCD to write polices ad guidelines for school net distribution, malaria commodities, iCCM, social behavioral change and the revised malaria strategy (KII, NMCD, Kampala). MAPD trained health service providers and laboratory personnel on malaria diagnosis and treatment, conducted External Quality Assessments to assess the quality of laboratory testing EQA and conduct site mentorships with DHMTs; support data analysis and data use, track stock status and ensure routine LLIN distribution at ANC and EPI. Other approaches employed by MAPD include supporting the Directly Observed Treatment in IPTp, disseminations of HMIS tools and rolling out of malaria guidelines. Based on the existing data, MAPD has contributed to improvement in Malaria in Pregnancy indicators, LLIN, ANC uptake among pregnant women, as well as testing fever cases before treating them for malaria (KII, National level stakeholders, District participants). Learning: To achieve its intended objectives, PMI needs to rethink the malaria approach in line with the changing malaria context and landscape. PMI should therefore repackage its approach to malaria prevention, diagnosis and treatment cognizant of the low, medium and high prevalence malaria zones. There is need for MAPD to scale up data use to inform program design; carry out root cause analysis and develop a diversified and integrated program as opposed to parallel programming. MAPD should also integrate community engagement interventions in malaria prevention and treatment; scale up social behavioral change targeted approaches to malaria prevention and treatment and ensure data quality improvement (Participants, Learning Review). From the interactions with the national level stakeholders, MAPD is experiencing some structural, institutional and management challenges that affect relationships with the Ministry of Health and the districts (KII, National Level stakeholders, Kampala). There is need for MAPD to intensify joint planning meetings, support supervision visits, death audits, incidence response for malaria and increase on regular interactions with the districts. MAPD already does many of these activities, but agrees that planning and follow up could USAID UGANDA MAPD MIDTERM LEARNING REVIEW 34 be improved. At the national level, MAPD should find ways to ensure that NMCD engagement becomes stronger. (Observation, Participants, Learning Review). Integrated Approaches RHITES SW is an HIV and AIDS integrated program that provides integrated support to national, district, facility and community level interventions. At the National level, RHITES projects train health care workers to implement the highest quality services routed in globally-recommended service guidelines, supports integrated funding initiatives, Partnerships, Integrated guidelines & tools and Integrated data collection platforms. At district level, RHITES works with the districts to support integrated supportive supervision visits, tools and workplans development and data reviews. Health Facility level support under RHITES comprises of ensuring improved client flow, setting up multidisciplinary management teams, expanding family centered care and integrated tools and reporting. RHITE works with the communities to facilitate integrated outreaches, coordination with local partners and strengthening data gathering. (Desk review, Observation, Participants, Learning Review) The Learning Review Team heard that RHITES SW uses a multi-pronged model for improving malaria by establishing a cadre of resident malaria mentors in their districts, conducting regular and frequent site malaria mentors with DHMT and mentors, conduct weekly data reviews to guide malaria case management and track stock status for commodities, carry out targeted interventions for malaria (RHITES SW Staff). RHITES SW mentorship approach strives at strengthening capacity of health workers in data collection, analysis and use, participate in HMIS tools dissemination and integrate activities with other supported project work. (Observation, Participants, Learning Review) Learning There are differences in technical assistance delivery approaches, packaging and targeting of malaria interventions between MAPD and RHITES SW (Observation, Participants, Learning Review). Whereas MAPD is a malaria only project, RHITES SW is an integrated HIV and AIDS program that also delivers on malaria interventions. However, there is evidence of positive trends for malaria prevention, diagnosis and treatment in MIP in OPD, IPD and CFR data in RHITES SW, as opposed to MAPD data for the similar period as depicted in the diagram below: Figure 10: MAPD and RHITES (Source: DHIS, 2019, Extracted and analyzed by USAID SITES) 0.0% 5.0% 10.0% 15.0% 20.0% Apr to Sep 2014 Oct to Mar 2015 Apr to Sep 2015 Oct to Mar 2016 Apr to Sep 2016 Oct to Mar 2017 Apr to Sep 2017 Oct to Mar 2018 Apr to Sep 2018 Oct to Mar 2019 MALARIA IN PREGNANCY TRENDS %OPD Malaria in Pregnancy % IPD Malaria in Pregnancy CFR. 0.0% 2.0% 4.0% 6.0% 8.0% 10.0% Apr to Sep 2014 Oct to Mar 2015 Apr to Sep 2015 Oct to Mar 2016 Apr to Sep 2016 Oct to Mar 2017 Apr to Sep 2017 Oct to Mar 2018 Apr to Sep 2018 Oct to Mar 2019 MALARIA IN PREGNANCY TRENDS %OPD Malaria in Pregnancy % IPD Malaria in Pregnancy CFR. 35 USAID UGANDA MAPD MIDTERM LEARNING REVIEW V. CONCLUSIONS AND LEARNINGS CONCLUSIONS Question 1: There have been discernable gains made, and in particular, gains in IPTp3 could be attributed to MAPD project. However, there are other players also working to improve the same indicators, like Results-based financing, and specific attributions cannot be ascertained without additional analysis. Question 2: MAPD has immensely contributed to the prevention and treatment of malaria in Uganda. However, there is need to understand the factors driving net use and the declining use of nets, need to improve on coordination, planning, collaboration and networking that are necessary for timely access to commodities and service delivery. MAPD should also scale up interactions with DHMTs and engage the private sector as stakeholders in malaria prevention and elimination. Question 3: The 52 districts covered by MAPD are mostly in the buffer zone or low malaria zone, with the exception of 11 districts in the West Nile Region. As such, MAPD covers just slightly more than half of the population with malaria, yet it receives the majority of malaria activity-level funding from PMI. The Learning Team realized the need for geographic realignment to improve programmatic cost effectiveness in efforts to decrease the malaria burden in Uganda. Question 4: There are efforts by the district level to ensure that they collect, analyze and make use of the data. However, the quality of data that is being collected and analyzed as well as the capacity of the districts to make meaning of the data for malaria still need improvement. MAPD should improve on its working relationship with NMCD and continue to actively participate in national thematic technical working groups for malaria prevention and elimination. MAPD needs to work with NMCDto find ways of strengthening the VHTs so that they can integrate iCCM in their service delivery package. Question 5: MAPD and RHITES implement similar approaches to malaria prevention, diagnosis and treatment. However, efforts by RHITES SW are more regular and engage more closely with the national and district stakeholders, DHMTs and at Health facility level. RHITES SW takes cognizance of the national level stakeholders and managers in planning, collaboration and coordination processes; intensively engages and acknowledges National and district level roles; DHMTs are actively engaged in malaria programming, prevention, diagnosis and treatment. The DHMTs interviewed expressed a desire to work similarly with MAPD. SUMMARY LEARNINGS President’s Malaria Initiative (PMI) • Strengthen community engagement – having invested in case management at facility level for about 13 years now, PMI needs to consider maintaining the good achievements at facility level but go beyond the facility and take the message of malaria prevention and treatment more rigorously at the community level. There is also a need to review the pertinence of limiting community case management to children under five, if justified by epidemiological data. USAID UGANDA MAPD MIDTERM LEARNING REVIEW 36 • Strengthen Social Behavior Change (SBC) – this is in relation to the above community engagement aspect. PMI needs to consider revisiting the SBC approach to a more novel and effective one beyond the traditional SBC interventions rather than focusing more on social and individual level behavior change. The most pressing issues that need to be addressed through SBC are the issue of net use, IPTp use, early self-reporting when febrile. • Strengthen quality reporting at all levels in support of PMI's focus on data use for decision making, with a focus on data use at the facility and community levels. • Better define, streamline, and strengthen capacity building at all levels through joint planning and implementation with GoU stakeholders, as well as supportive supervision from the central level to districts, from districts to health facilities, and from health facilities to communities. • Consider intervention prevalence-based zooming i.e. a stratified differentiated intervention package depending on prevalence as reported by the current Malaria Indicator Survey (MIS). Emerging low prevalence areas in the central region would ideally have different interventions as compared to high prevalence areas such as West Nile. • Respond to the question of an increase in mobile populations such as refugees and migrant workers traveling from high prevalence to low prevalence areas who are a potential reservoir that can drain the impact currently created by PMI interventions or even contribute to an epidemic or upsurge. • Anecdotal evidence from the review indicates that nets are lasting less than three years with most Focus Group Discussions (FGDs) reporting 1-2 years of usable life. This emphasizes the urgency of launching the MAPD durability study, as the NMCD and malaria donors would be required to rethink UCC and routine net distribution strategy and timing, if the results concurred. Rethink PMI's work with VHTs. Interviews with FGDs revealed varying amounts of allowances, particularly transport and lunch allowances, provided by different actors from USD 8-15 per sitting or per month or per quarter. All FGDs agreed that the amounts given are low and the highest amount suggested was $27 (UGX 100,000/=) per quarter. PMI needs to review allowances provided to VHTs by its implementing partners to create harmony in amounts given as the budget allows. In addition, there's need to discuss these allowances in light of sustainability of the program i.e. the ability of GOU or district or community structures to maintain the allowances and the activities when the program ends. Malaria Action Program For Districts (MAPD) Result 1: Effective malaria prevention programs implemented in support of the National Malaria Reduction Strategic Plan (UMRSP 2014-2020) LLIN: a) Sustainable model of LLIN delivery to the HFs through the districts to the HF (rather than MAPD support). a. The model is correct. However, as in transition from old MAPD led model, need to improve support to this change in some districts. Currently MAPD supports districts MAPD Learning Review 37 that find this model challenging especially to hard to reach or high-volume facilities. To improve: i. MAPD should map this need and link specific activities to each district challenges and document it better to showcase this support. ii. MAPD will time delivery LLINs to medicines and NMS delivery to the last mile to support the move of LLINs to HFs iii. Communication should have been clearer and stronger to the districts prior to transition to promote clear understanding and ownership. iv. Ideas on how to improve this will be gathered from the districts b. School LLINS – consistency of interventions is required for effectiveness and acceptance of MAPD – when plans change engagement should be improved (see point 3) however changes in these activities can lead to damaged acceptance of MAPD even with good understanding from all partners. c. LLIN use – establishing LLIN use needs to improve using data Q. Do people attach the same or different importance to the nets given through different mechanisms e.g. school, ANC or EPI. i. Need to track these different LLINs linked to their delivery and linked to use. Can do this through LQAS working with SITES, periodic household surveys, health facility exit interviews? MIP: a) IPTP performed well generally – MAPD can learn what it did here for this performance and transition into other program areas b) MIP cases high though IPTP3 high and LLIN ANC high, and reported LLIN use of pregnant women high a. MAPD will investigate this situation and conduct linked appropriate actions c) MAPD will continue to advocate for SP to fully get on the MOH high value commodity list Result 2: Effective malaria diagnosis and treatment activities implemented in support of the National UMRSP a) MAPD has performed well at HF level, need to improve at community level. b) Referral documentation and feedback needs to be ensured including tracking of tools, and referral feedback etc. a. MAPD will work with PMI, SITES, MOH, etc. on provision of referral fomrs c) ICCM a) ICCM is effective in reducing incidence a. MAPD is willing to increase its community presence however key barriers/bottlenecks to this model of implementation need to be effectively solved. b) VHT structure is present and they are willing to work a. MAPD will leverage on this system more in its non-ICCM areas in terms of SBC, line listing for upsurges/deaths etc. b. To motivate VHTs there are some financial and in-kind contributions to consider though there are opportunities to work with them without this. b) Improvement is needed in terms of community data and use at HF, District and Central level a. MAPD will strengthen data collection and use from HFs using district leadership and follow up and data use itself, to ensure improvements USAID UGANDA MAPD MIDTERM LEARNING REVIEW 38 c) Framework for VHT quality improvement is needed – MAPD will develop with MOH and other partners and roll out. d) Commodities for iCCM – a key challenge to iCCM roll out and sustainability. a. MAPD will work with all parties to unblock bottle necks including glove provision b. MAPD will work with districts for HF-iCCM quantifications at HF level c. MAPD will support the understanding of HFs and Districts through all its activities in terms of adopting the new strategy to see VHT work as part of HF work (same catchment) d. MAPD will roll out CCM for under 5s (as permission from MOH just secured verbally last week). e. MAPD will investigate CCM for above 5s if youth data support this in its next year’s plan. Result 3: Build capacity of the National Malaria Control Program (NMCD) and District Health Management Teams (DHMTs) to effectively manage malaria activities and sustain malaria gains a) Policy: Most policies are in place now as supported by MAPD in PY1-3. Still to finalize some – e.g. VHT supervision. Dissemination of these are done sufficiently within MAPD districts b) District Capacity Building: Some gaps in understanding, ownership, use and follow-up c) MAPD will improve planning by making these tools more alive, ensuring district goals, targets and performance management plans are periodically reviewed jointly with action plans set and followed. d) MAPD will use these plans and the national UMRSP in all discussions with the districts e) MAPD will strengthen the districts to follow up how HF and community activities are feeding into their targets. MAPD will help the districts advocate for fulfillment of their malaria budgets to other supporting actors. f) MADP will align with District Planning times, rather than USAID planning cycle. g) MAPD will enact recommendations from its capacity building assessment for building leadership, management etc. and see how these really link to improved malaria care rather than general management issues. • Engagement and Communication needs improvement a) An engagement and communication plan will be developed and staffs capacity assessed and built for representation and engagement. b) Communication to gain consensus and understanding rather than information sharing. c) Actions will be documented well. Timely communications e.g. for changes to plans will be signed by project and receiver • Granting/Sub-Contracting: a. Performance based sub-contracting –MAPD to clarify scope of this activity now that budget/funds realigned. • Private Sector Strategy b. MAPD to develop how to do this beyond the professional associations – seeing opportunities at district level. Proposed interventions will be in work plan MAPD Learning Review 39 Cross-Cutting M+E a) Indicators for tracking community activities e.g. community health clubs, needs developing further with effective tracking systems. b) MAPD to help the districts and national level develop and use District Bulletins c) Engagement with the NMCD will occur so that they understand MAPD district performance better, leading to decision making and supportive action. MAPD will share progress through meetings with the Program Lead (monthly) and by holding quarterly meetings with NMCD wider members, and other stakeholders. d) Improve NMCD use of data and District use –need to improve the ability to analyze, e.g. linkage of indicators, burden info, prevalence info etc. which then lead to program decisions e) Data use at HFs – strengthened. Big gap seen in the ability of HFs to use their data for decision making. MAPD will improve this through capacity building and promoting use and follow up of this from the district. SBC Need to improve and see how MAPD can strengthen this key aspect. a) Need to improve consistency, intensity, reach, and messaging. b) MAPD will improve its M+E strategy for SBC and link this to decision making. Learning a) Strengthen intra-MAPD learning e.g. VHT reporting well in Masaka but not in neighboring Lyantonde Other: MAPD will identify the areas (districts, sub-counties etc.) that are highest need for MAPD to operate in and propose changes (if needed) to PMI. All data will be used for this including new MIS prevalence data. MAPD will continue its targeted approach to implementation directing actions where needed most. National Malaria Control Division (NMCD) NMCD should scale up its role for supporting policy review and development within the context of changing malaria landscape. This will enable partners to have access to frameworks, policies and guidelines that address current trends in malaria prevention and treatment and quality service delivery (KII, National Level Stakeholders, Kampala). The NMCD needs to strengthen the Village Health quality improvement standards and facilitate iCCM for improved malaria prevention, diagnosis and treatment. NMCD should ensure that VHT commodities supply chain is improved, challenges of reporting among VHTs addressed to reduce on misinterpretation and double counting. (KII, NMCD, Global Fund, Kampala). NMCD should scale up its role for supporting policy review and development within the context of changing malaria landscape. This will enable partners to have access to frameworks, policies and guidelines that address current trends in malaria prevention and treatment and quality service delivery (KII, National Level Stakeholders, Kampala). In addition, there is need for NMCD to strengthen the Village Health quality improvement standards and facilitate iCCM for improved malaria prevention, diagnosis and treatment. NMCD should ensure USAID UGANDA MAPD MIDTERM LEARNING REVIEW 40 that VHT commodities supply chain is improved, challenges of reporting among VHTs addressed to reduce on misinterpretation and double counting. (KII, NMCD, Global Fund, Kampala). Private sector malaria reduction are still very weak and will require innovation and intensified action considering the majority of patients first seek care in private sector. Challenges is persist in reporting, adherence to test and treat, affordability of quality RDTs, LLINs and IRS commodities. NMCD and partners have to scale up malaria reduction efforts in the private sector. Scale up of IPTp3 did not translate in reduction of malaria in pregnancy. Other efforts must simultaneously be scaled up to reach the vulnerable group and review continuously the effectiveness of IPTp NMCD should develop tailored strategies for mobile populations case in point was Mityana where the malaria hotspots had many immigrants that were not using mosquito nets and uptaking other malaria interventions. Encourage partners and districts to leverage from other partners and interventions through integration to maximize efforts as was witnessed with Isingiro district and RHITES SW The districts did not have targets and were also not aware of UMRSP goals. NMCD to support MAPD/partners to develop district specific goals and targets in line with the Uganda malaria reduction strategic plan. Challenges persist in net use and malaria prevention practices. NMCD to review of BCC implementation by partners as it is not translating to practice and ownership of malaria control by communities. NMCD to provide guidance to partners on stratification and focus on malaria hotspots to have maximum benefit from investment in malaria. There were interventions that were spreading too thin and having minimal effect. A number of activities were implemented without good monitoring and evaluation such as school net distribution, school drama clubs and as such could not demonstrate results. NMCD to endure that activities implemented by partners have good M& E systems to demonstrate impact. Close collaboration and implementation by NMCD, districts and MAPD is very crucial to achieve the project and UMRSP goals. ADDITIONAL RECOMMENDATIONS FROM CONSULTANTS USAID should reconsider the added value of MAPD in districts where RBF is being implemented and the same indicators are being improved by both interventions. As part of this analysis, it may be useful to evaluate when did it come in which districts, select the MAPD districts and review results before and after RBF, and compare the timing to that of MAPD interventions. This type of review may help USAID in thinking through value-added assessment of USAID interventions when a new intervention is introduced in the same geographic area, and designed to affect the same indicators. MAPD should consider its approach to maintaining gains in case management even as it pivots towards a stronger community approach. MAPD Learning Review 41 MAPD should ensure that NMCD has technical input in the development of activities, deciding frequency and hotspot areas to focus, and monitoring and evaluation of the activities. USAID UGANDA MAPD MIDTERM LEARNING REVIEW 42 ANNEXES ANNEX 1: INCEPTION REPORT ANNEX 2: TOOLS ANNEX 3: LIST OF PARTICIPANTS ANNEX 4: LIST OF DOCUMENTS REVIEWED MAPD Learning Review 43 Annex 1: Inception Report July 7, 2019 This publication was produced at the request of the United States Agency for International Development. It was prepared independently by Rachel Jean-Baptiste, MPH, PHD, Oxford Epidemiology Services LLC; Jarvice Sekajja, PhD. INCEPTION REPORT Learning Assessment of USAID/Uganda Malaria Action Plan for Districts (MAPD) Project USAID UGANDA MAPD MIDTERM LEARNING REVIEW 44 INCEPTION REPORT Learning Assessment of USAID/Uganda Malaria Action Plan for Districts (MAPD) Project July 7, 2019 DISCLAIMER The author’s views expressed in this publication do not necessarily reflect the views of the United States Agency for International Development or the United States Government. MAPD Learning Review i TABLE OF CONTENTS TABLE OF CONTENTS .................................................................................................................... i ACRONYMS........................................................................................................................................ ii 1. INTRODUCTION, BACKGROUND AND CONTEXT ..................................................0 2. PURPOSE OF LEARNING REVIEW .......................................................................................0 LEARNING REVIEW PURPOSE...............................................................................................................1 LEARNING REVIEW QUESTIONS.........................................................................................................1 3. LEARNING REVIEW TEAM......................................................................................................1 EXTERNAL CONSULTANTS ...................................................................................................................1 USAID LEARNERS ......................................................................................................................................2 GOU NMCD LEARNERS ..........................................................................................................................2 4. METHODOLOGY.......................................................................................................................2 Design of the learning assessment .........................................................................................................2 sampling strategy........................................................................................................................................4 Data Analysis................................................................................................................................................4 Ethical Considerations................................................................................................................................5 Limitations of the learning assessment..................................................................................................6 5. PREPARATIONS FOR FIELD WORK....................................................................................6 6. TIMELINE AND DELIVERABLES .............................................................................................6 7. ANNEX 1 – DATA COLLECTION TOOLS ...........Error! Bookmark not defined. 8. ANNEX II – WORKPLAN FOR MAPD LEARNING REVIEWError! Bookmark not defined. USAID UGANDA MAPD MIDTERM LEARNING REVIEW ii ACRONYMS CDFUCommunication for Development Foundation Uganda DHMTDistrict Health Management Team DHODistrict Health Officer DODevelopment Objective FGDFocus group discussion GoUGovernment of Uganda HMISHealth Information Management System iCCMIntegrated community case management IDIInfectious Diseases Institute IPImplementing Partner IPTpIntermittent preventive treatment in pregnancy IRIntermediate Result KIIKey informant interview MAPDMalaria Action Plan for Districts M&EMonitoring and Evaluation MoHMinistry of Health MISMalaria Indicator Survey NGONon-governmental organization NMCDNational Malaria Control Division PIPerformance improvement PMIPresident’s Malaria Initiative PMPPerformance management plan SOWScope of Work SSSSocial and Scientific Systems UMRSPUganda Malaria Reduction Strategic Plan USAIDUnited States Aid for International Development WHOWorld Health Organization USAID UGANDA MAPD MIDTERM LEARNING REVIEW 0 1. INTRODUCTION, BACKGROUND AND CONTEXT The Malaria Action Plan for Districts (MAPD) project is a 5-year contract (2016-2021) funded by the President’s Malaria Initiative (PMI) through the United States Agency for International Development (USAID) and UKAid through the Department for International Development (DFID). The project is implemented by the Malaria Consortium, in partnership with Jhpiego, Banyan Global, Communication for Development Foundation Uganda (CDFU), Deloitte Uganda, Infectious Diseases Institute (IDI) and Infectious Diseases Research Collaboration (IDRC). MAPD aims to improve the health status of the Ugandan population by reducing childhood and maternal morbidity and mortality due to malaria with special focus on women and children. MAPD supports the National Malaria Control Program’s (NMCD) Uganda Malaria Reduction Strategic Plan (UMRSP) 2014-2020. Over the five-year period, MAPD is expected to produce three key results: Result 1: Effective malaria prevention programs implemented in support of the National Malaria Control Strategy. This result focuses specifically on at least 85% of pregnant women receiving three or more doses of IPTp, and 85% of pregnant women and children under 5 years sleeping under an insecticide-treated net (ITN). Result 2: Effective malaria diagnosis and treatment activities implemented in support of the National Malaria Strategy. This result specifically focuses on working with Village Health Teams to implement iCCM in the highly endemic districts in the Central region; improving diagnostic capacity, and improving health provider capacity for the management of simple and severe malaria. Result 3: Capacity of NMCD and DHMTs to manage and sustain efficient malaria activities in focus districts built. This result focuses specifically on strengthening capacity of DHMTs to set and meet capacity improvement targets, manage stock efficiently, and ensure health workers are trained in the diagnosis and treatment of simple and severe malaria. It also focused on demonstrating value for money, meaning that cost-eficiency for delivering malaria services can be demonstrated. To achieve these results, MAPD, led by a Ugandan team, employs a health systems-thinking approach towards implementing results-oriented, field-tested strategies that build on predecessor Stop Malaria Project, also led by Malaria Consortium. MAPD aims to improve the health status of the Ugandan population by reducing childhood and maternal morbidity and mortality due to malaria with special focus on women and children. MAPD supports the National Malaria Control Program’s (NMCD) Uganda Malaria Reduction Strategic Plan (UMRSP) 2014- 2020. The project covers 52 districts in central, mid-western and west Nile regions of Uganda, reaching an estimated population of 13 million Ugandans. MAPD is now in its third year of implementation, and in light of changing scenario of malaria in Uganda, USAID found it necessary to assess the effectiveness of MAPD’s current approach in order to identify major bottlenecks and challenges that may require adaptation in the design, interventions, or management including geography, delivery method or scale. 2. PURPOSE OF LEARNING REVIEW MAPD Learning Review 1 LEARNING REVIEW PURPOSE The purpose of this learning review is for different teams to learn what can be done differently to increase the chances of achievement of anticipated results. The review will assess the extent to which activities of MAPD are on track, and at the same time provide an opportunity for USAID staff to learn how the quality of malaria service are being improved (or not) at different project sites. The findings of this learning review are expected to inform various ways that the PMI team can adjust and improve MAPD’s current approach. Specifically, this assessment will allow PMI to gather evidence that can inform strategies and direction of the project workplan in years 4 and 5, and of PMI’s support to the GOU’s malaria control strategy post-MAPD. LEARNING REVIEW QUESTIONS This assessment has been designed to answer five primary questions: a. Does performance (HMIS and MIS) data suggest a discernible association between MAPD investments and increased availability, and uptake of quality malaria prevention, diagnosis and case management services? b. To what extent do institutional, structural, management and leadership factors (and others identified during the review) influence improvements in service delivery in the MAPD districts? c. Is the geographical coverage and scale of MAPD adequate to significantly influence achievement of the goal to get Uganda to pre-elimination stage by 2020? d. What is the evidence for improved fulfillment (or lack of improvement) of the role of GOU institutions (NMCD and DHMT) in quality improvement of malaria prevention and treatment services? e. How do you compare approaches by MAPD to those of RHITES? 3. LEARNING REVIEW TEAM EXTERNAL CONSULTANTS Dr. Rachel Jean-Baptiste: Team Leader will provide the overall technical direction for the Learning Review. In particular, she will be responsible for 1) providing the team with technical leadership, 2) managing the team’s activities, 3) ensuring that all deliverables are met on time, 4) leading learning discussions, briefings and presentations to USAID. She will provide the overall quality assurance on learning issues, including methods, development of data collection instruments, protocols for data collection, data management and data analysis. She will oversee the training of all engaged in data collection, ensuring high level of reliability and validity of data being collected. She will also ensure that all qualitative and quantitative data are analyzed accordingly to meet the needs of this learning assessment. Dr Jarvice Sekajja: Local Conslutant responsible for assessing the project’s activities and results within the greater Ugandan context. She will assess the project’s activities and Performance Management Plan as they relate to the NMCD Strategic Plans and other related documents, and will examine MAPD’s coordination capacity with all stakeholders. She, too, will conduct key USAID UGANDA MAPD MIDTERM LEARNING REVIEW 2 informant interviews with stakeholders, including the Ministry of Health National Malaria Control Division and other government officials, the USAID and MAPD project management team, district management team, and health facility staff. She will be part of all planning and briefing meetings, data collection, data analysis, development of learning presentations, and writing of the learning assessment report. She will design a comprehensive plan for field data collection, establish linkages and contact points critical to the data collection, as well as advise on coordination, monitoring and documentation during the review. USAID/SITES staff member Kenneth Kasule will analyze MAPD indicators available on Uganda DHIS, and coordinate logistics of site visits. Ms Immaculate Baseka will support note taking. Four research assistants, including three SITES staff members and one external consultant will support focus group discussions and exit interviews at health facilities. USAID LEARNING REVIEW PARTICIPANTS Dr Kassahun Belay, Team Leader, PMI Dr Mame Niang, PMI Advisor Dr Joel Kisubi, PMI, IRS Dr Norbert Mubiru, PMI Logistics Dr Gloria Sebikaari, COR, MAPD Joseph Mwangi, Team Leader, SI Edgar Agaba, SI Advisor Solome Sevume, SI Advisor GOU NMCD LEARNING REVIEW PARTICIPANT Dr. Damian Rutazana 4. METHODOLOGY DESIGN OF THE LEARNING ASSESSMENT The design of this learning assessment is cross-sectional, with a purposive sampling approach to selecting key informants, districts and sites, and multiple methods of data collection. The Team’s approach is flexible, inclusive and comprehensive. Data collection will include the following: Review of Project documents, including MAPD quarterly and annual reports, work plans, Performance Management Plans, MAPD technical proposal, NMCD Strategic Plan, PMI MOP 2016-2019, among others. MAPD reports and planning documents have been provided. As additional relevant documents are identified, the Learning Team will request them and they will be reviewed. Review of these documents will allow the Learning Team to obtain a clear picture of what MAPD has done, and some indication into strengths and weaknesses of the project. MAPD Learning Review 3 Key Information Interviews (KII) using semi-structured interview guides: These will be tailored to the group of individuals being interviewed to elicit information to: a) validate and, where possible, verify project approaches, interventions, and achievements and their current technical and strategic appropriateness; b) secure opinions and perceptions of project implementation effectiveness and issues, and identifying gaps in project activities; c) obtain first￾hand reports on training received, data systems changed, and overall capacity building in data collection, analysis, validation, reporting and use; d) determine how stakeholders and beneficiaries interact with the project, regarding issues of leadership, GOU ownership, coordination, and collaboration; and, e) determine how the project has influenced malaria pre-elimination goals of the GOU. Focus Group Discussions (FDGs) also using semi-structured guides will focus on aspects of malaria prevention, among beneficiaries, including barriers and facilitators of LLIN use. Among VHT, FGDs will focus on challenges and opportunities for iCCM. Analysis of DHIS Data As described above, we will analyze indicators of each of the Results (IPTp 2, 3; MIP; LLIN distribution; RDT; and malaria treatment) In addition to document reviews, other data collection methods will include Key Informant Interviews (KIIs) at national, district and health facilities, Exit Interviews at health facilities, and focus group discussions among community beneficiaries and Village Health Teams (VHTs). Data collection instruments are included in the Annex. Table 1 summarizes these methods and targeted individuals. Table 1: Data Collection Plan for Learning Method Targeted stakeholders/ Respondents No. Key Informants At National Level • NMCD • Global Fund • DFID • MAPD, including leadership within Malaria Consortium • WHO • JMS • PS (if possible) • USAID PMI At District Level • DHO • DHE • District Leadership (CAO. LCV) • District Malaria focal person • Biostatistician At Health Facility Level • Nursing officer (or Medical Officer) • Community based services department • Entomologist • HMIS Focal person RHITES • COPs- RHITES 18 Focus Groups Beneficiaries- pregnant women, women with children under 5 (one FGD per district) VHTS – integrated community management of Malaria- 8-10 VHTs (one FGD per district) 12 USAID UGANDA MAPD MIDTERM LEARNING REVIEW 4 Exit Interview Beneficiaries at Health Facilities: Youth, men and women (pregnant women, women with children under 5 (lactating), pregnant women) who show up with chief complaint of “fever” 10 Analysis DHIS data IPTp 2, IPTp 3, incidence of Malaria in Pregnancy, LLIN continuous distribution through ANC clinic, use of RDT on fever cases, malaria treatment on RDT positive cases, malaria treatment on RDT negative cases; These analyses will be conducted for both MAPD and RHITES projects Learning From 3PM to 5PM of each day, external consultants will facilitate a discussion among the learning participants to flesh out what was learned and its potential impact, guided by the following set of questions: • What was learnt? • What are the implications for the workplan? • Any need for modifications? • What modifications in program design and implementation are required? • Reflection on the questions being asked and ways of improving on the questions for better responses. At the district level, the review team will conduct district learning meetings and provide all learning participants an opportunity to discuss misrepresentations or other concerns about the findings. SAMPLING STRATEGY Selection Criteria for districts: Six districts were selected by USAID to represent the MAPD proximity. They considered what is feasible in one week that also provides a good overview of project implementation areas. The team will visit districts in Central and Western Uganda, but will not be able to reach West Nile during this visit. Selection Criteria for sites: In each district, the large referral hospital was selected and one other Health Center 3 or HC 4 that is proximal to it to minimize travel time. When there were multiple health facilities with relatively equal distances, the facility with the largest malaria prevalence was selected. Selection criteria for Key Informant Interviews, exit interviews, and FGDs: To conduct the qualitative interviews of key informants at the central level, district and facility levels, , and exit interviews, sampling will be purposive and snowball. The list of key informants is generated by USAID. If, during the course of our interaction, we learn of an additional person we should interview, their name will be added to the list. During the hours that the team is at the health facility, team members will approach any patient leaving the clinic for an exit interview, selecting only those who were at the clinic due to fever for interviews. DATA ANALYSIS The data will be analyzed on the basis of the five learning questions for the assessment. FGDs and KIIs will be analyzed qualitatively using a Qualitative Data Analysis Matrix. The team will analyze the emergence of opinions, perceptions, and issues. The data will be synthesized to determine recurrent themes and issues from the collected data, as well as from the daily learning MAPD Learning Review 5 reviews. Where appropriate these data will be presented in tables. Time series trend lines will be created from the DHIS data for the specific indicators of interest. Frequencies and averages will be produced from the exit interviews, disaggregated by gender. Table 3: Overview of Data Analysis Evaluation Question Data Analysis Method 1. Descriptive characteristics of respondents in evaluation data collection Quantitative descriptive data: -mean age -% female -% per district -%Stakeholder: USG, IP, GoU Central, GoU district, GoU health facility 2. Does performance (HMIS and MIS) data suggest a discernible association between MAPD investments and increased availability, and uptake of quality malaria prevention, diagnosis and case management services? Trend line of IPTp2, 3, LLIN distribution at ANC clinics, RDT, and malaria treatment from 2014 to 2019 July for MAPD districts, and all RHITES districts 3. To what extent do institutional, structural, management and leadership factors (and others identified during the review) influence improvements in service delivery in the MAPD districts? Qualitative summary of emerging themes, distilled and validated through daily learning discussions Triangulation between various data sources 4. Is the geographical coverage and scale of MAPD adequate to significantly influence achievement of the goal to get Uganda to pre-elimination stage by 2020? Comparison of current geographic scale of MAPD districts with Uganda MIS 2019 data on malaria prevalence Analysis of similarities and deviations Summary of qualitative discussions around geographic coverage from national and district stakeholders, noting any significant differences in thought process. 5. What is the evidence for improved fulfillment (or lack of improvement) of the role of GOU institutions (NMCD and DHMT) in quality improvement of malaria prevention and treatment services? Qualitative summary of stakeholder perceptions of GOU Engagement and of document review; Quantitative analysis of data use (who uses the data, what data are they using, where is data use happening, when does it happen, how does it happen, and for what reason is data used)- from districts and health facilities; 6. How do you compare approaches by MAPD to those of RHITES? Qualitative summary highlights of key differences in approaches of integrated RHITES projects vs malaria-only MAPD project The final report will list each learning question followed by Findings, Analysis, Conclusions, and Recommendations section. The end summary will focus on priority issues for USAID to address and major lessons learned. ETHICAL CONSIDERATIONS USAID UGANDA MAPD MIDTERM LEARNING REVIEW 6 The Learning Team will implement a policy of informed consent for all key informant interviews, exit interviews, and focus group discussions (see Annex 3 for an example), and all interviews will be done on a voluntary basis. Interviewees will be given the option to opt-out of particular questions or the whole interview, if at any time they believe a response would contain sensitive information. The information provided as part of these interviews and discussions will not be linked to any specific person in the Final Report and all information provided will be kept confidential and used for planning purposes only. Only general identifying information (organization, geographical unit, gender, and age if reported voluntarily) will be utilized. Any information that could be directly linked to an individual will not be used. Only members of the Learning Team will have access to the transcripts and raw data. The Final Report will be a synthesis of the Team’s analysis drawn from interviews from numerous respondents. Any included quotes to highlight particular issues will not include names. We will not collect data from minors, nor will we collect personally identifiable information directly from patient records or anywhere else, though health facility registers may be reviewed. LIMITATIONS OF THE LEARNING ASSESSMENT Selection only of districts and health facilities that are geographically accessible is a limitation to understanding the MAPD project overall. This is due to limited resources and short time frame, and as such, the data collected will not be generalizable to all of MAPD. Including USAID, MAPD, and MOH central staff in discussions may limit what is said at district and health facilities. However the opportunity for USAID, MAPD and MOH staff to learn from whatever is visible outweighs the potential bias. 5. PREPARATIONS FOR FIELD WORK The assessment will be carried out by the Learning Team in cooperation with PMI, MAPD, and NMCD teams. To ensure quality of data collection, the Learning Team leader will establish clear guidelines for data collection, specifically for how to conduct interviews. The Local Consultant will brief team members on the KII and FGD guides, and processes for their analysis. SITES is organizing all logistical support, including setting up appointments with district and health facility contacts for the six districts to be visited. Accordingly, the Learning Assessment will include the following steps: 1) Finalization of data collection tools; 2) Formation of two data collection teams who will conduct visits to three districts each; 3) Review and lite training on tools; 4) Conducting the data collection with quality control checks interspersed; 5) Facilitating learning of PMI, NMCD and MAPD teams 6) Data entry and analysis; and, 7) Report writing. 6. TIMELINE AND DELIVERABLES Date Tasks and Deliverables 6/30/2019 Team Lead arrives in Uganda 7/1/2019 In brief with USAID/Uganda MAPD Learning Review 7 7/1/2019 Finalize tools 7/2-7/4 Conduct KIIs in Kampala, conduct learning reviews with USAID staff 7/5/2019 Presentation of Inception Report and Preliminary Learnings to USAID/Uganda 7/5/2019 Write and submit inception report to USAID/Uganda 7/7-7/12/2019 Data collection in 6 districts, 12 sites 7/12/2019 Travel back to Kampala 7/13-7/16/2019 Summarize learning; prepare findings for USAID/Uganda 7/17/2019 Present findings to USAID/Uganda 7/18-7/30/2019 Submit draft report 8/15/2019 Submit final report USAID UGANDA MAPD MIDTERM LEARNING REVIEW 8 Annex 2: Tools July 16, 2019 MAPD KII – KAMPALA INFORMED CONSENT Thank you for giving us the time to speak with you today. My name is _____________________ As you know, we are here to as part of a team that is conducting a Learning assessment for MAPD and malaria pre￾elimination in Uganda. The purpose of this Learning assessment is to gather evidence of MAPD’s achievement to date in your district, and identify major roadblocks and challenges that require attention of all stakeholders as we move towards malaria pre￾elimination. Our role will be to provide recommendations to the USG to plan for improvements in their current strategy. The purpose of our discussion is to gain a better understanding of the approaches used in MAPD for Malaria control. Using the learning assessment questions as a guide, we will be asking you about 1) performance of malaria and HMIS reporting, 2) leadership factors and 3) engagement of the GOU in malaria pre-elimination. If there are staff members who are particularly suited for specific questions, we would appreciate the opportunity to include them in the interview as part of the group or separately. All of the answers you provide will remain confidential, and will be summarized and included in our report. No information shared will be connected to an individual without your consent. Your participation is completely voluntary. You can choose to stop the interview at any time, or you may refuse to answer any questions. There will not be any negative consequences whatsoever. Do you have any questions before we begin? Do we have your consent to conduct the interview? [ ] Yes [ ] No Gender: #M #F 1 1. Let’s talk about the MAPD project. We are mid-term. Can you help us review which of these activities you have completed? MAPD Learning Review 9 2. What are some challenges you have faced? 3. To what extent do you think results in IPTp and case management are due to MAPD? 4. What are some structural, managerial, and leadership factors that have impacted your work so far? 5. Now that you’ve had a chance to implement, are these the right intervention mix for the districts where you are? 6. When you think of reducing malaria burden in Uganda, do you think you are implementing in the right districts? 7. How would you describe your relationship with the NMCD? 8. How would you describe your relationship with districts leadership? District politicians? 9. What are some key learnings you have done through this learning assessment? USAID UGANDA MAPD MIDTERM LEARNING REVIEW 10 Key Informants Questionnaire- MAPD Learning USAID Uganda DISTRICTS Question/Information Required 00 INFORMED CONSENT Thank you for giving us the time to speak with you today. My name is _____________________ As you know, we are here to as part of a team that is conducting a Learning assessment for MAPD and malaria pre-elimination in Uganda. The purpose of this Learning assessment is to gather evidence of MAPD’s achievement to date in your district, and identify major roadblocks and challenges that require attention of all stakeholders as we move towards malaria pre-elimination. Our role will be to provide recommendations to the USG to plan for improvements in their current strategy. The purpose of our discussion is to gain a better understanding of the approaches used in MAPD for Malaria control. Using the learning assessment questions as a guide, we will be asking you about 1) performance of malaria and HMIS reporting, 2) leadership factors and 3) engagement of the GOU in malaria pre-elimination. If there are staff members who are particularly suited for specific questions, we would appreciate the opportunity to include them in the interview as part of the group or separately. All of the answers you provide will remain confidential, and will be summarized and included in our report. No information shared will be connected to an individual without your consent. Your participation is completely voluntary. You can choose to stop the interview at any time, or you may refuse to answer any questions. There will not be any negative consequences whatsoever. Do you have any questions before we begin? Do we have your consent to conduct the interview? [ ] Yes [ ] No Date: ___________________________ Organization __________________ Gender: #M #F Designation/Job Titles: _______________________________________ 01 Let’s start the Interview by talking about the MAPD Project. Has your DISTRICT interacted with MAPD? Yes No If YES, Please provide details of the type of interaction: __________________________________________________________________________ __________________________________________________________________________ __________________________________________________________________________ __________________________________________________________________________ ____ 1a What’s been working really well with MAPD Project? -please elaborate on factors that have helped __________________________________________________________________________ __________________________________________________________________________ __________________________________________________________________________ _______________________________________________________________________ 1b What has not been working as well as you would have hoped? MAPD Learning Review 11 __________________________________________________________________________ __________________________________________________________________________ __________________________________________________________________________ __________________________________________________________________________ __________________________________________________________________________ _____ -what factors have contributed to this? __________________________________________________________________________ __________________________________________________________________________ __________________________________________________________________________ __________________________________________________________________________ ____ 2 Let’s talk about malaria in your district. What is the prevalence of Malaria in your here? ___________________ Did you do a needs analysis for malaria? What were the key findings? __________________________________________________________________________ __________________________________________________________________________ __ -Does the district have a malaria prevalence goal? If yes, what is it? [ ] Yes [ ] No __________________________________________________________________________ __________________________________________________________________________ ______________________________________________________________ What approach have you used to address those key findings? __________________________________________________________________________ ___________________________________________________________________what has been the role of MAPD in supporting you in this process? ____________________________________________________________________ 03 Do you have a malaria workplan? Can we see it? (can we have a copy of it)? [ ] Yes [ ] No Did you have a workplan for malaria before the district started to work with MAPD? __________________________________________________________________________ __________________________________________________________________________ _ 04 As you know, MAPD has three objectives: 1) to build capacity at the district and facility level; 2) to improve malaria case management, and 3) to support malaria prevention. To what extent have MAPD investments contributed to: 1) Capacity building at the district level 2) Malaria case management 3) Malaria prevention USAID UGANDA MAPD MIDTERM LEARNING REVIEW 12 Please share some examples to support your views __________________________________________________________________________ __________________________________________________________________________ __________________________________________________________________________ ___ Here is MAPD data for selected indicators. What are your thoughts about this data? __________________________________________________________________________ __________________________________________________________________________ __________________________________________________________________________ ___ Do you use this data? (If yes, please share examples of how you have used this data) Can you tell us about policy that the district has adapted and technical approaches that the district have put in place to help the district reduce its malaria burden? _________________________________________________________________________________ _________________________________________________________________________________ _________________________________________________________________________________ _________________________________________________________________________________ __________________________________________________________________ 05 How do you engage with MAPD? –is there joint planning, joint implementation? Joint Review? Joint reporting? Joint use of evidence? Etc… 06 When you think about reducing malaria burden for your district, are the interventions promoted by MAPD the right ones for your district? _________________________________________________________________________________ _________________________________________________________________________________ _________________________________________________________________________________ _____ 07 In you view, what are three best ways that MAPD could better support your districts to reduce the burden of malaria in your district? __________________________________________________________________________ __________________________________________________________________________ __________________________________________________________________________ ___ What would you say is the minimum package of interventions that could help your district to reduce its malaria burden? __________________________________________________________________________ __________________________________________________________________________ __________________________________________________________________________ ___ MAPD Learning Review 13 08 How do you engage the schools in malaria prevention activities? __________________________________________________________________________ __________________________________________________________________________ __________________________________________________________________________ __________________________________________________________________________ ____ 09 According to the Uganda malaria national strategic plan 2014-2020, Malaria elimination requires a multi-sectoral approach. To what extent has the district engaged other sectors in malaria work? __________________________________________________________________________ __________________________________________________________________________ __________________________________________________________________________ ___ USAID UGANDA MAPD MIDTERM LEARNING REVIEW 14 Key Informants Questionnaire- National Level MAPD Learning USAID Uganda WHO Uganda Question/Information Required 00 INFORMED CONSENT Thank you for giving us the time to speak with you today. My name is _____________________ As you know, we are here to as part of a team that is conducting a Learning assessment for malaria pre-elimination in Uganda, with a focus on the PMI-funded Malaria Action Plan for Districts (MAPD) project and how to improve it moving forward. The purpose of this Learning assessment is to gather evidence of achievement and identify major roadblocks and challenges that require attention of all stakeholders as we move towards malaria pre-elimination. Our role will be to provide recommendations to the USG to plan for improvements in their current strategy. The purpose of our discussion is to gain a better understanding of the approaches used in Malaria control. Using the learning assessment questions as a guide, we will be asking you about 1) performance of HMIS reporting, 2) leadership factors and 3) engagement of the GOU in malaria pre-elimination. If there are staff members who are particularly suited for specific questions, we would appreciate the opportunity to include them in the interview as part of the group or separately. All of the answers you provide will remain confidential, and will be summarized and included in our report. No information shared will be connected to an individual without your consent. Your participation is completely voluntary. You can choose to stop the interview at any time, or you may refuse to answer any questions. There will not be any negative consequences whatsoever. Do you have any questions before we begin? Do we have your consent to conduct the interview? [ ] Yes [ ] No Date: ___________________________ Organization __________________ Name of respondent :______________________________ Age _____ Gender: M F Designation/Job Title: _ _____________________________________ 1 Let’s start the interview by understanding a bit about the role of WHO in supporting Uganda towards pre-elimination. 2 Are you familiar with the Malaria Action Plan for Districts (MAPD) project, funded by PMI and DFID? MAPD Learning Review 15 3 To what extent have MAPD investments contributed to availability and uptake of malaria prevention, diagnosis and treatment and case management services in Uganda? Please share some examples to support your views ________________________________________________________________________ ________________________________________________________________________ ____ 4 Here is current malaria prevalence map (from MIS 2019). What are your thoughts about this data? Are the current interventions (and their spread) the right ones? ________________________________________________________________________ ________________________________________________________________________ ________________________________________________________________________ ______ 5 Let’s talk about pre-elimination of malaria. Can you tell us about policy and technical approaches that have been put in place to help Uganda reach pre-elimination? See above__________________________________________________________________________ ________________________________________________________________________________ ________________________________________________________________________________ ____ 6 What is the minimum package of intervention that should be offered for high, medium, low burden districts? (stratification) What are some institutional, structural, management and leadership factors that are influencing improvements in service delivery for malaria (particularly in the districts where MAPD operates? 7 What do you see as emerging barriers to malaria pre-elimination in Uganda? ________________________________________________________________________ ________________________________________________________________________ ________________________________________________________________________ ______ 8 In you view, what are three best ways that PMI could support pre-elimination efforts? 9 Let’s discuss iCCM – we know it is one of the strategies to be used to reduce child mortality, but it has not taken off as quickly as one would like. What are the issues? ________________________________________________________________________ ________________________________________________________________________ ________________________________________________________________________ ________________________________________________________________________ ________ 10 In what ways can the GOU’s role in malaria pre-elimination be improved? ________________________________________________________________________ ________________________________________________________________________ ________________________________________________________________________ USAID UGANDA MAPD MIDTERM LEARNING REVIEW 16 ________________________________________________________________________ ________ 11 As we end the interview, what has given you the most hope about malaria since 2016? 12 On a scale of 1(low) to 10(high), how optimistic are you about Uganda moving towards malaria pre-elimination in the next five years? Thank you so much! Do you have any questions for us? MAPD Learning Review 17 Key Informants Questionnaire- National Level MAPD Learning USAID Uganda (Used with GF, NMCD) Question/Information Required 00 INFORMED CONSENT Thank you for giving us the time to speak with you today. My name is _____________________ As you know, we are here to as part of a team that is conducting a Learning assessment for malaria pre-elimination in Uganda. The purpose of this Learning assessment is to gather evidence of achievement and identify major roadblocks and challenges that require attention of all stakeholders as we move towards malaria pre-elimination. Our role will be to provide recommendations to the USG to plan for improvements in their current strategy. The purpose of our discussion is to gain a better understanding of the approaches used in Malaria control. Using the learning assessment questions as a guide, we will be asking you about 1) performance of HMIS reporting, 2) leadership factors and 3) engagement of the GOU in malaria pre-elimination. If there are staff members who are particularly suited for specific questions, we would appreciate the opportunity to include them in the interview as part of the group or separately. All of the answers you provide will remain confidential, and will be summarized and included in our report. No information shared will be connected to an individual without your consent. Your participation is completely voluntary. You can choose to stop the interview at any time, or you may refuse to answer any questions. There will not be any negative consequences whatsoever. Do you have any questions before we begin? Do we have your consent to conduct the interview? [ ] Yes [ ] No Date: ___________________________ Organization __________________ Name of respondent :___________________________________________ Age _____ Gender: M F Designation/Job Title: _______________________________________ 1.0 Let’s start the Interview by talking about the MAPD Project. Has your organization interacted with MAPD? Yes No If YES, Please provide details of the type of interaction: ________________________________________________________________________ ________________________________________________________________________ ________________________________________________________________________ ________________________________________________________________________ ________ [ [ ] GoU National [ [ ] USG/PMI [[ [ ] MAPD Staff USAID UGANDA MAPD MIDTERM LEARNING REVIEW 18 1.a What’s been working really well with this project? (Probe- something that makes you feel like ‘wow- we are really contributing to public health in Uganda’) -please elaborate on factors that have helped 1.b What has not been working as well as you would have hoped? -what factors have contributed to this? 1.c Which areas should be improved upon so that the country moves toward malaria pre-elimination? ________________________________________________________________________ ________________________________________________________________________ ________________________________________________________________________ ______ 2 2a. 3. To what extent have MAPD investments contributed to availability and uptake of malaria prevention, diagnosis and treatment and case management services in Uganda? Please share some examples to support your views ________________________________________________________________________ ________________________________________________________________________ ____ Here is current malaria prevalence map (from MIS 2019). What are your thoughts about this data? Are the MAPD interventions the right ones? ________________________________________________________________________ ________________________________________________________________________ ________________________________________________________________________ _____ Let’s talk about pre-elimination of malaria. Can you tell us about policy and technical approaches that they have put in place to help Uganda reach pre-elimination? ________________________________________________________________________________ ________________________________________________________________________________ ________________________________________________________________________________ ________________________________________________________________________________ ________ 3a. In you view, what are three best ways that PMI could support pre-elimination efforts? ________________________________________________________________________ ________________________________________________________________________ ________________________________________________________________________ ________________________________________________________________________ ________ 3b. How can these efforts be integrated in the current MAPD program? What would it take? ________________________________________________________________________ ________________________________________________________________________ ____ 4 Let’s discuss iCCM – we know it is one of the strategies to be used to reduce child mortality, but it has not taken off as quickly as one would like. What are the issues? (prompt about implementation, data and reporting…) MAPD Learning Review 19 __to what extent has the iCCM approach contributed to the reduction in child mortality (what approaches have they used to make it work? Why does it not work) ________________________________________________________________________ ________________________________________________________________________ ________________________________________________________________________ ________________________________________________________________________ ________ 5. To what extent does MAPD effectively influence the use of ACT for treating confirmed malaria cases ________________________________________________________________________ ________________________________________________________________________ ________________________________________________________________________ ________________________________________________________________________ ________________________________________________________________________ __________ 5a. In what ways had MAPD contribute to malaria pre-elimination in Uganda? ________________________________________________________________________ ________________________________________________________________________ ________________________________________________________________________ ________________________________________________________________________ ________________________________________________________________________ __________ 6 What approaches are being used by the GOU to reach pre-elimination? How effective are those approaches? And what should be done differently? ________________________________________________________________________ ________________________________________________________________________ ________________________________________________________________________ ______ 6a. What do you see as emerging barriers to malaria pre-elimination in Uganda? ________________________________________________________________________ ________________________________________________________________________ ________________________________________________________________________ ______ 6b How has the engagement with schools been going? Do you see added advantages? ________________________________________________________________________ ________________________________________________________________________ ________________________________________________________________________ ________________________________________________________________________ ________________________________________________________________________ ________________________________________________________________________ ___________________________________________________ 6c In what ways can the GOU’s role in malaria pre-elimination be improved? ________________________________________________________________________ ________________________________________________________________________ ________________________________________________________________________ ________________________________________________________________________ ________ 6d What is the minimum package of intervention that should be offered for high, medium, low burden districts? (stratification) High: _____________________________________________________________ USAID UGANDA MAPD MIDTERM LEARNING REVIEW 20 Med: _____________________________________________________________ Low: ______________________________________________________________ MAPD Learning Review 21 Key Informants Questionnaire- National Level MAPD Learning USAID Uganda JMS Question/Information Required 00 INFORMED CONSENT Thank you for giving us the time to speak with you today. My name is _____________________ As you know, we are here to as part of a team that is conducting a Learning assessment for malaria pre-elimination in Uganda, with focus on the Malaria Action Plan for Districts (MAPD) project funded by the President’s Malaria Initiative (PMI). The purpose of this Learning Assessment is to gather evidence of achievement and identify major roadblocks and challenges that require attention of all stakeholders as we move towards malaria pre-elimination. Our role will be to provide recommendations to the USG to plan for improvements in their current strategy. The purpose of our discussion is to gain a better understanding of the approaches used in Malaria control. Using the learning assessment questions as a guide, we will be asking you about 1) performance of HMIS reporting, 2) leadership factors and 3) engagement of the GOU in malaria pre-elimination. If there are staff members who are particularly suited for specific questions, we would appreciate the opportunity to include them in the interview as part of the group or separately. All of the answers you provide will remain confidential, and will be summarized and included in our report. No information shared will be connected to an individual without your consent. Your participation is completely voluntary. You can choose to stop the interview at any time, or you may refuse to answer any questions. There will not be any negative consequences whatsoever. Do you have any questions before we begin? Do we have your consent to conduct the interview? [ ] Yes [ ] No Date: ___________________________ Organization __________________ Name of respondent: ___________________________________________ Age _____ Gender: M F Designation/Job Title: _______________________________________ 1 2 As you know, supply chain plays a major role in the success of PMI efforts. Its current flagship project relies on availability of RDTs and ACTs in order to improve case management. MAPD has been operational since 2016. What’s been working really well with the PMI supply chain specifically, between 2016 and now? -please elaborate on factors that have helped 3 What has not been working as well as you would have hoped? -what factors have contributed to this? 4 Which areas should be improved upon so that the country moves toward malaria pre-elimination? _________________________________________________________________________________ _________________________________________________________________________________ _________________________________________________________________________________ _____________________________________________________ USAID UGANDA MAPD MIDTERM LEARNING REVIEW 22 5 6 To what extent have MAPD investments contributed to improving stock management districts and at health facilities? Please share some examples to support your views Probe: lead time, buffer stock, expiry management, pilferage, stock out, etc… _________________________________________________________________________________ _______________________________________________________________________ Here is current malaria prevalence map (from MIS 2019). What are your thoughts about this data? Are the current amount of commodities being distributed in accordance with the needs as seen here? _________________________________________________________________________________ ______________________________________________________________________ 7 In you view, what are three best ways that PMI could support pre-elimination efforts? Recent pmi nets that came in were quarantined, not distributes; may have contributed to recent spikes in malaria _________________________________________________________________________________ _________________________________________________________________________________ _________________________________________________________________ 8 How can these efforts be integrated in the current MAPD program? What would it take? _________________________________________________________________________________ ___________________________________________________________________ 9 Let’s discuss the new ERP system. What are the issues around its implementation? PILFERAGE-tell us about it __to what extent do you think it will help, and what will it help with? -will it help to prevent pilferage? (how much pilferage is there?) _________________________________________________________________________________ _______________________________________________________________________ 10 Tell us about future plans to improve supply chain for malaria _________________________________________________________________________________ _________________________________________________________________________________ ____ 11 What do you see as emerging barriers to malaria pre-elimination in Uganda? _________________________________________________________________________________ _________________________________________________________________________________ __________________________________________________________________ 12 How has distributions of LLINs through schools been going? Do you see added advantages? _________________________________________________________________________________ _________________________________________________________________________________ ____ 13 In what ways can the JMS’s role in malaria pre-elimination be improved? MAPD Learning Review 23 _________________________________________________________________________________ _________________________________________________________________________________ ____ 14 On scale of 1(low) to 10(high), how optimistic are you about Uganda moving to pre-elimination within the next five years? Facility Number: Interviewer Code: Provider SERIAL Number: [FROM STAFF LISTING FORM] Provider Sex: (1=MALE; 2=FEMALE) Provider Status: (1=Assigned; 2=Seconded) READ THE FOLLOWING CONSENT FORM Good day! My name is . We are here on behalf of [USAID]] conducting a study to assist the government in knowing more about health services in [COUNTRY].` Now I will read a statement explaining the study. You were selected to participate in this study. We will be asking you several questions about the types of services that you received today. The information you provide us may be used by the [MOH], other organizations or researchers, for planning service improvements or further studies of services. Neither your name nor that of any other health worker respondents participating in this study will be included in the dataset or in any report; however, there is a small chance that any of the respondents may be identified later. Still, we are asking for your help to ensure that the information we collect is accurate. You may refuse to answer any question or choose to stop the interview at any time. However, we hope you will collaborate with the study. Do you have any questions about the study? Do I have your agreement to proceed? SIGNATURE OF INTERVIEWER INDICATES INFORMED CONSENT WAS PROVIDED. 101 May I begin the interview now? YES. . . . . . . . . . . . . . . . NO. . . . . . . . . . . . . . . . 1 2 END 102 I would like to ask you some questions about your educational Background YEARS. . . . . How many years of education have you completed in total, starting from your primary, secondary and further education? 103 What is your current occupational category or qualification? GENERALIST MEDICAL DOCTOR. . . . . . . . . . . . . . . . . . . . 01 For example, are you a registered nurse, or generalist SPECIALIST MEDICAL DOCTOR . . . . . . . . . . . . . . . . . . . . 02 medical doctor or a specialist medical doctor? NON-PHYSICIAN CLINICIAN . . . . . . . . . . . . . . . . . . . . . . . . 03 USAID UGANDA MAPD MIDTERM LEARNING REVIEW 24 NURSING PROFESSIONAL . . . . . . . . . . . . . . . . . . . . . . 05 ASSOCIATE DEGREE NURSE. . . . . . . . . . . . . . . 06 MIDWIFERY PROFESSIONAL. . . . . . . . . . . . . . . . . . 07 [list will be country specific - must be ASSOCIATE DEGREE MIDWIFE. . . . . . . . . . . . . . . . 08 extensive, with no need for "other"] ENROLLED NURSE / ENROLLED MIDWIFE . . . . . . . . . . . 09 LABORATORY SCIENTIST. . . . . . . . . . . . . . . . . . . . . . . . . . 13 LABORATORY TECHNOLOGIST. . . . . . . . . . . . . . . . . . . . . 14 LABORATORY TECHNICIAN/ASSISTANT. . . . . . . . . . . . . 15 NO TECHNICAL QUALIFICATION/NURSE AIDE. . . . . . . . 95 OTHER 96 104 What year did you graduate (or complete) with this qualification? YEAR IF NO TECHNICAL QUALIFICATION (103=95), ASK: What year did you complete any basic training for your current occupational category? 105 In what year did you start working in this facility? YEAR 108 Are you a manager or in-charge for any clinical services? 1 YES NO. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1. GENERAL TRAINING / MALARIA / NON-COMMUNICABLE DISEASES 200 First I want to ask you about some general training courses. Have you received any in-service training (i.e., since you started working) or any training updates in any of the following topics [READ TOPIC] YES, YES, NO WITHIN OVER IN￾SERVICE MAPD Learning Review 25 IF YES, ASK: Was the training or training update within the past 24 months PAST 24 MONTHS TRAINING OR or more than 24 months ago? 24 MONTHS AGO UPDATES 01 Standard precautions, including hand hygiene, cleaning and disinfection, 1 2 3 waste management, needle stick and sharp injury prevention, or safe injection practices? 02 Any specific training related to injection safety practices? 1 2 3 03 Health Management Information Systems (HMIS) or reporting requirements for any service? 1 2 3 I will now ask you a few questions about services you personally provide in your current position in this facility and any in-training or training updates you may have received related to that service. Please remember we are talking about services you provide in your current position in this facility. 202 In your current position, and as a part of your work for this facility, do you YES. . . . . . . . . . . . . . . . 1 personally provide any services that are designed to be NO. . . . . . . . . . . . . . . . 2 youth or adolescent friendly? i.e., designed with the specific aim to encourage youth or adolescent utilization? 203 Have you received any in-service training or training updates on topics YES, WITHIN PAST 24 MONTHS. . . . . . . 1 specific to youth or adolescent friendly services? YES, OVER 24 MONTHS AGO. . . . . . . 2 IF YES, was the training update within the past 24 months or more than 24 months ago? NO TRAINING OR UPDATES. . . . . . . . . . 3 MALARIA 204 In your current position, and as a part of your work for this facility, do you YES. . . . . . . . . . . . . . . 1 Personally diagnose and/or treat malaria? NO. . . . . . . . . . . . . . . 2 205 Have you received any in-service training or training updates on topics YES. . . . . . . . . . . . . . . 1 207 related to diagnosis and/or treatment of malaria? NO. . . . . . . . . . . . . . . 2 206 Have you received any in-service training or training updates in any of the following topics [READ TOPIC] YES, YES, NO WITHIN OVER IN-SERVICE IF YES: Was the training or training update within the past 24 months or more than PAST 24 MONTHS TRAINING OR 24 months ago? 24 MONTHS AGO UPDATES 01 DIAGNOSING MALARIA IN ADULTS 1 2 3 USAID UGANDA MAPD MIDTERM LEARNING REVIEW 26 02 DIAGNOSING MALARIA IN CHILDREN 1 2 3 03 HOW TO PERFORM MALARIA RAPID DIAGNOSTIC TEST 1 2 3 04 CASE MANAGEMENT / TREATMENT OF MALARIA IN ADULTS 1 2 3 05 CASE MANAGEMENT / TREATMENT OF MALARIA DURING PREGNANCY 1 2 3 06 INTERMITTENT PREVENTIVE TREATMENT OF MALARIA IN PREGNANCY 1 2 3 07 CASE MANAGEMENT / TREATMENT OF MALARIA IN CHILDREN U5 1 2 3 MAPD Learning Review 27 2. MALARIA PREVENTION SERVICES 300 In your current position, and as a part of your work for this facility, do you YES. . . . . . . . . . . . . . . . 1 personally provide LLINs to pregnant women? NO. . . . . . . . . . . . . . . . 2 301 In your current position, and as a part of your work for this facility, do you YES. . . . . . . . . . . . . . . . 1 personally provide any IPTp to pregnant women? NO. . . . . . . . . . . . . . . . 2 If Yes, what is your approach to IPTp? _________________ _________________ _________________ _________________ _________________ ________________ 302 In your current position, and as a part of your work for this facility, are you YES. . . . . . . . . . . . . . . . 1 personally engaged in stock management of malaria commodities? NO. . . . . . . . . . . . . . . . 2 When was the last time that this health facility experienced a stockout for malaria commodities? What was stocked out? What was the effect of the stockout on your patients? What approaches have you employed to prevent future stockouts? _________________ _________________ _________________ _________________ _________________ _________________ _________________ _________________ _________________ _________________ 303 How do you work with Village Health Teams to implement integrated community case management of malaria? _________________ _________________ _________________ _________________ _________________ _________________ __________ 400 304 Tell us about your malaria data. How do you use it? __________________________ __________________________ __________________________ __________________________ __________________________ __________________________ __________________________ USAID UGANDA MAPD MIDTERM LEARNING REVIEW 28 __________________________ __________________________ ______ In what ways does the data from the VHTs influence malaria prevention and management in this health facility? __________________________ __________________________ __________________________ __________________________ __________________________ __________________________ __________________________ __________________________ _______ 3. DIAGNOSTIC SERVICES 700 In your current position, and as a part of your work for this facility, YES. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 do you personally conduct laboratory tests? NO. . . . . . . . . . . . . . . . 800 . . . . . . . . . . . . . . . . . 2 CIRCLE 'NO' IF THE PROVIDER ONLY COLLECTS SPECIMENS. 701 Please tell me if you personally conduct any of the following tests as part of your work in YES NO this facility 06 Malaria microscopy 1 2 703 Have you received any in-service training or training updates in any of the following topics [READ TOPIC] YES, YES, NO WITHIN OVER IN-SERVICE IF YES: Was the training or training update within the past 24 months or more than PAST 24 MONTHS TRAINING OR 24 months ago? 24 MONTHS AGO UPDATES 07 Malaria microscopy 1 2 3 MODIFIED ON JULY 7, 2019 29 801 Now I would like to ask you some questions about supervision you have personally received from the DHMT. Did you receive technical support or supervision in your work from the DHMT? YES, IN THE PAST 3 MONTHS. . . . . . . . . . . . 1 8 0 4 IF YES, ASK: When was the most recent time? YES, IN THE PAST 4-6 MONTHS. . . . . . . . . . . . . 2 YES, IN THE PAST 7-12 MONTHS. . . . . . . . . . . . . 3 YES, MORE THAN 12 MONTHS AGO. . . . . . . . 4 NO. . . . . . . . . . . . . . . . . . . 5 802 How many times in the past six months has your work been supervised? 803 The last time you were personally supervised, did your supervisor do any of the following: YES NO DK 01 Check your records or reports? CHECKED RECORD 1 2 8 02 Observe your work? OBSERVED WORK 1 2 8 03 Provide any feedback (either positive or negative) FEEDBACK 1 2 8 on your performance? 05 05 04 Give you verbal or written feedback that you were doing VERBAL PRAISE 1 2 8 your work well? 05 Provide updates on administrative or technical PROVIDED UPDATES 1 2 8 USAID UGANDA MAPD MIDTERM LEARNING REVIEW 30 700. Can you tell us how integrated community case management for malaria (iCCM) is going at your health facility? -probe: commodities to VHTs -probe: reporting from VHTs; integrate reporting in DHIS 2 701. What has MAPD done to support iCCM in this community? 702. How do you think iCCM can be improved? 703. Can you describe continuous LLIN distribution in ANC clinics – how is it going? What’s working well? What needs improvement? 704. What do you think it would take to eliminate malaria from your catchment area? Thank you very much for your time! Do you have any questions for us? MODIFIED ON JULY 7, 2019 31 SICK CHILD (Under 5 year) CARE TAKER EXIT INTERVIEW CHILD CARETAKER EXIT INTERVIEW FACILITY IDENTIFICATION TYPE OF FACILITY ( ): R R H o s p H C 3 HC4 NAME OF THE FACILITY: LOCATION OF CLINIC FACILTIY: INFORMATION ABOUT INTERVIEW DATE ___________________ DAY MONTH YEAR Name of interviewer _____________________ Interviewer code Client’s Code INFORMED CONSENT: READ TO CLIENT: Hello, I am_____________. We are representing [IMPLEMENTING ORGANIZATION]. We are conducting a study of the care received by patients at health facilities in [COUNTRY] in order to improve the services this facility offers. We would like to ask you some questions about your experiences here today. Please know that whether you decide to allow this interview or not is completely voluntary and will not affect services you receive during any future visit. You may refuse to answer any question, and you may stop the interview at any time. Information from this interview may be provided to researchers for analyses, but neither your name nor the date of services will be on any shared information, so your identity will remain completely confidential. Do you have any questions for me? Do I have your permission to continue with the interview? YES NO-end 2 0 1 Interviewer's signature DAY MONTH YEAR (Indicates respondent's willingness to participate) 32 1. Information About Visit – SICK CHILD CARETAKER NO. QUESTIONS CODING CLASSIFICATION GO TO DAY MONTH YEAR 100 May I begin the interview? CLIENT AGREES ............................................ 1 CLIENT REFUSES ........................................... 2 END 101 RECORD THE TIME THE INTERVIEW STARTED (HOUR) (MINUTE) 102 What is the name of the sick child? (NAME): _ CLIENT AGE 103 (What month and year was [NAME] born?) MONTH .......................................................... DON'T KNOW MONTH............................... 98 YEAR ............................................................... DON'T KNOW YEAR ...........98 104 How old is [NAME] in completed months? AGE IN MONTHS ............................................ DON'T KNOW .......................................... 98 SIGNS AND SYMPTOMS OF CURRENT ILLNESS 105 Has [NAME] had fever with this illness or any time in the past two days? _____ (YES) ................................................................. 1 (NO)................................................................. 2 (DON'T KNOW).............................................. 8 106 Has [NAME] had a convulsion with this illness? (YES) ................................................................. 1 (NO)................................................................. 2 (DON'T KNOW).............................................. 8 107 Does [NAME] have cough or difficulty breathing with this illness? (YES) ................................................................. 1 (NO)................................................................. 2 (DON'T KNOW).............................................. 8 108 Can [NAME] drink, eat or breastfeed? (YES) ................................................................. 1 (NO)................................................................. 2 (DON'T KNOW).............................................. 8 109 Does [NAME] vomit everything when he/she eats or breastfeeds during this illness? (YES) ................................................................. 1 (NO)................................................................. 2 (DON'T KNOW).............................................. 8 110 Has [HE/SHE] had watery and frequent stools with this illness or any time in the past two days? (YES) ................................................................. 1 (NO)................................................................. 2 (DON'T KNOW).............................................. 8 111 Has [HE/SHE] been excessively sleepy during this illness? (YES) ................................................................. 1 (NO)................................................................. 2 (DON'T KNOW).............................................. 8 MODIFIED ON JULY 7, 2019 33 112 For what other reason(s) did you bring [NAME] to this health facility today? CIRCLE ALL ITEMS THE RESPONDENT MENTIONS PROBE: Anything else? (EAR PROBLEMS) ....................................................A (A SKIN SORE/PROBLEMS)..........................................B (INJURY)............................................................... C (OTHER, SPECIFY) ................................................... X (NO OTHER REASON) ..............................................Y 113 Has [NAME] been brought to this facility before for this same illness? IF YES, ASK: How long ago was that? (WITHIN THE PAST WEEK) .........................................1 (WITHIN THE PAST 2-4 WEEKS) ...................................2 (MORE THAN 4 WEEKS AGO)......................................3 (NO)......................................................................4 (DON'T KNOW) ...................................................8 115 114 How many days ago did the illness for which you brought [NAME] here begin? IF LESS THAN 1 DAY, ENTER 00 (DAYS AGO) .................................................... (DON'T KNOW) .......................................... 98 INFORMATION PROVIDED TO CARETAKER 115 Did the provider tell you what illness [NAME] has? (YES) ................................................................. 1 (NO)................................................................. 2 (DON'T KNOW).............................................. 8 116 IF YES, what did they say? ______________________________ ______________________________ 117 What would you do if [NAME] does not get completely better or becomes worse? (RETURN TO FACILITY)....................................... 1 (GO TO OTHER FACILITY).....................................2 (GO TO OTHER HEALTH WORKER OR PHARMACY) ........3 (GO TO TRADITIONAL HEALER)...........................4 (NOTHING, JUST WAIT) ...........................................5 (DON'T KNOW)...................................................8 118 Did the Provider tell you about any signs or symptoms you may see for which you must immediately bring the child back? IF YES, ASK; Can you tell me what these are? IF NECESSARY, PROBE: Were there any serious symptoms or danger signs for which you were told to bring (name) back immediately? (FEVER).................................................................A (A BREATHING PROBLEM) ........................................ B (BECOMES SICKS)....................................................C (BLOOD IN STOOL) ................................................D (VOMITING) ...........................................................E (POOR/NOT EATING)...............................................F (POOR/NOT DRINKING) ..........................................G (OTHER, SPECIFY)-------.........................................X (NO) .................................................................... Y (DON'T KNOW)..................................................... Z 119 Did the provider tell you anything about bringing [NAME] back to the health facility for follow-up or non-emergency reasons? IF YES: Why were you to return? (to get MORE MEDICINES).....................................A (IF SYMPTOMS INCREASE OR BECOME WORSE) ............. B (FOLLOW-UP APPOINTMENT) ....................................C (VITAMIN A SUPPLIMENTATION) ......................................D (LAB TEST RESULTS) ..................................................E (CHILD ADMITTED)...................................................F (ROUTINE IMMUNIZATION).......................................G (OTHER, SPECIFY)-------.........................................X (NO) .................................................................... Y (DON'T KNOW)..................................................... Z TREATMENT AND CARETAKER COMFORT LEVEL 34 120 Did the provider give or prescribe any medicines for [NAME] to take at home? (YES, GAVE MEDICINE).....................................1 (YES, GAVE PRESCRIPTION) .............................2 (GAVE MEDICINE AND PRESCRIPTION)..........3 (NO)..................................................................4 124 121 ASK TO SEE ALL MEDICATIONS THAT THE CARETAKER RECEIVED AND ANY PRESCRIPTIONS THAT HAVE NOT YET BEEN DISPENSED. CIRCLE THE RESPONSE DESCRIBING THE MEDICATIONS AND PRESCRIPTIONS YOU SEE. (HAS ALL MEDICINES) .......................................1 (HAS SOME MEDICINE, SOME UNFILLED PRESCRIPTIONS) ...................................................2 (NO MEDICINES SEEN, HAS PRESCRIPTIONS ONLY) 3 122 What was the medicine prescribed? ____________________________ 123 Did a provider at the facility explain to you how to provide these medicines to [NAME] when you go home? IF “2” OR “8” SEND CLIENT BACK TO PROVIDER AT THE END OF THE INTERVIEW (YES) ................................................................. 1 (NO)................................................................. 2 (DON'T KNOW).............................................. 8 124 Do you feel comfortable or confident that you know how much of each medication to give [NAME] each day and for how many days to give it? IF “2” OR “8” SEND CLIENT BACK TO PROVIDER AT THE END OF THE INTERVIEW (YES) ................................................................. 1 (NO)................................................................. 2 (DON'T KNOW).............................................. 8 125 Has [NAME] been given a dose of any of these medications here at the facility already? (YES) ................................................................. 1 (NO)................................................................. 2 (DON'T KNOW).............................................. 8 126 Did [NAME] receive an injection for treating the sickness here at the facility today? IF NO, CHECK PRESCRIPTIONS AND RECORD IF THERE IS A PRESCRIPTION FOR AN INJECTION (YES, RECEIVED INJECTION) ........................................1 (YES, RECEIVED PRESCRIPTION FOR INJECTION)..............2 (NO) .....................................................................3 (DON'T KNOW) ...................................................8 127 Did anyone at the health facility weigh [NAME] today? (YES) ................................................................. 1 (NO)................................................................. 2 (CANNOT REMEMBER) ..........................................8 128 Did anyone talk to you today about [NAME]’s weight and how [NAME] is growing? (YES) ................................................................. 1 (NO)................................................................. 2 (CANNOT REMEMBER) ..........................................8 129 Did any provider ask you today about the types of foods and amounts that you normally feed [NAME] when [NAME] is not sick? (YES)................................................................... 1 (NO) .................................................................. 2 (CANNOT REMEMBER) ..........................................8 130 What did the provider tell you about feeding solid foods to [NAME] during this illness? (GIVE LESS THAN USUAL)...........................................1 (GIVE SAME AS USUAL) ..............................................2 (GIVE MORE THAN USUAL).........................................3 (GIVE NOTHING) .....................................................4 (DID NOT DISCUSS)..................................................6 (NOT CERTAIN).......................................................8 131 What did the provider tell you about giving fluids (or breast milk, if the child is breastfed) to [NAME] during this illness? (GIVE LESS THAN USUAL)...........................................1 (GIVE SAME AS USUAL) ..............................................2 (GIVE MORE THAN USUAL).........................................3 (GIVE NOTHING)......................................................4 (DID NOT DISCUSS)..................................................6 (DON'T KNOW) ...................................................8 132 Was [NAME] given a vaccination today? IF YES, ASK TO SEE THE HEALTH CARD OR BOOKLET TO VERIFY. (YES OBSERVED)....................................................1 (REPORTED, NOT SEEN)........................................2 ((NO)...................................................................3 (DON'T KNOW) ...................................................8 REFERRAL MODIFIED ON JULY 7, 2019 35 133 Did the provider instruct you to take [NAME] to see another provider or to a laboratory in this facility for a finger or heel stick for blood to be taken for a test? (YES)................................................................... 1 (NO) .................................................................. 2 134 134 Did you take [NAME] to the provider or laboratory for the finger or heel stick? (YES)................................................................... 1 (NO) .................................................................. 2 134 135 Were you told the result of the test that was done? (YES)................................................................... 1 (NO) .................................................................. 2 136 What was the result? (check to see the results in exercise book) [ ] RDT + [ ] RDT – [ ] Microscopy malaria ++ [ ] Microscopy malaria -- 137 Did the provider instruct you to take [NAME] to see a provider in another facility, or for a laboratory test outside of this facility, for further care for [NAME]? (YES)................................................................... 1 (NO) .................................................................. 2 136 138 Regarding this referral, please tell me: (YES) (NO) (DON'T KNOW) 01 Were you given any paper or record to take with you for the referral? 1 2 8 02 Were you told where to go for the referral? 1 2 8 04 Were you told why you are to go for the referral? 1 2 8 05 Do you intend to go to this (these) referral(s)? 1 2 8 139 Did you take [NAME] to see another health provider before coming here? IF YES, ASK: Whom did you see and where? CIRCLE ALL THAT APPLY (YES DRUG SHOP)................................................A (YES PRIVATE CLINIC)............................................... B (YES, Village Health Worker) C (YES TRADITIONAL HEALER) ................................D (OTHER).............................................................X (SAW NO ONE) ................................................... Y 36 2. Client Satisfaction No. Question (Response options) SKIP Pattern 201 How long did you wait between the time you arrived at this facility and the time you were able to see a provider for the consultation? (MINUTES)...................................................... (SAW PROVIDER IMMEDIATELY) ................ 000 202. Now I am going to ask about some common problems clients have at health facilities. As I mention each one, please tell me whether any of these were problems for you today, and if so, whether they were major or minor problems for you. No. Question (MAJOR PROBLEM) (MINOR PROBLEM) (NO PROBLEM) (DON’T KNOW) (NA) 202_01 Time you waited to see a provider 1 2 3 8 9 202_02 Ability to discuss problems or concerns about children’s illness 1 2 3 8 9 202_03 Amount of explanation you received about the problem or treatment 1 2 3 8 9 202_04 Privacy from having others see the examination or hear my consultation 1 2 3 8 9 202_05 Availability of medicines at this facility 1 2 3 8 9 202_06 The hours of service at this facility, i.e., when they open and close 1 2 3 8 9 202_07 The number of days services are available to you 1 2 3 8 9 202_08 The cleanliness of the facility 1 2 3 8 9 202_09 How the staff treated you 1 2 3 8 9 For the following statements, would you read to respondents and have them reply YES, NO, DON’T KNOW OR NOT APPLICABLE | (Circle number) NO. QUESTIONS (YES) (NO) (DON’T KNOW) (NA) 202_10 Service Providers here are good about explaining the reasons for medical tests 1 2 8 9 202_11 I have to pay more for my medical care than I can afford 1 2 8 9 202_12 The staff treated me in a very friendly and courteous manner 1 2 8 9 202_13 Service Provider sometimes ignore what I tell them 1 2 8 9 202_14 I am able to get medical care whenever I need it 1 2 8 9 MODIFIED ON JULY 7, 2019 37 202_15 I am satisfied with the services you received today 1 2 8 9 202_16 Will you recommend this clinic to a friend or family member? 1 2 8 9 203 Is this the closest health facility to your home? (YES)......................................................1 (NO)......................................................2 (DON'T KNOW)...................................8 301 204 What is the main reason you did not go to the facility nearest to your home? (INCONVIENT OPERATING HOURS) ... 1 (BAD REPUTATION) ........................ 2 (DON’T LIKE PERSONNEL).............. 3 (OTHER, SPECIFY) ................................... 6 3. Client Personal Characteristics NO. QUESTIONS CODING CLASSIFICATION GO TO Now I am going to ask you some questions about yourself. I would like to have your honest responses as this information will help to improve services in general. 300 Gender Male 1 Female 2 301 Age of care taker ___________ 302 What is your relationship to [SICK CHILD]? (MOTHER)....................................... 1 (FATHER)......................................... 2 (SISTER)............................................ 3 (AUNTS/UNCLE) ............................. 4 (GRAND MOTHER/FATHER) ......... 5 (OTHER).............................................. 6 303 [If Mother or Father]-How many children do you have? ____________________ 304 Have you ever attended school? (YES)....................................................... 1 (NO)....................................................... 2 304 305 How old are you? ____________________ 306 What is the highest level of school you completed? (NONE)............................................ 1 (PRIMARY)........................................ 2 (SECONDARY) ................................ 3 (COLLEGE OR HIGHER) .......................... 4 305 307 Do you know how to read or how to write? (YES, READ AND WRITE) . .………. ... 1 (YES, READ ONLY) ………………..…. 2 (NO). …….. ................................................3 308 What do you do for a living? ___________________________________ 309 RECORD THE TIME THE INTERVIEW ENDED . (HOURS) (MINUTE) THANK YOU VERY MUCH FOR TAKING THE TIME TO ANSWER MY QUESTIONS. ONCE AGAIN, ANY INFORMATION YOU HAVE GIVEN WILL BE KEPT COMPLETELY CONFIDIENTIAL. HAVE A GOOD DAY. 38 Interviewer's comments: MODIFIED ON JULY 7, 2019 39 FGD with Female parents age 15-49 of children under five Question/Information Required 00 Date of interview: _____________/____________/_____________ Name of FGD moderator__________________________________ Name of note taker______________________________________ Name of observer_______________________________________ Thank you for making the time to talk with me today. The USAID/Uganda has asked Oxford Epi to conduct an evaluation of the performance of its malaria activities in the region. This evaluation is meant to serve a dual purpose: (1) to learn to what extent the project’s objectives and goals—specifically iCCM—have been achieved; and (2) to inform of new activities that will move the country towards pre elimination of malaria. We are meeting with several groups of Village Health Workers including yours. Our questions are organized to obtain a good overview of how malaria activities functioned operationally. As such, we will be asking you questions about 1) uptake of malaria prevention diagnosis, treatment, and case management activities at the community level. 2) what are structural or leadership barriers that keep you from doing your job at 100% of your satisfaction. 3)What more you could do to improve malaria prevention in your area. All of the answers you provide will remain confidential, and will be summarized and included in our report. We will also be conducting interviews and focus group discussions with other stakeholders, and will be triangulating all findings. The final report will be shared with you through USAID. Before we begin, I want to let you know that any information or examples we gather during this focus group discussion will not be attributed to any specific person or institution, unless you tell us that you would be willing to have your responses to be either quoted by you in the report, or otherwise attributed to you. You are also free to not respond to any of our questions or stop being part of the discussion at any time. Our discussion will take about one hour. Before we go further, do you have any questions about this interview? [ ] Consent provided _________ [Interviewer/Recorder initials] Do you have any questions before we begin? May we continue with the interview? [ ] Yes [ ] No 00 FGD session participants profile Ages________________________________ # Females_____________ #Males________________ Let’s begin by talking about the MAPD Project. 40 01 1a What has been your experience with the promotion ILLNs. 1b What has been your experience with promotion of malaria treatment (ACT)? 1c If you obtained LLINs and malaria treatment products where did you obtain them and what motivated you to do so? 02 How many families are you serving in your villages? How many of them have bed nets? How many of them use bed nets consistently? Treating malaria When a mother came to you with a child under five who had fever most recently, how did you take care of that child? Preventing malaria in CU 5 What are some approaches you’ve applied to prevent malaria in the community? Which approach was the most effective? why? What keeps people from sleeping under a bed net sometimes? 4 03 How do VHTs work with health center professionals around malaria prevention and malaria case management? What has worked well? What needs to be improved? 04 What kind of support do you feel you need to do your job better as a VHT? 05 What is your involvement with iCCM? 06 Would you say the referral system is functional? How often do you refer children for care? Where do you refer them? How effective is the referral process? Do you ever get feedback to that the patient arrived? That patient was cared for? Would you say the referral system is functional? 4 MODIFIED ON JULY 7, 2019 41 Thank you very much! If you think of anything else, please do not hesitate to get in touch with us. 42 MAPD Learning Assessment FGD with Female parents age 15-49 of children under five Question/Information Required 00 Date of interview: _____________/____________/_____________ Name of FGD moderator__________________________________ Name of note taker______________________________________ Name of observer_______________________________________ Thank you for making the time to talk with me today. The USAID/Uganda has asked Oxford Epi to conduct an evaluation of the performance of its malaria activities in the region. This evaluation is meant to serve a dual purpose: (1) to learn to what extent the project’s objectives and goals have been achieved; and (2) to inform of new activities that will move the country towards pre elimination of malaria. We are meeting with several groups of Village Health Workers including yours. Our questions are organized to obtain a good overview of how malaria activities functioned operationally. As such, we will be asking you questions about 1) uptake of malaria prevention diagnosis, treatment, and case management activities at the community level. 2) what are structural or leadership barriers that keep you from doing your job at 100% of your satisfaction. 3)What more you could do to improve malaria prevention in your area. All of the answers you provide will remain confidential, and will be summarized and included in our report. We will also be conducting interviews and focus group discussions with other stakeholders, and will be triangulating all findings. The final report will be shared with you through USAID. Before we begin, I want to let you know that any information or examples we gather during this focus group discussion will not be attributed to any specific person or institution, unless you tell us that you would be willing to have your responses to be either quoted by you in the report, or otherwise attributed to you. You are also free to not respond to any of our questions or stop being part of the discussion at any time. Our discussion will take about one hour. Before we go further, do you have any questions about this interview? [ ] Consent provided _________ [Interviewer/Recorder initials] Do you have any questions before we begin? May we continue with the interview? [ ] Yes [ ] No Before we go further, do you have any questions about this interview? [ ] Consent provided _________ [Interviewer/Recorder initials] Do you have any questions before we begin? May we continue with the interview? [ ] Yes [ ] No MODIFIED ON JULY 7, 2019 43 00 FGD session participants profile Ages___________________________________________________________ # Females_____________ Let’s begin by talking about the MAPD Project. 01 1a How do you prevent malaria? 1b. How do you treat Malaria? 1c. How did you learn about malaria prevention and treatment? (national radio, local radio, Television, Mobile Video Units, interpersonal communications)? 02 Do you have a bed net? If yes, what motivated you to get it? Was it easy? If no, why not? 1 03 Treating illness When your child had a fever most recently, how did you take care of it? Preventing illness in CU 5 Last night, did you sleep under a net? Y/N Last night, did your children under 5 sleep under a net? Y/N Last night, did your children over 5 sleep under a net? Y/N 4 04 What do you think of the referral system? Does it work? How could it be improved? 4 Thank you very much! If you think of anything else, please do not hesitate to get in touch with us. 44 Annex 3: LIST OF PARTICIPANTS National Level Participant Office/ Position Organization District Dr. Jimmy Opigo Ass. Commissioner Health Services, Malaria Division MOH- Malaria Division Kampala Dr. Henry Katamba National Facilitator- Global Fund MOH- National Disease control Kampala Dr. Godfrey Magumba East & Southern Africa Programmes Director Malaria Consortium Kampala Ms. Grace Namata Sagi Deputy Programme Manager - DFID DFID Kampala Dr. Robinah Lukwago Health Advisor - UK Department for International Development UK Department for International Development Kampala Ms. Joanita N Lwanyaga JMS Kampala Mr. Paul Ssenyonga JMS Kampala Dr. Charles Katureebe National Professional Officer - Malaria WHO Kampala District Level Participant Office/ Position Organization District Ayot Peter Gwom SHE Masaka District Local Government Masaka Dr. Birungi Zziwa Godfrey Medical Officer Masaka District Local Government Masaka Magoba Alice Health Educator Masaka District Local Government Masaka Musisi Lilian N DCDO Masaka District Local Government Masaka John Baptist Bwanika SMEA MAPD Kampala Sam Siduda Gudoi COP MAPD Kampala Seemba Hood SACAO Masaka District Local Government Masaka Dr. Musisi Stuart DHO Masaka District Local Government Masaka Martin Seruyange Biostatistician Masaka District Local Government Masaka Lt Andrew Kandiho RDC Masaka District Local Government Masaka Kisekka Janat LCV vice chairperson Masaka District Local Government Masaka Yiga Martin Paul DCAO Lyantonde District Local Government Lyantonde Kyobe Elija Benon DQI/FP Lyantonde District Local Government Lyantonde Kansiime Benon ADHO (EH) Lyantonde District Local Government Lyantonde Lugira Simon Busuulwa A.I.M.O Lyantonde District Local Government Lyantonde Nabisubi Grace EMTCT/FP Lyantonde District Local Government Lyantonde Nakiwala Annet DMMS Lyantonde District Local Government Lyantonde Nababi Gorreth DSFP Lyantonde District Local Government Lyantonde Naggayi Deborah DDI Lyantonde District Local Government Lyantonde Lukyamuzi Ronald DHMIS Officer Lyantonde District Local Government Lyantonde Jjuko Joseph LCV Vice chairperson Lyantonde District Local Government Lyantonde Mutesi Merabu Biostatistician Lyantonde District Local Government Lyantonde Dr.Nkanika Moses DHO Lyantonde District Local Government Lyantonde Karema Fred DMFP Lyantonde District Local Government Lyantonde Mutyaba Michael DLFP Lyantonde District Local Government Lyantonde Dr. Namubiru Mary Technical Director USAID RHITES SW Mbarara Mugumya Lawrence DHSS USAID RHITES SW Mbarara Natumanya K Eliab Director SI USAID RHITES SW Mbarara Wasike Samuel P.O.M&E USAID RHITES SW Mbarara Dr. Edward Bitarakwate COP USAID RHITES SW Mbarara Dr. Edson Tumushere DHO Isingiro District Isingiro Muhwezi Stephen VCO Isingiro District Isingiro MODIFIED ON JULY 7, 2019 45 Nayebare Solome Biostatistician Isingiro District Isingiro Monday Justus MFP Isingiro District Isingiro Kajungu Clemmy ADHO MCH Isingiro District Isingiro Jennifer Mugisha STA./FH USAID RHITES SW Isingiro Kaija John Paul Entomologist Kabarole District Local Government Kabarole Muhenda Stephen Health Officer Kabarole District Local Government Kabarole Byaruhanga Chris Biostatistician Kabarole District Local Government Kabarole Kisembo Michael ACAO Kabarole District Local Government Kabarole Monday Christopher DCDO Kabarole District Local Government Kabarole Kihika Margaret Vice Chairperson LCV Kabarole District Local Government Kabarole Kisembo Brian DHE Kabarole District Local Government Kabarole Dr. Asiimwe Solomon Ag DHO Kabarole District Local Government Kabarole Rwobuhinga Richard LCV Chairperson Kabarole District Local Government Kabarole Ssempala Emmanuel DCAO Kabarole District Local Government Kabarole Balinda Resty Mboijana District councilor Kabarole District Local Government Kabarole Katumba Enid DCAO Mityana District Local Government Mityana Lwasampijja Fred DHO Mityana District Local Government Mityana Muzira David DCDO Mityana District Local Government Mityana Faith Nakiyimba MFP Mityana District Local Government Mityana Zalwango Margaret Intern Biostatistician Mityana District Local Government Mityana Nampijja Justine Biostatistician Mityana District Local Government Mityana Luzige Joseph LCV Chairperson Mityana District Local Government Mityana Kisakye Ivan Commercial Officer Mityana District Local Government Mityana VHTs Interviewed District Male Female Total Lyantonde 0 7 7 Masaka 8 20 28 Mbarara 5 9 14 Isingiro 2 5 7 Mityana 1 5 6 Total no. of VHTs 16 46 62 Pregnant Women 15-49 District No. of participants Total Kabarole 18 18 Lyantonde 13 13 Masaka 28 28 Mbarara 20 20 Mityana 07 07 Total no of participants 86 46 Annex 3: List of Documents Reviewed 1. Malaria Action Program for Districts Activity Monitoring, Evaluation, and Learning Plan. Resubmitted February 21, 2017 to PMI. 2. Malaria Action Program for Districts. Project Implementation Plan October 1, 2016 to September 30, 2017- District Focused Activities. 3. Malaria Action Program for Districts (MAPD). Quarterly Report. August 19, 2016 to September 30, 2016. 4. Malaria Action Program for Districts (MAPD). Quarterly Report. October 1, 2016 to December 31, 2016. 5. Malaria Action Program for Districts (MAPD). Quarterly Report. January 1, 2017 to March 31, 2017. 6. Malaria Action Program for Districts (MAPD). Quarterly Report. April 1, 2017 to June 30, 2017. 7. Malaria Action Program for Districts (MAPD). Quarterly Report. July 1, 2017 to September 30, 2017. 8. Malaria Action Program for Districts (MAPD). Quarterly Report. October 1, 2017 to December 31, 2017. 9. Malaria Action Program for Districts (MAPD). Quarterly Report. January 1, 2018 to March 31, 2018. 10. Malaria Action Program for Districts (MAPD). Quarterly Report. April 1, 2018 to June 30, 2018. 11. Malaria Action Program for Districts (MAPD). Quarterly Report. July 1, 2018 to September 30, 2018. 12. Malaria Action Program for Districts (MAPD). Quarterly Report. October 1, 2018 to December 31, 2018. 13. Malaria Action Program for Districts (MAPD). Quarterly Report. January 1 to March 31, 2019. 14. Malaria Action Program for Districts. Work Plan October 1, 2016 to September 30, 2017. 15. Malaria Action Program for Districts. Work Plan October 1, 2017 to September 30, 2018. 16. Malaria Action Program for Districts. Work Plan October 1, 2018 to September 30, 2019. 17. Malaria Action Program for Districts. Year 1 Report. October 1 2016 to September 30, 2017. 18. Malaria Action Program for Districts. Year 2 Report. October 1 2017 to September 30, 2018. 19. Malaria Clinical Services Mentorship Guide and Toolkit. National Malaria Control Program. Ministry of Health, Uganda. April 2019. 20. Uganda Malaria Indicator Survey, 2014-2015. Uganda Bureau of Statistics Kampala, Uganda; National Malaria Control Programme Uganda Ministry of Health Kampala, Uganda; Uganda Malaria Surveillance Project Molecular Laboratory Mulago Hospital Kampala, Uganda; ICF International Rockville, Maryland, USA. 21. USAID Uganda Country Development Coorporation Strategy 2016-2021 22. President’s Malaria Initiative, Uganda Malaria Operational Plan FY 2019 MODIFIED ON JULY 7, 2019 47 23. President’s Malaria Initiative, Uganda, Malaria Operational Plan 2018 24. Malaria Epidemic Map MIS 2009 25. Malaria Consortium Technical Proprosal-resubmitted, 2016. 26. MAPD Statement of work. USAID. 2016. 27. Project Performance Indicator with five year targets. MAPD. 28. Severe malaria case management in Uganda: A rapid assessment of management of severe malaria at health centers in Jinja District, Uganda. Ministry of Health, Uganda. 29. The DHS Program. Summary Malaria Indicators 2006 -2016. 30. Uganda Malaria Reduction Strategic Plan 2014-2020. The Republic of Uganda. Ministry of Health. Plot 6, Lourdel Road, Nakasero. P.O. Box 7272. Kampala, Uanda. May 2014. 31. Ye, Yazoume; Omumbo Judy; Ssekitooleko J, Shah, JA. USAID/Uganda Baseline Assessment for the Malaria Action Program for Districts (MAPD). Final Report. Sept 2017. 32. PMI. President’s Malaria Initiative. Fighting Malaria and Saving Lives. Uganda Profile 2018.