1 HealthPartners Uganda Health Cooperative Child Survival Project Bushenyi District, Uganda Cooperative Agreement Number GHS-A-00-05-00031-00 September 30, 2005 to September 29, 2010 Final Evaluation Report Report Authors: Garth Osborn, Independent Consultant Jolene Mullins, Country Director Tanzania, WellShare International December 31, 2010    2 Table of Contents Acronym and Abbreviation List ................................................................................................. 3 A. Executive Summary ................................................................................................................... 5 B. Project Overview ........................................................................................................................ 9 C. Data Quality: Strengths and Limitations .................................................................................. 13 D. Presentation of Results ............................................................................................................. 15 E. Discussion of Results ............................................................................................................... 19 F. Discussion of Potential for Sustained Outcomes, Equity, Community Health Worker Model, and Global Learning ................................................................................................................ 28 G. Conclusions and Recommendations ........................................................................................ 35 Annex 1: Results Highlight: Opening the door to another source of innovation: USAID’s ........ 38 Annex 2: List of Publications Relevant to the Project .................................................................. 39 Annex 3: Project Management Evaluation ................................................................................... 40 Annex 4: Work Plan Table ........................................................................................................... 45 Annex 5: Rapid CATCH Table ..................................................................................................... 48 Annex 6: Final KPC Report .......................................................................................................... 49 Annex 7: Community Volunteer Health Worker and Health Worker Training Matrixes ............ 78 Annex 8: Evaluation Team Members ........................................................................................... 79 Annex 9: Evaluation Assessment Methodology ........................................................................... 80 Annex 10: Persons Interviewed and Contacted during the Final Evaluation ............................... 82 Annex 11: Special Reports ............................................................................................................ 84 Annex 12: Project Data Form ....................................................................................................... 90 Annex 13: Grantee Plans to Address Final Evaluation Findings .................................................. 97 Annex 14: Grantee Response to Final Evaluation Findings ......................................................... 99    3 Acronym and Abbreviation List ANC Antenatal Care AMTSL Active Management of Third Stage Labor CBHW Community-Based Health Worker CDD Control of Diarrheal Disease BCC Behavior Change Communication BCI Behavior Change Intervention BOD Board of Directors CE Community Education CHeFA-EA Community Health Financing Association of East Africa CHF Community Health Financing C-IMCI Community Based Integrated Management of Childhood Illness CORE Child Survival Collaborations and Resources Group CORP Community Owned Resource Persons CSHGP Child Survival and Health Grants Program CSTS Child Survival Technical Support DDHS District Director of Health Services DHS Demographic and Health Survey DHT District Health Team DIP Detailed Implementation Plan FP Family Planning GOU Government of Uganda HFA Health Facility Assessment HIV Human Immunodeficiency Virus HP HealthPartners HQ Headquarter (Minnesota based) HSD Health Sub District HST Health Services Training HW Health Workers ICCM Integrated Community Case Management IMCI Integrated Management of Childhood Illness IMR Infant Mortality Rate IPT Intermittent Preventive Treatment IR Intermediate Results ISA Institutional Strengths Assessment ITN Insecticide-treated Nets KPC Knowledge, Practice and Coverage Survey LQAS Lot Quality Assurance Sampling MANGO Management Accounting for Non-Governmental Organizations M&E Monitoring and Evaluation MCH Mother and Child Health MCP Malaria Control Program (by MOH) MMR Maternal Mortality Rate MNC Maternal Newborn Care MOH Ministry of Health    4 NGO Non-government Organization ORS Oral Rehydration Solution OVC Orphans and Vulnerable Children PHC Primary Health Care PMTCT Preventing Mother to Child Transmission (of HIV) PPC Post Partum Care PPH Post Partum Hemorrhage PSI Populations Services International PUR Water Filtration System PVO Private Voluntary Organization RCT Routine Counseling and Testing (Updated from VCT) SCM Standard Care Management STD Sexually Transmitted Disease TA Technical Assistance TBA Traditional Birth Attendant TRM Technical Reference Materials TOT Training of Trainers UCBHFA Uganda Community Based Health Financing Association UHC Uganda Health Cooperative UHIS Uganda Health Information System UPMA Uganda Private Midwives Association WRA Women of Reproductive Age    5 A. Executive Summary Project Description: HealthPartners has been implementing its Uganda Health Cooperative (UHC) Child Survival Project in Bushenyi District in rural southwestern Uganda since October 2005. At that time, Bushenyi District was made up of five health sub districts with an estimated population of 759,000 and a 2% annual growth rate. HealthPartners sought to serve 34,500 women of reproductive age (WRA) and 15,500 infants and children under the age of five for a total beneficiary population of 50,000 over the life of the project. In addition the project sought to enlist 14,000 WRA and children under five in prepaid health schemes by 2010. The project goal was to link child survival interventions and partnerships to prepaid health schemes, building on the existing structure to sustainably reduce morbidity and mortality for WRA and children under 5. The project focused on five objectives: 1. Reduce incidence of malaria in Bushenyi District for children under 5 & pregnant women 2. Reduce incidence of diarrhea in Bushenyi District for children under 5 3. Increase % of pregnant women receiving improved ANC, delivery and post partum care 4. Build local organizational capacity to manage health schemes 5. Improve health care management especially for WRA and children under 5 The project was a partnership between HealthPartners (a US-based health insurance cooperative), the Bushenyi DHT, the UHC Board and stakeholders, including 29 UHC member groups and a dozen local health care providers. Project strategies included the following:  Development and support of a sustainable, locally-run health insurance cooperative.  Behavior change communication to improve health-related knowledge, practices and access to care at the community level.  Capacity building of health workers and community volunteers through training and supervision in IMCI, MNC and the new Ugandan Village Health Team model.  Demand creation for essential health resources, including ITNs, ORS sachets, PUR sachets, covered water containers and others. Key Findings/Results: Working within the parameters of a “traditional” Child Survival project, HealthPartners has established a prepaid insurance cooperative that covers the primary health care and maternal newborn care services for over four thousand people – something that has never been attempted in this type of setting before or through the CSHGP. It has developed a locally-run governance structure to oversee it and recruited a dozen health care facilities to provide the direct services – which are benefiting from their participation. Even if this cooperative only serves a relatively small percentage of the overall population directly in Bushenyi, these are people who would not otherwise receive the financial security that comes with insurance. In addition, the data shows that the program benefitted non-UHC members as well. Summary of Major Project Accomplishments Objective #1: Reduce incidence of malaria in Bushenyi District for children under 5 and pregnant women Project Inputs Activities Outputs Outcomes    6 ITNs and ITN demo kits Training materials and volunteer incentives (t￾shirts, bikes, awards, etc.). BCC and IEC materials. Trainers and supervisors. Train, support and supervise CORPs and VHTs in IMCI. Advocate with DHT to distribute ITNs. Train DHT staff in supply management. 5,001 ITNs donated and 174 sold. Community BCC sessions promoting proper ITN use. GOU ITN distribution Aug/Sept 2010. % of children >5 who slept under an ITN the previous night increased from 32% to 54% % of mothers who slept under an ITN the previous night increased from 19% to 48%. Objective #2: Reduce incidence of diarrhea in Bushenyi District for children under 5 PUR sachets and WaterGuard and ORS demo kits Training materials and volunteer incentives (t￾shirts, bikes, awards, etc.). BCC and IEC materials. Trainers and supervisors. Train, support and supervise CORPs and VHTs in WASH. Promote improved access to safe water Engage local leadership to promote and follow up on community adoption of improved pit latrines Promotion of PUR sachets and WaterGuard. 6,784 sachets of PUR and 3,939 sachets of ORS distributed Community BCC sessions promoting hand washing % of children under 2 with diarrhea in the last two weeks decreased from 55% to 33%. Community BCC sessions promoting hand washing % of caretakers who usually wash hands before feeding children increased from 15% to 36% and % of caretakers who usually wash hands after attending to a child who has defecated increased from 8% to 21%. Community BCC sessions promoting latrines % of households with access to a covered pit latrine increased from 19% Partnership with local to 77%. leaders to support uptake of improved latrines Objective #3: Increase % of pregnant women receiving improved ANC, delivery and post partum care Training materials and volunteer incentives (t￾shirts, bikes, awards, etc.). BCC and IEC materials. Trainers and supervisors. Train, support and supervise CORPs and VHTs in MNC. Community BCC sessions promoting RCT services. % of pregnant women seeking RCT services increased from 34% to 96%. Train, support and supervise CORPs and VHTs in MNC. Community BCC sessions promoting breastfeeding. % caretakers counseled on breastfeeding increased from 38% to 73% Mama Kits Training materials and volunteer incentives (t￾shirts, bikes, awards, etc.). BCC and IEC materials. Trainers and supervisors. Train, support and supervise CORPs and VHTs in MNC. Advocate with DHT to distribute Mama Kits. Distribution of Mama Kits. Community BCC sessions promoting safe birth plans. % of women who delivered with a skilled health professional as verified by maternal card/ book increased from 47% to 68% Objective #4: Build local organizational capacity to manage health schemes Supervisory checklists. Training materials/supplies Trainers and supervisors. Train DHT and project staff in supportive supervision. Joint supervisory visits by DHT and project staff. 89% of HWs are being regularly supervised.    7 Transportation Objective #5: Improve health care management especially for WRA and children under 5 Trainers and technical experts Developed user manuals for BOD, new member groups, volunteers & providers. Recruit members, train and provide technical/ administrative support to UHC Board. UHC BOD capacity increased from ‘minimal’ to ‘promising/strong’ per review of BOD minutes Marketing materials Technical expertise in BCC Develop targeted marketing strategy using BEHAVE Framework model. Engage providers, group members, and DHT staff in marketing membership to the UHC. Enlist 12 health care providers in the UHC network UHC membership increased from 1,783 to 4,227 Financial reporting system established. Audit Training of UHC group leaders and providers (Data Entrants) in UHC financial system. Production and analysis of financial reports. Provide feedback. 56% (16/29) of UHC groups reported revenues>expenses. Summary Conclusions:  Community members are better equipped to prevent and manage fevers and diarrhea, and understand when medical care is necessary for these ailments and maternal health.  The project’s community volunteer network has extended the reach of primary and reproductive health into communities and linked them to facility-based services.  A locally-run health insurance cooperative has been established and is running at a surplus with annual stakeholder contributions deposited in a reserve fund managed by the board of directors. Its future success depends on expanding its membership while continuing to provide consistent quality services within a fluid national policy environment that is now determining the future of health care financing in Uganda.  As a first-time CSHGP grantee, the project has a few technical issues to address in its planning and data management, which are detailed later in this report. Summary Recommendations: 1. The project needs to advocate for an on-going commitment of support for VHTs from the communities they serve, the health providers they assist, and the local authorities they answer to in each of the five new districts. This should be part of a detailed exit strategy the project needs to develop and implement within its few remaining months. 2. The project should advocate for a multi-sector approach that fosters links among NGOs and government programs – increasing community access to services (especially for men’s participation in ANC and safe delivery planning along with their partners), sharing information, and finding new sources of support for the volunteer structure. 3. Several recommendations were identified for increasing UHC membership, some of which have already been considered and others are new. a. Incentivize ‘satisfied customers’ to act as marketers, promoters and mentors to new groups. b. Re-explore the option of covering care for chronic illnesses at least on a limited, trial basis. The costs of these services could be covered through a tiered membership structure. c. Explore the option of allowing individuals to join at a reduced rate from families.    8 d. Identify other health service providers in major urban areas (i.e. Kampala, Mbarara, Masaka) as members requested coverage when traveling “out of area”. 4. HealthPartners should reapply for membership in the CORE Group regardless of its future involvement with the Child Survival and Health Grants Program, as this would be highly beneficial to both.    9 B. Project Overview Project Goal, Objectives and Intermediate Results: Results Framework Goal: Link child survival interventions and partnerships to prepaid health schemes, building on the existing structure to sustainably reduce morbidity and mortality for WRA and children under 5. Community Social Dimension Objective 1: Reduce incidence of malaria in Bushenyi district for children under 5 and pregnant women Community Social Dimension Objective 2: Reduce incidence of diarrhea in Bushenyi district for children under 5 Community Social Dimension Objective 3: Increase % of pregnant women receiving improved ANC, delivery and post partum care Local Organizational Dimension Objective 4: Build local organizational capacity to manage health schemes1 Health Services Dimension Objective 5: Improve health care management especially for WRA and children under 5 IR 1: Increase ITN use IR 2: Increase timely, appropriate malaria treatment IR3: Increase IPT IR 1: Improve recognition by caretakers of warning signs that child needs treatment IR 2: Improve safe water access & use IR 3: Increase safe water practices and hand washing IR 4: Improve sanitation practices IR 5: Increase intervention for severe diarrhea and dehydration IR 6: Increase zinc supplementation (On hold for MOH approval) IR 1: Increase demand for ANC IR 2: Increase demand for RCT IR 3: Improve caretaker knowledge of nutrition, child spacing, and danger signs IR 4: Increase access to safer birthing IR 1: Increase capacity of Board IR 2: Growth of health scheme IR 3: Increase ability of health scheme to financially cover service and administration IR 1: Reduce incidence of stock outs IR 2: Improve use of IMCI and SCM IR 3: Improve ANC IR 4: Improve postpartum care IR 5: Improve delivery (including the promotion of “clean practices and AMTSL) IR 6: Improve self assessment ability Project Location: HealthPartners Child Survival project was implemented in Bushenyi District in rural southwestern Uganda, located 400 kilometers from the national capital Kampala. Bushenyi District is bordered by Lake Edward and Democratic Republic of the Congo to the west, Kasese District to the north, Kamwenge and Ibanda Districts to the northeast, Mbarara                                                              1 NOTE: The position of objectives 4 and 5 was switched later in the project, as reflected in the M&E table below and throughout the remainder of this report.    10 District to the east, Ntungamo District to the south and Rukungiri District to the southwest. It covers approximately 4,293 square kilometers of flat grasslands and rolling hills. At the time of the DIP, Bushenyi District was made up of five counties (Buhweju, Ruhinda, Sheema, Bunyaruguru and Igara), 27 sub counties, 170 parishes and 2034 villages.2 Bushenyi was later divided into five separate districts in July 2010, each of which will have its own separate district government. Population: The population is represented by the Banyankole, Bakiga and Bakonjo tribes, the vast majority of whom live in rural villages. Most of the population is involved in agriculture and livestock – raising tea, coffee, sweet bananas, green bananas ‘matoke’ (the regional staple), beef and cotton. At the time of the DIP Bushenyi District had an estimated total population of 759,000 and a 2% annual growth rate. The female population was 395,000 and the under 5 population was 153,000. The population of infants was 33,000, 12-23 month olds 31,000, and children 24- 49 months 94,000. Thirty-seven thousand births were projected to occur in the first year of the project.3 HealthPartners sought to directly4 reach at least 34,500 WRA, 3,200 children 0-12 months, 3,100 children 12-23 months and 9,200 children 24-59 months old totaling a minimum of 50,000 women and infants over the five year life of the project. In addition the project sought to enlist 14,000 WRA and children under five in prepaid health schemes by 2010. Technical and Cross Cutting Interventions: The project included three child survival interventions: malaria at 50% of total effort, control of diarrheal diseases (DD) at 25%, and maternal newborn care (MNC) at 25%. Cross cutting interventions focused on building the capacity of a locally elected board of directors to manage a prepaid health insurance cooperative and the Bushenyi government District Health Team (DHT) in the administration of its services for mothers and children. The principle messages focused on:  Increased and proper use of ITNs by WRA and children under five.  Provision of IPT during ANC.  Timely care seeking in response to signs of malaria and dehydration.  Promoting the use of safe drinking water  Improved hygiene and hand washing  Improved sanitation practices  The importance of attending at least four ANC sessions per pregnancy.  Planning for a delivery at a health facility attended by a trained health professional.  The importance of attending post partum care.  The value of participating in the UHC. The project employed a variety of key strategies:                                                              2 http://bushenyi.go.ug/index.htm 3 Bushenyi Local Government Health Sector Budget Conference for FY 2006/2007 4 Indirectly through child survival education, drama shows, radio and other media, UHC expected to reach 100,000 or more    11  Behavior change communication (BCC) to develop strategies for increasing UHC membership and improving health behaviors related to the project’s three CS interventions.  Community mobilization by volunteer Community Owned Resource Persons (CORPs) and later the Volunteer Health Teams (VHT) to promote project services and convey the project’s health education messages at the village level. (See Section H for a more detailed description of project-supported volunteers.)  Training of CORPs in Community-IMCI and later, training VHTs on integrated community case management (ICCM), consistent with the MOH’s change in policy.  Health worker training on HW-IMCI.  Health worker and CORPs/VHT training on MNC.  Joint supportive supervision of health workers and CORPs/VHTs.  The design, development, marketing, administration and monitoring of a prepayment health insurance cooperative.  Create demand for and promote targeted community distribution of ITNs, ORS sachets, PUR sachets, covered water containers, WaterGuard, Mama Kits, iron tablets and zinc, once approved by the GOU/MOH. Project Design: The design of this project reflected its overall purpose, which was to test whether the availability of prepaid, locally managed health insurance would have an impact on child survival indicators specific to MNC and the two health problems that pose the highest toll on children in this resource-limited setting – malaria and diarrheal diseases. The project’s straightforward approach to child survival and DHT capacity building was juxtaposed against a health insurance scheme where groups could make quarterly premium payments so that basic curative care and delivery services would be available to them at a local NGO-run health facility at no further or very limited cost. For its community social and health service dimensions (See Results Framework above) the project was to be administered in six phases and was later streamlined to five, with each new phase covering health sub districts (HSD) within Bushenyi until the entire district was covered. In each phase, UHC and DHT staff collaborated to train and provide supportive supervision to health workers and community volunteers using MOH-approved curricula and messages on each of the project’s three CS interventions. Lessons learned and experiences were reviewed after each phase and used to improve the project’s work in the subsequent phases. Prior to the last phase, however, the national MOH policy switched from C-IMCI over to its own Village Health Team Model so the health workers and volunteers had to be trained in the new system across the entire project area. In response, the UHC met and worked closely with the DHT and a local Ugandan NGO, Integrated Community Based Initiatives (ICOBI), to share the responsibility for providing this training throughout the project area. The UHC covered two HSDs, the DHT covered two and ICOBI committed to training the remaining HSD in 2011. The project worked on its local organization dimension throughout the life of the project – building the capacity of the UHC’s Board of Directors to set and maintain a sustainable cost basis for the cooperative and focusing key activities on the expansion of membership in the cooperative. Partnerships: This project involved several partners, which helped to leverage resources, avoid duplication, and coordinate strategies and services. The principal partners were the Bushenyi    12 District Health Team and the UHC Board of Directors and stakeholders, which includes member groups and a dozen health care providers: Comboni Hospital Bubangizi Health Centre III Nyakasiro Health Centre III Butare Health Centre III Nyakashaka Health Centre II Humuza Clinic Mitooma Nursing Home St. Simon Clinic Rubaare Health Centre II St. Joseph Clinic Bushenyi Medical Center BB Clinic The project also coordinated and collaborated with the following organizations: International NGOs: Population Services International (PSI) and their Uganda affiliate PACE, Africare, Compassion International, Marie Stopes Uganda, Healthy Child Uganda (HCU) and Community Health Financing Association for East Africa (CHeFA – EA). Ugandan NGOs: Uganda Community Based Health Financing Association (UCBHFA), Uganda Red Cross, Integrated Community Based Initiatives (ICOBI), MACIS, Save for Health Uganda, Reproductive Health Uganda and the Catholic Diocese. Mission Collaboration: HealthPartners initiated work in Uganda in 1997 as a subcontractor on a USAID-funded grant to Land O’Lakes, a Minnesota-based dairy cooperative that had extensive prior experience in Uganda and throughout Africa. This was HP’s first organizational experience working outside the United States. It was recognized that HealthPartners’ expertise in fostering health care cooperatives would be a natural complement to the dairy cooperatives Land O’ Lakes was developing in rural Uganda. Since then HealthPartners has proactively sought to foster a relationship with the USAID Mission in Kampala by sharing all major project documents with them and inviting their staff to attend key events throughout the life of the project. Frequent turnover in the mission staff, including five Directors over the past dozen years, along with HealthPartners’ preference for posting its senior project staff in the field at the project site, has limited opportunities to foster this relationship, however, HealthPartners remains committed to developing this relationship further.    13 C. Data Quality: Strengths and Limitations Quality of the Data Presented in the M&E Table: This was HealthPartners’ first USAID￾funded Child Survival project and because of its design and the UHC component, it required a more complex HIS and monitoring system than most other Child Survival projects. To determine whether the availability of health insurance had an impact on the child survival indicators, the project’s M&E plan had to go beyond that used in a standard Child Survival project, which compares coverage in a beneficiary population across the life of the project. To test whether this project was achieving its community dimension objectives, it also had to compare health plan members to non-members who lived within the project area as well as a control group who had no contact to the project. The project used LQAS to measure its KPC community dimension indicators (Objectives 1-3). However, certain technical problems undermined the quality of some of the results and the conclusions that can be drawn from them, as detailed here and in the M&E table’s footnotes below. First, a sufficient sample size was not drawn for all the project indicators. LQAS requires a sample of approximately 95 for each indicator in order to measure coverage.5 While the project achieved 95 for several indicators, for five of the most critical ones it did not, including those focusing on the proper care of the sick child. This is an inherent difficulty in using LQAS, whenever a subset of the beneficiary population is being measured, for instance, children with fever within the previous two weeks. This problem was apparent in the project DIP, midterm evaluation report, and annual reports for years two and four, but not mentioned in reviewer or previous evaluator comments. Second, the project has reported coverage rates at the lot level where the sample size was only 19.6 LQAS is designed to determine whether a lot of 19 has achieved a set target or not, referred to as the ‘decision rule,’ and cannot be used to measure coverage. Third, the timing of when the KPC survey data was collected also could have had an impact on the quality of the results, particularly for the two indicators that measured incidence of fever and diarrheal disease over the previous two weeks since both are highly related to seasonality and weather.7 In order to meet CSHGP deadlines, the KPC for the baseline DIP was done in January, while the midterm and final evaluation KPCs were both done in June. This is an inherent problem within the scheduling of the CSHGP. Finally, the wording of some of the indicators in objectives four and five was changed between the baseline DIP and final evaluation results, which made the data difficult to compare. Use of Qualitative and Quantitative Data: The project used both qualitative and quantitative data to track overall progress and test the effectiveness of various volunteer incentive strategies as well as fee structures and marketing strategies for the UHC. For qualitative data the project used adapted versions of the Child Survival Sustainability Assessment (CSSA) tool to assess and track capacity building towards sustainability and the Institutional Strength Assessment (ISA) for monitoring the development of staff capacity. The UHC Board of Directors filled self assessments each quarter throughout the life of the program. For quantitative data the project                                                              5 19 randomly selected responses per lot X 5 lots = 95. 6 See Annex 1 in the Year Four Annual Report 7 Reference indicators 1.a and 2.a in the M&E table below.    14 used the KPC, health facility assessment (HFA), financial reports to track UHC fund performance, and monthly data tracking by volunteers on the number of BCC sessions and signatures turned in from session attendees. Use of District Health Care System Data: The DHT provided the project with the District Profile Health Management Information System 105 form, which it has used to plan interventions, trainings and community activities as well as monitor progress. Systematic Collecting, Reporting and Use of Data at all Levels: Initially HealthPartners set up a computer-based data information system for UHC providers, referred to as the Uganda Health Information System (UHIS). This system would be used to collect, tabulate and provide data reports UHC managers could use for planning, monitoring, supervision and evaluation. However, the project discovered early on that this system was not sustainable because of the lack of a consistent power supply for the computers and qualified staff to manage and maintain the system. Hence, the UHIS data tracking was replaced with monitoring the national Health Management Information System 105 form. Each project staff person, community-based volunteer and UHC group leader is responsible for collecting and reporting data on a regular basis, for which they receive feedback. VHTs track the BCC events they do, including sign-in sheets documenting each participant and the topics covered. Each UHC group leader collects premiums from his/her members and submits those to the provider along with a financial report. The UHC information is then combined by each provider’s Data Entrant with the number of members accessing care, the cost of care, co-pays, and diagnosis. Both the VHT and UHC reports are shared with the project staff, DHT and the UHC Board to track performance of the cooperative and the work of the VHTs; bill the providers for their fee to the reserve fund; and provide feedback to all that can be used to make adjustments. The effectiveness and uptake of health worker training were measured through self-assessments starting after the project midterm and continued through the final evaluation. The results are used to structure refresher trainings and supervisory visits. Use of Data to Make Management Decisions: The project developed a spreadsheet that compared the number of sessions and participants at volunteer-run BCC sessions with the timing and type of volunteer incentives offered in order to determine which incentives produced the highest level of volunteer performance. These results were then used to structure volunteer incentives in subsequent phases of the project. The project also uses the UHC financial reports to track the performance of each cooperative group and provider to identify which are operating at a surplus or deficit so appropriate follow up and support can be targeted as needed. In addition, the project used an adapted version of the BEHAVE Framework conducting a doer/non-doer analysis to develop a BCC plan for increasing UHC membership. At the end of each phase, the program shared KPC and HFA results with district and county stakeholders and incorporated lessons learned and feedback into the strategies that were used for the following phases.    15 D. Presentation of Results The project set out five objectives to achieve its goal, each of which had a related set of indicators for monitoring and evaluating progress. Indicators Data Source Baseline Value (%) Final Values Final Target (%) UHC Members (%) Non UHC Members (%) Objective 1: Reduce incidence of malaria in Bushenyi district for children under 5 and pregnant women 1.a. % of children under 2 with fever in the last two weeks KPC 44% 46% 58% 19% 1.b. % of children under 2 who slept under an ITN last night. KPC 32% 54% 57% 55% 1.c. % of mothers who slept under an ITN last night. KPC 19% 48% 40% 36% 1.d. % of children under 2 with fever in the last 2 wks who received anti malarial treatment KPC 71% 44%8 42%9 84% 1.e. % of pregnant women receiving IPT as verified by maternal card/ book KPC 27% 94%10 73%11 49% Objective 2: Reduce incidence of diarrhea in Bushenyi district for children under 5 2.a. % of children under 2 with diarrhea in the last two weeks KPC 55% 33% 34% 20% 2.b. % of mothers who know at least two signs that a child under 5 needs treatment. KPC 76% 81% 78% 84% 2.c. % of households who use improved water source (borehole, public tap, or protected dug well.) KPC 50% 64% 56% 60% 2.d. % of households with a designated hand washing station with a covered container for water KPC 24% 27% 10% 46% 2.e. % of caretakers who usually wash hands with soap before food preparation KPC 42% 45% 47% 64% 2.f. % of caretakers who usually wash hands before feeding children KPC 15% 36% 22% 35% 2.g. % of caretakers who usually wash hands after defecation KPC 63% 64% 57% 82% 2.h. % of caretakers who usually wash hands and after attending to a child who has defecated KPC 8% 21% 20% 30%                                                              8 The sample size for this indicator was 43, but needed to be at least 95 to achieve a statistically significant measurement of coverage using LQAS. Therefore the accuracy of this result cannot be substantiated. 9 The sample size for this indicator was 55, but needed to be at least 95. 10 The sample size for this indicator was 35, but needed to be at least 95. 11 The sample size for this indicator was 37, but needed to be at least 95.    16 Indicators Data Source Baseline Value (%) Final Values Final Target (%) UHC Members (%) Non UHC Members (%) 2.i. % of households who safely disposed of their child's feces the last time s/he passed stool KPC 68% 73% 55% 82% 2.j. % of households with access to a covered pit latrine KPC 19% 77% 58% 36% 2.k. % of children 0-23 months with diarrhea in the last two weeks who received ORS KPC 0% 26%12 19%13 30% 2.l. % of children aged 0-23 months who were offered more fluids during illness KPC 16% 48%14 25%15 N/A 2.m. % of children aged 0-23 months who were offered more food during illness KPC 42% 36%16 28%17 N/A 2.n. % of children aged 0-23 months who were offered more fluids and food during the illness KPC 2% N/A N/A 20% Objective 3: Increase % of pregnant women receiving improved ANC, delivery and post partum care 3.a. % of women with 4 ANC visits as verified by maternal card/ book. KPC 18% 46%18 58%19 30% 3.b. % of pregnant women seeking RCT services KPC 34% 96% 88% 44% 3.c. % caretakers counseled on breastfeeding KPC 38% 73% 61% 55% 3.d. % caretakers counseled on importance of child spacing KPC 95% 71% 55% 99% 3.e. % caretakers counseled on danger signs of pregnancy KPC 76% 76% 65% 90% 3.f. % of women who delivered with a skilled health professional as verified by maternal card/ book KPC 47% 68% 52% 65% Indicators Data Source Baseline Value (%) Final Value (%) Final Target (%) Objective 4: Improve health care management especially for WRA and children under 520                                                              12 The sample size for this indicator was 31, but needed to be at least 95. 13 The sample size for this indicator was 32, but needed to be at least 95. 14 The sample size for this indicator was 31, but needed to be at least 95. 15 The sample size for this indicator was 32, but needed to be at least 95. 16 The sample size for this indicator was 31, but needed to be at least 95. 17 The sample size for this indicator was 32, but needed to be at least 95. 18 The sample size for this indicator was 37, but needed to be at least 95. 19 The sample size for this indicator was 38, but needed to be at least 95. 20 This objective was originally Objective 5 in the DIP.    17 Indicators Data Source Baseline Value (%) Final Value (%) Final Target (%) 4.a. % of stock outs in the past 30 days21 HFA and self assessment performance scale HMIS 105 41% N/A 18% 4.b. % of HWs who have received IMCI training in the last 3 years HFA and self assessment performance scale 57% 37% 70% 4.c. % of HWs who have been supervised in IMCI within the last year22 HFA and self assessment performance scale 28% N/A 40% 4.d. % of HWs who have been supervised in IMCI regularly. HFA and self assessment performance scale N/A 89% 40% 4.e. % of HWs23 who have MOH policy and guidelines/protocol on antenatal and obstetric care services. HFA and self assessment performance scale 44% N/A 65% 4.e. % of facilities with MOH policy guidelines on ANC and obstetric services. HFA and self assessment performance scale N/A 75% 65% 4.f. % of facilities who provide information on nutrition& hygiene, ITNs, breastfeeding, STI/HIV/AIDs prevention, warning signs, post natal care. HFA and self assessment performance scale 94% 100% 98% 4.g. % of HWs who received training in AMTSL in the last 3 years HFA and self assessment performance scale 79% 43% 90% 4.h. % of HWs who filled out a self assessment in the last year Self assessment performance scale 0% N/A 25% 4.i. % of HW self assessments that showed higher ratings over the past year self assessment performance scale 0% N/A 18% Objective 5: Build local organizational capacity to manage health schemes 5.a. BOD capacity Review of BOD meeting minutes Minimal24 Promising to Strong (June 2010: 8.4)25 Strong                                                              21 This indicator was not reviewed in the final. Instead, stock outs were reviewed over a 6 month period for individual items rather than overall stock outs for the shorter period of 30 days. 22 “Within the last year” was changed to “regularly” from the DIP to the final evaluation. 23 “Health Workers” was changed to “facilities” from the DIP to the final evaluation. 24 See “Performance Scale” table on page 15 of the project’s MTE report 25 August 11, 2010 BOD Minutes    18 Indicators Data Source Baseline Value (%) Final Value (%) Final Target (%) 5.b. Size of UHC membership Scheme Manager Monthly Reports Minimal26 (January 2006: 1,783)27 Minimal to Emerging (January 2010: 4,227)28 Strong 5.c. Ability of scheme to financially cover services & administration costs Self Assessment and UHIS reports Minimal Of the 29 UHC groups functioning in July 2010, 16 operated at a surplus, 9 operated at a deficit & 4 did not report for the month, resulting in a net surplus: 485,008 Ugandan Shillings for that month.29 Strong                                                              26 See “Performance Scale” table on page 16 of the project’s MTE report 27 Jennifer Stockert August 18, 2010 email. 28 Jennifer Stockert August 18, 2010 email. 29 Written report to the UHC BOD for the month of July 2010 reviewed in their August 11, 2010 meeting.    19 E. Discussion of Results Objective 1: Reduce incidence of malaria in Bushenyi district for children under 5 and pregnant women How project implementation led to achievement: The project distributed 5,001 insecticide treated nets (ITNs) for free and sold 174 through year three. Initially each woman who delivered her baby with the assistance of a skilled birth attendant was provided with an ITN. The timing for the distribution was later switched to the fourth ANC visit, which was voiced as a concern by the MOH and community since the pregnant woman would not benefit from ITN protection through most of her pregnancy. 30 The project’s ITN distribution was stopped in year three due to the recognition that this was not a sustainable incentive for increasing ANC coverage and with the government’s announcement that it would be doing its own national mass distribution campaign, which was delayed. As a result, ITN coverage dropped from 80% at the project midterm to 61% by the final KPC, which still represented a significant increase from the baseline where no ITNs were found in the community. The coverage rate will likely increase again due to the mass distribution campaign for pregnant women and children under five that occurred in August/September 2010 at the end of project year five. While both the targets for ITN use have been exceeded, demand for ITNs continues as both health workers and VHTs report that the most common question they are asked is about the future availability of ITNs. The project used health education and promotion through project-trained volunteers (initially CORPS and later VHTs) and health workers to promote the proper use of ITNs by high risk groups, danger signs signaling the need for medical treatment and the importance of IPT at ANC. Clean water stations were established at health facilities for observed IPT uptake and ITN kits were provided to VHT parish coordinators for demonstration and community counter demonstrations during health improvement sessions. Health workers consistently stated that individuals with malaria symptoms would usually wait until cheaper and less effective local traditional treatments were exhausted before seeking medical care prior to the start of this Child Survival project, resulting in a more severe case. Now they were seeing fewer severe cases of malaria at health facilities because individuals were seeking care earlier upon recognition of symptoms. Several reasons were cited:  The availability of prepaid insurance through the UHC, which means there is no out-of – pocket charge for the consultation and treatment;  The increased promotion of ANC, including IPT, which has led to higher levels of attendance and decreased numbers of malaria cases in pregnant women;  Broader awareness of the malaria-related danger signs that signal the need for prompt medical care;  Stronger referral links between the VHTs working at the community level and the local health facilities; and,  The increased availability of ITNs for pregnant women and children under five.                                                              30 Field project staff and health workers said that ITNs were given at the fourth ANC visit as an incentive to attend all four ANC visits, which is in contrast with page 18 of the midterm evaluation report, which stated, “Change in ITN distribution policy from distribution to babies delivered with skilled birth attendant to distribution at 1st ANC so as to protect the mother during pregnancy when she is most vulnerable.”    20 Contextual factors: In year four of the project the MOH switched its focus from C-IMCI, which the project was using, over to its VHT manual which stresses use of ICCM for the initial treatment of fevers, especially in locations such as Bushenyi where physical access to health care facilities is a major challenge for large parts of the population. This change, though beneficial in the long run for Uganda, necessitated the retraining of all the health workers and volunteers in the district during the last year and a half of the project. Another challenge the project and communities faced were stock-outs of anti-malarials at health facilities and ITNs that were necessary to meet the demand the project was creating through its BCC strategies. This challenge was not unique to Bushenyi District, but was also mentioned by the DHO from neighboring Rukungiri District who is also experiencing stock-outs because the National Medical Stores do not always have the medicines and supplies available. 31 Health facilities run by faith-based NGOs were especially hard hit by the Government of Uganda’s new policy of no longer providing medicines to non-government facilities, which has increased shortages and therefore, provider costs. It is hoped that with ICCM more fevers can be treated at home so fewer clients will need facility-based care. Role of key partners: To a large extent the success of this intervention rested on the actions and resources of the DHT who the project advocated with to take over ITN distribution and the training/refresher training of health sub district supervisors on stock management. The training and supportive supervision of CORPs, VHTs, VHT parish coordinators and health workers has been done jointly by UHC, district trainers and health facility staff and is being transferred over to health facility staff and the DHT at the end of the project. VHTs and VHT parish coordinators will continue to be a part of HP’s Malaria Communities Program through 2012. Overall design factors that influenced results: Success in several of the project indicators on this and the other two community-dimension objectives (two and three) required that partners provide supplies and medicines, as well as additional services to meet the demand the project sought to increase.32 Specific to malaria, the promotion of ITNs and the importance of IPT at ANC needs to be timed to coincide with the availability of an accessible supply, otherwise, clients can get easily frustrated, and volunteers and health workers can lose credibility. Objective 2: Reduce incidence of diarrhea in Bushenyi district for children under 5 How project implementation led to achievement: The project sought to reduce incidence of diarrhea through VHT-organized health education and BCC sessions covering the warning signs of dehydration, the importance of using safe drinking water, promotion of hand washing, the introduction of hand washing stations at each toilet, the demonstration of ORS preparation, and improved sanitation practices. The project also introduced and distributed PUR sachets as available, established ORT stations at health facilities for demonstration to clients as part of health education at the health facility and as an easily accessible point for clients when they                                                              31 Notes from Dr. James Mukankusi’s August 2, 2010 interview with Dr. Zepher Karyabakabo, DHO for neighboring Rukungiri District. 32 See indicators 1.b-1.e, 2.c-2.k, 3.a and 3.f in the M&E table, each of which required some mix of supplies, medicines, and/or services from outside the project budget in order to achieve the indicator result.    21 come in with a child who has diarrhea as they await treatment/admission – first aid. The project also promoted the use of WaterGuard because it is locally available in drug shops. One of the most dramatic findings was the drop in incidence of diarrhea in children from the baseline KPC (55%), to the midterm (42%) and to the final (33%), with similar changes seen in UHC members and non-members. This is consistent with the increased proportion of households using an improved water source, which increased by 14% points, and the number of households with access to a covered pit latrine, which quadrupled from 19% to 77%. Treatment of children with diarrheal disease could not be accurately analyzed with the KPC results because of the insufficient sample size, however, health workers said that they were seeing fewer serious cases of dehydrated children being brought to health facilities. The difference found between UHC and non-UHC members in their knowledge and behaviors was relatively minor, which is understandable given that the focus of this intervention was on broadly targeted health education that was not restricted by UHC membership. Contextual factors: The effect the transition from IMCI to VHT and the stock-outs had on the project’s malaria intervention was similar to that for its diarrheal disease intervention. Inconsistent supply of ORS, PUR and WaterGuard was not able to keep up with the increased demand created through the community demonstrations done by the health volunteers. PUR and WaterGuard were not available for distribution by the end of the project, just for demonstration by the VHTs. HP did not have control over the social marketing or supply of these products. The project planned to promote zinc distribution, but was not able to do so since it was not approved by the MOH or included in the VHT training materials. Role of key partners: The training and supportive supervision of CORPS, VHTs, VHT parish coordinators and health workers has been done jointly by UHC, district trainers and health facility staff and is being transferred over to health facility staff and the DHT at the end of the project. Objective 3: Increase % of pregnant women receiving improved ANC, delivery and post partum care How project implementation led to achievement: The project’s MNC intervention focused on increasing attendance at ANC and PPC; promoting the planning for a safe delivery with a skilled attendant; increasing community awareness on the importance of RCT and PMTCT; training health workers and volunteers on MNC; and distributing Mama Kits. Attendance at ANC could not be accurately assessed with the KPC survey results. However, significant improvements in demand for RCT, counseling on breastfeeding, and the attendance of a skilled health professional at delivery that did come clearly through the KPC results implies that ANC utilization probably improved substantially as well during the project. The difference in results achieved between UHC members and non members was more dramatic for the project’s MNC intervention than for either the malaria or diarrheal disease interventions. Member use of a skilled health professional at delivery surpassed that for non-members by 16% points, which is probably a direct result of UHC member benefits that cover the associated costs. This is consistent with what was found in the percentage of UHC members who delivered at a    22 health facility, which was 68% for members and 52% for non-members. These are very important accomplishments that will have direct impact on maternal and newborn mortality. Contextual factors: The primary barrier to ANC, PPC and delivery at a health facility is distance, compounded by limited transportation, which could account for the 20% point increase in the percentage of members counseled on developing transportation plans to a delivery place as compared to7% points for non-members. To a lesser extent, recent stock-outs of ITNs and Mama Kits, both of which were used as incentives to attend ANC, could have limited the potential improvements in ANC attendance. The project also used its BCC activities to promote male interest in and support for perinatal care, as this was recognized as a cultural norm that needed to change as a key to increasing the effectiveness of reproductive health services. Role of key partners: The training and supportive supervision of CORPS, VHTs, VHT parish coordinators and health workers has been done jointly by UHC, district trainers and health facility staff and is being transferred over to health facility staff and the DHT at the end of the project. Objective 4: Improve health care management especially for WRA and children under 5: How project implementation led to achievement: The project focused on training health facility staff in resource stock maintenance; standard case management (SCM) for malaria and diarrhea following health worker-IMCI (HW/IMCI); and SCM for ANC, PPC and safe/clean delivery including active management of third stage of labor (AMTSL). A total of 32 Health Sub-District Supervisors were trained in stock maintenance and 203 health workers were trained in IMCI and MNC. (See Annex 7 for a complete listing of CORP, VHT and health worker training.) MOH training materials were used and MOH policy followed throughout. Staff self￾assessment results were used to inform follow up with health workers so their skills could be solidified through refresher trainings and quarterly joint supportive supervision. These were initially carried out by project and DHT staff, and then later taken on entirely by the DHT. Contextual and design factors that influenced results: This was the most difficult intervention to assess during the final evaluation because of the changes in some of the indicators from the baseline to the final, resulting in part from changes in government policy. As noted above, the primary reasons for the stock-outs could not be addressed through the resource stock maintenance training because they were due primarily to country-wide shortages. In 2005 MOH policy was to provide health facilities with PHC funds to procure the medicines and supplies they needed (pull method) on a regular basis making the skills in stock management a very important strategy to overcome the challenge of stock outs. Health facility in-charges were responsible for their orders and timely delivery. This policy was changed in 2008. Supplies are now received from National Medical Stores. However shortages are common and deliveries are not consistent with health facility orders and needs.  High turnover in staffing and the cost and difficulty of transport will likely continue to challenge the DHT in sustaining the capacity that has been built in its health workers.    23 Role of key partners: The training and supportive supervision of health workers has been done jointly by UHC and health facility staff and is being transferred over entirely to the DHT at the end of the project. Objective 5: Build local organizational capacity to manage health schemes How project implementation led to achievement: The purpose of this intervention was to establish a sustainable, locally-run health insurance plan that would provide health care services to membership groups in exchange for payment of quarterly premiums and co-payments. Plan membership grew by 237% net between January 2006 and January 2010. Year Total Plan Membership at January 1 2006 1,783 2007 3,985 2008 3,787 2009 4,099 2010 4,227 UHC members listed a variety of reasons they decided to participate in the cooperative. In focus group discussions they said that they had saved money overall since the cost of premiums and co-pays was less than what they normally would have had to pay out-of-pocket for care. Some told stories of families having to sell land to cover emergency medical bills so membership had calmed their fears of financial ruin. Increased member knowledge about the danger signs that signal the need to seek treatment and the availability of care that has already been paid for together has led members to seek care earlier before health problems worsened. Some said that membership in the cooperative gave them a sense of ownership in the plan since they had a channel to convey complaints and suggestions to their providers when they paid their quarterly premiums through their group leader, elected members to the UHC Board of Directors, and helped select care providers. In fact, greater contact with their health care provider also translated into increased opportunity for learning about health. Consistently throughout, stakeholders appeared to have a basic understanding of the services covered in the plan and how the plan worked, which is encapsulated in a set of four training manuals developed by the project in 2010 for providers, volunteers, stakeholders, and new member groups. The project enlisted 12 NGO-run, private health care providers that are responsible for providing UHC-covered health care services to the 29 UHC member groups made up of a total of 4,227 individuals by the end of the project. One health care provider had to be dropped for not meeting the terms of their agreement with the UHC which appeared to be discovered and resolved expeditiously and appropriately by the UHC and HealthPartners. The health care providers noted several benefits to their participation in the UHC, that early treatment reduced hospital costs for providers and the prepayment of premiums helped rationalize their revenue stream so they could more accurately budget and plan for emergencies and outbreaks. The provision of project supplied ITNs and Mama Kits for women attending ANC, ORT corners, and the availability of educational materials also drew people to health care facilities when they were available.    24 The key to the long term viability of the UHC is a growing membership. They have undertaken a ‘no missed opportunities’ approach to marketing their health plan, which now involves not only project staff, but BOD members, providers, VHT and current UHC members themselves, who reach out to enlist employers, schools and other potential community UHC groups. Under the direction of the UHC Board of Directors a surplus has been achieved in their reserve fund and for a majority of the providers, where premiums plus co-payments are now greater than the costs of treatment and provider membership fees. Contextual factors: The UHC has faced numerous challenges. Even though nearly everyone had heard horror stories about families who had suffered financial ruin due to the cost of dealing with medical emergencies, the concept of risk pooling or prepaid insurance was foreign to the area. As a result, health workers reported that some UHC members requested that their premiums be refunded at the end of a coverage period when they had not used any health care services and others would seek services when they were not ill in order to maximize their use of the benefits. When providers do notice a family or community accessing services for the same problem over and over again, the Data Entrant will notify the VHT/VHT parish coordinators who will follow￾up with further assessment and provide health education as needed. Another challenge is a majority of the district is living on subsistence agriculture which makes it difficult to consistently pay their premiums, which results in fluctuations in membership. Some communities have developed savings and loan associations to help but these are all in their earliest stages. Many expressed the hope that the UHC could add chronic conditions to the coverage and the costs of care at a referral facility when the primary facility lacks the capacity and/or supplies to provide the necessary treatment. Both issues have been discussed and researched by the project extensively. The concern is that a disproportionate number of individuals with pre-existing chronic conditions would seek coverage (i.e. adverse selection), which would result either in significantly higher premiums for all members or could even jeopardize the cooperative’s financial base. This is likely to remain an on-going debate and challenge for the UHC. As Uganda continues to develop and prosper, the country’s disease profile will include more long￾term chronic illnesses, such as cancers, diabetes and heart disease. A recent change in GOU policy means that private health facilities run by faith-based organizations are no longer eligible to receive subsidies on medicines and supplies from the National Medical Stores, which is going to result in an increase in costs for providers and potentially member groups. However, it is possible that the proposed expansion of ICCM could result in cost savings for providers that could help balance this out, as more cases of fever and dehydration are dealt with effectively at the household level without having to be brought into health facilities. Role of key partners: The roles of the key stakeholders are included in the graphic on the next page. In addition, the project’s UHC Manager is on the Board of Directors and the project coordinates closely with the Uganda Community Based Health Financing Association (UCBHFA), a national organization focused on advocacy, capacity building and sharing lessons learned. UHC is also a member of Community Health Financing Association for East Africa    25 (CHeFA-EA), a regional organization that brings together community health insurance organizations to share experiences, build consensus and increase capacity of the members. One NGO, Compassion International, works on HIV and covers the cost of UHC premiums for a group of orphans and vulnerable children (OVC). The DHT supports UHC efforts to ensure equal benefits for UHC member groups, VHT groups, and health care providers within participating districts by attending events and assisting with communication between parties. The community leaders’ role is to support UHC efforts by attending events, encouraging people to attend healthy behavior sessions led by VHT, encouraging groups to join UHC for improved health and assisting with accurate communication about the roles and responsibilities of UHC membership. The health workers’ role is to collect, compile, analyze data and share the results with the stakeholders. HW also plan for quarterly meetings and supervise VHTs through the health facility primary health care outreach activities. VHT roles are included in the graphic on the page below under volunteers. Role of key partners: The roles and responsibilities of the key stakeholders that are directly involved in the UHC are summarized in the following diagram.33                                                              33 UHC Stakeholders Handbook. Page 14.    26 Overall design factors that influenced results: The target was to have 14,000 WRA and children under 5 enrolled as members of the UHC by the end of the project, which would constitute between one to two percent of the total district population of 800,000+. It achieved 4,226 members by the end of the project including men, women and children, which is less than one percent. However, the KPC results that demonstrate improvements in non member Member‐elected Board of Directors Enforce stakeholder roles and responsibilities to sustain benefits and financial balance. Health Care Providers  Register and train new members  Take and/or collect photos to create ID cards for members  Keep IDs on file and check them when members come for care  Employ data entrant to track premiums and treatment costs  Contribute 10% of annual surplus to reserve fund each January  Only charge covered benefits to the cooperative    Attend quarterly meetings with groups to share data and address challenges  Attend the annual general membership meeting in January Focal Volunteers    Follow member roles and responsibilities and  Attend quarterly meetings with health workers and volunteers  Supervise volunteers and turn in premiums, current membership lists and data tracking forms.  Hand out cooperative registration forms. Volunteers  Follow member roles and responsibilities and  Train and support healthy behaviors at the village level and turn in data tracking forms  Attend quarterly meetings with health workers Group Leaders  Follow member roles and responsibilities and  Turn in current membership lists and member premiums in to health facility by the first day of Jan., Apr., July and Oct.    Members  Pay premiums to group leader by the 22nd day of the month in December, March, June, September  Contribute to reserve fund each January  Attend volunteer led sessions to improve health  Attend provider training, quarterly meetings and the annual general meeting Reserve Fund  Board meetings  Annual general meeting  Annual audit    27 knowledge and practices seem to imply a ‘coattail’ effect, where non members benefit from the project interventions as well, probably due most directly to the fact that both community and UHC volunteers were trained, and health workers from UHC and non-UHC providers received training and support supervision. Another challenge is the natural tendency for the UHC to become overly complicated in its policies, the services covered, fee structure and other factors, especially as it tries to expand its membership and accommodate the unique needs of different groups. This is already starting to become apparent as different groups are paying different premiums. While this provides an opportunity for groups in need to participate when they could not otherwise, it adds to the complexity of the plan and thus the burden and costs of administering it.    28 F. Discussion of Potential for Sustained Outcomes, Equity, Community Health Worker Model, and Global Learning F.1. Potential for Sustained Outcomes Sustainability Design and Goals: The project sought to achieve sustainable outcomes by building capacity in three interconnected dimensions that if successful, could improve the health of women and children now and into the future:  Community/Social would increase community knowledge about health, improve practices and expand access to appropriate health care services through BCC campaigns implemented at the community level by community health volunteers (CORPs and VHTs).  Health Services would build the capacity of health care workers and health care systems to provide more effective services at health facilities.  Local Organizational would strengthen and expand the membership of the Ugandan Healthcare Cooperative, which would increase member access to health care services and limit their exposure to financial risk. An adapted version of the CSSA framework was used to track and graphically represent progress on each of these dimensions over time, with each dimension measured by two proxy indicators as shown in the table below: Dimension Indicator # Data Range Baseline Mid 2007 EOP 2010 Goal Community Social % of children under 2 with fever in the last two weeks 1 63% 0% 44% 39% 46% 19% # of stakeholders including DDHS official, health facility administrators, health workers, leaders and members at the UHC Annual General Meeting. 2 0 400 0 99 130 Health Services % of HWs who have been supervised in IMCI within the last year 3 0% 100% 28% 93% 89% 40% Human Development Index which is a composite index including life expectancy, education and GDP all weighted by 1/3 to derive HDI. 34 4 0.3 0.5 0.38 0.398 .422 Local Organizational UHC annual board meeting notes address at least 1 MCH issue and include a balanced budget 5 1 5 1 2 3 5 # of UHC plan members. 6 3,500 14,000 1,783 4,075 4,226 14,000 The project achieved positive improvements on each of its sustainability indicators and if this trend can continue there is a strong likelihood that many of the project accomplishments will continue, as well. It is important to recognize, however, that this instrument measures progress                                                              34 International Human Development Indicators—UNDP    29 to date and not beyond the life of the project, so it is not a direct measure of what will continue once the project concludes. Status of the Project Phase Out Plan: Discussions have already started with the DHT, the UHC Board and at the quarterly VHT meetings about sustainability and what happens after the conclusion of the current project. The issue of phase out needs to be considered from each of the project’s three dimensions: Community/Social: The final evaluation KPC survey found evidence of positive improvements in several areas of knowledge, practice and access to care at the community level, which can last beyond the life of the project. The key to this happening is a vibrant VHT network that can not only continue to support and strengthen community level knowledge and practices, but also link the communities to the public and private health care services available in their areas. With the redistricting of the project area and the MOH’s switch from C-IMCI over to the VHT manuals, there is an opportunity and commitment by local authorities to continue to train and support VHTs with dedicated district budgets and supervision. The VHTs interviewed during the final evaluation expressed a strong desire to continue their work and asked for a formal exit strategy that will help ensure they can maintain their skills that have been strengthened by the project and offered suggestions for how that could be accomplished. The project has already taken several steps to see that this happens – training and supervising the VHTs in conjunction with the health workers and recently transitioning this responsibility over to them, as well as advocating with political leadership to incorporate VHT support into the plans and budgets in the new districts. Health Services: The district received funding from the national MOH to support Health Sector Strategic Plan II and is pursuing a strategy of decentralization, which will result in shifts in authority and responsibility for planning, administrating, financing and overseeing public health services from the national MOH over to district governments. The project’s training and supportive supervision in IMCI, MNC and stock management should complement these changes, however, for the time being the project has been asked to step back from this responsibility until the GOU completes its planning. In addition, each of the four newly formed districts has received funding and is in the process of creating a district-level government. This in itself is a positive step as it has the potential to allow the new DHTs more direct access to a smaller population living within a smaller area. Local Organizational: The project has strengthened the capacity of the UHC BOD to administer the health insurance scheme and started turning all oversight responsibilities over to the BOD to ensure a smooth and seamless transition by the end of the project. A 40-page manual was developed to assist in this process, Community Health Financing: The Uganda Health Cooperative Model Stakeholders Handbook, Sustaining a healthy community through partnership. This manual explains how the cooperative should be managed into the future, covering the key responsibilities, factors, requirements and resources necessary for sustaining the cooperative, including the necessary forms and templates for reporting, monitoring/supervising, marketing and administering the UHC. While written specific for Bushenyi, it is an invaluable resource on the development of prepayment, insurance plans in populations with limited financial resources, with useful lessons learned and best practices that are readily adaptable to other locations.    30 The UHC Board faces some potentially complex challenges, especially as a national task force investigates options for national health insurance and makes policy recommendations to the government that can affect UHC services and associated costs. While representatives from the UHC Board have been able to present the position of the UHC and make suggestions that have reached the national task force, ongoing tracking and reacting to these changes is going to remain difficult or impossible with the board situated in Bushenyi while the changes are being considered and adopted in Kampala. UHC’s continued participation on the UCBHFA can help. Project Success in building financial sustainability: Three of the project’s primary activities have on-going costs tied to them:  Training, support and supervision of the VHTs: The newly forming district administrations have verbally committed to incorporating VHTs into their future work plans and budgets.  The additional supply of ITNs, antimalarials and other supplies and medicines necessary to meet the increased demand for project-related preventive and curative health care: As noted above, even though the project has provided training in stock management for health sub-district supervisors, the lack of consistently available essential supplies is nationwide in scope and therefore, goes beyond the district and this project to address.  The administrative costs of the UHC BOD: The project has factored BOD administrative costs (meetings, allowances, transport, etc.) into the UHC financial plan so the gross revenues from insurance premiums, co-pays and provider fees will cover both the costs of the health care services as well as these administrative costs. As long as the UHC can operate at a surplus, these costs should continue to be covered. Project-built demand for services and mobilizing the community to influence health system planning: The project has sought to increase the demand for medical services and supplies primarily through its BCC campaigns and the marketing of health insurance through the UHC. Health education messages on the danger signs that require prompt medical care for malaria and diarrheal disease; the importance of ANC, PPC and a health facility-based delivery, have all contributed to the increased utilization of care services. However, as noted by administrators, health workers and the community alike, the health care system has not always been able to respond to this increased demand, which can lead to frustrations for clients and undermine the credibility of the VHTs. The community has two new channels to influence health system planning of both public and private health care providers. First, an inherent benefit of a well-organized cooperative is its ability to generate a compelling voice to advocate for the concerns and desires of its members that is ideally stronger than the sum of the individual member voices. Through their leaders, UHC group members can communicate to the UHC providers, which are all privately run NGOs. Unresponsive providers understand that they risk the loss of a consistent revenue stream if they choose to ignore the concerns of their UHC member clients. UHC members also have a voice through their election of members to the UHC Board of Directors. For the public sector, the VHTs can play a role in communicating community needs and input through their VHT parish    31 coordinators and then on to the health facilities and administrators during BCC sessions, supervisory visits and clinical care. Capacity building of local partners included joint training and supervision of volunteers and health workers; technical assistance and coaching for the UHC Board; development of user manuals for UHC providers, stakeholders, members and volunteers; and ‘on-the-job’ training by involving stakeholders in planning, administering and monitoring the project. The phasing of the project into each sub county allowed it to gradually step back as local health workers, administrators and VHT parish coordinators took on greater responsibility. The project could then track on how the initial sub-counties did once they were on their own and return to help fill in the gaps as needed. It also provided them with a list of lessons learned that they were able to use when implementing subsequent phases. Capacity building of the UHC Board required a longer view, simply because of the level of complexity inherent in the UHC, the region’s lack of familiarity with health insurance, and the time required for a board to become effective and motivated in their roles and responsibilities. Board members report developing skills in marketing, data collection, data analysis, decision-making, supervision, community mobilization, health education, planning and leadership. Components that will continue after the conclusion of this project with USAID support: The project submitted an application to the USAID Office of Development Partners for an expansion of the cooperative model linked to maternal child health and are waiting to hear back at the time of the writing of this report. At the end of project year three, the project received a four-year grant from US President’s Malaria Initiative focusing on the prevention and treatment of malaria, which will go into 2012. Should the extension grant not be awarded, HealthPartners’ Malaria Communities Program (MCP) provides an opportunity to continue to expand the child survival project’s malaria objective while remaining connected with the UHC Board so technical assistance can be provided to them, as needed. F.2. Equity Types of equity addressed: The question of equity is most directly relevant to the insurance scheme. The common challenge for any health insurance scheme is to keep it as simple as possible so members, providers and administrators can understand it, and administrative costs are kept at a minimum. But this needs to be balanced with serving those with unique needs and limited financial resources, especially in places like Bushenyi. Achieving equity often means that exceptions, incentives, cross-subsidies and other financial and administrative mechanisms need to be introduced so individuals and groups with unique needs have equal access to the benefits of membership. This adds complexity, which brings with it the potential for increased administrative costs, inefficiencies and inconsistencies. To address equity and also expand plan membership, the UHC Board has allowed some groups to join under different terms and at different rates, and allowed organizations to cover or subsidize the premiums for vulnerable groups such as people living with HIV/AIDS and orphans and other vulnerable children. This is an important ‘tool’ for helping to achieve greater equity in access, but again, it brings with it a system that can be challenging to administer and track. This is not necessarily negative, but it does need to be approached carefully and with forethought if the health insurance system is to grow into a long term operation.    32 Equity was also addressed in the project’s BCC plans, which considered the needs of women, mothers-in-law, youth and men separately to ensure that the IEC strategies were effectively targeted so each group would have access to project interventions. The project also sought gender equity in the recruitment of VHTs and achieved a gender balance of close to 50%, exceeding its target of at least one third of trained VHTs being females. Use of evidence to target interventions or strategies to address inequities: The UHC’s approach to addressing inequities appears to have been done on an informal case-by-case basis. While Board decisions are made by consensus, these do not appear to be based on an explicit set of criteria for determining what exceptions should be made for each group. Measurement of changes in inequities and results achieved: No formal mechanism for measuring changes in equities has been used and is probably beyond the practical capacity of the project. F.3. Community Health Workers The Project’s CHW model and its contribution to success: The project’s initial CHW model involved the training and support of volunteer Community Owned Resource Persons (CORPs). Each CORP was either chosen by their community or elected by his or her UHC member group. The project then supported the training of CORPs on C-IMCI, MNC and other health topics so they could organize and lead community mobilization and health education sessions, often in conjunction with other activities where they could draw the largest audience, such as church services, community events, and school activities. (See annex 7 for a list of volunteer trainings.) As a means of verification of sessions and match contribution, each CORP was also responsible for requesting that all the individuals attending each session sign-in. The sign-in sheet was counter signed by the local authority (LC1) and then reported by the CORP on a monthly basis so performance could be tracked. This proved difficult in some communities where some thought the CORP was getting an allowance. Late in year three the project changed its CHW strategy from CORPs over to VHTs and VHT parish coordinators in order to be consistent with the national MOH’s switch from C-IMCI over to its Village Health Team Model, which expanded the home-based care provided by volunteers through the adoption of ICCM. VHT parish coordinators are the link between the local health facility, local authorities and the VHTs who work directly out in the community. The VHT parish coordinators are responsible for training and supporting VHTs; collecting and consolidating reports from the VHTs for submission to the health facilities; and community mobilization and networking. Each VHT covers 25-30 households or a UHC member group, which is more realistic than what was expected of the original CORPs, each one being responsible for covering a parish. VHTs are responsible for organizing BCC sessions, doing home visits, providing very basic primary health care services, and reporting to their VHT parish coordinator. Each VHT parish coordinator meets with his/her VHTs monthly and with the health workers, local leaders and other VHT parish coordinators on a quarterly basis where they get feedback, training and the opportunity to share experiences.    33 Retention is a challenge for all community-based volunteer programs. At the final evaluation 65% of the CORPs trained in Phase I had held at least one BCC session within the past four months; 65% from Phase II; 80% from Phase III; 83% from Phase IV; 100% from Phase V; and 43% of the UHC member group CORPs. The project recognized the need for incentives to maintain volunteer motivation and involvement. Each volunteer was provided with a t-shirt upon starting training so they could be identified by their community or their peers in their UHC member group. Exceptional volunteers, who demonstrated a strong commitment to the program and their clients, received a bicycle from the project at their six month refresher training. Commitment was determined by consistent recordkeeping and reporting; the number of health education sessions the volunteer organized and the number of attendees who participated; and, consistent attendance at trainings and quarterly supervisory meetings. As a further incentive the project also offered discounted premium rates for membership into the UHC plan to all VHTs and VHT parish coordinators. The project also facilitates quarterly meetings involving the volunteers and health workers where they can share their experiences, report on results, receive training on new health topics and come to consensus on next steps and new strategies. The final evaluation found much support for the volunteers from both public and private health facility staff, as well as the communities and groups they are serving. Local authorities have also committed verbally to including VHT support in their district health plans and budgets. The project has worked to change initial misconceptions that the volunteers were being paid over to an acknowledgement by the community that it has an important responsibility to support their VHTs. The VHTs voiced a strong commitment to continue their work after the conclusion of the child survival project, including their quarterly meetings, the trainings, and the mobilization and education of the community about health. VHT Supervision beyond the end of the Project: All volunteer supervision was done jointly by health facility and project staff throughout most of the project. With the creation of four new districts and DHTs, and the MOH’s transition from C-IMCI to VHT, responsibility for supervision was transferred per the VHT model to the DHT, health workers and the VHT parish coordinators with support by community leaders who have been included in quarterly meetings at their request. F.4. Contribution to Global Learning HealthPartners and its Bushenyi Child Survival project bring two perspectives that are new to the world of child survival. This is the first and only attempt the authors are aware of where a locally-run health cooperative has administered a prepaid insurance program that covers primary health care and delivery services in a setting with significantly limited resources. While from one viewpoint it seems counter-intuitive to set up a prepayment insurance system in a location where few families earn more than a few dollars a day and the government is mandated to provide many of the same services for free, this project has met with enough success to warrant further exploration, analysis and operational research. With a monitoring system that is technically sound, an extension grant could answer the fundamental question posed in the project goal of whether a health insurance scheme, even if it involves a relatively small portion of the overall population, can have a lasting impact on the health status of women and children living in those same communities for members and non members.    34 HealthPartners comes from a very different background and experience than most of the other CORE member organizations and CSHGP grantees. While it is a cooperative by legal definition, HP’s background is from the private business sector where it has a proven track record in sustainable health care financing while improving the health of its members and generating profits for the providers it works with in the United States. As CORE and its member organizations continue to branch out to forge public-private partnerships, corporations such as HealthPartners can provide a very practical and grounded perspective on what works, what the private sector can contribute and how they can be engaged. Recommend for scale up? Working within the parameters and the goal and objectives of a “traditional” Child Survival project, this project has established a prepaid insurance cooperative that covers the primary health care and maternal newborn care services for over four thousand people – something that has never been attempted in this type of setting before. It has developed a locally-run governance structure to oversee it and recruited a dozen health care facilities to provide the direct services – which are benefiting from their participation. Even if this cooperative only serves a relatively small percentage of the overall population directly in Bushenyi, these are people who would not otherwise receive the financial security that comes with insurance. The true potential demand for this coverage is not fully known in Bushenyi, where a recent survey found that less than half the population of the district who are not UHC members have heard about the UHC. It is also worth exploring the viability of such a cooperative in other types of settings, such as districts that have a larger urban population or more employers, which can naturally form into member groups.    35 G. Conclusions and Recommendations Successes:  The project established technical skills within the CORPs and later the VHTs and VHT parish coordinators, which have extended the reach of life-saving primary health care services into the communities and linked them more effectively to the formal, facility￾based health care services available in the District.  Through increased knowledge, improved practices and expanded access community members are able to do more at home to prevent and manage fevers and diarrhea, and they understand when medical care becomes necessary and where to go for it.  A health insurance plan has been developed, honed, and institutionalized that is now self￾run and has begun to function at a surplus. Data points to a positive impact on the health of women and children.  The project has proactively enlisted all stakeholders in marketing UHC membership, including health workers, community leaders, local authorities, VHTs and VHT parish coordinators, current UHC members, etc. The momentum is there for this to continue. Challenges:  UHC Board members said their greatest challenges have not yet been completely resolved. o The future success of the UHC is subject to a fluid national policy environment to which it has little or no input. The recent policy switch on the supply of medicines to faith-based private health care facilities is going to have an impact on provider operating costs, the UHC balance sheet and member premiums. It is not clear what other changes the Government of Uganda will undertake as it continues to grapple with how best to extend, enhance and finance health care services for its people through the national health insurance system which is currently under development. o The most consistently heard request through nearly every channel during this evaluation was community demand for including chronic diseases under the coverage, to the extent that some have left the plan for this reason.  Several of the project indicators and strategies rely heavily on the actions of other stakeholders for success, especially through the provision of supplies and medicines. When they do not produce, the project is challenged to reach its targets.  The project’s M&E system is not producing the data necessary to accurately measure several of its indicators.    36  Although much discussion and effort has gone into a smooth and gradual transition to local ownership the project lacks a phase-out plan that succinctly details the steps that need to be taken to conclude the project over the next few weeks. Recommendations:35 1. The project needs to advocate for an on-going commitment of support for VHTs from the communities they serve, the health providers they assist, and the local authorities they answer to each of the five districts. a. The redistricting of the project area provides an opportunity to advocate for the formal incorporation of the VHT structure into the government’s annual work plans and budgets. b. Beneficiaries need to be repeatedly reminded that VHTs are unpaid volunteers and their long-term value to the community depends on the support provided by both the health care system and the people they serve. c. Incentives such as discounted or waived UHC premiums, exemption from community work, and simple recognition should continue to be pursued, while new incentives need to be constantly explored, tested and scaled up when proven effective. d. Continued interaction and coordination between the UHC and the MCP is a unique opportunity to mobilize leaders at all levels in sustained support of the VHTs beyond 2012. 2. The request by the stakeholders (VHTs, HWs, UHC members and district/community leaders) for a detailed exit strategy specific to the future of the VHTs, implies a need for further discussion on how the transition will be finalized. The MCP provides an opportunity to strengthen consensus over the next couple of years so both are more comfortable with the plan. 3. A multi-sector approach should be initiated to create links to other NGOs and government programs for greater community access, especially involving men, for the sharing of health information during meetings of other sector programs, (i.e. poverty elimination, agriculture, etc.) and the potential of additional support to the volunteer structure. This can be done over the year through the end of the MCP. 4. The results of the final evaluation KPC should not be used for a baseline if Health Partner’s child survival project extension application is granted. The survey should be done over. Consideration should be given to using 30-cluster sampling rather than LQAS, especially if indicators measuring care practices of the sick child are included. 5. Several recommendations were identified for increasing UHC membership, some of which have already been considered and others are new. It is important to recognize the fluidity in the insurance market, as the Government of Uganda makes changes in the health care services it covers, new funding becomes available for different health care services, and new NGOs enter the ‘market’ to provide services at subsidized rates or for free. It is entirely                                                              35 At the writing of this report HealthPartners had not been informed by the CSHGP Unit on the status of their application for a five year extension grant. Hence this section includes recommendations that should be addressed regardless of the future of their project and those which should only be addressed if the extension application is granted or if it can be bundled in with Health Partner’s Malaria Communities Project, which runs through 2012.    37 likely that the UHC will need to change and adapt its package of services in order to stay in business. a. Incentivize ‘satisfied customers’ to act as marketers, promoters and mentors to new groups. b. Re-explore the option of covering care for chronic illnesses at least on a limited, trial basis. This might be necessary to the long-term survival of the UHC should the Government of Uganda or other international NGOs begin to provide UHC-covered services for free or at lower cost. The costs of these services could be covered through a tiered membership that involves different co-payments and/or premiums. c. Explore the option of allowing individuals join at a reduced rate from families. d. Identify other health service providers in major urban areas (i.e. Kampala, Mbarara, Masaka) as members requested coverage when traveling “out of area”. 6. HealthPartners should reapply for membership in the CORE Group, as this would be highly beneficial to both, regardless of its future involvement with the Child Survival and Health Grants Program. Changes in Grantee Organizational Capacity This was HealthPartners’ first USAID-funded Child Survival project and Uganda is the first country the organization has worked in outside of the United States. Both headquarters and field staff noted the steep learning curve at the start of the project that required learning CSHGP program strategies, tools and technologies, while starting up the project in the field. Staff appreciated the New Grantees’ Workshop and the Mini University for the DIP review, and report using the skills developed there not only in designing this project, but also their subsequent cooperative development grant and malaria control program plans. The project’s field staff in Uganda completed an adapted version of CSTS’s Institutional Strength Assessment (ISA) tool at the baseline for the DIP and repeated it at the conclusion of each of the five phases of the project. The following table shows that scores remained generally high throughout. Latent Construct Baseline End of Project Management Practices and Governance 78% 86% Administrative Infrastructure and Procedures 80% 79% Organizational Learning 80% 90% Financial Resource Management 89% 86% Human Resource Management 90% 88% Average Score 83% 86% Field staff also said that participation in this project increased their skills in program planning, supervision, communication, training of trainers, computer software systems, budget management, data collection, analysis, behavior change communication, marketing and health scheme management.    38 Annex 1: Results Highlight: Opening the door to another source of innovation: USAID’s Child Survival and Health Grants Program (CSHGP) has contributed numerous technological and organizational innovations that have moved the science and practice of child survival forward since 1985 – the KPC survey, the BEHAVE Framework, data-driven program design and monitoring, to name a few. It has also moved our collective thinking forward on capacity building, sustainability, partnership, data-driven program planning and monitoring. The CSHGP, its 43 grantees and the CORE Group (an innovation itself) have all contributed by testing new approaches and sharing the results. Each new NGO has added to this mix, however, all had come from a similar background – the world of humanitarian, NGOs that trace their origins to faith-based institutions and charities. In 2005 the CSHGP cast its net further by providing its first grant to HealthPartners, a health maintenance organization. HealthPartners was one of the first consumer-governed health insurance cooperatives in the U.S. that has grown to 1.25 million members since 1957. It came from a ‘business’ background that balances member services with its financial health – an equally valid purpose and perspective. Health care financing is a universal challenge countries are now grappling with worldwide. As developing countries begin to explore options, a theoretical debate has ensued on the advantages and disadvantages of public, private not-for profit, and for-profit health insurance.363738 It is critical for NGOs and governments alike to understand the benefits and costs of each of these approaches, whether the government is going to administer its own public program, regulate a private insurance industry or some mix of both. While the mix is likely to be different from country to country, the lessons learned and potential pitfalls are universal. HealthPartners’ Ugandan Health Cooperative (UHC) in Bushenyi District is one of the very few trials of a not￾for-profit health insurance cooperative in a rural, African location and it is the only one that has sought to determine the impact a UHC-like model would have on child survival interventions. The UHC model showed several signs of success. Premium-paying membership expanded from 1,783 in 2005 to 4,227 in 2010, a 237% increase. It has also established and trained a Ugandan Board of Directors that has taken on responsibility for overseeing the Cooperative and enlisted a dozen NGO-run health facilities to administer the health care services, the majority of which are functioning at a surplus. Membership has improved utilization of essential health care services. The percent of women who delivered with a skilled health professional increased from 47% to 68% and RCT increased from 34% to 96% – both critically important indicators for maternal and child health, and both significantly higher than the rates found in the non-member population. The UHC project has demonstrated the potential to benefit the people of Bushenyi directly, as well as other organizations focused on developing financially sustainable health care programming in Africa. A recent survey found that over half of Bushenyi’s population has yet to learn about the UHC. Further, the UHC has the potential of yielding much more useful lessons learned and best practices on approaches to health care financing as it strengthens and hones its data collection systems. HealthPartners brings a unique background and expertise to do this.                                                              36 Pauly, M. et al, Private Health Insurance in Developing Countries, Health Affairs, 25, no. 2 (2006): 369-379. 37 Drechsler, D. and Jutting, Johannes, Is there a role for private health insurance in developing countries?, OECD Development Center, Paris. 38 Community-Based Health Insurance Schemes in Developing Countries, WHO, 2003.    39 Annex 2: List of Publications Relevant to the Project Author/Presenter Title Venue Date George Halverson (Former CEO, HealthPartners) Book: “Health Care Coops in Uganda: Effectively Launching Micro￾Health Groups in African Villages” Perfect Paperback 2006 Joanne Silberner interviewing George Halvorson (Former CEO, HealthPartners), Scott Aebischer (Senior VP HealthPartners) and Joy Batusa (Director UHC) Radio Interview : “Tracking a Health-Care Experiment in Uganda” National Public Radio http://www.npr.org/templates/story/story.php?storyId=69155 66 January 2007 Dr. Grace Namaganda, (Director UHC) Presentation: “Community Financing for Health Services.” Social Health Insurance Workshop, Sponsored by HEPNET, the Health Economics and Policy Network in Africa, in Cape Town, South Africa May 2007 Dr. Grace Namaganda (Director UHC) Presented paper on social health insurance HEP Net Workshop, University of Cape Town, South Africa May 2007 Dr. Grace Namaganda, (Director UHC) Panel Discussion: “How Qualitative Research Can Successfully Inform Program Design and Implementation” CSHGP Mini-University, Johns Hopkins Bloomberg School of Public Health in Baltimore, MD. June 2007 Dr. Grace Namaganda, (Director UHC) Presentation: “Use of formative research in programming” CSHGP Mini-University, Johns Hopkins Bloomberg School of Public Health in Baltimore, MD. June 2007 Dr. James Mukankusi (Director UHC) Presented paper: HIV/AIDS and its Impact on Community Based Health Financing Seventh Annual Community Health Financing Workshop sponsored by the Uganda Community Based Health Financing Association and Community Health Financing Association for Eastern Africa (CHeFA-EA), May 2009    40 Annex 3: Project Management Evaluation Planning: The support and involvement of several individuals and groups was sought for the project’s baseline assessments and the development of the DIP, including staff from the USAID Uganda Mission, MOH, Makerere University Kampala, Africare, Population Services International (PSI), the DDHS, health workers and local CBOs. Africare and PSI brought years of experience planning and administering USAID-funded child survival projects, which was especially helpful. Both project and partner staff were comfortable with the DIP overall and the work plan specifically. They agreed that they were good references to turn to in answering questions about project implementation strategies and for monitoring progress against indicators. The DIP was an especially important resource for maintaining project continuity when there was turnover in senior management positions, as occurred with this project. Some changes were made in the wording of certain indicators and strategies within the DIP in order to coincide with changes in Government of Uganda policy, for instance the change from CORPs to VHTs, and the change in the project phases. Supervision of Project Staff: The supervisory system within the field and between the field and headquarters was sufficient and appeared to improve over the course of the project, especially as the CS Director was switched from being Kampala-based over to being field based, which allowed for better and more frequent communication and feedback within the district. Headquarters and the field have regular communication and headquarters staff made annual site visits at key points during the life of the project. All HP staff are reviewed annually using Corporate Responsibility Score Cards. Project staff worked closely with the District Health Team in planning, organizing and carrying out joint supportive supervision of health workers and volunteers, including CORPs, VHT parish coordinators and VHTs. Supervisory checklists were used to help ensure consistency in quality and with accepted standards. As the project has progressed, responsibility for VHT supervision has transitioned to Health Workers with support from local leaders under the leadership of the DHT. The primary challenge the district faces in sustaining this level of supervision is the limitation in human resources and transportation. Hopefully, the redistricting of Bushenyi will help to address this, as each DHT will be covering a substantially smaller service area. Human Resources and Staff Management: Each project staff person has a copy of the organizational personnel policies and an up-to-date individual Corporate Responsibility Score Card used to track their progress against the project work plans. Formal personnel reviews are done annually and progress is monitored monthly. The UHC Board has five years of training, including a year when they have overseen the plan essentially on their own, and a detailed operational handbook that clearly sets out its roles and responsibilities, as well as those of the other UHC stakeholders. Given the uniqueness of the UHC and the role of its Board it is difficult to predict whether it will be able to continue to follow these guidelines after the conclusion of the project, especially if there are significant changes in the policy environment. HP’s Malaria Communities Project will allow HealthPartners to continue to provide technical assistance to the Board at least through 2012 if situations arise.    41 In general, staff morale, cohesion and working relationships have improved over the life of the project as more qualified staff have been recruited and become familiar with the UHC and the CSHGP. Dips in morale have occurred with the changing of project directors until they are able to get up to speed. The project has experienced some turnover in key positions, which can slow a project down but also bring new skills and perspectives to the team. HealthPartners invests in long term in-country leadership by hiring all local staff. The project has had three CS Directors over its five year life. The first two were officed in Kampala and made regular trips to the field. The first Child Survival Director is now the Country Manager for Family Health International, the second Director is the Senior Advisor, Human Resource Manager at the Ministry of Health. The previous Child Survival Monitoring and Evaluation Coordinator is now employed by the Ministry of Gender, Labor and Social Development. The current CS Director was a practicing physician and administrator at Comboni Hospital, which has been a long-term UHC provider so he came to his new role with a practical understanding of the UHC system, the staff and the local area. Six out of ten of the senior field staff have been with the project since the DIP. There has been no turnover at headquarters, except for George Halverson, the HealthPartners CEO who fostered the relationship with Land O’ Lakes that led to the Child Survival project. The Technical Backstop and Uganda Program Manager have remained the same throughout. Reportedly the UHC Board of Directors has experienced a remarkable level in continuity with all members having completed their term of office. Financial Management: HealthPartners projects that their CS Project should come in either on or slightly below budget and that there were sufficient funds to cover project activities and obligations throughout the life of the project. No changes were required in the budget, except for minor movements between line items, as allowed for by USAID. Fund transfers have been timely. HP does not request or receive indirect cost rate reimbursement—100% of grant funding is spent directly on the project. On every project HP has exceeded the match contribution requirement. With the exception of one part-time program manager (time divided between projects) and accounting support (5-10% time depending upon the level of funding) all grant funds are spent on in-country sustainable capacity building and development. HealthPartners senior level expertise ranging from the board of directors to legal, marketing, community relations, customer service, the Foundation, provider and physician services departments is donated as match contribution. As of June 2010, HP cost share on the CSHGP was 35% or $438,929. The agreement cost share was 25% or $312,500.  The challenge has been with the Government of Uganda’s ability to provide all the supplies, medicines and equipment necessary to meet the increased demand created through its BCC campaigns and community mobilization. The prospects for financial sustainability of the UHC structure and the VHT network appear to be reasonably strong. The UHC is operating at a surplus and the process of redistricting and capacity building will hopefully result in enough financial and human resources for the newly formed districts so they can successfully continue the support of the VHTs and VHT parish coordinators. A major focus in building the capacity of the UHC Board and providers has been to increase their understanding of how an insurance cooperative works from a financial standpoint – what needs to be in place to properly manage the funds and what level of income is required to    42 sustain the UHC overall. At each Board meeting they review the progress of the member groups and providers to ensure that they are meeting their financial obligations and reporting requirements, as well as track on their overall performance comparing revenues to costs so progress can be tracked and adjustments be made as needed. Project staff, UHC Board members and provider representatives were consistent in how they described the UHC financials, including the collection of premiums from group members, the transfer/documentation of funds to the providers, the structure and use of the reserve fund, and the frequency and types of financial reports. An external auditor had just completed the first audit of the UHC funds prior to the start of the final evaluation, however, the report was not available for review by the Final Evaluation Team before completing this final evaluation report. Logistics: Logistics had no major impact on project implementation, especially after the current Project Director was stationed in Bushenyi instead of Kampala. Major purchases and vehicle repairs have to be done in Kampala which is a one day drive from Bushenyi and therefore this is planned for effectively in advance to coincide with meetings and other business that has to be done in the capital city. Information Management: The project has developed a practical and user-friendly system for tracking and reporting on the performance of UHC plan members and providers (objective 5). As noted in the Data Quality section above, however, technical problems in the use of LQAS in collecting its KPC data limited the accuracy and usefulness of some of these results (objectives 1-3). Changes in the wording of several indicators measuring change in health care management made it difficult to track progress on this intervention over time (objective 4). Data has been collected, reported and used consistently on a timely basis. It has been used by the project to set premium rates and provide feedback to UHC member groups and providers, including the identification of one provider that had to be dropped from the UHC for not following the terms of the UHC agreement. Comparing VHT performance to various incentive options has provided useful information to the project on how it can best utilize limited resources to increase volunteer commitment and performance. At the request of the UCBHFA, HealthPartners and Oracle Corporation had developed a database specifically for tracking data for health cooperatives called the Uganda Health Information System (UHIS) prior to the start of this Child Survival project. The Child Survival project then planned to expand this database so that it would also track and report on its child survival interventions as well. However, due to the lack of computers, a stable power source and trained staff it was later determined that the UHIS was not sustainable as a computer-based system. As a more appropriate option, the project established a hard copy data system for tracking VHT performance and reporting to health workers. District health workers, VHTs and VHT parish coordinators have been trained and are now using this system to track the number of health education/BCC sessions done, the number of attendees and the topics covered. Once the technical issues have been addressed in the data, this project’s monitoring and impact data should be reanalyzed and shared with a broad audience including governments, donors, academics and other NGOs that are interested in the financing of primary and reproductive health care services in settings with limited financial resources and health care infrastructure.    43 Technical and Administrative Support: As a new CSHGP grantee, HealthPartners sought and received technical support from a wide variety of sources, including CSTS, Africare, PSI, the Institute of Public Health at Makerere University, MANGO, and the Ugandan MOH. In addition, project staff attended various workshops and conferences that were directly related to the project, including the DIP Mini University, the New Grantee Orientation, CORE’s Qualitative Research Methods Workshop, the Community Health Financing Workshop and a site visit hosted by UCBHFA to Tanzania where project staff shared experiences with other organizations implementing insurance schemes. HealthPartners was not a member of the CORE Group, which might have been another source of useful technical support and assistance. As a respected health insurance cooperative, HealthPartners was also able to call upon its own internal technical resources and expertise in support of the project from the headquarters project staff and others. Headquarters staff communicated frequently with the field staff throughout the life of the project on internet and conference calls and made annual trips to the field at key points during the life of the project, which field staff considered a high point in the life of the project. A major focus of these field trips was to build the capacity of the UHC Board so it would be able to administer and oversee the Cooperative. The Board received TA from HealthPartners in data tracking, marketing, financial management, community mobilization, logistics and health education. Further TA needs included budgeting, research methods, and resource utilization and management. Management Lessons Learned: Throughout the final evaluation, staff, volunteers and community members familiar with the project were invited to share lessons they had learned from their involvement with the project. Following is a list of their responses:  “Behavior change takes considerable time.”  “Supervision requires tolerance and patience.”  “Good administration and team work are crucial to success.”  “Involvement of all stakeholders, beneficiaries, and partners in planning, monitoring and feedback helps avoid duplication and builds consensus so resources are leveraged to achieve better outcomes.”  “Using lessons learned to guide planning avoids wasting time with interventions that are not working and builds on those that actually work.”  “The key to success of the UHC is mobilization and marketing.”  “The transition to sustainability needs to be a gradual process.”  “The UHC should target pre-existing groups and groups with strong leadership for membership in the Cooperative.”  “The testimony of existing UHC members is a powerful recruitment tool.” In addition, the following lessons learned have been generated from the results described throughout this report:  It is often difficult in rural settings to recruit staff who meet all the technical requirements of a specific position. However, they can bring an understanding of the local situation and    44 the people involved that someone from outside will not have and also be less likely to move away.  The success of project interventions should not be dependent on inputs that are outside the control of the project.  Use of LQAS should be reconsidered whenever there are several skips in the questionnaire. In these situations it might be advisable to use 30-cluster sampling, even though it requires a larger sample size.  Comparing volunteer performance with various incentive strategies is an excellent way to identify which incentives will work.  Timely and positive feedback can increase the rates and quality of reporting, especially with volunteers.  It is important to review the baseline indicators whenever making changes to a project in order to determine if they can still be used to identify the new benchmarks towards achieving success. If not, a new baseline might need to be tabulated from the existing data or an entirely new baseline might need to be measured, which can be costly and time consuming.    45 Annex 4: Work Plan Table Activities Objective Met? Activity Status Change Health Plan Benefit Scheme Met Conducted cost pricing analysis for chronic care coverage. Board determined it was too expensive for local stakeholders. Promote Health Plan Membership Met Conducted doer non doer analysis and developed BCC plan to promote health plan membership. Implemented plan but no change was seen. Uganda Health Information System (UHIS) Met Below Hired HealthPartners Information Technology specialist who converted the database platform to Word to remove the need to pay annual Oracle subscriptions. She traveled to Uganda to train stakeholders. Computer break downs and staff turn over made support of the intervention costly. There were long delays in data entry and data transfers for support supervision making data use less effective. Cost analysis led to determination that this was not a sustainable intervention and emphasis should be put instead on improving existing paper based Health Management Information Systems.  Change Platform Met Above  Develop Specs for system Met Done  Installer/trainer to Uganda Met January 2007  Contract service Met Akatukunda Duncan  Collect and analyze monthly reports Met January 2007-2008  Follow up training to close gaps Met February 2007-2008  Board/committee training on reports Met February 2007, Providers April 2007  Follow up (9 months) Met September 2007  Follow up (18 months) Met Cost analysis, discontinued intervention Intervention Planning:  Develop M&E Tools Met  Develop BCC/Community Education Met  Develop Health Services Training Met  Health Plans Mobilization Met Phase I: Igara County East and West Met Completed in February 2007 Community Mobilization Training  Malaria  Reduce Incidence of DD  MNC  Sanitation  HIV/AIDS and STDs  Review  LQAS survey and update Mobilization Met Provider Training  Stock order plans  SCM of malaria  SCM of DD  SCM of ANC  SCM postnatal & delivery (AMTSL)  Self assessment training (& review)  HFA and update provider training Met Annual CS Workshop/ISA/Training Met    46 Activities Objective Met? Activity Status Phase II: Ruhinda County Met Completed in October 2007  Community Mobilization Training (Same as above – updated as needed) Met  Provider Training (Same as above – updated as needed) Met Annual CS Workshop/ISA/Training Met Phase III: Sheema County South Met Completed in June 2008  Community Mobilization Training (Same as above – updated as needed) Met  Provider Training (Same as above – updated as needed) Met Midterm Evaluation & Annual CS Workshop Met Subsidized Health Plan Delayed due to board concerns about adding complexity before membership growth and sustainable cost balance Board developed work plan in 2008 detailing sustainable cost balance.  Review Phase 1-3 Assessments Met Assessed growth; conducted focus group discussions first followed by doer non doer analysis. Developed BCC plan specific to UHC growth before subsidized implementation.  Financial assessment to determine targets Met BCC plan still did not increase membership. Decided to offer VHT reduced rates as a sustainable community owned subsidy instead so VHT could 1) promote UHC from experience, 2) be models of healthy behavior—early treatment seeking 3) have a sustainable reward through local stakeholders  Mobilize subsidy beneficiaries Met Agreement for support signed by DHT and programs. Implemented for all VHT in Bushenyi. VHT slow to join, recognized need for BCC training tool.  Implement subsidy structure for phases 1-3 Met Developed training tool for VHT, implemented in 2010.  Assess subsidy Uptake is still slow. Efforts to support signing new VHT groups is ongoing.  Revise system and adjust/expand Met Ongoing. Phase IV: Sheema County North Met Integrated and done with Sheema south as phase III  Community Mobilization Training (Same as above – updated as needed) Met  Provider Training (Same as above – updated as needed) Met    47 Activities Objective Met? Activity Status Annual Stakeholder Workshop/ISA Met Phase V: Buhweju County Met Completed in March 2009  Community Mobilization Training (Same as above – updated as needed) Met  Provider Training (Same as above – updated as needed) Met Phase VI: Bunyaruguru County Met Completed in September 2009  Community Mobilization Training (Same as above – updated as needed) Met  Provider Training (Same as above – updated as needed) Met Final Assessment Met Done in June – Sept 2010 Coordinate Transition Plans for Program On going Final Stakeholder Workshop Met Done in August 2010    48 Annex 5: Rapid CATCH Table 6. Rapid catch indicators Indicators Baseline UHC n=95 (%) Non-UHC n=95 (%) 6.1 % of children 0-23 months who are underweight (-2SD from the median weight for age, according to the WHO/NCHS reference population) 27.2 21.8 27.4 6.2 % of children age 0-23 months who were born at least 24 months after the previous surviving child 65 70.6 78.2 6.3 % of children aged 0-23 months whose births were attended to by a skilled health personnel 47.4 74.7 52.6 6.4 % of mothers of children aged 0-23 months who received at least 2 tetanus toxoid injections before the birth of their youngest child 65.7 53.7 53.7 6.5 % of infants aged 0-5 months who were exclusively breastfed in the last 24 hours 100 94.1 100 6.6 % of infants aged 6-9 months receiving breast milk and complementary foods 66.7 62.5 66.7 6.7 % of children 12-23 months who were fully vaccinated against the five vaccine preventable diseases before their first birth day (measles was used as a proxy indicator for complete immunization) 76.7 73.3 43.5 6.8 % of children 12-23 months who received a measles vaccine 76.7 45.1 46.2 6.9 % of children 0-23 months who slept under ITN the previous night (in malaria- risk areas only) 31.6 47.4 38.9 6.10 % of mothers who know at least two signs of childhood illness that indicate the need for treatment 75.8 81.1 77.9 6.11 % of sick children age 0-23 months who received increased fluids and continued feeding during an illness in the past 2 weeks 7.8 16.7 9.1 6.12 % of mothers to children age 0-23 who cite at least 2 known ways of reducing the risk of HIV infection 64.2 83.2 78.9 6.13 % of mothers to children age 0-23 months who wash their hands with soap/ash before food preparation, before feeding children, after defecation and after attending to a child who has defecated 2.1 5.3 4.2    49 Annex 6: Final KPC Report CHILD SURVIVAL END OF PROGRAM KNOWLEDGE, PRACTICE AND COVERAGE AND HEALTH FACILITY ASSESSMENT BUSHENYI DISTRICT June, 2010 Authors: Michael Oturu, Monitoring and Evaluation Coordinator Dr. Mukankusi James, Child Survival Project Director Jennifer L. Stockert, Program Manager Uganda Health Cooperative    50 TABLE OF CONTENTS ABBREVIATIONS AND ACRONYMS .......................................................................................... EXECUTIVE SUMMARY ............................................................................................................. 1 BACKGROUND ............................................................................................................................. 2 A-Project Location and Background of the Area ......................................................................... 2 B-Characteristics of the Target Beneficiary Population .............................................................. 2 C-Socio-Economic Indices .......................................................................................................... 2 D-National Standards/Policies Regarding Maternal and Child Health ........................................ 3 E-Overview of the UHC Child Survival Project .......................................................................... 4 F-Objectives of the Evaluation .................................................................................................... 5 G-Process and Partnership Building ............................................................................................ 5 METHODS AND MATERIALS ..................................................................................................... 7 RESULTS ...................................................................................................................................... 16 DISCUSSION ................................................................................................................................ 36 Prevention of malaria ................................................................................................................. 36 Sanitation, hygiene and control of diarrheal diseases ................................................................ 36 Maternal and newborn care ........................................................................................................ 38 Health care management ............................................................................................................ 40 CONCLUSIONS............................................................................................................................ 42 Malaria ...................................................................................................................................... 42 Sanitation, hygiene and control of diarrheal disease ................................................................ 42 Maternal and newborn care ....................................................................................................... 42 Health plans ............................................................................................................................... 42 RECOMMENDATIONS ............................................................................................................... 43 Annex 1: KPC Tool – Child Survival Program End of Project Questionnaire ......................... 44 Annex 2: Health Facility Assessment Questionnaire ................................................................ 59 Annex 3: CS End of program evaluation - Data collection plan June 2010 ............................. 66 Annex 4: Check list for interviewers ........................................................................................ 70 Annex 5: Daily feedback form – for interviewers .................................................................... 71 Annex 6: Checklist for supervisors ........................................................................................... 72 Annex 7: Quality - control checklist ......................................................................................... 73 Annex 8: Daily interview form – for supervisors ..................................................................... 74 Annex 9: Permission letters ...................................................................................................... 75 Annex 10: Training time table .................................................................................................. 77    51 ABBREVIATIONS AND ACRONYMS AMTSL Active Management of Third Stage Labor ANC Ante Natal Care BCC Behavior Change Communication CORP Community Owned Resource Persons CU5 Children Under 5 DHT District of Health Team EPI Epidemiological Information System HFA Health Facility Assessment HIV Human Immunodeficiency Virus HW Health Workers IEC Information Education and Communication IMCI Integrated Management of Childhood Illness IMR Infant Mortality Rate IPT Intermittent Preventive Treatment KPC Knowledge, Practice and Coverage Survey LLIN Long Lasting Insecticide Treated Nets LQAS Lot Quality Assurance Sampling MMR Maternal Mortality Rate MNC Maternal Newborn Care MOH Ministry of Health NGO Non-Government Organization ORS Oral Rehydration Solution PMTCT Prevention of Mother to Child Transmission RCT Routine Counseling and Testing SCM Standard Care Management TBA Traditional Birth Attendant UBOS Uganda Bureau of Statistics UDHS Uganda Demographic and Health Survey UHC Uganda Health Cooperative USAID United States Agency for International Development VHT Village Health Team WHO World Health Organization WRA Women of Reproductive Age 1 EXECUTIVE SUMMARY HealthPartners, the Uganda Health Cooperative (UHC) and the Bushenyi District Health Team (DHT) made great strides to improve health care and healthy behavioral practices in Bushenyi district from 2005- 2010. The Uganda Health Cooperative Child Survival (UHC/CS) program trained 341 community volunteers using behavior change communication and integrated management of childhood illness (IMCI) resources; trained 94 health workers in maternal newborn care and 109 health workers in IMCI. The program goal was to reach 34,500 women of reproductive age and 15,500 infants and children under the age of five for a total beneficiary population of 50,000 over the life of the project. Incentives were provided to volunteers who held four or more health improvement sessions each month. Over 400,000 people attended volunteer-led sessions as verified by signatures collected after each session. The DHT was supportive of the program, attending meetings, events and health worker training to encourage best care practices and improvements throughout the district. Knowledge, Practice and Coverage and Health Facility Assessments were conducted at the project baseline, after each phase of implementation and at the end of the project to measure impact. Results showed that the percentage of children under 5 who slept under a long lasting insecticide treated bednet (LLIN) the previous night increased from 32% to 54% and the percentage of mothers who slept under a LLIN the previous night increased from 19% to 48%. The percentage of children under 2 with diarrhea in the last two weeks decreased from 55% to 33%. The percentage of caretakers who usually wash hands before feeding children increased from 15% to 36% and the percentage of caretakers who usually wash hands after attending to a child who has defecated increased from 8% to 21%. The percentage of households with access to a covered pit latrine increased from 19% to 77%. The percentage of pregnant women seeking routine testing and counseling for HIV/AIDS services increased from 34% to 96%. The percentage of caretakers counseled on breastfeeding increased from 38% to 73% and the percentage of women who delivered with a skilled health professional as verified by maternal card/ book increased from 47% to 68%. Eighty-nine percent of health workers are being regularly supervised. Results showed that more UHC members adopted healthy preventive and treatment seeking behaviors compared to non-members. Compared to non-members, UHC members had lower rates malaria (46% / 58%) and diarrhea (33% / 34%) in the previous two weeks more UHC members delivered their babies will a skilled health professional (68% / 52%.) Many results exceeded end of project goals however a few indicators dropped from the baseline. A variety of programmatic and environmental factors influenced the outcomes. The following narrative details the method of interventions, data collection, results, discussion and recommendations for the UHC/CS end of program evaluation.    2 BACKGROUND A‐Project Location and Background of the Area HealthPartners has been working in Uganda since 1997. HealthPartners Uganda Health Cooperative Child Survival (UHC/CS) project is located in Bushenyi District in the South Western part of Uganda about 400 km from the country’s capital, Kampala. The project has its head office in Kampala with a field office in Bushenyi. Bushenyi district has five health sub districts; Buhweju, Ruhinda, Sheema, Bunyaruguru and Igara. These are further divided into several sub-counties and parishes up to LC 1 (villages) levels. The population of the district according to counties is summarized in the table below. Table: District Population by Health Sub District COUNTY POPULATION Buhweju 95,400 Bunyaruguru 117,100 Igara 236,700 Ruhinda 185,000 Sheema 207,400 Total 841,600 Source: 2009 Uganda Bureau of Statistics Mid-Year Population Projections B‐Characteristics of the Target Beneficiary Population According to the 2009 statistical abstract published by the Uganda Bureau of Statistics (UBOS), the 2009 mid-year population projection for Bushenyi district is 841,600 with a sex ratio of 92 males per 100 females and a growth rate of 2% per annum. Bushenyi’s population is overwhelmingly rural with only 5% of the district total population classified as urban. Bushenyi district has a very young population with children below 5 years constituting about 20% of the total district population. Under the Child Survival and Health Grant Program, UHC planned to directly reach at least 50,000 women and infants in Bushenyi District where the overall likelihood of dying in infancy is greatest. This district has high prevalence rates of malaria and diarrhea with 26.9% and 30.6% of infants having fever and diarrhea respectively in the last two weeks, and use of LLINs at a dismally low 7% (UBOS 2006). C‐Socio‐Economic Indices According to the Demographic Household and Health Survey (UDHS 2006), South Western Uganda has a literacy rate of 67.6%. The higher literacy rate is due to improvements in enrolments and the functional adult literacy program. The Universal Primary Education program also had its impact on the literacy rates country wide. Uganda remains one of the poorest countries ranking 157 on the global Human Development Index (United Nations Development Programme, Human Development Report 2009.) The main economic activity is agriculture, accounting for over 90% of all economic activities in the district. Most agricultural produce is sold outside the district, usually in the capital city Kampala. Much of this agricultural production is by small scale farmers who generally do not have access to credit. Consequently, it has proven difficult for them to break out of the vicious cycle of poverty.    3 At current mortality levels, one in every 13 Ugandan children dies before reaching age one, while one in every seven does not survive to their fifth birthday. Infant mortality declined from 89 deaths per 1,000 live births (UDHS 2000-2001) to 75 (UDHS 2006) while under 5 mortality declined from 158 deaths per 1,000 live births to 137. The maternal mortality ratio is 435 maternal deaths per 100,000 live births with a variation from 345 to 524 (2006 UDHS.) These high rates are a result of severe malaria, pneumonia, anemia, diarrhea and poor handling of mothers before and after delivery and poor feeding/ sanitation practices for the new born. Whereas the Ministry of Health (MOH) has ensured that there are health facilities up to parish level (Health Center II), the baseline Health Facility Assessment (HFA) revealed that most of these facilities were not adequately stocked with essential medicines or adequately staffed. The terrain of the area also acts as an impediment to easy access to these facilities. D‐National Standards/Policies Regarding Maternal and Child Health The national policy classifies maternal and child health under one cluster. This classification emphasizes the link between maternal and child health mortality and the cumulative nature of health problems throughout the lifecycle. The cluster consists of five elements: Sexual and Reproductive Health, Newborn Care, Common Childhood Illnesses, Immunization and Nutrition. The cluster comprises of the following maternal and newborn health services: preconception care, ANC, post abortion care, intra￾partum care, emergency obstetric care, care of the new born and post natal care (MOH Uganda 2005). The main objective under this cluster is to contribute to a level of reduction in maternal, neonatal and young child mortality that is commensurate with the timely achievement of the Poverty Eradication Action Plan targets and related Millennium Development Goals. Sexual Reproductive Health Rights Specific MOH targets for this element include: Increase the proportion of deliveries by skilled attendants from 38 to 50%; Reduce the unmet need for emergency obstetric care from 86% to 40%; Increase antenatal care (ANC) attendance-- 4 visits per pregnancy from 42 to 50%, Increase the Contraceptive Prevalence Rate (CYP) from 23% to 40% (increase CYP from 223,686 per annum to 500,000 per annum); Reduce the percentage of teenage pregnancy rates from 37 to 20% (MOH 2005). Newborn Health and Survival Under this element the MOH targets to reduce the proportion of children with low birth weight by 30% and to reduce the proportion of neonates seen in health facilities with septicemia/severe disease by 30%. Core interventions include: provision of essential care during pregnancy including Tetanus Toxoid immunization, proper nutrition including iron/foliate supplements and prevention and treatment of maternal infections such as malaria, sexually transmitted diseases, infection control during and after delivery including the distribution of Mamakits (safe birthing resources), provision of essential care during the postnatal period including promotion of immediate and exclusive breast-feeding, thermal control, clean cord practices and vitamin A supplementation among other interventions (MOH 2005). Management of Common Childhood Illness Integrated Management of Childhood Illness (IMCI) is a key strategy for delivery of integrated child health services through improved health worker (HW) skills in assessment and management of malaria, acute respiratory infections, diarrhea and malnutrition, which contribute to over 70% of overall child mortality. The strategy also focuses on improving health system issues that affect care for children in health facilities and working to improve key family care practices that have the highest potential for child survival, growth and development    4 Core interventions for common childhood illnesses include improvement of HW skills in managing childhood illness using IMCI guidelines; community treatment of fever/malaria, diarrhea and pneumonia; family care practices message dissemination (care seeking, disease prevention, home treatment and compliance); integrated sustained outreach services and bi annual Child Days; and provision of comprehensive management of pediatric human immunodeficiency virus (HIV) and support (MOH 2005). E‐Overview of the UHC Child Survival Project HealthPartners helped the community to develop a community-owned health insurance cooperative (co￾op) in Uganda in 1997 through a United States Agency for International Development (USAID) sub-grant from Land O’ Lakes. Through the health co-op it was discovered that the major causes of morbidity and mortality in the Bushenyi community were preventable diseases. In September 2005, HealthPartners UHC was awarded a USAID Child Survival and Health Program grant. The goal is to link child survival interventions to the health co-op, building the capacity of a network of health stakeholders to sustainably reduce morbidity and mortality for women of reproductive age (WRA) and children under 5 (CU5) in Bushenyi district. To achieve this objective, UHC adopted the child survival sustainability assessment strategy with slight variations on community/social, heath services, and local organizational dimensions as the primary model for the program. Community and Social Dimension UHC addressed three objectives under this dimension: 1- to reduce incidence of malaria in children under 5 and pregnant women in the district; 2- to reduce the incidence of diarrhea in children under 5 in the district and 3- to increase the percentage of pregnant women receiving improved ANC, delivery and post natal care. Under the community social dimension, UHC implemented various interventions to improve the health status of the population by mobilizing communities on behaviors that have been proven to reduce morbidity and mortality and removing barriers to these behaviors. Community Owned Resource Persons (CORP) were trained to educate communities on maternal and new born care, prevention, identification and early health care seeking behavior. When the Ministry of health changed strategies and resources to the Village Health Team (VHT) model in 2009, UHC/CS partnered with the district and other projects to support VHT implementation. As a result in the last phase, the project trained 190 VHTs and supported the transition of previously trained CORP to the role of Parish VHT Coordinator, mentors for newly trained VHT. UHC/CS conducted a distribution campaign in 2007 where pregnant women and mothers of children under 5 who were members of the health cooperative received free long lasting insecticide treated bednets (LLINs.) Health care providers throughout the district were encouraged to provide free LLINS to pregnant women at second intermittent preventive treatment (IPT) and Mamakits at the fourth ANC visit. These interventions were aimed at encouraging mothers to seek ANC and delivery services at the health facilities in the district. Health Services Dimension The main objective under this dimension is to build capacity of providers to offer IMCI to ensure quality service delivery and reduction of childhood and maternal morbidity and mortality. The health workers were trained using MOH guidelines and protocols. Training was done in collaboration with staff from the MOH, Bushenyi DHT and other partner organizations. Information, education and communication (IEC) guidelines and protocols were given to volunteers and HW as teaching aides and reference materials.    5 Under this dimension, UHC/CS also aimed to demonstrate to providers the ability of prepaid health plans to address social health insurance for adoption by the MOH to cover the poorest populations. Groups in Bushenyi district were identified and mobilized to contribute premiums which are given to providers beforehand to enable them access health care when they need it. Local Organizational Dimension Under this dimension, UHC/CS had the following objectives: 1-build knowledge and capacity of UHC board to enable them to competently run UHC, 2- strengthen trust and ties between providers and community groups to enable best practices and continued coverage for members, 3- build incentive for membership, leadership and provider participation, 4- build capacity of UHC team, stakeholders and partners. To achieve the above objectives, UHC members elected a board of directors from members of UHC health plans. The board was trained and supported to develop and maintain a sustainable cost balance and benefits for cooperative health stakeholders-- private health care providers, and employer, school and other member groups. Model of Implementation Implementation was phased by health sub district in order to allow feedback and lessons learned from staff, stakeholders, partners, consultants and the management team to be incorporated in the way forward for the next phase. Before beginning the interventions in the district, an HFA and baseline KPC survey using lot quality assurance sampling (LQAS) methodology were carried out. Phase I started in Igara, Phase II in Ruhinda, Phase III Sheema, Buhweju was covered in Phase IV and Bunyaruguru in Phase V. F‐Objectives of the Evaluation General objective The objective of this evaluation is to assess progress in health status and health service delivery in Bushenyi district in comparison to end of project targets to determine UHC/CS project impact, lessons learned and to help local stakeholders continue to improve the health of the community. Results from this assessment will be shared with USAID Child Survival and Health Grants team, the Uganda Mission, other programs and grantees and with local stakeholders. Specific objectives 1. To assess the impact of malaria interventions for pregnant woman and CU5 in comparison to baseline data 2. To assess the impact of diarrhea interventions for CU5 in comparison to baseline data 3. To assess the impact on the percentage of pregnant women receiving improved ANC, delivery and post partum care compared to baseline data 4. To compare the difference of the impact project interventions had on UHC members compared to non-members G‐Process and Partnership Building The end of program KPC involved different stakeholders and partners. The DHT contributed to the drafting of the proposal for the survey, the selection of volunteers for data collection, consultation on reporting of results, and dissemination and use of the results. The DHT provided one car and a driver for data collection while the local council’s chairmen with permission from the Bushenyi district chairman and Chief Administrative Officer assisted in the selection of households for the study. The In-Charges of the health facilities with permission from the District Health Officer authorized their staff to release the information to the interviewers as per the HFA questionnaire. The    6 interviews were carried out by UHC/CS staff together with volunteers from Bushenyi to build local capacity to carry out community surveys in the district. Due to limited financial resources it was not possible to involve more partners as would have been desirable.    7 METHODS AND MATERIALS Background to the study area For the last five years HealthPartners UHC/CS project team has been implementing child survival interventions in Bushenyi district to reduce morbidity and mortality for WRA and CU5. Study design The end of program evaluation employed a cross-sectional survey of household and health facility configuration where quantitative descriptive data was collected directly from households and health facilities. Data was collected between 21st and 25th of June 2010. Study population Mothers of children aged 0-23 months constituted the study population at household level for the KPC and health facilities providing ANC services in the district comprised the study population for the HFA. Having stayed in the district for the past 12 months was used as an inclusion criterion for mothers of children aged 0-23 months old. Health facilities included were level of health center III and above. Health center II and I were not included. Sampling technique and sample size determination: There were five health sub districts in Bushenyi: Buhweju, Bunyaruguru, Igara, Ruhinda and Sheema which were all included in the survey, each constituting a supervision area. The sample size was determined using LQAS study design where 19 respondents were selected from each supervision area. A random sample of 95 households from UHC members were also selected to determine if there is any significant difference between the project’s impact on UHC and non-UHC members in the program area. Selecting the villages to be studied Village lists for Bushenyi district were obtained from Uganda Bureau of Statistics. Systematic sampling was used in the selection of villages in each supervision area. A sampling interval for each supervision area was calculated and with the use of random number tables, a random start was obtained. The sampling interval was used to obtain the 19 villages from which the interviews were carried out. The list of the villages selected for the interviews are attached in Annex 3. Selecting households to be studied The interviewers obtained household lists from the village chairpersons and where lists were not available the interviewers sought help of the local leaders to draw village maps showing the key features. These maps were then further divided into smaller sections (3-5) and each section was assigned a number. One part of the village was randomly selected. The local leader helped the interviewers to list all households in the selected section. Households were assigned numbers and with the use of the table of random numbers one household was randomly selected. The village leaders guided the interviewer to the selected household. At the household, the interviewer inquired if there was a mother with a child aged less than two years, with both of them living there. If such a respondent existed then the interviewer sought their permission to administer the questionnaire. If there was more than one such respondent the interviewer randomly selected one respondent to interview. If no such respondent existed, the interviewer went to the nearest household and continued that way until an eligible respondent was found. Questionnaire development    8 The questionnaire used for the survey was an adaptation of the one that was approved for monitoring the child survival program. Changes were made on how the questions were phrased based on lessons learned from the evaluation done at the end of previous phases. Four questions to capture data on the health plans were added. The questionnaire was further reviewed by the data collection team during training to ensure uniform understanding of the questions and pre tested to ensure clarity and validity of the questions. Most of the questions in the original questionnaire were translated into Runyankore, the local language, during the baseline. So for each of the questions in the monitoring questionnaire the corresponding translated question was used to formulate the translated questionnaire. After the pre test changes were applied to the questionnaire. The questionnaire had six major sections: Background Information, Malaria, Diarrhea, Sanitation and Hygiene, Maternal and Newborn Care, Malaria Prevention and Health Plans. KPC Questionnaire Indicator Definitions Objective 1: Reduce incidence of malaria in Bushenyi district for children under 2 and pregnant women. Indicator Numerator Denominator % of children under 2 with fever in the last two weeks Number of children with fever in the last two weeks ( Q28.1) All children studied % of children under 2 who slept under an LLIN last night. Number of children who slept under an LLIN last night (Q35.A) All children studied % of mothers who slept under an LLIN last night. Number of mothers who slept under an LLIN last night (Q35.c) All mothers of children studied % of children under 2 with fever in the last 2 wks who received anti malarial treatment Number of sick children who received anti malarial treatment (Q32.A,B,C,D,E,F,G,H) Number of children who had fever in the last two weeks (Q28.1) % of pregnant women receiving IPT as verified by maternal card/ book Number of mothers who received IPT 1 and 2 (13.1 AND 2 ) verified by maternal card Number of Mothers who sought prenatal care and have their maternal cards (Q4.2,3AND 4) % of children under 2 with fever in the last 2 weeks who sought treatment on the same day Number of sick children who sought treatment same day (Q31.1) Number of children who had fever in the last two weeks (Q28.1) % of households with children 0-23 months that own at least one mosquito net/LLIN Number of households with mosquito nets (Q33.1) All households in the survey Objective 2: Reduce incidence of diarrhea in Bushenyi district for children under 5 Indicator Numerator Denominator % of children under 2 with diarrhea in the last two weeks Number of children with diarrhea in the past 2 weeks (Q39.1) All children in the survey % of mothers who know at least two signs that a child under 5 needs treatment. Number of mothers who mentioned at least 2 signs that a child needs treatment (Q24.TWO ANSWERS 2-8) All mothers in the survey    9 % of care takers/mothers who know at least one signs that a child under 2 needs treatment Number of mothers who mentioned at least 2 signs that a child needs treatment (Q24.ONE ANSWER 2-8) All mothers in the survey % of households who use improved water source (borehole, public tap, or protected dug well.) Number of mothers that reported improved water sources (Q41.1, 2 OR 3) This referred to Borehole, Public taps, Protected spring All households in the survey % of households with a designated hand washing station with a covered container for water Number of households where hand washing facilities were observed (Q46.1) All households in the survey % of caretakers who usually wash hands with soap before food preparation Number of mothers who reported washing hands Before food preparation (Q47.BEFORE FOOD) All mothers in the survey % of caretakers who usually wash hands before feeding children Number of mothers who reported washing hands Before feeding children (Q47.BEFORE FEEDING CHILDREN) All mothers in the survey % of caretakers who usually wash hands after defecation Number of mothers who reported washing hands After defecation (Q47.AFTER DEFECATION) All mothers in the survey % of caretakers who usually wash hands and after attending to a child who has defecated Number of mothers who reported washing hands After attending to a child who has defecated (Q47.AFTER ATTENDING TO A CHILD WHO HAS DEFECATED) All mothers in the survey % of households who safely disposed of their child's feces the last time s/he passed stool Number of caretakers who dispose off children’s feces (Q44.1 and 2) All households in the survey % of households with access to a covered pit latrine Number of household with access to pit latrines (Q45.1, 2, OR 3) All households in the survey % of children 0-23 months with diarrhea in the last two weeks who received ORS Number of children with diarrhea in the last 2 weeks who received ORS (Q40.2) Number of children with diarrhea in the past 2 weeks (Q39.1) % of children aged 0-23 months who were offered more fluids during illness Children who were offered more fluid during illness (Q36.3) Number of children with diarrhea in the past 2 weeks (Q39.1) % of children aged 0-23 months who were offered more food during illness Children who were who offered more to eat during illness (Q27.3) Number of children with diarrhea in the past 2 weeks who can eat (Q39.1) Objective 3: Increased % of pregnant women receiving improved ANC, delivery and post partum care. Indicator Numerator Denominator % of women with 4 ANC visits as verified by maternal card/ book. Mothers who had at least 4 ANC visits as verified by maternal card/ book(Q4.4) Mothers with maternal card/ book (Q3.1) % of pregnant women seeking RCT services Mothers who sought VCT services (Q16.1) All mothers in the survey % caretakers counseled on Mothers counseled on breastfeeding All mothers in the survey    10 breastfeeding (Q8.breastfeeding=I) % caretakers counseled on importance of child spacing Mothers counseled on importance of child spacing (Q8.child spacing=1) All mothers in the survey % caretakers counseled on danger signs of pregnancy Number of mothers counseled on danger signs during pregnancy (Q8.danger signs=1) All mothers in the survey % of pregnant women counseled on where to deliver Number of pregnant mothers counseled on where to deliver (Q8.where to deliver=1) All mothers in the survey % of pregnant women counseled on transport plans to delivery place Number of pregnant mothers counseled on transport plans to delivery place(Q8.transport plans to delivery place=1) All mothers in the survey % of women who delivered with a skilled health professional as verified by maternal card/ book Number of mothers (verified) who delivered with a health professional (Q19.1 or 2 or 2 or 4 or 5) All mothers in the survey % of women who delivered at a health facility Number of mothers who delivered at a health facility (Q18.2,3 OR 4) All mothers in the survey % of caretakers mothers who know at least 2 danger signs during pregnancy Number of mothers who know mention at least two pregnancy danger sign (Q11. any 2(1, and 2, and 3, and 4) All mothers in the survey % of pregnant women who used a birth kit Number of pregnant mothers who used a birth kit (Q20.1) All mothers in the survey Objective 4: Improved health care management especially for women of reproductive age and children under 5 Indicators Definition of an indicator Denominator Numerator % trained in VCT in the past 3 years Number of staff involved directly with client counseling diagnosis services (Section 1: QN 101 ) Number of staff received in￾service training related to VCT in the past three years (Section 1: QN 102 ) % received in-service training in PMTCT Number of staff involved directly with client counseling diagnosis services (Section 1: QN 101 ) Number of staff received in￾service training related to PMTCT in the past three years (Section 1: QN 103 ) % received training in management of 3rd stage labour in last 3 year Number of staff involved directly in management of antenatal/third stage labor and Emergency Obstetrics care services (Section 3: QN 301 ) Number of staff received in￾service training related to management of 3rd stage labor (Section 3: QN 302 )    11 % of staff received in-service training in reproductive health life savings skills within the last 3 year Number of staff involved directly in management of antenatal/third stage labor and Emergency Obstetrics care services (Section 3: QN 301 ) Number of staff received in￾service training related to management of 3rd stage labor (Section 3: QN 303 ) % of facilities with MOH policy guidelines on ANC and Obstetric services All facilities in survey that provided ANC service Number of facilities with MoH guideline (QN323) % of staff received in-service training in IMCI within the last 3 year Number of staff involved directly in management of IMCI services (Section 4: QN 401 ) Number of staff received in￾service training related to IMCI (Section 3: QN 402) % facilities supervised for IMCI regularly All facilities in survey that provide IMCI services Number of facilities supervised regularly for IMCI (Q 403=1) Rapid Catch Indicator Key Indicators Definition of an indicator Denominator Numerator 1. Percentage of children age 0– 23 months who are underweight (-2 SD from the median weight￾for-age, according to the WHO/NCHS reference population) Number of children age 0– 23 months in the survey who were weighed (Q.A.1) Number of children age 0–23 months whose weight (Q.B) is – 2SD from the median weight of the WHO/NCHS reference population for their age 2. Percentage of children age 0– 23 months who were born at least 24 months after the previous surviving child Number of children age 0– 23 months in the survey who have an older Sibling (Q.2.E) Number of children age 0–23 months whose date of birth is at least 24 months after the previous sibling’s date of birth (Q.2.E) 3. Percentage of children age 0– 23 months whose births were attended by skilled health personnel Number of children age 0– 23 months in the survey (95 or 19) Number of children age 0–23 months with responses =1 (‘doctor’), 2 (‘nurse/midwife’), or 3 (‘auxiliary midwife’) for Q.19. 4. Percentage of mothers with children age 0–23 months who received at least two tetanus toxoid injections before the birth of their youngest child Number of mothers of children age 0– 23 months in the survey (95 or 19) Number of mothers of children age 0–23 months with responses=2 (‘twice’) or 3 (‘more than two times’) for Q.10 5. Percentage of children age 0–5 months who were exclusively breastfed during the last 24 hours Number of infants age 0–5 months in the survey (Q.2.C 0- 5 months only) Number of infants age 0–5 months with only response=A (‘breastmilk’) for Q.E    12 6. Percentage of children age 6–9 months who received breastmilk and complementary foods during the last 24 hours Number of infants aged 6–9 months in the survey (Q.2.C 6- 9 months only) Number of infants age 6–9 months with responses= A (‘breastmilk’) and D (‘mashed, pureed, solid, or semi-solid foods’) for Q.E 7. Percentage of children age 12– 23 months who are fully vaccinated (against the five vaccine preventable diseases) before the first birthday Number of children age 12– 23 months (Q.2.C 12-23 months only) in the survey who have a vaccination card that was seen by the interviewer (response=1 ‘yes, seen by interviewer’ for Q.F.1) Number of children age 12–23 months who received Polio3 (OPV3), DPT3, and measles vaccines (Q.G.5 and 8 and 9) before the first birthday, according to the child’s vaccination card (as documented Q.F.1) 8. Percentage of children age 12– 23 months who received a measles vaccine Number of children age 12– 23 months in the survey (Q.2.C 12-23 months only) Number of children age 12–23 months with response=1 (‘yes’) for (Q.H.1) 9. Percentage of children age 0– 23 months who slept under an insecticide-treated net (in malaria risk areas) the previous night Number of children age 0– 23 months in the survey (95 or 19) Number of children age 0–23 months with ‘child’ mentioned among responses to Q.35.A AND Q.36.1 10. Percentage of mothers with children age 0–23 months who cite at least two known ways of reducing the risk of HIV infection Number of mothers of children age 0– 23 months in the survey (95 or 19) Number of mothers of children age 0–23 months who report at least two of the signs listed in 2 through 8 for Q,17 11. Percentage of mothers with children age 0–23 months who report that they wash their hands with soap/ash before food preparation, before feeding children, after defection, and after attending to a child who has defecated Number of children surveyed who were reportedly sick in the past two weeks (children with any responses other than I (‘none”) for Q.25) Number of mothers of children age 0–23 months who mention responses 1 through 4 for Q.47 12. Percentage of mothers of children age 0–23 months who know at least two signs of childhood illness that indicate the Number of mothers of children age 0– 23 months in the Number of mothers of children age 0–23 months who mention at least two of the responses that relate to safer sex or practices involving blood for Use responses for Question 24 answers 2- 8    13 need for treatment survey (95 or 19) 13. Percentage of sick children age 0–23 months who received increased fluids and continued feeding during an illness in the past two weeks Number of mothers of children age 0– 23 months in the survey (95 or 19) Number of children age 0–23 months with response=3 (‘more than usual’) for Q.26 AND response=2 (‘same amount’) or 3 (‘more than usual’) for Q.27 UHC Indicator Key Indicators Definition of an indicator Denominator Numerator % of respondents who have heard about UHC # of non-member respondents (Q50.2) # of non-UHC respondents (Q50.2) who have heard about UHC (Q48.1) Reasons why respondents who have heard about UHC did not join. # of non-member respondents (Q50.2) who have heard about UHC (Q48.2) # of respondents who answered with reasons why they did not join tallied separately by reason. (Q51.1 or 2 or 3 or 4 or 5 or 6) Average amount of money respondent spent on care, counseling and/or drugs (total cost) at a clinic, hospital, pharmacy and/or other health treatment? Total # of questionnaire respondents for each group separately (Q50.1 and Q50.2.) Average amount of money spent on care, counseling and/or drugs (total cost) at a clinic, hospital, pharmacy and/or other health treatment? (Q52.1 or 2 or 3 or 4 or 5 or 6 or 7) Training of the evaluation team The survey team consisted of eight UHC staff and fourteen student volunteers from Bushenyi district. This team was responsible for collecting data for both the HFA and KPC surveys. The team underwent a one day refresher training on LQAS methodology and how to carry out HFA surveys. During this training the purpose of the survey was explained, the roles and responsibilities of supervisors and interviewers, proper interviewing and supervision techniques and quality control procedures in the field were discussed. The KPC and HFA questionnaires were reviewed item by item to ensure uniform understanding with particular emphasis on the skip patterns and special instructions. After the training, the team was divided into nine pairs of interviewers with two supervisors. One supervisor was in charge of four groups while the other took charge of five groups. Each interviewing team (consisting of one male and one female interviewer with at least one of them fluent in Runyankore) was asked to carry out at least two KPC interviews and one HFA as a pre test in Sheema County. Using the feedback from the pre test, the questionnaires were further reviewed before making final copies to be used for data collection. Survey team Team A Team B Team C Team D Team E Edidah Kananga Amelia Namanya (Team Grace Karungi (Team Leader) Arthur Kanyanyeru (Team Leader) Dr. James Mukankusi (Team Leader)    14 (Team Leader) Leader) Matsiko Mudashir Nabaasa Gloria Baylon Owaruhanga Janepher Bashabomwe Peter Byamukama Muhwezi Robert Immaculate Ainembabazi Simon Bagonza Benson Musinguzi Doreen Arinaitwe Nuwe Blick Jude Asiimwe Martin Mahwezi Joyce Akello Herbert Asiimwe Michael Oturu Dr. Owembabazi Wilberforce Data collection and quality control procedures Data was collected over five days starting from 21st – 25th, June, 2010 for Bushenyi district and two days (28th and 29th, June 2010) for the control in Rukungiri district. Face to face interviews and observation were the two major methods of data collection that were used. In each health sub district or supervision area, 19 household interviews were carried out with all of the interviews totaling to 95 for UHC members, 95 for non-UHC members and 19 for the control group. Due to the difficult terrain in some supervision areas and the long distances between health sub districts it was not possible to assign a group to a supervision area. Therefore three groups went to one supervision area per day to collect both KPC and HFA data for non-UHC member while two groups collected data on UHC members since these are concentrated in one county. Each group was able to carry out 4-7 household interviews a day. The team leaders coordinated the data collection exercise. This entailed observation of at least one interview per interviewer and giving feedback on performance and areas that needed improvement. Although the supervisors were not able to observe every interview conducted, they reviewed every questionnaire for completeness and errors while still in the field so as to resolve any problems identified. The survey team had daily briefing and debriefing meetings to discuss the planned activities and to share experiences and challenges from the field at the end of the day to come up with appropriate and timely solutions. Training of the survey team, pre testing, and reviewing the questionnaires, assigning supervisors to the data collection teams and having daily briefing and debriefing meetings were measures aimed at improving the quality of data collected. To ensure quality data entry and analysis, the questionnaires were pre-coded and numbered when entered just in case they needed to be referenced. Data quality checks were incorporated in the data entry program to minimize errors. A data entrant was trained on the use of Epidemiological Information System (EPI) Info for windows version 3.5.1 software which was used for data capture and analysis. Data cleaning was done before analysis. Data management /data analysis Since the study variables did not involve generating complex statistics, data was analyzed using EPI Info software version 3.5.1 for windows statistical package. The data entry screen was created and pre-tested using data from the questionnaires used during pre-test and the design errors were corrected to allow for smooth final data entry. Analysis entailed the determination of descriptive statistics for the variables under study. Results are presented in narrative form, tables and graphs as was found appropriate.    15 Challenges faced 1. The exercise was conducted during a farming season, it was therefore difficult to find the respondents at home especially during morning hours. At time interviewers had to wait for the mothers to be called from the gardens or walked there to carryout the interview. This resulted in each interview taking longer than had been planned. 2. The other perennial challenge is difficult terrain with poor road network. Interviewers had to walk long distances to locate the villages and respondents thus minimizing the number of interviews that could be accomplished per day. 3. Poor communication network was also a challenge in cases where interviewers needed to consult their supervisors. 4. Identifying respondents especially in the tea estates was very challenging. Most estate workers do not live with their families at the tea estates. This called for following the identified respondents to their villages for those workers who come from Bushenyi district. Lessons learnt 1. The whole survey team going to a health sub district in a day was motivating to members as they knew that they would complete all the interviews in that supervision area without having to go back the next day. This was particularly evident for remote and hard to reach supervision areas. 2. Majority of community members were cooperative and willing to guide the interviewers or act on behalf of LC officials in case they were not available for the village mapping, household and respondent selection. 3. The exercise was a great learning opportunity for both UHC staff and the volunteers to do evaluations and in particular to master the LQAS methodology. 4. Student volunteers proved to be committed to the whole exercise and this eliminated the problems of curb stoning. 5. Carrying out the pretest was instrumental in imparting the required skills to the interviewers and supervisors especially those who were using this methodology for the first time. 6. HFA and KPC data collection needs to be done by different teams to ensure consistency and data quality. The KPC team needs at least one of the interviewers to be well trained. Recommendations 1. Survey teams are encouraged to storm a particular supervision area as a team and complete it in a day. 2. To ensure data quality, all LQAS surveys should have an adequate number of supervisors. 3. Students should be encouraged to volunteer as interviewers so as to build local capacity and minimize program evaluation costs. 4. Where sufficient funds are available, each supervisor should be located a vehicle and interviewers adequately facilitated with umbrellas, boots, lunch and air time for communication.    16 RESULTS In this chapter the results of end of program KPC survey are presented. The presentation of the results starts with demographic characteristics of the respondents and the study units followed by a summary of all the indicators studied and finally detailed results for each objective are presented. Of the 95 mothers interviewed under UHC members, 64% were 25 years and above and 36% were below 25 years. Their ages ranged from 19 - 40 years with a median being 26 years. On the other hand, 58% of the non-UHC members were mothers aged 25 years and above while 42% were below 25 years. Their ages ranged from 18 – 40 year with a median age of 26 years. For UHC members, 45.3% of the children studied were male and 54.7% were female. Their ages ranged from 0-23 months and the median age was 13 months. While for non-UHC members, 55.8% of the children studied were male and 44.2% were female. Their ages ranged from 0-23 months and the median age was – months. Table 1: Results per indicator Objective 1: Reduce incidence of malaria in Bushenyi district for children under 2 and pregnant women. Indicators Baseline UHC (%) Significance compared to baseline Non￾UHC (%) Significance compared to baseline Significance UHC vs. non-UHC EOP Goal 1.1 % of children under 2 with fever in the last two weeks 44.2 46.3 57.9 S 19 1.2 % of children under 2 who slept under an LLIN last night. 32 53.7 S 56.8 S NS 55 1.3 % of mothers who slept under an LLIN last night. 19 48.4 S 40 S NS 36 1.4 % of children under 2 with fever in the last 2 wks who received anti malarial treatment 71 44.2 41.8 NS 84 1.5 % of pregnant women receiving IPT as verified by maternal card/ book 27 94.3 S 73 S S 49 1.6 % of children under 2 with fever in the last two weeks who 0 23.3 S 16.4 S NS    17 sought treatment the same day Objective 2: Reduce incidence of diarrhea in Bushenyi district for children under 5 Indicators Baseline UHC % Significance compared to baseline Non￾UHC (%) Significance compared to baseline Significance UHC vs. non-UHC EOP Goal 2.1 % of children under 2 with diarrhea in the last two weeks 55 32.6 S 33.7 S NS 20 2.2 % of mothers who know at least two signs that a child under 5 needs treatment. 76 81.1 NS 77.9 NS NS 84 2.3 % of care takers/mothers who know at least one sign that a child under 5 needs treatment. 97.9 97.9 NS 2.4 % of households who use improved water source (borehole, public tap, or protected dug well.) 50 64.2 S 55.8 NS NS 60 2.5 % of households with a designated hand washing station with a covered container for water 24 27.4 NS 9.5 S S 46 2.6 % of caretakers who usually wash hands with soap before food preparation 42 45.3 47.4 NS 64 2.7 % of caretakers who usually wash hands before feeding children 15 35.8 22.1 S 35    18 2.8 % of caretakers who usually wash hands after defecation 63 64.2 56.8 NS 82 2.9 % of caretakers who usually wash hands and after attending to a child who has defecated 8 21.1 20 NS 30 2.10 % of households who safely disposed of their child's feces the last time s/he passed stool 68 72.6 NS 54.7 S S 82 2.11 % of households with access to a covered pit latrine 19 76.8 57.9 S 36 2.12 % of children 0- 23 months with diarrhea in the last two weeks who received ORS 0 25.8 18.8 NS 30 2.13 % of children aged 0-23 months who were offered more fluids during illness 15.8 48.4 NS 25.0 NS NS 2.14 % of children aged 0-23 months who were offered more food during illness 42.1 35.5 S 28.1 NS NS Objective 3: Increased % of pregnant women receiving improved ANC, delivery and post partum care. Indicators Baseline UHC Significance compared to baseline Non￾UHC (%) Significance compared to baseline Significance UHC vs. non-UHC EOP Goal 3.1 % of women with 4 ANC visits as verified by maternal card/ book. 18 45.9 S 57.9 S NS 30    19 3.2 % of pregnant women seeking RCT services 34 95.7 S 88.4 S S 44 3.3 % caretakers counseled on breastfeeding 38 72.6 61.1 S 55 3.4 % caretakers counseled on importance of child spacing 95 70.5 S 54.7 S S 99 3.5 % caretakers counseled on danger signs of pregnancy 76 75.8 NS 65.3 S S 90 3.6 % of pregnant women counseled on where to deliver 69.5 89.5 S 76.8 NS S 3.7 % of pregnant women counseled on transport plans to delivery place 45.3 73.7 S 63.2 S S 3.8 % of women who delivered with a skilled health professional as verified by maternal card/ book 47.4 68.4 S 51.6 NS S 65 3.9 % of women who delivered at a health facility 66.3 53.7 3.10 % of caretakers mothers who know at least 2 danger signs during pregnancy 44.2 38.9 3.11 % of pregnant women who used a birth kit 77 78.9 NS 72.6 NS NS Objective 4: Improved health care management especially for women of reproductive age and children under 5 years old. Indicators Definition of an indicator       Baseline Denominator Numerator Percent EOP % trained in VCT in the past 3 years    153 54 35.3    20 % received in-service training in PMTCT    153 62 40.5 % received training in management of 3rd stage labour in last 3 year 79 111 48 43.2 90 % of staff received in-service training in reproductive health life savings skills within the last 3 year 111 42 37.8 % of facilities with MOH policy guidelines on ANC and Obstetric services 44 36 27 75.0 65 % of staff received in-service training in IMCI within the last 3 year 57 167 61 36.5 70 % facilities supervised for IMCI regularly    36 32 88.9 40 % of stock outs in the past 6 months    % of ACT stock outs in the past 6 months    36 22 61.1 % of injectable contraceptives stock outs in the past 6 months    36 31 86.1 % of ORS stock outs in the past 6 months    36 10 27.8 % of cotrimoxazole / septrin stock outs in the past 6 months    36 19 52.8 % of facilities who provide information on nutrition& hygiene, LLINs, breastfeeding, STI/HIV/AIDs prevention, warning signs, post natal care.    36 36 100.0 Objective 5: Health Plan Indicators    Baseline EOP UHC EOP Non-UHC 5.1 % of respondents who have heard about UHC 0 100 51.6 5.2 Reasons why respondents who have heard about UHC did not join 0 NA Don’t know about it 38.8 Don’t know how to join 34.7 Don’t have a group to join with 8.2 Think it is too expensive 6.1 Don’t think it works 2.0 Other …………(%) 10.2 5.3 Average amount of money respondents spent on care, counseling and or drugs (total costs) at a clinic, hospital, pharmacy and other health treatment. 0 <= 5,000……….. 42.1% 5,001-10,000…. 21.1% 10,001-20,000… 11.6% 20,001-50,000…. 18.9%    21 50,001-100,000… 4.2% Not sure……… 2.1% 0 NA <= 5,000………..% 35.8 5,001-10,000….% 22.1 10,001-20,000…% 22.1 20,001-50,000….% 12.6 50,001-100,000…% 3.2 Not sure………% 4.2 5.4 % of respondents who sought timely health care 0 88.4 72.7 6. Rapid catch indicators Indicators Baseline UHC (%) Non-UHC (%) 6.1 % of children 0-23 months who are underweight (- 2SD from the median weight for age, according to the WHO/NCHS reference population) 21.8 27.4 6.2 % of children age 0-23 months whowere born at least 24 months after the previous surviving child 70.6 78.2 6.3 % of children aged 0-23 months whose births were attended to by a skilled health personnel 27.2 74.7 52.6 6.4 % of mothers of children aged 0-23 months who received at least 2 tetanus toxoid injections before the birth of their youngest child 65.7 53.7 53.7 6.5 % of infants aged 0-5 months who were exclusively breastfed in the last 24 hours 100 94.1 100 6.6 % of infants aged 6-9 months receiving breast milk and complementary foods 67.7 62.5 66.7 6.7 % of children 12-23 months who were fully vaccinated against the five vaccine preventable diseases before their first birth day (measles was used as a proxy indicator for complete immunization) 67.7 73.3 43.5 6.8 % of children 12-23 months who received a measles vaccine 67.7 45.1 46.2 6.9 % of children 0-23 months who slept under LLIN the previous night (in malaria- risk areas only) 47.4 38.9 6.10 % of mothers who know at least two signs of childhood illness that indicate the need for treatment 75.8 81.1 77.9    22 6.11 % of sick children age 0-23 months who received increased fluids and continued feeding during an illness in the past 2 weeks 47.8 16.7 9.1 6.12 % of mothers to children age 0-23 who cite at least 2 known ways of reducing the risk of HIV infection 64.2 83.2 78.9 6.13 % of mothers to children age 0-23 months who wash their hands with soap/ash before food preparation, before feeding children, after defecation and after attending to a child who has defecated 2.1 5.3 4.2 Objective 1: To assess the impact of malaria interventions for pregnant woman and CU5 in comparison to baseline data Table 2: Comparing malaria indicator results with EOP targets Indicators Baseline UHC Non￾UHC EOP Goal UHC Comments Non- UHC Comments % of children under 2 with fever in the last two weeks 44.2 46.3 57.9 19 Target not achieved Target not achieved % of children under 2 who slept under an LLIN last night. 32 53.7 56.8 55 Target not achieved Target exceeded % of mothers who slept under an LLIN last night. 19 48.4 40 36 Target exceeded Target exceeded % of children under 2 with fever in the last 2 wks who received anti malarial treatment 71 44.2 41.8 84 Target not achieved Target not achieved % of pregnant women receiving IPT as verified by maternal card/ book 27 94.3 72.9 49 Target exceeded Target exceeded % of children under 2 with fever in the last two weeks who thought treatment the same day 0 22.7 22.5 Significant achievement Significant achievement % of households with children 0- 23 months that own at least one mosquito net/LLIN 0 61.1 63.2 Significant achievement Significant achievement It can be noted from the table 2 that the program attained or exceeded end of program goals in three out five indicators for UHC and non-UHC members that were measured at baseline and on which end of program goals were set to drive the intervention. It is important to note that the baseline included only non-UHC members. Among the driving factors for the malaria epidemic is the use of LLINs, and we can observe here that although 61.1% and 63.2% of the households for UHC and non-UHC members respectively owned at least one LLIN, incidence of malaria is still high in the district. Further analysis of those who slept under the net the night before the interview are presented in table 3.    23 Table 3: Persons who slept under a net the night before the interview UHC Members Non-UHC Members Who slept under LLIN Count % of responses Who slept under LLIN Count % of responses Child 51 53.7 Child 54 56.8 Mother 46 48.4 Mother 38 40.0 Partner 26 27.4 Partner 24 25.3 Other children 25 26.3 Other children 16 16.8 From table 3, it can be seen that 53.7% and 56.8% of the responses show that it is the child who slept under the net the night before the interview for UHC members and non-UHC members respectively. Results also show that 29.3% and 33.3% of the nets that the interviewers observed from UHC and non￾UHC household members respectively were damaged giving mosquitoes access to those sleeping under them. This may be a contributing factor to the increased incidence of malaria in the previous two weeks. Figure 1: Percentage of households with at least one net versus percentage of children sleeping under nets Table: 4 Children who had fever in the last two weeks and received anti – malarial treatment Indicator UHC (%) Numerator Denominator Non-UHC (%) Numerator Denominator % of children who had fever in the last two weeks 46.3 43 95 57.9 55 95 % of children with fever in the last two weeks who received anti malarial treatment 42.2 19 43 41.8 23 55    24 Of the 46.3% children who had fever from the UHC members’ families, 86% sought treatment or advice and 23.3% sought treatment the same day. For non-UHC members, out of the 57.9% who had fever, 76.4% sought treatment or advice but only 16.4% sought treatment/advice the same day. When asked where they went first for treatment, the figure below presents their responses. Figure 2: Place where mothers first sought treatment or advice From the results in figure 2, it can be noted that 89.2% of the mothers who are UHC members sought treatment from either a clinic or NGO health facility whereas only 2.7% went to a pharmacy. This is because the UHC service providers are private health facilities. Figure 2 above also shows that the majority of non-UHC members sought treatment from clinics and government health facilities (59.5% and 21.4% respectively). It is also important to note that 4.8% of the non-UHC members sought treatment/advice from their friends/relatives. Objective 2: To assess the impact of diarrhea interventions for CU5 in comparison to baseline data An assessment of the incidence of diarrhea in under five results are as seen in Table 5. Table 5: Diarrhea indicator results compared baseline values and end of project targets. Indicators Baseline UHC (%) Non￾UHC EOP Goal UHC Comments Non- UHC Comments 2.1 % of children under 2 with diarrhea in the last two weeks 55 32.6 33.7 20 Target not achieved Improved but target not achieved 2.2 % of mothers who know at least two signs that a child under 5 needs treatment. 76 81.1 77.9 84 Target not achieved Improved but target not achieved    25 2.4 % of households who use improved water source (borehole, public tap, or protected dug well.) 50 64.2 55.8 60 Target achieved Target not achieved 2.5 % of households with a designated hand washing station with a covered container for water 24 27.4 9.5 46 Target not achieved Improved but target not achieved 2.6 % of caretakers who usually wash hands with soap before food preparation 42 45.3 47.4 64 Target not achieved Target not achieved 2.7 % of caretakers who usually wash hands before feeding children 15 35.8 22.1 35 Target achieved Target not achieved 2.8 % of caretakers who usually wash hands after defecation 63 64.2 56.8 82 Target not achieved Target not achieved 2.9 % of caretakers who usually wash hands and after attending to a child who has defecated 8 21.1 20 30 Target not achieved Improved but target not achieved 2.10 % of households who safely disposed of their child's feces the last time s/he passed stool 68 72.6 54.7 82 Target not achieved Target not achieved 2.11 % of households with access to a covered pit latrine 19 76.8 57.9 36 Target exceeded Target exceeded 2.12 % of children 0-23 months with diarrhea in the last two weeks who received ORS 0 25.8 18.8 30 Target not achieved Target not achieved 2.13 % of children aged 0-23 months who were offered more fluids during illness 15.8 48.4 25.0 - 2.14 % of children aged 0-23 months who were offered more food during illness 42.1 35.5 28.1 - From the table, it can be noted that there is considerable improvement in most of the diarrhea indicator performance from the baseline although most of the targets were not achieved; sanitation and hygiene are still a major problem in this district. There was an impressive increase from 19% to 76.8% and 57.9% of the households for UHC members and non-members respectively reported that they had access to a covered pit latrine. Hand washing after attending to a child who has defecated and before feeding children however still remaind a major a problem with only 21.1 and 20% of UHC members and non￾UHC members respectively reporting that they washed their hands after helping a child who had defecated. Only 35.8 and 22.1% of the members and non members reported that they had washed their hands before feeding children. Safe disposal of children’s faeces is also still poor with only 72.6 and 54.7% of the members and non-members respectively reporting that they had safely disposed off the child’s faeces into the toilet. The figure below shows the results of mothers washing hands with soap and water. Figure 3: Hand washing practices by category compared with baseline and target    26 From figure 3 it is evident that washing hands before feeding children and after attending to a child who has defecated requires attention for improvement for both UHC and non-UHC members. From observation, 27.4 and 9.5% of the members and non members respectively had a designated hand washing facility with a covered container for water. Sixty four point two (64.2) and 55.8% of UHC and non-UHC members reported having improved water source with both groups reporting boiling (88.5 and 56.6% for UHC and non-UHC members respectively) as a major method of making this water safe. Of the 32.6 and 33.7% of the children with diarrhea for UHC and non-UHC members respectively in the last two weeks, 25.8 and 18.8% for UHC and non-UHC members respectively were treated with ORS. The figure below compares the performance of UHC members and non members feeding practices for sick children against baseline. It can be seen that UHC members gave their sick children more fluids and more food during illness compared to non members. The feeding practice of giving more food during illness declined compared to the baseline. Reasons for this will be explored in consultant interviews and focus group discussions.    27 Figure 4: Feeding practices for sick child Objective 3: To assess the impact on the percentage of pregnant women receiving improved ANC, delivery and post partum care compared to baseline data. Table 6: Maternal and newborn care indicators Indicators Baseline UHC Non￾UHC EOP Goal UHC Comments Non- UHC Comments 3.1 % of women with 4 ANC visits as verified by maternal card/ book. 18 45.9 57.9 30 Target exceeded Target exceeded 3.2 % of pregnant women seeking RCT services 34 95.7 88.4 44 Target exceeded Target exceeded 3.3 % caretakers counseled on breastfeeding 38 72.6 61.1 55 Target exceeded Target exceeded 3.4 % caretakers counseled on importance of child spacing 95 70.5 54.7 99 Target not achieved Target not achieved 3.5 % caretakers counseled on danger signs of pregnancy 76 75.8 65.3 90 Target not achieved Target not achieved 3.6 % of pregnant women counseled on where to deliver 69.5 89.5 76.8 - 3.7 % of pregnant women counseled on transport plans to delivery place 45.3 73.7 63.2 - 3.8 % of women who delivered with a skilled health 47.4 68.4 51.6 65 Target exceeded Target not achieved    28 professional as verified by maternal card/ book 3.9 % of women who delivered at a health facility 66.3 53.7 - 3.10 % of caretakers mothers who know at least 2 danger signs during pregnancy 44.2 38.9 3.11 % of pregnant women who used a birth kit 77 78.9 72.6 Results show that 96.8 and 92.6% of the UHC and non-UHC members who are mothers of children age 0-23 went for prenatal care with the majority of them (94.7 and 87.5% for UHC and non-UHC respectively) seeing a nurse/ midwife. 5.6% of the non-UHC members reported to have seen a TBA or a community health worker. Mothers were asked for their maternal cards and card retention was found to be low at 38.9 and 40% for UHC and non-UHC members respectively (figure 5). Figure 5: Ownership of maternal card Although the mothers reported to have attended more ANC visits and received at least 2 IPTs, results indicate that only 38.9 and 40% could be verified on the maternal cards (figures 5 and 6). Figure 6: Number of ANC visits as verified by maternal card    29 Figure 7: Number of IPTs as verified by maternal card It is also worth noting that 5.4 and 2.6% of the mothers (UHC and non-UHC members respectively) who had ANC cards did not have IPT recorded on their maternal cards. Mothers’ knowledge on danger signs during pregnancy that indicate the need to seek health care was assessed and the results are illustrated in figure 8.    30 Figure 8: Knowledge dangers signs during pregnancy While the majority of mothers reported that they were counseled on where to deliver (89.5 and 76.8% for UHC and non-UHC members respectively), actual health facility deliveries were lower at 66.3 and 53.7% for UHC and non-UHC members respectively. Details of place of delivery are in the figure 9 below. Figure 9: Place of delivery    31 From figure 9 it can be seen that a significant number of mothers (18.9 and 28.7% for UHC and non-UHC members respectively) gave birth at home. Of the mothers who delivered at health facilities, they mentioned the following as the people who assisted them during delivery. Objective 4: to assess the status of health service delivery in the district An assessment was done covering 36 health facilities from the level of health center III and above to find out the specific services that are provided and the organization of service delivery at the health facilities in the district. The services that were looked at are HIV counseling and testing and prevention of mother to child transmission, antenatal & emergency obstetric care services, integrated management of childhood illness, diagnosis and treatment of malaria, and commodity management. Although all the facilities surveyed reported that they offered all the above mentioned services, results show a limited number of personnel received the recommended frequency of in-service training and support supervision to offer these services. Results of the health care management indicators are shown below. Table 7: Health care management results No. Indicators Definition of an indicator Comments Baseline Den. Num. Percent CI (±) EO P 4.1 % trained in VCT in the past 3 years 153 54 35.3 7.85 4.2 % received in-service training in PMTCT 153 62 40.5 8.0 4.3 % received training in management of 3rd stage labour in last 3 year 79 111 48 43.2 9.55 90 Target not achieved 4.4 % of staff received in-service training in reproductive health life savings skills within the last 3 year 111 42 37.8 9.35 4.5 % of facilities with MOH policy guidelines on ANC and Obstetric services 44 36 27 75.0 15.05 65 Target achieved 4.6 % of staff received in-service training in IMCI within the last 3 year 57 167 61 36.5 7.55 70 Target not achieved 4.7 % facilities supervised for IMCI regularly 36 32 88.9 11.5 40 Target achieved % of stock outs in the past 6 months 4.8 % of ACT stock outs in the past 6 months 36 22 61.1 16.7 4.9 % of injectable contraceptives stock outs in the past 6 months 36 31 86.1 12.4 4.10 % of ORS stock outs in the past 6 months 36 10 27.8 15.5 4.11 % of cotrimoxazole / septrin stock outs in the past 6 months 36 19 52.8 17.05    32 4.12 % of facilities who provide information on nutrition& hygiene, LLINs, breastfeeding, STI/HIV/AIDs prevention, warning signs, post natal care. 36 36 100.0 0.0 From the table 7, 35.3 and 40.5% of the staffs received in-service training in HIV counseling and testing and in prevention of mother to child transmission of HIV in the past three years respectively. Another 37.8% received in-service training in reproductive health life saving skills within the past three years while only 43.2% received training in management of 3rd stage of labor which is far below the EOP target of 90% and the benchmark of 79%. It is however, important to note that 88.9% of the facilities were regularly supervised for IMCI surpassing end of program goal of 40 percent. Results also show that 75% of the health facilities had MOH policies and protocols/ guidelines on antenatal, prenatal and obstetric services exceeding the program target of 65%. Evaluation of commodity management at the health facilities revealed stock outs of ORS and ACTs (Coartem) in the district in the last six months prior to the survey stood at 61.1 and 27.8% respectively. These are considered key drugs in the treatment of diarrhea and malaria. Objective 5: To compare the difference of the impact project interventions had on UHC members compared to non-members. Respondents were asked if they had ever heard about or were members of health plans and results of this assessment are in the table 7. Table 8: Health plan indicator results No. Indicator Baseline EOP UHC (%) EOP Non-UHC 5.1 % of respondents who have heard about UHC 0 100 51.6 5.2 Reasons why respondents who have heard about UHC did not join 0 NA Don’t know about it 38.8 Don’t know how to join 34.7 Don’t have a group to join with 8.2 Think it is too expensive 6.1 Don’t think it works 2 Other …………(%) 10.2 5.3 Average amount of money respondents spent on care, counseling and or drugs (total costs) at a clinic, hospital, pharmacy and other health treatment. 0 <= 5,000 42.1 5,001-10,000 21.1 10,001-20,000 11.6 20,001-50,000 18.9 50,001-100,000 4.2 Not sure 2.1 0 NA <= 5,000………..% 35.8    33 5,001-10,000….% 22.1 10,001-20,000…% 22.1 20,001-50,000….% 12.6 50,001-100,000…% 3.2 Not sure………% 4.2 5.4 % of respondents who sought timely health care 0 88.4 72.7 From table 8, it can be seen that 51.6% of the mothers interviewed who are non-UHC members had heard about health plans. Comparing the timely treatment seeking behavior of mothers, 88.4 and 72.7% of UHC and non-UHC members respectively reported that they sought timely treatment for their children. The survey also sought to compare the costs incurred by the respondents in accessing treatment, 42.1% of the UHC members reported to have spent up to approximately US$2 for accessing treatment services within 2 weeks prior to the survey, while 35% of non-UHC members reported to have spent approximately the same amount of money. 32.7% of the members spent approximately between US$ (2- 10) compared to 44.2 non-members spending approximately the same amount of money. 3.2 and 4.2% of UHC and non-UHC members reported spending approximately between US$ 25 – 50. Respondent who had heard about UHC were asked to give reasons why they did not join, table 8 below presents the findings: Table 9: Reasons why respondents who heard about health plans did not join Reason Count % Don’t know about it 19 38.8 Don’t know how to join 17 34.7 Don’t have a group to join with 4 8.2 Think it is too expensive 3 6.1 Don’t think it works 1 2 Others 5 10.2 Total 49 100.0 From the results presented in table 9 above, most of the mothers 73.5% who are currently non-UHC members either don’t know about UHC or don’t know how to join. Among the sources of information where the respondents heard about UHC, radio 44.9% was the main source of information. Other sources of information were health center 16.3%, friends 14.3%, UHC members 10.2% and others accounted for 8.2%. It is important to note that local leaders and the church were the least sources of information for those who had heard about UHC accounting for 4.1 and 2% respectively. Table 10: Rapid catch indicators Rapid catch indicators Indicators Baseline UHC (%) Non-UHC (%)    34 6.1 % of children 0-23 months who are underweight (- 2SD from the median weight for age, according to the WHO/NCHS reference population) 21.8 27.4 6.2 % of children age 0-23 months who were born at least 24 months after the previous surviving child 70.6 78.2 6.3 % of children aged 0-23 months whose births were attended to by a skilled health personnel 27.2 74.7 52.6 6.4 % of mothers of children aged 0-23 months who received at least 2 tetanus toxoid injections before the birth of their youngest child 65.7 53.7 53.7 6.5 % of infants aged 0-5 months who were exclusively breastfed in the last 24 hours 100 94.1 100 6.6 % of infants aged 6-9 months receiving breast milk and complementary foods 67.7 62.5 66.7 6.7 % of children 12-23 months who were fully vaccinated against the five vaccine preventable diseases before their first birth day (measles was used as a proxy indicator for complete immunization) 67.7 73.3 43.5 6.8 % of children 12-23 months who received a measles vaccine 67.7 45.1 46.2 6.9 % of children 0-23 months who slept under LLIN the previous night (in malaria- risk areas only) 47.4 38.9 6.10 % of mothers who know at least two signs of childhood illness that indicate the need for treatment 75.8 81.1 77.9 6.11 % of sick children age 0-23 months who received increased fluids and continued feeding during an illness in the past 2 weeks 47.8 16.7 9.1 6.12 % of mothers to children age 0-23 who cite at least 2 known ways of reducing the risk of HIV infection 64.2 83.2 78.9 6.13 % of mothers to children age 0-23 months who wash their hands with soap/ash before food preparation, before feeding children, after defecation and after attending to a child who has defecated 2.1 5.3 4.2 From table 10 above, it can be seen that 21.8 and 27.4% of the children aged 0-23 months for UHC and non-UHC members respectively are underweight (below -2SD from the median weight for age, according to WHO/NCHS reference population). It can also be seen that only 16.7 and 9.1% of the children aged 0- 23 months that were sick in last two weeks for UHC and non-UHC members respectively received increased fluids and continued feeding during illness. Mother’s knowledge of at least two signs of childhood illness that indicate the need for treatment was assessed, it was found out that 81.1 and 77.9% of UHC and non-UHC members respectively know at least two sign of childhood illness that indicate the need for treatment. Immunization is one way of reducing mortality in children associated with five vaccine preventable diseases. Assessment of children aged 12-23 months showed that among those who    35 had their immunization cards, 73.3 and 43.5% (for UHC and non-UHC members respectively) of those whose cards were seen by the interviewers had completed vaccination against five vaccine preventable diseases. It is also important to note that from the assessment of mother’s hand washing practices that only 5.3 and 4.2% of UHC and non-UHC members respectively washed their hands before food preparation, before feeding children, after defecation and after attending to a child who has defecated.    36 DISCUSSION In this chapter the results are discussed and compared with finding from other studies like results from the 2006 UDHS, the annual sector performance report and also with the national targets as outlined in the Health Sector Strategic Plan II 2009/10-2014/15(HSSP II.) The discussion is presented according to the key intervention areas. Prevention of malaria Malaria remains the single most common cause of mortality and morbidity in Uganda accounting for 30- 50% of outpatient burden and 35% of hospital admissions (MOH 2007). One of UHC’s child survival objectives was to reduce the incidence of malaria in Bushenyi district for children under five and pregnant women. To ensure the achievement of this objective UHC promoted the demand for LLINs and distributed LLINS free to members of health schemes and to pregnant women at health facilities during the early years of the program. LLINs were also sold at subsidized prices. All LLINs distributed were long lasting in order to reduce the need for re treatment. The community was mobilized against malaria through training on proper LLIN use, the importance of ANC and receiving IPT 2 times during pregnancy, early recognition and care seeking for fever, compliance with prescribed treatment and appropriate malaria recognition and case management at the provider level. Results from the survey show that the percentage of children with fever in the last two weeks was high at 46.3 and 57.9% for UHC and non-UHC members respectively compared to the program target of 19% and the 2006 UDHS national level of 41%. The prevalence of malaria was high and this may be associated with a low net coverage of 61.1 and 63.2% for UHC and non-UHC members respectively compared to the national target of 70%. The low net coverage may be attributed to lack Global Fund free net distribution and high prices of nets in the market. The high malaria prevalence could also mean that either the few available nets are not being used correctly or treated frequently enough. Results further show that 53.7 and 56.8% of children under 2 for UHC and non-UHC members respectively actually slept under a net the night before the survey. There is need to continue behavior change communication (BCC) efforts to make sure that mothers adopt the practice of preventing children from mosquito bites by sleeping under LLINs. In A Qualitative Study on the Practices and Their Behavioral Determinants among Women, Men and Health Workers in Bushenyi District, July 2006, the Child Survival program team found that pregnant women do not sleep under nets because they lack information on proper net use; they also believed that nets can cause death to human beings sleeping under them and complained that nets cause discomfort. This finding was the basis of the program’s behave framework which emphasized training on proper use before distribution of nets. LLIN use especially for children under five and pregnant women is emphasized because they are particularly susceptible to severe disease and anemia. One of the objectives of the HSSP II is treatment of CU5 within 24 hours of onset of fever. The survey found that although 90.5% of the children with fever sought treatment, only 23.3 and 16.4% sought the treatment on the same day for UHC and non-UHC members respectively. This is below the program and national targets of 84% and 80% respectively. Sanitation, hygiene and control of diarrheal diseases Through volunteers UHC/CS mobilized and trained the community on warning signs of diarrhea and dehydration, home management of diarrhea, ORS information and proper preparation, importance of continued feeding and when to seek treatment. The community also received BCC support on safe    37 disposal of human feces, improved hygiene including hand washing practices, safe water sources, safe water use and storage. Poor sanitation and hygiene has remained a major predisposing factor to the high disease burden in Uganda with 70-80% attributed to poor sanitation, hygiene and poor living conditions (MOH 2007). The chances of getting diarrhea are high among those who use contaminated water, unhygienic practices in food preparation and have poor excreta disposal. The UDHS of 2006 found that children who lived in households with non improved toilet facilities were more likely to get diarrhea than those with improved toilet facilities. In Bushenyi, the percentage of households with an improved water source, bore hole, public tap and protected spring/well was found at 64.2 and 55.8% for UHC and non-UHC members respectively. Safe excreta disposal however performed poorly with only 72.6 and 54.5% of the mothers interviewed disposing off the children’s excreta hygienically for UHC and non-UHC members respectively i.e. dropped into toilet or rinsed away and water discarded into toilet. This performance is much lower than the UDHS 2006 finding for the Southwestern region and national coverage level of 86.5% and 77% respectively. The level is also below the set program target of 82%. Hand washing practices were found to be poor in the district compared to the program targets. Only 27.4 and 9.5% of the households studied for UHC and non-UHC members respectively had a designated hand washing facility while 64.2 and 56.8% of the mothers for UHC and non-UHC members respectively washed their hands with water and soap after defecation as compared to the program target of 82% and still very few (21.1 and 20% for UHC and non-UHC members) mothers washed their hands with soap and water after attending to a child who has defected. During the survey majority of the mothers mentioned other times when they washed their hands with soap and water and the most prominent of these was “after digging” for both UHC and non-UHC members. To them a soiled hand is more dangerous warranting to be washed with soap and water than one that is not visibly dirty yet potentially more dangerous. The poor hand washing practices can be explained by findings from the qualitative study. At the baseline, mothers said they do not was hands with water and soap for most of the recommended times because: they did not see anything bad with children’s feces, they also believed that a person should only wash hands if he/she has touched feces, some thought that using soap for washing hands is waste of soap while some actually did not know the dangers of not washing hands or poor disposal of children’s feces. These findings suggest that change in behavior is occurring at a slow rate and therefore calls for concerted effort from all partners to ensure that these simple yet key behaviors are adopted if the health of the community is to be improved. Some of the lessons learnt from Kaliro district which is highlighted as a success story in improving sanitation in the HSSP I were: 1. Collaboration between district departments and political leadership is crucial for improving sanitation 2. There is need for a deliberate effort from all departments to achieve a given objective 3. Mobilization using local media like FM stations is very useful 4. There is need to sustain the efforts so that communities appreciate the role of appropriate sanitation (HSSP I 2006/07) The same deliberate effort by all partners especially the local government and DHT was agreed upon during the dissemination workshop as a strategy to improve the sanitation. Prevalence and management of diarrhea    38 Diarrheal diseases remain among the five major causes of morbidity and mortality in Uganda (MOH 2007). The number of children with diarrhea in the last two weeks in Bushenyi district was found to be high at 32.6 and 33.7% for UHC and non-UHC members respectively compared to the program target of 20% and the Southwestern level of 30.6% (UDHS 2006). The major complication of diarrhea is dehydration which can be easily treated with oral rehydration solution (ORS). During community mobilization sessions the volunteers trained mothers on how to prepare ORS and the importance of increased fluids and feeding for a child with diarrhea so as to reduce dehydration and minimize the adverse consequences of diarrhea. Despite the fact that ORS has been found to be a simple and effective method of managing dehydration in diarrhea cases, its uptake is still low in Bushenyi district at 25.8 and 18.8% for UHC and non-UHC members respectively compared to end of project target of 30%. In A Qualitative Study on the Practices and Their Behavioral Determinants among Women, Men and Health Workers in Bushenyi District, July 2006, the Child Survival program team found that barriers to the use of ORS include the belief by mothers that ORS is merely water and therefore does not work, also that ORS increases water in a child who has a lot of water already. The other reason why mothers do not use ORS was found to be the fact that some of them cannot afford it. Feeding practices for a sick child were better compared to the UDHS findings of 2006. The survey found out that 48.4 and 25% of the children (for UHC and non-UHC members respectively) aged 0-23 months were offered more fluids during illness. This was better performance compared to the UDHS national finding of 20.4%. This level was also higher than the program target of 20%. The other key behavior that is promoted by the program is giving the same or more to eat for a sick child. This behavior was achieved based on the survey results which show that 35.5 and 28.1% of the sick children for UHC and non-UHC members respectively were given more than usual to eat when they were sick compared to a program goal of 20%. This too is much higher than what was found in the UDHS results of 2006 where only 7% were given more than usual to eat. Another important result worth noting is that the majority of the mothers 97.9% for both UHC and non￾UHC members know at least one crucial sign that indicates a child needs treatment. This could improve the mother’s health care seeking behavior for the child not allowing illness to progress. This makes treatment easy. However, more training and BCC is still needed for mothers on seeking appropriate treatment for the children, importance of ORS in the management of diarrhea and reemphasizing the importance of giving a sick child more to drink and to eat. This is a role that has to be done by every stakeholder but the program and the DHT in particular may have to consider innovative ways to make ORS more easily accessible to the community. Maternal and newborn care The UHC/CS objective under this intervention area is to increase the percentage of women receiving improved ANC, delivery and post partum care. Major activities include the mobilization and training of mothers and the community at large through the CORP on the importance of planning for safe birth, ANC, routine counseling and testing, prevention of mother to child transmission of HIV and training on maternal and newborn care (MNC). To encourage ANC and delivery at health facilities, UHC distributed free Mamakits to mothers at the IPT2 and LLINs to babies born at health facilities. Health workers are also given refresher training in reproductive health and life saving skills and active management of third stage of labor so as to ensure standard care management of the mothers.    39 Results from the assessment show that 96.8 and 92.6% of the mothers studied (UHC and non-UHC members respectively) attended at least one ANC visit with 94.7 and 87.5% (for UHC and non-UHC members respectively) of them seeing a nurse or midwife. This result is comparable to the finding from the UDHS of 2006 where nationally 94% of the women received ANC from qualified health personnel with 84% seeing a nurse or midwife. UDHS results for South Western Uganda where Bushenyi district is found showed 91.4% of the mothers receiving ANC from skilled health personnel with 70.4 seeing a nurse or midwife. This finding is encouraging meaning that at least mothers in Bushenyi are able to have their medical problems during pregnancy detected early and treated. Given that Bushenyi is a rural district it is not surprising therefore that the most common health worker seen for ANC is a nurse or midwife. The fact that some of the mothers reported to have seen a traditional birth attendant (TBA) for ANC raises concern in regard to the quality of ANC that can be offered by the TBA and therefore training of the mothers on the importance of seeing a skilled health worker for ANC should be continuous. TBAs should also be encouraged to refer mothers to skilled health personnel since according to the national policy guidelines for sexual and reproductive health and rights TBAs are not to provide ANC services. In line with World Health Organization (WHO) guidelines, the MOH recommends that a woman who is having a normal pregnancy attends four antenatal care visits. The MOH target is to increase the attendance of four visits per pregnancy from 42 to 50%. This indicator was at 47% according to the results from the UDHS of 2006. Results from the KPC survey found that only 34.3% of mothers attended the recommended four ANCs as compared to the project target of 85% and the national target of 50%. The low ANC visits observed during the KPC could partly be due to omissions especially in instances where mothers had books instead of maternal cards. So in case one book was misplaced only those visits recorded in the available book were recorded. Nevertheless it is obvious that the project target which was set basing on the baseline results is very high. This is because during the baseline the four ANC visits were based on mother’s recall and the results were therefore high. The MOH’s target is to have 60% of pregnant women receiving a complete dose of IPT2. By the end of 2006/2007 FY the coverage was at 42%. In Bushenyi, the study results show that IPT2 coverage is at 67.6 and 65.8% for UHC and non-UHC members respectively which are a big improvement from the 2006 UDHS finding of 24.6% for Southwestern Uganda. This is further supported by the fact that most of the mothers interviewed insisted that they had actually got the recommended two IPTs but in line with the study protocol only those seen on the cards/books were recorded. Despite the fact that most of the mothers reported to have been counseled on breastfeeding, child spacing, danger signs during pregnancy, where to deliver, transport plans to delivery place and having a birth kit and even if in Bushenyi the performance for the above variables was found to be better than the findings of the 2006 UDHS, the district performance was still below the set project targets. This is most likely due to heavy workloads experienced by the health workers therefore not having adequate time to counsel mothers. Nevertheless effort to ensure that all mothers attending ANC do not miss this opportunity should be put in place. One strategy would be to carry out group counseling during ANC. Counseling mothers on danger signs during pregnancy is essential if mothers are to recognize early the need to seek heath care during pregnancy. From the KPC survey it was found that only 75.8 and 65.3% of the mothers (for UHC and non-UHC members respectively) were counseled on danger signs during pregnancy compared to the target of 90%. Furthermore it was clear from interviewing the mothers that very few knew the crucial danger signs during pregnancy like bleeding, fever, shortness of breath, swelling of face, hands and feet. Most of the mothers mentioned other signs like abdominal pain, stomach pain, vaginal discharge, vomiting which are not necessarily the most critical signs. There is need therefore to scale up the counseling on danger signs if mothers are to seek health care early and hence improve their health and that of their babies.    40 It was encouraging to find that 66.3 and 53.7% of the mothers (for UHC and non-UHC members respectively) delivered at a health facility; this is much higher than was found in the UDHS of 2006 where nationally only 41% of the mothers delivered at a health facility with the figure for South Western Uganda being 31.3%. Assistance during childbirth is an important variable that influences the birth outcome and the health of the mother and the infant. This is because the skills and performance of the birth attendant determine whether or not he or she can manage complications and observe hygienic practices. So it was also encouraging to find that 83.6 and 90.2% of the mothers (for UHC and non-UHC members respectively were assisted by trained health personnel. The national target is 50%. It is worth noting that although the percentage of mothers delivering with TBA has reduced from the 17% stated in the national policy guidelines and service standards for sexual and reproductive health and rights, a sizable proportion 3.2 and 12.3% (for UHC and non-UHC members respectively) of the mothers delivered with a TBA. Further sensitization on the importance of delivering with a skilled health personnel therefore needs to be re emphasized so as to improve birth outcomes and the health of the mothers and their babies. Like has been found in many studies for mothers who delivered at home, it was found that a significant number of mothers 18.9 and 28.7% delivered at home. The majority were assisted by either their mothers or mothers-in-law. This finding has significant bearing on who to target when delivering key behavior change messages. It is important for other ongoing and any forthcoming programs and all stakeholders to note the importance of the mothers and mothers-in-law and target them for the trainings. This is because they will determine whether or not the mothers choose to deliver with skilled personnel or not. Health care management The UHC/CS objective under this intervention area is improved health care management especially for women of reproductive age and children under 5. Major activities included training of in-service health personnel in reproductive health life saving skills, Integrated Management of Childhood Illnesses, active management of 3rd stage of labor and support supervision for IMCI. A total of 203 health workers were trained in reproductive health life saving skills, integrated management of childhood illness (IMCI), and active management of 3rd stage of labor. The program worked together with the Bushenyi DHT to conduct regular support supervision and follow-up visits for trained health workers in IMCI and MNC to ensure that they practiced the skills they acquired in the training. Results show that less than half of the health workers have received in-service training in the five major areas of; HIV counseling and testing (35.5%), PMCT (40.5%), IMCI, active management of the 3rd stage of labor (43.2%) and reproductive health life saving skills (37.8%). Support supervision and follow-up reports show that some of the trained health workers left the district to serve in other districts and organizations, others were affected by internal transfers to manage lower health units which were not targeted by the survey and a few died. Furthermore, the survey did not target those who were trained to answer the questionnaires but anyone who was available at the health facility at the time of the visit. With few staffs having the skill to deliver the service, it means overload and this can affect the quality of the service offered. Of the total number of facilities surveyed, 75% were found to be having the MOH policy guidelines on ANC and obstetrics services. This is a good sign that the health workers can deliver these services according to national standards. In addition, 88.9% of the facilities reported that they were regularly supervised for IMCI. Findings also revealed high stock out rates for ACTs and ORS. Stock out of ACTs was found to be at 61.1% and ORS at 27.8%. ACTs and ORS are essential drugs which are recommended by MoH for treatment of malaria and diarrhea respectively in children under 5 years. With high stock outs of essential    41 drugs, it means children under 5 years old will not be able to access treatment within 24 hours of onset of fever which can lead to complications that result in deaths. It is important for the stakeholders to work together to improve stock management and commodity tracking at all levels to minimize stock outs of essential drugs. One of UHC’s objectives is to build local organizational capacity to manage health schemes and growth of the schemes is fundamental in the achievement of this goal. The end of program target is to have at least 16,000 members which is approximately 2% of the total population of Bushenyi. This target has not been achieved and is compounded by many challenges including the scarcity of providers especially in rural areas. Marketers report having identified interested groups in some areas but then fail to find an appropriate provider for them. It should be noted that the most well distributed health facilities in Bushenyi are government health facilities but the services in these hospitals is free hence making government facilities unsuitable for the prepaid health schemes. In addition the marketers report difficulty in decision making by groups to join the plans, high drop outs of members, unaffordability of premiums by some sections of the community, limited number of active groups and the fact that prepaid health schemes and health insurance is a new concept and its benefits are not as tangible as some of the challenges they face. Although the findings from the survey show that there is a better treatment seeking behavior among UHC members compared and non members, comparing the spending patterns for UHC members and non￾members when accessing treatment seems to suggest that there is no significant difference in the spending patterns of UHC members and non members when accessing treatment services. This may be because the majority of government health facilities which are fairly uniformly distributed in the district offer free medical treatment.    42 CONCLUSIONS Malaria 1. The incidence of malaria among children under 2 is still high in the district and the mothers’ disease preventive behavior is still low in that almost half of the children are not sleeping under LLINs. 2. IPT uptake by expectant women as verified by the maternal card has increased. 3. Treatment seeking behavior for mothers of children under 2 within 24 hours of onset of fever is still low. 4. Number of households that own LLINs is still low Sanitation, hygiene and control of diarrheal disease 1. Access of the district population to improved water source is fairly good. 2. A good number of mothers were found to be knowledgeable on at least two danger sign that a child needs treatment. 3. Poor sanitation characterized by poor fecal disposal, poor hand washing practices, and low coverage with latrines that have a solid slab remain a major challenges in the district. 4. Prevalence of diarrhea in district has reduced though the target has not been achieved. 5. Home management of diarrhea in the district is poor with minimal use of ORS and caretakers are not giving more fluids to children with diarrhea Maternal and newborn care 1. Mothers’ ANC seeking behavior is good, however very few mothers are attending the recommended four ANC visits. Also maternal card retention is still very poor. 2. Uptake of RCT services is high among mothers attending ANC. 3. Delivery at health facility and with a skilled birth attendant has improved though a significant number of mothers are still giving birth at home or with TBAs. 4. Majority of the mothers are not knowledgeable on the crucial danger signs during pregnancy that indicate the need to seek medical treatment. Health plans Overall results show that the program impact was stronger for UHC member than non-members. Low membership is a major challenge to the sustainability of the health plans and this is made worse by limited availability and distribution of providers and the fact that majority of the population are not knowledgeable how the schemes work.    43 RECOMMENDATIONS 1. The DHT should be commended for the indicators that showed improvements 2. The DHT and other stakeholders in district need work together to improve access to essential drugs and increase knowledge on how to prepare ORS as it is a key life saving resource during diarrhea. 3. VHT need to emphasize the need for caretakers to provide more fluids to children with diarrhea because this replaces fluids lost from diarrhea and reduces the potential of death from dehydration. Warning signs of when a child needs treatment also needs to be reemphasized. 4. The DHT needs to continue following up HW to ensure that HW make time to give all women key messages on preparing for child birth, child spacing, breast feeding and other key considerations. 5. VHT need to continue sensitizing mothers on the importance of seeing skilled HW for ANC. They should also encourage the TBAs to refer pregnant women to skilled health personnel since according to the national policy guidelines for sexual and reproductive health and rights, TBAs are not mandated to provide ANC services. 6. There is need to scale up counseling on danger signs during pregnancy that indicate need for treatment. 7. Further sensitization on the importance of delivering with skilled health personnel needs to be reemphasized so as to improve birth outcomes and the health of their babies. 8. All stakeholders in the district need to work together to improve the stock management and commodity tracking at all level to minimize stock outs of essential drugs.    44 Annex 1: KPC Tool – Child Survival Program End of Project Questionnaire INFORMED CONSENT Hello. My name is ______________________________, and I am working with the Uganda Health Cooperative. We are conducting a survey and would appreciate your participation. I would like to ask you about your health and the health of your youngest child under the age of two. This information will help the Uganda Health Cooperative to plan health services and assess whether it is meeting its goals to improve children’s health. The survey usually takes 30 minutes to complete. Whatever information you provide will be kept strictly confidential and will not be shown to other persons. Participation in this survey is voluntary and you can choose not to answer any individual question or all of the questions. However, we hope that you will participate in this survey since your views are important. At this time, do you want to ask me anything about the survey? RESPONDENT AGREES TO BE INTERVIEWED . . . …………….. . . 1 RESPONDENT DOES NOT AGREE TO BE INTERVIEWED . . . . . . . 2 END Interviewer’s Name:……………………………….. Signature…………………………………… Sub-County Name………………………….............. Village Name………………………………. Questionnaire Number……………………................ Supervisor’s Name…………………………. RECORD INTERVIEW DATE Day Month Year 1. How old are you? ________ (in complete years) 2.( A-D). What is the name, sex, date of birth of the youngest child that you gave birth to and is still alive? BE SURE THIS CHILD IS 0- 23 MONTHS OLD. (E). What is the name, sex, date of birth of the next youngest child that you have given birth to? NAME B- SEX C-AGE (Months) D-DATE OF BIRTH A 1. MALE 2. FEMALE __ __/___ __/___ ___ DD MM YY E 3. MALE 4. FEMALE __ __/___ __/___ ___ DD MM YY THE FOLLOWING QUESTIONS PERTAIN ONLY TO YOUNGEST CHILD A. May I weigh (NAME)? 1. YES 2. NO SKIP TO Q.C    45 B. IF MOTHER AGREES, WEIGH THE CHILD AND RECORD WEIGHT BELOW. RECORD TO THE NEAREST TENTH. ___ ___ . ___ KILOGRAMS C. Did you ever breastfeed (NAME)? 1. YES 2. NO SKIP TO Q.E D. How long after birth did you first put (NAME) to the breast? 1. IMMEDIATELY/WITHIN FIRST HOUR AFTER DELIVERY 2. AFTER THE FIRST HOUR E. I would like to ask you about the types of liquids and foods that (NAME) consumed yesterday during the day or at night. Did (NAME) have. . .READ EACH OF THE FOLLOWING AND PLACE A CHECK MARK IN THE BOX NEXT TO EACH ITEM CONSUMED. LIQUID/FOOD CONSUMED IN LAST 24 HOURS? A Breastmilk B Plain water C Other liquids D Mashed, pureed, solid or semi solid foods E Anything else? SPECIFY F. Do you have a card where (NAME’S) vaccinations are written down? IF ‘YES’ASK ‘May I see it please?’ 1. YES, SEEN BY INTERVIEWER 2. NOT AVAILABLE (lost/misplaced, not in home) SKIP TO Q.H 3. NEVER HAD A CARD SKIP TO Q.H 8. DON’T KNOW SKIP TO Q.H G. RECORD INFORMATION EXACTLY AS IT APPEARS ON (NAME’S) VACCINATION CARD. Vaccination Day Month Year 1. BCG 2. Polio 0 3. Polio 1 4. Polio 2 5. Polio 3 6. DPT 1 7. DPT 2 8. DPT 3 9. Measles 10. Vitamin A H. Did (NAME) ever received an injection to prevent measles? 1. YES 2. NO    46 8. DON’T KNOW Maternal and Newborn Care I would like to ask you about the time when you gave birth to (NAME). NO QUESTIONS AND FILTERS CODING CATEGORIES SKIP 3 Do you have a maternal health card for your pregnancy with (NAME)? Yes, Seen .. . . . . . . ………………... . . . 1 Not Available . . . . . . . ……………….. 2 Never Had A Card .. . ……………….. . 3 5 5 4 LOOK AT CARD AND RECORD THE NUMBER OF ANTENATAL VISITS WHILE MOTHER WAS PREGNANT WITH (NAME). 1 Visit 2 Visit 3 Visit 4 or more Visit 5 Did you see anyone for antenatal care while you were pregnant with (NAME)? Yes………………………………………1 No…………………………………..….. 2 9 6 IF YES: Whom did you see? Doctor ... ……………………………….A Nurse/Midwife……………………….... B Anyone else? PROBE FOR THE TYPE OF PERSON AND RECORD ALL PERSONS MENTIONED BY THE MOTHER. Other Person Traditional Birth Attendant…………..C Community Health Worker ……….…D Family Member………….…………...E Other _________________________ F (Specify) 7 How many times did you attend Antenatal service during (NAME) pregnancy? NUMBER OF TIMES ………………………… 8 During your prenatal check, were you counseled on the following: Where to deliver? A Yes B No Planning transport to place of delivery? C Yes D No Having a birth kit? E Yes F No Danger signs of pregnancy? G Yes H No Breastfeeding? I Yes J No Child Spacing? K Yes L No 9 Before you gave birth to (NAME) did you receive an injection in the arm to prevent the baby from getting tetanus, that is, spasms after birth? (CONFIRM FROM MATERNAL CARD/BOOKLET) 1. Yes Seen a Card 2. Yes Card Not Available 3. No 4. Can’t recall 11 10 How many times did you receive such an injection? 1. Once 2. Twice 3. More Than Two Times 4. Can’t recall    47 11 What symptoms during pregnancy indicate the need to seek health care? RECORD ALL MENTIONED. Fever……………..……………………..1 Shortness Of Breath. …………………...2 Bleeding . . . . . . . ……………… ……..3 Swelling of Body/Hands/Face……….....4 Other_____________ ……………5 (Specify) 12 When you were pregnant (NAME), did you take any drugs to prevent you from getting malaria? (CONFIRM FROM MATERNAL CARD/BOOKLET) Yes Seen A Card. . . . . . .. …………..… 1 Yes Card Not Available. . ……………..2 No ………………………….................. 3 15 15 13 LOOK AT CARD AND RECORD THE NUMBER OF IPT WHILE MOTHER WAS PREGNANT WITH (NAME). 1. IPT 1 2. IPT 2 3. IPT 3 4. No IPT 14 Which drug did you take? RECORD ALL MENTIONED Fansidar . . . . .. . . . . . . . . . ………...….. 1 Chloroquine . . . . . .. . . . ………….…...2 Don’t Know……………...…………….3 Other…………………………………...4 (Specify) 15 During your prenatal check, were you given any counseling on HIV/AIDS? 1 Yes 2 No 16 During your pregnancy, did you test for HIV/AIDS? 1 Yes 2 No 17 What can a person do to avoid getting AIDS or the virus that causes AIDS? DO NOT PROMPT. CIRCLE ALL MENTIONED. 1. Nothing 2. Abstain From Sex 3. Use Condoms 4. Limit Sex to One Partner/Stay Faithful to One Partner 5. Limit Number of Sexual Partners 6. Avoid Sex with Prostitutes 7. Avoid Sex with Persons Who Have Many Partners 8. Avoid Intercourse with Persons of the Same Sex 9. Avoid Sex with Persons Who Inject Drugs Intravenously 10. Avoid Blood Transfusions 11. Avoid Injections 12. Avoid Kissing 13. Avoid Mosquito Bites 14. Seek Protection from Traditional Healer 15. Avoid Sharing Razors, Blades 16. Other_______________________ (Specify)    48 18 Where did you give birth? IF SOURCE IS HOSPITAL, HEALTH CENTER, OR CLINIC, WRITE THE NAME OF THE PLACE _________________________________ Home.……………............................ 1 Government Health Facility. ……. . 2 NGO Health Facility.. .. ……………3 Clinic ……………………………….4 Traditional Birth Attendant. .. .......... .5 Other________________7 (Specify ) 19 Who assisted with the delivery of (Name)? VERIFY WITH MATERNAL CARD. Anyone else? PROBE FOR THE TYPE(S) OF PERSON(S) AND RECORD ALL MENTIONED. IF RESPONDENT SAYS NO ONE ASSISTED, PROBE TO DETERMINE WHETHER ANY ADULTS WERE PRESENT AT THE DELIVERY. Doctor……………………………..1 Nurse/Midwife……....................... .2 Auxiliary Midwife………………....3 TBA……………………...……….. 4 Community Health Worker………..5 Family Member…..…………….... 6 Other……………………………….7 No one……… ……………….… .. 8 20 Did you use a mama kit while giving birth to (NAME) 1 Yes 2 No 21 After (Name) was born, did any health care provider or traditional birth attendant check on (Name’s) health? Yes……………………1 No…………………... 2 23 22 Who checked on (*Name’s) health at that time? Anyone else? PROBE FOR THE MOST QUALIFIED PERSON AND RECORD ALL MENTIONED. Doctor…………………………………1 Nurse…................................................. 2 Midwife……………………...……..... .3 Auxiliary Midwife………………….....4 Other Health Staff with midwifery skills…………………………………. .5 Trained TBA…………………….…….6 Trained Community Health Worker.... 7 TBA………………………………...... 8 Community Health Worker………....... 9 Relative/Friend……………………......10 No one…………………………….......11 23 How long after the birth of (Name) did the first check take place? IF LESS THAN ONE DAY, CIRCLE 0 AND RECORD HOURS; IF LESS THAN ONE WEEK CIRCLE 1 AND RECORD DAYS; IF MORE THAN 6 DAYS CIRCLE 2 AND RECORD WEEKS. Hours 0 Days 1 Weeks 2 Can’t recall………..…998    49 Integrated Management of Childhood Illnesses (IMCI) 24. Sometimes children get sick and need to receive care or treatment for illness. What are the sign of illness that would indicate that a child needs treatment? DO NOT PROMPT. CIRCLE ALL MENTIONED. 1. Don’t know 2. Looks unwell or not playing normally 3. Not eating or drinking 4. Lethargic or difficult to wake 5. High fever 6. Fast or difficult breathing 7. Vomits everything 8. Convulsions 9. Other _____________________ (Specify) 25. Did (NAME) experience any of the following in the past two weeks? CIRCLE ALL MENTIONED BY RESPONDENT. A. Diarrhea 1- Yes 2- No (Mark Q.39) B. Blood in Stool 1- Yes 2- No C. Cough 1- Yes 2- No D. Difficult Breathing 1- Yes 2- No E. Fast Breathing/Short, Quick Breaths 1- Yes 2- No F. Fever 1- Yes 2- No (Mark Q.28) G. Malaria 1- Yes 2- No H. Other ________________ 1- Yes 2- No I. None Skip to Q.33 1- Yes 2- No 26. “When (NAME) was sick, was he/she offered less than usual to drink, about the same amount, or more than usual to drink?” 1. Less Than Usual 2. Same Amount 3. More Than Usual 27. When (NAME) was sick, was he/she offered less than usual to eat, about the same amount, or more than usual to eat? 1. Less Than Usual 2. Same Amount 3. More Than Usual Malaria Prevention NO QUESTIONS AND FILTERS CODING CATEGORIES SKIP 28 Did (NAME) experience fever in the past two weeks (CHECK FROM Q. 25) Yes . . . . . . . . . . . . . . . . . . . . . . . . . . . . .1 No . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2 33 29 Did you seek advice or treatment for (NAME’S) fever? Yes . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 No . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2 33 30 Where did you first go for advice or treatment? IF SOURCE IS GOVERNMENT, NGO OR CLINIC, WRITE THE NAME OF THE PLACE Government Health Facility. ……. . ….1 NGO Health Facility . . . ….. . . . . . . .. ..2 Clinic …………………………………..3 Other Source Traditional Practitioner. . … . .. ............ 4 Shop . . . . . . . . . . . . . . . . … . ….. . .. . ..5    50 (NAME OF PLACE) Pharmacy . . . .. . . . . . . . . . . …. . . . . …. 6 Community Distributors …………...…..7 Friend/Relative . . . . . . …………….…. 8 Other____________________________9 (Specify) 31 How long after you noticed (NAME’S) fever did you seek treatment from that person/place? Same Day . . . . . . . . . . . . . . . . . . .. . . . 1 Next Day . . . . . . . . ... . . . . . . . . . . . . . . 2 Two Days . . . . . . . .... . . . . . . . . . . . . . . 3 Three Or More Days... . . . . . . . . . . . . .. 4 32 Which medicines were given to (NAME) for his/her fever? CIRCLE ALL MEDICINES THAT WERE GIVEN. IF MOTHER IS UNABLE TO RECALL DRUG NAME(S), ASK HER TO SHOW THE DRUG(S) TO YOU. IF SHE IS UNABLE TO SHOW YOU THEM, SHOW HER TYPICAL ANTI-MALARIALS AND HAVE HER IDENTIFY WHICH WERE GIVEN. Chloroquine . . .………..........…… …..A Fansidar . . . . . ……….......………...….B Camaquinne . ..……….......……...……..C Quinine . . . . .. . . ………….…………...D Artenum………………..……...……….E Coartem……………….…………….….F Metakelfin………………………....…...G Homapak……………………..….....…..H Other Drugs Aspirin…………………….………….....I Panadol………………………….............J Co-trimoxazole……………….………...K Other_________________________ L (Specify) 33 Do you have a mosquito net in your house? Yes . . . . . . . . . . … . . . . . . . . . . . . . . . . ..1 No . .. . . . . . . . . . . . . . . . . . . . . . . …. . . 2 39 34 Where did you get/buy this mosquito net from? Health Units . . . . . . . . . . . . . . . . . . . ….. 1 Shop/Market . . . . . . . . . . . . . …. . . . …..2 Pharmacy /Drug Store………………….3 Hawkers .……………………………….4 Given To Me Free……...……………….5 Others …………………………………. 6 (Specify) 35 Who slept under a mosquito net last night? RECORD ALL MENTIONED. Child (Name) A-Yes B-No Myself C-Yes D-No Other (Specify) E-Yes F-No 36 Was the mosquito net mentioned above pre￾treated? (Name type of net) Yes………………………………………1 No……………………………………….2 Don’t know………………………….… 3 38 37 How long ago was the mosquito net last soaked or dipped in a liquid treated to repel mosquitoes or bugs? IF LESS THAN 1 MONTH AGO, RECORD 00 MONTHS. IF LESS THAN 2 YEARS AGO, RECORD MONTHS AGO. IF 12 MONTHS AGO, PROBE FOR EXACT NUMBER OF MONTHS. Months More than 2 years ago………………1 Don’t know………………………….2 38 ASK TO SEE THE MOSQUITO NETS FOR HOLES OR TEARS. NO HOLES/TEARS= GOOD CONDITION. VISIBLE HOLES Good Condition . . . . . . . . …... . . .... . .. 1 Damaged . . . . . . . . . . . . . . . .. ……. . . . 2    51 AND/OR TEARS=DAMAGED. (CHILD’S NET MOSTLY) Diarrhea, sanitation and hygiene NO QUESTIONS AND FILTERS CODING CATEGORIES SKIP 39 Has (NAME) had diarrhea in the past two weeks? (CHECK Q.25) Yes…………………………… 1 No…………………………….. 2 41 40 What was given, if anything, to treat the diarrhea? Anything else? RECORD ALL MENTIONED Nothing ……………………...... 1 ORS Packet…………………….2 Home-Made Fluid. . . . .. . . . …. 3 Pill or Syrup …………………...4 Injection………………………..5 (Iv)Intravenous Fluid…………..6 Home Remedies/Herbal Medicines………………………7 Other _____________________ 8 (Specify) 41 What is the main source of drinking water for members in this household? Borehole……………………….. 1 Public Tap………………………2 Protected Spring.….…….…...….3 Unprotected Spring..……..……..4 Rainwater Collection……………5 Surface Water (River/Pond/Lake/Dam/Stream)..6 Other______________________7 (Specify) 42 Do you do anything to make your water safe for drinking? Yes……………………………... 1 No…………………….…………2 44 43 What do you do? PUR.………………….………....1 Water guard…...……...…………2 Boiling…………..….….………..3 Other______________________ 4 (Specify) 44 The last time (name) passed stool, where were the feces disposed off? IF “WASHED AWAY”, PROBE WHERE THE WASTE WAS DISPOSED IF “DISPOSED” PROBE WHERE IT WAS DISPOSED OF SPECIFICALLY Dropped Into Toilet Facility……1 Rinsed Away Water Discarded Into Toilet Facility………………………….2 Water Discarded Into Sink Or Tub Connected To Drainage System......................................... 3 Disposed Into Solid Waste/ Trash………………………….....4 Somewhere in the yard…….........5 Outside premises………….......... 6 Buried…….…………………..…7 Did nothing…………………….. 8 Other______________________ 9 (Specify)    52 45 What kind of toilet facility does this household use? (CONFIRM BY OBSERVING) Flush toilet…………………….. 1 Ventilated Improved Pit latrine.. 2 Covered pit latrine…………….. 3 Simple pit latrine without slab... 4 Bucket latrine…………………... 5 No facility (field/bush/plastic bag) etc……………………………….6 46 Observation. Is there a hand washing device such as a tap, basin, bucket, or sink? This item should be within view or brought by the respondent within one minute. If item is not present within one minute, tick no even if brought later. Yes……………………………1 No……………………………..2 47 When do you wash your hands with soap? DO NOT PROMPT: CIRCLE ALL MENTIONED BY RESPONDENT. 1. Before food preparation 2. Before feeding children 3. After defecation 4. After attending to a child who has defecated 5.Other_____________________ (Specify) Prepaid health coverage-UHC 48. Have you ever heard about Uganda Health Cooperative? 1- Yes 2- No ………………. SKIP to Q.52 49. IF YES, where from? ______________________________ (Source/ Institution) Local leader ………………………………… 1 Church ………………………………….2 UHC members………………………………… 3 Friends ………………………………………… 4 Health centre ………………………..……… 5 Radio ……………………………….. 6 Other ____________________________............7 Specify 50. Are you a member of UHC? 1- Yes ………………. Skip To Q.52 2- No 51. If NO Why not?…………………….. I don’t know about it ……………….…..1 I don’t know how to join…………….….2 I don’t have a group to join with …….…3 It is too expensive………………………4 I don’t think it works……………………5 Other __________________________ ..6 Specify    53 52. In the past two weeks how much did you spend on care, counseling and/or drugs (total cost) at a clinic, hospital, pharmacy and/or other health treatment? Less than 5,000………………………….………………..1 Between 5,001-10,000……………………………………2 Between 10,001-20,000…………………………………..3 Between 20,001-50,000…………………………………..4 Between 50,001-100,000…………………………………5 More than 100,000………………………………………..6 Cannot guess/not sure……………………………………..7 Thank you for your time. END Child Survival Key Objective 1: Reduce incidence of malaria in Bushenyi district for children under 2 and pregnant women. Indicator Numerator Denominator % of children under 2 with fever in the last two weeks Number of children with fever in the last two weeks ( Q28.1) All children studied % of children under 2 who slept under an LLIN last night. Number of children who slept under an LLIN last night (Q35.A) All children studied % of mothers who slept under an LLIN last night. Number of mothers who slept under an LLIN last night (Q35.c) All mothers of children studied % of children under 2 with fever in the last 2 wks who received anti malarial treatment Number of sick children who received anti malarial treatment (Q32.A,B,C,D,E,F,G,H) Number of children who had fever in the last two weeks (Q28.1) % of pregnant women receiving IPT as verified by maternal card/ book Number of mothers who received IPT 1 and 2 (13.1 AND 2 ) verified by maternal card Number of Mothers who sought prenatal care and have their maternal cards (Q4.2,3AND 4) % of children under 2 with fever in the last 2 weeks who sought treatment on the same day Number of sick children who sought treatment same day (Q31.1) Number of children who had fever in the last two weeks (Q28.1) % of households with children 0-23 months that own at least one mosquito net/LLIN Number of households with mosquito nets (Q33.1) All households in the survey Objective 2: Reduce incidence of diarrhea in Bushenyi district for children under 5 Indicator Numerator Denominator % of children under 2 with diarrhea in the last two weeks Number of children with diarrhea in the past 2 weeks (Q39.1) All children in the survey % of mothers who know at least two signs that a child under 5 needs treatment. Number of mothers who mentioned at least 2 signs that a child needs treatment (Q24.TWO ANSWERS 2-8) All mothers in the survey % of care takers/mothers who know at least one signs that a child under 2 needs Number of mothers who mentioned at least 2 signs that a child needs All mothers in the survey    54 treatment treatment (Q24.ONE ANSWER 2-8) % of households who use improved water source (borehole, public tap, or protected dug well.) Number of mothers that reported improved water sources (Q41.1, 2 OR 3) This referred to Borehole, Public taps, Protected spring All households in the survey % of households with a designated hand washing station with a covered container for water Number of households where hand washing facilities were observed (Q46.1) All households in the survey % of caretakers who usually wash hands with soap before food preparation Number of mothers who reported washing hands Before food preparation (Q47.BEFORE FOOD) All mothers in the survey % of caretakers who usually wash hands before feeding children Number of mothers who reported washing hands Before feeding children (Q47.BEFORE FEEDING CHILDREN) All mothers in the survey % of caretakers who usually wash hands after defecation Number of mothers who reported washing hands After defecation (Q47.AFTER DEFECATION) All mothers in the survey % of caretakers who usually wash hands and after attending to a child who has defecated Number of mothers who reported washing hands After attending to a child who has defecated (Q47.AFTER ATTENDING TO A CHILD WHO HAS DEFECATED) All mothers in the survey % of households who safely disposed of their child's feces the last time s/he passed stool Number of caretakers who dispose off children’s feces (Q44.1 and 2) All households in the survey % of households with access to a covered pit latrine Number of household with access to pit latrines (Q45.1, 2, OR 3) All households in the survey % of children 0-23 months with diarrhea in the last two weeks who received ORS Number of children with diarrhea in the last 2 weeks who received ORS (Q40.2) Number of children with diarrhea in the past 2 weeks (Q39.1) % of children aged 0-23 months who were offered more fluids during illness Children who were offered more fluid during illness (Q36.3) Number of children with diarrhea in the past 2 weeks (Q39.1) % of children aged 0-23 months who were offered more food during illness Children who were who offered more to eat during illness (Q27.3) Number of children with diarrhea in the past 2 weeks who can eat (Q39.1) Objective 3: Increased % of pregnant women receiving improved ANC, delivery and post partum care. Indicator Numerator Denominator % of women with 4 ANC visits as verified by maternal card/ book. Mothers who had at least 4 ANC visits as verified by maternal card/ book(Q4.4) Mothers with maternal card/ book (Q3.1) % of pregnant women seeking RCT services Mothers who sought VCT services (Q16.1) All mothers in the survey % caretakers counseled on breastfeeding Mothers counseled on breastfeeding (Q8.breastfeeding=I) All mothers in the survey    55 % caretakers counseled on importance of child spacing Mothers counseled on importance of child spacing (Q8.child spacing=1) All mothers in the survey % caretakers counseled on danger signs of pregnancy Number of mothers counseled on danger signs during pregnancy (Q8.danger signs=1) All mothers in the survey % of pregnant women counseled on where to deliver Number of pregnant mothers counseled on where to deliver (Q8.where to deliver=1) All mothers in the survey % of pregnant women counseled on transport plans to delivery place Number of pregnant mothers counseled on transport plans to delivery place(Q8.transport plans to delivery place=1) All mothers in the survey % of women who delivered with a skilled health professional as verified by maternal card/ book Number of mothers (verified) who delivered with a health professional (Q19.1 or 2 or 2 or 4 or 5) All mothers in the survey % of women who delivered at a health facility Number of mothers who delivered at a health facility (Q18.2,3 OR 4) All mothers in the survey % of caretakers mothers who know at least 2 danger signs during pregnancy Number of mothers who know mention at least two pregnancy danger sign (Q11. any 2(1, and 2, and 3, and 4) All mothers in the survey % of pregnant women who used a birth kit Number of pregnant mothers who used a birth kit (Q20.1) All mothers in the survey Objective 4: Improved health care management especially for women of reproductive age and children under 5 Indicators Definition of an indicator Denominator Numerator % trained in VCT in the past 3 years Number of staff involved directly with client counseling diagnosis services (Section 1: QN 101 ) Number of staff received in￾service training related to VCT in the past three years (Section 1: QN 102 ) % received in-service training in PMTCT Number of staff involved directly with client counseling diagnosis services (Section 1: QN 101 ) Number of staff received in￾service training related to PMTCT in the past three years (Section 1: QN 103 ) % received training in management of 3rd stage labour in last 3 year Number of staff involved directly in management of antenatal/third stage labor and Emergency Obstetrics care services (Section 3: QN 301 ) Number of staff received in￾service training related to management of 3rd stage labor (Section 3: QN 302 ) % of staff received in-service training in reproductive health life savings skills within the last 3 year Number of staff involved directly in management of antenatal/third stage labor and Emergency Obstetrics care services (Section 3: QN 301 ) Number of staff received in￾service training related to management of 3rd stage labor (Section 3: QN 303 )    56 % of facilities with MOH policy guidelines on ANC and Obstetric services All facilities in survey that provided ANC service Number of facilities with MoH guideline (QN323) % of staff received in-service training in IMCI within the last 3 year Number of staff involved directly in management of IMCI services (Section 4: QN 401 ) Number of staff received in￾service training related to IMCI (Section 3: QN 402) % facilities supervised for IMCI regularly All facilities in survey that provide IMCI services Number of facilities supervised regularly for IMCI (Q 403=1) Rapid Catch Indicator Key Indicators Definition of an indicator Denominator Numerator 1. Percentage of children age 0– 23 months who are underweight (-2 SD from the median weight￾for-age, according to the WHO/NCHS reference population) Number of children age 0– 23 months in the survey who were weighed (Q.A.1) Number of children age 0–23 months whose weight (Q.B) is – 2SD from the median weight of the WHO/NCHS reference population for their age 2. Percentage of children age 0– 23 months who were born at least 24 months after the previous surviving child Number of children age 0– 23 months in the survey who have an older Sibling (Q.2.E) Number of children age 0–23 months whose date of birth is at least 24 months after the previous sibling’s date of birth (Q.2.E) 3. Percentage of children age 0– 23 months whose births were attended by skilled health personnel Number of children age 0– 23 months in the survey (95 or 19) Number of children age 0–23 months with responses =1 (‘doctor’), 2 (‘nurse/midwife’), or 3 (‘auxiliary midwife’) for Q.19. 4. Percentage of mothers with children age 0–23 months who received at least two tetanus toxoid injections before the birth of their youngest child Number of mothers of children age 0– 23 months in the survey (95 or 19) Number of mothers of children age 0–23 months with responses=2 (‘twice’) or 3 (‘more than two times’) for Q.10 5. Percentage of children age 0–5 months who were exclusively breastfed during the last 24 hours Number of infants age 0–5 months in the survey (Q.2.C 0- 5 months only) Number of infants age 0–5 months with only response=A (‘breastmilk’) for Q.E 6. Percentage of children age 6–9 months who received breastmilk and complementary foods during the last 24 hours Number of infants aged 6–9 months in the survey (Q.2.C 6- 9 months only) Number of infants age 6–9 months with responses= A (‘breastmilk’) and D (‘mashed, pureed, solid, or semi-solid foods’) for Q.E    57 7. Percentage of children age 12– 23 months who are fully vaccinated (against the five vaccine preventable diseases) before the first birthday Number of children age 12– 23 months (Q.2.C 12-23 months only) in the survey who have a vaccination card that was seen by the interviewer (response=1 ‘yes, seen by interviewer’ for Q.F.1) Number of children age 12–23 months who received Polio3 (OPV3), DPT3, and measles vaccines (Q.G.5 and 8 and 9) before the first birthday, according to the child’s vaccination card (as documented Q.F.1) 8. Percentage of children age 12– 23 months who received a measles vaccine Number of children age 12– 23 months in the survey (Q.2.C 12-23 months only) Number of children age 12–23 months with response=1 (‘yes’) for (Q.H.1) 9. Percentage of children age 0– 23 months who slept under an insecticide-treated net (in malaria risk areas) the previous night Number of children age 0– 23 months in the survey (95 or 19) Number of children age 0–23 months with ‘child’ mentioned among responses to Q.35.A AND Q.36.1 10. Percentage of mothers with children age 0–23 months who cite at least two known ways of reducing the risk of HIV infection Number of mothers of children age 0– 23 months in the survey (95 or 19) Number of mothers of children age 0–23 months who report at least two of the signs listed in 2 through 8 for Q,17 11. Percentage of mothers with children age 0–23 months who report that they wash their hands with soap/ash before food preparation, before feeding children, after defection, and after attending to a child who has defecated Number of children surveyed who were reportedly sick in the past two weeks (children with any responses other than I (‘none”) for Q.25) Number of mothers of children age 0–23 months who mention responses 1 through 4 for Q.47 12. Percentage of mothers of children age 0–23 months who know at least two signs of childhood illness that indicate the need for treatment Number of mothers of children age 0– 23 months in the survey (95 or 19) Number of mothers of children age 0–23 months who mention at least two of the responses that relate to safer sex or practices involving blood for Use responses for Question 24 answers 2- 8 13. Percentage of sick children age 0–23 months who received increased fluids and continued feeding during an illness in the Number of mothers of children age 0– 23 months in the Number of children age 0–23 months with response=3 (‘more than usual’) for Q.26 AND response=2 (‘same amount’) or 3 (‘more than usual’) for Q.27    58 past two weeks survey (95 or 19) UHC Indicator Key Indicators Definition of an indicator Denominator Numerator % of respondents who have heard about UHC # of non-member respondents (Q50.2) # of non-UHC respondents (Q50.2) who have heard about UHC (Q48.1) Reasons why respondents who have heard about UHC did not join. # of non-member respondents (Q50.2) who have heard about UHC (Q48.2) # of respondents who answered with reasons why they did not join tallied separately by reason. (Q51.1 or 2 or 3 or 4 or 5 or 6) Average amount of money respondent spent on care, counseling and/or drugs (total cost) at a clinic, hospital, pharmacy and/or other health treatment? Total # of questionnaire respondents for each group separately (Q50.1 and Q50.2.) Average amount of money spent on care, counseling and/or drugs (total cost) at a clinic, hospital, pharmacy and/or other health treatment? (Q52.1 or 2 or 3 or 4 or 5 or 6 or 7)    59 Annex 2: Health Facility Assessment Questionnaire INTEGRATED HEALTH FACILITY ASSESSMENT: IDENTIFICATION PARTICULARS Official use only 1. LQAS No. ________ OUT OF TOTAL SAMPLE__________ 2. DISTRICT: 3. COUNTY: 4. HEALTH SUB-DISTRICT: 5. FACILITY: 6. FACILITY CODE: 7. FACILITY TYPE: Please circle appropriate category 1=Regional hospital; 2=District hospital; 3=Health center IV; 4=Health center III 8. OPERATING AUTHORITY: Please circle appropriate category 1=Government; 2=Non-government Organization; 3=Private Sector; 4=CBO, 5=FBO 9. FACILITY LOCATION: Please circle appropriate category 1=urban; 2=rural 10. NAME OF HEAD OF FACILITY: 11. TITLE OF HEAD OF FACILITY: 12. DATE OF INTERVIEW: 13. NAME OF INTERVIEWER: ASSESSMENT OF SERVICES OFFERED FROM HEAD OF INSTITUTION First I want to ask some general questions about specific services that are provided and the organization of service delivery at this facility. Then, I would like to speak with the person or people responsible for providing each service your facility offers to ask for some more detailed questions for each service. Does this facility offer the following services? ENTER YES OR NO IN BOX FOR RESPONSE Note: If service not offered, do not ask for in-charge staff for that service, and skip that service’s section Who is responsible for providing this service that I may talk to today? SECTION SERVICE Yes/No NAME & TITLE OF PRESENT, IN￾CHARGE STAFF TO BE INTERVIEWED TODAY 1 HIV Voluntary Counseling & Yes……1    60 Testing and PMTCT No…….2 2 Antenatal & Emergency Obstetric Care Services Yes……1 No……..2 3* Integrated Management of Childhood Illness (IMCI) Yes……1 No……..2 4* Diagnosis & Treatment of Malaria Yes…….1 No………2 5 Commodity Management (Drug store or Pharmacy) Yes…….1 No………2 6 Health Plan Yes……..1 No………2 * NOTE: IMCI and malaria services may not be provided through a single department. If this is the case, ask Head of institution to direct you to the person she or he feels is most likely to have knowledge about the facility’s services in these areas. SECTION 1: HIV: VOLUNTARY COUNSELING & TESTING (VCT) AND PMTCT SERVICES NAME OF RESPONDENT: _____________________ TITLE OF RESPONDENT: _____________________ QUESTION CODING CATEGORIES 101 How many staff are directly involved with client counseling, diagnosis or management for this service? INCLUDE ALL STAFF WHO HAVE RESPONSIBILITY FOR ANY OF THESE ACTIVITIES, EXCLUDING THE LAB TECHNICIAN. 102 Among these staff, how many have received any in-service training related to VCT within the last 3 years? 103 Among these staff, how many have received any in-service training related to PMTCT within the last 3 years? 104 Observe if the posters are displayed openly, translated in the local language and at least 5 posters Adequate……………………. 1 Inadequate……………………2 105 Does the facility have any visual aids to use when providing VCT or PMTCT services? YES……………. 1 NO…………….. 2 106 Is HIV counseling and testing for pregnant women upon request, required or recommended? Upon request….. 1 Required……….. 2 Recommended…. 3 107 For pregnant women who test positive, do you offer ARVs (antiretrovirals) for the prevention of HIV transmission from the woman to her child? YES……………. 1 NO…………….. 2 108 Does the facility offer post-test infant feeding counseling to pregnant women who test positive for HIV? YES……………. 1 NO…………….. 2 109 Do you have a private space for delivering VCT and PMTCT services? YES……………. 1 NO…………….. 2 110 Ask: Can I see it? Adequate …………….1    61 If seen, is it spacious; private; and clean? (meets the criteria= adequate; doesn’t meet the criteria= inadequate) Inadequate……………2 111 Does the facility have a register or other record where you record information on clients who receive this service? YES……………. 1 NO…………….. 2 112 Of those tested in the last six months, how many tested positive for HIV? Total: Males: Females: 113 How many of the overall clients tested for HIV were pregnant women? Total: 114 How many of the clients who tested positive for HIV were pregnant women? Total: 115 Does this facility have the MOH policy and guidelines/protocols on HIV/AIDS VCT services? Ask to see. Yes, seen by Interviewer…………1 No, Not seen by interviewer………2 116 Does this facility have the MOH policy and guidelines/protocols on PMTCT? Ask to see. Yes, seen by Interviewer…………1 No, Not seen by interviewer………2 SECTION 2: LABORATORY SERVICES NAME OF RESPONDENT: _____________________ TITLE OF RESPONDENT: _____________________ QUESTION CODING CATEGORIES 201 What is the qualification of the person who is in charge of the quality of the laboratory work? Lab assistant certificate…...1 Lab technician diploma …..2 Lab technologist diploma or higher.………. 3 None……………………... 4 Other_________________6 202 Does the facility have guidelines or protocols for how the laboratory is to conduct diagnostic testing for HIV? YES……………………...1 NO …………..…………..2 203 Ask to see. Yes seen…………………..1 No not seen……………….2 204 Does this facility have the laboratory capacity to conduct HIV tests? YES……………………...1 NO …………..…………..2 205 Does the laboratory have a separate register where laboratory test results for HIV tests are recorded? YES……………………...1 NO …………..…………..2 206 Does the facility have all equipment and reagents required to conduct a malaria test today? YES……………………...1 NO …………..…………..2 207 Is there a separate register where laboratory test results for malaria are recorded? YES……………………...1 NO …………..…………..2    62 SECTION 3: ANTENATAL, PRENATAL AND MERGENCY OBSTETRIC CARE SERVICES NAME OF RESPONDENT: _____________________ TITLE OF RESPONDENT: _____________________ QUESTION CODING CATEGORIES 301 How many staff are directly involved with management of antenatal and/or third stage of labor/ emergency obstetric care services? INCLUDE ALL STAFF WHO HAVE RESPONSIBILITY FOR ANY OF THESE ACTIVITIES, EXCLUDING THE LAB TECHNICIAN. 302 Among these staff, how many have received training in active management of the third stage of labor in the last 3 years? 303 Among these staff, how many have received in-service training in reproductive health life saving skills within the last 3 years? Does this facility provide the following services: 304 Administration of antibiotics by injection or by intravenous infusion? YES……………………...1 NO …………..…………..2 305 Administration of anticonvulsants by injection or by intravenous infusion? YES……………………...1 NO …………..…………..2 306 Administration of anticonvulsants for pre-eclampsia and eclampsia by injection or by intravenous infusion? YES……………………...1 NO …………..…………..2 307 Manual removal of placenta? YES……………………...1 NO …………..…………..2 308 Removal of retained products of conception (e.g., manual vacuum aspiration)? YES……………………...1 NO …………..…………..2 309 Assisted vaginal delivery? YES……………………...1 NO …………..…………..2 310 Intravenous fluids? YES……………………...1 NO …………..…………..2 311 Provision of iron and folic acid supplements YES……………………...1 NO …………..…………..2 312 Examination of the mother to evaluate the pregnancy (fetal growth and maternal health) YES……………………...1 NO …………..…………..2 313 Surgery or caesarian section? YES……………………...1 NO …………..…………..2 314 Blood transfusion? YES……………………...1 NO …………..…………..2 315 Does this facility provide the following products to expectant mothers? A. Mama Kits B. Making a Birth Plan prior to delivery YES…………………………….1 NO……………………………..2 YES…………………………….1 NO……………………………..2 Does this facility provide the following Information: 316 Proper nutrition and hygiene for baby and mother YES…………………………….1 NO……………………………..2    63 317 Malaria prevention/ Insecticide Treated Nets use YES…………………………….1 NO……………………………..2 318 Importance of continued breastfeeding and child welfare YES…………………………….1 NO……………………………..2 319 Prevention of STI/HIV/AIDS YES…………………………….1 NO……………………………..2 320 Warning signs of pregnancy and complications YES…………………………….1 NO……………………………..2 321 Post natal care and family planning YES…………………………….1 NO……………………………..2 322 Dangers of self medication and use of traditional medicines during labor YES…………………………….1 NO……………………………..2 323 Does this facility have the MOH policy and guidelines/protocols on antenatal, prenatal and obstetric care services? Ask to see. Yes, seen by Interviewer…………1 No, Not seen by interviewer………2 SECTION 4: INTEGRATED MANAGEMENT OF CHILDHOOD ILLNESS (IMCI) NAME OF RESPONDENT: _____________________ TITLE OF RESPONDENT: _____________________ QUESTION CODING CATEGORIES 401 How many staff are directly involved with management of IMCI services? INCLUDE ALL STAFF WHO HAVE RESPONSIBILITY FOR ANY OF THESE ACTIVITIES, EXCLUDING THE LAB TECHNICIAN. 402 Among these staff, how many have received any in-service training related to IMCI within the last 3 years? 403 Does this facility receive regular supervision for IMCI? YES…………………………………….…...1 NO …………..……………………………..2 404 When was your last IMCI supervision? This month …………………..…………..…1 This quarter …………………...………...….2 In the last 6 months …………………….…...3 In the last year .………………….…………4 405 What are the danger signs that indicate that a child needs urgent medical care? CIRCLE ALL THAT IS MENTIONED – DO NOT PROMPT Convulsions …………………………..….1 Loss of consciousness .…………….……..2 Severe anemia or “lack of blood” shown by pale lips or palms .….……….3 Difficulty in breathing …………………...4 Extreme weakness (unable to sit or stand) ……………………..……….…5 Vomiting everything/severe vomiting ………………………………..6 Child not able to drink or breastfeed ……………………………….7    64 SECTION 5: DIAGNOSIS & TREATMENT OF MALARIA NAME OF RESPONDENT: _____________________ TITLE OF RESPONDENT: _____________________ QUESTION CODING CATEGORIES 501 How many staff are directly involved with management of malaria? INCLUDE ALL STAFF WHO HAVE RESPONSIBILITY FOR ANY OF THESE ACTIVITIES, EXCLUDING THE LAB TECHNICIAN. 502 Among these staff, how many have received any in-service training in management of severe and complicated malaria within the last 3 years? 503 During their antenatal checks, are pregnant mothers provided with Antimalarial tablets like Fansidar? YES………………………………………..1 NO………………………………………....2 504 Does the facility provide LLINs for pregnant mothers to protect them and the baby from getting malaria? YES……………………………….……….1 NO……………………………..…………..2 SECTION 6: COMMODITY MANAGEMENT (DRUG STORE OR PHARMACY) NAME OF RESPONDENT: _____________ TITLE OF RESPONDENT: _____________________ QUESTION CODING CATEGORIES 601 In the last 6 months, were there any stock outs for: ORS YES…………………………….1 NO……………………………..2 Cotrimoxazole /Septrin YES…………………………….1 NO……………………………..2 Mebendazole YES…………………………….1 NO……………………………..2 Amoxicillin oral YES…………………………….1 NO……………………………..2 Naladixic acid YES…………………………….1 NO……………………………..2 Ciprofloxacin YES…………………………….1 NO……………………………..2 Doxycycline YES…………………………….1 NO……………………………..2 Metronidazole / Flagil YES…………………………….1 NO……………………………..2 Injectable contraceptives YES…………………………….1 NO……………………………..2 Contraceptive Pills YES…………………………….1 NO……………………………..2    65 Condoms YES…………………………….1 NO……………………………..2 Chloroquine YES…………………………….1 NO……………………………..2 Fansidar YES…………………………….1 NO……………………………..2 ACT (Coartem) YES…………………………….1 NO……………………………..2 Quinine YES…………………………….1 NO……………………………..2 Injectable diazepam (valium) YES…………………………….1 NO……………………………..2 Ergometrine YES…………………………….1 NO……………………………..2 Anesthesia YES…………………………….1 NO……………………………..2 Homa Pak YES…………………………….1 NO……………………………..2 Oxytocin YES…………………………….1 NO……………………………..2 Misoprostol YES…………………………….1 NO……………………………..2 Zinc YES…………………………….1 NO……………………………..2 Iron/Foliate YES…………………………….1 NO……………………………..2 LLIN YES…………………………….1 NO……………………………..2 Mama Kits YES…………………………….1 NO……………………………..2 Drug Proper storage—FOR INTERVIEWER INFORMATION ONLY OBSERVE STORAGE—IS THE DRUG STORED PROPERLY? YES NO 12. Oxytocin / Pitocin 10 units Is it stored in 15-30 degrees C and protected from freezing? (Not more than six months from the time of manufacture) 13. Ergometrine .2 mg Is it protected from light and from freezing? (Protected from the light) 14. Misoprostol 400- 600 mcg Is it at room temperature, in a closed container? THANK YOU!    66 END OF CHECKLIST Annex 3: CS End of program evaluation ‐ Data collection plan June 2010 Supervisio n Area Parishes Villages Team Leaders Health Centers Vehicle Buhweju 21/06/2010 1. Kajumbura 2. Katongo 3. Nyakaziba 4. Rukiri 5. Karembe Bihanga s/c 1. Kyoma I 2. Ruzinga 3. Kyiiha 4. Nyamashannju 5. Kyambugu Edidah (Team A) Bihanga HC III MCP UAJ 096Z 1. Kirambi 2. Nyakitoko 3. Rwajere 4. Rwanyamabare 5. Rushayo Burere s/c 1. Kigango 2. Omukashenyi 3. Katagata 4. Mutongo 5. Kamuhiga Herbert (Team B) Burere HC III Nyakashaka HC III H1 1. Bitsya 2. Kasharara 3. Katara 4. Kitega 5. Mushasha Karungu s/c 1. Kasana 2. Karungu I 3. Rwemisha 4. Mpiija II 5. Kininga Grace (Team C) Karungu HC III H2 1. Kashenyi 2. Kibimba 3. Kyeyare 4. Rwengye Rwengwe s/c 1. Kashenyi 2. Kibimba 3. Kyankuba 4. Nyakishojwa A Arthur (Team D) Butare HC III Nsiika HC IV CS UAJ 605E UHC Members Dr. James (Team E) H3 Bunyarugu ru 22/06/2010 1. Kakaari 2. Katerera 3. Kyabakara 4. Mugyera 5. Mwongerwa 6. Katanda 7. Munyonyi Katerera s/c 1. Kakaari I 2. Kacu 3. Katabago II 4. Kaiita 5. Kagogoro I 6. Kanyatanga 7. Munyomyi II Edidah (Team A) Katerera HC III MCP UAJ 096Z 1. Kazinga Kichwaba s/c 2. Kichwamba 3. Kirugu 4. Rumuri 5. Kikumbo Katunguru / and 1. Kawombe Kichwaba s/c 2. Omumasaka A 3. Kafuru II 4. Kataara I 5. Kirugu II Herbert (Team B) Katunguru HC III Kichwamba HC III H1    67 1. Butoha 2. Buzenga 3. Mushumba 4. Ndangaro 5. Ndekye 6. Nyabubare 7. Nyakiyanja Ryeru s/c 1. Busonga II 2. Kamacumu I 3. Nyakakaka B 4. Kyanika II 5. Mugogo III 6. Bururuma 7. Ndangara Grace (Team C) Rugazi HC III H2 UHC Members Arthur (Team D) CS UAJ 605E UHC Members Dr. James (Team E) H3 Ruhinda 23/06/2010 1. Karimbiro 2. Kigarama 3. Rusheregyenyi 4. Kati 5. Kiyanga 6. Rwoburunga Bitereko s/c 1. Kabare I 2. Omukibare 3. Busheregyenyi Kanyabwanga s/c 4. Kati A Kiyanga s/c 5. Kashasha 6. Kashambya Edidah (Team A) Bitereko HC III Nyakatsiro HC III Kanyabwanga HC III Rwoburunga HC III MCP UAJ 096Z 1. Mayanga 2. Nyakatete 3. Ryengyerero 4. Rwanja 5. Bukongoro 6. Nyakizinga 7. Ryakitanga Kabira 1. Buyayo 2. Runyinya 3. Rutooma B 4. Rwakutaka Mutara s/c 5. Kinyemi I 6. Rutooma 7. Ryakitanga Central Grace (Team C) Kabira HC III Mutara HC III H1 1. Mushunga 2. Rukararwe 3. Bitooma 4. Kirembe 5. Bukari 6. Kyanzaire Mitooma s/c 1. Nyakahita 2. Kabarore 3. Kagongo 4. Rubaasa Kanshenshero s/c 1. Bweza 2. Kashenshero II Amelia (Team B) Mitooma HC IV Kashenshero HC III Bubangizi HC III H2 UHC Members Arthur (Team D) CS UAJ 605E UHC Members Dr. James (Team E) H3 Sheema Bugongi s/c Edidah Bugongi HC MCP    68 24/06/2010 1. Karera 2. Rugarama 3. Buraro 4. Kasaana West 5. Kyarushakara 6. Muhito 7. Kishabya 1. Rwamurungura 2. Ruhorobero II Kitagata s/c 3. Kikore II 4. Kihanga I 5. Bwooma III 6. Nyamiyaga Shuuku s/c 7. Kamweru (Team A) III Hope Medical Center HC III Shuuku HC IV UAJ 936Z 1. Kanyinasheema 2. Kiziba 3. Kyabandara 4. Nyakashambya 5. Rwamujojo Kagango s/c 1. Rwabutura 2. Rubanga 3. Kibutamo 4. Rugongi 5. Nyamitooma Amelia (Team B) Kabwohe HC IV Kabwohe BMC Mushanga HC III H1 1. Kitojo 2. Kyangundu 3. Kyangyenyi 4. Rweibare 5. Katooma 6. Kyabuharambo 7. Masheruka Kyangyenyi s/c 1. Rugarama II 2. Kyabahija I 3. Kahama 4. Rweibare Kigarama s/c 5. Rwengiri I 6. Bubare 7. Kangore Grace (Team C) Kigarama HC III H2 UHC Members Arthur (Team D) CS UAJ 605E UHC Members Dr. James (Team E) H3 Igara 25/06/2010 1. Bumbaire 2. Kainamo 3. Kiyaga 4. Ruharo 5. Buyanja 6. Karaaro 7. Kitwe 8. Rutooma Bumbaire s/c 1. Rwakati 2. Nyarurambi 3. Nyabwina 4. Ruyayo Kyeizoba s/c 5. Rukukuru 6. Karaaro 7. Rwentuha 8. Misingano Grace(Te am C) Kabushaho HC III Kyeizooba HC III MCP UAJ 096Z    69 1. Bijengye A 2. Katikatmwe 3. Nyeibingo 4. Ward II 5. Kakanju 6. Mazinga Kyamugimbi s/c 1. Bijengye A 2. Kacence 3. Ihanda B.Town Council 4. Ruhandagazi Cell Kakanju s/c 5. Kakanju B 6. Ryamabengwa Amelia (Team B) Kyamugimbi HC III Bushenyi HC III Kakanju HC III BMC HC III H1 1. Kigoma 2. Nkanga 3. Bitooma 4. Kyamuhunga 5. Nshumi Nyabubare s/c 1. Nyamiko 2. Birimbi Kyamuhunga s/c 3. Kashororo 4. Nyamashobe 5. Nyakabare Edidah (Team A) Nyabubaare HC III KIU Hospital Bitooma HC III Ishaka Adventist Hospital H2 UHC Members Arthur (Team D) CS UAJ 605E UHC Members Dr. James (Team E) Comboni Hospital H3 Teams and car allocation Team A – MCP – UAJ 096Z 1. Kananga Edidah – Team Leader 2. Julius Bejuna - Rukungiri 3. Matsiko Mudashil 4. Muhwezi Robert - Rukungiri 5. Nuwe Blick Team D – CS UAJ 605E 1. KanyanyeruArthur – Team Leader 2. Janepher Bashabomwe - Rukungiri 3. Musinguzi Benson 4. Akello Joyce Team B - Car Hire 1 (H1) 1. Herbert – Team Leader 2. Nabaza Gloria 3. Immaculate - Rukungiri 4. Asimwe Jude Team E – Car Hire 3 (H3) 1. Dr. Mukankusi James – Team Leader 2. Byamukama Peter Waiting list 3. Owaruhanga Baylon 4. Namanya Amelia Team C – Car Hire 2 (H2) 1. Karungi Grace – Team Leader 2. Bagonza Simon 3. Arinaitwe Doreen - Rukungiri 4. Muhwezi Martin Technical Backstop: Dr. Owembabazi Coordination: Michael Oturu    70 Annex 4: Check list for interviewers No. Survey Items Yes No 1 Car Available 2 Fuel/money for fuel 3 Money for guides 4 Lunch and water for self 5 Introductory Letter 6 Copies of LQAS questionnaires 7 Table of random numbers) 8 Papers and pencils for village Mapping 9 Maternal card 10 Digital Camera 11 Daily Interview Form – for interviewers 12 Note book 12 Airtime for group leaders    71 Annex 5: Daily feedback form – for interviewers Date:……………………………………………………………………………………… Names of Interviewers:…………………………………………………………………… ……………………………………………………………………………………………. Parishes and villages and visited: ………………………………………………………………………………………………………... ……………………………………………………………………………………………………….. ………………………………………………………………………………………………………... ………………………………………………………………………………………………………… Number of households visited: …………………………………………………………… Number of households visited without respondent: ……………………………………… Number of households visited where respondent refused to participate: …………………………. Number of interviews completed: ……………………………………………………………….. Number of interviews observed by Supervisor: ………………………………………….. Problems encountered today ……………………………………………………………… ……………………………………………………………………………………………… ……………………………………………………………………………………………… ……………………………………………………………………………………………… Lessons learnt today………………………………………………………………………... ……………………………………………………………………………………………… ……………………………………………………………………………………………… ……………………………………………………………………………………………… Unusual events or situations today ………………………………………………..…………………………………………… Recommendations ………………………………………………………………………… ……………………………………………………………………………………………… ………………………………………………………………………………………………    72 Annex 6: Checklist for supervisors No. Survey Items Yes No 1 List of selected Villages in each Supervision Area 2 List of Interviewers and their contacts and assignments 3 Random selection tools (Random table of numbers) 4 Extra copies of KPC Questionnaires 5 Ensuring that Interviewers in the Team have all Survey Items 6 Quality Control Checklists 7 Household lists/ Village maps 8 Daily Interview Form – for supervisors 9 Colored pen or pencil 10 Air time    73 Annex 7: Quality ‐ control checklist Observe and evaluate a minimum of one (1) interview conducted by each Interviewer in your Interview Team each day. Use this form as you observe the Interviewers. While you are observing, do not talk with the Interviewer being evaluated. Completely fill in this form. When the Interviewer has finished conducting the interview, review this form with him/her in private. Quickly discuss each point, pointing out both the Interviewer’s strong points and the areas where improvement is needed. Remember: The purpose of this form is to document the quality of the interviews, encourage the Interviewer to continue doing what she/he is doing correctly, and to improve the Interviewers’ performance. Name of Interviewer:….……………………Name of supervisor:…………………………… Supervision Area:………………………………Village:………………………………………… Date:…………………………………………………Time:…………………………… No. DID THE INTERVIEWER: Performed correctly? Yes No 1 Select the household correctly? 2 Select the respondent correctly? 3 Introduced the purpose of the survey correctly? 4 Introduced him/ herself correctly? 5 Correctly filled the KPC Questionnaire? 6 Speak clearly during the interview? 7 Ask leading questions that might have influenced the respondents Answers? 8 Read questions exactly as they were written? 9 Follow the skip patterns correctly? 10 Prompt the respondent for all answers (say: Anything else) 11 Checked all items for Observation (Child/mother cards, net etc) 12 Keep time on the interview 13 Acknowledge the respondent after the interview On the scale of 1 (needs follow up training) to 10 (Excellent), I rate the interviewer’s performance during this interview as follows (Circle one) 1 2 3 4 5 6 7 8 9 10 Approximate Duration of the Interview: ……………………………………………………. Minutes General notes ………………………………………………………………………………………... Supervisor’s Signature:…………………… ……………………………………………………………….    74 Annex 8: Daily interview form – for supervisors Name of Supervisor:…………………………Date………………………………... Names of Interviewers: _____________________________________________________________________________________ _____________________________________________________________________________________ ______________________________________________ Parishes and Villages visited: ______________________________________________________________________________ ______________________________________________________________________________ ____________________________________________________________ Number of households visited: _______________ Number of households visited without respondent: _________ Number of households visited where respondent refused to participate: ______________ Number of interviews completed: ____________________________________________ Number of interviews observed by Supervisor: _______________________________ Problems Encountered Today Solutions Assistance Needed Lessons learnt Unusual events or situations Recommendations    75 Annex 9: Permission letters May 7th, 2010 The Chief Administrative Officer, Bushenyi District Local Government P.O. Box 1, Bushenyi Dear Sir, Re: Permission to carry out Health Facility Assessment and Household Survey On behalf of Uganda Health Cooperative, I am writing to request for your permission to carryout Health Facility Assessment and Household survey in the District’s Health Facilities and selected households respectively. The exercise is scheduled to place between June 16th – 30th, 2010. Uganda Health Cooperative has operated in Bushenyi district since 2005 where it has trained and facilitated Health Workers and Community Own Resource Persons (CORP) in Integrated Management of Childhood Illnesses (IMCI), Maternal and New Born Care (MNC), Malaria and HIV/AIDS. The aim of this exercise is to assess the impact of the project’s intervention throughout the district as we work with various partners striving to improve the health of the communities in Bushenyi District. Your cooperation in this regard will be highly appreciated. Sincerely, Mukankusi James (Dr.) UHC/CS Director, CC. All Sub County Chiefs CC. LC III Chairpersons           Uganda Health Cooperative             Plot 2, Ibis Vale, Off Prince Charles Drive,             P.O Box 27659, Kampala, Uganda          Tel: 256‐41‐342353  Fax: 256‐41‐258678    76 May 7th, 2010 The Chief Administrative Officer, Rukungiri District Local Government P.O. Box 1, Bushenyi Dear Sir, Re: Permission to carry out Household Survey On behalf of Uganda Health Cooperative, I am writing to request for your permission to carryout Household survey in the selected households respectively in Rubaabo county. The exercise is scheduled to place between June 28th – 30th, 2010. Uganda Health Cooperative has operated in Bushenyi district since 2005 where it has trained and facilitated Health Workers and Community Own Resource Persons (CORP) in Integrated Management of Childhood Illnesses (IMCI), Maternal and New Born Care (MNC), Malaria and HIV/AIDS. During the baseline, Rubaabo County in Rukungiri district was used as the control. We are now conducting end of the project evaluation and this exercise is aimed at assessing the control to be able to compare the results with those of the intervention area to determine whether the project impacted on the communities. Your cooperation in this regard will be highly appreciated. Sincerely, Mukankusi James (Dr.) UHC/CS Director, CC. All Sub County Chiefs CC. LC III Chairpersons           Uganda Health Cooperative             Plot 2, Ibis Vale, Off Prince Charles Drive,             P.O Box 27659, Kampala, Uganda          Tel: 256‐41‐342353  Fax: 256‐41‐258678    77 Annex 10: Training time table Date Time Topic Facilitator Tuesday 15th June 2010 9:00AM￾9:30AM 1 – General introduction to the program Objectives of the evaluation Grace / Dr. James 9:30AM￾11:00AM 2 – The Evaluation Methodology LQAS sampling method Village selection protocol Household selection protocol Respondent selection protocol Michael 11:00AM￾11:30PM TEA BREAK All staff / Volunteers 11:30AM￾1:00PM 3 Roles of Supervisors and interviewers Proper interviewing techniques Data Quality assurance Michael 1:00PM￾2:00PM LUNCH All staff / Volunteers 2:00PM:5:00P M 4 - Reviewing the KPC Questionnaire 5 - Reviewing the HFA Questionnaire Arthur / Michael / Grace Wednesday 16th June 2010 8:30AM￾9:30AM Preparing For the pretest Michael 9:30AM￾1:00PM Pre tests All staff / volunteers 2:00PM￾3:00PM Feedback from the pretests and correcting of questionnaires Michael 5:00PM on wards Print and photocopy questionnaires Michael / Arthur / Herbert    78 Annex 7: Community Volunteer Health Worker and Health Worker Training Matrixes Community Volunteer Health Worker Training Phase (2010 District) Volunteer Female Male Total Government, Grantee or Community elected Focus of training Igara (Bushenyi) CORP 8 15 23 Grantee IMCI, MNC and CHF Ruhinda (Mitooma) CORP 17 17 34 Grantee IMCI, MNC and CHF Sheema (Rubirizi) CORP 6 17 23 Grantee IMCI, MNC and CHF UHC Group CORP 8 15 23 Grantee IMCI, MNC and CHF Buweju (Kibingo) CORP 8 15 23 Grantee IMCI, MNC and CHF Bunyaruguru (Nsiika) VHT 107 83 190 Community* VHT 9 modules and CHF VHT parish coordinator 2 23 25 Community* VHT 9 modules and CHF TOTAL 156 185 341 % 46% 54% 100% *: Elected by the community during sensitization per Village Health Team model Health Worker Training County (Phase) MNC IMCI Female Male Total Female Male Total Igara 8 1 17 19 6 25 Ruhinda 18 7 25 13 8 21 Sheema 18 5 23 21 5 26 Buweju 16 4 20 14 5 19 Bunyaruguru 15 2 17 9 9 18 Total 75 19 94 76 33 109    79 Annex 8: Evaluation Team Members  Jolene Mullins, Evaluation Team Leader/Field  Garth Osborn, Evaluation Team Backstop  Dr. James Mukankusi, Director CS/UHC  Edward Mpairwe, UHC Board Chair  Grace Karungi, CS/UHC Health Services Trainer  Edidah Kananga, CS CE/CLC  Amelia Namanya, UHC Manager  Arthur Kanyaryeru, UHC Marketer and Trainer  Michael Oturu, M&E Coordinator  Samuel Ibanda, DHT/MOH Representative  Imelda Tumwine, VHT Representative  Mugabe Henry, VHT Parish Coordinator  Pelly Katabazi, Enrolled Nurse, Nyakasiro Health Centre III  Sister Rose Mary Kyatukwire, Provider Representative  Jude K. Asiimwe, Translator Responsibilities of the Team Leader were split between two individuals who have extensive experience in Child Survival. Jolene Mullins was responsible for leading the Final Evaluation Team through the field work phase of the evaluation while Garth Osborn was responsible for planning, developing the questionnaire guides and writing the report. Both Ms. Mullins and Mr. Osborn worked under the auspices of WellShare International (formerly Minnesota International Health Volunteers), one of the first NGO participants in the CSHGP.    80 Annex 9: Evaluation Assessment Methodology The Final Evaluation Team used multiple methods to review this five year child survival project. Project staff implemented KPC and HFA surveys in June and July so the results would be ready for review by the Final Evaluation Team members well before the field work phase. An extensive list of project documents and data was made available to the two external evaluators for review of this project, commensurate with the complexity of the project strategies and interventions, including the project DIP, annual reports, midterm evaluation reports, UHC Board meeting materials, UHC technical guides, etc. Based on a review of this data and these documents, a set of interview and focus group discussion guides were developed for use during the field work to ensure that all the necessary information was collected from the key stakeholders. (See Annex 10 for a list of individuals and groups interviewed.) These guides were filled out and emailed back to Minnesota for review and inclusion in this report. A participatory approach has been used throughout the evaluation, especially during the field work, where each of the team members was invited to participate in each activity. Findings and recommendations were determined through consensus of the team based on the review and discussion of the information collected. A draft of this report was prepared and submitted to HealthPartners for review and comment. The final draft was then submitted to HealthPartners for formatting and submission to USAID and to share with partners. Field Work Bushenyi District - Uganda August 9 – 20, 2010 Date Activity Person Responsible Location Attendees 9 Meet Jolene at Hotel, Interviews  Project Administrator/Accountant Travel to Bushenyi Dr. James and Julius Lydia Kiwanuka Kampala Dr. James, Julius and Jolene 10 Introductory meeting in office Review KPC/HFA results, discuss plans and questions Interviews  CS Director  CS Staff  District Health Officer  MOH Rep Dr. James UHC office Evaluation Team and all staff 11 Site Visit  Provider site visit to see UHC check in, data centers and services Interviews  UHC Provider Administrator  UHC Data Entrant Dr. James Provider(s) Comboni Hosp, Nyakasiro HC Jolene, Dr. James, UHC Board Chair, Amelia and Arthur    81 Date Activity Person Responsible Location Attendees 12 Site Visit  UHC Monthly Board Meeting (detailed data on UHC performance and activities will be shared) Interviews  UHC Board Chair and board members Amelia UHC office Jolene, CS Staff and UHC Board 13 CS Stakeholder workshop to present KPC/HFA results, project summary and way forward Dr. James Mothers Union Hall, Katungu, Bushenyi Evaluation Team, Staff, DHT, stakeholders 15  Drama show (usually held on Weekends for bigger turn up) Arthur Kigarama Catholic Parish Jolene, Arthur 16 Site Visit  Health Worker Quarterly Meeting to sustainably support VHT Interviews  Health Worker(s) Dr. James Provider Rwengwe S/C-Buhweju, Jolene, Grace and/or Edidah TBD Interviews Mission Representative Dr. James TBD Jolene and Dr. James 17 Focus Group Discussion(s): UHC members Interviews  Focal VHT(s) Site Visit  Focal VHT Behavior Change Session, VHT demo kits Dr. James Gongo Bitooma parish Kyamuhunga Dr. James, Amelia and Jolene 18 Focus Group Discussion  Non UHC members Interviews  Focal VHT(s) Dr. James Field Kitagata Jolene, Dr. James, Grace and/or Edidah 19 Conclusions and lessons learned Dr. James Office Jolene, Evaluation team and CS staff 20 Return Jolene to airport Julius Entebbe Julius and Jolene    82 Annex 10: Persons Interviewed and Contacted during the Final Evaluation HealthPartners Headquarters:  Scott Aebischer, Senior Vice President, Customer Service and Product Innovation, (Technical Backstop on this project)  Jennifer Stockert, Uganda Program Manager HealthPartners Uganda Program:  Dr. James Mukankusi, CS Director, HealthPartners/Bushenyi  Grace Karungi, CS Health Services Trainer, HealthPartners  Michael Oturu, M&E Coordinator, HealthPartners  Edidah Kananga, CS CE/CLC, HealthPartners  Amelia Namanya, UHC Manager, HealthPartners  Arthur Kanyaryeru, UHC Marketer and Trainer, HealthPartners  Lydia Kiwanuka, Accountant/Administrator, HealthPartners/Kampala UHC Board of Directors:  Sister RoseMary Kyatukwire, UHC BOD member, Administrator/Accountant, Nyakasiro Health Center III  Apollo Mugisha, UHC BOD Secretary/ Community Leader, Comboni Hospital  David Agaba, UHC BOD Vice Chair, Rubare Health Center II UHC Partner Providers:  Christopher Baine, Acting Senior Enrolled Nurse, Pediatric Ward, Kitagata District Hospital  Julius Barigye, Data Entrant, Comboni Hospital  Elia Sendahura, Health Assistant, Bihanga HCIII  Sister Christine Nyanakanwagi, UHC Provider InCharge, Nyakasiro Health Center III  Dr. Allen Twesigomwe, Physician, Comboni Hospital (A UHC Provider)  Elia Sandahura, Health Assistant, Bihanga HCIII VHTs, VHT parish coordinators and CORPs:  Wilson Tukwatanise, VHT Parish Coordinator/CORP, Muhito Parish  Cossy Mirembe, VHT Parish Coordinator/CORP, Rukondo Parish  Mathias Kalemba, VHT Parish Coordinator/CORP, Kyeibanga Parish  Yoram Tukamuhebwa, VHT Parish Coordinator/CORP, Kasana East Parish  Asaph Rubahuriza, VHT Parish Coordinator, Handwashing Ambassador and CBD, Bitooma Parish  Pelly Katabazi, Enrolled Nurse, Nyakasiro Health Centre III  Bagonza Simon, Health Assistant, Kyamuhunga Sub County DHT & MOH:  Dr. Kenya Mugisha, Director General of Health Services (Head of Health Services in Uganda)    83  Samuel Ibanda, Senior Health Trainer, Central MOH-Child Health Division  Dr. Celestine Barigye, District Health Officer, Bushenyi District  Wycliffe Turyasingura, Acting DHO-Buhweju District, MOH/Buhweju District local government Others:  Gordon Tumweyambise, LC-1 Representative, Muhito Parish  Karen LeBan, Executive Director, The CORE Group Focus Group Discussions:  13 UHC group members including their Chairman from Comboni Hospital  45 community members during a VHT presentation  18 members of the project’s drama troupe  39 VHTs    84 Annex 11: Special Reports A. UHC Board Work Plan 2010-2011 Activity Goal Persons/ responsi bilities Time Frame Resources needed M&E Follow up IR1: Increased capacity of the board to manage the health scheme 1. Hold Bi￾monthly board meeting s To review UHC performance and BOD Chairm an Oct, Dec 201 0 Feb 2011 Apr, Jun, 2011 Aug , 201 1 ‐ UHC performa nce tools ‐ Previous minutes ‐ Support supervisi on reports ‐ Minut es of the meeti ng ‐ Actio n plan and follow up notes 2. Hold AGM To give a feed back to UHC members about the performance of the coop and elect new office bearers Board Secreta ry Jan 2011 ‐ Previous AGM minutes ‐ Facilitati on ‐ Venue ‐ Audit reports 3. Train board member s on strategi c plannin g To increase the capacity of the board to develop a strategic plan to guide the cooperative achieve its goals Board Chairm an Feb 2011 ‐ Training materials ‐ Venue ‐ Meals ‐ Transport facilitatio n ‐ Traini ng Report s ‐ Follow up plans 4. Train board member s on project proposa l develop ment To increase board capacity to identify gaps and source for funding to close these gaps Board Secreta ry Nov 201 0 ‐Training materials ‐Venue ‐Meals ‐Transport facilitation 5. Develo p UHC resourc e mobiliz ation action plan To develop a tool to guide the board fill resource gaps identified Board Secreta ry Nov 201 0 ‐Stationery ‐Transport facilitation ‐Meals ‐Venue ‐UHC strategic plan 6. Conduc t board study tours To share lessons learned with others to improve UHC Board Treasur er ‐Transport and per diem facilitation UCBHFA plans to fund this trip to    85 Activity Goal Persons/ responsi bilities Time Frame Resources needed M&E Follow up ‐ Luwero. Contact person is Prossy. 7. Train board member s on financia l manage ment Increase board capacity to handle financial responsibilities of the coop for sustainability Board Treasur er Dec 201 0 ‐Stationery ‐Transport facilitation ‐Meals ‐Venue ‐UHC strategic plan, audit reports IR2: Growth of schemes 1. Conduc t commu nity drama shows To reach out to potential members for the health schemes Kamas hengye ro (Board membe r) Oct, Nov , Dec , 201 0 Jan, Feb, Mar, 2011 Apr, May, Jun, 2011 Jul, Aug , Sep t, 201 1 ‐ Drama group ‐ Transport ‐ Stationery , brochures ‐ Venue ‐ Mobilizati on of communit ies Contact Bwera Drama group for scheduling , practice and shows 2. Hold radio talk shows quarterl y To market the cooperative and its benefits to communities Board Chairm an Oct, 201 0 Jan, 2011 April, 2011 July , 201 1 ‐ Talk show particip ants transpor t ‐ Radio spot Contact radio stations for concession s and free airtime 3. Sensitiz e commu nity on the benefits of the scheme To reach out to potential members for the health schemes UHC BOD membe rs, provide rs Oct, Nov , Dec , 201 0 Jan, Feb, Mar, 2011 Apr, May, Jun, 2011 Jul, Aug , Sep t, 201 1 ‐ Brochur es ‐ Posters ‐ UHC stakehol der handboo k Reminders to be sent to the providers 4. Train UHC member s on IGAs Improve household incomes to ensure members can pay premiums on time for health care Board treasur er Oct, 201 0 Jan, 2011 March, 2011 Ma y, 201 1 ‐VSLA manual ‐Stationery ‐Venue ‐Trainers facilitation ‐Participant s transport refund Follow up of the VSLA trainings done before further roll out IR3: Increased ability of scheme to financially cover services and administrative costs 1. Monthl To hold all UHC Oct, Nov, Dec Jan, ‐ Transpor Develop    86 Activity Goal Persons/ responsi bilities Time Frame Resources needed M&E Follow up y support supervis ion to UHC provide rs stakeholders responsible for fulfilling their roles and responsibilities for sustainability BOD membe rs 2010 2010 , 201 0 2010 t ‐ Stationer y ‐ Lunch allowanc es quarterly plan which can be incorporat ed into the MCP travel schedule 2. Collect reserve fund and member ship fees To collect contributions from members and providers for management of the cooperative UHC treasur er Marc h 15th, 2011 ‐ Annual provider/ group performa nce report ‐ Stationer y ‐ Receipt books 3. Conduc t annual audit To ensure UHC accountability and financial sustainability UHC treasur er Jan 2011 ‐ Receipts and cash book ‐ Expense documen ts ‐ Year end bank statemen t ‐ Audit firm To be done in Dec/Jan before the AGM 4. Renewa l of IDs To have all members follow the guidelines in getting services and eliminate misuse Data entrant s Dec , 2010 ‐ Camera ‐ Up dated member lists ‐ Transpor t for DE ‐ Compute r and laminato r ‐ Stationer y Annual updates in December with the renewal for Jan premiums and quarterly with any new changes in the family/gro up    87 B. Annual UHC Board 2011-12 Operating Budget Item Unit cost # of units # of times (Days) Freque ncy Amount IR1: Increased capacity of the board to manage the health scheme 1.1: Hold quarterly board meetings 0 0 1 4 0 Break tea 2,000 9 1 4 72,000 Lunch contribution 5,000 9 1 4 180,000 Transport refund 20,000 9 1 4 720,000 Sub Total 972,000 1.2: Hold 2011 Annual General Meeting Venue hire 50,000 1 1 1 60,000 Public address sound system 100,000 0 0 0 0 Refreshments with snacks 3,000 58 1 1 174,000 Transport refund for participants 1,000,000 1 1 1 695,000 Awards/certificates 300,000 1 1 1 300,000 Sub Total 1,229,000 1.3: Train board members on strategic planning Break tea 2,000 9 1 1 18,000 Lunch contribution 5,000 9 1 1 45,000 Transport refund 20,000 9 1 1 180,000 Sub Total 243,000 1.4: Train board members on project proposal development Break tea 2,000 9 1 1 18,000 Lunch contribution 5,000 9 1 1 45,000 Transport refund 20,000 9 1 1 180,000 Sub Total 243,000 1.5: Develop UHC resource mobilization action plan 1.6: Conduct board study tours Accommodation and meals 2 15 40,000 1 1,200,000 Transport refund to and from UHC 2 15 10,000 1 300,000 Field transport 3 1 250,000 1 750,000 Airtime for coordination 1 1 50,000 1 50,000 Sub Total 2,300,000 1.7: Train board members on financial management Break tea 2,000 9 1 1 18,000 Lunch contribution 5,000 9 1 1 45,000 Transport refund 20,000 9 1 1 180,000 Sub Total 243,000 IR1 Total 5,751,000 IR2: Growth of schemes 2.1: Conduct communityMDD shows 300,000 2 12 1 7200000 Sub Total 7200000 2.2:Hold quarterly radio talk shows 300,000 1 4 1 1200000 Sub total 1200000 2.3: Sensitize community members on the benefits of the scheme 0    88 Lunch contribution for board members 5000 2 12 1 120000 Transport refund to and from UHC 20,000 2 12 1 480000 Sub Total 600000 2.4: Train UHC members on IGAs 0 Purchase of VSLA kits (select 2 poorly performing groups) 125000 2 1 1 250000 Training on VSLA methodology (CVM) 0 0 0 1 0 Sub Total 250000 IR2: Total 9250000 IR3: Increased ability of scheme to financially cover services and administrative costs 3.1: Conduct monthly support supervision to UHC providers Lunch contribution for board members 5000 2 10 4 400,000 Transport refund to and from UHC 20,000 2 10 4 1,600,000 Sub Total 2,000,000 3.2: Collect reserve fund and membership fees Lunch contribution for board members 5000 1 1 4 20000 Transport refund to and from UHC 20,000 1 1 4 80000 Sub Total 100000 3.3:Conduct annual audit 300,000 1 1 1 300000 Sub Total 300000 3.4:Renewal of IDs Lunch contribution for board members 5000 2 10 1 100000 Transport refund to and from UHC 20,000 2 10 1 400000 Sub Total 500000 IR3: Total 2900000 IR4: Administrative costs: Preparation steps for reserve fund transactions Purchase of UHC self inking Stamp 80,000 1 1 1 80000 Purchase a preprinted voucher booklets 40,000 1 1 1 40000 Purchase of carbonated reciept book 40,000 1 1 1 40000 Purchase of carbonated invoice book for UHC 40,000 1 1 1 40000 Office rent (being paid by HealthPartners) 400,000 12 0 1 0 Creation of UHC BOD members' Desk Purchase of computers for documentation 1,200,000 0 1 1 Donated from CS Desktop tables 200,000 2 1 1 400000 Chairs 100,000 2 1 1 200000 Wooden shelves for keeping UHC files 250,000 1 1 1 250000 Others: 0 Transport refund (whoever comes to do UHC official work at office e.g follow up an activity) 20,000 2 1 4 160000 IR4: Sub Total 1,210,000 Grand Annual Total Budget Available Budgeted Remaining Project Expenses 8,400,000 8,400,000 0 UHC/MCP Support 7,462,983 5,389,000 2,073,983 UHC Reserve Fund 7,462,983 2,501,000 4,440,983 Total Funds available 23,325,966 16,811,000 6,5174,966    89 C. Cascade of Sensitization and Training Step Participants Duration Facilitators Outputs 1. Development of policies, guidelines and work aids  NMCP and partners (MCMTWG)  Consultancies Depends on work load Consultancies MCMTWG  Policies, guidelines and work aids 2. Training national HBMF trainers  Experts on malaria including the MCMTWG 1 day NMCP and partners  national HBMF trainers  National HBMF strategic work plan 3. Sensitization of district level leaders Representatives of all forms of district level leadership. (e.g. technocrats; political leaders, cultural leaders, religious leaders, non-governmental organizations, community based organizations) 1 day National HBMF Trainers  Supportive district leaders  District HBMF strategic work plan 4. Training district HBMF trainers Health workers selected from all over the district (2 per SC; 2 per HSD; 2 from the office of the DHO) 2 days National HBMF Trainers  District HBMF trainers 5. On-site training of health workers All health workers 1 day District HBMF trainers supervised by National HBMF trainers  Health workers who feel responsible for HBMF who also serve as supervisors of CMDs 6. Sensitization of sub-county level leaders Representatives of all forms of sub-county level leadership (e.g. technocrats; political leaders, cultural leaders, religious leaders, non-governmental organizations, community based organizations) 1 Day District HBMF trainers supervised by national HBMF trainers  Supportive sub-county leaders who also serve as community mobilisers  SC HBMF strategic work schedule 7. Sensitization at community level and selection of volunteers to be trained as CMDs All community members (LC1 Council) ½ day Community mobilisers supervised by health workers  Sensitized communities  Selected CMDs (volunteers) 8. Training of CMDs Selected CMDs (2 per community) 2 Days District HBMF trainers supervised by national HBMF trainers  Trained CMDs 9. Official launch All stakeholders and the general public 1 day Minister of Health  Supportive public 10. Support supervision, monitoring & evaluation    90 Annex 12: Project Data Form    91    92    93    94    95    96    97 Annex 13: Grantee Plans to Address Final Evaluation Findings During the last nine months of HealthPartners Child Survival (CS) program staff roles were transitioned to local stakeholders with ongoing support incorporated in HealthPartners Uganda Health Cooperative Malaria Communities Program (UHC/MCP) work plan for 2011. The timing of the new districting and national support for strengthening decentralized systems was perfect. A district coordination planning tool was developed in March 2010. Meetings were scheduled with new District Health Teams (DHT) to present CS End of Project results, to discuss newly appointed district leaders’ priorities and planned interventions and to share how HealthPartners UHC/MCP could support their continued capacity building and systems creation efforts. Each DHT with support from UHC/MCP agreed to continue support supervision of the Health worker (HW)/Village Health Team (VHT) link through quarterly meetings. These meetings are important for continued capacity building and sustained motivation. The transition from UHC/CS support to UHC/MCP support began with quarterly meetings between the DHT/HW/and Parish VHT where these stakeholders led the planning for VHT quarterly meetings. Several areas were identified for focus including: VHT data collection and use of data to make results based decisions; roll out of the new VHT model, expectations and resources; and linking village leaders to this system for community level promotion and sustainable support of VHT services. In partnership with the Uganda Mission and with Mission approval, UHC/MCP developed consolidated VHT integrated community case management data tracking tools applying the Ministry of Health register format while enabling district data collectors to use a simple Excel spreadsheet to automatically aggregate VHT reports from Parish levels, to Health Sub District, and District quarterly and annual reports for easy data analysis. UHC/MCP will propose 2010- 2011 Memorandums of Understanding to each DHT offering to support district data collection systems and data collector capacity building using this tool to enable them to submit complete, accurate, on-time reports and to use data to make results based decisions. As noted in the report, community leaders help build support for VHT by encouraging villagers to recognize value of VHT, to use their services and to support their continued efforts with in￾kind contributions like helping to clean VHT compounds, bringing soap or other small gifts when seeking advice, or watching children while VHT counsel neighbors. District Health Officers have agreed to further encourage this level of support by including leaders in district coordination meetings, and acknowledging religious and political leader contributions in public forums. The Uganda Health Cooperative offers discounted health plan membership to VHT. This is a sustainable effort that was not subsidized since VHT are informed and practice preventive health and early treatment seeking at a higher rate than the rest of the community and because VHT promote joining UHC. By providing discounted member rates to VHT, VHT can promote the health plan from first hand experience.    98 The end of HealthPartners Child Survival program marked an important transition for UHC stakeholders and the Board of Directors. Throughout the program, UHC activity expenses were catered for by grant funding. This enabled the co-op to test setting premiums and co-payment rates and membership rules to sustain the cooperative cost balance and to accrue a reserve fund for future operating expenses. As part of the Child Survival exit strategy, the UHC board developed their own work plan for 2010-11 with a matching budget (Annex 11 A and B.) From the budget it can be seen that the UHC reserve fund will cover the cost of their annual general meeting, quarterly board meetings and one annual audit. Technical capacity building support will continue from HealthPartners UHC/MCP but the topics will be selected and scheduled by the UHC board. UHC/MCP agreed to match the UHC Board reserve fund in Oct. 2010 (for support in 2011) and in Oct. 2011 (for support in 2012) to provide incentive for the board to keep the reserve fund as high as possible. Consultant recommendations for UHC management were shared with the UHC Board of Directors including carefully and selectively discounting premium rates for targeted groups; revisiting a benefit package that includes chronic care on a test case basis; and exploring the option to allow individuals to join at a reduced rate from families. The following recommendations: incentivize ‘satisfied customers’ to act as marketers, promoters and mentors to new groups; and identify other health service providers in major urban areas (i.e. Kampala, Mbarara, Masaka) as members requested coverage when traveling “out of area,” were also shared but were not recommended since extensive test cases have proven a lack of return on investment in the former and lack of practical ability in this resource poor setting to sustain administrative overhead to track, monitor and manage payments and reimbursement for referrals. HealthPartners does not currently intend to reapply for CORE group membership but does recommend that in the future, the CORE group consider accepting new grantees as members even if when they apply they do not seem to have experience that would make them an asset to the group. New grantees in particular may have differing approaches and perspectives that over the course of a project could be valuable to the CORE group.    99 Annex 14: Grantee Response to Final Evaluation Findings HealthPartners is grateful to the USAID Child Survival and Health Grants program (CSHGP) and technical support teams for making it possible for new grantees to share their experience and to extend private resources to impact sustainable development. The CSHGP process, requirements, timelines and support were well balanced to build HealthPartners capacity and to monitor and improve performance throughout the life of the project. The New Grantee Orientation where expectations for developing the Detailed Implementation Plan (DIP) were shared and the DIP Workshop breakout sessions, peer and technical reviews were particularly helpful. As a result of this support HealthPartners adopted the Child Survival Sustainability Assessment strategy, Behavior Change Communication, the BEHAVE framework, and learned to reference and apply resources and tools like Knowledge, Practice and Coverage (KPC) 2000+, Lot Quality Assurance Sampling, Technical Resource Manuals, Integrated Management of Childhood Illness (IMCI) and Community-IMCI. The project team also greatly appreciated the technical support provided by the Child Survival Technical Support team, the Maternal Child Health Integrated Program (MCHIP) and the field site visit, report and consistent support from Elaine Menotti. The final evaluation process was well guided by clear expectations and resources. As a result, we were able to collaborate with a local development partner, WellShare International. We were impressed with the research, process, report and lessons that we learned from working with Garth Osborn and Jolene Mullins. HealthPartners also learned two valuable lessons from partners that impacted our Child Survival project in the last two years and that may be helpful to report. First, the Ministry of Health Village Health Team trainer’s guide and participant’s manuals included a Cascade of Sensitization and Training (Annex 11.C) that detailed the roles of partners and the process of communication, capacity building and support for sustainable development. This was a concise planning matrix that included communication to stakeholders at every level and included timelines which were helpful especially in terms of reminding the team that support supervision, as a transition to sustainable local stakeholder uptake of interventions, is a process. HealthPartners learned to apply this cascade strategy to each program intervention. Additionally, HealthPartners did not have the technical capacity to develop training resources throughout much of the Child Survival project. We were aware of this limitation and relied on existing resources like IMCI for our capacity building. In 2009 HealthPartners staff was invited to attend Criterion Referenced Instruction (CRI) training supported by USAID’s Office of Development Partners. As a result of this training we were in a position to develop the Uganda Health Cooperative Stakeholders’ Handbook and Provider, Volunteer, and New Member Trainers’ Guides that helped us to overcome some of the challenges we were facing to that point. The guides were tested and improved by the UHC Board. Lessons learned from UHC stakeholder training were also incorporated to improve the tools and Debra Prosnitz from MCHIP provided technical recommendations that were particularly helpful. These resources are included with this final evaluation in case future CSHGP grantees are interested in adopting or adapting them to link child survival interventions to community-owned health financing cooperatives. The lesson that HealthPartners learned as a result of this final evaluation regarding our data quality was disappointing since throughout the project we felt our monitoring and evaluation was quite rigorous and we were proud of how we used data to make results based decisions. By    100 reviewing and comparing data sets from the baseline, midterm, end of phase monitoring and the final evaluation we learned another difficult lesson. At the baseline we did not disaggregate data to prioritize the phases of implementation by health sub district (HSD) based on need. In retrospect we should have started in Bunyarguru where Malaria rates were the highest and hand washing was the lowest. Our baseline KPC included 95 respondents from all five HSDs without disaggregation. After this, each monitoring KPC included the HSD where implementation had recently finished. End of Phase I results, 95 respondents from Igara HSD, were compared to the baseline full district results. In hindsight this was the location where performance was already the highest. Phase II results, 95 respondents from Ruhinda HSD, were also compared to the baseline full district results. Midterm data included 95 respondents from the first three HSDs— again our results looked excellent. At the End of Phase IV KPC we were shocked to see huge declines; Phase V results were similar. We spent that last portion of our project focusing interventions on Buhweju and Bunyaruguru HSDs to close gaps but we had lost the benefit that spending the extra years of support supervision where it was most needed would have had. It is difficult to report such an egregious mistake however we recognize the importance of documenting and sharing all of our lessons learned. HealthPartners continues to believe strongly in our development hypothesis: interventions will be more sustainable and will have a greater impact if linked to community-owned, stakeholder￾managed, sustainable health co-ops. HealthPartners has received a five year Cooperative Development grant through USAID to empower rural stakeholders in Southwestern Uganda to develop partnerships with measures of accountability linking private health care providers, community groups and local public and private health stakeholders so that by 2015 30,000 women age 15-49 and 85,000 total beneficiaries will have access to quality health care through sustainable community-owned health financing cooperatives. We are grateful for the five years that we had to learn from and work with the Child Survival and Health Grants Program and we will incorporate the many tools and lessons that we have learned through this experience to increase our development impact in the future.