May 27, 2015 This Proposal was produced for review by the United States Agency for International Development. It was Afya Jijini by the Evaluation Services and Program Support team of International Business & Technical Consultants Inc. EVALUATION SERVICES AND PROGRAM SUPPORT (ESPS) A Mid-term Performance Evaluation of Afya Jijini Activity EVALUATION REPORT AUGUST 10, 2018 This report is produced for United States Agency for International Development. It was prepared by International Business and Technical Consultants, Inc. i USAID/Kenya and East Africa Evaluation Services and Program Support A Mid-term Performance Evaluation of Afya Jijini Activity Prepared for: Health, Population & Nutrition Office United States Agency for International Development/Kenya Office of Health, Nutrition and Population C/O American Embassy Unite Nations Avenue, Gigiri P.O. Box 629, Village Market 00621 Nairobi, Kenya Prepared by: International Business & Technical Consultants, Inc. 8618 Westwood Center Drive Suite 400 Vienna, Virginia 22182 United States Authors: Dr. Swati Sadaphal, Team Leader and Project Director, ESPS Gilbert Alouch, Social Scientist Expert Dr. Makeba Shiroya-Wandabwa, Public Health Expert Mercelline Ogolla, Reproductive, Maternal and Child Health Expert Mwaura Njuguna, Senior M&E Expert DISCLAIMER The authors’ views expressed in this report do not necessarily reflect the views of the United States Agency for International Development or the United States Government. ii Table of Contents ACKNOWLEDGMENTS .......................................................................................................................................IV ACRONYMS ........................................................................................................................................................V GLOSSARY .......................................................................................................................................................VIII EXECUTIVE SUMMARY ......................................................................................................................................IX 1 INTRODUCTION....................................................................................................................................... 1 1.1 EVALUATION PURPOSE AND OBJECTIVES..............................................................................................................1 1.2 EVALUATION QUESTIONS..................................................................................................................................1 1.3 INTENDED AUDIENCE .......................................................................................................................................1 2 BACKGROUND......................................................................................................................................... 1 2.1 PROJECT DESCRIPTION .....................................................................................................................................1 2.2 PROJECT CONTEXT...........................................................................................................................................2 3 METHODOLOGY ...................................................................................................................................... 3 3.1 EVALUATION DESIGN .......................................................................................................................................3 3.1.1 Sampling strategy and selection criteria............................................................................................3 3.1.2 Data collection methods.....................................................................................................................4 3.1.3 Data management .............................................................................................................................5 3.2 DATA ANALYSIS AND TRIANGULATION.................................................................................................................6 3.3 EVALUATION ACTIVITIES AND TIMELINES..............................................................................................................6 3.4 LIMITATIONS ..................................................................................................................................................6 4 FINDINGS & CONCLUSIONS .....................................................................................................................7 4.1 EVALUATION QUESTION 1..........................................................................................................................7 4.1.1 Sub-purpose One: Increased Access and Utilization of Quality HIV Services......................................7 4.1.2 Sub-purpose Two: Increased Access and Utilization of Focused MNCH, FP, WASH and Nutrition Services 10 4.1.3 Sub-purpose Three: Strengthened and Functional County Health Systems .....................................14 4.1.4 Cross-Cutting Elements ....................................................................................................................18 4.2 EVALUATION QUESTION 2........................................................................................................................21 4.3 EVALUATION QUESTION 3........................................................................................................................23 4.4 EVALUATION QUESTION 4........................................................................................................................24 5 RECOMMENDATIONS............................................................................................................................ 25 6 ANNEXES............................................................................................................................................... 29 6.1 ANNEX 1: SCOPE OF WORK.......................................................................................................................29 6.2 ANNEX 2: SAMPLE SELECTION STRATEGY.................................................................................................44 6.3 ANNEX 3: EVALUATION PLAN MATRIX......................................................................................................49 6.4 ANNEX 4: LIST OF DOCUMENTS REVIEWED..............................................................................................62 6.5 ANNEX 5: DATA COLLECTION TOOLS ........................................................................................................63 6.6 ANNEX 6: LIST OF PERSONS INTERVIEWED.............................................................................................159 6.7 ANNEX 7: PROGRESS TO ACHIEVEMENT TABLES ....................................................................................162 iii List of Figures and Boxes Figure 1: Results Framework for Afya Jijini Activity...................................................................................2 Box 1: A Snap-Shot of Health Indicators Statistics in Nairobi County .......................................................3 Figure 2: Afya Jijini Focus County and Sub-Counties.................................................................................4 Figure 3: MTCT Rates at AJ Supported HFs................................................................................................7 Figure 4: Viral Load Suppression Rates at AJ Supported HFs ....................................................................7 Figure 5: Summary of AJ Activity’s Strengths, Weakness, Opportunities & Threats (SWOT) at Year 3 ..20 List of Tables Table 1: Afya Jijini eMTCT Strategies and Interventions: ..........................................................................9 Table 2: Review of PMTCT HEI Cohort Data across *8 Afya Jijini Supported Health Facilities................10 Table 3: A Summary of Afya Jijini MNCH Interventions for Postnatal Care ............................................13 Table 4: Examples of Policies and Tools Developed by AJ.......................................................................14 Table 5: Referral services interventions implemented by AJ ..................................................................16 Table 6: AJ Consortium Partners, Roles and Responsibilities..................................................................22 iv ACKNOWLEDGMENTS The evaluation team would like to acknowledge the following individuals: • International, national, and local implementing partners of the Afya Jijini Activity who provided useful insights into what did and did not work during the project period 2015-2018. • Nairobi City County government officials from health and other sectors, members of County and Sub￾County Health Management Teams, and in-charges at selected health facilities were gracious in sharing their insights, despite competing demands on their time. • ESPS/IBTCI Kenya staff (Dr. Norbert Rakiro, ESPS Chief of Party; Paul Mwai, ESPS Monitoring and Evaluation Advisor; Rosemary Were, ESPS Program Support Manager; and all ESPS administrative personnel) provided technical and administrative support to the evaluation team during all phases of the evaluation process. • IBTCI Home Office staff, we acknowledge Juan Carlos Alegre, Former ESPS Activity Director; Victoria Sesay, Program Associate; and Salima Mutima, Technical Advisor all who assisted in the proposal development, evaluation process and review of the report. • USAID/Kenya counterparts—in particular, Teresa Simiyu, John Bernon and Washington Omwomo- who offered guidance and support throughout the evaluation. Lastly, the authors extend heartfelt appreciation to community stakeholders, namely maternal, newborn, and child health clients; people living with HIV who are clients of comprehensive care clinics; youth; and facility doctors, nurses and community health workers, who participated in surveys, interviews and group discussions. v ACRONYMS AGYW Adolescent Girls and Young Women AJ Afya Jijini ART Antiretroviral Therapy ANC Antenatal Care AWP Annual Work Plan BCC Behavior Change Communication CBO Community-Based Organizations CCC Comprehensive Care Clinics CDC Centers for Disease Control and Prevention CDCS Country Development and Cooperation Strategy CDH County Director of Health CHAK Christian Health Association of Kenya CECH County Executive Committee for Health CHMT County Health Management Team CHA Community Health Assistant CHC Community Health Committees CHV Community Health Volunteers CME Continuous Medical Education c-MNH Community Maternal Neonatal Health COR Contracting Officer’s Representative CU Community Units DHIS2 District Health Information System 2 DQA Data Quality Assessment DREAMS Determined, Resilient, Empowered, AIDS-Free, Mentored, and Safe Women DHIS District Health Information Systems EID Early Infant Diagnosis EMONC Emergency Obstetric and Neonatal Care ESPS Evaluation Services & Program Support ET Evaluation Team FGD Focus Group Discussion FHMC Facility Health Management Committee FP Family Planning GBV Gender Based Violence GOK Government of Kenya HCW Health Care Worker HEIs HIV Exposed Infants HF Health Facility HIV/AIDS Human Immuno-deficiency Virus/ Acquired Immuno-deficiency Syndrome HPN Health, Population & Nutrition HPP+ Health Policy Plus HRH Human Resources for Health HTS HIV Testing Services HVF High Volume Facilities IBTCI International Business & Technical Consultants, Inc. ICA Institutional Capacity Assessment ICC Inter-agency Coordination Committee IYCF Infant and Young Child Feeding vi KMC Kangaroo Mother Care KAP Knowledge, Attitudes and Practices KDHS Kenya Demographic and Health Survey KEMRI Kenya Medical Research Institute KII Key Informant Interview KP Key Population LMIS Logistics Management Information System LOE Level of Effort LOP Life of Project LVF Low Volume Facilities M&E Monitoring and Evaluation MCTC Mother-to-Child Transmission MEDS Missions for Essential Drugs and Supplies MNCH Maternal Neonatal and Child Health MOH Ministry of Health MPDSR Maternal, Perinatal, Death Surveillance and Response MSM Men who have Sex with Men NASCOP National AIDS & Sexually Transmitted Infections (STIs) Control Programme NCC Nairobi City County NCCHSSIP Nairobi City County Health Sector Strategic and Investment Plan NOPE National Organization of Peer Educators OI Opportunistic Infection OJT On-the Job Training PATH Program for Appropriate Technologies in Health PCR Polymerase Chain Reaction PLHIV People Living with HIV PMTCT Prevention of Mother-to-Child Transmission PNC Postnatal Care PPH Post-partum Hemorrhage PrEP Post-Exposure Prophylaxis PSSGs Psychological Support Groups QA/QI Quality Assurance and Quality Improvement QIT Quality Improvement Teams RH Reproductive Health SCHMT Sub County Health Management Team SCHSF Sub-County Health Stakeholder’s Forum SOP Standard Operating Procedure SOW Scope of Work SWOT Strengths, Weakness, Opportunities, and Threat TB Tuberculosis TO Task Order TOCOR Task Order Contract Officer Representative TWG Technical Working Group TPM Team Planning Meeting USAID United States Agency for International Development VCT Voluntary Counseling and Testing VMMC Voluntary Medical Male Circumcision VL Viral Load WASH Water Sanitation and Hygiene vii WHO World Health Organization WIT Work Improvement Teams viii GLOSSARY Activity: USAID-funded program; referred to in this report as Afya Jijini or AJ. Antiretroviral drugs (ARVs): Tested and approved drugs that prevent HIV (and other retroviruses) from replicating. Antiretroviral therapy (ART): Use of a combination of ARVs to achieve viral suppression. CD4: Also known as T-helper cells: A form of white blood cell that is important for immune system functioning; used to determine the stage of HIV infection. Community Health Assistant (CHA): An employee of the Government of Kenya, a trained health worker who supervises the work performed by Community Health Workers assigned to a particular Community Health Unit. Community Health Unit: Within Kenya’s health system, a level 1 health unit comprising about 5,000 individuals, with oversight by a Community Health Assistant (CHA), supported by a cadre of Community health volunteers; fulcrum of the Community Health Strategy. Community Health Volunteer (CHV): An individual, male or female, recruited and trained to provide basic home-based and community-based health services; community mobilization and referral is a central function, with a focus on maternal and child health, community hygiene and sanitation, and family planning. Each CHV is assigned to a specific Community Health Unit and supervised by a Community Health Assistant; generally regarded as a volunteer though some CHVs receive stipends. Continuing Medical Education (CME): In-service training and updating of knowledge and skills to maintain a certain standard of clinical proficiency for different cadres of health professionals. County Health Management Team (CHMT): Entity created under devolution to provide technical and management coordination and oversight of health service delivery within a particular county. Mentorship: A form of strengthening the capacity of health service providers and/or technical staff through one-to-one pairings with technical advisers and SCHMTs. Operations research: Application of scientific principles to test programmatic solutions (tools, strategies) to implementation challenges and/or service delivery problems. Opportunistic infections (OIs): Various types of infections (e.g., viral, bacterial, fungal) associated with a weakened immune system. Quality improvement (QI): A series of techniques and/or methods employed to maximize high standards and performance at health service delivery sites and/or by persons involved in community￾based service delivery. Sub-County Health Management Team (SCHMT): Under Kenya’s devolved governance system, provides coordination/oversight of community health services. In theory, it is akin to the District Health Management Team, an entity that existed before devolution. ix EXECUTIVE SUMMARY The United States Agency for International Development Kenya and East Africa (USAID/KEA) commissioned Evaluation Services and Program Support (ESPS) to conduct a mid-term evaluation of the Afya Jijini (hereafter, referred as AJ) Activity, a three-year contract (September 2015 through August 2018) with two option years funded by USAID/KEA. The Activity is implemented by a consortium of five partners led by IMA World Health. Overall objectives of the mid-term evaluation are to: 1) review implementation approaches and determine the extent to which the Activity is on￾track in achieving its intended outcomes; 2) identify what needs to change to achieve expected results by the end of the Activity; and 3) provide recommendations to improve any weak areas and to sustain areas of strength. The evaluation utilized non-experimental, mixed methodology (qualitative and quantitative) approaches. The evaluation team (ET) utilized secondary data sources, including district health information system (DHIS) and early infant diagnosis (EID) databases, to understand the Activity’s progress-to-target achievements by year three of the Activity’s implementation. Trend analysis; strengths, weaknesses, opportunities and threats (SWOT) analysis; and contribution analysis through case study approach were used to understand what worked and what did not work. A total of 73 key informant interviews (KIIs), 30 focus group discussions (FGDs), nine panel discussions and 487 knowledge, attitude and practice (KAP) surveys of beneficiaries were conducted between May and June, 2018. A validation workshop held with the stakeholders helped ground evidence for developing conclusions and recommendations. Purposive sampling of KII and FGD participants and a small sample size for KAP surveys limit the ability to make generalizations; therefore, extrapolating findings and conclusions should be done with caution. To mitigate this limitation, the ET interviewed diverse stakeholders and health service users/clients. Additionally, the ET developed case studies to highlight AJ’s key strengths and weak areas that need further programmatic assessment. AJ is an integrated program to support the Nairobi City County (NCC) in strengthening its health system. Its strategic purpose is to improve and increase access and utilization of quality health services by strengthening service delivery and institutional capacity of health systems. About 70% of AJ funding is earmarked for sub-purpose one (i.e. to increase access and utilization of quality HIV/AIDS services) and 30% for sub-purpose two (i.e. to increase access and utilization of maternal, newborn and child health (MNCH), family planning (FP), water, sanitation and hygiene (WASH) and nutrition services). Sub-purpose three (i.e. to strengthen and build a functional County Health System) is cross-cutting and supported by funds from sub-purpose one and two. The trend analysis of performance indicators suggests that mother to child transmission (MTCT) rates for HIV at AJ supported sites (n=35) progressively declined from 5.1% in year 0 (Oct 2014 - Sept 2015) to 2.0% in year 3 (Oct 2017 – June 2018). AJ supported 42 health facilities (HFs) to deliver antiretroviral therapy (ART) services. The trend analysis of viral load (VL) suppression data from these facilities (n=40) suggest that VL suppression rates remained constant at 86% for the last three years, failing to achieve the target of 90% or above. Other indicators not yet achieved include ART retention rates at 60 months, HIV exposed infants (HEI) receiving polymerase chain reaction (PCR) test within two months, and people living with HIV (PLHIV) linked to care and treatment services. Under sub-purpose one, AJ support for HIV testing services (HTS) at HFs resulted in high HIV testing rates at antenatal clinics (ANCs). Linkage of positive infants to care and treatment is low (15%) and at 18 months a significant percentage (44%) of infants are either of unknown status or lost to follow up. AJ supported tuberculosis (TB) clinics by updating staff guidelines for TB screening and TB/HIV co￾infection treatment, supporting sample sputum transport for GeneXpert diagnosis, and HIV testing x through adherence counsellors. AJ introduced appointment dairies and phone defaulter tracing by peer educators and mentor mothers to improve defaulter tracing. However, respondents from County/sub-county health management teams (C/SCHMT) felt that defaulter tracing is insufficient because active community case finding mechanisms, including linkage with community through community health volunteers (CHVs), are not available. Under sub-purpose two, analysis of AJ performance indicators show that targets were achieved on pregnant women completing at least four ANC visits, childbirth under supervision of a skilled care provider, and full immunization rates for children under one year of age. By April 2018 (year 3 of activity), output indicators were not achieved for newborn receiving care by skilled health care workers (HCWs) within two days of birth, and modern contraceptive use among women of reproductive age. A decrease (40%) in reported cases of maternal death at targeted HFs was achieved, but a decrease in newborn death rates at targeted HFs was not. It was difficult to measure progress towards nutrition targets as the Activity tracks the numbers of children reached with the 11 High Impact Nutrition Interventions (HiNi), while the contractual progress indicator tracks infant and young child nutrition (IYCN) guideline compliance at AJ-supported health facilities. Moving forward, the Activity should track the number of facilities integrating HiNi into service provision to be able to measure progress towards compliance with the IYCN guidelines. MNCH beneficiaries were generally satisfied with health service availability at AJ-supported HFs, but many preferred private facilities due to poor staff attitude, shortage of staff, long waiting time, and non-availability of essential medication. Adolescent girls and young women (AGYW) FGD respondents benefited from the AJ- supported the Determined, Resilient, Empowered, AIDS-Free, Mentored, and Safe women (DREAMS) project package which provided capacity building in asset building and entrepreneurship, easy availability of condoms and pre-exposure prophylaxis (PrEP), and gender based violence (GBV) prevention interventions including safe spaces. Some challenges identified include gender issues and misconceptions about PrEP, and a lack of dignity kits at HFs caring for GBV survivors. Voluntary medical male circumcision (VMMC) services are provided mostly to school going boys of non-sexually active age-groups. Under sub-purpose three, AJ supported capacity building of the Nairobi CHMT, ten SCHMTs, four high volume facilities (HVFs) and four low volume facilities (LVFs). Respondents were generally satisfied with AJ support in developing policies, strategies, and tools including, annual work plans (AWPs) at the county and sub-county level. AJ also recruited staff and CHVs in the HFs. Reported weaknesses include problems with communication and accountability (i.e. some interventions were started then stopped without communicating or meeting the needs of CHMT/SCHMTs). Problems with payments/ reimbursements reported by CHVs in year one and two had improved in year three. HF in-charges and HCWs greatly appreciated the support they received from the Activity in: quality assurance/quality improvement (QA/QI) trainings, tools, and standard operating procedures (SOPs); supportive supervision and the formation of work improvement teams (WITs). They reported being empowered to carry out QA/QI activities independently. In its cross-cutting work, AJ established coordination and collaboration committees and technical working groups (TWGs) to support the development of various policies and frameworks for strengthening county health systems. KIIs with the CHMT reported that AJ partnership with the United States government (USG) partners works well with no misunderstandings. With USG partners, AJ supported the development and dissemination of human resources for health (HRH) policy, IQCare (an electronic medical record system), lab sample networking, and commodity securities and management. Partnership with other donor organizations is limited. xi The AJ Activity has been trying some innovative service delivery approaches including ‘We Men care’, ‘men priority clients’ and ‘male champion’ models to improve male involvement in MHCH/FP services, access and utilization of HTS and other health services. To decrease long wait times and decongest maternities during weekdays, AJ piloted the ‘MCH7/7’ model where maternities open on weekends. AJ supported staff reimbursements, site operations and logistics for these innovative models. In the absence of formal assessment or evaluation, the ET could not assess health outcomes improvements or achievements of these models. In conclusion, AJ seems to be on-track on achieving its intended results in many aspects. Access to HIV testing, care and treatment has increased; however, gaps exist in follow-up and linkages from testing to treatment services. There are missed opportunities to initiate pregnant HIV positive women on ART and follow-up on mother-baby pair through at least 18 months after birth. The quality of MTCT services is sub-optimal. Gender, youth, and key population activities are limited in terms of coverage and utilization of services. Tools and methods for quality service delivery are not fully assimilated within institutions. AJ has focused on strengthening county and sub-county governance (e.g. work planning), and monthly data reporting. There are gaps in the referral system. To an extent, the Activity has enhanced institutional capacities to analyze, implement and evaluate development activities and improve informed decision-making processes using tools. With high staff turnover, new staff are continuously added yet training is weak with an absence of refresher or follow-up training. Coordination mechanisms are in place, but need further strengthening to ensure sustainability. CHMT reported taking a much stronger role in partnership coordination of government and donor￾supported programs. AJ is working at strengthening management capacity at the SCHMT level, but more work is needed. There is a need for further assessments of program innovations to confirm efficacy before scale up. AJ’s approach of working with relevant local community based organizations (CBOs), CHVs and private institutions is appropriate and necessary to ensure sustainability of activities and services beyond the life of the Activity. It remains unclear how the Activity will address sustainability of innovative models in the next two years. Unless these service delivery models are adapted by the local government or mainstreamed, institutional sustainability cannot be ensured, especially if AJ stops its support. The evaluation concludes with recommendations. AJ should develop a strategy to address barriers to uptake of services in public HFs and to refocus its resources for underperforming interventions. AJ could create centers of excellence in each of the ten sub-counties to model integrated health facilities with improved clinical management, referral systems, and community linkages. To improve on coordination, a communication framework should be designed taking into consideration the various stakeholders on the ground, their concerns, their sensitivity to government priorities, and their expected involvement in implementation. AJ should conduct operations research/impact assessments of innovative service delivery models and synthesize key results and lessons learned for national/county stakeholders. AJ and CHMT should jointly prepare and implement sustainability plans for innovative models. Starting with the year four work plan, a clear sustainability and exit strategy should be agreed upon between AJ, CHMT and USAID. In design and implementation of any future innovations, USAID should also consider a phased-implementation approach that includes transition timelines, so that towards the end of each innovative model implementation local stakeholders are driving the process with USAID implementing partners taking on an advisory role. 1 1 INTRODUCTION 1.1 Evaluation Purpose and Objectives International Business & Technical Consultants, Inc. (IBTCI) implements the Evaluation Services and Program Support (ESPS) activity that provides monitoring and evaluation services to the United States Agency for International Development/Kenya/East Africa (USAID/KEA). In March 2018, ESPS received a task order from USAID/KEA to conduct a mid-term performance evaluation of the Afya Jijini (hereafter, referred to as AJ) Activity. The main purpose of the evaluation is to provide recommendations which will form the basis for potential adjustments to program activities or management, as needed, and document any higher-level issues for USAID/KEA and/or relevant ministries in Kenya to consider for any future activities. The overall objectives of this mid-term evaluation are to: 1) review implementation approaches for each output; determine the extent to which the Activity is on track in achieving its intended outcomes; and determine what does and does not work well; 2) identify what needs to change to achieve expected results and outcomes by the end of the Activity; and 3) provide recommendations about any required changes that will enable the Activity to improve any weak areas and to sustain areas of strength. 1.2 Evaluation Questions The evaluation questions, as described in the scope of work (Annex 1), are: 1. What progress has the Activity made towards the achievement of the Activity’s midterm mandates/targets/outcomes/milestones? With regard to the activity design, what are the main constraints and/or weaknesses, opportunities and threats that need further attention? 2. What coordination and collaboration mechanisms/structures has the Activity established and/or strengthened among key stakeholders (i.e. service delivery implementing partners, relevant Government of Kenya (GOK) ministries including Social Services)? How effective have these coordination/collaboration structures been in achieving program objectives? 3. What sustainable and innovative service delivery models does the Activity currently implement? What do the Ministry of Health and other relevant stakeholders think in terms of their long￾term use? 4. What are the key programmatic and management recommendations that the mission could consider for mid-course changes to the current program design? 1.3 Intended Audience The primary intended audience of this evaluation includes the Afya Jijini implementing partner (IP) IMA World Health and its consortium partners, USAID/KEA mission staff and the GOK, specifically Nairobi County Government/Ministry of Health, which coordinate and interact with Afya Jijini on a regular basis. 2 BACKGROUND 2.1 Project Description The U.S. Mission to Kenya’s HIV/AIDS interagency team under the President’s Emergency Plan for AIDS Relief (PEPFAR), which is composed of the Department of State, the Department of Defense, the U.S. Centers for Disease Control and Prevention (CDC), and the U.S. Agency for International Development (USAID), began Human Immuno-deficiency Virus/ Acquired Immuno-deficiency Syndrome (HIV/AIDS) programming in 2003 under the oversight of the Office of the Global AIDS Coordinator. The U.S. Mission collaborates with the Government of Kenya (GOK) through the Ministry of Health, the 2 National AIDS & Sexually Transmitted Infections (STIs) Control Programme (NASCOP), and other line ministries. The AJ Activity is a three-year (September 1, 2015 – August 31, 2018), $34,913,036 contract with two option years. The purpose of the AJ Activity is to improve and increase access and utilization of quality health services in Kenya through strengthened service delivery and institutional capacity of county health systems. Specifically, the Activity aims to improve key health service coverage in the informal settlements of Nairobi City County. The Activity is implemented by a consortium of five partners: IMA World Health (lead partner), the Center for HIV Prevention and Research at University of Nairobi, the Mission for Essential Drugs and Supplies (MEDS), the National Organization of Peer Educators (NOPE) and the Christian Health Association of Kenya (CHAK). The activity goal is to contribute to improved county-level accountability, institutional capacity, leadership, and management of health service delivery (Country Development Cooperation Strategy (CDCS) Sub-Intermediate Result 2.2.2). Afya Jijini is an integrated program designed to support the County Government of Nairobi in strengthening its health system in the implementation of HIV/AIDS, Mother Newborn and Child Health/ Family planning (MNCH/FP), water sanitation and hygiene (WASH), and nutrition services. It builds on previous work carried out by APHIAplus programs to better support the Nairobi City County (NCC) government, focusing on health service utilization among populations living within informal settlements. The strategic goal of the program is to improve county-level capacity and management of health service delivery. The strategic purpose is to improve and increase access and utilization of quality health services in NCC through strengthened service delivery and institutional capacity of health systems. The results framework for the Activity is provided in Figure 1. The Activity had three Sub-purposes: • Sub-purpose one: Increase access and utilization of quality HIV services. • Sub-purpose two: Increase access and utilization of focused MNCH/FP, WASH and nutrition services. • Sub-purpose three: Strengthen and build a functional County Health System. About 70% of AJ funding is earmarked for Sub-purpose one and 30% for Sub-purpose two. As Sub￾purpose three is cross-cutting, it is supported by funds from Sub-purpose one and two. To enhance the achievement of its ambitious targets and deliverables, the Activity collaborates closely with the GOK, development partners (Global Fund, DANIDA, UNICEF) and stakeholders at different levels: inter￾agency coordination committees (ICC) and technical working groups (TWG) at the national level; Sub- /County Health Management Teams (S/CHMTs); the Sub-county Health Stakeholder Forum (SCHSF); Facility Health Management Committees and; Community Health Committees. 2.2 Project Context Figure 1: Results Framework for Afya Jijini Activity 3 HIV/TB/MNCH/ FP/WASH/Nutrition HIV/TB statistics: TB prevalence per 100,000 persons – 558 HIV prevalence rate (2012) – 3.3% New HIV infections – 13,510 MNCH statistics: Neonatal Mortality Rate – 39 deaths per 1,000 live births Infant Mortality rate (IMR) – 39 deaths per 1,000 live births Under 5 Mortality Rate – 52 per 1,000 live births Fully Immunized population < 1 year – 97% Median age at first marriage for women - 22.1 years Median age at first birth among women - 22.2 years. FP/Postnatal Care statistics: Total Fertility Rate (TFR) – 2.7 Contraception prevalence – 38.7% Birth delivered at health facilities – 78.9% Postnatal care from SBA – 71% Family size for women = 3.0 children Unmet FP need = 11% Nairobi County’s population (3,138,369 in 2009) is projected to increase to 4,390,158 in 2018. This population is distributed across 17 administrative sub-counties, which have been merged into 10 health sub-counties for ease of health administration (NCCa, 2017). The population distribution shows that the age group of under 15 years accounts for 40.7% of the total population, the young adult age group of 15–29 years accounts for 38.6%, and the reproductive age groups of 15–49 years accounts for 40.7% with figures of 22% and 18% for males and females respectively. The age group of over 60 years accounts for 2% of the population (KNBS, 2009). The four most densely populated sub-counties are Mathare, Embakasi North, Ruaraka and Kamukunji with population densities of over 20,000 people per square kilometer. The least densely populated sub-counties are Westlands, Langata, Kasarani and Embakasi East. Furthermore, it is estimated that about 58% of Nairobi’s population lives in slums or slum-like conditions (UN Habitat, 2010) with about 55,000 refugees and asylum seekers living in the city. Box 1 provides a snap-shot of disease statistics in Nairobi. In Nairobi slums, mortality rates for children under five years old are double the city’s average and greater than those in rural areas (UN Habitat, 2010). Studies have indicated that children living in slums are more likely to die from pneumonia, diarrhea, malaria, measles or HIV/AIDS than children from wealthier parts of the same city or rural areas. Nairobi’s informal settlements are mostly unplanned, resulting in inadequate infrastructure (e.g. roads, water, sanitation, sewerage, drainage, and electricity), temporary and haphazard building solutions, poor housing, high occupation density, and very low levels of public services. Clean water, adequate sanitation, and hygiene are visibly substandard. This was confirmed in a consumer survey which illustrated informal settlement residents’ general discontent with services (MWI, 2007). In Nairobi’s informal settlements, an estimated 24% of residents have access to household toilet facilities while 67% rely on shared facilities and 6% have no access to toilets (World Bank, 2006). 3 METHODOLOGY 3.1 Evaluation Design This mid-term evaluation adopted a descriptive, non-experimental, mixed-method performance evaluation approach that employed a multiple cross-sectional study design. This approach provided the ability to review and hear different inputs for the same questions and triangulate data from multiple sites and types of stakeholders. 3.1.1 Sampling strategy and selection criteria Box 1: A Snap-Shot of Health Indicators Statistics in Nairobi County Source: KDHS, 2014; Kenya Tuberculosis Survey, 2016 4 Figure 2: Afya Jijini Focus County and Sub-Counties The evaluation used a purposeful sample selection technique based on the AJ Activity’s partners’, sub-contractors’, and beneficiaries’ input. The respondents were selected purposively in consultation with AJ Activity partners and USAID/KEA to ensure that actors who were most informed about AJ interventions were included so as to provide meaningful and reliable data for the evaluation exercise. Site selection: The sites for evaluation were restricted to Nairobi (AJ Focus County) and its sub-counties (Figure 2). For administrative and design purposes, the evaluation team stratified the Nairobi County into four clusters. Due to cost considerations, and the geographical spread of the health facilities, the evaluation team selected 50%, i.e. eight sub-counties of the 16 sub-counties supported by the Activity. These are – Westlands, Langata, Kasarani, Ruaraka, Embakasi South, Embakasi North, Makadara, and Mathare sub counties. Respondent selection: Tabl11 (see Annex 2) provides the listing of respondent categories as well as the number of respondents interviewed within each category. A total of 73 key informant interviews (KIIs), 30 focus group discussions (FGDs), nine panel discussions and 487 Knowledge, Attitude and Practice (KAP) surveys of beneficiaries were conducted between May and June, 2018. Please see further details of respondent selection methodology, respondent type categories and numbers of interviews, discussions and survey respondents in Annex 2. 3.1.2 Data collection methods A summary table linking evaluation questions to corresponding information on data source and data collection and analysis methods is presented in the Evaluation Plan Matrix (see Annex 3: Evaluation Plan Matrix). This midterm evaluation data collection was carried out by a six-person evaluation team (ET) – Team Leader (TLs- Jean Capps (led the data collection phase) and Dr. Swati Sadaphal), Public Health Specialist (PHS- Dr. Makeba Shiroya-Wandabwa), Reproductive, Maternal, Neonatal & Child Health Expert (RMNCH- Mercelline Ogolla), Social Scientist (SS- Gilbert Alouch) and Senior Monitoring & Evaluation Expert (Sr. M&E- Peter Mwaura). All personnel signed a non-conflict of interest form. The evaluation team primarily utilized the following data collection methods to address the evaluation objectives and questions. Document review: A content analysis of project reports and other relevant documents was completed to assess the appropriateness of the technical approach; understand inputs, outputs and synergies among various interventions; and determine the extent of their contributions in achieving the objectives of the AJ Activities. This content analysis identified themes for triangulation with other data collected and analyzed as part of the evaluation. Baseline estimates and progress at year 3 for the Activity’s performance indicators were obtained from the District Health Information System 2 (DHIS2), the Early Infant Diagnosis (EID) database, and the community–led total sanitation (CLTS) database and were 5 reconstructed from the facility register entries. A complete list of these documents is attached to the report as Annex 4. Document review provided the basis for designing the qualitative and quantitative data collection tools (see Annex 5). All tools were submitted to USAID/KEA for approval prior to deployment. Key informants interview (KII): A total of 73 KIIs were conducted. Interviews were conducted either in a one-on-one session or in a group setting (with 2-3 participants). The KIIs were conducted face-to￾face or by phone. Semi-structured KII guides were utilized to obtain: i) key informants perspectives on the effectiveness of technical and management approaches, contributions, gaps, and overlaps with related interventions; ii) insights into the achievements, challenges and sustainability of AJ activities; iii) the extent to which crosscutting interventions influence activity progress and outcomes; and iv) the extent to which Afya Jijini activities/results supported the Nairobi County to become more credible and build their confidence and ability for informed decisions. A complete list of persons interviewed is attached to the report as Annex 6: List of Persons Interviewed. Focus group discussions (FGD): A total of 30 FGDs were conducted. Semi-structured FGD guides were utilized. FGD respondents included the following four target groups: Maternal Neonatal and Child Health (MNCH) clients; Comprehensive Care Clinic (CCC) clients; youth (15-24 years) beneficiaries of HIV prevention services offered under the Determined, Resilient, Empowered, AIDS-Free, Mentored, and Safe women (DREAMS) project; and community health volunteers (CHV). The FGD sessions provided insights critical for grounding evidence emerging from the other data sources. Panel discussions: A cross-sectional institutional capacity assessment (ICA) of Nairobi County and eight sub-county teams (S/CHMTs) was conducted. A structured qualitative assessment questionnaire was administered for each S/CHMT in a panel discussion format. The objective of these capacity assessments was to assess: a) S/CHMT’s, health facilities’ and community health units’ ability to sustain the Activity’s achievements beyond the life of the project; b) S/CHMT’s ability to organize communication strategies; c) S/CHMT’s ability to plan and coordinate implementation of effective multi-sectoral partnerships for health; d) S/CHMT’s and community units’ ability to manage social protection initiatives; e) health facilities’ ability to run and sustain quality improvement systems; and f) community units’ ability to mobilize communities in the targeted counties. Knowledge Attitudes and Practices (KAP) surveys: A quantitative KAP survey was administered among four AJ target groups: MNCH clients (n=98), CCC clients (n=97); youth (15-24 years, n=162), CHVs (n=92) and health workers (n=38). These surveys were used to capture data on knowledge, attitude, behavior and practice, relevant to the Activity’s performance indicators. 3.1.3 Data management The Content-Data Abstraction Template was used for organizing the data and information obtained during the desk review and qualitative interviews. The transcripts were tracked in a transcript log indicating interviewer’s name, respondent details, date conducted and date transcript received. Quantitative KAP surveys were conducted using Android-enabled mobile phones with SurveyToGo Dooblo data collection software. Four Research Assistants were trained to administer KAP surveys and the data were uploaded onto a remote secure server the day of data collection. Quality assurance for surveys was provided by Sr. M&E Expert. The database had data entry screens with skip patterns and data/value ranges programmed into it. This ensured that the data were consistent at the point of interview. The data were then exported into SPSS for statistical data analysis. Verbal consent was taken from all respondents. The evaluation team ensured protection of data confidentiality by only submitting summaries of qualitative data transcripts to USAID as part of the raw data and presenting data in this report without linking any personal identification details of respondents. No personal identifying data 6 such as national identity number and residential address of respondents were collected during the KAP surveys. Names of respondents were removed prior to the data analysis. 3.2 Data Analysis and Triangulation Primary qualitative data sources included interview and discussion transcripts. Thematic analysis categories and sub-categories were developed, modified and extended on the basis of emergent themes. The qualitative information was then coded using Atals-ti software, compared, and re-categorized as new themes emerged. Primary quantitative data sources included KAP surveys. The data were analyzed using SPSS software version 23. Cross- tabulation analysis for knowledge, attitudes and practices, and behaviors indicators was done for different categories of respondents. Content analysis of background documentation was used for data triangulation. Quantitative trend analysis was used for the Activity’s performance monitoring indicators under each sub-purpose of the AJ Activity. Additionally, triangulation was done by comparing findings across data sources and across respondents. Strengths, Weakness, Opportunities and Threats (SWOT) analysis focused on the Activity’s design to identify internal factors (strength and weakness) and external factors (opportunities and threats) that may have contributed to the observed outcomes. The ET developed a logic model for contribution analysis to help draw conclusions about the AJ Activity’s efforts towards the achievement of the observed health outcomes. The analysis focused on the annual work plans and progress reports to document the intended interventions, implementation strategies, resource allocation, gaps in outputs, and the observed health outcomes. The conclusions and recommendations were drafted after data triangulation and interpretation. 3.3 Evaluation Activities and Timelines Team planning meetings: Prior to the field work, the ET conducted a six-day team-planning phase during which it finalized the evaluation methodology, data collection tools, data analysis plan and work plan. In-brief meeting with IP: A meeting with IMA World Health was conducted to identify respondents, relevant background documents and data, and to receive contextual information about Activity design and implementation. USAID In-brief presentation: An in-brief meeting was organized between the ET and USAID/KEA prior to the fieldwork. Field data collection & analysis: The data collection period lasted about five weeks in May-June, 2018. The team performed the data analysis between July 9 and July 25, 2018. Data validation meeting: The ET conducted a stakeholder meeting with the AJ IP team and the Nairobi County and sub-county staff to objectively review the preliminary findings, seek further contextual insights into the ET’s observations, and seek recommendations for the mid-course corrections to program implementation and design. USAID out-brief presentation: A synopsis of the findings was shared with USAID/KEA during the ESPS/USAID out-brief meeting on July 23, 2018. 3.4 Limitations Some limitations to the evaluation design must be acknowledged. First, because purposive sampling methods were used for selecting KII and FGD participants, the analysis does not make inferences or generalize issues to the general population of stakeholders. Second, because key informants constituted the primary source of information, the interview data were subject to personal biases, opinions, and recollection (i.e. respondent bias). To mitigate this, the ET used the desk review period to design appropriate probing/follow-up questions. Interviewer bias was mitigated by daily team debriefs when the team members compared their transcripts. Rigor and validity of findings in the methodology was ensured through triangulation of interview and document sources, appropriate sampling of KII and FGD participants, and validation of both the data collected and their interpretation throughout the data collection and analysis processes. These methods collectively reduced bias due to the subjective nature of qualitative methods. There is no comparison group – the performance evaluation focused on 7 5.1% 5.2% 2.8% 2.0% 0.0% 1.0% 2.0% 3.0% 4.0% 5.0% 6.0% Year 0 (Oct 14 - Sep 15) Year 1 (Oct 15 - Sep 16) Year 2 (Oct 16 - Sep 17) Year 3 (Oct 17 - Jun 18) MTCT rates in AJ supported sites (n=35) Source: EID database Figure 3: MTCT Rates at AJ Supported HFs intervention areas (i.e. sub-counties within the NCC) and did not take into account other counties where the technical interventions have not been delivered. The evaluation required a review of documents and reconstruction of baseline estimates and follow-up. Some of the respondents from IP and GOK were new and not familiar enough with the activity to provide historic information. The team faced difficulty in obtaining the data on all relevant indicators as not all indicators are reported consistently to the database (e.g. tuberculosis (TIBU) data could not be accessed at all). The analysis thus factored in all these dynamics and documented instances where outcomes measures are grossly affected. The sample size for quantitative surveys was small and the respondents were health facility clients or service users so the results are not generalizable and therefore, extrapolating findings and conclusions should be done with caution. To mitigate this limitation, the ET conducted qualitative interviews with a diverse type of stakeholders and health service users/clients at the community and facility levels. Additionally, developed case studies to highlight AJ’s key strengths and weak areas that need further programmatic assessment. 4 FINDINGS & CONCLUSIONS 4.1 EVALUATION QUESTION 1 What progress has the Activity made towards the achievement of the program’s midterm mandates/targets/outcomes/milestones? With regard to the activity design, what are the main constraints and/or weaknesses, opportunities and threats that need further attention? 4.1.1 Sub-purpose One: Increased Access and Utilization of Quality HIV Services AJ supported a total of 43 health facilities (HFs) for elimination of mother to child transmission (eMTCT) services, 78 HFs for HIV testing services (HTS), 42 Comprehensive Care Centers (CCCs), 7 HFs for VMMC for HIV prevention, and 36 HFs for tuberculosis (TB) services. The early infant diagnosis (EID) database showed that MTCT rates at AJ supported sites (n=35) at year 3 (Oct 2017- June 2018) was 2% (target: less than 5%) and had progressively declined from year 0 (Oct 2014 - Sept 2015). This is an achievement (Figure 3). The activity is on￾track to achieve Nairobi County MTCT Rate target of less than 3% by 2019. AJ supported 42 HFs provided antiretroviral therapy (ART) services. Trend analysis of viral load (VL) suppression data from these facilities (n=40) suggest that VL suppression rates are 86% (target: 90% or above) and the target was not achieved yet (Figure 4). 86% 86% 85% 86% 82% 83% 84% 85% 86% 87% 88% 89% 90% 91% Year 0 (Oct 14- Sep 15) Year 1 (Oct 15- Sep 16) Year 2 (Oct 16- Sep 17) Year 3 (Oct 17- Jun 18) Viral load suppression rates in AJ supported sites (n=40) Source: DHIS2 Viral load suppression rates Target Figure 4: Viral Load Suppression Rates at AJ Supported HFs 8 Analysis of AJ performance indicators results show that AJ achieved output indicators related to eMTCT, HIV prevention, TB/HIV co-infection. Indicators related to HIV care and treatment services were not achieved by the end of month (April 2018) in year three of the AJ Activity. Indicators not achieved include ART retention rates at 60 months, HIV exposed infants (HEI) receiving polymerase chain reaction (PCR) test within two months and people living with HIV (PLHIV) linked to care and treatment services. Please see Annex 7 for the details. County Health Management Teams (CHMT), Sub-county Health Management Teams (SCHMTs) and health facility in-charges reported AJ supported dissemination and updating of HCWs on revised HTS guidelines for pregnant and breastfeeding mothers. In addition, they reported AJ had recruited PMTCT nurses and supported the transportation of dried blood samples for EID to KEMRI laboratories. Healthcare workers at Kangemi CCC, Westlands CCC and Mukuru reported health facility based Mentor Mothers were recruited to provide peer education, psychosocial support and phone defaulter tracing to HIV+ pregnant women. The CHMT respondents reported that there is still a high proportion HEI with unknown status and loss to follow up; they attributed this to the mobile, migratory nature of people in informal settlements and postulated that linked electronic medical record systems would improve linkages. Only one out of the six comprehensive care centers (CCCs) visited had integrated immunization and family planning with the PMTCT clinic. HCWs at the clinic reported the integration improved work efficiency and clients were more adherent to clinic visits since the integrated clinic offered a ‘one stop shop’ for PMTCT, FP and Child welfare services such as immunization. Respondents from CHMT and three HFs reported that the AJ supported VL sample networking increased and improved the VL testing uptake. HCWs interviewed at six CCCs reported AJ had supported distribution of VL tracking registers and supported fast tracking of VL sample collection by employing nurses to take samples from eligible patients identified through file audits. Concerns were raised about poor suppression rates (< 90%) by CCC HCWs (non-AJ paid worker at AJ supported site) and they attributed this to poor adherence. Integration of HTS services at antenatal clinics (ANCs), Labor & Delivery ward and Post Natal Clinic at select health facilities (n=19) is completed. AJ also supported the dissemination of revised HTS guidelines for pregnant and breastfeeding mothers. CCC clients’ discussants in all the facilities sampled, reported satisfaction in availability of HIV testing, care and treatment services. However, they reported issues with staff attendance and time of service provision. Clinical staff at the CCC reporting late to work were cited in three out of eight facilities visited (i.e. Mathare North Health Centre, Kasarani Health Centre and Kariobangi North Health Centre). This was compounded by a shortage of CCC staff as in the case of one facility which reported to have only one clinical officer who occasionally reports late or not at all. CCC services were reported to commence between 9 or 10 am, although clients would prefer to be served earlier. Some clients reported longer wait times due to late opening of the clinics or extended tea breaks by HCWs. In one health facility, it was reported that at times the CCC starts serving clients at mid-day. AJ support to TB clinics included recruitment of adherence counsellors and nurses; updating staff on guidelines for TB screening and TB/HIV co-infection treatment; provision of job aids and algorithms; and support for sample sputum transport for GeneXpert diagnosis and HIV testing through adherence counsellors. Health facility in-charges and HCWs at TB Clinic reported AJ had engaged CHV at the health facilities which supported health education, TB screening, TB drug refills, telephone defaulter tracing and cough monitoring. The CHV cough monitors were supplied with presumptive TB registers. However, hospital-based CHVs reported challenges linking with community-based CHVs to trace TB defaulters or for active case finding particularly of children who are at high risk of obtaining TB from infected adults. SCHMT reported that the AJ model engaged CHVs at the individual level rather than via existing MOH structures specifically the Community Unit and Community Health Assistants (CHAs, who are CHV supervisors). This lack of engagement with existing MOH community structures was 9 perceived as one of the reasons for suboptimal defaulter tracing and active TB case finding in the community. The SCHMT noted the lack of distinct engagement with the existing MOH community systems as a challenge for sustainability after AJ exits. AJ progress reports indicated 60% of high volume facilities had integrated TB clinics with ART. Nonetheless, county and sub-county HMT reported there was still a challenge with linking HIV-positive TB patients from TB clinics to HIV CCCs. Health facility based CHVs were mainly involved in cough monitoring at waiting bay, TB screening (mainly adults), dispensing drugs and health education at the facility. AJ engaged peer educators and mentor mothers for adherence support (through appointment diaries) and phone defaulter tracing. However, some respondents from SCHMT and TB clinics felt that the phone defaulter tracing is insufficient for optimal contact and defaulter tracing. The CHMT and health facility in-charge respondents expressed concern that when the AJ Activity ends the AJ employed nurses, adherence counsellors and support to VL sample networking system will likely stop. All SCHMTs voiced their concern about high turnover of AJ HIV technical and program management staff. They stated this affected quality of care, initiation and continuity of innovative programs. The C/SCHMT and non-AJ HCWs at CCC proposed greater emphasis on building capacity of frontline HCWs at CCCs through more advanced technical mentorship which would enable them to address more challenging and difficult cases. The ET developed a case-study to further understand AJ’s contributions, strengths, weakness, opportunities and threats for sub-purpose one interventions in PMTCT. Case study 1: eMTCT and HEI outcomes at Afya Jijini supported health facilities Context: Kenya is one of 22 PMTCT priority countries and has embraced the UNAIDS super-fast track framework to end new HIV infections among children (Start Free) and keep mothers alive; keep adolescents HIV free (Stay free); and end pediatric and adolescent AIDS (AIDS free) by 2020 (UNAIDS, 2017). As at 2015, Kenya had successfully reduced mother-to-child transmission of HIV to 8.3%. Kenya hopes to eliminate it by 2020 and receive validation of eMTCT and congenital syphilis as a public health problem by 2021(GOK, Ministry of Health (MOH), 2016). AJ eMTCT Strategies and interventions: AJ is expected to contribute to averting new HIV infections and achieve an AIDS-free generation through increasing ART uptake and eMTCT. Table 1 below summaries various interventions AJ implemented from Sept 2015 – April 2018 to contribute to the elimination of mother to child HIV Transmission. Table 1: Afya Jijini eMTCT Strategies and Interventions: eMTCT Activity: HIV Testing and Counselling Services to Pregnant Women HIV+ receive ART Exposed infants receive PCR 2 months after birth MTCT Rate at 18 months < 5% eMTCT Strategies & Interventions Dissemination of HTS Guidelines for pregnant & lactating mothers; Employment of HTS counsellors to support HIV testing at high volume facilities; Integration of HIV Testing into ANC at Training of HCWs on ‘Test and Treat’ as well as recommended ARVs for pregnant & breastfeeding mothers; Integration of ART services at 19 high volume facilities; Support ART commodity availability at MCH clinics; HIV testing at 6 weeks immunization at Child Welfare Clinic. Training of HCWs in HEI cohort analysis as quality improvement intervention for Initiation of Health Facility Work Improvement Teams to improve quality of PMTCT 10 19 health facilities; HTS offered as an opt out to all pregnant women at first ANC visit Employment of eMTCT Nurses to support timely initiation of ART tracking and follow up of HEIs Source: extracts from AJ year1, year 2 and Year 3 annual work plan and progress reports PMTCT Cascade analysis: PMTCT Cascade analysis from 35 AJ eMTCT sites between Sept 2017 and June 2018, showed that more than 93% of pregnant women received HIV testing services at ANC, suggesting HTS strategies are effective. However, 18% of pregnant HIV+ women did not start ART and hence did not have timely initiation on ART regimen. AJ needs to improve linkage of pregnant HIV+ women to ART. About 95% of HEIs received infant prophylaxis which is commendable. Although AJ supports EID sample transport network only 64% of HEIs had PCR results in less than eight weeks. For the period of cascade analysis, 121 HEIs were positive with MTCT rate of 5%, however only 15% were initiated on ART. HEI Cohort analysis: To further understand HEI outcomes, we reviewed HEI cohort data from eight AJ supported HFs from Sept 2015 to January 2018. Table 2 summarizes data and results. Table 2: Review of PMTCT HEI Cohort Data across *8 Afya Jijini Supported Health Facilities No. of HEI registered in six cohorts in 8 HF (N) #HEI Negative at 18 months (%) # HEIs who Died (%) # HEI Positive at 18 months (%) HEI with Unknown HIV status at 18 Months (%) # HEI Lost to follow Up at 18 Months (%) # HEI Transfer Out (TO) Health Facility (%) 375 193 (52) 5 (1) 13 (4) 131 (35) 34 (9) 12 (3) * Mukuru, Makadara, Bahati, Mathare North, St Mary’s, Kangemi, Kariobangi South & Kasarani; Source: HIV Exposed Infant Cohort register Only 4% of HEI were positive at 18 months; however 35% have an unknown HIV status while 9% were lost to follow up. Status of 44% of HEI in these cohorts is unknown, yet HIV support systems such as mentor mothers, peer educators and PMTCT psychosocial support groups have been implemented. Conclusion of case-study 1: AJ support for HTS at HFs resulted in high testing rates at ANC. Linkage of positive infants to care and treatment is low (15%) and at 18 months 44% are either of unknown status or lost to follow up. There is a need to review the current AJ linkage and defaulter tracing strategies. There are missed opportunities to initiate pregnant HIV+ women on ART and follow-up of mother-baby pair through at least 18 months after birth. 4.1.2 Sub-purpose Two: Increased Access and Utilization of Focused MNCH, FP, WASH and Nutrition Services Sub-purpose two services included: antenatal care; family planning (FP); maternity care; newborn care; immediate postpartum care; management of childhood illnesses; immunization; post-natal care; nutrition education, assessment and counselling services; and promotion of safe drinking water and sanitation practices. However, there were variations in the number of services offered by HFs. AJ covered 66 maternities, 252 child care clinics, 252 FP clinics, 190 nutrition facilities and 21 villages in Embakasi, Starehe, Ruaraka, Makadara and Kasarani sub-counties for WASH. 11 Analysis of AJ performance indicators under sub-purpose two shows that indicators have been achieved on pregnant women completing at least four ANC visits, childbirth under supervision of a skilled care provider, and full immunization rate for children under one year of age. Output indicators related to newborns receiving care by skilled health worker within two days of birth, and modern contraceptive use among women of reproductive age were not achieved by the end of April 2018 in year three of the AJ Activity. Please see Annex 7 for details. Decrease in reported cases of maternal deaths at targeted HFs by 40% was achieved, but decrease in newborn death rates at the targeted HFs by 30% is not likely to be achieved by year three. The establishment of the Maternal, Perinatal, Death Surveillance and Response (MPDSR) processes at the county, sub-county and HF level resulted in increased reporting of perinatal deaths. Nutrition indicators could not be assessed. Data from the DHIS indicates that facilities report inconsistently on some of the 11 High Impact Nutrition Interventions (HiNi) indicators. AJ Activity data provided progress of each HiNi indicator through the life of the project but data were not available on whether or not AJ-supported facilities complied with infant and young child nutrition (IYCN) guidelines. It was difficult to measure progress towards nutrition targets as the Activity tracks the numbers of children reached with the 11 HiNi, while the contractual progress indicator tracks IYCN guideline compliance at AJ-supported health facilities. Moving forward, the Activity should track the number of facilities integrating HiNi into service provision to be able to measure progress towards compliance with the IYCN guidelines. KAP survey with MNCH clients indicate that 77% received counselling on benefits of breastfeeding while attending the ANC clinic, 78% received counselling on infant feeding for the first 6 months while attending the maternity, and 48% received counselling on lactation. Data was unavailable on the proportion of children under 5 years with diarrhea receiving ORT (ORS and Zinc) for treatment; however, the DHIS and project reported on the number who receive ORS only. While this indicator had improved gradually and points to a positive health seeking behavior by mothers, to reduce diarrheal cases the Activity may need to re-evaluate the WASH strategy to improve the availability of safe water sources and sanitation services at the household level. Child immunization services were available and consistent across all sampled facilities. Complete immunization coverage (Activity’s contractual indicator) already exceeds national targets and is largely not a function of AJ activities. All HFs visited by the ET provided ANC services. Quantitative data from project reports show that the number of women attending at least four ANC visits trended upward except for a decline during year two when doctors and nurses went on strike. KAP survey results showed that the percentage of pregnant women receiving counselling for MTCT and nutrition is high (86% and 83%, respectively). However, pregnant women receiving counselling on dangers of smoking (49%), pregnancy complications (63%), and intimate partner violence (40%) were low. Family planning services were available in all but one of the selected high volume facilities. While the project data indicate that about 1209 CHVs were trained in the family planning modules, only a few were engaged as community based distributors (CBDs). During the qualitative interviews, it was reported that there were periodic stock outs of family planning commodities affecting CBDs the most. In addition, long-acting reversible contraceptives (LARC) are not provided on a routine basis due to lack of skilled personnel to offer the method; in some areas these methods are offered during outreaches by a non USG partner. At the health facility level, there is a clear lack of integration of family planning services in HIV service provision. In addition, while the Activity’s contractual indicator is to increase the number of women of reproductive age using modern contraceptives, the Activity reports on Couple Years of Protection (CYP) which is based on the volume of all contraceptives sold or distributed free of charge to clients. It is therefore difficult to indicate progress towards the FP target. 12 The ET could not find relevant data from the DHIS or HF registers to assess progress towards achievements of output 2.4: WASH and output 2.5: Nutrition. The ET reviewed results from a recently conducted survey in Nairobi county slum areas (Concern Worldwide Kenya, 2017). These survey results were used as an independent data source to assess achievement of AJ progress on WASH indicators. The survey was conducted in May 2017 in partnership with the Ministry of Health, Feed the Children and Concern Worldwide and was funded by UNICEF. The target geographical area was the slums in Nairobi County. Specifically, the survey covered the following slums; Makadara (Viwandani), Embakasi East (Mukuru), Lan’gata (Kibera), Westlands (Githogoro/Kibagare), Dagoretti (Kawangware/Gatina), Ruaraka (Korogocho), Kasarani (Gitare), Starehe (Mathare), Embakasi West (Kayole Soweto), Kamukunji (Kiambio/ Majengo). The survey showed that 52% (496) of households reported treating water before drinking, with 61% (304) boiling the water, 45% (223) using chemicals, and only 0.8% (4) using pot filters. The ET used the Nairobi County DHS Survey (2014) as a baseline; it showed that in 2014 about 45% of respondents used a method to treat water before drinking. Comparing these two data sources shows an increase of 7% in the prevalence of practicing correct use of recommended household water treatment methods in May 2017 (or around third quarter of year two of AJ). AJ target is to increase correct use by 10% in year three, by Sept 2018. KIIs with CHAs reported a reduction in the number of mothers giving birth at home. The CHVs have also been conducting MNCH health education activities that have been reported and will continue even if AJ were to exit since they have already been empowered. In addition, AJ has also supported mentor mothers who conduct home visits to AGYW and link girls with safe spaces for GBV services and provide referrals for MNCH and FP services. FGD with AGYW in Embakasi revealed that some women and girls do not use any FP method due to religion while others reported myths including one that “any unmarried woman using FP will never give birth once married.” Others are discouraged from using FP methods by their husbands. The use of PrEP by AGYW has been embraced, however, others do not use it due to fear of potential reprisals from their spouses or boyfriends; however, some who suspect their partners of having extramarital relationships reported using PrEP in secrecy. However, it was reported that limited awareness about PrEP especially among men and even other AGYW is a challenge to its wide use. In the case of MNCH services, respondents also reported challenges in health-seeking behavior related to culture, gender, and health system issues. Another challenge is non-availability of drugs at the facility at all times. MNCH client FGD discussant in Mathare North reported “my child was sick but I did not bring him to the facility since I knew from experience that they will tell me to go buy drugs”. In Kangemi it was reported that families mostly prefer to go to chemists or private clinics because there are no drugs at the facility. Similar findings were reported in FGDs at Bahati and Makadara. At Makadara: “these staff will send you to a particular laboratory outside the facility regardless of your illness even before they review your clinic card”. Mothers who deliver at night (Mathare North and Mukuru) are not served any meal, and when referred to Pumwani hospital in most cases the ambulance does not have fuel so the patient has to find their way. The facility normally serves the first ten mothers and tells the rest to go back home. In all surveyed facilities, MNCH services are only available during morning hours, after 12.30 the HCWs were reported not to attend to children. Not all MNCH services were provided by all the health facilities visited. Mathare North Health Centre does not conduct caesarian deliveries, has no nursery for underweight or pre-term babies, and does not deliver first born babies but refers these women to Pumwani. MNCH clients reported lack of sufficient numbers of nurses, particularly, in maternities. AJ respondents reported constraints in maternal death reviews and subsequent corrective action planning. Data shows that no sampled high volume facilities provided EmONC services. The respondent reported a challenge in making blood transfusions available for serious cases of post-partum hemorrhage (PPH) and in linking the bleeding PPH mother to comprehensive EmONC sites with adequate supply of 13 blood and essential medications. Another gap is the lack of health education to the mothers on all pregnancy and newborn related health complications and on when to seek care immediately. The ET developed a case-study to further explore and understand AJ’s contribution, strengths, weakness, opportunities and threats for Sub-purpose two interventions in the postnatal care. Case study 2: Newborns in AJ target areas evaluated by skilled HCW within two days of birth Context: Nairobi City County has the highest county-specific Neonatal Mortality Rates (39 per 1000 live births) in the country (KDHS, 2014). On average, 72% of mothers in Nairobi receive postnatal care (PNC) checkup for the mother within the first two days of birth; however, only 31% of Kenyan women in the lowest wealth quintile, and 55% of mothers under the age of 20 years receive such care (KDHS, 2014). Data form the DHIS-2 show that Langata and Kamukunji, Ruaraka and Dagoretti sub-counties yield large numbers of reported neonatal deaths. AJ strategies and interventions: Table 3 presents the interventions implemented by the AJ Activity under sub-purpose two specifically related to post-natal newborn care. Table 3: A Summary of Afya Jijini MNCH Interventions for Postnatal Care Year one (2015-16) Year two (2016-17) Year three (2017- 18) • Engagement of reproductive health field Assistant nurses in high volume facilities • Training/Mentorship/on-the-job trainings (OJT) and continued medical education seminars (CMEs) on elements of essential newborn care/emergency obstetrics and neonatal care (EmONC)/MNH module • Distributed (EmONC) equipment to 52 maternities, key MNH supplies and job aids; basic MNH equipment to 78 facilities • County consultation on MCH7/7 weekend operational hours at 23 high-volume ANC/MNCH clinics in informal settlements • Establishment of Maternal Perinatal Death Surveillance and Response (MPDSR) Committees/Establishment of work improvement teams (WITs) • Mentorship ,OJTs and CMEs on EmONC; Kangaroo Mother Care (KMC); use of chlorhexidine; essential newborn care; Health education on PNC provided by CHVs • Procurement of additional stopgap consumables • Supported participation in dissemination of a chlorhexidine study done by Partner for Appropriate Technologies in Health (PATH) • Referral by CHVs for cases of home deliveries within the first 48 hours (or soon as possible) • Human resource for health (HRH) support St. Francis Community Hospital • Technical assistance to WITs • Mentorship and OJT on various topics related to maternal and neonatal PNC (10 facilities) • Distributed PNC registers • Supported a midwife to conduct community-level follow-up to postnatal mothers • Referral by CHVs for cases of home deliveries within the first 48 hours (or soon as possible) Analysis results: Program data indicate that the Activity is on-track to achieve performance indicators with an increase seen in the number of newborns evaluated within 48 hours in year three. In year two, facilities provided PNC for 20,282 newborns, which increased to 22,571 in the first quarter of year three. In year two, the nurses’ strike affected service delivery in public facilities and resulted in an influx of patients to private facilities. Due to the increase in patients, mothers were discharged more frequently before 48 hours after delivery. Qualitative data suggest that although women prefer facility delivery, home deliveries still continue. Clients reported long waiting hours at facilities. KAP survey results among MNCH clients showed that about 90% of the postnatal mothers indicated their babies had been examined prior to discharge and only 30.6% returned to the facility within 2 weeks. Counselling on high impact interventions like keeping the baby warm and cord care are low (56% and 52% respectively) despite the trainings and CMEs done by AJ. There is also low knowledge (50%) among mothers about recognizing signs that the baby requires immediate health care. Source: extracts from AJ year1, year 2 and Year 3 annual work plan and progress reports 14 Table 4: Examples of Policies and Tools Developed by AJ 1. Guidelines a) Guidelines for the Operations of Hospital Management Boards, Health Centers and Dispensary Management Board 2. Frameworks a) Partnership Engagement Framework b) Medium Term Expenditure Framework (MTEF) Report, 2015-2019 3. Plans a) County AIDS/HIV Strategic Plan b) Drug resistant TB (DRTB) Emergency Response Plan c) Transition Plan d) Health Strategic Plan and Investment Plan 4. Policies a) County Health Policy 5. Strategies a) Quality Improvement Coaching Strategy 6. Tools a) Adolescent HIV Baseline Assessment Tool b) Human resources for health (HRH) Audit Tool Source: Afya Jijini project reports Conclusion of case study 2: AJ supported capacity building of HCWs (training, OJT, CME) on elements of essential newborn care, capacity building of CHVs to improve their ability to follow-up on home deliveries and referral. However, there is need to address barriers to uptake of postnatal services. There is an opportunity to improve awareness among mothers on importance of postnatal care at the community and facility level. Health worker structured mentorship sessions and close supervision could improve PNC counselling services. 4.1.3 Sub-purpose Three: Strengthened and Functional County Health Systems Country ownership: AJ supported Nairobi CHMT, ten sub-counties, four high volume facilities (HVF) and four low volume facilities (LVF) to strengthen the county health system. Table 4 presents some examples of policies, strategies and tools developed and disseminated by AJ. AJ supported annual work plan (AWP) development meetings at the county level and new AWP tools in sub-counties in 2016, 2017, and 2018. A CHMT discussant reported that they used tools and data for quarterly work planning with AJ support. Respondents specifically appreciated Leadership and Management trainings and suggested extending financial and management trainings to more HF in-charges. One panel discussant reported: “Leadership management training helped me in my program and I am able to do program based budgeting I didn’t have other management skills so I am a better manager since then.” CHMT respondents reported that they feel confident to develop strategic heath policies and work plans independent of any external support but support at the SCHMT has just started and continued capacity building support is still needed. Human resources for health: In terms of staffing and technical capacity building of the County Health System, AJ supported 44 HFs (public, faith-based organization and private). A total of 362 personnel (including 29 clinical officers, 26 eMTCT nurses, 10 lab technologists, 60 HTS counsellors, 21 records and information officers, 36 peer educators, 51 mentor mothers and 131 CHVs. The ET noted the following findings: • AJ engaged temporary nurses, phlebotomists and adherence counsellors helped to fast track VL sample collection. • Training on new ART guidelines for pregnant and breastfeeding mothers, and integration of ART services into MNCH/eMTCT Clinics for same day ART initiation for HIV+ pregnant women at some facilities. • Engagement of mentor mothers for peer education and psychosocial support at most HFs. • Engagement of sub-grantee (e.g. St John Hospital) to recruit and support community mentor mothers to increase retention and defaulter tracking. • HTS providers trained in adherence support and Facility Based Psychosocial Support Groups (PSSGs) created specifically for non-VL suppressed patients. • Sensitization of HCWs on TB-HIV integration. 15 • AJ facilitated different trainings to CHVs, mentor mothers, youth, and AGYW, something that participants greatly appreciated. However, no certificates have been awarded or given to the trainees, so there is no evidence that they attended such trainings Health products and technologies: AJ supported strengthening of laboratory network for 42 care and treatment centers for VL, CD4, EID and TB testing using GeneXpert. AJ also supported blood sample transport for EID to Kenya Medical Research Institute (KEMRI) Laboratories. While facility in￾charges appreciated improved HIV testing rates at TB clinics through AJ-supported adherence counsellors, they reported a backlog of sputum testing due to a faulty GeneXpert machine and delays in TB diagnosis due to unreliable results. Respondents also noted that medical waste disposal (transport from facility to incinerator) has broken down and the roles and responsibilities around waste disposal were unclear. There was an apparent assumption by SCHMT that AJ had promised to pick up and dispose of the waste and had not followed through. Respondents from AJ noted constraints due to limitations of budget allocations for waste management; they considered it a threat to quality health services that was outside of AJ’s control: “There was not much resolution to that (medical waste disposal) the county just keeps on asking as to resolve that problem and we say we do not have the budget allocation for it.” MNCH clients were generally satisfied with the nutritional products and services they receive from HFs including: nutritional counselling, plum peanuts, enriched porridge flour and taking of nutritional assessment measurements. However, few cases of clients being advised to go buy nutritional products due to stock-out at the facilities were reported. Nutrition activities do not focus on chronic malnutrition (stunting), a major contributor to child mortality in NCC. CCC clients appreciated the reduction in the frequency of visits by receiving drugs that last for longer periods, averaging 3 months, although in Kariobangi North health center, clients are given drugs that last for 6 months. Availability and improvements in VL testing frequency in most facilities was reported as an improvement. For example, in Kangemi health center, a discussant reported that “previously we would be sent for testing services outside the facility and we were expected to pay Kenya Shillings 500 for the service, but now we get the viral load test done at the facility at no cost.” None of the facilities sampled for CCC clients interviews had a resource or information center with books, pamphlets, leaflets or any other type of information, education or communication (IEC) materials that clients could read as they wait or take home. Out of 8 facilities, only Mathare North health center had a television that clients watched as they wait for services. Most CCC clients with opportunistic infections (OIs) buy drugs from chemists and only visit HFs when their condition deteriorates. This happens despite sensitization by HCWs of CCC clients (in Mathare North, Kariobangi North, Makadara and Kasarani HFs) to see a doctor first when they fall sick and not use any other drugs unless prescribed at the CCCs. A CCC client discussant in Kasarani health center reported that “we are forced to buy own medication when we fall sick because whenever we go to the facility for medication, we are referred to buy the drugs from chemists.” Performance monitoring and evaluation systems: AJ supported Quality Assurance and Quality Improvement (QA/QI) teams in ten sub-counties and ten HVFs. CHMT respondents reported that they interact with AJ in sub-county QA/QI activities and appreciate the support they receive for HF work improvement teams (WITs) which monitor various HF outcomes. All health facility in-charges interviewed confirmed that viral load uptake and suppression results are discussed at the monthly WITs meetings promoting them to monitor results. AJ carried out file audits and listed out all eligible patients to carry out VL on all patients according to new guidelines. At the county level, they conduct Quarterly Data Review Meetings and discuss data during support supervision visits. A weakness was AJ’s lack of planning and communicating with county leaving them without needed information. CHMT respondent noted that AJ has supported the County QI TWG, trained SCHMT QI focal persons and moved to HF QI teams. Not all HFs have QI teams, including private facilities. Kenya Quality Model 16 Table 5: Referral services interventions implemented by AJ for Health is used as a guideline for QI. Much of the QI work is currently “in progress” and not yet complete. There are about 103 WITs now in place and AJ has plans to develop more over time. AJ respondents noted that in the past QI was seen as something separate. WITs are now better at identifying gaps and understand that “it isn’t easy to tackle everything at one time. They have to go by the idea of prioritization and avoid becoming overwhelmed.” AJ promoted the use of QI Dashboard which the respondents found useful. WITs also have a client representative or a community representative on HF QITs. On the question of sustainability, facility in-charges think QI activities by WITs will continue after the project ends. Earlier AJ made payment to QI coaches from sub-county and visited HFs for supervision. These payments were stopped and the focus shifted to forming HF-based QI Champions to ensure sustainability. There could be a struggle to continue county and sub-county supervision which relied heavily on funding to hold meetings and to move from one point to the other for supervision visits. AJ reported they prioritize QI needs based on high-volume health facilities. AJ respondents are aware of existing gaps in the quality of care provision at these sites and prioritized resources for the highest impact in terms of volume. Above findings on QI were corroborated with a health workers (HWs) survey (n=38). Fifty percent of surveyed had received QI capacity building or coaching in the past two years from AJ. The ET developed a case-study to further explore and understand AJ’s contribution, strengths, weakness, opportunities and threats for sub-purpose three interventions in supporting a functional referral system. Case study 3: Functional referral system in AJ target areas Context: A functional referral system has five strategic areas including: i) Client movement: Transfer of client to an appropriate level of care where their health needs can be addressed in an efficient and cost￾effective way; ii) Services and expert movement: Transfer of health care service provider to provide specialized services that might not otherwise be available or not routinely offered at the lower level; iii)Specimen movement: Transfer of patient’s specimens for diagnostic purposes, which avoids the need to move the client; iv) Client parameter movement: Transfer of client information to appropriate levels of the health system for supportive diagnosis or management guidance, mostly involving innovative ICT services; and v) .Coordination and management of referral services: Effective management and coordination of the county referral networks to provide linkages across the different levels of the health system across the county. Referral services interventions implemented by AJ: During the last three years, AJ planned to implement or continues to implement a serous of interventions summarized in Table 5 that would contribute to the improvement of referral services in the county either directly or indirectly. Year one (2015-16) Year two (2016-17) Year three (2017-18) • Conduct county health and HSS stakeholder mapping to guide resource leveraging, linkages, referral mapping, and project HSS interventions • Support a stakeholder coordination mechanism at the county and sub￾county level • Document experiences from selected health facilities in working with community health workers/volunteers in • Implement and strengthen an electronic reporting system at target sites with capacity to upload data to national commodity platforms • Collaborate with various USG and non- USG National mechanisms to provide various services including supply chain system strengthening, increase commodity data visualization and utilization at county • Training of 50 healthcare workers on referral protocols • Printing of referral protocols and feedback mechanism • Support County Referral Hospitals (4) to develop a facility-based training plan • Implement the 5S approach 17 Year one (2015-16) Year two (2016-17) Year three (2017-18) key service areas to identify promising practices on CHV management • Support population of HR data into the HRIS (iris) system for all project sites through working closely with Capacity Bridge with an emphasis on use of HR data for management and decision making • Implement and strengthen a reporting system using EMR (IQ care) • Provide tools for waste management and facilitate linkages for waste disposal • Support lab linkages for specimen referrals • Support the county to establish a sample handling and processing hub within one centrally located laboratory • Strengthen defaulter tracking system (Use of SMS platform link to EMR) and sub-county level, support pharmaceutical and Lab commodity management electronic systems and on implementation of IQCare supply chain module, among others • Continue daily sample networking for viral load, CD4 and GeneXpert TB diagnostic tests. • Institutionalize monthly and quarterly DQAs at facility and sub￾county levels and Support the county quarterly data review meetings at the county level • Facility network improvement support • Strengthen and integrate community health information systems with other key health service delivery data • Support CHVs in providing monthly information to facility to improve the work flow and commodity storage at health facilities • Implement use of the IQCare supply chain module for dispensing and reports generation for decision￾making at high-volume health facilities • Collaborate with relevant stakeholders in supporting county monitor and evaluate priority health service delivery areas effectively • Strategic EMR scale-up to 29 project-supported C&T sites to full EMR functionality • Support community facility referrals by tracking effective uptake of RMNCH services using CHVs. Source: extracts from AJ year1, year 2 and Year 3 annual work plan and progress reports Analysis results: AJ supported the development of referral services and products including the county health sector referral strategy and investment plan (2017-2022) and the referral directory which was already in use. Lab linkages for specimen referrals improved turnaround time for getting lab results. The Activity increased the number of motorcycle riders from two to four, each in charge of the daily collection of specimens from at least 10 health facilities. In year three, the Activity initiated advocacy to establish model health facility sites or centers of excellence in each of the 10 sub-counties. Since year one, the AJ in collaboration with Health Policy Plus project has scaled up implementation of IQcare electronic medical records system (EMR) in 29 health facilities with IQcare infrastructure and antivirus. Most facilities are high volume and include 50% of all 16 facilities sampled during this mid-term review. The Activity continues to hire, train and use CHVs linked to health facilities as cough monitors, TB and ART defaulter tracing, and peer educators. According to CHVs interviewed during FGDs, “There are now fewer cases of children born who are HIV+ as we refer mothers to PMTCT,” and “We teach the mothers on the importance of early clinic attendance and as such the incidences of mothers giving birth at home has reduced.” The use of safe spaces for HIV/AIDS prevention and treatment services by adolescent girls and young women (AGYW) was hailed as a great idea by some of the AGYW interviewed. Further, the accompaniment of AGYW who test HIV+ to health facilities of their choice by mentors for care and treatment was reported to reduce stigma. The good referral work by CHVs was corroborated by KAP survey findings: 97% of CHVs (n=92) provide family planning information to anyone in their communities, 99% referred pregnant women in the community to HFs for regular antenatal care services, while 97% referred youths or high risk groups for HIV testing services. However, IQcare EMR has not been fully implemented even in the 29 facilities where it has been installed; it is operational only in 11 sites. All 29 sites have data, but experience challenges scaling up its use at the CCCs and HTS points due to low uptake by HCWs. Although CHVs support the community to facility linkages and referrals, it is not working effectively and CHAs feel left out. According to CHAs interviewed, “We use a lot of our time and resources in mobilization and supervision 18 of CHVs implementing AJ supported activities yet we do not get even allowances for airtime from the programme.” Health workers survey (n=38) results showed that 55% of HCWs reported they have a written protocol for referral and use it as a quick reference, 24% do referrals but do not have any established protocol, and 21% do not follow any protocol. Surveyed health worker highlighted logistics challenges for functional referral systems. About 50% reported that timely transportation is not available, 21% reported patients do not accept referral due to affordability, 19% reported other challenges, and 11% reported geographic inaccessibility. Conclusion of case study 3: Overall, when focusing on the five strategic referral areas, the percentage of high volume HFs that have functional referral systems could not be adequately assessed due to lack of complete data. Only the movement of MNCH clients and samples were used as proxy indicators. However, although positive contribution of AJ’s referral services work is already felt by stakeholders especially in specimen movement and community to facility linkages, more work is still necessary especially under EMR and the other strategic areas. 4.1.4 Cross-Cutting Elements Youth: AJ’s sub-partner NOPE implemented The Determined, Resilient, Empowered, AIDS-free, Mentored and Safe Women (DREAMS) package in the AJ supported zones of the Mukuru area of Embakasi South sub-county targeting AGYW in a total of 11 sites and being scaled-up to Westland. The package of activities included establishing and identifying safe spaces, mobilizing and recruiting AGYW for HIV prevention, and liaising with key stakeholders such as county and sub-county MOH, Ministry of Education, Ministry of Public Service, Youth and Gender Affairs, other partners in implementing youth HIV and AIDS activities, and reproductive health programs up to the ward level. These activities span across sub-purpose 1 and 2. DREAMS also supported three facilities to offer comprehensive sexual GBV services. NOPE respondents reported that there was a shortage of post-rape care kits since the project does not support the purchase of commodities and they needed to rely on government supplies in the HFs. Another challenges mentioned was completing the referral of AGYW and young boys who needed health services due to insufficient and overburdened community HWs and mentor mothers. The FGD among AGYW respondents noted that they have benefited from AJ’s DREAMS package support especially capacity building in asset building and entrepreneurship. However, parents with boys of the same age groups in slum areas with similar environment and challenges feel non-inclusion is discrimination by the DREAMS. From discussions with AGYW, it emerged that myths exits such as “if one uses FP before they give birth then they will never conceive.” Youth and AGYW regard knowledge and training on the use of condoms and PrEP as a great benefit. However, there were reported challenges in obtaining condoms. According to male volunteers in the 15-24 years age group who participate in HIV prevention programs in Ruaraka, health facilities are not offering youth-friendly services. Most youth prefer to get sexual and reproductive health commodities from their friends rather than going to health facilities. Girls reported that they are now empowered and able to use PrEP and distribute condoms. “We even carry our own female condoms” reported an AGYW discussant. AGYW discussants reported that although PrEP is a good idea, girls fear using it openly for fear of their husbands or boyfriends. Further, the belief that PrEP is meant for commercial sex workers impedes its utilization. One AGYW reported “If your husband discovers that you are using PrEP one may end up being chased away from her marriage.” A senior CHMT respondent reported that the County conducted PrEP assessment among adolescent girls and found messaging is not clear. Respondents mentioned that little has been done on behavior change communication (BCC) about PrEP use and continuing risky behavior. Further sensitization on use and availability of PrEP is a needed. According to AGYW interviewed in Embakasi, “Girls who do not belong to groups are not aware on the 19 existence of PrEP services.” Further, more sensitization is still necessary to discount myths and misconception on use of PrEP as a family planning method or on associating its use with prostitution. HTS for adolescent girls was reported as sub-optimal. According to a GOK respondent, “No impact of AJ DREAMS, yet work with other DREAMS teams in county and a lot is happening.” Other key populations: Following the discontinuation of sub partner University of Nairobi in February 2018, AJ is currently not implementing standardized HIV prevention packages specific to other key populations (KP) including sex workers, drug users and men who have sex with men (MSM). HCWs in KP hotspots such as Kangemi reported that KPs seek HIV services at CCC and are not segregated from the rest of the CCC Clients. However their prevention and care needs are quite distinct and CCC HCWs did not feel they had sufficient knowledge and skill to handle KPs at CCCs. Currently, AJ has a partnership with LINKAGES project which provides key population services in the catchment areas of the AJ-supported health facilities. Gender: AJ supported the establishment of VMMC services to the following facilities: Mathare North Health Center, Reuben Medical Center and Jericho Heath Center. Support included the provision of VMMC surgical equipment and engaging a VMMC surgical team, comprised of a clinical surgeon, a nurse, a hygienist, and a voluntary counseling and testing (VCT) counselor. VMMC at high volume facilities and outreaches in the community targeting pockets of non-circumcising communities were conducted. AJ also conducted mobilization through Imams and religious leaders. However, CHMT reported that VMMC services mainly capture school going young adolescents of 10–15 years and are not targeting enough sexually active young men over 15 years of age. The constraints are no demand creation or adjustment of service delivery to increase uptake of older males for VMMC. CHMT reported that AJ supported the quarterly gender TWG meetings and gender issues are not yet included as a QI in HFs. CHMT reported that they have scaled up GBV in four facilities. The respondent noted that gender is an “orphan baby with many mothers” and the primary challenge is to provide GBV survivors safe spaces or shelters. Shelters for adult GBV victims have not been built by NCC government, so women are returned to the environment where they were victimized. The CHMT respondent corroborated with what a NOPE respondent reported: “We don’t have ‘Dignity Kits’, AJ supports lab, RH, and HIV services for GBV survivors. Since 2016, early in the AJ project, they were doing GBV, but we have not been able to synchronize or work plans together. It needs to be strategic, but it isn’t.” CHMT reported that the team is working on gender mainstreaming related to things such as sexual harassment, HIV, staff recruitment and sensitization. These topics are included in the HRH policy which was developed in coordination with Capacity and AJ. Sustainability: AJ conducted capacity building in service delivery and program management, targeting different stakeholders including youths, AGYW, CHVs, mentor mothers, HCWs, SCHMT and CHMTs. Capacity building has so far been conducted through classroom training, support to CME, on-job mentorship and supportive supervision, technical assistance in the formation and/or strengthening of various TWGs and committees, and support for the creation of community support groups. The following are highlights of perceptions on sustainability and capacity building: 1) Health workers appreciated trainings support received for FP, nutrition, GBV, WASH, MNCH, RH, PMTCT, cancer screening, TB defaulter tracing, disability, HIV- adherence counselling, data collection and reporting; and 2) youth mentors have been trained on mentorship, entrepreneurship, life skills, evidence based interventions (e.g. healthy choices for a better future and SASA! -A community mobilization approach developed by Raising Voices for preventing violence against women and HIV). Consortium partners, facility in-charges, and CHMT and SCHMT respondents perceived the following as either already sustainable or likely to be sustainable even if AJ were to exit:  CHVs have acquired knowledge in sensitization, FP and MNCH and will continue with community work including home visits. 20  The lab management supervisory tool was revised and pretested with the help of AJ and will be launched and used in the county for a long time.  The revised AJ supported HRH guidelines will be used for a long time.  Support groups created among youth and AGYW through AJ’s facilitation work with strengthening of CBOs already in place such as St Johns, among others.  County commodity security committee strengthened by AJ is functional and meets quarterly with membership including USG and non-USG implementing partners through the county chairmanship. AJ has lobbies private sector players including Nairobi Hospital and Lancet Laboratories to join the committee for logistical and technical support.  AJ supported the formation of DREAMS advisory committee. It is already functional in Mukuru and meets quarterly. The committee is made up government representatives, local CBOs, children’s department representative, school head teacher representative and CBOs dealing with GBV.  Self-esteem among youth and AGYW has improved.  At the HF level, SOPs for service provision, lab and pharmacy developed through AJ’s support are in place and will continue to be used. Figure 5: Summary of AJ Activity’s Strengths, Weakness, Opportunities & Threats (SWOT) at Year 3 Strengths • Project design-intention to promote integration of services • Technical expertise of consortium prime and sub￾partners • Trainings, supervision and support to disseminate guidelines, SOPs, tools • Leadership and Management Training • Production of the Medium Term Expenditure Framework Budget 2015-2019 • Functional system of specimen transport – improve time to diagnosis • Setting up QI/MPDSR committee: Work Improvement Teams (WIT) • Health service delivery innovation models; psychosocial support • Coordination Committees, including Community advisory and Technical working groups Weaknesses • Poor integration of services (MNCH/FP into HIV) • Technical expertise of consortium – not utilized to a full extent • HF and community linkages are not functional in some areas • Reputation: ‘Afya madeni’, perception that AJ is riding on other’s achievements • Lack of communication with county and facility managers • Accountability – new intervention/plans started but no follow-up or continuity • High-turnover of staff employed by AJ at the HF level • Health outcomes of DREAMS hard to track; many DREAMS activities are not health and hard to report in relation to sub-purposes Opportunities • Trainings for technical skills and mismatch for supply of commodities/financial support –there is an opportunity for alignment of support to reduce mismatch • Costed HRH plan included in up-coming budget. If received, significant numbers of essential health personnel will be hired. • OR Studies/ impact assessment of innovative models and dissemination of results • Involve local stakeholders to advise modification of service delivery model/packages • Scaling-up key population and youth services for comprehensive coverage • Mother-baby pair follow-up • Referral linkages optimization Threats • Poor facility infrastructure for infection control/commodity storage • Stock-out of essential medicines/kits • Delays in disbursements from county to HFs • Staff turn-over, systemic HRH issue • Doctors and nurses strike in year 2 • ‘Afya madeni’- locals do not want to work for AJ • Harmonization of staff pay/benefits between different sources of personnel is not yet completed. • Cultural issues- prayers over seeking care in a HF, misconception for FP methods • Gender issues- misperceptions about PrEP use by girls, insecurity issues for women use of HFs • Population movement – loss to follow-up 21 Conclusions: Since program design does not consider the whole PMTCT Cascade, there is loss to follow up. AJ and Nairobi County MTCT Rates are not aligned. There is an opportunity to follow-up mother-baby pairs until 18 months to reduce loss to follow up for treatment. CHVs play a significant role in provision of TB services at TB clinics. However, training and supervision structure, under AJ, is unclear. Uptake of ANC services and skilled delivery has improved. However, access to long-term FP and modern FP methods availability is a challenge. Nutrition activities do not focus on chronic malnutrition (stunting). AJ has focused on strengthening county and sub-county governance (work planning), coordinating (TWGs) and monthly data reporting. Health products and technologies strengthening is working well, M&E and knowledge management support to county, sub-counties and CBOs needs structuring and improvement. There are gaps in the referral system. To some extent the Activity has enhanced institutional capacities to analyze, implement and evaluate development activities and to improve informed decision-making processes using tools. Weaknesses in training include no refresher or follow-up training despite high staff turnover with new staff being continuously added. There is still a sustainability issue regarding uptake of tools, training and QI methodology which rely heavily on continuous updating and technical expertise. However, the willingness and ability of individuals trained by AJ to pass along these skills and knowledge to their colleagues is vested with their institutions. Gender, youth, and key population activities are limited in terms of coverage and need to scale-up to improve access and utilization of services. Tools and methods are not fully assimilated within the institutions. Health outcomes of DREAMS are yet to be tracked; many DREAMS activities are not health-related and hard to report in relation to sub-purposes results. VMMC does not adequately demand creation for males over 15 years of age. 4.2 EVALUATION QUESTION 2 What coordination and collaboration mechanisms/structures has the Activity established and/or strengthened among key stakeholders (Service delivery implementing partners, relevant GOK ministries including Social Services)? How effective have these coordination/collaboration structures been in achieving program objectives? Coordination with GOK: CHMT reported that AJ supported the formation and/or strengthening of various committees or TWGs at the county and sub-county levels. To form this coordination structure, AJ advocated or brought in other implementing partner stakeholders to collaborate. The coordination meetings resulted in joint revision and development of policy and tools (e.g. collaborative development of HRH manual and guidelines through the leadership of CHMT). AJ worked closely with HFs, SCHMT and CHMT in various areas including trainings, supervision, facilitating sample networking, and commodity forecasting and quantification. AJ supported the county to hold HRH audit review dissemination meetings; this helped provide a platform for the county, sub-county and facility HMT’s to discuss the findings and identify areas that they could strengthen to improve their HRH capacity. AJ and IntraHealth (the lead) helped with the HRH policy. CHMT reported that the policy is yet to be disseminated to county government and other partners. According to a CHMT respondent, “We need to see if we can take it to the next level.” According to a CHMT respondent, “It is very easy to meet the AJ team, they are proactive and accessible especially if an activity had been planned a team easy to work with.” GOK respondents noted weaknesses in areas of work planning with S/CHMT. The program-based planning and budgeting training for the CHMT to be implemented jointly with HP+ had not taken place yet due to logistical and coordination challenges. There were perceptions that AJ initiates mini-projects but does not follow through. When these projects stop, there is no communication on ‘why and when’ with S/CHMT. AJ respondents reported that they do their work plan using CHMT work plan, but do not have a chance earlier in the cycle to collaborate on something that needs to go into both plans so they can be supported. 22 Coordination with private sector and civil society: Local provincial administration officials including village elders and chiefs who are members of DREAMS advisory board helped with identifying potential program beneficiaries. Identification of potential mentor mother candidates was done by sub￾county community focal points. Coordination with private sector is not evident much. Kenya Commercial Bank (KCB) helped with the disbursements of unconditional cash transfers and financial literacy training to DREAMS beneficiaries. Coordination within consortium: Review of project documents reveals that each had clearly defined roles and responsibilities according to sub-purposes. See roles are provided in Table 6. Table 6: AJ Consortium Partners, Roles and Responsibilities National Organization of Peer educators (NOPE) To oversee DREAMS activities and youth and gender cross-cutting activities in the Afya Jijini Project. University of Nairobi Centre for HIV Prevention and Research (UON-CHIVPR) To develop implementation approaches to boosting uptake and provision of quality health services for key populations, with a particular emphasis on HIV, FP/RH, and MNCH. Christian Health Association of Kenya (CHAK) To develop and implement an overarching strategy to assist health facilities provide the full continuum of HIV services, MNCH services, including family planning, targeted community engagement, and facility health system support. Mission for Essential Drugs Services (MEDS) To strengthen the Nairobi County’s health system in the following areas: supply chain management (forecasting, procurement, etc.), laboratory quality assurance and strengthening, and quality assurance strategies. AJ engaged consortium partners in joint work plan development on project start-up, joint review of consortium partner scope of work, and the recruitment of consortium partner key personnel. All AJ program staff from prime and consortium partners operated from IMA World Health offices. AJ had two levels of engagement with consortium partners. Technical staff level met monthly or as necessary to discuss activity implementation status, challenges and next steps. Quarterly leadership meetings discuss policy issues emanating from program implementation and administration, with IMA World Health as the secretary. The consortium is still in place except for the discontinuation of UON-CHIVPR in February 2018. Coordination with other donors: Coordination with other donor partners is limited. A county commodity security committee meets quarterly with membership including AJ, and other USG and non￾USG implementing partners through the chairmanship of the county. Coordination with other USG partners: AJ coordinated with other USG partners. This was confirmed by interviews with the CHMT, which reported the partnership works well and there have been no misunderstandings involving AJ and other organizations. The organizations that AJ was reported to coordinate and collaborate with in activity implementation include:  John Snow International (JSI) in the commodities security committee through the US CDC￾funded lab improvement project  University of Maryland on joint commodity quantification training and as member of the commodity security committee TWG member  Health Policy Plus (HPP+, implemented by Palladium) on technical support in IQCare EMR and pharmacy module for monthly reporting. HPP+ provides software, hardware set-up, back-up security and computer maintenance. AJ provided data assistants for data entry and management. 23 CHMT raised issues of sustainability of this support when activity ends and of the acceptability and use of EMR by clinicians for data keeping.  CHAI in supporting sub-county pharmacists and commodity management training including OJT and KEMSA to supply commodities to AJ supported public health facilities.  LINKAGES project provides care and treatment services to KPs in communities linked to facilities supported by AJ. According to respondents from LINKAGES, intensive collaboration with AJ was initiated at the beginning of year two and has progressed very well. While AJ’s support to drop-in centers has been overwhelming and functional in Nairobi County, LINKAGES support to Kiambu has experienced enormous challenges.  Health Commodities and services Management (HCSM) implemented by MSH. Although Nairobi was not one of the counties supported by HCSM, AJ reached out to them Most of USAID’s meetings with implementing partners are normally on “targets” discussions making some partners focus less on collaboration. Conclusions: Coordination mechanisms are in place, but need further strengthening to ensure sustainability. CHMT is taking a stronger role in partnership coordination of government and donor￾supported programs. AJ is working to strengthen management capacity at SCHMT level, but more needs to be done. Though a service delivery partner, AJ is supposed to implement all tasks (e.g. health financing, policy, governance, HRH) creating overlaps with roles supposed to be performed by national mechanisms thereby decreasing efficiency. Private-public partnership involvement will likely make continuation of various committees and TWGs sustainable. 4.3 EVALUATION QUESTION 3 What sustainable and innovative service delivery models is the activity currently implementing? What do the Ministry of Health and other relevant stakeholders think in terms of their long-term use? Findings AJ activity has been trying some innovative service delivery approaches. “We Men Care” model was introduced in 17 facilities to improve male involvement in MNCH after a pilot in four health facilities. Men have poor health seeking behavior so getting them to the facility was a challenge. Most men who did accompany their wives or partners to ANC were left out because of messaging and information was targeted only to pregnant women. AJ created stand-alone spaces within the facility where men who accompany women meet a male champion. This strategy differs from traditional male partner involvement strategies in that it provides separate, male-friendly services that are attractive to the client. The men are taken through topics including birth planning, nutrition, danger signs in pregnancy, HTS for men, body mass index for men, nutritional counseling, and referral within the HF and FP. When asked about “We Men Care” model CHMT respondents answered: “We are not feeling its results at our level.” CHMT recognizes the need for male involvement, new ways to reach men in the ANC clinic and appropriate messaging. One of the CBOs, St John’s at Kariobangi North HC, uses the term “mentor fathers.” These mentors are paid a monthly allowance of Kenya Shillings (Kshs) 8,000. Project documents show a progressive increase in the number of men attending “We Men Care” dedicated spaces and being refereed to appropriate services. However, health outcomes achieved are unclear from project reports. The “men priority clients” is a pilot model that takes 100 men partners of priority clients through a two-day dialogue on norms change training to improve retention of mothers and partners. Baseline training for the first batch has been done. DREAMS uses safe spaces as sites where AGYW receive holistic packages of services including HTS, GBV screening, post-test and trauma counselling, PrEP and referrals in an atmosphere free of stigma and other challenges. By May 2018, AJ had 11 safe spaces sites in Mukuru but was expanding to Westlands. 24 As one FGD AGYW discussant reported “We fear the community but don’t fear ourselves at the safe spaces.” These safe spaces are rented by AJ and therefore may not continue after funding ends. As presented in section 4.1.4, the CHMT considers unavailability of GBV survivors’ safe spaces or shelters as a major challenge. Shelters for adult GBV victims have not been built by NCC government, and currently looking for support from non-government agencies. CHV household visits for follow-up and provision of condoms were considered innovative especially by youth and AGYW who fear getting such commodities from public places including HFs due to stigma. CHV activities are directly paid by the Activity. MCH7/7 is a model where the maternal and child health clinic remains open all days of the week including Saturdays and Sundays. The model was created with an aim to decongest health facilities during weekdays thereby reducing patient waiting time. AJ paid staff reimbursements for the weekend work. The clients reported MCH7/7 is a good initiative they would like to see continue. Conclusions: A few program innovations are being piloted and there is a need for further assessments and consultations to confirm efficacy before scale up. AJ’s approach of working with relevant local CBOs, CHVs and private institutions is appropriate and necessary to ensure sustainability of activities and services beyond life of the Activity. Technical assistance from AJ is appreciated by all levels of stakeholders and the demand from beneficiaries is there. It remains unclear exactly how the activity will address sustainability of innovative models. Unless these service delivery models are owned, adapted or mainstreamed by the local government institutional sustainability is uncertain, especially if the AJ stops supporting institutions or discontinues staff reimbursements. 4.4 EVALUATION QUESTION 4 What are the key programmatic and management recommendations that the mission could consider for mid-course changes to the current program design? The findings presented below are the key programmatic and management challenges observed during the AJ program implementation which require mid-course correction Community outreaches: AJ initiated weekend outreaches at some HFs, an activity that community members appreciated, especially mothers who work weekdays. The clinics also handle weekend emergencies common in the slum areas. Other services including VMMC and HTS were also offered. However, outreaches were discontinued after six months in year one. There is a lack of identification of CHVs supporting AJ interventions at the community level and in schools. CHV respondents reported hostility including being sent out of schools or even arrested. CHVs lack basic tools including bags, work in hostile and muddy environments especially during home visits, and cover long distances while exposed to hostile weather. Human resources and Staff professional training and mentorship: HCWs seconded to HFs have improved quality of services as observed by CHV and client respondents. However, in 7 of 8 HFs respondents reported long CCC and MNCH patient waiting times, a limited number of patients being seen by HCWs per day, and long turnaround times for results and referrals to other facilities for basic services due to lack of specialists. AJ sub-grantee budgets are limited so they cannot hire enough staff despite the workload expected to be delivered. CHMT reported the current HIV services mentorship model is not responsive to the technical needs of CCCs: “we are not feeling their technical support on the ground … there is no one-on-one mentorship of our MOH CCC HCWs.” At the validation meeting, the CHMT proposed that AJ supports setting up central technical support to link various CCC staff to HIV experts. USAID contract mechanism: AJ reported that the activity cannot purchase items or build infrastructure under its contract. Stock-out of basic drugs and supplies for laboratory tests was reported 25 in 6 of 8 AJ supported facilities visited by the ET. There is a shortage of post-rape care kits and dignity kits for rape survivors (last stock received from the county in Feb 2017). Hospitals only offer overnight shelters to such adults; although 17 rescue centers were planned for Nairobi, none has been constructed. AJ introduced use of Aquatab for drinking water treatment but the support was discontinued after a community demand was created. One of the panel discussant stated “Irrespective of how much I train if I don’t have weighing scales for nutrition am not going to do a good job with reducing stunting, underweight and chronic malnutrition.” As part of HSS, some health facilities placed requests to AJ through the CHMT for support with minor repairs and renovations. Requests were made in year one but had not been acted upon two years later. Data deporting tools: Inadequate or total lack of data collection and reporting tools/ cards, especially community to facility reporting tools makes it challenging to report on nutrition and other indicators. According to one CHA, “AJ program expect CHVs to share detailed information on households and AGYW in their areas of jurisdictions without tools to collect or summarize the data/information.” Some facilities ask CHVs to photocopy tools using their own funds. Tools allocated by the national government to Nairobi County facilities are inadequate, and most are for HIV data collection. Moreover, AJ does not have the mandate or budget to print data collection tools for use at health facilities besides occasional minor photocopying. Feedback from consortium members and other USG partners: AJ had a difficult year one and many senior management staff were replaced. AJ has a USAID contract mechanism and not a cooperative agreement, reducing their flexibility in program design and implementation. IMA World Health faced difficulty due to non-familiarity with restrictions and approval processes required under USAID contract mechanisms. Currently, sub-grantees or consortium partners do not meet with USAID representatives directly but through the prime partner. Activities including training the CHMT in program-based planning and budgeting by HPP+ together with AJ has not taken place due to lack of clarification on the budget for HPP+ staff level of effort (LOE). AJ activity has very little or no branding unlike other USAID funded projects and programs. Conclusions: There were internal weaknesses as well as external threats which resulted in programmatic and management challenges for AJ during the program implementation. Going forward, these challenges need to be tackled for the remainder of program duration. 5 RECOMMENDATIONS Progress design and implementation  AJ should strengthen integration of MNCH and FP services into HIV services through staff and beneficiary sensitization, tools, and trainings.  AJ should develop a strategy to address barriers to uptake of services in public HFs (i.e. poor staff attitude, long wait time, non-availability of essential drugs, poor infrastructure, lack of IEC material, and lack of referral).  Within the community components (i.e. DREAMS, youth, nutrition, WASH, MNCH, FP), AJ should contract short term technical assistance to update IEC materials for BCC within the program.  AJ should refocus efforts and resources for underperforming interventions (i.e. linkages from HIV testing to care/treatment services, loss to follow-up, defaulter tracing, inter-facility and facility-community linkages for referral, infection, prevention and control and medical waste management). AJ could create centers of excellence in each of the 10 sub-counties to model health facilities with improved clinical management, referral systems, sample collections, 26 community linkages and supply chain. AJ should shift from target (quantity) focus to quality focus to improve quality of care in supported HFs.  AJ should develop a cohesive MNCH/FP/nutrition strategy using high-impact, evidence-based interventions that correspond with Kenyan National Policies (e.g. IMCI, Thousand Days Strategy) and supports NCC adaptation of the Community Health Strategy.  There is insufficient time for AJ to develop a comprehensive community component, but AJ should support the SCHMT in developing community-facility linkages.  AJ should work with S/CHMT to design a comprehensive sustainable mentorship program that is responsive to the needs at CCCs.  AJ M&E team should refocus on capacity building of county, sub-county, HF and CBOs for data management and data quality.  USAID should strongly consider including BCC components in the urban slum context as a component in the design of future programs. Coordination and collaboration  AJ should clarify types of support provided at each HF type, updating the list frequently.  AJ and the CHMT should jointly meet with SCHMT and relevant HFs to orient (or reorient) to the roles, responsibilities and expectations for project assistance. This process should be updated frequently as the activity phases out.  Consortium partners should gain a better understanding of implementation models from their fellow consortium partners through joint cross visits.  USAID should come up with measurable and reportable “collaboration” indicators.  Referrals in health facilities should be strengthened so that means of transport for mothers/patients referred to other facilities are available.  To improve on project/activity coordination, a communication framework should be designed taking into consideration the various stakeholders on the ground, their concerns and sensitivity to the project/activity in relation to government priorities and their expected involvement in implementation.  To minimize risk of duplication, parallelism and double funding, USAID should hold coordination meetings with other development agencies to agree on contribution level.  USAID should hold consultative meetings and agree on the scope of each intervention and each office’s level of involvement/contribution/reporting requirements. In addition, the project/activity should start with a participatory needs assessment of each sub-county ministry, and adopt an agreed upon systematic assessment schedule. Sustainable and innovative models  Conduct operations research or impact assessments and synthesize key results and lessons learned into briefers for national and county stakeholders.  AJ and CHMT should jointly prepare and implement sustainability plans for models that are most likely to be sustained after AJ project funding ends.  In design and implementation of future innovations, USAID should consider a phased￾implementation approach that factors in transition timelines, so that towards the end of each innovative model implementation, the local stakeholders are driving the process with USAID Implementing partner taking an advisory role on technical matters. Program management  Program progress reporting should focus on what AJ implemented in compliance with USAID reporting requirements including the project M&E indicators which are outcome-based rather than output-based. 27  AJ team should be more integrated to ensure optimal resource use and sharing including data review and use for decision making.  AJ management should clarify with USAID what is allowable under the contract (or potential modifications, including budget implications) until end of the project.  AJ team should focus on building strong internal finance operations and management systems, branding, and partner communication.  Starting with the year four work plan, a clear sustainability and exit strategy should be agreed upon between AJ, CHMT and USAID. 28 REFERENCES Concern Worldwide. (May 2017). Nutrition survey conducted in the slums of Nairobi County. Kenya: Concern Worldwide. Government of Kenya, Ministry of Health (GOK MOH). (2016). Kenya framework for elimination of mother-to-child transmission of HIV and syphilis, 2016-2021. Kenya: Ministry of Health. Kenya AIDS Indicator Survey (KAIS). (2012). Kenya AIDS indicator survey 2012. Kenya: KAIS. Kenya Demographic Health Survey (KDHS). (2014). Kenya Demographic Health Survey, 2014. Kenya: KDHS. Kenya National Bureau of Statistics (KNBS). (2009). Kenya national housing and population Census, 2009. Kenya: KNBS. Ministry of Water and Irrigation (MWI). (2007). Citizens’ report card of urban water, sanitation and solid waste services in Kenya. Kenya: MWI. Nairobi City County (NCCa). (2017). Nairobi city county health sector strategic and investment plan – 2013/2014 – 2018/2019 (Revised 2017). Kenya: NCC. UN-Habitat. (2011). UN Habitat annual report, 2010. Kenya: UN-HABITAT. UNAIDS. (2017). Start free, stay free, AIDS free: 2017 progress report. Retrieved July 27, 2018 from http://www.unaids.org/sites/default/files/media_asset/JC2923_SFSFAF_2017progressreport_en.pdf World Bank. (2006). Kenya - Inside informality: Poverty, jobs, housing, and services in Nairobi’s slums. Washington, DC: World Bank. 29 6 ANNEXES 6.1 ANNEX 1: SCOPE OF WORK INTRODUCTION Nairobi, capital of Kenya, is the main commercial center of the country with a well-developed infrastructure, including modern financial and communications systems. It hosts the country’s largest industrial center which accounts for almost 20% of the gross domestic product (GDP). Nairobi County’s population of 3,138,369 in 2009 is projected to increase to 4,390,158 in 2018. This population is distributed in 17 administrative sub-counties, which have been merged to 10 health sub￾counties for ease of health administration.1 It borders Machakos County, Kiambu County and Kajiado County and covers an area of 695 square kilometers. The population distribution shows the age group of the under 15 accounting for 40.7% of the total population, the young adult age groups of 15–29 years accounting for 38.6%, the reproductive age groups of 15– 49 years accounting for 40.7% with figures of 22% and 18% for males and females respectively. The proportion of those over 60 years is 2% of the population.2 Many citizens work in the city during the day, but reside in the neighboring counties such as Machakos, Kiambu and Kajiado. This transit day population is estimated at about one million people. The four most densely populated sub-counties are Mathare, Embakasi North, Ruaraka and Kamukunji with population densities of over 20,000 people per square kilometer. The least densely populated sub￾counties are Westlands, Langata, Kasarani and Embakasi East. Furthermore, it is estimated that about 58% of Nairobi’s population live in slums or slum-like conditions (UN Habitat 2010) and that there are about 55,000 refugees and asylum seekers living in the city (UNHCR 2013). Nairobi and other cities in Kenya have been unable to keep up the pace of planning and developing its infrastructure to meet the demands of a growing population with the population (4 to 5 percent average growth per year since the 1990s). Rural migration and the natural growth of the urban population have been largely absorbed in the fast growing urban informal and unplanned settlements.3 Migrants and long-standing residents aspire for improved livelihoods, but instead are often trapped in congested informal settlements, where housing costs may be lower but living conditions are precarious. Additionally, rising costs in Nairobi’s formal areas result in some residents shifting to informal settlements to make ends meet. Already, an estimated 60% of Nairobi’s inhabitants live in these informal settlements, which constitute only five percent of the city’s residential land. High population densities and sub-standard living conditions in these congested areas make accommodating the needs of the growing settlements a daunting task.4 The informal settlements are varied in size, population, ethnic, social and cultural character, but share one common denominator – poverty. Average monthly income in Nairobi informal settlements is estimated to be KSH 3,000 (approximately USD$30), leaving little room for savings or investments.5 1 Nairobi City County Health Sector Strategic and Investment Plan – 20 13/20 14 – 20 18/20 19 (Revised 20 17) 2 KNBS, 20 0 9 Kenya National Housing and Population Census 3 Strategic Guidelines for Improving Water and Sanitation Services in Nairobi’s Informal Settlements, 20 0 9. 4 Strategic Guidelines for Improving Water and Sanitation Services in Nairobi’s Informal Settlements, 20 0 9. 30 HIV/TB/MNCH/ FP/WASH/Nutrition HIV/TB statistics: TB incidence per 100,000 persons – 514 HIV prevalence rate (2012) – 3.3% New HIV infections – 13,510 MNCH statistics: Neonatal Mortality Rate – 39 deaths per 1,000 live births Infant Mortality rate (IMR) – 39 deaths per 1,000 live births Under 5 Mortality Rate – 52 per 1,000 live births Fully Immunized population < 1 year – 97% Median age at first marriage for women - 22.1 years Median age at first birth among women - 22.2 years. FP/Postnatal Care statistics: Total Fertility Rate (TFR) – 2.7 Contraception prevalence – 38.7% Birth delivered at health facilities – 78.9% Postnatal care from SBA – 71% Family size for women = 3.0 children Unmet FP need = 11% 6.1.1.1 HEALTH SERVICE DELIVERY The Nairobi City County Health Sector Strategic and Investment Plan (NCCHSSIP) 2013/2014 – 2018/2019 highlights Nairobi County’s increasing burden of non-communicable and communicable diseases including HIV/AIDS and tuberculosis (TB). Hence, there is need for concerted efforts to improve the health of the city residents.6 NCCHSSIP 2013/2014 – 2018/2019 focuses on six health sector objectives: 1) elimination of communicable conditions, 2) halting and reversing the rising burden of non-communicable conditions, 3) reducing the burden of violence and injuries, 4) provision of essential health care, 5) minimizing exposure to health risk factors, and 6) strengthening collaboration with health-related sectors. The seventh objective that is specific to Nairobi County is: 7) improving emergency, referral and rehabilitative services. The County government is determined to improve access to quality health services with a special focus on the urban slums through targeted interventions. HIV prevalence was consistently higher among women than men in all counties. Significant changes in HIV prevalence were observed among women in Nairobi, declining from 11.9% in 2003 to 5.9% in 2012. Since 2003, HIV prevalence among men in Nairobi County declined significantly from 7.8% in 2003 to 3.3% in 2012.7 The major obstacles to child survival in the developing world include malnutrition and the risks associated with low birth weight and high fertility. A serious problem exists in the informal settlements in Nairobi, as a result of children dying from common illness and infections, parasitic diseases, infections that are attributed to poor nutrition. In Nairobi slums, mortality rates for children under five years old are double the city’s average and greater than those in rural areas, according to UN Habitat, the U.N. agency for human settlements. Studies have indicated that children living in slums are more likely to die from pneumonia, diarrhea, malaria, measles or HIV/AIDS than those from wealthier parts of the same city and rural areas. Sanitation is a human right and a key component of primary prevention to ensure better health (WHO 2014). According to the WHO (2014) report, in the African Region 45% of the population uses either Box 2: A Snap-Shot of Health Indicators Statistics in Nairobi County Source: KDHS, 2014; Kenya Tuberculosis Survey, 2016 31 shared or unimproved facilities, and a quarter practices open defecation. The majority of those practicing open defecation live in rural areas or urban informal settlements.5 Nairobi’s informal settlements are mostly unplanned, resulting in inadequate infrastructure (roads, water, sanitation, sewerage, drainage, and electricity), temporary and haphazard building solutions, poor housing, high occupation density, and very low levels of public services. Clean water, adequate sanitation, and hygiene are visibly substandard. This was confirmed in a consumer survey which illustrated informal settlement residents’ general discontent with services.6 In Nairobi’s informal settlements, an estimated 24% of residents have access to household toilet facilities: a ventilated improved pit (VIP) latrine, an ordinary pit latrine, or a flush toilet. Sixty-eight percent rely on shared facilities, while 6% have no access to toilets.7 The Kenya Demographic and Health Survey (KDHS) 2014 showed that children in urban areas (31%) are more likely to be fed appropriately than their rural counterparts (17%) and that the likelihood of children being fed according to the recommended UNICEF Infant and Young Child Feeding (IYCF) guidelines increases with age. Adherence to IYCF feeding practices was highest in Nairobi (39%).8 6.1.1.2 COUNTY HEALTH SYSTEM The most significant feature of the Constitution of Kenya 2010 is the introduction of a devolved system of government, which provides for one (1) national government and forty-seven (47) county governments. The governments at the national and county levels are “distinct and interdependent” and are expected to undertake their relations through “consultation and cooperation.”9 Kenya Health Policy 2014-2030 takes cognizance of the specific functions assigned to the two levels of governments, with the county governments responsible for county health services, including county health facilities and pharmacies; ambulance services; promotion of primary healthcare; licensing and control of undertakings that sell food to the public; cemeteries, funeral parlors and crematoria; and refuse removal, refuse dumps and solid waste disposal.10 There is evidence that health systems that can deliver services equitably and efficiently are critical to achieve improved health status.11 Thus, many global initiatives now incorporate attention to health systems strengthening in the support they provide to countries. However, increased attention to health systems strengthening requires a sound monitoring strategy. Accordingly, in 2007, WHO proposed a framework describing health systems in terms of six core components or building blocks: i) service 6 Citizens’ Report Card of urban water, sanitation and solid waste services in Kenya, MWI, 20 0 7 7 Inside Informality: Poverty, Jobs, Housing, and Services in Nairobi’s Informal Settlements. May 20 0 6. S. Gulyani, Debrata Talukdar, Cuz Potter. 8 KDHS 20 14 9 Article 6 of the Constitution of Kenya 20 10 10 Kenya Health Policy 2014–2030 11 WHO 2010: Monitoring the building blocks of health systems: a handbook of indicators and their measurement strategies 32 delivery, ii) health workforce, iii) health information systems, iv) access to essential medicines, v) financing, and vi) leadership and governance.12 Development aid should be tailored to supporting the county capacity to ensure effectiveness and to avoid unintended distortions. Thus, supporting and capacity building of county health management is a central element of preparing and implementing any kind of support.13 6.1.1.3 EVALUATION OBJECTIVES The objectives of the Afya Jijini mid-term evaluation are to: 1. Review implementation approaches for each output and determine the extent to which the program is on-track in achieving its intended outcomes by finding out: • What is working well? What does the program need to do more of? • What is not working well? What does program need to do less of? 2. Identify what the program needs to change to achieve expected results and outcomes by the end of the program. 3. Provide recommendations based on the key findings and conclusions about any required changes that will enable the program to improve any weak areas identified and to sustain areas of strength. The contractor must undertake the following tasks: • Task I: Effectiveness: Evaluate Afya Jijini program design, performance, contract objectives and implementation to date holistically for each funding pocket and program objectives. • Task II: Efficiency: Evaluate the mix of activities and funding allocated to Afya Jijini program. • Task III: Cross-cutting elements: Evaluate Afya Jijini’s course of implementation in regards to: o Youth, Young Women, and Other Marginalized Groups o Key Populations o Gender o Sustainability and Capacity Building o Coordination, Alignment and Collaboration • Task IV: Factors of success: Describe the program’s key success factors in terms of field level implementation, program systems, and program management. What are the promising lessons and approaches for expanding the integrated health service delivery to additional sites? • Task V: Obstacles: Describe the program’s key obstacles in terms of field-level implementation, program systems, and program management. • Task VI: Sustainability: Evaluate Afya Jijini’s contributions to building sustainable health service delivery in Kenya and MOH ownership of health programs. • Task VII: Beneficiary relations: Evaluate the program’s success at delivering quality health services to beneficiaries. 12 Everybody’s business: Strengthening health systems to improve health outcomes. WHO’s framework for action. Geneva: World Health Organization; 2007 (http://www.who.int/healthsystems/strategy/everybodys_business.pdf). 13 European Commission. Tools and methods series: Reference document No 1. Institutional Assessment and Capacity Development, Why, what and how? September 2005 33 • Task VIII: Linkages, coordination and leveraging: Evaluate the program’s success at establishing linkages between and among stakeholders. • Task IX: Performance monitoring: Evaluate Afya Jijini performance monitoring systems. • Task X: Management systems within the program: Evaluate the factors that have contributed to or inhibited the functionality and performance of the Afya Jijini program partnership, which includes four sub-partners and the Ministry of Health. Evaluate the performance and effectiveness of the prime, sub-partners and sub-awardees in the Afya Jijini program coalition and their contribution towards realization of the program objectives. • Task XI: USAID’s program management: Evaluate USAID's management of the Afya Jijini program. • Task XII: Evaluate relevance of the program in the prevailing/current health systems context (devolved system of governance) For each of the above tasks, the evaluation team must assess progress and provide management recommendations informed by data and analysis resulting from the proposed evaluation activities. These recommendations will provide the basis for potential adjustments to program activities or management, as needed, and frame any higher-level issues for USAID/KEA and/or relevant ministries in Kenya. TECHNICAL APPROACH In order to achieve the evaluation objectives described above, the Contractor must use a multiple cross￾sectional study design for conducting a performance mid-term evaluation using a mixed method approach. The study design will obtain: i) qualitative data through document review, key informant interviews (KII), panel discussions, and focus group discussions (FGD); ii) quantitative data through exit surveys (with health facility beneficiaries – MNCH and CCC clients), and a knowledge, attitude and practice survey with youths (15-24 years) and community health workers; iii) routine data from the District Health Information System (DHIS2); and iv) data from an institutional capacity assessment of county health management teams (CHMT), health facilities and community based organizations (CBO). The proposed evaluation design along with data collection methods and the analytical approaches must be used to answer the following USAID evaluation questions described below. 6.1.1.4 EVALUATION QUESTIONS • What progress has the Activity made towards the achievement of the program’s midterm mandates/targets/outcomes/milestones? • With regard to the activity design, what are the main constraints and/or weaknesses, opportunities and threats that need further attention? • What coordination and collaboration mechanisms/structures has the Activity established and/or strengthened among key stakeholders (Service delivery implementing partners, relevant GOK ministries including Social Services)? • How effective have these coordination/collaboration structures been in achieving program objectives? • What sustainable and innovative service delivery models is the activity currently implementing? • What does the Ministry of Health and other relevant stakeholders think in terms of their long￾term use? • What are the key programmatic and management recommendations that the mission could consider for mid-course changes to the current program design? In addition to the above, USAID Evaluation Questions Matrix provides a detailed list of questions/sub￾questions that collectively address the objectives of this midterm evaluation, while Performance Target Indicators provide a list of outcome indicators. 34 6.1.1.5 GEOGRAPHICAL SCOPE OF THIS MIDTERM EVALUATION The Afya Jijini midterm evaluation must be restricted to Nairobi,14 Afya Jijini’s focus county/sub counties.15 For administrative and design purposes, the contractor shall stratify the Nairobi county into four clusters, as described below: Figure 6: Map of Nairobi County Cluster I will include Westlands, Dagoretti North, Dagoretti South, Langata and Kibra** sub-counties; Cluster II will cover Roysambu, Kasarani and Ruaraka subcounties; Cluster III will include Embakasi North, Embakasi South, Embakasi East, Embakasi West and Embakasi Central; and Cluster IV will cover Starehe, Kamukunji, Mathare and Makadara sub counties. ** Service coverage did not include Kibra sub county (ref: USAID/KEA, Afya Jijini Contract). 6.1.1.6 DESCRIPTION OF EVALUATION METHODS DESK REVIEW: The contractor must review selected documents on technical and managerial aspects related to each Activity’s output. These documents shall include but are not limited to: ESPS technical proposal/Scope of Work; Afya Jijini Contract; Activity’s Monitoring & Evaluation Plans; Annual and 14 R Core Team (20 16). R: A language and environment for statistical computing. R Foundation for Statistical Computing, Vienna, Austria. URL https://www.R-project.org/. – R “ggplot2” - library GPL 3 License Copyright (C) 20 0 7 Free Software Foundation, Inc. http://fsf.org/. Kenya county shapefiles source: http://mapeastafrica.com/countries/east-africa-shapefiles/kenya-shapefiles/ - shapefile projections in World Geodetic System 1984 (WGS84) Coordinate Reference System; 15 Statement of Work – Request for Task Order Proposal (October 24, 20 17) 35 Quarterly Reports, among others. A comprehensive list of documents can be found in Project Documents. Additional documents will be agreed upon between USAID, Afya Jijini and ESPS. Objective: Information obtained during the desk review will give the contractor Evaluation Team familiarity with the Activity’s scope and interventions prior to fieldwork, as well as a basis to design further probing questions that directly/indirectly answer the evaluation question. KEY INFORMANT INTERVIEWS (KII)/PANEL DISCUSSIONS: A cross-sectional design using qualitative methods must be used to conduct KII and panel discussions with consortium members and various stakeholders at the county and national level. INSTITUTIONAL CAPACITY ASSESSMENT (ICA): This evaluation must include a cross sectional institutional capacity assessment (ICA) of county health management teams (CHMT), facility management teams and community-based organizations. The objective of these capacity assessments will be to assess the capacity of the a) CHMT, health facilities and community health units’ ability to sustain the Activity’s achievements beyond the life of the project; b) CHMT’s ability to organize communication strategies; c) CHMT’s ability to plan and coordinate implementation of effective multi-sectoral partnership for health; d) CHMT’s and community units’ ability to manage social protection initiatives; e) health facilities’ ability to run and sustain quality improvement systems; and f) Community units to mobilize communities in the targeted counties. This ICA shall provide insights into the sustainability of the program (see section 2.5), as implied by intermediate result (IR) 2.2.2 - Improved county-level accountability, institutional capacity, leadership and management of health and education service delivery of the Country Development Cooperation Strategy’s (CDCS). FOCUS GROUP DISCUSSIONS (FGD): The contractor Evaluation Team must conduct FGDs with the following four target groups: Maternal Neonatal and Child Health (MNCH) clients; Comprehensive Care Clinics (CCC) clients, and youths (15-24 years) – beneficiaries of HIV prevention services offered under the Determined, Resilient, Empowered, AIDS-Free, Mentored, and Safe women (DREAMS) project; and community health volunteers (CHV). The FGD sessions will provide insights critical for grounding evidence emerging from the quantitative and qualitative data. The objectives of the FGDs are summarized in Objectives of FGDs. KNOWLEDGE, ATTITUDE AND PRACTISE SURVEY: This evaluation must also include a cross-sectional survey involving four target groups – MNCH, CCC beneficiaries; youth (15-24 years), and CHV. These surveys will be used to capture data on beneficiaries’ relations, and knowledge, attitude, behavior and practice, relevant to the Activity’s performance indicators. 6.1.1.7 SAMPLING STRATEGY To ensure valid representation from multiple stakeholders when conducting the Afya Jijini midterm evaluation, a systematic multi-stage stratified sampling approach must be used in the selection of health facilities and participants for this evaluation. SELECTION OF INSTITUTIONS/ORGANIZATIONS, SELECTION OF SUB-COUNTIES First, the County of Nairobi must be stratified into four regions (see section 2.2 Geographical scope of this midterm evaluation). Note: Inclusion of sub-counties within a region is based on their proximity to each other, as shown below: Table 7: Selected Sub-Counties 36 Cluster 1 Cluster 2 Cluster 3 Cluster 4 Westlands Roysambu Embakasi South Makadara Dagoretti North Kasarani Embakasi North Kamukunji Dagoretti South Ruaraka Embakasi Central Starehe Langata Embakasi East Mathare Kibra** Embakasi West ** Service coverage did not include Kibra sub county (ref: USAID/KEA, Afya Jijini Contract). Due to cost considerations, and the geographical spread of the health facilities, ESPS must select 50%, i.e. eight sub-counties (on rounding up to the next whole number) of the 16 sub-counties supported by the Activity. These are – Westlands, Langata, Kasarani, Ruaraka, Embakasi South, Embakasi North, Makadara, and Mathare sub counties. SELECTION OF HEALTH FACILITIES AND LINKED COMMUNITY UNITS Health Facilities: Within the selected sub-counties, health facilities must be stratified by level of care (hospitals, health centers and dispensaries), ownership (private or public) and location (informal or formal settlement). Community Units: All CUs that are linked to the sampled health facilities must be selected for this evaluation. Inclusion Criteria: All county, sub-county hospitals will be purposively included in the sample; health centers (ANC/PMTCT client load > 500/annum), and dispensaries (ANC/PMTCT client load >200/annum); the sample will include between 15 – 25% of the high-volume facilities. The distribution of sampled health facilities (supported by the Activity/sampled for the midterm evaluation) can be found in Sampling Frame for the Health Facilities. SELECTION OF COMMUNITY-BASED ORGANIZATIONS Within the selected sub-counties, a maximum of eight CBOs supporting HIV prevention activities for youth and young women must be purposively selected for this evaluation to ensure effective representation of the Activity’s geographic coverage. The selection shall be based on following inclusion criteria. Inclusion criteria: 1) total time of support in years, 2) total number of youths that it supports, 3) geographical location (proximity to the sampled health facilities and/or community unit). SELECTION OF COLLABORATING INSTITUTIONS Collaborating/partner institutions must be purposively selected for this evaluation. These shall include, but not limited to, the following: Table 8: Illustrative List of Collaborating Institution Target Group Collaborating Organization ▪ USG agencies/supported mechanism PEPFAR, USAID/KEA, CDC, ▪ USG supported mechanism KEMSA, HCSM, and Kenya PHARMA ▪ Consortium partners IMA World Health (Prime), Center for HIV Prevention and Research (CHIVPR)- University of Nairobi, Missions for Essential Drugs & Supplies (MEDS), National Organization of Peer Educators (NOPE), and Christian Health Association of Kenya (CHAK); 37 ▪ Nairobi County Government/Ministry of Health CHMT/SCHMT; National Ministry of Health departments – National AIDS & STIs Control Program (NASCOP), Division of Family Planning, Malaria and Community Health; ▪ Donor community UNICEF, WHO, DANIDA and World Bank SELECTION OF PARTICIPANTS KEY INFORMANTS Key informants/panel discussion participants must be purposively selected for this evaluation. Representatives of key institutions, government departments, developmental partners as well as IP/consortium members who are most informed about Afya Jijini Activity shall be sampled. The final sample size will depend upon representation and allocated time/cost of fieldwork. The List of Key Informants and Panelists provides a detailed list (including location and contact information) of all relevant stakeholders that will be interviewed by the evaluation team. Table 9 below provides a summary of the key informant categories by location. Table 9: Estimated Number of Respondents for KII and Panel Discussions Target group Estimated # KII County Government ▪ National Government Ministries/Departments 6 ▪ Health Management Teams - CHMT/SCHMT 9 Implementing Partner ▪ Prime Partners 5 ▪ Consortium Members 5 USG Agencies ▪ USAID 5 ▪ Centers for Disease Control & Prevention (CDC) 1 ▪ Department of Defense (DOD) 1 USG Supported Mechanisms 5 Other Donor Communities ▪ Developmental Partners16 4 Sub-Total 41 Key Activity’s beneficiaries ▪ Health Facility In-charges/Health Management Committees 8 ▪ Community unit in-charges 8 ▪ CBO unit in-charges 8 Sub-Total 24 Total 65 FGD PARTICIPANTS AND KAP/EXIT SURVEYS INTERVIEWEES A systematic sampling approach must be used to select participants for FGDs and Knowledge, Attitudes and Practice survey (KAP) interviews. The contractor Evaluation Team shall liaise with the MNCH clinic, 16 See Annex 06 – List of Key Informants/Panel Discussion Participants 38 CCC, CU and CBO in-charges to estimate the average number of clients seen per day (for facility beneficiaries), and the average number of youth (15-24 years) that participated in HIV prevention services in a month (for CBO supporting evidence-based interventions targeting youths) and the number of community health workers supporting community work within each sampled community unit. • To select health facility beneficiaries - clients attending CCC and MNCH clinics, a systematic random sampling will be applied, with a sampling interval of ‘k’, where, • k = N / (sample size of 7), and • N is average number of clients per target group.17 • The selection criteria must take into consideration the duration of support (i.e. minimum of two years). Participants for CHV FGDs shall be required to have been supporting the activity for a minimum of six months. For FGDs, the contractor shall interview seven participants for each session. For the KAP survey, the estimated sample size for each target beneficiary group is 12. 17 Target group: The average number of clients seen per day OR the average number of Youth (15-24 years) that participated in HIV prevention services in a month OR the number of CHV supporting community work within each sampled facility 39 Table 10: Estimated Number of FGDs/KAP Sessions and Participants Program Beneficiaries FGD KAP Sessions # Participants Sessions # Participants ▪ Health Facilities – MNCH 8 56 8 96 ▪ Health Facilities – CCC 8 56 8 96 ▪ CU – CHVs 8 56 8 96 ▪ CBO – Youths 8 56 8 96 Total 32 224 32 384 6.1.1.8 DATA MANAGEMENT AND ANALYSIS DATA MANAGEMENT QUALITATIVE DATA: A Content Abstraction Guide must be used to abstract relevant data/information from the project documents during desk review. Using this information, the team will develop semi-structured Key Informant/FGD Guides that will be used consistently during fieldwork. These tools must be submitted to USAID for approval prior to commencement of fieldwork. Audio recorders must be used during the interview sessions, and later transcribed by a team of transcribers. QUANTITATIVE DATA: For Capacity Assessment data, an automated spreadsheet shall be designed for data entry and analysis of each parameter of interest. A 5-point Likert Scale must be used to provide an overall score for each parameter. DHIS2 Program data will first be downloaded into a Microsoft Excel spreadsheet before exporting it into SPSS for analysis.18 For the KAP SURVEY, data shall be collected using Android-enabled mobile phones. The SurveyToGo Dooblo Software shall be used to administer the survey questionnaire.19 Each respondent must be allocated a unique nine-digit study numbers. At the end of each day of data collection, the Evaluation Team must upload data from the day’s survey sessions onto a remote secure server. The database shall have data entry screens with skip patterns and data/value ranges programmed into it. This will ensure that the data is consistent at the point of interview. The data shall then be exported into SPSS for statistical data analysis. A code book (reference manual of all variables) will be generated. DELIVERABLES: All raw data shall be submitted to USAID on an optical drive. DATA ANALYSIS: The contractor must utilize the following analytical domains and methodologies in response to the USAID evaluation questions. To the extent possible, all results shall be disaggregated by sex. DOMAIN 1: ACTIVITY’S ACHIEVEMENT/MILESTONES Trend analysis and models for count data shall be conducted for all quantitative indicators over the period under review (Sep, 2015 – Dec, 2017). These approaches will provide insights on the Activity progress toward the program’s midterm targets. 18 IBM Corp. Released 2016. IBM SPSS Statistics for Windows, Version 24.0. Armonk, NY: IBM Corp. 19 SurveyToGo Dooblo: Data Collection Software for Mobile Android & Tablets Surveys: Dooblo Ltd. Kfar Sava, Israel 40 ▪ Baseline Estimates – will be obtained from the District Health Information System 2 (DHIS2), or Early Infant Diagnosis (EID) database, and/or reconstructed from the facility data from the quarter (June – Aug, 2015), i.e. prior to the Activity award. ▪ Coverage Estimation – the observed indicator value will be compared to the projected target (at midterm), by conducting a two-sided hypothesis test at 5% level of significance.20 ▪ Prediction/Forecasting: The fitted models will be used to predict options Year 4 and 5, and in making determination if the Life of Project (LOP) targets are appropriate. Strengths, Weakness, Opportunities and Threats (SWOT) analysis will focus on the Activity’s design to identify internal factors (strength and weakness), and external factors (opportunities and threats), that may have contributed to the observed outcomes. Internal Strengths Activity’s capabilities Weakness Activity’s limitations External Opportunities External factors that can be exploited Threats External factors that are likely to challenge performance Positive Negative Analysis of external factors will evaluate the relevance of the Activity in the current devolved system of governance. Contribution Analysis: The contractor Evaluation Team must develop a logic model for contribution analysis (Figure 7) to understand the development hypothesis and help drawing conclusions about Afya Jijini’s efforts towards the achievement of the observed health outcomes. Three case studies (one under each sub purpose) will be explored. The analysis must then focus on the annual work plans for years 1– 3, and Activity’s input (resources) to document the intended interventions, implementation strategies, resource allocation, gaps in outputs and the observed health outcomes. Figure 7: Logic Model for Contribution Analysis This exercise will also document the roles and responsibilities of the consortium members, coordination efforts with other stakeholders with the county, and linkages with other USAID and Government of Kenya (GoK) national mechanisms; and their effect on the expected health outcomes. The process will 20 Null hypothesis: Observed value = projected target; vs Alternative hypothesis: Observed < Target or Observed > Target INPUTS/ RESOURCES ACTIVITIES/ INTERVENTIONS OUTPUTS OUTCOME INTERMEDIATE RESULTS ASSUMPTIONS 41 also document parallel activities by other developmental partners during the LOP, so as to isolate the contribution of other donors. The analysis must explore the intermediate results, planned interventions and the expected change on the beneficiaries’ behavior including knowledge, practices and adoption of healthy behaviors, specifically: health seeking behavior for HTC, seeking STI treatment, condom negotiation and use and linkage to care and treatment. This shall be used to triangulate findings on target versus achievement. The analysis will also test the validity of the development hypothesis “If health and human capacity in Kenya are sustainably strengthened, then Kenyans will be able to effectively participate in and contribute to the transformation of their governance and economy” to infer the Activity’s contribution. DOMAIN 2: COORDINATION AND COLLABORATION MECHANISMS Stakeholders’ Matrix: The contractor Evaluation Team must categorize the various stakeholders into four main target groups, namely: Government of Kenya, private sector and civil society, donor community and USG as described in Figure 8. The analysis must focus on the existence of collaboration mechanisms/ structures between the Activity and each of the four target groups above, as well as internal coordination, i.e. within the consortium. In addition, the analysis shall explore the feasibility of networks – supply chains management, laboratory network, and referral systems. Figure 8: Potential Coordination and Collaboration Mechanisms Content & Triangulation Analysis: This analytical approach will further be applied to ascertain the effectiveness of every potential collaboration structure within and without the consortium. DOMAIN 3: SUSTAINABILITY Institutional Capacity Assessment: Analysis along the Activity’s sub-purpose 3 – strengthened and functional county health system – must guide this midterm’s assessment on sustainability, using the following parameters/thematic areas: County ownership: leadership and governance, budgetary allocations, leverage (external sources of funding); Human resources for health: staffing/technical capacity of the health management team); Health products and technologies: laboratory network/systems strengthening and supply chain management and inventory management practices; as well as Performance monitoring and evaluation systems: joint work planning, accurate program data, data quality assessment and functional referral systems). To this end, analysis of institutional capacity assessment data (see section 2.3 Description of Evaluation Methods) must provide insight into the ability of the county government to replicate and continue with the program’s activities without further USG 42 investments. A gap analysis must be carried out to identify areas of weakness, as well as compare the current structures/performance and desired performance. Critical Success Factors (CSF): Analysis of various aspects of the Activity, including but not limited to the management structure (leadership, roles and responsibilities, standard operating procedures), designed interventions, program M&E systems, coordination/relationships (internal & external), robust supply chain management, beneficiaries’ satisfaction and adoption of products/services, barriers to accessing and health services by various groups, and the Activity’s operations within devolved government system, that collectively ensure a functional county health system will provide insights into sustainability of the program. Content and Triangulation Analysis: This analytical approach shall be applied to ascertain the appropriate of various factors that could potentially improve the performance of the Activity. Qualitative data from the KII and FGD, as well as quantitative data from the surveys, will be systematically triangulated to document the Activity’s success factors. 2.6 LIMITATIONS Desired response bias: The key informant interviews may yield responses that the Evaluation Team perceive. To mitigate this, the contractor must use the desk review period to design appropriate probing/follow-up questions. Response shift bias: This will be mitigated by specifying the period under review and focusing on the Activity’s contribution during this period. Evaluators’ bias: will be mitigated through cross-referencing and triangulating each evaluator’s perceptions. No comparison group – the performance evaluation must focus on interventions areas (sub-counties within the Nairobi County) and must not take into account other counties where the technical interventions have not been delivered. The evaluation requires a review of documents and reconstruction of baseline estimates and follow-up. This approach assumes 1) the availability of data on all relevant indicators, 2) that indicators are measured consistently, 3) completeness of program data, and 4) the activity’s scope and scale has remained the same over the LOP. The analysis shall factor in all these dynamics and document instances where outcome measures are grossly affected. 3.0 TASK ORDER MANAGEMENT PLAN AND DELIVERABLES 3.1 TO IMPLEMENTATION A phased implementation approach to this task order is hereby proposed. The sections below describe in detail the requirement of each phase. A summary is provided in section 3.4 – Estimated Level of Effort (LOE). 3.1.1 PHASE I – PREPARATORY & TEAM PLANNING DESK REVIEW: The contractor must conduct an extensive document review on selected documents (see section 2.3 Description of review methods) prior to commencement of fieldwork. The objective of this task will be to gather information that will be used to design further probing questions, and to update the list of key informants. This task will take 10 workdays. TEAM PLANNING MEETING: The contractor will hold a 6-day team planning meeting (TPM), culminating with an in-brief meeting between the Evaluation Team and USAID/KEA HPN prior to the start of fieldwork. The TPM will be held at IBTCI’s ESPS Nairobi Office. During the TPM, the Evaluation Team will discuss the objectives of this evaluation, design the evaluation tools, develop the work/logistics plans, and prepare for the USAID/ESPS in-brief meeting. ESPS will make prior arrangements with the IP to participate in the TPM. 43 Deliverables/LOE: Finalization of the assessment tools, work plan and logistics. The final work plan must be submitted to USAID for approval. The maximum level of effort (LOE) for Phase I is 18 days (including travel time). 3.1.2 PHASE II - FIELDWORK The contractor will utilize a phased-implementation approach for this evaluation. First, the team must meet with the IP before meeting with other stakeholders. Interviews with other stakeholders will be scheduled depending on their availability but within the approved implementation work plan/schedule of activities. Deliverables/LOE: Successful completion of fieldwork (conducting all scheduled interviews and surveys), submission of raw data, and providing progress reports to the Task Order Contract Officer Representative (TOCOR) as agreed. The level of effort (LOE) for Phase II is 12 days (including travel time). 3.1.3 PHASE III - POST FIELDWORK This phase will be dedicated to data analysis, consolidating all findings, and report writing. The team must hold a validation meeting with the Afya Jijini team to discuss evaluation findings. The report shall be structured according to the four evaluation questions. A synopsis of the findings shall be shared with USAID/KEA during the ESPS/USAID out-brief meeting before the team departs from Nairobi. ESPS must submit the final evaluation report to USAID/KEA and to the Development Experience Clearinghouse (DEC) as provided in the activity contract. Deliverables/LOE: These will include, a validation meeting with Afya Jijini team, submission of final report. The maximum LOE for this phase is 32 workdays. 3.2 ESPS MANAGEMENT OF THE TASK ORDER Overall Coordination: The ESPS Chief of Party (COP) will be the primary point of contact for USAID/KEA, assuming a central role in ensuring the midterm evaluation is being conducted as proposed, deadlines are adhered to and the final report is submitted on time. The frequency of periodic update briefings with USAID/KEA and the TOCOR throughout the assignment will be discussed at the first review meeting and the agreed upon schedule will be reflected in an updated implementation work plan. ESPS Support Team: The COP will be assisted by the ESPS Finance Manager and Program Support Manager located in IBTCI’s ESPS Nairobi Office. The support team will be actively involved in: (i) communication with and obtaining a written confirmation from the participating stakeholders and implementing partner; and (iii) development of logistical plans. ESPS Technical Team Support: The ESPS Senior M&E Advisor and the ESPS Public Health Specialist will provide day-to-day management of the Evaluation Team; guide the team to ensure that the proposed methodology, data collection plan, and data analysis methods approved by USAID are adhered to. They will provide technical directions to the Evaluation Team as necessary; avail the relevant documents needed for the desk review and participate in data analysis as needed. They will monitor all field activities and contractual deliverables stipulated in the Scope of Work (SOW), Implementation Work Plan and consultant’s Terms of Reference (TOR). The IBTCI Home Office will play an important role in ensuring that the quality of deliverables applicable to this evaluation meets USAID/KEA standards and performance evaluation requirements. This will be conducted through peer review of all deliverables by the IBTCI Home Office programmatic team. 44 6.2 ANNEX 2: SAMPLE SELECTION STRATEGY Table 11 below represents the final respondent categories and number of interviews, discussions and survey conducted during the mid-term evaluation. Table 11: The Final Respondents’ Categories and Number of Interviews, Discussions and Surveys Conducted Respondent categories No. of KIIs No. of FGDs No. of Panels No. of Surveys Ministry of Health & County Government  County/Sub-county Health Management Teams - CHMT/SCHMT 21 9 Implementing Partner  Prime Partner – IMA World Health & Consortium Members 17 • Christian Health Association of Kenya (CHAK) • Mission for Essential Drugs & Supplies (MEDS) • National Organization of Peer Educators (NOPE) • Center for HIV Prevention & Research (CHIVPR) USG Agencies/supported mechanisms  USAID 1  Supported Mechanisms 8 • IntraHealth (HRH Capacity) • Health Policy Plus (HPP+) • Health Commodities and services Management (HCSM) • KEMSA (Medical Commodities Program) • NILINDE • AMREF Health Africa • LINKAGES Other Donor Communities  UNICEF, DANIDA 2 Key Activity’s beneficiaries  Community Health Volunteers (CHVs) 7 92  Community Health Assistant (CHA) 2  Community-based organization (CBO) unit in-charges 1  Health Facility In-charge 8  Health Care Workers 13 38  Maternal, newborn and child health (MNCH) Clients 10 98  Comprehensive Care Clients (CCC)Clients 6 97  Youth 4 162 45 Respondent categories No. of KIIs No. of FGDs No. of Panels No. of Surveys  Adolescent Girls and Yong Women (AGYW) 3 Total 73 30 9 487 Sampling strategies To ensure valid representation from multiple stakeholders when conducting the Afya Jijini mid-term evaluation, a systematic multi-stage stratified sampling approach was used in the selection of health facilities and participants for this evaluation. Selection of Sub-counties First, the County of Nairobi was stratified into four Clusters (Table 12). Due to cost considerations, and the geographical spread of health facilities, ESPS proposed to select 50%, i.e. eight of the 16 sub-counties supported by the Activity. Table 12: Sub-Counties of the County of Nairobi Cluster 1 Cluster 2 Cluster 3 Cluster 4 Westlands Roysambu Embakasi South Makadara Dagoretti North Kasarani Embakasi North Kamukunji Dagoretti South Ruaraka Embakasi Central Starehe Langata Embakasi East Mathare Kibra** Embakasi West ** Service coverage did not include Kibra sub county (ref: USAID/KEA, Afya Jijini Contract). Note: Inclusion of sub-counties within a cluster is based on their proximity to each other, see Table 13: Table 13: Sub-Counties of the County of Nairobi Selected for the AJ Mid-Term Evaluation Cluster 1 Cluster 2 Cluster 3 Cluster 4 Westlands Embakasi South Makadara Kasarani Embakasi North Ruaraka Langata Mathare Selection of Health Facilities: Within the selected sub-counties, health facilities were stratified by level of care (hospitals, health centers and dispensaries), ownership (private or public) and location (informal or formal settlement). Based on the sampling frame of HFs, there were 16 HFs in the selected/sampled sub counties. As we decided to cover 50% (8 HF to conduct the FGDs and the KAP survey). The criteria for the selection of the Health facilities was done as follows: 1. Selected HF represent the 4 clusters and the sub counties selected (Table 14), 2. The selected HF were covering the following services MNCH, FP/RH, WASH, Nutrition, PMTCT and HIV C&T 3. Inclusion Criteria: health centers (ANC/PMTCT client load > 500/annum), and dispensaries (ANC/PMTCT client load >200/annum); the sample include between 15 – 25% of the high-volume facilities. 46 Table 14: Selected HFs for the CCC and Youth Beneficiary Interviews or Surveys - Sampled Sub￾Counties Identified by symbol* Cluster 1 Cluster 2 Cluster 3 Cluster 4 Westlands Ruaraka Embakasi South Makadara 1.Kangemi Health Center 3. Mathare North HC 5.Mukuru HC 7. Bahati HC Embakasi North Langata Kasarani 6.Kariobagi HC Mathare 2.St Mary Mission Hospital * 4. Kasarani HC * 8.Makadara HC This gives us a total of 8 Session of the KAP and FGDs with total of the 96 participants. Table 15: Selected HFs for the MNCH Beneficiary Interview or Surveys - Sampled Sub-Counties Identified by symbol* Cluster 1 Cluster 2 Cluster 3 Cluster 4 Westlands Ruaraka Embakasi South Makadara 1.Kangemi Health Center 2.Lianas Clinic Dispensary * 5. Mathare North HC 9.Mukuru HC 13. Bahati HC 10.Embakasi Medical Center 14. Mbotela Clinic* 6. Babadogo HC Embakasi North Kasarani Langata 7.Kasarani HC * 11.Kariobagi HC Mathare 3.St Mary Mission Hospital* 4.St Mary immaculate* 8.St Peter Dispensary 12.St Clare Medical * 15.Makadara HC 16. Huruma Dispensary * All CUs that are linked to the sampled health facilities were selected for this evaluation. Selected Community Units associated with health facilities selected are presented in Table 16. Table 16: Selected HFs for the CU Interviews - Sampled Sub-Counties Cluster 1 Cluster 2 Cluster 3 Cluster 4 Westlands Ruaraka Embakasi South Makadara 1.Kangemi Health Center (2 CU) 3. Mathare North HC (1 CU) 5.Mukuru HC (2 CU) 7. Bahati HC (1 CU) 4.Uzima Dispensary Embakasi North Langata 6.Kariobagi HC (1CU) Mathare 2.St Mary Mission Hospital 8.Makadara HC (1CU) CUs supporting HIV prevention activities for youth and young women were purposively selected for this evaluation to ensure effective representation of the Activity’s geographic coverage. The selection was based on following inclusion criteria. Inclusion criteria: 1) total time of support in years, 2) total number of youths that it supports, 3) geographical location (proximity to the sampled health facilities and/or community unit). 47 Selection of Participants: Key Informants Key informants/panel discussion participants were purposively selected for this evaluation. Representatives of key institutions, government departments, developmental partners as well as IP/consortium members who are most informed about Afya Jijini Activity were sampled. The final sample size depended upon representation and allocated time/cost of fieldwork. See Table 1 above. FGD Participants and KAP Surveys interviewees A systematic sampling approach was used to select participants for FGDs and Knowledge, Attitudes and Practice survey (KAP) interviews. The Evaluation Team liaised with the MNCH clinic, CCC, CU and CBO in-charges to estimate the average number of clients seen per day (for facility beneficiaries), and the average number of youth (15-24 years) that participated in HIV prevention services in a month (for CBO supporting evidence-based interventions targeting youths) and the number of community health workers supporting community work within each sampled community unit. • To select health facility beneficiaries - clients attending CCC and MNCH clinics, a systematic random sampling was applied, with a sampling interval of ‘k’, where, • k = N / (sample size of 7), and • N is average number of clients per target group.21 • The selection criteria took into consideration the duration of support (i.e. minimum of two years). Participants for CHV FGDs were required to have been supporting the activity for a minimum of six months. Table 17 represents the estimated sample size for each target beneficiary group for FGD and KAP survey. Table 18 represents the final sample achieved for the KAP survey by the type of respondents in each selected health facility. Each FGD had about 6-8 participants. Participants in FGD and KAP surveys are unique respondents. Table 17: Estimated Number of FGDs/KAP Sessions and Participants Program Beneficiaries FGD KAP Sessions # Participants Sessions # Participants  Health Facilities – MNCH 8 56 8 96  Health Facilities – CCC 8 56 8 96  CU – CHVs 8 56 8 96  CBO – Youths 8 56 8 96 Total 32 224 32 384 21 Target group: The average number of clients seen per day OR the average number of Youth (15-24 years) that participated in HIV prevention services in a month OR the number of CHV supporting community work within each sampled facility 48 Table 18: Final Sample for KAP Survey Interviewees Health Facility Name Respondent groups CCC Clients Total MNCH clients CHVs Youth Bahati HC 12 13 13 10 48 Kangemi HC 13 12 12 12 49 Kariobagi HC 12 12 14 12 50 Kasarani HC 12 12 15 12 51 Makadara HC 12 12 12 12 48 Mathare North HC 12 13 12 13 50 Mukuru HC 12 12 12 79 115 St Mary Mission Hospital 12 12 2 12 38 Total 97 98 92 162 449 49 6.3 ANNEX 3: EVALUATION PLAN MATRIX EVALUATION KEY QUESTION 1: What progress has the Activity made towards the achievement of the program’s midterm and mandates/targets/outcomes/milestones? With regard to the activity design, what are the main constraints and/or weaknesses, opportunities and threats that need further attention? REVIEW SUB-QUESTION TYPE OF EVIDENCE DATA COLLECTION SAMPLING OR SELECTION APPROACH DATA ANALYSIS METHOD SOURCE METHOD Effectiveness 1.1 To what extent is the program on course to meet the life of program targets? 1.2 To what degree do the program outputs represent the maximum possible given the inputs in each program area? Efficiency 1.3 To what degree do the program outputs represent the optimal combination to address HIV/AIDS, MNCH, nutrition and FP/RH in the program’s country context? 1.4 To what degree are planned activities and corresponding budgets adequate and appropriate to achieve targets and objectives? Contribution & Exploratory Contribution and Exploratory Contribution & Exploratory Analytical and Contribution Project Documents County/facility minutes DHS2 CPs, CHMTs, HFs, SCHMTs, LIP – Youth, DP HFBs, CHVs, Project Documents CPs, CHMTs, HFs, SCHMTs, LIP – Youth, DP HFBs, CHVs, Project Documents CPs, CHMTs, HFs, SCHMTs, LIP – Youth, Document Review Minutes Review System review KIIs FGDs Document Review KIIs FGDs Document Review KIIs FGDs Document Review KIIs As appropriate Purposive sampling Purposive sampling Purposive sampling As appropriate As appropriate Purposive sampling As appropriate As appropriate Trend Analysis Contribution Analysis 50 EVALUATION KEY QUESTION 1: What progress has the Activity made towards the achievement of the program’s midterm and mandates/targets/outcomes/milestones? With regard to the activity design, what are the main constraints and/or weaknesses, opportunities and threats that need further attention? REVIEW SUB-QUESTION TYPE OF DATA COLLECTION SAMPLING OR DATA 1.5 How well have the program resources been used? 1.6 What opportunities exist to accelerate program impact with a modified mix of activities or funding? 1.7 Are there cost-efficiencies that could be achieved? Cross-cutting elements 1.8 To what degree do these principles or requirements need further attention or modification during the second half of Afya Jijini program implementation? 1.9 Assess the degree to which the program is complying with USAID’s mandatory requirements such as visibility, gender integration and environmental compliance. For example, 1.9.1 How is the program addressing gender inequalities/inequities in service provision, service delivery and behavior change for the beneficiaries? 1.9.2 Assess the adequacy of Afya Jijini’s program gender strategy? 1.9.3 To what degree is the program adhering to the implementation of the environmental compliance and mitigation plan? Beneficiary relations 1.10Does services coverage (location and range of services offered) represent an efficient use of program resources? Exploratory Exploratory and Comparative Comparative Analysis Analytical DP HFBs, CHVs, Project Documents CPs, CHMTs, HFs, SCHMTs, LIP – Youth, DP HFBs, CHVs Project Documents CPs, CHMTs, HFs, SCHMTs, LIP – Youth, DP HFBs, CHVs Project Documents CPs, CHMTs, HFs, SCHMTs, LIP – Youth, DP HFBs, CHVs Project Documents CPs, DGDs, CHMTs, SCHMTs, LIP – Youth, FGDs Document Review KIIs FGDs Document Review KIIs FGDs Document Review KIIs FGDs Document Review KIIs Purposive sampling As appropriate As appropriate Purposive sampling As appropriate As appropriate Purposive sampling As appropriate As appropriate Purposive sampling As appropriate As appropriate Exploratory Analysis Comparative Analysis Content & Triangulation Analysis Content & Triangulation Analysis 51 EVALUATION KEY QUESTION 1: What progress has the Activity made towards the achievement of the program’s midterm and mandates/targets/outcomes/milestones? With regard to the activity design, what are the main constraints and/or weaknesses, opportunities and threats that need further attention? REVIEW SUB-QUESTION TYPE OF DATA COLLECTION SAMPLING OR DATA 1.11How do program beneficiaries view the following: 1.11.1 hours of operation, 1.11.2 professionalism of staff, 1.11.3 quality of care, 1.11.4 confidentiality, 1.11.5 ease of accessing multiple services, 1.11.6 availability of product. 1.12What factors influence the decision to use or not use services? Performance monitoring and impact 1.13What M&E system does Afya Jijini use in monitoring progress and trends? 1.14Does the M&E plan facilitate utilization of data for improved performance? 1.15Has the program established reasonable methods of gathering the data necessary to monitor and evaluate progress and indicator data? 1.16Is the performance management plan developed and used to provide timely and high-quality data? If not, why not? Analytical Analytical and Exploratory Analytical and Exploratory Analytical and Exploratory Analytical and Exploratory Exploratory DP HFBs, CHVs Project Documents CPs, CHMTs, SCHMTs, LIP – Youth, DP Project Documents CPs, CHMTs, SCHMTs, LIP – Youth, DP CPs, CHMTs, SCHMTs, LIP – Youth, DP CPs, CHMTs, SCHMTs, LIP – Youth, DP CPs, CHMTs, SCHMTs, LIP – Youth, DP Project Documents Document Review KIIs KIIs KIIs KIIs Document Review KIIs Mini-KAP survey FGDs Purposive sampling As appropriate As appropriate Purposive sampling As appropriate Purposive sampling Purposive sampling Purposive sampling Purposive sampling Comparative Analysis Content & Triangulation Analysis Content & Triangulation Analysis Content & Triangulation Analysis 52 EVALUATION KEY QUESTION 1: What progress has the Activity made towards the achievement of the program’s midterm and mandates/targets/outcomes/milestones? With regard to the activity design, what are the main constraints and/or weaknesses, opportunities and threats that need further attention? REVIEW SUB-QUESTION TYPE OF DATA COLLECTION SAMPLING OR DATA Management systems within the program 1.17Did consortium members develop a set of common criteria for excellence in internal consortium management, such as compliance to donors and each other and programmatic, financial and managerial accountability for service delivery to communities and/or program participants? 1.18Did consortium members agree to a set of appropriate financial, administrative and managerial processes and procedures that are based on consortium needs and each agency’s strengths? Did they develop an operations manual documenting these processes and procedures to remain in compliance with the national law and donor requirements? 1.19Did the consortium members define their roles based on each agency’s capacities and the program’s needs? 1.20Does the consortium or partnership have guidelines for conflict resolution, communication and decision￾making that reinforce transparency and accountability at all levels? How have gender-equitable programming and integration been presented to consortium members, staff and partners? 1.21Do consortium members conduct business in a transparent, timely and respectful fashion and work to build a consortium based on trust and mutual respect, consistently supporting positive interpersonal behavior? 1.22Does the consortium have an open and transparent process for decision making and arbitration? Can it be improved? 1.23Do consortium members put the needs and identity of the consortium ahead of individual organizational needs when representing the consortium? 1.24Does the consortium or partnership allocate adequate Contribution Contribution & Exploratory Analytical Exploratory & Contribution DPs, CPs, CHMTs, SCHMTs, LIP - Youth HFBs, CHVs, LIP - Youth HFBs, CHVs, LIP – Youth CPs, CHMTs, HFs, LIP -OVC, CHAs, SCHMTs HFBs, CHVs, LIP – Youth Project Documents CPs, CHMTs, HFs, LIP -OVC, CHAs, SCHMTs Project Documents KIIs FGDs Document review KIIs Document review KIIs Document review KIIs Document review KIIs As appropriate Purposive sampling Systematic sampling Purposive sampling Purposive sampling Purposive sampling As appropriate Purposive sampling Content & Triangulation Analysis Content & Triangulation Analysis Comparative Analysis Contribution Analysis Content & Triangulation 53 EVALUATION KEY QUESTION 1: What progress has the Activity made towards the achievement of the program’s midterm and mandates/targets/outcomes/milestones? With regard to the activity design, what are the main constraints and/or weaknesses, opportunities and threats that need further attention? REVIEW SUB-QUESTION TYPE OF DATA COLLECTION SAMPLING OR DATA resources to monitoring, evaluation, learning and knowledge management systems; support staff in learning, change and innovation; and create and sustain a culture that continually improves its management practice from lessons learned, both failures and successes? 1.25Did the consortium members periodically meet to review program progress and implementation challenges? Exploratory Exploratory Analytical Analytical Analytical CPs, CHMTs, HFs, LIP -OVC, CHAs, SCHMTs Project Documents CPs, CHMTs, HFs, LIP -OVC, CHAs, SCHMTs Project Documents CPs, CHMTs, HFs, LIP -OVC, CHAs, SCHMTs Project documents CPs, DPs Project documents CPs, DPs Document review KIIs Document review KIIs Document review KIIs Document review KIIs Document review KIIs Document review KIIs Document review As appropriate Purposive sampling As appropriate Purposive sampling As appropriate Purposive sampling As appropriate Purposive sampling As appropriate Analysis Exploratory Analysis Contribution Analysis Contribution Analysis Content & Triangulation Analysis 54 EVALUATION KEY QUESTION 1: What progress has the Activity made towards the achievement of the program’s midterm and mandates/targets/outcomes/milestones? With regard to the activity design, what are the main constraints and/or weaknesses, opportunities and threats that need further attention? REVIEW SUB-QUESTION TYPE OF DATA COLLECTION SAMPLING OR DATA Analytical Analytical Analytical & Exploratory Analytical & Exploratory Analytical & Exploratory Project documents CPs, DPs Project documents CPs, DPs Project documents CPs, DPs Project documents CPs, DPs Project documents CPs, DPs Project documents CPs, DPs KIIs Document review KIIs Document review KIIs Purposive sampling As appropriate Purposive sampling As appropriate Purposive sampling As appropriate Purposive sampling As appropriate Purposive sampling As appropriate Purposive sampling As appropriate Content & Triangulation Analysis Content & Triangulation Analysis Content & Triangulation Analysis Content & Triangulation Analysis 55 EVALUATION KEY QUESTION 1: What progress has the Activity made towards the achievement of the program’s midterm and mandates/targets/outcomes/milestones? With regard to the activity design, what are the main constraints and/or weaknesses, opportunities and threats that need further attention? REVIEW SUB-QUESTION TYPE OF DATA COLLECTION SAMPLING OR DATA Project documents CPs, DPs Purposive sampling As appropriate Purposive sampling Content & Triangulation Analysis Content & Triangulation Analysis Content & Triangulation Analysis Content & Triangulation Analysis 56 EVALUATION KEY QUESTION 2: What coordination and collaboration mechanisms/structures has the Activity established and/or strengthened among key stakeholders (Service delivery implementing partners, relevant GOK ministries including Social Services)? How effective have these coordination/collaboration structures been in achieving program objectives? REVIEW SUB-QUESTION TYPE OF EVIDENCE DATA COLLECTION SAMPLING OR SELECTION APPROACH DATA ANALYSIS SOURCE METHOD METHOD Sustainability 2.1 What is Afya Jijini’s contribution to developing a comprehensive integrated health program in Nairobi City County? 2.2 How does the Ministry of Health, sub-Counties and county show signs of improving their ability to manage services and operations (managerial, financial, technical etc.)? 2.3 How does health service delivery continue to receive technical assistance under Afya Jijini? Linkages, coordination and leveraging 2.4 How effectively has Afya Jijini program coordinated, managed and linked sub-county, facility and community￾based services? 2.5 Has the creation of these partnerships been effective? Analytical Analytical Analytical Analytical & Contributory Analytical Project documents CPs, DPs Project documents CPs, DPs Project documents CHMT, SCHMTs, CPs, DPs, LIP – Youth, CHAs Project documents CHMT, SCHMTs, CPs, DPs, LIP – Youth, CHAs Project documents CHMT, SCHMTs, CPs, DPs, LIP – Youth, CHAs Project documents CHMT, SCHMTs, CPs, DPs, LIP – Youth, CHAs Document Review KIIs Document review KIIs Document Review KIIs Document Review KIIs Document Review KIIs As appropriate Purposive sampling As appropriate Purposive sampling As appropriate Purposive sampling As appropriate Purposive sampling As appropriate Purposive sampling Contribution Analysis Content & Triangulation Analysis Contribution Analysis Content & Triangulation Analysis Content & Triangulation Analysis 57 EVALUATION KEY QUESTION 2: What coordination and collaboration mechanisms/structures has the Activity established and/or strengthened among key stakeholders (Service delivery implementing partners, relevant GOK ministries including Social Services)? How effective have these coordination/collaboration structures been in achieving program objectives? REVIEW SUB-QUESTION TYPE OF EVIDENCE DATA COLLECTION SAMPLING OR SELECTION APPROACH DATA ANALYSIS SOURCE METHOD METHOD 2.6 Are there best practices or approaches to developing sector linkages that should receive additional attention or resources during the second half of the program? 2.7 Assess Afya Jijini program’s wider participation in terms of collaboration and coordination within the partnership, other USG programs/programs, and other key donor programs/programs, and stakeholders in health service delivery arena. Performance monitoring and impact 2.8 What else is going on in the health sector/environment that might account for the changes or the program achieving its objectives? Analytical Analytical & Contribution Analytical & Contribution Project documents CHMT, SCHMTs, CPs, DPs, LIP – Youth, CHAs Project documents CHMT, SCHMTs, CPs, DPs, LIP – Youth, CHAs Document Review KIIs Document Review KIIs Document Review KIIs As appropriate Purposive sampling As appropriate Purposive sampling As appropriate Purposive sampling Content & Triangulation Analysis Content & Triangulation Analysis Contribution Analysis 58 EVALUATION KEY QUESTION 3: What sustainable and innovative service delivery models is the activity currently implementing? What does the Ministry of Health and other relevant stakeholders think in terms of their long-term use? REVIEW SUB-QUESTION TYPE OF EVIDENCE DATA COLLECTION SAMPLING OR SELECTION APPROACH DATA ANALYSIS SOURCE METHOD METHOD Factors of success 3.1 What are the lessons learned from successful interventions that merit continuation or replication? 3.2 What practices, products and tools should be considered for dissemination? Sustainability 3.3 Are these activities likely to continue without further USG investments? 3.4 What investments or approaches would promote services delivery program sustainability? 3.5 To what extent is the MoH/County team demonstrating ownership and commitment to sustaining accomplishments under Afya Jijini? Beneficiary relations 3.6 What is the success of community volunteers in helping the program achieve its objectives? Analytical Analytical Contribution Contribution Contribution & Analytical Contribution & Analytical Project Document DPs, CPs, CHMTs, SCHMTs Project Document DPs, CPs, CHMT, SCHMTs Project Documents CPs, CHMTs, SCHMTs Project Documents CPs, CHMTs, SCHMTs Project Documents CPs, CHMTs, SCHMTs, CHAs Project Documents CPs, CHMTs, SCHMTs, CHAs Project Documents Document Review KIIs Document Review KIIs Document Review KIIs Document Review KIIs Document Review KIIs As appropriate Purposive sampling As appropriate Purposive sampling As appropriate Purposive sampling As appropriate Purposive sampling As appropriate Purposive sampling Content & Triangulation Analysis Content & Triangulation Analysis Content & Triangulation Analysis Contribution Analysis Content & Triangulation Analysis 59 EVALUATION KEY QUESTION 3: What sustainable and innovative service delivery models is the activity currently implementing? What does the Ministry of Health and other relevant stakeholders think in terms of their long-term use? REVIEW SUB-QUESTION TYPE OF EVIDENCE DATA COLLECTION SAMPLING OR SELECTION APPROACH DATA ANALYSIS SOURCE METHOD METHOD Management systems within the program 3.7 Which strategies have worked in encouraging partner participation and involvement, and which did not? Contribution & Analytical CPs, CHMTs, SCHMTs, CHAs Document Review KIIs Document Review KIIs As appropriate Purposive sampling As appropriate Purposive sampling Content & Triangulation Analysis Content & Triangulation Analysis EVALUATION KEY QUESTION 4: What are the key programmatic and management recommendations that the mission could consider for mid-course changes to the current program design? REVIEW SUB-QUESTION TYPE OF EVIDENCE DATA COLLECTION SAMPLING OR SELECTION APPROACH DATA ANALYSIS SOURCE METHOD METHOD Effectiveness 4.0 Is the life of program targets appropriate, given the implementation environment and the evolution of service delivery in Kenya for HIV/AIDS, MNCH, FP/RH and Nutrition? If not, the evaluation team should make specific recommendations for contractual modifications, such as life of program targets, tasks or other aspects of the contract Obstacles 4.1 What approaches, lessons learned from other settings, or program modifications might address these obstacles? Contribution & Exploratory Contribution & Exploratory Project Document DPs, CPs, CHMTs, SCHMTs, CHAs Project Document DPs, CPs, CHMTs, Document Review KIIs Document Review KIIs As appropriate Purposive sampling As appropriate Purposive sampling Content & Triangulation Analysis Content & Triangulation Analysis 60 EVALUATION KEY QUESTION 4: What are the key programmatic and management recommendations that the mission could consider for mid-course changes to the current program design? REVIEW SUB-QUESTION TYPE OF EVIDENCE DATA COLLECTION SAMPLING OR SELECTION APPROACH DATA ANALYSIS SOURCE METHOD METHOD 4.2 Are there any obstacles that require policy changes or dialogue? 4.3 Are there obstacles that threaten program performance or sustainability? Performance monitoring and impact 4.4 What suggestions does the team have for improving the M&E system? Management systems within the program 4.5 How effective is the partnership in planning, developing implementation strategies, reviewing Contribution & Exploratory Contribution & Exploratory Exploratory & Analytical Contribution & Analytical SCHMTs, CHAs Project Documents DPs, CPs, CHMTs, SCHMTs, CHAs Project Documents CPs, DPs, CHMTs, SCHMTs, CHAs Project Documents CPs, DPs, CHMTs, SCHMTs, CHAs Project Documents CPs, DPs, CHMTs, SCHMTs, CHAs Project Document Review KIIs Document Review KIIs Document review KIIs Document review As appropriate Purposive sampling As appropriate Purposive sampling As appropriate Purposive sampling As appropriate Contribution Analysis Contribution Analysis Content & Triangulation Analysis Content & Triangulation Analysis 61 EVALUATION KEY QUESTION 4: What are the key programmatic and management recommendations that the mission could consider for mid-course changes to the current program design? REVIEW SUB-QUESTION TYPE OF EVIDENCE DATA COLLECTION SAMPLING OR SELECTION APPROACH DATA ANALYSIS SOURCE METHOD METHOD monitoring reports, documenting and sharing lessons learned, and making programmatic policy decisions? Provide specific recommendations to improve the quality of partnership. USAID’s program management 4.6 Are there any management actions or steps that might enhance program implementation and success? Contribution & Analytical Documents CPs, DPs, CHMTs, SCHMTs, CHAs KIIs Document review KIIs Purposive sampling As appropriate Purposive sampling Content & Triangulation Analysis 62 6.4 ANNEX 4: LIST OF DOCUMENTS REVIEWED 1. ESPS Technical Proposal/Scope of Work; 2. Afya Jijini Cooperative Agreement; 3. Activity’s Monitoring & Evaluation Plans (MEL Plans); 4. Annual and Quarterly Reports; 5. Financial Reports; 6. USAID Gender Analysis and Action Plan (2012); 7. National Monitoring and Evaluation Framework towards the Prevention of and Response to Sexual and Gender Based Violence in Kenya (2014); 8. DREAMS Core Package of Interventions Summary; 9. Health Sector Working Group Report Medium Term Expenditure Framework Budget For The Period 2014/2015 - 2019/2020; 10. Nairobi City County HIV & AIDS Strategic Plan (2015/2016 - 2018/2019); 11. Nairobi City County Health Sector Strategic and Investment Plan 2013/2014 – 2018/2019 63 6.5 ANNEX 5: DATA COLLECTION TOOLS AFYA JIJINI MID TERM EVALUATION Key informant Interview Guides (Version: May 16, 2018) Prime Partner (IMA) KII/Panel Discussion Guide Introduction & Consent Good morning/afternoon. My name is ______________________, I am an independent Consultant contracted by IBTCI on behalf of USAID/Kenya. We are working with IBTCI to conduct a mid-term review (MTR) on behalf of USAID/Kenya for the Afya Jijini Program. Your Ministry/organization/ department/ County/ Sub-county/hospital/ community unit has been selected as one of the key stakeholders of Afya Jijini program to assist with information to support the MTR. The interview will take approximately 90 minutes. We are, therefore, grateful for your participation. I will be taking notes/recording during our discussion, however, no individual information or names will be included in the final report to ensure confidentiality. Do you agree to participate in the survey? Can I now proceed? Title of Respondent ________________________________________ 1. What is the role of your organization in the Afya Jijini strategy? Progress towards achieving targets 1. Has the role of your organization changed over the lifetime of the project? Have you been given additional activities? Have you had any activities eliminated?(Copy of contract modification if any) 2. To what extent is Afya Jijini/Consortium partners on course to meet the life of program targets? 3. To what degree do the program outputs represent the optimal combination to address HIV/AIDS, MNCH, nutrition and FP/RH in the program’s country context? a) To what degree are planned activities and corresponding budgets adequate and appropriate to achieve targets and objectives? b) How well have the program resources been used? Are there cost-efficiencies that could be achieved? c) What opportunities exist to accelerate program impact with a modified mix of activities or funding? d) To what degree do these principles or requirements need further attention or modification during the second half of Afya Jijini program implementation? 4. Are they still relevant? Assess the degree to which the program is complying with USAID’s mandatory requirements such as visibility (branding and communication), gender integration and environmental compliance. For example, a) How is the program addressing gender inequalities/inequities in service provision, service delivery and behavior change for the beneficiaries? b) Assess the adequacy of Afya Jijini’s program gender strategy? c) To what degree is the program adhering to the implementation of the environmental compliance and mitigation plan? 5. Does services coverage (location and range of services offered) represent an efficient use of program resources? 6. How do program beneficiaries view the following: 64 a) Hours of operation, b) Professionalism of staff, c) Quality of care, d) Confidentiality, e) Ease of accessing multiple services, f) Availability of product. 7. Describe the M&E system used by Afya Jijini in monitoring progress and trends and how you fit in. a) Does the Afya Jijini M&E system facilitate utilization of data for improved performance? (Probe for costing and completeness) b) Is the performance management plan developed and used to provide timely and high-quality data? If not, why not? (IMA) c) Does Afya Jijini have a complete and costed M&E plan that facilitates utilization of data for improved performance? Explain by citing specific examples. d) Has the program established reasonable methods of gathering and managing data necessary to monitor and evaluate progress and indicator data? e) Is the performance management plan developed and used to provide timely and high-quality data? If not, why not? f) Which organizations/ departments have so far received Afya Jijini program’s M&E Support? Explain. g) Explain how Afya Jijini’s M&E activities are linked and integrated within the National M&E System, if any. h) Are there any challenges/ limitations Afya Jijini has so far experienced that has limited ability to provide effective performance monitoring supports to sub-grantees, MOH and health facilities? Explain i) What suggestions do you have for improving Afya Jijini’s M&E system in the next phase of the program? 8. Has Afya Jijini established and supported health commodity inventory management practices, commodity forecasting and supply planning at the county level? Health facilities? Explain (Meds). 9. Did consortium members develop a set of common criteria for excellence in internal consortium management, such as compliance to donors and each other and programmatic, financial and managerial accountability for service delivery to communities and/or program participants? 10. Did consortium members agree to a set of appropriate financial, administrative and managerial processes and procedures that are based on consortium needs and each agency’s strengths? Did they develop an operations manual documenting these processes and procedures to remain in compliance with the national law and donor requirements? 11. Did the consortium members define their roles based on each agency’s capacities and the program’s needs? 12. Does the consortium or partnership have guidelines for conflict resolution, communication and decision-making that reinforce transparency and accountability at all levels? How have gender￾equitable programming and integration been presented to consortium members, staff and partners? 13. Do consortium members conduct business in a transparent, timely and respectful fashion and work to build a consortium based on trust and mutual respect, consistently supporting positive interpersonal behavior? 14. Does the consortium have an open and transparent process for decision making and arbitration? Can it be improved? 15. Do consortium members put the needs and identity of the consortium ahead of individual organizational needs when representing the consortium? 16. Does the consortium or partnership allocate adequate resources to monitoring, evaluation, learning and knowledge management systems; support staff in learning, change and innovation; and 65 create and sustain a culture that continually improves its management practice from lessons learned, both failures and successes? 17. Did the consortium members periodically meet to review program progress and implementation challenges? 18. Please share with me some of the strengths, weaknesses, opportunities and threats in the implementation of Afya Jijini program. 19. How many high volume health facilities in Nairobi County supported by Afya Jijini conducted data quality assessments annually? 20. How many health facilities in Nairobi County supported by Afya Jijini that conducted data quality assessments implement data quality improvement plans? Share sample of DQI 21. Out of the high volume facilities supported by Afya Jijini in Nairobi County, how many have functional referral systems? Coordination and collaboration mechanism 1. What is Afya Jijini’s/consortium partners’ contribution to developing a comprehensive integrated health program in Nairobi City County? 2. How does the Ministry of Health, sub-Counties and county show signs of improving their ability to manage services and operations (managerial, financial, technical etc.)? 3. How effectively has Afya Jijini program coordinated, managed and linked sub-county, facility and community-based services? 4. Has the MOH team at the county and sub-county levels been part of the implementation of Afya Jijini program? (Probe- Does the MOH team at the county and sub-county levels feel they are part and own products and services provided by Afya Jijini so far? Explain. Does the MOH at the county level have systems in place to sustain accomplishments under Afya Jijini so far? Explain) 5. Has the creation of these partnerships (between MOH and AJ) been effective? 6. Are there best practices or approaches to developing sector linkages that should receive additional attention or resources during the second half of the program? 7. Assess Afya Jijini program’s wider participation in terms of collaboration and coordination within the partnership, other USG programs/programs, and other key donor programs/programs, and stakeholders in health service delivery arena. 8. What else is going on in the health sector/environment that might account for the changes or the program achieving its objectives? Sustainability and innovation of service delivery models 1. Are AJ/ Consortium partner supported activities likely to continue without further USG investments? a) To what extent is the MoH/County team demonstrating ownership and commitment to sustaining accomplishments under Afya Jijini? b) What other investments or approaches would promote services delivery program sustainability? 2. What are the lessons learned from interventions implemented by AJ/Consortium partners that merit continuation or replication in the next phase? a) What practices, products, tools should be considered for dissemination? b) What models and implementation strategies by Afya Jijini should be considered for wider dissemination? (To whom?) 3. What is the success of community volunteers in helping the program achieve its objectives? 4. Which strategies have worked in encouraging partner participation and involvement, and which did not? 66 Programmatic and management recommendations 1. Is the life of program targets appropriate, given the implementation environment and the evolution of service delivery in Kenya for HIV/AIDS, MNCH, FP/RH and Nutrition? If not, what recommendations would you make? (Probe for contractual modifications, such as life of program targets, tasks or other aspects of the contract) 2. How effective is the partnership in planning, developing implementation strategies, reviewing monitoring reports, documenting and sharing lessons learned, and making programmatic policy decisions? Provide specific recommendations to improve the quality of partnership. 3. What approaches, lessons learned from other settings, or program modifications might address these obstacles? 4. Are there any obstacles that require policy changes or dialogue? 5. Are there obstacles that threaten program performance or sustainability? 6. Are there any challenges/ limitations Afya Jijini has so far experienced that has limited ability to provide effective performance monitoring supports to sub-grantees, MOH and health facilities? Explain. What suggestions do you have for improving Afya Jijini’s M&E system? 7. To what extent USAID’s management structures (for oversight, guidance and direction on overall vision) enhanced program implementation and success? 8. What recommendations would you propose for improvement of the current Afya Jijini program design and implementation during the second half of the program phase for better results and sustainability?(Probe: Management actions from Prime and USAID) Date: Sign: AFYA JIJINI MID TERM EVALUATION Consortium Partners KII/Panel Discussion Guide Introduction & Consent Good morning/afternoon. My name is ______________________, I am an independent Consultant contracted by IBTCI on behalf of USAID/Kenya. We are working with IBTCI to conduct a mid-term review (MTR) on behalf of USAID/Kenya for the Afya Jijini Program. Your Ministry/organization/ department/ County/ Sub-county/hospital/ community unit has been selected as one of the key stakeholders of Afya Jijini program to assist with information to support the MTR. The interview will take approximately 90 minutes. We are, therefore, grateful for your participation. I will be taking notes during our discussion, however, no individual information or names will be included in the final report to ensure confidentiality. Do you agree to participate in the survey? Can I now proceed? Title of Respondent ________________________________________ 1Name of consortium Partner Organization_____________________________ 2. What is the role of your organization in the AJ strategy? 3. How long has your organization been involved in AJ project? 67 Progress towards achieving targets 4. Has the role of your organization changed over the lifetime of the project? Have you been given additional activities? Have you had any activities eliminated?(Copy of contract modification if any) 5. To what extent is Afya Jijini/Consortium partners on course to meet the life of program targets? 6. To what degree do the program outputs represent the optimal combination to address HIV/AIDS, MNCH, nutrition and FP/RH in the program’s country context? e) To what degree are planned activities and corresponding budgets adequate and appropriate to achieve targets and objectives? f) How well have the program resources been used? Are there cost-efficiencies that could be achieved? g) What opportunities exist to accelerate program impact with a modified mix of activities or funding? h) To what degree do these principles or requirements need further attention or modification during the second half of Afya Jijini program implementation? 7. Are they still relevant? Assess the degree to which the program is complying with USAID’s mandatory requirements such as visibility (branding and communication), gender integration and environmental compliance. For example, d) How is the program addressing gender inequalities/inequities in service provision, service delivery and behavior change for the beneficiaries? e) Assess the adequacy of Afya Jijini’s program gender strategy? f) To what degree is the program adhering to the implementation of the environmental compliance and mitigation plan? 8. Does services coverage (location and range of services offered) represent an efficient use of program resources? 9. How do program beneficiaries view the following: g) Hours of operation, h) Professionalism of staff, i) Quality of care, j) Confidentiality, k) Ease of accessing multiple services, l) Availability of product. 10. Describe the M&E system used by Afya Jijini in monitoring progress and trends and how you fit in. j) Does the Afya Jijini M&E system facilitate utilization of data for improved performance? (Probe for costing and completeness) k) Is the performance management plan developed and used to provide timely and high-quality data? If not, why not? (IMA) 11. Has Afya Jijini established and supported health commodity inventory management practices, commodity forecasting and supply planning at the county level? Health facilities? Explain (Meds). 12. Did consortium members develop a set of common criteria for excellence in internal consortium management, such as compliance to donors and each other and programmatic, financial and managerial accountability for service delivery to communities and/or program participants? 13. Did consortium members agree to a set of appropriate financial, administrative and managerial processes and procedures that are based on consortium needs and each agency’s strengths? Did they develop an operations manual documenting these processes and procedures to remain in compliance with the national law and donor requirements? 14. Did the consortium members define their roles based on each agency’s capacities and the program’s needs? 15. Does the consortium or partnership have guidelines for conflict resolution, communication and decision-making that reinforce transparency and accountability at all levels? How have gender- 68 equitable programming and integration been presented to consortium members, staff and partners? 16. Do consortium members conduct business in a transparent, timely and respectful fashion and work to build a consortium based on trust and mutual respect, consistently supporting positive interpersonal behavior? 17. Does the consortium have an open and transparent process for decision making and arbitration? Can it be improved? 18. Do consortium members put the needs and identity of the consortium ahead of individual organizational needs when representing the consortium? 19. Does the consortium or partnership allocate adequate resources to monitoring, evaluation, learning and knowledge management systems; support staff in learning, change and innovation; and create and sustain a culture that continually improves its management practice from lessons learned, both failures and successes? 20. Did the consortium members periodically meet to review program progress and implementation challenges? 21. Please share with me some of the strengths, weaknesses, opportunities and threats in the implementation of Afya Jijini program. Coordination and collaboration mechanism 1. What is Afya Jijini’s/consortium partners’ contribution to developing a comprehensive integrated health program in Nairobi City County? 2. How does the Ministry of Health, sub-Counties and county show signs of improving their ability to manage services and operations (managerial, financial, technical etc.)? 3. How effectively has Afya Jijini program coordinated, managed and linked sub-county, facility and community-based services? 4. Has the MOH team at the county and sub-county levels been part of the implementation of Afya Jijini program? (Probe- Does the MOH team at the county and sub-county levels feel they are part and own products and services provided by Afya Jijini so far? Explain. Does the MOH at the county level have systems in place to sustain accomplishments under Afya Jijini so far? Explain) 5. Has the creation of these partnerships (between MOH and AJ) been effective? 6. Are there best practices or approaches to developing sector linkages that should receive additional attention or resources during the second half of the program? 7. Assess Afya Jijini program’s wider participation in terms of collaboration and coordination within the partnership, other USG programs/programs, and other key donor programs/programs, and stakeholders in health service delivery arena. 8. What else is going on in the health sector/environment that might account for the changes or the program achieving its objectives? Sustainability and innovation of service delivery models 1. Are AJ/ Consortium partner supported activities likely to continue without further USG investments? • To what extent is the MoH/County team demonstrating ownership and commitment to sustaining accomplishments under Afya Jijini? • What other investments or approaches would promote services delivery program sustainability? 69 2. What are the lessons learned from interventions implemented by AJ/Consortium partners that merit continuation or replication in the next phase? • What practices, products, tools should be considered for dissemination? • What models and implementation strategies by Afya Jijini should be considered for wider dissemination? (To whom?) 3. What is the success of community volunteers in helping the program achieve its objectives? 4. Which strategies have worked in encouraging partner participation and involvement, and which did not? Programmatic and management recommendations 1. Is the life of program targets appropriate, given the implementation environment and the evolution of service delivery in Kenya for HIV/AIDS, MNCH, FP/RH and Nutrition? If not, what recommendations would you make? (Probe for contractual modifications, such as life of program targets, tasks or other aspects of the contract) 2. How effective is the partnership in planning, developing implementation strategies, reviewing monitoring reports, documenting and sharing lessons learned, and making programmatic policy decisions? Provide specific recommendations to improve the quality of partnership. 3. What approaches, lessons learned from other settings, or program modifications might address these obstacles? 4. Are there any obstacles that require policy changes or dialogue? 5. Are there obstacles that threaten program performance or sustainability? 6. Are there any challenges/ limitations Afya Jijini has so far experienced that has limited ability to provide effective performance monitoring supports to sub-grantees, MOH and health facilities? Explain. What suggestions do you have for improving Afya Jijini’s M&E system? 7. To what extent USAID’s management structures (for oversight, guidance and direction on overall vision) enhanced program implementation and success? 8. What recommendations would you propose for improvement of the current Afya Jijini program design and implementation during the second half of the program phase for better results and sustainability?(Probe: Management actions from Prime and USAID) Date: AFYA JIJINI MID TERM EVALUATION Key Informant Interview (KII)/ Panel Discussion Guide. County/ Sub-County Health Management Teams (CHMT/SCHMT) Introduction & Consent Good morning/afternoon. My name is ______________________, I am an independent Consultant contracted by IBTCI on behalf of USAID/Kenya. We are working with IBTCI to conduct a mid-term review (MTR) on behalf of USAID/Kenya for the Afya Jijini Program. Your Ministry/organization/ department/ County/ Sub-county/hospital/ community unit has been selected as one of the key stakeholders of Afya Jijini program to assist with information to support the MTR. The interview will take approximately 90 minutes. We are, therefore, grateful for your participation. I will be taking notes/recording during our discussion, however, no individual information or names will be included in the final report to ensure confidentiality. 70 Do you agree to participate in the survey? Can I now proceed? Title of Respondent ________________________________________ Progress towards achieving targets 1. Has the MOH team at the County/ Sub-County level been part of the implementation of Afya Jijini program? Explain by giving specific examples a) Does the MOH team at the county level feel they are part and own products and services provided by Afya Jijini so far? Explain 2. Can you name activities where Afya Jijini has collaborated with you? a) Out of the interventions you have listed, which ones do you consider successful? b) For the interventions that you consider successful, which ones merit continuation or replication? c) In your view, are there models and implementation strategies by Afya Jijini that should be considered for wider dissemination? Please explain. d) Which activities so far implemented by Afya Jijini are likely to continue further without USG assistance once the program is exited? Explain why. e) Can you comment on the cost-effectiveness of any of these interventions? f) Are there any interventions that you think could have been implemented differently Explain. 3. What specific products or tools (SOPs, policies, guidelines etc.) has Afya Jijini so far developed, disseminated or distributed to County, Sub-county, health facilities, CBOs or CUs? 4. Does the MOH team at the county level have systems in place to sustain accomplishments under Afya Jijini so far? Explain 5. Do you have County HRH Stakeholders’ Coordination Committees? Which ones and when were they established? a) What is the composition and membership of the HRH committee and how frequently do they meet? Kindly share/ refer to HRH committee meeting minutes. 6. Do you have a mechanism for health commodity forecasting and supply planning? Was this established with support from Afya Jijini? When was it established? 7. Has Afya Jijini established health commodity inventory management practices at the county level/ sub-county and health facility level? Explain. 8. Has Afya Jijini supported high volume health facilities in Nairobi County to conduct data quality assessments annually? If yes, how many facilities? a) How many high volume health facilities in Nairobi County have been supported by Afya Jijini to implement data quality improvement plans? 9. Out of the high volume facilities supported by Afya Jijini in Nairobi County, how many have functional referral systems? Coordination and collaboration mechanism 10. How can you describe the linkages between Afya Jijini and the CHMT/ SCHMT? a) How effectively has Afya Jijini program coordinated, managed and linked sub-county office, health facility and community based services? Has the creation of these partnerships been effective? b) Any suggestions’ on how Afya Jijini may improve or strengthen linkages between the CHMT/SCHMT and among other stakeholders for sustainability? c) Are there best practices or approaches to developing sector linkages that should receive additional attention or resources during the second half of the Afya jijini program? d) How have synergies, collaboration or coordination between Afya Jijini and different stakeholders within the MOH at the national, county and sub-county levels been perceived so far? 71 11. What is your assessment of Afya jijini’s program’s wider participation in terms of collaboration and coordination with CHMT/SCHMTS, USG programs, other key donor programs and stakeholders in health service delivery arena? 12. Has Afya Jijini program established other reasonable methods of gathering and managing data necessary to monitor and evaluate progress and indicator data? a) Comment on Afya Jijini program’s M&E Support to the CHMT, SCHMT and Health facilities. b) Are there any challenges/ limitations Afya Jijini has so far experienced that has limited its ability to provide effective performance monitoring supports to the CHM, SCHMT and health facilities? Explain 13. At this stage, do you think the implementation strategies employed by Afya Jijini have been effective and could lead to sustainability? 14. What are the main Strengths or weaknesses, opportunities and threats that have so far affected the project’s implementation and its design? 15. To what extent has USAID’s management structures (for oversight, guidance and direction on overall vision) been perceived to be effective and efficient in producing better health outcomes and accountability for results 16. Any recommendations you would propose for improvement of the current Afya Jijini program design and implementation during the second half of the program phase for better results and sustainability? 17. What are the main Strengths or weaknesses, opportunities and threats that have so far affected the project’s implementation and its design? Partnerships for Governance and Strategic Planning Services 1. Has the Afya Jijini project contributed to building your leadership and governance capacity? If so, how? If not, what more could be done to strengthen leadership and governance? 2. Has the Afya Jijini project done anything to improve leadership and governance at the health facility level? Please give examples. 3. Has the Afya Jijini project strengthened the Health Workforce Coordination Committees at County, Sub County and Facility Level? Please give examples 4. Can you provide any other examples where the project has helped you strengthen partnerships and coordination to provide better health services at the County, Sub-County or Facility level? Human Resources for Health Strengthening 1. Can you name any Afya Jijini contributions to improve workforce planning to ensure adequate number and skill mix of health workers at facility level? 2. Has Afya Jijini assisted health facilities to develop an HRH management and development system? 3. Has Afya Jijini contributed County and sub-county level Health and Pharmaceutical Technologies strengthening such as strengthening commodity management and information systems at the county, sub-country and facility levels? 4. At the facility level, has Afya Jijini contributed to improved commodity management systems management including information systems, patient safety and quality assurance? 5. Has Afya Jijini assisted you to collaborate with supply chain system strengthening implementing partners to improve commodity reporting and use of data (JSI, KEMSA, CHAI, UMB, AMREF)? Strategic M&E and Quality Improvement Systems 1. Has Afya Jijini assisted you to improve collection and use of data at the County, Sub-County and/or health facility level? If yes, please provide examples. 72 2. Has the project done any activities intended to improve Quality Improvement (QI) processes at the County Level? Sub-County Level? Facility Level? Community Level? Date: Sign: AFYA JIJINI MID TERM EVALUATION Key Informant Interview (KII)/ Panel Discussion Guide. USG Agencies: USAID Good morning/afternoon. My name is ______________________, I am an independent Consultant contracted by IBTCI on behalf of USAID/Kenya. We are working with IBTCI to conduct a mid-term review (MTR) on behalf of USAID/Kenya for the Afya Jijini Program. Your Ministry/organization/ department/ County/ Sub-county/hospital/ community unit has been selected as one of the key stakeholders of Afya Jijini program to assist with information to support the MTR. The interview will take approximately 45- 60 minutes. We are, therefore, grateful for your participation. I will be taking notes during our discussion, however, no individual information or names will be included in the final report to ensure confidentiality. Do you agree to participate in the survey? Can I now proceed? Title of Respondent ________________________________________ 1. Are there any successful interventions so far implemented by Afya Jijini ? Explain by giving specific examples. 2. Which activities that have been implemented to date by Afya Jijini are likely to continue further without USG assistance once the program has ended? Please explain why. 3. Are there any best practices or approaches to developing sector linkages that should receive additional attention or resources during the second half of the Afya Jijini program? 4. In your opinion, has Afya Jijini’s integration model worked either for or against the achievement of results in each of the program’s key service delivery programs areas (HIV/AIDS, TB, RMNCH, WASH)? Has the model resulted in any cost savings? 5. What are the main constraints or weaknesses, opportunities or threats that have so far affected the project’s implementation and its design? 6. What are your perceptions on the effectiveness and efficiencies of Afya Jijini’s prime partner, IMA’s management model? 7. What are the perceptions on Afya Jijini’s implementation strategies’ effectiveness and whether they could lead to sustainability? Can you comment on the cost-effectiveness of these strategies? 8. Are there best practices or approaches to developing sector linkages that should receive additional attention or resources during the second half of the Afya jijini program? 9. Have synergies, collaboration or coordination between different program areas and/or between different USG activities (national level mechanisms) contributed if any, to the observed health outcomes? If yes, how? If no, why not? 10. From your perspective, to what extent has USAID’s management structures (for oversight, guidance and direction on overall vision) been perceived to be effective and efficient in producing better health outcomes and accountability for results? 73 11. What are some of the programmatic and management recommendations that you would propose for to improve the current Afya Jijini program design and implementation during the second half of the program? Date: Sign: 74 AFYA JIJINI MID TERM EVALUATION Key Informant Interview (KII)/ Panel Discussion Guide. USG Agencies: CDC/DOD Good morning/afternoon. My name is ______________________, I am an independent Consultant contracted by IBTCI on behalf of USAID/Kenya. We are working with IBTCI to conduct a mid-term review (MTR) on behalf of USAID/Kenya for the Afya Jijini Program. Your Ministry/organization/ department/ County/ Sub-county/hospital/ community unit has been selected as one of the key stakeholders of Afya Jijini program to assist with information to support the MTR. The interview will take approximately 30-45 minutes. We are, therefore, grateful for your participation. I will be taking notes during our discussion, however, no individual information or names will be included in the final report to ensure confidentiality. Do you agree to participate in the survey? Can I now proceed? Title of Respondent ________________________________________ 1. Has your agency collaborated with Afya Jijini? If yes, in which areas? 2. Can you name any successful interventions that have been implemented so far by Afya Jijini? Explain by giving specific examples. If there have been successful interventions, can you comment on the cost-effectiveness of any of them? 3. Are there activities that have been implemented by Afya Jijini so far that are likely to continue without USG assistance once the program is ended? If yes, please explain why. If no, why do you think they will not continue 4. Are there best practices or approaches to developing sector linkages within the Afya Jijini project that should receive additional attention or resources during the second half of the program? 5. Has Afya Jijini’s integration model worked for and/or against the achievement of results in each of the program’s key service delivery programs areas (HIV/AIDS, TB, RMNCH, WASH)? Do you think the integration model has resulted in any combination of interventions resulting in cost￾savings? If yes, can you provide examples? 6. Have synergies, collaboration or coordination between different program areas and/or between different USG activities (national level mechanisms) contributed if any, to the observed health outcomes? If yes, how? If no, why not? Have they resulted in any cost savings? If yes, please explain. 7. Do you have any recommendations that you would propose for improvement of the current Afya Jijini program design and implementation during the second half of the program to support sustainability of any positive program outcomes? Date: AFYA JIJINI MID TERM EVALUATION Key Informant Interview (KII) Guide USG Supported mechanisms Good morning/afternoon. My name is ______________________, I am an independent Consultant contracted by IBTCI on behalf of USAID/Kenya. We are working with IBTCI to conduct a mid-term review (MTR) on behalf of USAID/Kenya for the Afya Jijini Program. 75 Your Ministry/organization/ department/ County/ Sub-county/hospital/ community unit has been selected as one of the key stakeholders of Afya Jijini program to assist with information to support the MTR. The interview will take approximately 30-45 minutes. We are, therefore, grateful for your participation. I will be taking notes during our discussion, however, no individual information or names will be included in the final report to ensure confidentiality. Do you agree to participate in the survey? Can I now proceed? Organization Name: ____________________________ Title of respondent___________________________________________________ 1. Is your organization now, or has it in the past, been involved in the implementation of activities in the AJ project? 2. How long has your Organization collaborated with Afya Jijini and in which areas? 3. Are there any successful interventions so far implemented by Afya Jijini? Was your organization involved in any of these successes? Explain by giving specific examples 4. Which activities so far implemented by Afya Jijini are likely to continue further without USG assistance once the program is exited? Explain why. 5. How would you describe the relationships between Afya Jijini and your organization/ department? 6. Comment on Afya jijini’s program’s wider participation in terms of collaboration and coordination with your organization, other USG programs and other key donor programs and stakeholders in health service delivery arena. 7. Any suggestions’ on how Afya Jijini may improve or strengthen collaboration and coordination between and among stakeholders for sustainability? 8. What are the main constraints or weaknesses, opportunities and threats that have so far affected the project’s implementation and its design? 9. Are there best practices or approaches to developing sector linkages that should receive additional attention or resources during the second half of the Afya jijini program? 10. How has synergies, collaboration or coordination between different program areas and/or between different USG activities (national level mechanisms) contributed if at all, to the observed health outcomes? 11. What recommendations would you would propose for any improvements of the current Afya Jijini program design and implementation during the second half of the program phase for better results and sustainability. AFYA JIJINI MID TERM EVALUATION Key Informant Interview (KII) Guide. National Government Ministries/ Departments Good morning/afternoon. My name is ______________________, I am an independent Consultant contracted by IBTCI on behalf of USAID/Kenya. We are working with IBTCI to conduct a mid-term review (MTR) on behalf of USAID/Kenya for the Afya Jijini Program. Your Ministry/organization/ department/ County/ Sub-county/hospital/ community unit has been selected as one of the key stakeholders of Afya Jijini program to assist with information to support the MTR. The interview will take approximately 45- 60 minutes. We are, therefore, grateful for your participation. I will be taking notes/recording during our discussion, however, no individual information or names will be included in the final report to ensure confidentiality. Do you agree to participate in the survey? Can I now proceed? 76 Title of Respondent ________________________________________ 1. How long has your Ministry/division/department collaborated with Afya Jijini and in which areas? 2. Are you aware of any sustainable and innovative service delivery models that Afya Jijini is currently implementing or has implemented? Please explain. 3. Are there any successful interventions so far implemented by Afya Jijini that merit continuation or replication? If yes, explain by giving specific examples. Can you comment on which ones are the most cost-effective? Were there some that were successful, but not cost-effective? 4. Which activities that have been so far implemented by Afya Jijini are likely to continue without USG assistance once the program is exited? Explain why. 5. Can you describe any linkages between Afya Jijini and your ministry/ division/department? Please give examples. 6. Can you describe Afya jijini program’s wider participation in terms of collaboration and coordination within the MOH at the Sub-County, County and national levels in health service delivery arena? 7. Any suggestions’ on how Afya Jijini may improve or strengthen linkages between and among stakeholders for sustainability? 8. Are there best practices or approaches to developing sector linkages that you think should receive additional attention or resources during the second half of the Afya jijini program? 9. How have synergies, collaboration or coordination between Afya Jijini and different stakeholders within the MOH at the national, county and sub-county levels been perceived so far? 10. Do you have any recommendations that you would propose for improvement of the current Afya Jijini program design and implementation during the second half of the program phase for better results, cost-effectiveness and sustainability, especially for those aspects of the program that are expected to continue after the program ends? Date: AFYA JIJINI MID TERM EVALUATION Key Informant Interview (KII) Guide. CBO In charges Good morning/afternoon. My name is ______________________, I am an independent Consultant contracted by IBTCI on behalf of USAID/Kenya. We are working with IBTCI to conduct a mid-term review (MTR) on behalf of USAID/Kenya for the Afya Jijini Program. Your Ministry/organization/ department/ County/ Sub-county/hospital/ community unit has been selected as one of the key stakeholders of Afya Jijini program to assist with information to support the MTR. The interview will take approximately 45- 60 minutes. We are, therefore, grateful for your participation. I will be taking notes during our discussion, however, no individual information or names will be included in the final report to ensure confidentiality. Do you agree to participate in the survey? Can I now proceed? Name of CBO___________________________________________________ Title of Respondent ________________________________________ 1. How long has your organization collaborated with Afya Jijini and in which areas? 2. Are there any successful interventions so far implemented by Afya Jijini that merit continuation or replication? If yes, explain by giving specific examples. 77 3. What models and implementation strategies so far implemented by Afya Jijini should be considered for wider dissemination? 4. What products or tools (SOPs, policies, guidelines etc) has Afya Jijini so far developed, disseminated or distributed to Sub-county, County, health facilities, CBOs or CUs? 5. Which activities so far implemented by Afya Jijini are likely to continue further without USG assistance once the program is exited? Explain why? 6. What kind of data management and general M&E support has your organization received from Afya Jijini? Explain. 7. Has this CBO conducted quarterly data review and use meeting(s)? When was the last time? What was the role of Afya Jijini, if any during the meeting? What is the frequency of the meetings? 8. Any suggestions’ on how Afya Jijini may improve or strengthen linkages between and among stakeholders for sustainability? 9. Are there best practices or approaches to developing sector linkages that should receive additional attention or resources during the second half of the Afya jijini program? 10. Has Afya Jijini established reasonable methods of gathering and managing data necessary to monitor and evaluate progress and indicator data including the data collected by your CBO? 11. Are there any challenges/ limitations Afya Jijini has so far experienced that has limited ability to provide effective performance monitoring supports to your organization? Explain 12. What suggestions do you have for improving Afya Jijini’s M&E system including support to partners including your organization? 13. What are the main constraints or weaknesses, opportunities and threats that have so far affected the project’s implementation and its design? 14. Recommendations you would propose for improvement of the current Afya Jijini program design and implementation during the second half of the program phase for better results and sustainability? Date: Sign: 78 AFYA JIJINI MID TERM EVALUATION Key Informant Interview (KII) Guide. Community unit (CU) In charges/CHA Good morning/afternoon. My name is ______________________, I am an independent Consultant contracted by IBTCI on behalf of USAID/Kenya. We are working with IBTCI to conduct a mid-term review (MTR) on behalf of USAID/Kenya for the Afya Jijini Program. Your Ministry/organization/ department/ County/ Sub-county/hospital/ community unit has been selected as one of the key stakeholders of Afya Jijini program to assist with information to support the MTR. The interview will take approximately 45- 60 minutes. We are, therefore, grateful for your participation. I will be taking notes during our discussion, however, no individual information or names will be included in the final report to ensure confidentiality. Do you agree to participate in the survey? Can I now proceed? Name of CU: ________________________________________________________ Title of Respondent ________________________________________ 1. How long has your CU collaborated with Afya Jijini and in which areas? 2. Has your CU seen improvements as a result of assistance from Afya Jijini? Please list what they are. 3. Are there any successful interventions so far implemented by Afya Jijini that merit continuation or replication? If yes, explain by giving specific examples. 4. What models and implementation strategies so far implemented by Afya Jijini should be considered for wider dissemination? 5. What practices implemented by Afya Jijini should be considered for wider dissemination? 6. Which activities so far implemented by Afya Jijini are likely to continue further without USG assistance once the program is exited? Explain why? 7. When was the last time this CU conducted quarterly data review and use meeting? What was the role of Afya Jijini, if any during the meeting? What is the frequency of the meetings? 8. Any suggestions’ on how Afya Jijini may improve or strengthen linkages between and among stakeholders for sustainability? 9. To what extent, if any has Afya Jijini’s work with CHWs helped them to serve as link between communities and health facilities, to ensure continuum of care from the household to health facilities;? 10. What specific capacity-building as Afya Jijini done with CHWs? Which specific interventions? 11. Who mentors or supervises the CHWs for these new skills? 12. Comment on any innovative approaches that were developed to increase the access and utilization of health services at community and facility levels. 13. Recommendations you would propose for improvement of the current Afya Jijini program design and implementation during the second half of the program phase for better results and sustainability? AFYA JIJINI MID TERM EVALUATION Key Informant Interview (KII) Guide. Health facility in- charges Good morning/afternoon. My name is ______________________, I am an independent Consultant contracted by IBTCI on behalf of USAID/Kenya. We are working with IBTCI to conduct a mid-term review (MTR) on behalf of USAID/Kenya for the Afya Jijini Program. Your Ministry/organization/ department/ County/ Sub-county/hospital/ community unit has been selected as one of the key stakeholders of Afya Jijini program to assist with information to support the 79 MTR. The interview will take approximately 45- 60 minutes. We are, therefore, grateful for your participation. I will be taking notes during our discussion, however, no individual information or names will be included in the final report to ensure confidentiality. Do you agree to participate in the survey? Can I now proceed? Name of health facility: _________________________________________________ Title of Respondent ________________________________________ 1. In which departments of your facility did Afya Jijini collaborate with you? (Please list all) 2. What specific activities has Afya Jijini been involved in your facility? Is this in service delivery, capacity building or both? HIV Services • Kindly describe the HIV package of care, support and treatment you provide at this CCC for PLHIV. Probe for clear SOC, availability of guidelines, registers and summary documents (HIV care & treatment) • What strategies has Afya Jijini supported you in to ensure ART for children, adolescents and adults? (probe for each category) • What strategies has Afya Jijini supported you to help retain children, adolescents and adults in HIV care and treatment? (probe for each category) {HIV Care & Treatment} • List the areas of support you have received from Afya Jijini in PMTCT? Probe capacity building on MTCT, sample collection, follow up mother- baby pairs {probe mentor – mothers}, cohort data registers, HTC services and linkage to CCC from PMTCT. • What are your current MTCT rates are 18months? Kindly show me your cohort registers? • What has been of most successful (probe for innovations)? What area would you require more support (probe what has not worked well) (HIV/MTCT) • Please tell me HIV prevention interventions you have for key population’s i.e. adolescent girls, sexual workers, MSMs? {HIV/Key Populations} • Has AJ provided support for VMMC service? Please be specific on the package of support for VMMC. Probe for strengths, opportunities and constraints (HIV/VMMC) • Do you have a system to identify TB in PLHIV? Describe the TB/HIV package of care. Probe for TB infection control, TB screening, TB testing, Initiation of ART, Patient monitoring at the community level , contact tracing. {HIV/TB Co infection} • How are HIV positive clients from TB clinic linked to CCC? Probe for intrafacililty referral or interfacility referral, missing patients between the clinics {HIV/TB Co infection} Integration of MNCH/HIV services • Do you have system to notify maternity on HIV+ status of pregnant to ensure PMTCT? Please describe the system. • Please describe follow‐up system for HIV (+) mothers giving birth in your facility? • Please describe system to ensure integration of HIV (+) mothers in HIV care? • Please describe follow‐up system for babies born from HIV (+) mothers in your facility? • Please describe system to ensure integration of babies born from HIV (+) mothers in HIV care if needed? • Please describe system to ensure integration of HIV (+) mothers in HIV care. 80 Health Information System • What are the major gaps in availability, completeness and accuracy of medical documentation in your facility? • Does this facility have a health information management system? Explain • Does this facility conduct data review and use meetings? How often? • What was the role of Afya Jijini, if any during the meeting? • Has Afya Jijini assisted you to improve collection and use of data? Explain. MPDSR • Does this facility have an MPDSR committee? What is the composition of the committee? • When was it established? What was the role of Afya Jijini, if any? • How often do you meet? • In the last one month how many maternal deaths have occurred and how many have been reviewed? How many perinatal deaths have occurred and how many have been reviewed? • What mechanism is in place to implement the recommendations of reviews? Quality Improvement • Does this facility have resources for process improvement? • Does your facility have QI team(s) or other internal quality improvement structures? What routine activities are the QI team involved in? • How does your facility monitor the progress of quality improvement? How frequently does your facility (or its QI team) monitor the QI progress? • Can you describe your leadership and support to quality improvement efforts in your facility. In your opinion, what are the main barriers to provide high quality medical services in your facility? • Does your facility regularly measure patient satisfaction through any method? Referrals and Linkages • Does this health facility have a functional referral systems? Has the Afya Jijini project provided any technical assistance to strengthen these systems? Explain. • If you had to urgently refer 10 patients to the hospital, on average how many from them would be able to use facility transport? • Is there an established procedure to issue referral note each time you refer the patient? What information is usually included in referral note? • Do you care for patients referred from lower level facilities? Do you think you receive enough information from referring facilities in order to provide timely care for these patients? • Do you have established communication with lower level facilities about back‐referred patients? • Does this facility have community units linked to it? If yes how many? • Has Afya Jijini been engaged in capacity-building with CHWs linked to your facility? If yes, in what health intervention areas? Who mentors or supervises the CHWs to implement any new skills? Supply chain 81 • Does this facility have an established health commodity inventory management practice? Explain. Has the Afya Jijini project provided any technical assistance to strengthen these systems? • Does this facility have a mechanism for health commodity forecasting and supply planning? When was it established? • Has Afya Jijini had any role in assisting this facility to collaborate with supply chain system strengthening implementing partners to improve commodity reporting and use of data Human Resource for Health • Does this facility have a HRH management and development system? Has the Afya Jijini project provided any technical assistance to strengthen these systems? Explain. • Has Afya Jijini contributed to improving workforce planning to ensure adequate number and skill mix of health workers at facility level? Explain. Leadership and governance • How has the Afya Jijini project contributed to building your leadership and governance capacity? What more could be done to strengthen leadership and governance? • Does this facility have a Health Workforce Coordination Committees? Has the Afya Jijini project provided any technical assistance to strengthen these committee? Explain. • Can you provide any other examples where the project has helped you strengthen partnerships and coordination to provide better health services? 3. Are there any successful interventions so far implemented by Afya Jijini that merit continuation or replication? If yes, explain by giving specific examples. 4. What practices implemented by Afya Jijini should be considered for wider dissemination? 5. Which activities so far implemented by Afya Jijini are likely to continue further without USG assistance once the program is exited? Explain why? 6. Are there best practices or approaches to developing sector linkages that should receive additional attention or resources during the second half of the Afya jijini program? 7. Any changes/ support this health facility has received through the partnership with Afya Jijini. 8. Recommendations you would propose for improvement of the current Afya Jijini program design and implementation during the second half of the program phase for better results and sustainability? Date: Sign: 82 AFYA JIJINI MID TERM EVALUATION Key Informant Interview (KII) Guide. Health care workers (CCC/ MNCH in charges/ staff) Good morning/afternoon. My name is ______________________, I am an independent Consultant contracted by IBTCI on behalf of USAID/Kenya. We are working with IBTCI to conduct a mid-term review (MTR) on behalf of USAID/Kenya for the Afya Jijini Program. Your Ministry/organization/ department/ County/ Sub-county/hospital/ community unit has been selected as one of the key stakeholders of Afya Jijini program to assist with information to support the MTR. The interview will take approximately 45- 60 minutes. We are, therefore, grateful for your participation. I will be taking notes during our discussion, however, no individual information or names will be included in the final report to ensure confidentiality. Do you agree to participate in the survey? Can I now proceed? Name of health facility: ___________________________________________________ Title of Respondent ________________________________________ 1. Please list areas where Afya Jijini has been working with you. Is this in direct service delivery, capacity building, or both? Please be specific. MNH services • Kindly describe the package of care you provide at this MCH clinic. • Has Afya Jijini provided any support in the provision of MNCH services? • What strategies has Afya Jijini supported you to help retain mothers to attend at least 4 ANC visits? • Has Afya jijini supported any innovations in this department? • What area would you require more support (probe what has not worked well) • How are HIV positive mothers linked to CCC? Probe for intrafacililty referral or interfacility referral, missing patients between the clinics HIV Services • Kindly describe the HIV package of care, support and treatment you provide at this CCC for PLHIV. Probe for clear SOC, availability of guidelines, registers and summary documents (HIV care & treatment) • What strategies has Afya Jijini supported you in to ensure ART for children, adolescents and adults? (probe for each category) • What strategies has Afya Jijini supported you to help retain children, adolescents and adults in HIV care and treatment? (probe for each category) {HIV Care & Treatment} • List the areas of support you have received from Afya Jijini in PMTCT? Probe capacity building on MTCT, sample collection, follow up mother- baby pairs {probe mentor – mothers}, cohort data registers, HTC services and linkage to CCC from PMTCT. • What are your current MTCT rates at 18months? Kindly show me your cohort registers? • What has been of most successful (probe for innovations)? What area would you require more support (probe what has not worked well) (HIV/MTCT) • Please tell me HIV prevention interventions you have for key population’s i.e. adolescent girls, sexual workers, MSMs? {HIV/Key Populations} 83 • Has AJ provided support for VMMC service? Please be specific on the package of support for VMMC. Probe for strengths, opportunities and constraints (HIV/VMMC) • Do you have a system to identify TB in PLHIV? Describe the TB/HIV package of care. Probe for TB infection control, TB screening, TB testing, Initiation of ART, Patient monitoring at the community level, contact tracing. {HIV/TB Co infection} • How are HIV positive clients from TB clinic linked to CCC? Probe for intrafacililty referral or interfacility referral, missing patients between the clinics {HIV/TB Co infection} Integration of MNCH/HIV services • Do you have system to notify maternity on HIV+ status of pregnant to ensure PMTCT? Please describe the system. • Please describe follow‐up system for HIV (+) mothers giving birth in your facility? • Please describe system to ensure integration of HIV (+) mothers in HIV care? • Please describe follow‐up system for babies born from HIV (+) mothers in your facility? • Please describe system to ensure integration of babies born from HIV (+) mothers in HIV care if needed? • Please describe system to ensure integration of HIV (+) mothers in HIV care. Referral and Linkages • Is there an established procedure to issue referral note each time you refer the patient? What information is usually included in referral note • Do you care for patients referred from lower level facilities? Do you think you receive enough information from referring facilities in order to provide timely care for these patients? • Do you have established communication with lower level facilities/communities about back‐ referred patients? • Does this facility have a community unit? Do you work closely with the Community units linked this facility? Explain. • Has Afya Jijini been engaged in capacity-building with CHWs linked to your facility? If yes, in what health intervention areas? Who mentors or supervises the CHWs to implement any new skills? Quality Improvement • When was the most recent improvement cycle (PDSA) you completed (individually or as part of the QI team) within your facility • Please briefly describe the most recent improvement (PDSA) cycle you completed within your facility related to maternal, newborn or child care (individually or as part of the QI team) • Please describe the steps you performed during the last PDSA cycle Improvement Objective Plan Do Study Act 84 Supply chain • Does this facility have an established health commodity inventory management practice? Explain. • Have you experienced stockouts of essential medicines? Please specify. How long did the stock out last? Health Information Systems • What are the major gaps in availability, completeness and accuracy of medical documentation in your facility? • How does this department management data? Explain • Has Afya Jijini assisted you to improve collection and use of data? Explain. • Does this facility conduct data review and use meetings? How often 2. What products or tools (SOPs, policies, guidelines etc.) has Afya Jijini so far developed, disseminated or distributed to Sub-county, County, health facilities, CBOs or CUs? 3. Are there any successful interventions so far implemented by Afya Jijini that merit continuation or replication? If yes, explain by giving specific examples. 4. What practices implemented by Afya Jijini should be considered for wider dissemination? 5. Any support you have received as a health care worker through the partnership with Afya Jijini.? Comment on quality and how it has impacted on your service delivery. 6. Recommendations you would propose for improvement of the current Afya Jijini program design and implementation during the second half of the program phase for better results and sustainability? Date: Sign: FOCUSED GROUP DISCUSSION (FGD) GUIDE. Target Group(s): MNCH & CCC Clients Good morning/afternoon. My name is ______________________, I am an independent Consultant contracted by IBTCI on behalf of USAID/Kenya. We are working with International Business and Technical Consultants Inc. (IBTCI) to conduct an assessment, on behalf of USAID/Kenya for the Afya Jijini Program. We are visiting selected health facilities, including this one, supported by the Afya Jijini Program. All your responses will be kept strictly confidential and your details will not feature anywhere in the report. There is no right or wrong answer. Instead, we want to get your honest opinion on the true picture of the programme; how it has been implemented, lessons learnt and areas of improvement for the future programme. The interview will take approximately 60 minutes. We are, therefore, grateful for your participation. I will be taking notes and taping our discussion for later transcription, however, as stated above, no individual information or names will be included in the final report to ensure confidentiality. Do you agree to participate in the survey? Can I now proceed? 85 If respondents agree to be interviewed, proceed with the interview. If any respondent does not agree to be interviewed, thank the respondent and excuse him/her. MNCH FGD Questions: The FGD will target clients have who received MNCH services at the facility. Explain that MNCH services include Pregnancy, Delivery and Postpartum (Mother and Child) services 1. Which MNCH services are provided by this health facility? (Probe on availability of essential newborn care and resuscitation, nutrition, safe and clean water at point of use, and prevention and management of childhood illnesses) 2. What nutrition services do you provide to pregnant women and children in this facility? (Probe: Nutritional counselling, recipes, etc.) 3. What is your opinion about the MNCH (Immunization services, FP services, ANC service etc.) and nutritional services provided? 4. Does this health facility provide reliable and consistent MNCH services? Explain. 5. What are your comments on the availability of skilled birth attendants in this health facility? 6. What FP services are available in this health facility? 7. Which family planning methods are available in this health facility? Which methods are most popular? 8. Are there any challenges or barriers to accessing and utilization of MNCH services? Explain. 9. Are there challenges you encounter in providing nutritional support and supplements? Which ones? 10. Comment on the capacity of this health facility to provide client-centered, humane and respectful care; 11. How would you describe your own health care seeking behavior? What usually prompts you to seek medical care? What about the health care seeking behavior of the general population in your neighborhood? 12. Any suggestions on how to improve nutritional support and MNCH services in this facility that may be supported by Afya Jijini in partnership with CHMT, SCHMT and the facility HMT? CCC FGD Questions: 1. Kindly comment on the availability of HIV/AIDS treatment services for the people who need the services in this facility 2. What nutrition services are provided to CCC clients in this facility? (Probe: Nutritional counselling, recipes, etc.) 3. Comment on the capacity of this health facility to provide reliable and consistent high quality HIV preventive and curative package? 4. Are there any economic or geographic barriers to accessing and utilizing HIV/AIDS care and treatment services that have been reduced as a result of accessing the services from this facility? Explain. 5. Comment on the capacity of this health facility to provide client-centered, humane and respectful HIV care. 6. Share your experience on access to resources, information and services to improve utilization of HIV/AIDS care and treatment services from this health facility; 7. What comments do you have on your own health care seeking behavior? What about the health care seeking behavior of the general population in your neighborhood? 8. Any challenges you have so far had when seeking HIV/AIDS care and treatment services from this facility? 9. Are there any gender related expectations from the community that may encourage risk taking behaviors towards HIV/AIDS among the boys and men? Explain 86 10. Any suggestions on how to improve HIV/AIDS care and treatment services in this facility that may be supported by Afya Jijini in partnership with CHMT, SCHMT and the facility HMT? 87 FOCUSED GROUP DISCUSSION (FGD) GUIDE Target Group(s): CBO/ Community Units (CUs) Good morning/afternoon. My name is ______________________, I am an independent Consultant contracted by IBTCI on behalf of USAID/Kenya. We are working with International Business and Technical Consultants Inc. (IBTCI) to conduct an assessment, on behalf of USAID/Kenya for the Afya Jijini Program. We are visiting selected community-based organizations (CBOs) and community units (CUs) in partnership with Afya Jijini and yours has been selected. All responses will be kept strictly confidential and your details will not feature anywhere in the report. There is no right or wrong answer. Instead, we want to get your honest opinion on the true picture of the programme; how it has been implemented, lessons learnt and areas of improvement for the future programme. The interview will take approximately 60 minutes. We are, therefore, grateful for your participation. I will be taking notes and taping our discussion for later transcription, however, as stated above, no individual information or names will be included in the final report to ensure confidentiality. Do you agree to participate in the survey? Can I now proceed? If respondents agree to be interviewed, proceed with the interview. If any respondent does not agree to be interviewed, thank the respondent and excuse him/her. Community Based Organizations (CBOs) Targeting 15-24 year old youths that participated in HIV prevention program activities 1. Have you received any inputs or support from the implementation of this program? If so, please specify. 2. Are there any social, economic, and geographic barriers that are impeding access and utilization of HIV prevention services? Explain 3. Are there cultures or behaviors that discourage youth from accessing health services? Explain 4. Are there any barriers to access of HIV health services among the youth? Explain 5. Explain if there are any innovative approaches so far developed or being used to increase the use of quality HIV prevention services at community and facility levels? 6. Explain any activities currently implemented through Afya Jijini support that are likely to continue even without their assistance if they were to leave. 7. Are there any successful interventions so far implemented by your CU through support of Afya Jijini that can be replicated elsewhere? Explain by giving specific examples. 8. Share any other lessons learnt so far during the implementation of your prevention activities? Explain. 9. Please share your suggestions on how Afya Jijini may improve or strengthen linkages between and among stakeholders for sustainability? 10. What should Afya Jijini do differently from now onwards to help you get better results in your prevention activities? Community Units (CUs) CHVS/CHVs supporting HIV and non-HIV related community work within the CU. 1. Have you received any support (training, mentorship, materials etc.) from Afya Jijini to help in data collection, management or reporting? 2. Comment on the linkages between Afya Jijini and your community unit. Is there any value addition to the linkages? 88 3. Do you have any suggestions on how Afya Jijini may improve or strengthen linkages between and among stakeholders for sustainability? Explain 4. Are there any documents (SOPs, guidelines, tools etc.) developed by Afya Jijini that you are using to support implementation of your activities? Explain. 5. Explain any activities currently implemented through Afya Jijini support that are likely to continue even without their assistance if they were to leave. 6. Are there any successful interventions so far implemented by your CU through support of Afya Jijini that can be replicated elsewhere? Explain by giving specific examples 7. Share any other lessons learnt so far during the implementation of your prevention activities? Explain. 8. What should Afya Jijini do differently from now onwards to help you get better results in your work? 89 County Institutional Capacity Assessment Tool (updated May 18, 2018) A. LEADERSHIP AND GOVERNANCE 1. What successes and challenges, if any, have you experienced in implementing the county health strategic plan? 2. What is the role of Afya Jijini in developing the plan and contributing to its achievement? 3. What additional capacity would strengthen implementation across the county (probe for capacity in individual knowledge, skills, behaviors and attitudes as well as the structures, policies, systems and procedures of the organization and system as a whole)? 4. Briefly describe the communication strategy of the county. What mechanisms/tools exist for communication within each department? (Probe for between departments, with County Assembly Health Committees etc) 5. What mechanisms/tools exist for communication between county and health development partners and/or implementing partners like Afya Jijini? 6. Do we have any form of agreements between county and health development partners and/or implementing partners that support delivery of health services? Is there a policy to guide collaborations? Please describe. 7. What mechanisms are in place to promote regular dialogue between County Health Department leadership and the different health actors such as health development partners, implementing partners, MCAs, religious/community leaders, private sector and sub-county health administrators? 8. Who is involved in budget formulation and their role, health sector performance reviews? 9. What are the strategies for building leadership capacity of health care managers and practitioners at the county and sub-county level? B. HEALTH WORKFORCE 10. Briefly describe County Health Department’s strategy for health work force attraction, recruitment and retention at all levels? a) Do you have an operation plan to attract and recruit new workforce? Please describe. b) Has the county reached any agreements/ contracts with pre-service institutions to train and recruit new workforce? Please describe c) Has county conducted periodic assessments of workforce needs and priorities? Please describe. 11. Briefly describe the County Health Department’s strategy to mobilize and distribute health workforce based on each sub-county’s and health facilities’ needs. a) How are the needs assessed? b) Who is involved in the needs assessment? c) How often is a workforce needs assessment conducted? 12. Briefly describe the County Health Department’s health work force planning. a) How has the county adopted staffing based on norms, standards and guidelines? b) What strategies are being used in the mobilization of resources to meet staffing gaps? c) Does the county have a mechanism in place to measure on regular basis the staffing gaps at all levels of health care delivery? explain 13. Briefly describe the mechanisms in place to promote accountability and transparency in the workforce. a) Are there clear guidelines in the job descriptions about staff roles and responsibilities? Please describe one or more? b) How often are these guidelines reviewed and implemented? 14. What mechanisms are in place to address workforce absenteeism and poor productivity? 15. What types of trainings have been provided by the county in the past year? EXCLUDING vertical programs and implementing partners. a) Who were trained? 90 b) Who determines the staff to be trained? c) How were the training needs identified? d) Who initiated/ requested the training? e) Who conducted the training? f)How was the training funded? 16. Please describe the county health department’s policy to strengthen existing workforce through vertical programs. a) Is there an operational plan for in-service training? b) How are in-service training needs identified? c) How often are in-service trainings delivered? Is there an operation plan to retain existing workforce? d) Do county health staff that complete requisite in-service trainings get incentives? 17. Does the county health department have a centralized Training Unit to address training needs for the county health staff? How is training currently coordinated and documented? a) How are training needs and training programs or opportunities matched? b) What records are kept on in-service training for individual health workers? c) What do you think are the major pre-service training problems facing the county? d) What do you think are the major in-service training problems facing the county? e) What kind of assistance does the county need to coordinate and document training? 18. What are three priority performance areas most in need of strengthening within the county health department that relate to HRH? 19. What are the successes and major challenges for strengthening health workforce? (probe for each vertical program (HIV/AIDS, TB/HIV, RMCH, Nutrition) and the county as a whole)? C. HEALTH INFORMATION SYSTEMS 20. Does the county have an integrated Health Information System that includes indicators, data elements and sources, frequency of collection, data flow, data validation rules and quality assessment guidance/protocol? 21. How has this system been rolled out to sub-counties and facilities? 22. Does the county have a system for monitoring and evaluation of county programs that details priority health impact and outcome level indicators at a minimum that presents plans on how data will be collected for monitoring, evaluating, disseminating and using analyzed data, that clearly spells out roles and responsibilities, capacity building and county stakeholders’ data review forums? 23. How has this plan been rolled out to sub-counties and facilities? 24. Who has the primary responsibility for collecting data for routine health information, vital statistics, disease surveillance and health surveys systems? 25. Who has the primary responsibility for submitting/entering data and validating it from these data systems? 26. To what extent has the county health department institutionalized Ministry of Health’s National Data Quality Protocol and Standards? 27. What is the process for data quality assessment and how often is it conducted by county health department? By Sub-county health administrators’ offices? 28. Where is health data stored at the county and sub-county levels? 29. How often is routine health data analysis presented to senior managers for discussion, field monitoring/supportive supervision, problem solving and decisions? 30. How often is performance information presented to County Health Department leadership for discussion, problem solving and decision making? Provide examples of how reviewed performance data have been used to identify opportunities to improve services. 31. How often is health data used in reviewing/evaluating the success and/or failure of county health programs and strategies? 91 32. How often is health data used in the formulation of policy and/or incremental re-adaptation of existing programs and strategies? 33. What role does the CHMT play in promoting and/or facilitating the use of health data for management decision making at county level? D. ACCESS TO ESSENTIAL MEDICINES & OTHER HEALTH COMMODITIES 34. Describe the procedures for implementing and supervising supply chain services in the county? a) Describe the way through which the county ensures availability and use of required guidelines, protocols and tools for product selection, quantification, commodity reporting, use, support supervision and M&E at all levels of service delivery in the county? b) Briefly describe how supply chain data is used to help decision making at county/sub-county and facility level; and how the county ensures that systems for collecting data from lower levels and feedback loop from higher levels is in existence, adequate and continuously being improved. c) Does the process of supportive supervision for service delivery incorporate supervision for supply chain service/commodity management at health facility level? Explain. 35. Describe the procedures for monitoring and reporting supply chain performance at all levels in the county? a) In which specific ways does the county take a whole-market approach in strengthening commodity management systems for the county? (ie inclusion of non-government health sub￾sector (eg faith-based )that offer services within the county) b) How does the county ensure trend graphs on key supply chain performance indicators are maintained as a measure of quality of supply chain services rendered in the county? eg stock￾out rates, stocking according to plan, reporting rates, and commodity disposal due to expiration. c) How is equity ensured in commodity distribution and dispensing? In other words, what procedures are used to make sure that essential medicines and health commodities are distributed/ issued out according to need? d) How does the county ensure improved access to quality and affordable essential medicines and other health commodities? (Consider systems for commodity quantification and supply planning, inventory management tools, commodity information management, commodity financing and procurement, and financing for continuous improvement of supply chain systems) 36. How are commodity needs identified? a) How are the county, sub-county and health facility needs identified? b) What role does National Government agencies/institutions play in assessing county commodity needs? c) What happens after commodity needs are identified? How are requests made? 37. What is the role of development partners and CHMT for health and/or implementing partners like Afya Jijini in procuring essential medicines? 38. What is the proportion of county spending on commodities as % of total county health spending? 39. Describe the procedures adopted for proper storage of essential medicines and other health commodities(county, sub-county and health facilities) 40. What is the role of community-based groups and networks in community commodity distribution? 41. What mechanisms does the county use to assure quality for medicines and other health commodities within the county level? 42. Does the county have in place a pharmacovigilance system? If so, since when? If not, is there a plan to develop/put in place such a system? Please describe. 43. What systems does the county have in place for medical waste management? 92 E. DELIVERING ESSENTIAL HEALTH SERVICES 44. What mechanisms are in place to involve community stakeholders, sub-county health officers and partners in planning for service delivery? 45. Has the county conducted a formal exercise to plan for health services? a) How often is planning conducted? b) Is there a general Annual Work Plan? c) Do you have unit-specific and or Vertical Programs specific Annual Work Plans? How were they developed and shared? d) Who is involved in the planning process? e) How is the planning process organized? 46. How are priority service areas identified? a) Is service delivery reflective of priority health needs per county health strategic plan? b) What policies do you have in place to ensure service delivery targets priority health needs? Please describe. 47. What mechanisms exist in place for supervision of sub-county health facilities? a) Is supervision focused on medical audits or coaching and performance improvement or both? b) How often is supervision conducted? c) How are supervision needs determined? (needs-based or regularly scheduled?) d) Who conducts the supervision visits? e) Is there clarity about levels of supervision (who supervises who) and reporting? f) What tools are used to conduct supervision? g) How is supervision findings used? h) Are supervision results linked to any type of reward/recognition/incentives system? i) What are the challenges to conducting supervision? 48. What mechanisms exist for improving quality of care through the health system? What are the gaps in quality of care in the system? What are some of the successes in improving quality of care? a) What indicators are used to measure service quality? b) What kind of mechanism exists to assess quality of care regularly and who is in charge to monitor this? c) Are there QI teams in place at the community, facility and/or sub-county levels? d) How is county supporting QA/QI in the private sector? 49. What is the county’s capacity towards delivering Essential Health Services Package (EHSP)? 50. Which services are the strongest? 51. Which services present the most challenges? 52. How do you identify targets? Please list some of your targets. (probe for Maternal and new born, child health, FP/RH, HIV/AIDS) 53. Where are you with your targets for maternal and newborn services 54. What assistance do you need to reach your targets? 93 USAID-AFYA JIJINI-HCW KAP-May-18 Organization: International Business & Technical Consultants Inc. Report Date: 5/15/2018 3:07:49 PM FACILITY IDENTIFICATION Question ID Question Answer 1 Facility name: Babadogo HC Bahati HC Embakasi Medical Center Huruma Dispensary Kangemi Health Center Kariobagi HC Kasarani Health Centre Lianas Clinic Dispensary Makadara HC Mathare North HC Mbotela Clinic Mukuru HC St Clare Medical St Mary immaculate St Mary Mission Hospital St Peter Dispensary 2 Facility type COUNTY HOSPITAL SUB-COUNTY HOSPITAL HEALTH CENTRE DISPENSARY MISSION HOSPITAL CHURCH HOSPITAL OTHER (SPECIFY) 3 Type of sector GOVERNMENT FAITH-BASED/MISSION/CHURCH OTHER SPECIFY 4 Facility location (Locality) RURAL URBAN 5 County 6 Sub-county Langata Kasarani Embakasi South Embakasi North Makadara Mathare Ruaraka Westlands 7 Village/ neighborhood 8 Today's Date. 9 Respondent Name Knowledge Attitude and Practice questionnaire for health care providers Question ID Question Answer 10 If you voluntarily agree to participate Yes No 11 REASONS FOR REFUSAL OUTRIGHT REFUSAL THE INTERVIEW IS TOO LONG 94 HAVE OTHER COMMITMENTS SERIOUSLY ILL OTHER(SPECIFY) 12 Respondent groups Health worker in CCC Health worker in MNCAH SECTION 1: General Module Question ID Question Answer 13 GM1. Your age in years: 14 GM 2. Your gender: MALE FEMALE 15 GM3. Your Specialty: a) Obstetrician/Gynecologist b) Pediatrician/Neonatologist c) Doctor(other) d) Medical Officer e) Clinical officer f) Nurse 16 GM4. In Which year did you complete your clinical training? 17 GM5. In which year did you start working at this facility? 18 GM6. As part of your work in this facility, which services are you providing today? a) Adolescent Services b) Family Planning c) Antenatal Care d) Labor and delivery (maternity) services e) Newborn services f) Child welfare (immunizations, growth monitoring) services g) Nutrition information and counselling h) Comprehensive care for HIV SECTION 2: Adolescent services Question ID Question Answer 19 AD1. Please check all services you provide to adolescent, for any of the following conditions or needs for information counseling and clinical management in the following areas: (a) Normal growth and pubertal development (b) Pubertal delay (c) Precocious puberty (d) Mental health and mental health problems (e) Nutrition, including Anemia (f) Physical activity (g) Adolescent‐specific immunization (h) Menstrual hygiene and health (i) At least one Family planning and contraception – oral contraceptive pills, IUDs, condoms, emergency contraceptive pills, implants, injectable contraceptives (j) Safe abortion (where legal), and post‐ abortion care (k) Antenatal care and emergency preparedness, delivery and postnatal care (l) Reproductive tract infections/ sexually transmitted infections (m) HIV (n) Sexual violence (o) Family violence (p) Bullying and school violence 95 (q) Substance use and substance use disorders (r) Injuries (s) Skin problems (t) Chronic conditions and disabilities (u) Endemic diseases (v) Common conditions during adolescence (fatigue, abdominal pain, diarrhea, headache) 20 (a) Normal growth and pubertal development Information Counselling Clinical Management Referral Guidelines / decision support tools NONE 21 (b) Pubertal delay Information Counselling Clinical Management Referral Guidelines / decision support tools NONE 22 (c) Precocious puberty Information Counselling Clinical Management Referral Guidelines / decision support tools NONE 23 (d) Mental health and mental health problems Information Counselling Clinical Management Referral Guidelines / decision support tools NONE 24 (e) Nutrition, including Anemia Information Counselling Clinical Management Referral Guidelines / decision support tools NONE 25 (f) Physical activity Information Counselling Clinical Management Referral Guidelines / decision support tools NONE 26 (g) Adolescent‐specific immunization Information Counselling Clinical Management Referral Guidelines / decision support tools NONE 27 (h) Menstrual hygiene and health Information Counselling Clinical Management Referral Guidelines / decision support tools NONE 96 28 (i) Modern methods of family planning e.g. oral contraceptive pills, IUDs, condoms, emergency contraceptive pills, implants, injectable contraceptives Information Counselling Clinical Management Referral Guidelines / decision support tools NONE 29 (j) Safe abortion (where legal), and post‐ abortion care Information Counselling Clinical Management Referral Guidelines / decision support tools NONE 30 (k) Antenatal care and emergency preparedness, delivery and postnatal care Information Counselling Clinical Management Referral Guidelines / decision support tools NONE 31 (l) Reproductive tract infections/ sexually transmitted infections Information Counselling Clinical Management Referral Guidelines / decision support tools NONE 32 (m) HIV Information Counselling Clinical Management Referral Guidelines / decision support tools NONE 33 (n) Sexual violence Information Counselling Clinical Management Referral Guidelines / decision support tools NONE 34 (o) Family violence Information Counselling Clinical Management Referral Guidelines / decision support tools NONE 35 (p) Bullying and school violence Information Counselling Clinical Management Referral Guidelines / decision support tools NONE 36 (q) Substance use and substance use disorders Information Counselling Clinical Management Referral Guidelines / decision support tools NONE 37 (r) Injuries Information 97 Counselling Clinical Management Referral Guidelines / decision support tools NONE 38 (s) Skin problems Information Counselling Clinical Management Referral Guidelines / decision support tools NONE 39 (t) Chronic conditions and disabilities Information Counselling Clinical Management Referral Guidelines / decision support tools NONE 40 (u) Endemic diseases Information Counselling Clinical Management Referral Guidelines / decision support tools NONE 41 (v) Common conditions during adolescence (fatigue, abdominal pain, diarrhea, headache) Information Counselling Clinical Management Referral Guidelines / decision support tools NONE 42 AD2. Have you ever received the following training in adolescent health care? (a) Communication skills to talk to adolescents? (b) Communication skills to talk to adult visitors/community members (c) The policy on privacy and confidentiality (d) Clinical case management of adolescent patients? (e) Orientation on the importance of respecting the rights of adolescents to information and health care that is provided in a respectful, non‐judgmental and non‐ discriminatory manner? (f) Policies and procedures to ensure free or affordable service provision? (g) Data collection, analysis and use for quality improvement? (g) None 43 AD3. Are you aware of the following SOPs/ guidelines For services that should be provided in the facility and in the community? (a) Referral guidelines/SOPs? (b) Policy/SOPs for a planned transition from pediatric to adult care? (c) Guidelines/SOPs on informed consent? (d) Guidelines/SOPs on providing services to all adolescents irrespective of their ability to pay, age, sex, marital status or other characteristics? (e) Guidelines/SOPs on providing free, or 98 affordable, services to adolescents? (f) Guidelines/SOP on measures to protect the privacy and confidentiality of adolescents? (g) Not Aware 44 AD5. Do you think the working hours in this facility are convenient for adolescents? Yes No 45 AD6. Have you ever trained any of the following groups in these areas? (a) Outreach workers in adolescent health care? (b) Adolescents in providing certain services, for example, health education for peers, counselling? (c) Never trained 46 AD11. Have you ever involved any of the following groups in these activities? (a) Adolescents in the planning, monitoring and evaluation of health services? (b) Adolescents in any aspects of service provision? (c) Vulnerable groups of adolescents in the planning, monitoring and evaluation of health services and service provision? (d) Never PRACTICES Question ID Question Answer 47 ADP1. When you see an adolescent client for services or counselling do you: 48 (a) Introduce yourself first to the adolescent? Always Most of the times Sometimes Never 49 (b) Ask the adolescent what he/she likes to be called? Always Most of the times Sometimes Never 50 (c) Ask the adolescent who he/she has brought with him/her to the consultation? Always Most of the times Sometimes Never 51 (d) Explain to adolescents that are accompanied that you routinely spend some time alone with the adolescent towards the end of the consultation? Always Most of the times Sometimes Never 52 (e) Ask the adolescent permission to ask the accompanying person(s) their opinions/observations? Always Most of the times Sometimes Never 53 (f) Obtain, in cases when an informed consent from a third party is required, the adolescent’s assent to the service/procedure? Always Most of the times Sometimes Never 54 (g) Ensure that no one can see or hear the adolescent client from outside during the consultation or counselling? Always Most of the times Sometimes Never 55 (h) Ensure that there is a screen between the consultation and examination area? Always Most of the times 99 Sometimes Never 56 (i) Assure the adolescent client that no information will be disclosed to any one (parents/other) without his/her permission? Always Most of the times Sometimes Never 57 (j) Explain to the adolescent client the conditions when you might need to disclose information, such as in situations required by law,1 and if that is the case you will inform him/her of the intention to disclose unless doing so would place them at further risk of harm? Always Most of the times Sometimes Never 58 (k) Keep all records/lab test reports under lock and key or password protected if in the computer? Always Most of the times Sometimes Never 59 ADP2. During a consultation with an adolescent client, do you routinely take a psychosocial history such as: 60 (a) Asking the adolescent questions about home and relationships with adults? Always Most of the times Sometimes Never 61 (b) Asking the adolescent questions about school? Always Most of the times Sometimes Never 62 (c) Asking the adolescent questions about his/her eating habits? Always Most of the times Sometimes Never 63 (d) Asking the adolescent questions about sports or other physical activity? Always Most of the times Sometimes Never 64 (e) Asking the adolescent questions about sexual relationships? Only adolescents of an appropriate age. Always Most of the times Sometimes Never 65 (f) Asking the adolescent questions about smoking, alcohol or other substances? Always Most of the times Sometimes Never 66 (g) Asking the adolescent questions about how happy he/she feels, or other questions about his/her mood or mental health? Always Most of the times Sometimes Never 67 ADP3. Would you provide the following services to all adolescents regardless of sex, age, marital status or ability to pay? (a) Hormonal contraceptives (b) Condoms (c) STI treatment (d) HIV testing and counselling (e) Medical termination of pregnancy/abortion (where legal) (f) None 100 68 ADP4. How confident do you feel about your knowledge of how to provide care to adolescents? 1) Confident 2) Somewhat/not confident 3) Not confident 69 ADP5. Has any adolescent been denied services within last 12 months because of (a) Recent stock‐outs? (b) Malfunctioning/unavailable equipment? (c) Other (specify) SECTION 3: Family Planning Question ID Question Answer 70 FP 1: Which contraception methods do you provide to women in this facility? a) Combined oral contraceptive pills b) Progestin‐only contraceptive pills c) Combined injectable contraceptives d) Male condoms e) Female condoms f) Intrauterine contraceptive device g) Menstrual hygiene and health h) Implant i) Emergency contraceptive pills 71 FP2. Have you ever received the following training or refresher training in Family Planning? a) Implant insertion b) IUCD insertion c) None 72 FP3. Are you aware of the following SOPs/ guidelines for provision of Family planning services in the facility and in the community? a) Screening for clients who want to initiate Combined oral contraceptives b) Screening for clients who want to initiate DMA/Net-en c) Screening for clients who want to initiate contraceptive implants d) Screening Clients Who Want to initiate Use of the Copper IUCD e) How to Be Reasonably Sure a Client is Not Pregnant f) Screening for cervical cancer g) WHO Medical Eligibility criteria for contraceptive use h) Postnatal family planning timeline i) Not aware 73 FP4. Do you use guidelines or decision support tools, for example, job aids or algorithms, for information, counselling and provision of the following methods? 74 (a). Combined oral contraceptive pills Information Counselling Provide method Referral None 75 (b). Progestin-only contraceptive pills Information Counselling Provide method Referral None 76 (c). Combined injectable contraceptives Information Counselling Provide method Referral 101 None 77 (d). Male condoms Information Counselling Provide method Referral None 78 (e). Female condoms Information Counselling Provide method Referral None 79 (f). Intrauterine contraceptive device Information Counselling Provide method Referral None 80 (g). Menstrual hygiene and health Information Counselling Provide method Referral None 81 (h). Implant Information Counselling Provide method Referral None 82 (i). Emergency contraceptive pills Information Counselling Provide method Referral None 83 FP5. On a scale of 1 to 5, please rate the effectiveness of the following FP options (rate 1 for the least effective and 5 to the most effective, do not repeat the numbers) (a) Withdrawal (b) Hormonal Implants (c) Combined Injectable Contraceptives (CICs) (d) Standard Days Method (e) Diaphragm 84 FP6. Based on you understanding, please indicate whether the following statements are true or false about emergency contraception pills (ECP) 85 (a). It is the only methods that can help prevent pregnancy after a woman has had unprotected sex. True Not True 86 (b). Is safe for regular use True Not True 87 (c). A woman using ECP repeatedly should receive additional family planning counseling to select the most appropriate continuous method. True Not True 88 (d). Is ineffective after 36 hours of unprotected sex True Not True 89 (e). Is safe to use in postpartum period True Not True 90 (f). Must be used within 5 days (120 hours) True 102 of unprotected sex. Not True 91 (g). It cannot be used in rape victims True Not True 92 (h). Does not disrupt existing pregnancy True Not True 93 (i). Is not safe for a woman living with HIV/AIDS True Not True 94 (j). Cannot be used together with antiretroviral (ARV) medicines True Not True SECTION 4: Ante Natal Care Question ID Question Answer 95 ANT1. Which services do you provide to pregnant women in this facility? (a) Nutritional assessment of the pregnant woman (b) Performing RDT for Malaria (c) Testing for HIV infection (d) Distribution of Nutrition Supplements (Iron, Folic acid) (e) Pelvic examination (f) Cervical Cancer screening (g) Immunization services (h) IV administration of medications (i) Initial treatment for pre‐eclampsia (j) Intermittent Prevention of Malaria in pregnancy (k) Provision of ART for PMTCT 96 ANT2. Which trainings have you received last 12 months? (a) Focused Antenatal Care (b) Obstetric Emergency training for complications of pregnancy and their management (c) Updated WHO recommendations on positive pregnancy experience (d) Screening and Management of Pre‐ eclampsia (e) Screening and Management of Gestational Diabetes (f) Screening for HIV infection and prevention of Mother to Child Transmission (PMTCT) (g) Prevention and referral for preterm birth (h) Danger signs during pregnancy, labor and childbirth Education (i) Nutritional assessment of the pregnant woman (j) Nutrition and Lifestyle Counseling for ANC (k) Birth Preparedness Counselling (l) Clinical enquiry on Intimate Partner Violence and Domestic violence (m) Supportive response for Intimate Partner Violence (n) Malaria in Pregnancy (o) interpersonal communication/counselling skills/cultural competence (p) None of the above 97 ANT3. Do you use guidelines or decision (a) Routine screenings of antenatal care 103 support tools, for example, job aids or algorithms, for information, counselling and clinical management in the following areas: (b) Screening and Management of hypertension (c) Screening and Management of proteinuria (d) Prevention and treatment of malaria (e) Screening and Management of Gestational Diabetes (f) Screening and Management of Preeclampsia (g) Screening and Management of preterm labor (h) Prevention of Mother to Child Transmission of PMTCT 98 ANT4. How comfortable are you to independently manage or decide on referral for the following conditions? 99 (a) Screening and Management of hypertensive disorders Very comfortable Somewhat Comfortable Not comfortable 100 (b) Screening and Management of high Urinary Protein Very comfortable Somewhat Comfortable Not comfortable 101 (c) Screening and Management of high Blood Glucose Very comfortable Somewhat Comfortable Not comfortable 102 (d) Screening and modification of behavioral risk factors Very comfortable Somewhat Comfortable Not comfortable 103 (e) Fever and Infections Very comfortable Somewhat Comfortable Not comfortable 104 (f) Vaginal Bleeding Very comfortable Somewhat Comfortable Not comfortable 105 ANT5. Are you aware of the following policies, protocols, SOPs/guidelines? (a) Routine ANC Screenings (b) Task shifting to provide quality Antenatal Care (c) Counseling on Intimate Partner Violence (d) Sexual and Reproductive Health Policy (e) Not aware KNOWLEDGE Question ID Question Answer 106 ANK1. How many basic visits should a pregnant woman have? 107 ANK2. Which dietary supplements are recommended for all pregnant women according to WHO? (a) Iron (b) Vitamin C (c) Folic acid (d) Calcium (e) Vitamin A (f) Zinc (g) Multiple Micronutrients (h) Vitamin B6 (i) Vitamin E (j) Vitamin D 108 ANK3. What will be your approach to a) Two doses of a tetanus toxoid‐containing 104 prevent maternal and neonatal tetanus in pregnant women who never had any vaccination with toxoid‐containing vaccine? vaccine (TT‐CV) one month apart with the second dose given at least two weeks before delivery. (b) Three doses of a tetanus toxoid‐ containing vaccine (TT‐CV) one month apart with the last dose given at least two weeks before delivery. (c) One dose as soon as possible, the second 6 months after delivery and two more doses, in the two subsequent years or during two subsequent pregnancies. (d) One dose of a TT‐CV during each subsequent pregnancy to a total of five doses PRACTICES Question ID Question Answer 109 ANP2. What is your practice regarding ANC patient notes? (a) Patient notes are usually kept in the facility organized in systemic way, that makes them easily accessible during next visits (b) Patient notes are kept in the facility; however, we open new one at each visit as it is difficult or impossible to find the previous records (c) The only facility held medical information is record in the registers (d) Each pregnant woman carries her own Patient notes during pregnancy (e) Other specify 110 ANP3. Which Conditions do you routinely screen at least once during ANC visits (a) Anemia (b) Asymptomatic bacteriuria (c) Pregnancy-related Hypertension (d) Proteinuria during the pregnancy (e) Gestational diabetes mellitus (f) Tobacco use (g) Exposure to second‐hand smoke (h) Substance use (i) HIV Infection (j) Syphilis (k) Tuberculosis (l) Malaria (m) Helminthiasis (deworming) (n) Iron insufficiency (o) Intimate partner violence 111 ANP4. If you screen for intimate partner/domestic violence do you think the privacy and time allocated for the visit are adequate for disclosure? 1) Yes 2) No 3) N/A (I never screen for IPV/DV) 112 ANP5. If patient confirms intimate partner violence do you have referral system in place to support her? Yes No 113 ANP6. What fetal assessment you routinely perform on at least one ANC visits (a) abdominal palpation for the assessment of fetal growth (b) symphysis‐fundal height (SFH) measurement for 105 the assessment of fetal growth (c) Routine antenatal cardiotocography (d) ultrasound scan before 24 weeks of gestation (e) Doppler ultrasound of fetal blood vessels 114 ANP7. Which Measures do you routinely offer for all pregnant women to prevent malaria: (a) No routine interventions except health education (b) Performing Rapid Diagnostic Test for Malaria (c) Distributing Insecticide treated nets at ANC visit (d) Intermittent preventive treatment with sulfadoxine‐pyrimethamine (IPT‐SP) 115 ANP8. When you usually start IPT‐ SP a) At the first ANC visit b) As early as possible in the second trimester c) As early as possible in the third trimester d) At 20th week of gestation e) Other specify 116 ANP9. What is the spacing the treatment doses in IPT‐SP (a) At least a week (b) At least 14 days (c) At least a month (d) At least 6 weeks 117 ANP10. What is your approach for the management of physiologic symptoms of pregnancy? (a) I do not spend much time on them as they are benign self‐limited conditions (b) I never screen for them unless pregnant woman presents as complaint, in this case I take time to ensure them that they are not dangerous for her or for their future baby. (c) I routinely screen for these symptoms as they might be important for the pregnancy experience (d) I routinely screen for these symptoms as can point to the potential pregnancy complications (e) The management options of these symptoms, should be strictly based on the evidence of their effectiveness (f) The management options of these symptoms, should be strictly based on the evidence of their effectiveness and local availability (g) The management of these symptoms based on women’s preferences SECTION 5: Elimination of Mother to Child transmission of HIV Question ID Question Answer 118 MTCT1. Please select all services you provide to pregnant HIV women in this facility? (a) HIV testing (b) Nutritional assessment and distribution of nutrition supplements (c) Recognizing and responding to danger signs during pregnancy, (d) Birth preparedness including skilled birth attendance, (e) Post-natal care including immunization, family planning and maternal and infant nutrition; (f) HIV prevention and treatment (g) Health checks to treat and prevent new infections during pregnancy, (h) ART for those who are HIV 106 positive (i) ART monitoring of ART (j) ARV prophylaxis and follow-up for HEIs (k) HIV Prevention interventions for special populations (l) Screening for Gender Based Violence (GBV) 119 MTCT2. Please Indicate all trainings you received last 12 months? (a) Focused Antenatal Care (b) Updated WHO recommendations on positive pregnancy experience (c) Screening for HIV infection and prevention of Mother to Child Transmission (PMTCT) (d) Danger signs during pregnancy, labor and childbirth Education (e) Nutritional assessment of the pregnant woman (f) Birth Preparedness Counselling (g) Clinical enquiry on Intimate Partner Violence and Domestic violence (h) Supportive response for Intimate Partner Violence (i) interpersonal communication/counselling skills/cultural competence 120 MTCT3. Do you use guidelines or decision support tools, for example, job aids or algorithms, for information, counselling and clinical management in the following areas? 121 a). Focused Antenatal Care Yes No Don't Know 122 b). HIV Testing & Counselling Yes No Don't Know 123 c). Positive Health, Dignity & Prevention Yes No Don't Know 124 d). Prevention and treatment of malaria Yes No Don't Know 125 e). Prevention of Mother to Child Transmission of PMTCT Yes No Don't Know 126 MTCT4.Please describe your personal level of comfort to independently carry out the following tasks and / or manage the following conditions: 127 (a). HIV Testing & Counselling of Pregnant Women Very comfortable Somewhat Comfortable Not comfortable 128 (b). Deliver the Positive Health, Dignity and Prevention Package Very comfortable Somewhat Comfortable 107 Not comfortable 129 (c). Clinical evaluation of Pregnant Woman Very comfortable Somewhat Comfortable Not comfortable 130 (d). Appropriate Prevention and Treatment of infectious disease in Pregnant women Very comfortable Somewhat Comfortable Not comfortable 131 (e). Management of HIV Exposed Infant Very comfortable Somewhat Comfortable Not comfortable KNOWLEDGE Question ID Question Answer 132 MTCT1: A 25 yr old pregnant woman tests HIV positive at your ANC. What Preventive and Treatment PMTCT package of care would you offer her: • Treat her for TB because TB is common in PLHIV • Maternal immunization • Start ART after clinical staging and establishing CD4 count • Iron, folate and multivitamins • Syndromic STI treatment if indicated • Antimalarial and ITNs • ART should be initiated in all pregnant and breastfeeding women living with HIV, regardless of gestation, WHO clinical stage and at any CD4 cell count and continued lifelong. • Initiate TB prophylaxis with IPT (isoniazid 300 mg once daily for 6 months) 133 MTCT 2: A mother brings her 2-week baby to the Post Natal Clinic for review after delivering a private health facility. The mother declines to be tested but permits you to carry out a HIV antibody test which is found to be positive. What next steps would you take? • Inform the mother that her baby is HIV positive and needs to start ART immediately; • Refer the mother to a HTS counsellor for counselling and testing; • Start infant ARV prophylaxis i.e. AZT+NVP and follow up as HIV￾exposed infant; • Collect blood HIV DNA PCR at the next 6-week immunization visit; • Tell mother she should stop breastfeeding and buy formula milk for her baby; • Counsel on post – partum contraception methods emphasizing dual protection to prevent avoid new/re-infection and unplanned pregnancies; 134 MTCT3: Mark each statement as True or False: 135 (a) All pregnant women should be tested for HIV at first ANC visit True Not True 136 (b) In the third trimester all women who tested HIV True 108 negative at the first ANC visit should be tested again for HIV Not True 137 (c) Pregnant women should be tested for HIV at every trimester True Not True 138 (d) Pregnant women should be tested for HIV during labor and delivery True Not True 139 (e) At 6 weeks postnatal clinic on all breastfeeding mothers (unless known HIV positive) should be tested for HIV True Not True 140 (f) All pregnant and breastfeeding women who are not tested, opt-out or decline HIV testing during the first contact should be offered HIV counselling and testing in subsequent visits True Not True 141 (g) All HIV positive and breastfeeding women enrolled into care should receive counselling and support (assisted disclosure), case managed linkage and follow-up including comprehensive care and treatment (lifelong ART) True Not True 142 (h) HIV testing and counselling cannot be offered to spouses/partners of HIV positive pregnant and breastfeeding mothers True Not True 143 (i) HIV testing and counselling cannot be offered to children of HIV positive pregnant and breastfeeding mothers True Not True 144 (j) All HIV positive mothers should be encouraged and supported to exclusively breastfeed for the first six months of life, introducing appropriate complementary foods at six months and continue BF up to AT LEAST 12 months of the infant’s life and preferably up to 24 months True Not True PRACTICE Question ID Question Answer 145 MTCT 1. If pregnant woman tests HIV positive for the first time, what are your routine actions? a) Conduct CD4 count b) Conduct Viral load c) Conduct clinical staging of HIV infection d) Evaluate for ART e) Counsel on Prevention of Mother to Child transmission (PMTCT) f) Start ARV treatment g) Refer to the HIV clinic 146 MTCT 2. Do you have system to notify maternity on HIV+ status of pregnant to ensure PMTCT? Yes No 147 MTCT3. What actions during labor and delivery would you take in an HIV+ woman to prevent/ reduce mother‐to‐child transmission of the virus? (a) Minimize vaginal examinations, use aseptic techniques to conduct delivery, (b) avoid artificial rupture of membranes, (c) monitor labor and avoid prolonged labor by use of the partograph, (d) avoid unnecessary genital tract trauma. (e) Where available, consider elective Caesarean section prior to onset of labor if the VL in late pregnancy (after 36 weeks gestation) is ≥ 1000 copies/ml. (f) Provide ARV prophylaxis to woman in early labor (g) Wipe nose, mouth, eyes of newborn with 109 gauze, suction only if necessary (h) No routine episiotomy (i) Minimize instrument delivery (j) Hibitane vaginal cleansing (k) Avoid milking cord/ immediate clamp cord (l) Active mgt of 3rd stage labor (m) Provide ARV prophylaxis to infant (n) Don't know 148 MTCT 3: What is management of HIV positive woman during labor and delivery 149 MTCT 4: If you refer women to CCC what is your communication to the receiver facility (a) I Write free form referral note with all clinical information that I consider important for a patient to take to HIV clinic (b) I Write standard referral note with all information required by protocol for a patient to take to HIV clinic (c) Support staff verifies whether woman got admitted in HIV clinic (d) Patient brings standard record from HIV clinic with all necessary information (e) Patient brings free‐form record from HIV clinic with all necessary information SECTION 6: Maternal (obstetric) Question ID Question Answer 150 OBT1. Please describe all services you provide to the mothers in labor in this facility: (a) Normal delivery services; (b) IV administration of antibiotics; (c) Parenteral administration of oxytocic drug; (d) Parenteral administration of anticonvulsants; (e) Assisted vaginal delivery; (f) Manual removal of placenta; (g) Manual removal of retained products; (h) Advanced surgical services (including C‐ section), (i) Blood transfusion (j) Counseling for family planning (k) IUD insertion and/or removal (l) Implant insertion and/or removal (m) Performing tubal ligation (n) Clinical management of FP methods, including managing side effects (o) Family planning for HIV positive women (p) Referral for FP services not available in your facility (q) Cervical Cancer Screening 151 OBT2. In the past 12 months, did you receive any training (initial or refresher) on the following topics: (a) Routine care for labor and normal vaginal delivery (b) Positive impact of the presence of a chosen companion during labor and birth (c) Non‐pharmacological and pharmacological pain relief during labor and birth (d) Interpersonal and cultural competence in providing emotional support during labor and birth (e) Identification and management of obstetric 110 emergencies (f) Screening and Management of Preeclampsia/eclampsia g) Management of PPH (h) Emergency obstetric care (EmONC)/Life‐ saving skills (LSS) (i) Management of prolonged and Obstructed labor (j) Recognition and management of maternal peri‐ partum infections (k) Preterm birth/Labor (care of mothers and babies) (l) Removal of placenta or products of conception (D&C, vacuum aspiration, etc.) (m) Manual removal of placenta (n) Special delivery care practices for preventing mother‐to‐child transmission (PMTCT) of HIV/AIDS (o) Assisted vaginal delivery (apply vacuum or forceps) (p) Resuscitate a newborn with bag and mask (HBB) (q) Cervical Cancer Screening (r) Maternal death or near miss reviews/audits (s) Standard infection control and precautions for transmission (t) Harmful practices and unnecessary interventions (u) Referral protocols and guidelines (v) Clinical hand‐over policy and communication of important information for hand‐over, referral or discharge (w) NONE 152 OBT3. Do you use guidelines or decision support tools, for example, job aids or algorithms, for clinical management in the following areas? (a) Screening and Management of Preeclampsia/eclampsia (b) Diagnosis and Management of Obstructed labor (c) Treatment of women with or at risk for infections (d) Preterm birth/Labor (care of mothers and babies) (e) Administration of Antenatal Corticosteroids (f) Screening for Cervical Cancer (g) Diagnosis and management of pph (h) NONE 153 OBT4. Please describe your personal level of comfort in independently managing the complications listed in the table below; 154 (a) Obstructed Labor Very comfortable Somewhat Comfortable Not comfortable 155 (b) PPH Very comfortable Somewhat Comfortable Not comfortable 156 (c) Eclampsia Very comfortable 111 Somewhat Comfortable Not comfortable 157 (d) Maternal Sepsis (risk of infections) Very comfortable Somewhat Comfortable Not comfortable 158 (e) Preterm Labor Very comfortable Somewhat Comfortable Not comfortable 159 OBT5. Are you aware of the following policies, SOPs/guidelines at your facility? (a) On verbal and written hand‐over of women at shift changes (b) On verbal and written hand‐over of women during intra‐facility transfer (c) On verbal and written hand‐over of women on referral to other facilities (d) On verbal and written hand‐over of women at discharge (e) zero‐tolerance non‐discriminatory policies with regard to mistreatment of women (f) NONE KNOWLEDGE Question ID Question Answer 160 OBK1. What are the indications for Prophylactic antibiotics during labor a) If there is maternal fever b) If it has been 18 hours or more since Rupture of membranes (ROM) c) If the mother has been in active labor for > 24 hours d) If there is a prior history of neonatal sepsis in a previous pregnancy 161 OBK2. Which item below is the single best way to prevent sepsis in both mother and child a) Administer IV antibiotics during labor if there is ROM > 8 hours b) Use sterile or high‐level disinfected sheets during birth c) Wash the vulva and perineum with antiseptic solution d) Wash your hands appropriately before every patient contact 162 OBK3. Which conditions below must be satisfied for safe administration of antenatal steroids? (a) gestational age assessment can be accurately undertaken; (b) preterm birth is considered imminent; (c) adequate childbirth care is available (including the capacity to recognize and safely manage preterm labor and birth); (d) the preterm newborn can receive adequate care if needed (including resuscitation, thermal care, feeding support, infection treatment and safe oxygen use). (e) Blood transfusion can be provided if necessary (f) Advanced surgical services (including C Section) is available in the facility 163 OBK4. Please tell me whether these statements are true or false 164 (a) Active labor is diagnosed when a woman has regular contractions and her cervical dilatation True Not True 112 is at least 2 cm 165 (b) The cervix should dilate at an average rate of at least 1 cm per hour True Not True 166 (c) A normal time range for the second stage of labor (between full cervical dilatation and delivery) is 3‐4 hours True Not True 167 (d) For active labor to be effective, the frequency of uterine contractions should be at least every 6‐7 minutes True Not True 168 OBK5. For the following items, please check whether you 1=agree, 2= have no opinion; 0= not agree. 169 (a) It is appropriate for a woman to walk around during labor agree, have no opinion; not agree 170 (b) Women should be allowed to eat and drink during the labor agree, have no opinion; not agree 171 (c) Choice of position for vaginal delivery should be made by healthcare professional based on mother’s and fetus condition agree, have no opinion; not agree 172 (d) Neonatal resuscitation skills are important for all obstetric providers agree, have no opinion; not agree 173 (e) Patient privacy should be a priority even when the maternity is very busy agree, have no opinion; not agree 174 (f) Confidentiality of patient information is not a problem in my maternity agree, have no opinion; not agree 175 (g) It is best to avoid discussing complicated obstetric decisions with a woman in labor as she may become anxious or not be able to understand agree, have no opinion; not agree 176 (h) Patients should have a companion with them during labor agree, have no opinion; not agree 177 (i) Patients should have a companion with them during delivery agree, have no opinion; not agree 178 (j) A woman should choose her preferred delivery position agree, have no opinion; not agree 179 The questions below are about treatment options in hypothetical clinical situations; please answer as you see best: 180 OBK6. A woman begins bleeding 20 minutes after delivering a healthy baby boy; she has had no complications during labor or delivery. You measure her blood loss as 550 cc. Your first action(s) would be a) Massage uterus fundus b) Examine vagina and perineum for lacerations c) Administer a uteronic (IV or IM) d) Begin IV fluids 181 OB7. Mrs. A. is 30 weeks pregnant. She was referred from lower level facility to Referral Hospital with the diagnosis severe preeclampsia. She was given loading dose of (a) Repeat loading dose (b) Stabilize blood pressure with diuretics (c) Encourage her to continue the pregnancy until at least 32 weeks to allow for fetal 113 magnesium sulfate 5 hours ago in the referring facility. On admission she still presents with severe headache, dizziness and blurred vision. On examination her blood pressure is 160/110 and she has 3+ protein in her urine. What should be the plan of management? maturation (d) Interrupt the pregnancy as soon as possible (e) Give 5 g of 50% magnesium sulfate solution with 1 mL of 2% lignocaine in the same syringe by deep IM injection into alternate buttocks every four hours 182 OBK9. What actions are appropriate for a woman who presents with, or develops heavy bleeding postpartum from atonic/uncontracted uterus? (a) Massage the fundus (b) Empty urinary bladder (c) Give uterotonics IM or IV (d) Perform bimanual compression of uterus (e) Perform abdominal compression of aorta (f) Start IV fluids (g) Take blood for hemoglobin, grouping and x‐matching (h) Insert condom tamponade (i) Refer to doctor or hospital (j) Raise foot of bed (k) Don't know 183 OBK10.When should membranes be ruptured artificially by the provider? (a) At start of second stage (b) Immediately prior to delivery when they are bulging in vagina (c) Routinely during active phase of labor (d) As part of augmentation of labor (e) Upon admission for all women (f) To check color of fluid/liquor when fetal distress is noted (g) Not to be ruptured (h) Don't know PRACTICES Question ID Question Answer 184 OBP1. How many deliveries did you INDIVIDUALLY manage in the past month? 185 OBP2. Have you ever used a partograph? Yes No 186 OBP3. When was the last time you used a partograph? a) Never b) Within Past Week c) Within Past Month d) Within Past 6 Months e) Over 6 Months Ago 187 OBP4. How often you use a partograph in your clinical practice? a) Always b) For most of the times c) At half of the cases d) Less than half of the cases e) Very rarely 188 OBP5. Do you normally allow a birth companion to be present during a woman’s labor? Yes No 189 OBP6. Do you normally allow a birth companion to be present during a woman’s delivery? Yes No 190 OBP7. Who can be a companion during a delivery? a) Only close relative (mother, husband) b) Only relative, but not important how close c) Any female 114 d) Anybody of pregnant woman’s choice e) Other specify 191 OBP8. Do you provide orientation sessions or have information materials (written or pictorial) to orient the companion on their role? Yes No 192 OBP9: which procedures do you perform routinely for all your patients during labor and delivery (a) Artificial rupture of membranes (b) Active management of third stage of labor (c) Episiotomy (d) Perineal shaving (e) Maternal blood pressure monitoring (f) Administration of prophylactic antibiotics to women in labor (g) Enema (h) Fetal heart rate monitoring 193 OBP10. For which obstetric interventions, if any, do you ask a patient to sign a consent form: (a) NONE (b) Generic consent form at time of admission (c) Episiotomy (d) Cesarean section (e) Augmentation or induction of labor (f) Other (Please specify) 194 OBP11. How often do you usually check a woman’s blood pressure during the first stage of uncomplicated labor? a) At admission and after delivery only b) At least every 2 hours from admission c) At least every 4 hours from admission d) At admission, only 195 OBP12. How often do you usually check a woman’s Heart Rate during the first stage of uncomplicated labor: At admission and immediately at the beginning of second stage 2) At least every hour from admission during the first stage 3) At least every 30 minutes from admission during the first stage 4) At admission, only 196 OBP13. How often do you usually check a woman’s temperature the first stage of uncomplicated labor: a) At admission and immediately at the beginning of second stage b) At least every 2 hours from admission during the first stage c) At least every 4 hours from admission during the first stage at admission, only 197 OB14. How often do you usually check a woman’s urinalysis the first stage of uncomplicated labor? a) At admission and immediately at the beginning of second stage b) At least every 6 hours from admission during the first stage c) At least every 4 hours from admission during the first stage d) At admission, only 198 OBP15. How often do you usually check contractions the first stage of uncomplicated labor: a) At admission and immediately at the beginning of second stage b) At least every 2 hours from admission c) At least every 4 hours from admission d) At admission, only 199 OB16. How often do you usually perform vaginal exam of women the first stage of uncomplicated labor? a) At admission and after delivery only b) At least once in every 2 hours from admission to discharge c) At least once in every hour from admission to discharge 115 d) At least once in every 4 hours from admission to discharge e) Once in 4 hour or less frequently f) At admission only 200 OBP17. To which category of women, you administer Uterotonic: a) Women with anemia b) Women with prior history of PPH c) Women with hematologic bleeding disorder d) All women after vaginal delivery e) Women > age 35 201 OBP18. If you need to administer uterotonic how you space it after vaginal birth? Within 5 minutes after delivery of fetus; Immediately after delivery of placenta Within one minute after delivery It depends on the weight of the fetus 202 OBP19. What is average length of stay in your facility after uncomplicated vaginal birth? 203 OBP20. Did you have even one severe pre‐ eclampsia or eclampsia patient in the health facility who did not receive the full dose of magnesium sulfate because of a stock‐out? Yes No 204 OBP21. Did you have even one patient who needed emergency C‐Section and but could not receive due to lack of supplies or staff trained to conduct caesarean section? Yes No 205 OBP 22. For which reason did they not receive the emergency C-section? Lack of infrastructure to offer C-section. Due to lack of supplies Due to lack of staff trained to conduct caesarian section Overcrowded theatre schedule 206 OBP23. Do women receive Family Planning Counselling before they are discharge? Yes No 207 OBP24. Does your facility usually provide postnatal contact with a skilled health‐care provider to mothers and babies 48–72 hours after normal birth Yes No 208 OBP25. If your facility usually provides postnatal contact with a skilled health‐care provider to mothers and babies 48–72 hours after normal birth, how it is organized? Provider (specify ___________) delivers home visits The women have follow up visit scheduled at discharge) Other specify 209 OBP26. Do you routinely screen any group of your patients for cervical cancer? Yes No 210 OBP27. At what point do you screen your patients for cervical cancer? (a) In the maternity before the discharge (b) At postnatal contact (c) At ANC visit (d) Other, specify 211 OBP28: Generally, what do you use for cord care at the facility? (a) Chlorhexidine (b) Methylated spirit (c) Other (Specify) SECTION 7: Maternal-Neonatal Question ID Question Answer 212 NET1. Which services do you provide to the newborns? (a) Essential newborn care (b) Breastfeeding support (c) Neonatal resuscitation 116 (d) Management of Possible Severe Bacterial Infections; (e) Care of preterm babies, (f) Kangaroo Mother Care (g) Nasogastric tube insertion (h) IV administration of Medications (i) Management of poor feeding (j) Management of neonatal Jaundice (k) Management of congenital anomalies (l) Routine postnatal care (m) Vaccination 213 NET2. In the past 12 months, did you receive any training (initial or refresher) on the following topics: (a) Essential newborn care (e.g., cord care, warming, early and exclusive breastfeeding) (b) Resuscitation of newborns not crying or breathing at birth (Helping Babies Breathe) (c) Care of the very small newborn (d) Care of the sick newborn (e) Recognition and management of suspected newborn infections (f) Management of Possible Signs of Bacterial infection (PSBI) in newborns and young infants (g) Nutrition /feeding counselling (h) Routine vaccination (i) Integrated care of common newborn conditions (INCI) (j) Kangaroo Mother Care (k) Standard infection control and precautions for transmission (l) Harmful practices and unnecessary interventions (m) Clinical hand‐over policy and communication of important information for hand‐over, referral or discharge (n) Referral protocols and guidelines (o) Interpersonal communication/counselling skills/cultural competence (p) None 214 NET3. Do you use guidelines or decision support tools, for example, job aids or algorithms, for clinical management in the following areas? (a) Essential newborn care (e.g., cord care, warming, early and exclusive breastfeeding) (b) Resuscitation of newborns not crying or breathing at birth (c) Care of the very small newborn (d) Care of the sick newborn (e) Management of Possible Signs of Bacterial infection (PSBI) in newborns and young infants (f) Referral criteria for sick or premature babies (g) None 215 NET4. Did you ever have the opportunity to practice Newborn resuscitation skills using a newborn anatomic model/doll (e.g. NeoNatalie) after you were trained? Yes No 216 NET5. When was the last time when you practiced newborn resuscitation, skills using a newborn anatomic model/doll (e.g. neonatally)? 117 217 NET6. Please describe your personal level of comfort in independently managing the complications listed in the table below: 218 (a) Asphyxia/ Resuscitation Very comfortable Somewhat Comfortable Not comfortable 219 (b) Newborn sepsis Very comfortable Somewhat Comfortable Not comfortable 220 (c) Low birth weight or Prematurity Very comfortable Somewhat Comfortable Not comfortable 221 (d) Possible signs of Bacterial Infections Very comfortable Somewhat Comfortable Not comfortable 222 NET7. Are you aware of the following policies, SOPs/guidelines at your facility? (a) On verbal and written hand‐over of newborns at shift changes (b) On verbal and written hand‐over of newborns during intra‐facility transfer (c) On verbal and written hand‐over of newborns on referral to other facilities (d) On verbal and written hand‐over of newborns at discharge (e) Zero‐tolerance non‐discriminatory policies with regard to mistreatment of newborns (f) NONE 223 NET8. How satisfied or dissatisfied are you with the communication during clinical hand‐over of newborns among members of the health care team in the health facility? a) Extremely dissatisfied b) Somewhat dissatisfied c) Nor satisfied not dissatisfied d) Somewhat satisfied e) Extremely satisfied KNOWLEDGE Question ID Question Answer 224 NEK1. A baby is quiet, limp and not breathing at birth. What should you do? a) Dry the baby thoroughly b) Shake the baby c) Throw cold water on the face d) Hold the baby upside down 225 NEK2. A baby is born through meconium‐stained amniotic fluid. Which statement is TRUE? a) Stimulate the baby and then clear the airway b) Meconium cannot be inhaled into the lungs c) Clear the airway before drying the baby d) All babies born through meconium‐ stained amniotic fluid can receive routine care 226 NEK3. A newborn baby is quiet, limp and not crying. The baby does not respond to steps to stimulate breathing. What should you do next? a) Slap the baby’s back b) Hold the baby upside down c) Squeeze the baby’s ribs d) Begin ventilation 227 NEK4. Which of the following statements about ventilation with bag and mask is TRUE? a) The mask should cover the eyes b) Air should escape between the mask and face 118 c) Squeeze the bag to produce gentle movement of the chest d) Squeeze the bag to give 80 to 100 breaths per minute 228 NEK5. Which of the following signs need to be monitored in a baby during the first few hours after birth? a) Breathing b) Movement c) Temperature d) Feeding status e) Urine output 229 NEK6. A baby’s chest is not moving with bag and mask ventilation. What should you do? a) Stop ventilation b) Reapply the mask to get a better seal c) Slap the baby’s back d) Give medicine to the baby 230 NEK7. You can stop ventilation if a) Baby is blue and limp b) Baby’s heart rate is 80 per minute c) Baby’s heart rate is 120 per minute and the chest is not moving d) Baby’s heart rate is 120 per minute and the baby is breathing or crying 231 NEK8. What should you do to keep the baby clean? a) Wash your hands before touching the baby and help the mother wash her hands before breastfeeding b) Reuse the suction device before cleaning c) Keep the umbilical cord tightly covered d) Do not touch the baby 232 NEK9. Which factors should prompt close observation for symptoms of sepsis in a newborn: (a) Low newborn temperature (< 35 C) (b) Post‐dates infant (c) Prolonged rupture of membranes before delivery (d) History of neonatal sepsis in a sibling 233 NEK10. The first step in thermal protection of newborns is: a) Completely dry and cover the baby immediately following the birth b) Completely dry the baby after the umbilical cord has been cut c) Place a hat over the baby’s head d) Wrap the baby immediately in a blanket 234 NEK11. If a newborn does not spontaneously breathe at birth within 60 seconds what is the most important first action: a) Begin bag and mask ventilation as quickly as possible b) Assess for a heart rate c) Suction the baby’s mouth and nose d) Dry and stimulate the newborn e) None of the above 235 NEK12. The first step of the pre‐cleaning (decontamination) process of newborn resuscitation equipment is: a) Boil in water for 10‐20 minutes b) Soak in activated glutaraldehyde, then rinse well with boiled water c) Wash with soap and water d) Wiping the outside of the devices with a gauze soaked with 0.5% chlorine solution or soak all parts in 0.5% chlorine solution for 10 minutes e) Steam autoclave (sterilization) 236 NEK13. Correct steps of reprocessing the reusable a) Clean, dissemble, high level 119 bag, mask and manual suction device is disinfection or sterilization, proper storage before next use and reassemble b) Pre‐clean, disassemble, clean, reassemble and subject equipment to High level disinfection or sterilization and proper storage before the next use c) Dissemble, pre‐clean, reassemble, clean, high level disinfection or sterilization and proper storage before the next use d) Pre‐clean when wearing gloves, clean, dissemble, high level disinfection or sterilization, reassemble, proper storage before the next use 237 NEK14. You are stationed at a Health Centre as a midwife, and you deliver a mother who has a fever of 38.5°C during labor. What is the recommended next step of management? 1) Treat mother for malaria and allow baby and mother to go home 2) Investigate and treat mother for infection 3) Give the new born an oral dose of Septrin syrup for five days 4) Give the new born IM/IV Ampicillin and Gentamycin for 2 days, reassess and continue antibiotics only if there are signs of sepsis 5) Treat mother for infection and give a stat dose of X‐pen to the baby and follow up after 2 days 6) 2 and 4 above 7) 1 and 4 above 238 NEK15. Which newborn can be placed in Kangaroo Mother Care? 1) Clinically stable baby weighing 2000‐ 2500 g 2) Any baby weighing over 2000 g 3) Clinically stable baby weighing less than 2000 g 4) Babies born at 30‐34 weeks of gestation 239 NEK16. What interventions are included in Kangaroo Mother Care? 240 (a) Baby should be wearing only diaper True Not True 241 (b) Mother needs special clothing for KMC True Not True 242 (c) Baby should be placed between the mother’s breasts in an upright position, chest to chest True Not True 243 (d) Baby should be placed under the mother’s breasts in horizontal position, facing upwards to facilitate breastfeeding on demand True Not True 244 (e) Mother should carry baby with one hand during all KMC session True Not True PRACTICE Question ID Question Answer 245 NEP1. How many newborns have you resuscitated in the last month? 246 NEP2. As part of your routine practice what is your Ensure they are breathing 120 first action after the baby is born? Dry thoroughly Tie and clap the umbilicus Place them in skin to skin contact with the mother 247 NEP3. When do you place the newborns for the first time at mother’s abdomen and chest after drying and clapping the cord? Immediately after birth After first medical assessment After BCG vaccination Not routinely 248 NEP4. Usually how long do you keep babies with skin to skin contact with mothers after the birth? At least 15 minutes At least for half an hour At least an hour At least two hours 249 NEP5. When do you usually clamp or tie and cut umbilical cord during routine care? Immediately after the baby is born Around 1‐3 minutes after birth After the placenta is delivered Before a baby has cried 250 NEP6. When do you usually initiate breastfeeding during routine care? Immediately after birth Within first 30 minutes Within the first hour Within first 24 hours 251 NEP7. Which vaccinations, if any, do you usually provide to newborns (a) no routine vaccinations for newborn (b) BCG (c) Polio (d) Hepatitis B 252 NEP8. In your maternity, where do newborns usually stay after delivery most of the time: with their mother in a nursery with nurses 253 NEP9. For the following questions, please indicate whether you Agree, Do not have an opinion or Do not agree. 254 (a) If there are no medical problems for the newborn, the mother should decide whether her newborn stays with her (at her bedside) after delivery until discharge agree, have no opinion; not agree 255 (b) If a mother has had a cesarean it is usually best to encourage her to mix bottle-feeding with breastfeeding so that she can recuperate adequately. agree, have no opinion; not agree 256 (c) It is appropriate to place a newborn directly onto the mother’s abdomen immediately after birth agree, have no opinion; not agree 257 NEOP13. What basic equipment and supplies must be available to ensure the baby receives appropriate immediate care after birth? (a) 2 dry warm towels or cloths (b) Sterile blade or scissors (c) Sterile or disposable cord ties/ clamps (d) Cap for baby (e) Source of warmth: heating lamp or incubator (f) Self‐inflating ventilation bag (g) Newborn face mask size 1 (h) Newborn face mask size 0 (i) Penguin suction/Mucus extractor/ suction/ bulb syringe (j) Flat surface (k) Clock or watch with seconds 258 NEOP14. When a baby is delivered and there is no complication, what care is important to give them immediately after birth and in the first hour? (a) Dry the baby thoroughly (b) Ensure baby was breathing/ crying (c) Provide thermal protection: place skin 121 to skin with mother (d) Once placed skin‐to‐skin with mother, cover with dry towel (e) Clamp and cut cord with sterile blade/scissors (f) Monitor newborn breathing every 15 min (g) Monitor newborn temperature by hand every 15‐30 min (h) Ensure mother initiates breast feeding within 1 hour 259 NEOP15. When a baby is delivered and there is no complication, what care is important to give the baby in the first 90 minutes after birth? (a) Assess/examine newborn within 90 mins (b) Measure temperature (c) Weigh newborn (d) Provide eye care (e) Provide cord care (f) Give Vit K (can be later, if too busy) 260 NEOP16. What are the danger signs in the newborn that require immediate attention of the care taker and provider? (a) Not feeding (b) Too hot/cold (c) Convulsions (d) No movement (e) Chest in‐drawing or fast breathing (f) Yellow palms or soles of feet 261 NEOP17. What are the signs and symptoms of possible severe bacterial infection in a newborn? (Revised according to the new sepsis guideline) (a) Not able to feed since birth or stopped feeding well (b) Breathing difficulties/ severe chest in‐ drawing (c) Hypothermia (less than 35.5 °C) (d) Hyperthermia (38 °C or greater) (e) Breathing rating >60/minute (f) Convulsions (g) Movement only when stimulated or no movement at all SECTION 8: Child Care Question ID Question Answer 262 CHT1. What services you provide to the children under five? (a) Well‐child visits (including monitoring of growth and nutrition) (b) Nutrition Clinic (c) Outpatient visits of sick children for common childhood conditions (IMCI) (d) HIV infection testing for children (e) HIV infection testing for mothers (f) Childhood immunization 263 CHT2. In the past 12 months, did you receive any training (initial or refresher) on the following topics: (a) Integrated care of young Infant (<2months) conditions (IMCI) (b) Integrated care of common childhood conditions (IMCI) (c) Nutrition /feeding/ breastfeeding counselling (d) Assessment, treatment, counselling for diarrhea, (e) Testing, treatment, counselling for HIV infection (f) Malnutrition prevention, diagnosis and 122 treatment (g) Childhood immunization (h) Other, specify (I) NONE 264 CHT3. Do you use guidelines or decision support tools, for example, job aids or algorithms, for clinical management in the following areas? a) IMCI chart book b) Assessment, treatment, counselling for diarrhea, c) Assessment, treatment, counselling for malaria d) Screening, treatment, counselling for HIV infection e) Malnutrition prevention, diagnosis and treatment 265 CHT4. Please describe your personal level of comfort in independently managing the complications listed in the table below; 266 (a) Severe dehydration Very comfortable Somewhat Comfortable Not comfortable 267 (b) HIV Very comfortable Somewhat Comfortable Not comfortable 268 (c) Severe Malnutrition Very comfortable Somewhat Comfortable Not comfortable KNOWLEDGE Question ID Question Answer 269 CHK1. A 1 year old presents with a low‐ grade fever, and watery diarrhea, yellow‐ greenish in color. Your assessment reveals a slow skin pinch and sunken eyes, Temp is 37.60c. The baby is irritable but is eager to drink. Other findings are normal. What recommended classification should you document in the OPD register? Diarrhea Clinical Malaria Diarrhea, no dehydration Diarrhea with some Dehydration. Diarrhea and Clinical Malaria 270 CHK2. From the above scenario (CK13), what would be your plan of management? Give oral cotrimoxazole, ORS (plan A), Zinc, counsel mother on danger signs and follow up in 5 days. Give oral cotrimoxazole, ORS (plan B) Zinc, counsel mother on danger signs and follow up in 5 days. Give ORS, Zinc (Plan B), and counsel mother on danger signs and follow up in 5 days. Admit or refer for further management Give an Antimalarial, Septrin, ORS (Plan B) Zinc, counsel mother on danger signs and follow up in 5 days. 271 CK3. What are the indications for the use of antibiotics in diarrhea? Greenish diarrhea Very Watery diarrhea Diarrhea lasting more than 3 days Diarrhea with occasional episodes of vomiting. None of the above 272 CHK4. An exclusively breastfed 5 weeks old infant is presented to you with a history Give Zinc and Encourage frequent breastfeeding and follow up. 123 frequent watery stool. The mother explains the young infant is not vomiting and is feeding well. Assessment reveals an active baby with no signs of dehydration and normal examination findings. Which of the following would you recommend? Encourage frequent breastfeeding and follow up Give Zinc, an appropriate oral antibiotic with frequent breast-feeding and follow up. Prescribe cotrimoxazole syrup, encourage frequent breast-feeding and follow up. Give Zinc and ORS. Give ORS or clean water in addition to breast milk if follow up is not available 2 and 6 2 and 5 SECTION 9: Comprehensive Care for HIV Question ID Question Answer Question ID Question Answer 273 TB/HIV1. Please select all TB/HIV coinfection services you provide to PLHIV at this facility? (a) HIV Testing & Counselling Services (b) Screening for TB (c) Isoniazid Prophylaxis (d) TB Treatment (e) ART for TB/HIV Co-infection 274 TB/HIV2. Please Indicate all trainings you received last 12 months? (a) TB Counselling and Testing (b) Isoniazid Preventive Therapy (IPT) (c) Diagnosis and Treatment (d) ART for TB/HIV Co-infection (e) TB Infection Control 275 TB/HIV3.Please describe your personal level of comfort to independently carry out the following tasks and / or manage the following conditions: 276 a). Screening of PLHIV (ICF) Very comfortable Somewhat Comfortable Not comfortable 277 b). Provision of ART for TB/HIV Coinfection Very comfortable Somewhat Comfortable Not comfortable 278 c). Provision of Isoniazid Preventive Therapy Very comfortable Somewhat Comfortable Not comfortable 279 d). Follow up of Patients on INH Very comfortable Somewhat Comfortable Not comfortable 280 e). Management of Complications of TB/HIV Coinfection Very comfortable Somewhat Comfortable Not comfortable KNOWLEDGE Question ID Question Answer 281 TB/HIV1. What are the symptoms of TB in children? (a) Cough of any duration (b) Fever (c) Failure to thrive or poor weight gain (d) Lethargy, less playful than usual (e) Contact with a TB case 282 TB/HIV2. Indicate if question is True or False 283 (a) Symptom-based TB screening using the ICF tool should only be on PLHIV who are coughing. True Not True 284 (b) Patients who screen positive (presumptive TB True 124 cases) should have sputum smear test or GeneXpert Not True 285 (c) Patients who screen negative should be allowed to go home on antibiotics and reviewed in 6 months True Not True 286 (a) Patients who screen negative should be evaluated for isoniazid preventive therapy (IPT) True Not True PRACTICE Question ID Question Answer 287 TB/HIV1: In which situation will you NOT provide Isoniazid Prophylaxis Therapy (a) Active tuberculosis disease (b) Active hepatitis (c) Active substance abuse and/or regular and heavy alcohol consumption (d) Symptoms of peripheral neuropathy (e) Poor adherence to CPT, ART or clinic appointments (f) Poor understanding of IPT by parent/caregiver (g) Infants < 1 year who have direct exposure to TB contacts (h) Past history of TB and/or (i) current pregnancy 288 TB/HIV2: What is your practice regarding management of recently child (less than 3yrs) diagnosed with TB/HIV and are not yet on ART (a) Start patient on ART and CPT as part of the comprehensive package of care for PLHIV. (b) Start anti-TB immediately (c) Wait for CD4 count to improve then start anti-TB (d) Initiate ART as soon as anti-TB medications are tolerated, preferably within 8 weeks (e) Use “super-boosted” LPV/r by adding additional ritonavir suspension to manage the drug interaction between LPV/r and rifampicin SECTION 10: Others Question ID Question Answer 289 CHK5. A One-year old infant is brought to you for an OPD visit. Your assessment reveals a Middle Upper Arm Circumference (MUAC) of 109mm. No other abnormalities are seen on examination. You offer Ready to Use Therapeutic Foods (RUTF) and the child completes the portion given. What is the most likely Classification? Complicated Severe Acute Malnutrition Uncomplicated Severe Acute Malnutrition Moderate Acute Malnutrition No Acute malnutrition None of the above 290 CHK6. The following measurement would indicate a classification of Severe Acute Malnutrition in a 1 years old infant. MUAC 118mm Weight of 8 kg Weight for Height below ‐2 Z score All the above None of the above 291 REFERRAL 292 OS1. Do you refer the severely‐ill patients? Yes No 293 OS2. Does your facility have protocol or any Yes, we have written protocol that is available 125 established procedure that defines when and where to refer patients if needed? for quick reference Yes, we follow established procedure, but it is not written anywhere No 294 OS3. Do you have any procedure of communication with the receiving facility? Yes No 295 OS7. What is main reason that prevent you from referring a severely‐ill patient to a higher level facility/hospital? Parent does not accept referral due to geographic accessibility Parent does not accept referral due to affordability Timely transportation to higher level facility is not possible Other specified in previous question 296 OS9. Please write down all information that usually include in referral note (e.g. Patients’ name….)? 297 OS14. How do you communicate with other health care providers taking care of your patient? (e.g. specialist to specialist or primary care to specialist)? Usually patients do not have multiple providers Communication usually occurs via written notes in the patent records or specific visit notes Communication usually occurs via direct verbal communication Communication usually occurs via a combination of verbal and written communication It is not common practice for different providers of an individual patient to communicate in any way. Other specify 298 OS16. In your facility, is there any system or procedure to track patients who do not come for a scheduled follow up visit? Yes No 299 OS17. How much are you satisfied with the communication during clinical hand‐over among members of the health care team in the health facility? Extremely dissatisfied Somewhat dissatisfied Nor satisfied not dissatisfied Somewhat satisfied Extremely satisfied 300 HI1. Approximately how much time would it take for you to retrieve information from patient’s previous outpatient visit in this facility? 301 HI2. How difficult it is to retrieve information from patient’s previous outpatient visit in this facility a) It depends on the type of visit patient had (e.g. ANC visits, well‐child visit, FP visits, outpatient care of sick patient and etc.) b) It is generally easily accessible to me for review during the patients’ outpatient visit c) It is generally not possible or extremely difficult to retrieve information on previous outpatient visit in this facility. d) The only information on previous outpatient visits I may get is from patients’ standardized card or passport e) The only information on previous visits I may get is from non‐standardized patients’ notebook 302 HI3. Does standardized medical information in your facility allow you to document Yes No 126 essential aspects of RMNCHA/HIV care? 303 CQI 304 QI2 Does your facility have an established continuous quality improvement process? My facility has a continuous QI process My facility has some internal QI initiatives but not a continuous QI process QI process in my facility is mostly initiated by external agents (coaches, supervisors) Other, specify MY facility does not have any QI process 305 QI9. How many quality improvement team meetings activities have you been engaged/participated in the preceding six months? 306 QI11. Have you ever received QI capacity building or coaching Yes, during last 6 month Yes, during last 7‐12 months Yes, during last 13‐24 months >24 months ago Do not remember Never 307 QI12. If yes, who provided the QI capacity building or coaching? a. Clinical officer from District/County Health Management Team b. Administrative officer from District/County Health Management Team c. Central MoH d. Member of professional association e. Representative of donor‐funded project (Specify) f. Other, please, (specify) MPDSR Question ID Question Answer 308 DA1 Is you facility routinely conducting reviews of patient death and near misses? Yes No 309 DA2. If yes, please, select all death or near‐misses reviews that is regularly conducting in your facility a). Maternal death review b). Neonatal death review c). Perinatal death review d). Review of maternal near‐misses e). Review of neonatal near misses 310 DA5. Do you have Maternal or Perinatal Death Audit Committee or any other structure that reviews death audits? Yes No 127 USAID-AFYA JIJINI-Beneficiary KAP-May-18 Organization: International Business & Technical Consultants Inc. Report Date: 5/15/2018 5:25:49 PM HIV, MNCH, WASH, NUTRITION FACILITY IDENTIFICATION Question ID Question Answer 1 Facility name: Babadogo HC Bahati HC Embakasi Medical Center Huruma Dispensary Kangemi Health Center Kariobagi HC Kasarani HC Lianas Clinic Dispensary Makadara HC Mathare North HC Mbotela Clinic Mukuru HC St Clare Medical St Mary immaculate St Mary Mission Hospital St Peter Dispensary 2 Facility type COUNTY HOSPITAL SUB-COUNTY HOSPITAL HEALTH CENTRE DISPENSARY MISSION HOSPITAL CHURCH HOSPITAL OTHER (SPECIFY) 3 Type of sector GOVERNMENT FAITH-BASED/MISSION/CHURCH OTHER (SPECIFY) 128 4 Facility location (Locality) RURAL URBAN 5 County 6 Sub-county Embakasi South Embakasi North Kasarani Langata Makadara Mathare Ruaraka Westlands 7 Village/ neighbourhood 8 Date 9 Respondent Name APPENDIX 1: CONSENT FORM Question ID Question Answer 10 voluntarily agree to participate Yes No 11 REASONS FOR REFUSAL OUTRIGHT REFUSAL THE INTERVIEW IS TOO LONG HAVE OTHER COMMITMENTS SERIOUSLY ILL OTHER (SPECIFY) 12 TIME INTERVIEW STARTED: 13 In what month and year were you born? 14 GENDER OF THE RESPONDENT MALE FEMALE 15 How old are you now? 16 What is the highest level of schooling you attended? Primary Post-Primary 129 Secondary College University Other 17 What is your religion? Roman catholic Protestant /Other Christian Muslim Other region 18 What is your marital status now? Single Married Widowed Separated Other 19 Respondent groups MNCH clients CCC Clients Youth CHWs A)Knowledge Question ID Question Answer 20 Have you heard of HIV/AIDS Yes No Don’t know/not sure HIV/STIs Question ID Question Answer 21 Having one uninfected, faithful partner prevents HIV/AIDS Yes No Don’t know/not sure 22 One can use condoms to prevent HIV/AIDS Yes No 130 Don’t know/not sure 23 One Cannot get HIV/AIDS from mosquitoes Yes No Don’t know/not sure 24 People can get HIV because of witchcraft or other supernatural means? Yes No Don’t know/not sure 25 One cannot get HIV/AIDS from sharing food with infected person Yes No Don’t know/not sure 26 A healthy-looking person can have HIV/AIDS Yes No Don’t know/not sure 27 A mother can transmit HIV/AIDS to her child during pregnancy and delivery Yes No Don’t know/not sure 28 A mother can transmit HIV/AIDS to her child through breastfeeding Yes No Don’t know/not sure 29 There are special drugs that a doctor or a nurse can give to a woman infected with HIV to reduce the risk of transmission to the baby? Yes No Don’t know/not sure 30 Can a wife propose condom use when the husband has STI? Yes No (explain) 31 Have you been exposed to GBV? Yes No (explain) Antenatal care Question ID Question Answer 32 How many times have you been pregnant? Parity Gravida 33 How many ANC visits should a woman 131 have? 34 Can you name all items on which you have been counseled during the ANC visits at least once HIV/PMTCT Birth preparedness (how to prepare for birth) Pregnancy Complication preparedness Family planning services Healthy spacing between the pregnancies Danger signs (when to seek medical care) Healthy Diet Physical Activity Dangers of smoking Benefits of breast Feeding Intimate partner violence/domestic violence 35 Please name all the danger signs in pregnancy when you need to be taken to hospital or health center immediately Vaginal bleeding Convulsions Severe headaches with blurred vision Fever and is too weak to get out of bed Severe abdominal pain Fast or difficult breathing 36 Please name all danger signs in pregnancy when you need to come to health center as soon as they can Fever Foul smelling discharge from her vagina Abdominal pain Feels ill Swelling of fingers, face and legs Maternity care Question ID Question Answer 37 Please specify all topics anyone counseled you about since you arrived in the maternity -What your baby needs to drink/eat for first 6 months -What your baby needs to drink/eat for first 6 months 132 -Nutrition & Hygiene -Cord care -Maintaining Lactation -Keeping baby warm & clean -Communication and play with the baby -Which signs to watch for (danger signs) in my baby -Which signs to watch for (danger signs) in yourself -Where to go in case emergency for myself -Where to go in presence of danger signs in my baby -When and where I need to follow up for myself -When and where I need to follow up for my baby -Healthy spacing between the pregnancies -Family Planning options available to me now Post natal care Question ID Question Answer 38 How many postnatal contacts with skilled health-care provider at home or facility did you have 39 When did you have your first postnatal contact after the \childbirth? 40 Were you counselled on FP method at any of postnatal contacts? Yes No 41 Did you accepted/choose FP method at any of these visits? Yes No 42 Have you ever received cervical cancer screening? Yes No 43 Is your child exclusively breastfed now? Yes 133 No 44 How long was your child breastfed? Family planning Question ID Question Answer 45 Which methods of FP have you heard of? Condom Oral contraceptives IUD/spiral Injection Calendar method Withdrawal method Breast feeding Male sterilization Female sterilization Diaphragm Other (specify) 46 Have you ever heard about any emergency contraceptive method? Yes No 47 What is the ideal age space between children? one year two years three to five years Five years or more Don't know Immunization Question ID Question Answer 48 Has your baby ever received any immunizations? Yes No Don’t know/not sure 49 Do you have a mother and child health (MCH) booklet? If so, may I see it? Yes No, lost/misplaced it No, never had one 134 50 At what age should your baby receive the following vaccines? BCG OPV 1st OPV 2nd OPV 3rd DPT-HepB-HiB 1st DPT-HepB-HiB 2nd DPT-HeB-HiB 3rd PCV 10 (Pneumoccal Vaccine 10) 1st PCV 10 2nd PCV 10 3rd Measles Yellow fever* Doesn’t know 51 Look at the mother and child health (MCH) booklet and write the dates when vaccines were administered 52 a). BCG (dose below 1 year) 53 b). OPV 0 54 c). OPV 1 55 d). OPV 2 56 e). OPV 3 57 f). DPT, HEP, HIB 1st dose 58 g). DPT, HEP, HIB 2nd dose 59 h). DPT, HEP, HIB 3rd dose 60 i). Pneumococcal1st dose 61 j). Pneumococcal2nd dose 62 k). Pneumococcal3rd dose 63 l). Rotavirus 1 64 m). Rotavirus 2 65 n). Measles (9 mo.) 66 o). Yellow fever 135 No MCH booklet Question ID Question Answer 67 Has (NAME) received any vaccinations that are not recorded in the mother￾child booklet, including vaccinations received in a national immunization day campaign? Yes No Don’t know/not sure 68 Please tell me if (NAME) received any of the following vaccinations: A BCG vaccination against tuberculosis that is, an injection in the arm or shoulder that usually causes a scar? Yes No Don’t know/not sure 69 Polio vaccine, that is, drops in the mouth? Yes No Don’t know/not sure 70 How many times was the polio vaccine received? 71 A Pentavalent vaccination that is an injection given in the thigh, sometimes at the same time as polio drops? Yes No Don’t know/not sure 72 How many times was a Pentavalent vaccination received? 73 A measles injection- that is, a shot in the right upper arm at the age of 9 months or older - to prevent him/her from getting measles? Yes No Don’t know/not sure Nutrition Question ID Question Answer 74 Have you heard of exclusive breastfeeding? Yes No 75 If yes, what does exclusive breastfeeding mean? Exclusive breastfeeding means that the infant gets only breast milk and no other liquids or foods Other (specify) Don’t know 76 What is the first food a new-born baby should receive? Only breast milk Other (specify); 136 Don’t know 77 How long should a baby receive nothing more than breast milk? From birth to six months Other (specify) Don’t know 78 When should a mother start adding foods to breastfeeding? start adding earlier than 4 months of age start adding between 4-6 months of age start adding later than 6 months of age doesn’t know 79 Why do you think breast milk is the only food recommended for infants up to six months old? Because breastmilk provides all the nutrients and liquids a baby needs in its first six months Because babies cannot digest other foods before they are six months old Other (specify) Don’t know 80 How often should a baby younger than six months be breastfed or fed with breast milk? On demand, whenever the baby wants Other (specify) Don’t know 81 What are the benefits for a baby if he or she receives only breast milk during the first six months of life? He/she grows healthily Protection from diarrhoea and other infections Protection against obesity and chronic diseases in adulthood protection against other diseases (specify) Other (specify) Don’t know 82 How long is it recommended that a woman breastfeeds her child? Six months or less 6-11 months 12-23 months 24 months and more Other (specify) Don’t know 83 Why is it important to give foods in Breast milk alone is not sufficient 137 addition to breast milk to babies from the age of six months? (enough)/cannot supply all the nutrients needed for growth/from six months, baby needs more food in addition to breast milk Other (specify) Don’t know Household water treatment Question ID Question Answer 84 What is the MAIN source of drinking water for members of your household? Piped water; Public tap/standpipe; Tube well/borehole (& pump); Protected dug well; Protected spring; Rain water collection; UNHCR Tanker; Unprotected spring, Unprotected dug well, Small water vendor, Tanker truck, Bottled water, Surface water (e.g. river, pond), Other (specify) Don’t know 85 What specific kind of water container do you use USUALLY store your drinking water? Jerry can Bucket Clay pots Drum/barrel Bottles Basin Other (specify); 86 Do you treat your water? Yes No 138 87 If Yes, which treatment do you use regularly? Not applicable Solar disinfection Cloth filtration Sedimentation Chlorine/water guard/bleach Water filter (bio sand/ceramic) Boiling 07 Others 88 If No, why? Not applicable It is expensive Water is safe Do not know how to treat We are used to the water already Other (specify) Sanitation and hygiene Question ID Question Answer 89 Does your family have access to a sanitation facility (toilet/latrine) Yes No 90 If YES, who helped you construct the latrine? Not applicable Self Local authority/Govt NGO (specify) Other (specify) 91 If Yes, which type of sanitation facility is it? Flush to piped sewer system; Flush to septic system; Pour-flush to pit; VIP/simple pit latrine with floor/slab; 05=Composting/dry latrine; Flush or pour-flush elsewhere; Pit latrine without floor/slab; Service or bucket latrine; Hanging toilet/latrine; No facility, field, bush, plastic bag 139 92 How many households share this sanitation facility? Not shared (1 HH) Shared family (2 HH) Communal toilet (3 HH or more) Public toilet (in market or clinic etc.) Don’t know 93 If No Latrine, where your family members go for defecation? Public latrine Neighbour’s latrine Plastic bag Dig a hole/cat hole Creeks/canal/river Bush/backyard/field Other (specify) 94 Where is baby’s/infant’s feces usually being thrown? Toilet Bury Garbage pit Bush throw it on the ground/field River/canal/creek Other (specify) 95 Kindly give me the key times you usually wash your hands? Before eating After eating After defecation After latrine use Before feeding child After handling rubbish After handling baby’s diaper/faeces Before food preparation After handling animals Other (specify); 96 What do you usually use in washing hands? Water only Water & sand/leaves Water & Soap 140 Water & Ash Other (specify) 97 If the answer is 1 (water only), what is the MAIN factor that prevents your family from using soap? Not applicable Washing with soap takes time Soap is not a practice even before Negligence/laziness Water alone cleanses the hand expensive to buy soap Other (specify); B)Attitude Question ID Question Answer 98 Should the positive HIV/AIDS status of family member be kept secret? Yes No Don’t know/not sure 99 Do you believe a teacher with HIV/AIDS should be allowed to keep teaching? Yes No Don’t know/not sure 100 Can you buy fresh vegetables from a shopkeeper with HIV/AIDS? Yes No Don’t know/not sure 101 Have you disclosed your HIV status to: Your partner Your relative Your friend None Others-specify 102 Attending antenatal clinics as and when due are very essential aspect of safe motherhood initiative Yes No Don’t know/not sure 103 Regular antenatal services prevent complication during labour Yes No Don’t know/not sure 141 104 a). During the last pregnancy, did you plan where you would deliver the baby? Yes No Don’t know/not sure 105 b). Where did you plan to deliver the baby? Facility Home 106 c). For what reasons did you prefer to deliver in health facility? 107 d). For what reasons did you prefer to deliver at home? 108 Do you think unmarried young girl should know about reproductive health and family planning? Yes No Don’t know/not sure 109 What is your perception towards discussing reproductive health and family planning with adolescents? Not common in our society Embarrassment/shameful Is normal to discuss 110 Have you ever discussed FP with your spouse/partner? Yes No 111 What is/are your views about contraceptive methods? I have used contraceptives without any problems I have used contraceptives in spite of problems It is a problem to use it has side effects it is against nature I don’t like to use it I never used it Other (specify) 112 Is it good to breastfeed your baby exclusively for six months? Yes No Don’t know/not sure 113 Is it difficult to breastfeed your baby exclusively for six months? Yes No Don’t know/not sure 114 If yes, why? 142 115 Is it good to breastfeed your baby on demand (that is when the baby wants to feed?) Yes No Don’t know/not sure 116 If No, why? 117 If Yes, why? C)Practice Question ID Question Answer 118 How many ANC visits have you attended? 119 Would you recommend a friend to come to this facility for antenatal care services? Yes No 120 If yes, Why? 121 If no, Why? 122 Did you go for antenatal care visits during the last pregnancy? Yes No 123 At what gestational age did you first go for the antenatal checkup during the last pregnancy? 124 How many times did you go for antenatal care during your last pregnancy? 125 During your last ANC visit, did the provider ask you about a) Any complaints you have b) Diet c) Physical activity d) Smoking status e) Whether someone is smoking in the same room f) Alcohol g) Substance abuse h) Intimate partner violence 126 During your last visit did the provider do any of the following? a) Measure blood pressure b) Weigh you using a scale c) Listen to your abdomen for fetal heart d) Palpate your abdomen to measure 143 size of the baby 127 Can you name all tests your provider did during last (current) pregnancy ? a) Blood tests b) Complete blood count c) Hemoglobin testing d) Blood grouping e) Blood RH f) HIV test g) Syphilis test h) Fasting plasma glucose i) 1h plasma glucose j) 2h plasma glucose k) Urine test l) Urine for protein m) Urine for bacteria n) Urine tests for glucose 128 Have you had ultrasound at least once during last pregnancy? Yes No Don’t know/not sure 129 How old was the pregnancy when you had your first ultrasound during last (current) pregnancy? Weeks Months 130 Have you been vaccinated against tetanus during the last (current) pregnancy? Yes No Don’t know/not sure 131 If yes how many doses of vaccine have you received during the last (current) pregnancy? 132 Have you received iron tablets and folic acid during the pregnancy in the health facility? Yes No Don’t know/not sure 133 During the pregnancy, have (did) you received treatment to prevent malaria in the health facility? Yes No Don’t know/not sure 134 How many times did you receive 144 treatment for malaria? 135 Did your provider ever discussed importance of sleeping under insecticide treated nets (ITN)? Yes No Don’t know/not sure 136 Did you receive ITN from the facility? Yes No Don’t know/not sure 137 Where did you deliver your child in your last pregnancy? Facility Home 138 Were you examined vaginally at admission? Yes No Don’t know/not sure 139 Was your Blood Pressure measured at admission? Yes No Don’t know/not sure 140 Was the baby's heart rate listened to at admission? Yes No Don’t know/not sure 141 Did you have a family member/friend with you at all times during admission? Yes No Don’t know/not sure 142 Did each member of the delivery team introduce and identify him or herself when they came into the room? Yes No Don’t know/not sure 143 Did health workers explain to you any examination or procedure before performing them? Yes No Don’t know/not sure 144 Did health workers ask your permission before performing any examination or procedure? Yes No Don’t know/not sure 145 Did you have a companion of your choice with you during your labor? Yes No 145 Don’t know/not sure 146 Did you receive an enema prior to delivery? Yes No Don’t know/not sure 147 Did your health care provider do an episiotomy before you delivered Yes No Don’t know/not sure 148 Were different birth positions discussed with you by your provider Yes No Don’t know/not sure 149 Did provider discussed with you that you could have food and drink during labor? Yes No Don’t know/not sure 150 Were you examined vaginally during labor? Yes No Don’t know/not sure 151 Did your health care provider do an episiotomy before you delivered Yes No Don’t know/not sure 152 Were you encouraged to walk around during the first stage of labor? Yes No Don’t know/not sure 153 Was your baby in skin to skin contact immediately after delivery Yes No Don’t know/not sure 154 Was your baby kept with you in your room for almost the whole time you were in the hospital? Yes No Don’t know/not sure 155 What were you feeding your baby at the hospital? Formula Breast milk Other 156 When were you asked to initiate breastfeeding after delivery? Never 146 immediately after birth In first hour after birth between 1-6 hours after birth > 24 hours after birth I don't remember 157 How often were you advised to breastfeed your baby? Once in every 3 hour with 6 hour night rest Once in every 3 hour day and night Once in every 4 hour with 8 hour night rest Once in every 4 hour day and night On demand as much as possible day and night Other specify 158 After your child was born how long did you stay in the facility? 159 After your child was born how many times did medical staff check on you? 160 How many times was your blood pressure measured? 161 How many times was your temperature measured? 162 How many times was your heart rate or pulse measured? 163 Did your baby receive full clinical examination before discharge (ask to specify how (s)he was examined and consider as full only if at least was undressed, heart and lung were listened, abdomen palpated, weight/height measured, reflexes checked? Yes No 164 Are you using family planning? Yes No 165 If yes, which methods are you using now? Condom Oral contraceptives IUD/spiral Injection Calendar method 147 Withdrawal method Breast feeding Male sterilization Female sterilization Diaphragm Other (specify) 166 Have you received family planning counselling from a CHV in the last 12 months? Yes No Don’t know/not sure Nutrition Question ID Question Answer 167 After the delivery, when did you breastfeed your baby for the first time? Never breastfed Within 1 hour of delivery 1-8 hours following delivery 8-24 hours following delivery Greater than 24 hours following delivery Does not remember when first breastfed 168 How long did you exclusively breast feed without liquid supplements? days weeks months 169 When did you introduce solids? days weeks months A)Knowledge Question ID Question Answer 170 Have you heard of HIV/AIDS Yes No Don’t know/not sure 148 HIV/STIs Question ID Question Answer 171 Having one uninfected, faithful partner prevents HIV/AIDS True False Don’t know/not sure 172 One can use condoms to prevent HIV/AIDS True False Don’t know/not sure 173 One Cannot get HIV/AIDS from mosquitoes True False Don’t know/not sure 174 People can get HIV because of witchcraft or other supernatural means? True False Don’t know/not sure 175 One cannot get HIV/AIDS from sharing food with infected person True False Don’t know/not sure 176 A healthy-looking person can have HIV/AIDS True False Don’t know/not sure 177 A mother can transmit HIV/AIDS to her child during pregnancy and delivery True False Don’t know/not sure 178 A mother can transmit HIV/AIDS to her child through breastfeeding True False Don’t know/not sure 179 There are special drugs that a doctor or a nurse can give to a woman infected with HIV to reduce the risk of transmission to the baby? True False Don’t know/not sure B)Attitude Question ID Question Answer 149 180 Should the positive HIV/AIDS status of family member be kept secret? Yes No Don’t know/not sure 181 Do you believe a teacher with HIV/AIDS should be allowed to keep teaching? Yes No Don’t know/not sure 182 Can you buy fresh vegetables from a shopkeeper with HIV/AIDS? Yes No Don’t know/not sure 183 Can you care for a relative with HIV/AIDS at home? Yes No Don’t know/not sure 184 Have you disclosed your HIV status to: Your partner Your relative Your friend None Others-specify C)Practice Question ID Question Answer 185 Have you been using a condom consistently during sexual intercourse in the last 12 months? Yes No Don’t know/not sure 186 How many sexual partners have you had in the last 12 months 187 Have you been diagnosed for an STI? Yes No Don’t know/not sure 188 a). Have you received treatment for an STI Yes No Don’t know/not sure 189 Have you sought medicine from a traditional healer for an STI? Yes No 150 Don’t know/not sure 190 Have you sought treatment for an STI from a NON-QUALIFIED person or from a place other than a health facility? Yes No Don’t know/not sure 191 Can a wife propose condom use when the husband has STI? Yes No (explain) 192 In the last 12 months, how many doses of ARVs have you missed/forgotten? 193 Are you a member of any psychosocial support group? Yes No 194 Have you had unprotected penetrative vaginal or anal sex with any partner whose HIV status is unknown to you? Yes No Don’t know/not sure 195 Have you had unprotected penetrative vaginal or anal sex with any partner whose HIV status is known to you without protection? Yes No Don’t know/not sure 196 Have you had unprotected sex with a “one night stand” or somebody you have just met or known for less than three months (or had sex “just for fun”)? Yes No Don’t know/not sure 197 Have you had unprotected sex with someone with whom you have had no previous intimate relationship? Yes No Don’t know/not sure 198 Have you performed oral sex on someone or has oral sex act been performed on you? Yes No Don’t know/not sure 199 Have you had sexual intercourse in exchange for a gift, a favour, a promise of something or cash? Yes No Don’t know/not sure 200 Have you exchanged a gift, a favour, a promise or cash to have sex? Yes No 201 Next page A)Knowledge 151 Question ID Question Answer 202 Have you heard of HIV/AIDS Yes No Don’t know/not sure Please indicate whether the following statements regarding HIV/STIs is true, or false, or whether you don't know. Question ID Question Answer 203 Having one uninfected, faithful partner prevents HIV/AIDS True False Don’t know/not sure 204 One can use condoms to prevent HIV/AIDS True False Don’t know/not sure 205 One Cannot get HIV/AIDS from mosquitoes True False Don’t know/not sure 206 People can get HIV because of witchcraft or other supernatural means? True False Don’t know/not sure 207 One cannot get HIV/AIDS from sharing food with infected person True False Don’t know/not sure 208 A healthy-looking person can have HIV/AIDS True False Don’t know/not sure 209 A mother can transmit HIV/AIDS to her child during pregnancy and delivery True False Don’t know/not sure 210 A mother can transmit HIV/AIDS to her child through breastfeeding True False Don’t know/not sure 152 211 Have you ever been beaten or coaxed into sex? Yes No B)Attitude Question ID Question Answer 212 Should the positive HIV/AIDS status of family member be kept secret? Yes No Don’t know/not sure 213 Do you believe a teacher with HIV/AIDS should be allowed to keep teaching? Yes No Don’t know/not sure 214 Can you buy fresh vegetables from a shopkeeper with HIV/AIDS? Yes No Don’t know/not sure 215 Can you care for a relative with HIV/AIDS at home Yes No Don’t know/not sure 216 Have you disclosed your HIV status to: Your partner Your relative Your friend None Others-specify 217 Should the positive HIV/AIDS status of family member be kept secret? Yes No Don’t know/not sure C)Practice Question ID Question Answer 218 Have you used a condom during sexual intercourse in the last 12 months? Yes No Don’t know/not sure 219 How many sexual partners have you had 153 in the last 12 months 220 Have you been diagnosed for an STI? Yes No Don’t know/not sure 221 Have you sought treatment for an STI from a NON-QUALIFIED person or from a place other than a health facility? Yes No 222 Have you sought medicine from a traditional healer for an STI? Yes No 223 a). Have you received treatment for an STI Yes No 224 In the last 12 months, how many doses of ARVs have you missed/forgotten? 225 Are you a member of any psychosocial support group? Yes No 226 Have you had unprotected penetrative vaginal or anal sex with any partner whose HIV status is unknown to you? Yes No Don’t know/not sure 227 Have you had unprotected penetrative vaginal or anal sex with any partner whose HIV status is known to you without protection? Yes No Don’t know/not sure 228 Have you had unprotected sex with a “one night stand” or somebody you have just met or known for less than three months (or had sex “just for fun”)? Yes No Don’t know/not sure 229 Have you had unprotected sex with someone with whom you have had no previous intimate relationship? Yes No Don’t know/not sure 230 Have you performed oral sex on someone or had oral sex act performed on you? Yes No Don’t know/not sure 231 Have you had sexual intercourse in exchange for a gift, a favour, a promise of something or cash? Yes No Don’t know/not sure 154 232 Have you exchanged a gift, a favour, a promise or cash to have sex? Yes No 233 Have you shared any drug paraphernalia with friends or fellow drug users? Yes No 234 Have you shared an injection with anybody? Yes No Don’t know/not sure 235 Have you shared a razor blade? Yes No Don’t know/not sure 236 Have you ever been tested for HIV? Yes No Don’t know/not sure 237 If positive were you referred for care and treatment at the health facility? Yes No Don’t know/not sure 238 Next Page... A)Knowledge Question ID Question Answer 239 Have you heard of HIV/AIDS Yes No Don’t know/not sure HIV/STIs Question ID Question Answer 240 Having one uninfected, faithful partner prevents HIV/AIDS True False Don’t know/not sure 241 One can use condoms to prevent HIV/AIDS True False Don’t know/not sure 155 242 One Cannot get HIV/AIDS from mosquitoes True False Don’t know/not sure 243 People can get HIV because of witchcraft or other supernatural means? True False Don’t know/not sure 244 One cannot get HIV/AIDS from sharing food with infected person True False Don’t know/not sure 245 A healthy-looking person can have HIV/AIDS True False Don’t know/not sure 246 A mother can transmit HIV/AIDS to her child during pregnancy and delivery True False Don’t know/not sure 247 A mother can transmit HIV/AIDS to her child through breastfeeding True False Don’t know/not sure 248 Women should have at least four antenatal check-ups in the life of a pregnancy True False Don’t know/not sure 249 Immunizations protects the baby from infections and diseases True False Don’t know/not sure B)Attitude Question ID Question Answer 250 Should the positive HIV/AIDS status of family member be kept secret? Yes No Don’t know/not sure 251 Do you believe a teacher with HIV/AIDS should be allowed to keep teaching? Yes No 156 Don’t know/not sure 252 Can you buy fresh vegetables from a shopkeeper with HIV/AIDS? Yes No Don’t know/not sure 253 Can you care for a relative with HIV/AIDS at home Yes No Don’t know/not sure 254 Have you disclosed your HIV status to: Your partner Your relative Your friend None Others-specify 255 Would you recommend pregnant women to attend ANC visits at the health facility? Yes No Don’t know/not sure 256 Attending antenatal clinics as and when due are very essential aspect of safe motherhood initiative Yes No Don’t know/not sure 257 Regular antenatal services prevent complication during labour. Yes No Don’t know/not sure 258 Would you recommend pregnant woman to deliver at home? Yes No Don’t know/not sure 259 Is it acceptable if the child misses some immunization schedules? Yes No Don’t know/not sure C)Practice Question ID Question Answer 260 Have you referred suspected STI clients in the past 12 months? Yes No 157 261 Have you referred youths/high risk groups for HIV testing in the past 12 months? Yes No 262 Have you linked the HIV positive clients to care and treatment in the past 12 months? Yes No 263 Have you referred HIV positive clients to psychosocial support groups? Yes No 264 Have you referred pregnant women in the community to the health facility for regular antenatal care services in the past 12 months? Yes No 265 Have you provided family planning information to anyone in the community in the last 12 months? Yes No 266 Did you have a job aid to enable you provide information on family planning? Yes No 267 Have you provided family planning counselling to anyone in the community in the last 12 months? Yes No 268 Have you provided a family planning method to anyone in the community in the last 12 months? Yes No 269 Did you refer any one for family planning services at a health facility? Yes No 270 Did you experience a stock out of family planning commodities in the last 12 months? Yes No 271 Have you participated in any outreach community based services? Yes No 272 If yes Please specify which services are made available during outreach visits? (a) Antenatal Care (b) Rapid Diagnostic Test for Malaria (c) HIV testing (d) Family planning (e) Immunization (f) Other, specify 273 What key health messages have you given to women who have just given birth? (Both at the facility and community level) 158 274 What key health messages have you given to women attending ANC clinic? (Both at the facility and community level) Responses include importance of attending ANC danger signs in pregnancy malaria prevention nutrition birth preparation Anaemia 275 What key health messages have you given to women who have just been discharged from the health facility? Responses include importance of PNC KMC cord care using chlorhexidine breastfeeding immunizations Maintaining Lactation Keeping baby warm & clean nutrition hygiene 276 What key nutrition messages have you given to women who have children below 5years of age? 277 What key hygiene promotion messages have you given to women (both at the facility and in the community? 278 How do you track mothers who do not come for a scheduled follow up visit? End Question ID Question Answer 279 We have come to the end of our interview. do you have any comments? Yes No 280 Thank you for your Participation. 281 End Time 159 6.6 ANNEX 6: LIST OF PERSONS INTERVIEWED No. Name Position Organization 1 Dr. Dan Wendo Chief of Party Afya Jijini Program 2 Mary Kariuki Deputy Chief of Party Afya Jijini 3 June Mwende Human Resources for Health Advisor Afya Jijini Program 4 Samuel Orina M&E Team Afya Jijini 5 Kenneth Meme VMMC Afya Jijini 6 Grace Gitau Quality Improvement Afya Jijini 7 Gladys Someren RH/FP Advisor Afya Jijini 8 Dr Jonathan Kiliko Head of Customer Services Missions for Essential Drugs & Supplies (MEDS) 9 George Nzioka HSCA Missions for Essential Drugs & Supplies (MEDS) 10 Mary Muia, Director of Programs National Organization of Peer Educators (NOPE) 11 Dr. Cyprian N. Kamau Head of Health Programme Christian Health Association of Kenya (CHAK) 12 Lucy Njoki In charge ST. Johns Community Based Organization 13 Gladys Mwende In charge Health rights int. 14 Dr. Janet Muriuki for, Country Director INTRAHEALTH 15 Alice Olawo Senior Program Officer LINKAGES 16 Dr. Osman Warfa Head of NCAH Unit Division of Family Health: -Unit of Neonatal Chid and Adolescent Health 17 Dr. MASINI, Enos Okumu WHO 18 Stephen Muchiri Chief of Party Palladium HP+ 19 Douglas Onyancha Program Manager USAID KEMSA 20 Patrick Mwangi, ICT incharge of LMIS USAID KEMSA 21 John Kibuchi Procurement Manager - Program & Strategic Partnership KEMSA 22 Denis Wanyama AMREF Health Africa (KCO) 23 Victoria Mwenda Dept. of Nutrition UNICEF Kenya 24 Daniel Tewolde Dept. of Nutrition UNICEF Kenya 25 Kate Vorley Chief of Party NILINDE 160 No. Name Position Organization 26 Dr. Hellen Bonuke Team Lead DANIDA 27 Kassim Lupao CONCERN WORLDWIDE 28 Dr. Thomas Ogaro Ag. Chief Officer of Health (COH) & County Director of Health Services (CDHS) CHMT 29 Lisbeth Kageni Program Manager Centre for HIV Prevention and Research (CHIVPR) - University of Nairobi 30 John Muturi Finance Analyst Centre for HIV Prevention and Research (CHIVPR) - University of Nairobi 31 Teresa Simiyu Agreement Officer’s Representative (AOR) USAID/KEA 32 Francis Mutua County HRH CHMT 33 Eric Inda County WASH Coordinator CHMT 34 Dr. Carol Ngunu County AIDS/STI Coordinator (CASCO) CHMT 35 Esther Kiambati County Reproductive Health Coordinator (RHCo) CHMT 36 Dr. Lucina Koyio Deputy Director Health Policy Planning and Research (DDHPPR) CHMT 37 Esther Mogusu County Nutrition Officer (CNO) CHMT 38 Dr. Gregory Miyanga County Partnership Coordinator CHMT 39 Susan Omondi County Quality Improvement Systems Officer CHMT 40 Felistus Referrals CHMT 41 Kimani Nganga County Health Products & Technologies CHMT 42 Kabuga EPI Immunization CHMT 43 Maureen Muganda Deputy County Health Records Information Officer (DCHRIO) CHMT 44 Alfred Owiti Deputy County Director of Health CHMT 45 Judy Macharia County Community Strategy focal point CHMT 46 Laibuta Kaberia County Nursing Officer CHMT 47 Rosaline Makabana County Gender Specialist CHMT 48 Faith Kiruthi Adolescents CHMT 161 No. Name Position Organization 49 Alice Kimani County Health Records Information Officer/Monitoring and Evaluation (CHRIO/ME) CHMT 50 Leah Ruto Maternal, Neonatal and Child Health Coordinator (MNCH) CHMT 51 Elizabeth Mweni TB CHMT 52 Faith Kirethi SNO CHMT 53 Hannah Gethura SCO CHMT 54 Veronica Njeri CEDC CHMT 55 Erick Meda Wash Coordinator CHMT 56 Wanjiku Kabiru CCCO CHMT 57 Eunice Musau DDHA CHMT 58 George Mugo Med. Eng. CHMT 59 Charity N. Manene ACNO CHMT 60 Lillyan M. Mutua CHPC CHMT 61 Nahashon Marebe CMLC CHMT 62 Florence Kabuga SNO CHMT 63 Raphael Muli CDSC CHMT 64 Dr. Judy Gichuki SCMOH MOH Makadara Sub-County 65 Dr. Moses Owino SCMOH MOH Embakasi South Sub-County 66 Dr. Sahra Mohamed SCMOH MOH Lang'ata Sub-County 67 Dr. Samuel Kadivane SCMOH MOH Embakasi North Sub-County 68 Dr. Samuel Kadivane SCMOH MOH Kasarani Sub-County 69 Dr. Nyawira Nyagah SCMOH MOH Mathare Sub-County 70 Dr. Asmaa Awadh SCMOH MOH Westlands Sub-County 71 Dr. Juma Yasin SCMOH MOH Ruaraka Sub-County 72 Charles O. Lwanga Former Technical Advisor Health Commodities and Services Management (HCSM) Project 162 6.7 ANNEX 7: PROGRESS TO ACHIEVEMENT TABLES (Data source: DHIS program data) Green: Achieved or on-track; Yellow: Partially achieved; Red: Not achieved Description Indicator Baseline (Oct 2014 - Sep 2015) Year 1 (Oct 2015 - Sep 2016) Year 2 (Oct 2016 - Sep 2017) Year 3 (Oct 2017 - Apr 2018) Target at year 3 Percentage achievement as of 30 Jun 2018 Sub-Purpose 1: Increased Access and Utilization of Quality HIV Services Output 1.1: eMTCT Services MTCT rates at 18 months of age of the child. 5.2% (326/5963) 5% (172/3132) 2.3% (95/4150) 2.1% (123/5880) Under 5% Although 2.1% positivity rate is below the target rate of less than 5%. The indicator is still considered partially achieved due to a high number of HEI lost to follow-up Number of HIV-exposed infants with a documented outcome by 18 months of age disaggregated by outcome type. 326 172 95 123 Number of HIV-exposed infants who were born 24 months prior to the reporting period and registered in the birth cohort. 5963 3132 4150 5880 Retention of adults, youth 15 - 24 and pediatrics in care & treatment at 60 months 35% 26% 24% 75% 32.3% 163 Description Indicator Baseline (Oct 2014 - Sep 2015) Year 1 (Oct 2015 - Sep 2016) Year 2 (Oct 2016 - Sep 2017) Year 3 (Oct 2017 - Apr 2018) Target at year 3 Percentage achievement as of 30 Jun 2018 Number of ART S patient registered (Oct 2015 to June 2016 Cohorts) 56650 64311 45923 29713 Number of on ART Patient at 12 Months (Oct 2015 to June 2016 Cohorts) 19912 16684 11127 Pregnant women reached with HTC services 96% 94% 94% 88% 95% 92% Number of pregnant women with known HIV status (includes women who were tested for HIV and received their results) 102828 108294 95465 72940 Number of new ANC in targeted facilities (includes facilities 106596 115359 101824 83223 HIV+ pregnant women receive most efficacious regimen for prophylaxis 72% 93% 81% 82% 95% 86% is achieved by quarter three of year 3 and it is unlikely that the target to (increase by 10%) will be achieved by the end of year 3. Number of HIV-positive pregnant women who received antiretrovirals to reduce risk of mother-to￾child-transmission during pregnancy and delivery. 4467 6,400 5,750 4394 Number of HIV-positive pregnant women identified in the reporting period (including known HIV￾positive at entry) 6225 6,898 7,073 5355 Exposed infants receive PCR within 2 months after birth 47% 52% 57% 52% Number of infants who had 95% 55% a virologic HIV test within 2 months of birth during the 2949 3620 4036 2802 164 Description Indicator Baseline (Oct 2014 - Sep 2015) Year 1 (Oct 2015 - Sep 2016) Year 2 (Oct 2016 - Sep 2017) Year 3 (Oct 2017 - Apr 2018) Target at year 3 Percentage achievement as of 30 Jun 2018 reporting period Number of HIV- positive pregnant women identified during the reporting period (include known HIV￾positive women at entry into PMTCT) 6225 6,898 7,073 5355 Output 1.2: HIV Care and Support Services PLHIV (adults, youth 15 – 24 and children) linked to care and receive either clinical assessment, CD4 or Viral load testing per national guidelines Number of the HEI infants linked to care/ Number of the HEI infants positive at 18 months (proxy indicator for above) 35% 15% 22% 2% 90% 2% Number of the HEI infants linked to care 43 58 65 2 Number of the HEI infants positive at 18 months 123 395 293 121 Output 1.3: HIV Treatment Services Eligible PLHIV (adults, youth 15 – 24 and pediatrics) initiated on ART 77% 69% 73% 73% 80% 91% Number of adults and children newly enrolled on antiretroviral therapy (ART) 5449 23658 21309 10690 165 Description Indicator Baseline (Oct 2014 - Sep 2015) Year 1 (Oct 2015 - Sep 2016) Year 2 (Oct 2016 - Sep 2017) Year 3 (Oct 2017 - Apr 2018) Target at year 3 Percentage achievement as of 30 Jun 2018 Number of PLHIV newly diagnosed HIV-positive (know their status 7058 34365 29352 14636 Output 1.4: HIV Prevention, HTC and VMMC Services Key populations (SW, MSM, Sexually Active Young Women 15 – 24) complete standardized HIV prevention interventions NA NA NA NA 80% NA Eligible males circumcised as part of VMMC for HIV prevention 255% 1371% 183% 60% 305% Number of males circumcised as part of the voluntary medical male circumcision (VMMC) for HIV prevention program within the reporting period 10126 10,359 10,299 11,394 Number of eligible males in the target population 4,058 751 6,224 Output 1.5: TB/HIV Co￾infection Services PLHIV screened for TB at HIV sites 417250 473305 389750 80% NA TB patients tested for HIV at TB sites 60% 48% 46% 72% 95% 75% Number of new and relapsed TB cases with documented HIV test results. 3028 746 151 526 Total number of new and relapsed TB cases. 5082 1544 331 735 Co-infected TB/HIV patients enrolled in care 90% 93% 98% 93% 95% 98% Number of TB cases with documented HIV-positive status who start or continue ART 1707 539 134 126 166 Description Indicator Baseline (Oct 2014 - Sep 2015) Year 1 (Oct 2015 - Sep 2016) Year 2 (Oct 2016 - Sep 2017) Year 3 (Oct 2017 - Apr 2018) Target at year 3 Percentage achievement as of 30 Jun 2018 Total number of registered new and relapsed TB cases with documented HIV+ status 1901 579 137 135 Sub-Purpose 2: Increased access and utilization of focused Maternal, Newborn, and Child Health (MNCH), Family Planning (FP), and Water, Sanitation and Hygiene (WASH) and Nutrition Services Decrease in reported cases of newborn deaths at targeted health facilities 35 per 1000 24 per 1000 30 Per 1000 27 per 1000 30% (24.5 per 1000) 27 per 1000 Number of reported cases of newborn deaths at targeted health facilities within the reporting period 2566 1873 1940 1565 Number of live births at targeted health facilities within the reporting period 72405 76804 63576 58202 Decrease in reported cases of maternal deaths at targeted health facilities 1 per 1000 1 per 1000 1 per 1000 1 per 1000 40% 1 per 1000 Number of reported cases of maternal deaths at targeted health facilities within the reporting period 41 38 24 27 Number of live births at targeted health facilities within the reporting period 69839 74931 61636 56637 Output 2.1: Maternal and Newborn Health Services Eligible health facilities offering the 8 EmONC signal functions NA NA NA NA 40% NA Pregnant women who attend 4 ANC visits 58% 55% 60% 63% 50% 126% 167 Description Indicator Baseline (Oct 2014 - Sep 2015) Year 1 (Oct 2015 - Sep 2016) Year 2 (Oct 2016 - Sep 2017) Year 3 (Oct 2017 - Apr 2018) Target at year 3 Percentage achievement as of 30 Jun 2018 Number of women who received at least 4 ANC visits during the latest pregnancy during the project year 61717 63731 61308 52547 Number of expected pregnancies within the targeted catchment areas within the reporting period 106596 115359.00 101824 83223 Percentage of deliveries that assisted with skilled care 94% 102% 82% 72% 55% 131 % Number of deliveries in a given year attended by a skilled birth attendant (SBA) such as a doctor, nurse, or midwife within a reporting period 69839 74931 61636 56637 Number of expected deliveries in the facility catchment area 73938 73522 75011.00 78535.00 Maternal deaths reviewed at targeted health facilities 39% 71% 71% 100% 70% 143% Number of maternal deaths reviewed at targeted health facilities 16 27 24 27 Number of total maternal deaths at targeted facilities 41 38 34 27 Newborns in target areas evaluated by skilled HCW within 2 days of birth 0% 21% 26% 22% Number of newborns who 70% 32% received postnatal care within two days of childbirth in USG-supported programs 0 16051 16051 12712 168 Description Indicator Baseline (Oct 2014 - Sep 2015) Year 1 (Oct 2015 - Sep 2016) Year 2 (Oct 2016 - Sep 2017) Year 3 (Oct 2017 - Apr 2018) Target at year 3 Percentage achievement as of 30 Jun 2018 Number of livebirths in targeted facilities 69839 74931 61636 56637 Perinatal deaths reviewed at targeted health facilities NA NA NA NA 70% Data not reported in DHIS Number of perinatal deaths reviewed at targeted health facilities within reporting period 2566 1873 1940 1565 Data not reported in DHIS Total number of perinatal deaths at targeted facilities within the reporting period Not reported in DHIS Not reported in DHIS Not reported in DHIS Not reported in DHIS Data not reported in DHIS Output 2.2: Child Health Services Children <1 year who are fully immunized 83% 84% 84% 94% 80% 118% Children <5 years with diarrhea who receive ORT (ORS & Zinc) for treatment Not reported in DHIS Not reported in DHIS Not reported in DHIS Not reported in DHIS 80% Data not reported in DHIS Output 2.3: Family Planning Services Increase in number of women of reproductive age using modern contraceptives 52% 56% 40% 22% 63% 35% Number of women of reproductive age (WRA) using modern family planning method 255535 273535 217163 164200 Total number of women 15- 49 years (reproductive age) within the catchment area 490596 491593 544014 736774 Output 2.4: Water, Sanitation & Hygiene Services Decrease in targeted population practicing open defecation 0% 0% 9% 11% Decrease by 2% Need a survey to asses this indicator Number of households NA NA NA NA 169 Description Indicator Baseline (Oct 2014 - Sep 2015) Year 1 (Oct 2015 - Sep 2016) Year 2 (Oct 2016 - Sep 2017) Year 3 (Oct 2017 - Apr 2018) Target at year 3 Percentage achievement as of 30 Jun 2018 without access to a basic toilet facility (optional): Number of households in a catchment area NA NA NA NA Increase in target areas practicing correct use of recommended household water treatment technologies NA NA NA NA Increase by 10% Need a survey to asses this indicator Output 2.5: Nutrition Services Health facilities complying with Infant and Young Child Nutrition guidelines 0% 0% 0% 0% 80% 0% Sub-Purpose 3: Strengthened and Functional County Health Systems Output 3.1: Partnerships for Governance and Strategic Planning Services Functional County Health Stakeholders’ coordination & collaboration mechanism established Yes Yes, based on qualitative interview and document review Annual County Health Program Review Forums conducted Yes Yes, based on qualitative interview and document review Output 3.2: Human Resources for Health Services County Functional County HRH Stakeholders’ Coordination Committees established Yes Yes, based on qualitative interview and document review Quarterly meetings of the HRH Stakeholders Coordination Committees held Yes Yes, based on qualitative interview and document review 170 Description Indicator Baseline (Oct 2014 - Sep 2015) Year 1 (Oct 2015 - Sep 2016) Year 2 (Oct 2016 - Sep 2017) Year 3 (Oct 2017 - Apr 2018) Target at year 3 Percentage achievement as of 30 Jun 2018 Output 3.3: Health Products & Technologies Functional health commodities inventory management practices at county/targeted health facilities established Yes Qualitative and quantitative data shows; there were reports of stock-out of essential medication Functional mechanism for accurate forecasting and supply planning in place Yes Qualitative and quantitative data shows that forecasting system is not established; there were reports of stock￾out of essential medication Output 3.4 Strategic M&E Systems Percentage of high volume health facilities that report complete and accurate program data through DHIS2 50% Percentage of high volume health facilities that conduct quarterly data review and use forums 70% Qualitative and quantitative data shows that Quality improvement system is not established in each HF yet Percentage of high volume health facilities that conduct data quality assessments annually 30% Qualitative and quantitative data shows that Quality improvement system is not established in each HF yet Percentage of health facilities that conducted data quality assessments implement data quality improvement plans 80% Qualitative and quantitative data shows that Quality improvement system is not established in each HF yet 171 Description Indicator Baseline (Oct 2014 - Sep 2015) Year 1 (Oct 2015 - Sep 2016) Year 2 (Oct 2016 - Sep 2017) Year 3 (Oct 2017 - Apr 2018) Target at year 3 Percentage achievement as of 30 Jun 2018 Percentage of high volume facilities that have functional referral systems 45% Qualitative and quantitative data shows that functional referral system is not ready yet ET observations of program strategy, implementation and monitoring  Project strategy is largely facility-focused but included community-based activities without clear connection to CHAs and the facilities where they are based.  Inadequate data management and data use for program planning and adjustment.  Difficult Year one when senior management staff was replaced.  The residual mistrust over payment commitments remains in some areas especially with CHVs and nick-named AJ as “Afya Madeni” which translates as (Afya prone with debts). Although all who cited the challenge were unanimous that they have noted improvements in payment delays over the last one year, they felt that some improvement is still necessary. However, it is still a threat for AJ to recruit new CHVs due to the past reputation.  Lengthy nurse’s strike in 2017 closed many HFs and required many year two activities to be postponed until year three.  Baseline survey focused on informal settlements only and was never approved. It is unclear if data collected can be used.  Yearly targets versus percentage milestones in contract PMP make basis for measurement of progress unclear; there were constant changes (additions) of PMP indicators and targets. For meaningful tracking of progress, AJ Activity should maintain consistency in tracking both contractual and other indicators and targets. This will ensure that trends of additional indicators introduced in the course of implementation by the donor and original indicators.  Contractual indicators, PEPFAR and national indicators definitions are not in sync. Lack of common quantitative measurements of progress towards project goals and objectives makes drawing objective conclusions based on activity PMP very difficult.