0 USAID HIV CLINICAL SERVICES TECHNICAL ASSISTANCE PROJECT (UTAP) MID-TERM EVALUATION REPORT IntraHealth International 7/31/2017 1 CONTENTS Acronyms ........................................................................................................................................................................... 3 Introduction....................................................................................................................................................................... 4 Background ....................................................................................................................................................................... 4 Geographic Scope ...................................................................................................................................................... 4 Expected Results .........................................................................................................................................................5 Objectives of the mid-term evaluation ................................................................................................................... 6 Evaluation Design ...........................................................................................................................................................7 Population.....................................................................................................................................................................7 Sample and Sampling Procedure.........................................................................................................................7 Data Collection Instruments and Data Sources ..............................................................................................8 Data Collection Procedures ................................................................................................................................. 10 Pre-Testing of Instruments and Procedures.................................................................................................. 10 Data Management.................................................................................................................................................. 11 Data Analysis............................................................................................................................................................. 11 Ethical Considerations ........................................................................................................................................... 12 Privacy and Confidentiality.............................................................................................................................. 12 Potential Risks ...................................................................................................................................................... 13 Informed Consent............................................................................................................................................... 13 Evaluation Results and Discussion ......................................................................................................................... 13 HIV Clinical Prevention .......................................................................................................................................... 14 SIMS Prevention Indicator .................................................................................................................................... 14 Domain 7. HIV Testing and Counseling (HTC) ......................................................................................... 14 Prevention Programmatic Indicator ................................................................................................................. 17 HIV Treatment ........................................................................................................................................................... 18 SIMS HIV Treatment Indicator ............................................................................................................................ 18 Domain 1. ART ..................................................................................................................................................... 18 Programmatic HIV Treatment Indicators ........................................................................................................ 21 HIV Drug Resistance Early Warning Indicators (HDREWI) Results ........................................................ 33 HIV Program management.................................................................................................................................. 35 SIMS Domain 21 – Performance Management....................................................................................... 35 SIMS Domain 15 -Medication Management............................................................................................ 40 2 SIMS Domain 20- Site Management -Finance and Planning............................................................. 42 UTAP Targets and Achievements ...................................................................................................................... 44 UTAP Expenditure Costs ....................................................................................................................................... 47 HIV Drug Resistance Early Warning Indicators (HDREWI) Results ........................................................ 49 Conclusions .................................................................................................................................................................... 51 HIV Clinical Prevention.......................................................................................................................................... 51 HIV Treatment........................................................................................................................................................... 51 HIV Program Management.................................................................................................................................. 52 Early Warning Indicators....................................................................................................................................... 52 Recommendations....................................................................................................................................................... 53 Annexes............................................................................................................................................................................ 55 ANNEX A. Programmatic Indicators District and Facility Specific Result Tables ............................. 55 Annex A1. Number Tested and Received results and Percent HIV Positive by Facility, District, HTC_TST and HTC_POS, MTE .......................................................................................................................... 55 Annex A2. Percent of ART Patients Alive and On Treatment 12 Months After Initiation Of Treatment, TX_RET MET .................................................................................................................................... 56 Annex A3. Percent Viral Load Documentation by District and Health Facility, TX_VL MTE .... 57 Annex A4. Percent of New and Relapsed TB cases put on ART while of TB Treatment by Facility, District, TX_RET MTE .......................................................................................................................... 59 Annex A5. Percent Viral Suppression by District and Health Facility, TX_PVLS MTE................. 60 ANNEX B. SIMS Domains MTE Assessment CEE Results by District..................................................... 62 Annex B1. HTC SIMS Domain Assessment CEE Colour Score Results............................................. 62 Annex B2. ART SIMS Domain Assessment CEE Colour Score Results ............................................. 64 Annex B3. Performance Management SIMS Domain Assessment CEE Colour Score Results66 Annex B4. QM/QI SIMS Domain Assessment CEE Colour Score Results ....................................... 67 Annex B5. Medication Management SIMS Domain Assessment CEE Colour Score Results .. 69 ANNEX C. Adult and Paediatric EWI Site Specific Results Extracted From 2016 MOHSS National EWI Assessment Report for UTAP MTE........................................................................................................... 71 Annex C1. Adult EWI Site Specific Results ................................................................................................. 71 Annex C2. Paediatric EWI Site Specific Results ........................................................................................ 74 3 ACRONYMS ANC Antenatal Care ART Antiretroviral Therapy ARV Antiretroviral CBO Community Based Organization CCN Council of Churches Network CEE Core Essential Element COP Community of Practice COP PEPFAR Country Operation Plan CTX Cotrimoxazole DH District Hospital EQA External Quality Assessment EWI Early Warning Indicator FBO Faith Based Organization HCW/HW Health (Care) Worker HIVDREWI HIV Drug Resistance Early Warning Indicator HRH Human Resources for Health HTC HIV Testing & Counseling ICT Information, Communication, Technology IHI IntraHealth International ODK Open Data Kit PEPFAR President’s Emergency Plan for AIDS Relief PMP Performance Management Plan PMTCT Prevention of Mother to Child Transmission PT Proficiency Testing QM/QI Quality Management/Quality Improvement RTK Rapid Test Kit SIMS Site Improvement through Monitoring System TB Tuberculosis USAID US Agency for International Development UTAP USAID HIV Clinical Services Technical Assistance Project MTE Mid Term Evaluation 4 INTRODUCTION The USAID HIV Clinical Services Technical Assistance Project (UTAP) is a four-year project, started in February 2015, with an overall goal of strengthening non-governmental and public health facilities' contribution to the national HIV response through the delivery of high quality, accessible HIV prevention, care and treatment services in order to achieve reduced morbidity and mortality and a reduction of new HIV infections. UTAP’s goal is in line with the Government of the Republic of Namibia’s priority to improve the quality of life of PLHIV and have them live longer by implementing treatment, care and support interventions’1 . In the first year, UTAP worked in five districts - Andara, Nyangana, Oshikuku and Odibo sub-district in Engela, in addition to supporting HIV testing services at CCN New Start Center in Windhoek and providing technical support to St. Mary’s Hospital in Rehoboth. In its second year, the project’s scope and geographic coverage were modified by expanding HIV prevention, care and treatment services to areas of high HIV burden designated as PEPFAR priority regions and hotspots in Namibia. As a result, in addition to the districts that are already implementing UTAP programs since 2015 in the four regions (Kavango, Khomas, Ohangwena, Omusati, and Oshikoto), two new districts namely Tsumeb and Omuthiya in Oshikoto region and one hot spot (Grootfontein Hospital and clinic) in Otjozondjupa region were added in 2016 UTAP has expanded HIV prevention, care and treatment services to areas of high HIV burden and contributes to achieving HIV epidemic control in Namibia through: a) accelerating the treatment services to address unmet need for antiretroviral therapy (ART) by targeting facility￾based HIV prevention/testing services and linkage to care; b) improving early initiation of ART, adherence, and retention among persons living with HIV to achieve viral load suppression; c) improving the quality of clinical care using standardized tools; and d) generating real-time and quality data at health facilities. UTAP activities are intended to support the Government of Namibia in achievement of the targets set out in the National Strategic Framework. UTAP carried out two baseline assessments, one for initial project districts and another for expansion districts, in 2015 and 2016 respectively, forming the basis for comparing results against mid-course performance as will be determined by this mid-term evaluation (MTE). BACKGROUND UTAP activities target priority populations selected for enrollment in treatment, including patients receiving HIV care who are eligible for ART, TB/HIV co-infected patients not on ART, pregnant women receiving PMTCT but not on ART, HIV positive children below 5 years, discordant couples, and newly identified PLHIV from in-patient and out-patient services. Geographic Scope UTAP targets the following districts in the PEPFAR priority regions/districts and one hotspot: Kavango East (Andara and Nyangana); Ohangwena (part of Engela District); Omusati (Oshikuku); 1 National Strategic Framework for HIV and AIDS response in Namibia 2010/11-2015/16 5 Oshikoto (Onandjokwe, Tsumeb and Omuthiya); and one urban hotspot (Grootfontein Hospital and Clinic). For the duration of the agreement, UTAP is supporting 74 health facilities at various levels in the supported districts: Table 1. Number and Type of UTAP-Supported Health Facilities and Community Sites, by Region and District ( n=74) Region District Hospitals Health Centers Clinics Community Sites Total by District Kavango East Andara 1 0 8 0 9 Nyangana 1 0 7 0 8 Ohangwena Odibo 0 1 7 2 10 Oshikoto Onandjokwe 1 3 7 0 11 Tsumeb 1 0 4 2 7 Omuthiya 1 0 7 3 11 Omusati Oshikuku 1 2 13 0 16 Otjozondjupa Grootfontein 1 0 1 0 2 TOTALS 7 6 54 7 74 Expected Results Expected Result 1: Achieved access to HIV treatment coverage to 80% of PLWHIV in the targeted districts by 2018 Expected Result 2: Strengthened capacity of HCWs to deliver high quality, HIV prevention, care and treatment clinical services Expected Result 3: Utilized interoperable health information systems to improve quality assurance, efficiency, and sustainability of integrated HIV care and treatment services in public and NGO health care facilities. Expected Result 4: Increased technical, management and financial capacity of health care NGOs to sustain their contribution to the national HIV response To achieve the project goal and expected results, IntraHealth and its partners guide project efforts to achieve the following major milestones: 1. Provider initiated HIV testing and counseling is implemented at the primary and secondary health care facilities 2. Health workers providing quality HIV services integrated with MCH/FB/RH/TB 3. Districts have increased HIV treatment coverage to 80% of the PLHIV 4. District health systems support delivery of critical package of services customized to the needs of the population (differentiated health care model) 6 5. Faith based organization-supported facilities operate efficiently with improved financial, management, administrative, data use, procurement, asset management systems. Based on the review of a range of relevant documents in Namibia, IntraHealth designed a Performance Management Plan (PMP) to establish a systematic process to monitor and evaluate the achievements of the technical assistance project. This M&E system is generating evidence that is currently being used by MoHSS health care administrators, health workers, and IntraHealth program staff to make mid-course corrections to achieve desired program outcomes. By way of consolidating the evidence base of the PMP, IntraHealth has set out to conduct a MTE to determine the progress of the project after two years of implementation. Purpose of the Mid-term Evaluation This evaluation will be conducted as required of IntraHealth by the four-year UTAP contract agreement with USAID Namibia. The MTE will assess mid-term progress towards the achievement of the project objectives and outcomes as specified in UTAP documents, and assess early signs of project success or failure with the goal of identifying the necessary changes to be made in order to set the project on-track to achieve its intended results. The MTE should also identify problems and indicate solutions for remedial actions. OBJECTIVES OF THE MID-TERM EVALUATION The table below summarizes the objectives of the mid-term evaluation and their corresponding expected results (ER) of UTAP No Objectives Correspondence to expected results (ER) 1 Site’s ability to manage basic capacity building interventions (in￾service training performance) to strengthen HWs HIV clinical care competences ER 2 2 Track the facilities that have standards/guidance on identifying staff performance and creating opportunities to enhance capacity (performance review, training schedule, training tracking, supportive supervision) ER 2 3 Track facilities that have standards and QM/QI initiatives and IT systems in place to improve performance, efficiency and sustainability of services (clinical mentoring, QM/QI plan, quality monitoring and reporting – linkages between clinical services) and reporting ER 3 4 Identify the health care organizations that have the skilled manpower to properly plan and budget for and manage HIV clinical services (HRH, finance, administrative and management systems) ER 4 5 Track the extent to which health facilities have achieved acceptable levels of performance on key HIV clinical care programmatic indicators ER 2 6 Determine the level of utilization (expenditure) of inputs by program/management areas ER 5 7 EVALUATION DESIGN The evaluation has employed a quantitative explorative and descriptive approach where characteristics of interest will be measured and described in figures with the view to achieving the stated objectives. For programmatic indicators, the reference period for performance evaluation was April 2016 to March 2017 covering the last two quarters of COP15 and the first two quarters of COP16. Population Fifty-two public and faith based health facilities (hospitals, health centres and clinics) providing ART clinical services in the eight UTAP districts constituted the target for the MTE. Health facilities that do not provide ART services were excluded. In general, health workers working in these health facilities providing HIV treatment service constitute the target population for the mid-term evaluation. The following cadres are included under the term ‘health worker’: nurses, doctors, pharmacists, laboratory technicians, data workers, health assistants, health facility administrators, human resources officers, finance officers and facility managers. Table 2. UTAP-supported Facilities that Provide ART Services, by Region and District Region District Hospitals Health Centers Clinics Community Sites Total by District Kavango East Andara 1 0 7 0 8 Nyangana 1 0 4 0 5 Ohangwena Odibo 0 1 4 0 5 Oshikoto Onandjokwe 1 3 7 0 11 Tsumeb 1 0 2 0 3 Omuthiya 1 0 6 0 7 Omusati Oshikuku 1 2 8 0 11 Otjozondjupa Grootfontein 1 0 1 0 2 TOTALS 7 6 39 0 52 Sample and Sampling Procedure Among the 52 public and faith based health care facilities providing ART clinical services in the eight UTAP support districts, seven hospitals, three health centers and 13 health clinics were included in the mid-term evaluation. The sampling of health facilities was done in two strata: hospital strata and health center/clinic strata. Since there is only one hospital in each of the seven districts, they were all enrolled automatically. Since Odibo, the eighth district, has no hospital in the UTAP program, the only health center, Odibo, was taken automatically. 8 In strata two, in the five districts that have no health center, two clinics were selected at random and in the three districts that have one or more health centers, one health center and one clinic were selected randomly. Where there is only one health center or one clinic in a district, that health center or clinic was included automatically. The procedures for the sampling of individuals and/or documents for review in any selected health facility are specifically described in the data collection tools of the different SIMS domains, program indicators, and cost centers. Table 3. UTAP Supported Facilities Sampled for the MTE, by Type and District District Hospital Health Center Clinic Total Andara Andara District Hospital - 1. Divundu 2. Bagani 3 Nyangana Nyangana DH - 1 .Mbambi 2. Katere 3 Odibo - Odibo HC 1. Hamukoto Wa kapa 2. Okatope 3 Onandjokwe Onandjokwe DH Onyanya 1. Elombe 3 Tsumeb Tsumeb DH - 1. Tsinsabis 2. Catherine Bullen 3 Omuthiya Omuthiya DH - 1. Omuntele 2. Onyuulaye 3 Oshikuku St. Martins DH Okalongo 1. Onheleiwa 3 Grootfontein Grootfontein DH - 1. Grootfontein clinic 2 TOTAL 7 3 13 23 Data Collection Instruments and Data Sources Objective 1 The Site Improvement through Monitoring Systems (SIMS) tool developed by PEPFAR for assessing performance management was used for the mid-term evaluation. The in-service training CEE measures sites’ ability to keep written record of training logs, have selection criteria for what staff members attend trainings and sites’ ability to provide staff coverage while their staff members are at training. Objectives 2-4 The Site Improvement through Monitoring Systems (SIMS) tool developed by PEPFAR for assessing the performance and quality of HIV services was used for the mid-term evaluation. The use of SIMS plays an important role as it is the tool that is recommended for use in monitoring the progress of site level HIV programs during implementation of most USAID 9 programs including the UTAP project. These tools were also used in the baseline assessment for UTAP. SIMS uses checklists which assess individual facility’s performance against the current PEPFAR recommendation on different areas of the continuum of care and support services. The responses which depict site performance are color coded to display the critical nature of the challenge/success in performance of the specific indicator with dark-green being the best performance, light-green needs minimal improvement, yellow needs substantial improvement, and red indicates the indicator requires urgent attention. Among more than 20 domains of SIMS, only seven domains those that are believed to give strategic indication of the level of performance of services corresponding to UTAP’s expected result areas were included. These are:  SIMS domain 1. ART  SIMS domain 7. HIV Testing and Counseling  SIMS domain 15. Medication Management  SIMS domain 20. Site Management - Finance and Planning  SIMS domain 21. Performance Management  SIMS domain 23. Site Management - Quality Management/Quality Improvement Broadly, the SIMS tool explored the following dimensions of HIV clinical services: availability and application of performance and quality standards, compliance to national/global standards, availability and use of quality monitoring and improvement system, availability and use of information system for M&E of UTAP. Depending on the checklist item, facility-held service registers and individual HIV patient/client medical records in paper or electronic forms served as the sources of data for SIMS indicators. SIMS Facility Assessment Scoring Scheme Core Essential Element (CEE) Color Score Description Dark green (4) Surpasses Basic Expectations Light green (3) Meets Basic Expectations Yellow (2) Needs Improvement Red (1) Needs Urgent Remediation 10 For some SIMS items, health workers, facility in-charges, and other facility-held documents served as data sources. Objective 5 A data extraction form was designed to obtain data items that are needed to construct the numerators and denominators of a select list of programmatic indicators; HIV testing services; antiretroviral therapy initiation, retention in care, viral load documentation and viral suppression, and treatment of TB/HIV co-infection. A combination of facility held service registers and individual medical records in paper or electronic forms were the sources of data. For HIV drug resistance Early Warning Indicators2 , secondary data from the most recent (2016) national assessment done by the MOHSS was used. Objective 6 Quarterly reports on expenditures submitted to USAID by IH Namibia finance office in Windhoek was used to obtain data for UTAP program cost analysis. Table 4. Summary of data sources for mid-term evaluation Paper-based registers Electronic registers - HIV Counseling and Testing Registers - HIV Rapid Testing Log Book - ART register - TB register - Quarterly COP reports - Quarterly expenditure reports -HIV Counseling and Testing -Electronic patient management system (ePMS) Data Collection Procedures Data was collected by four teams of enumerators contracted from the University of Namibia. Each team, comprised of four data collectors, where one served as a supervisor as well in each group, was responsible for evaluating two UTAP districts. The data collectors conducted interviews with health workers and facility managers, reviewed electronic or paper-based health care registers and individual patient medical records as applicable. Data collectors and supervisors were thoroughly trained; detailed procedures were provided in the various sections of the data collection instruments and with each data item to be collected. Pre-Testing of Instruments and Procedures All evaluation tools and procedures were tested at one district hospital (Rehoboth) outside the UTAP evaluation areas. Based on observations during the pilot some changes were made including dropping certain program indicators listed in the evaluation protocol. 2 Report on Early Warning Indicators of HIV Drug Resistance in Namibia; MOHSS, 2017 (Unpublished) 11 Data Management Data was collected from the various sources and directly entered in real time onto tablets loaded with data entry templates developed with Open Data Kit (ODK) software. One data collector along with a supervisor in each team ensured that electronically captured data is saved and secure, assessment data is of good quality (complete and accurate) and give feedback to data collection teams at the earliest possible time before the data is sent to the central IT server. All tablets were collected and kept with the central IT coordinator and data collectors were denied access code one week after field work as soon as data cleaning and verification was completed. One IH Namibia IT staffer oversaw and coordinated the functionality of mobile devices and ODK software from Windhoek office; this IT person kept open communication with field teams and gave feedback and technical support. To protect the confidentiality of participating individuals and data safety, the assessment team was assigned a tablet that is secured with an access code; furthermore, only specific members of the team were given access. Once the data is saved it went to the secure central IT server and was only to be accessible to IntraHealth’s HIS technical advisor and IT coordinator. ODK was used to capture site level data and no individual patient information was entered. Data Analysis Since the assessment was intended to provide information on the performance of the district clinical HIV program in line with the stated objectives, quantitative descriptive information was analyzed. Objective 1: The number (and percentage) of health facilities that have achieved red, yellow, light and dark green color scores on SIMS performance management in – service training indicators. The results will be displayed on a dashboard to indicate levels of performance by facility and district. Objectives 2-4: The number (and percentage) of health facilities that have achieved red, yellow, light and dark green color scores on SIMS indicators. The results will be displayed on a dashboard to indicate levels of performance ranging from ‘Surpasses expectations’ to ‘Needs Urgent Remediation’, then compare baseline results with MTE to determine whether UTAP is doing the same, better or worse. In comparing SIMS baseline results with MTE at district level, first, the total number of CEE scores obtainable is calculated by multiplying the number of health facilities in a district by the number of CEEs of any SIMS domain to get 100% as a denominator, and second, the number of CEEs that Meet or Surpass basic expectation is counted as a numerator. Dividing the numerator by the denominator gives the proportion (percent) of CEEs that Met or Surpassed basic expectation. Large differences are noted between assessment results of MTE and HIV drug resistance Early Warning Indicator assessment conducted in 2016 (EWI16) in some districts; a possible explanation is difference in the data sources used between the two methodologies. 12 Objective 5: The number (and percentage) of health facilities that have achieved satisfactory performance levels per national/international standards on key programmatic indicators, including HIV drug resistance early warning indicators, then to compare with baseline results to determine where UTAP at mid-point is doing better or worse or the same. Objective 6: The extent to which UTAP technical and financial input has been utilized (expenditure) by programs areas. Comparison between MTE and Baseline and between MTE and EWI Results MTE vs Baseline/EWI Color signal MTE better than Baseline/EWI Green MTE about the same as Baseline/EWI Yellow MTE worse than Baseline/EWI Red Difference Size Representation Interpretation 0% – 5% MTE = Baseline/EWI no or minimal difference 6% - 15% MTE > or < Baseline/EWI modest difference 16% - 35% MTE >> or << Baseline/EWI big difference 36% + MTE >>> or <<< Baseline/EWI excessive difference Ethical Considerations The UTAP MTE employed the same protocol as the UTAP baseline assessment that was reviewed and approved by the Namibian MOHSS IRB in 2016. The MOHSS Directorate of Special Programs considered the baseline a project assessment and not research. Thus, by extension, this midterm evaluation is also considered an internal project assessment not necessitating additional approval from an external review board. Privacy and Confidentiality The data sources for the two major methodologies – SIMS and programmatic indicator assessment – are individual patient/client and facility medical records. The MTE ensured that no patient/client-identifying information are extracted and recorded in data collection tools. The mobile data collection tool was only used to capture site level data and no individual patient information was entered. While individual patient/client and facility medical records were reviewed to assess clinical performance, the MTE ensured that no patient/client-identifying information are extracted and 13 recorded in data collection tools. Data collection teams and their supervisors have received training on careful handling and respecting of patient/client information. Teams requested from facilities a dedicated private space or room when handling records and clinic documents and made sure that these documents are kept secure from the public while in their care. To protect the privacy of participating individuals and data safety, the assessment team was assigned a tablet that is secured with an access code; furthermore, only specific members of the team were given access to the data base in ODK. The names of health care workers and other facility staff who participated in individual or group interviews were not recorded. Once the data is saved it went to the secure central IT server and became accessible to the MTE lead consultant and IntraHealth Namibia IT Officer only. Potential Risks It was determined that since the MTE identified facilities in the report, it may be possible to guess the identity of participating health workers and other facility staff putting them in low risk. Unlike research, M&E needs to identify gaps and their sources with the aim of targeting those areas/departments and workers with support for improvement. This being the aim, there is a possibility that some managers may use this information to wrongly reflect unfavorably on those targets. To minimize this risk, the evaluation team emphasized the aim of the evaluation in the consent and when findings are shared with stakeholders. Overall, the information collected is considered low-risk since it is focused on facility assessments. It is strongly believed that conducting the mid-term assessment will promote development of a facility level performance improvement action plan by mobilizing technical and resource support. Informed Consent Each health worker and other facility staff who participated in individual and group interviews received an informed consent form and asked to verbally agree to being interviewed prior to beginning. No names were recorded on this consent form. EVALUATION RESULTS AND DISCUSSION In this section, the findings from the SIMS, Programmatic and Early Warning Indicators data reviews will be presented and discussed. The presentation is divided into HIV Clinical Prevention, HIV Treatment, and HIV Program Management sections. For each indicator, findings on all the sampled health facilities in the eight districts are presented and discussed in aggregate in the main body of the report and individual district and/or facility results are summarized in the Annex. The results of the MTE are discussed in relation to expected performance levels and against baseline results and, where applicable, findings of the National HIV drug resistance EWI report will be compared against similar program indicator results of the MTE. 14 HIV Clinical Prevention SIMS Prevention Indicator Domain 7. HIV Testing and Counseling (HTC) HIV testing and counseling service is the first step towards meeting the goal of HIV epidemic control by educating and encouraging people who do not know their HIV status to get tested. The service is critical to identify HIV positive clients early and link them to Care and Treatment. National HIV programs are required to develop guidelines and standards to be strictly followed by all HTC service providers with focus on health care workers in medical care facilities. All 23 health facilities were evaluated against this SIMS indicator. A large majority of facilities met or surpassed expectations on all but two CEEs of HTC domain: HTC referrals and HIV proficiency testing. On HIV proficiency, nine facilities were below expectation (scored a yellow or red) and on HTC referral CEEs, 15 facilities scored Needs Urgent Remediation (red) or Improvement (yellow). This 61% fail rate in HTC referrals affects the rest of the clinical cascade, and intervention efforts should be prioritized. Table 5. Comparison of SIMS HTC Domain Assessment Results between MTE and Baselines, All UTAP Districts SIMS Domain 7. HTC MTE 2017 Baseline 2015 Baseline 2016 Number % Number % Number % Comment Met or surpassed basic expectation 128 80 72 64.3 45 64.3 MTE >> Baseline Failed to meet basic expectation 33 20 40 35.7 25 35.7 Total 161 100 112 100.0 70 100.0 1 1 0 12 2 0 0 3 1 3 3 7 0 0 1 5 0 4 2 1 17 18 16 20 4 12 22 6 7 . 1 C O M P L I A N C E W I T H N A T I O N A L A L G O R I T H M 7 . 2 H I V T E S T I N G Q A 7 . 3 H T C D O C U M E N T A T I O N 7 . 4 H T C R E F E R R A L S 7 . 5 H I V P R O F I C I E N C Y T E S T I N G 7 . 6 S U P P L Y C H A I N R E L I A B I L I T Y 7 . 7 H T C S A F E T Y M E A S U R E S FIG 1. HTC - FACILITY SIMS CEE COLOR SCORE (N=23) Needs Urgent Remediation Needs Improvement Meets Basic Expectations Surpasses Basic Expectstions 15 The 23 health facilities assessed on the seven CEEs of the SIMS HTC domains by the MTE produced a total of 161 scores, of which 128 (80%) represented meeting or exceeding basic expectations. This finding is certainly a much better performance compared to the 64% reported as meeting or exceeding basic expectations in both the 2015 and 2016 baseline assessments. Table 6. Comparison of SIMS HTC Domain Assessment Results Between MTE and Baseline by District Andara, Omuthiya, Oshikuku and Tsumeb districts demonstrated sizeable improvement in percentage of CEEs that Met or Surpassed expectation in the MTE compared to baseline; Odibo, Onandjokwe and Grootfontein districts remained essentially the same. Nyangana district worsened by 15 percentage points in the MTE from its baseline. Compliance with National Testing Algorithm and Strategy Each site should perform rapid HIV testing in accordance with national testing algorithms and strategies. This CEE assesses the extent to which UTAP facilities comply with the Namibian national two-test algorithm. Review of the 20 most recent entries where the first rapid test was HIV positive in the HTC register/rapid testing logbook found out that, in one health facility’s register only <14 (<70%) entries did follow the two-test algorithm, i.e., >30% were not compliant Needing Urgent Remediation (red), and 14-18 (70%-90%) entries in three health facilities’ registers/log-books complied with the two-test algorithm Needing Improvement. The rest of the assessed facilities (19) had logbooks where at least 95% of entries were compliant and had written testing protocols present. HTC UTAP District % CEEs Met or Surpassed Basic Expectations Comment MTE Baseline Andara 81 57 MTE >> Baseline Nyangana 71 86 MTE < Baseline Odibo 81 86 MTE = Baseline Omuthiya 81 69 MTE > Baseline Onandjokwe 76 79 MTE = Baseline Oshikuku 81 64 MTE >> Baseline Tsumeb 90 48 MTE >>> Baseline Grootfontein 79 79 MTE = Baseline 16 HIV Testing Quality Assurance Each site should have QA procedures in place for HIV rapid testing that include direct observation, the use of standardized laboratory logbooks, running quality controls on rapid test kits, and annual refresher trainings for HTC providers. One health facility Needs Remediation (red) because it did not use one or more of the five tools/approaches that ensure and monitor the quality of rapid HIV testing, and one facility Needs Improvement (yellow) because it did not have a site manager or laboratorian to observe and document that its HTC provider is conducting rapid testing to monitor proficiency. Most of the facilities (21/23) Met or Surpassed basic expectations (green) because they complied with the HTC tools and approaches to ensure and monitor the quality of rapid testing; their HTC registers/rapid testing logbooks were reviewed quarterly and/or HIV testing staff have completed annual refresher trainings. HTC Documentation Each site should maintain accurate and complete standard data collection and reporting tools for all nationally required HTC indicators. Three of the 23 facilities Need Improvement (yellow) because their HTC registers did not meet all of the criteria for complete HTC register documentation (using registers provided by the MOHSS, legible and complete entries, updated weekly, registers used for routine facility reporting, and registers stored in a secure location that maintains confidentiality); while the remaining 20 facilities Surpassed Basic Expectation (dark green) as they review their HTC registers at least monthly and use them to inform HTC services. HTC Referrals to HIV Care and Treatment All service delivery points that provide HTC should have a standardized system for tracking successful referral of HIV-infected clients to HIV care and treatment services. Twelve of the 23 health facilities did not have a system in place with standard tools to track referrals to care and treatment and Need Urgent Remediation (red) and three health facilities had <40% successful referrals (yellow) failing to meet basic expectations. Four health facilities Met Basic Expectations (light green) and four other facilities Surpassed Basic Expectations (dark green) with successful tracking of 60-89% and over 90% of referrals, respectively demonstrating that they know their clients were linked to HIV care and treatment services. Site Level HIV Proficiency Testing Whenever proficiency testing (PT) is available the sites should participate in (and pass) proficiency testing (PT/EQA) for HIV testing. Two health facilities Need Urgent Remediation (red) because they were not enrolled in proficiency testing and seven facilities Need Improvement (yellow) because they completed between 50-79% of PT panels or they completed >80% PT panels but did not take corrective 17 actions when they failed proficiency testing. Two facilities took corrective actions when they failed PT, which means they met Basic Expectations (light green). The remaining 12 facilities achieved 100% score on their most recent PT panel exceeding Basic Expectations (dark green). Prevention Programmatic Indicator Number HIV Tested and Received results and Number HIV Positive (HTC_TST, HTC_POS) The number of people who tested for HIV, received their results and were positive were obtained from each MTE sampled health facility’s monthly report for each month from April 2016 to March 2017. The number reported was verified against the number from HTC source documents and where the discrepancy between the two was greater than 10% for any two selected months, the number from the source document was recounted by the MTE team and entered in the data extraction form. Table 7. Number HIV Tested and Received results and Percent HIV Positive by District During April 2016 – March 2017, MTE (n=51, 281) District Number of HIV Tested Number HIV results received Number HIV (+) result received % HIV+ Andara 3712 3712 310 8.4% Nyangana 4285 4285 202 4.7% Odibo 5390 5312 282 5.3% Oshikuku 9462 9406 499 5.3% Onandjokwe 14629 14623 709 4.8% Omuthiya 4692 4662 267 5.7% Grootfontein 4999 5176 285 5.5% Tsumeb 4112 4094 273 6.7% All Districts 51281 51270 2827 5.5% The number of individuals who tested for HIV is greater than the number who received their results in five districts and was the same in two districts as would be expected. In one district however, the number of individuals who received their HIV test results was more than the number who were tested. This should be an error in reporting by the facilities or an error in counting by our data collectors. 18 The overall HIV positive rate in all eight UTAP districts is 5.5%. Andara district documented the highest HIV positivity rate (8.4%) followed by Tsumeb (6.6%) and Nyangana district documented the lowest (4.7%) followed by Odibo (5.7%). Similarly, UTAP Q1 and Q2 2017 reports showed that Andara and Nyangana districts consistently documented the highest and lowest HIV positivity respectively. The HTC data on the 23 sampled facilities and eight districts is shown in Annex A1. HIV Treatment SIMS HIV Treatment Indicator Domain 1. ART The four CEEs under the Anti-Retroviral Therapy domain represent critical aspects of ART retention that, if not in place, greatly hinder the ability of facility staff to track and monitor ART patients and ensure continued adherence. 0.0% 1.0% 2.0% 3.0% 4.0% 5.0% 6.0% 7.0% 8.0% 9.0% 0 2000 4000 6000 8000 10000 12000 14000 16000 Fig. 2. Number Tested and Percent HIV Positive by District, MTE Number of HIV Tested % HIV+ 19 All 23 health facilities were able to report against this domain. Overall, of the 92 CEE colour score cards available on the ART SIMS domain, 70 (76%) were light or dark green indicating majority of the facilities meeting or surpassing basic expectation. On the other hand, 22 (24%) were red or yellow scores indicating failure to meet basic expectation. The best and worst results among the four ART CEEs was obtained on Adherence Support and Patient Tracking with 20/23 and 16/23 assessed facilities meeting or surpassing basic expectation respectively. Table 8. Comparison of SIMS ART Domain Assessment Results between MTE and Baselines, All UTAP Districts As can be seen in the table (8), 70 (76%) CEES Met or Surpassed basic expectations showing a sizeable improvement over the two baseline assessments on ART SIMS indicators, in which the comparable figures for 2015 and 2016 baselines were 62.5% and 50% respectively. 3 0 3 4 4 3 3 2 8 2 4 5 8 18 13 12 1.1 PATIENT TRACKING (ART) 1.2 ADHERENCE SUPPORT 1.3 ART MONITORING 1.4 SUPPLY CHAIN RELIABILITY FIG. 3. ART FACILITY SIMS CEE COLOR SCORE (N=23) UTAP MID-TERM EVALUATION Need Urgent Remediation Needs Improvement Meets Basic Expectations Surpasses Basic Expectstions SIMS Domain 1. ART MTE 2017 Baseline 2015 Baseline 2016 Number % Number % Number % COMMENT Met or surpassed basic expectation 70 76 30 62.5 18 50 MTE > Baseline Failed to meet basic expectation 22 24 18 37.5 18 50 Total 92 100 48 100.0 36 100.0 20 Table 9. Comparison of SIMS ART Domain Assessment Results Between MTE and Baselines, by District Nyangana did worse in the MTE than the baseline while the other seven districts did better than (5) or were more or less similar to (2) the baseline on percent of CEEs that met or surpassed basic expectations of ART SIMS domain. Patient Tracking This CEE represents the minimum requirements for patient tracking – each ART site should have a standard procedure for identifying and tracking ART patients who have defaulted on their appointments. The system should be fully implemented and contain the following core elements: defined staff roles/responsibilities, procedures for patient identification and tracking, and standardized documentation. A majority of sites 16 (70%) did perform satisfactorily on patient tracking, i.e., they Met or Surpassed basic expectation. Three of the 23 facilities did not have standard procedures for identifying and tracking ART patients who have defaulted on their appointments, thus necessitating Urgent Remediation (red). Four facilities Need Improvement (yellow score) because their ART patient tracking documentation was not complete and did not show evidence of defaulted ART patients brought back into care. Eight facilities used tracking results to update site indicators, such as loss to follow up earning Light Green score, while another eight had written SOP for identifying and tracking defaulters to earn a Surpasses Basic Expectations (dark green) score. Adherence Support Each site that provides ART should have a standard adherence support protocol for providing and documenting all the following core elements: 1) Adherence counselling prior to ARV treatment initiation; 2) Routine adherence assessments during ARV therapy; and 3) Counselling interventions for patients with poor adherence. UTAP District % CEEs Met or Surpassed Basic Expectations Comment MTE Baseline Andara 83 87 MTE = Baseline Nyangana 67 100 MTE << Baseline Odibo 67 50 MTE >> Baseline Omuthiya 100 50 MTE>>> Baseline Onandjokwe 67 63 MTE = Baseline Oshikuku 75 63 MTE > Baseline Tsumeb 67 50 MTE>> Baseline Grootfontein 88 63 MTE >> Baseline 21 Three facilities did not meet expectations with Needs Urgent Remediation (red) score because they did not have any adherence support procedures in place and two health facilities obtained a Needs Improvement (yellow) score because their sample of patient records showed that less than 60% had any documentation of an adherence assessment at the last clinical visit. The remaining 18 facilities that achieved a Meets or Surpasses Basic Expectations (light or dark green) demonstrated adherence assessment documentation for at least 80% of a sample of patient records. ART Monitoring All patients on antiretroviral therapy (ART) should receive routine monitoring for treatment failure through regular assessment of CD4 and/or viral load per national guidelines, and results should be documented in the medical record. Three of the 23 facilities had less than 50% of charts reviewed had documentation of CD4 and/or VL within the appropriate interval per the national guidelines, thus requiring Urgent Remediation (red), and three facilities had 50-69% of reviewed patient charts having CD4 and/or viral load test results documented, earning a Need Improvement (yellow) score. Two facilities had CD4 and/or viral load documentation in 70-89% of charts earning a Met Basic expectation (light green) score, while 18 facilities surpassed basic expectation because they had CD4 and/or viral load documented in at least 90% of patient charts and were able to go a step further and present written SOP or a clinic algorithm for monitoring patients on ART and responding to CD4 and viral load test results. Supply Chain Reliability In order to ensure patients are fully compliant to their treatment, each site should have a reliable supply of ARVs. Four of the 23 health facilities scored a Needs Urgent Remediation (red) because they experienced a stock-out of ARVs in the past three months that resulted in an interruption of ART for patients; two health facilities had a stock-out which did not interrupt ART therefore indicating “Needing Improvement (yellow)”. Five health facilities had to give patients appointments at shorter intervals due to decreased ARV supply and the remaining 12 had no problem with ARV supply at all achieving Meets (light green) and Surpasses (dark green) Basic Expectations scores respectively. Programmatic HIV Treatment Indicators Number of Adults and Children Newly Enrolled on ART (TX_NEW) The indicator measures the ongoing scale-up and uptake of ART programs and it is critical to monitor along with number of patients currently on ART and the progress of the HIV services cascade, specifically the successful linkage between HIV diagnosis, enrollment in care services and initiating ART. This is particularly important in the context of current revisions to Namibia’s ART eligibility. 22 Table 10. Number of Adults and Children Newly Enrolled on ART by Age Group, April 2016 - March 2017, MTE (n=3087) A total of 3087 new patients were enrolled on ART between April 2016 to March 2017, of which only 184 (6%) of them belonged to pediatric age group. 8% 8% 10% 21% 13% 20% 10% 10% 0% 5% 10% 15% 20% 25% Fig. 4 Percent Adults Newly Started on ART, April 2016 - March 2017 by District, MTE (n=2903) Age Group Number Percent Adult 2903 94% Pediatric 184 6% Total 3087 100% 23 As can be seen in figures 4 and 5, the highest proportion of new adult and pediatric ART patients were from Oshikuku district (21% each age group) followed by Onandjokwe district (20% each age group) while the lowest proportion were from Andara and Nyangana districts for adult (8%) and Grootfontein (3%) and Tsumeb districts (6%) for pediatric patients. Percentage of ART Patients with Viral Load Results Documented within the Past 12 Months (TX_VL) This programmatic indicator seeks to measure facilities’ observance of national viral load testing policies. Viral load testing is the standard assessment of clinical and immune status. Periodic viral load testing after initiation of ART is also a means of monitoring ART adherence. Those facilities that ensure consistent access to ART and continuous patient adherence to medication should see clients with low viral loads. The Namibian National viral load testing guideline stipulates that adult ART patients should have viral testing every six months during the first year of ART initiation and one viral load testing every 12 months in the second year of initiation of ART and thereafter. Pediatric ART patients should have one viral load testing done every six months. Table 11. Documentation of Viral Test Results During April 2016 – March 2017, UTAP Mid-Term Evaluation Viral Load Documentation Yes Number (%) No Number (%) At least one Viral Load Documented (n=343) 294 (85.7) 49 (14.3) Viral Load Properly Documented (n=342) 194 (56.7) 148 (43.3) 14% 12% 10% 21% 15% 20% 6% 3% 0% 5% 10% 15% 20% 25% Fig 5. Percent Pediatric Newly Started on ART, April 2016 - March 2017 by District, MTE (n=184) 24 In the mid-term review a total of 343 ART patient records (279 adult and 63 pediatric) were sampled from 23 health facilities (7 hospitals, 3 health centers and 13 clinics) for review of viral load documentation per national guidelines. Of this total, 148 (43.3%) including 49 that don’t have a single viral load result over 12 months of review, do not have proper viral test result documented per national guideline. Fifty seven percent of adult and 54% of pediatric ART patient records reviewed have viral load results correctly documented per national guideline that is higher than the 2015 baseline, which was 45% of adult and 30% of pediatric ART patient records. Table 12. Documentation (Completion) of Viral Load during April 2016 – March 2017 by Districts, UTAP Mid-Term Evaluation (n=342) District Viral Load Not Correctly Documented Viral Load Correctly Documented Total Number Percent Number Percent Number (%) Andara 17 40.5 25 59.5 42 (100) Nyangana 11 52.4 10 47.6 21 (100) Odibo 22 35.5 40 64.5 62(100) Oshikuku 23 38.4 37 61.6 60(100) Onandjokwe 24 49.0 25 51.0 49(100) Omuthiya 7 28.0 18 72.0 25(100) Grootfontein 20 55.6 16 44.4 36(100) Tsumeb 24 51.1 23 48.9 47(100) All Districts 148 43.3 194 56.7 342(100) Omuthiya district has the highest (72%) ART patient records with correctly documented viral test results and Grootfontein district has the lowest (44%). The 2016 baseline assessment for the three expansion districts reported the highest (67%) correct documentation of viral load according to the national guideline for Grootfontein and the lowest for Tsumeb and Omuthiya districts (23% and 25% respectively). The mid-term findings on proper viral load documentation per national guideline also called viral load completion was compared with results of the 2016 national ARV resistance Early Warning Indicator (EWI) assessment, which used ePMS and EDT as its data source. Among 21 of the 23 mid-term involved facilities for which EWI result was available on ADULT VL completion, only one facility (Grootfontein clinic, 100% (2 of 2 patient charts reviewed)) obtained excellent score (Green >70%) on ePMS based EWI assessment, and on EDT based EWI assessment, only two facilities (Grootfontein clinic 100% and Odibo health center 74% patient charts reviewed) scored Green while all the rest got poor performance score (Red < 70%) or data was not available (Amber = NA) on both EDT and ePMS assessments. Along a similar trend, the mid￾term evaluation also found only two of the 21 facilities scored excellent (Omuntele clinic 100% 25 (2 of 2 patient charts reviewed) and Omuthiya hospital 72% (13 of18 patient charts reviewed)) on VL completion. Percentage of adults and children on treatment 12 months after initiation of antiretroviral therapy (TX -RET) One of the critical service quality indicators is the program level of retention of the clients. High retention is one important measure of program success, specifically in reducing morbidity and mortality, and is a proxy for overall quality of the ART program. For this assessment, we have considered this indicator as the percentage of adults and children known to be on treatment 12 months after initiation of antiretroviral therapy (ART). Table 13. Percentage of adults and children known to be alive and on treatment 12 months after initiation of antiretroviral therapy in all the reviewed districts (n=297) Districts No (Not Still on Treatment) Yes (Still on Treatment) Total Number Percent Number Percent Number Andara District 12 24.0 38 76.0 50 Nyangana District 3 8.6 32 91.4 35 Odibo District 6 19.4 25 80.6 31 Oshikuku District 6 15 34 85 40 Omuthiya District 2 8.0 23 92.0 25 Onandjokwe District 8 18.6 35 81.4 43 Tsumeb 7 20 28 80 35 Grootfontein District 10 26.3 28 73.7 38 District Total 54 18.2 243 81.8 297 The overall retention rate on ART treatment is 81.8 % in all the districts. Omuthiya District retained the highest percent (92.0 %), followed by Nyangana District (91.4 %); Grootfontein District (73.7 %) and Andara district (76%) had the lowest retention on treatment rate. Among those sampled eighty-two percent of 216 adults and 88 percent of 16 pediatric ART patients were on treatment 12 months after initiation of ART. Facility level retention rates are presented in Annex A2. 26 Table 14. Comparing MTE Retention on ART at 12 Months to 2015 (baseline) and 2016 (mid-point) National EWI results and UTAP Baseline (2016) Main ART Site EWI 2014 EWI 2016 UTAP Baseline** UTAP MTE Comment* Andara Hospital 79% 75% NA 76% MTE = EWI16 = EWI14 Nyngana Hospital 80% 58% NA 91% MTE >> EWI14 >> EWI16 Odibo Health center 89% NA NA 81% MTE < EWI14 Oshikuku Hospital 83% 86% NA 85% MTE = EWI16 = EWI14 Onandjokwe Hospital 77% NA NA 81% MTE = EWI14 Omuthiya Hospital 97% 79% 72% 92% MTE >> Baseline; EWI16 << EWI14 Tsumeb Hospital 78% 75% 93% 80% MTE < Baseline; EWI16 = EWI14 Grootfontein Hospital 61% 71% 19% 74% MTE >>> Baseline; EWI16 > EWI14 *EWI2016 retention scoring applied to UTAP MTE column: >85=green; 75-85%=Amber; <75=Red Comparing the MTE with EWI and baseline where data is available, in Nyangana, Omuthiya and Grootfontein districts retention rate was higher than baseline and/or the 2016 EWI is higher than the 2014 EWI while in Andara, Oshikuku and Onandjokwe districts, the MTE retention rate was about the same as EWI’16 and/or EWI’14. Odibo and Tsumeb districts achieved less on MTE than baseline or EW14 retention rates. Although Grootfontein improved retention in MTE from baseline, 74% still shows poor performance by EWI standards. The fact that the UTAP baseline for Grootfontein is very different from the 2014 EWI while the MTE gave a similar assessment to the 2016 EWI shades doubt on its accuracy. Apart from that, large differences observed between MTE and EWI16 assessment results could be explained by differences in data sources used. Percentage of HIV-positive new and relapsed registered IB cases on ART during TB treatment (TB_ART) This indicator will measure the ability of programs to accomplish linkage between TB and HIV programs. Because PLHIV who develop active TB disease are often seriously ill or at an advanced stage of immunosuppression, the timely initiation of ART among registered TB cases that are HIV-positive is a priority intervention to prevent TB-related mortality among PLHIV and to reduce HIV transmission. 27 Table 15. Percentage of HIV-positive new and relapsed registered TB cases on ART during TB treatment in all health facilities/districts HIV Positive and on ART while on TB Treatment? Number % Can’t determine 13 5.9 No 5 2.2 Yes 203 91.9 Total 221 100 Table 16. Percentage of HIV-positive new and relapsed registered TB cases on ART during TB treatment by District (n=221) Overall, out of the 221 new and relapsed TB-HIV co-infected patient files sampled from 21/23 health facilities in the eight UTAP districts for the mid-term review, 203 (91.9%) have been found to be on ART while on anti-TB treatment as is required by the national TB-HIV management standard. This is slightly higher than the 2015 baseline assessment of the initial five UTAP HIV Positive and on ART while on TB Treatment District Can’t Determine No Yes Total Number Percent Number Percent Number Percent Number Andara District 0 0 0 0 21 100 21 Grootfontein 0 0 1 5.3 19 94.7 20 Nyangana 0 0 0 0 20 100 20 Odibo 2 5.5 0 0 34 94.4 36 Omuthiya 7 25.9 1 3.7 19 70.4 27 Onandjokwe 0 0 0 0 34 100 34 Oshikuku 1 4.2 0 0 23 95.8 24 Tsumeb 3 7.7 3 7.7 33 84.6 39 District Total 13 5.9 5 2.2 203 91.9 221 28 districts, which was 89%, and the same as the 2016 baseline assessment result of the three UTAP expansion districts, 92%. Tsumeb and Grootfontein districts that achieved 95% and 100% in the 2016 baseline assessment placed 85% and 95% of HIV positive TB patients on ART while on anti-TB treatment in this mid￾term evaluation respectively, and Omuthiya district showed a slight decrease from 74% to 70% between the baseline and mid-term assessment periods. Andara, Nyangana and Onandjokwe districts demonstrated the best performance at 100% while Omuthiya district scored the worst at 74% in this mid-term evaluation. Onandjokwe district made considerable improvement from 71% to 100% between the 2015 baseline and MTE. Facility specific results of the MTE on percent of New and Relapsed TB cases put on ART while on TB Treatment, TX_RET are presented in Annex A4. Percentage of ART Patients with a Viral Load Result Documented within the Past 12 months with a Suppressed Viral Load (1000 copies/ml) (TX_PVLS) Viral suppression is such an ultimate outcome indicator for the success of Anti-retro viral therapy of an individual HIV patient as well as the success of the implementation of HIV treatment program at facility and regional/national levels that monitoring its performance level is critical. The indicator allows us to assess the extent to which the ART program is positively impacting on the clinical outcomes of patients on treatment. Overall, among 403 individual ART patient charts reviewed for clients on ART at 12 months and with documented viral test result between April 2016 to March 2017 in the 23 sampled health facilities, 367 (91%) had viral load suppressed to less than 1000 copies/ml. It needs to be noted 91% 9% Fig. 6. Percent Virally Suppressed ART Patients, UTAP MTE (n=403) Suppressed Not Suppressed 29 that a high viral suppression of 92% is achieved on the backdrop of a viral load completion rate of 86%. Table 17. Percent Distribution of Viral Suppression of <1000 copies/ml ART Patients in UTAP Districts (TX_PVLS), MTE (n=403) Districts No Viral Load not Suppressed Yes Viral Load Suppressed Total Number Percent Number Percent Number Andara 6 10.0 54 90.0 60 Nyangana 11 23.4 36 76.6 47 Odibo 2 3.6 53 96.4 55 Omuthiya 6 10.0 54 90.0 60 Onandjokwe 4 7.7 48 92.3 52 Oshikuku 1 2.2 45 97.8 46 Tsumeb 1 1.7 57 98.3 58 Grootfontein 5 20.0 20 80.0 25 District Total 36 8.9 367 91.1 403 In the above table, Tsumeb district had the highest percent (98.3 %) of viral load suppression among the patients under ART treatment, followed by Oshikuku district (97.8 %), and Nyangana District has the lowest (76.6 %). Table 18. Percent Viral Suppression of Sampled UTAP Facilities Assessed by National EWI and UTAP MTE EWI Colour Scoring scheme: Percent viral suppression EWI 2016 EDT and ePMS MTE 2017 Chart Review Number % Number % Green >90% 8 67% 19 83% Amber 75%-90% 3 25% 2 9% Red <75% 1 8% 2 9% Total 12* 100% 23 100% *11 MTE sampled facilities have no EWI data on viral load suppression Comparing the results of national EWI assessment that used data from EDT and ePMS with that of MTE findings that used ART patient booklet review as a source of data for viral load test result shows that 19 (83%) of the 23 sampled UTAP facilities scored green on MTE while eight of 12 UTAP MTE sampled facilities (75%) scored green by the 2016 National EWI assessment. 30 Table 19. Comparing MTE viral load suppression to 2015 (baseline) and 2016 (mid-point) National EWI results, by District Main ART Site EWI 2014 EWI 2016 UTAP MTE Comment* Andara Hospital 83% 75% 90% MTE > EWI14 > EWI16 Nyangana Hospital 83% 100% 77% MTE << EWI16 >> EWI14 Odibo Health center 53% NA 96% MTE >>> EWI14 Oshikuku Hospital 62% 90% 98% MTE > EWI16 >> EWI14 Onandjokwe Hospital 66% NA 92% MTE >> EWI14 Omuthiya Hospital 83% 95% 90% MTE = EWI16 > EWI14 Tsumeb Hospital 60% 92% 98% MTE > EWI16 >>> EWI14 Grootfontein Hospital 67% 84% 80% MTE = EWI16 >> EWI14 *EWI2016 viral suppression scoring applied to UTAP MTE column: >90=green; 75-90%=Amber; <75=Red Six districts achieved 90% or more viral load suppression and two districts, namely, Nyangana and Grootfontein achieved 77% and 80% respectively. Comparing the MTE results with the 2014 and 2016 EWI results with ART sites in this table shows there is considerable improvement of the viral load suppression demonstrating good outcomes of the ART programs. Large differences are noted between assessment results of MTE and EWI16 in some districts; a possible explanation is the difference in data sources used between the two methodologies. Table 20. Percent Viral Suppression of ART Patients by Age Group, MTE (n=403) Age Group Virally Suppressed # (%) Virally Not Suppressed #(%) Total # (%) Adult 341 (93.2) 25 (6.8) 366 (100) Paediatric 26 (70.3) 11 (29.7) 37 (100) Both Age Groups 367 (91.1) 36 (8.9) 403 (100) Viral load suppression is much higher among adults (93%) compared to paediatric ART patients (70%). Difference in retention of ART patients does not appear to explain the low VL suppression in paediatric patients as higher percentage (88% vs 82%) of paediatric patients were on treatment 12 months after initiation of ART. Proportion of ART Patients who completed TB Preventive Therapy (TB_PREV) This indicator measures the performance of HIV programs in scaling up TB preventive therapy with the goal of preventing progression to active TB disease among PLHIV and decreasing ongoing TB transmission in the population. This indicator is measured only for the MTE as it was not available during baseline. 31 Completion of TB preventive therapy is the worst performance indicator with only 30% of the 306 ART patient care booklets completed TBPT during a six- month period. 1% 69% 30% Fig. 7. ART Patients Percent Completion of TB Preventive Therapy (TB_PREV), all Districts, MTE (n=306) Indeterminate no yes 32 Table 21. Proportion of ART Patients who Completed TB Preventive Therapy, UTAP MTE (n=306) Indeterminate Not Completed Yes Completed District Number % Number % Number % Total Andara 1 2.9 30 85.7 4 11.4 35 Nyangana 0 0 19 95 1 5.0 20 Odibo 0 0 26 66.7 13 13.3 39 Oshikuku 0 0 28 47.5 31 52.5 59 Omuthiya 0 0 52 86.7 8 13.3 60 Onandjokwe 0 0 24 55.8 19 44.2 43 Tsumeb 2 7.4 17 63 8 29.6 27 Grootfontein 1 4.3 16 69.6 6 26.1 23 Total 4 1.3% 212 69.3% 90 29.4% 306 All districts but Oshikuku with 53%, have less than half of their ART patients completed TBPT between October 2016 to March 2017. The lowest performance was recorded by Nyangana (5%) followed by Andara 11%. 33 HIV Drug Resistance Early Warning Indicators (HDREWI) Results The UTAP MTE uses the results of the 2016 EWI assessment that was carried out nationwide by the MOHSS. Data for EWIs are extracted from existing electronic medical and pharmacy records such as EDT and ePMS. The purpose of collecting data on HIVDR EWIs is to assess the extent to which ART programs are functioning to minimize the emergence of HIVDR and optimize patient care. In 2016, Namibia abstracted five WHO EWIs for adult and paediatric patients from all ART sites (50 main sites and 199 outreach and Integrated Management of Adolescent and Adult Illness (IMAI) sites). These are: On-time Pill Pick-up, Retention in Care, Pharmacy Stock-outs, Dispensing Practices, and Viral Load Suppression. Fig. 8. A scoring scheme used by the 2016 MOHSS HDR EWI assessment. The results of the 2016 HDREWI for the 23 UTAP health facilities sampled for the mid-term evaluation is presented and discussed in relation to the 13 UTAP health facilities for which EWI data was available to compare against the facilities that were included in the then baseline assessment as follows. 34 Table 22. Percent Distribution of Scores of Health Facilities on Four Key Adult EWIs in 2016 and 2014 National assessments EWI 2016 EWI 2014 Color score Number % Number % Comment On Time Pill Pick Up Green >90% 3 14% 2 15% 2016 = 2014 Amber 80-90% 7 33% 4 31% 47% = 46% Red <80% 11 52% 7 54% 2016 = 2014 Viral Suppression Green >90% 8 66.7% 19 82.6% 2016 = 2014 Amber 75-90% 3 25.0% 2 8.7% 91.7% = 91.3% Red <75% 1 8.3% 2 8.7% 2016 = 2014 Retention at 12 months Green >85% 3 23% 4 40% 2016 << 2014 Amber 75-85% 7 54% 5 50% 73% << 90% Red <75% 4 31% 1 10% 2016 >> 2014 No Stock Out in 12 Months Period Green 100% 3 14% 7 54% 2016 <<< 2014 Red <100% 18 86% 6 46% 2016 >>> 2014 On the 2016 EWI assessment that corresponds to the time frame of the MTE, overall, the percentage of health facilities that scored Green is way below fifty percent for all but Viral suppression EWI, where 67% of facilities scored Green. A Red score was highest on Stock out (86%) followed by On Time Pill pick up (52%) in 2016 EWI assessment. The Green and Amber scores are combined to compare the 2016 (the closest to MTE) with the 2014 (baseline) EWI assessment results. On that basis, for On-Time Pill pick up and Viral Suppression EWI, the results were similar between the two periods while UTAP facilities performed a lot worse on retention at 12 months and No Stock Out EWIs in 2016 than in 2014, 73% vs 90% and 14% vs 54% respectively. Drug resistance will not necessarily result immediately if an indicator shows non-optimal performance; however, achieving the best possible performance as measured by these indicators will help to minimize preventable HIVDR. The results of the 2016 EWI for the 21 health facilities sampled for the MTE is extracted in a table and presented in Annex C. 35 HIV Program management SIMS Domain 21 – Performance Management Performance management involves measuring, monitoring and enhancing staff performance. This includes job descriptions, performance appraisal and education opportunities. The CEEs for this domain assess supervisory visits to sites, training of sites employees and staff performance evaluations. All twenty-three facilities were able to be evaluated on performance management. Thirty facilities (43%) of this domain met or surpassed expectations, as indicated by the light and dark green in the graph. However, thirty-nine CEEs (57%) need either improvement or urgent remediation. Out of the Performance Management CEEs, both staff performance and supportive supervision have the best results, each having 11/23 facilities meeting or exceeding expectations. In-service training had the worst results with 15/23 facility needing improvement. Table 23. Comparison of SIMS Performance Management Domain Assessment Results between MTE and Baselines, All UTAP Districts SIMS Domain 1. ART MTE 2017 Baseline 2015 Baseline 2016 Comment Number % Number % Number % Met or surpassed basic expectation 30 43 14 29 10 33 MTE > Baseline Failed to meet basic expectation 39 57 34 71 20 67 Total 69 100 48 100 30 100 11 12 3 1 3 9 6 4 7 5 4 4 21.1 Staff performance 21.2 In-service training 21.3 Supportive supervision FIG. 9. PERFORMANCE MANAGEMENT FACILITY (N=23) SIMS INDICATOR CEE COLOR SCORES Needs Urgent Remediation Needs Improvement Meets Basic Expectations Surpasses Basic Expectstions 36 Forty-three percent% of SIMS Performance Management CEEs met or surpassed basic expectations signalling an improvement from the two baseline assessments. Table 24. Comparison of SIMS Performance Management Domain Assessment Between MTE and Baselines, UTAP Districts During the mid-term review, two districts (Odibo, Tsumeb) performed worse than and one district (Grootfontein) remained the same as the baseline. Although five districts improved over the baseline (green), the proportion of CEEs that met or surpassed basic expectation is less than 80% in all of them, calling for UTAP’s attention in performance improvement. UTAP District % CEEs Met and Surpassed Basic Expectations Comment MTE Baseline Andara 78 67 MTE > Baseline Nyangana 44 0 MTE >>> Baseline Odibo 0 17 MTE << Baseline Omuthiya 44 27 MTE >> Baseline Onandjokwe 67 50 MTE >> Baseline Oshikuku 33 25 MTE > Baseline Tsumeb 22 33 MTE < Baseline Grootfontein 67 67 MTE = Baseline 37 Supportive Supervision This CEE measures if each site received routine supportive supervision from the PEPFAR implementing partners with timely and constructive feedback. These supervisory visits focus on meeting staff needs for management support, logistics, mentoring and continuing education. Eleven sites (48%) met or exceed expectations for supportive supervision. Three facilities (13%) needed urgent remediation for not having had a supervisory visit in the last 3 months or at least four times a year, and nine facilities (39%) needed improvement because they did not receive and keep written reports of the supervisory visit. Seven sites (30%) met expectations by having documented QI processes to make improvements based on the notes from the supervisory visit. Finally, 4 sites (17%) exceeded expectations by having documented improvement from the deficiencies noted in the supervisory visits. In-service Training The in-service training CEE measures sites’ ability to keep written record of training logs, have selection criteria for what staff members attend trainings and sites’ ability to provide staff coverage while their staff members are at training. Eight facilities (35%) met or exceeded in-service training expectations. Twelve facilities (52%) need urgent remediation because they did not keep written records of those they sent to participate in in-service trainings. Three facilities (13%) need improvement because they did not update these training logs within the last three months. Four facilities (17%) met expectations, and an additional 4 facilities (17%) exceeded expectations. To meet expectations a facility had to have criteria in which they chose who went to trainings, and to exceed expectations a site had to have standard operating measures to have adequate staff coverage while a staff member was away at training. Staff Performance This CEE assesses facility’s ability to measure and support health workers’ performance. This includes job descriptions, individual performance reviews of health workers, and an ability to create opportunities for health workers to improve performance when they are found to be lacking. Forty-eight percent of all facilities met or surpassed expectations for staff performance. Eleven facilities (48%) need urgent remediation because they did not have job descriptions available for every staff member involved in HIV linked services, and one facility (4%) needs improvement for not having every staff member involved in HIV linked services participate in individual performance evaluations. Six facilities (26%) met and 5 facilities (22%) exceeded expectations. Those that just met expectations could improve by providing support to staff members who have a performance development plan. 38 SIMS Domain 23-Site Management- Quality Management/Quality Improvement Quality Management and Quality improvement are important indicators for accessing a site’s ability to assess the quality or ability to provide patients with the care they need. Quality management involves developing, implementing and sustaining a quality program, while quality improvement is a site’s ability to assess their quality and make their care and services better where necessary by routinely reviewing health data. Twenty-three facilities were assessed for QM/QI SIMS. Twenty-two (48%) CEEs showed facilities meeting or exceeding expectations. Twenty-four CEEs indicate (52%) needs urgent remediation or needs improvement, as indicated by the yellow and red scores in the graph above. Of the two CEEs that are part of the SIMS QM/QI domain, facilities performed better on HIV QM/QI systems with 12/23 facilities meeting or exceeding expectations. In comparison, of the other CEE - Utilization of Data in QI - only had 10/23 facilities met or exceeded expectations. District level QM/QI SIMS domain results are presented in Annex B4. Table 25. Comparison of SIMS QM/QI Domain Assessment Results between MTE and Baselines, All UTAP Districts SIMS Domain 1. ART MTE 2017 Baseline 2015 Baseline 2016 Comment Number % Number % Number % Met or surpassed basic expectation 22 48 6 21 3 14 MTE >> Baselines Failed to meet basic expectation 24 52 22 79 19 86 Total 46 100 28 100 22 100 9 6 2 7 2 8 10 2 23.1 HIV QM/QI system 23.2 Utilization of data in QI FIG. 10. HIV QM/QI FACILITY (N=23) SIMS INDICATOR CEE COLOR SCORES Needs Urgent Remediation Needs Improvement Meets Basic Expectations Surpasses Basic Expectstions 39 UTAP facilities have shown some improvements in HIV QI/QM SIMS domain over the baseline assessments in 2015 and 2016. The MTE showed that 48% of facilities met or exceeded expectations, which is an improvement from the 21% and 14% from the 2015 and 2016 baseline assessments, respectively. On the flip side, it is disconcerting to see the majority (52%) of CEEs on QM/QI assessment failed to meet basic expectation. Table 26. Comparison of SIMS QM/QI Domain Assessment Results between MTE and Baselines, UTAP Districts UTAP District % CEEs Met and Surpassed Basic Expectations Comment MTE Baseline Andara 67 0 MTE >>> Baseline Nyangana 83 100 MTE < Baseline Odibo 17 0 MTE > Baseline Omuthiya 50 0 MTE >>> Baseline Onandjokwe 100 25 MTE >>> Baseline Oshikuku 33 50 MTE << Baseline Tsumeb 33 12 MTE >> Baseline Grootfontein 75 50 MTE >> Baseline Nyangana and Oshikuku showed lower proportion of CEEs that met or surpassed basic expectations than the baseline on QM/QI SIMS domain. However, despite the lower than baseline result, Nyangana’s performance is the second best. Among the other six districts that showed improvement over the baseline, Andara, Odibo, Omuthiya and Tsumeb district’s performance is still less than 70%, highlighting the immensity of work required to improve the quality of facility based HIV care in UTAP project areas. Utilization of Data in QI This CEE represents a facility’s ability to have a system for reviewing and using performance data to inform implementation for QI initiatives. This includes reviewing performance data on a routine basis, making sure that a diverse group of stakeholders are involved in QI initiatives, an 40 ability to identify capabilities in QI activities, and recognition of excellent performance when staffs are doing a good job with QI initiatives. Ten sites (43%) met or exceeded in utilization of data in quality improvement. Six facilities (26%) need urgent remediation, as indicated in red, because they did not review any performance data to identify gaps and form quality improvement initiatives. Seven facilities (30%) scored yellow, and need improvement. This was because they either did not collect and review data to make QI indicatives (n=1), did not document QI actives and initiatives (n=3) or they did not engage stakeholders from the community in forming QI plans (n=3). Eight facilities (35%) met expectations, as indicated by light green; however they could have exceeded expectations by recognizing excellent performance among staff. Two facilities (9%) did recognize excellent performance among staff, and therefore exceeded expectations, as indicated by the dark green score. HIV QI/QM System This CEE measures if sites have a QM/QI system with routine monitoring of the quality of services. This includes that sites have a QI team that meets regularly, follow a facility QM/QI plan and routinely review performance and delivery standards. Twelve sites (52%) met or exceeded expectations for QI/QM Systems. Nine facilities (39%) need urgent remediation because they had no QM or QI activities in place. Two sites (9%) did not have a functional QM/QI committee or team, and therefore scored yellow and need improvement. Two sites (9%) met basic expectations and scored a light green by having the QM/QI team but not having a written QM/QI plan being implemented with defined staff roles and responsibilities. The remaining 10 facilities (43%) did have written plans, and therefore exceeded expectations, scoring a dark green. SIMS Domain 15 -Medication Management The two CEEs for Medication Management include medication dispensing and supply chain. Medication dispensing plays a valuable role in identifying patients who have missed medication refills and allow patient tracking and target adherence interventions for patients. Supply chain is a critical CEE for a site to make sure they are able to order new medication that is running low in a timely manner or emergency medication when it runs out early. This is important for the site’s ability to provide the needed medications to its patients. 41 Only twenty one out of twenty-three facilities that participated in the midterm evaluation had data on Medication Management. Overall, there were 30 (71%) CEE that were light green or dark green, indicating they met or exceeded expectations for that CEE. This leaves 12 (29%) needing improvement, as indicated by yellow and red. Of the two CEEs for SIMS Medication Management, supply chain management performed stronger than Medication Dispensing. Supply chain management had 17/21 facility meeting or exceeding expectations, while Medication Dispensing only had 13/21 facilities meeting or exceeding expectations. Table 27. Comparison of SIMS Medication Management Domain Assessment Results between MTE and Baselines, All UTAP Districts SIMS Domain MTE 2017 Baseline 2015 Baseline 2016 Comment Number % Number % Number % Met or surpassed basic expectation 30 71 15 75 14 70 MTE = Baseline Failed to meet basic expectation 12 29 5 25 6 30 Total 42 100 20 100 20 100 The MTE shows that 71% of CEEs met or exceeded expectations for the Medication Management SIMS Domain, but did not show improvement from the 2015 or 2016 baseline assessments, which had 75% and 70% of CEEs meeting or exceeding expectations respectively. District level Medication Management SIMS domain results are presented in Annex B5. 2 3 2 5 2 3 15 10 15.1 Supply chain management 15.2 Medication Dispensing FIG. 11. MEDICATION MANAGEMENT FACILITY (N=21) SIMS INDICATOR CEE COLOR SCORES Needs Urgent Remediation Needs Improvement Meets Basic Expectations Surpasses Basic Expectstions 42 Medication Dispensing This CEE represents whether a facility had standard medication dispensing protocol and maintains good dispensing registers in order to identify patients who missed pick up for medications and create adherence interventions. Most sites (57%) met or exceeded expectations on medication dispensing. Three facilities (14%) did not have any documentation of medications dispensed, meaning they are sites that need urgent remediation. Five facilities (24%) need improvement because their dispensing records were not up to date. Three facilities (14%) earned a light green score of meeting basic expectations because they had written protocol for medications dispensed and documented, while 10 facilities (48%) received a dark green score of surpassing expectations, by routinely identifying patients who had missed medication refills. Supply Chain Management This CEE measures whether sites have inventory management protocols for ARVs, cotrimoxazole and HIV rapid tests kits, as well as understand how to submit routine and emergency orders to maintain adequate stock within minimum and maximum stock levels. Seventy four percent of sites met or exceeded expectations for supply chain management. Two sites (10%) need urgent remediation, because these sites did not have inventory management tools. Two sites (10%) did not resubmit to refill medications in a timely manner in the last 3 months, earning them a yellow score, or a score indicating they need improvement. Two sites (10%) showed they met basic expectations but could improve on maintaining minimum and maximum stock levels for their mediations and supplies, and having procedures to order emergency or off schedule supplies. The remaining 71% exceeded expectations, and had all the abilities listed above. SIMS Domain 20- Site Management -Finance and Planning Accurate and reliable facility finance and planning is important to ensure there are enough staff and resources to adequately care for HIV positive patients. Only nine of the 23 health facilities assessed for the MTE claimed to manage their finances and use those financial reports for managing facility budgets and strategic planning; of these, six exceeded expectations. On adequate staffing of clinicians to treat HIV positive patients, all 23 facilities met or surpassed expectation. 43 HRH Staffing Each site that provides HIV or HIV-linked services should have adequate number of staff to provide quality services. Clinicians in vast majority of facilities (21/23) see an average of less than 30 HIV positive patients (dark green) per day while 2/23 see 30-50 patients a day (light green). Financial Management Each site that is responsible for planning, managing and accounting for its expenditures should have financial monitoring systems that meet financial management criteria at the operational level. Fourteen public health facilities reported that their budget is approved, executed and reported on by their respective regional health offices. The only financial report they produce is on the revenue generated from the services they provide. The remaining nine facilities claimed they manage their budget; it was understood that these facilities are faith based health facilities and not public as the procedure is the same across all regions. Among these nine, six have financial reports at their facilities, use their financial records to generate financial reports; reports comply with national financial management criteria and are used for budgeting and strategic planning. 1 0 2 0 2 6 21 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% 20.1 Financial Management 20.2 HRH Staffing FIG.12. FINACE AND PLANNING FACILITY (N=23) SIMS INDICATOR CEE COLOR SCORES Needs Urgent Remediation Needs Improvement Meets Basic Expectations Surpasses Basic Expectstions 44 UTAP Targets and Achievements From the outset UTAP has been supporting the National Health Information system (HIS) in the project districts and health facilities where they operate. Data collected through the HIS produces periodic reports which are utilized for HIV program performance review at different levels. The MTE has obtained from UTAP M&E all the reports that have been submitted to USAID quarterly starting from the third quarter of FY2015 (April – June 2015) to the second quarter of FY2017 (January – March 2017) covering roughly the first two years of the four-year life of UTAP. This section presents the performance levels on key UTAP indicators as reported to USAID. For the majority of the indicators, the percentages are calculated by comparing quarterly targets set by USAID against reported counts of achievements in respective quarters and for some indicators, the percentages are achievements levels of the indicators themselves calculated by dividing their numerators to the denominators. In reviewing targets, it was noted that the annual targets and budgets have changed from first year to the second year because of the expansion of UTAPs award districts from initial five to eight districts in the second year, and, as a result, the quarterly targets used in this analysis have changed accordingly. Performance levels have been increasing from Q3 2015 to Q2 2017 for all the indicators depicted in the above graph with exception of HTC-TST and PMTCT_STAT that decreased slightly. By Q1 2017 all indicators have achieved well above 95 percent except PMTCT_EID that got 60 percent and was consistently low throughout the two years. 73% 96% 139% 94% 111% 197% 163% 160% 100% 100% 89% 93% 98% 136% 86% 42% 42% 36% 53% 44% 60% 65% 11% 52% 74% 63% 56% 61% 100% 99% 98% 103% 95% 93% 96% 98% 100% 0% 50% 100% 150% 200% 250% Q 3 APR -J U N 1 5 Q 4 J U L - S E P 1 5 Q 1 O C T -D E C 1 5 Q 2 J A N- M A R 1 6 Q 3 APR -J U N 1 6 Q 4 J U L - S E P 1 6 Q 1 O C T -D E C 1 7 Q 2 J A N- M A R 1 7 FIG. 13. PERCENT ACHIEVED AGAINST TARGETS/EXPECTATIONS FOR SELECTED UTAP INDICATORS FROM APRIL 2015 TO MARCH 2017 HTC_TST PMTCT_STAT PMTCT_EID TX-NEW TB_STAT 45 UTAP has set new targets for all indicators for 2018; end of project targets through January 2019 (COP18) are not known at this point. Because of these moving targets, the MTE could not determine the achievements against the end of project target and, standing at mid-point, be able to appraise the level of effort that will be required to get to the finish line. To find out whether UTAP performed as stated by its Expected Result 1 and reached USAID set targets in its Project Management Plan (PMP), we compared the actual number of HIV positive patients currently on ART (TX_CURR) at the end of quarter 2 of 2017 (mid-term) against UTAP target for TX_CURR and the 80% of estimated number of PLHIV in the UTAP districts. Table 28. Percent of UTAP TX_CURR Target Achieved by Q2 FY17, MTE District Target Number Current on ART, end of 2017 Actual Number Current on ART, Q2 FY17 Percent Achieved Target vs Actual, Q2 FY17 Andara 3300 1833 56% Nyangana 3304 2134 65% Odibo in Engela* 7552 2672 65% Oshikuku 10809 8602 80% Omuthiya 2310 5200 225% Onandjokwe 15104 11642 77% Tsumeb 2407 2845 118% Grootfontein 2407 2085 87% All Districts** 47200 37013 78% * Odibo sub-district’s target appears to be the target for Engela district **District total does not add up to All Districts target Two quarters remaining before the end of 2017, overall, UTAP has succeeded in placing 78% of targeted HIV positive patients on ART which is a remarkable achievement. While Omuthiya and Tsumeb districts achieved well over 100%, Andara and Nyangana districts are at less than 70% raising concern as might be the case with Odibo. However, the correctness of the target labelled for Odibo sub-district needs to be sorted out with USAID as it appears to represent the target for the whole of Engela district, to which Odibo is a part and, under normal estimation, constitutes only 25%. If 25% of Engela was the target, Odibo’s achievement would be much higher pushing UTAP’s achievement up with it. 46 Table 29. Percent of 80% of Estimated number of PLHIV Reached by UTAP Q2 FY17 TX_CURR, MTE Two quarters remaining to the end of 2017, overall, UTAP shows a short fall of 6373 PLHIV. UTAP’s total number of newly enrolled ART patients in Q3 of 2017 is 1109 and at this rate, it takes 5-6 quarters for UTAP to reach 80% of its estimated number of PLHIV. It has to be noted however, that this gap would change to excess if the correct number for Odibo and Engela is sorted out with USAID. Five out of eight UTAP districts have exceeded the estimated 80% PLHIV, but three districts, namely, Andara, Odibo and Oshikuku show short falls ranging from 203 to 10470. District 80% of Estimated Number of PLHIV, end of FY17 Actual Number Current on ART, Q2 FY17 Gap between Estimated 80% PLHIV end of FY17 and Actual Q2FY17 Andara 2693 1833 -860 Nyangana 2006 2134 128 Odibo in Engela* 13142 2672 -10470 Oshikuku 8805 8602 -203 Omuthiya 4250 5200 950 Onandjokwe 8194 11642 3448 Tsumeb 2421 2845 424 Grootfontein 1875 2085 210 All Districts** 43386 37013 -6373 * Odibo sub-district’s 80% estimated number of PLHIV appears to be that of Engela district **District total does not add up to All Districts target 47 UTAP Expenditure Costs Along with the quarterly performance report, UTAP submits to USAID its expenditure analysis report. The MTE obtained all nine quarterly expenditure analysis reports for the period January 2015 to March 2017 and presents the results in the following graphs. During the review period covered by the MTE, UTAP spent US$10,280,530 of which the highest (33%) was for program management followed by facility based care and treatment (FBCTS) services (27%) and the lowest was on health systems (5%). 27% 7% 5% 19% 9% 33% Fig. 14. Percent Districution of UTAP Total Expenditure by Program Area, MTE FBCTS PMTCT HTC Health System Strategic Info Program Mgmt 48 Figure 15 shows that spending started at higher level during the first quarter largely due to initial investment in project vehicles, and dropped during subsequent quarters until it picked up later in Q1 of 2016 for all cost items marking the start of the implementation of the UTAP expansion plan including the addition of three new districts. After a sharp drop in spending thereafter in Q3 of 2016, spending has been on the increase every quarter ever since. Cost of pre-fabs and hiring of new staff through sub-contracts contributed to this increase. A significantly higher proportion of the total expenditure for the three HIV clinical services (FBCT, PMTCT, HTC) was spent on recurrent than investment costs ranging from a ratio of 1:6 for 0 100000 200000 300000 400000 500000 600000 700000 800000 900000 Expenditure USD Fig. 15. TREND IN QUARTERLY EXPENDITURE (US$) JANUARY 2015 - MARCH 2017 BY UTAP PROGRAM AREAS, MTE FBCTS PMTCT HTC Health sys Strategic Info Program Mgmt 0 500000 1000000 1500000 2000000 2500000 3000000 FBCT PMTCT HTC Fig. 16. COMPARISON BETWEEN TOTAL INVESTMENT AND RECURRENT EXPENDITURES IN US$ OF UTAP'S THREE CLINICAL SERVICE AREAS, MTE Investement Recurrent 49 PMTCT to 1:12 for HTC. Investment costs cover purchase of vehicles, pre-fab housing, medical equipment and refurbishment of health facilities, while recurrent cost mainly covers salaries, benefits and utilities. HIV Drug Resistance Early Warning Indicators (HDREWI) Results The UTAP MTE employs the results of the 2016 EWI assessment that was carried out nationwide by the MOHSS. Data for EWIs are extracted from existing electronic medical and pharmacy records such as EDT and ePMS. The purpose of collecting data on HIVDR EWIs is to assess the extent to which ART programs are functioning to minimize the emergence of HIV drug resistance and optimize patient care. In 2016, Namibia abstracted five WHO EWIs for adult and pediatric patients from all ART sites (50 main sites and 199 outreaches and Integrated Management of Adolescent and Adult Illness (IMAI) sites). These are: On-time Pill Pick-up, Retention in Care, Pharmacy Stock-outs, Dispensing Practices, and Viral Load Suppression. Fig. 17. Scoring scheme used by the 2016 MOHSS HDREWI assessment. The results of the 2016 HDR EWI for the 23 UTAP health facilities sampled for the mid-term evaluation is presented and discussed in relation to the 13 UTAP health facilities for which EWI data was available to compare against the facilities that were included in the then baseline assessment as follows. 50 Table 30. Percent Distribution of Color Scores of Health Facilities on Four Key Adult EWIs in 2016 and 2014 National assessments EWI 2016 EWI 2014 Color score Number % Number % Comment On Time Pill Pick Up Green >90% 3 14% 2 15% 2016 > 2014 Amber 80-90% 7 33% 4 31% 47% > 46% Red <80% 11 52% 7 54% 2016 = 2014 Viral Suppression Green >90% 8 66.7% 19 82.6% 2016 > 2014 Amber 75-90% 3 25.0% 2 8.7% 91.7% > 91.3% Red <75% 1 8.3% 2 8.7% 2016 < 2014 Retention at 12 months Green >85% 3 23% 4 40% 2016 << 2014 Amber 75-85% 7 54% 5 50% 73% << 90% Red <75% 4 31% 1 10% 2016 >> 2014 No Stock Out in 12 Months Period Green 100% 3 14% 7 54% 2016 <<< 2014 Red <100% 18 86% 6 46% 2016 >>> 2014 On the 2016 EWI assessment that corresponds to the time frame of the MTE, overall, the percentage of health facilities that scored Green is way below fifty percent for all but Viral suppression EWI, where 67% of facilities scored Green. A Red score was highest on Stock out (86%) followed by On Time Pill pick up (52%) in 2016 EWI assessment. The Green and Amber scores are combined to compare the 2016 (the closest to MTE) with the 2014 (baseline) EWI assessment results. On that basis, for On-Time Pill pick up and Viral Suppression EWI, the results were similar between the two periods while UTAP facilities performed a lot worse on retention at 12 months and No Stock Out EWIs in 2016 than in 2014, 73% vs 90% and 14% vs 54% respectively. The results of the 2016 EWI for the 21 health facilities sampled for the MTE is extracted in a table and presented in Annex C. 51 Conclusions Generally, there is a clear indication that UTAP has progressed well according to its stated objectives and targets and when compared against baseline results on many of the indicators assessed. On the other hand, there are a few areas that UTAP has lagged and needs to pick up effort to improve performance in the remaining second half of the project. Access to PLHIV UTAP has succeeded in placing 78% of targeted HIV positive patients on ART and 85% of estimated number of PLHIV in its project area set for the end of 2017, which is a remarkable achievement. The remaining two quarters should increase the achievement against planned targets well over 90%. The fact that Andara and Nyangana districts achieved less than 70% of target raises concern. Given the number of newly enrolled ART patients reported in Q3 of 2017, i.e., 76, 126, and 216 for Andara, Odibo and Oshikuku districts respectively, it takes Andara up to 11 quarters, and Oshikuku one quarter to close the gaps. Odibo sub-district’s target and estimated number of PLHIV appears to be that of its parent district, Engela resulting in apparently higher performance gap and lower achievements. HIV Clinical Prevention UTAP is performing well on HIV testing and counselling as witnessed from the SIMS and programmatic indicator results. A large majority of facilities met or surpassed expectations on all but two CEEs of HTC SIMS domain: HTC referrals and HIV proficiency testing. Quarterly achievements against targets are progressing well above 100%. HIV Treatment Overall, UTAP’s interventions have certainly contributed to improvements as demonstrated by percentage of SIMS CEEs that met or surpassed basic expectations on HIV treatment. The finding that six out of eight districts also did better on the MTE than the baseline is remarkable. Andara and Nyangana districts results showed MTE SIMS ART domain performance had decreased compared to their baseline SIMS scores. Seven of the 23 health facilities failed to meet basic expectations on ART SIMS Patient Tracking CEE. The fact that 14% of patient booklets reviewed had no single viral load result documented after 12 months on ART and that only 57% had properly documented viral load as per national guideline is concerning. Judging by EWI cut-off for retention at 12 months, 82% for all assessed UTAP facilities on the MTE is a good performance. This breaks into seven districts doing as well as (3 districts) or better than (4 districts) UTAP baseline or EWI 2014 retention rates. UTAP facilities placed more than 90% of TB patients that are HIV positive on ART while they were on TB treatment; apart from Omuthiya all districts achieved 85% and more. This is a remarkable performance. 52 A 29% TB preventive therapy completion is the poorest performance of all indicators assessed by this MTE and the worst achievers are Nyangana and Andara districts. An overall viral suppression of 92% for UTAP facilities sampled for the MTE is an excellent result. Judged on EWI scores for viral suppression, Tsumeb, Omuthiya, Oshikuku, Odibo, Onandjokwe district, Andara district scored green implying above expectation performance; Grootfontein and Nyangana districts scored amber implying a satisfactory performance. HIV Program Management UTAP facilities showed some improvement in percent meeting or exceeding basic expectations on performance management CEEs on the MTE from very low (poor) baseline. However, the fact that 57% of CEEs failed to meet basic expectation currently shows that performance management has a long way to go, especially with documentation of in-service trainings at facility level. Although UTAP facilities showed some improvement in percent of CEE that met or surpassed basic expectation on the QM/QI SIMS assessment for MTE, the fact that 52% CEEs failed to meet basic expectation is disconcerting. Eight facilities failed to meet basic expectation on medication and four on Supply Chain Management lowering the overall percent that met or surpassed expectation CEEs down to 71% in the MTE. UTAP’s investment in human resource coupled with decentralization of ART services to smaller/peripheral health facilities through NIMART has made considerable impact on staffing patterns as all health facilities assessed on SIMS domain Finance and Staffing had adequate number clinicians to take care of HIV positive patients. This is a major accomplishment of UTAP. Public health facilities in UTAP areas do not manage their budget. This will adversely affect results of efforts aimed at improving performance and quality of services. Early Warning Indicators Overall, UTAP health facilities in 2016 did poorly with a predominance of red score on On-Time pill pick up and No Stock-out EWIs. In fact, the percent scored red in 2016 is the same as on On￾Time Pick up but worse on No Stock-out EWI than 2014. UTAP facilities placed more than 90% of TB patients that are HIV positive on ART while they were on TB treatment; apart from Omuthiya all districts achieved 85% and more. This is a remarkable performance. An overall viral suppression of 92% for UTAP facilities sampled for the MTE is an excellent result. Judged on EWI scores for viral suppression, Tsumeb, Omuthiya, Oshikuku, Odibo, Onandjokwe district, Andara district scored green implying above expectation performance; Grootfontein and Nyangana districts scored amber implying a satisfactory performance. 53 RECOMMENDATIONS  The remaining two quarters of 2017 should increase the UTAP achievement against target in terms of PLHIV currently on ART well over the 80% target. Andara and Nyangana should be prioritized in terms of reaching out to PLHIV in their districts with focus on finding and enrolling new HIV positive patients on treatment. The target as well as number of PLHV for Odibo and Engela needs to be reconciled in consultation with USAID.  There is a need to relook into procedures and practices pertaining to HTC referrals at facility level. With current decentralization of HIV clinical services to peripheral facilities improving access and reducing waiting time, HTC referral shall improve significantly in the coming periods. To maintain and ensure the quality of HIV testing, national standards of proficiency testing should be strictly followed.  Quarterly achievements against targets are progressing well above 100% on HTC program area although the yield of HIV positives is low. UTAP might have to look for new strategies to reach out to hard to reach populations to avoid lower coverages due to higher targets set by USAID for the 2018 fiscal year and beyond.  Andara and Nyangana districts will have to be prioritized to improve their overall SIMS ART domain performance. Urgent intervention is required to turn around poor performance on ART SIMS Patient Tracking CEE as seven of the 23 health facilities failed to meet basic expectations. A site specific intervention to improve results will have to be developed.  UTAP needs to take serious steps to improve viral load test result documentation in patient care booklets and/or ePMS in general and to enforce the standard viral load testing of adult and paediatric ART patients per national guideline.  There is up to 18% improvement needed on retention of ART patient at 12 months of initiation of ART in UTAP areas. Closer investigation and support is needed to change the low retention rate (red) in Grootfontein district using QI to demonstrate if changes lead to improvements.  UTAP shall keep up the good work on handling HIV-TB co-infected patients – placing HIV positive TB patients on ART as early as diagnosed. Attention should be given to Omuthiya’s situation to get it out of the low achievement on the TB_ART indicator.  TB preventive therapy has to be prioritized as a matter of urgency; UTAP has to re￾strategize and mobilize its clinical mentorship and M&E system to strengthen it with a focus on Nyangana and Andara districts.  While strengthening ART services to continue the high viral load suppression observed in the MTE into the remaining two years of UTAP, special attention needs to be given to raise the viral suppression score of Grootfontein and Nyangana districts from Amber to Green.  To improve results on Performance Management SIMS domain, UTAP has to re-examine existing processes thoroughly and come up with better procedures to improve the situation with the involvement of the facilities themselves and district/regional teams. 54  UTAP should review what worked and what did not work and reinvigorate QM/QI initiatives with new practical approaches and procedures to be implemented during the second and last half of UTAP.  As much as a success story is to invest in human resource for better HIV program outcomes, careful planning is needed to ensure its sustainability after UTAP. UTAP shall work closely with MOHSS to prepare exit strategy to ensure transition of function of some of the HCWs, mentors and information personnel using the new HIV-WISN results.  It may be difficult for UTAP to influence change on lack of budget management by health facilities as it is a longstanding government policy.to keep budget management centralized.  Effort should be made to improve adherence to ART through On-Time pick up and ensure the availability of HIV drugs all the time. UTAP needs to look in to lessons learned and determine whether action is required at the various levels including national level. 55 ANNEXES ANNEX A. Programmatic Indicators District and Facility Specific Result Tables Annex A1. Number Tested and Received results and Percent HIV Positive by Facility, District, HTC_TST and HTC_POS, MTE District/Health Facility Number of HIV Tested Number HIV results received Number HIV (+) result received % HIV+ Andara Hospital 3158 3158 270 8.5% Bagani Clinic 238 238 11 4.6% Divundu Clinic 316 316 29 9.2% Andara Total 3712 3712 310 8.4% Nyangana Hospital 3547 3547 172 4.8% Katere Clinic 268 268 12 4.5% Mbambi Clinic 470 470 18 3.8% Nyangana Total 4285 4285 202 4.7% Odibo_HC 3103 3030 196 6.3% Hamukoto Wakapa Clinic 1058 1053 29 2.7% Okatope Clinic 1229 1229 57 4.6% Odibo Total 5390 5312 282 5.2% St_Martins Hospital 6251 6244 332 5.3% Okalongo HC 2378 2349 130 5.5% Onheleiwa clinic 833 813 37 4.4% Oshikuku Total 9462 9406 499 5.3% Onandjokwe Hospital 12529 12529 610 4.9% Onyanya_HC 1599 1593 73 4.6% Elombe Clinic 501 501 26 5.2% Onandjokwe Total 14629 14623 709 4.8% Omuthiya Hospital 2914 2914 196 6.7% Onyuulaye HC 868 863 28 3.2% Omuntele Clinic 910 885 43 4.7% Omuthiya Total 4692 4662 267 5.7% Grootfontein Hospital 2109 2286 156 7.4% Grootfontein Clinic 2890 2890 129 4.5% Grootfontein Total 4999 5176 285 5.7% Tsumeb Hospital 2398 2394 150 6.3% 56 Catherine Bullen Clinic 980 966 60 6.1% Tsinsabis Clinic 734 734 63 8.6% Tsumeb Total 4112 4094 273 6.6% All Districts Total 51281 51270 2827 5.5% Annex A2. Percent of ART Patients alive and on Treatment 12 Months after Initiation of Treatment, TX_RET MET TX_RET District/Health Facility Not Still on Treatment Still on Treatment Tota l # # % # % Andara Hospital 9 45.0 11 55.0 20 Divundu Clinic 2 11.1 16 88.9 18 Bagani Clinic 1 8.3 11 91.7 12 Andara District Total 12 24.0 38 76.0 50 Nyangana Hospital 1 4.8 20 95.2 21 Mbambi Clinic 2 28.6 5 71.4 7 Katere Clinic 0 0 7 100 7 Nyangana District Total 3 8.6 32 91.4 35 Odibo HC 3 15 17 85 20 Hamukuto Wakapa clinic 2 20 8 80 10 Okatape clinic 1 100 0 0 1 Odibo District Total 6 19.4 25 80.6 31 Omuthiya Hospital 2 10 18 90 20 Onyuulaye HC 0 0 2 100 2 Omuntele clinic 0 0 3 100 3 Omuthiya District Total 2 8 23 92 25 Onandjokwe Hospital 3 15 17 85 20 57 Onyanya HC 5 25 15 75 20 Elombe clinic 0 0 3 100 3 Onandjokwe District Total 8 18.6 35 81.4 43 Oshikuku Hospital 2 10.5 17 89.5 19 Okalongo HC 4 20 16 80 20 Onheleiwa clinic 0 0 1 100 1 Oshikuku District Total 6 15 34 85 40 Tsumeb Hospital 5 26.3 14 73.7 19 Catherine Bullen clinic 0 0 6 100 6 Tsinsabis clinic 2 20 8 80 10 Tsumeb District Total 7 20 28 80 35 Grootfontein Hospital 4 20 16 80 20 Grootfontein clinic 6 33.3 12 66.7 18 Grootfontein District Total 10 26.3 28 73.7 38 All Districts Total 54 18.2 243 81.8 297 Annex A3. Percent Viral Load Documentation by District and Health Facility, TX_VL MTE District/Health Facility Viral Load Not Properly Documented Viral Load Properly Documente d Total # # % # % Andara Hospital 10 37.0 17 63 27 Divundu Clinic 7 70.0 3 30.0 10 Bagani Clinic 0 0.0 5 100 5 Andara District Total 17 40.5 25 59.5 42 Nyangana Hospital 11 57.9 8 42.1 19 58 Mbambi Clinic 0 0 2 100 2 Katere Clinic Nyangana District Total 11 52.4 10 47.6 21 Odibo HC 15 37.5 25 62.5 40 Hamukuto Wakapa clinic 7 31.8 15 68.2 22 Okatape clinic Odibo District Total 22 35.5 40 64.5 62 Omuthiya Hospital 5 27.8 13 72.2 18 Onyuulaye HC 0 0.0 1 100.0 1 Omuntele Clinic 2 33.3 4 66.7 6 District Total 7 28.0 18 72.0 25 Onandjokwe Hospital 13 72.2 7 27.8 18 Onyanya Health Center 9 48.9 13 59.1 22 Elombe Clinic 2 28.6 5 71.4 7 District Total 24 49.0 25 51.0 49 Oshikuku Hospital 9 30.0 21 70.0 30 Okalongo HC 14 48.3 15 51.7 29 Onheleiwa clinic 0 0.0 1 100.0 1 Oshikuku District Total 23 38.4 37 61.6 60 Tsumeb Hospital 10 52.6 9 47.4 19 Catherine Bullen clinic 9 45 11 55 20 Tsinsabis clinic 5 62.5 3 37.5 8 Tsumeb District Total 24 51.1 23 48.9 47 Grootfontein Hospital 13 59.1 9 40.9 22 Grootfontein clinic 7 50 7 50 14 Grootfontein District Total 20 55.6 16 44.4 36 All Districts Total 148 43.3 194 56.7 342 59 Annex A4. Percent of New and Relapsed TB cases put on ART while of TB Treatment by Facility, District, TX_RET MTE TB_ART District/Health Facility Can’t Determine NoT on ART while on TB Treatment On ART while on TB Treatment Tota l # # % # % # % Andara Hospital 0 0 0 0 19 100 19 Divundu Clinic 0 0 0 0 1 100 1 Bagani Clinic 0 0 0 0 1 100 1 Andara District Total 0 0 0 0 21 100 21 Nyangana Hospital 0 0 0 0 20 100 20 Mbambi Clinic 0 0 0 0 2 100 2 Katere Clinic NA Nyangana District Total 0 0 0 0 2 100 2 Odibo HC 0 0 0 0 20 100 20 Hamukuto Wakapa clinic 0 0 0 0 3 100 3 Okatape clinic 2 15.4 0 0 11 84.6 13 Odibo District Total 2 5.5 0 0 34 94.4 36 Omuthiya Hospital 6 30 1 5 13 65 20 Onyuulaye HC 0 0 0 0 1 100 1 Omuntele clinic 1 16.7 0 0 5 83.3 6 Omuthiya District Total 7 25.9 1 3.7 19 70.4 27 Onandjokwe Hospital 0 0 0 0 20 100 20 Onyanya HC 0 0 0 0 11 100 11 Elombe clinic 0 0 0 0 3 100 3 60 Onandjokwe District Total 0 0 0 0 34 100 34 Oshikuku Hospital 0 0 0 0 4 100 4 Okalongo HC 1 7.1 0 0 13 92.9 14 Onheleiwa clinic 0 0 0 0 6 100 6 Oshikuku District Total 1 4.2 0 0 23 95.8 24 Tsumeb Hospital 3 15.0 0 0 17 85 20 Catherine Bullen clinic 0 0 0 0 8 100 8 Tsinsabis clinic 0 0 3 27.3 8 72.7 11 Tsumeb District Total 3 7.7 3 7.7 33 84.6 39 Grootfontein Hospital 0 0 0 0 1 100 1 Grootfontein clinic 0 0 1 5.3 18 94.7 19 Grootfontein District Total 0 0 1 5.3 19 94.7 20 All Districts Total 13 6.4 5 2.5 185 91.1 203 Annex A5. Percent Viral Suppression by District and Health Facility, TX_PVLS MTE District/Health Facility Virally Not Suppressed Virally Suppressed Total # (%) # % # % Andara Hospital 5 25% 15 75% 20 Divundu Clinic 1 5% 19 95% 20 Bagani Clinic 0 0% 20 100% 20 Andara District Total 6 10% 54 90% 60 Nyangana Hospital 2 10.5% 17 89.5 19 Mbambi Clinic 8 42.1% 11 57.9% 19 Katere Clinic 1 11.1% 8 88.9% 9 Nyangana District Total 11 23.4% 36 76.6% 47 61 Odibo HC 2 10% 18 90% 20 Hamukuto Wakapa clinic 0 0% 21 100% 21 Okatape clinic 0 0% 14 100% 14 Odibo District Total 2 3.6% 53 96.3% 55 Omuthiya Hospital 4 20% 16 80% 20 Onyuulaye HC 1 5% 19 95% 20 Omuntele clinic 1 5% 19 95% 20 Omuthiya District Total 6 10% 54 90% 60 Onandjokwe Hospital 2 10% 18 90% 20 Onyanya HC 2 10.5% 17 89.5% 19 Elombe clinic 0 0% 13 100% 13 Onandjokwe District Total 4 7.7% 48 92.3% 52 Oshikuku Hospital 1 5.3% 18 94.7% 19 Okalongo HC 0 0% 14 100% 14 Onheleiwa clinic 0 0% 6 100% 6 Oshikuku District Total 1 2.6% 38 97.4 39 Tsumeb Hospital 0 0% 18 100% 18 Catherine Bullen clinic 1 5% 19 95% 20 Tsinsabis clinic 0 0% 20 100% 20 Tsumeb District Total 1 1.7% 57 98.3% 58 Grootfontein Hospital 5 26.3% 14 73.7% 19 Grootfontein clinic 0 0% 6 100% 6 Grootfontein District Total 5 20% 20 80% 25 All Districts Total 36 9.1% 360 90.9% 396 62 ANNEX B. SIMS Domains MTE Assessment CEE Results by District Annex B1. HTC SIMS Domain Assessment CEE Colour Score Results District HTC SIMS CEE Scores Need Urgent Remediation Need Improvement Meet Basic Expectations Surpass Basic Expectations 7.1 Compliance with national algorithm 1 0 0 2 7.2 HIV testing QA 0 1 0 2 7.3 HTC documentation 0 0 0 3 7.4 HTC referrals 1 1 0 1 7.5 HIV proficiency testing 0 0 1 2 7.6 Supply chain reliability 0 0 0 3 7.7 HTC safety measures 0 0 3 0 Andara District Total 2 2 4 13 7.1 Compliance with national algorithm 0 0 0 3 7.2 HIV testing QA 1 0 0 2 7.3 HTC documentation 0 0 0 3 7.4 HTC referrals 3 0 0 0 7.5 HIV proficiency testing 1 1 0 1 7.6 Supply chain reliability 0 0 0 3 7.7 HTC safety measures 0 0 3 0 Nyangana District Total 5 1 3 12 7.1 Compliance with national algorithm 0 0 1 2 7.2 HIV testing QA 0 0 2 1 7.3 HTC documentation 0 0 0 3 7.4 HTC referrals 2 0 1 0 7.5 HIV proficiency testing 0 2 1 0 7.6 Supply chain reliability 0 0 0 3 63 7.7 HTC safety measures 0 0 3 0 Odibo District Total 2 2 8 9 7.1 Compliance with national algorithm 0 1 0 2 7.2 HIV testing QA 0 0 0 3 7.3 HTC documentation 0 1 0 2 7.4 HTC referrals 2 0 0 1 7.5 HIV proficiency testing 0 0 0 3 7.6 Supply chain reliability 0 0 0 3 7.7 HTC safety measures 0 0 0 3 Omuthiya District Total 2 2 0 17 7.1 Compliance with national algorithm 0 0 0 3 7.2 HIV testing QA 0 0 1 2 7.3 HTC documentation 0 1 0 2 7.4 HTC referrals 3 0 0 0 7.5 HIV proficiency testing 0 1 0 2 7.6 Supply chain reliability 0 0 1 2 7.7 HTC safety measures 0 0 2 1 Onandjokwe District Total 3 2 4 12 7.1 Compliance with national algorithm 0 0 0 3 7.2 HIV testing QA 0 0 1 2 7.3 HTC documentation 0 1 0 2 7.4 HTC referrals 0 0 2 1 7.5 HIV proficiency testing 1 2 0 0 7.6 Supply chain reliability 0 0 0 3 7.7 HTC safety measures 0 0 1 2 Oshikuku District Total 1 3 4 13 7.1 Compliance with national algorithm 0 1 0 2 64 7.2 HIV testing QA 0 0 0 3 7.3 HTC documentation 0 0 0 3 7.4 HTC referrals 0 1 1 1 7.5 HIV proficiency testing 0 1 0 2 7.6 Supply chain reliability 0 0 0 3 7.7 HTC safety measures 0 0 3 0 Tsumeb District Total 0 2 4 14 7.1 Compliance with national algorithm 0 1 0 1 7.2 HIV testing QA 0 0 1 1 7.3 HTC documentation 0 0 0 2 7.4 HTC referrals 1 1 0 0 7.5 HIV proficiency testing 0 0 0 2 7.6 Supply chain reliability 0 0 0 2 7.7 HTC safety measures 0 0 2 0 Grootfontein District Total 1 2 3 8 Annex B2. ART SIMS Domain Assessment CEE Colour Score Results District ART SIMS CEE Scores Need Urgent Remediation Need Improvement Meet Basic Expectations Surpass Basic Expectations 1.1 Patient Tracking (ART) 0 0 1 2 1.2 Adherence Support 0 0 0 3 1.3 ART Monitoring 0 2 1 0 1.4 Supply Chain Reliability (Adult ARVs) 0 0 0 3 Andara District Total 0 2 2 8 65 1.1 Patient Tracking (ART) 0 0 3 0 1.2 Adherence Support 0 2 0 1 1.3 ART Monitoring 1 0 0 2 1.4 Supply Chain Reliability (Adult ARVs) 0 1 1 1 Nyangana District Total 1 3 4 4 1.1 Patient Tracking (ART) 1 0 1 1 1.2 Adherence Support 0 0 0 3 1.3 ART Monitoring 1 0 1 1 1.4 Supply Chain Reliability (Adult ARVs) 1 1 0 1 Odibo District Total 3 1 2 6 1.1 Patient Tracking (ART) 0 0 1 2 1.2 Adherence Support 0 0 0 3 1.3 ART Monitoring 0 0 0 3 1.4 Supply Chain Reliability (Adult ARVs) 0 0 1 2 Omuthiya District Total 0 0 2 10 1.1 Patient Tracking (ART) 0 1 0 2 1.2 Adherence Support 0 0 1 2 1.3 ART Monitoring 0 1 1 1 1.4 Supply Chain Reliability (Adult ARVs) 2 0 1 0 Onandjokwe District Total 2 2 3 5 1.1 Patient Tracking (ART) 2 0 1 0 1.2 Adherence Support 0 0 0 3 1.3 ART Monitoring 0 0 0 3 1.4 Supply Chain Reliability (Adult ARVs) 1 0 1 1 66 Oshikuku District Total 3 0 2 7 1.1 Patient Tracking (ART) 0 3 0 0 1.2 Adherence Support 0 0 1 2 1.3 ART Monitoring 1 0 0 2 1.4 Supply Chain Reliability (Adult ARVs) 0 0 1 2 Tsumeb District Total 1 3 2 6 1.1 Patient Tracking (ART) 0 0 1 1 1.2 Adherence Support 0 1 0 1 1.3 ART Monitoring 0 0 1 1 1.4 Supply Chain Reliability (Adult ARVs) 0 0 0 2 Grootfontein District Total 0 1 2 5 Annex B3. Performance Management SIMS Domain Assessment CEE Colour Score Results District Performance Management SIMS CEE Scores Need Urgent Remediation Need Improveme nt Meet Basic Expectation s Surpass Basic Expectations 21.1 Staff performance 0 0 2 1 21.2 In-service training 2 0 1 0 21.3 Supportive supervision 0 0 2 1 Andara District Total 2 0 5 2 21.1 Staff performance 0 1 1 1 21.2 In-service training 1 0 1 1 21.3 Supportive supervision 1 2 0 0 Nyangana District Total 2 3 2 2 21.1 Staff performance 3 0 0 0 21.2 In-service training 3 0 0 0 67 21.3 Supportive supervision 1 2 0 0 Odibo District Total 7 2 0 0 21.1 Staff performance 1 0 2 0 21.2 In-service training 2 0 1 0 21.3 Supportive supervision 1 1 1 0 Omuthiya District Total 4 1 4 0 21.1 Staff performance 1 0 0 2 21.2 In-service training 0 1 0 2 21.3 Supportive supervision 0 1 0 2 Onandjokwe District Total 1 2 0 6 21.1 Staff performance 3 0 0 0 21.2 In-service training 1 1 1 0 21.3 Supportive supervision 0 1 2 0 Oshikuku District Total 4 2 3 0 21.1 Staff performance 2 0 1 0 21.2 In-service training 2 1 0 0 21.3 Supportive supervision 0 2 1 0 Tsumeb District Total 4 3 2 0 21.1 Staff performance 1 0 0 1 21.2 In-service training 1 0 0 1 21.3 Supportive supervision 0 0 1 1 Grootfontein District Total 2 0 1 3 Annex B4. QM/QI SIMS Domain Assessment CEE Colour Score Results District HIV QM/QI SIMS CEE Scores Need Urgent Remediation Need Improveme nt Meet Basic Expectations Surpass Basic Expectations 23.1 HIV QM/QI system 1 0 1 1 68 23.2 Utilization of data in QI 0 1 1 1 Andara District Total 1 1 2 2 23.1 HIV QM/QI system 2 0 0 1 23.2 Utilization of data in QI 2 1 0 0 Nyangana District Total 4 1 0 1 23.1 HIV QM/QI system 2 0 0 1 23.2 Utilization of data in QI 2 1 0 0 Odibo District Total 4 1 0 1 23.1 HIV QM/QI system 0 2 0 1 23.2 Utilization of data in QI 0 1 2 0 Omuthiya District Total 0 3 2 1 23.1 HIV QM/QI system 0 0 0 3 23.2 Utilization of data in QI 0 0 2 1 Onandjokwe District Total 0 0 2 4 23.1 HIV QM/QI system 2 0 0 1 23.2 Utilization of data in QI 0 2 1 0 Oshikuku District Total 2 2 1 1 23.1 HIV QM/QI system 2 0 0 1 23.2 Utilization of data in QI 2 0 1 0 Tsumeb District Total 4 0 1 1 23.1 HIV QM/QI system 0 0 1 1 23.2 Utilization of data in QI 0 1 1 0 Grootfontein District Total 0 1 2 1 69 Annex B5. Medication Management SIMS Domain Assessment CEE Colour Score Results District Medication Management SIMS CEE Scores Need Urgent Remediation Need Improvement Meet Basic Expectations Surpass Basic Expectations 15.1 Supply chain management 0 1 0 2 15.2 Medication Dispensing 1 0 0 2 Andara District Total 1 1 0 4 15.1 Supply chain management 0 1 0 1 15.2 Medication Dispensing 1 1 0 0 Nyangana District Total 1 2 0 1 15.1 Supply chain management 1 0 0 2 15.2 Medication Dispensing 1 0 1 1 Odibo District Total 2 0 1 3 15.1 Supply chain management 0 0 0 3 15.2 Medication Dispensing 0 1 2 0 Omuthiya District Total 0 1 2 3 15.1 Supply chain management 0 0 1 2 15.2 Medication Dispensing 0 1 0 2 Onandjokwe District Total 0 1 1 4 15.1 Supply chain management 1 0 0 2 15.2 Medication Dispensing 0 1 0 2 Oshikuku District Total 1 1 0 4 15.1 Supply chain management 0 0 0 2 15.2 Medication Dispensing 0 1 0 1 70 Tsumeb District Total 0 1 0 3 15.1 Supply chain management 0 0 1 1 15.2 Medication Dispensing 0 0 0 2 Grootfontein District Total 0 0 1 3 71 ANNEX C. Adult and Paediatric EWI Site Specific Results Extracted From 2016 MOHSS National EWI Assessment Report for UTAP MTE Annex C1. Adult EWI Site Specific Results Region Main Site Num ber Main Site /Outreach Site EWI 1:On time pill Pick up EDT alone EWI 1:On time pill Pick up with ePM S EWI 2: Retent ion at 12 Month s EDT alone EWI 2: Retent ion at 12 Month s with ePMS EWI 3: Pharm acy Stoke out EWI 4: Dispen sary Practic es EWI 5: Viral Load Suppres sion at 12 Months EDT Alone EWI 5: Viral Load Suppres sion at 12 Months EDT with ePMS EWI 5a: Viral Load comple tion at 12 Months EDT Alone EWI 5: Viral Load comple tion at 12 Months EDT with ePMS Kavango 12 Andara Hospital 1593/1 934 (82%) 874/1 072 (82%) 284/38 5 (74%) 256/34 7 (74%) 12/12 (100%) 0/920 (0%) 3/4 (75%) 3/4 (75%) 4/284 (1%) 4/256 (2%) Andara Hospital 590/69 4 (85%) 150/1 79 (84%) 94/130 (72%) 87/118 (74%) 12/12 (100%) 0/155 (0%) 1/1 (100%) 2/2 (100%) 1/94 (1%) 2/87 ( 2%) Divundu Clinic 277/32 5 (85%) 139/1 57 (89%) 50/60 (83%) 42/52 (81%) 12/12 (100%) 0/130 (0%) NA NA 0/50 (0%) 0/42 (0%) Old Bagani Clinic 145/19 7 (74%) 80/11 8 (68%) 33/42 (79%) 26/33 (79%) 12/12 (100%) 0/100 (0%) 2/2 (100%) 1/1 (100%) 2/33 (6%) 1/26 (4%) Mbambi Clinic 127/14 7 (86%) 108/1 20 (90%) 1/9 (11%) NA 7/12 (58%) 0/100 (0%) NA NA 0/1 (0%) NA Katere Clinic 16/37 (43%) 11/31 (35%) 7/7 (100%) 4/4 (100%) 7/12 (58%) 0/37 (0%) NA NA 0/7 (0%) 0/4 (0%) 15 Nyangana Hospital 1627/1 930 425/5 17 245/31 3 (78%) 225/28 7 (78%) 7/12 (58%) 0/512 (0%) 3/3 (100%) 4/4 (100%) 3/245 (1%) 4/225 (2%) 72 (84%) (82%) Nyangana Hospital 1276/1 513 (84%) 176/2 15 (82%) 226/29 3 (77%) 210/27 2 (77%) 7/12 (58%) 0/175 (0%) 3/3 (100%) 4/4 (100%) 3/226 (1%) 4/210 (2%) Ohangw ena 29 Odibo Health Center 1837/2 010 (91%) NA 248/30 8 (81%) NA 8/12 (67%) 0/273 (0%) 177/183 (97%) NA 183/248 (74%) NA Odibo Health Center 1756/1 917 (92%) NA 238/29 6 (80%) NA 8/12 (67%) 0/180 (0%) 175/181 (97%) NA 181/238 (76%) NA Okatope Clinic 57/106 (54%) 54/90 (60%) NA NA 6/12 (50%) 0/75 (0%) NA NA NA NA Omusati 36 St' Martins Hospital (Oshikuku Hospital ) 3955/4 422 (89%) 276/3 20 (86%) 441/52 8 (84%) 427/49 4 (86%) 10/12 (83%) 0/406 (0%) 209/229 (91%) 199/222 (90%) 229/441 (52%) 222/427 (52%) St' Martins Hospital (Oshikuku Hospital ) 3828/4 239 (90%) 220/2 39 (92%) 395/47 6 (83%) 385/44 8 (86%) 10/12 (83%) 0/210 (0%) 192/211 (91%) 181/203 (89%) 211/395 (53%) 203/385 (53%) Okalongo HC 2658/2 930 (91%) 223/2 43 (92%) 320/36 1 (89%) 325/35 7 (91%) 8/12 (67%) 0/200 (0%) 145/155 (94%) 141/153 (92%) 155/320 (48%) 153/325 (47%) Onheleiwa Clinic 0/2 (0%) 0/2 (0%) NA NA 10/12 (83%) 0/2 (0%) NA NA NA NA Oshikoto 43 Onandjokwe Hospital 5047/8 352 (71%) NA 815/10 90 (75%) NA 3/12 (25%) 2/520 (0%) 459/487 (94%) NA 487/815 (60%) NA Onandjokwe Hospital 5704/7 843 (73%) NA 796/10 63 (75%) NA 3/12 (25%) 0/215 (0%) 450/478 (94%) NA 478/796 (60%) NA 73 Onyaanya HC 11/22 (50%) NA 0/1 (0%) NA 3/12 (25%) 0/22 (0%) NA NA NA NA 42 Omuthiya Hospital 2496/3 797 (66%) 341/7 11 (48%) 490/66 8 (73%) 446/56 7 (79%) 8/12 (67%) 0/603 (0%) 211/227 (93%) 197/207 (95%) 227/490 (46%) 207/446 (46%) Omuthiya Hospital 2182/3 060 (71%) 151/2 40 (63%) 424/58 0 (73%) 386/49 3 (78%) 8/12 (67%) 0/173 (0%) 186/201 (93%) 174/184 (95%) 201/424 (47%) 184/386 (48%) Omuntele Clinic 123/32 0 (38%) NA 9/15 (60%) NA 3/12 (25%) 0/130 (0%) 5/5 (100%) NA 5/9 (56%) NA Onyulaye Clinic 128/24 5 (52%) 76/12 9 (59%) 15/21 (71%) 14/18 (78%) 8/12 (67%) 0/120 (0%) 5/5 (100%) 5/5 (100%) 5/15 (33%) 5/14 (36%) 45 Tsumeb Hospital 1865/2 183 (85%) 243/3 23 (75%) 294/41 8 (70%) 284/38 0 (75%) 7/12 (58%) 0/280 (0%) 144/157 (92%) 143/156 (92%) 157/294 (53%) 156/284 (55%) Tsumeb Hospital 1806/2 072 (87%) 185/2 14 (86%) 276/38 0 (71%) 266/35 5 (75%) 7/12 (58%) 0/180 (0%) 136/148 (92%) 135/147 (92%) 148/276 (54%) 147/266 (55%) Tsinsabis Clinic 59/111 (53%) 58/10 9 (53%) 18/29 (62%) 18/25 (72%) 7/12 (58%) 0/100 (0%) 8/9 (89%) 8/9 (89%) 9/18 (50%) 9/18 (50%) Catherine Bullen (Oshivelo) Clinic 468/55 7 (84%) 163/1 86 (88%) 65/86 (76%) 61/72 (85%) 7/12 (58%) 1/155 (1%) 31/31 (100%) 26/26 (100%) 31/65 (48%) 26/61 (43%) Otjozond jupa 46 Grootfontein Hospital 1578/1 901 (83%) 231/2 85 (81%) 233/38 0 (61%) 229/32 1 (71%) 11/12 (92%) 0/334 (0%) 51/64 (80%) 51/61 (84%) 64/213 (27%) 61/229 (27%) Grootfontein Hospital 1524/1 809 182/2 13 228/32 7 (70%) 225/30 5 (74%) 11/12 (92%) 0/180 (0%) 48/60 (80%) 49/58 (84%) 60/228 (26%) 58/225 (26%) 74 (84%) (85%) Grootfontein Poly Clinic 4/13 (31%) 4/4 (100% ) 3/33 (9%) 2/3 (67%) 11/12 (92%) 0/13 (0%) 2/3 (67%) 1/2 (50%) 3/3 (100%) 2/2 (100%) N.B. Hamukuto Wa Kapa and Elome clinics are sampled in MTE but have no data in 2016 EWI assessment report Annex C2. Paediatric EWI Site Specific Results Region Main Site Num ber Site: Main Site/ Outrech site EWI 1:On time pill Pick up EDT alon e EWI 1:On time pill Pick up with ePM S EWI 2: Retent ion at 12 Month s EDT alone EWI 2: Retent ion at 12 Month s with ePMS EWI 3: Pharm acy Stoke out EWI 4: Dispen sary Practic es EWI 5: Viral Load Suppres sion at 12 Months EDT Alone EWI 5: Viral Load Suppres sion at 12 Months EDT with ePMS EWI 5a: Viral Load comple tion at 12 Months EDT Alone EWI 5: Viral Load comple tion at 12 Months EDT with ePMS Kavango 12 Andara Hospital 142/1 75 (81%) 131/1 63 (80%) 27/30 (90%) 26/29 (90%) 9/12 (75%) 0/176 (0%) NA NA 0/27 (0%) 0/26 (0%) Andara Hospital 56/71 (79%) 55/69 (80%) 9/9 (100%) 9/9 (100%) 9/12 (75%) 0/71 (0%) NA NA 0/9 (0%) 0/9 (0%) Divundu Clinic 25/30 (83%) 21/25 (84%) 7/8 (88%) 7/8 (88%) 9/12 (75%) 0/31 (0%) NA NA 0/7 (0%) 0/7 (0%) Old Bagani Clinic 18/19 (95%) 14/15 (93%) 5/5 (100%) 5/5 (100%) 9/12 (75%) 0/19 (0%) NA NA 0/5 (0%) 0/5 (0%) Mbambi Clinic 13/14 (93%) 13/14 (93%) 2/2 (100%) NA 7/12 (58%) 0/14 (0%) NA NA 0/2 (0%) NA 75 Katere Clinic 1/1 (100 %) 2/2 (100 %) 1/1 (100%) 1/1 (100%) 7/12 (58%) 0/1 (0%) NA NA 0/1 (0%) 0/1 (0%) 15 Nyangana Hospital 176/2 00 (88%) 127/1 41 (90%) 15/30 (50%) 14/28 (50%) 7/12 (58%) 0/151 (0%) NA NA 0/15 (0%) 0/14 (0%) Nyangana Hospital 133/1 53 (87%) 97/10 9 (89%) 13/27 (48%) 12/26 (46%) 7/12 (58%) 0/100 (0%) NA NA 0/13 (0%) 0/12 (0%) 29 Odibo HC 212/2 26 (94%) NA 21/26 (81%) NA 7/12 (58%) 0/132 (0%) 15/16 (94%) NA 16/21 (76%) NA Ohangwe na Odibo Health Center 190/2 04 (93%) NA 21/24 (88%) NA 7/12 (58%) 0/110 (0%) 15/16 (94%) NA 16/21 (76%) NA Okatope Clinic 10/20 (50%) 3/6 (50%) NA NA 5/12 (42%) 0/20 (0%) NA NA NA NA Omusati 36 Oshikuku (St' Martins) Hospital 428/4 70 (91%) 162/1 80 (90%) 70/73 (96%) 70/74 (95%) 8/12 (67%) 0/184 (0%) 45/49 (92%) 44/48 (92%) 49/70 (70%) 48/70 (69%) Oshikuku Hospital 420/4 57 (92%) 156/1 69 (92%) 66/69 (96%) 66/70 (94%) 8/12 (67%) 0/145 (0%) 42/46 (91%) 41/45 (91%) 46/66 (70%) 45/66 (68%) 34 Okalongo HC 427/4 59 (93%) 159/1 63 (98%) 88/93 (95%) 72/76 (95%) 3/12 (25%) 0/145 (0%) 48/51 (94%) 44/47 (94%) 51/88 (58%) 47/72 (65%) Okalongo HC 427/4 59 (93%) 159/1 63 (98%) 88/93 (95%) 72/76 (95%) 3/12 (25%) 0/145 (0%) 48/51 (94%) 44/47 (94%) 51/88 (58%) 47/72 (65%) Oshikoto 43 Onandjokwe Hospital 510/8 38 (61%) NA 141/17 9 (79%) NA 5/12 (42%) 0/266 (0%) 88/102 (86%) NA 102/141 (72%) NA 76 Onandjokwe Hospital 470/7 32 (64%) NA 125/16 2 (77%) NA 5/12 (42%) 0/160 (0%) 77/89 (87%) NA 89/125 (71%) NA Onyaanya HC 2/4 (50%) NA NA NA 5/12 (42%) 0/4 (0%) NA NA NA NA 42 Omuthiya Hospital 305/4 16 (68%) 108/2 52 (67%) 86/98 (88%) 83/92 (90%) 3/12 (25%) 1/241 (0%) 28/37 (76%) 28/36 (78%) 37/86 (43%) 36/83 (43%) Omuthiya Hospital 257/3 39 (76%) 123/1 49 (83%) 66/74 (89%) 69/74 (93%) 3/12 (25%) 1/130 (1%) 23/30 (77%) 23/30 (77%) 30/66 (45%) 30/69 (43%) Omuntele Clinic 14/65 (22%) NA 6/7 (86%) NA 5/12 (42%) 0/65 (0%) 3/5 (60%) NA 5/6 (83%) NA Onyulaye Clinic 24/42 (57%) 22/39 (56%) 2/3 (67%) 1/2 (50%) 3/12 (25%) 0/42 (0%) NA NA 0/2 (0%) 0/1 (0%) 45 Tsumeb Hospital 90/10 5 (86%) 83/95 (87%) 20/26 (77%) 18/24 (75%) 8/12 (67%) 0/83 (0%) 6/10 (60%) 6/10 (60%) 10/20 (50%) 10/18 (56%) Tsumeb Hospital 83/97 (86%) 76/86 (88%) 16/22 (73%) 14/20 (70%) 8/12 (67%) 0/75 (0%) 6/10 (60%) 6/10 (60%) 10/16 (63%) 10/14 (71%) Tsinsabis Clinic 7/8 (88%) 7/9 (78%) 4/4 (100%) 4/4 (100%) 8/12 (67%) 0/8 (0%) NA NA 0/4 (0%) 0/4 (0%) Catherine Bullen (Oshivelo Clinic) 40/45 (89%) 39/42 (93%) 7/7 (100%) 7/7 (100%) 6/12 (50%) 0/45 (0%) 2/3 (67%) 2/3 (67%) 3/7 (43%) 3/7 (43%) Otjozond jupa 46 Grootfontein Hospital 88/10 4 (85%) 81/94 (86%) 15/22 (73%) 16/22 (73%) 12/12 (100%) 0/86 (0%) 4/6 (67%) 3/7 (71%) 6/15 (40%) 7/16 (44%) Grootfontein Hospital 85/98 (87%) 78/89 (88%) 14/19 (74%) 15/19 (79%) 12/12 (100%) 0/75 (0%) 4/5 (80%) 5/6 (83%) 5/14 (36%) 6/15 (40%) N.B. Hamukuto Wa Kapa, Elome and Grootfontein clinics are sampled in MTE but have no data in 2016 EWI assessment report 77