August 2020 This publication was produced at the request of the United States Agency for International Development. It was prepared independently by ME&A, Inc. and ICF as subcontractor. MID-TERM PERFORMANCE EVALUATION OF THECAMBODIA MALARIA ELIMINATION PROJECT Final Report MID-TERM PERFORMANCE EVALUATION CAMBODIA MALARIA ELIMINATION PROJECT August 2020 This publication was produced at the request of the United States Agency for International Development Developed under Agreement Number: AID-OAA-I-15-00024/72044220F00001. It was prepared independently by: Leopoldo Villegas, MD, DTM&H, MSc, DrPH, AdvDPHM (ME&A) Bopha Kong, LLM (ME&A) Young Pheak, MDM (ICF) Contractor: ME&A, Inc. 4350 East-West Highway, Suite 210 Bethesda, MD 20814 Tel: 240-762-6296 www.MEandAHQ.com Subcontractor: ICF International DISCLAIMER The author’s views expressed in this publication do not necessarily reflect the views of the United States Agency for International Development or the United States Government. ABSTRACT The five-year Cambodia Malaria Elimination Project (CMEP) aims to intensify malaria control and elimination activities through technical assistance and support to the National Center for Parasitology, Entomology, and Malaria Control (CNM) to further develop, refine, and evaluate an evidence-based Model Elimination Package for malaria, for implementation in other appropriate operational districts (ODs). The Royal Government of Cambodia’s (RGC) goals are eliminating Plasmodium falciparum (Pf) and mixed infections (Pf + P. vivax) and zeroing death by 2020, and eliminating malaria infections by 2025. CMEP is aligned with the national malaria strategies, Malaria Elimination Action Framework (MEAF) Phase 1. The evaluation aimed to determine CMEP’s results, identify lessons learned and best practices, and make recommendations for future USAID malaria programs. The evaluation team (ET) conducted a desk review of materials, interviews with key stakeholders, focus group discussions with direct beneficiaries, and a survey with OD and health facility service providers and at-risk populations. The ET found CMEP’s interventions and activities have: 1) impacted malaria transmission after implementation of the 1-3-7 elimination model in Sampov Loun OD, significantly reducing falciparum malaria cases; 2) strengthened national capacities to assess, plan, train, supervise, and monitor programs; 3) provided Social and Behavior Change Communication interventions at the OD level linking affected communities to improved malaria (and health wellness) services; and 4) reduced stock outs and built capacity to address malaria commodity disruptions. Recommendations include: supporting increased access to malaria programming support for key and vulnerable populations; expanding quality improvement of malaria programs; increasing emphasis on additional approaches for eliminating vivax malaria; and improving outcome and impact indicator reporting in the current malaria elimination scenario. APPRECIATION The evaluation team (ET) wishes to acknowledge helpful guidance from the United States Agency for International Development (USAID) Cambodia team for the Cambodia Malaria Elimination Project (CMEP), especially the Task Order Contract Officer’s Representative, Ms. Adriana Harvey. The ET would like to express appreciation to the CMEP team that organized our interviews and field visits on very short notice and in often very challenging circumstances. We are grateful for the Cambodian National Center for Malaria Control, Parasitology, and Entomology (CNM) Director and Deputy Director and focal (CMEP) person for their support in the evaluation process. In addition, we would like to thank the Provincial Health Department and Operational District (OD) representatives, health facility managers, health workers, implementing partners, and donor organization managers that took time from their very demanding schedules to meet with the team and gave us thoughtful and detailed answers to our questions. The ET also appreciates the support from USAID headquarters, the U.S. President’s Malaria Initiative, and the U.S. Centers for Disease Control and Prevention’s staff responsiveness to queries. We would especially like to thank the groups of Village Malaria Workers, Mobile Malaria Workers, and village/settlement/community leaders for their warm welcomes into their communities and for taking the time to answer important questions about the benefits they derived from improved malaria and health services that they received from the project across the six ODs visited during the mid-term evaluation. Lastly, the ET would like to thank ME&A’s (Wondimu Wolde, Alex Dow, and Mirela McDonald) and ICF’s (Debra Prosnitz, Andrew Andrada, and Diadier Diallo) Home Office experts who helped with the review and data analysis of the report. CONTENTS EXECUTIVE SUMMARY.................................................................................................................................. i 1.0 EVALUATION PURPOSE AND QUESTIONS............................................................................ 1 1.1 Evaluation Purpose................................................................................................................ 1 1.2 Evaluation Questions............................................................................................................ 1 2.0 PROJECT BACKGROUND ............................................................................................................. 2 3.0 EVALUATION METHODOLOGY................................................................................................. 4 3.1 Sampling................................................................................................................................... 5 3.2 Data Collection Methods.................................................................................................... 5 3.3 Data Analysis.......................................................................................................................... 7 3.4 Evaluation Limitations........................................................................................................... 7 4.0 FINDINGS BY EVALUATION QUESTION................................................................................. 8 4.1 EQ1: To what extent has the CMEP project developed a scalable, evidence-based elimination model in SPL and supported its widespread dissemination and replication in other target ODs in Cambodia?............................................................... 8 4.2 EQ2: To what extent has CMEP supported high-quality malaria control and prevention interventions in five ODs and expanded transitional ODs, where gaps in coverage or quality may exist? ................................................................................... 16 4.3 EQ3: Does the current national malaria surveillance system serve its intended purpose to detect, immediately notify, investigate, respond, and support appropriately for malaria elimination and control activities? . ................................. 24 4.4 EQ4: To what extent has the CMEP project built capacity of the CNM/MOH to manage, intensify, and sustain malaria control and elimination efforts, particularly at the PHD and OD levels?.............................................................................................. 26 4.5 EQ5: What factors, both internal and external of CMEP, have facilitated or constrained project performance and how can these factors be addressed? ...... 33 5.0 CONCLUSIONS AND RECOMMENDATIONS..................................................................... 38 5.1 Conclusions ......................................................................................................................... 38 5.2 Recommendations.............................................................................................................. 41 ANNEXES ....................................................................................................................................................... 46 Annex 1: Statement of Work........................................................................................................ 47 Annex 2: CMEP Mid-Term Evaluation Schedule ....................................................................... 58 Annex 3: Evaluation Design Matrix.............................................................................................. 61 Annex 4: Documents Reviewed ................................................................................................... 66 Annex 5: List of Participants in Focus Group Discussions, Mini-Survey, and Key Informant Interviews.......................................................................................................... 70 Annex 6: Illustrative Data Collection Tools .............................................................................. 74 Annex 7: Bios and Summary Information of Team Members.............................................. 133 Annex 8: CMEP Performance Indicator Tracking Table (PITT).......................................... 134 Annex 9: Indicative List of Selected Protocols, Guidelines, and Standards Developed with CMEP Support by Subject and Type(s) of Assistance..................................... 157 Annex 10: CMEP Testing and Treatment (FY 2017-Fy 2020) ............................................. 159 Annex 11: CMEP Testing and Treatment by OD (Fy 2017-fy 2020)................................. 160 Annex 12: Mini-Survey Results ................................................................................................... 162 LIST OF TABLES Table 1: Sample of KIIs by Respondent Category ..................................................................................... 5 Table 2: Focus Group Discussions by Operational Districts.................................................................. 6 Table 3: CMEP Operational Districts Starting Dates and Elimination Timeline.............................. 12 Table 4: Internal and External Factors Affecting CMEP’s Performance ............................................ 33 LIST OF FIGURES Figure 1: Number of Malaria Indigenous and Imported Cases in SPL OD 2015-2018 .................. 10 Figure 2: Malaria Cases Per Operational District and Year of CMEP Implementation ................. 13 Figure 3: Total Number of Malaria Cases Among CMEP Operational Districts, 2016-2020 ....... 14 Figure 4: Percentage Share Contribution of the Malaria Cases Among CMEP Operational Districts, 2016-2020 ........................................................................................................................... 14 Figure 5: Percentage of At-Risk Population Satisfaction With CMEP Malaria Activities in Which They Know or Have Participated .................................................................................................... 23 Figure 6: Percentage of Health Service Providers Opinion on Usefulness of Training.................. 28 Figure 7: Percentage of Health Service Providers’ Opinion on Supervision Usefulness (N=89) . 29 Figure 8: Percentage of Service Providers’ Opinion on Trainings’ Success...................................... 30 Figure 9: A. Country Funding Landscape 2005-2017 (All Donors) and B. Domestic Funding Only 2018-2023 ............................................................................................................................................. 36 Figure 10: CMEP’s Expansion and Proposed Sustainable Steps........................................................... 37 ACRONYMS Acronym Definition ACT Artemisinin-Based Combination Therapy AHEAD Action for Health Development AOP Annual Operational Plan API Annual Parasite Incidence AS-MQ Artesunate-Mefloquine BCC Behavior Change Communication BKN Bakan BMGF Bill and Melinda Gates Foundation BTB Battambang CAP Control and Prevention Malaria Project CBO Community-Based Organization CDC Centers for Disease Control and Prevention CHAI Clinton Health Access Initiative CMEP Cambodia Malaria Elimination Project CNM Cambodian National Center for Malaria Control, Parasitology, and Entomology COP Chief of Party CSO Civil Society Organization D Day d Deficiency DHA-P Dihydroartemisinin-Piperaquine DOT Directly Observed Therapy EDAT Early Diagnosis and Treatment EQ Evaluation Question ET Evaluation Team FGD Focus Group Discussion FY Fiscal Year G6PD Glucose-6-Phosphate-Dehydrogenase GF Global Fund GFATM Global Fund to Fight AIDS, Tuberculosis, and Malaria HC Health Center HCW Health Care Worker HF Health Facility HQ Headquarters HR Human Resources IP Implementing Partner IPC Interpersonal Communication ITN Insecticide-Treated Net KII Key Informant Interview KRK Krakor LLIN Long-Lasting Insecticide-Treated Bed Net LLIHN Long-Lasting Insecticide-Treated Hammock Net Acronym Definition M&E Monitoring and Evaluation MDR Multidrug Resistant MEAF Malaria Elimination Action Framework MEP Model Elimination Package MIS Malaria Information System MMP Mobile and Migrant Population MMW Mobile Malaria Worker MOH Ministry of Health MRS Maung Russey NGO Non-Governmental Organization NTG National Treatment Guideline OD Operational District OEC Oephan Ecole Corp OR Operational Research PDFA Partners for Health Development Pf Plasmodium Falciparum PHD Provincial Health Department PITT Performance Indicator Tracking Tool PKV Phnom Kravanh PLN Pailin PMI U.S. President’s Malaria Initiative POC Point of Care PP Private Provider PPM Public-Private Mix PSI Population Service International PSM Procurement Supply Management Pv Plasmodium vivax Q Quarter QI Quality Improvement RDT Rapid Diagnostic Test RF Results Framework RGC Royal Government of Cambodia SBCC Social and Behavior Change Communication SLDPQ Single Low-Dose Primaquine SOP Standard Operating Procedure SOW Statement of Work SPL Sampov Loun SPM Sampov Meas TMK Thamar Kaul TO Task Order TOCOR Task Order Contracting Officer’s Representative UNOPS United Nations Office for Project Services URC University Research Co., LLC Acronym Definition USAID United States Agency for International Development USG United States Government VMW Village Malaria Worker WHO World Health Organization Y Year i EXECUTIVE SUMMARY EVALUATION PURPOSE The purpose of the performance evaluation of the United States Agency for International Development (USAID) Cambodia Malaria Elimination Project (CMEP) is: 1) to assess activity implementation and performance to date and identify key bottlenecks and constraints; and 2) to make actionable recommendations for improvements needed to meet the objectives. USAID will use the lessons learned and challenges identified to develop future projects, with the intention to replicate successful approaches. The results will be shared with the USAID Mission, implementing partners (IPs), the National Center for Parasitology, Entomology, and Malaria Control (CNM) at the Ministry of Health (MOH), and different stakeholders. Furthermore, the recommendations will be relevant for the remaining years of CMEP, the new follow-on project, and possibly to inform CNM’s update of the Malaria Elimination Action Framework Phase II. PROJECT BACKGROUND CMEP is implemented in Western Cambodia and led by University Research Co., LLC (URC) in collaboration with the MOH of the Royal Government of Cambodia (RGC). CMEP is a five-year award intended to intensify malaria control and elimination activities by providing technical assistance and support to the CNM to further develop, refine, and evaluate an evidence-based Model Elimination Package (MEP) for malaria, which can then be implemented in other appropriate Operational Districts (ODs). CMEP is currently being implemented in nine ODs in three provinces (Pursat, Battambang, and Pailin). In Year 1 (Y1), the project was implemented in six ODs: Sampov Loun (SPL), Battambang (BTB), Maung Russey (MRS), Thamar Kaul (TMK), Phnom Kravanh (PKV), and Krakor (KRK). Starting in Quarter 3 (Q3) (May 2018) of Y2, the project expanded to three additional ODs (total of nine ODs), including Pailin (PLN), Bakan (BKN), and Sampov Meas (SPM). Three transitional ODs graduated to elimination ODs (BTB, MRS, and TMK) with a total of seven out of nine ODs that reached elimination status by 2020. At the time of the evaluation PKV and KRK ODs are still transitional ODs. 1 CMEP actively supports the development, implementation, and monitoring of OD annual operational plans (AOPs) and introduction of data visualization to support data use in malaria control and elimination. CMEP uses a health OD-centered approach to build capacity and joint supervision at all levels in the health system to strengthen both the quality of care and routine reporting. In order to increase reach to at-risk-populations, the project team works with community networks to: improve access to and quality of early diagnosis and treatment (EDAT); provide support to strengthening systems for passive and active surveillance; support distribution of insecticide-treated nets (ITNs); implement a range of social and behavior change communication (SBCC) activities; and build central and local-level capacities in the health system for sustainability. CMEP’s period of performance is October 26, 2016, through October 25, 2021. The activity funding is $16,543,857. 1 PKV and KRK ODs are implementing partial Plasmodium falciparum (Pf) elimination activities as approved by the U.S. President’s Malaria Initiative (PMI) in April 2020. ii EVALUATION METHODOLOGY The evaluation was conducted between March and July 2020 and used a mixed-methods approach that included: 1) a review of key documents and available data; 2) key informant interviews (KIIs2 ) with CNM officials, IPs, health facility (HF) managers and staff, provincial health department (PHD) managers, private providers (PPs), and United States Government (USG) personnel3 (including USAID, the U.S. President’s Malaria Initiative [PMI], and the Centers for Disease Control and Prevention [CDC]), (123 total); 3) focus group discussions (FGDs) with service providers and at￾risk populations (14 FGDs; 62 participants); and 4) a survey with 203 health service providers and at-risk populations that benefited from and/or participated in the project’s activities. The field work was conducted in six operational districts (SPL, BTB, PLN, PKV, KRK, and SPM), at the beginning of the rainy season at a time when many people were moving to the plantations. There are several limitations that affected implementation of the evaluation, including COVID-19 related challenges (Team Leader working remotely), internal travel restrictions, and limited access to CNM data. FINDINGS Evaluation Question 1 (EQ1): To what extent has the CMEP project developed a scalable, evidence-based elimination model in SPL and supported its widespread dissemination and replication in other target ODs in Cambodia? The evaluation found that CMEP’s elimination model was effectively developed and implemented in SPL during Y1, achieving 100 percent of the targets for the 1-3-7 elimination model—cases notified within one day, investigated within three days, and responded to within seven days—for 101 identified cases. By Y3, CMEP results from the 1-3-7 elimination activities showed that 97 percent of cases were notified within one day and 100 percent of those cases were investigated within three days. There were no data reported in the CMEP performance indicator tracking tool (PITT) for the responses within seven days in SPL. CNM did not provide guidance on collecting and reporting on the seven-day response until June 21, 2019 (CMEP Y4), which affected reporting on this indicator nationally. 4 The implementation of the 1-3-7 activities over the last three years led to a 69 percent reduction of the annual parasite incidence (API)5 of all malaria species. Parasite-specific API in SPL showed a 99, 100, and 40 percent reduction of Plasmodium falciparum (Pf), Plasmodium vivax (Pv), and mixed cases (Pf+Pv), respectively. The number of severe malaria cases per 100,000 population dropped 100 percent, from 5.54 (baseline) to 0 in Y3; similarly, the test positive rate decreased from 4 percent in Y1 to 1 percent in Y3. The operationalization of the model involved staff at all levels of the health system (PHD, OD, HF, PP) and communities’ members (village malaria workers [VMWs], mobile malaria 2 KIIs included: USG personnel; IP officers; PHD, CNM, and OD staff; service providers (HF and PP); and the CMEP team. 3 USG personnel are also referred to in this evaluation as USG respondents in the KIIs. 4 Until June 21, 2019 (CMEP Y4), there was no direction or guidance (or standard operation procedures, tools), from CNM, thus seven-day active foci investigation in the whole country was zero. The actual work on Day 7 response for active foci investigation started post June 21, 2019 which is CMEP Y4. 5 API is based in a country-specific definition that differs from the international established World Health Organization (WHO) definition. Cambodia includes all malaria cases as part of API definition without distinguishing indigenous from imported cases. iii workers [MMWs], community-based organizations [CBOs], malaria patients, and at-risk populations). Between October 2016 and May 2020, the 1-3-7 model and additional interventions in SPL reported 35,698 malaria tests carried among suspected cases from which 432 were positive; 99.5 percent of cases received effective treatment (data from the national Malaria Information System [MIS]). Although there is some evidence of indigenous and imported malaria cases in SPL in CMEP reports, the ET did not have access to the malaria database that had indigenous and imported malaria cases disaggregated and, consequently, the ET was unable to assess malaria-free areas in other ODs. Overall results of CMEP’s 25 performance indicators showed progress over time (see Annex 8 – PITT).6 There were several challenges at the beginning of CMEP implementation including poor quality of deliverables (data quality problems, weak presentation, structure, and writing errors), delayed submissions, and low expenditure rate of funds (see EQ5). By the end of Y1, the above challenges were successfully addressed and a new Chief of Party (COP) and a Senior Monitoring and Evaluation (M&E) Officer were hired in Y2 (January 2018). VMWs and MMWs have been trained and supported by the project to promote behaviors using effective SBCC methods to promote healthy outcomes, and methods to prevent diseases and/or mitigate their effects. VMWs/MMWs were credited as playing a key role with affected communities, settlements, villages, and mobile migrant groups and their families to improve malaria service provision (diagnosis and treatment) and successfully promote care-seeking for prevention. CMEP expanded its activities to nine ODs by Y2, including PLN, BKN, and SPM with BTB, MRS, and TMK moved to elimination OD status. 7 Malaria diagnosis, treatment, and prevention services for all affected and vulnerable populations have been strengthened. Between October 2016 and May 2020, CMEP’s and other malaria stakeholders’ activities in the nine ODs identified 257,836 suspected malaria cases that were tested, of which 33,065 were positive. Overall, the number of Pf, Pv, and mixed malaria confirmed cases were 8,773, 23,558, and 734, respectively.8 There was a malaria epidemic in 2017-2018 with a significant monthly increase of cases in KRK and PKV; mostly vivax malaria cases. CNM led the development of an intensification plan to address this malaria outbreak in collaboration with partners and with CMEP support. The reduction of multidrug resistant (MDR) falciparum malaria (Pf only and mixed falciparum-vivax) cases was highlighted as a key achievement for CMEP’s work (overall a 97 percent reduction for Pf only and 94 percent reduction for mixed infections).9 Data from CMEP PITT show overall diagnosis and treatment rates (Annex 8). 6 There were variations in the selected ODs moving from transition to elimination to elimination status over the years. By 2020, all ODs have an elimination status as per CNM criteria. The CMEP M&E Plan was revised in March 2018 and CMEP indicators were refined and aligned to the CNM MIS. 7 The phasing of malaria elimination was based on the stratification and the operational capacity of the national program to transition into elimination in the MEAF 2016-2020. Elimination-targeted ODs: cluster of ODs with API under 1 per 1,000 population including neighboring ODs with API at the lower threshold of 1-10 per 1,000 population chosen because of operational feasibility due to them being part of the same province, low absolute number of infections, and/or their multidrug resistance status. Transitional ODs: cluster of ODs with average API higher than the elimination-targeted ODs, with available evidence of multidrug resistance. These are positioned for elimination targeting in the subsequent year. Burden Reduction ODs: API over 10 per 1,000 population, regardless of evidence of MDR. Malaria Free ODs: Historically, considered to be without local transmission. 8 Based on the MIS data from October 2016 until May 2020. 9 Ibid. iv EQ2: To what extent has the CMEP project supported high-quality malaria control and prevention interventions in five ODs and expanded transitional ODs, where gaps in coverage or quality may exist? The evaluation found there was increased access to effective malaria interventions, especially for vector control and case management. All USG respondents noted that CMEP’s work has been effective in the expansion of high quality interventions. Two USG respondent remarked “CMEP’s role setting up a reference model for the malaria response in Cambodia” included “a highly-skilled team developing new technical tools adapted to local context at the OD level.” There were new interventions added during implementation of the CMEP activities: introduction of low-dose primaquine (SLDPQ) for falciparum malaria; piloting of radical cure of vivax malaria (in two provinces10) and the introduction of qualitative point of care (POC)11 testing of glucose-6-phosphate￾dehydrogenase (G6PD) deficiency in selected HFs; as well as the continued distribution of long lasting insecticide-treated bed nets (LLINs) and hammock nets (LLIHNs). KIIs with USG, CMEP, and the IP commented on the need to increase focus on vivax malaria elimination issues since vivax malaria was more prevalent in all ODs. Vivax malaria requires different approaches that should be implemented before outbreaks start appearing; vivax malaria transmission occurs before people present clinical symptoms. Most service providers and at-risk populations rated the interventions implemented by CMEP in their villages, settlements, or among mobile groups very highly. Expansion of the elimination model and other malaria activities occurred in a phased manner. In Y3, malaria diagnoses with either microscopy or rapid diagnostic tests were increased as per evidence observed in the performance indicators. The overall number of tests among suspected cases increased from 19,723 in Y1 to 90,789 in Y4.12 In contrast, the DOT progress over the period of the project varied overtime as the positive cases were decreasing and the DOT conducted on confirmed cases was being influenced by the movement and mobility of the patients thus the percentage showed a slight downward curve. (86 percent, 74 percent, and 87 percent for Y1, Y2, and Y3, respectively) with variations in reporting completeness among ODs by Y3 (SPL=98 percent; BTM=87 percent; MRS=82 percent; PLN=87 percent, and TMK=97 percent).13 VMWs and MMWs were taught by staff from CMEP, PHD, OD, and HF how to improve malaria control and elimination services in their catchment areas including by promoting healthy lifestyles. The conducted pilot is on implementing PQ radical cure 14 days treatment with G6PD testing at selected HFs in two provinces of CMEP, and the activities to scale up nationally are planned from October 2020; 14 this affects the radical cure of vivax malaria that includes primaquine. EQ3: Does the current national malaria surveillance system serve its intended purpose to detect, immediately notify, investigate, respond, and support appropriately for malaria elimination and control activities? CMEP developed a surveillance strategy to support the national MIS, including a list of protocols, guidelines, and standards at the national and regional level that were supported, to various extents, by the project. Staff from HFs, VMWs, MMWs, IPs, and others confirmed that CMEP’s 10 Implemented by CMEP in Battambang and Pailin provinces since November 2019. 11 Qualitative POC G6PD d is being done only at the HF level (only in selected HFs). 12 MIS data available until May 2020. 13 There were no data reported from BKN and SPM in the CMEP PITT. 14 By 2021, Pv radical cure with quantitative testing at selected HFs will scale up across the whole country. v involvement in surveillance has significantly improved timeliness of the processes (times for detecting, diagnosis, providing treatment, notification, and response) and, more importantly, data quality. CNM representatives at the central and provincial level specifically cited project contributions in data quality improvement, timeliness, and data use for decision-making, 15 For example, data quality improvement was incorporated into the M&E workshops and strengthened the capacities of HF, VMW, MMW, OD, and PDH staff. Those trainings and monthly meetings resulted in better recording and reporting of malaria data by nearly all ODs (all ODs met their targets except MRS). Data procedures have been streamlined and led by the CNM, and the project’s inputs have significantly supported local teams at the OD level to capture the national malaria indicators. Most of CMEP’s performance indicators are sourced from the MIS. Approval and dissemination of this data is a sensitive issue that requires clearance from CNM leadership, which was not obtained for this evaluation. The ET considers that data sharing limitations affects the transparency of the current real malaria strata (malaria stratum by indigenous cases), thus, having significant implications to guide future investment and approaches (control vs. elimination vs. prevention of reintroduction). One hundred (100) percent of notification units are reporting on a monthly basis to the MIS in CMEP ODs,16 and stakeholder meetings are routinely conducted to analyze and discuss progress and bottlenecks in the malaria response. The malaria epidemiology is changing, with mobile migrant populations (MMPs) migrating between ODs and provinces presenting a huge and ongoing challenge for malaria control in Cambodia. Data sharing among provinces is limited, with annual malaria review meetings conducted only at the central level. National stakeholders told the ET that the malaria stakeholder working groups meet monthly and annually to present and discuss all malaria-related activities and work plans; however, no representatives of malaria￾affected communities seem to be attending those annual meetings. The effected communities attend meetings and participate in malaria activities. The routine malaria meetings are held at the PHD, OD, and HF level only (monthly, quarterly). EQ4: To what extent has the CMEP project built capacity of the CNM/MOH to manage, intensify, and sustain malaria control and elimination efforts, particularly at the PHD and OD levels? CMEP has integrated capacity building activities within the structure at CNM, PHD, and OD levels. CMEP’s capacity building approach addresses quality improvement (QI) capacity building and supportive supervision combined with implementation of interventions targeted and delivered to key and vulnerable populations for malaria (including forest goers and MMPs). CMEP has developed national capacities to assess, plan, train, provide supportive supervision, and monitor programs at the local level. Coverage of quality malaria services was low for some indicators at baseline and access to improved services for most of the at-risk population is far from universal. All ODs (100 percent target achieved in Y1) developed AOPs and budgets for their malaria activities with support from CMEP. The number of health care workers (HCWs) 15 For example: 1) Poor quality submitted deliverables containing notable issues, including insufficient content, deficiencies in required technical details and analytical rigor; data quality problems; and weak presentation, structure, and writing errors; 2) Development of the Y1 work plan took nearly three months to complete due to back and forth discussions; 3) The submitted Environmental Monitoring and Mitigation Plan (EMMP) and Pesticide Evaluation Report and Safe Use Action Plan (PERSUAP) documents were very poor in terms of quality of work and understanding of the requirements. 16 Ninety-nine (99) percent reporting rate nationwide of the MIS assessed by mid-2019. vi trained in case management (with artemisinin-based combination therapy [ACT]) increased from 531, 390, and 773 for Y1, Y2, and Y3 respectively (cumulative 1,694 trained HCWs). Similarly, the number of HCWs trained in malaria laboratory diagnosis with USG funds increased from a baseline of 464 to a total of 1,725 by Y3 (562, 390, and 773 in Y1, Y2, and Y3, respectively). Furthermore, the percentage of planned supervisory visits conducted by national, PHD, and OD staff at HFs, VMW/MMWs, and PPs in targeted ODs had slow progress in Y1 and Y2 (58 percent and 68 percent, respectively) and surpassed its targets in Y3 (130 percent achieved). In this epidemiological setting in Cambodia, malaria epidemics are common and may require a rapid response with trained personnel at the local level. The availability of trained personnel and local capacities is important for the epidemic response and this includes VMWs, MMWs, HFs, civil society organizations (CSOs), and PPs. Maintaining a cadre of skilled personnel at the local level remains a challenge due to the high turnover of human resources at the OD level. In this epidemiological context, these personnel play key roles in strengthening the responses for malaria elimination and prevention reintroduction. The RGC is at the early stages of planning decentralization of health services, however, there is still a lack of clear information about roles and responsibilities, process, or timelines for putting this governance structure in place. EQ5: What factors, both internal and external of CMEP, have facilitated or constrained project performance and how can these factors be addressed? KII, FGD, mini-survey, and desk review results show there are a number of factors that affect CMEP’s performance. Factors that enhance the performance of the project but are outside its control include the high level political will17 at MOH and CNM and the cooperation between malaria stakeholders and government entities, both of which are acknowledged as important enablers in the work CMEP is doing. Additional factors enhancing CMEP’s impact include the use of the RGC’s structures, procurement, and supply systems, and the coordination of routine meetings (monthly and annual reviews). Performance factors under CMEP’s control include the new key personnel; high number of skilled staff; good working relationship with VMWs, MMWs, OD, PHD, and CNM staff; and performance of subcontractors. Certain inhibiting factors of CMEP progress were both outside and within the project’s control. Within CMEP’s control there were important issues in Y1 and Q1 of Y2 (15 months) (poor quality of reports, delays of deliverables, low expenditure rate of funds, management issues, the lack of a more proactive advocate approach towards the vivax malaria agenda with CNM, and the lack of a highly technical malaria expert as part of the project team). These negatively affected performance for the first fifteen months of the project. Other inhibiting factors that were outside CMEP control are related to indicator definitions (i.e., API, outbreaks), the delayed inclusion of vivax approaches, high human resources (HR) turnover, low salaries, lack of a diversified job market, lack of motivation of local staff, high internal migration, cross-border movements, delayed and limited implementation of new vivax malaria approaches. 18 long bureaucratic delays at CNM, and issues managing CNM’s short notice requests were also among the inhibitors of CMEP’s performance. All respondents—USG, IP, VMW, MMW, PHD, and OD—reported limited impact of COVID-19 on CMEP activities, with the most mentioned consequence being delayed meetings. 17 Political will is demonstrated by the commitment towards the malaria elimination agenda and support of the Minister of Health visiting CMEP’s project, although there is no significant funding earmarked to support this agenda further. 18 Qualitative G6PD d test have been piloted among vivax cases in males only in selected HFs. There are plans to implement G6PD d quantitative POC test in 2021. vii Despite the availability of a CMEP sustainability plan and exit strategy, discussions with CNM and malaria stakeholders about these have not yet occurred. 19 With regard to financial sustainability, domestic funding from the RGC averages $4 million per year between 2021-2025, and the country is heavily dependent on external funding. There is no new funding earmarked to cover all the gaps in the Malaria Elimination Action Framework (MEAF) 2021-2025 proposed budget. CONCLUSIONS Overall, CMEP has implemented the malaria activities tied to its Results Framework (RF). The evidence-based model for malaria elimination in SPL has been effective in reducing 66 percent of the overall malaria cases (reduction of Pf, Pv, and Mixed of 97 percent, 29 percent, and 94 percent, respectively). Expansion of the elimination model was done in a phased manner and nine ODs have moved into an elimination status as per the CNM strategy and MEAF 1. National stakeholders consider that CMEP has been crucial in this elimination process, supporting local and national-level staff as well as other IPs. The inclusion of high-quality interventions and additional HF and community approaches, along with strengthening access to case management and prevention with vector control, had mutually reinforcing effects. All respondents stated that CMEP’s support has been pivotal in the strengthening of the malaria elimination in Western Cambodia, particularly MDR falciparum malaria, a global threat. CMEP’s results and achievements are noticed by the stakeholders and, in many ODs, the malaria response is improved as a result of the project’s interventions. More work remains to be done in the areas of vivax control and elimination and treatment with radical cure and management of vivax outbreaks. There is a need for a more detailed stratification and harmonization of indicators with global guidance as well as a tailored approach for imported and indigenous cases using surveillance as the core intervention. Based on the malaria funding request for the GFATM, the RGC has plans for decentralization of the malaria program activities to the OD level; however, this process is in early stages. CMEP developed national capacities to assess, plan, train, provide supportive supervision, and monitor programs at the CNM and PHD/OD levels. Coverage of quality malaria services was low for some indicators at baseline20 and access to improved services for most of the at-risk population is far from universal. Further progress will require more granular epidemiological analysis of the indigenous cases, identifying the risk factors and estimating the population of mobile and migrant workers at the OD level. Without this key information CNM and CMEP will not be able to confirm malaria elimination of indigenous cases, as per World Health Organization (WHO) policy. The internal migration dynamics need to be better understood to scale up several health systems strengthening (HSS) components (i.e., readiness of health services, health workforce and procurement supply management, financing, and leadership/governance) that were not CMEP’s focus. There is enough epidemiological malaria information to identify and assess proxies of population mobility using the CNM malaria database on case classification. RECOMMENDATIONS The ET presents five overarching recommendations based on the evaluation findings. The first three recommendations are particularly crucial to maintain gains in malaria control and 19 In confirmation with CMEP COP, the sustainability and exit plan was approved by USAID in the third quarter (Q3) of Y2. 20 Based on the PITT. viii elimination and to guide future investments. For each overarching recommendation, the ET has specified high-, medium-, and low-priority actions. 1. Expand and strengthen the implementation of the 1-3-7 evidence-based elimination model. High Priority • Continue reaching at-risk communities where they are by increased mobilization of VMWs, MMWs, and CSOs supporting access to malaria interventions using tailored and evidence-based mixed approaches. • Focus on CMEP core activities, capacity building, surveillance, and supervision rather than entomologic surveillance activities. • All malaria partners should consider the inclusion of high-level technical malaria expertise on malaria elimination and prevention of reintroduction in the next malaria investments in Cambodia. Medium Priority • Continue supporting CSOs through the grants program to encourage locally-driven innovation to improve service provision to MMPs. Low Priority • Continue to support country ownership working in collaboration with CNM and other malaria stakeholders in the implementation of technically sound interventions and risk-mitigation strategies. 2. Increase data analysis and information use for strategic decision-making. High Priority • Conduct an impact evaluation of the evidence-based malaria elimination model and case studies on each of the interventions implemented under CMEP to inform scale-up. • Provide technical support in conducting a cost-effectiveness assessment of malaria interventions for vivax malaria transmission only, for example reassess LLINs as a strategy in areas with outdoor biting, use of targeted mass drug administration, etc. Medium Priority • Showcase CMEP achievements within USAID Headquarters (HQ) and participate in webinars and talks with other stakeholders within the USG. CMEP should also make recommendations to stakeholders about what to carry forward in the next phases. • Increase dissemination of CMEP achievements in international meetings/events and scientific conferences in the United States. Low Priority • Participate actively in annual scientific conferences venues in South East Asia (i.e., Tropical Medicine conferences). 3. Review and harmonize malaria indicators and update the malaria profile. High Priority • Conduct a rapid assessment of targeted OD profiles based on indigenous malaria cases only and identify malaria-free areas using MIS data. • Report on the API indicator disaggregating by using indigenous cases only as an alternative reporting in the CMEP M&E Plan or use the number of indigenous malaria cases as recommended by PMI. Medium Priority • Continue supporting the MEAF 2 M&E framework in collaboration with malaria stakeholders, including the development of new tools, standard operating procedures (SOPs), and technical strategies. Low Priority • Use information on malaria-free areas per targeted ODs as an advocacy tool in conjunction with all malaria partners and the RGC. ix 4. Strengthen the early detection and prompt response to upsurge of malaria cases. High Priority • Provide technical support to differentiate the malaria profile among MMPs. • Conduct a rapid assessment of the main biological, social, and behavioral risk factors associated with malaria among MMPs, including identification of barriers for service provision. • Conduct a cost-effectiveness assessment of vivax interventions proposed in the MEAF including, for example, the use of targeted mass drug administration. Medium Priority • Continue providing technical support to PHD, OD, and HF staff and to VMWs, MMWs, PPs, and CSOs to fully manage any upsurge of malaria before transition following national normative guidance. Low Priority • Provide technical and financial support maintaining an stockpile of antimalarial drugs, rapid diagnostic tests (RDTs), LLINs, and logistic field support. CMEP should continue supporting commodity forecasting, procurement, and distribution of commodities in a way that ensures extra (contingency) supplies at certain levels (national, OD, or HF) to avoid stock outs. 5. Support sustainable approaches for service delivery and system strengthening. High Priority • Share and reassess (if needed) the CMEP transition and exit strategy with CNM and malaria stakeholders. • Consider supporting sustainable country ownership of health systems reforms by assessing readiness for malaria elimination at the OD level in preparation for transition. • Continue to support the network of VMWs, MMWs, and PPs for malaria and assess their potential contribution to other health-related issues in the planning stages of decentralization. • Provide technical support to the RGC to explore novel alternatives for resource mobilization at the local and central level, including multisectoral private and public organizations. Medium Priority • Reassess annual needs (HR) and coverage of VMWs/MMWs in hard-to-reach areas. • Continue support and engagement with CSOs working with hard-to-reach populations. Low Priority • Support HCW and service providers’ accreditation processes and continuing health education credits in the health system. • Continue communicating the malaria prevention of transmission and/or reintroduction (for malaria￾free areas) mindset to communities, services providers and policymaking managers using different advocacy and SBCC tools. 1 1.0 EVALUATION PURPOSE AND QUESTIONS 1.1 EVALUATION PURPOSE The purpose of the Cambodia Malaria Elimination Project (CMEP) mid-term performance evaluation is to: 1) assess activity implementation and performance to date, identify key bottlenecks and constraints, and make actionable recommendations for improvements needed to meet the activity’s intended objective; and 2) identify successes of the activity, areas for improvement, and any course corrections required for the remainder of the project. The evaluation is intended to provide an independent view of how the project has been implemented from a wide variety of perspectives, especially the extent that coverage with evidence-based malaria control and elimination interventions has been achieved. The evaluation addresses the activity’s flexibility and effectiveness in adjusting to any changes that took place in its operational areas within the Royal Government of Cambodia (RGC) or within implementing partners (IPs) during the life of the activity. Results from this evaluation are intended to be used by the United States Agency for International Development Cambodia (USAID/Cambodia) Office of Public Health and Education and the U.S. President’s Malaria Initiative (PMI) headquarters (HQ) team, as well as the malaria community within Cambodia and across the region. Evaluation results will further be shared with local stakeholders, such as CMEP IPs, the Cambodia Ministry of Health (MOH), other organizations working on malaria in Mekong subregion, the World Health Organization (WHO), the Global Fund to Fight AIDS, Tuberculosis, and Malaria (GFATM), the Asian Development Bank (ADB), the United Nations Office for Project Services (UNOPS), and other donor or advocacy organizations. 1.2 EVALUATION QUESTIONS The CMEP mid-term performance evaluation seeks to answer the following five core evaluation questions (EQs): EQ1: To what extent has the CMEP project developed a scalable, evidence-based elimination model in Sampov Loun (SPL) and supported its widespread dissemination and replication in other target Operational Districts (ODs) in Cambodia? EQ2: To what extent has the CMEP project supported high-quality malaria control and prevention interventions in five ODs and expanded transitional ODs, where gaps in coverage or quality may exist? Please consider the extent to which and how the activity has achieved and maintained universal coverage of at-risk populations with proven vector control and case management interventions. EQ3: Does the current national malaria surveillance system serve its intended purpose to detect, immediately notify, investigate, respond, and support appropriately for malaria elimination and control activities? Please elaborate the extent of CMEP involvement and any gaps required to improve the current system, if any. 2 EQ4: To what extent has the CMEP project built capacity of the Cambodian National Center for Malaria Control, Parasitology, and Entomology (CNM)/MOH to manage, intensify, and sustain malaria control and elimination efforts, particularly at the Provincial Health Department (PHD) and OD levels? EQ5: What factors, both internal and external of CMEP, have facilitated or constrained project performance and how can these factors be addressed? a. Measures that have been put in place to contribute to or promote sustainability. Factors that still need to be addressed. b. The efficiency and effectiveness of the design and project management arrangements and oversight between the prime recipient and sub-partners (and sub-grantees) for achieving project objectives. 2.0 PROJECT BACKGROUND Cambodia is moving towards achieving malaria elimination by 2025. The country has had a significant decline in malaria cases which coincides with the introduction of a mix of interventions: artemisinin-based combination therapies (ACT) in 2000, increased ownership of long-lasting insecticide-treated nets (LLINs), and improved case management (diagnosis and treatment) through the network of village malaria workers (VMWs) established in 2004. Overall, Cambodia has a low malaria transmission with few exceptions in communities surrounding the forested areas, such as the Krakor (KRK) and Phnom Kravanh (PKV) ODs in Pursat province. Most confirmed malaria cases are reported among males (15 to 49 years of age), who live and/or work in forested areas or on private farms near the forest. Significant movement of populations also contributes to malaria transmission. Historically treatment for Plasmodium falciparum (Pf) malaria has been complicated by the emergence and spread of antimalarial drug resistance. First-line treatment for Pf has included artemether-lumefantrine, dihydroartemisinin-piperaquine (DHA-P), and most recently artesunate-mefloquine (AS-MQ) over the last 10 years. Although AS-MQ appears to remain an effective treatment, multidrug resistant (MDR) Pf remains a global threat and it requires investments and faster implementation of elimination interventions adapted to the local conditions. Malaria due to Plasmodium vivax (Pv) is also prevalent in Cambodia, and it is more complex and challenging in terms of control and elimination. CMEP is a five-year award intended to intensify malaria control and elimination activities by providing technical assistance and support to the CNM to further develop, refine, and evaluate an evidence-based Model Elimination Package (MEP) for malaria, which can then be implemented in other appropriate ODs. In Year 1 (Y1), the project was implemented in six ODs: SPL, Battambang (BTB), Maung Russey (MRS), Thamar Kaul (TMK), PKV, and KRK. Starting in Quarter 3 (Q3) (May 2018) of Y2, the project expanded to three additional ODs (total of nine ODs), including Pailin (PLN), Bakan (BKN), and Sampov Meas (SPM). Three transitional ODs graduated to elimination ODs (BTB, MRS, and TMK) with a total of seven out of nine ODs that reached elimination status by 2020 (PKV and KRK ODs were still transitional ODs at the time 3 of this evaluation). CMEP is currently being implemented in nine ODs in three provinces (Pursat, Battambang, and Pailin).21 CMEP is implemented by University Research Co., LLC (URC) and local civil society organizations (CSOs) in partnership with the MOH for a total commitment of $16.5 million. The period of performance is October 26, 2016 to October 25, 2021. CMEP uses a health OD-centered approach to build capacity for ongoing malaria control and elimination. The CMEP team actively supports the development, implementation, and monitoring of OD annual operational plans (AOPs) with joint supervision at all levels to strengthen both the quality of care and routine reporting. CMEP works with community networks to improve access to and quality of early diagnosis and treatment (EDAT), provides support to strengthening systems for passive and active surveillance, supports distribution of LLINs, and implements a range of social and behavior change communication (SBCC) activities to stimulate positive malaria preventive and treatment seeking behaviors for at-risk populations. CMEP aimed to accomplish this goal through four intermediate results: 1) an evidence-based malaria elimination model in SPL developed and replicated in other target ODs; 2) high-quality malaria control and prevention interventions supported; 3) national malaria surveillance systems and monitoring and evaluation (M&E) appropriate for malaria elimination and control activities strengthened; and 4) capacities of MOH to manage, intensify, and sustain malaria control and elimination efforts built at the OD level. In order to achieve these objectives, CMEP is following a strategic approach to control and eliminate malaria with seven components: • Component I: Elimination in the context of decentralization: building OD/PHD capacity – seeks to strengthen the capacities of the local health staff by working with CNM to provide training and technical assistance to PHDs and ODs in a recently evolved malaria program (from a vertical to decentralized program). • Component II: Malaria surveillance in real-time reporting and monitoring linked with geospatial analysis for improving targeting – seeks to increase rapid notification of individual cases, geo-referencing cases’ foci investigation, and prompt response in target elimination ODs. • Component III: Approach to targeting of intervention – focuses on effectively targeting interventions to those who need them, including for example LLINs, behavior change communication (BCC), and specific interventions for forest goers. CMEP sub-contracts to local non-governmental organizations (NGOs) to promote malaria education and outreach campaigns in remote areas where health facilities and CMEP staff have limited reach. • Component IV: High-quality diagnosis – seeks to continue using rapid diagnostic tests (RDTs) and operationally assess the role of the new high-sensitive RDTs (operational research, OR). 21 Since November 2019, CMEP has been requested by CNM to provide technical assistance to new areas, outside CMEP’s ODs. 4 • Component V: Identify hotspots and intensify outreach activities to target high-risk populations including mobile and migrant populations (MMPs) and forest goers – seeks to continue identifying more workplaces for MMPs in forest areas and plantations. • Component VI: Entomological surveillance for malaria elimination – seeks to continue implementing entomological surveillance activities in targeted areas as per national guidelines. • Component VII: Use of prompt and efficacious treatment for all malaria cases – focuses on ensuring timely efficacious treatment of all confirmed uncomplicated and severe malaria cases according to national malaria treatment guidelines (AS-MQ), including private providers (PPs) per current national policy. This component also focuses on the utilization of low dose Primaquine to reduce the transmissibility of Pf and radical cure for Pv. Through the above efforts, CMEP helps Cambodia to build constructive dialogue within Cambodia and the Mekong region on malaria elimination and expanding knowledge and opportunities for key malaria epidemiologists and health professionals, students, researchers, policymakers, and end users to network and share information using educational events as a platform. Finally, the institutionalization of evidence-based activities and services was manifested through the project support and advocacy for policy reforms on malaria control and elimination—and health—to better facilitate a conducive environment for project best practices. 3.0 EVALUATION METHODOLOGY The performance evaluation was conducted by ME&A in collaboration with ICF, as subcontractor. The team consisted of one international Team Leader (ME&A) and two local experts (one from ME&A and the other from ICF). This team was supported by five experts with expertise in M&E, public health, malaria, and data management from both ME&A’s and ICF’s Home Offices. Due to the COVID-19 pandemic, the Team Leader could not travel to Cambodia and the evaluation team (ET) adjusted its workplan and methodology22 in order to allow for remote data collection and monitoring. The ET ensured the successful implementation of the evaluation despite significant limitations.23 The ET used a mixed-methods evaluation design, employing a combination of qualitative and quantitative methods. Qualitative methods included desk review of key project documents; review of secondary sources of information and analysis, including documents published by the RGC and other donors, and malaria technical documents; and key informant interviews (KIIs) and focus group discussions (FGDs) with health providers, beneficiaries, community members, and other activity stakeholders. A list of key documents consulted can be found in Annex 4. Quantitative methods included implementation of a mini survey and reviewing, analyzing, and visualizing existing CMEP quantitative data. Participation in the KIIs, FGDs, and survey was voluntary, and the ET received verbal informed consent prior to proceeding with the discussions (see data collection tools in Annex 6). 22 This was the first partially remote evaluation conducted by ME&A and USAID/Cambodia. 23 Including all preventives measures as recommended by USAID Cambodia, WHO, and Cambodia MOH. 5 3.1 SAMPLING The criteria for selecting ODs for the evaluation included areas with more than one year of CMEP project implementation. By 2020, seven of the nine targeted ODs have moved to elimination (two ODs remain as transitional ODs) and the selected ODs for field visit were SPL, BTB, PLN, PKV, KRK, and SPM. Those selected ODs have different implementation timelines; however, they all complied with the minimum criteria of more than one year of CMEP project implementation. In cooperation with USAID/Cambodia and the CMEP team, the proposed list of KIIs and FGDs was finalized, considering potential replacements, if needed. 3.2 DATA COLLECTION METHODS24 3.2.1 Desk Review The desk review started with a review of project documents provided by USAID and secondary data from documents pertaining to: relevant aspects of the activity and malaria control and elimination in Cambodia; the latest global health and malaria materials on evidence-based elimination; and reports from other malaria projects in Cambodia, the Mekong subregion, and the world. The review included evaluation reports from CMEP’s predecessor, the Control and Prevention (CAP) Malaria project as well as the regional Resistance Artemisinin Initiative (RAI) grant from GFATM. A list of documents reviewed is presented in Annex 4. 3.2.2 Key Informant Interviews KII respondents were selected using purposive sampling. They were administered to five groups with experience on malaria in Cambodia and/or knowledge on CMEP activities: staff of the United States Government (USG) agencies and initiatives (USAID, PMI, Centers for Disease Control and Prevention [CDC]), staff at supported health facilities (HFs) and PPs, officials at the CNM, PHD, and OD levels, IPs, and others as well as CMEP team members. Overall, 123 KIIs were conducted. Table 1: Sample of KIIs by Respondent Category Respondent Category Total Number of Interviews Female Male USG: USAID, PMI, CDC in Cambodia and Washington, DC metro area 6 3 3 CMEP: URC Cambodia and Washington, DC metro area 11 3 8 IPs & Others: WHO, Population Service International (PSI), Clinton Health Access Initiative (CHAI), Donor-Funded Projects, Academia, Research Centers, NGOs, and Independent Experts 8 3 5 PHD/OD Staff (selected ODs), including CNM staff 40 7 33 HFs’ Staff and PPs (selected ODs) 58 13 45 Total 123 29 94 24 The field visits to carry out KIIs, FGDs, and mini surveys in the selected ODs complied with USAID, WHO, and Cambodia guidelines for prevention of COVID-19. Social distancing – 6 feet apart, regular handwashing, meetings with less than 10 people, and mask use were implemented for the duration of fieldwork. All recommendations were followed by the local ET and participants. 6 The purpose of the KIIs was for the ET to gain insights into informants’ experiences with CMEP, their perceptions of the activity, how the malaria interventions are being implemented, CMEP’s relevance to the national elimination response, the effectiveness of the project’s approaches, and the likelihood that changes supported by the project will be sustained. In six ODs and at the central level in Phnom Penh, the ET conducted face-to-face and remote KIIs, including relevant PHD and OD government stakeholders. Interviews were conducted with the support of project’s local personnel and used structured data collection questionnaires. Two local ET members translated the data collection tools into Khmer, an additional ET member not involved in the tools’ development, translated selected tools back—Khmer into English. This was done as part of quality control to ensure that the meaning of the questions was maintained. All local ET members were native Khmer speakers; pre-testing of the tools was conducted before field work in Rolead and Pursat and final modifications were conducted accordingly. See Annex 5 for a list of individuals and/or officers interviewed. 3.2.3 Focus Group Discussions The FGDs were conducted with two groups: VMWs/mobile malaria workers (MMWs) and malaria patients/at-risk populations. Participants were selected using purposive sampling. The number of FGDs in selected ODs is shown in Table 2. Overall, 14 FGDs were conducted instead of the originally proposed 12 FGDs. Table 2: Focus Group Discussions by Operational Districts ODs VMWs/MMWs Malaria Patients Total FGDs Female Male SPL 1 1 2 5 3 BTB 2 1 3 4 6 PLN 2 1 3 4 8 PKV 1 3 4 6 15 KRK 1 1 2 2 9 Total 7 7 14 21 41 All FGDs were conducted in person and each of them took approximately 60 minutes. Due to seasonal agricultural activity and the beginning of the rainy season, few malaria patients were available for interviews; however, the ET was able to interview at-risk populations to compensate for this. Topics covered during FGDs were directly related to CMEP’s efforts in eliminating malaria. 3.2.4 Mini Survey The qualitative component of this evaluation was complemented by a mini survey with 203 VMWs/MMWs and malaria patients/at-risk populations that, using purposive sampling, sought to gain insights into the project’s results and outcomes from a diverse group of beneficiaries and stakeholders. Data collection tools were translated into Khmer, reviewed by native speakers on the ET, and pre-tested in project ODs not selected in the evaluation sample. The survey was programmed into Kobo Toolbox (https://www.kobotoolbox.org/) and administered to respondents using tablets. Data were uploaded to the cloud at the time of collection and monitored in real-time, remotely. In addition to collecting information on types of training and attitudes on the usefulness and quality of training (“yes” or “no” questions in the survey), the ET 7 also included the use of Likert-style numeric codes in order to judge the intensity of the participants’ opinions. Participation in the surveys was voluntary and responses were anonymous to encourage frank and open answers. 3.2.5 Quantitative CMEP Project Data CMEP’s 25 performance indicators were assessed and compared against baseline. Data on suspected malaria, confirmed malaria, parasite species, and treatment were analyzed in all nine ODs. 3.3 DATA ANALYSIS The ET conducted analysis of the quantitative and qualitative data gathered to answer the five EQs. 3.3.1 Qualitative Data Data collected were systematically summarized to draw out trends and key findings and analyzed by identifying thematic areas that emerged from the aggregated responses from individuals and groups to common questions asked to each kind of informant. Where outliers were found, they were identified separately in the evaluation results. The ET triangulated these key findings from the KIIs and FGDs with results from the quantitative survey to identify areas of agreement and any differences. 3.3.2 Quantitative Data The performance indicators and the mini-survey data were analyzed using Excel and IBM SPSS Statistics software to generate frequency and summary statistics, including measures of central tendency and contingency tables and charts/graphs showing how different respondent groups responded to different survey questions and whether these differences are significant. 3.4 EVALUATION LIMITATIONS The evaluation had several limitations, including: • Finding malaria patients at HFs and settlements/villages was difficult due to a significant reduction of malaria at the selected ODs. • There were logistic challenges as the evaluation field work coincided with the beginning of the rainy season. There were many people moving to the plantations and not many visiting the HFs at the time of the ET’s visits to the facilities. • Difficulties in obtaining permission letters from the CNM to conduct the data collection on time initially delayed the field work. • Data sharing is still a sensitive issue in the CNM, limiting the quantitative data available for more in-depth analyses and for more targeting of the investments. • Some of the major international partners (UNOPS, GFATM) and some IPs were unavailable. • Due to the current COVID-19 pandemic, the Team Leader was unable to travel to Cambodia. The ET mitigated these limitations using mixed approaches, for example HFs and settlements with low or no malaria cases were replaced by the nearest HF and OD with malaria cases 8 diagnosed. This was particularly important for the FGDs. To maximize field work, the ET coordinated, scheduled, and started activities (KIIs, FGDs, survey) very early in the morning. Due to the delays in getting the official letter, the ET worked with USAID to facilitate the communication with CNM and this caused a one-week delay in the workplan. However, the ET was able and ready to start activities the next day. Due to COVID-19, malaria partners and the ET acknowledged the need to reinforce the prevention measures and all teams followed the RGC, WHO, and USAID guidance for prevention of COVID-19. 4.0 FINDINGS BY EVALUATION QUESTION (Note: this report focuses on findings that most accurately respond to the EQs and should not be considered a comprehensive list of all CMEP activities in the project Results Framework (RF). These can be found in detail in the CMEP quarterly and annual reports submitted to USAID. Findings are listed by priority to USAID). 4.1 EQ1: TO WHAT EXTENT HAS THE CMEP PROJECT DEVELOPED A SCALABLE, EVIDENCE-BASED ELIMINATION MODEL IN SPL AND SUPPORTED ITS WIDESPREAD DISSEMINATION AND REPLICATION IN OTHER TARGET ODS IN CAMBODIA? Cambodia has a heterogenous malaria transmission across different ecological zones with permanent and mobile population groups living or traveling around the forest fringe as the main key and vulnerable populations. The highest transmission occurs in areas in the north and north east bordering Vietnam and Lao People’s Democratic Republic (Lao PDR), and in the west of the country bordering Thailand. Malaria cases are perennial and peak during the rainy season which lasts from June to November. Based on an updated malaria profile in 2020 in the Malaria Elimination Action Framework (MEAF) 2021-2025, 10 ODs in seven provinces were responsible for 80 percent of the malaria cases in 2019. Ninety (90) percent of those confirmed cases were estimated to be related to forested areas as well as MMPs (occupational risk). Access to health services in those areas is limited and illegal activities along border areas are common. Data from the CNM Malaria Information System (MIS) and documents obtained from CNM, CMEP, and PMI showed that Pf and Pv persist in the country. The ratio between Pf and Pv changed in 2017, and vivax malaria is the predominant parasite in the malaria transmission since 2018. By the end of 2019, vivax malaria was responsible for 84 percent of the overall malaria cases (26,981/31,971). 9 In Cambodia, the malaria epidemiology and its response have been evolving over the years. In the early MEAF 2016-2020,25 the CNM stratified ODs into four sub-categories: elimination-targeted, burden reduction, transitional, and malaria-free (Box 1). A further review in the newly developed MEAF 2021-2025 used a different stratification method. For example, in this revised MEAF, 6,422 villages in endemic ODs were stratified into six categories (no risk, low, medium, risk, high, and highest) using two covariates: village-level annual parasite incidence (API) 26 for 2018 and 2019, and the percentage of forest cover within a three-kilometer radius of the center of each village. In Cambodia, malaria predominantly affects males 15-49 years of age, with API in adult males five times higher than adult females. 4.1.1 Evidence-Based Malaria Elimination Model Based on CNM and PMI reports, CMEP began implementing a basic essential package of activities developed for malaria elimination (also known as the MEP) as a pilot in 2015. The MEP includes EDAT; implementation of individual case reporting, case and foci investigation,27 response, and distribution of LLINs;28 and interpersonal communication (IPC). This MEP has been known as the “1-3-7” approach which is based on the successful work of the similar approach implemented in China but adapted to the Cambodian context. The CMEP 1-3-7 approach refers to “all malaria cases detected should be reported to a health center within one day using the SMS Alert System for real￾time reporting, a case investigation should be conducted (including determination of whether the case was locally-acquired or imported) within three days, and case finding and response activities should be implemented within seven days for all cases.” Additional to the 1-3-7 approach, the MEP includes a 28-day Pf treatment follow-up.29 The malaria response activities include screening of all household members of each confirmed case (and co-travelers) and suspected cases tested among neighbors (around 10 households) alongside providing health education and top-up of insecticide-treated nets (ITNs). The evidence-based malaria elimination model was first introduced in SPL OD between July 2015 and January 2017, during a transition between the Y5 of CAP-Malaria30 (Cambodia) and Y1 of CMEP.31 SPL is an OD in Battambang Province in northwestern Cambodia. By 2015, the health system in SPL had nine HFs (also known as health centers [HCs]), one former district hospital, one referral hospital, 168 VMWs, and 32 PPs. Based on the PMI and CAP-Malaria final reports, between 2015-2016, there were 348 malaria cases reported. Most cases were reported by HFs (224 confirmed cases or 64 percent) and the rest by VMWs (124 or 36 percent). The reported cases by the 1-3-7 approach were: Day 1 (D1)= 293 (85 percent); D3=279 (81 percent), and D7=289 (84 percent). The majority of cases (47 percent) were reported during the December￾February period. Key milestones for the activities in SPL included the integration of the 1-3-7 approach into the CNM MIS database (in January 2016) and the introduction and use of a new ACT AS-MQ in February 2016, as first-line treatment for falciparum malaria. 25 MEAF 2016-2020 is the strategic plan since the start of CMEP. 26 In Cambodia, the API indicator definition uses all malaria cases without distinction (indigenous versus imported) and is different from the recommendation by the WHO. 27 Investigation of passively detected cases. 28 This includes bed LLINs and hammock LLINs (LLIHNs). 29 CAP-Malaria Final Evaluation Report. 30 CAP-Malaria was a regional project including Burma, Cambodia, and Thailand. 31 URC continues with the implementation of the Monitoring, Evaluation, and Learning (MEL) Package in SPL under CMEP since October 2016. 10 Data from the evidence-based elimination model (pre-CMEP) showed a significant impact on malaria transmission. A 33 percent reduction of overall malaria incidence was observed in SPL OD with an API change from 3.26 in 2014-2015 to 2.17 in 2015-2016.32 Data for 1-3-7 activities were recorded in an Excel datasheet until their integration into the CNM MIS database in January 2016. Based on CMEP annual reports and the PMI Malaria Operational Plan (MOP), the last indigenous case in SPL was reported in March 2016, while all confirmed cases were imported. 4.1.2 Implementation of the Evidence-Model Elimination Model in SPL OD Under CMEP CMEP began activities in October 2016 conducting a pre-implementation situation analysis to assess the extent of implementation and the effectiveness of the 1-3-7 approach. Results from this assessment by the CMEP team found “a better case management and malaria control in SPL OD since its inception, with an increased case notification, investigation and response.” While the assessment concluded that “significant reduction in the API and reduced number of Pf and mixed malaria cases over the last two years was attributed to 1-3-7 elimination efforts,” the ET considers, however, that this effect seems to be more related to “contribution” rather than attribution. The CNM current API definition used to measure impact at the OD level included all malaria cases (indigenous and imported) and, as a denominator, the total population at the OD, which is different from the WHO-recommended definition.33 Based on the CNM and CMEP annual reports, other malaria activities were implemented with a focus on MMPs. For instance, LLIN distribution, quality assurance/quality control through regular supportive supervision, malaria education, and strengthening the VMW network were some of the activities implemented in SPL. Since April 2016, no indigenous cases of malaria have been reported in SPL OD (see Figure 1) and two marked “peaks” of imported malaria cases can be seen every year, one in June-July and another in December-January. Those peaks seem to be related to the monsoon and harvesting season; however, the ET did not have access to the epidemiological CNM MIS data to confirm the mobilization of those populations nor their occupations. Figure 1: Number of Malaria Indigenous and Imported Cases in SPL OD 2015-2018 Source: CMEP Annual Report. 32 Higher reduction among falciparum cases. 33 WHO recommends the measure of API as the total number of malaria cases (indigenous) divided by the population at-risk. Population at-risk measured by mid-year annually. 11 During the first year of implementation, CMEP reported 100 percent achievement in most performance indicators (CMEP Performance Indicator Tracking Tool [PITT], Annex 8). For example, the percentage of malaria cases that were successfully notified within 24 hours, investigated within three days, and responded to within seven days, were all reported as 100 percent (101/101). However, only 86 percent (87/101) of confirmed cases received directly observed therapy (DOT) malaria treatment according to national treatment guidelines (NTGs).34 Between October 2016 and May 2020, overall there were 4,900 suspected cases tested with 148 positive cases (3 percent positivity rate) in SPL. Of these, 109 (73.6 percent), 38 (25.7 percent), and one (0.7 percent) were confirmed cases of Pv, Pf, and mixed, respectively. Age and sex disaggregated data for malaria cases in SPL in Y1 showed 94.4 percent among patients aged more than 15 years and 88.4 percent were male. Young adult males are at increased malaria exposure in Cambodia, and other Mekong subregion countries, as the primary demographic group engaged in specific occupations (forest goers, farming) and migration patterns (internal and external migration). Infections among males are also less likely to be promptly diagnosed and treated due to gendered behavior (i.e., late presentation in health services for men compared to women). SPL reached a “malaria-free status” in April 2019, after three years with no indigenous malaria. However, the API used in-country classifies it differently (see challenges and constraints section EQ5). In the KIIs with PHD and OD staff, there was a general consensus about the efficiency and efficacy of the elimination model implemented in SPL OD. Ninety-six (96) percent (22/23) of the PHD and OD managers and officers interviewed considered that the elimination model was successful in SPL. “The CMEP project is very successful based on their achievements in malaria indicators.” – IP KII “Malaria cases have dropped by at least 80 percent in the Krakor OD.” – PHD KII As part of the response package, reactive case detection (RCD) was also implemented in SPL OD. Results from screening during case investigations of 101 index cases indicated that there were no positive malaria cases among 178 index household members and 26 surrounding index households. The evidence-based malaria elimination model is cost-effective. CMEP implements its activities using government structures (PHD, OD, HF staff and offices) and logistic systems (i.e., transport and supply system for malaria commodities), and produces mostly digital products (limited printing). A cost-effective analysis of the SPL elimination model was conducted in 2018 by a third￾party contractor of USAID. An empirical micro-costing analysis, using data reported by CMEP, examined the cost of malaria elimination activities in SPL OD. This study assessed government opportunity costs of implementing the program and projected the incremental cost-effectiveness of additional Pf and mixed (Pf/Pv) malaria cases averted under CMEP implementation compared to a no-CMEP proxy. The reported total cost35 of malaria elimination activities in SPL OD was $883,096, including all activities aimed at eliminating malaria. Also, this study estimated that the cost to maintain the malaria elimination status beyond the period of observation was approximately $110,600 per annum total, or less than $0.64 per resident in the OD if the program 34 Based on CNM MIS and CMEP Performance Indicator Tracking Tool (PITT) data. 35 Note: From July 2015 to March 2018. Did not include malaria prevention and control activities from the CAP￾Malaria Project. 12 was continued with the current IP (URC). The large difference between Pf and Pv case cost effectiveness was driven by the cost of treatment of Pf cases being almost a factor of 10 higher than for cases of Pv.36 That is, applying the same set of interventions to the relatively cheaper-to￾treat Pv cases, means that additional cases of Pv identified are less cost-effective than identifying expensive-to-treat Pf/mixed cases. Those estimations did not account for the additional economic benefits realized from the burden of malaria averted or the positive externalities of the program. CMEP elimination activities in SPL were cost-efficient and conferred additional effectiveness compared to the standard government implementation of malaria control based on estimated budgets in the MEAF 2016-2020. 4.1.3 Dissemination and Replication of the Elimination Model in Other ODs In Y1, the project was implemented in six ODs: SPL, BTB, MRS, TMK, PKV, and KRK. Beginning in Y2, the project expanded to a total of nine ODs, including PLN, BKN, and SPM. Three transitional ODs graduated to elimination ODs (BTB, MRS, and TMK). Based on the successful results of SPL and situational analyses, the implementation of the 1-3-7 malaria elimination model in SPL OD continued and expanded to BTB, MRS, PLN, and TMK ODs from 2018 onwards (Table 3). With 90 percent of all cases imported from nearby provinces in Cambodia or from Thailand, the recommendations from the SPL’s situational analysis highlighted the crucial role of the VMW network, strengthening malaria case management training for health care workers (HCWs), and the strengthening of the PP program to address MMPs. Furthermore, as part of quality improvement (QI) system, regular and structural supportive supervision, meetings, and workshops were conducted at all points of care (POCs) to improve quality care (see EQ2 for details). By 2020, CMEP has been supporting the implementation of the 1-3-7 malaria elimination model in all targeted elimination ODs37 and completed the required situational analysis exercises for those ODs. Table 3: CMEP Operational Districts Starting Dates and Elimination Timeline Province OD Start of Activities Remarks Battambang SPL October 2016 Elimination activities from the start BTB October 2016 TMK October 2016 Started elimination activities in May 2018 MRS October 2016 Pursat PKV October 2016 Start Pf elimination in April 2020 KRK October 2016 SPM October 2017 Started elimination activities in October 2019 BKN October 2017 Pailin PLN September 2018 Elimination activities from the start Source: CMEP report. “The SPL as a model was very helpful through using the 1-3-7 method and a national reference. The CMEP project helped the whole country as the CMEP team is very knowledgeable and skilled in malaria elimination.” – IP KII 36 Radical cure with primaquine and glucose-6-phosphate-dehydrogenase (G6PD) deficiency testing for vivax malaria were not included. 37 All nine CMEP ODs reached elimination criteria in 2019. 13 “The effectiveness of the CMEP has led to a decrease in the malaria cases, for example, in June 2020 in Sampov Meas OD, there are only seven cases, while before there were hundreds of cases.” – OD staff KII “The SPL elimination model has been a collaborative effort between various technical staff.” – USG KII “For example in Pursat, in high burden areas, they did an excellent job reducing the malaria cases, especially Pf, in such a very challenging area to work.” – IP KII Between Y1-Y4 (until May 2020), there were 432 malaria confirmed cases and 430 treated (99.5 percent) in SPL. The overall number of malaria cases has been decreasing from 148 in Y1 to 23 in Y4. The distribution of parasite species of the malaria cases showed a higher number (and proportion) of Pv infections (361; 83.5 percent), followed by Pf (70; 16 percent), and mixed infections (1; 0.23 percent). Ninety-four (94) percent of all malaria cases were among people more than 15 years of age and 84 percent were males. Figure 2 shows the number of malaria cases by year of CMEP implementation. Figure 2: Malaria Cases Per Operational District and Year of CMEP Implementation Source: CNM MIS. Note: Y4 until May 2020. There is a significant decrease in malaria over the years; however, in Y2, an increase of cases has been observed in several ODs. This unusual upsurge of malaria cases went up further causing an important outbreak that reshaped the malaria transmission in the country. In 2018, malaria stakeholders developed a 12-month intensification plan to address this outbreak38 39 including a 38 CNM used a definition of outbreak based on the estimation of the mean + SD methods recommended by WHO. This method should include only indigenous cases. 39 The goal of the intensification plan phase 1 was to identify where malaria cases were coming from in these areas (forest goer, traveler, etc.) identifying 3-5 villages or sites with extremely high amounts of cases to set up MMWs similar to VMWs in order to test and treat cases. The primary target was the migrant population, particularly those that work in remote areas. 14 more comprehensive and proactive approach. Given Cambodia’s goal of Pf elimination by 2020, a second intensification plan (15-month) was also developed to mitigate the malaria burden among forest goers and hard-to-reach populations. Figures 3 and 4 show the total number of confirmed malaria cases (including all species) among the nine CMEP ODs and the percentage of shares by OD. Figure 3: Total Number of Malaria Cases Among CMEP Operational Districts, 2016-2020 Source: CNM MIS data. Insert shows the number of cases by parasite species among the nine ODs. Figure 4: Percentage Share Contribution of the Malaria Cases Among CMEP Operational Districts, 2016-2020 Source: CNM MIS data. From all interviews with USG, IP, PHD, and OD staff, service providers, and CNM leadership, the ET found that there is a consensus on the successful implementation of the elimination model in SPL and other ODs. However, CNM senior officials highlighted in discussions about sustainability and future funding that there are concerns at CNM about CMEP’s human resources (HR) and financial capacities to effectively scale up the work in the ODs (see inhibitor factors, 15 EQ5). There is a high turnover of HR in Cambodia, especially at the OD level, and CNM senior managers highlighted the need to continue supporting HR due to a high number of staff retiring at CNM. In three instances, PHD and OD staff told the ET during the interviews that learning from SPL OD was “beneficial.” This learning process from peers that have the knowledge and experience of implementing the 1-3-7 elimination was “considered important for the new ODs in a planning stage.” After the success of the 1-3-7 elimination model in SPL, staff involved in malaria control and elimination from other target ODs visited SPL OD to share the lessons learned with their colleagues, particularly in how the work was done and the challenges faced while implementing the model. Based on the responses received during KIIs, PHD and OD staff found that the effectiveness of the elimination model was seen in the full adherence to the 1-3-7 strategy by HCWs and other service providers (PPs) and its impact on the fall of malaria cases. Additionally, 96 percent of interviewees (22/23) noted that there was a shortening time interval between notification, investigation, and response, and this was seen as a good acceptance of this model by health professionals. Other key areas of effectiveness highlighted by the respondents (HF, PP, PHD, and OD) included the speeding up of LLIN distribution, RDTs, screening, and communication and awareness campaigns at the OD, HC, and VMW levels. The CMEP PITT (Annex 8) shows the main indicators per fiscal year and the differential change since baseline40or first and last result reported per indicator. Based on the PITT, the overall percentage of suspected malaria cases receiving a parasitological test was high at baseline (98 percent) and remained at 98 percent in 2019, just slightly lower than the 100 percent target. The lag in meeting the target for this indicator is likely due to low reported testing by PPs in 2019 (5 percent) and slight lags in SPL and BTM ODs, each testing 97 percent in 2019. There are also challenges (i.e., delays in transmitting data) with VMWs mobile reporting in target ODs. There were no differences in testing by sex and age. For more details on malaria testing rates, diagnoses, and treatments by OD per year, see Annexes 10 and 11. Appropriate treatment in the five elimination ODs lagged below targets in 2018 and 2019. The overall percentage of all confirmed cases receiving appropriate malaria treatment under DOT was only 74 percent in 2018 and increased to 87 percent in 2019, compared to a 92 percent target. This lag behind targets was consistent across providers and species with the exception of the two mixed infections recorded in 2019 receiving appropriate treatment with DOT (100 percent). SPL was the only elimination OD that met 2019 annual targets, with 98 percent of confirmed cases receiving appropriate malaria treatment under DOT (Annex 8). Similar trends were seen on the percentage of all diagnosed malaria cases reported to the appropriate responding team within 24 hours and the percentage of all confirmed malaria cases investigated and reported within three days of diagnosis. Targets were met for both indicators in 2017 and then experienced lag in 2018, improving progress in 2019 but not reaching the 100 percent targets. The percentage of all diagnosed malaria cases reported to the appropriate responding team within 24 hours were lowest in 2018 and 2019 in PLN (23 percent and 85 percent, respectively) and MRS (48 percent and 77 percent, respectively) while both SPL and TMK were high performing ODs, both achieving 97 percent in 2019. Similarly, PLN and MRS had lower performance in the percentage of all confirmed malaria cases investigated and reported 40 In case of missing values at baseline or Y3, the percentage of change was calculated taking into account the first and last reported result per indicator. 16 within three days of diagnosis in 2018 and 2019, while both SPL and TMK achieved 100 percent in 2019 (Annex 8). Regarding the percentage of confirmed active foci investigated in which an appropriate response was initiated within seven days, CMEP only has 2017 data for SPL OD and reached 100 percent. CMEP fell short of the annual 100 percent supervision targets for PPs, with 97 percent and 78 percent of registered PPs that received supervision at least twice in the last 12 months in 2018 and 2019, respectively. In 2018, TMK was the only OD to reach the 100 percent target, and in 2019 MRS was significantly behind at only 48 percent followed by PLN at 70 percent (Annex 8). “Since 2018, the malaria cases decreased from thousands of cases to hundreds of cases per month in 2019.” – OD staff KII “CMEP should focus on the core strength, surveillance, and capacity building” and “...ensure expansion with clear plans.” – USG KII “CMEP should plan for transition in advance.” and “There should be clear [Standard Operating Procedures] SOPs, guidance to pass over to other partners, I would say ‘Prepare better for transition.’ This is the time to document the success of western Cambodia, we need to be vigilant, we do not want re-introduction and there is still a lot to do in order to eliminate all malaria cases.” – USG KII All respondents in the two FGDs organized by the ET were asked if they had noticed any change in the malaria status in the community since the project began. One respondent said that there were “less malaria cases, more nets and fewer people getting sick and dying from malaria.” Other respondents also noted that they went for medical care more frequently because HCWs were nicer to them and gave them more useful information and answered their questions. Another respondent mentioned noted that “nowadays no one is dying because they have malaria testing and treatment services in the villages.” Despite the outbreak during 2017-2018, the malaria elimination model was successfully implemented in all nine ODs with different impact results. One of the major achievements, after the replication of the model and the add-on of tailored approaches during the epidemic is the reduction of Pf hot spots. MDR falciparum malaria cases dropped 97 percent based on MIS data (until May 2020). 4.2 EQ2: TO WHAT EXTENT HAS THE CMEP PROJECT SUPPORTED HIGH-QUALITY MALARIA CONTROL AND PREVENTION INTERVENTIONS IN FIVE ODS AND EXPANDED TRANSITIONAL ODS, WHERE GAPS IN COVERAGE OR QUALITY MAY EXIST? PLEASE CONSIDER THE EXTENT TO WHICH AND HOW THE ACTIVITY HAS ACHIEVED AND MAINTAINED UNIVERSAL COVERAGE OF AT-RISK POPULATIONS WITH PROVEN VECTOR CONTROL AND CASE MANAGEMENT INTERVENTIONS. CMEP conducted a comprehensive situational analysis of malaria control operations in five transitional elimination ODs (BTB, TMK, PKV, KRK, and MRS) in April 2017 and transitional elimination ODs (BKN and SPM) in March 2018. These analyses aimed to identify gaps in malaria service provision with detailed description of the access to essential commodities, service delivery infrastructure, HR availability, and governance. Initial challenges identified in most of the ODs included interruption of VMW/MMW activities supported by GFATM, the public-private mix (PPM) program, the lack of educational materials, difficulties in providing malaria commodities among some providers (i.e., PPM), and limited information on LLIN coverage in some areas. The 17 recommendations given from those challenges highlighted the need to strengthen capacities of service providers (VMWs, PPM, HCWs) at all levels to improve quality care through regular and structural supportive supervision, meetings and workshops and informed each AOP and CMEP workplan. 4.2.1 High Quality Malaria Interventions CMEP supports CNM and provincial and OD staff in the improvement of malaria activities in the target ODs, including early diagnosis and treatment, prevention (LLINs, SBCC41) and capacity building interventions (supportive supervision and M&E). Case Management: Diagnosis and Treatment Cambodia pursues universal access and coverage of high quality early diagnosis and effective treatment services delivered through public HFs, trained VMWs at the community level, PPs,42 military/police, and selected border check-points according to the national guidelines. Malaria diagnosis in the country is based on microscopy or RDTs. No data are available to provide the percentage of the population living in targeted, at-risk villages who had fever in the last two weeks and had a diagnostic test for malaria according to national policy (CMEP PITT). Between October 2016 and May 2020, there were 257,836 tests performed in CMEP target areas. CMEP PITT reports an overall 60 percent reduction (comparing Y3 vs. baseline) on the test positivity rate in all ODs, ranging from 40 percent (PKV) to a 100 percent (PLN) reduction. CMEP supported a package of interventions in PKV and KRK ODs, which were promptly initiated after detecting the malaria upsurge (Box 2). During the malaria outbreak (2017- 2018), CMEP conducted foundational work addressing the upsurge43 intensifying case detection, treatment, vector control, and SBCC activities among MMPs. This work served as the basis for the development of the first intensification plan to manage the outbreak and reduce malaria cases in the seven highest burden provinces, led by CNM and stakeholders. The first intensification plan was focused on strengthening program management and coordination and implementing a more aggressive intervention approach to address the most at-risk populations (forest goers and other MMPs). PMI funded the intensification plan activities in Pursat province. After analysis of the 2018 data, CNM and partners agreed that further work was necessary to control the malaria resurgence Box 2: Package of Interventions to Respond to the Malaria Upsurge • Public announcements on malaria prevention • ITN assessment and top ups, where needed • Health education via IPC and distribution of SBCC • Mass screening, testing and treatment • Touchpoint intervention and peer to peer education • MMW service provision in hotspots Source: CMEP Report. 41 Includes IPC and village-based health education campaigns. 42 Until 2018. 43 The terms epidemic, upsurge, and outbreak used in this document have the same meaning, an unusual increased number of malaria cases. 18 and planned further work. The second intensification plan44 was developed for a longer period (15 months, October 2019-December 2020) and is still ongoing. Malaria testing increased 2.5-fold (30,732 tests in 2018 to 78,770 tests in 2019) at HFs and in the community 85 percent of tests were carried out by MMWs and VMWs.45 The first intensification plan resulted in almost 75 percent reduction of Pf and mixed (Pf+Pv) cases nationwide (compared to the same period 2018). In CMEP ODs, the total number of monthly malaria cases has been decreasing, from 1,372 in October 2018) to 104 in May 2020 (cumulative 11,713 cases, all species).46 CNM senior leadership and USG officials confirmed that CMEP works effectively, in collaboration with local teams and other sectors (military, forestry), to strengthen the malaria response. In KIIs with PHD managers, the ET was told that malaria cases had increased in a few ODs; 22 out 23 PHD managers said that CMEP has been key to identify the best malaria interventions for the most at-risk populations. Most IP respondents (5/6; 83 percent) cited that the CMEP success is related to their approach to diagnosis, treatment, training, and use of data for stratification. A HF staff respondent added that “it is not only about cooperation and work with communities, it is also the strong support CMEP provides to HFs and the key role of VMWs and MMWs in screening and searching for malaria cases.” Annual reports indicate that CMEP reached 100 percent of the annual targets for the percentage of suspected malaria cases that received a parasitological test, except in PKV and KRK where 99 percent received tests in 2017 and again in PKV in 2019.47 There was a deviation among PPs, where only 93, 98, and 54 percent of suspected malaria cases received a parasitological test in 2017, 2018, and 2019, respectively. This low achievement in 2019 among PPs was due to the restrictions to provide diagnosis and treatment services in 2018.48 CMEP builds local capacities in malaria microscopic diagnosis. For instance, in Battambang and Pursat provinces, 24 microscopists from HFs, former district hospitals, and referral hospitals were trained under the Internal Competency Assessment Course (ICAC) to improve quality assurance in malaria diagnosis via microscopy examination (Fiscal Year [FY] 2018). In Cambodia, patients with positive results receive a DOT regimen. Uncomplicated Pf malaria cases were treated with DHA-P from July 2015 to January 2016. However, due to treatment failures with DHA-P, the regimen for uncomplicated malaria cases was switched to AS-MQ in early 2016. Those with treatment failure by the Day 28 follow-up were deemed to be drug￾resistant and were treated with quinine plus tetracycline. Pregnant women in their first trimester were treated with quinine as well. Annex 8 shows the percentage of all confirmed cases received the appropriate malaria treatment49 under DOT reported through technical supervision for BKN, 44 The intensification plan was extended to Y2 continuing to target and mitigate the malaria burden among forest goers and hard-to-reach populations. 45 Based on the CMEP report; the ET did not have access to the detailed data. 46 Based on CNM MIS. 47 No data provided on numerator and denominator. 48 Malaria services from PPs were shifted to HFs after the restriction in 2018. 49 Based on current national malaria treatment guidelines. 19 PKV, KRK, and SMS ODs. Achievements for this indicator were high at 99 percent in 2017 and 2018, and 98 percent in 2019, although just below the 100 percent annual targets.50 There were 734 Pf/mixed confirmed cases from Y1-Y4 (May 2020) in all target ODs (17, 332, 76, and 2 in Y1, Y2, Y3, and Y4, respectively). CMEP made measurable achievements in the percentage of all confirmed Pf/mixed cases that received appropriate malaria treatment under DOT by VMWs/MMWs in the two non-elimination ODs; two other transitional ODs—SPM and BKN—moved to elimination in October 2019, CMEP Y4. For more details, Annex 12 shows the timeliness of malaria testing and treatment of mini-survey respondents per sample OD. The annual target of 100 percent was achieved in three (BKN, KRK, and SMS) of the four (75 percent) ODs in 2018 and 2019. Progress lagged in PKV with 76, 89, and 82 percent of all confirmed Pf/mixed cases that received appropriate malaria treatment under DOT by VMWs/MMWs in 2017, 2018, and 2019, respectively. There were no notable differences by age or sex for this indicator. CMEP has supported the pilot activities in malaria case management in target ODs. This includes an assessment of the performance of POC tests to screen for glucose-6-phosphate dehydrogenase (G6PD) deficiency (d), and a study to assess the feasibility of the scale up of radical cure treatment for Pv. The WHO recommends that a single 0.25mg/kg primaquine low-dose primaquine (SLDPQ) be given to patients with Pf to reduce transmission without G6PDd testing. However, in Cambodia, where prevalence of severe variants of G6PDd (e.g., Vien Chang variant) are common, those policies were not implemented earlier due to safety concerns. Preliminary results indicated that SLDPQ was likely to be safe to administer without G6PD testing and, based on national evidence (provided by the CAP-Malaria project), CNM recommended its use in July 2017. CMEP supported the introduction and roll-out of SLDPQ in patients with uncomplicated Pf infections51 in 2017. A monitoring of adverse events was included during this introduction of SLDPQ in areas targeted for malaria elimination. CMEP provided additional technical assistance to CNM in elaborating the Malaria Surveillance Manual and job aids to support safe use of SLDPQ. Annex 8 shows the number of ACTs and RDTs purchased by other partners that were distributed with USG funds, and the number of ITNs purchased by USG funds that were distributed.52 CMEP, along with CNM, CHAI, and WHO, facilitated the decision on the radical treatment for vivax malaria (14-day primaquine) and roll out of G6PD testing. The CMEP technical team played a key role in developing the relevant tools (SOPs, treatment regimen, job aids, training guidance, HF case register, etc.). A pilot study was initiated late 2019 in four provinces out of which CMEP works in two provinces (Battambang and Pailin).53 CNM requested CMEP expansion of the pilot into eight provinces covering 20 ODs; however, due to finances and HR constraints, CMEP proposed—along with PMI—to support CNM in three provinces covering five ODs in Y4. Data from OD visits and interviews indicate stakeholders’ strong appreciation for CMEP’s collaborative working style and processes. One IP interviewee told the ET that the “CMEP team excelled in leading many activities, and in the Pv radical cure meetings” and another also mentioned 50 More comprehensive data for this indicator are reported under objective 1, sourced from the MIS for the five elimination ODs. 51 Who were G6PD-deficient and non-deficient. 52 Including type of ITNs. 53 Pilot began November 2019. 20 that “CMEP showed a high level of expertise that was acknowledged by all partners.” The majority of USG interview respondents cited the introduction of the Pv radical cure as a milestone in reaching the point of a national recommendation on the radical cure. Although this decision was “welcomed,” one USG interviewee remarked that “CMEP could have provided this expertise earlier on and probably the lack of high level malaria expertise was a limiting factor.” Another interviewee from the IP recognized the importance of addressing vivax malaria and cited the “need to understand the role of asymptomatic vivax infections in maintaining the transmission and challenging further the malaria elimination.” In response from the CMEP team, there were contract sections mentioning the vivax radical cure; for example, the ET was told that CMEP contract does not require the project to lead implementation of radical cure for vivax. The contract specifies that “CNM may elect to implement both low dose primaquine (PQ) for P. falciparum and radical curative doses of PQ for P. vivax.” CMEP took the initiative to follow up on CNM’s implementation of these efforts in project Y2. The ET confirmed that databases from CMEP showed the incorporation of variables registering information on DOT, G6PD d testing results, SLDPQ, primaquine 14-days, and adverse events. CMEP builds capacities of key staff at the national, provincial, and district levels54 as well as among targeted PPs. As part of improving access to and quality of malaria diagnostic and treatment services, CMEP has been providing technical assistance, including supportive supervision, on-the￾job training, supply monitoring, and regular meetings. Annex 8 shows the number and percentage of supportive supervision per fiscal year and provider. Monthly meetings for VMWs/MMWs were initiated in four transitional ODs and expanded further to other ODs. Under CMEP, VMWs reported information on malaria cases, received supplies of RDTs/ACTs from HFs, and received on-the-job training for diagnosis and treatment, registration and reporting, and SBCC. In the KIIs, all HF respondents (100 percent) told the ET that they participated in training, supervision, and monthly meetings. For instance, one HF manager remarked that “CMEP has the budget to support VMWs and MMWs to work in all areas in the villages. It is a good point that CMEP covers all people in villages, people have knowledge about malaria and then malaria decreases.” Regarding the CMEP’s work, an OD director cited “VMW/MMW capacity building” as the foundation of the project’s effectiveness that makes the model replicable and covers at-risk populations. Half of PHD and OD staff respondents (12/23; 52 percent) indicated that a key benefit for OD and PHD planning is the CMEP’s ability to support and equip people with proper preventive tools and give the ability to know if malaria cases are domestic or imported faster, which results in better planning and decision-making. They said that the reduction of malaria motivated them to engage more with the project and their CNM colleagues to do what they could within their positions to improve conditions and services. The ET found that 96 percent (22/23; one non￾respondent) of PHD and OD respondents considered that, overall, CMEP was effective especially in covering at-risk populations. However, there was no consensus on what constitutes at-risk (vulnerable) populations. 54PMI also assisted in establishing a cadre of VMWs as a critical extension of the public health system at the village level. 21 “CMEP should have regular meetings with PPs to continue strengthening the network and allow them to provide malaria testing.” – PP KII “Before, people were dying of malaria and now we rarely see any person dying because there are adequate malaria testing and treatment services in the village.” – FGD PP CMEP facilitated and supported training of PPs; 100 percent of providers registered in five transitional ODs in FY 2017 (BTB=70, MRS=50, TMK=50, PKV=30, and KRK=38). Training topics for PPs include national policy, uncomplicated malaria diagnosis and treatment, management and referral, tracking and reporting, and monitoring RDTs and ACTs. CMEP’s PP supervision report provides data on the percentage of registered PPs that received supportive supervision at least twice during the past 12 months for four ODs in 2018 and 2019. Overall, 90 and 50 percent of PPs received bi-annual supervision visits in 2018 and 2019, respectively. In 2019, no PP received these visits in PKV, only 50 percent in BKN, and 56 percent in KRK. During the field visit to the six ODs, it was confirmed that PPs were not allow to (and did not) provide malaria diagnosis and treatment services. FGDs with PP participants (7/7; 100 percent) reported an improvement of their knowledge on malaria diagnosis, treatment, and prevention during the training they participated in and they were very enthusiastic about the results on malaria. PPs were providing malaria diagnosis and treatment until October 2018. CMEP has made improvements in the percentage of HFs and VMWs/MMWs that received structured supervision and feedback on malaria at least twice in the last year. CMEP works with CNM and stakeholders to ensure proper use of the M&E reporting forms including monthly registration reports for VMWs, HFs, and PPs and the supervision checklist. Overall in 2017, only 16 percent of the facilities and VMWs/MMWs received the bi-annual structured supervision, increasing to 72 percent in 2018 and 87 percent in 2019, although still short of 100 percent annual targets. CNM central and PHD units carried out supportive supervision to HFs, PPs, and VMWs in the target ODs; the CMEP team provided support for these visits. Supervision visits were held by seven CNM units and CNM central leadership55 during the first year of CMEP implementation. In Y3 only, 155 supervisory visits to HFs were conducted by PHD and CNM staff (109 percent; 155/142). In the Cambodian context, continuous availability of malaria commodities is critical for the effective management of malaria moving towards malaria elimination. HFs are restocked based on an inventory check and verification of expiration dates.56 HF records (patient registers and prescription forms) are checked to ensure consistency with the consumption of antimalarial treatments. CMEP is working at the central and local levels to improve capacities in stock management and detect and quickly respond to potential stock outs. This support involves weekly visits, stock checks, and logistical support for relocating of supplies in all nine target ODs. CMEP uses their mHealth application (via laptops) that allows them to geotag facilities for monitoring. This web-based stock status reporting is also being used by CNM and other stakeholders during monthly meetings. Based on data from CMEP reports, stock outs are minor and restocking seems to be addressed promptly by local teams. 55 Including CNM director visit to PKV and KRK ODs in September 2017. 56 No information was provided about commodity (RDTs, ACTs) waste due to expiration. 22 Malaria Prevention Interventions: Vector Control and SBCC “The most important intervention is having VMWs working at the village where they help decrease malaria cases through village education and distribution of mosquito nets.” – Malaria patient As a primary vector control strategy, the CNM distributes LLINs and long-lasting insecticide￾treated hammock nets (LLIHNs) to all populations at risk in prioritized areas, based on risk strata. Cambodia has a strong culture of bed net use, and net ownership is believed to be high, especially in farming areas (based on CNM reports). However, the ET did not find evidence of population coverage data nor did it have access to the Cambodia Malaria Survey57 dataset to confirm this claim. The CMEP PITT, which gets population-based survey indicators from the Cambodia Malaria Survey, does not have data available on the percentage of people living in target villages who slept under an ITN/LLIN during the previous night. Although, based on WHO’s latest malaria country profile,58Cambodia claimed to have reached 100 percent of the at-risk population protected with ITN, the ET did not find evidence59 to support this claim. Between Y1-Y3, CMEP has distributed overall 455,124 nets (including all types of nets, LLINs, and LLIHNs). In addition to ITN distribution, CMEP provides support to CNM to strengthen their entomology capacities. CMEP funds entomological surveillance activities and provides support for entomological monitoring at one fixed sentinel site in PKV, Pursat Province. With technical assistance (from external consultants and the CDC), CMEP supported CNM in the development of a malaria entomology strategy, SOPs, and a training curriculum for the CNM, PHD, and OD staff. Other support includes training in entomology related subjects targeted to HF, PHD, and OD staff and VMWs and entomological surveillance field work. Two USG official interviewees made observations on entomology activities, one specifically citing “the need for better relationship between CNM and CMEP with a focus on the remaining period of the project.” Along the same line, one IP respondents considered that “CMEP should be focusing their work in what they are good at, surveillance, case management (diagnosis and treatment), and capacity building” and another highlighted “the need to target more activities that will support further the elimination target of Pf, this might include dropping entomology activities.” Another four interviewees (one USG official, one IP officer, and two CMEP staff members) told the ET that there are challenging issues with short notice requests by CNM and the administrative process involved which affect the performance of the work (see EQ5 for additional information). Social and Behavior Change Communication for Malaria SBCC is a key strategy in both the Cambodia MEAF for 2016-2020 and 2021-2025. CMEP is implementing comprehensive SBCC approaches aiming to improve treatment-seeking for malaria within 24 hours at service providers and malaria prevention. SBCC includes IPC and village-based health education campaigns. CMEP created, tested, and distributed SBCC materials to high risk populations, including VMW job aids and three different types of posters aimed at promoting ITN use among forest workers, seeking malaria testing, treating with VMWs, and treatment compliance with VMWs, villagers, and workers. For more details on at-risk population knowledge 57 The Cambodia malaria survey is not a full Malaria Indicator Survey, but rather a targeted malaria survey that is based off the malaria indicator survey methodology. 58 https://www.who.int/malaria/publications/country-profiles/profile_khm_en.pdf?ua=1 accessed on July 18, 2020. 59 Based on the information provided and desk review. 23 and participation in CMEP malaria activities, see Annex 12. SBCC activities have been included in different training and across the implementation of activities. “CMEP provides support to VMW for community awareness. It is good to raise people’s awareness about malaria.” – Malaria patient CMEP’s approach goes well beyond the more familiar Information, Education, and Communication (IEC) “health education approach” and supports uptake and sustaining desired behaviors by having repeated contacts with those targeted for behavior change through VMW/MMW touch points, and participation in community/settlement/village health campaigns. FGDs with malaria patients and at-risk population in six visited ODs (with two to five participants each) showed that there was positive feedback from participants and other community members on the awareness activities. In those FGDs, all malaria patients and at-risk populations knew details about the health awareness campaign (including malaria) in their communities or settlements. In two FGDs, participants from affected communities told the ET that they learned a lot from those activities, they were trained on the malaria symptom, treatment, where to do a test, they learned how to prevent malaria, and they were able to disseminate knowledge to other people. One respondent also said that “VMWs/MMWs encouraged them to do a test quickly if they have fever.” At-risk population mini-survey respondents in six visited ODs reported high levels of satisfaction with CMEP activities. Figure 5 shows at least 75 percent of respondents surveyed who know or have participated in each CMEP activity reported being “very satisfied.” Distribution of LLINs and behavior change communication and awareness activities scored the highest with 84 and 85 percent of respondents, respectively, reporting being “very satisfied.” Figure 5: Percentage of At-Risk Population Satisfaction With CMEP Malaria Activities in Which They Know or Have Participated 24 4.3 EQ3: DOES THE CURRENT NATIONAL MALARIA SURVEILLANCE SYSTEM SERVE ITS INTENDED PURPOSE TO DETECT, IMMEDIATELY NOTIFY, INVESTIGATE, RESPOND, AND SUPPORT APPROPRIATELY FOR MALARIA ELIMINATION AND CONTROL ACTIVITIES? PLEASE ELABORATE THE EXTENT OF CMEP INVOLVEMENT AND ANY GAPS REQUIRED TO IMPROVE THE CURRENT SYSTEM IF ANY. As stated by the WHO, malaria surveillance is “the continuous and systematic collection, analysis and interpretation of malaria-specific data, and the use of that data in the planning, implementation and evaluation of public health practice.” The Global Technical Strategy for Malaria 2016-2030 calls for the evolution of malaria surveillance into a “core intervention.” This transformation is necessary for malaria-endemic countries and in those countries that achieved elimination but with risk of re-establishment of transmission. When surveillance is effectively used to monitor cases and deaths, it helps implementers to identify geographic areas and/or population groups that are most affected as well as design strategic plans to achieve impact. In malaria elimination settings, it is being acknowledged that surveillance plays a critical role. 4.3.1 Surveillance, Monitoring, and Evaluation (SM&E) “CMEP has been strengthening the MIS, contributing to quality and timely reporting of surveillance.” – USG KII The malaria response in Cambodia has achieved a significant impact in recent years and this success of the malaria response is multifactorial.60 The national malaria response has a strong leadership that works with national institutions and structures and multiple malaria stakeholders to support interventions at the central and OD levels. The health system has a network of VMWs, MMWs, and PPM that play a key role in malaria diagnosis and treatment, particularly for the most at-risk populations. In 2018, the role of PPs was phased out, shifting the responsibility for the malaria response to the public health sector. By the end of 2019, 38 percent of the malaria cases diagnosed in the country were treated by VMWs and the rest by HFs. The CNM MIS was fully operational in 2018 and it is able to track malaria case testing, diagnosis, and treatment. More recent improvements and incremental updates also capture surveillance for elimination data, real time reporting, and commodity stock management reporting; however, not all modules are fully functional. Data are discussed monthly among local PHD, OD, and HF staff as well as VMWs/MMWS and partners. An annual review meeting is conducted at the central level with all PHD managers and IPs to review the data and adjust technical decisions. As part of the strengthening of the malaria surveillance and M&E at the central and local levels, CMEP conducted training sessions.61 Training topics included M&E basic concepts, data management (collection, entry, analysis, and reporting) and how all data are integrated into the CNM MIS and MOH Health Management Information System (HMIS). CMEP’s support to CNM at the OD level was focused on strengthening functional capacity of OD teams to respond to malaria outbreaks as defined by the National Strategies (MEAF 2016-2020, 2021-2025). The updated surveillance for the malaria elimination manual did not provide specific instructions on outbreak detection and response. 60 Between 2011 and 2019 there has been an estimated reduction of 68 percent from almost 100,000 cases in 2011 to 31,921 in 2019. Similarly, malaria deaths dropped from 90 reported deaths in 2011 to 0 deaths reported in 2018 and 2019. 61 The majority of CMEP performance indicators are obtained from the CNM MIS. 25 Data from the CMEP PITT and MEAF (2016-2020 and 2021-2025) show heterogenous malaria transmission based on the API. In this epidemiological context, accurate and timely routine data from the sub-national level are essential to identify and manage cases, plan strategic and tailored sub-national level responses, and inform policy decisions. CMEP used a PITT to monitor indicators throughout the life of the project. The CMEP PITT tracks 25 indicators and reports the targets and achieved results by quarter and OD for each indicator.62 The ET desk review noted that in FY 2017 the CMEP team provided inputs to the Surveillance Technical Working group in elaborating the surveillance manual for malaria elimination.63 Based on the CMEP PITT information, SPL was the only OD conducting trainings on the use of case investigation forms with 14 percent and 100 percent achieved targets in Y1 and Y2, respectively. By Y2 (FY 2018), BTB, TMK, and MRS ODs also achieved 100 percent for the percentage of all health facilities in target elimination/pre-elimination ODs trained in the use of case investigation forms. No training data for 2019 were available. The CMEP annual report and KIIs indicated that no malaria elimination training was conducted in Y3 (see Annex 8); however, in the Y3 CMEP workplan there was an activity planned to train 35 dedicated staff on program management, planning, and technical guidance for malaria elimination. 64 There have been no major changes in the overall percentage of all HFs and VMWs/MMWs in target ODs that are submitting surveillance data on time when comparing baseline versus Y3 achievements (overall FY1=93 percent vs. FY3=93 percent). For example, all selected ODs except PKV (FY1=97 percent, FY3=86 percent) and BTB (FY1=97 percent, FY3=94 percent) had a reduction of the achieved result compared with baseline. Based on CMEP’s Y3 annual report, the reason for late submission was due to some VMWs not being present in the regular meetings and VMWs showed lower performance against this indicator with 91 percent of them submitting surveillance data on time in line with the national guidelines (compared with HFs submitting 99.7 percent on time). CMEP conducted five three-day trainings on malaria surveillance. Three trainings were conducted in four ODs during Y1 (TMK, SPL, BTB, and MRS) and the second and third trainings were conducted at BTB and MRS OD (total number of participants = 163). Almost two-thirds (5/8; 62.5 percent) of IP interviewees in the KIIs acknowledged that CMEP strengthened surveillance capacities at the OD level and the majority of interview respondents cited that the VMWs/MMWs play a critical role in the malaria response. One IP highlighted that “CMEP needs to incorporate more VMWs/MMWs into future malaria work.” PPs have been referring suspected malaria cases to HFs.65 For instance, based on CMEP reports and indicator tracking66 (PITT, Annex 8), there was a decrease in this indicator between Y2 and Y1 (overall change Y2 vs. Y1 achieved = -17 percent).67 The percentage of PPs submitting referral 62 The disaggregation varies among those indicators but usually included an OD disaggregation. 63 CMEP supported translation of the manual into Khmer; the manual was endorsed by MOH/CNM on August 31, 2017. 64 Based on CMEP Y3 annual report “There were no malaria elimination surveillance training courses conducted in Year 3 (these were previously conducted in Year 1 and Year 2). However regular refresher training on elimination surveillance took place within regular OD, HFs and VMW meetings.” 65 Since the publishing of the Pakras in 2018, PPs are not allowed to diagnose and treat malaria patients. 66 All CMEP indicators in PITT include fiscal year targets. 67 Only one out of six ODs (TMK) achieved its targets under this indicator in Y2. 26 data has been consistently high over the first three years of the project, at 93 percent overall in 2019. Although falling just behind the 98 percent targets overall and per OD, the percentage of PPs submitting referral data in 2019 was high in all ODs ranging from 85 percent in PLN to 100 percent in TMK. CMEP met the Y1 and Y2 targets for the percentage of target ODs whose case data including notification and response are linked to or integrated to the MIS. The project did not meet the 2019 target of seven out of nine ODs (78 percent), achieving linkage/integration in only five of the nine ODs (56 percent) (Annex 8). CMEP met its annual targets in each OD for having a functional system to detect and respond to all malaria outbreaks in target ODs in place as defined by the MEAF. KII respondents, including OD staff, agreed that CMEP facilitation of surveillance activities “was beneficial” and “increased data quality and timely reporting.” CMEP has been providing support to CNM to strengthen their capacity to conduct entomologic surveillance. In Y3, CMEP, with technical assistance from the CNM Entomology Unit, conducted eight monthly sentinel entomological collections. Respondents from the CMEP team reported that they conducted an internal data quality assessment in Y3. Based on the CMEP reports, results from the assessment demonstrated overall improvement in data quality at the OD level. A CMEP internal data quality assessment was carried out for two ODs in Y3, Q1 (KRK and MRS ODs). The results, reported in the CMEP Y3 annual report, showed improvement in data quality at the OD level. Ninety-three (93) percent of all operating providers (VMWs, HFs, and PPs) submitted surveillance data on time according to national guidelines suggesting success of the various CMEP M&E and surveillance trainings, and instructions provided in meetings. CMEP’s support to CNM at the OD level focused on strengthening the functional capacity of OD teams to respond to malaria outbreaks as defined by the MEAF. The updated surveillance for the malaria elimination manual did not provide specific instructions on outbreak detection and response. CNM defines a malaria outbreak as the mean of malaria cases in a particular month that exceeds the mean of monthly malaria cases derived from a minimum five-year data set (excluding abnormal years) plus 2 SD (an epidemic threshold). The ET did not have data to assess whether this threshold includes indigenous cases only (the recommended approach by WHO) or include all cases (indigenous and imported). 4.4 EQ4: TO WHAT EXTENT HAS THE CMEP PROJECT BUILT CAPACITY OF THE CNM/MOH TO MANAGE, INTENSIFY, AND SUSTAIN MALARIA CONTROL AND ELIMINATION EFFORTS, PARTICULARLY AT THE PHD AND OD LEVELS? With decentralization of health care services, it is extremely difficult for Cambodia to eliminate malaria effectively and sustainably. At the PHD level and some ODs, there are shortages of people with the time and skills necessary to provide inputs to operationalize malaria actions. Furthermore, there is high turnover of staff, mainly due to low salaries and lack of a diversified job market. Often trained staff in Cambodia engage in activities other than malaria prevention, control, and elimination. This is attributed to misallocation, or lack of motivation, to undertake malaria control activities. In the six visited ODs, most of PPs are also staff from the HF. The CMEP capacity building strategy was developed to guide the implementation of interventions aimed at addressing performance gaps at all levels of the health system to ensure effective malaria 27 response in line with the MEAF. As part of its capacity building interventions and to optimize sustained improvements in malaria service delivery, CMEP applies various interactive techniques and blended learning approaches that are combined with QI efforts. This includes coaching or supervision, which ensure engagement of the providers and allow opportunity for continuous constructive feedback as well as simulated and learning practices that are planned and delivered on a low-dose, high-frequency (LDHF) basis. CNM senior management team members told the ET that CMEP contributed (along with other partners) to several national guides, SOPs, and protocols that were revised and submitted for approval by the CNM. Of those that were submitted, the protocols on SLDPQ for Pf and radical cure for Pv were referred as “good quality” and used as “national guidance” by IP managers, as confirmed by respondents in the KIIs. Additionally, CMEP increases its cost-effectiveness by developing digital products with limited printing of technical documents. The dissemination of materials is based on approved national protocols and policies by CNM. The MEAF 2016-2020 and its renewed version, 2021-2025, were developed with several partners and all malaria approaches were incorporated into the final draft. A list of selected policies, guidelines, and standards with CMEP contributions is included in Annex 9. True institutionalization can only take place in Cambodia when updated protocols are standardized and disseminated; HF staff, VMWs, and MMWs are trained; and new approaches are implemented, supervised, and sustained. KIIs with the CNM, PHD, and OD staff confirmed universal appreciation for the overall capacity building approach used by CMEP, including CMEP’s involvement in developing malaria training curricula. The CMEP team does not take full credit for any of these efforts and said that progress is the result of many partners. IP stakeholders reported that technical meetings68 represent an excellent opportunity to actively contribute and move these guidelines forward. Those meetings are frequently attended by “almost” all malaria partners; however, the final approval of those documents takes time due to mainly national administrative processes (KIIs from IP and CMEP) and CNM is responsible for the national approval. CMEP, CNM, PHD, and OD staff said that they jointly identified skilled national personnel from across the ODs to conduct training-of-trainers (TOTs) for key malaria topics. CMEP participated in national malaria technical working groups (TWGs) and facilitated meetings of the CNM and stakeholders to raise national awareness about key malaria areas for intervention. Several respondents from PHD, OD, HFs, and IPs, as well as USG officials, confirmed that they had attended these meetings. KIIs with PHD personnel, service providers (HF staff, VMW, MMWs, PPs), and trainees cited activities and approaches reported in CMEP reports. The project provided support for training activities at all levels in the health system with focus on target ODs. CMEP successfully met annual targets of 100 percent each year of project implementation for the percentage of AOPs developed that include malaria activities and budgets (see Annex 8). When assessing the usefulness of trainings among service providers in the mini-survey (N=83), only PPs have a lower rate (78 percent compared with 100 percent among VMW/MMWs, HFs, and others69) (Figure 6). 68 There are monthly, quarterly, and annual meetings to discuss progress and bottlenecks and plan next steps. 69 Others refer to PP outside the CMEP network. 28 Figure 6: Percentage of Health Service Providers Opinion on Usefulness of Training The percentage of planned supervisory visits conducted by national, PHD, and OD staff at public HFs in all targeted ODs remained relatively consistent over the first three years of the project, falling short of targets at 86 percent in 2019. The percentage of planned supervisory visits to VMWs/MMWs and PPs in all targeted ODs improved substantially from 2016 to 2019, with over 100 percent of visits conducted to each group in 2019, exceeding targets. The frequency of supervision visits made by PHD and CNM staff gradually improved over the first three years of the project, falling short of targets for the percentage of planned supervisory visits conducted by PHD and CNM staff to all targeted ODs in 2017 and moving on to exceed targets for supervision in all ODs in 2019 (Annex 8). Data on supervisory visits from CMEP reports were confirmed by PHD and OD interview respondents. CMEP worked with OD and CNM staff to ensure an uninterrupted supply of malaria commodities at the ODs and detect and respond to any stock outs or potential stock outs. For example, through HF visits, the stock status was assessed using the CMEP mHealth70 application. CMEP conducted a total of 160 visits to HFs for stock monitoring in FY 2019 and potential stock outs were able to be immediately resolved. In Y3, ACT stock outs were observed twice in two facilities. RDT stock outs were reported three times in three HFs (CMEP Y3 annual report). Interviews with HF staff confirmed that stock is usually reallocated from other ODs/areas by the CMEP and/or OD teams. Improvements of malaria practices and procedures, measured through periodic supervisory visits, were made by the CNM and PHD staff themselves (including senior managers); CMEP supported mentoring and coaching national-level and PHD trainers. HF managers and staff members said that the supervisory visits (supported by CMEP) are “very valuable.” The ET noted examples of process improvements supported by CMEP during OD visits and KIIs and group interviews with HCWs. Among 84 service providers that participated in the mini survey conducted by the ET (84/89), 88 percent, considered their experience with CMEP supervision very useful. MMWs, HFs, and other providers that participated (VMW N=24; HF N=30; Other N=11) rated CMEP supervision as very useful (90 percent or more) (Figure 7). Among PPs, the overall rating was lower with only 70 CMEP mHealth application monitors commodities availability among HFs. 29 79 percent (N=19; Female=50 percent; Male=87 percent) reporting supervision as very useful. These scores were contextualized in KIIs with CMEP, HF, and PP respondents which indicated that PPs were not allowed to provide malaria services (conduct diagnosis and treat malaria patients).71 Figure 7: Percentage of Health Service Providers’ Opinion on Supervision Usefulness (N=89) 4.4.1 Health Worker Trainings “…priority training, trainer-of-trainers, and TA [technical assistance] contributed to improved technical and managerial skills.” – USG KII “…CMEP helped to increase the knowledge of malaria prevention (i.e., malaria prevention, use of mosquito nets, malaria prevention information to other communities’ members.” – FGD VMW Multiple indicators showed improvement toward training conducted by CMEP. The total number of trainees in malaria diagnosis for the Y1-Y3 period was 1,725 (1,004 in VMWs/MMWs and 721 in Public HFs). After a lag in 2018, CMEP exceeded 2019 targets for the number of HCWs trained in malaria laboratory diagnostics (RDTs or microscopy) with USG funds (see Annex 8). A larger proportion of HCWs trained by CMEP were male than female (Male= 948 or 54.95 percent; Female=777 or 45 percent).72 CMEP achieved similar results for the number of HCWs trained in case management with ACTs with USG funds. Between Y1 and Y3, there were 1,694 HCWs trained, the majority (59 percent) were VMWs and MMWs (1,004/1,694) and 54 percent were male (925/1,694). The number of HCWs at public HFs that received this ACT training increased from 186 in Y1 to 690 (cumulative) in Y3. PHD and OD KII respondents shared that they knew and commend the type of training implemented by CMEP. For example, one respondent mentioned that “the type of training[s] for VMWs and MMWs are focusing on topics like national policies, malaria diagnosis and treatment, referral system, supply, tracking and reporting and they are effective.” In half of the FGDs with malaria patients (7/14), participants noted an improvement in their malaria knowledge after participating in CMEP activities. In three of the 14 FGDs, participants 71 None of the service providers rated supervision as “not useful” in the mini-survey with service providers. 72 No sex disaggregated data provided among service providers. 30 reported that sharing malaria prevention information with other community members was important. Other notable comments shared during FGDs with malaria patients include reports of better knowledge of malaria symptoms, diagnosis, testing and treatment, and better knowledge of malaria prevention by the use of mosquito nets. Furthermore, malaria patients considered CMEP activities “a success” noting that the training “helped them understand better how to prevent and treat malaria.” In five of the FGDs (5/7, 71 percent) with service providers, respondents shared that they appreciated the training and workshops they participated in, in which “they developed their skills on malaria symptoms, testing, treatment, record keeping and reporting.” Service provider respondents from the survey (89/203; 44 percent) included VMWs, MMWs, PPs, HF staff, and others. Slightly more than two-thirds were male (60/89; 67 percent); the percentage of respondents by OD were BTB=8 (9 percent), KRK=20 (23 percent), PLN=15 (17 percent), PKV=14 (16 percent), SPL=25 (28 percent), and SPM=7 (8 percent). Three-quarters of respondents (67/89) commented about the training received by CMEP staff (including national trainers) or colleagues and the training was considered appropriate and useful. “Increased knowledge on malaria prevention skills” was cited by a respondent as the primary benefit from the training among providers. Additionally, they recommended that future trainings should be more frequent (including refreshers) due to the high turnover of staff, particularly at the OD level. All VMWs, MMWs, HFs, and others rated CMEP training as very successful (100 percent); however, in the PP group this rating was lower (71 percent). Service provider (VMWs=24; MMWs=10; PPs=29, HF staff=22; Others73=8) respondents that participated in any CMEP training were asked to rate the success of the training. Most rated the trainings as “very successful” (MMWs, VMWs, and Others = 100 percent) with exception among HFs and PPs (see Figure 8). Figure 8: Percentage of Service Providers’ Opinion on Trainings’ Success 73 Others refer to PP outside the CMEP network. 31 FGDs (7/14; 50 percent) also confirmed service providers’ satisfaction with CMEP training activities. VMW participants appreciated the trainings and considered that they developed new skills on malaria symptoms, testing, treatment, recording information, and reporting. Data from CMEP and CNM reports, supported by PHD and OD-level KII respondents, indicate that CNM and MOH officials “appreciate CMEP’s work.” On June 3-4, 2019, the CMEP team conducted and facilitated the visit of the Cambodian Minister of Health Prof. Mam Bun Heng to Pursat province, to PKV at the triangle area of three provinces of Pursat, Kg Speu, and Kg Chhnang. The MOH delegation consisted of 68 participants including the Minister and MOH officials, Pursat provincial governor and deputy governor, chairman of provincial council, CNM director and subordinates, three PHD directors, the CMEP team, USAID/PMI, and other IPs. The Minister visited a forest location to physically witness and talk to the forest goers. During this visit, malaria testing was conducted by MMWs and by Forest Rangers; the Minister interacted with those groups. This official visit is the first ever visit of the Minister of Health to any partner location and is considered a “success” for CMEP and CNM. During the KIIs, the IPs also highlighted their appreciation and raised the Minister’s visit as a “crucial step” for political will and support of the malaria elimination response. CMEP has been supporting the capacity building of PPs; however, after the introduction of the MOH “Pakras”74 for PPs on April 13, 2018, no training was done. CNM has clear regulations about not engaging PPs to test and treat malaria cases. CMEP follows those regulations and maintains the bi-monthly meetings and supporting supervision meetings to encourage PPs to refer all suspected and actual malaria patients to get tested and treatment in the public HFs. While understanding the financial interest of PPs, there is also a need to effectively use all available resources (financial, human, logistics, etc.) in this evolving malaria epidemiological context. Planning for future malaria scenarios should take into account any health system decentralization process, access to services for remote hard-to-reach populations, and the management of epidemics (i.e. vivax malaria upsurges). The ET noted that PPs are not allowed to provide malaria services since 2018, however, there is a long term issue on sustainability and all malaria stakeholders should assume that not all service providers (including PPs) will be fully or even partially funded by donors after malaria elimination is achieved and prevention of reintroduction is carefully monitored. This long term strategic vision should be linked to the upcoming decentralization process (which is in early stage) and the MEAF 2021-2025. Furthermore, despite any restriction in service provision, the ET recommends having all malaria providers (including PPs) continuously updated in malaria diagnosis, treatment, and epidemic containment activities (Note: This statement does not represent an endorsement of PPs to conduct routine malaria diagnosis and treatment, it is a cautionary recommendation in this evolving epidemiological setting in which having a cadre of trained workers could contribute to special events, for example during epidemics (see recommendations). 74 A regulation issued by a Minister. 32 “…There are major challenges to capacity building at CNM, PHD and OD levels that include constant training needs, high staff turnover and difficulties in filling positions quickly.” – CMEP staff “There are three key issues that I would highlight as challenges in the capacity building: high staff turnover at PHD level, variation on capacities among ODs and dependence on donor funding.” – USG KII The CMEP capacity building training, along with specific malaria technical training were initially developed for the CAP-Malaria and the ET found that CMEP significantly contributed to expand those trainings. CMEP also contributed to a more integrated approach into the national structure of health services. 33 4.5 EQ5: WHAT FACTORS, BOTH INTERNAL AND EXTERNAL OF CMEP, HAVE FACILITATED OR CONSTRAINED PROJECT PERFORMANCE AND HOW CAN THESE FACTORS BE ADDRESSED? A. Measures that have been put in place to contribute to or promote sustainability. Factors that still need to be addressed. B. The efficiency and effectiveness of the design and project management arrangements and oversight between the prime recipient and sub-partners (and sub-grantees) for achieving project objectives. Table 4: Internal and External Factors Affecting CMEP’s Performance Hindered Enhance External Ambit of the Project External ∉ COVID-19 pandemic ∉ Indicators’ definitions ∉ High turnover of staff ∉ Low salaries, growth opportunities, and motivation ∉ High internal mobilization and/or migration ∉ High cross border movements ∉ Malaria transmission for vivax, more difficult to eliminate ∉ Limited implementation of radical cure, delayed policies ∉ Changing farming context ∉ Data sharing issues at CNM ∉ Financial sustainability ∉ Missed OR in vivax control and elimination ∉ Cost of implementation logistic ∉ Environmental challenges ∉ Lack of cooperation among MMPs ∉ Long bureaucratic delays ∉ Management of CNM’s short notice requests and expectations ∉ Type of nets and preferences among at-risk populations ∉ Quality of reporting ∉ Low expenditure rate of funds ∉ Delayed deliverables ∉ Management issues ∉ Indicator definitions ∉ Sustainability and exit strategy ∉ Delayed inclusion of vivax approaches ∉ New key personnel ∉ High number of skilled personnel ∉ Strong collaboration at local level ∉ Performance of sub￾contractors ∉ OR in case management ∉ Excellent work with VMWs/ MMWs, and HFs, OD and PHD staff ∉ mHealth application and bottleneck approach for solving stock outs ∉ Good reputation in the target areas ∉ Monthly meetings ∉ Annual reviews ∉ Good working collaboration with CNM leadership ∉ Cooperation of HF, OD, PHD, and CNM staff ∉ Multisectoral collaboration with malaria partners ∉ Strategic use of government structures ∉ Use of the RGC procurement supply management (PSM) system ∉ Collaboration with other sectors: education, military and forestry. ∉ Network of providers and intrinsic collaboration: VMW, MMW, PP, and HF ∉ High level political will Source: Evaluation findings. 34 Table 4 shows internal and external factors affecting CMEP’s performance. There were several factors that interviewees reported as CMEP’s performance enhancers outside the control of the project. For example, the high level political will at MOH, CNM, and the cooperation between malaria stakeholders and government entities are acknowledged as important enablers in the work CMEP is doing. Also, the project uses RGC’s structures, procurement and supply systems, and the routine meetings (monthly and annual reviews). Under CMEP’s control, factors mentioned that improved the performance include the new key personnel, high number of skilled staff, good work relationship with VMWs, MMWs, OD, PHD, and CNM staff, and performance of subcontractors. The role of VMWs/MMWs reaching out to hard-to-reach populations were highlighted by all KIIs as a major achievement in the malaria response in CMEP-supported areas. Although the project implements its work at the subnational level, its presence is acknowledged at the national level, where it is considered a key partner with skilled staff by the MOH, U.S. Ambassador, IPs, and others. The use of the mHealth application to monitor commodities, the selection and close monitoring of CSOs, the high number of skilled personnel at the OD level, and the interactions between malaria stakeholders during monthly/annual meetings were also considered important enablers of the CMEP’s performance. Certain inhibiting factors that respondents reported were both outside and inside the project’s ambit of control. Under CMEP’s control there were important issues in Y1 and Q1 of Y2 (15 months) (poor quality of reports, delays of deliverables, low burning rate of funds, and management issues). All of those early issues in Y1 significantly affected CMEP performance and were solved with the incorporation of a new Chief of Party (COP) and a Senior M&E Officer in the project. In the KIIs, Washington-based USG stakeholders shared common reported dissatisfaction with project management and technical leadership during Y1 of the project, which resulted in delays and poor quality of the deliverables. “Despite being URC, the same incumbent of CMEP’s predecessor, the CAP-Malaria Project, the project has had several issues in the first two years that required a lot of time, attention and follow-up from different teams, this was unexpected…fortunately those issues were successfully solved…” – USG KII In interviews, USG respondents cited the project’s weaknesses as the lack of a more proactive advocate approach towards the vivax malaria agenda with CNM and the lack of a highly technical malaria expert as part of the project implementation team. However, overall there is consensus among USG, IP, and PHD staff that the addition of the new COP in Y2 made significant improvements in the project. Some USAID and IP respondents felt that managing Pf malaria in Cambodia (a global threat) requires high-level expertise, and all respondents (from USG and the IP) agreed that it is necessary that all stakeholders bring their best experts to achieve malaria elimination. “…the lack of a high level malaria expert in the CMEP is usually compensated with additional workload from USG agencies, this should be something to consider in future projects.” – USG KII “…USG agencies have sent their senior malaria experts in various opportunities in the to support CMEP during difficult times [beginning of CMEP implementation] and when there are issues that require strong expertise.” – USG KII 35 “URC has been the incumbent for CAP Malaria and also for CMEP. However, the start-up of the project was painful; they [caught up] with activities in Y2. The CMEP project is doing its best. The 1-3-7 model has been an iterative process, with strong collaboration with country teams and sharing inputs in the elimination package. For this 1-3-7 strategy the role of VMWs/MMWs has been strategic and crucial.” – USG KII “…planning for transition should be developed in advance. There should be clear SOPs, guidance to pass over to other partners, I would say ‘Prepare better for transition.’ This is the time to document the success of western Cambodia, we need to be vigilant, we do not want re￾introduction and there is still a lot to do to eliminate all malaria cases.” – USG KII All malaria stakeholder groups appreciate CMEP’s collaborative working style, particularly to technical groups and other collaboration platforms. All USG and IP officials interviewed (100 percent) stated that the selection of CMEP’s new COP in 2018 has been a significant improvement for the project. The quality of the work has been improved and the work of CMEP is acknowledged as “high quality” by stakeholders. There is a high level of participation of CMEP teams at the central, PHD, and OD levels in monthly, quarterly, and annual meetings, all of which are well-attended. All respondents from the IP highly appreciated CMEP’s participation in technical group meetings. “CMEP teams are highly-skilled at the central and OD level, we are all grateful of the inputs of CMEP’s project” – IP KII Issues related to indicators’ definitions (i.e., API, outbreaks), the delayed inclusion of vivax approaches, and sustainability issues were identified as affecting performance of the project. There were additional factors outside CMEP’s control that were also cited as negatively affecting CMEP’s activities. Other factors cited by interviewees include the high HR turnover, low salaries, limited career growth opportunities, lack of motivation of local staff, high internal migration, cross-border movements, delayed and limited implementation of new vivax malaria approaches, long bureaucratic delays, and issues managing CNM’s short notice requests. In interviews, USG and IP respondents cited that entomology remains a challenging area for CMEP, considering that “CNM’s expectations are high and there are still complex obstacles with short notice of requests, including entomology activities.” Population movements (within Cambodia and across the border with Thailand) are hindering CMEP’s performance and this was outside the control of the project. Human mobilizations seem to be concentrated in “waves” and, while the intensification plans have partially addressed malaria control and elimination among those groups, these mobilization patterns are changing over time. There is a need to improve the use of data for decision-making more effectively, including updating the profile of the suspected and confirmed cases. Most cases are due to vivax and are mostly imported cases; this requires a further strategic assessment to further prioritize activities. IPs are aware that the success in the malaria elimination requires a list of steps, two of them are strategic use of data and management. 4.5.1 Limited Impact of COVID-19 Pandemic on CMEP Implementation The COVID-19 pandemic is ongoing in many countries and territories, and cases in Cambodia were anticipated. The first case of COVID-19 was confirmed on January 23, 2020. The RGC announced the shutdown of malls, public gatherings, pubs, cinema halls, and other public locations that host people in large gatherings. With the increase of cases, CNM suspended all national-level 36 meetings on March 18. In relation to malaria-related activities, all events starting the week of March 23 at the PHD and OD levels were suspended, including all face-to-face CMEP and stakeholders’ meetings. As of March 24, 2020, Cambodia reported 91 cases of COVID-19, and the CMEP team prepared a business continuation and contingency plan (sent to USAID on March 26). This plan includes COVID-19 transmission scenarios and mitigation activities for the safety of project staff, malaria commodities stock out monitoring, and office security. CMEP teams at the central and local levels followed the RGC, WHO, and USAID preventive measures. No COVID-19 cases were reported among CMEP staff; PHD, OD, and HF staff; nor VMWs/MMWs. By July 10, Cambodia had a total of 141 confirmed cases of COVID-19. All respondents (USG, IP, VMW, MMW, PHD, and OD) reported limited impact of COVID-19 on CMEP activities, only delayed meetings. HF respondents from the six visited ODs, confirmed that malaria services provision has not been interrupted, however, patients’ visits to HFs have decreased. 4.5.2 Sustainability Issues The RGC currently receives financial support mainly from the Global Fund (GF), PMI, and the Bill and Melinda Gates Foundation (BMGF) (see Figure 9). Figure 9: A. Country Funding Landscape 2005-2017 (All Donors) and B. Domestic Funding Only 2018-2023 Source: WHO Malaria Country profile.75 GF funding request 2020. The GF remains the major donor for malaria control and elimination and Cambodia is part of the Regional Artemisinin-Resistance Initiative 2 Elimination (RAI2E) grant for 2018-2020 and also receives support for the national malaria response. CHAI provides technical assistance to CNM with support from the BMGF. Cambodia is heavily dependent on international funding for the national malaria response. The new MEAF 2021-2025 aims to eliminate malaria by 2025 requiring an investment of $141.7 million for the five years. However, domestic funding from the RGC averages $4 million per year between 2021-2023 and there is no new funding earmarked to cover the gaps in the MEAF proposed budget. 75 https://www.who.int/malaria/publications/country-profiles/profile_khm_en.pdf?ua=1 37 CMEP developed a sustainability plan and exit strategy based on USAID’s principles, which defines sustainability as “the capacity of a host country entity to achieve long-term success and stability and to serve its clients and consumers without interruption and without reducing the quality of services after external assistance ends.”76 The purpose of a sustainability and exit plan is to ensure that host￾country partners and beneficiaries are able to maintain project results and impacts after a project ends. In terms of programmatic sustainability, CMEP’s activities are designed and implemented jointly with local ODs, and PHD and CNM staff to the extent possible, with the goal of sustainability at the end of the project. In interviews with CMEP managers, they highlighted that CMEP’s activities have been implemented “mostly” with the staff from PHD, OD, and CNM and “preparing the ground to ensure gradual takeover of all those activities currently supported through the project by CNM.” However, the funding for logistics and mobilization of the activities is mainly provided by CMEP and there is no future RGC funding earmarked to ensure a sustainable transition and continuation of activities after CMEP ends. For instance, there is no clear view how the RGC will absorb some of the CMEP activities outside staff salary and commodities nor is there information on partners’ contribution to the MEAF 2021-2025. A respondent from CNM remarked that “CMEP has not discussed yet any handing over of the activities and it will be very challenging to stop the support of VMWs, MMWs and CSO as they play a key role in the elimination in the CMEP’s ODs.” CMEP plans to phase out some of the key activities as shown in Figure 10 and CMEP’s exit plan includes a list of activities to be handed over to the ODs on planned dates. The ET considers that the exit plan should be discussed in detail with CNM and malaria stakeholders during the monthly meetings and based on a readiness assessment of the OD staff; some of those activities can be handed over in a phased manner to those ODs that are ready. Figure 10: CMEP’s Expansion and Proposed Sustainable Steps Source: CMEP Sustainability Plan and Exit Strategy, Version 4 (Final version), March 6, 2018. 76 https://www.usaid.gov/what-we-do/global-health/hiv-and-aids/technical-areas/aid-investment#section3 38 Six out of eight (75 percent) malaria implementers and other stakeholders interviewed stated they had no knowledge of CMEP’s sustainability plan. Similarly, three CNM senior leadership staff cited the lack of knowledge about the handing over of CMEP’s activities. In response to the proposed CMEP handover of activities presented in Y4-Y5, three USG KII respondents considered that some of these activities could be handed over now to those ODs with better capabilities and strong skills and oversight from PHD and CNM. 4.5.3 Oversight Between the Prime Recipient and Sub-Partners CSOs play a crucial role in malaria control and elimination in Cambodia, especially in providing services to remote and hard-to-reach communities. The CMEP team worked with qualified CSOs to build their capacities. A CMEP-funded small grant program was initiated in May 2017 to support engagement with CSOs in malaria response. A local CSO, Action for Health Development (AHEAD), was selected and contracted in September 2017 to work in SPL and TMK. AHEAD implemented activities in SPL targeting 59 villages and 10 HF catchment areas, including LLIN distribution, conducting malaria health education campaigns, mapping malaria mobile population settlements, referring suspected malaria cases to HFs, and malaria situation monitoring in their area of work. In May 2018, AHEAD expanded activities to TMK. Another CSO also began working in the malaria response: Partners for Health Development (PDFA) that supported malaria activities in BTB and MRS in 2018 and Oephan Ecole Corp (OEC) which worked in MRS (started working in Y3). Based on an established set of indicators and targets, AHEAD improved its performance considerably by Y3 and aligned well with OD structures and CMEP support, OEC was not able to meet indicator targets, consistently performed poorly (in achieving targets in two indicators), and, ultimately, CMEP ended their contract. Based on the target achieved and presented in the CMEP annual reports, the ET considers that the performance of AHEAD and PDFA reached most of their targets. The small grant program and close CMEP follow-up of CSO subcontractors was effective and efficacious. 5.0 CONCLUSIONS AND RECOMMENDATIONS 5.1 CONCLUSIONS 5.1.1 EQ1 The implementation of the 1-3-7 malaria elimination approach was effective and contributed to the elimination of malaria in SPL OD. The interventions included in the MEP were effectively manageable at the community and settlement levels as well as at the HF level. CMEP strengthened the capacities to implement the 1-3-7 model at the OD-level, although those capacities varied by OD. Since April 2016, all cases of malaria diagnosed in SPL have been classified as imported, indicating interruption of local transmission which represents a criterion for elimination (three years without indigenous cases). The effectiveness of the model in other ODs varies with a significant impact (as measured by API, positivity rate, and number of cases) towards reduction of falciparum and mixed (Pf+Pv) infections. The elimination model 1-3-7 could be improved by proactively introducing, expanding, and adapting treatment77 interventions to target vivax malaria 77 Malaria diagnosis in Cambodia includes all species (i.e., vivax infections). 39 and effective management of outbreaks that were not initially included in the original design of the model. CMEP is supporting a pilot for radical cure using G6PD d testing and primaquine for vivax malaria. Assessing the real impact of the elimination model requires access to detailed programmatic data and separating indigenous from imported cases. The ET understands that CNM has sufficient data to assess in detail this impact that could provide stakeholders with additional granular strategic direction for future investments, however, the ET did not have access to these detailed epidemiological data. The API definition used by CNM to monitor malaria elimination progress differs from the international policy guidance from the WHO (including indigenous and imported cases as part of API) which might overestimate the malaria transmission in the country. Despite initial delays in the first 15 months (Y1 and Q1 of Y2), CMEP effectively supported the continuation and expansion of the evidence-based malaria elimination model to another target OD. CMEP activities are cost effective compared to the no-CMEP proxy, as shown through an incremental cost-effectiveness of $28 for every additional Pf or Pf/Pv mix malaria case averted. The elimination model should be further integrated into government practice to institutionalize elimination activities to reach the goal of eliminating malaria in Cambodia by 2025 (all species) with a priority on eliminating the hotspots of Pf.78 Overall, CMEP has implemented the malaria activities tied to its RF. The evidence-based model for malaria elimination in SPL has been effective in reducing 66 percent of the overall malaria cases in target ODs (reduction Pf, Pv and Mixed of 97 percent, 29 percent, and 94 percent, respectively). Expansion of the elimination model was done in a phased manner and nine ODs have moved into an elimination status based on the RGC MOH criteria. National stakeholders consider that CMEP has been crucial in this elimination process by supporting local and national￾level staff as well as other IPs. The inclusion of high-quality interventions and additional HF and community approaches, along with strengthening access to case management and prevention with vector control, had mutually reinforcing effects. All respondents stated that CMEP’s support has been pivotal in strengthening the malaria elimination efforts in Western Cambodia, particularly MDR falciparum malaria, a global threat. CMEP’s results and achievements are noticed by stakeholders and, in many ODs, the malaria response has improved as a result of project interventions. More work remains to be done in the areas of vivax control and elimination and treatment with radical cure and management of vivax outbreaks. There is a need for a more detailed stratification and harmonization of indicators with global guidance as well as a tailored￾approach for imported and indigenous cases using surveillance as the core intervention. The RGC has plans for decentralization of the malaria program activities to the OD level; however, this process is in the early stages. 5.1.2 EQ2 The evaluation found CMEP increased access to effective malaria interventions, especially for vector control and case management. There were new interventions added during implementation of the CMEP activities: introduction of SLDPQ for Pf, piloting of radical cure of vivax malaria, and the introduction of G6PD deficiency testing. CMEP supported and expanded the implementation of high-quality malaria interventions in a phased manner in target ODs. The overall number of tests among suspected cases increased; and there was slow progress on the overall percentage of malaria cases that received DOT treatment as per national treatment guidelines, although 100 percent of cases were treated. During the malaria outbreak in 2017- 78 Cambodia commitment of Pf elimination is 2020 and Pv by 2025. 40 2018, CMEP played a key role with other stakeholders in the development of first and second intensification plans where interventions were targeted and tailored to MMPs. MMWs were taught how to improve malaria control and elimination services in their catchment areas including healthy lifestyles. There is a need to fully implement vivax malaria control and elimination approaches in all ODs to achieve further impact and there are several activities ongoing from CMEP, CNM and malaria stakeholders to do so. 79 5.1.3 EQ3 CMEP has successfully built capacities at all central (CNM) and local levels (OD, PHD, HF, VMW, MMW, and CSO) to manage, promptly respond, and sustain malaria control and elimination efforts. The project developed a surveillance strategy to support the national MIS and provided the ET with a list of protocols, guidelines, and standards at the national and regional levels that were supported, to various extents, by the project. Staff from HFs, VMWs, MMWs, IPs, and others confirmed CMEP’s involvement in surveillance has significantly improved timeliness of the processes (times for detecting, diagnosis, providing treatment, notification, and response), and, more importantly, data quality. CNM representatives at the central and province levels specifically cited project contributions in data quality improvement, timeliness, and data use for decision￾making. Data procedures have been streamlined and led by the CNM and the project’s inputs have significantly supported local teams at the OD level to capture the national malaria indicators. Most of CMEP’s performance indicators are sourced from the CNM MIS. Approval and dissemination of data is a sensitive issue that requires clearance from CNM leadership, which was not obtained for this evaluation. The malaria surveillance system detects, notifies, investigates, responds, and supports malaria elimination activities. One hundred (100) percent of notification units are reporting on a monthly-basis to the MIS in CMEP ODs;80 and stakeholder meetings are routinely conducted to analyze and discuss progress and bottlenecks in the malaria response. The malaria epidemiology is changing with MMPs migrating between ODs/provinces and across the border with Thailand, presenting a huge challenge for malaria elimination in Cambodia. National stakeholders told the ET that the malaria stakeholder working groups meet monthly and annually and all malaria-related activities and work plans are presented and discussed. However, data sharing among provinces is limited with annual malaria review meetings conducted only at the central level. There is a need to use the MIS data to characterize malaria profiles among indigenous and imported cases and tailor the responses. 5.1.4 EQ4 CMEP has integrated capacity building activities within the structure at the CNM, PHD, and OD levels. CMEP’s capacity building approach addresses QI capacity building and supports supervision combined with implementation of interventions targeted and delivered to the most at-risk population, forest goers, and MMPs. CMEP has developed national capacities to assess, plan, train, provide supportive supervision, and monitor programs at the local level. Coverage of quality malaria services was low for some indicators at baseline81 and access to improved services for most of the at-risk population is far from universal. Further progress will require more granular epidemiological analysis of the indigenous cases, identifying the risk factors and estimating the 79 Following the pilot in CMEP 2 provinces, the scale up of P.vivax intervention with PQ radical cure and G6PD testing for the entire country will commence from October 2020. 80 Ninety-nine (99) reporting rate nationwide of the CNM MIS assessed by mid-2019. 81 Based on the PITT. 41 population of mobile and migrant workers at the OD level. The internal migration dynamics need to be better understood to scale up several health systems strengthening (HSS) components that were not CMEP’s focus. In this epidemiological setting, malaria epidemics will be common and they will require a rapid response with trained personnel at the local level. The availability of trained local capacities are important for the epidemic response and this includes VMWs, MMWs, HFs, CSOs, and PPs. Maintaining a cadre of skilled personnel in this context will strengthen the malaria elimination and prevent reintroduction. 5.1.5 EQ5 There were several factors that have affected CMEP’s performance. For example, some factors that enhance the performance of the project but are outside its control include: the high level political will at MOH and CNM and the cooperation between malaria stakeholders and government entities, which are acknowledged as important enablers in CMEP’s work. Additional factors enhancing CMEP’s efforts also include the use of the RGC’s structures, procurement and supply systems, and the coordination of routine meetings (monthly and annual reviews). Factors under CMEP’s control that improved the performance include the new key personnel, high number of skilled staff, good working relationship with VMWs and MMWs and OD, PHD, and CNM staff as well as the performance of subcontractors. Certain inhibiting factors of CMEP’s progress were both outside and inside the project’s ambit of control. Under CMEP’s control there were important issues during the first 15 months of implementation (poor quality of reports, delays of deliverables, low burning rate of funds, management issues, the lack of a more proactive advocate approach towards vivax malaria agenda with CNM, and the lack of a highly technical malaria expert as part of the project team) that negatively affected performance. Outside factors which was not under CMEP control were related to indicator definitions (i.e., API, outbreaks), the delayed inclusion of vivax approaches, high HR turnover, low salaries, limited career growth opportunities, lack of motivation of local staff, high internal migration, cross￾border movements, delayed and limited implementation of new vivax malaria approaches, long bureaucratic delays, and issues managing CNM’s short notice requests were among of inhibitors of CMEP’s performance. All respondents (USG, IP, VMW, MMW, PHD, and OD) reported limited impact of COVID-19 on CMEP activities, only delayed meetings. Despite the availability of a CMEP sustainability plan and exit strategy, sustainability discussions with CNM and other malaria stakeholders have not yet occurred and planned to be done in Y5. With regards to financial sustainability, domestic funding from the RGC will average $4 million per year between 2021- 2025, and the country is heavily depending on external funding. There is no new funding earmarked to cover all the gaps in the MEAF 2021-2025 proposed budget, even taking into account recent approved RAI3 GF grants. A simplified readiness assessment of OD capacities is recommended during the current fiscal year, which will be helpful to guide the progressive handover of some of the activities in advance. 5.2 RECOMMENDATIONS The ET considers that the first three recommendations to be crucial to maintain gains in malaria control and elimination and guide future investments. While the other recommendation presented are important, these are less critical. All sub-recommendations have been prioritized as high, medium, and low. 42 1. Expand and strengthen the implementation of the 1-3-7 evidence-based elimination model and geographic reach. CMEP’s implementation of the 1-3-7 strategy (case notification within one day, investigation within three days and response within seven days) in SPL and expansion in targeted ODs has proven feasible and achieved significant impact on reducing malaria cases and strengthening service provision. CMEP in collaboration with CNM and malaria stakeholders has been effectively adapting the interventions and approaches to the local context and malaria epidemiology. The 1- 3-7 elimination model has produced positive outcomes that are welcomed and accepted by affected communities, service providers, CNM managers, PHD and OD staff, and other malaria partners. High Priority • Continue reaching at-risk communities where they are by increased mobilization of VMWs, MMWs, and CSOs supporting access to malaria interventions using tailored and evidence￾based mixed approaches. • Focus on CMEP core activities, capacity building, surveillance, and supervision rather than entomologic surveillance activities. • All malaria partners should consider the inclusion of high-level technical malaria expertise on malaria elimination and prevention of reintroduction in the next malaria investments in Cambodia. Medium Priority • Continue supporting CSOs through the grants program to encourage locally-driven innovation to improve service provision to MMPs. Low Priority • Continue to support local- and central-level ownership working in collaboration with affected communities and other malaria stakeholders in the implementation of technically sound interventions. 2. Increase data analysis and information use for strategic decision-making. CMEP has a considerable amount of output and outcome data that need to be translated into strategic information. This could help not only Cambodia but also other countries in the Mekong subregion and other malaria epidemiologic scenarios in Latin America. There is a need to assess the effectiveness of some interventions in vivax malaria and the most effective implementation channels. CMEP’s challenges and achievements should be shared in a wider global health community. High Priority • Conduct an impact evaluation of the evidence-based malaria elimination model and case studies on each of the interventions implemented under CMEP to inform scale up. • Provide technical support in assessing the cost-effectiveness of malaria interventions for vivax malaria transmission only, for example reassess LLINs as a strategy in areas with outdoor biting, use of targeted mass drug administration, etc. 43 Medium Priority • Showcase CMEP achievements within USAID HQ and participate in webinars and talks with other stakeholders within the USG. • Increase dissemination of CMEP achievements in international meeting and scientific conferences in the United States. Low Priority • Participate actively in scientific annual conferences venues in South East Asia (i.e., Tropical Medicine conferences). 3. Review and harmonize malaria indicators and update the malaria profile. Additional to the global recommended indicators by the WHO Global Malaria Program, there is also a list of PMI indicators to monitor the malaria response. PMI recommends the use of the number of malaria cases (indigenous cases) as a better indicator for areas in the pre￾elimination/elimination spectrum (low/very low transmission) than the current indicator being used by CNM. There is a need to differentiate indigenous cases from imported to measure progress and impact. For example, the use of all cases (indigenous and imported) to classify API might overestimate malaria burden and misguide investments. The ET noted that API included in MEAF 1 and 2 included all malaria cases. Additionally, CMEP should establish a clear distinction between those areas with less and more than three years without indigenous malaria cases reported (considered a malaria-free area) in the target ODs. Furthermore, malaria profiles for each OD should be updated using indigenous cases by location of probable/likely infection and differentiate them from imported malaria cases (outside Cambodia); these updated profiles should guide the next investments on malaria elimination and/or prevention of reintroduction. High Priority • Conduct a rapid assessment of targeted OD profiles based on indigenous malaria cases only and identify malaria-free areas using MIS data. • Report on the API indicator disaggregating by using indigenous cases only as an alternative reporting in CMEP M&E Plan or use the number of indigenous malaria cases as recommended by PMI. Medium Priority • Continue supporting the MEAF 2 M&E framework in collaboration with malaria stakeholders, including the development of new tools, SOPs and technical strategies. Low Priority • Use information on malaria-free areas per targeted ODs as an advocacy tool in conjunction with all malaria partners and the RGC. 4. Strengthen the early detection and prompt response to upsurge of malaria cases. Malaria transmission has been significantly reduced in CMEP target ODs and the areas are now in the elimination spectrum. Vivax infections are now the most prevalent and the intense population movements contribute to malaria epidemics. CMEP has strengthened the epidemic 44 response and still plays a key role in the intensification plans 1-2; however, there is a need to differentiate and even predict some of the upsurges in vivax cases using data for action. High Priority • Provide technical support to differentiate the malaria profile among MMPs. • Conduct a rapid assessment of the main biological, social, and behavioral risk factors associated with malaria among MMPs, including identification of barriers for service provision. • Consider investigating the most cost-effective approaches to address vivax malaria outbreaks proposed in MEAF 2 including, for example, the use of targeted mass drug administration. Medium Priority • Continue providing technical support to PHD, OD, and HF staff and to VMWs, MMWs, PPs, and CSOs to fully manage any upsurge of malaria before transition following national normative guidance. Low Priority • Provide technical and financial support maintaining an stockpile of antimalarial drugs, RDTs, and LLINs as well as logistic field support. CMEP should continue supporting commodity forecasting, procurement, and distribution of commodities in a way that ensures extra (contingency) supplies at certain levels (national, OD, or HF) to avoid stock outs. 5. Support sustainable approaches for service delivery and system strengthening. The RGC is planning decentralization of health services and while this process is in the early stages, there is a need to maintain and continue supporting networks of service providers in malaria elimination activities. CMEP has been fully or partially supporting VMWs, MMWs, PPs, and CSOs working in and reaching out to the most hard-to-reach populations in the targeted ODs; however, more sustainable approaches are necessary. CMEP and malaria partners should assume that those networks will not be fully supported by local- and central-level governments and all stakeholders should work to identify more sustainable alternatives. This should include, for example, reviewing the number of trained service providers needed per OD per year as well as reducing the frequency of meetings from monthly to quarterly. Lessons from recent malaria outbreaks highlighted the huge risk they represent with the potential to erase all previous gains in the malaria elimination response. To address those outbreaks, local governments will need to be able to promptly and effectively respond to those epidemics to prevent deaths. In the current elimination scenario with more vivax infections and high mobilizations among MMPs, those outbreaks are likely to continue. The high turnover of PHD, OD, and HF staff still remains a challenge and CMEP should advocate for quality improvement linked to HCW accreditation processes and to continuing health education (CME) credits for the mix of health workers at all levels of the health system, also including other service providers such as PPs, VMWs, MMWs, and CSOs. Financial sustainability is also considered a high risk which should be addressed by providing technical support to local- and central-level policymakers in identifying multisectoral mechanisms for resource mobilization at all administrative levels. High Priority 45 • Share and reassess (if needed) the CMEP transition and exit strategy with CNM and malaria stakeholders. • Consider supporting sustainable country ownership of health systems reforms by assessing readiness for malaria elimination at the OD level in preparation for transition. • Continue to support the network of VMWs, MMWs, and PPs for malaria and assess their potential contribution to other health-related issues in the planning stages of decentralization. • Provide technical support to the RGC to explore novel alternatives for resource mobilization at the local and central levels, including multisectoral private and public organizations. Medium Priority • Reassess annual needs (HR) and coverage of VMWs and MMWs in hard-to-reach areas. • Continue support and engagement with potential CSOs working with hard-to-reach populations. Low Priority • Support HCW and service provider accreditation processes and continuing health education credits in the health system. • Continue communicating the malaria prevention of transmission and/or reintroduction (for malaria-free areas) mindset to communities, service providers, and policymaking managers using different advocacy and SBCC tools. 46 ANNEXES 47 ANNEX 1: STATEMENT OF WORK SECTION C – DESCRIPTION / SPECIFICATIONS/STATEMENT OF WORK C.1 BACKGROUND Over the last decade, many of Cambodia’s key health indicators have improved as the country’s economy has developed. Cambodia has also made huge progress in malaria prevention and control and is poised to move from control to elimination by 2025. Malaria deaths have decreased dramatically from 94 deaths in 2011 to 0 deaths reported in 2018. The country has seen a gradual decline in malaria since 2011, but experienced an increase in cases reported in 2017 and 2018. Despite these successes, the spread of artemisinin resistance and adequately reaching mobile populations remain challenges to achieving the national elimination goals. Geographically, malaria is endemic in 21 out of Cambodia’s 25 provinces. The incidence is highest in the Northeastern parts of the country and lower in the Western provinces. Peak malaria transmission is between July and November which is a transition period from the hot to the rainy season. Malaria predominantly impacts males 15-49 years of age (incidence rates in adult males are five times higher than adult females) while only 3.7 percent of cases are in children less than five years of age. Many of the affected men are mobile workers who move from low to high transmission areas and lack access to malaria services and education making them vulnerable to malaria infection. C.1.1 CMEP Award Information Project/Activity Title: Cambodia Malaria Elimination Project (CMEP) Award/Contract Number: AID-442-C-17-00001 Award/Contract Dates: October 26, 2016 through October 25, 2021 Project/Activity Funding: $16,543,857.00 Implementing Organization(s): University Research Co., LLC (URC) Project/Activity COR: Dr. Rida Slot C.1.2 CMEP Program Objectives The CMEP has four primary objectives: 1. Develop a scalable, evidence-based elimination model in Sampov Loun (SPL) and support its dissemination and replication for malaria elimination in Cambodia; 2. Support scale-up of high-quality malaria control and prevention interventions in five to eight additional Health Operational Districts (ODs), where gaps in coverage or quality exist; 3. Strengthen national malaria surveillance systems and M&E appropriate for malaria elimination and control activities; 4. Build capacity of Ministry of Health (MOH) to manage, intensify, and sustain malaria control and elimination efforts particularly at the OD level. This five-year award is intended to intensify malaria control and elimination activities by providing technical assistance and support to the National Center for Parasitology, Entomology, and Malaria 48 Control (CNM) to further develop, refine, and evaluate an evidence-based Model Elimination Package for malaria, which can then be implemented in other appropriate ODs. Under the award, the Contractor was tasked to develop a malaria elimination package and to pilot it initially in one OD, SPL in Battambang Province. During the course of the activity, the Model Elimination Package—with all of its tools, technical guidelines, manuals, standard operating procedures (SOPs), best practices, and lessons learned—will be evaluated and costed before being provided to CNM and other partners for adoption and onward implementation, as appropriate. In addition, during the first year of the project, the Contractor was tasked to work initially in five target ODs to bring malaria transmission levels down to a point where the activity can initiate malaria elimination activities using the tools and approaches developed in SPL. As outlined in the contract (section C.5 Program Objectives, Tasks, and Targets), each of the four objectives specifies tasks, deliverables, and expected results. The activity interventions must be conducted in accordance with the following principles: a. Support existing OD Office systems and partners. To the extent practicable, the Contractor will use the existing health system and strengthen the ability of public and private providers and NGOs to support malaria elimination activities. The Contractor will avoid setting up parallel, project-specific systems. This may also include collaborating with other partners who are strengthening systems that affect the delivery of malaria activities, including logistics, information systems, and other service delivery systems. b. Maintain linkages at national and provincial level. While the main focus of activities is at the OD level, the Contractor will also play a critical role in providing technical leadership at the national level. The Contractor must also collaborate closely with the MOH and the CNM in the planning and implementation of activities. In addition, Cambodia has many NGOs, donors, and partners. The Contractor’s, in addition to USAID’s, presence at the national level will assist President’s Malaria Initiative (PMI) with central level coordination of activities of all PMI-supported partners. c. Program reach. Implementation will be done in a phased approach. In Year 1, the Contractor will pilot and implement elimination activities in SPL OD in Battambang Province and malaria control activities in five (5) other ODs selected in collaboration with CNM and PMI. In consultation with PMI/USAID and CNM, the Contractor may be required to expand to as many as three (3) additional ODs by the third year of the project. The Contractor is expected to establish a presence in each OD although consolidation of management functions in regional offices may be used to decrease management costs and improve efficiency. d. Build capacity and sustainability. The Contractor is expected to provide technical leadership and lead implementation while building capacity and skills at the district, health facility, and community level without creating parallel management or reporting structures. The intent is to equip districts to take over ownership and implementation of all malaria control activities and all elimination activities by the end of the CMEP award implementation. e. Gender considerations. The group that is at highest risk of morbidity and mortality related to malaria consists of male mobile/migrant workers that are in search of employment in the forests and plantations located in the west of the country. The contractor must 49 develop a gender-sensitive communication strategy and targeted messages to reach these workers (and their families) when they arrive in the OD in search of employment and while they are employed. The contractor will also support the targeted ODs to distribute long-lasting insecticidal nets (LLINs) to mobile/migrant workers and to the resident populations that reside within two kilometers of forested areas to increase LLIN ownership and use as stated in task under Objective 1 in section C.5.1 of the award. Gender also plays a critical role in determining access to treatment and care for malaria and use of preventative measures such as LLINs. Populations working in forested areas but residing in households of permanent dwellers may be the source of infection to women and children who reside in the village. Gender norms and behaviors influence access to LLINs within the home if mosquito nets are insufficient to cover every sleeping space. Power dynamics within the household further impact the ability to seek preventative care and treatment. The contractor shall address gender-related barriers to prevention and treatment in all of its interventions. f. Coordination with other donors and partners. The CMEP implementing partner will coordinate with other implementing partners and Malaria district teams in the target ODs on activities supported by other donors/partners (e.g., Global Fund, Asian Development Bank, World Health Organization, and the Bill & Melinda Gates Foundation) to avoid overlapping activities or duplicate funding. C.1.3 CMEP Geographical Coverage In Year 1, the project was implemented in six (6) ODs: SPL, Battambang (BTB), Maung Russey (MRS), Thamar Kaul (TMK), Phnom Kravanh (PKV), and Krakor (KRK). Beginning in Year 2, the project expanded to a total of nine (9) ODs, including Pailin (PLN), Bakan (BKN), and Sampov Meas (SPM). Three transitional ODs graduated to elimination ODs (BTB, MRS, and TMK). The CMEP project is currently being implemented in nine (9) ODs in three (3) provinces (Pursat, Battambang, and Pailin). C.2 STATEMENT OF WORK C.2.1 Purpose of the Evaluation The primary purpose of this mid-term performance evaluation is to assess activity implementation and performance to date, identify key bottlenecks and constraints, and make actionable recommendations for improvements needed to meet the activity’s intended objectives. The findings will be used to provide feedback to USAID/Cambodia, the PMI, the national malaria program, partners, and relevant stakeholders on the successes of the activity, areas for improvement, and any course corrections required for the remainder of the project. The evaluation will also help to inform program design for future awards. C.2.2 Purpose, Audience, and Application The primary audience of this evaluation is the USAID/Cambodia Office of Public Health and Education (OPHE) and the interagency PMI headquarters team. The secondary audiences will be the national malaria program of Cambodia, the implementing partners of CMEP, and other stakeholders and donors working towards malaria elimination in Cambodia. 50 C.2.3 Evaluation Questions 1. To what extent has the CMEP project developed a scalable, evidence-based elimination model in SPL and supported its widespread dissemination and replication in other target ODs in Cambodia? 2. To what extent has the CMEP project supported high-quality malaria control and prevention interventions in five ODs and expanded transitional ODs, where gaps in coverage or quality may exist? Please consider the extent to which and how the activity has achieved and maintained universal coverage of at-risk populations with proven vector control and case management interventions. 3. Does the current national malaria surveillance system serve its intended purpose to detect, immediately notify, investigate, respond, and support appropriate for malaria elimination and control activities? Please elaborate the extent of CMEP involvement and any gaps required to improve the current system if any. 4. To what extent has the CMEP project built capacity of CNM/MOH to manage, intensify, and sustain malaria control and elimination efforts, particularly at the Provincial Health Department (PHD) and OD levels? 5. What factors, both internal and external of CMEP, have facilitated or constrained project performance and how can these factors be addressed? a. Measures that have been put in place to contribute to or promote sustainability. Factors that still need to be addressed. b. The efficiency and effectiveness of the design and project management arrangements and oversight between the prime recipient and sub-partners (and sub-grantees) for achieving project objectives. C.2.4 Evaluation Approach and Methodology The evaluation team must develop an evaluation approach and methodology to answer the proposed evaluation questions. The evaluation team must use applied mixed-methods (qualitative and quantitative), as appropriate. The evaluation team must provide the rationale behind the methods selected, why they are appropriate, any known methodological/data limitations, and how they provide rigor to answering the evaluation questions. The evaluation team is expected to engage with a range of stakeholders, including but not limited to relevant Royal Government of Cambodia, PMI headquarters, USAID/Cambodia, development partners, and implementing partner counterparts. The approach and methods proposed by the evaluation team are subject to USAID/Cambodia’s approval as part of the evaluation work plan. USAID/Cambodia will make the following documents such as work plans, Monitoring, Evaluation, and Learning (MEL) plans, progress reports, technical reports, Data Quality Assessment reports, available to the evaluation team before commencement of the award to inform the evaluation design and planning. These documents are intended for use only by the evaluation team and not to be shared with any third parties, unless otherwise instructed by USAID/Cambodia. Data must be disaggregated by sex, as relevant. The evaluation team must take into account gender issues when analyzing data and identify any potential gender inequality pertaining to each evaluation question as appropriate. 51 Data Collection and Analysis USAID/Cambodia requests that the evaluator complete the following table as part of its detailed design and evaluation plan. Please note that another format may be used if the table is not preferred, but any chosen format should contain all the information specified for each question. Evaluation Question Data Source Data Collection Method (including sampling methodology, where applicable) Data Analysis Method C.3 EVALUATION SCHEDULE 52 C.4 DELIVERABLES AND REPORTING REQUIREMENTS C.4.1 Evaluation Work Plan Within 14 calendar days (two weeks) of the effective date of the task order, the Contractor will submit the proposed Evaluation Work Plan to the Task Order Contracting Officer’s Representative (TOCOR). The Evaluation Work Plan must include: a. The anticipated schedule for the evaluation and logistical arrangements; b. Proposed submission date of each deliverable in accordance with the schedule included in section IV; c. The evaluation team members, including names of key personnel and proposed enumeration team composition, delineated by roles and responsibilities; and d. Evaluation design including: 1. A detailed evaluation design matrix that links the evaluation questions in the SOW to data sources, methods, and the analysis plan; 2. All data collection instruments or their main features; 3. The list of potential interviewees and organizations; and 4. Known limitations to the evaluation design. USAID/Cambodia will provide consolidated comments on the Evaluation Work Plan through the TOCOR within seven (7) calendar days (one week) of receipt of it. The Contractor must submit a revised Evaluation Work Plan to the TOCOR for approval within seven (7) calendar days (one week) of receipt of comments. The Contractor must have an approved Evaluation Work Plan in place before proceeding with field work. C.4.2 In-Briefing The evaluation team is required to have an in-brief with USAID/Cambodia in person to discuss the team’s understanding of the evaluation, initial assumptions, evaluation questions, methodology, and work plan, and to answer questions the team may have. This must happen before the fieldwork begins. 53 C.4.3 Field Work – Quantitative and Qualitative Data Collection: The evaluation team must complete all field work activities within four weeks following the approval of the Evaluation Work Plan by the TOCOR. It is expected that the team will be able to finish actual field work within three (3) weeks and they have one more week for data verification if needed. C.4.4 Regular Updates on the Field Work The evaluation team leader must brief the TOCOR on progress made with the field work on a weekly basis, in person or by electronic communication in an informal report to be sent via email, agreed upon in advance. Any delays or complications must be communicated to the TOCOR as early as possible to allow for quick resolution and to minimize any disruptions to the evaluation. C.4.5 Exit Briefings The evaluation team must hold an exit briefing at the USAID Mission in Phnom Penh after field work and prior to leaving the country. The debriefings should include initial findings from the evaluation through a PowerPoint presentation, including a discussion of key evaluation findings only, with preliminary recommendations for possible modifications to program approaches, results, or activities for discussion with USAID/Cambodia staff. The evaluation team must consider USAID/Cambodia and the CMEP implementing partner comments received during the exit briefings and incorporate them appropriately when drafting the evaluation report. C.4.6 Draft Evaluation Report The draft evaluation report must be consistent with the guidance provided in Section C.6 Evaluation Report Format. The report must address each of the questions identified in the SOW and any other issues the evaluation team considers having a bearing on the objectives of the evaluation. The Contractor must submit the draft Evaluation Report to the TOCOR within 49 calendar days (seven (7) weeks) following the approval of the Evaluation Work Plan. Once the draft evaluation report is submitted, the TOCOR will disseminate the draft report to USAID/Cambodia and to the CMEP implementing partner to review and comment on. USAID/Cambodia will provide comments on the report within 14 calendar days (two weeks) following the receipt of the draft report (one set of comments from USAID/Cambodia and another from the implementing partner). C.4.7 Final Evaluation Report and Cleaned Data The evaluation team must address/incorporate all comments/suggestions and submit the Final Report to the TOCOR within 14 calendar days (two weeks) of receipt of comments from USAID on the draft report. All data and records must be submitted in electronic form in an easily readable format, organized and documented for use by those not fully familiar with this evaluation. C.5 EVALUATION TEAM COMPOSITION The evaluation team must be composed of an Evaluation Team Leader, a National Evaluation Advisor, and a Logistics/Program Assistant. Either the Evaluation Team Leader or the National Evaluation Advisor must have experience in gender analysis in order to ensure integration of gender in responding to each evaluation question as appropriate. The evaluation team must consist of members of both sexes. 54 C.5.1 Evaluation Team Leader This is a senior-level evaluation advisor who will lead the evaluation team to design the evaluation, develop the evaluation plan, interact with USAID/Cambodia and stakeholders, implement the evaluation, including qualitative data collection and analysis, and handle and submit all required deliverables. The Team Leader will have appropriate educational level and work experience in conducting similar health-related performance evaluations. Qualifications: ∉ The education and experience requirements must meet the senior level as defined in Section B.7.3 “Minimum Qualifications” of the IDIQ contract. ∉ Minimum of 8 years of experience in public health, which included experience in implementation of malaria programs, with a Ph.D., or 10 years of experience with a JD/ABD, or 12 years of experience with a MS/MA. ∉ Post-graduate degree in public health, epidemiology, statistics, or another relevant field. ∉ Demonstrated experience leading health sector project/program evaluation/assessments, utilizing both quantitative and qualitative methods. ∉ Demonstrated experience in public health and malaria programming, program design, and program evaluation. ∉ Demonstrated skills in planning, facilitation, and consensus building. ∉ Excellent interpersonal skills, including experience successfully interacting with host government officials, civil society partners, and other stakeholders. ∉ Demonstrated skills in project management. ∉ Demonstrated organizational skills and ability to keep to a timeline. ∉ Proficiency in written and spoken English is required, with demonstrated report writing experience. ∉ Experience working in Asia and/or Southeast Asia is required, and experience in Cambodia is highly desirable. ∉ Familiarity with USAID policies and practices: o Evaluation policy o Results frameworks o Performance monitoring plans C.5.2 National Evaluation Advisor The National Evaluation Advisor will support the Evaluation Team Leader in designing, planning, and implementing the evaluation, including quantitative and qualitative data collection and analysis. In addition to that, s/he will ensure contextual relevance of the design and implementation of the evaluation. Qualifications: ● At least 5 years of experience in M&E procedures (program monitoring, evaluation, and/or research) and implementation. ● Master’s Degree in Public Health, Statistics, Epidemiology, Social Studies, or another related field. ● Experience assessing, monitoring, and/or evaluating health programs—malaria programs in Cambodia is preferred. ● Experience in design and implementation of evaluations and/or assessments. 55 ● Strong knowledge, skills, and experience in qualitative and quantitative analytic tools. ● Experience implementing and coordinating others to implement surveys, key informant interviews, focus groups, observations, and other evaluation and assessment methods that assure reliability and validity of the data. ● Experience in data management. ● Able to analyze quantitative and qualitative data, which will be primarily descriptive statistics, cross-tabulations, and triangulation. ● Experience using statistical analytic software. ● Able to review, interpret, and reanalyze as needed existing data pertinent to the evaluation. ● Demonstrated data interpretation and presentation skills. ● Experience working in Asia and/or Southeast Asia is required, and experience in Cambodia is highly desirable. ● Proficient in written and spoken English. ● Local hire. ● Demonstrated experience writing evaluation and/or assessment reports. ● Familiarity with USAID M&E policies and practices: o Evaluation policies o Results frameworks o Performance monitoring plans C.5.3 Logistics/Program Assistant The Logistics/Program Assistant will support the Assessment Team with all logistics and administration to allow them to carry out this evaluation. S/he will also serve as translator/interpreter, as needed. S/he will work under the guidance of the Assessment Team Leader to make preparations, arrange meetings and appointments. S/he will conduct programmatic administrative and support tasks as assigned and ensure the processes moves forward smoothly. S/he may also be asked to assist in translation of data collection tools and transcripts, if needed. Qualifications: ∉ Knowledge of key actors in the health sector and their locations including MOH, donors, and other stakeholders. ∉ Able to efficiently liaise with hotel staff, arrange in-country transportation (ground and air), arrange meeting and workspace as needed, and ensure business center support, e.g., copying, Internet, and printing. ∉ Local hire. ∉ Good command of English and Khmer. C.6 EVALUATION REPORT FORMAT The final evaluation report must include: ∉ An abstract; ∉ Executive summary; ∉ Background of the local context and the activity being evaluated; ∉ Evaluation purpose and evaluation questions; ∉ Evaluation methodology and limitations; and 56 ∉ Findings, conclusions, and recommendations. For more detail, see “How-To Note: Preparing Evaluation Reports” (https://www.usaid.gov/sites/default/files/documents/1870/How-to-Note_Preparing-Evaluation￾Reports.pdf) and A Mandatory Reference for ADS Chapter 201 (https://www.usaid.gov/ads/policy/200/201mah). Further guidance is provided in the Evaluation Toolkit (https://usaidlearninglab.org/library/evaluation-report-template). The executive summary must be 2-5 pages in length and summarize the purpose, background of the project being evaluated, main evaluation questions, methods, findings, conclusions, recommendations, and lessons learned. The evaluation methodology must be explained in detail in the report. Limitations to the evaluation must be disclosed in the report, with particular attention to the limitations associated with the evaluation methodology. The annexes to the report must include: ∉ Evaluation SOW; ∉ Detailed description of the evaluation design and methods; ∉ Statements regarding significant unresolved differences of opinion by funders, implementers, and/or members of the evaluation team, if any; ∉ All data collection and analysis tools used in conducting the evaluation, such as surveys, interview questions, checklists, and discussion guides; ∉ All sources of information, properly identified and listed (including documents reviewed, sites visited, and key informants, assuming they gave permission to be identified); ∉ Signed disclosure of any conflict of interest forms for all evaluation team members, by including a statement by evaluation team members that attest to a lack of conflict of interest or describes existing conflicts of interest relative to the project being evaluated; and ∉ Summary information about evaluation team members, including qualifications, experience, and role on the team. In accordance with ADS 201.3.5.18(B) Evaluation Utilization, the Contractor will make the final evaluation report publicly available through the Development Experience Clearinghouse within 90 calendar days of the approval by USAID. C.7 CRITERIA TO ACCESS EVALUATION QUALITY USAID will evaluate the quality of evaluation report based on the Mandatory Reference for ADS Chapter 201, Criteria to Ensure the Quality of the Evaluation Report, as sited in this link (https://www.usaid.gov/ads/policy/200/201maa). The draft and final evaluation reports will be evaluated against the following criteria to ensure the quality of the evaluation report. ∉ Evaluation reports should represent a thoughtful, well-researched, and well-organized effort to objectively evaluate the strategy, project, or activity. ∉ Evaluation reports should be readily understood and should identify key points clearly, distinctly, and succinctly. ∉ The Executive Summary of an evaluation report should present a concise and accurate statement of the most critical elements of the report. 57 ∉ Evaluation reports should adequately address all evaluation questions included in the SOW, or the evaluation questions subsequently revised and documented in consultation and agreement with USAID. ∉ Evaluation methodology should be explained in detail and sources of information properly identified. ∉ Limitations to the evaluation should be adequately disclosed in the report, with particular attention to the limitations associated with the evaluation methodology (selection bias, recall bias, unobservable differences between comparator groups, etc.). ∉ Evaluation findings should be presented as analyzed facts, evidence, and data and not based on anecdotes, hearsay, or simply the compilation of people’s opinions. ∉ Findings and conclusions should be specific, concise, and supported by strong quantitative or qualitative evidence. ∉ If evaluation findings assess person-level outcomes or impact, they should also be separately assessed for both males and females. ∉ If recommendations are included, they should be supported by a specific set of findings and should be action-oriented, practical, and specific. See the Evaluation Report Review Checklist from the Evaluation Toolkit (https://usaidlearninglab.org/library/evaluation-report-checklist-and-review-template) for additional guidance. END OF SECTION C 58 ANNEX 2: CMEP MID-TERM EVALUATION SCHEDULE March 2020 Sunday Monday Tuesday Wednesday Thursday Friday Saturday 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 Kick-Off meeting with USAID 24 25 Mission CO/COR receive technical approach and contract modification request 26 27 28 29 30 New technical approach, schedule, and contract modification approved by CO 31 Doc review/ Analysis design/Prepare Work Plan April 2020 Sunday Monday Tuesday Wednesday Thursday Friday Saturday 1 Doc review/ Analysis design/Prepare Work Plan 2 Doc review/ Analysis design/Prepare Work Plan 3 Doc review/ Analysis design/Prepare Work Plan 4 5 6 Doc review/ Analysis design/Prepare Work Plan 7 Doc review/ Analysis design/Prepare Work Plan 8 Doc review/ Analysis design/Prepare Work Plan 9 Doc review/ Analysis design/Prepare Work Plan 10 Doc review/ Analysis design/Prepare Work Plan 11 12 13 Submit draft Work Plan 14 15 16 17 18 19 20 Schedule stakeholder meetings 21 Schedule stakeholder meetings 22 Schedule stakeholder meetings 23 Schedule stakeholder meetings 24 Schedule stakeholder meetings 25 26 27 Receive USAID feedback 28 Incorporate feedback 29 Incorporate feedback 30 Incorporate feedback 59 May 2020 Sunday Monday Tuesday Wednesday Thursday Friday Saturday 1 Incorporate feedback 2 3 4 Submit Final Work Plan 5 Finalize and translate tools 6 Finalize and translate tools 7 Finalize and translate tools 8 Final Work Plan approved 9 10 11 In-Brief with USAID 12 Training of local ET/Data Quality/Pre-test 13 Training of local ET/Data Quality/Pre-test 14 Training of local ET/Data Quality/Pre-test 15 Training of local ET/Data Quality/Pre-test 16 Training of local ET/Data Quality/Pre-test 17 18 19 20 21 22 23 Field testing/Review tools 24 25 Data collection 26 Data collection 27 Data collection 28 Data collection 29 Data collection 30 Data collection 31 June 2020 Sunday Monday Tuesday Wednesday Thursday Friday Saturday 1 Data collection 2 Data collection 3 Data collection 4 Data collection 5 Data collection 6 Data collection 7 8 Data collection 9 Data collection 10 Data collection 11 Data collection 12 Data collection 13 Data collection 14 15 Data collection 16 Data collection 17 Data collection 18 Data collection 19 Data collection 20 Data collection 21 22 Data analysis and develop presentation 23 Data analysis and develop presentation 24 Data analysis and develop presentation 25 Data analysis and develop presentation 26 Out-Brief with USAID 27 28 29 Data analysis and report writing 30 Data analysis and report writing 60 July 2020 Sunday Monday Tuesday Wednesday Thursday Friday Saturday 1 Data analysis and report writing 2 Data analysis and report writing 3 Data analysis and report writing 4 5 6 Data analysis and report writing 7 Data analysis and report writing 8 Data analysis and report writing 9 Data analysis and report writing 10 Data analysis and report writing 11 12 13 Data analysis and report writing 14 Data analysis and report writing 15 Data analysis and report writing 16 Data analysis and report writing 17 Submit Draft Evaluation Report 18 19 20 21 22 23 24 25 26 27 28 29 30 31 Receive USAID Comments August 2020 Sunday Monday Tuesday Wednesday Thursday Friday Saturday 1 2 3 Incorporate comments 4 Incorporate comments 5 Incorporate comments 6 Incorporate comments 7 Incorporate comments 8 9 10 Incorporate comments 11 Incorporate comments 12 Incorporate comments 13 Incorporate comments 14 Submit Final Evaluation Report 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 61 ANNEX 3: EVALUATION DESIGN MATRIX EQs Additional Questions Data Sources Data Collection Methods Analysis Methods 1. To what extent has the CMEP project developed a scalable, evidence￾based elimination model in SPL and supported its widespread dissemination and replication in other target ODs in Cambodia? 1.1. To what extent have the project activities contributed to achieving the project objectives and results? Where the targets achieve on the annual basis? If not, what were the main challenges and how were there managed? 1.2. To what extent would these results have occurred without the project? Why? 1.3. What were/are the main challenges or obstacles in terms of achieving program outcomes, and how have they been addressed? 1.4. How sustainable are CMEP results? Why? 1.5. Are there aspects of the program that could have been done better (what/how)? 1.6. What was the performance of the malaria elimination model on case detection, notification, classification of cases, and response? If not, what were the main bottlenecks? 1.7. Were malaria interventions executed on time (as planned)? 1.8. How was the compliance with the 1-3-7 approach? 1.9. To what extent has the annual parasite incidence (API) changed over the course of the model implementation progress? What were the main challenges in the different ODs? how does that compare to other similar regions, or to the national change? National legislative framework, relevant national policies Analytical reports and assessments Reporting documentatio n M&E data Program staff and stakeholders Independent industry experts Desk study: Review of project documents, legal/ policy framework, analytical materials, statistical data Semi-structured interviews and/or group discussions with all identified stakeholders and industry experts Systematic document review Qualitative thematic analysis Discursive analysis Comparative analysis of actual vs. planned results (disaggregated by activities, ODs, and target groups) Observational data analysis 62 EQs Additional Questions Data Sources Data Collection Methods Analysis Methods 2. To what extent has the CMEP project supported high-quality malaria control and prevention interventions in five ODs and expanded transitional ODs, where gaps in coverage or quality may exist? Please consider the extent to which and how the activity has achieved and maintained universal coverage of at-risk populations with proven vector control and case management interventions. 2.1. How relevant are the program’s objectives with regard to the national malaria priorities? 2.2. Which critical areas are not covered by the program? 2.3. Were all key stakeholders considered by the program? What, if any, duplication of activities was identified during implementation? If yes, how was the correction measures implemented? 2.4. How many ODs have implemented the package of intervention considered in the elimination model? What is the selection process for the ODs? 2.5. How is the coordination with other stakeholders responsible for procurement and distribution of malaria commodities? 2.6. To what extent has the CMEP project benefited key and vulnerable populations? 2.7. What were the main gaps in intervention coverage addressed by CMEP? 2.8. What is the level of attainment of the targets (achieved vs. proposed) at the OD level? 2.9. What is the level of Government support (for malaria interventions) at the national and local level (ODs) for the malaria program? 2.10. To what extent has CMEP strengthen capacity for malaria interventions (case management, vector control, SBCC, etc.)? National legislative framework, relevant national policies Analytical reports and assessments Reporting documentatio n M&E data Program staff, stakeholders, and beneficiaries Desk study: Review of project documents, legal/ policy framework, analytical materials, statistical data Semi-structured interviews and/or group discussions with all identified stakeholders and industry experts Systematic document review Qualitative thematic analysis Descriptive statistical analysis Gap analysis Comparative analysis of actual vs. planned results (disaggregated by activities, ODs, and target groups) 63 EQs Additional Questions Data Sources Data Collection Methods Analysis Methods 3. Does the current national malaria surveillance system serve its intended purpose to detect, immediately notify, investigate, respond, and support appropriate for malaria elimination and control activities? Please elaborate the extent of CMEP involvement and any gaps required to improve the current system, if any. 3.1. Aside from the CMEP, which other projects or initiatives provide assistance in malaria surveillance principles and practices? These may include, for example, projects or initiatives funded by USAID, other donors or international organizations, or by your government. Please describe this assistance. 3.2. What coordination mechanisms exist in the national malaria program at the local and/or national levels? What is coordinated and how (information sharing, join events, division of interventions, etc.)? 3.3. How could the coordination between these projects and initiatives be improved? 3.4. What gaps remain in improving the national malaria surveillance system? How could project assistance be improved to fill these gaps? 3.5. Is the national malaria surveillance system able to report real-time? What are the main challenges? 3.6. To what extent the completeness/timeless of surveillance data is reported as planned? 3.7. Is the information from the surveillance system being used to stratify/re-stratify ODs? Please describe the process. 3.8. Are surveillance data regularly analyzed? At what level (PDH, ODs, Central) does the analysis occur? What are the dissemination channels used for decision-making? 3.9. Does the malaria surveillance integrate other malaria data? For example, are the entomology and drug resistant data integrated in the national surveillance system? At what level does the integration occur? 3.10. What are key lessons learnt in strengthening the surveillance system? What best practices have been introduced? Program staff, stakeholders’ interviews Representativ es of other donor-funded initiatives and international organizations Independent industry experts Reporting documentatio n M&E data Semi-structured interviews/group discussions Analysis of project documents and M&E data Revision of documents of donor￾funded initiatives and international organizations Systematic document review Qualitative thematic analysis Descriptive statistical analysis Comparative analysis of actual vs. planned results (disaggregated by activities, ODs, and target groups) Observational data analysis 64 EQs Additional Questions Data Sources Data Collection Methods Analysis Methods 4. To what extent has the CMEP project built capacity of CNM/MOH to manage, intensify, and sustain malaria control and elimination efforts, particularly at the PHD and OD levels? 4.1. What has the CMEP achieved in terms of capacity￾building? Were the annual targets achieved in Y1, Y2, and Y3? 4.2. What gaps remain in improving malaria elimination knowledge and capacity? How could project assistance be improved to fill these gaps? 4.3. What are the primary challenges in applying the improved knowledge or capacity acquired from CMEP project assistance? 4.4. What were the main challenges in building capacities of the CNM and PHD/OD staff? Are those challenges different? 4.5. To what extent has the CMEP contributed to the health sector capacity-building? 4.6. Who was responsible for the selection of key malaria capacity-building? What is the mechanism of approval of specific trainings involving new tools or approaches? Which factors were critical for decision-making? 4.7. In the first three years, how successful has CMEP project been in supervisory capacity-building? What are the main challenges in the malaria coordination, management and leadership at the central level and at the PHD/OD level? 4.8. What have been the main lessons learned while building capacities at the PHD/OD level? CMEP project documents Project staff and stakeholders’ interviews Independent industry experts Reporting documentatio n M&E data Analysis of project documents and M&E data Semi-structured interviews and/or group discussions Observational data analysis Qualitative thematic analysis Descriptive statistical analysis Comparative analysis of actual vs. planned results (disaggregated by activities, ODs, and target groups) 65 EQs Additional Questions Data Sources Data Collection Methods Analysis Methods 5. What factors, both internal and external of CMEP, have facilitated or constrained project performance and how can these factors be addressed? a. Measures that have been put in place to contribute to or promote sustainability. Factors that still need to be addressed. b. The efficiency and effectiveness of the design and project management arrangements and oversight between the prime recipient and sub-partners (and sub-grantees) for achieving project objectives. 5.1. What hindered or enhanced performance during the last three years of implementation of CMEP? 5.2. How did the project ensure cost-effectiveness? Could you think about more cost-effective options? 5.3. How sustainable are CMEP project-supported activities? Why? Could you recommend more sustainable alternatives? 5.4. Is there any exit strategy for the CMEP project? 5.5. What are the main malaria capacity constraints related to financial or human resources (HR) at the PHD/OD level? 5.6. To what extent does the CMEP project collaborate and coordinate with other implementing partners? 5.7. How has the CMEP project been affected by external factors? For example, reliance on other partners for delivery of key services (i.e., acquisition of malaria commodities, etc.). 5.8. How is COVID-19 pandemic affecting the implementation of activities? What are the prevention/safety measures considered to protect staff, health workers, and key populations? 5.9. Do you expect delays in the implementation of all activities? Which key interventions would be more affected than others? 5.10. Is there any national malaria sustainability plan in Cambodia? National legislative framework, relevant national policies Analytical reports and assessments Reporting documentatio n M&E data Program staff and stakeholders Independent industry experts Desk study: Review of project documents, legal/ policy framework, analytical materials, statistical data Semi-structured interviews and/or group discussions with all identified stakeholders and industry experts Revision of documents of donor￾funded initiatives and international organizations Qualitative thematic analysis Discursive analysis Comparative analysis of actual vs. planned results (disaggregated by activities, ODs, and target groups) Observational data analysis 66 ANNEX 4: DOCUMENTS REVIEWED 1. ACT watch annual report 2013 2. Ariey F, Witkowski B, Amaratunga C, Beghain J, Langlois A-C, et al. (2014) A molecular marker of artemisinin-resistant Plasmodium falciparum malaria. Nature 505: 50–55. 3. Artemisinin Resistant Malaria in the Greater Mekong Sub-region: Options for the Global Fund, The Global Fund, December 2012. 4. Bannister-Tyrrell M, Krit M, Sluydts V, et al. Households or Hotspots? Defining Intervention Targets for Malaria Elimination in Ratanakiri Province, Eastern Cambodia. J Infect Dis. 2019. 5. Bourdier et al., Malaria and population dynamics in Cambodia ethnographic investigations in three remote areas (Pailin, Samlaut andTrapaeng Prasat). 6. Cambodia Malaria Elimination Action Framework 2016-2020; 2021-2025. 7. Cambodia President's Malaria Initiative MOP 2016-2020 8. Canavati SE, Lawford HL, Fatunmbi BS, Lek D, Top-Samphor N, Leang R, et al. Establishing research priorities for malaria elimination in the context of the emergency response to artemisinin resistance framework-the Cambodian approach. Malar J. 2016;15:120. 9. CMEP Annual Reports Y1-Y3 10. CMEP Annual workplans Y1-Y4 11. CMEP contract modifications 1-8 12. CMEP M&E Plan 2016; revised 2018 13. CMEP Operational Research list of activities 2017-2020 14. CMEP technical strategies 15. CNM Malaria annual reports 2018, 2019. 16. CNM Malaria Information System, malaria morbidity indicators October 2016-May 2020 17. CNM Malaria Intensification Plan 1 and 2 18. CNM Mobile & Migrant Population in the context of Malaria Elimination, Operational Manual, 2018. 19. CNM National Treatment Guidelines for Malaria in Cambodia, December 2014 20. CNM. Cambodia Malaria Survey 2013. 21. Cox J, Sovannaroth S, Soley LD, Ngor P, Mellor S, Roca-Feltrer A. Novel approaches to risk stratification to support malaria elimination: an example from Cambodia. Malar J. 2014;13(1):1–10. https://doi.org/10.1186/1475-2875-13-371. 67 22. Cui L, Yan G, Sattabongkot J, Cao Y, Chen B, et al. (2012) Malaria in the Greater Mekong Subregion: Heterogeneity and complexity. Acta Tropica 121: 227–239. 23. Disease Surveillance for Malaria Control: An Operational Manual, WHO, 2012. 24. Feng J, Liu J, Feng X, Zhang L, Xiao H, Xia Z. Towards malaria elimination: monitoring and evaluation of the “1-3-7” approach at the China-Myanmar border. Am J Trop Med Hyg. 2016;95(4):806–10. 25. Final Report CAP-Malaria: Burma, Cambodia, Thailand and Regional, 2016. 26. Global Fund New Funding Mechanism funding requests and annexes 2014-2016; 2021- 2022 27. Global Fund, Mitigating the impact of COVID-19 on countries affected by HIV, Tuberculosis and Malaria, 2020. 28. Guyant et al., Malaria and the mobile and migrant population in Cambodia: a population movement framework to inform strategies for malaria control and elimination. Malaria J. 2015 14:252 29. Guyant P, Canavati SE, Chea N, et al. Malaria and the mobile and migrant population in Cambodia: a population movement framework to inform strategies for malaria control and elimination. Malar J. 2015;14(1):1–15. 30. Imwong et al., The spread of artemisinin-resistant Plasmodium falciparum in the Greater Mekong Subregion: a molecular epidemiology observational study Lancet Infect Dis 2017 31. Institut Pasteur du Cambodge. Analyse of blood spot samples collected in Pailin for evidence of cytochrome b status (normal/mutant). 2013. 32. Kheang ST, Lin MA, Lwin S, et al. Malaria case detection among Mobile populations and migrant Workers in Myanmar: comparison of 3 service delivery approaches. Glob Heal Sci Pract. 2018;6(2):384–9. 33. Laxminarayan R. Act now or later? Economics of malaria resistance. Am J Trop Med Hyg. 2004;71(2 Suppl):187-95. 34. Long-lasting Insecticidal Nets for Malaria Prevention: A Manual for Malaria Programme Managers, WHO, 2007. 35. Lu G, Liu Y, Beiersmann C, Feng Y, Cao J, Müller O. Challenges in and lessons learned during the implementation of the 1-3-7 malaria surveillance and response strategy in China: a qualitative study. Infect Dis Poverty. 2016; 5(1):1–11. https://doi.org/10.1186/s40249-016-0188-8. 36. Malaria Elimination: A Field Manual for Low and Moderate Endemic Countries. Global Malaria Program, WHO, 2007. 68 37. Maude RJ, Nguon C, Ly P, Bunkea T, Ngor P, Canavati de la Torre SE, et al. Spatial and temporal epidemiology of clinical malaria in Cambodia 2004-2013. Malar J. 2014;13:385. 38. Mekong Malaria III, Southeast Asian Journal of Tropical Medicine and Public Health, Vol 44, Sup 1, 2013. 39. Methods for Surveillance of Antimalarial Drug Efficacy, WHO, 2009. 40. Mid-term Review of Mekong Malaria Partners Activities FY2010 in the GMS: Activities and Progress Made from Oct 2010 - April 2011, RDMA, April 2011. 41. Mon A, Kyaw M, Kathirvel S, et al. “Alert-audit-act”: assessment of surveillance and response strategy for malaria elimination in three low endemic settings of Myanmar in 2016. Trop Med Health. 2018;46:1–9. 42. Parker DM, Tripura R, Peto TJ, Maude RJ, Nguon C, Chalk J, et al. A multi-level spatial analysis of clinical malaria and subclinical Plasmodium infections in Pailin Province, Cambodia. Heliyon. 2017;3(11):e00447. 43. Rapid Assessment Reports, Tak, Trat and Ranong, Thailand, URC, 2012. 44. Regional Action Plan for Malaria Control and Elimination in the Western Pacific (2010- 2015), WHO Western Pacific Regional Office, 2010. 45. Sen ZS, Sen ZS, Zhang L, et al. China’s 1-3-7 surveillance and response strategy for malaria elimination: is case reporting, investigation and foci response happening according to plan? Infect Dis Poverty. 2015;4(1):1–9. 46. Shafique, M and George, S (2014) Positive deviance: An asset-based approach to improve malaria outcomes www.malariaconsortium.org/learningpapers. 47. Singhasivanon P. Mekong malaria. Malaria, multi-drug resistance and economic development in the greater Mekong subregion of Southeast Asia. Southeast Asian J Trop Med Public Health. 1999;30 Suppl 4:i-iv, 1-101. 48. Strategy to address migrant and mobile populations for malaria elimination in Cambodia, 2013. 49. Suaya JA, Shepard DS, Chang MS, Caram M, Hoyer S, Socheat D, et al. Cost-effectiveness of annual targeted larviciding campaigns in Cambodia against the dengue vector Aedes aegypti. Trop Med Int Health. 2007;12(9):1026-36. 50. The National Strategic Plan For Elimination of Malaria in the Kingdom of Cambodia 2011– 2025. 2011. http://www.cnm.gov.kh/userfiles/file/N Strategy Plan/National Strategy in English.pdf. 51. Tin-Oo P et al. Gender, mosquitoes and malaria: implications for community development programmes in Laputta, Myanmar. Southeast Asian Journal of Tropical Medicine and Public Health, 32(3):588-594. 69 52. University Research Co. Project fact sheet: CAP-Malaria/Cambodia. 2016, 2018, 2020 53. USAID Health Policy Project. Health financing profile: Cambodia 2016. 54. Van Eijk AM, Ramanathapuram L, Sutton PL, et al. What is the value of reactive case detection in malaria control? A case-study in India and a systematic review. Malar J. 2016;15(1):1–13. 55. Wang D, Chaki P, Mlacha Y, et al. Application of community-based and integrated strategy to reduce malaria disease burden in southern Tanzania: the study protocol of China-UK￾Tanzania pilot project on malaria control. Infect Dis Poverty. 2019;8(1):4–9. 56. Wang D, Cotter C, Sun X, Bennett A, Gosling RD, Xiao N. Adapting the local response for malaria elimination through evaluation of the 1-3-7 system performance in the China￾Myanmar border region. Malar J. 2017;16(1):1–10. 57. WHO, IOM, Population Mobility and Malaria, 2017 58. World Health Organization, Global Malaria Programme. A Framework for Malaria Elimination.; 2017. 59. World Health Organization. Entomological surveillance for malaria elimination in Cambodia. Phnom Penh, Cambodia; 2017. 60. World Health Organization. Malaria Country Profile: Cambodia. Geneva; 2016-2019 61. World Health Organization. Strategy for malaria elimination in Greater Mekong Subregion (2015-2030) Phnom Penh, Cambodia; 2015. 70 ANNEX 5: LIST OF PARTICIPANTS IN FOCUS GROUP DISCUSSIONS, MINI￾SURVEY, AND KEY INFORMANT INTERVIEWS Numbers of FGD Participants by Target Group and Gender Target Female Male Total At-risk Population 4 27 31 Battambang 3 3 Krakor 1 5 6 Pailin 3 3 Phnom Kravanh 3 13 16 Sampov Loun 3 3 Service Provider 17 14 31 Battambang 4 3 7 Krakor 1 4 5 Pailin 4 5 9 Phnom Kravanh 3 2 5 Sampov Loun 5 5 Grand Total 21 41 62 List of Mini Survey Participants by Target Group, OD, and Gender Target Female Male Total At-risk population 45 69 114 Battambang 3 1 4 Pailin 11 8 19 Phnom Kravanh 9 9 18 Sampov Loun 17 34 51 Sampov Meas 5 17 22 Service provider 29 60 89 Battambang 8 8 Krakor 7 13 20 Pailin 9 6 15 Phnom Kravanh 5 9 14 Sampov Loun 6 19 25 Sampov Meas 2 5 7 Grand Total 74 129 203 71 List of Key Informant Interviews by Target and Gender Target Participant's Name Position/Role Sex CMEP Sorn Sophal CMEP Team Leader Female CMEP Pheakdey Finance Manager Female HFSPP Ms. Ke Saoran Service Provider Female HFSPP Men Phalla OPD – Health Center Female HFSPP Phim Sinoeun OPD – Health Center Female HFSPP Som Sotheary Service Provider Female HFSPP Ms. Chuon Sophea Krakor Health Center Deputy Director Female HFSPP Long Sinat Lab staff of Soun Komar HC Female HFSPP Loeur Pirun Health Center Director Female HFSPP Mom Bopha Phal Medicine Store Manager Female HFSPP Soth Pov Health Center Director Female HFSPP Seur Sophon Birth Spacing Unit Staff Female HFSPP Keo Ravy Service Provider Female HFSPP Say Chanra OPD in Sya HC Female HFSPP Khoun Sokean PPM Female PHDOD Ieng Vichida Technical Chief Female PHDOD Dr. Horm Lida Child Health Manager Female PHDOD Aing Soklin Medicine Store Manager Female PHDOD Mol Sinat MIS Officer Female PHDOD Dr. Thavrin Vice Director Female PHDOD Doch Chanthy Midwife – Health Care Manager Female PHDOD Um Sarith Malaira Program Manager Female CMEP Alexandra Fisher CMEP Project Coordinator Female IP Sara Canavatti Business Development Manager Female IP Yucheng Tsai Malaria Associate Female IP Moul Vanna Executive Director AHEAH Female USG Lenna Neat Arango Team Leader Infectious Diseases USAID Cambodia Female USG Rida Slot Project Management Specialist Female USG Jimee Hwang CDC HQ Backstop Mekong Region Female CMEP Krung Siv CMEP Team Leader Male CMEP Kun Sidavy CMEP Team Leader Male CMEP Mang Say CMEP Team Leader Male CMEP Chun Phally CMEP Team Leader Male CMEP Dr. Sokomar Technical Male HFSPP Pheurn Van PPM Male HFSPP Heng Venhong Bak Chen Chien Health Center Director Male HFSPP Hean Kim Nath Service Provider Male HFSPP Chan Bunna Health Center Director Male HFSPP E. Lim Sim Service Provider Male HFSPP Ny Sinoeun Health Center Vice Director Male HFSPP Son Long Service Provider Male HFSPP Khuy Samnang Health Center Director Male HFSPP Srun Theany Prognil Health Center Acting Director Male HFSPP Tep Siya Health Center Director Male HFSPP Tum Puthea Phlov Meas Health Center HF Male HFSPP Duem Te Malaria Program Manager Male HFSPP Sambuk Vuthy Trengh Health Center HF Male HFSPP Chin Earng Communicable Decease Unit Staff Male HFSPP Heav Samnang Service Provider Male HFSPP Kroch Sokhom OPD – Health Center Male 72 Target Participant's Name Position/Role Sex HFSPP Chheoy Sophal Service Provider Male HFSPP Thoeun Kosal OPD – Health Center Male HFSPP Noun Norin PPM Male HFSPP Teouk Da Service Provider Male HFSPP Moa Seangdy Health Center Director Male HFSPP Nem Choun Nimol OPD – Health Center Male HFSPP Phou Born Health Center Director Male HFSPP Nguon Sarik Health Center Director Male HFSPP Khut Sokhoeun Serei Meanchey Health Center, Malaria Program Officer Male HFSPP Khim Samoun Serei Meanchey Health Center Director Male HFSPP Chim Sameth Health Center Director Male HFSPP Nou Prathna Service Provider Male HFSPP Tiem Sambo Service Provider Male HFSPP Sim Vathana Sampov Luon Health Center Director Male HFSPP Tim Vanna Sampov Luon Health Center Staff Male HFSPP Mr. Len Luogn Kroko Health Center Director Male HFSPP Dr. Hor Choeung PPM Male HFSPP Khun Sen Trang Health Center Director Male HFSPP Chak Touch Lab staff of Trang HC Male HFSPP Soy Rieng Service Provider Male HFSPP Seng Eak Service Provider Male HFSPP Soun Seng PPM Male HFSPP Sok Sam Pon PPM Male HFSPP Ny Thavy PPM Male HFSPP Ke Khorn Service Provider Male HFSPP Choun Sokcheat PPM Male HFSPP Som Lay Drug Repository Officer Male HFSPP You Sro PPM Male HFSPP Bun ngeam Bak Chen Chien Health Center Director Male IP Chao Sophea Executive Director Male PHDOD Mr. Kim A Chief of Drug Repository Male PHDOD Iem Samnang Malaria Program Manager Male PHDOD Chheng Charlyheng HIS Officer Male PHDOD Um Bunthy Chief of Technical in charge of malaria program, Sampove Loun OD Male PHDOD Kong Sovann MIS Officer Male PHDOD Po Ly Vice Chief of Technical Bureau and Head of VMW and PPM Unit Male PHDOD Tho Sochanta CNM Vice-Director and Chief of Entomology Unit Male PHDOD Dr. Mao Sovan Vice Pursat Referral Hospital Male PHDOD Lek Dysoley CNM Vice-Director Case Management and Operation Research Male PHDOD Dr. Kuy Dy PHD Director Male PHDOD Dr. Chan Davoeng Malaria Program Manager Male PHDOD Dr. Sok Ratha OD Director Male PHDOD Dr. Nop Sopina RH Director Male PHDOD Sam Vat Rin Admin/Data Entry Male PHDOD Nuo Sophanak Malaria Program Manager Male PHDOD Tang Chihuot Deputy Director of Malaria Program Male PHDOD Tep Phalla OD Vice Director Male PHDOD Vann Van Technical staff Male 73 Target Participant's Name Position/Role Sex PHDOD Chim Tha Technical staff Male PHDOD Koeun Viro Child Health Care Male PHDOD Dr. Tol Bunkea Epidemiology Unit Manager Male PHDOD Pao Manut SPM OD director Male PHDOD Yos Pov OD Director Male PHDOD Tat Veivath BTB OD Deputy Director Male PHDOD Dr. Yok Sovan Vice PHD Male PHDOD Muy Moravy Chief of Accountant Male PHDOD Haong Chhei Chief of Drug Repository Male PHDOD Sles You PHD Director Male PHDOD Meas Mony OD staff in charge of HMIS Male PHDOD Mey Dyno OD Director Male PHDOD Kim Sokhan Deputy Director of Pailin PHD Male PHDOD Phon Vuthy Technical staff Male PHDOD Khiev Patt HIS Officer Male CMEP Sharon Thangadurai Chief of Party CMEP Male CMEP Paul Bacon M&E Strategic Information Male CMEP Ranjith De Alwis Deputy Chief of Party Male IP Lawrence Barat Senior Technical Advisor PMI Impact Malaria Project PSI Male IP Chy Say Medical Officer, Malaria Unit Male IP David Papworth Regional Country Director PSM TO2 Male IP Benoit Witkowski Head Malaria Unit Male USG Sory But Bugget Officer CMEP Male USG Michael Thigpen Malaria Resident Adviser Male USG John E. Gimnig Research Entomologist Male Target Female Male Total CMEP 3 8 11 HFSPP 13 45 58 IP 3 5 8 PHDOD 7 33 40 USG 3 3 6 Grand Total 29 94 123 74 ANNEX 6: ILLUSTRATIVE DATA COLLECTION TOOLS INTERVIEW GUIDE FOR CMEP TEAM 75 76 77 INTERVIEW GUIDE FOR STAKEHOLDERS Public Health Districts/Operational Districts 78 79 80 INTERVIEW GUIDE FOR STAKEHOLDERS USAID, PMI and CDC 81 82 INTERVIEW GUIDE FOR STAKEHOLDERS Implementing & Technical Partners: WHO, Global Fund, UNOPS, PSI, AFRIMS, CSO, Chemonics, CHAI, and Others 83 84 INTERVIEW GUIDE Village Malaria Workers and Mobile Malaria Workers 85 86 INTERVIEW GUIDE Health Facilities’ Staff and Private Providers 87 88 MINI-SURVEY Malaria Patients and At-Risk Populations at OD Levels Who Have Benefited from CMEP Interventions 89 90 MINI-SURVEY Malaria Service Providers at OD Levels Who Have Benefited From CMEP Interventions 92 93 94 95 FOCUS GROUP DISCUSSION 96 ឧបសម�័ន�៦: ឧបករណ៍ស្រមាប់្របមូលទិន�ន័យ េសចក�ីែណនោំសមា� សន៍ស្រមាប់្រក� មការងោរ CMEP 97 98 99 100 101 េសចក�ីែណនោំសមា� សន៍ស្រមាប់អ�កពាក់ព័ន� មណ� លសុខភាពសាធារណៈ្រស� ក/្រស� ក្របតិបត�ិ 102 103 104 105 េសចក�ីែណនោំសមា� សន៍ស្រមាប់អ�កពាក់ព័ន� USAID, PMI និង CDC 106 107 108 109 េសចក�ីែណនោំសមា� សន៍ស្រមាប់អ�កពាក់ព័ន� ៃដគូបេច�កេទសនិងអនុវត�: WHO, Global Fund, UNOPS, PSI, AFRIMS, CSO, Chemonics, CHAI, and others 110 111 112 113 េសចក�ីែណនោំសមា� សន៍ស្រមាប់អ�កពាក់ព័ន� អ�កេធ��រ ើ ស�័្រគចិត�ជំង្រ ឺគន� �ញ់ និងអ�កេធ�ើ�រចលតជ័ ងំ ឺ្រគន� �ញ់ 114 115 116 េសចក�ីែណនោំសមា�សន៍ ស្រ�ប់បគុ �លិកស�ុ ភិ�ល និងអ�កផ�ល់េស�ឯកជន 117 118 119 មីនី េសើេវ អ�កផ�ល់េស�ជងំ ឺ្រគន� �ញ់េ�ក្រមិត្រសក� ្របតិបត�ិែដល�អ�កទទួលផល�� ល់ពីកិច�អន��គមន៍របស់គេ្រ�ង CMEP 120 121 122 123 124 125 126 127 128 មីនី េសើេវ អ�កផ�ល់េស�ជងំ ឺ្រគន� �ញ់េ�ក្រមិត្រសក� ្របតិបត�ិែដល�អ�កទទួលផល�� ល់ពីកិច�អន��គមន៍របស់គេ្រ�ង CMEP 129 130 131 132 133 ANNEX 7: BIOS AND SUMMARY INFORMATION OF TEAM MEMBERS Team Leader, Dr. Leopoldo Villegas, M.D., D.T.M.&H., M.Sc., Dr.P.H., Adv.D.P.H.M., is a global health expert with over 25 years of experience implementing and evaluating malaria and infectious/tropical diseases programs worldwide. He has worked on all aspects of malaria control and elimination—diagnosis, treatment, control, implementation science, M&E, targeting interventions, financial investments, and others—for a range of international donors, including the World Bank, USAID, WHO, GF, and others. Has served as Evaluations Expert/Team Leader in several evaluations primarily in malaria and infectious diseases sector including integration with HIV, tuberculosis, and strengthening of health services in developing countries on three continents. He has extensive experience working for USAID￾funded malaria evaluations. Dr. Villegas holds five degrees, three of them (D.T.M.&H., M.Sc., and Dr.P.H.) from the London School of Hygiene and Tropical Medicine. National Evaluation Advisor, Mr. Young Pheak, is a Cambodian national with over 10 years of experience in M&E focused in the areas of social protection and gender equality and public health, among others. He has proven expertise in the field of M&E and has worked on the design, implementation, and management of a range of evaluation-related programs. He has conducted baseline, mid-term, and final evaluations of donor-funded projects across various regions of Cambodia for a variety of clients including USAID, World Bank, European Union (EU), WHO, etc. He has experience in conducting evaluations, participating in five evaluations of USAID-funded projects in the last five years. Mr. Pheak holds an M.D.M. from Norton University, Cambodia, and a B.A. in Psychology from the Royal University of Phnom Penh. Logistic/Program Assistant, Ms. Bopha Kong, LLM, is a Cambodian national with extensive experience providing data collection and translation support for research and evaluation in Cambodia. She holds a master’s degree in Primate Law from the Royal University of Law and Economics. Malaria Advisor, Ms. Debra Prosnitz, MPH, PMP, is a gender advisor and global health and development professional with over ten years of experience in malaria, integrated community case management (iCCM), community health, program design, monitoring, evaluation, and surveillance (SME), health information systems, data quality assessment, capacity building, and the intersection of climate and health. She managed the monitoring and evaluation of the World Health Organization’s Rapid Access Expansion (RAcE) program and is the technical lead for PMI Measure Malaria core activities. Ms. Prosnitz has an MPH from Emory University and a BA from Smith College. Data Analyst, Andrew Andrada, MSPH, is a surveillance, monitoring, and evaluation (SME) specialist with over five years of experience in conducting country malaria surveillance system assessments and malaria program evaluations. He provides SME technical assistance to malaria￾endemic countries through the PMI Measure Malaria project. He has also conducted trainings and managed national surveys in sub-Saharan Africa and South Asia. Mr. Andrada holds an MSPH from Johns Hopkins School of Public Health. 134 ANNEX 8: CMEP PERFORMANCE INDICATOR TRACKING TABLE (PITT) No. Indicator Data Source Disaggregation – Baseline Value FY 2017 FY 2018 FY 2019 % Change Baseline FY 2019* Target (6 ODs) Actual Target Actual FY 2018 Target Actual FY 2019 Impact 1 Annual Parasite Incidence (API) per 1,000 population (disaggregated by species and OD and sex) - HMIS web-based (Population, Cases reported by HF) - MIS web-based (Cases reported by VMW) Overall 3.68 3.32 4.65 2.25 10.62 1.47 6.09 65 By species Pf 2.32 2.04 2.41 1.16 3.12 0.58 0.73 -69 Pv 1.20 1.14 2.07 1.01 7.20 0.85 5.29 340 Mixed 0.17 0.14 0.17 0.09 0.30 0.04 0.07 -57 PM NA NA NA NA NA NA NA By OD and Species Sampov Loun 1.82 1.45 0.88 1.10 1.07 0.77 0.56 -69 Pf 0.83 0.62 0.26 0.42 0.14 0.21 0.01 -99 Pv 0.92 0.77 0.61 0.65 0.92 0.55 0.55 -40 Mixed 0.07 0.05 0.01 0.04 0.01 0.02 0.00 -100 PM NA NA NA NA NA NA NA Battambang 1.57 1.23 0.92 0.92 1.20 0.62 0.50 -68 Pf 0.69 0.52 0.30 0.35 0.24 0.17 0.03 -96 Pv 0.77 0.63 0.59 0.52 0.95 0.42 0.47 -39 Mixed 0.11 0.08 0.03 0.06 0.02 0.03 0.00 -100 PM NA NA NA NA NA NA NA Thmar Koul 0.26 0.21 0.13 0.15 0.27 0.11 0.16 -40 Pf 0.11 0.08 0.06 0.06 0.05 0.03 0.00 -96 Pv 0.12 0.10 0.06 0.08 0.20 0.07 0.15 27 Mixed 0.03 0.02 0.01 0.02 0.02 0.01 0.00 -100 PM NA NA NA NA NA NA NA 127 No. Indicator Data Source Disaggregation – Baseline Value FY 2017 FY 2018 FY 2019 % Change Baseline FY 2019* Target (6 ODs) Actual Target Actual FY 2018 Target Actual FY 2019 Maung Russei 1.23 0.96 1.41 0.70 2.30 0.46 1.28 4 Pf 0.65 0.49 0.69 0.33 0.50 0.16 0.10 -84 Pv 0.50 0.41 0.64 0.34 1.75 0.28 1.17 133 Mixed 0.08 0.06 0.08 0.04 0.04 0.02 0.01 -88 PM NA NA NA NA NA NA NA Kravanh 26.85 20.81 30.82 14.96 120.71 9.28 63.74 137 Pf 18.44 13.83 16.77 9.22 38.07 4.61 8.17 -56 Pv 7.49 6.29 13.27 5.28 79.07 4.44 54.72 631 Mixed 0.92 0.69 0.78 0.46 3.57 0.23 0.85 -7 PM NA NA NA NA NA NA NA Krakor 15.99 12.79 20.32 9.61 37.22 6.42 23.20 45 Pf 11.40 8.55 11.01 5.70 10.28 2.85 2.87 -75 Pv 3.82 3.67 8.31 3.52 25.90 3.38 20.08 426 Mixed 0.77 0.58 1.01 0.39 1.04 0.19 0.24 -69 PM NA NA NA NA NA NA NA Bakan 1.53 N/A N/A 0.90 1.09 0.56 1.61 5 Pf 1.02 N/A N/A 0.51 0.18 0.26 0.05 -95 Pv 0.37 N/A N/A 0.32 0.88 0.27 1.56 321 Mixed 0.14 N/A N/A 0.07 0.04 0.04 0.01 -95 PM NA NA NA NA NA NA NA Sampov Meas 2.17 N/A N/A 1.40 2.28 0.96 3.04 40 Pf 1.15 N/A N/A 0.58 0.47 0.29 0.36 -69 Pv 0.99 N/A N/A 0.81 1.74 0.67 2.63 166 128 No. Indicator Data Source Disaggregation – Baseline Value FY 2017 FY 2018 FY 2019 % Change Baseline FY 2019* Target (6 ODs) Actual Target Actual FY 2018 Target Actual FY 2019 Mixed 0.03 N/A N/A 0.02 0.06 0.01 0.05 76 PM NA NA NA NA NA NA NA Pailin 1.28 N/A N/A 0.89 0.62 0.65 0.75 -41 Pf 0.46 N/A N/A 0.23 0.00 0.12 0.04 -91 Pv 0.78 N/A N/A 0.64 0.61 0.52 0.71 -9 Mixed 0.04 N/A N/A 0.02 0.01 0.01 0.00 -100 PM NA NA NA NA NA NA NA By sex Male 3.31 2.98 4.15 2.02 9.76 1.32 5.57 68 Female 0.38 0.34 0.50 0.23 0.86 0.15 0.52 37 2 Number of severe cases per 100,000 per mid-year population (disaggregated by OD and sex) - HMIS web-based (Severe case in IPD in HO2) Overall 1.70 1.45 1.28 0.85 1.50 0.43 0.51 -70 By OD Sampov Loun 5.54 4.16 0.59 2.77 2.88 1.39 0.00 -100 Battambang 2.15 1.61 2.11 1.08 3.37 0.54 1.28 -40 Thmar Koul 1.30 0.98 0.00 0.65 0.00 0.33 0.00 -100 Maung Russei 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0 Kravanh 0.00 0.00 1.11 0.00 2.17 0.00 0.00 0 Krakor 2.23 1.67 5.47 1.12 3.21 0.56 2.10 -6 BTB Provincial Hospital 0.34 0.26 0.67 0.17 0.84 0.09 0.00 -100 PST Provincial Hospital 6.91 5.18 3.14 3.46 2.62 1.73 0.64 -91 Bakan 0.73 N/A N/A 0.37 0.00 0.18 0.00 -100 Sampov Meas 0.00 N/A N/A 0.00 0.00 0.00 0.00 NA 129 No. Indicator Data Source Disaggregation – Baseline Value FY 2017 FY 2018 FY 2019 % Change Baseline FY 2019* Target (6 ODs) Actual Target Actual FY 2018 Target Actual FY 2019 Pailin 2.99 N/A N/A 1.50 0.00 0.75 1.48 -51 By sex Male 1.36 1.12 0.94 0.75 0.91 0.37 0.45 -67 Female 0.34 0.33 0.34 0.22 0.59 0.11 0.06 -82 3 Test positivity rate - HMIS web-based (Cases/tests reported by HF) - MIS web-based (Cases/test reported by VMW) Overall 25% 20% 25% 18% 24% (16285/67387) 14% 10% (9494/92515) -60 By OD Sampov Loun 5% 4% 3% 4% 1% (185/13304) 3% 1% (98/12634) -80 Battambang 14% 12% 7% 10% 4% (464/11791) 8% 2% (195/12170) -86 Thmar Koul 11% 9% 5% 8% 5% (64/1233) 6% 3% (38/1479) -73 Maung Russei 21% 18% 18% 15% 15% (483/3271) 12% 6% (273/4307) -71 Kravanh 42% 36% 49% 29% 45% (11117/24612) 23% 25% (5991/24057) -40 Krakor 51% 43% 46% 36% 33% (3480/10512) 28% 12% (2212/19167) -76 Bakan 24% N/A N/A 17% 12% (155/1288) 13% 7% (233/3577) -71 Sampov Meas 46% N/A N/A 32% 43% (295/687) 25% 12% (402/3450) -74 130 No. Indicator Data Source Disaggregation – Baseline Value FY 2017 FY 2018 FY 2019 % Change Baseline FY 2019* Target (6 ODs) Actual Target Actual FY 2018 Target Actual FY 2019 Pailin 15% N/A N/A 11% 6% (42/689) 8% 0% (52/11674) -100 Objective 1: Develop a scalable, evidence-based elimination model in SPL and support its widespread dissemination and replication in other target ODs 1 % of suspected malaria cases receiving a parasitological test (disaggregated by type of provider, sex and age group) - HMIS web-based (Cases/tests reported by HF) - MIS web-based (Cases/tests reported by VMW) Overall 98% 99% 100% 100% 99% 100% 98% 0 By provider type Public HFs 100% 100% 100% 100% 100% 100% 100% 0 VMWs/MMWs 100% 100% 100% 100% 100% 100% 100% 0 Private providers 94% 97% 99% 100% 93% 100% 5% -95 By sex (only public HFs) Male 100% 100% 100% 100% 100% 100% 100% 0% Female 100% 100% 100% 100% 100% 100% 100% 0% By age group (only public HFs) <5 100% 100% 100% 100% 100% 100% 100% 0% 5-14 100% 100% 100% 100% 100% 100% 100% 0% ≥15 100% 100% 100% 100% 100% 100% 100% 0% By OD Sampov Loun 98% 99% 100% 100% 99% 100% 97% -1 Battambang 100% 100% N/A 100% 98% 100% 97% -3 Thmar Koul 100% 100% N/A 100% 95% 100% 100% 0 Maung Russei 100% 100% N/A 100% 100% 100% 100% 0 Pailin 100% N/A N/A 100% 100% 100% 100% 0 Bakan 100% N/A N/A N/A N/A N/A N/A NA Sampov Meas 100% N/A N/A N/A N/A N/A N/A NA 131 No. Indicator Data Source Disaggregation – Baseline Value FY 2017 FY 2018 FY 2019 % Change Baseline FY 2019* Target (6 ODs) Actual Target Actual FY 2018 Target Actual FY 2019 2 % of all confirmed cases receiving the appropriate malaria treatment under DOT according to the NTGs (disaggregated by type of provider, species, sex, and age group) - MIS/1-3-7 database (1-3-7 data for 5 elimination ODs) Overall 77.14% (270/350) 85% 86% (87/101) 91% 74% (509/688) 92% 87%(570/656) 13 By provider type Public HFs NA 75% 83% (59/71) 90% 72% (304/424) 91% 87%(395/453) 5 VMWs/MMWs NA 95% 93% (28/30) 95% 85% (173/203) 95% 86%(175/203) -8 Private providers NA NA NA 95% 52% (32/61) 95% NA NA By species Pf NA 87% 88% (28/32) 92% 85% (68/80) 92% 82%(32/39) -6 Pv NA 81% 87% (58/67) 85% 72% (434/600) 90% 87%(536/615) 7 Mixed NA 87% 50% (1/2) 92% 88% (7/8) 92% 100%(2/2) 15 PM NA NA NA NA NA NA NA By sex Male NA 85% 87% (81/93) 90% 73% (453/617) 91% 87%(526/605) 0 Female NA 85% 75% (6/8) 90% 79% (56/71) 91% 86%(44/51) 1 By age group 132 No. Indicator Data Source Disaggregation – Baseline Value FY 2017 FY 2018 FY 2019 % Change Baseline FY 2019* Target (6 ODs) Actual Target Actual FY 2018 Target Actual FY 2019 <5 NA 85% No case (0/0) 90% 100% (3/3) 91% 50%(1/2) 5-14 NA 85% 83% (5/6) 90% 74% (31/42) 91% 76%(26/34) -8 ≥15 85% 86% (82/95) 90% 74% (475/643) 91% 88%(543/620) 2 By OD Sampov Loun 78% (220/281) 85% 86% (87/101) 91% 83% (154/185) 92% 98%(96/98) 26 Battambang 64% (32/50) 85% N/A 91% 60% (120/201) 92% 87%(169/195) 36 Thmar Koul NA 85% N/A 91% 69% (20/29) 92% 97%(37/38) Maung Russei 95% (18/19) 85% N/A 91% 79% (196/247) 92% 82%(223/273) -14 Pailin NA N/A N/A 91% 73% (19/26) 92% 87%(45/52) 19 Bakan N/A N/A N/A N/A N/A N/A N/A Sampov Meas N/A N/A N/A N/A N/A N/A N/A 3 % of all diagnosed malaria cases are reported to the appropriate responding team within one day (24 hours) (disaggregated by type of provider, sex, and age group) - MIS/1-3-7 database (1-3-7 data for 5 elimination ODs) Overall 91.81% (258/281) 95% 100% (101/101) 98% 69% (474/688) 100% 87%(568/656) -5 By provider type Public HFs 89.18% (165/185) 93% 100% (71/71) 100% 67% (285/424) 100% 83%(377/453) -7 VMWs/MMWs 96.87% (93/96) 97% 100% (30/30) 100% 76% (155/203) 100% 94%(191/203) -7 Private providers NA NA NA 93% 56% (34/61) 100% NA 133 No. Indicator Data Source Disaggregation – Baseline Value FY 2017 FY 2018 FY 2019 % Change Baseline FY 2019* Target (6 ODs) Actual Target Actual FY 2018 Target Actual FY 2019 By sex Male 93.67% (237/253) 95% 100% (93/93) 98% 68% (422/617) 100% 86%(523/605) -9 Female 75% (21/28) 95% 100% (8/8) 98% 73% (52/71) 100% 88%(45/51) 17 By age group <5 No case (0/0) 95% No case (0/0) 98% 67% (2/3) 100% 50%(1/2) 5-14 63% (5/8) 95% 100% (6/6) 98% 79% (33/42) 100% 91%(31/34) 44 ≥15 93% (253/273) 95% 100% (95/95) 98% 68% (439/643) 100% 86%(536/620) -8 By OD Sampov Loun 91.81% (258/281) 95% 100% (101/101) 98% 91% (169/185) 100% 97%(95/98) 6 Battambang NA N/A N/A 98% 75% (151/201) 100% 94%(183/195) 25 Thmar Koul NA N/A N/A 98% 100% (29/29) 100% 97%(37/38) -3 Maung Russei NA N/A N/A 98% 48% (119/247) 100% 77%(209/273) 60 Pailin NA N/A N/A 98% 23% (6/26) 100% 85%(44/52) 270 Bakan NA N/A N/A N/A N/A N/A N/A Sampov Meas NA N/A N/A N/A N/A N/A N/A 4 % of all confirmed malaria cases are investigated, - MIS/1-3-7 database (1-3-7 data for 5 elimination ODs) Overall 95.01% (267/281) 96% 100% (101/101) 100% 59% (404/688) 100% 84%(552/656) -12 By provider type 134 No. Indicator – classified (local versus imported), and reported within 3 days of diagnosis (disaggregated by type of provider, sex, and age group) Data Source Disaggregation Baseline Value FY 2017 FY 2018 FY 2019 % Change Baseline FY 2019* Target (6 ODs) Actual Target Actual FY 2018 Target Actual FY 2019 Public HFs 94.05% (174/185) 96% 100% (71/71) 100% 64% (270/424) 100% 86%(390/453) -9 VMWs/MMWs 96.87% (93/96) 96% 100% (30/30) 100% 50% (101/203) 100% 80%(162/203) -13 Private providers NA 96% 100% 54% (33/61) 100% NA By sex Male 96.83% (245/253) 96% 100% (93/93) 100% 59% (361/617) 100% 84%(509/605) -13 Female 78.57% (22/28) 96% 100% (8/8) 100% 61% (43/71) 100% 84%(43/51) 7 By age group <5 No case (0/0) 96% No case (0/0) 100% 100% (3/3) 100% 50%(1/2) NA 5-14 63% (5/8) 96% 100% (6/6) 100% 55% (23/42) 100% 79%(27/34) 25 ≥15 96% (262/273) 96% 100% (95/95) 100% 59% (378/643) 100% 85%(524/620) -11 By OD Sampov Loun 95.01% (267/281) 96% 100% (101/101) 100% 94% (173/185) 100% 100%(98/98) -11 Battambang NA N/A N/A 100% 49% (98/201) 100% 89%(173/195) -11 Thmar Koul NA N/A N/A 100% 79% (23/29) 100% 100%(38/38) -11 Maung Russei NA N/A N/A 100% 43% (105/247) 100% 73%(199/273) -11 Pailin NA N/A N/A 100% 19% (5/26) 100% 85%(44/52) -11 135 No. Indicator Data Source Disaggregation – Baseline Value FY 2017 FY 2018 FY 2019 % Change Baseline FY 2019* Target (6 ODs) Actual Target Actual FY 2018 Target Actual FY 2019 Bakan NA N/A N/A N/A N/A N/A N/A NA Sampov Meas NA N/A N/A N/A N/A N/A N/A NA 5 % of confirmed active foci investigated in which an appropriate response was initiated within 7 days (disaggregated by type of provider, sex, and age group) - MIS/1-3-7 database (1-3-7 data for 5 elimination ODs) Overall 100% 100% 100% 100% NA 100% NA NA By provider type Public HFs 100% 100% 100% 100% NA 100% NA NA VMWs/MMWs 100% 100% 100% 100% NA 100% NA NA By sex Male 100% 100% 100% 100% NA 100% NA NA Female 100% 100% 100% 100% NA 100% NA NA By age group <5 100% 100% No case (0/0) 100% NA 100% NA NA 5-14 100% 100% 100% 100% NA 100% NA NA ≥15 100% 100% 100% 100% NA 100% NA NA By OD Sampov Loun 100% 100% 100% 100% NA 100% NA NA Battambang NA N/A N/A 100% NA 100% NA NA Thmar Koul NA N/A N/A 100% NA 100% NA NA Maung Russei NA N/A N/A 100% NA 100% NA NA Pailin NA N/A N/A 100% NA 100% NA NA Bakan NA N/A N/A N/A N/A N/A N/A NA Sampov Meas NA N/A N/A N/A N/A N/A N/A NA 136 No. Indicator Data Source Disaggregation – Baseline Value FY 2017 FY 2018 FY 2019 % Change Baseline FY 2019* Target (6 ODs) Actual Target Actual FY 2018 Target Actual FY 2019 6 % of registered private providers received supervision at least twice during the last 12 months -PPM supervision report (Project database) Overall NA 100% 100% (32/32) 100% 97% 100% 78% -22 By OD Sampov Loun NA 100% 100% (32/32) 100% 92% 100% 96% -4 Battambang NA N/A N/A 100% 99% 100% 80% -9 Thmar Koul NA N/A N/A 100% 100% 100% 92% -8 Maung Russei NA N/A N/A 100% 98% 100% 48% -51 Pailin NA N/A N/A 100% NA 100% 70% NA Bakan NA N/A N/A N/A N/A N/A N/A NA Sampov Meas NA N/A N/A N/A N/A N/A N/A NA Objective 2: Support high quality malaria control and prevention interventions 1 % of suspected malaria cases that received a parasitological test (disaggregated by type of provider, sex and age group) - HMIS web-based (Cases/tests reported by HF) - MIS web-based (Cases/tests reported by VMW) Overall 100% 100% 99% 100% 100% 100% 100% 0% By provider type Public HFs 100% 100% 100% 100% 100% 100% 100% 0 VMWs/MMWs 100% 100% 100% 100% 100% 100% 100% 0 Private providers NA 100% 93% 100% 98% 100% 54% 0 By sex (only public HFs) Male 100% 100% 100% 100% 100% 100% 100% 0 Female 100% 100% 100% 100% 100% 100% 100% 0 By age group (only public HFs) 137 No. Indicator Data Source Disaggregation – Baseline Value FY 2017 FY 2018 FY 2019 % Change Baseline FY 2019* Target (6 ODs) Actual Target Actual FY 2018 Target Actual FY 2019 <5 100% 100% 100% 100% 100% 100% 100% 0 5-14 100% 100% 100% 100% 100% 100% 100% 0 ≥15 100% 100% 100% 100% 100% 100% 100% 0 By OD Kravanh 100% 100% 99% 100% 100% 100% 99% -1 Krakor 100% 100% 99% 100% 100% 100% 100% 0 Bakan 100% N/A N/A 100% 100% 100% 100% 0 Sampov Meas 100% N/A N/A 100% 100% 100% 100% 0 2 % of population living in targeted, at risk villages who had fever in the last two weeks and had a diagnostic test for malaria according to national policy within 24 hours of the onset of fever (disaggregated by sex) -Cambodia Malaria Survey (CMS) Overall 89.7% (Domain 1,CMS) NA(CMS) NA(CMS) 90% NA(CMS) NA By sex Male NA NA(CMS) NA(CMS) NA NA(CMS) NA Female NA NA(CMS) NA(CMS) NA NA(CMS) NA By OD Kravanh NA NA(CMS) NA(CMS) NA NA(CMS) NA Krakor NA NA(CMS) NA(CMS) NA NA(CMS) NA Bakan NA N/A NA(CMS) NA NA(CMS) NA Sampov Meas NA N/A NA(CMS) NA NA(CMS) NA 3 % of all confirmed cases received the appropriate malaria treatment under DOT according to the NTGs - Quarterly Technical Supervision (TS) report to HF and PPs Overall 97% (614/635) 98% 99% (744/754) 100% 99% (1815/1826) 100% 97.7% (1783/1825) 1 By OD Kravanh 95% (407/427) 97% 99% (365/369) 100% 100% (934/935) 100% 97.9% (821/839) 3 138 No. Indicator – (disaggregated by OD, type of provider, sex, and age group) Data Source Disaggregation Baseline Value FY 2017 FY 2018 FY 2019 % Change Baseline FY 2019* Target (6 ODs) Actual Target Actual FY 2018 Target Actual FY 2019 Krakor 99% (207/208) 99% 99% (228/231) 100% 99% (523/529) 100% 97.6% (527/540) -1 Bakan NA N/A N/A 100% 100% (58/58%) 100% 97.2% (172/177) -3 Sampov Meas NA N/A N/A 100% 96.34% (79/82) 100% 97.8% (263/269) 2 By provider type Public HFs 97% (614/635) 98% 99% (744/754) 100% 98% (1628/1658) 100% 97.7% (1783/1825) 1 Private providers NA 98% NA 100% 100% (208/208) 100% NA NA By sex Male 97% (603/623) 98% 99% (659/667) 100% 99% (1688/1698) 100% 97.7% (1670/1710) 1 Female 92% (11/12) 97% 98% (85/87) 100% 99% (127/128) 100% 98.3% (113/115) 7 By age group <5 NA 100% NA 100% 100% (12/12) 100% 100% (5/5) 0 5-14 86% (18/21) 95% 75% (27/36) 100% 94% (61/65) 100% 96.2% (50/52) 1 ≥15 97% (596/614 98% 100% (717/718) 100% 100% (1742/1749) 100% 97.7% (1727/1767) 1 139 No. Indicator Data Source Disaggregation – Baseline Value FY 2017 FY 2018 FY 2019 % Change Baseline FY 2019* Target (6 ODs) Actual Target Actual FY 2018 Target Actual FY 2019 4 % of all confirmed Pf/Mixed cases receive the appropriate malaria treatment under DOT according to the NTGs by VMWs/MMWs (disaggregated by OD, sex, and age group) - MIS web-based (VMW monthly data) Overall 89% (933/1045) 91% 85% (1472/1725) 94% 92% (1715/1855) 95% 88% (730/833) 1 By OD Kravanh 100% (575/575) 100% 76% (720/949) 100% 89% (1055/1188) 100% 82% (480/583) -18 Krakor 76% (358/470) 90% 99% (702/707) 93% 100% (633/634) 95% 100% (234/234) 32 Bakan NA N/A N/A 93% 100% (2/2) 95% 100% (3/3) 0 Sampov Meas NA N/A N/A 93% 100% (5/5) 95% 100% (13/13) 0 By sex Male NA 91% NA 94% NA 95% NA NA Female NA 91% NA 94% NA 95% NA NA By age group <5 NA 90% NA 90% NA 93% NA NA 5-14 NA 90% NA 90% NA 93% NA NA ≥15 NA 92% NA 93% NA 96% NA NA 5 % of people living in target village who slept under an ITN/LLIN during the previous night (disaggregated -VMW monitoring reports of ITN - Response investigation -CMS (2019) Overall 59.5% (Domain 1, CMS) NA(CMS) NA(CMS) 88% NA(CMS) NA By sex Male NA NA(CMS) NA(CMS) NA NA(CMS) NA 140 No. Indicator – by sex and pregnancy) Data Source Disaggregation Baseline Value FY 2017 FY 2018 FY 2019 % Change Baseline FY 2019* Target (6 ODs) Actual Target Actual FY 2018 Target Actual FY 2019 Female NA NA(CMS) NA(CMS) NA NA(CMS) NA Pregnant women 62.5% (Domain 1, CMS) NA(CMS) NA(CMS) 88% NA(CMS) NA By OD Kravanh NA NA(CMS) NA(CMS) NA NA(CMS) NA Krakor NA NA(CMS) NA(CMS) NA NA(CMS) NA Bakan NA N/A NA(CMS) NA NA(CMS) NA Sampov Meas NA N/A NA(CMS) NA NA(CMS) NA 6 % of registered private providers that received supportive supervision at least twice during the past 12 months (disaggregated by OD) -PPM supervision report (Project database) Overall NA 96% 0% 100% 90% (61/68) 100% 50% -44 By OD Kravanh NA 96% 0% 100% 87% (26/30) 100% 0% -100 Krakor NA 96% 0% 100% 92% (35/38) 100% 56% -39 Bakan NA N/A N/A 100% NA 100% 49% NA Sampov Meas NA N/A N/A 100% NA 100% 100% NA 7 % of HFs and VMWs/MMWs received structured supervision and feedback on malaria at least twice in the last year - Reports of TS to public HFs & VMWs/MMWs Overall 43% 95% 16% 100% 72% (393/548) 100% 87% (230/265) 102 By provider type Public HFs 53% 100% 37% 100% 99% (78/79) 100% 98% (44/45) 85 141 No. Indicator – (disaggregated by type of provider and OD) Data Source Disaggregation Baseline Value FY 2017 FY 2018 FY 2019 % Change Baseline FY 2019* Target (6 ODs) Actual Target Actual FY 2018 Target Actual FY 2019 VMWs/MMWs 33% 90% 13% 100% 67% (315/469) 100% 85% (186/220) 158 By OD (HF/VMW) Kravanh 60% 95% 20% 100% 93% (141/151) 100% 82% (84/103) 37 Krakor 47% 95% 0% 100% 69% (74/107) 100% 88% (76/86) 87 Bakan 20% N/A N/A 100% NA 100% 100% (39/39) 400 Sampov Meas 0% N/A N/A 100% NA 100% 84% (31/37) NA 8 Number of ITNs, RDTs, and ACTs purchased by other partner that were distributed with USG funds (disaggregated by LLIN, ACT, and RDT) - Delivery notes - Distribution reports LLINs 631 0 0 230,800 206,319 0 10,903 217222 LLIHNs 19 17,300 8,355 98,000 87,464 0 5,540 101359 ACTs NA 57,728 57,728 10,000 43,353 10,000 0 101081 RDTs NA 9,850 9,850 5,000 17,000 5,000 0 26850 9 Number of ITNs, RDTs, and ACTs purchased by USG funds that were distributed (disaggregated by LLIN, ACT, and RDT) - Delivery notes - Distribution reports LLINs 18,600 20,000 17,624 32,300 16,039 25,840 52,251 85914 LLIHNs 11,760 0 0 31,400 12,944 25,120 37,685 50629 ACTs 0 0 0 0 0 0 0 0 RDTs 2121 3,000 0 1,750 4,750 1,750 2,925 7675 142 No. Indicator Data Source Disaggregation – Baseline Value FY 2017 FY 2018 FY 2019 % Change Baseline FY 2019* Target (6 ODs) Actual Target Actual FY 2018 Target Actual FY 2019 Objective 3: Strengthen national malaria surveillance systems and M&E appropriate for malaria elimination and control activities 1 % of all HFs in target elimination/ pre-elimination ODs are trained in the use of case investigation forms (disaggregated by OD) -Training report Overall 100% 100% 14% 100% 103% 100% NA 400 By OD Sampov Loun 100% 100% 14% 100% 115% 100% NA 0 Battambang NA NA NA 100% 100% 100% NA NA Thmar Koul NA NA NA 100% 100% 100% NA NA Maung Russei NA NA NA 100% 100% 100% NA NA Kravanh NA NA NA NA NA NA NA NA Krakor NA NA NA NA NA NA NA NA Bakan NA N/A N/A NA NA 100% NA NA Sampov Meas NA N/A N/A NA NA 100% NA NA Pailin NA N/A N/A 100% 100% 100% NA NA 2 % of HFs, VMWs/MMWs in target ODs are submitting surveillance data on time according to national guidelines (disaggregated by OD and type of provider) - HMIS web-based (Inventory report for public HFs) - MIS web-based (VMW received reports) Overall NA 92% 93% 95% 88% 98% 93% 0 By OD Sampov Loun NA 92% 92% 95% 90% 98% 94% 2 Battambang NA 92% 97% 95% 90% 98% 94% -3 Thmar Koul NA 92% 97% 95% 99% 98% 100% 3 Maung Russei NA 92% 88% 95% 80% 98% 97% 10 Kravanh NA 92% 97% 95% 80% 98% 86% -11 Krakor NA 92% 92% 95% 91% 98% 93% 1 Bakan NA N/A N/A 95% 95% 98% 98% 3 Sampov Meas NA N/A N/A 95% 99% 98% 99% 4 Pailin NA N/A N/A 95% 82% 98% 85% 4 143 No. Indicator Data Source Disaggregation – Baseline Value FY 2017 FY 2018 FY 2019 % Change Baseline FY 2019* Target (6 ODs) Actual Target Actual FY 2018 Target Actual FY 2019 By provider type Public HFs NA 95% 99% 97% 100% 100% 100% 1 VMWs/MMWs NA 92% 87% 95% 97% 97% 91% 5 Private providers NA 90% 95% 93% 79% 96% NA -17 3 % of Private Providers submitting referral data on a quarterly basis (disaggregated by OD) PP Referral Slips (quarterly basis) Overall NA 90% 95% 93% 79% 96% NA -17 By OD Sampov Loun NA 90% 85% 93% 78% 96% NA -8 Battambang NA 90% 100% 93% 82% 96% NA -18 Thmar Koul NA 90% 93% 93% 98% 96% NA 5 Maung Russei NA 90% 92% 93% 68% 96% NA -26 Kravanh NA 90% 100% 93% 63% 96% NA -37 Krakor NA 90% 94% 93% 72% 96% NA -23 Bakan NA N/A N/A 93% NA 96% NA NA Sampov Meas NA N/A N/A 93% NA 96% NA NA Pailin NA N/A N/A 93% NA 96% NA NA 144 No. Indicator Data Source Disaggregation – Baseline Value FY 2017 FY 2018 FY 2019 % Change Baseline FY 2019* Target (6 ODs) Actual Target Actual FY 2018 Target Actual FY 2019 4 % of target ODs whose case data including notification, and response is linked and/or integrated to the Malaria Information System -MIS web-based (Elimination data entry form built in MIS) Overall 17% (1/6) 17% (1/6) 17% (1/6) 56% (5/9) 56% 78% (7/9) 56% (5/9) 229 5 Functional system to detect and respond to all malaria outbreaks (defined by exceeding pre￾defined outbreak thresholds) in target ODs is in place as defined in the national strategy (disaggregated by OD) -Project report Overall 1 1 2 5 9 9 9 800 By OD Sampov Loun 1 1 1 1 1 1 1 0 Battambang 0 0 1 1 1 1 1 100 Thmar Koul 0 0 0 1 1 1 1 100 Maung Russei 0 0 0 1 1 1 1 100 Kravanh 0 0 0 0 1 1 1 100 Krakor 0 0 0 0 1 1 1 100 Bakan NA N/A N/A 0 1 1 1 100 Sampov Meas NA N/A N/A 0 1 1 1 100 Pailin NA N/A N/A 1 1 1 1 100 Objective 4: Build capacity of MOH to manage, intensify, and sustain malaria control and elimination efforts particularly at the OD level 1 % of Annual Operational Plans (AOPs) that include malaria activities (and budgets) are developed -OD AOPs Overall 67% 100% 100% 100% 100% 100% 100% 49 145 No. Indicator Data Source Disaggregation – Baseline Value FY 2017 FY 2018 FY 2019 % Change Baseline FY 2019* Target (6 ODs) Actual Target Actual FY 2018 Target Actual FY 2019 2 Number of health workers trained in case management with artemisinin based combination therapy (ACTs) with USG funds* (disaggregated by type of provider and sex) -Training report Overall 428 (FY 14&15) 444 531 660 390 444 773 1,694 By provider type Public HFs 113 186 160 229 96 186 434 690 VMWs/MMWs 315 258 371 431 294 258 339 1,004 By sex Male 205 NA 285 NA 172 NA 468 925 Female 223 NA 246 NA 218 NA 305 769 3 Number of health workers trained in malaria laboratory diagnostics (rapid diagnostic tests (RDTs) or microscopy) with USG funds* (disaggregated by type of provider and sex) -Training report Overall 464 (FY 14&15) 444 562 660 390 444 773 1,725 By provider type Public HFs 149 186 191 229 96 186 434 721 VMWs/MMWs 315 258 371 431 294 258 339 1,004 By sex Male 239 NA 308 NA 172 NA 468 948 Female 225 NA 254 NA 218 NA 305 777 4 % of planned supervisory visits conducted by national, PHD, and OD staff at health facilities, VMWs/MMWs and PPs in all - Reports of regular TS to public HFs & VMWs/MMWs&PPs Overall 71% 90% 58% 95% 68% (1957/2858) 100% 130% (1171/904) 83 By provider type Public HFs 62% 90% 85% 95% 89% (374/418) 100% 86% (330/384) 39 146 No. Indicator – targeted ODs (disaggregated by type of provider and OD) Data Source Disaggregation Baseline Value FY 2017 FY 2018 FY 2019 % Change Baseline FY 2019* Target (6 ODs) Actual Target Actual FY 2018 Target Actual FY 2019 VMWs/MMWs 56% 90% 80% 95% 62% (1251/2023) 100% 185% (664/358) 230 Private providers 96% 90% 22% 95% 80% (332/417) 100% 109% (177/162) 14 By OD (HF/VMW/PPs) Sampov Loun 90% 90% 74% 95% 74% (519/705) 100% 102% (86/84) 13 Battambang 73% 90% 73% 95% 77% (520/680) 100% 138% (276/200) 89 Thmar Koul NA 90% 55% 95% 84% (115/136) 100% 107% (77/72) 95 Maung Russei 75% 90% 45% 95% 64% (206/322) 100% 130% (99/76) 73 Kravanh 30% 90% 56% 95% 79% (313/396) 100% 191% (172/90) 537 Krakor 47% 90% 44% 95% 57% (227/395) 100% 123% (155/126) 162 Bakan 30% N/A N/A 95% 28% (27/95) 100% 141% (110/78) 370 Sampov Meas 50% N/A N/A 95% 40% (30/75) 100% 108% (93/86) 116 Pailin 50% N/A N/A 95% 0% (0/54) 100% 112% (103/92) 124 147 No. Indicator Data Source Disaggregation – Baseline Value FY 2017 FY 2018 FY 2019 % Change Baseline FY 2019* Target (6 ODs) Actual Target Actual FY 2018 Target Actual FY 2019 5 % of planned supervisory visits conducted by PHD and CNM staff to all targeted ODs (disaggregated by OD) - Reports of regular TS ODs Overall NA 90% 40% 95% 89% (97/109) 100% 109% (155/142) 173 By OD Sampov Loun NA 90% 50% 95% 80% (12/15) 100% 106% (17/16) 112 Battambang NA 90% 36% 95% 100% (15/15) 100% 100% (16/16) 178 Thmar Koul NA 90% 43% 95% 71% (10/14) 100% 107% (15/14) 149 Maung Russei NA 90% 29% 95% 100% (15/15) 100% 106% (17/16) 266 Kravanh NA 90% 43% 95% 121% (17/14) 100% 138% (22/16) 221 Krakor NA 90% 43% 95% 114% (16/14) 100% 113% (18/16) 163 Bakan NA N/A N/A 95% 57% (4/7) 100% 106% (17/16) 86 Sampov Meas NA N/A N/A 95% 71% (5/7) 100% 100% (16/16) 41 Pailin NA N/A N/A 95% 38% 3/8) 100% 106% (17/16) 179 148 157 ANNEX 9: INDICATIVE LIST OF SELECTED PROTOCOLS, GUIDELINES, AND STANDARDS DEVELOPED WITH CMEP SUPPORT BY SUBJECT AND TYPE(S) OF ASSISTANCE Title Level of Contribution CMEP Contribution Cambodia Malaria Elimination Action Framework 2016-2021 National • Participated in development • Approval • Implementation • Indirect support • Advocacy Cambodia Malaria Elimination Action Framework 2021-2025 National • Participated in development • Approval • Implementation • Indirect support • Advocacy CMEP Capacity Building Strategy Provincial, National • Participated in development • Approval • Implementation • Indirect support • Advocacy CMEP Technical Strategy For Social and Behavior Change Communication Provincial, National • Participated in development • Approval • Implementation • Indirect support • Advocacy CMEP Technical Strategy Reducing Malaria among Mobile and Migrant Populations (MMPs) Provincial, National • Revision • Implementation • Indirect support • Advocacy CMEP Technical Strategy for Early Diagnosis and Appropriate Treatment Provincial, National • Development • Revision • Implementation • Indirect support CMEP Technical Strategy For Strengthening Malaria Surveillance within CMEP National, Regional • Development • Revision • Implementation • Indirect support CMEP Sustainability Plan and Exit Strategy National, Regional • Development • Revision • Implementation • Indirect support Cambodia Malaria Implementation Plan 1 National • Facilitated the process • Revision • Indirect support • Advocacy Cambodia Malaria Implementation Plan 2 National • Facilitated the process • Revision • Indirect support • Advocacy 158 Title Level of Contribution CMEP Contribution The Mobile and Migrant Population (MMP) in The Context of Malaria Elimination Operational Manual National • Revision • Implementation • Indirect support • Advocacy Cambodia National Malaria Annual Reports 2017, 2018, 2019 National • Facilitated the process • Revision • Indirect support • Advocacy National CNM foci investigation protocol, SOP and tools National • Support conducting field testing • Indirect support • Advocacy Pv radical treatment and roll out of G6PD testing along with 14-day PQ administration and rigorous case monitoring/follow up National • Facilitated the decision • Developed the relevant tools (SOP, treatment regimen, job aids, training guidance, HF case register) Source: CMEP reports, Evaluation results. This list does not include all documents supported by CMEP 159 ANNEX 10: CMEP TESTING AND TREATMENT (FY 2017-FY 2020) OD FY 2017 FY 2018 FY 2019 FY 2020 Tested Diagnosed Treated Tested Diagnosed Treated Tested Diagnosed Treated Tested Diagnosed Treated Bakan 108 23 23 1,367 216 209 3,415 245 239 7,659 33 33 Battambang 4,194 348 317 7,656 441 418 13,290 196 196 7,325 50 49 Krokor 3,371 1,850 1,845 8,674 3,034 2,997 19,209 2,228 2,225 25,197 697 691 Phnom Kravanh 5,507 2,869 2,686 21,379 10,593 9,624 23,738 6,241 5,974 25,754 966 921 Maung Russei 923 242 234 2,493 466 463 4,246 263 260 2,912 61 60 Pailin 121 40 37 1,073 120 119 10,808 54 52 5,571 18 16 Sampov Loun 4,900 148 147 10,761 163 162 12,562 98 98 7,475 23 23 Sampov Meas 304 232 222 1,033 497 448 3,410 405 400 7,771 94 92 Thma Koul 295 20 20 779 50 50 1,431 32 32 1,125 9 9 160 ANNEX 11: CMEP TESTING AND TREATMENT BY OD (FY 2017-FY 2020) Testing and Treatment per OD (FY 2017) 0 1000 2000 3000 4000 5000 6000 Bakan Battambang Krokor Phnom Kravanh Maung Russei Pailin Sampov Luon Sampov Meas Thma Koul Tested Diagnosed Treated Testing and Treatment (FY 2018) 0 5000 10000 15000 20000 25000 Bakan Battambang Krokor Phnom Kravanh Maung Russei Pailin Sampov Luon Sampov Meas Thma Koul Tested Diagnosed Treated 161 Testing and Treatment (FY 2019) 0 5000 10000 15000 20000 25000 30000 Bakan Battambang Krokor Phnom Kravanh Maung Russei Pailin Sampov Luon Sampov Meas Thma Koul Tested Diagnosed Treated Testing and Treatment (FY 2020) 0 5000 10000 15000 20000 25000 Bakan Battambang Krokor Phnom Kravanh Maung Russei Pailin Sampov Luon Sampov Meas Thma Koul Tested Diagnosed Treated 162 ANNEX 12: MINI-SURVEY RESULTS At-Risk Population Knowledge of Malaria Activities (n=114) 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% LLINS Distribution Malaria Diagnosis Malaria Treatment Distribution of Malaria Treatment Distribution of RDTs Interpersonal Communications Awareness Training Sampov Meas Phnom Kravanh Battambang Pailin Sampov Luon Krakor At-Risk Men Participation in Malaria Activities (n=61) 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% LLINS Distribution Malaria Diagnosis Malaria Treatment Distribution of Malaria Treatment Distribution of RDTs Interpersonal Communications Awareness Training Sampov Meas Phnom Kravanh Battambang Pailin Sampov Luon Krakor 163 At-Risk Women Participation in Malaria Activities (n=43) 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% LLINS Distribution Malaria Diagnosis Malaria Treatment Distribution of Malaria Treatment Distribution of RDTs Interpersonal Communications Awareness Training Sampov Meas Phnom Kravanh Battambang Pailin At-Risk Men Received Most Recent Malaria Test (n=50) 31% 71% 60% 70% 31% 14% 50% 33% 20% 5% 15% 14% 33% 20% 20% 23% 50% 33% 5% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Sampov Meas Phnom Kravanh Battambang Pailin Sampov Loun Krakor Within the past 3 months Within the past 6 months Within the past year One or more years ago 164 At-Risk Women Received Most Recent Malaria Test (n=24) 50% 75% 71% 88% 50% 100% 25% 29% 13% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Sampov Meas Phnom Kravanh Battambang Pailin Sampov Loun Within the past 3 months Within the past 6 months Within the past year One or more years ago *No women in Krokor were surveyed When At-Risk Men Received Most Recent Malaria Treatment (n=29) 13% 100% 100% 20% 25% 0% 0% 10% 25% 33% 30% 38% 100% 67% 40% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Sampov Meas Phnom Kravanh Battambang Pailin Sampov Loun Krakor When at-risk men received most recent malaria treatment (n=29) Within the past 3 months Within the past 6 months Within the past year One or more years ago *A total of seven women were surveyed having received treatment in the last three years, two of which received treatment in the last three months located in Sampov Meas and Phnom Kravanh.