December 2011 This publication was produced for review by the United States Agency for International Development. It was prepared by David Nelson (Team Leader), Rose Schneider, Pedro Jesus Mendoza Arana of Social Impact, Inc (SI) and Elvira Beracochea of MIDEGO, Inc. EVALUATION USAID/Peru Millenium Challenge Corporation (MCC) Immunizations Threshold (ITP) Assessment Final Report US Embassy, Peru via Flickr D DECEMBER 2011 This publication was produced for review by the United States Agency for International Development. It was prepared by David Nelson (Team Leader), Rose Schneider, Pedro Jesus Mendoza Arana of Social Impact, Inc (SI) and Elvira Beracochea of MIDEGO, Inc. USAID/PERU MILLENIUM CHALLENGE CORPORATION (MCC) IMMUNIZATIONS THRESHOLD PROGRAM (ITP) ASSESSMENT FINAL REPORT USAID/PERU MILLENIUM CHALLENGE CORPORATION (MCC) IMMUNIZATIONS THRESOLD PROGRAM (ITP) ASSESSMENT FINAL REPORT DISCLAIMER The authors’ views expressed in this publication do not necessarily reflect the views of the United States Agency for International Development or the United States Government. This document was submitted by Social Impact, Inc., with MIDEGO Inc., to the United States Agency for International Development under USAID Contract No. RAN-I-00-09-00019-00; Task Order #AID-527-TO-11-00001. Social Impact, Inc. 2300 Clarendon Boulevard Suite #300 Arlington, VA, 22201 Tel: (703) 465-1884 Fax: (703) 465-1888 info@socialimpact.com TABLE OF CONTENTS EXECUTIVE SUMMARY............................................................................................................................... i BACKGROUND.............................................................................................................................................. 1 METHODOLOGY.......................................................................................................................................... 3 FINDINGS ........................................................................................................................................................ 5 Overall Achievements of MCC-ITP........................................................................................................... 6 Status of immunization programs at end of MCC-ITP.........................................................................10 Differences in regions/networks ..............................................................................................................19 Status of regional programs and networks today....................................................................................20 Other issues affecting the status of vaccination programs....................................................................21 CONCLUSIONS.............................................................................................................................................22 Factors that facilitated success in the regions/networks.......................................................................22 Factors that inhibited success in the regions/networks ........................................................................23 Factors that inhibited sustainability of achievements ............................................................................23 RECOMMENDATIONS..............................................................................................................................24 LESSONS LEARNED ..................................................................................................................................25 ANNEXES.......................................................................................................................................................27 Annex A - The proposed improvement and sustainability action plans for each Selected Region.....28 Annex B - Statement of Work (SOW) for the Assessment, including modifications to the SOW .....45 Annex C - Methodological notes, including all tools used in conducting the evaluation - questionnaires, checklists, and discussion guides ........................................................................................57 Annex C.1 – Data Collection Instruments..............................................................................................57 Annex C.2 C-DHS Vaccination Coverage Analysis..............................................................................77 Annex D List of Documents Consulted ...................................................................................................... 83 Annex E List of Contacts ...............................................................................................................................86 Annex F Regional comparison table .............................................................................................................95 Annex G Other Relevant Information...................................................................................................... 104 Annex G.1 Maps of Sites Visited........................................................................................................... 105 Annex G.2 Status of Cold Room Installations ................................................................................... 109 Annex G.3 Regional Pending Actions - Amazonas ............................................................................ 110 Annex G.4 Reporte de la Encuesta ....................................................................................................... 111 Annex G.5 Threshold Programs Lessons learned ............................................................................... 112 Annex G.6 MCC Peru Indicators .......................................................................................................... 112 ACRONYMS AUS Universal health insurance scheme BCG Tuberculosis vaccination CD Compact disc C-DHS Continuous Demographic and Health Survey CI Confidence Interval COP Chief of Party DGE Directorate General of Epidemiology DIGEMID Directorate General for Drugs and Medical Supplies DIREMID Regional Directorate for Drugs and Medical Supplies DIRESA Regional Health Directorate DNI National identification document DPT3 Diphtheria-Pertussis-Tetanus vaccine, 3rd dose EDA Acute diarrheal disease (“Enfermedad diarréica aguda”) EPI Expanded Program on Immunizations GIS Geographic information system GOP Government of Peru HC Health center HEO Health and Education Office of USAID/Peru HIS Health information system IB Itinerant brigade IEC Information-Education-Communication IIIS Integrated Immunization Information System INEI National Institute of Statistics and Informatics ITP Immunization Threshold Program of MCC KAP Knowledge-Attitude-Practice MCC Millennium Challenge Corporation MED Monitoring and Evaluation of Decentralization MEF Ministry of Economics and Finance MMR Measles-Mumps-Rubella vaccine MOH Ministry of Health NGO Non-governmental organization NIS National immunization strategy OGEI General Office of Statistics and Informatics PAHO Pan American Health Organization PAN Articulated Nutritional Program (“Programa Articulado Nutricional”) PARSALUD Program for Support of Health Reform, The World Bank PCM President´s Council of Ministers PDA Personal Digital Assistant PDF Portable Document Format for printing PIP Public investment project POI Operational immunizations plan (“Plan operativo para inmunizaciones”) PpR Budget by Results (“Presupuesto por resultados”) PRONAA National Food Assistance Program (“Programa Nacional de Asistencia Alimentaria”) RCM Rapid coverage monitoring RIS Regional immunization strategy RM Ministerial resolution (“Resolución ministerial”) RO Regular Budget (“Recursos Ordinarios”) SI Social Impact, Inc. SIGA Integrated System for Administrative Management SIS Integrated Health Insurance scheme SISMED Drug information system SNIP National System for Public Investment SOW Scope of work SPSS Statistical Package for the Social Sciences SWOT Strengths-Weaknesses-Opportunities-Threats TOT Training of Trainers TPM Team planning meeting UHF Ultra High Frequency UNICEF United Nations Children´s Fund USAID United States Agency for International Development VDIM Virtual Diploma for Immunization Management VP Vaccine preventable WHO World Health Organization i USAID/PERU MCC-IPT Assessment Report – Executive Summary EXECUTIVE SUMMARY BACKGROUND The Millennium Challenge Corporation’s Immunization Threshold Program (MCC-ITP) for Peru was implemented by the United States Agency for International Development in Peru (USAID/Peru), and from October 2008 through February 2011, a USAID/Peru contractor implemented the program under the following directives: “. . . [E]xpand coverage of one-year old infants who have received immunization against measles and the full series of three diphtheria, pertussis and tetanus (DPT3) vaccinations, so that immunization rates of 95% are reached in each of Peru's regions and, thus, nationally. This assistance will focus intensive efforts on eight regions with dispersed rural populations where coverage rates are significantly below this goal. In addition, this activity will support strengthening of key systems required for the Peru to sustain high vaccination rates throughout the country: training, cold chain management and logistics, and information collection and analysis. The work will be implemented with the government health sector at the national, regional and local levels.”1 In 2006, the MCC scorecard to establish eligibility for compact grants used four indicators for “Investing in People.” One of these was Immunization Rates (IR) for measles-mumps-rubella (MMR) and DPT3, based on World Health Organization (WHO) statistics (Annex G.6). In 2007, the MCC scorecard for Peru showed an IR of 82% for 2005, the most recent year of available statistics, after declining from over 95% in 1999. The MCC-ITP goal was to achieve and sustain a 95% national immunization rate according to WHO statistics. In turn, the official WHO statistics are largely based on Ministry of Health (MOH) coverage reports. In order to reach the immunization threshold, the MOH designed a strategy to complement the National Immunization Strategy based on three components: 1. Increased vaccination in eight target regions through use of Itinerant Brigades (IBs)to access dispersed and excluded populations 2. Strengthened vaccine logistics such as cold chain management 3. Creation of a national integrated immunization information system (IIIS) The MCC-ITP also assisted in the design and production of communications material to generate demand for vaccinations and provided technical assistance in the planning, budgeting, and financing of immunizations operations. The MCC-ITP was conducted during a period of rapid change in roles and responsibilities at the central and regional levels due to decentralization, which rapidly transferred many health system functions from the central MOH to regional governments which had, and still have, uneven capacities to carry out these functions and adapt to new roles and responsibilities. 1 MCC Immunization Project, RFTOP 527-08-009, page 3, USAID/Peru, 2008. USAID/PERU MCC-IPT Assessment Report – Executive Summary ii Frequent political turnover brought to power new officials with little experience in management of health issues. Their inexperience caused delays in budget approval and changes in regional health priorities. A further complication was the rapid introduction of new vaccines into an already-crowded vaccination schedule. The number of vaccines increased from four to fifteen in a matter of two years, challenging financing, staff, logistics, and storage and complicating families’ understanding of their children’s vaccination needs. These changes required additional personnel and training, increased financing, installation of cold chain equipment, and effective communication with the community. This report examines the status of the national immunization program within four selected regions and compares current status with end-of-project status to identify factors that inhibit or facilitate sustained achievements in a program of this nature. It also identifies lessons learned, and recommends next steps for the selected regions. METHODOLOGY Assessment methods included document review, surveys, interviews, SWOT (strengths, weaknesses, opportunities, and threats) analyses, and direct observation. Four regions were chosen for in-depth study, in accordance with the scope of work (SOW) and after discussions with USAID and MOH counterparts (two regions with the highest and two with the lowest vaccination coverage rates, according to the 2010 Continuous Demographic and Health Survey, C-DHS): Cusco, Apurimac, Amazonas, and Puno. The Assessment Team studied: (1) the end-of-program status of the immunization system as documented in the implementing contractor’s (Abt Associates) Final Report, interviews, and other sources; (2) accomplishments of the MCC-ITP according to Abt reports, correspondence, and interviews; (3) the current status of the regional immunization systems as documented in MOH and regional documents, interviews, and observations; (4) the reported and observed strengths, weaknesses, opportunities and threats to the immunization program in four selected regions; and (5) draft regional sustainability plans for the selected regions, developed at a national workshop with regional and national participants. Interviews with key informants were conducted, using contacts suggested by USAID and local counterparts, including USAID staff, former MCC-ITP staff, United Nations Children´s Fund (UNICEF), central government officials, Regional Health Directorates (DIRESA), regional and local government authorities, health facilities staff, and itinerant brigades. Despite repeated requests for interviews, Pan American Health Organization (PAHO) officials in Washington D.C. and Lima were unavailable to the Team. Staffs of health networks and micro-networks were observed and interviewed in order to assess the program based upon the eleven performance criteria stipulated in the SOW (see Annex B), and to assess the quality of the information system. The Team conducted SWOT analyses with DIRESA, the regional health directorate, personnel to understand success factors and constraints and to identify critical steps for improving and sustaining the immunization system. iii USAID/PERU MCC-IPT Assessment Report – Executive Summary After site visits, the Assessment Team consolidated and analyzed findings from the selected regions and presented them to USAID/Peru. On November 8, 2011 the Team held a workshop for key regional stakeholders, central MOH authorities, UNICEF, PAHO, National Institute of Statistics and Informatics (INEI), and USAID staff to present findings and elicit commitments to sustain and strengthen the immunization system. The meeting validated the critical steps identified in the regions and established a time schedule for carrying them out. FINDINGS 1. The MCC-ITP fulfilled contractual obligations, delivered high-quality products, and made important contributions to the national immunization strategy (NIS) and regional immunization strategies (RIS). However, due to the brief implementation period and delays, design adjustments and mid-term corrections could not be made to address the rapidly changing context brought about by decentralization, increasing complexity of vaccination schedule, and new financing mechanisms. Some tasks were still in process when the program closed and many of these tasks have not been completed. 2. Four important studies—logistics, norms and procedures, knowledge-attitudes-practices (KAP), and information systems—were completed under the MCC-ITP, giving a clear view of the NIS at the time of the program. Many of the studies’ findings were used as inputs to the principal activities of MCC-ITP, including strengthening the cold chain; training IB members and RIS staff; designing and implementing the IIIS; and creating an information￾education-communication (IEC) toolbox (these studies were also used by other MOH strategies and directorates, as well as by the Ministry of Economics and Finance (MEF). 3. The MCC-ITP created and implemented a Virtual Diploma Immunization Management (VDIM) course from which 856 health professionals graduated. Many of those trained (67% of a spontaneous-response national survey) are still working in immunizations and 98% consider that it improved their competency. However, the course was not institutionalized in the MOH or a university and has not been continued. 4. Staffs in the selected regions were competent, motivated and qualified for the complex task of managing the region’s immunization program. Many were trained through the VDIM. 5. The NIS became stronger and very complex during the MCC-ITP, demonstrating: increased staff and numbers of vaccines; tighter control of supplies and cold chain; better use of social communication; and closer monitoring of coverage with the “analytic” tables and Rapid Coverage Monitoring. Much of the improvement to NIS operations was supported by the MCC-ITP through studies, training, and introduction of norms—for example, most regional vaccination coordinators were trained through the VDIM, which was based on then-current vaccination standards. 6. The monitoring and evaluation of decentralization (MED) of 2009 provided semi￾quantitative assessments of DIRESA’s assumption of regional government functions in health. The baseline assessment found Puno (30%) and Amazonas (32%) slightly below the median of 34% and Cusco (34%) and Apurimac (36%) slightly above it (minimum was 20% USAID/PERU MCC-IPT Assessment Report – Executive Summary iv and maximum was 62%).2 7. The cold chain has improved in 14 of the regions, which now have functioning cold rooms and use data loggers to monitor refrigerator and freezer temperatures. MCC-ITP-trained technicians in the four selected regions are effectively using the MOH’S drug information system (SISMED) and the Ministry of Finance’s Integrated System for Administrative Management (SIGA) information systems. Frequent turnover among regional health directors was a source of instability, but second-level managers were stable, experienced, and highly trained. For example, the RIS coordinators in Amazonas and Apurimac had been in their positions for seven and nine years, respectively, and both had completed the VDIM course. 8. The 145 IBs trained and equipped by the MCC-ITP were a key component of the MCC-ITP strategy, intended to reach over 500,000 rural Peruvians. However, because of their low numbers and the difficult terrain to cover, they could not reach more than a small fraction of that population. In comparison with IBs during the MCC-ITP implementation period, current IBs suffer from high turnover, reduced training, and budget restrictions and as a result are incomplete, fewer in number, and tend to work in fixed sites rather than as mobile teams. 9. IBs are a politically important outreach service, but are being abandoned in some regions in favor of facility-based, extra-mural service delivery, which is perceived as being more cost effective. In three of the four selected regions, IBs have decreased from 42 to 26 in number. 10. The formalization of the IIIS in Ministerial Resolution RM 614 was delayed and came very late in the MCC-ITP so that IIIS implementation was not completed by the end of the program. Without a champion for IIIS implementation to conduct training and monitor its use, the IIIS has not been implemented beyond the initial pilot areas and is ignored in most regions. 11. Despite increased numbers of vaccine doses reportedly administered during the MCC-ITP, the MOH vaccination statistics for the public sector for 2008, 2009 and 2010 showed that coverage did not reach the 95% safety threshold. DOSES/YEAR 2008 2009 2010 COVERAGE/YEAR 2008 2009 2010 DPT3 494478 551504 548965 DPT3 92% 93% 92% This is because the 2007 census corrected the technically questionable 2005 census and, as a result, the population increased by nearly 10% in 2009. This raised the coverage denominator in almost direct proportion to the increased number of vaccine doses administered between 2008 and 2010, wiping out any improvement in coverage achieved by the increased number of vaccine doses applied. 12. The MOH national vaccination statistics that were the basis for MCC-ITP design assumptions and were relied upon for program monitoring cover only public facilities and do not capture vaccinations provided by the private sector, which account for 23% of vaccinations, according the C-DHS.3 2 Ministerial Resolution RM 739-2009/MINSA, Nonetheless, the MOH reports over 90% coverage for 3 The team processed 2010 C-DHS data. v USAID/PERU MCC-IPT Assessment Report – Executive Summary the whole country—which means that either many children are receiving more than one dose of a given vaccine, or reported doses are exaggerated. In either case, the Team believes that MOH-reported coverage rates are overestimated by nearly 20%. 13. The C-DHS reports national vaccination coverage on a cohort of children born 18 to 29 months before the survey. Thus, the 2010 C-DHS could not provide coverage information for children born and vaccinated in 2010 and could not serve to evaluate the impact of the MCC-ITP. The program’s impact on coverage rates will not be known until 2012, when the 2011 C-DHS findings of immunization coverage for the 2010, one-year-old cohort will be published. 14. The C-DHS sample is too small to give statistically valid comparisons of vaccination coverage between regions. Thus, the apparent differences in coverage rates between regions cited in the SOW are inconclusive. 15. Regional teams and service providers in the selected regions, who have identified and vaccinated virtually all children in their catchment areas, uniformly observe MOH vaccination norms. They all complain that their assigned populations (based on 2007 census figures) do not reflect their true catchment populations, because of migrations and seasonal movements. Because of this assessment, INEI recently acknowledged the problem. The NIS also recognizes this problem, and is working with regional and network immunization teams to start local child censuses that will correct it. 16. According to the “analytic” tables produced by immunization services in health facilities and used by RIS coordinators to monitor coverage, (i.e., the parallel immunization information system) there are real coverage differences between facilities. These are the result of different individual leadership and management skills and are being addressed in each region through training and personnel actions. CONCLUSIONS 1. Importance of timely studies for program design. Sizable time lags between design and implementation allowed for significant changes in circumstances and affected validity of design assumptions. 2. Validity of coverage indicators. Vaccination coverage indicators for a given year (e.g. 2009) as published by the MOH and estimates from C-DHS are not comparable or reliable because: - MOH data are incomplete; - Coverage rate denominators are unreliable due to migrations and seasonal movements; - C-DHS estimates cover vaccinations delivered to a child cohort 18 to 29 months before the year in question. Furthermore, the C-DHS estimates for 2010 cannot establish end￾of-project vaccination status. 3. Discrepancy between numbers of doses and coverage. The NIS reported increases in numbers of doses of DPT3 and MMR from 2007 to 2009. While these numbers may be overestimated, the corrected 2007 census increased the population and therefore, the vaccination coverage denominator, thus erasing any improvement in coverage rates. 4. Vaccination coverage differences between regions. The C-DHS estimates do not give statistically significant differences between regions and apparent differences are probably due USAID/PERU MCC-IPT Assessment Report – Executive Summary vi to population migrations and seasonal movements. The one-time household survey (CUANTO) conducted by MCC-ITP did not publish confidence intervals, so it is unknown whether cited differences in coverage estimates between regions are statistically significant. 5. Vaccination coverage differences between facilities. Differences in immunization performance between facilities are likely to be the result of individual management and leadership skills, which are monitored by regional immunization coordinators who intervene with training and management actions. 6. Trained personnel is the most significant outcome. The large number of qualified professionals working in key positions of the immunization system is, in great part, thanks to the MCC-ITP VDIM. In general, the VDIM diploma has been considered in making assignments and advancements within the immunization system. 7. Ability to affect national systems and processes (including procurement, training, staffing, financing, and epidemiologic surveillance) that affect sub-national program performance and decentralized operations have been formalized in norms and ministerial resolutions, in part due to MCC-ITP training and advocacy. 8. The IIIS, developed with MCC-ITP assistance, has not been sustained. The lack of a functioning, effective, national information system impedes the monitoring of public and private vaccinations. Regional immunization coordinators have developed their own manual and computerized “parallel” systems, during and after MCC-ITP, to meet local management information needs in public facilities. However these systems do not receive information from the private sector and are not inputs to the national health information system (HIS), and so cannot contribute to a national vaccination-coverage-rate indicator. 9. Itinerant brigades as a vaccination strategy. The planned number of 145 IBs could not and did not significantly increase vaccination coverage in the country. Currently, regional government support for IBs in the selected regions is limited because of cost and perceived lack of benefits and they are not a significant component of the NIS, despite announced central-level support. 10. Roadmaps, vision, and critical steps. At the end of this assignment, each of the selected regions had a simple, actionable plan to improve their performance and sustain their training and cold chain management. Each plan was tailored to their unique circumstances and the findings of this assessment. The regional plans are provided in Annex A. The Assessment Team identified the following factors that affected sustained success: 1. Immunization Management course prepared national and regional immunization staff at a high level and these personnel are operating the immunization system effectively and according to MOH norms. 2. The President´s Council of Ministers (PCM) was a strong champion for the MCC-ITP and continued pressure for completion of cold room installations nationally after close of the program.4 4 The PCM has been a champion for cold room installation during and after the MCC-ITP (they were active while the Team was working in October.) vii USAID/PERU MCC-IPT Assessment Report – Executive Summary 3. The 30-month implementation period was too short for institutionalization of many functions necessary for sustained success of the program. 4. The VDIM was implemented by MCC-ITP staff, rather than through local academic institutions that might have institutionalized the course and made it more sustainable. 5. The IIIS was officially sanctioned too late to be fully implemented and has fallen into disuse in most regions. 6. Effects of decentralization. Regional governments, which under decentralization have autonomy for prioritizing and budgeting immunizations activities within their regions, vary in their capacities and support for immunizations. Additionally, the frequent turnover of regional health directors5 was a source of instability, but second-level managers were stable, experienced and highly trained. They have sustained the success of interventions implemented by the MCC-ITP in all four selected regions. Additionally, the MOH’s dual role as both official reporter of national vaccination coverage and steward of regional vaccination provision has not been regulated fully. The MOH does not have a clear strategy to receive information about and report on the vaccinations provided by institutions outside of the public sector nor does it have a consistent decentralized management model. RECOMMENDATIONS • A national, nominal, information system capable of collecting and reporting all public and private vaccinations should be finalized. • The NIS’ analytic immunization information system should be standardized and implemented on MCC-ITP-donated computers for day-to-day management of vaccinations. • The VDIM should be institutionalized, either in a university or through the MOH. Immunization has become very complex and all staff involved should take the course. • IBs as a strategy for service outreach to dispersed and excluded populations should be evaluated, including cost and effectiveness in reaching these populations, as well IB and staff roles within DIRESAs and contribution to immunization coverage. • The IEC toolbox provides an important resource for overcoming cultural barriers to vaccination. It should become part of the VDIM in order to be used sustainably at operational levels. LESSONS LEARNED • Given the short nature of MCC’s threshold programs, flexibility in program design and implementation of activities is imperative in order to achieve objectives. 5 Regional health directors were reported to have changed frequently in the last nine months as a result of the regional elections which changed regional administrations (e.g. Puno with 3). The Team also found that people perceived greater instability due to this turnover, however, beyond those perceptions the team does not have additional evidence that turnover is greater now than before decentralization. USAID/PERU MCC-IPT Assessment Report – Executive Summary viii • Monitoring and evaluation indicators should measure the strategies’ effectiveness and should be timely and reliable. • Design assumptions are critical to achieving desired outcomes in the short time of a threshold program. Sizable time lags between design and implementation allow for significant changes in circumstances and affect validity of design assumptions. • The legal basis for interventions is very important for institutionalization, but delays in achieving legal status can endanger implementation, especially in the short timeframe of a threshold program. • The involvement of local authorities can increase sustainability and improve local decision￾making, local management, and service delivery processes. • Unreliable coverage indicators made monitoring of project outcomes and measurement of impact difficult. This coincides with the first of the lessons learned as reported by MCC on December 16, 2010:6 • Untested assumptions in the MCC-ITP design, (e.g., the location of unvaccinated children) and incomplete diagnoses, (e.g., the number of functional IBs at the beginning of the MCC￾ITP) led to outcomes and impacts below those planned in project preparation. (MCC Lesson Two: “Link threshold programs to indicators and goals that are actionable and measurable within a relatively short period of time.” 7 • For the MCC-ITP, the creation of a national immunization information system was overly ambitious, given the institutional impediments, delays in official sanctioning, and complexity of the undertaking, although the consultative process did allow MCC-ITP to change from the HIS platform to the Integrated Health Insurance scheme (SIS) in order to achieve pilot testing. (MCC Lesson Three: “Deepen diagnostic and feasibility analysis and identify the connection of activities to outputs, outcomes and impacts during program preparation.”) 8 • The MCC-ITP outcome goal of “Integrated Immunization Information System improved and implemented in the MOH and 1,246 Health Centers” was inappropriate for the two-year (later increased to 30 months) timeframe. MCC Lesson Four: “Be more selective when determining program interventions and establish a consultative process to tailor focus areas.”) 9 • It was assumed that the MOH and NIS would buy in to the activities of the MCC-ITP, such as training for IBs, cold chain technicians and the VDIM. No assurance for this was built into the program design, beyond the efforts to gain ministerial sanction for the IIIS. MCC Lesson Six: “Build in sustainability assurances during program design and early implementation.” “Establish outcome-level goals that are appropriate for the timeframe.” 10 6 MCC Threshold Program Lessons Learned, . 7 Ibid. 8 Ibid. 9 Ibid. 10 Ibid. 1 USAID/PERU MCC-IPT Assessment Report BACKGROUND The Millennium Challenge Corporation (MCC) Immunization Threshold Program (ITP) was designed to help the Government of Peru (GOP) achieve the immunization threshold on the MCC scorecard in order to make the country eligible for a MCC Compact grant. During 2006–2007 in Peru, the President’s Council of Ministers (PCM) coordinated the planning for a threshold program to achieve “green” scores in two components: (1) anti-corruption and (2) immunization coverage. The MCC-ITP effort was financed by MCC and the Government of Peru (GOP) financed expansion of the national immunization program. The United States Agency for International Development in Peru (USAID/Peru) implemented the MCC-ITP, and from October 2008 through February 2011, a USAID/Peru contractor implemented the program under the following directives: “. . . [E]xpand coverage of one-year old infants who have received immunization against measles and the full series of three diphtheria, pertussis and tetanus (DPT3) vaccinations, so that immunization rates of 95% are reached in each of Peru's regions and, thus, nationally. This assistance will focus intensive efforts on eight regions with dispersed rural populations where coverage rates are significantly below this goal. In addition, this activity will support strengthening of key systems required for the Peru to sustain high vaccination rates throughout the country: training, cold chain management and logistics, and information collection and analysis. The work will be implemented with the government health sector at the national, regional and local levels.”11 In 2006, the MCC scorecard to establish eligibility for compact grants used four indicators for “Investing in People.” One of these was Immunization Rates (IR) for measles-mumps-rubella (MMR) and DPT3, based on World Health Organization (WHO) statistics (Annex G.6). In 2007, the MCC scorecard for Peru showed an IR of 82% for 2005, the most recent year of available statistics, after declining from over 95% in 1999. The MCC-ITP goal was to achieve and sustain a 95% national immunization rate according to WHO statistics. In turn, the official WHO statistics are largely based on Ministry of Health (MOH) coverage reports. In order to reach the immunization threshold, the MOH designed a strategy to complement the national immunization strategy (NIS), based on three components: 1. Increased vaccination in eight target regions through use of Itinerant Brigades (IBs) to access dispersed and excluded populations; 2. Strengthened vaccine logistics such as cold chain management; and 3. Creation of a national integrated immunization information system (IIIS)12. The MCC-ITP also assisted in the design and production of communications material to generate demand for vaccinations and provided technical assistance in the planning, budgeting, and financing of immunizations operations. 11 MCC Immunization Project, RFTOP 527-08-009, page 3, USAID/Peru, 2008. 12 Ibid. pg 19. USAID/PERU MCC-IPT Assessment Report 2 The MCC-ITP was conducted during a period of rapid change in roles and responsibilities at the central and regional levels, due to decentralization that quickly transferred many health system functions from the central MOH to regional governments with uneven capacities to carry out these functions and adapt to new roles and responsibilities. Frequent political turnover brought to power new officials with little experience in management of health issues. Their inexperience caused delays in budget approvals and changes in regional health priorities. A further complication was the rapid introduction of new vaccines into an already-crowded vaccination schedule. The number of vaccines increased from four to fifteen in a matter of two years, challenging NIS financing, staff, logistics, and storage and complicating families’ understanding of their children’s vaccination needs. These changes required additional personnel and training, increased financing, installation of cold chain equipment, and effective communication with the community. Peru is divided administratively into 24 regions. Each region has a Regional Health Directorate (DIRESA) that oversees regional public health programs and the provision of health services through a number of health facilities organized into two to four operationally autonomous networks, each with 5 to 10 micro-networks consisting of a main health center and 10 to 20 health facilities. This report examines the status of the national immunization program within four selected regions and compares current status with end-of-project status to identify factors that inhibit or facilitate sustained achievements in a program of this nature. It also identifies lessons learned and recommends next steps for the selected regions. PURPOSE OF THE ASSESSMENT AND ASSESSMENT QUESTIONS The purpose of this Assessment is to inform USAID/Peru, the MCC, and the GOP on the current status of national, regional, and local immunization programs in Peru and of changes since the close of the MCC-ITP; to identify factors that facilitate and/or challenge success of medium-term projects, such as the MCC-ITP; and to provide actionable recommendations to increase sustainability of MCC-ITP achievements and advances in immunizations. Specific objectives of the assessment were to: • Determine the current functionality of the immunization system at national, regional, and local levels, including those national systems and processes that affect sub-national program performance or operations; • Assess the implementation of immunization program plans since the close of the MCC￾ITP; • Identify factors that affect sustained success of interventions like the MCC-ITP; and • Recommend a critical path for improvement and sustainability at regional and local levels. The questions to be answered in this Assessment, detailed in Annex B, are summarized below, in three areas of focus: 3 USAID/PERU MCC-IPT Assessment Report 1) Regional Immunization Programs. In each selected region, the assessment shall appraise the region’s capacity to deliver, and performance in delivery of, basic vaccines for children. a. What were the effects and achievements of the MCC-ITP in the region (at the end of the program, February 2011)? b. What is the post-ITP status of development of the regional immunization programs (as of October 2011)? c. How has the regional immunization program progressed since the close of the MCC￾ITP (what happened between February and October 2011)? 2) Immunization Performance in the Health Networks. Analysis of strengths, weaknesses, opportunities and threats (SWOT) at the local level. 3) Improvement and Sustainability Plan. Action plans to serve as roadmaps for other regional and local governments to use in strengthening their respective immunization programs. In discussions with USAID/Peru, it was agreed that a roadmap would consist of a vision for the future of the immunization programs and the critical steps needed to achieve this vision. A further clarification of USAID’s interest identified factors that might explain differences in coverage levels between regions and networks. METHODOLOGY The Assessment Team developed its methodology with emphasis on the complex requirements of the assignment. Team members initially reviewed the scope of work (SOW), the Social Impact proposal and the MCC-ITP planning documents, with special attention to questions posed by USAID in the SOW. In-depth document review focused special emphasis on several MCC-ITP baseline assessments,; the MCC-ITP contractor’s (Abt Associates) quarterly reports; consultant assessments of the supply system and information system; and a mid-term evaluation, using the project’s Results Framework and the three strategic objectives as a guide. The Team, working with Social Impact staff and technical division, held a series of virtual Team planning meetings (TPM) that confirmed the approach of the assessment, identified additional key documents, and selected key contacts for interviews and data collection in the U.S. and in Peru. TPMs included in-depth discussions of the focus of the assessment and the rationale and organization of the questions in the SOW, and further identified documents and individuals as sources of additional information. During review of key documents, the Team recognized that although the SOW’s questions emphasized the regions, the methodology needed to assess each level for its support to the three project focus areas: service delivery, logistics/supply, and information systems. The Team reviewed baseline studies, tools produced, coverage datasets and program reports for a deeper understanding of the situation. In-depth interviews were conducted in the U.S. with MCC-ITP contractors, including Abt Associates, Johns Hopkins Communications Project, and the mid-term team at CAMRIS USAID/PERU MCC-IPT Assessment Report 4 International. The Team’s Peruvian member made contact with key informants for advance notice and scheduling of interviews in Peru, including the Pan American Health Organization (PAHO), CARE, MOH, Ministry of Economics and Finance (MEF) and the United Nations Children Fund (UNICEF). In the initial meeting with USAID/Peru, the discussion clarified USAID’s priorities, specific interests, and concerns for the assessment. USAID emphasized lessons learned from the MCC-ITP to define what should be done for better results in Peru’s ongoing immunization program and how to improve the MCC threshold process. USAID expressed particular interest in how to strengthen regional immunization systems and capacity within the complex decentralization taking place and voiced concern about the discrepancies between MOH and National Institute of Statistics and Informatics (INEI) coverage figures, the expansion of the number of vaccines, and decentralization’s effect on the vaccination program. SITE SELECTION The Team, working with Social Impact staff and technical division, evaluated coverage levels as presented in the SOW to select the regions with highest and lowest rates, while trying to maintain some geographic dispersion. The Team conferred with the Peruvian consultant for advice on security concerns in the possible selected regions and then proposed to USAID/Peru the following regions for intensive study: • Amazonas • Apurimac • Cusco • Puno This selection was reviewed during the first Team-briefing meeting and was approved by USAID/Peru (see Annex G.1 for maps of sites visited in the four selected regions). The Team carried out in-depth key informant interviews with the (former) Peruvian MCC-ITP chief of party (COP), Team members, and consultants, using key questions and prompts to cover the three main areas (services, logistics, and information). The Team also conducted an in-depth interview with the supply-chain-assessment13 Team, and with representatives of other USAID￾supported health projects, USAID/Health Policy and USAID/Quality Healthcare. Introductory meetings were held with MOH Office of Foreign Cooperation and the Health Services Directorate. The Team prepared a short presentation to brief MOH officials in preparation for the in-depth interviews of and data collection from individual divisions and departments that support immunizations. The Team met with the INEI to clarify coverage issues as well as concepts and practices surrounding the Continuous Demographic and Health Survey (C-DHS). The Team also collected databases from INEI and processed information from the 2010 C-DHS, using the Statistical 13 “Plan de Mejora de Cadena de Suministro de Vacunas del MOH,” Dr. Hernán García, Dra. Cecilia Lengua 5 USAID/PERU MCC-IPT Assessment Report Package for the Social Sciences (SPSS) to evaluate regional vaccination coverage and associated factors. The Team developed tools for data collection at the levels of the regional government (President and Social Development Manager); regional health team (Director General, Director of Health Services, Immunization Coordinator and other departments); supply system manager; facility Teams; and IBs. These tools are provided in Annex C. Two surveys attempted to gather information from the other twenty regions of Peru. An instrument was faxed by the PCM to all presidents of regional governments, and another to the regional health directors. None responded. Another survey questionnaire was e-mailed to the list of 1884 registrants in the Virtual Diploma Course on Immunization Management (VDIM). Two-hundred-thirty-two participants responded (14%); their responses were tabulated and notable quotations were extracted for this report, in Annex G.4. THREATS TO VALIDITY The methodology had the following limitations: • The validity of generalizing Team observations in four regions to the seventeen regions supported by the MCC-ITP was a limitation. Direct observations and data collection were done in four regions; however, these were complemented with data and information collected on other regions and on the national status of the immunization program in document reviews, through analysis of technical reports and interviews with national authorities. • Analysis and validation of the data on immunization coverage was a key issue. The Team cross-checked data from the C-DHS, health information system (HIS), Integrated Health Insurance system (SIS) and IIIS in order to “interpret the data available, attempt to ascertain and adjust for possible biases and arrive at the most accurate estimate of immunization coverage.”14 FINDINGS The most salient results of the Team’s document review, interviews and direct observations are developed in the following sections regarding: • Overall achievements of MCC-ITP • Status of immunization programs at end of MCC-ITP • Differences in regions/networks 14 USAID Immunization Essentials: A Practical Field Guide, 2003, Bos, Edward and Amie Batson, “Using Immunization coverage Rates for Monitoring Health Sector Performance: Measurement and Interpretation Issues”. HPN Discussion Paper, Washington, D.C., The World Bank. August 2000. USAID/PERU MCC-IPT Assessment Report 6 • Status of regional programs and networks today • Other issues affecting the status of the NIS OVERALL ACHIEVEMENTS OF MCC-ITP The overall, general findings on achievement of the MCC-ITP are summarized in the following paragraphs. Specific findings related to the three MCC-ITP design components are presented in the later sections, as follows: 1) Ensure immunizations of rural children, 2) Strengthen immunization management and logistics systems and 3) Strengthen Immunization Information System in MOH at central and decentralized levels. The MCC-ITP fulfilled contractual obligations, delivered high-quality products, and made important contributions to the NIS and regional immunization strategies (RIS). However, due to the brief implementation period and external delays, design adjustments and mid-term corrections could not be made to address the rapidly changing context brought about by decentralization, increasing complexity of vaccination schedule, and new financing mechanisms. Some tasks were still in process when the program closed and many of these tasks have not been completed. Four important studies were completed under the MCC-ITP, giving a clear view of the NIS at the time of the program: logistics, norms and procedures, knowledge-attitudes-practices (KAP), and information systems. Many of the findings from these studies were used as inputs to the principal activities of MCC-ITP, including strengthening the cold chain, training of IB members and RIS staff, design and implementation of the IIIS, and creation of an information-education-communication (IEC) toolbox (these studies were also used by other MOH strategies and directorates as well as the MEF). The MCC-ITP created and implemented a Virtual Diploma for Immunization Management (VDIM) course, from which 856 health professionals graduated. Many of those trained (67% of a spontaneous-response national survey, Annex G.4) are still working in immunizations and 97% consider that it improved their competency. However, the course was institutionalized in neither the MOH nor a university and has not been continued. The Team observed and interviewed staff in the selected regions who were competent, motivated, and qualified for the complex task of managing the region’s immunization program. Many were trained through the VDIM. Ensure immunizations of rural children One MCC-ITP priority was immunization of rural children using the IB strategy. Five-member IBs provide basic health services to dispersed and excluded rural populations and were to be strengthened by the program through the training and equipping of IBs. 7 USAID/PERU MCC-IPT Assessment Report 145 IBs trained and equipped The MCC-ITP trained 786 members of IBs with a series of seven training modules. The IBs are composed of five professionals—physician, nurse, dentist, obstetrician, and practical nurse. The MCC-ITP also trained sixty-six trainers to continue preparation of future IBs. The MCC-ITP equipped 146 IBs in all seventeen supported regions with twenty-two types of equipment in varying amounts. Equipment included blood pressure monitors, glucose meters, stethoscopes, fetascopes, fetal Dopplers, manual resuscitation equipment, minor surgery kits, and other items. The MCC-ITP also provided dental tools, growth-monitoring instruments, and camping equipment and clothing for the IBs who traveled to and stayed in remote areas where shelter and cooking facilities were not available. Equipment initially was purchased by the MCC-ITP at the sum of $931,680,15 with replacement of worn and damaged equipment to be the responsibility of regional governments and DIRESAs. In September 2008, at the beginning of implementation, the MCC-ITP found that only 23 IBs were operational, instead of the 145 indicated in the program design. This required strong advocacy by the MCC-ITP for the recruitment and training of additional brigades and special financing by the GOP, all of which the MCC-ITP managed to achieve in the first months of the program. The 146 trained and equipped IBs ensured the vaccination of the rural children in their coverage areas. However, the number of children vaccinated by the IBs could not be determined, because the reporting system aggregated data from IBs with those of the health facility from which they worked. Strengthen immunization management and logistics systems The NIS became stronger and very complex during the MCC-ITP with support from the GOP and MCC-ITP: • 7,000 health staff were contracted or appointed to health facilities to manage vaccinations, replacing health technicians • The number of vaccines in the national immunization schedule increased from four to fifteen • National and regional drug management authorities exerted tighter control over supplies and cold chain • Regional immunization coordinators made better use of social communication • Regional immunization coordinators monitored coverage more closely with the analytic tables and Rapid Coverage Monitoring Much of the strengthening of management and logistics systems for the NIS was supported by the MCC-ITP through studies, training and introduction of norms—for example, most regional vaccination coordinators were trained through the VDIM, which was based on then-current vaccination standards. 15 Procurement Report, USAID/ITP, Abt Assoc., Lima 2009. USAID/PERU MCC-IPT Assessment Report 8 Studies of Immunization Management, Cold Chain and Logistics, Information System and KAP Four important studies were completed under the MCC-ITP, giving a clear vision of the NIS at the time of the program: logistics, norms and procedures, knowledge-attitudes-practices (KAP), and information systems. Many of the findings from these studies were used as inputs to the principal activities of MCC-ITP, including strengthening the cold chain, training of IB members and RIS staff, design and implementation of the IIIS, and creation of an IEC toolbox. (These studies were also used by other MOH strategies and directorates as well as the MEF). Virtual Diploma course on Immunization Management The VDIM was completed by 856 professionals, most of whom received certificates for their studies (63% of a spontaneous-response, national e-mail survey, Annex 5). The course covered six, key areas of management of immunization programs and was highly appreciated by participants. The course was created and conducted by MCC-ITP personnel using the MOH website platform to disseminate course materials and receive assignments from students. MCC-ITP recruited, selected and contracted facilitators who monitored student performance and graded their work. The University of Trujillo certified academic credit. Despite its success, the course was not institutionalized and has not been continued. Cold Chain Inventory, Cold Chain Training The MCC-ITP updated the cold chain inventory by training cold chain technicians in all regions to enter information on cold chain assets procured after 2007 into the MEF´s Integrated System for Administrative Management (SIGA-patrimonio) information system. Additionally, staff of Regional Directorates for Drugs and Medical Supplies (DIREMID) in each region was trained to use the drug information system (SISMED) to track stocks of vaccines, order new stocks and ensure adequate provision of vaccination inputs, based on RIS annual plans. The PCM advocated for installation and operation in all regions of cold rooms that were purchased by the GOP in 2006. The cold rooms ensure safe warehousing of heat-sensitive vaccines received by the regions prior to their distribution and use in health facilities. IEC Toolbox After completing the KAP study, the MCC-ITP achieved the planned communications and advocacy goals with creation of the IEC toolbox, which included print and electronic materials in five languages for adaptation and use in all regions. 9 USAID/PERU MCC-IPT Assessment Report The IEC toolboxes were distributed to regions in the closing days of the MCC-ITP and, subsequently, to some health networks, but without the benefit of training in their use and adaptation. Strengthen Immunization Information System in MOH at central and decentralized levels IIIS integrated in MOH and 1246 health centers The MCC-ITP partially achieved this output. It designed, created, and pilot tested an Integrated Immunization Information System (IIIS) as a subsystem of the Integrated Health Insurance System. The IIIS was officially sanctioned in October 2010 for national implementation. Pilot tests in seven micro-networks were completed successfully and statistics-information staffs in seventeen regions were trained in the IIIS. Computers were purchased and distributed, one each, to 1,246 health facilities. These computers had Windows Office and antivirus software. However, they did not have printers (at the request of the MOH) and, in many cases, did not have Internet connections. Nor did they have dedicated software to monitor vaccinations in the health units. In the pilot tests, the IIIS collected vaccination information for each child vaccinated in MOH facilities, including data for children not eligible for SIS insurance. It produced local vaccination reports for individual children, as well as facility-level coverage. Web reports were tested to ensure that geo-referenced information of vaccination coverage and cold chain functionality could be plotted on detailed maps. However, the SIS did not issue usernames and passwords to RIS coordinators for them to be able to access the IIIS database. The IIIS never did capture vaccination information from non-MOH services, either public or private sector. 145 IBs have PDAs and radios and use them properly The MCC-ITP procured and distributed 145 personal digital assistants (PDAs) to DIRESAs for distribution to the IBs in their region. Although the original MCC-ITP program specified procurement of radios for the IBs, the MOH determined that radios were already operational in remote, fixed health facilities and that distribution to the IBs would duplicate installed capacity. At MOH’s request, the project was amended to include donation of PDAs for each IB instead of the radios. MCC-ITP created specialized software for the PDAs in order to register and report vaccinations administered by the IBs and trained in the use of the PDAs and software. The specialized software interfaced with the IIIS-SIS, but not with the MOH’s HIS. GIS implemented in eight (selected) regions The MCC-ITP created a geographic information system (GIS) to map vaccination coverage and cold chain functionality, down to the micro-network level. The GIS was implemented on the WHO HealthMapper platform and a detailed geo-referenced database was created for eight regions. Training was held for regional statistics-information staff, as well as MOH epidemiology personnel. The HealthMapper software and GIS database were distributed to the trained staff, along with a detailed manual for operation. USAID/PERU MCC-IPT Assessment Report 10 However, after the pilot test, the SIS did not issue usernames and passwords to RIS coordinators or epidemiology staff for them to be able to access the IIIS database and create georeferenced vaccination information. STATUS OF IMMUNIZATION PROGRAMS AT END OF MCC-ITP Political support and priority accorded to vaccinations—national level The GOP is strongly committed to vaccinations, as indicated by actions mentioned in the following paragraphs. Vaccinations are an integral part of the GOP’s strategic commitment to decrease chronic malnutrition in children through the Articulated Nutritional Program or Programa Articulado Nutricional (PAN). The justification is that rotavirus and pneumococcus immunization will reduce diarrheal and respiratory disease incidence and prevalence and thus avoid repeated, acute malnutrition crises that result in chronic malnutrition. The PCM, the high-level, Peruvian counterpart of the MCC, sponsored and coordinated the design and submission of the MCC-ITP proposal. During the ITP , the PCM provided support to both components (immunizations and anti-corruption) and after the close of the MCC-ITP, the PCM continued to monitor the installation of regional cold rooms, insisting that regional authorities commit funds and contract construction and connections. The Assessment Team met repeatedly with the PCM manager for the ITP and received support from the PCM office to contact regional authorities and organize the national immunization workshop as a part of this assessment. Ministerial Resolution RM 457-2009/MOH of July 9, 2009 signaled MOH’s approval of the “National Scheme and Calendar of Vaccination,” designed to provide and distribute fourteen basic vaccines in the public and private sectors, and included definition of coverage indicators, their analysis, and reporting. This resolution defined the roles and responsibilities of MOH, DIRESAs, and health facilities. The MEF has ensured that the high political priority assigned to vaccinations is backed by necessary financial resources. The MEF has established the Budget-for-Results (PpR) mechanism to define results and cost all necessary inputs for the government’s high-priority, strategic programs. The Assessment Team found that all the regional immunization coordinators accessed the MEF website and effectively managed proposals for PpR funding, having been trained and advised by MCC-ITP. Regional government political support and priority Interviews with regional government authorities found political support for vaccinations. The regional president of Amazonas identified “vaccinations at the 100% level” as one of his three health priorities for 2012. He mentioned radio messages as a method to increase community interest and action to immunize the children and discussed using IBs to vaccinate dispersed, indigenous populations. He also spoke of his desire to work with mayors to orient them on vaccination and other health priorities, and to recognize those who improve health/vaccination indicators. 11 USAID/PERU MCC-IPT Assessment Report In Apurimac, the manager of social development indicated his three health priorities to be: decreasing chronic malnutrition; decreasing intra-family violence; and increasing the population with national identification documents (DNIs) for access to services. His target for vaccination coverage was 95%; when meeting with local mayors, he used their coverage indicators to urge municipal and district governments to increase their involvement in and support for vaccinations, health, and social programs. He received monthly reports of budget execution and used these data to ask health and education sectors about their performance. He collaborated with the MCC-ITP, was aware of staffing and training needs, and followed the immunization reports. He collaborated with the RIS Coordinator and will sign off on the RIS local census (needed to establish local populations for coverage – the coverage rate denominator). When made aware of the delay in cold room installation, he committed to working with the building contractor to complete the installation quickly. The regional government in Puno is very supportive of immunizations as a priority. The Team met with the development secretary and the representative for health issues, who confirmed that they are working with the coordinator of the RIS and the Regional Health Council, particularly the regional director of health and the regional president, to support vaccinations. Local political support and priority Some local officials provide support to vaccinations in public announcements and assistance with installation of cold rooms. However, the Assessment Team’s interviews confirmed the need to educate them about the importance of vaccinations in order to step up their leadership role in urging increased community response to vaccination campaigns. In Amazonas, the DIRESA identified the need to engage local authorities more actively, by seeking additional funding from mayors through the Municipal Incentives account, for example, for health facility/cold chain related and similar construction. Currently, structured orientation/training on health priorities is not provided to local authorities, who generally do not serve on health/social committees. DIRESA staff possess limited training on advocacy and limited skills and experience to engage local authorities and increase their support for vaccinations. While not widespread in the four Selected Regions, in some districts and towns, political and community support for vaccinations was not forthcoming and there were even negative reactions and actual blockages. For example, the Hepatitis B vaccination campaigns were opposed by some local authorities in Condorcanqui, Amazonas, and teachers in Puno were reported to oppose vaccinations because they believe that this vaccine causes cancer. The Thimerosal preservative in the influenza vaccine also provoked resistance from some local authorities and communities, and some religious leaders. Local health and government authorities have tried to respond to negative reactions from teachers, religious leaders, and others, but do not have convincing, consistent messages. Funding mechanisms The variety of funding sources for vaccinations presents a challenge to NIS and RIS coordinators and the network managers, who may not be aware of them or mechanisms to access them. USAID/PERU MCC-IPT Assessment Report 12 At the national level, funding for the NIS comes from the regular budget (RO), and covers the purchase of vaccines (using the PAHO’s revolving fund mechanism), purchase of supplies, the distribution of vaccines to regional storage points, as well as supervision and regulation activities. Funding for regional programs is more diverse, because it depends on regional or local capacities and priorities and includes: 1. Regular Budget (for payroll personnel) 2. Budget-for-Results, which covers short-term, contract personnel, operational expenses, and cold chain improvement. 3. Participatory Budgets, a mechanism mandated by law whereby regional or municipal governments (which can also be the DIRESA or health networks), may propose projects to be financed by the regional or local government budget. Some regions and networks utilize this to improve cold chains and health facilities. 4. Minor public investment projects (PIP Menor) is another mechanism whereby the National Investment System (SNIP) allows public agencies to apply for additional funds for investment purposes. The NIS has prepared a template that helps the RIS to apply for funds that can be used, for example, for cold chain renewal. Norms and Procedures The norms and procedures for immunization programs are extensive and provide information and guidance to regions on procedures for different activities, types of vaccines, rapid monitoring. Since the close of the MCC-ITP, the GOP has issued a supreme decree (SD) and three ministerial resolutions (RM) regarding immunizations. Two of the resolutions, Supreme Decree 006-2011-SA and RM 311-2011-MOH, promote increased immunizations awareness and activities by establishing a “National Vaccination Day” and a “National Vaccination Week,” respectively. RM 565-2011 describes procedures for mop-up campaigns for polio and MMR, including the Rapid Coverage Monitoring activity. RM 70-2011 includes a new vaccine against human papiloma virus and revised scheduling of pneumococcus vaccination in the national vaccination schedule. Human Resources In 2009, the NIS obtained a special earmark from the MEF to contract additional, recently trained health personnel to implement the national vaccination scheme at a time when new vaccines were being introduced and coverage levels had been slipping for the past four years. Regions used these funds to triple the number of health staff in health facilities, in many cases replacing health technicians, who had little training in vaccination procedures. Improved quality of service, including observation of norms for placement of injections and improved counseling, is attributed to this increase in quantity and quality of professional staff. 13 USAID/PERU MCC-IPT Assessment Report Managerial and administrative capacity to plan and implement and monitor immunizations The regional immunization coordinators of the selected regions had all taken the VDIM course and were very well versed in all aspects of planning, implementation, and monitoring of their RIS. All had prepared annual operational plans, had presented budgets using the PpR system, had planned and conducted trainings for facility staff and cold chain operators, and were monitoring not only vaccination coverage at network and micro-network levels, but also their levels of budget execution and achievement of results specified in the PpR. Additionally, they indicated specified staffing levels and competencies for facilities and participated in contracting, as well as evaluation, of vaccination personnel. Similarly, many of the network immunization coordinators of the selected regions had taken the VDIM course. They, in turn, had prepared their own annual operational plans and budgets in the PpR system, trained facility staff, and monitored vaccination coverage rates in their network and micro-networks, as well as achievement of results planned in the PpR. At the facility level, vaccination supervisors were well trained and experienced in administering vaccines, counseling, and recording their activities. All were up to date on the national vaccination schedule and monitored their cold chain to assure availability of vaccines, syringes and associated materials. The Team noted, however, that the assistants to the vaccination supervisors had more limited training and were less able to manage the full range of the vaccination program responsibilities. When discussed with NIS managers, they agreed that managerial capacity at the network level needed to be strengthened for more in-depth capacity. Information and epidemiological surveillance The MOH Epidemiology Directorate (DGE) has a four-person unit devoted to surveillance of vaccine preventable (VP) diseases. The DGE issues weekly epidemiological bulletins with a section on VP diseases, including whooping cough, diphtheria and tetanus (DPT), flaccid paralysis (polio), measles, and mumps, as well as acute diarrheal disease and respiratory infections, especially pneumonia. They do not participate in decisions about which vaccines are needed, as discussed in the section below. Degree of protection achieved, level of risk and epidemic potential There are two principal concerns reported by the VP disease surveillance unit: 1. Pockets of unvaccinated populations, especially in border regions (the measles outbreak in Ecuador is of major concern because of reportedly low levels of MMR vaccination rates in Condorcanqui and Jaen, on Peru’s northern border). 2. The number of susceptible persons who do not react to vaccination, i.e., do not make antibodies after vaccination and are therefore not protected by the vaccine against disease. Based on international experience16 16 For example, Deficiency of the Humoral Immune. Response to Measles Vaccine in Infants Immunized at Age 6 Months, JAMA, 1998. http://jama.ama-assn.org/content/280/6/527.full.pdf , the DGE estimates that in Peru there are 300,000 such USAID/PERU MCC-IPT Assessment Report 14 persons susceptible to mumps and measles and perhaps 500,000 susceptible to polio.17 Complete serological studies have not been conducted in Peru. The National Institute of Health has suggested that a serological study could be conducted to determine levels of susceptibility and reasons for lack of immunological response. Such a study might cost as little as $10,000.18 Involvement of national surveillance system in detecting and controlling vaccine-preventable (VP) diseases The VP disease surveillance unit is responsible for monitoring and investigating all reported cases of VP diseases from over 7,000 health facilities in the country, through trained regional epidemiologists. The VP disease surveillance unit commented on the selection and introduction of new vaccines in the national immunization schedule, pointing out that baseline immunological studies were not performed because Peru does not have laboratory diagnostic capability to analyze the DNA of disease genotypes. Thus, decisions about new vaccines were based on studies from other countries, which have other epidemiological conditions. They opined that vaccination should be focused on diseases that cause, or potentially cause, high mortality. However, the DGE does not participate in the NIS Advisory Committee, which makes decisions about which vaccines to include in the national schedule, and therefore does not have a voice in changes to the vaccination strategy. When the Assessment Team asked about the MCC-ITP’s impact on increasing vaccination levels, the VP surveillance unit stated that vaccination coverage rates were three to four percent higher in the MCC-ITP target regions. To support the national VP surveillance system, the MCC-ITP program developed the GIS information system and trained professionals in the DGE in its use. Unfortunately, as explained above, after training VP surveillance personnel did not have access to the IIIS and GIS databases to conduct the surveillance. Cold chain functionality The PCM has been insisting vigorously on installation of the 52 cold rooms purchased in 2008 by the MOH for central and regional sites. Since the purchase price only covered equipment and not installation, each region was responsible for construction and electrical connections. Annex G.2 provides data from NIS on the status of the installation of the cold rooms across all regions with data on the sites, numbers of cold rooms, and functionality, as well as the status of the construction or placement within existing structures. This report shows 29 (56%) of the cold rooms functioning and nine (17%) classified as not functioning (i.e. correctly installed and functional but missing some external component or service, such as a voltage stabilizer). Another 14 (26%) cold rooms were considered non-functional (i.e. not completely built or installed) for various reasons. 17 Interview with VP disease surveillance unit, DGE/MOH, Lima 2011. 18 Interview with César Cabezas, Director of National Institute of Health, Lima 2011. 15 USAID/PERU MCC-IPT Assessment Report To assess the status of cold rooms in the four selected regions, the Team met with the PCM, Directorate of Drugs and Medical Supplies (DIGEMID), NIS and regional staff. Table 1 shows the current status of the cold rooms in the selected regions visited. Table 1: Status of cold rooms in selected regions REGION COLD ROOMS AS OF OCTOBER 2011 STATUS AMAZONAS Completed October 2010 Operating APURIMAC Not functioning Installed but lacking electrical connection CUSCO Two operating since 2010 Operating PUNO Non-functional Building under construction To complete the electrical installation of the cold room in Apurimac, the contractor will be contacted by the regional manager for social development and it should be functioning by the end of 2011. In Puno, due to the construction of a new building to house the cold room, the cold room is non-functional. The installation in Puno is expected to be functioning in 2012. In 2008, the responsibilities of the DIGEMID for cold chain management were transferred to the regional governments, where the cold chain coordinator in each DIRESA now controls the inventory of refrigerators and freezers for vaccination storage throughout the region. In an upcoming meeting, the Intergovernmental Health Commission will discuss strengthening regional staff competencies to manage the cold chain and DIGEMID plans to issue a directive in December 2011 that will clearly identify the appropriate cold chain roles and functions at the central, regional and local levels. The Team’s observations of the cold chain functionality in the sixteen facilities in the four regions was done with a structured assessment tool and revealed the following: • All of the sixteen facilities visited store vaccines in refrigerators and freezers that are new (between two and five years old), are electric, and function adequately (consistent temperatures according to norms). In some areas of the regions, e.g., rural Apurimac and Puno, there are still a number of domestic/household refrigerators; reportedly, these are to be replaced within the next year. All cold chain equipment is registered in the MEF´s Asset Information System (SIGA patrimonial). Maintenance is provided by the DIREMID on a rapid-response basis in all regions to resolve problems with the cold chain equipment. Preventive maintenance programs are not in place, although in Apurimac, a preventive maintenance plan is nearly completed. All staff report cleaning and defrosting the refrigerators—some each month, others when ice forms inside. In general, there is not a schedule posted for this cleaning and defrosting. • All staff in the facilities reported that they have adequate stock for routine vaccinations. However, a small percentage does not have adequate stock for campaigns in their micro￾networks. Most micro-networks and facilities’ staff report that they can easily get additional stock from their DIRESA and brief stock-outs were reported in only a few establishments. USAID/PERU MCC-IPT Assessment Report 16 Most reported that they had been trained in cold chain management and all staff report using MOH norms as their guide to managing the cold chain. These norms are available in summary form in hard copy in all establishments and some have a full version of the norms as an electronic copy. • All the refrigerators and freezers had thermometers on the front; most also had small thermometers inside the refrigerators and freezers, and all have data loggers to monitor the temperatures of the equipment. The refrigerators and freezers did not have alarms; however, a new type of refrigerator, soon to arrive, reportedly comes with alarms. All the establishments visited were monitoring and recording temperatures, which were found to be within the acceptable range for safe vaccine storage. Only a few facilities reported damaged vaccines, which were returned to the DIRESA. During the visit to one rural micro-network, a power outage occurred and the Team observed staff frequently monitoring the temperature in the refrigerator, which rose rapidly. The staff prepared for the transfer of vaccines to thermos boxes and monitored the temperature of the vaccines in the thermos boxes using the data logger. (The temperature should not have risen as rapidly as it did and this machine was to be checked by the technician.) • Only a very few sites visited had received voltage stabilizers to protect the cold chain equipment from spikes in electric power that could damage the equipment. A number of the freezers and refrigerators were connected to extension cords instead of directly into outlets, and often wires were crisscrossed across the floor near the equipment. Few sites had marked the outlet/plug “do not unplug,” and in two rooms, the refrigerators and freezers were in areas where unauthorized persons could enter. For further data on cold chain functioning at the regional level, the Team obtained the “February 2011 Inventory of Cold Chain Equipment” for the Apurimac region. The data revealed that there were, as of February 2011, 241 refrigerators across 305 facilities (some have two). Another 64 establishments do not have refrigerators, perhaps due to the small number of children in their catchment area. The vast majority of the refrigerators are powered by electricity and fifteen are solar powered. Of the 241 refrigerators in the region, 131 (54%) were put into service before 2000. Biosecurity practices The Team observed biosecurity practices as part of its assessment of the facilities, using the structured survey tool. The Team found that all facilities used disposable syringes and most had autoretractible syringes, at least for the tuberculosis vaccination (BCG). Availability varied across facilities. Most immunization services discarded their syringes (without recapping them) into MOH￾provided, cardboard security boxes made with a small hole at the top for discarding used syringes, but not allowing a hand to enter. Most immunization services discarded empty vaccine ampoules in the security boxes with the syringes. Some facilities, especially in the urban areas, did not have a place to discard or destroy filled security boxes. In rural areas, the security boxes were put into fenced pits and sometimes were set on fire by staff. A few urban facilities took their filled security boxes to a nearby hospital to have them incinerated with the hospitals’ other refuse. Others are awaiting agreements with municipalities to dispose of medical waste. In one facility, the staff was storing the filled security boxes because the 17 USAID/PERU MCC-IPT Assessment Report hospital did not have an incinerator or other disposal mechanism and they feared putting the filled security boxes into the regular trash collection. Achievements in social communication and user satisfaction. NIS and MCC-ITP worked closely to develop activities to strengthen and improve communications surrounding vaccinations. The MCC-ITP social communication component was developed using evidence from the MCC-ITP KAP study, presented in 201019 to MOH technical personnel, including regional staff and other organizations. This baseline study was conducted with the express purpose of providing information to health professionals at several levels and to IB teams regarding the knowledge, attitudes, beliefs and actions of mothers, fathers, family members, local leaders, teachers, and others that affect their acceptance and seeking of vaccinations. This study was a key component of the MCC-ITP baseline assessment of the immunization system and was the evidence base for later work with the media to increase their commitment and their actions in support of immunizations. The final quarterly report from MCC-ITP stated that “community-level communication and education campaigns for behavior change regarding immunizations were implemented in all seventeen regions.”20 Print and video communications materials, customized to specific findings of the KAP study, currently are being used in vaccination campaigns. In addition, materials for nominal community monitoring of vaccinations (vigilancia comunitaria) were delivered to regional governments and have been distributed to health facilities at the community level. Finally, a political advocacy video was created to draw national attention to and promote understanding of the work of IBs and the need to reform the health-care model in areas where “itinerant brigades are the only alternative.” An IEC Toolbox21 was developed as a guide for health staff to communicate with patients; other tools were for communications staff and the media. The Team reviewed the Toolbox and found a series of targeted instructions developed for social actors, media, health workers, and users, families, and communities. Television spots were developed with four messages in five languages; the same message was also prepared as radio spots in five languages. Material in the toolbox repeated the primary message, “La mamá sabe—una es ninguna. Todas las vacunas para su edad. ¡Vacúnalo!” A brochure developed by the national Health Promotion Directorate contained a series of eighteen themes for school-aged children, with lessons taught by their teachers. The children, in turn, were to discuss these themes with their parents at home. The brochure was included in the toolbox, even though vaccination was the very last theme. The message, to receive all vaccinations before two years, did not support the focus of NIS, which promotes all vaccines according to age. The Team’s observations in the four regions found very limited availability and use of the IEC Toolbox, although reportedly, 200 copies were distributed by the MCC-ITP. A few copies were available, mostly in the DIRESAs and urban micro-networks, but interviews revealed that they had not been used to develop communication or health promotion plans or activities. However, the 19 “Si Su Niño Esta sano, Nosotros Somos Mejores, Conocimientos, Actitudes y Practicas en Turno a la Vacuna, Estudio Cualitativo,” Programa MCC-ITP , Abt Assoc., Lima 2011 20 Tercer Informe Treimestral, Programa MCC-ITP , Abt Assoc. Lima 2010 21 Caja de Herramientas Comunicacionales. Programa MCC-ITP , Abt Assoc. Lima 2011 USAID/PERU MCC-IPT Assessment Report 18 communications director of DIRESA/Apurimac has used the brochures and other tools, in Spanish and in Quechua. She informed the Team that she has plans to train 100 newspaper media persons and give them the messages after these are properly adapted for the region. The technical norm for IBs22 noted that the training would include themes of health promotion and intercultural approaches. One of the seven MCC-ITP training modules for IB teams was focused on communications. The one IB team interviewed did not use the IEC tools in its work. In health facilities, the Team observed health staff communicating with parents whose children were being vaccinated. They communicated actively, explaining the vaccinations, discussing any adverse reactions to be expected, counseling on home-based care of fever, swelling, and pain. Nurses also reviewed the vaccination schedule with the parents, explaining and marking the date of the next visit on the growth chart. No negative communications by staff were observed and, in general, their interactions were warm and supportive to the parents. The Team’s discussions with parents after the vaccinations found that, although the nurse explained the vaccination schedule, parents were not able to say which vaccines had been administered. This perhaps is due to the large number of vaccines and to names unrelated to the diseases they prevent. The Team also observed that communications were generally one-way, from the staff to the mother, and staff did not appear to be able to draw mothers into dialogue. Regional capacity to adapt to new demands generated by health reform, decentralization, technologies, insurance, etc. RIS coordinators and related staff show strong interest in addressing the new demands generated by the many changes in the health and political systems and in technologies. For example, in Puno, the DIRESA vision and mission statements prioritized the broader integrated/family health intervention as defined in the MOH family health model23. The demands of decentralization, health insurance schemes, and other technologies also require adaptation. For example, RIS staff develops and adapts the regional plan based on national norms to produce their RIS, which defines implementation mechanisms: staffing, timelines, budgets, supervision, and indicators for monitoring. Some of the regions are further along in this adaptation process. These regional adaptations were collected in the operational immunization plans (POI) in the four regions visited. In the Puno POI, the Team found well-developed presentations based on MOH norms regarding immunization schedule, priority vaccines, historical coverage by districts (Pentavalente 3 and SPR 2003–2009), RIS organizational and functional structure, staffing, and resources. The POI included the targets, strategies, activities and the specific targets by quarter and persons assigned as responsible, as well as a calendar of activities and a budget. The Puno POI demonstrates how staff adapts, assuming responsibility under decentralization for defining the specifics of their planning and programming, and assigning responsibilities for implementation. Another example is adoption of new technologies, such as autoretractable syringes, adaptation to increased storage requirements of mono-dose vaccines, the use of data loggers with digital computer 22Norma Técnica: Atención Integral de Salud a Poblaciones Excluidas y Dispersas. (pg 23) RM-478-2009/MOH, Lima 2009. 23 Modelo de Atención Integral de Salud Basado en familia y Comunidad, RM No. 464-2011, MINSA, Lima. 19 USAID/PERU MCC-IPT Assessment Report readouts to monitor temperatures of the horizontal refrigerators and freezers, and the use of cell phones to notify parents of pending vaccinations for their children. The Condorcanqui region adapted upright refrigerators so they would lie horizontally and adapted condensers to work with solar power. Personnel readily adopted distance-learning technology for the VDIM course and all staff interviewed had personal e-mail addresses, although many facilities were not yet connected to the Internet. The Amazonas DIRESA used technology to develop the NovaHIS software for its health information system. The regional president of Amazonas discussed the use of the bar codes on the DNI to give access to social services and allow tracking of immunizations and other services. DIFFERENCES IN REGIONS/NETWORKS An important consideration in the achievements of the MCC-ITP and this assessment was the effects of decentralization on regional immunization programs. The MOH implemented a monitoring and evaluation of decentralization (MED) program in 2009 to assess the assumption of regional government functions in health by DIRESAs. The baseline assessment found a median of 34% of regional competency: Puno (30%), Amazonas (32%), Cusco (34%) and Apurimac (36%). These levels compare to a minimum of 20% and maximum of 62% in other regions. Thus, in 2009, there were no great differences in regional management and administrative capacity between the selected regions at the time of the MED. The Assessment Team observed virtually universal compliance with national vaccination norms in networks, micro-networks and facilities which ensure that service delivery is uniform across regions and networks. The Team attributed differences between facilities to individual management and leadership qualities of staff. Regional management of the immunization strategy was also uniform. Coordinators had been trained in the virtual diploma course and were skilled in the use of the SIGA-PpR for planning and budgeting. Levels of staffing were proportional to served populations and monitoring plans were similar. All regions had annual operational plans, monitored vaccination coverage in networks and facilities, conducted training and moved staff to optimize skills. MOH vaccination norms are uniformly observed by regional Teams and service providers who have identified and vaccinated virtually every child in their catchment areas. They all complain that their assigned populations (based on 2007 census figures) do not reflect their true catchment populations because of migrations and seasonal movements. As a result of this assessment, the problem recently has been recognized by INEI. The NIS also acknowledges this problem and is working with regional and network immunization Teams to start making local child censuses that will correct it. Regarding the VDIM, almost 25% of participants gave their residence as Lima— with 5.2% from Puno, 2.8% from Cusco, 1.9% from Apurimac and 1.4 % from Amazonas. Many of the VDIM graduates are still working in immunizations (67% of a spontaneous-response, national e-mail survey, Annex 5) and 98% consider that it improved their competency. USAID/PERU MCC-IPT Assessment Report 20 The frequent turnover of regional health directors was a source of regional instability, but second￾tier managers were stable, experienced and highly trained. For example, the RIS coordinators in Amazonas and Apurimac had been in their positions for seven and nine years, respectively, and both had completed the VDIM course. According to the analytic tables produced by immunization services in health facilities and used by RIS coordinators to monitor coverage, (i.e., the parallel immunization information system) real coverage differences between facilities exist. These are the result of different individual leadership and management skills and are being dealt with in each region through training and personnel actions. The important differences between the selected regions are related to: • Adoption of the IIIS in Apurimac and rejection or disuse in Amazonas, Puno and Cusco. The rejection of the IIIS in Amazonas has more to do with the personal objectives of the Statistics Director and the implementation of a nominal HIS that he designed. In Puno and Cusco, it appears that disuse is due to lack of a champion to promote continued expansion from pilot micro-networks to the rest of the region. • Level of support to IBs. After the exceptional injection of funds from MEF in 2008 to regions in support of IBs, further responsibility for financing has been left to the regional governments and DIRESAs. Individual directors of health have had different views of the IBs and their frequent turnover has precluded establishment of stable policies. In the absence of clear priorities, the IBs have languished, with reductions in number in three regions (Amazonas, Apurimac and Puno) and only Cusco maintain its five IBs. • Installation and operation of cold rooms in Cusco and Amazonas vs. delayed installations in Apurimac and Puno. This difference is one of time: both Apurimac and Puno are completing their installations and both cold rooms will be operational in early 2012. • The use the IEC Toolbox by the institutional image communications officer in Apurimac and relative disuse in Amazonas, Puno and Cusco. The communications officer took individual initiative to utilize the Toolbox. In the other regions, the health promotion director held responsibility and had ignored the toolbox. • Presence of anti-vaccination groups in Puno and Amazonas, especially motivated by community opinion leaders, viz. teachers and religious figures. • Communications and advocacy with local governments. The differences observed were, again, more attributable to individual initiative than to institutional capacity. That being said, a clear, social-communication objective and appropriate materials and training could make a more uniform advocacy program. STATUS OF REGIONAL PROGRAMS AND NETWORKS TODAY The four observed, regional-immunization programs followed NIS norms, were well staffed, and have operational plans and excellent institutional support. All RIS coordinators are experienced and 21 USAID/PERU MCC-IPT Assessment Report well trained, and manage their programs effectively including staffing, planning and budgeting. All had good relations with regional and/or local governments. Of the VDIM graduates, 67% are still working in various immunization activities in their regions, according to the email survey conducted by the assessment (Annex 5) and 97% consider that it improved their competency. Cold chains are all effective (despite non-operational cold rooms in Puno and Apurimac) and stock outages are minimal. DIREMID staff in each selected region is monitoring temperatures using data loggers and is controlling vaccine stocks using the SISMED information system. These digital controls are more precise than analog thermometers and paper records provide a history of temperature changes over several days and weeks. These have been enthusiastically adopted by cold chain technicians after appropriate MCC-ITP training and have strengthened the immunization cold chain. The 145 IBs trained and equipped by the MCC-ITP were a key component of the MCC-ITP strategy, intended to reach over 500,000 rural Peruvians. However, because of their low numbers and the difficult terrain to cover, they could not reach more than a small fraction of that population. In comparison with IBs during the MCC-ITP implementation period, the current IBs suffer from high turnover, reduced training and budget restrictions and, as a result are incomplete, fewer in number and tend to work in fixed sites rather than as mobile teams. The delayed formalization of the Integrated Immunization Information System in Ministerial Resolution RM 614 came very late in the MCC-ITP so that implementation was not completed by the end of the program. Without a champion for IIIS implementation to conduct training and monitor its use, the IIIS has not been implemented beyond the initial pilot areas and is ignored in most regions. Issues yet to be resolved in all regions are: • Definition of regional and local coverage populations • Disposal of medical vaccination waste • Unification of remunerations for similar work under different contractual conditions OTHER ISSUES AFFECTING THE STATUS OF VACCINATION PROGRAMS Validity of Coverage Indicators: • Despite increased numbers of vaccine doses reportedly administered during the MCC-ITP, the MOH national vaccination statistics for 2008, 2009 and 2010 showed that coverage did not increase significantly and failed to reach the 95% safety threshold goal. DOSES/YEAR 2008 2009 2010 COVERAGE/YEAR 2008 2009 2010 DPT3 494478 551504 548965 DPT3 92% 93% 92% USAID/PERU MCC-IPT Assessment Report 22 This is because the 2007 census corrected the technically questioned 2005 census used as the baseline when designing the MCC ITP and, as a result, the population as reported in 2009 increased by nearly 10 percent. This raised the coverage denominator (which is calculated by the MOH General Office of Statistics and Informatics (OGEI) for future years) in almost direct proportion to the increased number of vaccine doses administered between 2008 and 2010, wiping out any improvement in coverage rate. • The MOH national vaccination statistics, which were the basis for MCC-ITP design assumptions and were relied upon for program monitoring, cover only public facilities and do not capture vaccinations provided by the private sector, which account for 23% of vaccinations, according the C-DHS24. Nonetheless, the MOH reports over 90% coverage for the whole country, which means that either many children are receiving more than one dose of a given vaccine, or reported doses are exaggerated. In either case, MOH-reported coverage rates are overestimated by nearly 20%. CONCLUSIONS FACTORS THAT FACILITATED SUCCESS IN THE REGIONS/NETWORKS 1. Trained personnel is the most significant factor for success. The large number of qualified professionals working in key positions of the immunization system is, in great part, thanks to the MCC-ITP VDIM. In general, the VDIM diploma has been considered in making assignments and advancements within the immunization system. 2. Excellent MCC-ITP regional coordinators facilitated success in the regions and gained support from both regional and local governments as well as network and facility staff. 3. Strong and continued, high-level support from the PCM and MOH was a factor in the success of achieving special financing for IBs and for insisting on cold-room installation in regions that had not done so by the end of the MCC-ITP. 4. The declaration of immunizations as a priority program under the strategic Programa Articulado Nutricional (PAN) to reduce malnutrition and the application of Budget-for￾Results to finance the NIS and RIS were key factors for continued and increased financing for immunizations in all the regions. 5. Ability to affect national systems and processes (including procurement, training, staffing, financing and epidemiologic surveillance) that affect sub-national program performance and decentralized operations have been formalized in norms and ministerial resolutions, in part due to MCC-ITP training and advocacy. 6. Need for manual and computerized “parallel” systems, during and after MCC-ITP, to meet local management information needs in public facilities, but these systems do not receive information from the private sector and are not inputs to the national HIS so cannot contribute to a national vaccination-coverage-rate indicator. 24 The team processed 2010 C-DHS data. 23 USAID/PERU MCC-IPT Assessment Report 7. Broad participation of health professionals in trainings that led to prioritization of the immunization system at the national and regional levels and effective operation of the immunization program. FACTORS THAT INHIBITED SUCCESS IN THE REGIONS/NETWORKS The MOH´s dual role as both official reporter of national vaccination coverage and steward of regional vaccination provision has not been fully regulated. The MOH does not have a clear strategy to receive information about and report on the vaccinations provided by institutions outside of the public sector nor does it have a consistent decentralized management model. 1. Reliance on government approval actions. The MOH took unexpectedly long to officially sanction the IIIS - too late for it to be fully implemented. As a result, it has fallen into disuse in most regions. The lack of a functioning, effective, national information system impedes the monitoring of public and private vaccinations. 2. Itinerant brigades as a vaccination strategy. The number of IBs (145 at the end of MCC￾ITP) could not and did not significantly increase vaccination coverage in the country. Currently, regional government support for IBs in the selected regions is limited because of cost and perceived lack of benefits and they are not a significant component of the NIS, despite announced central-level support. 3. The 30-month implementation period was too short for institutionalization of many functions, such as the IIIS and VDIM course, follow up was not pursued beyond the end of the ITP. 4. Effects of decentralization. Regional governments, which under decentralization have autonomy for prioritizing and budgeting immunizations activities within their regions, are not uniformly supportive of vaccinations. In addition, the frequent turnover of regional health directors25 8. Lack of timely studies for program design. Sizable time lags between design and implementation allowed for significant changes in circumstances and affected validity of design assumptions. was a source of instability, but second-tier managers were stable, experienced and highly trained. They have sustained the success of interventions implemented by the MCC-ITP in all four selected regions. FACTORS THAT INHIBITED SUSTAINABILITY OF ACHIEVEMENTS 1. The VIDM was not institutionalized by MOH or university, which would have made it more sustainable. 2. Institutional rivalries led to non-ownership of IIIS and disuse of reports. The SIS hosts the IIIS software but does not feel responsible for regional implementation or continued updates based on changes in norms and regulations. The OGEI is responsible for 25 Regional health directors were reported to have changed frequently in the last nine months as a result of the regional elections which changed regional administrations (e.g. Puno with 3). The Team also found that people perceived greater instability due to this turnover, however, beyond those perceptions the team does not have additional evidence that turnover is greater now than before decentralization. USAID/PERU MCC-IPT Assessment Report 24 immunization statistics from its own HIS and does not use SIS reports. There is no champion to continue use of the system. 3. Late approval of legal status of IIIS led to incomplete implementation. No further extension of the IIIS occurred after pilot tests at the end of the MCC-ITP. 4. Regional priorities not supportive of IB. In the face of competing financing requirements, IBs have become a second priority in three of the four regions. 5. Lack of valid of coverage indicators. The sustainability of vaccination achievements cannot be measured, because coverage indicators from the MOH and estimates from C￾DHS are not comparable: - MOH data are incomplete. - Coverage rate denominators (assigned population) are unreliable due to migrations and seasonal movements. - The C-DHS collects data on vaccinations delivered to a child cohort 18 to 29 months before the year in question. Thus, the C-DHS estimates for 2010 cannot establish MCC-ITP end-of-project vaccination status and cannot be compared to MOH figures for 2010. - Discrepancy between numbers of doses and coverage: The NIS reported increases in numbers of doses of DPT3 and MMR from 2007 to 2009. While these numbers may be overestimated, the corrected 2007 census increased the population and therefore, the vaccination coverage denominator, thus erasing any improvement in coverage rates. - Vaccination coverage differences between regions. The C-DHS estimates do not give statistically significant differences between regions and apparent differences are probably due to population migrations and seasonal movements. The one-time household survey conducted by MCC-ITP did not publish confidence intervals so differences in coverage estimates between regions are ambiguous. RECOMMENDATIONS • A national, nominal information system capable of collecting and reporting all public and private vaccinations should be finalized. • The NIS’ analytic immunization information system should be standardized and implemented on MCC-ITP-donated computers for day-to-day management of vaccinations in all of the regions. • The VDIM course should be institutionalized, either in a university or through the MOH. Immunization has become very complex and all staff involved should take the course. • IBs as a strategy for service outreach to dispersed and excluded populations should be evaluated, including cost and effectiveness in reaching these populations, as well IB and staff roles within DIRESAs and contribution to immunization coverage. • The IEC Toolbox could be an important resource to overcome cultural barriers to vaccination. It should become part of the VDIM, in order to be used sustainably at operational levels. 25 USAID/PERU MCC-IPT Assessment Report ROADMAPS, VISION AND CRITICAL STEPS At the end of this assignment, each of the selected regions had a simple, actionable plan to improve their performance and sustain their training and cold chain management. Each plan was tailored to their unique circumstances and the findings of this assessment. This report provides the regional plans in Annex A. LESSONS LEARNED • Given the short nature of MCC’s threshold programs, flexibility in program design and implementation of activities is imperative in order to achieve objectives. Adjust MCC Threshold plans according to changes in interval between design and implementation. • Ownership of program activities by the host country is imperative for sustainability and must be planned for in the design. Handover is not the same as ownership. The brief nature of MCC-ITP means that it is implemented at a much faster pace than host country institutional activities. • Monitoring and evaluation indicators should measure the strategies’ effectiveness and should be timely and reliable. Unreliable coverage indicators make monitoring of project outcomes and measurement of impact a difficult task. This coincides with Lesson one of Lessons Learned reported by MCC on December 16, 2010: 26 • Where the legal basis for interventions is important for institutionalization, but delays in achieving legal status can endanger implementation, especially in the short timeframe of a threshold program. “Link threshold programs to indicators and goals that are actionable and measurable within a relatively short period of time.” • Rigorously evaluate design assumptions. Design assumptions are critical to achieving desired outcomes in the short period of a threshold program. Sizable time lags between design and implementation may invalidate design assumptions. In the MCC-ITP design, untested assumptions (e.g. the location of unvaccinated children) and incomplete diagnoses (e.g. the number of functional IBs at the beginning of the MCC-ITP) led to outcomes and impacts below those planned in project preparation (Lesson two of MCC: “Deepen diagnostic and feasibility analysis and identify the connection of activities to outputs, outcomes and impacts during program preparation”). • For MCC-ITP, the creation of a national immunization information system was overly ambitious, given the institutional impediments, delays in official sanctioning, and complexity of the undertaking, although the consultative process did allow MCC-ITP to change from the Health Information System (HIS) platform to the SIS in order to achieve pilot testing (MCC Lesson three: “Be more selective when determining program interventions and establish a consultative process to tailor focus areas”). 26 MCC Threshold Program Lessons Learned, . USAID/PERU MCC-IPT Assessment Report 26 • The MCC-ITP outcome goal of “Integrated Immunization Information System improved and implemented in the MOH and 1,246 Health Centers”27 • Buy-ins and ownership should not be assumed. No assurance for activities such as training for IBs and institutionalization of the VDIM were built into the program design beyond the efforts to gain ministerial sanction for the IIIS. MCC Lesson six: “Build in sustainability assurances during program design and early implementation.” was inappropriate for the two￾year (later increased to 30 months) timeframe (MCC Lesson four: “Establish outcome-level goals that are appropriate for the timeframe”). 27 USAID/PERU MCC-IPT Assessment Report ANNEXES USAID/PERU MCC-IPT Assessment Report 28 ANNEX A - THE PROPOSED IMPROVEMENT AND SUSTAINABILITY ACTION PLANS FOR EACH SELECTED REGION As indicated in section A of this report, it was agreed with USAID/Peru that the “roadmap” specified in the section IV.3 of the SOW, would consist of a vision of the immunization programs for the future and the critical steps needed to achieve the vision. This definition is also understood to encompass the “critical pathway” mentioned in the same section of the SOW. VISION All Peruvians receive all the vaccinations included in the NIS immunization schedule according to their age. For children, vaccinations are part of integrated child health services. Mothers receive appropriate counseling and information and their questions are answered. Vaccinations in public sector facilities are correctly noted in the child health card and on MOH forms and vaccination data from public and private facilities are correctly entered into national information systems which then provide timely reports to health staff for monitoring and analysis. Regional action plans, including: • A critical pathway for plan implementation • Draft budget and financial plan • Draft regulatory framework • Timeline • Specific responsibilities • A communication plan • A mechanism to monitor and evaluate its implementation at the regional and local levels The Assessment Team facilitated SWOT analyses with regional and network staff in each visited region. On November 8, 2011, the Team gathered representatives from central and regional levels to discuss the outcome of the regional SWOT analyses and develop regional critical paths to achieve the 95% coverage goal in 2012. This meeting had the goal of achieving consensus on critical activities and the required support from national levels to achieve 95% coverage rates. This meeting was designed to be part of the ongoing annual planning process of the NIS and will be followed by macro-regional meetings over the following weeks leading to a national planning meeting December 14–18. 1. The critical pathway identified for each region was different based on their unique needs in the course of the decentralization process in progress. However, common steps were identified to strengthen the management of the NIS and RIS. 29 USAID/PERU MCC-IPT Assessment Report Common critical pathway steps: - Retain, recruit, train, and evaluate the performance of immunization staff. Recognition of good performance needs to be abundant and frequent. - Define a methodology to determine coverage denominators to manage health facility activities efficiently - Improve information system to gather and monitor numerator data for local facility management and for national oversight. The information system needs to be nominal and capture other providers besides public health sector facilities such as pharmacies, private clinics, armed forces and non-governmental organizations (NGOs). - Coordination of all stakeholders at the district and regional levels. This is a new activity for health staff that is not used to working with the local mayors and will require training and sharing of good practices. - Training and support to improve the budgeting by results process by each microrred, and monitoring of budget execution. - Coordinated promotion and communication activities for better informed demand of regular and timely immunization services. (When will Peru be able to decide to reduce need for costly campaigns in the future?) - Create cold chain preventive maintenance plans and prioritize equipment replacement 2. Draft budget and financial plan. The budget and financial needs are included in next year’s budget. There may be need from technical assistance to solve selected bottlenecks such as planning and implementation of an integrating communication activities, integrated facility operations and decentralized information management. 3. Draft regulatory framework. Peru has a very good regulatory framework that is in the process of being adapted to a decentralized and integrated delivery system. New norms will be required to empower regional levels to make informed decisions and respond to local needs, including: - Norm to create and use local population census to calculate coverage - Procedure to distribute INEI regional populations down to district and province - Norm for preventive cold chain maintenance - Norm to integrate child services of the Family Health Model in facilities - Standards for human resource management: contracting and remuneration - Procedures for operation of the “parallel system” for information - Norm for vaccination reporting from the private sector - Define mechanisms for local government funding via SNIP and Municipal Incentives - Procedures to identify, capture and share best practices USAID/PERU MCC-IPT Assessment Report 30 4. Timeline. All of the activities planned will be implemented in 2012. 5. Specific responsibilities. The specific responsibilities of the national and decentralized regional levels are clearly defined in the accompanying Critical Steps tables. The national level sets the policy, i.e. the “what” to do. The Regional levels decide how to do, i.e. the best way to deliver this based on their current situation and needs. The technical assistance role of the central level needs to shift from a prescriptive role to a coaching and facilitating role to help the regional level figure out the best course of action. 6. A communication plan (see section 2 of this Annex A) 7. A mechanism to monitor and evaluate its implementation at the regional and local levels. The “parallel system” can become the source of monitoring and evaluation information to be exported to the HIS. The NIS should help define a small number of indicators to monitor the quality of care and performance of the health facilities. 31 USAID/PERU MCC-IPT Assessment Report Planes de Mejoramiento y Sostenibilidad Regionales En este anexo se presentan los resultados de los análisis FODA en las regiones y del consenso FODA a nivel nacional. Asimismo se presentan los pasos críticos definidos por cada región y sus planes para implementarlos en 2012. De los planes regionales se desprenden las siguientes recomendaciones: 1. El plan de la ESNI debe reafirmar el rol normador y de asistencia técnica del MOH a las regiones y de las regiones a las redes y microrredes. Las RISs tienen diferentes niveles de necesidades técnicas para hacer cumplir las normas. Se necesitan normas e indicadores de gestión de la estrategia y un plan para su implementación y monitoreo. 2. Las funciones centrales de asistencia técnica, información y monitoreo deben apoyar la gestión regional. Las capacidades de planificación varían entre las regiones lo cual impide un planificación clara y orientada a los resultados. 3. Se hizo evidente la necesidad de integración al nivel operativo de las diversas estrategias, así como la definición operativa del modelo integrado de asistencia. Los establecimientos visitados no cuentan con manuales operativos que describan el funcionamiento de los establecimientos: descripción de funciones, procesos y actividades asistenciales, así como prácticas de auto-evaluación de la calidad de atención. 4. Existen numerosas experiencias buenas y efectivas en las REDESS y micorredes por lo que se debe facilitar y promover el proceso de intercambio de experiencias y buenas prácticas entre las RISs y entre redes. Esta información serviría para desarrollar planes que respondan a las FODA locales y resolver cuellos de botellas. 5. Dirección de Estadística e Informática debe prepara norma para el ajuste de población para la gestión de cada establecimiento. La falta de un denominador de población real, no basado en proyecciones impide una planificación realista, una buena gestión y el monitoreo efectivo del desempeño. USAID/PERU MCC-IPT Assessment Report 32 CONTEXTO: FODA NACIONAL FORTALEZAS OPORTUNIDADES 1. Estrategia fuerte y bien organizada 2. Esquema de vacunación ejemplar 3. Fuerte compromiso político y financiero (PpR) 4. Cadena de frío y capacidad de almacenamiento en mejoramiento 5. Compromiso político a nivel MOH nacional 6. Personal profesional técnico y comprometido 7. Existe demanda de vacunas 8. Cooperación internacional incluyendo los insumos del MCC-ITP 9. Trabajo integral 10. Desarrollo de capacidades de gestión en manejo de proyectos para el mejoramiento de la cadena de frío 11. Disponibilidad de insumos 12. Economías de escala y adquisición de biológicos mediante Fondo Rotatorio de OPS 13. Proceso de monitoreo, supervisión, y asistencia técnica constante a las regiones 14. Instrumentos para evaluar los indicadores son socializados en todas las regiones y son de uso permanente pare desarrollar capacidades de gestión 1. Descentralización 2. Recursos de capacitación disponibles 3. Computadoras disponibles 4. Materiales de comunicación y promoción disponibles 5. DNI 6. PpR 7. Radio y televisión 8. Lazos con instituciones y sociedad civil 9. Descentralización presupuesta y de funciones 10. Mesas de concertación en favor de la infancia DEBILIDADES AMENAZAS 1. Transferencia de personal capacitado 2. Formatería 3. Dificultades en estimación de indicadores de cobertura 4. Definición del doble rol del MOH 5. Necesidad de capacitación continua a nivel operativo 6. Implementación incompleta de PpR a nivel de todas las UE 7. Sobremeta y submeta de coberturas 8. Poca difusión del impacto de la estrategia de inmunizaciones 9. Corresponsabilidad y compromiso de las autoridades 10. Articulación para el sostenimiento de los avances del proyecto MCC-ITP 11. Poco tiempo asignado a la educación del paciente 12. Uso y manejo de indicadores de cobertura y deserción 13. Incompleto ciclo de gestión de la información para la gestión y toma de decisiones: Captación, procesamiento y presentación del dato, Análisis, Retroalimentación, toma de Decisiones e Ejecución de cambios 14. Avance lento del sistema informático integrado de inmunizaciones para la captación y procesamiento de información 15. Avance lento del sistema de información geográfica. 16. Insuficiente personal capacitado 17. Falta de incentivos laborales 18. Capacidad limitada de laboratorios para investigación de casos de inmunoprevenibles 19. Poca capacidad para replicar entrenamientos 20. Equipos administrativos no priorizan la ejecución presupuestal 1. Demanda limitada 2. Alto grado de recambio de autoridades 3. 30% no alcanzado, apoyo de otros sectores no-MOH no articulado 4. Gobierno locales 5. Rotación de Personal 6. Sobre y subestimación censal pare la gestión de los EE.SS. 7. Procesos descentralizados no han sido normados todavía 8. Metodología de distribución de la población por edad y sexo en los EE. SS. no se ha definido. 9. Necesidad de mecanismos y procesos de monitoreo que evidencien y reconozcan el cumplimiento 10. Formación universitaria con escaza salud colectiva (comunitaria) 11. Temor de los padres a la vacunación simultanea de varias vacunas 12. Gobiernos regionales retrasados en la transferencia presupuestal 33 USAID/PERU MCC-IPT Assessment Report AMAZONAS ANÁLISIS FODA FORTALEZAS OPORTUNIDADES 1. El personal son enfermera(o)s profesionales, capaces y dedicados comprometidos a la vacunación. 2. Las normas muy completas son aplicadas por el personal. 3. Amazonas instaló su cámara fría en Octubre de 2010. 4. Muchos de las establecimientos tienen refrigeradora y congelador funcionando y la mayoría tiene vacunas suficentes. Usan data loggers 5. Hay Planes Anuales (POI, Plan Táctico) Buenas prácticas de vacunación 6. Los reportes de información sobre coberturas existen y están actualizados. 1. En MEF hay buena disposición para adoptar las poblaciones de padrones locales. 2. Utilizar los equipos de computación y conectarlos al Internet 3. PAN, PPR y SIS priorizan las inmunizaciones 4. Colaboración con las instituciones de Formación Locales 5. Fomentar capacitaciones avanzadas para incrementar competencias y estimular al personal. 6. Estimular al personal a crear, analizar y promover soluciones creativas, Ej. Software de seguimiento a niños. 7. Conseguir más recursos – formar una unidad para hacer proyectos/propuestas - seguimiento DEBILIDADES AMENAZAS 1. Capacidad gerencial no parece profunda en las redes y microrredes. 2. El estímulo mediante la supervisión es limitado y el personal se siente aislado en sus esfuerzos. 3. Demasiado tiempo en llenado de los registros diferentes en los EESS. 4. La cadena de frío no tiene un plan de mantenimiento preventivo y de renovación. 5. Los IB se redujeron de 11 a 5 y algunos están trabajando en el puestos fijos 6. Comunicación continua por radio y televisión limitada para estimular la populación de buscar las vacunas. 7. Utilización limitada de computadoras y PDAs. 8. Enlaces interinstitucionales débiles. 1. La migración y otros movimientos de la población dificultan la vacunación completa de los niños. 2. El tiempo de las enfermeras utilizado en papelería se puede utilizar en supervisión y apoyo. 3. La movilidad del personal de salud afecta la calidad de servicios y relaciones con la población. 4. Hay grupos activistas en contra de las vacunas, que necesitan manejo especializado USAID/PERU MCC-IPT Assessment Report 34 Pasos Críticos: Amazonas • Implementar el SIII a nivel de todos los EE.SS del ámbito regional • Desarrollar programas educativo comunicacionales con enfoque intercultural para reducir las barreras de rechazo a las inmunizaciones, en población urbana y rural. • Fortalecer competencias en los profesionales de enfermería involucrados en las actividades de la ESI (capacitaciones, diplomados en Gerencia de inmunizaciones, abogacía, liderazgo y PIP) • Desarrollar políticas regionales en gestión de recursos humanos para el sector, que garanticen su permanencia en los EE.SS. • Involucramiento del personal de salud en la ejecución de los presupuestos por incentivos municipales, para garantizar el buen uso de los mismos. • Las ONG, dentro de su quehacer, deben verificar que las inmunizaciones se aplican en los niños. AMAZONAS PARTICIPANTES: ESTHER, JUAN, SOLEDAD, SARA Y LIZ ACTIVIDADES RESPONSABLE INDICADOR NECESIDADES Implementar el SIII en todos los establecimientos ODSIS Capacitación de digitadores Concordancia de la información: SIS, HIS y # de dosis Apoyo de la Dirección técnica nacional Diseño e implementación de un programa educativo que utilice las herramientas del maletín de comunicación Promoción de salud y redes 43 kits de promoción en las redes y programa en implementación Debe incluirse en el PpR de 2012 Fortalecer competencias DIRESA Capacitación con RIS Convenio DIRESA-Universidades Incluir en PpR, imprimir módulos y financiar los gastos de los participantes Políticas de gestión de RR.HH. Gobierno Regional￾DIRESA Nivelación de remuneraciones Asistencia del nivel nacional Involucrar el personal con el programa de incentivos municipales DIRESA, DISA y Municipios Número de municipios con PIM programado en inmunizaciones Plan regional de trabajo integrado con municipios distritales ONG verifica inmunizaciones DIRESA Concordancia del número de dosis y tipo de vacuna con niños atendidos por la ONG Listado de niños atendidos por ONG en pilotos y verificar 35 USAID/PERU MCC-IPT Assessment Report APURIMAC ANÁLISIS FODA FORTALEZAS OPORTUNIDADES 1. Hay stock suficiente en los establecimientos y el stock de la DIRESA está accesible. 2. El cuidado clínico, consejería y referencia se hace muy bien y de acuerdo a las normas de MOH. 3. La microrred no solo da servicios pero supervisa, da apoyo, y monitorea los establecimientos en su micro red. 4. El personal recupen niños con visitas a casas, spots por radio, colaboración con obstetrices. 5. En muchos comunidades, el personal conoce a todas las familias, y saben quienes y cuando tienen que vacunarse. 6. Hay monitoreo la cadena de frío constantemente, moviendo vacunas según el plan de contingencia. 1. En MEF hay buena disposición para adoptar las poblaciones de padrones locales. 2. Hay recursos en los presupuestos municipales que pueden ser convocados para las ESI. 3. El SIS/AUS paga por los servicios – incentivar producción y reportes, transparentar valores hacia micro-redes. 4. El SIS/AUS produce información valiosa de inmunizaciones y CRED. 5. Una relación entre la DIRESA y las escuelas de enfermería puede hacer la educación pre-servicio mas apropiada para las necesidades del sector. 6. El reconocimiento por DIRESA y autoridades al personal puede realzar la importancia de su trabajo. El reconocimiento debe ser por mérito. DEBILIDADES AMENAZAS 1. Población asignada está sobrestimada (denominador). Diferentes fechas de corte para la recolección de información. 2. Los sistemas son de recolección de datos y no producen información útil para la gerencia o los establecimientos. Muchos formatos para cada servicio. 3. La cámara fría todavía no funciona. El equipo de la cadena de frío a nivel de establecimientos no tiene mantenimiento preventivo y un plan de renovación de equipos. 4. Cambios de personal capacitado y experimentado debilitan fuertemente los sistemas de información. El recambio afecta a la confianza de la comunidad. 5. Capacitación no se replica en forma sistemática a nivel operativo. 6. La proporción del personal contratado es alta y los contractos son cortos, con grandes diferencias en la remuneración. Falta reconocimiento por logros y desempeño. 7. La Caja de Herramientas para promoción y comunicación tiene uso limitado 1. Las municipalidades se benefician de una población sobrestimada. 2. La formación de nuevos profesionales no responde a las necesidades del sector. USAID/PERU MCC-IPT Assessment Report 36 Pasos Críticos: Apurimac • Sistemas de Información – Plan de mantenimiento a equipos y software – Capacitación al personal de información y prestadores – Minimizar rotación del personal capacitado – Fortalecer capacidad de análisis del personal de microrred, socializar y tomar decisiones – Plan de recopilación, registro, análisis y almacenamiento de datos, difusión de información y toma de decisiones. – Conexión a Internet en todos los puntos de digitación. – Unificar sistemas y registros – exportación del SIS al HIS (RM614), ensayar una tarjeta DNI con banda magnética para ingreso a todo sistema de servicio social público. • Cadena de Frío – Arreglar con el contratista de la cámara de frío en 30 días – Plan de mantenimiento preventivo y reposición. – Priorizar equipamiento con cadena de frío y grupos electrógenos en distritos de acuerdo a la realidad. – Incluir financiamiento para cadena de frío en PPR. • Servicios – Mejorar asignación presupuestal. MINSA/Calidad que gestione con sectoristas de salud en MEF para financiar clima organizacional en PPR. – Equilibrar escalas de remuneración. – Mejorar eficiencia, eficacia y clima organizacional (comunicación interna para socializar esfuerzos de la DIRESA) trabajo en equipo. • Otros – Estabilizar criterios informáticos y programáticos en SIGA PPR – Fortalecer el sistema de referencia y contrarreferencia – Registrar vacunados con DNI a nivel nacional. 37 USAID/PERU MCC-IPT Assessment Report APURIMAC PARTICIPANTES: LIC. TEOFILIO CACERES, DR. CARLOS ALFREDO VILLANUEVA LUNA, DR. JORGE UCHUYA. DR. FREDY PAJUELO, Y LIC. ACTIVIDADES RESPONSABLE INDICADOR NECESIDADES Sistema Cadena de Frió Desarrollar Plan de mantenimiento preventivo y reemplazo—Cadena De Frió. ESRI y DIRESA Criterios para re-emplazamiento aprobado por ¿DARES? PpR financiamiento en 2012 Acceso a los fundos de municipios Iniciar compra de neveras según las prioridades # de neveras identificada para reemplazar, usando los criterio. PpR financiamiento en 2012 Acceso a los fundos de municipios Sistema - Recursos Humanos Desarrollar estudio de RH y Plan de reestructurar RH para estabilidad, salarios, beneficios, incentivos RH, ESRI, Gobierno Regional Asignaciones duran uno ano por lo menús Asistencia técnica experto RG, Incremento presupuesto PpR 2012 Estabilizar criterios informáticos y programáticos en SIGA PPR ¿?? ¿?? ¿?? Sistema de Información Fortalecer la implementación del SIII a nivel Regional Comité técnico implementador del SIII Concordancia de la información: SIS, Informe analìtico # de dosis Asistencia técnica del nivel Nacional Formar grupo de trabajo con estadística, ESRI, SIS y epidemiología Estadistica, SIS.ESRI Grupo de trabajo formado Plan Regional Hacer estudio/definición necesidades del sistema de información ESRI, SIS y Epidemiología Sistema nominal utilizado como norma aprobada Apoyo—SIS regional Intercambio con Macroregión ESRI, PCM Desarrollar módulos de análisis de la información Estadísticas, epidemiología ESRI, SIS Asistencia SIS y ESRI Capacitación al personal- recopilación y registro de datos ESRI, SIS regional Asistencia SIS Prestar apoyo técnico a las microrredes y establecimientos de salud ESRI Incremento de % de registro completada correctamente Incluir fondos en PpR 2012 Capacitación del personal nuevo en la recopilación y registro de datos SIS, ESRI Nº de personas capacitados Plan Regional de capacitación Fortalecer capacidades para el procesamiento de los datos brutos (Base de datos SIS) SIS. ESRI, Epidemiología Nº de personas capacitadas en procesamiento de datos Asistencia técnica del nivel Nacional USAID/PERU MCC-IPT Assessment Report 38 CUSCO ANÁLISIS FODA FORTALEZAS OPORTUNIDADES 1. Personal capacitado 2. Cámaras frías y data loggers 3. Calidad del dato 4. Abastecimiento oportuno de insumos y materiales 5. Sostenimiento de coberturas de vacunación. 1. Implementación de un sistema de información integral 2. Aparición de nuevas vacunas 3. Intensificar el avance del programa Identidad 4. Incluir ESNI dentro del PAN 5. Programa de Incentivos Municipales 6. Asistencia Técnica para el aprovechamiento 7. de las oportunidades (Ej. PpR) DEBILIDADES AMENAZAS 1. Sinceramiento de la información 2. Retraso en la información 3. Ausencia del registro nominal del niño 4. Alta población migrante 5. Exceso de formatos 6. Disposición de residuos post vacuna 7. Ausencia de un sistema de información único 8. La gestión de las inmunizaciones aun no esta 9. totalmente descentralizada. 10. El sistema de información no esta en red. 1. Población migrante 2. Disposición final de cajas seguras (Requiere apoyo de los municipios) 3. Sobrecarga y multiplicidad de labores para el personal. 4. Exceso de formatos 5. Sobreestimación de la población (INEI) Pasos Críticos: Cusco 1. Descentralización de los programas (Ej.. CRED si se ha podido hacer adecuación) 2. Programación de metas físicas que se aproxime más a lo real. 3. PpR ajustado a las necesidades 4. Retomar el trabajo extramural (la carga administrativa resta tiempo) 5. Difusión sostenida de los beneficios de la vacunación, especialmente por televisión nacional. 6. Fortalecer la consejería en cada contacto con la madre. 7. Implementación de la codificación HIS. 39 USAID/PERU MCC-IPT Assessment Report CUSCO PARTICIPANTES: LIC. TANIA SALDIVAR TAPIA, HILDA PILLCO, LILIAM LIMA ACTIVIDADES RESPONSABLE INDICADOR NECESIDADES 1. Programación de metas físicas de acuerdo a la población nominal versus HIS DIRESA, RIS, Redes y establecimientos # de redes con padrón nominal / total de Redes Apoyo técnico del INEI, RIS, colaboración de USAID y cooperantes, decisiones políticas, participación ciudadana Fortalecer el trabajo extramural de la RIS para el seguimiento de los niños DIRESA, RIS, Redes y establecimientos Tasa de deserción 2. Fortalecer el sistema de comunicación para la difusión de los beneficios de la vacunación ESNI, DIRESA Estudio de impacto de sintonía, tasa de deserción nacional Plan de apoyo de cooperantes Apoyo de ESNI Existe una estrategia nacional para la difusión de vacunas ESNI, DIRESA Fortalecer el trabajo comunitario a través de los COVICOS ESNI, DIRESA Empoderar a los gobiernos locales para apoyar la RIS ESNI, DIRESA 3. Fortalecer las capacidades de los recursos humanos a nivel operativo mediante la elaboración de planes descentralizados de recursos humanos ESNI, DIRESA # de personal capacitado/ total de personal # de personal certificado/ total de personal capacitado Elaborar plan de capacitación Plan de gestión de recursos humanos ESNI, DIRESA USAID/PERU MCC-IPT Assessment Report 40 PUNO ANÁLISIS FODA FORTALEZAS OPORTUNIDADES 1. Trabajo en equipo RIS-SIS-DIREMID-Estadística 2. Libro de Nacimientos 3. Seguimiento oportuno para CRED y Vacunas 4. El personal se capacita permanentemente para la Estrategia 1. Coordinación con Programa Familia Saludable - Fortalecer la programación en PpR 2012 - En los criterios de Familia Saludable, debería estar que todos los miembros de la familia estén vacunados, según edad - Convenio con Educación para que el niño tenga vacunas completas al matricularse, según edades - Incrementar el personal o las horas disponibles para Familia Saludable - Propiciar el fortalecimiento del Primer Nivel de Atención 2. Reconocer el esfuerzo del personal 3. Concientizar el personal de salud que vacunan a sus niños DEBILIDADES AMENAZAS 1. Necesidad de personal - Contrataciones (preferente, renovación) para enero 2012 - Evaluación del personal: Diciembre 2011 - Programación de necesidades: Noviembre 2011 - Capacitación - Coordinación con Colegios para que la capacitación )Ej.. Módulo) sean válidos para recertificación - Convenio con Universidades para que institucionalicen y den validez a la capacitación, o la incorporen en sus programas de pregrado 1. Población sobre-estimada 2. Demanda informada 3. Contra-campaña política desde el nivel central 41 USAID/PERU MCC-IPT Assessment Report Pasos Críticos: Puno 1. Adecuada programación y monitoreo del PpR 2. Compromiso político de las autoridades e involucramiento de las mamás. 3. Recursos Humanos (Contratos, etc.) 4. Reconocimiento de la población nominal. 5. Garantizar la dotación oportuna de biológicos. 6. Garantizar la cadena de frío. 7. Capacitación continua Indicadores 1. Coberturas 2. Número de personal 3. Rendimiento del personal / Productividad 4. Porcentaje de ejecución según lo programado 5. Concordancia entre utilización de biológicos y número de vacunados. 6. Concordancia entre recursos presupuestados y niños vacunados (Costo por niño protegido). USAID/PERU MCC-IPT Assessment Report 42 PUNO PARTICIPANTES: LIC. HONORATA ORTEGA B, LIC. ALICIA QUISPE QUISPE, LIC INGRID ROJAS CASTRO, MARIA DICONA ACTIVIDADES RESPONSABLE INDICADOR NECESIDADES Elaborar cuadro de necesidades de cada establecimiento para que sus PpR incluyan todos los componentes de la RIS Jefes de EE.SS y la coordinadora de inmunizaciones Número de EE.SS que cuentan con cuadro de necesidades/ total de EE.SS Realizar capacitaciones y reuniones técnicas pare elaborar el formato y los instructivos para elaborar el cuadro. Abogacía con participantes de PROMS y las autoridades regionales y de RED de salud: compromiso político PROMS, ESRP, RIS Renovación del contrato, aprobación de PIP menor Reuniones de coordinación internas previas Ampliación de plazas vacantes para enfermeras, técnicos de enfermería y digitadores Director general, RIS, Directores de REDES Mantener 171 enfermeras y agregar 50% más y digitadores para cada Establecimiento Tener un plan de necesidades de personal por REDESS y EE.SS Evaluación del desempeño del personal a fin de renovar sus contratos e identificar necesidades de capacitación RIS y Director de capacitación % de profesionales aprobados / total de personal evaluado Elaborar instrumentos de evaluación y auto-evaluación, listados de chequeo e indicadores de desempeño Formar convenios con universidades para que su formación responda a las necesidades y normatividad del MOH - urgente Nivel de Asistencia técnica y RIS Número y % de convenios firmados Reunión de Directores generales para coordinación a través del C.R.D. S. Elaborar un instrumento o software para evaluar el trabajo diario, y producción por PpR ESNI Instrumento elaborado y usado por la RIS Reunión nacional de programación y evaluación Elaborar plan de mantenimiento y renovación de la cadena de frío, 20% cada año durante 5 años por orden de prioridades Dirección General, RIS Plan elaborado y el 20% de los equipos renovados cada año Inventario actualizado cada año y análisis de prioridades Capacitación y evaluación trimestral con temas transcendentales como uso de data logggers, vacuna segura ESNI, Dirección regional, RIS Nivel de desempeño de personal en las supervisiones Realizar supervisiones de acuerdo al formato de evaluación 43 USAID/PERU MCC-IPT Assessment Report Communication Plan in support of the Regional Action and Sustainability Plans A communication plan will be developed in each region by a Team of ESRI. A regional Team comprised of ESRI, Communications, Promotion, Planning divisions and others will first clarify the objectives of the Communications Plan (CP) its audiences, what messages/support is intended and the methods and resources available. The purpose of the CP will be to assure support for the goals, activities and progress of the Regional Action Plans. As a first step the Team will assess the resources needed. They will identify the staff capacity, time available, the potential assignments, the needed budget, the technologies to be used, and the major activities needed. The Team will also identify technical assistance support needed and identify potential sources (Communications Office of MINSA, NGOs specializing in communications, local university communications experts) who can provide technical guidance to formulate the CP, the strategy and message development. A second step will be a clear identification of the audience/s to be reached with messages. For the four regions’ Action Plans, the core/first audience will be the “internal” audience of including ESRI, DIRIMED, DIRESA, and the immunization staff in establishments, AISPED Teams and others who need to receive the Communication Plan’s message of the Region’s mission, critical steps and the Action Plan to increase immunization coverage. The second audience is the regional and local municipal officials who will receive CP messages to orient and motivate them to support immunizations/Action Plans. A third major audience is the community leaders, teachers, families and others who influence mothers’ decisions about immunizations. These local actors will receive messages to inform and motivate them to actively promote immunizations, counteract negative beliefs and behaviors, and motivate their communities to actively seek vaccinations and actively cooperate with immunization campaign and barridas. The final and most crucial audience is the mother/parent; the one/s most directly involved in assuring that their children receive all vaccinations according to the schedule. The C-Team will conduct a rapid scan of the needs and resources available. Information about the audiences will be assembled from the MCC-ITP KAP study, other research, interviews with anthropology, sociology and public health experts and immunization services coordinators and managers. A series of key informant interviews and focus groups will be done (with modest amounts of technical assistance) to augment the evidence base for the development of messages of knowledge, attitudes, beliefs, barriers, etc. in each audience that affects immunizations. With this information, a clear message will be developed to inform, motivate, and promote actions to be taken with tailoring to each of the key audiences. Although the message is tailored, the clear, consistent content of the message will be presented across all audiences and will be focused promoting actions to achieve the result. For example, the messages to regional and municipal officials and media will inform and motivate them, encourage their support to routine vaccinations at health establishments and during campaigns and mop-ups and suggest actions they can take as officials and media persons to motivate communities, and provide logistics, publicity and other support to vaccination activities. The Team will work with local communications experts to develop messages and/or adapt ITP Communications toolbox materials for posters, and Mother’s histories, flipcharts, brochures, workbooks as well as radio and TV spots. Games for school children with immunization related USAID/PERU MCC-IPT Assessment Report 44 messages will also be developed to engage them in carrying messages home to their families. Staff will be encouraged use the tools to develop/adapt messages and the Team will provide feedback. To diffuse the messages by mass media, a survey of media outlets will be done, considering their capacity to broadcast to the priority audiences in the timeslots, language and with the “appeal " needed. The Team will orient and encourage media to broadcast messages, conduct on air interviews, act as public advocates and otherwise use their media to inform and motivate the public. Early in the CP process, the Team will identify technologies that will be used to support diffusion of messages. For example, to deliver messages and otherwise motivate facility staff, frequent Internet messaging (e.g. Facebook) will encourage staff to actively expand vaccination activities. Likewise posting the progress of the Action Plans activities on website and actively encouraging staff to visit these web sites will greatly increase the diffusion of information across regions, allowing staff to learn from other regions and create competition. The Team can encourage posting of pictures of immunization activities, graphs of coverage, and can encourage widespread posting of messages. The Team will meet regularly to monitor progress and rapidly reprogram and adapt as needed, based on their observations of the activities, and feedback they actively seek from staff, community leaders and other audiences. 45 USAID/PERU MCC-IPT Assessment Report ANNEX B - STATEMENT OF WORK (SOW) FOR THE ASSESSMENT, INCLUDING MODIFICATIONS TO THE SOW Contract No. RAN-I-00-09-00019 TO No. AID-527-TO-11-00001 Social Impact, Inc. Page No. 4 of 31 SECTION C – STATEMENT OF WORK I. PURPOSE The Millennium Challenge Corporation (MCC) Immunizations Threshold Program (ITP) Team within the Office of Health of USAID/Peru intends to conduct an assessment of selected regional immunizations programs (hereinafter the “Assessment”). The purpose of the Assessment is to inform USAID/Peru, the MCC, and the Government of Peru on current status of national and sub￾national (regional and local) immunizations programs since the close of the MCC-ITP, to identify factors that facilitate and/or challenge success of medium-term projects such as the MCC-ITP, and to provide actionable recommendations to increase sustainability of MCC-ITP achievements and advances in immunizations. Specific objectives of the Assessment are to: • Determine the current functionality of these immunizations systems at national and subnational (regional and local) levels, including those national systems and processes that affect sub-national program performance or operations; • Assess the implementation of immunization program plans since the close of the MCCITP; • Identify factors that facilitate and factors that challenge sustained success of interventions such as the MCC-ITP; and • Recommend critical pathways for improvement and sustainability at the subnational levels. The period of performance is estimated at no more than seventeen weeks, from August 31, 2011 to December 28, 2011. II. SCOPE The Assessment will cover from the close of the MCC-ITP in selected regions on or before February, 2011 through the current date. III. BACKGROUND From October 2008 through February 2011, USAID/Peru contracted with Abt Associates, Inc. to implement the MCC-ITP to support the Peruvian immunizations system and increase immunization rates. One of the main objectives of the MCC-ITP was to increase the vaccine coverage for measles and the third dose of diphtheria-pertussis-tetanus (DPT3) above the safety threshold of 95% in eight targeted regions: Amazonas, Apurimac, Ayacucho, Cusco, Huancavelica, Huánuco, Ica and Puno. During implementation the MCC-ITP faced numerous challenges, many related to the changes in roles and responsibilities caused by decentralization of Peru’s health system. At the beginning of the USAID/PERU MCC-IPT Assessment Report 46 program, the MCC-ITP’s initial assessment of the targeted regions highlighted a weakening of resources (such as human resources, medical equipment, vaccine storage equipment, supplies, and training) for health care facilities and questioned the accuracy of health information available. When MCC approved the Threshold Program for Peru in 2008, systems of the Ministry of Health (MOH) such as logistics and information were centrally managed and operated. The subsequent decentralization of Peru’s public health sector rapidly transferred many health system functions from the central MOH to regional governments. The regions had varying levels of management capacity to implement these new functions, and not all the aspects of the system were ready to function in a decentralized manner. For example, many regional governments in the regions targeted by the MCC-ITP had neither the information nor the capacity necessary to operate, plan, budget, and request resources to finance the human resources, vaccine supplies, and management of immunizations programs. Regional abilities to increase immunizations coverage rates were also affected by additional challenges, such as a dramatic expansion of Peru’s national vaccination schedule, changes in the official population count within regions, and MOH difficulties with timely procurement of vaccines and equipment. To overcome these challenges and achieve MCC-ITP goals, the program worked with the MOH and regional governments to build capacity and competency in health personnel; strengthen itinerant brigades in selected regions, i.e., roving medical Teams that bring basic health services, including vaccinations, to dispersed populations; analyze and improve vaccine logistics; develop regional immunizations improvement plans; and implement an integrated immunizations information system, at national and subnational levels, that relies on identification of patients by name (and therefore does not require a national identity card, which many rural Peruvians do not have). Across all 26 regions of Peru, the MCC-ITP donated 1,250 computers and provided technical assistance to improve and implement the integrated immunizations information system. (Please see http://inmunizaciones.programaumbral.pe/ for more information on MCCITP deliverables.) In the areas of behavior change, communication and advocacy, the MCC-ITP worked with regions to generate demand for vaccinations within target populations and to build capacity of local healthcare providers to communicate within a gender sensitive and culturally respectful manner. In the seventeen regions of Peru using itinerant brigades to reach rural populations and the MCC-ITP provided equipment and training for the brigades; technical assistance in design and implementation of communications strategies and regional immunizations plans; and technical assistance in the planning, budgeting, and financing of immunizations operations. In June 2010, USAID/Peru awarded CAMRIS International a contract for an interim assessment of the MCC-ITP. A copy of that final interim assessment is available for review through the Development Experience Clearinghouse (www.dec.gov) as Report Number 10-01-1018. The interim assessment concluded that, in general, MCC-ITP activities were being successfully implemented. After 30 months of operations, the MCC-ITP had made significant contributions to increase immunizations for measles (as part of the combined vaccine against measles, mumps and rubella, hereinafter called MMR), diphtheria, pertussis and tetanus (as part of the DPT, HB and Hib combined vaccine, hereinafter called Pentavalent), as shown by the increase in the administration of immunization doses for children under two. The assessment found that the MCC-ITP also had made important contributions to the advancement of the institutional framework to ensure sustainability in the immunizations improvement. 47 USAID/PERU MCC-IPT Assessment Report Despite the positive interim assessment findings, official data from Peru’s Demographic Health Survey (DHS) for the eight priority regions during 2008-2010 showed that expected coverage with DPT and measles did not reach the safety threshold of 95%, (please see table below), and results were uneven among the targeted regions. USAID/PERU MCC-IPT Assessment Report 48 49 USAID/PERU MCC-IPT Assessment Report The interim assessment and DHS data also revealed performance differences among MCC targeted regions, each of which received similar levels of program support. One hypothesis is that these variances in program success were caused by the discrepancy in readiness and capacity of regional authorities to assume responsibilities transferred to them under decentralization of the health sector. Such differences in regional preparedness would affect not only the immunizations program (Estrategia Nacional de Immunizaciones (ESNI)), but also other vertical national programs (such as those for tuberculosis and family planning) as they transition through decentralization. It is also important to examine the effect that decentralization and subsequent transition of authority/responsibility, as well as changes in national vaccine schedule, had on MCC-ITP implementation and outcomes. IV. OBJECTIVES The Assessment shall accomplish the following objectives: A. In at least four of the eight regions targeted by the MCC-ITP (no less than two with higher immunizations coverage rates and two with lower immunization coverage rates), hereinafter the “Selected Regions,” assess the current functioning of the regional immunizations system, including a review of any national-level support or resources (such as monitoring/supervision, training, stewardship/guidance, financing, technical support, equipment, and vaccine supplies) on which the regional program relies. The Assessment shall ascertain the level of continued implementation of immunization program plans in the Selected Regions following the completion of the MCC-ITP and identify successes and/or gaps in the systems; B. In each of the Selected Regions, perform a SWOT (strengths, weaknesses, opportunities and threats) analysis of the immunizations performance in a health network (local-level network of micro-networks, health facilities and itinerant brigades) with higher immunization coverage rates and in a health network with lower immunization rates in order to identify factors that facilitate and that challenge sustained success of medium-term interventions such as the MCC-ITP; and C. Develop, in consultation with regional authorities and other stakeholders, recommendations for critical pathways for improvement and sustainability for each assessed health network, and region. The recommendations shall be designed to serve as roadmaps and be adaptable for developing improvement plans for other regional and local governments in further strengthening their immunization programs and ensuring sustainability of MCC-ITP achievements. The Assessment shall address a number of specific issues of MCC and USAID interest. To ensure these topics are covered, USAID has developed a series of Assessment Questions (delineated below) to be answered by the Assessment. The Assessment shall cover issues and questions in the following three thematic areas: 1) Regional Immunizations Programs. In each Selected Region, the Assessment shall appraise the region’s capacity to deliver and performance in delivery of basic vaccines for children. a) What were the effects and achievements of the MCC-ITP in the region, including but not limited to: - Impact of the MCC-ITP in the region; USAID/PERU MCC-IPT Assessment Report 50 - Factors that hindered, facilitated, or otherwise influenced the effectiveness of the MCC￾ITP (such as regional readiness to assume responsibilities under decentralization); and - Reasons for difference in immunizations coverage outcomes within and among regions that received MCC-ITP support. b) What is the post MCC-ITP status of development of the regional immunizations program including but not limited to: - Political priority at the different levels; - Funding mechanisms at the different levels (national, sub-national); - Norms and procedures – implementation, monitoring and improvement; - Human resources – staffing quality and quantity; - Managerial and administrative capacity at national and sub-national level (regional, provincial, municipal or district) to plan, implement and monitor immunizations; - Information and epidemiological surveillance – data quality and use; - Degree of protection achieved, the level of risk and epidemic potential; - Involvement of national surveillance system for detecting and controlling vaccine￾preventable diseases; - Cold chain functionality – supply quantity and quality; - Biosecurity practices in the handling and disposal of used syringes and vials; and - Achievements in social communication and the degree of user satisfaction – service quality, quantity, and demand; and - Capacity to adapt to new demands generated by health sector reform, including decentralization, Universal Health Insurance, integration of immunizations into maternal and child health programs, and the inclusion of new technologies and vaccines. c) How has the regional immunizations program progressed since the close of the MCCITP, in terms of but not limited to: - Adherence to region-specific recommendations provided by MCC-ITP; - Demonstrated commitment to continued improvement of immunizations program; - Sustainability of results achieved at end of the MCC-ITP; and - Extent to which tools produced by the MCC-ITP are being used, and why are or aren’t they being used. 2) Immunizations Performance in the Health Networks. In each of the Selected Regions, perform a SWOT (strengths, weaknesses, opportunities and threats) analysis of the immunizations performance in a health network (local-level network of micro-networks, health facilities and itinerant brigades) with higher immunization coverage rates and in a health network with lower immunization rates in order to identify factors that facilitate and that challenge sustained success of medium-term interventions such as the MCC-ITP. The SWOT shall include analysis of network performance in immunizations, including but not limited to: • Coverage rate and compliance with vaccination schedules; • Managerial priority given to vaccinations; • Availability of vaccinations and necessary supplies; • Use of norms and procedures for vaccinations; • Human resources; • Information and epidemiological surveillance; 51 USAID/PERU MCC-IPT Assessment Report • Involvement of national or regional surveillance system for detecting and controlling vaccine-preventable diseases; • Cold chain functionality; • Biosecurity practices in the handling and disposal of used syringes and vials; • Achievements in social communication and the degree of user satisfaction; and • Factors (such as geographic, political, economic, social, readiness/capacity to implement system, and other aspects of decentralization) that facilitated and/or impeded continued improvements to immunizations program after the close of the MCC-ITP. 3) Improvement and Sustainability Plan. The Assessment shall identify weaknesses and gaps in the regional immunizations programs and in the functioning of those systems at the health network level. The Assessment report shall provide recommendations to improve immunizations and ensure sustainability. These recommendations will be developed in consultation with national, regional, and local input as appropriate, and will include regional and local action plans for each selected region. The action plans prepared shall be developed to serve as roadmaps and/or models for other regional and local governments to use in strengthening their respective immunizations programs. The action plans shall consider activities that are feasible within the system cost constraints and applicable in the context of the health sector reform (decentralization and Universal Health Insurance). The action plans must include: • A critical pathway for plan implementation; • Draft budget and financial plan; • Draft regulatory framework; • Timeline; • Specific responsibilities; • A communication plan; and • A mechanism to monitor and evaluate its implementation at the regional and local levels. V. METHODOLOGY The Contractor shall submit to USAID/Peru a Technical Proposal including a suggested methodology, actions, and timeline for completion of the Assessment. The methodology proposed should comply with USAID Evaluation Policy (http://www.usaid.gov/evaluation/). The Proposal shall include a case study approach and employ, at a minimum, an analysis of information and data obtained through document reviews, e.g., MCC-ITP reports, MOH reports, information from the DHS, and health network and micro network records, direct observation, and interviews with health stakeholders in the selected regions (USAID, MOH, other donors such as the United Nations Children’s Fund (UNICEF), regional and local governments, the USAID Contractor that implemented the MCC-ITP (Abt Associates, Inc.), GOP partners, health networks, micro-networks, health facilities, itinerant brigades, and communities. The proposed methodology shall allow and seek participation of stakeholders during the assessment process. To the extent practicable, data collection shall be systematic and findings and conclusions should be evidenced-based. USAID/PERU MCC-IPT Assessment Report 52 The proposed methodology shall allow the Contractor to analyze the functioning of the immunizations system from the national level down to the health post level, i.e., regional, network, micro-network, and health facility, and the sustainability of the MCC-ITP assistance in the selected regions. The Contractor shall prepare its proposal on the basis of travel to at least four regions to perform the Assessment. The regions selected for review will be confirmed by USAID/Peru. The following factors may be taken into consideration when determining field visits: security, geographical location, immunizations coverage performance, and number of MCC-ITP activities implemented in the region. Regarding analysis and comparison of immunization performance in health networks, the Proposal shall include criteria for selecting which health networks within each selected region to include in the Assessment. The health networks selected for review will be confirmed by USAID/Peru. The following factors may be taken into consideration when determining which networks to include in the Assessment: security, geographical location, immunizations coverage performance, number of MCC-ITP activities at the network. The networks selected for review should have at least one itinerant brigade. The Proposal shall include, in each of the selected regions and health networks, an in-depth analysis and interviews to identify strengths, weakness, opportunities, and threats. Proposed recommendations and action plans to remedy the problems identified should be elaborated in a participatory manner with stakeholders, e.g., regional authorities, health networks decisionmakers, users, etc. Documents to be reviewed by the Contractor shall include but are not limited to: • MOH national immunization plans • Regional immunizations plans for the selected regions • MCC-ITP project reports • MCC-ITP special studies, e.g., assessments of the immunization strategy at the regional level, assessments of the logistic system, immunization system, cost of itinerant brigades • Mid-term assessment of the MCC-ITP • DHS reports • Vaccination reports (MOH, selected regional health office, health network, micro network, health post) List of individuals/institutions to be interviewed include but are not limited to: • USAID • Peru’s Presidencia del Consejo de Ministros - Threshold Program Executive Directorate • MOH • UNICEF/Peru • Pan American Health Organization • Regional governments in selected regions • Regional Health Offices in selected regions • Heads of selected health networks 53 USAID/PERU MCC-IPT Assessment Report • Heads of health micro networks within selected networks • Itinerant brigades from selected health networks • Ministry of Economy and Finance • Other Donors VI. SCHEDULE Not more than seventeen weeks of services will be required to complete the services required under this Task Order Contract. The Contractor, on the basis of the information provided above, shall provide a detailed work plan. The work plan shall provide details of how all the various tasks and activities will be undertaken; the starting time and duration of each; location and staff resources for the duration of the services. The proposed work plan should be consistent with the technical approach and methodology, demonstrating that the offeror understands the SOW. The work plan may follow a schedule, included for illustrative purposes only, such as: Week 1: Team preparation. Week 2: Review of existing information, studies. Site selection for field visits. Week 3: Analysis of available data. Prepare data collection instruments. Week 4: Lima interviews. Week 5-6-7: Field visits. Week 8: Data processing. Week 9: Analyze findings. Week 10: Debrief to USAID and GOP counterparts. Week 11: Finalizing regional action plans. Week 12: Presentation of action plans to regions. Week 13: Submission of Draft Report. Week 15: Submission of Final Draft Report and Draft Peer Review Paper. Week 17: Submission of Final Report and Peer Review paper. VII. DELIVERABLES AND REPORTING REQUIREMENTS The Contractor shall produce the reports generally described in the sections below to the highest international technical standards, and compliance with USAID requirements. The following is the list of deliverables and reports expected from the Contractor, their contents, and general times of delivery. The Contractor shall propose a schedule for specific delivery dates in the Work plan. 1) Work plan: During mobilization, having studied the operational budget and logistics, the Contractor shall produce an updated work plan for review and approval by USAID within fourteen days from the start of the TO. The work plan shall be consistent with the TO. The workplan shall detail the execution, supervision, and progress of activities to be performed under the TO. The Contractor requires prior approval from the Contracting Officer (CO) to modify the Contract. The Contractor requires prior approval by the COTR to modify or revise the workplan. The work plan shall include: the Contractor’s understanding of the assignment, the objectives and desired end products; roles and responsibilities; description of the methodologies and data collection instruments to be used; description of the individual tasks methods, materials, personnel, procedures and proposed schedules and field visits to carry out the assessment and USAID/PERU MCC-IPT Assessment Report 54 respond to each research question. The Contractor shall not proceed with the assessment prior to USAID approval of the Assessment work plan. 2) Preliminary debriefing: Five days after completing the field visits, the Contractor will present an outline of preliminary findings and recommendations to the USAID MCC Team. The USAID MCC Team will provide comments orally, which the Contractor will use to finalize its draft report. 3) Preliminary report and debriefing: The Contractor will prepare a power point presentation in Spanish to present first to the USAID/Peru Team and then to MOH key stakeholders, in a debriefing meeting, the key findings and recommendations from the assessment, including the draft regional action plans and proposed presentation schedule for the regional plans. All recommendations should be presented as actionable recommendations and shall identify the recommended institution and/or office responsible. The Contractor shall collect comments and recommendations expressed in the debriefing meeting and shall incorporate them in the Draft Assessment Report. 4) Presentation of regional improvement and sustainability plans: Contractor shall present proposed regional action plans to improve and sustain basic child immunizations (as described in section IV. 3) to selected regions. 5) Draft Assessment Report: Contractor shall submit to USAID for review a draft assessment report in English within 13 weeks from the award date. USAID will provide comments within one week after receiving Draft Assessment Report. 6) Final Draft Report: Contractor shall submit to USAID for review a Final Draft Assessment Report in English and Spanish, one week after receiving USAID comments on the Draft Assessment Report. USAID will provide comments to these reports within one week after receiving the Final Draft Report. 7) Draft Peer Review Paper: Contractor shall prepare a paper, in English, on findings that complies with international standards for submission of a paper for peer review. Contractor will submit to USAID for review a Draft Peer Review Paper along with Final Draft Assessment Report. USAID will provide comments to the Draft Peer Review Paper within one week after receiving the document. 8) Final Report: The Final Assessment Report shall be submitted at the end of the services, o/a seventeen weeks after the award date. The Final Assessment Report shall be based on the information included in the draft Assessment Reports and responses thereto. Sections of the Final Assessment Report should include: a) Executive Summary: A two to five page summary of the purpose, background of the project, main assessment questions, methods, findings, conclusions, recommendations and lessons learned (if applicable) of the Assessment. This Executive Summary will be used by the Contractor as the abstract to accompany the Contractor’s submission of the Final Assessment Report to the USAID Development Experience Clearinghouse. 55 USAID/PERU MCC-IPT Assessment Report b) Assessment Report: In no more than 45 pages, the Assessment Report shall present a thoughtful, well-researched, evidence-based, and well-organized objective report that fulfills the purposes of the assessment. It shall discuss the major findings and the related issues and questions identified in Section III. Sections of the body of the Assessment report shall include: i) Purpose of the Assessment and assessment questions; ii) Summary of the methodology applied, data collection methods, and description of the sites selected. iii) Evidence/findings of the study concerning the assessment questions; iv) Briefly stated conclusions drawn from the findings, including lessons learned; and v) Recommendations based on the assessment’s findings and conclusions, presented with sufficient detail to enable actions to be taken by involved parties. c) Annexes: Annexes to the Final Report shall include: i) The proposed improvement and sustainability action plan for each Selected Region. Each regional action plan shall include a brief summary of the findings and conclusions of the assessment in the region, and the elements described in section IV. 3). ii) Statement of Work (SOW) for the Assessment, including modifications to the SOW, whether in technical requirements, evaluation questions, evaluation Team composition, methodology or timeline, requires prior approval of the Contracting Officer. The Contractor shall submit the request to the CO and the COTR for review. The workplan shall be consistent with the SOW and the COTR may approve revision(s) to the workplan. iii) Methodological notes. This section shall include all tools used in conducting the evaluation, e.g., questionnaires, checklists, and discussion guides. iv) List of documents reviewed. v) List of people interviewed. vi) Other relevant information. 9) Final Peer Review Paper: Contractor shall submit to USAID its paper on findings that complies with international standards for submission for peer review. The Final Peer Review Paper shall be submitted to USAID with the Final Assessment Report, o/a seventeen weeks after award date, and be based on information included in the Draft Peer Review Paper and USAID’s comments thereto. All reports and papers shall be considered draft until they are approved by USAID. The Assessment Reports shall meet a minimum of the following quality criteria: • Limitations to the assessment shall be disclosed in the report with particular attention to limitations associated with the evaluation methodology, i.e., selection bias, recall bias, unobservable differences between comparator groups, etc. • Statement of differences: when applicable, evaluation reports shall include statements regarding any significant unresolved differences of opinion either of USAID, counterparts, implementers, and/or members of the evaluation Team. • Assessment reports shall address all evaluation questions included in the SOW. • Findings will assess outcomes and impact on males and females, where applicable. USAID/PERU MCC-IPT Assessment Report 56 • Assessment findings shall be presented as analyzed facts, evidence, and data, and not based on anecdotes, hearsay or the compilation of people’s opinions. Findings shall be specific, concise, and supported by strong quantitative or qualitative evidence. • Sources of information shall to be properly identified and listed in an annex. • Recommendations shall be supported by a specific set of findings. • Recommendations shall be action-oriented, practical and specific, with defined responsibility for the action. The Final Assessment Report shall be produced in English in an electronic format and in five (5) print copies bound and one (1) original unbound and produced in Spanish in an electronic format and two (2) print copies bound and one (1) original unbound. The Final Peer Review Paper shall be produced in English in five (5) print copies bound and (1) original unbound. All reports and papers shall be produced in electronic form, eight (8) copies on CD/DVD, written in an application compatible with MS Windows & MS Office. 57 USAID/PERU MCC-IPT Assessment Report ANNEX C - METHODOLOGICAL NOTES, INCLUDING ALL TOOLS USED IN CONDUCTING THE EVALUATION - QUESTIONNAIRES, CHECKLISTS, AND DISCUSSION GUIDES ANNEX C.1 – DATA COLLECTION INSTRUMENTS Regional Data Collection Checklist Region:_________________________________ ACTIVITY DONE 1. Conduct initial meeting with President and Gerente de Desarrollo. Write down minutes. 2. Conduct initial meeting with Regional health Team: DIRESA, RIS, DIREMID, Estadística, ODSIS, IB, Epidemiologia, Promotion, Dept. Education, and MEF. Get agreement on Date and time for SWOT workshop and select networks to be assessed. Invite to accompany. Write down minutes 3. Show draft plan of activities and complete list, timing and sequence 4. Ask for 2011 Plan Regional de Inmunizaciones 5. Get copy of August 2011 HIS report 6. Get copy of August 2011 Epidemiological report 7. Conduct individual meetings/interviews and write down minutes 8. Visit Network 1 and Post 1 9. Visit Network 2 and Post 2 10. Interview at least one IB 11. Book venue and make coffee break arrangement for SWOT workshop 12. Conduct SWOT work shop 13. Get blank copy of local “certificado de vacunaciones” 14. Get copy of latest Supervision report from DIRESA to selected Networks 15. Keep list of people met with contact information 16. Agree on who will come to Lima for the November 8th workshop and make arrangements for transport, per diem and hotel 17. Keep list of documents collected USAID/PERU MCC-IPT Assessment Report 58 SWOT Attendance sign up REGION:___________________________________ FECHA:_________________ NOMBRE CARGO TELÉFONO EMAIL FIRMA 59 USAID/PERU MCC-IPT Assessment Report SOCIAL IMPACT MCC-ITP ASSESSMENT INTERVIEW AGENDA AND MINUTES DATE: Interviewee’s Name : Ministry of Health – Central Office Email : Phone : Interviewer (s): QUESTIONS / TOPICS NOTES DOCUMENT REF. What responsibilities does your office have for national level support of the immunization program, including the following: • Setting, national policy on vaccinations, reviewing or monitoring its implementation? • Developing long term planning for vaccination program? • Aggregating data (coverage and service delivery data), analyzing data, providing recommendations based on data? • Coordinating with international donors on vaccination programs? • Supporting development of regional action plans (HR, budgets, etc). ? Please discuss in more detail how you carry out the following responsibilities: • Monitoring/supervision • training • stewardship/guidance • financing • technical support, • equipment, and vaccine supplies • developing information systems • Analyze and improve vaccine logistics • Others What is the current functioning of the regional immunization systems? What data do you use to assess the systems functioning? How has the MOH sustained the results achieved under the MCC/ITP? What have been the lessons learned? Please describe changes/improvements planned for the immunization program in the selected regions. USAID/PERU MCC-IPT Assessment Report 60 INTERVIEW AGENDA AND MINUTES (CON’T) DATE: Interviewee’s Name : Ministry of Health – Central Office Email : Phone : Interviewer (s): QUESTIONS / TOPICS NOTES DOCUMENT REF. SWOT QUESTIONS What are the particular strengthens of your immunization program? What are the particular weaknesses of your immunization program? What opportunities do you see to improve your program? What are the major threats to improving and sustaining your immunization program? 61 USAID/PERU MCC-IPT Assessment Report SOCIAL IMPACT MCC-ITP ASSESSMENT INTERVIEW AGENDA AND MINUTES DATE: SEPTEMBER 13, 2011 Interviewee’s Name : Representives of President Council, Intergovernmental Commission of Ministers,and Office of Decentralization Email : Phone : Interviewer (s): QUESTIONS / TOPICS NOTES DOCUMENT REF. What repsonsibilties does your office have for national level support of the immunization program? Setting, national policy on vaccinations, reviewing or monitoring its implementation Developing long term planning for vaccination program Aggregating data (coverage and service delivery data), analyzing data ,providing recommendations based on data Coordinating with international donors on vaccination programs Supporting development of regional action plans (HR, budgets, etc). USAID/PERU MCC-IPT Assessment Report 62 INTERVIEW GUIDE FOR REGIONAL HEALTH TEAM 1) ¿Cuáles considera que son los logros más importantes del Programa MCC-ITP de Inmunizaciones? 2) ¿Cuáles de estos se mantienen hasta la fecha? 3) ¿Qué factores favorecieron estos logros en esta Región? 4) ¿Cuál es la cobertura de Inmunizaciones para Pentavalente? 5) ¿Tienen un Comité Técnico de la RIS? 6) Las diversas Redes tienen diferentes coberturas. ¿A qué atribuyen estas variaciones? 7) ¿Cómo apoya su gobierno regional (presidente, planeamiento, Ger. Des. Soc.) al programa de inmunizaciones y cómo monitorea las coberturas? ¿Con qué datos? 8) ¿Cómo apoyan los líderes comunitarios y las sociedad civil al programa de inmunizaciones y cómo monitorea las coberturas? a) ¿Desarrollan reuniones de concertación intrasectorial? b) ¿Se realizan audiencias públicas? 9) ¿Tiene la Región un Plan Regional de Inmunizaciones para el 2011? a) Si no lo tiene, ¿por qué? b) Si lo tiene, ¿tiene resolución de aprobación? c) Favor proporcionar una copia del plan, y de la Resolución. 10) ¿Cuándo fue la última evaluación del Plan Regional de Inmunizaciones? ¿Cuál fue el resultado de la evaluación? (pedir copia del Informe de la evaluación) 11) ¿Se elaboró un presupuesto PPR para vacunaciones? ¿Cuál es su nivel de ejecución? Qué dificultades se están presentando? 12) ¿Cómo vienen funcionando los reembolsos del SIS para el caso de inmunizaciones? 13) ¿Hay municipalidades que han participado del Programa de Incentivos Municipales, en lo referido a inmunizaciones? 14) ¿Cuándo fue la última supervisión que tuvieron por parte de la ESNI? (pedir una copia del informe) 15) ¿Existen planes anuales de inmunización a nivel de redes y municipios para 2011? Tienen aprobación? ¿Se están implementando? 16) ¿Se aplican las normas nacionales de la Estrategia de Salud Nacional Inmunizaciones (ESNI) en cuanto a calendarios y procedimientos? ¿Se monitorea su aplicación? Tienen alguna dificultad para la aplicación de las normas, en especial para Pentavalente3 y SPR? 17) ¿Han sufrido desabastecimiento de vacunas durante el 2011, especialmente DPT / SPR, in 2011? a. ¿Y han tenido algún desabastecimiento de jeringas u otros insumos? b. ¿Realizan un monitoreo de vacunas y jeringas entregadas a los establecimientos de salud? 18) ¿Están usando actualmente el SIII? 19) ¿Cuál es el último reporte de coberturas de inmunizaciones? Podrían proporcionarnos una copia? Cuál es su cobertura actual de Inmunizaciones como Región? Para Pentavalente? Y para SPR? Tienen alguna red que reporte menos del 95% o más del 100% de cobertura? Qué medidas han tomado al respecto? 63 USAID/PERU MCC-IPT Assessment Report 20) ¿Cuántas computadoras recibieron por parte del Programa MCC-ITP? ¿Se encuentran registradas en el SIGA? ¿En qué se están usando actualmente? 21) ¿Cuánto personal de la Región participó en la Capacitación Virtual? ¿Quiénes fueron? ¿Recibieron sus certificados? 22) ¿Tienen el Maletin de Materiales de Comunicación desarrollado por el Programa MCC-ITP? Cómo lo emplean? Han elaborado materiales nuevos basados en la metodología de MCC-ITP? 23) ¿Tienen un Plan Regional de Comunicación Social? Es un Plan separado o es parte del Plan Regional de Inmunizaciones? 24) ¿Se informa a los padres de familia o apoderados acerca de las reacciones post-vacunales? 25) ¿Existe algún movimiento de resistencia activa a la estrategia de inmunizaciones en su región? Explique. 26) ¿Cuántas IB tienen en su Región? Han aumentado o disminuido en comparación con el año pasado? ¿Están equipadas de acuerdo a la norma? ¿Han tenido problemas con el equipamiento? ¿Tienen problemas para reclutar o mantener el personal? Qué apoyo reciben las redes para mantener sus brigadas? Vienen usando las PDA? 27) ¿El Programa MCC-ITP, en su informe final, efectuó algunas recomendaciones para su región (Ver el informe final MCC-ITP). Podría comentarnos si se llegaron a aplicar? 28) ¿Podrían comentarnos algunas de las lecciones aprendidas en el proceso de implementar la descentralización en la Región? ENCUESTA SOBRE LA GESTIÓN DEL PROGRAMA DE INMUNIZACIONES EN ESTABLECIMIENTOS REGIÓN TIPO Centro de Salud Puesto de Salud NOMBRE DE LARED AREA Urbana Rural NOMBRE DEL ESTABLECIMIENTO USAID/PERU MCC-IPT Assessment Report 64 DATOS DE LA PERSONA RESPONSABLE DE INMUNIZACIONES EN EL ESTABLECIMIENTO Nombre Capacitado por el Programa MCC-ITP Si =1 No=2 Tiempo en este cargo desde (fecha) Teléfono y/o Correo electrónico para contactarlo si hay necesidad de aclaración PREGUNTAS SOBRE EL ESTABLECIMIENTO RESPUESTA ¿Tiene energía eléctrica? Si =1,No =2 Número de horas con energía eléctrica en un día normal ______ ¿Existe un generador eléctrico de energía? Si =1,No =2, No se sabe =9 ¿Ha usado el generador eléctrico el último trimestre? Si =1; No=2; No sabe=3 Si tiene refrigeradora a gas , ¿puede comprar gas en la localidad?: Si =1; No=2; No sabe=3; No tiene=4 ¿Accede a teléfono si le faltan vacunas o necesita insumos? En el establecimiento=1;En un centro comunitario=2; Celular personal=3; No accede =4; Otro=5 Especificar ¿Accede a comunicación por radio? En el establecimiento=1; En el Municipio=2; No accede =3 ; Otro= 4 Especificar ¿Tiene computadora en el establecimiento para registrar los datos de inmunizaciones? Si =1 No=2 ¿Usa la computadora para registrar los datos de inmunizaciones? Sí =1 No=2 Por qué? ¿Tiene impresora conectada a la computadora con tinta y papel? Sí =1 No=2 ¿Accede al Internet? Sí, en el establecimiento=1; Sí, en cabina=2; Sí, en otro lugar=3 (Especificar) ; No=4 COBERTURA DE VACUNACIONES Y CUMPLIMIENTO DEL ESQUEMA DE VACUNACIONES ¿Puede mostrar el plan regional de la estrategia de inmunizaciones? Sí =1 Obtenga una copia No=2 ¿Puede mostrar el plan operativo del establecimiento? Sí =1 Obtenga una copia No=2 65 USAID/PERU MCC-IPT Assessment Report ¿El plan tiene una sección para inmunizaciones? Si=1 No=2 ¿Cuál ha sido la población oficial para vacunar en el 2011 por grupo de edad? (La población oficial es aquella que fue proporcionada por la oficina de estadística de la DIRESA para el año 2011, se refiere al número de niños/personas por cohorte de edad) AÑO < DE 1 AÑO POBLACIÓN DE 1 AÑO POBLACIÓN DE 2 A 4 AÑOS OTRO (ESPECIFICAR) 2011 ¿Usted considera que la población oficial 2011 corresponde a su población observada? Sí =1 No=2 ¿Se ha realizado un censo local? Sí =1 No=2 Si hay un censo: escriba la población de <1 año_____________fecha de realizado_____________ ¿Hay un mapa de la zona de cobertura con población? Sí =1 No=2 Si lo hay, tome una foto Al momento de la encuesta: ¿Conoce cuántos niños faltan por vacunar con Pentavalente 3 para cumplir la meta del 2011? Si=1 No=2 Comentarios_______________________________________________________ ¿Existen mecanismos para recuperar a los niños que no se vacunaron oportunamente? Si=1 No=2 Comentarios_______________________________________________________ ¿Estima mensualmente el indicador de deserción de DPT3 o Pentavalente3? Si=1 en el mes Agosto de 2011 fue __________________________________________ No=2 Comentarios_______________________________________________________ USAID/PERU MCC-IPT Assessment Report 66 Datos de vacunación de pentavalente al mes de Agosto 2011 VACUNA < DE 1 AÑO POBLACIÓN DE 1 AÑO POBLACIÓN DE 2 A 4 AÑOS PENTAVELENTE SPR PRIORIDAD EN LA GESTIÓN DE LA RIS Sólo pensando en vacunación regular en 2011 (no incluye procedimientos de campaña ) PREGUNTA RESPUESTA 1. ¿En este establecimiento ¿Se ofrece servicios de vacunación todos los días? Sí =1; No=2; No sabe=3 Anote el horario de atención para vacunación regular: Desde ______ Hasta _____# Horas ____ Si no se ofrece servicios de vacunación todos los días, ¿Qué es lo que se lo impide? 2. ¿Existe personal para que el servicio de inmunizaciones no se interrumpa cuando sale la responsable de inmunizaciones a visitas domiciliarias o extramurales? Sí =1 No=2 3. ¿Se han gestionado recursos adicionales para gastos operativos de la vacunación regular? Sí =1; No=2; No sabe=3 4. En su opinión, ¿usted necesita mejorar la cobertura de vacunación regular en su establecimiento? Sí =1; No=2; No sabe=3 5. ¿Ha implementado alguna estrategia para incrementar las coberturas de vacunación? Sí =1; No=2; No sabe=3 6. ¿Qué estrategia ha usado para incrementar las coberturas? ¿La estrategia usada, ha sido efectiva? Explique: 67 USAID/PERU MCC-IPT Assessment Report RECURSOS HUMANOS Número de trabajadores en el servicio de inmunizaciones (personal en el establecimiento) NOMBRE CARGO 1=Responsable RIS 2=Personal asistencial de apoyo 3= Brigada AIDPED 4-Otro: (especificar) PROFESIÓN 1=Enfermera 2=Enfermera SERUMS 3=Técnico en Enfermería 4= dentista 5-Otro ¿Capacitado en los últimos dos años por el Programa MCC￾ITP? Si la respuesta es sí, ¿Qué curso? ¿Virtual? PREGUNTA RESPUESTA 1. ¿Se han realizado actividades de capacitación para el personal de esta red en el área de inmunizaciones in 2011? Si=1 No=2 Si la respuesta es sí, ¿en qué temas? 2. ¿Quién se encarga de las actividades de capacitación en su establecimiento? La misma responsable de inmunizaciones La encargada de capacitación del establecimiento. Encargada de capacitación de la DIRESA Otro 3. ¿Opera una Brigada IB en el área de su establecimiento? Sí =1; No=2 4. ¿A su establecimiento le corresponde recibir reportes de producción de vacunación a través de equipos IB? Sí =1; No=2. Si la respuesta es sí, ¿Después de cuántos días de realizada la atención le reporta? 5. ¿Cómo se registra la información de IB? Se integra la producción de IB al establecimiento La reporta por separado ¿Cuenta con datos de cobertura de las comunidades servidas por Brigadas? Sí =1; No=2 6. ¿Hace Ud. la supervisión de las Brigadas? Sí =1; No=2 7. ¿La supervisión de las brigadas es con frecuencia regular? Sí =1; No=2. Fecha de la última supervisión regular________ 8. ¿Cómo hace la supervisión? ¿Se usa algún formulario? Obtenga copia del formato Si necesita, escriba comentarios sobre el funcionamiento de las brigadas: USAID/PERU MCC-IPT Assessment Report 68 CADENA DE FRÍO CADENA DE FRÍO SÍ NO NA OBSERVACIONES ¿Cuenta con equipos de cadena de frio (refrigeradoras, congeladoras) que usa para almacenar vacunas? ¿Ha recibido equipos nuevos en el 2008 o 2009? ¿El inventario de cadena de frío está actualizado incluyendo los equipos nuevos? ¿Puede mostrarme su plan de mantenimiento preventivo o proyectos presentados? ¿Considera que las condiciones de su almacén son adecuadas y corresponden a la norma técnica? El local es propio Tiene grupo electrógeno Corresponden las dimensiones del área Tiene alarma sonora por falla eléctrica ¿Tiene vacuna dañada por altas o bajas temperaturas o vencida? ¿Cómo se descarta la vacuna dañada o vencida? ALMACENAMIENTO SÍ NO NA OBSERVACIONES ¿Puede mostrar un plan de contingencia ante cualquier evento que signifique una ruptura de la cadena de frío? Anote la fecha ¿Ha tenido suficiente capacidad de almacenamiento en el año? -Para la vacunación regular -Durante la campaña ¿Solo se almacenan vacunas en el refrigerador? Verifique que no se guarden alimentos, bebidas, medicamentos, muestras o hemoderivados. . ¿Tiene registro de mantenimiento de equipos? Anote la última fecha ¿Descongela y limpia el refrigerador? Anote cada cuánto tiempo. Verifique los registros de las fechas de descongelamiento en la hoja de control de temperatura. ROLES Y RESPONSABILIDADES SÍ NO NA OBSERVACIONES ¿Existe una persona de reemplazo del responsable para manejar y almacenar las vacunas? ¿Estas personas han recibido capacitación apropiada sobre cadena de frío? Han recibido alguna capacitación del programa MCC-ITP ¿Puede mostrar algún documento con instrucciones y políticas para la administración de la cadena de frío incluyendo: pedidos, recepción y almacenaje de vacunas y manejo de vacunas cuando el equipo se daña? 69 USAID/PERU MCC-IPT Assessment Report ¿Las fallas de la cadena de frío son documentadas y reportadas al responsable de inmunizaciones? Descripción del equipo donde se almacena la vacuna Pentavalente DESCRIBA ESCRIBA SEGÚN CORRESPONDA OBSERVACIONES Tipo de equipo Refrigerador = R; Refrigerador y Congelador = RC; Congelador = C Año de compra Menos de 2 años 2 a 5 años =1 6 a 10 años = 2 10 o más años = 3 Energía Eléctrica = E; Eléctrica y Gas = EG ; Eléctrica y kerosén = EK, Gas = G; Kerosén = K; Solar = F ¿Tiene estabilizador? Sí =1, No=2 Estado del equipo/reparación Funciona y usa para vacunas =OK No funciona por falta de combustible =1 No funciona porque falta reparación = 2 No funciona /obsoleto =3 Otro = 4 Especifique Fecha del último mantenimiento ____/_____/____ Día Mes ño No sabe ______ ¿Tiene termómetro? Sí =1, anote la marca y la temperatura No = 2 ¿Usa adecuadamente la hoja de control de temperatura? Sí =1, No =2 (Verificar si está marcada la temperatura al tiempo adecuado, por ejemplo, es media mañana no debe estar marcada la de la tarde) ¿El enchufe está marcado “no desconectar”? USAID/PERU MCC-IPT Assessment Report 70 DISPONIBILIDAD DE VACUNAS E INSUMOS NECESARIOS LISTA DE CHEQUEO RIS SÍ NO N/A OBSERVACIONES ¿Tuvo desabastecimiento de vacunas desde enero de 2011?: ¿Desabastecimiento de otros Insumos?: - Algodón - Jabón liquid - Jeringas - Cajas seguras ¿Tiene disponibilidad adecuada de vacunas con relación a la programación de las vacunas?: ESCRIBA EL STOCK DE ACUERDO AL KARDEX BCG STOCK Pentavalente STOCK Polio oral STOCK Fiebre Amarilla STOCK Sarampión, rubéola y paperas STOCK Hepatitis B STOCK Rotavirus STOCK Influenza STOCK Neumococo STOCK Otra, cuál(es)? STOCK ¿Existen mecanismos de contingencia en caso exista falta de vacunas e insumos? Precise cuáles 71 USAID/PERU MCC-IPT Assessment Report OBSERVE HASTA 5 NIÑOS SIENDO VACUNADOS CON LA VACUNA PENTAVELENTE Marque con una “X” si observa las siguientes prácticas OBSERVE LAS SIGUIENTES ACTIVIDADES 1 2 3 4 5 COMENTARIOS Niño/a tiene DNI Dosis: 1, 2, o 3 Edad en meses ¿El paciente tiene cita previa? ¿Se lava las manos antes de vacunar al niño/a? Observar la vacunación Identificación de la zona (muslo cara lateral) Asepsia Fijación Introducción de la aguja Aspirado Aplicación Consejería ¿Retapa la jeringa? ¿Utiliza jeringas descartables? ¿Registra en el carné del niño las vacunas aplicadas, fecha y lote de aplicación? ¿Se explica la información del carné al responsable del niño? ¿Se indica al responsable del niño la fecha en que le corresponde la próxima vacuna? ¿Se descarta la jeringa en un contenedor adecuado? USAID/PERU MCC-IPT Assessment Report 72 COMUNICACIÓN Y SATISFACCIÓN CON EL SERVICIO DE INMUNIZACIONES Entreviste a 5 a 10 madres luego de la vacunación de su hijo/a Sí =1; No=2 PREGUNTA 1 2 3 4 5 Sabe que vacuna(s) recibió el niño/a Sabe la dosis que recibió (1,2 o 3) Sabe cuándo debe volver por la siguiente dosis Sabe que otras vacunas le faltan Sabe que hacer si hay efectos secundarios Sabe que se registró en el carne de vacunación El trato a Ud. y al niño/a puede ser mejorado28 Tiene preguntas sobre la(s) vacuna(s) dadas su niño/a hoy Hay algo que le preocupe sobre las vacunas recibidas PREGUNTA 6 7 8 9 10 Sabe que vacuna(s) recibió el niño/a Sabe la dosis que recibió (1,2 o 3) Sabe cuándo debe volver por la siguiente dosis Sabe que otras vacunas le faltan Sabe que hacer si hay efectos secundarios Sabe que se registró en el carne de vacunación El trato a Ud. y al niño/a puede ser mejorado29 Tiene preguntas sobre la(s) vacuna(s) dadas su niño/a hoy Hay algo que le preocupe sobre las vacunas recibidas Sí =1; No=2 28 Si responde que si, pregunte como. 29 Si responde que si, pregunte como. 73 USAID/PERU MCC-IPT Assessment Report BIOSEGURIDAD 1. ¿Utiliza jeringas con aguja segura (autodestruible o autoretractable)? Sí =1 Describa________________________________ No=2; No sabe=3 2. Descarta las jeringas en (marque con un X la práctica más común, usual o regular- Solo una respuesta): a) Caja de seguridad b) Botellas plásticas/galones c) Botellas plásticas/galones con lejía d) Tacho de basura e) Cajas comunes de carton f) Bolsas plásticas 3. En los recipientes de descarte SE OBSERVA: Jeringas con agujas sin retapar Sí/No Sin información _____ Jeringas con agujas retapadas Sí/No Sin información _____ Jeringas sin agujas o sólo agujas Sí/No Sin información _____ 4. El establecimiento tiene un sistema para destrucción o disposición de ampollas usadas Sí =1 Describa________________________________ No=2; No sabe=3 5. ¿Dónde se descartan las cajas de seguridad con jeringas y agujas usadas? ________________________________________________________ 6. Describa otros recipientes de basura en el área de vacunación si los hay ________________________________________________________ NOTAS DEL ENCUESTADOR SOBRE EL ESTABLECIMIENTO DE SALUD 1. Tome fotografías del establecimiento de salud: a) Sala de espera si no b) Cámara de frio o refrigerador de vacunas si no c) Sala de vacunación si no d) Lavabo de manos si no e) Mesa de preparaciones si no USAID/PERU MCC-IPT Assessment Report 74 f) Cuaderno de registro de vacunaciones si no g) Computador en uso si no h) PDA en uso si no i) Mapa de cobertura si no j) Otras observaciones:______________________________________________ 2. Indique sus observaciones del centro o puesto: a) Limpieza si no ____________________________________________________ b) Lleno de pacientes esperando si no ____________________________ c) Impresión de poco personal si no___________________________________________ d) Vacunación es una prioridad si no _________________________________________ e) Conservan el patrimonio si no _____________________________ Otros comentarios y observaciones 75 USAID/PERU MCC-IPT Assessment Report ENCUESTA RÁPIDA DEL FUNCIONAMIENTO DE LA ESTRATEGIA DE INMUNIZACIONES POST PROGRAMA MCC-ITP Estimado Sr. Director de la Dirección Regional de Salud: El Programa MCC-ITP de Inmunizaciones tuvo lugar de noviembre de 2008 a febrero de 2010 con el objetivo de fortalecer la estrategia regional de inmunizaciones en 17 regiones. La presente encuesta financiada por la Agencia Internacional para el Desarrollo del Gobierno de Estados Unidos (USAID) tiene la finalidad de conocer la situación actual en materia de inmunizaciones en su región y los factores que contribuyeron a su mejoramiento y la sostenibilidad de las mejoras. Agradecemos que complete el siguiente formulario de acuerdo a la situación actual de la estrategia de inmunizaciones en su región. COMPONENTE IB ACTIVIDAD CANTIDAD 1. ¿Qué cantidad de brigadas había en Diciembre 2010? 2. ¿Qué cantidad de brigadas hay ahora? 3. ¿Cuántas brigadas aún tienen los equipos donados por el programa MCC-ITP? 4. ¿Cuántas brigadas les faltan equipos? 5. ¿Todas las brigadas usan sus PDAs? si no Por favor háganos saber sus observaciones sobre el funcionamiento del componente IB en la actualidad: ___________________________________________________________________ ___________________________________________________________________ PLANEAMIENTO Y MONITOREO ACTIVIDAD SI NO 1 ¿Se formuló un Plan Regional de Inmunizaciones para 2011? 2 ¿El presupuesto aprobado es suficiente? ¿Cuánto es?_________________ 3 ¿Se realiza el monitoreo de coberturas con la matriz elaborada por el programa MCC-ITP? 4 ¿Cuál es el porcentaje de cobertura de vacuna pentavalente3 al 31 de agosto 2011 5 ¿Cuál es el porcentaje de cobertura de vacuna SPR al 31 de agosto 2011? Por favor, háganos saber sus otras observaciones sobre el funcionamiento del componente de planificación y monitoreo: ___________________________________________________________________ ___________________________________________________________________ USAID/PERU MCC-IPT Assessment Report 76 SISTEMA DE INFORMACION ACTIVIDAD SI NO 1 ¿Usa Ud. el Sistema de Información Integrado de Inmunizaciones (SIII)? 2 ¿Todos los establecimientos de salud aplican el SIII? 3 ¿Se registra la vacunación de NO asegurados en formato único SIS en todos los EESS? 4 ¿Se elaboró el reporte de inmunizaciones del SIS del mes de Agosto de 2011? 5 ¿Puede Ud. elaborar reportes web con el sistema SIS? Por favor, háganos saber sus otras observaciones sobre el funcionamiento del Sistema de Información: ___________________________________________________________________ ___________________________________________________________________ SISTEMA DE SUMINISTROS ACTIVIDAD SI NO 1. ¿Los equipos de cadena de frio están registrados en el SIGA patrimonial? 2. ¿Ha tenido dificultad para almacenar todas las vacunas recibidas en 2011? 3. ¿Tiene instaladas cámaras de frío? 4. ¿La región tiene fondos para distribuir las vacunas en forma oportuna a todos los EESS? 5. ¿Hubo desabastecimiento de vacunas en la región en el trimestre de Julio-Agosto-Setiembre de 2011? 6. ¿Todas las computadoras donadas por el Programa MCC-ITP están aún funcionando en su región? Por favor, háganos saber sus otras observaciones sobre el funcionamiento de la Cadena de Frío en su región: ___________________________________________________________________ ___________________________________________________________________ 77 USAID/PERU MCC-IPT Assessment Report ANNEX C.2 C-DHS VACCINATION COVERAGE ANALYSIS The C-DHS estimates vaccination coverage in urban and rural locales and at the national level; however, in regions, the sample is too small to provide robust estimates. Since the C-DHS is a sample-based survey, results are only estimates of the true coverage value. It is conducted on a subsample of the standard, five-year DHS survey sample so it is important to consider the sample size and confidence intervals (CI= + two standard deviations, SD) in describing coverage estimates. In 2010, the national sample was 1747 persons under one year of age and the rural and urban samples were 611 and 1136 persons, respectively. Sample sizes for Selected Regions in this assessment were: Amazonas 83, Apurimac 79, Cusco 75, Puno 77. These sample sizes determined 95% CI that can be seen in Figs. 1 and 2. Figure 1 40 50 60 70 80 90 100 Coverage with DPT3 + 2SD Continuous DHS, Perú 2010 USAID/PERU MCC-IPT Assessment Report 78 Figure 2 The pink bands describe the CI around the estimated national coverage rates of 73.7+2.9% for DPT3 and 83.4+2.4% for MMR. In neither case did national rates reach the 95% goal in 2010. The points on the graphs are the estimated coverage rates for regions and urban/rural populations, while the vertical lines represent the 95% confidence limits (signifying that the true value lies within that range with 95% probablility). While five regions have estimated rates above and two have rates below the band, the CI of all but Apurimac lie within the band, signifying that they are not statistically different from the estimated national rate. Thus, while the national estimate is valid for the whole Peruvian population within the CI shown, inter-region comparisons of estimated coverage rates are indicative but not statistically significant. Furthermore, the C-DHS collects immunization information for children 18 to 24 months of age, i.e. at least one year before the survey. Thus, coverage reported in the 2009 C-DHS corresponds to vaccinations received in 2007-2008 (before MCC-ITP) and are not comparable to 2009 HIS administrative coverages published by the MOH. The source of information in the C-DHS is children´s health cards (“carnets”) when available and legible, or mothers´ recall. The increasing complexity of the immunization schedule confuses mothers recall of which vaccines were administered at birth, two, four, six and twelve months (the Team found that not even all nurses administering the vaccines remember the schedule). In addition, each health region has printed its own version of the carnet with slightly different contents and previous versions still circulate with the most recent ones, making data collection confusing at best. A further analysis of the C-DHS can be seen in a time series of estimated coverage. Fig 3 shows estimates of national coverage from the C-DHS of 2008 and 2009 (before MCC-ITP) and 2010 (corresponding to 2009, first full year of MCC-ITP). The 2011 C-DHS is not yet available to evaluate 2010 coverage in the final year of the MCC-ITP. Thus, at the end of MCC-ITP in February 50 60 70 80 90 100 Coverage with MMR + 2SD Continuous DHS, Perú 2010 79 USAID/PERU MCC-IPT Assessment Report 2011, the C-DHS could not provide conclusive, vaccination coverage information nor credible comparisons between and within regions. Figure 3 The MOH´s General Office of Statistics and Informatics (OGEI) operates the national Health Information System (HIS) which collects data on immunization services provided by the MOH and social security insitute (ESSALUD). MOH vaccination services cover approximately 67% of the population, ESSALUD 10% and the rest is covered by pharmacies, private clinics and practitioners30. The coverage indicator calculated by the OGEI divides the number of reported DPT3 and MMR vaccinations by the projected under one-year age population for the same period according to the 2007 census. Thus, even if the HIS reports all vaccinations provided by the public sector, it should logically not give much more than 77% coverage. Fig 5 shows HIS national coverage at 92% for DPT3 in 2009 (comparable to C-DHS 2010) which seems unreasonably high, especially in light of the C-DHS which estimates 73.7+2.9% for DPT3. 30 C-DHS 2010, analysis of survey data. 0 20 40 60 80 100 2008 2009 2010 2008 2009 2010 2008 2009 2010 DPT3 MMR ALL VACCINES Vaccination Coverage Rates DHS Perú, 2008, 2009, 2010 National Rural Urban USAID/PERU MCC-IPT Assessment Report 80 Figure 4 As can be seen in Figure 4, the regional results are also at odds with the C-DHS. For example, Huancavelica appears to have 63% coverage with DPT3 in HIS statistics for 2009 while the C-DHS estimates 92.1+6.7% (Fig 1). This is explained by several factors which are recognized by OGEI: • Late and incomplete reporting to the HIS which reduces the number of vaccinations administered (numerator) • Migrations which can cause over and under delivery of services (numerator) • Under and over estimates of census-projected populations (denominator) Given the uncertainties surrounding the previous two cases, in mid-2010 MCC-ITP undertook a household sample survey in the eight targeted regions (conducted by the Quanto Company) to provide a third, and presumably more credible, estimate of coverage rates. The eight-region sample was 2329 households (vs. 1747 in the C-DHS) which gave a slightly smaller CI for regional, urban/rural coverage estimates. The Quanto survey investigated children in three age cohorts: 6-11, 12-23 and 24-36 months of age and estimated regional DPT3 coverage in 2009 at 84% (no CI given), intermediate between the C-DHS and the HIS. Fig 5 compares the results for coverage statistics and estimates from the three sources for 2010. 0 50 100 150 Coverage with DPT3 2006- 2010 OGEI-MINSA, Perú 2006 2007 2008 2009 2010 81 USAID/PERU MCC-IPT Assessment Report Figure 5 It should be pointed out that surveys and censuses are cross-sectional, conducted at certain moments and reflect conditions prevalent at that moment in the population. However, populations migrate so that a survey or census conducted in March is not representative of conditions in August or December. On the other hand, the HIS captures all vaccinations provided during a year – it is longitudinal – but only for that part of the population served by the public sector. In summary, the cross-sectional, sample-based DHS gives coverage estimates for the whole country, down to the regional level within specified confidence limits and for a given moment in time. The HIS is accurate for the population covered by the public sector, primarily the poor and indigent, with recognized reporting flaws. The cross-sectional Quanto survey was a one-time effort, conducted in eight regions at a different time in an attempt to resolve the differences between the previous two. The temporal and population relations are shown in the following schematic: This Assessment concludes that the best available estimate of national coverage rates is the C-DHS but, because of the time lag between the cohort being investigated and time of vaccination, it is not useful for program monitoring or evaluation. In addition, the small regional samples do not allow for meaningful comparisons of the “targeted regions, each of which received similar levels of program support” and presumably should have had similar coverage rates. 40 60 80 100 DPT3 Coverage by Source Perú 2010 OGEI Cuanto DHS Total popu￾lation HIS SIS D H S Q u a n Population served by public sector (77%) Timeline 1 year USAID/PERU MCC-IPT Assessment Report 82 Timeline of MCC-ITP 2006 GOP requests support from MCC for a Threshold program January 2007 GOP-MCC Negotiations start and MOH committee develops proposal: Ma. Ana Mendoza, ESNI; Lidia Mendoza, OGEI; Hernan Roig, Coop Int. ; Jose Marconi, DGSP February 2007 GOP submits proposal November 2007 Informal news that proposal was approved March 2008 Proposal is approved and MOH enters agreement between GOP and USG March 2008 MOH appoints DGSP Director and ESNI Coordinator in charge of developing and coordinating the implementation of the ITP in coordination with USAID June 2008 GOP and USG sign agreement: S/35,585,000 to: - Increase coverage to 95% in 8 regions, 2. - Reduce to 1 week time for itinerant brigades to report, and - Reduce information errors by 50% USAID No. 527-0422: - Strengthen AIDPED brigades (program component 1) - Strengthen management capacity of cold chain and immunization through training (component 2) - Strengthen information system (component 3) October 2008 Abt Associates is awarded the contract and appoints Dr. Ada Pastor to manage the ITP November 12, 2008 Letter from Yehude Simon Munaro, PCM President correcting mistaken estimate for DPT3 coverage rate for 2007 from 82% to 94.7%, (and even as high as 96.9% due to late reporting) resulting from the substitution of Pentavalent vaccine for DPT. December 2008 PAHO confirms Peru’s DPT/Pentavalent 3 coverage as 94.7% March 27, 2009 Letter from Dr. Ugarte, MOH, to Paul Weisenfeld requesting changes to the projects SOW: - Communication: instead of radios, PDAs are requested for the IB and printers are requested to be purchased with the computers for the 8 target regions - Development of a module in the SIGA system for cold chain equipment that would update the current Excel-based inventory system, FOX-Pro license is no longer needed - Replace the creation of a new information system for the strengthening of the SIS to include recording and report of immunizations July 30, 2009 Email from Paul Weisenfeld, USAID mission director confirms Peru’s ineligibility for a MCC compact due to World Bank’s classification of Peru as a middle income country December 15, 2010 Vice-Minister Zarela Esther Solis Vazquez requests ITP to implement activities not yet carried out and to support : 1. Information system assistance, 2. Cold Chain Management Training, 3. Information tools training. Recommendations from the logistics study are considered no longer relevant. February 2011 ITP closes 83 USAID/PERU MCC-IPT Assessment Report ANNEX D LIST OF DOCUMENTS CONSULTED FROM INEI Encuesta Demográfica y de Salud 2006, INEI, Lima Encuesta Demográfica y de Salud 2007, INEI, Lima Encuesta Demográfica y de Salud 2008, INEI, Lima Encuesta Demográfica y de Salud 2009, INEI, Lima Encuesta Demográfica y de Salud 2010, INEI, Lima Perfil Sociodemográfico del Perú, Censos Nacionales 2007: XI de Población y VI de Vivienda, Instituto Nacional de Estadística e Informática, Lima 2008. Encuesta Demográfica y de Salud Familiar 2007-2008, Instituto Nacional de Estadística e Informática, Lima 2009 FROM USAID USAID Evaluation Policy Peru Threshold Program: Data Quality Review and Midterm Assessment, CAMRIS, Washington DC 2011. RFTOP MCC 527-08-09, Immunization program Population, Health, and Nutrition Technical Assessment of three USAID/Peru Health Projects Implemented by the Ministry of Health: Report No. 10-001-259, GHTech, 2010 USAID Country Health Statistical Report Peru December 2009 Human Resource Development in Health: System for the Development of Competencies in Peru, February 2010 Using Information for Local Health Management in Peru February 2010 DISCLAIMER tance and Support Contract 3- Global Health (TASC 3 – Global Health) IQC Checklist for Assessing USAID Evaluation Reports FROM MCC-ITP http://inmunizaciones.programaumbral.pe/preguntas01.asp http://inmunizaciones.programaumbral.pe/programa01.asp Close of MCP en Ancash – good suggestions for sustainability. http://inmunizaciones.programaumbral.pe/noticiasdesc.asp?id=111 USAID/PERU MCC-IPT Assessment Report 84 http://www.minsa.gob.pe/portada/esninm_default.asp - Immunization Strategy web page http://www.minsa.gob.pe/portada/esninm_default.asp - Immunization Coverage per MOH Threshold Immunization Program: Monitoring and Evaluation Plan: November 2008 Gerencia en Inmunizaciones – Módulos de Diploma Virtual, Yanina Loyola, 2010 Módulos de Capacitación, AIESPED Propuestas para la Construcción de un Modelo de Atención Inclusivo en Salud para las Poblaciones Rurales Excluidas y Dispersas. Noviembre 2010. Análisis del Proceso Logístico de las Vacunas en sus Diferentes Niveles: Diagnóstico de Suministro de Vacunas: Informe de Consultaría: Dr. Hernán García y Dra. Cecilia Lengua, Mayo 2010. Plan de Mejora de Cadena de Suministro de Vacunas del MINSA, Nivel Nacional y Regional Informe de Consultaría: Dr. Hernán García y Dra. Cecilia Lengua, Diciembre 2010. Guía SIGA Patrimonio, Sistema de Información Integrado de Inmunizaciones (SIII) Guía Metodologica para la Capacitación de Supervisores del SIII Sistematización de la experiencia exitosa en la obtención de recursos públicos adicionales para el financiamiento de las actividades de inmunización en el Perú. Informe Final Carlos Meza Dextre. Agosto 2010 Contract No. GHS-1-00-07-00003 TO 365, p. 21, Abt Assoc, USAID/Peru 2008 MCC - Threshold Country Program (Tcp) Peru, Quarterly Report Q2 FY 2011, January 1, 2011 – March 31, 2011, USAID/Perú Tercer Informe Trimestral, Programa ITP, Abt Associates Lima 2010 MCC-ITP Component 2: Threshold Immunization Program, Monitoring and Evaluation Plan, November 2008, USAID/Peru Estudio Diagnóstico de la Estrategia Nacional de Inmunizaciones del Ministerio de Salud, Evaluación Anual 2008 – 2009, Mayo 2010 Procurement Report, USAID/ITP, Abt Assoc. Lima 2009 Informe y propuesta técnica para la atención de salud en zonas excluidas y dispersas. ITP, Lima 2008. http://inmunizaciones.programaumbral.pe/recursos01.asp Informe De Cierre Programa Umbral De Inmunizaciones, Región Ancash: Resultado y Pendientes. http://inmunizaciones.programaumbral.pe/noticiasdesc.asp?id=111 Análisis Del Proceso Logístico De Las Vacunas En Sus Diferentes Niveles: Diagnóstico De Suministro De Vacunas. USAID/Perú, 2010 85 USAID/PERU MCC-IPT Assessment Report Estudio Diagnóstico De La Estrategia Nacional De Inmunizaciones (ESNI) Del Ministerio De Salud, USAID/Perú, 2009 ITP Final Report, Abt Assoc., Lima 2011 Plan de Mejora de Cadena de Suministro de Vacunas del MOH, Dr. Hernán García, Dra. Cecilia Lengua, MCC-ITP, Lima, 2010. Si Su Nino Esta sano, Nosotros Somos Mejores, Conocimientos, Actitudes y Practicas en Turno a la Vacuna, Estudio Cualitativo, Programa ITP, Abt Assoc. Lima 2011 Plan of Operations for the Threshold Program of Immunizations, Abt, Peru 2008-2010 Informe y propuesta técnica para la atención de salud en zonas excluidas y dispersas, María Tello Acho, Lima 2008. El Costo – efectividad de la Atención Integral de Salud para Poblaciones Excluidas y Dispersas (AISPED), Giovanni Alarcón, 21 de Enero de 2011 Millennium Challenge Corporation Threshold Country Program (TCP) Peru Quarterly Report Q2 FY 2011, January 1, 2011-March 31, 2011 Plan Regional de Mantenimiento de Equipos de Cadena de Frío 2012 – (draft) Noviembre 2011. FROM MEF Intervención Pública Evaluada: Servicio de Vacunación. MEF, Lima 2011 FROM MOH Resolución Ministerial No. 457-2009/MOH, Ministerio de Salud, Lima 2009. Resolución Ministerial No.614-2010/MOH, Ministerio de Salúd, Lima 2010. Norma Técnica: Atención Integral de Salud a Poblaciones Excluidas y Dispersas. RM-478- 2009/MOH, Lima 2009 Relación de participantes para el curso virtual de especialización de gerencia en inmunizaciones, MSDG, MINSA Plan Operativo Anual de Inmunizaciones,- región Amazonas, Chachapoyas, 20 Abril, 2011. PLAN REGIONAL DE LA ESTRATEGIA SANITARIA DE INMUNIZACIONES, Apurimac, Abancay, Diciembre 2010 Norma Técnicas de Salud que Establece el Esquema Nacional de Vacunación, MINSA/DGSP Instrumentos Para la Supervisión, (de Plan de Acción y Linimientos Técnicos de la Campaña de Vacunación para el Barrido de Polio y Sarampión –Rubéola, 2011. USAID/PERU MCC-IPT Assessment Report 86 ANNEX E LIST OF CONTACTS SOCIAL IMPACT MCC-ITP ASSESSMENT People Contacted NAME ROLE/ ORGANIZATION CONTACT INFO DATE CONTACTED BY Kathleen Novak Abt Associtates 301-347-5321 kathleen_novak@abtassoc.com 9/13/11 EB, RS Jim Heinzen Camris (703)548-6713 EB, RS Bruce Kay MCC Enrique Paz UNICEF - Panama 507-301-7403 epaz@unicef.org Luis Seminario USAID 988544422 lseminario@usaid.gov 9-26-11 DN,EB,RS,PM Mirian Choy PDP/USAID 980127328 mchoy@usaid.gov 9-26-11 DN,EB,RS,PM Theresa Herscowitz USAID 618-1267 therscowitz@usaid.gov 9-26-11 DN,EB,RS,PM Jo Jean Elenes Chief HEO/USAID 618-1267 9-26-11 DN,EB,RS,PM Myrian Sarco USAID 618-1267 9-26-11 DN,EB,RS,PM Dr. Ada Pastor USAID/MCC-ITP 9-26-11 DN,EB,RS,PM Dr. Sonia Hilser MINSA/CI 985945927 shilserv@minsa.gob.pe 9-27-11 DN,EB,RS,PM Arq. Hernan Roig MINSA/CI 3156600 ext 2412 hroiga@minsa.gob.pe 9-27-11 DN,EB,RS,PM Dr. Luz Illescas CAMRIS 989295094 luzillescasr@yahoo.com 9-27-11 DN Gerardo Morris PCM/Unbral 996293295 gmorris@pcm.gob.pe 9-27-11 DN,EB,RS,PM Julio Cesar Naveda PCM/Unbral 985044135 jnaveda@pcm.gob.pe 9-27-11 DN,EB,RS,PM Midori de Habich USAID/Políticas en Salud-Abt 4176000 mdehabich@polsalud.org 9-28-11 DN Patricia Mostajo USAID/Políticas en Salud-Abt 4176000 pmostajo@polsalud.org 9-28-11 DN Arturo Granados USAID/Políticas en Salud-Abt 4176000 agranados@polsalud.org 9-28-11 DN Dr. Oscar Cordón USAID/Quality Healthcare-Quimonics 985629907 ocordon@calidadensalud.pe 9-28-11 DN Dr. Lengua, Dr. Garcia 9-28-11 DN,EB,RS,PM Fabiola Quiroz UNICEF 9-29-11 DN Judith Aguinaga JHU-Abt 9-29-11 DN,EB,RS,PM Dr. Hamilton Garcia DGPS/MOH 10-4-11 DN, RS Laura Veramendi DGPS/MOH 10-4-11 DN,RS 87 USAID/PERU MCC-IPT Assessment Report NAME ROLE/ ORGANIZATION CONTACT INFO DATE CONTACTED BY Aquiles Vilches DG Epidemiología MINSA 10-4-11 DN,RS María Ticona DG Epidemiología MINSA 10-4-11 DN,RS Gladys Turpo DG Epidemiología MINSA 10-4-11 DN,RS Jorge Uchuya DG Epidemiología MINSA 10-4-11 DN,RS Edward Ochoa Responsable SIS DIRESA Cusco edwardochoav@hotmail.com Cel 984994800 13-oct-11 EB, PM Efrain Sarmiento Equipo Tecnico SIS DIRESA Cusco Cel 984622760 efrainsh4@hotmail.com 13-oct-11 EB, PM Jesús Fuentes Mujica Médico Jefe CS Manco Capac, Cusco 12-oct-11 EB, PM Efrain Gonzales Becerra Responsable Est. E Informatica CS Ttio, Cusco becerraefra@hotmail.com 12-oct-11 EB, PM Mario Cornejo Director General, DIRESA Cusco 10-oct-11 EB, PM Julio César Vargas Sub Director, DIRESA Cusco 10-oct-11 EB, PM Zonia Rozas Huacho Directora Ejecutiva de Salud Individual, Cusco Cel 984627566 zrozas@hotmail.com 10-oct-11 EB, PM Angelica Zapata Cusco leohuaran@hotmail.com 10-oct-11 EB, PM Javier Ramirez Gutierrez Responsable Cadena de Frio DIRESA, Cusco jrg_1111@hotmail.com 10-oct-11 EB, PM Pablo Grajeda Director de Inteligencia Sanitaria DIRESA Cusco 10-oct-11 PM Isabel Fuentes Directora de Estadisticas, DIRESA Cusco isabelf9@hotmail.com 10-oct-11 PM Erika Chalco Merma Responsable Etapa de Vida Niño, DIRESA Cusco docarri_e@hotmail.com 10-oct-11 EB, PM Tania Saldivar Tapia Responsable ESRI Cusco Cel 984572419 taniali6@hotmail.com 10-oct-11 EB, PM Mabel Sarmiento Huanca Coordinadora Estrategia No Transmisibles, DIRESA, Cusco notransmisiblescusco@yahoo.com.pe 10-oct-11 EB, PM Dario Navarro Mendoza Director de la Red Cusco Sur, Cusco 11-oct-11 EB, PM Lilian Karina Lima Grajeda Coordinadora ESI Cusco Sur, Cusco lilikg@hotmail.com 11-oct-11 EB, PM Marina Quiroga Canal Jefa ODIS Cusco Sur, Cusco marinaquiroga@hotmail.com 11-oct-11 EB, PM Juan Lopez Muñoz Gerente CLAS San Jerónimo, Cusco clas_sanjeronimo@yahoo.es 11-oct-11 EB, PM Daysi Sota Paz Responsable Inmunizaciones San Jeronimo, Cusco Cel 974350067 11-oct-11 EB, PM USAID/PERU MCC-IPT Assessment Report 88 Olga Apaza Zela Directora de la Oficina de Aseguramiento, Puno 17-oct-11 EB, PM NAME ROLE/ ORGANIZATION CONTACT INFO DATE CONTACTED BY Rudy Tico Chayña Director de Aseguramiento Público, Puno ticorudy@hotmail.com Tel 051-365303 17-oct-11 EB, PM Maria Linares ODSIS Puno mlinares@sis.gob.pe Tel 051-363677 17-oct-11 EB, PM Antonia Figueroa Ramos Encargada, Area de Análisis Epidemiológico, Puno tukyfigueroa@hotmail.com Cel 951711246 17-oct-11 PM Antonio Olazabal Chayña Director, Atención Integral de Salud, Puno olazabalch@hotmail.com Cel 951749229 17-oct-11 EB, PM Gabiana Ticona Cacasaca Enfermera, AIS Niño, REDESS Chucuito, Puno chevetomim@hotmail.com 19-oct-11 EB, PM Alicia Quispe Quispe Enfermera ESNI, REDESS Chucuito, Puno alicia_r63@hotmail.com 19-oct-11 EB, PM Wilder Cruz Ticona Director ODI, REDESS Chucuito, Puno Wcruztmd_900009@hotmail.com 19-oct-11 EB, PM Juan de Dios Velasquez Pimentel Estadistico, REDESS Chucuito, Puno Polayjvp_10@hotmail.com 19-oct-11 EB, PM Zenaida Cruz Ortega Farmaceutica, SISMED REDESS Chucuito, Puno zenaida_cruz@hotmail.com 19-oct-11 EB, PM Emilia Lupaca Valeriano Enfermera, AIS Adulto Mayor, Chucuito, Puno, Puno elura30@hotmail.com 19-oct-11 EB, PM Honorata Ortega B. Enfermera ESRI DIRESA Puno norita294@hotmail.com 19-oct-11 EB, PM Juana Pilco Monroy Enfermera ESNI Microrred Juli, Puno juanapilcomonroy@hotmail.com 19-oct-11 EB, PM Gloria Suana Muñoz Enfermera, Epidemiologia, Chucuito, Puno gloriairene@hotmail.com 19-oct-11 EB, PM 89 USAID/PERU MCC-IPT Assessment Report NAME POSITION ORG. ADDRESS PHONE CELL EMAIL Rofilia Ramirez Directora Técnica de Demografía e indicadores sociales (ENDES) INEI Gral. Garzón 654, Jesús María 610 5200/ 203 2640 Central 203 9280 Directo, 997 567 422 Rofilia Ramírez 433 8284 Jefatura, 652 0082 Demografía e Ind. Sociales patricia.huapaya@inei.gob.p e (Asist Dirección); rofilia.ramirez@inei.gob.pe; maria.manotupa@inei.gob.p e Dr. Fernando Gonzáles Responsable ENDES INEI Gral. Garzón 654, Jesús María 652 0000 433 4223 jrios@paho.org; jrios@paho.org Dra. Sonia Hilser Directora General de Cooperacion Internacional MINSA - Coop. Internacional Salaverry 801 315 6600 Ax 2408 shilserv@minsa.gob.pe, lvasquezq@minsa.gob.pe Eva. Guerrero Director General, Oficina General de Descentralizacion MINSA - Descentralización Av. Arequipa 810, 4 piso 988 030 320 eguerreroc@minsa.gob.pe Dr. Roly Cruz Jefe Estadística Inmunizaciones MINSA - DGE (edistica) Av. Salaverry 801, 2do. Piso Jesús María 315 6600 rcruz@minsa.gob.pe Manuel Vargas Director MINSA - DIGEMID - Dir General de Medicamentos Insumos y Drogas Calle Coronel Odriozola 111, San Isidro. 422 8456/422 9200 996 069 337 dg@digemid.minsa.gob.pe Carmen Cuevas, Pedro Ijarasca Funcionarios DIGEMID MINSA - DIGEMID Calle Coronel Odriozola 111, San Isidro. 422 8456/422 9200 Lic. Ma. Ana Mendoza Coord. Nacional MINSA - ESNI Av. Salaverry 801, Jesús María 315 6600, Ax 2664 952 324 679 mmendoza@minsa.gob.pe; plandeo@minsa.gob.pe Dr. Luis Alberto Huarachi Jose Carlos del Carmen, Director Adjunto Director MINSA- SIS Av. Carlos Gonzáles, Urb. Maranga, San Miguel (altura Metro, costado Edelnor) 6273800 Ax 1010, Directo 464 7238 562 0251 director Adj. Liz Meneces Sec. alhuarachi@sis.gob.pe, gzambrano@sis.gob.pe Leyda Sánchez Supervisora de Logística (se realizan todas las adquisiciones) No conocen a Vilma Vargas MINSA-Adquisiciones Av. Salaverry 801, Jesús María 315 6600 Ax 2305, 2306 Dr. Carlos Acosta Director General MINSA DG Salud de las personas Av. Salaverry 801, Jesús María Directo: 315 6687 /315 6600 Ax cacosta@minsa.gob.pe; dlopeze@minsa.gob.pe Hamilton García Díaz Director General MINSA - Promoción de la Salud Av. Salaverry 801, Jesús María 315 6600 Ax 2658 hgsrcia@minsa.gob.pe, tlazo@minsa.gob.pe Flor Garcia, Julio Canseco Oficina General de Planificación y Presupuesto MINSA - OGPP Av. Salaverry 801, Jesús María 315 6600 USAID/PERU MCC-IPT Assessment Report 90 NAME POSITION ORG. ADDRESS PHONE CELL EMAIL Pablo Cordova DG Estadística e Informática MINSA - DGEI Av. Salaverry 801, Jesús María 315 6600 Ax 2658 Lesly Aliaga Directora de Desarrollo Tecnológico MINSA - DGEI Av. Salaverry 801, Jesús María 315 6600 Ax 2658 Dr. Luis Suarez Ognio Director General MINSA Epidemiología 316 6600 Ax 4614239 Ax 113, 461 3679, 461|3687 lsuarez@dge.gob.pe pquevedo.dge.gob.pe Walter Curioso Vilchez Director General MINSA - Estadística e Informática Av. Salaverry 801, Jesús María 315 6600 Ax. 2301 wcurioso@minsa.gob.pe, fbarrenechea@minsa.gob.pe Eco. Leonardo Paz Puelles (derivó atención al Sr. Slor García) Director General (Jefe ) MINSA - Plaaneamiento e Inversiones (Planeamiento y Gestión Institucional) Av. Salaverry 801, Jesús María Directo 315 6678, central 315 6600 Ax 2826 rvalenciar@minsa.gob.pe Quim. Carmen Ponce Directora MINSA - Abastecimiento de recursos estratégicos en salud Jr. Nazca 548, Jesús María 628 5761, 66 y 67 cponce@minsa.gob.pe, melespuru@minsa.gob.pe Eco. Segundo Díaz Hernández Director General MINSA - Oficina General de Administración Av. Salaverry 801, Jesús María Directo 315 6680, central 315 6600 Ax 2115 sdiazh@minsa.gob.pe, eperezv@minsa.gob.pe Juan Pablo Silva Director General MEF - Dirección de Calidad del Gasto Junín 319 - Lima jpsilva@mef.gon.pe, mtejada@mef.gob.pe Washington Toledo Inmunizaciones OPS (PAHO) Los Pinos 251, Urb. Camacho, La Molinaa (costado Iglesia la Resurección, espalda Wong) 3195700, directo: 3195767 wtoledo@per.ops-oms.org; aplenge@per.ops-oms.org Sr. Ben Schneider Coordinador Umbral Av. Javier Prado Oeste 1661 San Isidro 421 6322 999 099 931 cberiso@comsa.com.pe Gerardo Morris PCM 447 2549, 242 0925 996 293 295 gmorris@pcm.gob.pe Judith Aguinaga Coordinaroa de Proyectos Redess Jóvenes (Johnn Hopkins) 2 de Mayo 315, San Isidro 628 6043 995 994 776, RPM #871 869 jaguinaga11@yahoo.com Mario Tavera Oficial de Salud UNICEF Pque. Melitón Porras 350, Miraflores 612 07 07 Ax 740 mtavera@unicef.org; rfernandez@unicef.org 91 USAID/PERU MCC-IPT Assessment Report NAME POSITION ORG. ADDRESS PHONE CELL EMAIL Danilo Fernandez 997 351 743 dfernandez39@yahoo.es DRA. ZONIA ROZAS Directora Ejecutiva Salud Individual DIRESA CUSCO 084-581564 ó 084-581562. 984 627 566 zrozas@hotmail.com DR. PABLO GRAJEDA Director Ejecutivo Inteligencia Sanitaria 984 603 297 diresacusco_dg@yahoo.es DANTE POLLOYQUERI MAMANI DIRESA PUNO JR. 7 DE JUNIO N° 352 URB. CHANU PUNO 951 613 253 Eco. José Arista Presidente Gobierno Regional Amazonas Prolongación Ortiz Arrieta 1250, Chachapoyas 041 477 796 / 478 131 jarista@regionamazonas.gog.pe, oampuero@regionamazonas.gog.pe Dr. Agusto Wong Vice Presidente Gobierno Regional Amazonas Prolongación Ortiz Arrieta 1250, Chachapoyas 42 477 796 / 478 131 Jenny Nuñez Gerente de Desarrollo Social Gobierno Regional Amazonas Prolongación Ortiz Arrieta 1250, Chachapoyas 42 477 796 / 478 131 Dr. Francisco Ramos Santillan Director General DIRESA Amazonas Jr. El Triunfo Cdra 3 s/n Chachapoyas 041 477 960 framos@diresamazonas.gob.pe, Dra. Janet Valle Sub-directora Regional de Salud, Amazonas DIRESA Amazonas Jr. El Triunfo cuadra 3 Chachapoya Jenny Nuñez Gerente de Desarrollo Social Gobierno Regional Amazonas Alfredo Diaz Jefe de comunicaciones DIRESA Amazonas Jr. El Triunfo cuadra 3 Chachapoyas Ponciano Alvarado, JC Saldaño Almacen Regional, Responsable de Patrimonio DIRESA Amazonas Jr. El Triunfo cuadra 3 Chachapoyas Agustin Alba Diaz Jefe de DIREMID DIRESA Amazonas Jr. El Triunfo cuadra 3 Chachapoyas Bernardino Villeras, Claudio Pinzón Responsable cadena de Frío, DIREMID DIRESA Amazonas Jr. El Triunfo cuadra 3 Chachapoyas Juanito Torres Jefe de Oficina Ejecutiva de Estadística e Informática DIRESA Amazonas Jr. El Triunfo cuadra 3 Chachapoyas USAID/PERU MCC-IPT Assessment Report 92 NAME POSITION ORG. ADDRESS PHONE CELL EMAIL Geovanna Pachas Coordinador AISPED DIRESA Amazonas Jr. El Triunfo cuadra 3 Chachapoyas Manuel Rivas Gerente de Estrategias de Salud DIRESA Amazonas Jr. El Triunfo cuadra 3 Chachapoyas Esther Manchena Responsable ESRI Regional DIRESA Amazonas Jr. El Triunfo cuadra 3 Chachapoyas Rosemery Poema Directora Micro-red Chachapoyas Micro-red Chachapoyas Chachapoyas NAME POSITION INSTITUTION CONTACT INFORMATION Dr. Roberto Romero Asesor a Gerencia Seguro Integral de Salud (SIS) Dr. Pedro Rojas Asistente SIS Dr. Miguel Avila Asistente SIS Martin Vasquez C. Consultor informático UMBRAL, responsable del IIIS y Gis Lima 989037784 Dra. Lengua, Dr. Garcia Consultores UMBRAL – sistema de inmunizaciones Lima Victor Rosangel, Huber Gonzalez Sectorialistas de Salud MEF 3115930 ext 2175 Hgonzalez@mef.gob.pe 93 USAID/PERU MCC-IPT Assessment Report NAME POSITION ORG. ADDRESS PHONE CELL EMAIL Sonia Paredes, Joaquin Tanca, Blanca Joaquim Responsable ESRI Micro-red Chachapoyas Chachapoyas José Raúl Ayala, Mario Santillan Responsable SIS, Informático Micro-red Chachapoyas Chachapoyas John Acuña Responsable ESRI Puesto Pipus Chachapoyas Richard Llanca Responsable ESRI Red Bagua Chachapoyas Dra. Martinez Abad, Abel Bustamante, Yesenia Barreto, Edward Ciezta Responsables ESRI Red Utcubamba Chachapoyas Manuel Rivas Coordinador UMBRAL DIRESA Amazonas Jr. El Triunfo cuadra 3 Chachapoyas Norma Cruz Directora de Promoxión de Salud DIRESA Amazonas Jr. El Triunfo cuadra 3 Chachapoyas Dr Eduardo Quezada Director de Epidemiología DIRESA Amazonas Jr. El Triunfo cuadra 3 Chachapoyas Rosalía Vargas Depto. Atención Integral de Salud DIRESA Amazonas Jr. El Triunfo cuadra 3 Chachapoyas Dr. Carlos Hernan Monteagudo Gonsalez Director General DIRESA Apurimac Av. Daniel A. Carrión, Abancay 083-321117, 083-323690 Carloshmg2@hotmail.com Carloshmg2@hotmail.com José Portacarrero, Liliana Santillan Responsables AISPED DIRESA Apurimac Av. Daniel A. Carrión, Abancay Rosa López P. OAUM DIREMID Apurimac Av. Daneil Alcides s/n 083-321117, 083-323690 rosemilopez@yahoo.es María I. Ccahuana T. Director Estadística DIRESA Apurimac Av. Daneil Alcides s/n 083-321117, 083-323690 Maria_ccahuana@yahoo.es Segundino Serrano L. Técnico en Cadena de Frío DIRESA Apurimac Av. Daneil Alcides s/n 083-321117, 083-323690 Teófilo Caceres Ll. Coordinador ESRI DIRESA Apurimac Av. Daneil Alcides s/n 083-321117, 083-323690 Tecalle.007@gmail.com Carmen Luz Luna G. Directora de Promoción DIRESA Apurimac Av. Daneil Alcides s/n 083-321117, 083-323690 Luzluna3@hotmail.com USAID/PERU MCC-IPT Assessment Report 94 Miriam Monzón C. Supervisora ESNI MINSA Miriam_monzonc@hotmail.com NAME POSITION ORG. ADDRESS PHONE CELL EMAIL Jorge Luis Ruíz P. Director de Atención Integral de la Salud DIRESA Apurimac Av. Daneil Alcides s/n 983672774 Geisizto15@hotmail.com Miryam Solórzano C. Responsable Estrategia Vacuanción Micro-red Centenario Abancay Miryam_25Z@hotmail.com Leoncio Yupanqui Responsable cadena de frío Micro-red Centenario Abancay Mayra Giannina Estrada A. Directora Imagen Institucional DIRESA Apurimac Av. Daneil Alcides s/n 083-321117, 083-323690 Mayrita-21@hotmail.com María Povez A. Coordinadora AISPED DIRESA Apurimac Av. Daneil Alcides s/n 083-321117, 083-323690 mapoveza@yahoo.es Darwin Pezo O. Director LRSP Abancay darwinpezo@hotmail.com Elsa Alfaro A. Responsable de PROMS Vida Sana DIRESA Apurimac Av. Daneil Alcides s/n 083-321117, 083-323690 Ecalfaro73@gmail.com Juan Saavedra R. Director Ejec. Recursos Humanos DIRESA Apurimac Av. Daneil Alcides s/n 083-321117, 083-323690 Juansr40@hotmail.com Carlos A. Villanueva L. DESP DIRESA Apurimac Av. Daneil Alcides s/n 083-321117, 083-323690 Carloespn19_83@hotmail.com Marcelina Gamarra C. C.S. Pueblo Joven Abancay Marcey260y99@hotmail.com Luzmila Consoyo F. Coordinador Etapa Niño DIRESA Apurimac Av. Daneil Alcides s/n 083-321117, 083-323690 luzmilacoflores@hotmail.com Aurora Cáceres Ll., Hilda Reinoso Responsable ESRI C.S. Tamburco Abancay A.C.LL.18@hotmail.com Agustina Chacón Digitadora SIS C.S. Tamburco Abancay Elías Gómez A. ODSIS DIRESA Apurimac Av. Daneil Alcides s/n 083-321117, 083-323690 egomez@sis.gob.pe Gabino Villafuerta Responsable ESRI Red Chalhuanco Chalhuanco, Apurimac Virgilio Cespedes, Rosemery Quispe Ex SIS, Enfermera ESRI Red Chalhuanco Chalhuanco, Apurimac José Lizárraga Gerente de Desarrollo Social Gobierno Regional Abancay, Apurimac 083-324084 983702820 jomolisto@hotmail.com 95 USAID/PERU MCC-IPT Assessment Report ANNEX F REGIONAL COMPARISON TABLE Table 2 beginning on the next page compares regions by Assessment Question. USAID/PERU MCC-IPT Assessment Report 96 Table 2. COMPARISON OF REGIONS BY ASPECTS OF ASSESSMENT QUESTION 1. COMPONENT/ ASPECT AMAZONAS APURIMAC CUSCO PUNO (a) What were the effects and achievements of the MCC-ITP in the region, program including but not limited to: Impact of the MCC-ITP in the region • 21 registered for VDIM course. • Cold room installed/functioning • All cold chain assets in SIGA-patrimonio • IEC Toolkit not used • IIIS totally blocked by Statistics • 17 registered for VDIM course. • Cold room not functioning • All cold chain assets in SIGA-patrimonio • IEC Toolkit being used by Communications Officer – Institutional Image • IIIS being used in Abancay sub-region • VDIM-11 people completed31 • AISPED equipment distributed to 8 IBs the course, but certificates have not been distributed yet. • 5 AISPED training workshops, 35 staff and 7 trainers trained • Entered 744 cold chain assets into regional inventory • Regional immunization plan developed and approved • 49 registered for VDIM course • 41 community health agents trained and child database created • None of the staff were able to complete VDIM course due to internet access problems • AISPED equipment distributed. • Four people trained in SIGA-patrimonial. 42 cold chain items were entered in the regional SIGA inventory • 8 IB (40 staff) trained and equipped and 10 AISPED trainers trained Factors that hindered effectiveness Changing health priorities due to frequent turnover of local officials • Increased complexity of vaccination schedule • Population has migrated and is overestimated. • Staff turnover • Population is over￾estimated, with denominator based on projections and not actual # of births • Inaccurate numerator due to high population mobility. • Staff turnover. Puno had 3 directors in the last 18 months. • Up to 33% underestimated population. Inaccurate denominator based on projections and not actual # of births. • Inaccurate numerator due to high population mobility 31 Cusco End of MCC-ITP Report 97 USAID/PERU MCC-IPT Assessment Report COMPONENT/ ASPECT AMAZONAS APURIMAC CUSCO PUNO Factors that facilitated effectiveness • Excellent ITP coordinator • VDIM • Trainings • Excellent ITP coordinator • VDIM • Trainings • Excellent ITP coordinator • Planning skills for Budget for Results. • Sound studies and evidence to tailor communication activities • 40 community health agents trained in communication • Excellent ITP coordinator • Planning skills for Budget for Results. • Tailored communication materials in various local languages • Communication workshops for community leaders and health staff. • Community Surveillance workshops in 25 facilities Reasons for difference in immunizations coverage outcomes within and among regions that received MCC-ITP support • No reasons can be given because C-DHS coverage reports are inconclusive and OGEI reports are distorted by incomplete reporting • No reasons can be given because C-DHS coverage reports are inconclusive and OGEI reports are distorted by incomplete reporting • No reasons can be given because C-DHS coverage reports are inconclusive and OGEI reports are distorted by incomplete reporting • No reasons can be given because C-DHS coverage reports are inconclusive and OGEI reports are distorted by incomplete reporting COMPONENT/ ASPECT AMAZONAS APURIMAC CUSCO PUNO (b) What is the post MCC-ITP status of development of the regional immunizations program including but not limited to: POLITICAL PRIORITY AT THE DIFFERENT LEVELS; • High priority accorded by regional president and DIRESA • High priority accorded by regional social development manager and DIRESA • High priority accorded by regional social development manager and DIRESA • High priority accorded by regional Social Development Manager and DIRESA FUNDING MECHANISMS AT THE DIFFERENT LEVELS (NATIONAL, SUB￾NATIONAL); • Excellent funding support from MEF via PpR. • Growing municipal and private sector support due to close communication with health staff. • Excellent funding support from MEF via PpR. • Weak municipal support due to inadequate communication with health staff. • Adequate funding. Expressed need to program funds appropriately • Adequate funding. Expressed need to improve capacity of networks to program funds and recruit staff USAID/PERU MCC-IPT Assessment Report 98 COMPONENT/ ASPECT AMAZONAS APURIMAC CUSCO PUNO NORMS AND PROCEDURES – IMPLEMENTATION, MONITORING AND IMPROVEMENT; • All service norms and procedures followed; IIIS reporting through SIS is blocked by Directoral decree; cold chain norms followed. • All service norms and procedures followed; SIS reporting is undergoing revision because of AUS; cold chain norms followed • All forms available and norms followed • All forms available and norms followed HUMAN RESOURCES – STAFFING QUALITY AND QUANTITY; • Regional and network personnel responsible for ESRI was trained with VDIM. Professional nurses have replaced health technicians in most vaccination sites – very competent. • Increased number of professional nurses. At least 5 took VDIM course and have replaced health technicians. Regional and network staff responsible for ESRI was trained with VDIM, competent • High staff turnover • Competent staff observed • High staff turnover • Short of staff nurses, expressed need to recruit and renew contracts • Observed staff competence MANAGERIAL AND ADMINISTRATIVE CAPACITY AT NATIONAL AND SUB-NATIONAL LEVEL (REGIONAL, PROVINCIAL, MUNICIPAL OR DISTRICT) TO PLAN, IMPLEMENT AND MONITOR IMMUNIZATIONS; • High managerial and administrative capacity at regional and network level. Facility level capacity weak in secondary positions. • High managerial and administrative capacity at regional and network level. Facility level capacity weak in secondary positions. • High managerial capacity but staff turnover • Expressed need to improve capacity at network and micronetwork levels • High managerial capacity but staff turnover. • Expressed need to improve capacity at network and micronetwork levels 99 USAID/PERU MCC-IPT Assessment Report INFORMATION AND EPIDEMIOLOGICA L SURVEILLANCE – DATA QUALITY AND USE; • The regional epidemiology office does not publish epidemiologic bulletins on the DIRESA website. It does monitor notifications and investigates outbreaks of VP diseases, but does not inform vaccination coverage • The most recent epidemiologic bulletin is of out of date. The last bulletin announced a 45% reduction in severe pneumonia to date, compared to the same time period in 2010. The EDAs also decreased by 15% of reported cases from 2010 and dysenteric diarrhea decreased by 17% compared with 2010. Vaccination coverage was not reported. • The epidemiologic bulletin is up to date. The latest issue reported a measles outbreak in Ecuador and the reintroduction of a wild virus of measles and emphasized the need “to strengthen surveillance and to reinforce prevention and control.” It also reported seven probable cases of rubella that were discarded based on laboratory results. The bulletin also announced that MMR vaccination coverage rate for the Cusco region did not reach 95% in 2010. • The most recent epidemiologic bulletin is out of date. The latest issue reported thirteen cases of rubella that were discarded based on laboratory studies and reported one case of whooping cough. Pneumonia reportedly decreased from forty￾nine cases in 2010 to thirty-three cases in 2011. Vaccination coverage was not reported. DEGREE OF PROTECTION ACHIEVED, THE LEVEL OF RISK AND EPIDEMIC POTENTIAL; • The pockets of unvaccinated population, especially in border regions, is of major concern because of low levels of MMR vaccination rates in Condorcanqui and Jaen, on Peru’s northern border. Children non￾responsive • The DGE estimates that there are 300,000 persons who do not make antibodies to mumps and measles and perhaps 500,000 susceptible to polio. If proportional to population, in Apurimac, this would amount to 4925 and 7158 children, respectively. • The DGE estimates that there are 300,000 persons who do not make antibodies to mumps and measles and perhaps 500,000 susceptible to polio. If proportional to population, in Cusco, this would amount to 12448 and 20747 children, respectively. • The DGE estimates that there are 300,000 persons who do not make antibodies to mumps and measles and perhaps 500,000 susceptible to polio. If proportional to population, in Puno, this would amount to 13479 and 22466 children, respectively. INVOLVEMENT OF NATIONAL SURVEILLANCE SYSTEM FOR DETECTING AND CONTROLLING VACCINE￾PREVENTABLE DISEASES; • The national VP disease surveillance unit is responsible for monitoring and investigating all reported cases of VP diseases from over 7000 health facilities in the country, through the regional epidemiologist in Amazonas who reports • The national VP disease surveillance unit is responsible for monitoring and investigating all reported cases of VP diseases from over 7000 health facilities in the country, through the regional epidemiologist in Apurimac who reports all • The national VP disease surveillance unit is responsible for monitoring and investigating all reported cases of VP diseases from over 7000 health facilities in the country, through the regional epidemiologist in Cusco who reports all • The national VP disease surveillance unit is responsible for monitoring and investigating all reported cases of VP diseases from over 7000 health facilities in the country, through the regional epidemiologist in Puno who reports all USAID/PERU MCC-IPT Assessment Report 100 all outbreaks of VP diseases. outbreaks of VP diseases. outbreaks of VP diseases. outbreaks of VP diseases. COLD CHAIN FUNCTIONALITY – SUPPLY QUANTITY AND QUALITY; • Cold room operating since Oct 2010, All of the facilities visited store vaccines in refrigerators and freezers that are between two and five years old, are electric, and function adequately (consistent temperatures according to norms). Local technicians are adapting domestic refrigerators for solar power • Cold room installed but lacking electrical connection. All of the facilities visited store vaccines in electric refrigerators and freezers that are between two and five years old and function with temperatures according to norms. In some areas of rural Apurimac, there are still a number of domestic/household refrigerators • Two cold rooms functioning since 2010. All of the facilities visited store vaccines in electric refrigerators and freezers that are between two and five years old and function with temperatures according to norms. All staff in the facilities reported that they have adequate stock for routine vaccinations. • Building to house cold rooms is under construction. All of the facilities visited store vaccines in electric refrigerators and freezers that are between two and five years old and function with temperatures according to norms. In some areas of rural Puno, there are still a number of domestic/household refrigerators BIOSECURITY PRACTICES IN THE HANDLING AND DISPOSAL OF USED SYRINGES AND VIALS; AND • All facilities used disposable syringes. Most immunization services discarded their syringes in MOH-provided, cardboard security boxes. In Pipus, the security boxes were put into a fenced and were set on fire by staff. • All facilities used disposable syringes. Most immunization services discarded their syringes in MOH-provided, cardboard security boxes. • All facilities used disposable syringes. Most immunization services discarded their syringes in MOH-provided, cardboard security boxes • All facilities used disposable syringes. Most immunization services discarded their syringes in MOH-provided, cardboard security boxes. In rural sites, the security boxes were put into a fenced and were set on fire by staff. ACHIEVEMENTS IN SOCIAL COMMUNICATION AND THE DEGREE OF USER SATISFACTION – SERVICE QUALITY, QUANTITY, AND DEMAND; AND • Communication is excellent. One nurse communicated particularly well with a pair of agitated parents who were sure that the rotavirus vaccination given at the last visit had given the child diarrhea. She patiently listened, provided information, discussed myths that others had told the mother, and explained • The institutional communications director of DIRESA/Apurimac has used the brochures and other tools, in Spanish and in Quechua. She informed the Team that she has plans to train 100 newspaper media persons and give them the messages after these are properly adapted for the region. Communications in • Health staff communicated actively, explaining the vaccinations, discussing any adverse reactions to be expected, counseling on home-based care of fever, swelling, and pain. However, parents were not able to say which vaccines had been administered. This perhaps is due to the large number of vaccines • The Team observed a nurse communicating with mothers and infants in a warm, welcoming way, explaining the vaccinations, showing the schedules, and vaccinating the children in a friendly and supportive manner. 101 USAID/PERU MCC-IPT Assessment Report why they were not true. She communicated in a calm and supportive manner with the parents, who then let her vaccinate the child. vaccination services were generally one-way, from the staff to the mother, and staff did not appear to be able to draw mothers into dialogue. and to names unrelated to the diseases they prevent. CAPACITY TO ADAPT TO NEW DEMANDS GENERATED BY HEALTH SECTOR REFORM, INCLUDING DECENTRALIZATI ON, UNIVERSAL HEALTH INSURANCE, INTEGRATION OF IMMUNIZATIONS INTO MATERNAL AND CHILD HEALTH PROGRAMS, AND THE INCLUSION OF NEW TECHNOLOGIES AND VACCINES. • The Condorcanqui region adapted upright refrigerators so they would lie horizontally and adapted condensers to work with solar power. Personnel readily adopted distance￾learning technology for the VDIM Diploma course. The DIRESA developed the NovHIS software for its health information system. The Regional President discussed the use of the bar codes on the DNI to give access to social services and allow tracking of immunizations and other services. • Personnel readily adopted distance￾learning technology for the Virtual Immunization Management Diploma course. Apurimac is notable for adaptation to increased storage requirements of mono￾dose vaccines, the use of data loggers with digital computer readouts to monitor temperatures of the horizontal refrigerators and freezers. • Personnel readily adopted distance￾learning technology for the Virtual Immunization Management Diploma course. The Cusco POI adapted MOH norms regarding immunization schedule, priority vaccines, historical coverage by districts, RIS organizational and functional structure, staffing and resources. • The Puno POI was adapted from MOH norms regarding immunization schedule, priority vaccines, historical coverage by districts, RIS organizational and functional structure, staffing and resources. USAID/PERU MCC-IPT Assessment Report 102 COMPONENT/ ASPECT AMAZONAS APURIMAC CUSCO PUNO (c) How has the regional immunizations program progressed since the close of the MCCITP, in terms of but not limited to: ADHERENCE TO REGION-SPECIFIC RECOMMENDATIO NS PROVIDED BY MCC-ITP; • Four Amazonas-specific recommendations out of 8 are currently being undertaken: -Use PpR to create clear incentives for analysis and use of information -Continue PpR budgeting to increase funding for operating expenses -Make/distribute copies of the VDIM modules for individual study. - 100% of children under five have DNI. • Three of ten Apurimac￾specific recommendations pending. 1. Form community surveillance committees. 2. Distribute VDIM certificates. 3. Redistribute assigned population to adjust denominator. Six out of seventeen pending recommendations reported to have been completed Eleven out of thirty-four pending recommendations reported to have been completed DEMONSTRATED COMMITMENT TO CONTINUED IMPROVEMENT OF IMMUNIZATIONS PROGRAM; • DIRESA/Amazonas has increased the number of nurse positions for the immunization program and has an annual operational immunization plan, as did Bagua and Utcubamba networks. These plans include a budget in PpR format. The new director of the DIRESA has agreed to report immunization coverage to the regional president. The regional president stated that immunizations were one of his three health priorities. • Apurimac has an annual operational immunization plan which includes a budget in PpR format. When the DIRESA could not install the cold room because the electrical supply was insufficient, a new transformer was purchased with Program for Support of Health Reforms (PARSALUD) funds. A new annual preventive maintenance plan, a communications plan, and actions to assure the rapid completion of the cold room demonstrated commitment to improving the immunization • Cusco has an annual operational immunization plan, as did both networks visited. These plans include a budget, usually in PpR format. • Puno budgeted 150,000 soles (over $55,000 USD) from the regional government in 2011 an • d a similar amount in the 2012 budget for the new DIRESA storage facility, where the cold room can finally be installed. Puno has an annual operational immunization plan, as did both networks visited. These plans include a budget, in PpR format. 103 USAID/PERU MCC-IPT Assessment Report COMPONENT/ ASPECT AMAZONAS APURIMAC CUSCO PUNO program. SUSTAINABILITY OF RESULTS ACHIEVED AT END OF THE MCC-ITP; AND • The MEF is supporting the Amazonas vaccination activities through PpR budgets as part of the priority, national strategy. The NIS is implemented through well-trained, experienced regional and network managers. • The VDIM-trained managers are sustaining results of the ITP. Funding is guaranteed by MEF support to the Apurimac vaccination activities through PpR. The SIS is working closely with RIS to continue IIIS operations. • VDIM-trained managers in key RIS positions. • PpR is used effectively by DIRESA and networks. • IEC Toolbox being used in five languages. • The VDIM-trained regional and network managers. • MEF support through PpR budgets as part of national nutrition strategy. EXTENT TO WHICH TOOLS PRODUCED BY THE MCC-ITP ARE BEING USED, AND WHY ARE OR AREN’T THEY BEING USED. • The IIIS rejected by the regional statistics office which has implemented NovHis. IEC toolboxes were not being used. The GIS was unknown • Data input to the IIIS is operating. No regional reports are being produced. The IEC toolbox being used by the communications office. No GIS. • IIIS is not operating due to lack of followup • AISPED trainers not in position anymore. • VDIM has not been offered again. • IIIS is not operating. • AISPED trainers not be in position anymore. • Some VDIM modules have been replicated by the RIS coordinator USAID/PERU MCC-IPT Assessment Report 104 ANNEX G OTHER RELEVANT INFORMATION 105 USAID/PERU MCC-IPT Assessment Report ANNEX G.1 MAPS OF SITES VISITED REGION AMAZONAS USAID/PERU MCC-IPT Assessment Report 106 REGION APURIMAC 107 USAID/PERU MCC-IPT Assessment Report REGION CUSCO USAID/PERU MCC-IPT Assessment Report 108 REGION PUNO 109 USAID/PERU MCC-IPT Assessment Report ANNEX G.2 STATUS OF COLD ROOM INSTALLATIONS USAID/PERU MCC-IPT Assessment Report 110 ANNEX G.3 REGIONAL PENDING ACTIONS AMAZONAS Increase demand for use of information and analysis in all sub-processes of vaccination by strengthening monitoring and evaluation systems. 1. Use the PpR to create clear incentives for analysis and use of information 2. Continue PpR budgeting to increase funding for operating expenses in the regions. 3. Establish clear guidelines for the introduction of new vaccines that include the assessment of financial sustainability not only for the purchase of vaccines, but for logistics at the regional level. 4. Evaluate the cost effectiveness of strengthening routine immunization vs. national campaigns. 5. Place the virtual training modules of the Immunization Management course on the DIRESA Amazonas website 6. Replicate the training program and materials for IBs at the regional level 7. The operational norm for IBs should consider training. 8. Place the training modules for IBs on the DIRESA Amazonas website. 9. Conduct replication and induction workshops for new immunization staff. 10. Ensure that 100% of children under five years have ID, which will identify and locate the 10–15% of children who are not vaccinated annually. 11. Expansion of GIS to all DIRESAs and other sectors (education). 12. Present the KAP study and socialize content. 13. Build regional capacity to adapt and reproduce IEC materials for the community from the program. 14. Implement IEC campaigns to strengthen immunization interventions at district and community levels. 111 USAID/PERU MCC-IPT Assessment Report ANNEX G.4 REPORTE DE LA ENCUESTA POR EMAIL SOBRE DIPLOMADO EN GESTIÓN DE INMUNIZACIONES Correos remitidos 1,884 Direcciones no validas 271 Correos con destinatario valido 1,613 Respuestas: 252 en total (Tasa de respuesta, 15.6%) 241 iniciaron el curso 2 que se negaron a responder las preguntas 9 refieren ser tutores o autoridades Completaron el Diploma 181 (72.4%), el 27.6% abandonaron el Diploma antes de completarlo, básicamente por dificultades de conectividad a Internet. De los que completaron, 113 recibieron su Diploma (62.4%), lo que implicaría que en nuestra muestra, 37.6% de los respondientes manifiestan no haber recibido su Diploma a pesar de haber completado el Curso. Even when this answer may not be representative of the total of course alumni, this would mean that there is a group, which is very uncomfortable about this. Alter one year of completion, almost 2/3, 67.1% (IC95 60.7 – 73.0%) is still working on Immunizations, and 97.4% (IC95 94.4 – 99.0%) consider the Diploma improved their competencies (regardless if they are or not currently in immunizations). 218 de los respondientes, incluyeron recomendaciones. De estas, las más frecuentes fueron: 50 (22.9%) Ampliar o repetir el Diploma para mas participantes. 41 (18.8%) Entregar los Diplomas a quienes concluyeron el curso. 34 (15.6%) Permitir a los que abandonaron, completar el Diploma. 25 (11.5%) Dictar actualizaciones para quienes ya hicieron el Diploma. 22 (10.1%) Recomendaciones pedagógicas (mayor cumplimiento por los tutores, etc) 12 (5.5%) Que se ofrezcan otros Diplomados virtuales. 7 (3.2%) Que se hagan reuniones presenciales periódicas. 7 (3.2%) Que se proyecte una Especialidad para los que ya hicieron el Diploma. 5 (2.6%) Profundizar o actualizar contenidos (Inmunobiologia, Data loggers, etc) 4 (2.1%) Otros Diplomados presenciales. 3 (1.4%) Modificar el documento para que tenga valor oficial. USAID/PERU MCC-IPT Assessment Report 112 ANNEX G.5 THRESHOLD PROGRAMS LESSONS LEARNED ANNEX G.6 MCC PERU INDICATORS Fact Sheet MCC Threshold Program Lessons Learned Introduction The Threshold Program, authorized under section 616 of the Millennium Challenge Act of 2003 (as amended), is intended to help candidate countries become eligible for an MCC Compact. Using the incentive of a potential MCC Compact, the Threshold Program is meant to encourage partner countries to design and undertake a chal￾lenging reform program. The first generation of individual country programs was designed to be short-term and provide countries with an opportunity to improve performance on MCC’s eligibility indicators through broad-based policy and institutional reforms. Design efforts for these early programs focused on rapid deployment of substantial technical and financial resources to generate ambitious results within a two-year window. Staff in the field worked aggressively to program substantial amounts of democracy and governance funding with few past models to which to refer for guidance. To assess whether the Threshold Program was achieving its policy and program objectives, MCC conducted a year-long review that analyzed the current portfolio, consulted a broad range of external stakeholders, and gath￾ered input from USAID, which typically works in partnership with MCC to implement the Threshold Program. MCC has also completed independent program evaluations of the threshold programs in Malawi and Zambia and an impact evaluation of the program in Burkina Faso. The findings from the review and these evaluations contributed to the development of a body of Threshold Program “lessons learned” that will be applied to future MCC threshold programs. MCC will continue consulta￾tions and will provide further detail on how these lessons will be applied. The Threshold Program has proved to be a useful tool for engaging non-eligible countries in constructive policy dialogues. Using a country threshold program to improve performance on MCC’s eligibility indicators within a narrow time frame, however, has not been effective in most cases. The Threshold Program has expanded and strengthened the U.S. Government’s dialogue with threshold country partners and has created an opportunity for MCC to support country-driven institutional reforms using the incentive of potential Compact eligibility. December 16, 2010 | www.mcc.gov MCC Threshold Program Lessons Learned, December 16, 2010 2 Past Threshold Programs have produced significant achievements, including a streamlined business registration process that helped lower the number of days to register a business from 39 to five in Albania; improved vaccina￾tion rates in Peru; and a newly formed anti-corruption unit that has tried and convicted three high-profile cases in Uganda. The program is also a valuable tool for providing information to MCC’s Board of Directors regarding a country’s commitment to reform and the prospects of partnership through an MCC Compact. By employing broad technical expertise from across the U.S. Government, MCC has been able to bring together the relative strengths of MCC, USAID, and other U.S. Government counterparts in a strong partnership for economic growth in its Threshold Program countries. MCC plans to continue to strengthen these partnerships in future threshold programs. Lessons Learned In response to the year-long review and the first set of evaluations, MCC has drawn the following key lessons that will be applied to the Threshold Program going forward: Lesson one: Link threshold programs to indicators and goals that are actionable and measurable within a relatively short period of time. While MCC eligibility indicators are useful for comparing peer countries’ performance on a range of policy mea￾sures for the purposes of selecting partner countries, they are generally ill-suited for the task of measuring the im￾pact of threshold programs. Because several of the eligibility indicators measure institutional performance broadly within an area of governance (for example, rule of law), and because programmatic interventions are necessarily focused in scope, these indicator scores reflect performance well beyond a threshold program’s interventions. As a result, it is often difficult to attribute changes in the eligibility indicators to program interventions or to measure progress in a timely manner. Future threshold programs will assist countries in becoming Compact-eligible by focusing on country-specific policy reforms linked to impediments to growth. Progress will be assessed against measurable metrics that partner governments can act upon within well-specified time frames. This will serve to strengthen the logic underpinning the Threshold Program and to more closely align it with MCC’s goal of creating policy environments conducive to reducing poverty through growth. Lesson two: Deepen diagnostic and feasibility analysis and identify the connection of activities to outputs, outcomes and impacts during program preparation. As a first step, where appropriate, MCC will carry out diagnostics of binding policy and institutional constraints to growth. These diagnostics will focus program design on reforms in key sectors. Feasibility of proposed activities will then be examined and will include an assessment of the related political economy. Finally, a clear program logic linking activities to outputs, outcomes and impacts will be articulated during the design phase. MCC Threshold Program Lessons Learned, December 16, 2010 3 MCC will devote increased resources and time to the preparation of threshold programs and will rely more heavily on in-house experts to develop programs. This strengthened due diligence process will allow MCC to more sys￾tematically analyze links between activities and expected impacts, and to calculate and weigh risks explicitly before approving a program. As a result, future country partners will benefit from good program design and realistic expectations of results. Lesson three: Be more selective when determining program interventions and establish a consultative pro￾cess to tailor focus areas. A more structured approach to program development will contribute to a more narrow focus on policy and institutional reforms relevant to growth. These reforms will be concentrated in areas in which governments are inclined to undertake reforms due to favorable political support and sufficient technical capacity. More narrowly￾defined programs will facilitate oversight and management. A broader consultative process will also serve as an additional measure of feasibility and help ensure that the reforms sought by the Threshold Program will have the necessary degree of country ownership and political support to be truly sustainable. Lesson four: Establish outcome-level goals that are appropriate for the timeframe. Threshold Programs often support policy and institutional reforms that require a gestation period before tangible results can be captured. In the first generation of programs, short timelines meant that external factors – such as vacancies in key host-country government agencies, normal political turnover, or lack of buy-in from new government leaders – could cause critical delays in implementation. In more recent threshold programs, MCC has integrated targets that are achievable in the two or three-year program time frame and reflect the partner country’s capacity and the anticipated gestation period. The process for setting higher-level targets, i.e., results emanating from trainings or the establishment of an anti-corruption unit, will continue to improve as due diligence efforts are expanded. Lesson five: Identify champions of reform at multiple levels, including the leaders of institutions targeted for reforms, and build in safeguards. When a threshold program targets a risky reform, political support must exist at multiple levels, ranging from the ministerial level to the leaders of a department or agency where the reform is specifically targeted. In addition to this support, attention must be focused on appropriate programmatic sequencing, particularly when enabling legislation is a precondition for the reform. When legislative action or the support of relevant leaders is uncertain, an interim assessment can help determine whether or not MCC assistance should continue. Recent programs, such as the Kyrgyz Republic’s threshold program, built in conditions so that program assistance would be released only when reform objectives had been met. MCC will continue to increase its focus on integrating such conditions in future threshold programs. MCC Threshold Program Lessons Learned, December 16, 2010 4 Lesson six: Build in sustainability assurances during program design and early implementation. Recent threshold programs are working closely with government counterparts to plan and allocate budgets for post-program sustainability plans. For example, an intervention focused on information technology requires that a partner government allocate funding and staff to operate the system and conduct troubleshooting during and after the threshold program. In the Liberia threshold program, funding for some renovation activities and equipment purchases is contingent upon the development of five-year maintenance and sustainability plan by each recipient community as part of the education component. Lesson seven: Increase oversight and monitoring and evaluation of projects to increase prospects for success and ensure learning during and after program implementation. The first generation of MCC threshold programs focused on anti-corruption and rule of law reforms – an area that suffers from a dearth of evidence on best practices, the lack of which amplifies the need for strong monitoring and impact evaluation. However, for the first generation of country threshold programs, baselines for later comparison and attribution of program results were generally not established. More recently, efforts have been made to better integrate stronger monitoring and evaluation (M&E) practices into Threshold Programs, including the design and implementation of impact evaluations. For instance, the Rwanda threshold program randomized the civic engage￾ment and local government capacity-building activities to improve conditions for a rigorous impact evaluation. MCC’s Threshold Program has recently made strides forward by integrating independent program monitoring in new threshold programs, as well as publishing three independent evaluations; two program evaluations are also underway. MCC will continue to strengthen its monitoring and evaluation and will use the growing body of evidence-based data to continue learning and improving future threshold programs. 2010-002-0480-02 Peru FY07 GNI/Cap: $2,610 LMIC Population: 27,562,390 Investing In People Ruling Justly Economic Freedom Sources: Freedom House Sources: World Health Org. Sources: World Bank Inst. Freedom House World Bank Institute World Bank Institute World Bank Institute World Bank Institute World Health Org. UNESCO/National Sources UNESCO/World Bank Intl. Finance Corp. Intl. Finance Corp. Heritage Foundation Intl. Monetary Fund National Sources 0 10 20 30 40 '02 '03 '04 '05 Median 25 Political Rights Data 32 (68%) 0 20 40 60 '02 '03 '04 '05 Median 40 Civil Liberties 42 (55%) Median 0.00 Control of Corruption -0.08 (39%) -2.0 - 1. 0 0.0 1. 0 2.0 '02 '03 '04 '05 Median 0.00 Government Effectiveness -0.44 (23%) -2.0 - 1.0 0.0 1.0 2.0 '02 '03 '04 '05 Median 0.00 Rule of Law -0.39 (16%) -2.0 - 1. 0 0.0 1. 0 2.0 '02 '03 '04 '05 Median 0.00 Voice and Accountability 0.01 (52%) -2.0 - 1.0 0.0 1.0 2.0 '02 '03 '04 '05 Median 0.00 Regulatory Quality 0.36 (74%) -2.0 - 1.0 0.0 1.0 2.0 '02 '03 '04 '05 Data -8.00 -5.50 -3.00 -0.50 2.00 4.50 '01 '03 '05 Median -1.37% Fiscal Policy -1.10% (61%) 20 40 60 80 10 0 '01 '02 '03 '04 '05 Median 92.5 Immunization Rates Data 82 (26%) 0 50 10 0 150 200 '02 '04 '06 Median 19.8 Cost Of Starting A Business 32.5 (28%) 0 50 10 0 150 200 '02 '04 '06 Median 33 Days To Start A Business 72 (13%) 20 40 60 80 10 0 '02 '04 '06 Median 62.1 Trade Policy 62.2 (52%) 0 10 20 30 '01 '03 '05 Max. 15 Inflation 1.60 (90%) 10 40 70 10 0 13 0 '00 '01 '02 '03 '04 Median 97.3 Girls' Primary Education Completion Rate 98.9 (75%) 0 3 6 9 12 15 '00 '01 '02 '03 '04 Median 3.66 Health Expenditures 2.29 (16%) 0 2 4 6 8 10 '02 '03 '04 '05 '06 Median 1.88% Primary Education Expenditures 0.99 (17%) 10/6/2006 How to Read this Scorecard: Each MCC Candidate Country receives a scorecard annually assessing performance in 3 policy categories: Ruling Justly, Investing in People, and Economic Freedom. For each indicator box (from top to bottom): the name of the indicator or policy, the country’s data, or score, indicated by a green or red line representing a passing or failing score. Next to the score, is the country’s percentile ranking in its respective Low Income or Lower Middle Income group (0% is worst; 50% is the median; 100% is best). Under the score/ranking, is the median score for the respective income group, above which countries have to score in order to pass the indicator. The white box represents a trend line of performance with each red dot assigned to a score on the vertical axis and the year on the horizontal access. The black line running through the dots represents the current year’s median. Data sources are below the box. For more information regarding the Millennium Challenge Account Selection Process and these indicators, please consult MCC’s website: www.mcc.gov