ASSESSMENT OF THE USAID/ARMENIA PRIMARY HEALTHCARE REFORM PROJECT November 2007 This publication was produced for the United States Agency for International Development, Armenia. It was prepared by Robert Taylor, Capri-Mara Fillmore, and Rosario Scarato, through the Global Health Technical Assistance Project, with contributions from Kimberly Waller, USAID/Armenia, and Mark Austin, USAID/Washington. ASSESSMENT OF THE USAID/ARMENIA PRIMARY HEALTHCARE REFORM PROJECT DISCLAIMER The authors’ views expressed in this publication do not necessarily reflect the views of the United States Agency for International Development or the United States Government. This document (Report No. 07-001-41) is not publicly available in printed or online versions. Additional information can be obtained from The Global Health Technical Assistance Project 1250 Eye St., NW, Suite 1100 Washington, DC 20005 Tel: (202) 521-1900 Fax: (202) 521-1901 info@ghtechproject.com This document was submitted by The QED Group, LLC, with CAMRIS International and Social & Scientific Systems, Inc., to the United States Agency for International Development under USAID Contract No. GHS-I-00-05-00005-00. i CONTENTS ACRONYMS, ABBREVIATIONS, AND FOREIGN TERMS ..................................................... ii EXECUTIVE SUMMARY ................................................................................................................... 1 I. BACKGROUND........................................................................................................................ 5 II. ASSESSMENT FINDINGS ........................................................................................................ 7 PROJECT ORGANIZATION, MANAGEMENT, AND CROSS-CUTTING ISSUES ...... 7 COMPONENT 1: EXPANSION OF PRIMARY HEALTHCARE REFORMS ..................10 COMPONENT 2: FAMILY MEDICINE ...................................................................................16 COMPONENT 3: OPEN ENROLLMENT .............................................................................27 COMPONENT 4: QUALITY OF CARE .................................................................................30 COMPONENT 5: HEALTHCARE FINANCING .................................................................33 COMPONENT 6: PUBLIC EDUCATION, HEALTH PROMOTION, AND DISEASE PREVENTION ...............................................................................................................................40 MONITORING AND EVALUATION ....................................................................................43 III. RECOMMENDATIONS .........................................................................................................45 FOR USAID ...................................................................................................................................45 For PHCR .......................................................................................................................................45 ANNEXES ..........................................................................................................................................51 ANNEX 1: STATEMENT OF WORK ....................................................................................51 ANNEX 2: WORK SCHEDULE ...............................................................................................57 ANNEX 3: TECHNICAL SUGGESTIONS ON THE SCIENCE AND PRACTICE OF FAMILY MEDICINE .....................................................................................................................59 ANNEX 4: PROVIDER SURVEY ..............................................................................................63 ANNEX 5: PEOPLE MET ...........................................................................................................67 ANNEX 6: REFERENCE MATERIALS .....................................................................................73 ii ACRONYMS, ABBREVIATIONS, AND FOREIGN TERMS AMD Armenian Dram ASTP Armenia Social Transition Program BBP Basic Benefits Package BMC Yerevan Base Medical College CHC Community health committee CME Continuing medical education CN Community nurse DHS Armenia Demographic and Health Survey 2000 EMG Emerging Markets Group, Ltd. (Prime Contractor, PHCR Project) FAP Feldsher Acousher Posts (Rural Health Post) FM Family medicine FN Family nurse FP/RH Family planning and reproductive health GH Tech Global Health Technical Assistance Project GOA Government of Armenia HPIU World Bank Health Project Implementation Unit HSSD Health and Social Security Department (marz-level health authority) JMF Jinishian Memorial Foundation, NGO Marz Regional governmental division MOE Ministry of Education MOH Ministry of Health NGO Non-governmental organization NHA National health accounts NIH National Institute of Health NOVA Project NOVA, NGO OE Open enrollment PE Public education and disease prevention PHC Primary healthcare PHCR Primary Healthcare Reform Project QOC Quality of care SHA State Health Agency SOP Standard of practice SOW Scope of work TA Technical assistance ToT Training of trainers UFMC Unified family medicine curriculum URI Upper respiratory infection USAID United States Agency for International Development WB The World Bank WHO World Health Organization Assessment of the USAID/Armenia Primary Healthcare Reform Project 1 EXECUTIVE SUMMARY In 2005, USAID/Armenia entered into a contract with the Emerging Markets Group, Ltd., launching the Primary Healthcare Reform Project (PHCR). PHCR’s strategic objective is to support health sector reforms designed to increase utilization of sustainable, high-quality, primary healthcare services—contributing to the improved health of all Armenian families and to a more productive workforce.1 On September 30, 2007, PHCR concluded the first two base years under its contract. In October 2007, at the request of USAID/Armenia, the Global Health Technical Assistance Project (GH Tech) fielded an assessment team to analyze the general performance of PHCR—including successes, constraints, and failures—and to make managerial and programmatic recommendations to USAID/Armenia on exercising its options for the contract’s three remaining years.2 With six major components and numerous expected outcomes, PHCR is ambitious and complex and poses significant organizational and managerial challenges. To address these challenges, the Emerging Markets Group has assembled a strong management team and an able and industrious staff. In addition, PHCR has overcome a year-long vacancy of its chief of party position to earn a reputation among its counterparts and peers as a respected, cooperative, flexible, and productive partner in implementing Armenia’s health reform agenda. To its credit, PHCR has made good progress in each of the component areas and is likely to achieve most, if not all, of its goals by project’s end. Major Accomplishments Thus far, PHCR has many accomplishments. It has renovated and equipped 39 health posts, with an additional 36 nearing completion, and others about to begin. The policy framework underpinning the health reform agenda has been further strengthened with additional legislation and regulations in development. Open enrollment is ahead of schedule and exceeding expectations. Increasing numbers of family physicians and community nurses have been trained and are now working in polyclinics, ambulatories, and health posts. Efforts to strengthen the quality of care are in development. In addition, 57 communities have received assistance to help mobilize community health committees and increase local understanding and support for health promotion and disease prevention. Challenges and Constraints While PHCR’s achievements are significant, a number of challenges remain. Most critically, and cutting across all components, is the need to further strengthen the in-country capacity to maintain and further develop the health reform agenda and to continue the numerous initiatives so ably launched during these last few years. Because of local inflation and devaluation of the dram, the project’s budget will not stretch as far as originally projected and some programmatic adjustments will be required. Of particular note, the project will need to reduce the number of health posts targeted for renovation. For continuity of leadership, USAID should reconsider the contractual provision calling for a turnover of PHCR leadership to local professionals in year four. 1 USAID/Armenia, Contract with Emerging Markets Group, Primary Healthcare Reform Project, Contracting Office, USAID/Armenia, September 30, 2005, . 2 USAID/Armenia, Statement of Work: Assessment of the Primary Healthcare Reform Project, USAID/Armenia, August 2007. Assessment of the USAID/Armenia Primary Healthcare Reform Project 2 The project’s comprehensive strategy to strengthen rural health posts—including physical upgrading, equipment, staff training, and community organization—should continue to be pursued. However, the long-term sustainability of health posts is still uncertain and will require continuing support and maintenance of all that is being done by PHCR, plus high-level political commitment supporting health posts as a principal mode of delivering primary health services in rural areas. The project also should select additional rural health posts based on their potential. Because of shortcomings in their training or the constraints of working in rural areas, family doctors and nurses do not always perform at hoped-for levels. The professional identities of family doctors and nurses and their roles in relation to other health practitioners are still unsettled. However, the performance-based remuneration system was developed and piloted by the Armenia Social Transition Program (ASTP) in 2004–5. In addition, PHCR has assessed the pilot results and made recommendations for a performance-based system for nationwide implementation. This has been presented to the new Minister of Health, and PHCR is waiting for approval to move forward. Key Recommendations Contract extension. Based on the project’s current exceptional performance, the assessment team recommends that USAID/Armenia exercise further option years according to Mission procedures for option year authorization, in so much as PHCR has continued high performance and effective collaboration with the Government of Armenia; and in accordance with available Mission funding levels, Mission priorities, and the new Mission strategy. Transfer of expertise. Cutting across all components is the need to strengthen the in-country capacity to maintain and further develop the health reform agenda and to continue the numerous initiatives so ably launched during these last few years. While there are many possible solutions to this transfer of knowledge and skills, one possible solution is to embed PHCR staff within the MOH to increase direct involvement and knowledge transfer. Another possibility is to create an NGO that provides technical expertise to the GOA. These and other options should be explored further. Fewer health posts. If necessary to stay within budget, PHCR should maintain its standards but reduce the targeted number of health posts and ambulatories to be renovated. Retaining the target, but reducing the amount or quality of the work done in each facility is not advisable. Health post sustainability. In light of the vulnerability of health posts, PHCR’s multi-intervention strategy—renovation, equipment, staff training, financing, and community organization—offers the best hope for ensuring the sustainability of health posts. In addition, PHCR should help to develop the capacity within the MOH to support and maintain these initiatives on an ongoing basis. Further strengthen family medicine and nursing curricula and training. PHCR should continue to use its cardiovascular training package methodology to reach national consensus on the teaching of other modules in the FM curriculum. This is a key element and perhaps PHCR’s most influential capacity-building role to improve medical education and increase skills for Armenia’s health providers. PHCR should also increase the hands-on experience for those using these packages, for example, by requiring that 50 blood pressures be taken with preceptor checking. Consider the realities of the rural physician and nurse in developing curricula tools. While attempting to provide state-of-the-art medical care, the curriculum, guidelines, and modules often ignore the current realities of polyclinic, rural ambulatory FM, and FAP work. It makes sense to Assessment of the USAID/Armenia Primary Healthcare Reform Project 3 acknowledge the gold-standard practices, but an alternative method needs to be indicated in the teaching tools when the gold-standard practice cannot be done or is not available. Role of the community nurse. The community nurse curriculum needs updating to be more suited to the actual capabilities and needs of the FAP nurse. A high priority is to standardize (in reality, not just on paper) the medications available at the FAP and be sure that each FAP nurse knows exactly when and how to use them. Community nurses should be considered a specialty with far greater responsibilities than those of family nurses and similar to public health nurses in the U.S. and elsewhere. Therefore, it should be expected that training and retraining would take longer. In the long term, PHCR should assist the MOH in declaring community nursing as a specialty, expanding the curriculum, and institutionalizing community nursing training within a nursing college. Such a nurse will have a role in future Armenian healthcare, even if the actual FAP post does not. Strengthened institutional capacity to manage open enrollment. Critical to sustaining the launching, continuation, and monitoring of OE is the ownership of the respective GOA agency and its ability to continue as well as to evolve and improve the PHC OE activities necessary for continuous reform. While GOA ownership and capacity improvement is currently incorporated in most PHCR activities, these activities should be reviewed and systematically addressed in the coming three years of PHCR. Each PHCR action should be planned and implemented jointly by the staff of PHCR and the respective GOA agency at the national, marz, and relevant sub-marz levels, with participation from providers—including public and private doctors, nurses and administrators—and the local community. An option that may be considered is to imbed PHCR OE activities within an agency of the GOA such as the MOH/SHA and the marzpetrans with the goal of increasing the respective agency’s ownership of PHCR activities and achievements, and institutionalize its capacity to continue these functions. This could be accomplished in a staged manner. A decree might be needed for a special office, along with a staged GOA budget for space, logistical support, and added staff. USAID support, while potentially diminished after three years, could possibility continue to support of a reduced level of technical staff as the technical capacity of the GOA agency increases. Strengthened institutional capacity to manage healthcare financing. USAID intermediate result 3.2.1 states, ―Critical to sustaining the financial system development and its analysis is the ownership of the respective GOA agency and its ability to continue as well as to evolve or better the PHC healthcare financing aspects of reform.‖ How GOA institutionalization, ownership, and capacity are achieved should be systematically addressed, with priorities in all activities and specific implementation plans incorporated in the coming three years of PHCR. Such planning actions should be undertaken jointly by the staff of PHCR and the respective GOA agencies at all levels, along with respective stakeholders in both the public and private sectors. An option that may be considered is to imbed PHCR financial unit within an agency of the GOA, such as the SHA, with the goal of increasing the agency’s ownership of the project activities and achievements and institutionalize its capacity to continue these functions. This could be accomplished in a staged manner. A decree may be needed for a special office, along with a staged GOA budget for space and logistical support and added staff. USAID support, while diminished after three years, could continue its support, with the number of technical staff reduced as appropriate with the increase in capacity of the GOA agency, and possibly a mechanism in place to provide short-term consultants on an as-needed basis. Assessment of the USAID/Armenia Primary Healthcare Reform Project 4 Assessment of the USAID/Armenia Primary Healthcare Reform Project 5 I. BACKGROUND Armenia’s health system is in transition, reflecting changes in society, economy, and government that began in 1991 when Armenia declared its independence from the Soviet Union. As with other Soviet countries, Armenia’s health system was centrally financed and managed, with the Ministry of Health (MOH) overseeing an extensive system of hospitals and clinics, including a system of well￾staffed health stations that served even the most remote communities. After independence, faced with severe economic difficulty and a health system that was outdated and overstaffed, the Armenian government was no longer able to guarantee free healthcare for all. As public moneys for health diminished, the emphasis shifted toward specialized medical services provided in hospitals and polyclinics, and the burden of financing healthcare began to shift to private citizens. One result of the transition was that the newly independent Armenia experienced little improvement in health status and several new problems began to emerge. In the 1980s, in the waning years of the Soviet Union, Armenia was reported to have the longest living people of all the Soviet republics. In the years immediately following independence, however, life expectancy in Armenia dropped slightly, although it has since recovered. Of particular concern, maternal and infant death rates remain high, due primarily to diminished access to care, the poor quality of health services, and weaning infants too early. Cardiovascular disease has now become the leading cause of death among those over 65 years of age. There has also been a disturbing increase in the incidence of antibiotic￾resistant tuberculosis. Smoking rates for Armenian men remain among the highest in the region and cancer is on the rise. These changes in health status are rooted in the way healthcare in Armenia is organized, delivered, and financed and, in 1996, this stimulated the Armenian government to launch a major program of health reform that included three major thrusts: improving access to primary healthcare services, improving health financing, and optimizing health facilities and personnel. The government’s reform agenda is the foundation upon which USAID’s health strategy in Armenia has been developed. In 2000, USAID launched its five-year ASTP to assist the MOH in developing, testing, and implementing a series of social protection initiatives, including primary healthcare reform. In the intervening years, ASTP-sponsored programs have helped establish a supportive regulatory framework for primary healthcare, departments of family medicine (FM) in medical schools, a unified curriculum for training doctors and nurses in FM, and piloted models of FM practice and open enrollment that have been adopted nationally. In 2005, as a follow-on to ASTP and to expand its accomplishments more broadly, USAID/Armenia entered into a contract with the Emerging Markets Group, Ltd., launching the Primary Healthcare Reform Project. PHCR’s strategic objective is to support health sector reforms designed to increase utilization of sustainable, high-quality, primary healthcare services— contributing to the improved health of all Armenian families and to a more productive workforce.3 To accomplish this objective, PHCR activities have been organized into six component areas: Component 1: Expansion of Primary Healthcare Reforms Component 2: Family Medicine Component 3: Open Enrollment 3 USAID/Armenia, Contract with Emerging Markets Group, Primary Healthcare Reform Project, Contracting Office, USAID/Armenia, September 30, 2005. Assessment of the USAID/Armenia Primary Healthcare Reform Project 6 Component 4: Quality of Care Component 5: Healthcare Financing Component 6: Public Education, Health Promotion, and Disease Prevention To manage PHCR and its complex array of components, Emerging Markets Group assembled a team of firms and personnel with a broad array of expertise and technical competence. There have been some changes since inception and the firms now include Emerging Markets Group, IntraHealth International, Overseas Strategic Consulting, American University of Armenia’s Centre for Health Services Research, and the Boston University School of Public Health. Social Sectors Development Strategies, an original member of the consortium, recently withdrew. At PHCR launch, Emerging Markets Group also mobilized a management team of international and national experts including a Yerevan-based chief of party, a deputy chief of party, and a technical staff of component team leaders. A local administrative staff was also assembled. A dedicated Washington-based project manager and administrative staff provide project quality assurance and administrative management and contract, accounting, and logistics support. In addition, transfer of project management to local professionals is planned for year four. The effective date of the contract was September 30, 2005, and PHCR management team was mobilized later that year. On September 30, 2007, the project concluded the first two base years under its contract. In October, 2007, at the request of USAID/Armenia, GH Tech fielded an assessment team to analyze the general performance of PHCR—including successes, constraints, and failures—and to make managerial and programmatic recommendations to USAID/Armenia on exercising its options for the contract’s three remaining years.4 4 USAID/Armenia, Statement of Work: Assessment of the Primary Healthcare Reform Project, USAID/Armenia, August 2007. Assessment of the USAID/Armenia Primary Healthcare Reform Project 7 II. ASSESSMENT FINDINGS The project has performed exceptionally well across all components, and USAID/Armenia should exercise its option to continue it for the three remaining years of its contract. While PHCR’s achievements are significant, a number of challenges remain. Most critically, and cutting across all components, is the need to further strengthen the in-country capacity to maintain and further develop the health reform agenda and to build on the momentum generated over these last few years. PROJECT ORGANIZATION, MANAGEMENT, AND CROSS-CUTTING ISSUES PHCR has overcome a year-long vacancy of its chief of party position to earn a reputation among its counterparts and peers as a respected, cooperative, flexible, and productive partner in implementing Armenia’s health reform agenda. With an able and industrious staff, the project has made good progress in each of the component areas and is likely to achieve most, if not all, of its goals by project end. Achievements Overcoming gap in leadership. In spite of an early gap in leadership, the project was able to move forward in each component area with a minimal loss of momentum. After contract signing, Emerging Markets Group promptly mobilized the PHCR management team. Nearly all key staff were in place by the end of 2005, working in each of the component areas by early 2006. Unfortunately, after several months on the job, the chief of party left the project. After that departure, the deputy chief of party, John Vartanian, was elevated to the position of acting chief of party and has ably served in that capacity for the past year. Just recently, in mid-October 2007, the new chief of party, Richard Yoder, joined PHCR team. Achieving strong collaboration. Of particular note is the collaborative relationship PHCR established with the World Bank. Early on, agreements were reached regarding which agency would take the lead in each of several program areas: open enrollment, primary healthcare (PHC) financing, independent group practice, and upgrading health posts, polyclinics, and ambulatories. The two agencies—especially staff of PHCR and the World Bank Health Project Implementation Unit— continue to coordinate their activities. Challenges and Constraints Inflation and budget constraints. Inflationary pressures, both domestic and international, threaten the project’s ability to reach all its programmatic targets. Local salaries and construction costs have been especially hard hit. Since project inception, the Armenian dram (AMD) has devalued by 30 percent against the U.S. dollar, and the cost of building materials in the domestic market has increased by more than 20 percent. Transfer of PHCR expertise to the MOH. In its first two years, PHCR has earned a reputation among its counterparts and peers as a respected, cooperative, flexible, and productive partner in implementing Armenia’s health reform agenda. In interviews conducted by the assessment team, it was evident that the project has worked hard to establish good working relationships with stakeholders at every level—USAID, other donors, subcontractors, the MOH, other governmental Assessment of the USAID/Armenia Primary Healthcare Reform Project 8 agencies, local health authorities, and local communities and health facilities. Invariably, those interviewed were informed on issues, enthusiastic, supportive of project goals, and appreciative of the respectful and intelligent relationship they had with PHCR and its staff. While the MOH is closely involved in the development of many tasks within PHCR’s scope, many critical tasks (as in the financing sector) are being carried out by PHCR on behalf of the MOH (for example, estimated age- and sex-specific capitation rates for the new payment system). This raises the question about the extent to which many of these tasks and their associated skills are effectively transferred to the MOH. Since many of these calculations will require annual adjustments based on new data, MOH staff will need to increase its capacity to conduct these activities themselves in the years to come. Adjustments and Modifications Modifying targets to reflect budget. Because of inflationary pressures, PHCR will need to adjust its programmatic targets, most notably the goal to upgrade 230 health posts. This issue is discussed more fully under Component 1. Democracy and health—potential new opportunities. Throughout this assessment, there were a number of activities identified that demonstrated strong democracy leadership at the grass-roots level. These cross-sector initiatives are an important source of programmatic growth and expansion, given FY08 and FY09 potential flat-lined budgets. Highlighting healthcare reform activities that have strengthened democracy and civil society in rural areas are excellent advocacy tools for the USAID health team to potentially access additional support or opportunities in the democracy and governance sector under USAID. Documenting the democratization of the healthcare system opens the door for project integration with democracy and governance initiatives funded by USAID. Major healthcare reforms currently supported by the project, such as open enrollment and performance-based financing, embrace democracy through the healthcare system. With open enrollment, the population is actively choosing its healthcare providers by registering and participating in the open enrollment process. Then increased utilization rates for specific providers paired with PHC monitoring (tracking key PHC indicators) represents the patient’s voice in the healthcare system. Implementation of key healthcare reforms, utilization, and quality will direct healthcare budgets and replace traditional Soviet budgeting systems based on population numbers or catchment areas. Open enrollment registration is already at 60% of the total population, demonstrating the strong individual will of the population to participate in the management of their own health. This democratic voice in the health sector needs to be captured as a cross-cutting success supported by AID. In addition, the public education component of the project has motivated CHCs and their respective non-governmental organizations (NGOs) to advocate for supporting neglected healthcare facilities and facilitating access to critical health information to rural communities. Not only is knowledge of health risk factors and prevention methods important, but the motivation to work with these risk factors at the community level is essential to successfully improving rural health. The NGOs and CHCs have demonstrated ability to advocate for the community with very little donor intervention, and this is a strong indicator of essential civil society growing out of healthcare reform efforts. Millennium Challenge Corporation (MCC) linkages. Potential synergies should be explored between MCC efforts and the healthcare reform efforts of PHCR and Project NOVA. Although Assessment of the USAID/Armenia Primary Healthcare Reform Project 9 MCC’s primary goal is focused on agriculture through improvement of irrigation and road systems, it is firmly rooted in working in the same remote rural areas as PHCR and NOVA projects. Because MCC will be improving one-third of the roads and water infrastructure in these rural areas, health projects may be able to tap into the improved roads or water systems to improve access and sustainability of FAPs and ambulatories in rural Armenia. This may be an option to explore with the development and renovation of rural health systems in Zone 2, and it can definitely be factored into Zone 3 implementation. Anti-corruption efforts intrinsic to healthcare reform and National Health Accounts. Central to PHCR healthcare reform efforts is diminishing corruption in the healthcare system in Armenia. This should be noted in USAID’s new anti-corruption initiatives, and the potential to complement current PHCR efforts could be explored as an alternative to fostering an independent USAID￾funded anti-corruption effort in health. Recommendations In moving forward, several management issues need to be addressed: extension of the contract, transfer of expertise, local PHCR leadership, and the visibility of FP/RH programming. Contract extension. Based on the project’s current exceptional performance, the assessment team recommends that USAID/Armenia exercise further option years according to Mission procedures for option year authorization in so much as PHCR has continued high performance and effective collaboration with the Government of Armenia; and in accordance with available Mission funding levels, Mission priorities and the new Mission strategy. Local project leadership. The contract provides for ―A planned transfer of project management to local professionals in the fourth year of the project in order to enhance Armenian management and reduce the cost of implementation.‖ The assessment team supports the intent of this provision but recommends that it not be implemented. There is much to do in the next three years and a change in leadership would be unnecessarily disruptive. This issue and its budgetary implications should be addressed and resolved by USAID and PHCR at the earliest opportunity. Transfer of expertise. While PHCR’s achievements are significant, a number of challenges remain. Most critically, cutting across all components, is the need to further strengthen the in-country capacity to maintain and further develop the health reform agenda and to continue the numerous initiatives so ably launched during these last few years. While there are many possible solutions to this transfer of knowledge and skills, one possible solution is to embed PHCR staff within the MOH to increase more direct involvement and knowledge transfer. Another possibility is to create an NGO that provides technical expertise to the GOA. These and other options should be explored further. As discussed below under Component 1, an additional contract consideration—the number of health posts to be targeted—also needs to be resolved. Assessment of the USAID/Armenia Primary Healthcare Reform Project 10 COMPONENT 1: EXPANSION OF PRIMARY HEALTHCARE REFORMS The major aim of Component 1 is to roll out primary healthcare reforms in systems, protocols, and procedures developed under ASTP, extending them nationwide to the most remote of rural areas. As originally defined in the contract, expected outcomes included: The renovation and equipping of 100 FAPs and approximately 130 ambulatories and polyclinics Increased service utilization, clinical care, and client satisfaction at targeted facilities A supportive policy and regulatory environment for implementation of reforms; Effective processing for national workforce planning and an incentive structure in place to attract professionals and providers to rural areas Improved budget, planning, policy, and payment systems at the national level Improved management, supervision, and financial oversight mechanisms at the facility level. As noted below, some of the targeted outcomes were modified after inception. Achievements In Component 1, the project has made good progress toward each targeted outcome, although important challenges still need to be addressed. Of particular note are PHCR’s efforts to: Renovate and equip health posts Measure impacts on health post utilization Further strengthen the policy framework supporting PHC reforms Strengthen the management of ambulatories and polyclinics Renovating and equipping health posts. Shortly after inception, PHCR’s goal to renovate and equip 100 FAPs and approximately 130 ambulatories and polyclinics was modified. In negotiation with the World Bank at USAID’s direction, it was agreed that PHCR should focus on the renovation and equipping of 230 health posts and ambulatories, while the World Bank’s Health Project Implementation Unit (HPIU) was to target ambulatories and polyclinics. Subsequently, PHCR began work with the MOH and regional health departments to develop a plan to select, renovate, and equip health posts in three geographic zones, to be phased in over five years. Through September 2007, PHCR has completed, or nearly completed, 75 PHC facilities. Earlier in the year, it completed renovating and equipping 39 health posts serving a combined population of 55,000 people in Zone 1 (Lori and Shirak marzes). Now, work is nearing completion on an additional 36 PHC facilities serving over 35,000 people in Zone 2 (Tavush, Gegharkunik, and Kotayk marzes). These newest health posts will be furnished and equipped shortly, then turned over to local community authorities. Preparations are also underway to begin expansion of the program in Zone 3. Increased service utilization. Early studies by Project NOVA suggest that provider performance and utilization do improve because of training, equipment provision and community education and mobilization activities. In a study in four marzes, nurses who received Safe Motherhood Clinical Skills Training improved their antenatal performance by an average of 26 percent and their Assessment of the USAID/Armenia Primary Healthcare Reform Project 11 postpartum performance by an average of 22 percent.5 NOVA’s baseline and end-line data previously collected in Shirak and Tavush marzes also suggest significant increase in the utilization of health services at health posts: for example, utilization of health posts for antenatal care increased six fold in communities receiving training of nurses, equipment and supplies, as well as supported by community mobilization and education activities6 . Still, these percentage increases reflect rather small increases in the absolute number of visits. Substantial increases in volumes have yet to be demonstrated. Anecdotal evidence gathered by the assessment team, through visits to an array of representative health posts, also suggests utilization has increased. However, it is too early to accurately measure increases in the utilization of health posts upgraded under PHCR or to determine which interventions—renovations, equipment, staff, community organization, or a combination thereof— most influence utilization. Nor is it possible to determine if the interventions and resulting changes in utilization are sustainable. Over these next three years, studies by PHCR’s monitoring unit designed to measure such changes in utilization and their influencing factors will undoubtedly provide a useful basis for evaluating PHCR interventions and for developing guidance for future efforts. Supportive policy framework. The project has been successful in expanding the policy framework established under ASTP. While FM obtained formal recognition in Armenia as early as 1997, it was not until 2003, with passage of the National Strategy on Primary Healthcare,7 that the foundation was laid for the development of primary healthcare, and FM was adopted as the preferred method of health service delivery. Decree 1533 set out the PHC Strategy (2003–2008) and approved the Pilot Project (2003–2005) for developing new methods for PHC administration and financing— legislation that underpinned USAID’s ASTP program and set the stage for PHCR. PHCR was quick to establish a close working relationship with the MOH’s legal department, which views PHCR as a valued partner in developing and evaluating policy proposals. Since Decree 1533, with PHCR’s aid, a number of policy initiatives have been introduced that reinforce or elaborate PHC reforms, most notably: Decree No. 497-N of April 2007, which legitimizes FM independent and group practice Decree No. 420-N (April 2006), which sets out rules for selecting and registering with PHC providers MOH Order 365 (2007) on open enrollment In addition, the project has been instrumental in assisting the legal department in developing a package of amendments that update and correct earlier legislation. The amendments are now in the process of formal approval by the government of Armenia (GOA) and are expected to receive legislative approval shortly. PHCR is also working with the MOH and other donors to develop other policies that would set work standards for family doctors; update and clarify the roles of family doctors and family nurses; and upgrade the formal status of nurses, including community health nurses. 5 Project NOVA, Project NOVA Overview, Power Point Presentation, October 2007. 7 RA Government Decree No. 1533, December 6, 2003. 7 RA Government Decree No. 1533, December 6, 2003. Assessment of the USAID/Armenia Primary Healthcare Reform Project 12 Improved facilities management. In November–December of 2006 and in July 2007, PHCR delivered six five-day training courses on PHC Management to 140 PHC facility managers from Yerevan, Lori, Shirak, Kotayk, Tavush, and Gegharkunik marzes. The curriculum included the following major topics: Financial management, which included financial planning and budgeting, costing of PHC services, financial analysis, tax accounting, reporting, and improving procurement procedures Healthcare system policy, reforms, and supportive legal and legislative framework, including quality of care management Human resource management, which included planning and organizing, leading and controlling, management styles, creating and maintaining teamwork, problem solving and handling conflicts, time management, data and information flows in organizations, and applying knowledge management framework in PHC facilities Strategic planning, which included stages of strategic plan development, SWOT analysis, problem tree analysis, resource analysis, first year action plan, and monitoring and evaluation As an output of training, participants from 92 polyclinics, ambulatories, and rural health centers in Yerevan, Lori, Shirak, Tavush, Gegharkunik, and Kotayk marzes developed and submitted three￾year development plans for their PHC facilities to their Health and Social Security Departments. During its visit, the assessment team interviewed managers and accountants from several representative PHC facilities, plus the heads of three marzes (Kotayk, Shirak, and Yerevan). All confirmed the value of the training they had received. Challenges and Constraints Inflation and budget constraints. As noted earlier, inflationary pressures, both domestic and international, threaten PHCR’s ability to reach all of its programmatic targets, especially the goal to upgrade 230 health posts. Based on experience thus far, it costs an average of $3,900 (excluding pre￾renovation design and engineering and other overhead costs) to upgrade one facility. If that unit cost held through project end, completing all 230 facilities would require approximately $900,000, about $100,000 under the fixed allocation of $1 million specified in the contract budget. To stay within the budget, the cost for the remaining 151 health posts cannot exceed an average of $4,582, about 17 percent higher than the average cost to date. However, since project inception, the Armenian dram (AMD) has devalued by 30 percent against the U.S. dollar, and the cost of building materials in the domestic market has increased by more than 20 percent. Health post sustainability. While it is clear that PHCR has done its work well in upgrading health posts, the future role of health posts and their long-term sustainability are uncertain. On the positive side, MOH officials at the federal level made it very clear to the team that PHCR’s focus on FAPs and their staff addresses a critical piece of the overall improvement of the Armenia healthcare system. However, no concrete plans or funding from the GOA to support or further maintain the health posts are in place to date. This indicates that there is no major commitment from the GOA thus far to improve accessibility and sustainability of rural health posts, and further USAID investments should be reviewed if sustainability of support is still in question. Conceptually, upgrading health posts is seen as a way to increase the accessibility and utilization of primary health services in rural areas. Renovated facilities also provide local communities with Assessment of the USAID/Armenia Primary Healthcare Reform Project 13 tangible evidence of health-related initiatives and are seen as the first step in mobilizing local participation and support. Based on visits to several representative health posts, the renovations work has been simple but adequate—new roofs where necessary, wall repairs and painting, new floors, new lighting, and new window treatments. Within the limits of local constraints, water services were also improved. Furnishings were simple and equipment minimal. The standards used seemed comparable to those used in the renovation of World Bank-sponsored projects. Not surprisingly, the numerous doctors, nurses, and community representatives interviewed by the assessment team were uniformly appreciative of the work done, claimed increased utilization as a result, and seemed committed to providing ongoing support. However, if health posts are to survive, and increase their utilization, a broad-scale effort will be required. As is being done now under PHCR, health posts will need to be upgraded, equipment provided, and trained staff put in place. Communities will need to be organized to use and support their local facilities. Importantly, financial resources will need to be mobilized to ensure the continued availability of needed supplies and the maintenance of facilities and equipment. Capacity building. Capacity building and advocacy to support rural health facilities are essential to ensure their sustainability beyond the life of the project. Building the capacity of key GOA counterparts, such as the MOH or the marz-level health department and the National Institute of Health (NIH), for training and maintenance of competency, is important to instill in the current system. In addition, advocacy to promote the importance of the role of the FAP and community healthcare in rural settings in particular, is a direction that may be pursued at this point in the project. PHCR is involved in capacity building and policy development; however, the FAPs and role of the community health nurse currently fall short of formal (legal) recognition. Moreover, GOA commitment to FAPs in the form of ensured budgetary support or legal rights to negotiate more productive relationships with ambulatories is poor. Both PHCR’s and NOVA’s focus on establishing community-based institutions is one step in addressing the future of FAPs in these communities. In interviews with two Community Health Committees in Shirak region health posts, there was a demonstrated community commitment to maintaining these facilities. For example, the newly renovated health post in Aregnadem suffered roof damage to their facility in a wind storm. The CHC quickly mobilized the community to donate materials and labor to repair the roof. Also, in the same community, the mayor has integrated the role of the FAP in her three-year mayoral strategy for the community. These groups are demonstrating both investment in their health as well as a practical application of local decision making and democratic process. Of critical importance to health post sustainability will be strong political and budgetary support at the highest level of government. Even with these efforts, time and economic development might well overwhelm some, and possibly many, health posts. Eventually, improved roads and easier transportation will encourage people to bypass health posts and go directly to ambulatories and polyclinics. In some areas, the smallest or weakest health posts in a system will close and others in close proximity may consolidate or expand to form new ambulatories. Certainly, some health posts will survive and even flourish. Those with the strongest community and financial support may continue indefinitely. Others may be subsidized because of the remoteness of their location or the unique characteristics of the population they serve. Assessment of the USAID/Armenia Primary Healthcare Reform Project 14 Adjustments and Modifications Collaboration on renovation of health posts and ambulatories. FAPs and ambulatory renovation and collaboration were explored throughout the assessment. While PHCR does collaborate with NOVA, the two projects are so closely linked they could take greater advantage of potential integrated activities to capitalize on their common goals. Although each project has a different technical focus, both PHCR and Project NOVA are involved in training staff and rehabilitating facilities in the rural health posts throughout the country. Because of contract timing, NOVA is geographically preceding PHCR with its renovation, equipment provision, and training activities. PHCR is currently following the NOVA project and upgrading FAPs in need of renovation/rejuvenation not covered by NOVA. This approach may not maximize sustainability, because PHCR criteria for FAP selection may not take into account maximum cost effectiveness and long-term sustainability with FAP renovations. In addition, FAPs that are avoided—the previously renovated NOVA FAPs—are not afforded the opportunity to form a CHC. NOVA￾renovated FAPs may miss the depth and breadth of PHCR training and the complementary community mobilization modules due to the current strategy for FAP selection practiced in Zone 1. As NOVA now begins to work in the southern marzes, focusing only on certain networks and geographical areas within these regions, and PHCR plans for its work in these marzes in years four and five, it is important to consider whether PHCR will work marz-wide or only in the select networks that have not been addressed by NOVA. Health post rehabilitation. The increasing costs of renovation, as noted above, will force PHCR either to decrease the breadth of renovations to future facilities or to decrease the overall quantity of facilities renovated while trying to maintain a certain quality of renovation. Although NOVA’s budget for renovations increased two fold in the South based on the results of its mid-term evaluation, it is still considerably smaller than PHCR’s. Although NOVA has ―first pick‖ of health posts to work with, they often have not been able to address as broadly the same level of rehabilitation as PHCR. To date, renovations done by NOVA have been divided from those done by PHCR, with no overlap. Work in the southern marzes (Zone 3) might require a change in order to develop more effective and efficient synergy between the two projects originally designed to complement each other with their overall programmatic activities. USAID and PHCR are currently weighing if it would make sense for PHCR to take NOVA’s renovation in the South to the next level by addressing a broader scope of infrastructure, equipment provision, community health nurse skill building, and major community awareness efforts in the same areas. While this would enable PHCR to stretch its budget, allowing the project to ―complete‖ more facility renovations, this practice could be problematic since each partner would likely count these facilities as renovated. Therefore careful consideration should be given to PMPs to ensure aggregate mission numbers are accurate. Criteria for selecting health posts. Criteria for FAP and potential ambulatory renovation and training must be reassessed to ensure that the most sustainable FAPs and ambulatories are fostered in the rural areas that need them the most. It is recommended that two key criteria should govern FAP and ambulatory decisions—first, the utilization of the facility, and second, the need for the facility. Health facilities that are currently highly utilized have already established a strong relationship with the community and developed the community’s reliance on the FAP or ambulatory facility, thereby facilitating primary healthcare in rural areas. Community relationships and utilization rates of local health facilities created at the grass roots level prior to donor intervention are most likely to be sustainable in the medium term. Therefore, an investment in those facilities should be prioritized. Assessment of the USAID/Armenia Primary Healthcare Reform Project 15 There may be some facilities identified that are not necessarily highly utilized or do not have strong relationships with the community. However, the community’s need for the facility may be high, such as in cases where rural health is poor. Traditional clinical FAPs and ambulatories may be renovated to attempt to improve the health services available to these high-risk communities. However, CHCs and other innovative methodologies to improve community knowledge, attitudes, and practices concerning primary healthcare should also be endorsed. The synergy of making health services (including NOVA’s MCH services) available, fostering demand and improved utilization of services, and increasing awareness of key health risks will promote improved health with rural populations. The communities with the highest need would benefit from NOVA MCH curriculum development, renovations by either project, and PHCR training coupled with CHC community mobilization. Strategic flexibility. If health posts identified in Zones 2 and 3 do not meet these two primary criteria of high utilization and poor health, then PHCR could be more flexible in its approach to working in the rural areas to maximize impact. Strategies that may be more appropriate could include: Complementing World Bank ambulatory renovations with CHCs Ensuring that ambulatories and polyclinics are fully stocked with critical PHC equipment, such as modern stethoscopes, blood pressure cuffs (potentially electronic ones), and glucometers, as well as lab functions for cholesterol checks that are now limited or absent from the current rural PHC systems Including heat or running water renovations where they were not fully developed in renovated ambulatories Working with NGOs to advocate with marz- and village-level governments to promote maintenance of ambulatories, hospitals, and polyclinics Training. In Zone 1 and 2, PHCR worked in the same geographical area as Project NOVA. In the Zone 3 rollout, PHCR’s geographical coverage may increase compared to NOVA’s limited coverage in the five southern marzes. What have been the benefits of each of these? Additional Questions: What are the costs and benefits of working in the same networks as Project NOVA? Considering NOVA’s work in FAPs and PHCR’s target of 230 FAPs, what portion of the remaining FAPs will be left in critical need? Or is it anticipated that the remaining facilities are in better condition or less used? Recommendations Fewer health posts. If necessary to stay within budget, PHCR should maintain its standards but reduce the targeted number of health posts and ambulatories to be renovated. Retaining the target, but reducing the amount or quality of the work done in each facility is not advisable. Selection of health posts. PHCR must be mindful that the existing network of health posts is vulnerable. The criteria used to select the health post that will be upgraded over the next three years should include consideration of those factors that are likely to influence long￾term viability—inaccessibility, community size and support, financing, staff availability, and the like. Assessment of the USAID/Armenia Primary Healthcare Reform Project 16 Coordination of community development strategies. USAID should encourage PHCR and Project NOVA to collaborate with USAID’s democracy and governance activities to encourage community-based strategic plans (to be determined) and ensure that the issue of FAP sustainability is a key component of these strategies. Health post sustainability. In light of the vulnerability of health posts, PHCR’s multi￾intervention strategy—renovation, equipment, staff training, financing, and community organization—offers the best hope for ensuring the sustainability of health posts. In addition, PHCR should help to develop the capacity within the MOH to support and maintain these initiatives on an ongoing basis. COMPONENT 2: FAMILY MEDICINE In Component 2, PHCR aims to consolidate and reinforce the government’s efforts to institutionalize not only the concept of FM, but also the basic tenets of affordable, quality, client￾focused healthcare. At inception, USAID expected the outcomes of PHCR contract to include: Autonomous, free-standing FM group practices 80 new practices 150 doctors and 670 nurses trained in FM The team was unable to ascertain if the ―80 new practices‖ referred to freestanding family medicine group practices. However, retraining of 150 doctors has been discarded, and emphasis is now placed on retraining of community rather than family nurses due to overlaps with the World Bank. At the same time, helping create independent Family Medicine practices was added. The contract now calls for the following approaches to be taken by PHCR: Help the government conceptualize and realize at least three distinct family practice models (polyclinic, rural FM, free-standing practice) Help develop a model for independent FM practice. Reinforce and institutionalize quality training in FM, nursing, and primary care Support the key policy and regulatory reforms (addressed in Component 1) The switch in emphasis to working with FAPs has limited PHCR’s ability to work with the first approach above, and currently only one free-standing family practice clinic exists in Armenia. No legal structure exists for freestanding family practice clinics. The third and fourth approaches remain targets for PHCR. Perhaps specific training for FAP nurses should be stated (although it is implicit in the changes). Achievements The family medicine team of PHCR works effectively together. The team gives the impression of general love of work and commitment to the goals of the project. They are oriented to give practical assistance in the marzes, and by holding training in the marzes, have made that training accessible to doctors and nurses who otherwise would not receive it. The sheer amount of training materials and policy papers produced, combined with the variety of the work that this team has accomplished, makes this perhaps the most complicated of all of PHCR components. PHCR family medicine achievements include: Assessment of the USAID/Armenia Primary Healthcare Reform Project 17 Development of three training packages to improve the content of unified family medicine curriculum (UFMC). These packages are up-to-date and were well-received by everyone, including the FM faculty members who conducted the retraining. (The subjects of the packages are cardiology, urinary tract infections, and dermatology. A training package on diabetes was close to completion when the USAID evaluation team was in the country.) Delivery of ―Chronic Disease Management in Primary Healthcare‖ training courses to 61 family physicians and clinical preceptors in two marzes and Yerevan. These courses, which served as a type of continuing medical education curriculum training, included some hands￾on training for preceptors in rural areas. Holding training of trainers (ToT) sessions for 48 faculty members at the trainings mentioned above Development of the unified family nursing curriculum and assuring its approval by the MOH and the Ministry of Education Development of the clinical teaching skills courses for FM and family nurse (FN) teaching institutions. These courses are greatly appreciated by those involved, and more are planned in this area. Delivery of ToT for UFNC at regional nursing medical colleges in Vanadzor and Gyumri Presentation of the cardiovascular nursing training package to 20 trainers Completion (in November 2007) of the 6½-month training of the first group of FAP nurses (130 nurses in Lori and Shirak) Upgrading of FN training departments in marz nursing colleges Development of draft guidance entitled, ―On Establishing and Registering Independent FM Practices in the Form of a Limited Liability Company‖ Challenges and Constraints The fact that no legal structure exists for free-standing family practice clinics makes the first approach—helping the government conceptualize and realize at least three distinct family practice models—impossible. There is also no corrected legal structure to support a real FM role within the polyclinics. Additionally, in spite of retraining, the physicians either do not feel comfortable with children when their original job was with adults (or vice-versa), or the overabundance of narrow specialists makes it impossible for a retrained physician to work as an FM doctor (the head physician of polyclinics still has to see that all of the narrow specialists have work). Finally, thus far the rural family physician has not been PHCR’s focal point because of its arrangement with World Bank to focus on FAPs. This constraint is not the fault of PHCR, but it emphasizes the challenge and need for PHCR to be involved in advocating for the legal changes. The primary challenge to the creation of freestanding practices is for PHCR to convince SHA to change contracts with actual practices, including how referrals are made, so that doctors will want to be involved with freestanding or independent family medicine clinics. Assessment of the USAID/Armenia Primary Healthcare Reform Project 18 Treatment Guidelines. There are no consistent guidelines or standards of procedures for nurses and physicians. Existing materials have been developed independently by previous donors and often do not reflect current, internationally accepted protocols. Current PHCR activities are addressing this by focusing on the development of practical job aids based on international standards, which can be easily referred to during practice. This activity has become a way to influence and update the guidelines. However, future attention should be given to address the updating and consistency of standards of practice in Armenia. This begins with defining ―guidelines,‖ ―standards,‖ and ―protocols‖ (i.e., are they the same?) and deciding who is responsible for writing, updating, and approving them. In the U.S., there are recognized groups that develop guidelines, such as the Academy of Pediatrics, the Association of Family Practitioners, the Diabetes Association, and others. With budget constraints, it seems unlikely that the guidelines will be developed without outside support, but perhaps the NIH and MOH could have a joint committee responsible for approval of guidelines, and that same committee could be involved in QA. The PHCR staff suggest consistent treatment guidelines can be a part of the QA initiative of the project. They feel these efforts will not be successful if there is no official board at the MOH or NIH responsible for protocol revision and updating. They feel that PHCR cannot do the entire job for MOH and other stakeholders. They suggest consideration of a joint QA committee at the policy￾making level that can address many of the QA challenges and treatment guideline development. Continuing Education. One potential area of concern the team noticed is the lack of concrete plans to maintain the nurses’ skills after they have completed their training in community nursing. While there are professional conferences and seminars organized by the Armenian Association of Family Physicians, these are financially prohibitive and often less applicable to community nurses. The team learned from the Yerevan Base Medical College (BMC) that the NIH is currently discussing a strategy to address future training needs. A few dollars of technical assistance could go a long way in helping them think through this process and plan accordingly. Family nurse education. The team’s visit to the BMC provided some new insights into the future of nurse training and workforce issues. Although the team was told that the BMC has the staffing capacity to provide a degree in family nursing (currently only available at BMC in Yerevan), they are restricted to training only for general nurses. Part of this dilemma is policy-based, however, with much of it based on the lack of laboratories, equipment, and other tangible requirements the school needs in order for the MOH to review the school’s accreditation. Additional donor resources were identified as a potential source of funding to address this facilities issue. An important discussion ensued about the number of nurses graduating from the school in the general nursing program and how many actually continued on to find work as a nurse, which was estimated at 35 percent. PHCR program developed three training packages containing UFMC course content and continuing medical education (CME) modules that are now institutionalized at the Yerevan State Medical University (YSMU) and NIH. All three of the training packages are up-to-date, in accordance with the format set by PHCR, and based on basic instructional design and IntraHealth PBL methodology even though they were developed in different modes. The method used by PHCR to create the training package for the cardiovascular diseases course for family physicians serves as a model for the development of other training packages. It included: A review of the evidence-based literature Discussions and presentations about local and international guidelines Assessment of the USAID/Armenia Primary Healthcare Reform Project 19 The formulation of a working group consisting of local experts in FM and cardiology— Armenia’s best in the field along with people who understand the role of family physicians— to create ownership for the training package and a responsibility to continually update the materials The involvement of all stakeholders in the development and implementation of guidelines and curricula at different all levels The quality of medical care delivery is a direct reflection of the education the medical providers receive. Some PHCR reviews on the teaching materials suggest that the contents for training on UFMC and UFNC topics are outdated. However, since the retraining program for family doctors and nurses is already well underway by the MOH and the World Bank, the PHCR team has sought to bring about better FM and nursing education by teaching instructors how to be better teachers using up-to-date training packages. Thus far, these instructors have been very pleased with the packages and have incorporated the packages into their retraining programs for UFMC and UFNC. Given constraints on changing a previously developed program that others have a stake in, the PHCR FM team’s approach shows good creativity in addressing this challenge. Understanding that much of the UFMC and UFNC retraining has been conducted by those with minimal hands-on experience, PHCR has developed marz-level clinical preceptors that allow the newly retrained doctors to have more hands-on work. However, the number days of work with a clinical preceptor cannot begin to make up for the lack of hands-on training in the rest of the retraining. Apparently, Armenian academicians still do not have an understanding about the amount of hands-on training required, and this attitude will continue to be a constraint in training until their understanding is changed. For example, in the last two years of medical school in most western countries (as well as during residency), there are few lectures (not more than one or two hour-long talks per week, and those could be optional) and no book learning (other than looking up information needed for care of specific patients). Training is entirely hands-on, under the observation of higher-level residents and attending who assure the correctness of medical care, tests, diagnoses, prescriptions, and procedures. USAID staff suggests that the general problem with both FM and FN training appears to be that those trained receive some knowledge and skills that they later do not, or cannot, apply in their practices. This means that training is disjointed from the practice; therefore, training needs for both doctors and nurses should be re-evaluated. Too often, real outcomes are not seen from FM or FN training; policy revision is under consideration, and there is thought being given to what can be done in addition to trainings to support doctors and nurses to expand the scope of their practices. One continuing challenge is encouraging exclusive breastfeeding in the first six months. According to the 2005 ADHS, 97% of all children are breastfed (an increase from 88% in 2000); median duration of any breastfeeding in Armenia is 10.5 months compared to 9 months reported by ADHS in 2000. However, median duration of exclusive and predominant breastfeeding is shorter than optimal (less than 1 month and 3 months, respectively). Exclusive breastfeeding is recommended by WHO and RA MOH during the first six months of a child’s life: although 2005 ADHS data suggests that a vast majority of children are breastfed (84%), only 1/3 are exclusively breastfed (33%) compared to 30% in 2000. Another constraint is a delay in getting the medical equipment, which appears to have been a stateside problem, but certainly has implications for training. Assessment of the USAID/Armenia Primary Healthcare Reform Project 20 The activities of the FM team are very demanding on the small staff of five. They advise PHCR financial team on performance-based financing and are intimately involved in the health education by PHCR health promotion team; additionally, they have responsibility for developing a quality assurance plan. One of their constraints is time and manpower. As PHCR goes into its final years, more budgeted funds for this team would be helpful to overcome this constraint. Adjustments and Modifications The modification of PHCR’s objective from the basic FM training of doctors and nurses to that of basic training of the FAP nurse has changed the approach of PHCR. One of the reasons that the World Bank was unable to include FAP nurses in their trainings was that these nurses, who often worked far in the countryside, were unwilling to go to Yerevan for training (a requirement of the World Bank family nurse retraining). The nurses have been very pleased that PHCR brought the nursing faculty to the marzes for their training. The scheduling of the training has also allowed them to maintain their work in the community. PHCR estimates that it will be able to meet its commitment of training 670 nurses by training FAP nurses in their marz regional nursing colleges. The freestanding or independent family medicine practices do not yet have a legal framework to exist. To the best of the team’s knowledge, none exists, although there was a mention of one in the north. The mechanism for supporting this lies in the hands of the GOA, and PHCR can only give advice on the legal framework. It seems highly likely that this part of PHCR’s work may not be accomplished in the time allotted, because it is dependent on bureaucratic change. Recommendations Curricula, modules, guidelines, training packages, and job aids. Although both PHCR and the World Bank are quite far along in their training of FM doctors and family nurses, and the curriculum looks terrific, there are still some concerns, including the following: Some confusion was expressed in the MOH about what a curriculum was, versus a module, guideline, job aid, or training package, and who has done what. The PHCR FM team is well aware of this problem and has noted that a centralized approach within the MOH in defining the role of clinical protocols, guidelines, and job aids is essential. PHCR suggests the joint creation of a glossary on training terminology to be used by the various organizations involved. While curricula may be considered teaching guidelines, they are not the same as clinical guidelines, which are standards of treatment that are actually stating the treatment or medication that should be used. The nomenclature is easily confused. Reviews conducted by PHCR for the nursing curriculum found that not all of what the NIH faculty taught was up-to-date, with the exception of subjects where teaching packages have been developed (by PHCR, the Eye Project, NOVA), which represented less than a quarter of the current training. Assessment of the USAID/Armenia Primary Healthcare Reform Project 21 The training experience is not hands-on at the level and intensity of an FM doctor in any Western European or North American country, although improvements in this area have been made, generally with PHCR input. The average U.S. medical student has been the principal provider, with preceptors watching, of more than 500 patients. The average U.S. FM resident has been the principal or sole provider—with preceptors with which they must discuss the case, though not always watching—of well over 2,000 patients, each of whom they have seen many times. Given these norms, it is easy to see why the hands-on learning of Armenian retrained doctors, estimated at less than 200 patients, seems minimal. Nursing students would have worked as principal nurse (while being observed) for more than 600 patients by the time they graduate. The curriculum was reviewed and supported by the World Organization of Family Doctors (of which the American Association of Family Physicians is a part). It is a good curriculum, but it does not guarantee hands-on training nor suggest how to teach it. Although the team did not receive a copy of the UFMC diary, seeing a patient with a disease is quite different from being the first-line care provider, which is the requirement of students and residents in Western countries. The training packages are up-to-date and well done. The methodology as developed by PHCR for the cardiovascular diseases course for family physicians is an excellent model of how training packages should be developed. In short, the development of training packages should include (with slight variation from the PHCR format): – Have the YSMU (or NIH or BMC) pull together Armenia’s best in the field, and people who understand the role of family physicians and nurses – Invite all stakeholders involved in guidelines and curriculum – Review the evidenced-based literature – Discuss and present the local and international guidelines – Field-test the guidelines and make addendum adjustments for the realities of the field (this part is not yet complete for the cardiovascular disease course) PHCR’s recommendation of a glossary of terms is a good one. Because it was mentioned during the team’s last day in country, the team was unable to ask if this was acceptable to MOH, which was asking for a conference. A brief conference with all stakeholders would be helpful, presenting various examples of curricula, guidelines, training packages, and the like to determine nomenclature, responsibility, and who approves what. PHCR should continue to use its cardiovascular training package methodology to reach national consensus on the teaching of other modules in the FM curriculum. This is a key element and perhaps PHCR’s most influential capacity-building role to improve medical education and increase skills for Armenia’s health providers. PHCR should also increase the hands-on experience for those using these packages, for example, by requiring that 50 blood pressures be taken with preceptor checking. Continuous medical education (CME). The Armenian FM academicians were asked about future needs they wanted to be met by PHCR’s CME. CME can come in three general formats: an intensive, all-encompassing review a selective review of a single topic deemed important to the country or the practitioner Assessment of the USAID/Armenia Primary Healthcare Reform Project 22 a written topic in which the physician reads an article and responds to questions about it (sometimes done on the computer) In the U.S., the family physician decides which type of CME he or she wants. The choice is often driven by whether or not the physician is going to take the written recertification exam that is required every seven years to demonstrate current knowledge (i.e., a physician would choose the all￾encompassing review in the year of the exam). The American Board of Family Physicians also requires computerized CME modules and a certain number of credit hours, gained through conference attendance or by reading articles and answering questions, every year. The training packages developed by PHCR actually had CME in mind and make a logical choice for CME in Armenia for the time being. Currently no formal credit system for CME exists in Armenia. Should a licensing system or board recertification program (e.g. under the AAFP) exist, a specific number of CME credits could be required for licensing or board renewal. PHCR should support the development of CMEs by using PHCR training packages, along with specific priority (possibly mandatory) subjects on a yearly basis. Consideration of the realities of the rural physician and nurse in developing Armenia curricula tools. While attempting to provide state-of-the-art medical care, the curriculum, guidelines, and modules often ignore the current realities of polyclinic, rural ambulatory FM, and FAP work. It makes sense to acknowledge the gold standard practices, but an alternative method needs to be indicated in the teaching tools (perhaps as an addendum) when the gold standard practice cannot be done or is not available. For example, for diabetes control, the hemoglobin A1C blood test is the gold standard for determining how well sugars are being controlled, since it reflects the degree of control over the previous three months. If hemoglobin A1C is not available, the alternative is to take blood sugars on an average day of activity and meals at fasting, one to two hours after each meal, and at bedtime. However, clinic hours make only fasting and one to two hours after breakfast and lunch the only possible times to test. This schedule of testing would need to be repeated every one to two months until target blood sugar levels approach an individual’s goals. When a glucometer is not available, a urine dip for sugar would be the next best substitute, although it is only positive if blood sugar is under very poor control. If urine sticks are not available, then checking for urine frequency and thirst or signs of hypoglycemia may be the only check available, and might be used as an indication for a patient to make the long trip to a laboratory. When guidelines and protocols do not take into consideration the local realities, it appears that the professionals who developed them are not fully aware of the situation. More appropriate guidelines and teaching tools could be developed if PHCR family doctor staff members spent time in the rural FAPs, ambulatories, and polyclinics shadowing the doctors and nurses. (Shadowing does not directly improve clinical skill deficits, but it would help PHCR tailor the training packages to what is most needed.) Family doctors in PHCR staff have already expressed their desire to do this but felt they did not have enough time. Some suggestions on how this type of observation should be conducted include: Taking notes and making observations in a non-judgmental way Observing how diagnoses are made and what treatments are given and determining what would make for better diagnoses or treatments Assessment of the USAID/Armenia Primary Healthcare Reform Project 23 Asking how medicine is used in the clinic Asking patients if they can go to the referral, take the blood test, or buy the medicine, and if so, how long it would take Inquiring about the likelihood of receiving free medication Checking the practicality of guidelines, performance indicators, and job aids Reviewing the log of patients and some records to determine who comes to the clinic, why they come, and what records are kept; Going on home visits to determine if doctors are always necessary or if nurses do this work Accompanying a doctor on monthly FAP visits Asking doctors, nurses, and patients for improvement ideas In the brief time spent with the PHCR FM team members conducting site visits in Vanadzor and Gyumri, it was observed that the medicines available in FAPs varied considerably, some nurses used the medicines while others did not, and some used them incorrectly. Few blood pressures were taken or recorded, and few nurses had thermometers, even though cardiovascular and infectious diseases were among the most common reasons for visits. A list of common illnesses seen in October 2007 at a single family physician ambulatory clinic is given in Annex 3. PHCR medical staff and medical consultants need to spend a few days in each zone shadowing FM doctors, FNs, community FNs, and patients in day-to-day work. Updated curricula/guidelines should reflect rural capabilities and include rural practitioners in the updating process, since they are stakeholders and can report on what is possible in their clinics. Once these tools are updated, PHCR should return to the rural areas, observe what happens when the tools are used, and, if necessary, change the tools based on their findings. Clinical skills in reading blood pressure. Concrete information from neighboring countries confirms the importance of attending to hypertension and associated cardiovascular disease in Armenia. More than 50 percent of adults in the Republic of Georgia had improperly controlled hypertension,8 and it may be reasonable to assume that Armenia has a similar rate. Furthermore, in Russia, the rates of cardiovascular mortality among men 35 to 49 years of age are more than five times that of American men of the same age.9 Although blood pressure reading is among the most basic of medical skills (and probably one of the most important in Armenia) and the first line in cardiovascular disease prevention, it appears that many doctors and nurses—even those who have gone through training—have not learned this skill. Observations in this area include: The assessment team watched a doctor take a blood pressure and saw doctors’ and nurses’ records blood pressures that were absolutely impossible—virtually all patients with a blood pressure of 120/80. Stethoscopes are of poor quality and it is difficult to hear through them, particularly when trying to listen through clothing, as was observed. Most doctors do not have a large-sized cuff, which is likely to become more and more necessary. Good equipment is not required, but it definitely makes it easier to learn how to perform this test correctly. 8 Vakhtang Y. Barbakadze, Levan G. Koblianidze , Nodar N. Kipshidze, Clarence E Grim, Carlene M. Grim, and Frederick Tavill, (2006). The Republic of Georgia High Blood Pressure Control Program. Ethn Dis.; 16 (2 Suppl 2):S2-61-5. 9 V. Fuster, J. Voute, M. Hunn, and S.C. Smith, (2007). Low Priority of Cardiovascular and Chronic Disease on the Global Health Agenda: A Cause for Concern. Circulation: 116: 1966-1970. Assessment of the USAID/Armenia Primary Healthcare Reform Project 24 Most medical records do not include many blood pressure readings. Doctors and nurses are capable of learning these skills, but have obviously not been taught them systematically, with hands-on work with patient after patient and with a mentor’s assistance. Even in developed countries, this skill has not always been perfected, and there are many skills that Armenian doctors and nurses need to master. However, since hypertension control is such a big problem and related to the most common cause of death in Armenia (cardiovascular disease), it seems that providing proper training on taking blood pressure would yield the greatest dollar return for any healthcare investment made in Armenia. Doctors and nurses (even those who have already been in the training) need to be systematically trained in taking blood pressure, perhaps requiring a certification. The American Heart Association has a training certification program that could be adapted for an Armenian training package for FM/FN retraining and CME. Role of the community nurse. The place of the community (FAP) nurse in the future of the Armenia health system is not yet insured beyond ten years. The community nurse position appears to have something of a parallel to the public health nurse in the U.S., which could suggest a role for the community nurse even if FAPs do not continue to be part of the Armenian healthcare system in the future. PHCR training allows nurses to receive the FN certificate, which is good. However, it is questionable whether most FAP and rural nurses will ever use the skills or have access to the necessary materials to perform certain procedures learned in training, or if certain health problems are so rare that the skills learned will be forgotten by the time they are needed. Although the community nurse curriculum has already been approved by the MOH and endorsed by the MOE, it is not too soon to begin updating it. Examples of some of these rarely used procedures are: Psychoactive drug education, which rural nurses say they are sure none of their patients use Occult blood reading, since no one has the cards to perform this Visual acuity, since there are no eye score charts Peak flow meters Cervical, vaginal, and urethral sample taking Pregnancy tests Electrocardiograms Foley catheter placement Intra-ocular pressure Ear and tonsil lavage Analgesia induction IV sedation On the other hand, thermometers are not in the FAPs, and there is no discussion of how to take an adult or baby’s temperature, in spite of the fact that febrile seizures are a main concern of doctors Assessment of the USAID/Armenia Primary Healthcare Reform Project 25 and infectious disease is one of the most common reasons for FAP visits. Directly observed therapy (DOT) is not mentioned, even with its stated emphasis in Armenia policy (although it is poorly implemented). DOT, like immunization, is best administered and in many cases only possible to be administered at the local level (FAP or ambulatory) with good follow-through. Perhaps worst of all, the curriculum does not include education about when and how to use the medicines that are currently in the FAP office. If a list exists of the FAP medications, the evaluation team never had access to it, and all FAPs differed as to what was available. The community nurse curriculum needs updating so that it is more suited to the actual capabilities and needs of the FAP nurse. A high priority would be to standardize (in reality, not just on paper) the medication available at the FAP and be certain that each FAP nurse knows exactly when and how to use them. Community nurses should be considered a specialty with far greater responsibilities than those of family nurses and similar to public health nurses in the U.S. and elsewhere. Therefore, it should be expected that training and retraining would take longer. For the long term, PHCR should assist the MOH in making a declaration of community nursing as a specialty, expand the curriculum, and institutionalize community nursing training within a nursing college. Such a nurse will have a role in future Armenian healthcare, even if the actual FAP post does not. Build the relationship between the FAP and the ambulatory. Although the existence of the FAP depends on an ambulatory center or a polyclinic and its associated doctors are supposed to have regular visits to the FAP and advise the FAP nurse when requested, there are a variety of relationships between the FAP and its associated clinic: Currently, doctors are required to visit FAPs one time per month, although some visit more often. A few doctors bring medicine with them, although most did not. The doctors with whom the evaluation team spoke spent half of their FAP time making home visits in the community. Polyclinics are supposed to supply FAP emergency medicine, but this varies by FAP as to what is given and when it is given. Any medical supplies—from gloves to glucometers—depend entirely on the benevolence of the supervisory clinic. Even though the polyclinic gets reimbursed for urine dips and glucose determinations, one manager we spoke to did not see the advantage of providing even minimal financial support or necessary supplies (for example, a glucometer or urine sticks) to have tests done at the FAP. There is no evidence in the FAP of planning for the efficient use of its doctor’s time. PHCR should support a more constructive a relationship between the FAP and its associated doctor, which could include the following: FAP nurses should schedule appointments and line up patients to see a doctor for specific reason on a specific date. Doctors’ reimbursements for glucometers and urine sticks should be used as an incentive for them to encourage FAP nurses to do this testing for them. Assessment of the USAID/Armenia Primary Healthcare Reform Project 26 Doctors should be encouraged to bring medicine to the FAP through a system where appointments are made in advance so they will know which medicines to bring. Home visits by the doctor should be limited to the disabled who cannot make it into a clinic. Emphasize MCH. Armenia has a number of health issues on the MCH area, including: High infant mortality rate. Although Armenia’s rate is among the highest in the region, as shown in the two DHS surveys, overall infant mortality decreased from 30 deaths per 1,000 live births in 2000 to 26 in 2005. Evidence suggests that significant proportion of the infant mortality is neonatal (20 per 1,000 live births and 17 in 2000 and 2005, respectively), which suggests that the problem is largely occurring during pregnancy or childbirth calling for more effective interventions during prenatal and perinatal period. High maternal mortality ratio. Maternal mortality ratio is an important indicator of the women health and their access to health services. The maternal mortality in Armenia currently at 31/100,000 (RA MOH, 2006) exceeds the average for Central and Eastern Europe (15/100,000) and is noticeably higher than in Western Europe (8.8/100,000). The current MMR is similar to the data reported in late 1980s, which suggests that Armenia experienced little improvement since then. Major causes of maternal deaths are hemorrhage (34%), hypertension (23%), infection (9%), and abortion. Significant proportion of maternal deaths (75%) is occurring after 28th week of gestation in labor and postpartum period. Abortion contributed to 8% of maternal deaths. Breastfeeding and anemia in children. According to DHS about 13% of babies receive milk other than breastmilk, an important contributor to anemia. According to ADHS data on the anemia rates for children 6-59 month, 37% of children suffer from anemia: with only 1% having severe anemia, and one-half – moderate anemia. However, DHS data also suggest that anemia among children increased 50% due to the increase in a moderate level of anemia. DHS data on anemia has to be interpreted with caution because of the equipment malfunction (see page 152 2005 ADHS) Lack of unified in-depth MCH training for community nurses. Currently, only a few of the community nurses who have been retrained as family nurses (either through World Bank or PHCR) have received the full MCH curriculum. The UFNC offers a shorter version of the NOVA self-paced 6-months program, with its 22-day classroom time, in which nurses receive training for fewer than 10 days over the course of a month. The shorter version is probably suitable for most family nurses, since reproductive and prenatal care is usually performed by an OB/GYN, or, in rare cases, an FM doctors. However, community nurse requires more MCH education than this. In spite of all of these MCH challenges, PHCR does not do as much MCH work as its sister organization, NOVA, and NOVA’s predecessor, PRIME II. PRIME II’s mission was to work with babies up to six months of age and to cover all aspects of reproductive health. PRIME and NOVA also have developed an excellent program for training the community nurse in Safe Motherhood Clinical Skills (MCH/RH/FP). Community nurses need a large educational component of MCH, which could be provided by increasing the duration of their training to include all 22 days (spread throughout the course of several months) of the NOVA training. Perhaps NOVA needs first to reassess whether all of their training is being used by FAP nurses. Assessment of the USAID/Armenia Primary Healthcare Reform Project 27 In spite of progress, national attention and educational campaigns are still needed to change cultural attitudes towards breastfeeding to encourage new mothers to do this longer. PHCR’s community education program could be used to spread the message that breast milk is good for babies. Consideration needs to be given to providing economic support, such as subsidized birth control pills or Depo Provera, to encourage women to use other forms of birth control instead of abortion. FAP nurses and family doctors need the legal ability to give or prescribe birth control. Training more OB/Gyn doctors to perform tubal ligations might make them less obstructive to allowing others to prescribe birth control, since abortions tend to be a major source of income. COMPONENT 3: OPEN ENROLLMENT Open enrollment (OE) is the process of individuals selecting and registering for a primary healthcare physician at a specific facility in order to both access primary care and, if needed, access specialists’ care not considered part of PHC services, such as referred diagnostic, specialized and inpatient services. The GOA and its relevant agencies, particularly the SHA, seem to be totally in support of OE and are preparing to have it be the basis for reimbursement for services within a short period of time—specifically, the beginning of 2008. While budgetary increases are evident, particularly for PHC, no specific funds have been cited for OE and, perhaps, such earmarking may not be necessary. Provider incentives for OE are obvious, since their salaries will be based on both the number and the quality of services provided. Population incentive to enroll is to insure their access to a caregiver of their choice as well as access to specialized care. While OE is viewed as a major element in sector reform by establishing the up-to-date population basis for each provider and introducing the element of provider selection, its impact will be constrained by the timely progress in all the major elements of system reform, including FM, quality of care, financing, and public education. OE implementation will facilitate data gathering and processing, referral to other levels of care, and introduction of performance-based provider reimbursement, improving service delivery and management efficiency. PHCR’s use of short-term consultants has been excellent, particularly the work of Ahsan Sadiq, which provides a thorough assessment of OE, including the benefits to all stakeholders and options for next steps. Achievements Achievements in the OE area include: Provision technical assistance to the MOH on 2008-2013 PHC strategy, quality of care for PHC services, and a rural incentive system, particularly as they pertain to OE Increased accessibility and use of PHC facilities by renovating facilities, a first step in community motivation Creation of a computerized health information system, including provision of hardware and software for registration, financing, management and planning, and training of staff in the use of the systems, as well as establishing a one-year help desk for users Active participation in the national OE coordination group Provision of training and materials to regional health authorities and their PHC facilities Organization of an Estonian study tour Assessment of the USAID/Armenia Primary Healthcare Reform Project 28 Support to the MOH in preparation of the legal and policy base for OE Increased public awareness through promotion efforts in most media Support for short-term technical assistance, including a report by Ahsan Sadiq and Frank Feeley that analyzes OE strengths, limitations, linkages, financing and recommendations for next steps, a comprehensive and useful document Current OE registration of about 60 percent of the population PHCR Achievement of Objectives and Goals The expected contractual outcomes are: Adoption of the concept of OE throughout the marzes. Backed by GOA degrees and supported by PHCR trainings, materials preparation, and promotion have resulted in marzes adopting OE nationally. Implementation of an OE information system to track facility use. PHCR developed a computerized health information system, providing necessary hardware, software, training, and help desk to track registrations and assist in the implementation process. Policies in place to support OE. PHCR supported the development of regulatory guidelines to implement OE and responded to the MOH and World Bank requirements for an earlier initiation of OE in the field by accelerating its contribution to the ancillary guidelines. Provider training and facilities renovation that improve clinic utilization and quality of care, through an OE approach. PHCR supported the development of simple quality of care indicators and the use of performance-based reimbursement of providers on a pilot basis and used it as the basis for impending GOA implementation nationally. Services rendered with a more client-oriented approach, and clients receiving improved quality of care. PHCR provided TOT for community health education and provided materials to improve the population’s understanding of the delivery of quality of care within PHC services, as well as provider training enhancement. Challenges and Constraints The following represent OE challenges and constraints for PHCR: Project impact will be negative for some stakeholders if some providers are passive in enrollment efforts or if their respective catchment area population considers them delivering a questionable quality of care, resulting in fewer numbers of registrations for those providers, and thus reducing their income. GOA flow of funds has not been identified as a limit to OE. OE alone will not change health outcomes without a balanced and timely system reform, as was previously stated, and, in rural areas, the availability of alternative providers from which selection can be made. PHCR has provided guidelines, information management, training, and patient education, particularly to underserved rural populations. However, public education on health matters, Assessment of the USAID/Armenia Primary Healthcare Reform Project 29 particularly in rural areas, is not institutionalized at either the national or the marz level. This area must be addressed by PHCR in the next three years, as discussed later. Adjustments and Modifications While the information system, guidelines, training, and help desk activities have been instrumental in the launching of OE, it is critical that these activities continue during the remaining three years of PHCR, and that these activities must be sustained and institutionalized within the GOA agencies. Therefore, PHCR activities that reinforce the ―passing of the baton‖ —developing the capacity and institutionalization of these activities within the GOA—should both continue and be systematically addressed. Perhaps PHCR activities could be curtailed unless carried out jointly with respective GOA agency participation at all levels within national, marz, and sub-marz organizations. PHCR should coordinate the criteria for selecting new community investments with plans for the MCC to improve road access and the availability of water. The latter infrastructure would complement and support the development and sustainability of health facilities. (See Component 5 regarding institutionalization and capacity enhancement of respective GOA agencies.) The existing PHCR contractual performance measures are realistic and achievable within their funding levels. Recommendations The following are recommended for PHCR to forward OE objectives: Strengthened institutional capacity to implement PHC reform, USAID intermediate result 3.2.1. Critical to sustaining the launching, continuation, and monitoring of OE is the ownership of the respective GOA agency and its ability to continue as well as to evolve and improve the PHC OE activities necessary for continuous reform. While GOA ownership and capacity improvement is currently incorporated in most PHCR activities, these activities should be reviewed and systematically addressed in the coming three years of PHCR. Each PHCR action should be planned and implemented jointly by the staff of PHCR and the respective GOA agency at the national, marz, and relevant sub-marz levels, with participation from providers—including both public and private doctors, nurses and administrators—and the local community. An option which may be considered is to imbed PHCR OE activities within an agency of the GOA such as the MOH/SHA and the marzpetrans with the goal of increasing the respective agency’s ownership of PHCR activities and achievements, and institutionalize its capacity to continue these functions. This could be accomplished in a staged manner. A decree might be needed for a special office, along with a staged GOA budget for space, logistical support, and added staff. USAID support, while potentially diminished after three years, could possibility continue to support of a reduced level of technical staff as the technical capacity of the GOA agency increases. A monitoring system that provides registrant feedback and prevents abuse in reporting OE numbers of enrollees and tracking performance-based reimbursement. This is an area where PHCR could provide technical input, particularly for PHC centers. A necessary issue to address is the use of a numerical identifier for each population member. Having such numbers assigned could facilitate development of a monitoring system. Assessment of the USAID/Armenia Primary Healthcare Reform Project 30 COMPONENT 4: QUALITY OF CARE The Quality of Care (QOC) component is the least developed of all the components to date, and the reason is obvious. A clear role and the specific method of FM education in Armenia had to be established first. The expected outcomes of this component in the initial contract were the following: The establishment of a family medical council The establishment of provider licensure procedures The establishment and use of facility accreditation procedures The adoption of internationally accepted standards and protocols for PHC Increased access to quality health services The use of state-of-the-art quality improvement methodologies Achievements The following are achievements to date: Commencement of data entry of licensing information for the MOH Production of a draft Quality Assurance Strategy package with implementation plan, the Armenian version of which has been presented to stakeholders for feedback, submitted to the MOH, and recently approved Delivery of training on PHC QOC in Zones 1 and 2 within the PHCR management training course Creation of a baseline household survey, which will, among other things, assess accessibility of health services Development of protocols, job aids, clinical guidelines and training packages as part of the QOC package, as discussed in Family Medicine Component #2 Challenges and Constraints To date, no Family Medical Council has been established. As mentioned previously, one would be helpful in determining whether specific guidelines, updated curricula, etc. are accepted. PHCR has mentioned that it is working on this; however, there is the constraint of adding an increased time burden on already busy professionals. Everyone would like formal licensing in Armenia; however, it has not been done, and it will require a considerable amount of legal work that includes evaluating the skills and knowledge of the many medical colleges in Armenia. Developing testing in itself is enormously time-consuming and has to be piloted; writers need training on developing simple questions that are not confusing. This may not yet be reasonable in Armenia. Nonetheless, a board certification may be possible for those who have received a specific training and have continued to have a certain number of hours of CME. In its limited time in country, the team was not directed to meet with any licensing board. Assessment of the USAID/Armenia Primary Healthcare Reform Project 31 The team did not meet with people working in facility accreditation; to the best of the team’s knowledge, the only movement in this direction has been the standardization of equipment and building standards by both the World Bank and PHCR. The QOC package is an ambitious program that depends upon training skills, such as supervision, that have never been previously developed in Armenia. The Supportive Supervision Training, considered key for supervisors and providers to implement QA, is used in the immunization program of the World Health Organization; however, evidence of its success in former Soviet countries could not be found. The vertical, top-down structure of the former Soviet medical system may create special challenges to the institution of supportive supervision, and require creativity in evaluation (for example, employees may have problems criticizing their supervisor in an evaluation). Additionally, the actual text of PHCR QOC package suggests controls and feedbacks that do not lend themselves to the supportive supervision. The recommendations below contain ideas of how to monitor quality without concern for overly punitive supervision. Groundwork for QOC chart review has been laid, and the PHCR proposal is quite detailed and idealistic. The amount of time allotted in each clinic to do all these steps is not realistic for something to be continuous, and may cause physician, patient and clinic manager resentment due to the time it takes away from patient care. The assessment team was told by USAID that not all of the Quality Assurance Strategy Package would be instituted at the same time, but that these tools would be phased in. Everyone involved seemed to understand that this was too ambitious in terms of getting it all done in the timeframe suggested. It is one thing to have it up and running in a pilot QOC project, but quite another to get regional or national acceptance. Tying QOC to information obtained from the institution of performance-based financing is a viable and efficient method of QOC. The indicators chosen for QOC overlap a great deal with the performance-based financing indicators. These indicators have been chosen based on the practicality of receiving accurate information. For example, since referral forms for fundoscopy of diabetics have to be written, counting the number of referrals is used as a QOC indicator (an eye exam is considered standard of care for diabetes). However, a measure of the control of diabetes, such as the gold-standard hemoglobin A1, is impossible in diabetes. To complicate matters further, a referral to an ophthalmologist does not mean that patients are receiving a good or even adequate fundoscopic exam, since many ophthalmologists do not have working ophthalmoscopes and most in rural areas had no cycloplegic medication for dilating eyes, according to one NGO working in the vision field, and which the reviewer found commonly the case in the district medical centers of other former Soviet countries. The constraints of services and tests available to most FM doctors in Armenia make the choices for QOC reportable indicators less than optimal. Adjustment and Modifications This component has had no adjustments or modifications to date as it is only just starting, although as the program stands, it is too ambitious to stay on schedule. This reviewer is not an expert in this field, as the tenor of the QOC is administrative rather than medical. However, to a physician who has worked in a number of countries, it seems the current plan would not likely be feasible for more than a trial in a highly supervised situation. The amount of time away from seeing patients to accomplish the QOC as it currently stands assures that it is not sustainable. One part of the QOC was to judge how well the doctors are following the guidelines; if those guidelines include tests and medicines not available, this type of QOC would only be demoralizing. Assessment of the USAID/Armenia Primary Healthcare Reform Project 32 For this reason, some ideas for self-assessment QOC liked by doctors in several countries are offered below. This is not meant to be a replacement or improvement of already pre-determined performance-based payment indicators, but are meant to make family doctors think about how they can improve their practice with what they have available to them. Recommendations Self-assessment. The self-assessment tool currently proposed in the Quality Assurance Strategy does not qualify in terms of medical care or analysis. This appears to be an underuse of the potential in self-assessment. Use of the job aid has been planned as a future self-assessment tool (though not stated as self-assessment in the packet reviewed by the team). This would be a good idea and is on a more complex level than the list given below. It would be necessary to know just what capabilities the physician and the patients have to follow the job aid (i.e. do they have the money to buy the medication?). QOC in the U.S. is variable, but it generally consists of physicians—and even FM residents as part of their residencies—reviewing their own charts. For example, during a lunch period, the physicians might review their diabetic patients, asking the following questions: Is the HgbA1C below <7? If not, are medicines being adjusted to go in that direction? Are blood pressures below 130/80? Has the patient had a foot exam, eye exam, ECG, and cholesterol panel in the last year? This may occur every three to six months. The physicians do not report their errors; however, doctors make notes to take care of any deficiencies during the patient’s next visit. This methodology, with questions particularly applicable to Armenia (i.e. the guidelines in Armenia, as mentioned above, are often impossible to follow because of lack of resources and materials), would be gentle and probably more acceptable than the supervisorial QOC of the strategy package. It could be something as simple as everyone checking the quality of their medical charting. A list of possible items to check is under ―medical charting‖ below. Clinic-wide indicators. Making use of the QOC indicators by having performance-based payment is reasonable, in spite of the limited number of indicators that can be chosen. One QOC practice using performance-based indicators typical in the U.S. is to have indicators, usually those which are normally computerized (like lab reports), reported for an entire clinic, and the same indicators are reported for other named or unnamed clinics that serve a similar (socio-economic and ethnic) clientele. The advantage is that rather than pointing the finger at one particular provider, the whole clinic wants to do things better together. Although performance-based payment actually reports on an individual basis, the same data can be used to compare between clinics; if one clinic were not doing as well, it would be important to understand why. Medical charting. Good medical charting is an important part of QOC, but currently, the charting is minimal and it is difficult to understand quality. The recently developed new medical chart will help assess quality; some additional suggestions are listed in Annex 3. PHCR needs to be cautious about overburdening clinics with chart checks and ensuring that supportive supervision is actually done (rather than reverting to punitive supervision, which was typical of the Soviet era). Using the QOC indicators as comparison between similar clinics can also Assessment of the USAID/Armenia Primary Healthcare Reform Project 33 be helpful. If Armenia wishes to emulate the gentler American self-assessment, below are questions (different from the QOC indications) that lend themselves to Armenia-available intervention and education: Percentage of babies breastfed solely until 6 months (percentage with no cow’s milk for 12 months) Percentage of adult visits with blood pressure taken Percentage of hypertensive and diabetic patients with urine screening Percentage of controlled hypertensives (being sure to use the lower targets for diabetes and/or renal disease) Percentage of babies with weight and height recorded Percentage of adults with BMI calculated Please note these are not suggested as substitutes for performance-based payments, which have been chosen because they are feasible and the medical provider cannot ―lie‖ about them. The above list is for self-assessment, which will be easier to check than the entire treatment guideline. However, they could not be ―proved‖ as easily as the performance-based indicators that have already been chosen. COMPONENT 5: HEALTHCARE FINANCING In the public sector, the GOA’s commitment to subsidize the cost of PHC based on enrollment makes it key to reform. The political and organizational will to move ahead exists, but there remains the completion and adaptation of some of the intended reforms, such as performance-based reimbursement operational procedures and the monitoring of these new reforms. This is precisely the area in which PHCR is contributing successfully and must continue its support in order to rationalize the process, minimizing unwanted effects. The emphasis of the GOA in subsidizing PHC services will be the major factor in improving health outcomes and decreasing disparities if it results in improving access to the complete package of basic healthcare, particularly in rural areas. The team could not conclude at what depth the basic PHC package was, in fact, delivered at the community level, except to function as a referral system to a more advanced level of care. What was observed, albeit from a small and insufficient sample, was that the delivery of basic PHC varied widely at the community level. The GOA is banking on OE to facilitate financial remuneration for services, but those payments must be both timely and rational (that is, based on cost estimates reflecting actual field values, particularly given cost changes over time resulting from inflation and other factors) for access to effectively improve health outcomes, particularly in rural areas. Effectiveness of provider reimbursement for PHC will be contingent upon GOA functional capacity to analyze information and develop evidence-based implementation policies and procedures (the operational procedures and their promulgation to relevant stakeholders) and a nascent internal capacity that PHCR is both developing and in part substituting for, that must be institutionalized within the GOA at all appropriate levels. While it appears that the GOA will support the development of private healthcare services, the nature of that support (in terms of details such as policies and procedures) is yet to be determined, although PHCR has drafted the initial guidance on set-up procedures, which is currently being considered by the GOA. The promotion of private facilities, such as subsidies, is an area in which Assessment of the USAID/Armenia Primary Healthcare Reform Project 34 continuous support might be needed in order to sustain future progress. It is noted that the World Bank is initiating promotion of the private sector, including the provision of facilities and equipment for 50 entities shortly. Performance-based reimbursement is a method of payment that motivates more effective and efficient delivery of services and is consistent with both the country’s needs and the goals of PHCR. The impact of performance-based reimbursement will be dependent upon the rational cost basis, transparency, consistency, and timeliness of implementation. These latter requirements can easily be hindered if bureaucracy results in disincentives, such as payment delays, costing updating delays, or undue paperwork. A monitoring system is needed to minimize such occurrences and provide feedback to all stakeholders necessary to have continuous system reform improvements. This activity is included in PHCR work plan for its third year to assist the MOH/SHA in the development of policy support and implementation of systems, including the development of a guide for monitoring and evaluation (M&E) of PHC performance, as well as a guide on evaluation of the performance of the PHC team within the PHC facility. This activity must include participation of both the national and local level relevant entities in the public and private sectors. While considerable discussion regarding proposed reimbursement impacts on medical doctors was heard, no parallel observations were made for the impact of performance-based reimbursement on nurses. However, the ASTP pilot program proposed formulas for the doctor, nurse, and administrative staff reimbursements, which were included in the remuneration order approved by the Minister of Health. Achievements PHCR has made substantial contributions through the work of its own staff in areas outlined below and through the work of short-term consultants. Works by Igor Sheiman, Frank Feeley, Ashan Sadiq, and Taryn Vian provide ample analysis and next-step recommendations for major sector improvements in the following areas: OE—switching from catchment-area-based to enrollment with healthcare providers – PHCR’s community promotional and educational campaigns have contributed to the population registration under the GOA enrollment efforts; however, no government institution was identified with internal capacity to sustain these enrollment efforts or to provide a sustained health promotion and disease prevention campaign, particularly in rural areas. – PHCR projections of potential enrollment issues have contributed to the GOA increasing flexibility in requirements for the conditions to initiate remuneration to providers (that is, the initial requirement of 70 percent is no longer required). – PHCR has proposed options and formulas to shift from catchment area to enrollment￾based financing. Management and information—support in system development and training – PHCR provided management and accounting training, as well as IT hardware, software, and training for information automation, including accounting software. Performance reimbursement—development of measures reporting forms, reward formulas, and draft implementation order Assessment of the USAID/Armenia Primary Healthcare Reform Project 35 – PHCR evaluated pilot experiments. – PHCR participated in the development of performance-based quantity and quality indicators and a payment formula, both of which are being considered for use in the reimbursement of PHC services for both doctors and nurses. – PHCR assessed the MIDAS-2 system for use in PHC setting to automate data collection and evaluation of performance measures. Medium-term expenditure framework—reporting progress towards health policy objectives – PHCR helped MOH develop performance measures for five quantity and five quality indicators for the next three years. Normative costs for PHC lab services – PHCR developed and obtained agreement with the MOH/SHA on a methodology for normative costing. – PHCR developed a computer model on agreed methodology. – PHCR refined BBP list of PHC lab and instrument services. – PHCR developed models for costs of 25 PHC services accepted by SHA and then were asked and completed 115 cost models for typical PHC lab services for use in expenditure remuneration. – PHCR educated SHA technical staff on specifics and use of models. National Health Accounts (NHA) – PHCR participated in the development of the NHA process for the GOA, developing methodology, accounting systems, and the need for analysis that identifies sub-group impacts, the latter of which will hopefully be included in future presentations. National and Regional Healthcare Planning – PHCR provided training and supported the participation of local entities’ planning efforts—initiating a bottom-up approach not typical in GOA practices. Contractual Requirements. As noted below, PHCR is generally on track regarding achieving its contractual requirements in this component. However, monetary exchange rates and local inflation may limit some achievements, particularly in financing future PHC facility renovation and maintenance. Improved and deepened awareness of the national budgetary and medium-term expenditure framework process, beyond the Minister’s level at the MOH. The outcome achieved was exemplified through its work with NHA, normative costing, training, financial accounting systems and the specific recommendations by cited short-term consultants. Improved health information systems related to patient tracking and case management. The following outcomes were achieved: – Provided hardware and software for PHC health data – Designed, tested, trained, and provided one-year help desk for implementation – Assessed encounter-based MIDAS-2 system for use in PHC settings Assessment of the USAID/Armenia Primary Healthcare Reform Project 36 Accounting procedure manuals and the development of a standard chart of accounts for service providers that complies with international financial reporting standards. The achieved outcome was the development of accounting guidelines and training on typical PHC transactions. Administrative staff (including heads of marz health departments) and facility managers who have been trained in sound financial management, cost accounting, and governance. The achieved outcome was training of 140 PHC managers and 52 chief accountants. Strengthened procurement systems, facility-based contracting, and performance￾based contracting mechanisms being implemented by the SHA. Achievements included the following: – Support for the development of performance and quantity- and quality-based indicators – Introduction of elements of cost accounting in PHC, including some cost elements in 22 facilities Enhanced, demonstrated SHA capacity to analyze provider performance reimbursement. The achieved outcome was assessment of current contracting mechanisms between the SHA and medical facilities to improve and augment the SHA’s role as a purchaser. However, it is unclear if the SHA has the capacity to implement the desired performance reimbursement in a timely, rational, and transparent manner without unnecessary paper requirements and simultaneously minimizing abuse. NHA Working Group has instituted new standards for data collection and analysis, including the identification of new data sources. Achieved outcomes include conducting an international donors’ health expenditure survey and starting evaluation of equity and access to care for different subgroups to show differential impact on health expenditures. However, additional emphasis is needed on analysis and policy implications. Completion of feasibility study for national health insurance program. A project decision was made to postpone the study based on short-term consultant advice and with the concurrence of USAID. However, if conditions change in the future, such a study may be initiated. Additional or separate USAID funding should be considered for such a study. Facilities and information technology systems upgraded at targeted sites. The achieved outcome was providing facility renovation and IT systems for PHC. Pilot programs expanded to include more sites; data collected from pilots included in the development of costing and performance-based contracting standards. Achieved outcomes include completion of pilot-site programs on facilities’ strategic plans. In 92 PHC facilities, three-year plans were developed, 140 PHC managers were trained in performance￾based budgeting, and their knowledge was incorporated in recommendations currently under consideration to expand the effort of performance-based reimbursement nationally. A survey of PHC facilities was conducted, and the level of technology availability and the condition of equipment and procedures used were analyzed. Standards for facility service costing established, approved, and disseminated; standards for performance-based budgeting established, approved and disseminated. Achieved outcomes include development of models for 115 lab diagnostic services, Assessment of the USAID/Armenia Primary Healthcare Reform Project 37 development of performance standards (5 quality and 5 quantity), and obtaining the MOH agreement to include these in the system for reimbursement in 2008. Train facility staff (facility directors and accounting staff) in target marzes on sound financial management principles, as well as standards for costing and performance￾based budgeting. The outcome achieved was the training of over 200 PHC managers and accountants. Procurement of a total of 390 computers and related equipment, including software. The achieved outcome was the installation of 235 computers in 204 sites along with software for accounting, management, and OE tracking. Challenges and Constraints A basic constraint on rationalizing the financing of PHC is the internal functional capacity of the GOA to provide the timely, evidence-based analysis needed to underpin and support policy and procedure development without donor assistance. Major improvements have been made by PHCR in identifying issues, developing information systems, creating guidelines, training, and promoting community education and participation. PHCR has been effective in filling those functions; nevertheless, it remains an outside agency of the GOA, which can both help and hinder. Nevertheless, the effectiveness of both PHCR staff inputs and their short-term consultancies has been outstanding both in terms of technical specification and GOA acceptance. Multiple GOA entities expressed considerable regard for PHCR’s support in the financial area, particularly through model development and training. While budgeting for PHC requirements may be increasing and enacting necessary laws and specific regulatory requirements may contribute to limiting or constraining PHCR effectiveness, such limits were not unduly observed. Project achievements have provided support for both budget increases and, more specifically, for implementing procedures necessary for healthcare reform at the PHC level. Healthcare providers who may be negatively affected by planned remuneration changes may create a negative backlash to the envisioned changes when enacted. Both providers and service recipients may also be confused by the changes and the new information requirements, particularly if requirements are burdensome, or if feedback (for example, reimbursement, or improved understanding of individual care delivery successes, constraints, and failures) is not timely. GOA practices regarding health financing are initiated from the top, with the decrees or regulations approved by several national governmental organizations, with little if any participation by local implementing units, nor are necessary budgets additions included for implementation. Therefore, a challenge and constraint is to include local sensitivities and knowledge in the preparation of directives and to provide, in a timely manner, the funds to implement the directive. The PHCR Project can help the national organizations include local inputs and, if necessary, temporarily bridge the information gap to help explain appropriateness and rationality of each PHC directive. Furthermore, the project should also help GOA introduce the need for adequate budgeting including actual funding, and rational timing of implementation in order to avoid false expectations and improve the realism of their directives. The GOA should not issue PHC directives on health financing without necessary budgetary and actual funding to support implementation, particularly those directives adding changes and workload to local staff. Preparation of PHC directives on health financing should include participation of local Assessment of the USAID/Armenia Primary Healthcare Reform Project 38 implementing staff and agencies. Perhaps directive preparation should include Marz presentation with selective involvement of local authorities. The GOA flow of funds may limit enactment of plans for healthcare reform, but they do not affect PHCR contractual obligations. While it is clear how financing is provided by PHCR for renovation of healthcare facilities, the maintenance of said facilities must be financed by other sources, such as a portion of GOA reimbursements for PHC services or community support. PHCR has calculated FAP maintenance costs and alerted the MOH/SHA to this issue. While the depreciation of the U.S. dollar and the inflation of local construction costs will limit future activities, particularly planned renovation of rural facilities, specific impacts have not been concluded. Potentially, fewer or less complete renovations may result. While PHCR is examining these possibilities and no conclusions have yet been made, this is an issue that needs to be addressed. Support of MIDAS-2 implementation should be included in the future PHCR planning, although it is not clear that in-depth support is feasible under existing PHCR budget constraints. Adjustments and Modifications Balanced efforts. PHCR is supporting various GOA institutions in quantifying and analyzing financial requirements and procedures necessary in reform of the PHC sector, including the estimated costing and operation of reimbursement procedures for PHC services. However, timely and balanced progress must be made in all the components of PHCR as well as parallel components within their respective GOA agencies at both national and sub-marz levels for the impact to improve health outcomes. While such improvement across all needed reforms requires time, the GOA seems ready to make such reforms and has committed to hastening these reforms. Nevertheless, it remains unclear whether all the required procedures (for example, the operational details) will be promulgated and monitored in a timely manner in order for them to be effective, minimizing untoward impacts. NHA and the National Health Survey. Additional support to the respective GOA agencies is required to link and analyze the data for policy, program, implementation, and operation impacts on PHC services, particularly for rural population subgroups such as pregnant women, children, the disabled, and the elderly. PHCR and GOA agency interaction. While PHCR staff has considerable contact with their respective agency staff, that interaction is that of a consultant and trainer as opposed to an integrated team member. While this mode of interaction has been very effective in the short term, continued consideration must be given to institutionalization of these activities within the GOA at all levels— national, marz, and community. PHCR, with its excellent, hardworking local staff and its use of short-term consultants, has provided critical input to the GOA reform of PHC. Its efforts are accepted and viewed with considerable respect. Most of its products are even assimilated by the GOA. PHCR staff are not only responsive but anticipate needs, initiating actions needed to implement reforms. They indeed have become a partner with the GOA in the implementation of PHC reform activities. This partnership has been instrumental in the launching of this reform. However, the question must be asked: At what point does this partnership become enabling, allowing the GOA to delay or compensate for the development of internal capacity to fulfill PHCR functions? What does that mean for PHCR? It certainly should not stop its activities, but it should Assessment of the USAID/Armenia Primary Healthcare Reform Project 39 systematically, in all its endeavors, push for counterpart participation. For example, when PHCR staff are in the field, they should work with and not visit local stakeholders and always have a national or local counterpart with them, one who has specific responsibilities in the ongoing activity, an activity that is not PHCR’s alone, but is their counterparts’ activity and responsibility in which PHCR staff assists. Target budget adjustments. Project budget limitations resulting from monetary exchange and local inflation should be examined and targets adjusted where appropriate. Specifically, the selection of sites for renovation should include assessment of the long-term viability of the site function. Consideration during site assessment when considering planed activity in the southern marz should include factors such as the lack of accessibility by a minimally defined number of people and the potential for a site’s services to be expanded. Future transportation and waterworks improvements, such as those planned under the MCC, should contribute to new site selection criteria. In addition, the long-term viability of rural FAPs should be assessed. Consideration might be given to the possibility that future improved road conditions could enable mobile units to replace some fixed facility sites, particularly for preventive measures, as well as general health education. Specific additional USAID funding could be made to the GOA to pilot such mobile health activities with the participation of PHCR. Private sector promotion. Consideration should be given to how the private sector PHC activities will be promoted and linked to the public sector activities. Identification of areas where private provider facilities could be feasible would allow for public PHC investments in alternative locations. This may be a subject for PHCR activity, both with its internal staff and with short-term consultants within the next three-year period. However, care must be taken to obtain the active participation of both respective GOA agencies and potential private sector providers, as well as of service users. This is also an area where additional USAID funding may be required, perhaps separate to PHCR. Recommendations Strengthened institutional capacity to implement PHC reform (USAID intermediate result 3.2.1) states, ―Critical to sustaining the financial system development and its analysis is the ownership of the respective GOA agency and its ability to continue as well as to evolve or better the PHC healthcare financing aspects of reform.‖ GOA institutionalization, ownership, and capacity should be systematically addressed, with priorities in all activities and specific implementation plans incorporated in the coming three years of PHCR. Such planning actions should be undertaken jointly by the staff of PHCR and the respective GOA agencies at all levels, with stakeholders in both the public and private sectors. An option to consider is imbedding PHCR financial unit within an agency of the GOA, such as the SHA, with the goal of increasing the agency’s ownership of project activities and achievements and institutionalizing its capacity to continue these functions. This could be accomplished in a staged manner. A decree may be needed for a special office, with a staged GOA budget for space, logistical support, and added staff. USAID support, while diminished after three years, could continue, with the number of technical staff reduced as appropriate with the increase in capacity of the GOA agency, and possibly a mechanism in place to provide short-term consultants on an as-needed basis. The GOA’s support of private sector provision of healthcare delivery has been mentioned and a guideline on procedural requirements has been drafted by PHCR. However, the specifics of the Assessment of the USAID/Armenia Primary Healthcare Reform Project 40 GOA promotion and assistance are not known. PHCR may be able to support the analysis and quantification of the need for the private sector provision of PHC, as well as the necessary specific support and public sector linkages. A monitoring system that provides feedback and inhibits abuse of both OE and performance￾based reimbursement is necessary and may be an area in which PHCR can provide technical input, particularly for PHC centers. The impact of the introduction of OE, performance, and quantity- and quality-based reimbursements on specific high-risk populations—such as pregnant women, infants and children, the elderly, the disabled, and others identified by the GOA as health priority groups—should be assessed and appropriate measures taken to ensure the effective delivery of needed services. Currently, PHC providers receive double the standard reimbursement for serving children rather than the adult population. Rationalization of such approach and expansion to other vulnerable or high-risk groups may need to be analyzed and specific reimbursement modifications proposed to the GOA. For example, PHCR can establish the guidelines for rational and objective reimbursement differentials among selected subgroups reflecting national priorities—such as reproductive-age and pregnant women—jointly with the respective MOH/SHA agencies. Currently, differential reimbursements10 are being considered, using 2006 hospital data that provides consistently higher differential rates for males than females for all age categories. Such statistics in no way reflect use and demand in local PHC facilities and should not be used. PHCR should consider undertaking a specific study to develop differential coefficients to adjust for age and sex for PHC provider reimbursement. The impact of OE and performance-based reimbursement (if enacted) should be assessed for both doctors and nurses, particularly for family and community providers. PHCR could establish the guidelines to assess and monitor such impacts, particularly as they pertain to community health centers and most particularly those in rural areas, and engage its respective GOA agency in order to institutionalize these efforts within a specific agency. COMPONENT 6: PUBLIC EDUCATION, HEALTH PROMOTION, AND DISEASE PREVENTION According to the PHCR contract, the expected outcomes for this component were the following: Campaigns promoting healthy behaviors, resulting in improved health-seeking behavior of the public Improved understanding of OE and increased acceptance of FM providers Local communities engaged in mobilization efforts to promote healthy lifestyles The finding of this evaluation team is that PHCR public education team has delivered even more than was promised. The need is great. What is required entails many more subjects in health education as well as the mobilization more community groups, expanding to ambulatory level. Achievements 10 Differential reimbursement (per age and sex) is not implemented yet by the SHA. The Project developed different coefficients to adjust for age and sex, making an assumption that the hospital admissions data can serve a s good predictor of population PHC needs. The 2006 hospital admissions data provided by the SHA was the closest proxy available. Assessment of the USAID/Armenia Primary Healthcare Reform Project 41 PHCR public education team has been an active group, developing and promoting public education, health promotion, and disease prevention projects, including: OE advertising and doctor training about OE and health reform Training reporters and media representatives on behavior change in health ToT programs for local NGOs to teach community mobilizing and building capacity OE education to stakeholders OE roll-out advertising, including mass distribution of text messages to one million subscribers Training to doctors and other stakeholders on health reform Training reporters and media representatives on health behavior change (51 journalists in Zones 1 and 2) Development of ToT on adult learning strategies ToT programs for local NGOs to teach community mobilization and capacity building, a detailed training building on already developed and successful programs Grants to local NGOs for community mobilization Descriptions of OE, BBP, and pharmaceuticals available free of charge—posted on the wall of every FAP visited and delivered to all PHCs nationwide ToT on community education packages for three subjects Public information booklets, leaflets, brochures, posters, DVDs, and TV and radio public service announcements 57 CHCs formed and received mobilization training PHCR generated real excitement in community groups about health, so much so that groups were asking for libraries of health information and training in more topics, including one mayor who remarked, ―Of course we are all interested in health. Who would not be interested in subjects about health?!‖ PHCR’s OE and BBP descriptions were posted on the walls of every FAP, ambulatory, and polyclinic visited, whether or not the clinic was a PHCR renovation project. Through the creative leadership of Ruzanna Melyan, this project has even succeeded negotiating free public service messages—virtually unheard of in Armenia. Challenges and Constraints Implicit in all USAID projects is the sustainability of the project’s impact or interventions after USAID is no longer involved, which suggests the need for ownership by the country involved. During the evaluation team’s visit, only one anecdotal concern cropped up: OE had been identified by the patient recruiter as a USAID program in the recruitment of a person who happened to be an MOH worker. MOH health promotion capacity. An overriding challenge exists for this PHCR team to create sustainability and local capacity when the MOH does not have a division of health education or health promotion. Although the MOH has a media advisor (whom this PHCR team includes regularly in its activities), no person or division is available to carry the torch once PHCR is completed. Presumably, this will be fine for the publicizing of health reform changes, since most of Assessment of the USAID/Armenia Primary Healthcare Reform Project 42 the reform will have been completed, but what about community health education and public information on health system changes after PHCR is completed? Funding. Constraints are largely financial. Flow of funds has not limited implementation, but it does limit the ability to continuously connect with the CHCs in the selected marz villages, which may decrease some of the enthusiasm that is currently present in the villages. Presumably, more funding could provide for a more comprehensive follow up on the initial village data collection. For example, how can one know if the CHC is effectively communicating its health messages to rest of the community without both a baseline and a follow-up questionnaire for the community (not just members of the CHC) on its topical health knowledge? Health-seeking behavior. Improved health-seeking behavior, as measured by increased medical visits, has been evident in Armenia with the provision of free primary care (as specified in the BBP) and probably cannot be attributed to PHCR. Finding an appropriate measure for evaluation may present a challenge. It should also be noted that one of the measurements used to denote a lack of health-seeking behavior was the failure to see a doctor for an upper respiratory infection (URI). Given the large number of URIs occurring in any population over the course of the year, it is not necessarily a good thing that more patients visit, unless these are high risk patients, such as the very young, very old, diabetic, asthmatics, and so forth. The percentage of visits for URIs in Armenia should not be considered bad if low, or good if high. Experiences from Medicaid (insurance for free medical care) in the U.S. shows doctor-dependence to be high, so much that one concludes population self-care for a URI is minimal, and the dependence results in more unnecessary antibiotic prescriptions (a URI, if uncomplicated, requires no antibiotics). Excess treatment with antibiotics is associated with increased drug resistance. Adjustments and Modifications This component has not had significant adjustments over the course of the project. The shift to FAP health post renovation may have made PHCR’s job easier, as it may be easier to mobilize smaller communities than the larger ones where ambulatories and polyclinics exist. Now is the time to think of modifications that will best use the considerable skills of the public education team. Recommendations Decreasing USAID logo size. Currently, the USAID logo is much larger than that of the MOH. Addressing the logo size may help with some of the ownership confusion discussed above. It might also help to have the ToT and the training on patient recruitment for OE performed solely by government employees, and not PHCR. Dissemination of health education. The public education team appears to have met all of its stated objectives and outcomes to date. However, some of the evaluation components need to be developed further, such as how to assess whether or not health education information is being disseminated to the rest of the village. Household survey. Possibly the only indicator for measuring the goal of improved health-seeking behavior within the renovated FAPs and mobilized CHCs may be the household survey, where Assessment of the USAID/Armenia Primary Healthcare Reform Project 43 intervention villages can be compared to non-intervention villages, as this would eliminate the bias of increased health-seeking behavior resulting from the free benefit package. Continuing CHC support. Some way of connecting with the CHCs over the years might be helpful (for example, an annual new topic or a newsletter), so the CHCs do not feel isolated once the renovation is completed. A methodology to assess the effectiveness and long-term viability of CHCs could be useful. CHC mobilization skills. The CHC mobilization skills should be considered part of community nurse education. However, some participants have been out of town at PHCR-sponsored marz community nursing training and have missed out on this valuable opportunity to develop their community leadership skills (a hands-on component of the very education in which they are participating). Promoting exclusive breastfeeding for the first six months. Through its CHC PHCR team could devote more effort, thought, and enthusiasm, to the promotion of exclusive breastfeeding, among other important MCH tasks (e.g. recognition of danger signs in pregnancy and postpartum period, recognition of newborn and infant death signs, benefits of family planning, etc.) . Among the ambulatories visited, it was clear that mobilization of their communities around health would be a great asset. This program could further help other PHCR activities by: Ensuring that FAP nurses do not miss mobilization training in their communities Developing a comprehensive breastfeeding promotion and education program or module for the CHCs (and nationwide) Expanding CHC mobilization to include NOVA-renovated FAPs and ambulatories Further emphasis on evaluation of promotional efforts, such as CHC dissemination of health information to the rest of the village (for example, the household survey looks at smoking attitudes) and of factors contributing to the viability of CHCs will be useful. MONITORING AND EVALUATION During these next three years, studies by PHCR’s monitoring unit designed to measure changes in health post utilization and influencing factors will undoubtedly provide a useful basis for evaluating project interventions and for developing guidance for future efforts. In addition to those parameters already being monitored, consideration should be given to monitoring the following topics: Monitoring OE registrants. A monitoring system that provides registrant feedback and inhibits abuse of both OE numbers and performance-based reimbursement is necessary and is an area in which PHCR could provide technical input, particularly for PHC centers. As mentioned preciously, the use of a numerical identifier for each population member could facilitate the development of a monitoring system. The impact of the introduction of OE, performance, and quantity- and quality￾based reimbursements on specific high-risk populations—such as pregnant women, infants and children, the elderly and others—should be assessed and appropriate measures taken to ensure the delivery of needed services. Currently, children are receiving double the standard amount reimbursed for the general population. Rationalization of such an approach and expansion to other vulnerable or high-risk groups may need to be analyzed and specific reimbursement modifications proposed to the GOA. PHCR can establish the guidelines for a rational and objective Assessment of the USAID/Armenia Primary Healthcare Reform Project 44 reimbursement differential among selected subgroups reflecting national priorities jointly with the respective MOH/SHA agencies. Measuring community satisfaction. The ability to measure a community’s satisfaction with its health services is an important step to improving quality of care and competition through client demand. Under the current work plan, M&E plans include client satisfaction surveys within select PHC facilities to determine the impact of project activities. However, this survey alone would not capture the possible interaction between quality of care and community education and mobilization activities being carried out. For example, one possible outcome is a decreased satisfaction of health post performance even after nurses have been trained and facilities have been upgraded. This result could be due to an increase in community demand for quality services as a result of community education activities. Thus, community awareness and demand outpaces the increased capacity of facilities and staff to deliver those services through FAPs. A client satisfaction survey would not be able to capture this nuance; however, a KAP survey among the communities where health posts exist would be able to capture these issues and inform the project of it success not only in improving quality of care, but also communities’ knowledge of healthy behaviors and service standards, both of which play important roles in improving health and health services nationwide. Measuring community health knowledge. For correct evaluation of the effectiveness of mobilization and community health education, PHCR needs to assess general community health knowledge before the community education component and afterwards. The household survey has questions on attitudes toward smoking and alcohol, as well as healthy lifestyles, which can be used for this evaluation. It is important to have community education include a module about the harmful effects of smoking—in which case the household survey will have a small amount of evaluation potential for community health education. Assessment of the USAID/Armenia Primary Healthcare Reform Project 45 III. RECOMMENDATIONS In summary, the assessment team recommends the following: FOR USAID Contract extension. Based on the project’s current exceptional performance, the assessment team recommends that USAID/Armenia exercise further option years according to Mission procedures for option year authorization in so much as PHCR has continued high performance and effective collaboration with the Government of Armenia; and in accordance with available Mission funding levels, Mission priorities, and the new Mission strategy. Local project leadership. The contract provides for ―A planned transfer of project management to local professionals in the fourth year of the project, in order to enhance Armenian management and reduce the cost of implementation.‖ The assessment team supports the intent of this provision but recommends that it not be implemented. There is much to do in the next three years, and a change in leadership would be unnecessarily disruptive. This issue and its budgetary implications should be addressed and resolved by USAID and PHCR at the earliest opportunity. Fewer health posts. If necessary to stay within budget, PHCR should be authorized by USAID to maintain its standards but reduce the targeted number of health posts and ambulatories to be renovated. Retaining the target but reducing the amount or quality of the work done in each facility is not advisable. Coordination of community development strategies. USAID should encourage PHCR and Project NOVA to collaborate with USAID’s democracy and governance activities to encourage community-based strategic plans (to be determined) and ensure that the issue of FAP sustainability is a key component of these strategies. Donor coordination. Communication and collaboration among donors and with the MOH has been strong, especially at the program level. Coordination meetings serve two major purposes—to ensure greater congruence of donor programming with government priorities, and to build in-country capacities to deal with donors more effectively. The assessment team recommends that the monthly high-level MOH/donor coordination meetings be reactivated. For PHCR Emphasis on capacity building. PHCR’s expertise has been essential in supporting the PHC reform agenda and in building enthusiastic support and understanding of the complex issues being addressed. Most critically, and, PHCR should continue to emphasize even more strongly capacity building across all components, strengthening the abilities of the MOH, the SHA, and other in-country planners and decision-makers to further develop and sustain the reforms now underway. Transfer of expertise. While PHCR’s achievements are significant, a number of challenges remain. Most critically, cutting across all components, is the need to strengthen further the in-country capacity to maintain and further develop the health reform agenda and to Assessment of the USAID/Armenia Primary Healthcare Reform Project 46 continue the numerous initiatives so ably launched during these last few years. While there are many possible solutions to this transfer of knowledge and skills, one possible solution is to embed PHCR staff within the MOH to increase more direct involvement and knowledge transfer. Another possibility is to create an NGO that provides technical expertise to the GOA. These and other options should be explored further. Health post sustainability. In light of their vulnerability, PHCR’s multi-intervention strategy—renovation, equipment, staff training, financing, and community organization— offers the best hope for insuring the sustainability of health posts. In addition, PHCR should help develop the capacity within the MOH and marz authorities to support and maintain these initiatives on an ongoing basis. Selection of health posts. PHCR must be mindful that the existing network of health posts is vulnerable. The criteria used to select the health posts and ambulatories that will be upgraded over the next three years should include consideration of those factors that are likely to influence long-term viability—inaccessibility, community size and support, financing, staff availability, and the like. Curricula and guidelines development. A conference with all stakeholders needs to present various examples of curricula and guidelines to determine nomenclature, responsibility, and who approves what. PHCR should continue to use its cardiovascular training package methodology to reach national consensus on the teaching of other modules in the FM curriculum. CME should be developed on a yearly basis by using PHCR training packages with specific mandatory subjects. Continuing Medical Education (CME). PHCR should support the development of CME by using PHCR training packages, along with specific priority (possibly mandatory) subjects on a yearly basis. Reflecting rural capabilities in the curriculum. PHCR medical staff and medical consultants need to spend a few days in each zone shadowing FM doctors, FNs, community FNs, and patients in day-to-day work environments. Updated curricula and guidelines should reflect rural capabilities and include rural practitioners in the process. Once these tools are updated, they should be tested by returning to the rural areas and observing what happens when the tools are used. If necessary, the tools should be changed to reflect what is learned. Blood pressure training. Doctors and nurses, even those who have already been in the training, need to be systematically trained in taking blood pressure and even required to receive certification. The American Heart Association has a training certification program that could be developed into an Armenian training package for FM and FN retraining and CME. Nursing curriculum development. A specific training package needs to be developed that includes, at minimum, all of the following: – The entire NOVA health post training – Education on available medicine in FAPs, after first standardizing what medicine is available – Participation in CHC mobilization Assessment of the USAID/Armenia Primary Healthcare Reform Project 47 Community nursing specialty. In the long term, PHCR should assist the MOH in the GOA declaration of a community nursing specialty, expand the curriculum, and institutionalize community nurse training within a nursing college. FAP nurse/doctor relationship. There is a role for PHCR to try to support and, if possible, formalize a constructive relationship between the FAP and its associated doctor, which could include the following strategies: – FAP nurses should line up patients to see doctors for specific reasons at scheduled appointment times. – Doctors should be encouraged to bring medicine to the FAP, told beforehand by FAP nurses which people they will see, why they will see them, and which medicines they are on, if any. – Home visits by the doctor should be limited to the disabled. Maternal and child health. The community nurse needs a large component of MCH, which could be provided by blending the NOVA training into PHCR training. Attention to changing cultural attitudes towards breastfeeding to encourage a longer duration of breastfeeding is important, and PHCR’s community training program should be used to get this message out. Consideration needs to be given to economic support for use of forms other than abortion for birth control. Integration of FM. While progress has been made in training family doctors and nurses, much still needs to be done to build acceptance of their professional role and identity, their acceptance as providers of valued health services in the community, and the integration of their practices into the system of healthcare providers. OE capacity building. Institutional capacity should be strengthened to promote PHC reform, a USAID intermediate result. Critical to sustaining the launching, continuation, and monitoring of OE is the ownership of the respective GOA agency and its ability to continue as well as to evolve and improve the PHC OE activities necessary for continuous reform. While GOA ownership and capacity improvement is currently incorporated in most PHCR activities, these activities should be reviewed and systematically addressed in the next three years of PHCR. Each PHCR action should be planned and implemented jointly by the staff of PHCR and the respective GOA agency, with provider and community participation. One option is to imbed PHCR OE activities within an agency of the GOA, such as the MOH/SHA and the marzpetrans, with the goal of increasing the respective agency’s ownership of PHCR activities and achievements and institutionalizing its capacity to continue these functions. This could be accomplished in a staged manner. A decree may be needed for a special office, with a staged GOA budget for space, logistical support, and added staff. USAID support, while diminished after three years, could possibility continue with a technical staff level that is reduced as the capacity of the GOA agency increases. Monitoring OE. A monitoring system that provides registrant feedback and inhibits abuse of both the OE numbers of enrollees and the tracking of performance-based reimbursement is necessary and may be an area in which PHCR can provide technical input, particularly for PHC centers. An exemplar issue that needs addressing is the use of a numerical identifier for each citizen, which could facilitate the development of a monitoring system. Assessment of the USAID/Armenia Primary Healthcare Reform Project 48 Quality Assurance self-assessment. The self-assessment tool currently proposed in the Quality Assurance Strategy does not qualify in terms of medical care or analysis. This appears to be an underuse of the potential in self-assessment. QOC in the U.S. is variable, but it generally consists of physicians—and even FM residents as part of their residencies— reviewing their own charts. Clinic-wide quality indicators. QOC can be stimulated by performance-based reimbursement, but as listed in Annex 3, the realities of rural practice make several of the chosen indicators not practical or useful. Making use of the QOC indicators by having performance-based payment is reasonable, in spite of the limited number of indicators that can be chosen. Medical charting. Good medical charting is an important part of QOC, but currently, the charting is minimal and it is difficult to understand quality. The recently developed new medical chart will help assess quality. Financial management capacity building. Strengthened institutional capacity to implement PHC reform, a USAID intermediate result 3.2.1 states, ―Critical to sustaining the financial system development and its analysis is the ownership of the respective GOA agency and its ability to continue as well as to evolve or better the PHC healthcare financing aspects of reform.‖ How GOA institutionalization, ownership, and capacity are achieved should be systematically addressed with priority in all activities and specific implementation plans incorporated in the coming three years of PHCR. Such planning action should be undertaken jointly by the staff of PHCR and the respective GOA agencies along with respective stakeholders. An option which may be considered is to embed PHCR financial unit within an agency of the GOA such as the SHA with the goal of increasing the respective agency’s ownership of the project activities and achievements and institutionalize its capacity to continue these functions. This could be accomplished in a staged manner. Perhaps a decree would be needed for a special office, along with a staged GOA budget for space and logistical support and added staff. USAID support while perhaps diminished after three years could continue with support of a technical staff level reduced as appropriate with the increase the capacity of the GOA agency. Provision for private sector. The GOA support of private sector provision of healthcare delivery has been mentioned, and a guideline on procedural requirement has been drafted by PHCR, but the specifics about GOA promotion and assistance are not known. PHCR could support both the analysis and quantification of the need for the private sector provision of PHC and the specific support and linkages to the public sector that would be necessary to achieve same. Monitoring performance-based reimbursement. The impact of reimbursements on specific high-risk populations—such as pregnant women, infants and children, the elderly, and other groups—should be assessed and appropriate measures should be taken to insure that they receive their services and that incentives are provided for the delivery of needed services. Currently, for children, PHC providers are receiving double the standard reimbursement for children than they receive for the adult population. Rationalization of such approaches and expansion to other vulnerable or high-risk groups may need to be analyzed and specific reimbursement modifications proposed to the GOA. PHCR can establish the guidelines for a rational and objective reimbursement differential among Assessment of the USAID/Armenia Primary Healthcare Reform Project 49 selected subgroups reflecting national priorities jointly with the respective MOH and SHA agencies. Currently, differential reimbursements are being considered, using 2006 hospital data that provides consistently higher differential rates for males than females for all age categories. Such statistics in no way reflect the use and demand in local PHC facilities and should not be used.11 Assessing the impact of OE and performance-based reimbursement. The impact of OE and performance-based reimbursement (if enacted) should be assessed for both doctors and nurses, particularly for family and community providers. The project could establish guidelines to assess and monitor such impacts—particularly as they pertain to community health centers and, most particularly, those in rural areas—and engage its respective GOA agencies in order to institutionalize these efforts within a specific agency. Community development of the ambulatory health post network. Community development efforts have been very effective in mobilizing community understanding and involvement, and have further potential to enhance harmony and coordination among all of the communities involved in a local network. Decreasing USAID logo size. Currently, the USAID logo is much larger than that of the MOH. Adjusting logo size might help with some of the ownership confusion discussed above. It might also help to have the ToT and the training on patient recruitment for OE performed solely by government employees, and not PHCR. Dissemination of health education. The public education team appears to have met all of its stated objectives and outcomes to date. However, some of the evaluation components need to be developed further, such as how to assess whether or not health education information is being dissemination to the rest of the village. Household survey. Possibly the only indicator for measuring the goal of improved health￾seeking behavior within the renovated FAPs and mobilized CHCs may be the household survey, where intervention villages can be compared to non-intervention villages, as this would eliminate the bias of increasing health-seeking due to the free benefit package. Continuing CHC support. Some way of connecting with the CHCs over the years might be helpful (for example, an annual new topic or a newsletter), so the CHCs do not feel isolated once the renovation is completed. A methodology to assess the effectiveness and long-term viability of CHCs could be useful. CHC mobilization skills. The CHC mobilization skills should be considered part of community nurse education. However, come participants have been out of town at PHCR￾sponsored marz community nursing training and have missed out on this valuable opportunity to develop their community leadership skills (a hands-on component of the very education in which they are participating). 11 Differential reimbursement (per age and sex) is not implemented yet by the SHA. The project developed different coefficients to adjust for age and sex, making an assumption that the hospital admissions data can serve a s good predictor of population PHC needs. The 2006 hospital admissions data provided by the SHA was the closest proxy available. (Source: PHCR) Assessment of the USAID/Armenia Primary Healthcare Reform Project 50 Assessment of the USAID/Armenia Primary Healthcare Reform Project 51 ANNEX 1: STATEMENT OF WORK Assessment of the Primary Healthcare Reform Project (USAID Revised GH Tech: May 16, 2007) 1. PURPOSE This assessment will provide input to the USAID/Armenia Social Reform Office and the mission in exercising option years for the Primary Healthcare Reform Project (PHCR). PHCR is a five-year (Sept. 2005 to Sept. 2010), $17 million project and represents the largest investment of USAID/Armenia in the health sector. PHCR is a performance-based, level of effort contract with two base years and three one￾year option periods. The goal of PHCR project assessment is to analyze the general performance of the project including successes, constraints and failures, and to make managerial and programmatic recommendations for the option year periods. It should also make recommendations for future USAID participation in the health sector reform arena. This assessment will contribute to the analytical understanding of USAID/Armenia’s engagement in health sector reforms, and will serve as a guide for project activities for the remainder of the project. 2. BACKGROUND During the Soviet era, Armenians had the longest life expectancy and one of the best developed healthcare systems of all the Soviet Republics. However, dire post-independence economic and social conditions presented a new context for service provision. The healthcare system was characterized by weak governmental commitment and financial support; a focus on expensive, tertiary curative care rather than primary and preventive medicine; high out-of-pocket expenditures on health; healthcare providers with inadequate clinical and managerial skills; and overstaffed facilities in disrepair with outdated equipment and insufficient supplies. A scarcity of resources constrained the government’s ability to implement reforms. Moreover, the historical legacy of an authoritarian, top-down approach to healthcare administration discouraged individual initiative and stifled management and institutional development at the regional and local levels. Considerable donor assistance and healthcare reform initiatives over the last 15 years have improved the situation somewhat, but most of the above mentioned conditions still prevail. Since independence the Ministry of Health has made some progress in moving away from tertiary care to Primary Healthcare (PHC). The National Strategy on Primary Healthcare in Armenia was approved in late 2003.12 The Strategy presented the country’s vision for PHC reform implementation and service delivery improvement. The main components comprise accessibility, equity, comprehensiveness, continuity of care, and coordination between different levels of the healthcare system. PHC in Armenia includes MCH care, immunization and treatment of common and infectious diseases, the provision of MCH care, and dispensing of necessary drugs and basic curative care. Today, Armenian’s suffer from the diseases common in developed countries: cardiovascular disease, cancer, and accidents, and from those common in developing countries such diarrhea and ARIs. The 2005 ADHS data suggest a decline in infant mortality (from 36 per 1,000 live births in 2000 to 26 in 2005); however, the results were not statistically significant so it is difficult to conclude a definite decline. In addition, many key MCH indicators remained stable or worsened from the 2000 to the 2005 DHS. Only 60 percent of children 12-23 months have received the six vaccinations recommended by WHO, compared to 76 in 2000. 37 percent of children under four months of age were exclusively breastfed compared to 45 percent in 2000. In 2000, the total contraceptive prevalence was 61 percent, but only 22 percent for modern methods. In 2005, contraceptive prevalence was 54 percent and the use of modern methods was 20 percent. These indicators compare poorly with data from other countries with similar per capita incomes and education status. 12 RA Governmental Decree No. 1533, on December 6, 2003. Assessment of the USAID/Armenia Primary Healthcare Reform Project 52 3. USAID/ARMENIA ASSISTANCE IN PHC REFORMS Since the 1990s, USAID/Armenia has supported initiatives to improve service delivery and management of health programs that directly benefit Armenia’s most vulnerable populations. In August 2000, USAID began its five-year Armenia Social Transition Program (ASTP) to assist the MOH to develop, test and implement a series of social protection initiatives including PHC reform. USAID/ASTP’s health sector activities included: design and implementation of the organizational and regulatory framework for family medicine as the predominant specialty for PHC; development and testing of open enrollment for the population to receive state-funded PHC services; support for MOH/State Health Agency (SHA) and health facility information systems; better targeting of vulnerable populations; reducing corruption through transparent contracting, cost accounting and financial management practices; and laying the groundwork for sustainable, national PHC coverage. USAID/ASTP's successes include: ASTP pilot sites formally recognized as national health system pilots; a new, unified curriculum for family medicine training developed and adopted; family medicine departments established; and population enrollment and quality improvement programs introduced in pilot sites. Moreover, the MOH's Primary Healthcare Strategy (for 2003-2008) and the recent National Health Policy draft document (for 2004-2015) incorporate many of the strategies being promoted and piloted under ASTP. To continue USAID’s PHC reform efforts, USAID initiated PHCR Project in 2005 as a follow-on to ASTP. PHCR builds on the healthcare reform successes of ASTP, and aims to expand key reform efforts across the country. 4. PHCR PROJECT PHCR supports the health sector reform efforts of the Ministry of Health to increase utilization of sustainable, high-quality primary healthcare services. The project has six main components: Reforming healthcare systems, policies, and procedures, and extending reforms nationwide Building clinical service capabilities through a Family Medicine approach by supporting Family Medicine Group Practices, updating curriculum and clinical standards, and training doctors and nurses in Family Medicine Improving quality of care by establishing provider licensure and facility accreditation procedures, and by putting in place state-of-the-art quality improvement methodologies. Increasing the equitable and efficient distribution of health resources through healthcare financing strategies and the use of National Health Accounts Building a more customer-service oriented system of clinic and provider choice through open enrollment Fostering improved health-seeking behavior through public health education and health promotion activities, including national campaigns on open enrollment and family medicine. PHCR team consists of Emerging Markets Group (Prime), IntraHealth International, Overseas Strategic Consulting, Ltd., Social Sectors Development Strategies (recently withdrew from the consortium), and American University of Armenia’s Centre for Health Services Research. ATTACHMENT: Table #1 at the end of this document shows main PHCR achievements and current activities. Assessment of the USAID/Armenia Primary Healthcare Reform Project 53 5. ASSESSMENT GOAL The goal of PHCR project assessment is to analyze the general performance of the project including: successes, constraints and failures, and to make managerial and programmatic recommendations for the option-year periods. It should also make recommendations for future USAID participation in the health sector reform arena. 6. METHODOLOGY AND TASKS The evaluators should consider a range of possible methods and approaches to collecting and analyzing the information which will be required to assess the evaluation goal above. Data collection methodologies will be discussed with and approved by USAID prior to the start of the assessment. To the extent possible, the approach taken to the assessment should be positive and participatory. The outcome should be a summary of progress to date and a set of recommendations to improve performance and ultimate impact. Tasks under the assessment include, but are not limited to: Review background materials such as the following: PHCR contract PHCR quarterly reports PHCR Workplan and PMP PHCR working products (e.g. training modules, IEC materials, etc) ASTP technical reports MOH orders Relevant governmental decrees USAID/Armenia Country Strategic Plan and FY07 OP MOH strategy paper on Primary Care development) RPM + Study on prescribing practices and drug management ADHS 2000 and 2005 Visit project implementation sites in Yerevan and two regions (Marzes) (Lori and Shirak) Collect information from key stakeholders: MOH, National Institute of Health (NIH), SHA, other international donors (e.g. WB, WHO). USAID-funded projects, clients and MOH service providers. Discussions with PHCR local office staff, EMG and other PHCR team headquarters’ representatives as well as USAID/Armenia team. In consultation with USAID, the assessment team should develop a list of questions that should be addressed in the assessment report. The list should include but is not limited to the following: For each project component listed below, answer the following questions: 1) Expansion of Primary Healthcare (PHC) reforms, 2) Family Medicine (FM), 3) Open Enrollment (OE), 4) Quality of Care (QOC), 5) Healthcare Financing (HF), 6) Public Education and Disease Prevention (PE)) What has been achieved and is PHCR on track towards achieving its end-of-project objectives and goals? What are the constraints or failures (if any)? Will this component have the desired impact? Should this component be modified in any way? Should the end of project targets be modified? Assessment of the USAID/Armenia Primary Healthcare Reform Project 54 Overall, to what extent are PHCR activities benefiting the project beneficiaries such as the MOH, National Institute of Health, State Health Agency and Yerevan municipality/Marz level Health Departments and patients? Are the contract deliverables realistic (or modest) with regards to USAID contract resources? Should they be modified in any way? Are there any recommendations for improved management of the project, either on the part of EMG or USAID. Recommendations would focus on management changes that would enhance the performance of the project. What are the team’s recommendations regarding the Mission’s options to extend PHCR activities up to September 2010? Component 1: Expansion of Reforms Are the health post renovations the most cost effective way of improving services in rural villages? Are villages, nurses, and ambulatories satisfied with the health post renovations? Component 2: Family Medicine Given the overlap with the WB funded Health Sector Modernization Project, the focus of PHCR in family medicine changed significantly from the contract SOW. Is the new approach to training the most likely to have an impact on public health? Are the topic areas appropriate? Is the project focusing sufficient efforts on the MAIN areas of morbidity and mortality in primary healthcare? Do they need to focus more on maternal and child health? Is the Unified Family Nurses Curriculum appropriate for rural nurses? Component 3: Open Enrollment Are sufficient resources (including STTA) being devoted to such a large undertaking? Do key stakeholders at the Ministry of Health appear ready and capable of implementing OE? Component 4: Quality of Care Is the implementation of the quality of care package feasible? Is it likely to have the desired impact of improving quality of care? Component 5: Healthcare Financing In order for Open Enrollment to have the desired impact of creating competition among providers by paying them based on the number of patients enrolled, the financing mechanism has to be in place. Does the team feel that the State Health Agency is on board with this and that the system is in place to do this? Component 6: Public Education Are the community mobilization efforts on track? What is the quality of the NGO and community mobilizer efforts? What is the quality of the education packages? How can the IEC materials developed by the project have an impact beyond the 100 communities targeted for community mobilization efforts? Management Are they any constraints to smooth implementation of the project in terms of financing, staffing etc? Are weekly meetings with the CTO sufficient? 7. TEAM COMPOSITION AND TEAM PLANNING MEETING The team should consist of three health experts: 1) a health finance specialist 2) a public health specialist (preferably a clinician) who is familiar with Family Medicine approaches in PHC and 3) a health policy specialist. Experts from USAID/Washington may also participate. Members of the USAID/Armenia and PHCR Project will also be available to the team as necessary. Qualifications: Regional experience would be a plus Russian and/or Armenian language skills Experience in implementing healthcare reform and in the project components such as OE, quality etc. Assessment of the USAID/Armenia Primary Healthcare Reform Project 55 Experience implementing projects and doing evaluations is necessary. Communication and writing skills are essential. The team will conduct a 2-day team planning meeting (TPM) upon arrival in Armenia and before starting the in-country portion of the assessment. The TPM will review and clarify any questions on the assessment SOW, draft an initial work plan, develop a data collection plan, finalize the assessment questions, develop the assessment report table of contents, clarify team members’ roles, and assign drafting responsibilities for the assessment report. The TPM outcomes will be shared with USAID/Armenia and the health team will participate in sections of the TPM. A local facilitator can be hired and the TPM can be held at PHCR. 8. TIMELINE AND LEVEL OF EFFORT USAID/Armenia anticipates that the entire assessment will be completed within 3 to 4 weeks. This includes preparation days, in-country work in Yerevan and in two regions (marzes) as well as drafting and finalizing the assessment report. If possible, the assessment should take place in mid-August (o/a August 20 for TPM). We expect that in-country work will take about 3 weeks. A 6 days work week is authorized while in-country. Level of Effort The estimated level of effort (LOE) for this assessment will be as follows: Document review and background preparation 3 days Travel days 3 days TPM 2 days Meetings with key stakeholders in Yerevan 5 days Field work (6 days field + 2 days finalization) 6 days Draft report 5 days Revisions and final report 4 days (TL); 1 day (team) Total LOE for each team member is estimated as follows: Team leader (health finance specialist) 30 person days Public health specialist 28 person days Health policy specialist 28 person days Local facilitator 3 person days Interpreter(s) TBD Total LOE: 89 9. LOGISTICS This assessment will be carried out by the GHTECH Project. The contractor should provide all other logistical arrangements such as in-country travel, airport pick-up, lodging and interpreters. PHCR project Deputy Chief of Party will be the point person at PHCR for the assessment team and will coordinate field visit arrangements and will arrange an appropriate escort for the assessment team during their field trips and meetings with ROA counterparts, if needed. PHCR will also provide administrative support in arranging for meetings. They will provide transport where practical. USAID/Armenia will provide key documents and materials for reading and help arrange the in-briefing and debriefing. PHCR CTO and Activity Manager will participate in the assessment as much as possible. Exact participation will be determined during the TPM but someone from USAID will participate in key meetings with the Ministry of Health and other stakeholders and at least some field visits. USAID/Armenia point of contact: Assessment of the USAID/Armenia Primary Healthcare Reform Project 56 Ann Hirschey (thru July 5th when on Home Leave) – (Operator) 374-10-46-47-00, x 4275 (Direct) 374-10-49-42-75 Email: ahirschey@usaid.gov Ruben Jamalyan (after July 5th) 10. DELIVERABLES 1. Present USAID with the framework for the evaluation on day three of the assessment. This will include the materials produced during the TPM. 2. Draft Report: The first draft of the final assessment report will be due at the end of the team’s visit (format: A4, font type: Times New Roman, font size 12). The draft report will include key findings and recommendations for mission review 3. Partner debriefings: The team will conduct one mid-term and two final debriefings. The mid-term debriefing will take place after the field visits to discuss preliminary findings with USAID. The first debriefing will be with USAID and the second with the larger stakeholder community (e.g. PHCR, and others TBD). The debriefing should present key findings and recommendations in a power point format. 4. Final Report: USAID/Armenia and PHCR will provide the team with comments on the draft report within 10 days of receiving it. The Final report will be due within 10 days after the team receives comments from USAID/Armenia and PHCR. USAID/Armenia requests both electronic vision of the final report (Microsoft Word or PDF format) as well as four hardcopies of the report. Draft Table of Contents Executive Summary Assessment Findings Overall Assessment Components 1 Components 2 Components 3 Components 4 Components 5 Components 6 Other observations Management Findings Observations about the USAID’s work in health sector reform Recommendations For USAID For PHCR For the option period Annexes List of people met Schedule Data collection instruments Scope of work 11. ESTIMATED BUDGET To be developed once team members are identified. Assessment of the USAID/Armenia Primary Healthcare Reform Project 57 ANNEX 2: WORK SCHEDULE DATE TIME ACTIVITY LOCATION Oct 3-5 Document Review Home offices Oct 6-7 Travel to Yerevan Transit Oct 8 All Day Team Planning Meeting Marriott, Yerevan Oct 9 9:00-13:00 Team Planning Meeting Marriott 13:00-13:30 USAID Briefing Marriott 14:30-18:00 PHCR Presentations PHCR Project Office Oct 10 9:00-12:00 PHCR Presentations & Discussion PHCR Project Office 13:00-14:45 Ministry of Health Interviews MOH 15:00-16:30 State Health Agency SHA Oct 11 10:00-11:00 Yerevan Municipal Health and Social Security Department HSSD, Yerevan 11:00-14:00 Ministry of Health Interviews MOH 15:15-16:30 Yerevan Polyclinic #5 Yerevan 17:00-18:00 Apaven Hamaynknerin (NGO) PHCR Project Office Oct 12 9:00-10:30 Health Project Implementation Unit (WB) HPIU Offices 11:00-13:00 Faculty Interviews: Base Medical College and National Institute of Health Yerevan Base Medical College 13:00-16:00 Project NOVA NOVA office 17:00-18:00 Jinishian Memorial Foundation (NGO) JMF office Oct 13-14 Report preparation Yerevan Oct 15 Depart 7:45 Field Visits Shirak Marz Oct 16 Field Visits Shirak & Lori Marz Oct 17 Return 18:00 Field Visits Lori Marz Oct 18 Report preparation Yerevan Oct 19 11:30-13:00 Yerevan Polyclinic #12 Yerevan 14:30-15:30 World Vision (NGO) World Vision Office 15:30-16:30 Kotayk Marz Health and Social Security Department PHCR Project Office 18:00-19:00 World Bank World Bank Office Oct 20-21 Report preparation Yerevan Oct 22 Morning Follow-up meetings Yerevan Afternoon Prepare Debriefing, Report preparation Oct 23 Morning Prepare Debriefing, Report preparation Afternoon USAID Debriefing USAID offices Oct 24 MOH Debriefing TBD Submit Draft Report Oct 25 Team returns home Transit Oct 29- Nov 1 USAID reviews draft report Yerevan Nov 2 USAID provides comments to Team Leader Nov 2-7 Report finalization, Team Leader Home Office Nov 7 Submit Final Report to Mission Assessment of the USAID/Armenia Primary Healthcare Reform Project 58 Assessment of the USAID/Armenia Primary Healthcare Reform Project 59 ANNEX 3: TECHNICAL SUGGESTIONS ON THE SCIENCE AND PRACTICE OF FAMILY MEDICINE Notes from Armenia clinic site visits by C.M.Fillmore, MD, MPH, MSc THE AMBULATORY MEDICAL CHART This chart is quite different from what I am used to; but there are a few suggestions that might help. Chronic diseases, hypertension Generally US/European medical charts have a list of chronic diseases and acute diseases (I think this is ―Regular Dispensary Check-ups‖ and ―Final Diagnosis‖ respectively). Listing of hypertension as a chronic disease is important, though I think it is probably not considered a dispensary registry problem, it needs to be: Hypertension is something the doctor/nurse can improve Hypertension implies the need for certain screenings—cholesterol, urine dip (checking kidney function--protein), EKG [in USA a patient with hypertension and no diabetes, chest pain, nor dyspnea would receive an EKG at first diagnosis and maybe not again for several years thereafter—not twice a year as in required in ], and glucose if patient is overweight (if on a diuretic) and obese. Screenings On the list entitled ―Screenings‖ I would remove ―Blood pressure screening‖—the way it is written suggests that blood pressure (b.p.) is only taken once per year, whereas b.p. should be taken virtually every doctor/nurse visit13. B.P. reflects hydration status and degree of illness and stress. Multiple recording of b.p.s will better overcome errors often made in these readings. I like that breast (I assume breast exam because mammography is not readily available) and gyn screenings are listed here (but PAP is virtually impossible in rural areas). Regular screenings might include full physical, cholesterol, anemia, blood sugar, BMI (are doctors given tables or formula BMI=kg/m2 [really needs a calculator]?) for adults: BMI <18.5 underweight BMI 18.5-24.9 normal weight BMI 25-29.9 overweight BMI>30 obesity Cholesterol. Cholesterol levels are not regularly determined, but this blood test is recommended in the UFMC (and performance-based indicators) for hypertensive and diabetic patients. Many patients have hypercholesterolemia without hypertension or diabetes, since Armenians even in the Diaspora have high rates of cardiovascular deaths, we can assume that many Armenians have high cholesterol without associated hypertension and diabetes. Certainly all smokers over 35 years old need cholesterol screening at least every 3-5 years. 13 It is assumed that b.p. will be taken on every visit, but emphasizing it as a screening means that even if a patient has not visited for years (common among middle-age populations), s/he will be invited for a check-up, and b.p. will be taken along with other important examinations on a preventive visit. The list of screenings for adult and pediatric population was developed by the MOH based on the U.S. Preventive Services Task Force, adapted for Armenia. (Source: PHCR) Assessment of the USAID/Armenia Primary Healthcare Reform Project 60 We need to know how widespread hypercholesterolemia is among adults in Armenia and what is the likelihood that a rural patient would be able to obtain/afford anti-cholesterol medication? (Alternatively, do doctors/nurses know in Armenia that 3 servings of fish/tuna/sardines/anchovies a week decreases triglycerides and to a lesser extent cholesterol and hence decreases risk of a cardiovascular event14? Where no medicine is available this may be a more reasonable alternative). Also BMI>30 among patients older than 40 years suggests the need to test for diabetes15 . Structure of medical records—generally we do more than one physical exam, at minimum every 5 years, so these sheets ―Primary medical examination…‖ should have a few copies. Each page with the ―Complaints, course or disease, physical examination, diagnosis‖ should have boxes for weight, bp, temp, pulse (or heart rate), respiratory rate to compel the doctor or assisting nurse to collect this information. Smoking information—because we know a physician regularly advising on quitting smoking does have a positive impact, Smoker yes/no Years Started Quit Date are often put in a prominent place on medical charts or at least under chronic medical problems. PERFORMANCE-BASED REIMBURSEMENT It is understood that the current suggested performance-based reimbursement (PBR) is still a working document. However, a great deal of attention to this aspect of physician reimbursement could make it a useful proxy for an abbreviated quality of care assessment. In the USA a patient with hypertension and no diabetes, no chest pain, nor dyspnea would receive an ECG at first diagnosis and maybe not again for several years thereafter—not twice a year16 as recommended in the Performance-based Reimbursement is definitely too much for this single risk patient. A diabetic definitely needs yearly EKGs with or without hypertension. My brief specific comments: • Immunization coverage—no refrigerator17, depends18 on national campaign and availability of shipments—better to do percent eligible actually receiving immunizations during campaign • Anemia testing—good, but able to get test done (perhaps the percent of children not starting cow’s milk before 12 months of age); never saw iron in any clinic med supply. • Pregnancy identification—more useful if clinic offered something during pregnancy (prenatal vitamins, vaginal infection testing), pregnancy testing kit/gloves? 14 http://www.aafp.org/afp/20070501/1365.html Accessed 20 October 2007. Or by journal: Oh RC, Lanier JB. Management of hypertriglyceridemia. Am Fam Physician. 2007 May 1;75(9):1365-71. 15 The non-pharmacological and pharmacological management of CVD risk factors based on risk stratification is included within the training materials developed by PHCR project, although not available for all physicians (Source: PHCR). 16 Twice a year is recommended by ―BBP service regulation‖ document, which is developed and approved by the MOH. PHCR￾developed Performance Based Reimbursement document recommends performing ECG for population with hypertension at least once a year (Source: PHCR). 17 All polyclinics and ambulatories are equipped with refrigerators. FAPs were provided with ―cold chain igloos‖ to implement immunization (Source: PHCR). 18 Depends on mandatory immunization schedule (Source PHCR). Assessment of the USAID/Armenia Primary Healthcare Reform Project 61 • Diabetic fundoscopy—more useful than thought, as long as the Eye Care Project is in Armenia (clinics or referral polyclinics having a working ophthalmoscope estimated at <50 percent)19 . • ECG – varies 1 to 2 ECGs a year for hypertension (too many?) and IHD (what about diabetes?) percent of clinics where ECG is possible? • Cholesterol—for IHD and diabetes (what about for hypertension?) where possible? Medicine available?20 19 Upon reaching the desired target, an indicator will no longer be considered in the list. 20 Medicines are available (although not all of them free of charge), and diets can be prescribed to reduce cholesterol levels. Assessment of the USAID/Armenia Primary Healthcare Reform Project 62 Assessment of the USAID/Armenia Primary Healthcare Reform Project 63 ANNEX 4: PROVIDER SURVEY Urgent: Please Complete And Return By Tomorrow Morning As MedFam positions itself as a strong competitor in the changing healthcare market, assessing the status of MedFam and redefining the priorities will be a major and recurring theme. Future success will be dependent upon the ability to implement and sustain major changes in the way we do business. To help in this effort, we would like you to share your thoughts, opinions and ideas about MedFam. Your honest and confidential responses will help prioritize our future efforts! Some Background 1. What are your functions/tasks at MedFam? Please be specific _________________________________________________________________ _________________________________________________________________ _________________________________________________________________ 2. What is your current position? Please circle: Receptionist Treatment Nurse Nurse Physician Other __________________________________________________________________ __________________________________________________________________ Work Environment Please answer the following items using the number associated with choice that comes closest to your own feelings: 1 2 3 4 5 Strongly disagree Disagree Neutral Agree Strongly agree 1. My job makes the best use of my skills and abilities _____ 2. I am made to feel a worthwhile, valued, and important part of the team _____ 3. The physicians encourage the staff to work together as a team _____ 4. There is good communication and cooperation between my colleagues ______ 5. I always know exactly what to do in order to do my job properly _____ 6. I have enough authority to accomplish the work that is expected of me _____ 7. I am encouraged to try new ways of accomplishing my job _____ 8. The workload in my job function is evenly and fairly distributed _____ Assessment of the USAID/Armenia Primary Healthcare Reform Project 64 9. The staff at MedFam take pride in their job performance _____ 10. Meeting patient expectations is the highest priority at MedFam_____ 11. Efficiencies and productivity are rewarded at MedFam _____ 12. There is continuous effort to improve group/work performance at MedFam_____ 13. I receive regular feedback from my supervisor on my work performance _____ 14. I am provided opportunities to increase my skills and knowledge through training and education. _____ 15. The physicians effectively communicate the future and direction of the practice. _____ 16. The organization makes the best use of technology to improve work methods. _____ 17. Everyone cares about accomplishing the goals of MedFam. _____ 18. I feel my job tasks contribute to the success of MedFam. _____ 19. There is good cooperation between me and my colleagues. _____ 20. MedFam encourages me and provides me the opportunity to improve my professional knowledge or job skills. _____ 21. Compared to other similar practices, MedFam is a good place to work. _____ 22. I have considered resigning in the last six months. _____ 23. Staff meetings provide me an opportunity to state my views _____ 24. What are the three things you like most about the changes that have taken place? 1. _____________________________________________________ 2. _____________________________________________________ 3. _____________________________________________________ 25. What are the three things you like the least about the changes that have taken place? 1. _____________________________________________________ 2. _____________________________________________________ 3. _____________________________________________________ 26. What feedback are you getting from patients? _____________________________________________________ _____________________________________________________ _____________________________________________________ _____________________________________________________ _____________________________________________________ Assessment of the USAID/Armenia Primary Healthcare Reform Project 65 27. What do you see as the top priorities for MedFam over the next few years? Please list in order of decreasing priority: 1. _____________________________________________________ 2. _____________________________________________________ 3. _____________________________________________________ 4. _____________________________________________________ 5. _____________________________________________________ 28. Please provide any other feedback that you feel is important to the success of MedFam. Assessment of the USAID/Armenia Primary Healthcare Reform Project 66 Assessment of the USAID/Armenia Primary Healthcare Reform Project 67 ANNEX 5: PEOPLE MET GOVERNMENT OF ARMENIA: Ministry of Health: Karine Saribekyan, Head of MCH Department Izabela Abgaryan, Head of Legal Department Ruzanna Yuzbashyan, Head of Primary Healthcare Department Hovhannes Margaryants, Advisor to the Minister of Health State Health Agency, MOH: Ara Ter-Grigoryan, Head of State Health Agency, Ministry of Health National Institute of Health: Samvel Hovhannisyan, Head of Family Medicine Faculty Yerevan State Medical University: Michael Narimanyan, Head of Family Medicine Chair Yerevan State Base Medical College Donara Hakobyan, Deputy Director, BMC Amalja Adamyan, Faculty, BMC Varduhi Arzumanyan, Faculty, BMC Varaduhi Akopyan, Faculty, BMC Narine Shekoyan, Faculty, BMC Karine Avetyan, Faculty, BMC Ervand Manukyan, Faculty, BMC Artur, Martirosyan, Faculty, BMC Anahit Gevorgyan, Faculty, BMC LOCAL GOVERNMENT AND HEALTH FACILITIES: Yerevan: Health and Social Security Department (HSSD): Armen Soghoyan, Head Robert Sukiasyan, Deputy Head Polyclinic #5, Yerevan: Jivan Sanosyan, Director Marine Mkrtchyan, Accountant Ignatia Polyclinic (Private): Ruben Meghrabyan, Director Ruzanna Matevosyan, Accountant Assessment of the USAID/Armenia Primary Healthcare Reform Project 68 Shirak Marz: Shirak Regional Hospital, Gyumri: Satenik Beybutyan, Ex-Head, HSSD, Shirak Marz Gyumri Nursing College: Manvel Martirosyan, Director Shushan Hakobyan, Deputy Director Three Faculty Four Community Nurses (students) Ajakits NGO, Gyumri: Heghine Mkrtchyan, President Arpine Porsughyan, Training Coordinator Vahe Tagvoryan, CHC Trainer Christine Aghabekyan, Logistics Kaps Community Health Post: Lorik Mkoyan, Community Mayor Sonja Grigorjan, School Director, Community Health Committee Vehanush Hovhannisyan, President, Meghvik NGO Anaksja Grigrajan, Visiting Doctor (FM) Varsenik Sargsyan, FAP Nurse, Kaps Health Post Aregnadem Community Health Post: Aghunik Hazryan, Community Mayor Anna Mhoyan, Visiting Doctor (FM), Amaria Health Center Manik Gabrielijan, Meg Sister Narine Ghazaryan, FAP Nurse, Aregnadem Health Post Voskehask Community Health Post: James Muradyan, Community Mayor Astghik Nozoyan, CHC Member Lena Manukyan, Physician Anahit Shmavonyan, FAP Nurse Karine Borceghyan, Nurse Ruzanna Mkhitoryan, Nurse Nune Karoutunyan, Nurse Jajur Community Ambulatory: Mary Danielyan, Director (FM) Seda Hossoyan, Physician Anahit Shekoyan, Family Nurse Sofya Karapetyan, Accountant Vardanush Arshakyan, Operator Assessment of the USAID/Armenia Primary Healthcare Reform Project 69 Lori Marz: Khnkoyan Community Health Post: Marine Lambaryan, Director, Mets Parni Ambulatory Ofelya Ispiryan, FAP Nurse, Khnkoyan Health Post Ashot Manukyan, Community Mayor, Khnkoyan Vahayn Karapetyan, Teacher, CHC Member Khanum Karapetyan, School Teacher, CHC Member Vardablur Ambulatory: Armine Gharajyan, Family Physician, Director and Clinical Preceptor Satenik Sahakyan, Accountant/Operator Gugark Community Central Polyclinic: Marine Simonyan, Director, Gugark Central Polyclinic Julietta Ulikhanyan, Accountant, Gugark Central Polyclinic Marfa Meliksetyan, Accountant, Gugark Central Polyclinic Nor Ughiner NGO, Vanadzor: Aza Aghabekyan, President Christine Chevondyan, Community Mobilization Coordinator Ghursal Community Health Post: Naira Poghosyan, FAP Nurse, Ghursal Health Post Mher Minasyan, Community Mayor Shushanik Mahrabyan, Community Member Garegin Karhanyan, School Director Artsvik Abrahamyan, Deputy Mayor Spitak Town Polyclinic: Armen Chobanyan, Director, Spitak Polyclinic, Vanadzor Marine Simonyan, Director Marfa Meliksetyan, Accountant Kotayk Marz: Health and Social Security Department: Gayane Martirosyan, Head, HSSD PRIMARY HEALTHCARE REFORM PROJECT: Richard A. Yoder, Chief of Party John Vartanian, Acting Chief of Party (Deputy Chief of Party) Gayane Gharagebakyan, Team Leader, Health Sector Reform and Open Enrollment Karine Gabrielyan, Team Leader, Family Medicine and Quality of Care Assessment of the USAID/Armenia Primary Healthcare Reform Project 70 Lusine Aydinyan, Team Leader, Healthcare Finance Ruzanna Melyan, Team Leader, Public Education and Disease Prevention Tsovinar Harutyunyan, Director, Monitoring and Evaluation Varuzhan Avanesyar, Public Education Specialist Susanna Mkrtchyan, Health Education Specialist, PE Team Christopher Wild, Short Term Technical Assistance, Public Education Armine Danielyan, Training Advisor, FM&QoC team Susanna Onanyan, Nursing Advisor, FM&QoC team Gagik Ghazakhetsyan, Field Office Manager PROJECT NOVA: Inna Sacci, Chief of Party Karen Adamyan, Maternal Health Specialist Lilit Hovakimyan, Clinical Manager JINISHIAN MEMORIAL FOUNDATION: Eliza Minasyan, Executive Director Zara Janibeyan, Health Program Coordinator APAVEN HAMAYNKNERIN (NGO): Karen Arakelyan, President, Yerevan WORLD VISION (NGO): Kristine Mickaelidi, Yerevan Avetik Harutyunyants Araks Hovhannisyan AJAKITS (NGO), GYUMRI: Heghine Mkrtchyan, President NOR UGHINER (NGO), VANADZOR: Tigran Torosyan, President WORLD BANK: Susanna Hayrapetyan, Senior Health Specialist, Armenia Office Sergey Khachatryan, Director, Health Project Implementation Unit (MOH) Assessment of the USAID/Armenia Primary Healthcare Reform Project 71 USAID: Ann Hirschey, USAID/Armenia Kimberly Waller, Health Programs Manager, USAID/Armenia Ruben Jamalyan, USAID/Armenia Mark Austin, PHN Strategy and Technical Advisor, USAID Washington OTHERS: Gina Holdar, Facilitator, Team Planning Meeting Gayane Simonyan, Translator Assessment of the USAID/Armenia Primary Healthcare Reform Project 72 Assessment of the USAID/Armenia Primary Healthcare Reform Project 73 ANNEX 6: REFERENCE MATERIALS General PHCR contract ASTP Reports # 84,85,86, 93, 103, 104, 109, 110 PHCR Work plans (Years 1&2) Quarterly reports (Q1, FY01; Q2, FY01; Q3, FY01; Q4, FY01; Q1, FY02; Q2, FY02) Quarterly bulletins (Q1, 2006; Q2, 2006; Q3, 2006; Q4, 2006; Q1, 2007) Health Sector Reform & Open Enrollment RA Law on Medical Aid and Services to the Population GOA Decree No. 1533-N 0n PHC Strategy 2003-2008 and Approval of Pilot Project Through 2003- 2005 for Development of New Methods pf PHC Administration and Financing GOA Decree No. 497-N on Family Medicine Independent and Group Practices GOA Decree № 420-N on PHC Provider Selection and Registration with Them MOH Order № 57-N on Approval of OE Registration and Shift Forms and Their Filling Instructions IRM document Family Medicine and Quality of Care Family Physician’s organizational provisions (Gov. Decree # 539 April 08, 2004) Unified Family Medicine Curriculum Report on Analysis of the Current State of Family Medicine Education in Armenia Report on Yerevan Based Training Sites Evaluation Unified Family Nursing Curriculum A Guide and Toolkit for Health Worker Training and Education “Learning for Performance” Programs (developed by IntraHealth HO) Training Package on UTI Quality of Care Package Medical Record Form with relevant regulation Job Aids on Hypertension and Stable Angina Concept Paper on the Improvement and Management of the Quality of Healthcare provided to the RA Population (Gov. Decree #46, 31.10. 2002). L. Crigler’s report on QA package implementation Healthcare Finance Document describing process of PHC budgeting and financing in RA Report on assessment and recommendations for PHC payment system in Republic of Armenia (developed by PHCR STTA Igor Sheiman) R. Feale’s report on HF component for Year 3 Workplan List of proposed PHC performance indicators 2004 NHA report Public Education BBP poster – 2006 BBP booklet – 2006 Drug poster – 2007 CHC Capacity Building Manuals: Advocacy Project Design and Management Assessment of the USAID/Armenia Primary Healthcare Reform Project 74 Behavior Change Communication CHC Health Training Manuals: Calcium and Healthy Bones Hypertension Prevention and Management Diabetes Prevention and Management Monitoring & Evaluation Baseline patient satisfaction report (investigating the level of patient satisfaction with care at target and several control facilities in Zone 1) Baseline facility assessment and self-assessment report (technical and performance assessment of target facilities in Zone 1) Baseline household health survey report (nationwide household survey of 2,310 respondents assessing the attitude, practice, and knowledge of the population with regard to open enrollment and family medicine, their perceived health status, use of early diagnostics and preventive services, accessibility and perceived quality of care, as well as the level of exposure to health education activities conducted by PHCR project) For more information, please visit http://www.ghtechproject.com/resources/ Global Health Technical Assistance Project 1250 Eye St., NW, Suite 1100 Washington, DC 20005 Tel: (202) 521-1900 Fax: (202) 521-1901 www.ghtechproject.com