ASSESSMENT OF THE AMERICAN INTERNATIONAL HEALTH ALLIANCE (AIHA) PROGRAM IN AZERBAIJAN Submitted to: U.S. Agency for International Development/Azerbaijan In response to: USAID Evaluation IQC No. AEP-I-00-00-00023-00 Task Order No. 842 Prepared by: Greg Becker – Team Leader Ivdity Chikovani, MD – Consultant Submitted by: Development Associates, Inc. 1730 North Lynn Street Arlington, VA 22209-2023 December 2003 Assessment of the American i December 2003 International Health Alliance (AIHA) Program in Azerbaijan TABLE OF CONTENTS Acronyms & Terms ..................................................................................................................ii Executive Summary................................................................................................................. iii 1. Background ...................................................................................................................1 2. Methodology.................................................................................................................3 2.1 Interviews.........................................................................................................3 2.2 Observations and Field Visits...........................................................................4 2.3 Focus Groups....................................................................................................4 3. Summary of Key Findings............................................................................................4 3.1 Scope of Work Specified Tasks........................................................................5 3.2 Improved Health Outcomes............................................................................10 3.3 Cost Effectiveness and Efficiency..................................................................12 3.4 Focus Areas of the Partnerships.....................................................................12 3.5 Sustainability ..................................................................................................13 3.6 Technical Assistance Outside of Partnership Expertise .................................14 4. Conclusions.................................................................................................................15 4.1 Cooperative Development and Learning........................................................15 4.2 Working With the System ..............................................................................15 4.3 Network Effects - Spread by Attraction .........................................................16 4.4 Systemic Behavioral Change ..........................................................................16 4.5 Sustainability Through Replication................................................................17 5. Recommendations.......................................................................................................19 5.1 Increased Attention to Epidemiological Priorities..........................................19 5.2 Structural Changes..........................................................................................20 5.3 Involvement with Other USAID Programs ....................................................20 Annex A Scope of Work ................................................................................................... A-1 Assessment of the American ii December 2003 International Health Alliance (AIHA) Program in Azerbaijan ACRONYMS & TERMS AIHA American International Health Alliance APO Azeri Partnership Organization GoA Government of Azerbaijan IDP/R Internally Displaced Person/Refugee MoH Ministry of Health NIS Newly Independent States PHC Primary Health Care Rayon District USAID U.S. Agency for International Development USPO U.S. Partnership Organization WHO World Health Organization Assessment of the American iii December 2003 International Health Alliance (AIHA) Program in Azerbaijan EXECUTIVE SUMMARY BACKGROUND Greg Becker and Ivdity Chikovani conducted an assessment of the AIHA/Azerbaijan partnership program during the period of September 22 to October 22, 2003. In-country field work was conducted between October 6 and 18, 2003. The Evaluators examined the three primary partnerships of Baku-Houston, Baku-Portland, and Baku-Richmond to determine the general accomplishments, constraints, and shortcomings of the efforts, and to offer advice on possible future USAID participation in partnership activities in Azerbaijan. METHODOLOGY The evaluation of AIHA’s activities in Azerbaijan was conducted using qualitative research methods of structured interviews, focus groups, document review, and observation. Limited quantitative analysis was conducted using secondary source information such as health facility records, GoA statistical information, and data from bilateral and multilateral donors and organizations. The state of health information in Azerbaijan appears to be more problematic than the norm for countries of the NIS. Much data is inaccurate prima facie, and contradictions between data sources are common. This has made quantification of the impact of the AIHA Partnerships difficult. FINDINGS The Azeri Health System as a whole appears to be somewhat behind the other countries of the Caucasus in its evolution from the Soviet legacy. The greatest deficit in this regard is the absence of financial reform of the system. The lack poses a significant impediment to the sustainability of improvements based on material resources, and has a debilitating effect on the morale of health workers. The most important findings specifically related to the Partnerships are: 4 The three AIHA Partnerships in Azerbaijan have been effective in achieving significant improvements to the health of the population of Azerbaijan, particularly women and IDP/Refugees: · Basic newborn resuscitation training resulted in a major decrease in infant death in targeted facilities. Development Associates, Inc. Assessment of the American iv December 2003 International Health Alliance (AIHA) Program in Azerbaijan · The creation of a neonatal training unit and capability at the Mir Kasimov center for high-risk deliveries provides significantly improved newborn protection for a major portion1 of the national population of women at this time of greatest peril. · The introduction of family medicine practices has resulted in significant measurable improvements in patient health in the areas of hypertension, gastric ulcers, asthma, and diabetes. · The success of the family medicine model has resulted in its official adoption by the MoH and the establishment of an Azeri post-graduate training program. · Azeri physicians establishing a Baku-Saberabad partnership have copied the AIHA Model. · Local demand for training in cardiovascular care resulted in this training being developed and delivered without direct partnership support. · The development of a program to teach breast health to physicians and nurses, and breast self-examination to women has provided an extremely low cost method of significantly impacting the pathology of this most prevalent cancer in women. · The Partnerships have fundamentally changed the power structure of the health system through the empowerment of area physicians and nurses to provide actual care to patients when their traditional roles have been limited to referrals rather than service delivery. · Behavioral changes in health workers generated by the Partnerships have been significant. Morale, comportment, competence, and commitment of participants are greatly improved. 4 The Evaluators believe that the Partnership Program in Azerbaijan is both cost effective and cost efficient: The impact of these organic changes to the medical structure of the country reaches more people more effectively than traditional efforts to bypass the medical establishment or through the externally stimulated development of NGOs designed to replace the existing structure. Calculations of costs and benefits should prove that the Partnership Model achieves more in terms of tangible lives saved and morbidity averted when compared to alternative models where benefits accrue on a net future value basis. 4 Most of the structural and health outcome impacts of the Partnership Model are sustainable by nature: The key factors are that these activities are driven by local 1 Presently 2.3% of all pregnancies are delivered at Mir Kasimov Hospital – most referred as high-risk pregnancies. The projection of wide coverage of major segments of the population is based on the protective nature of a tertiary care referral center. When assessing this coverage, an appropriate analogy is a neighborhood ambulance – all residents of the area have a higher level of protection due to the existence of the resource. Measurements of beneficiaries are thus greater than the actual number of patients who use the ambulance. Development Associates, Inc. Assessment of the American v December 2003 International Health Alliance (AIHA) Program in Azerbaijan demand stimulated by an expanded understanding of the possible, and that they are implemented through the medical establishment. Programs that are implemented through this self-powered mechanism are the essence of local ownership. 4 Epidemiological priorities are not always followed in the selection of Partnership activities: The number one scourge of Azerbaijan is cardiovascular disease. It is not sufficiently addressed. The life expectancy of men is significantly lower than that of women despite the risks of childbirth and is not addressed by the AIHA program. Maternal mortality during childbirth is still a grave problem that is being only marginally addressed as a part of more diffuse Women’s Wellness Center activities 4 Material resource sustainability is problematic: The Baku-Portland Partnership has developed an appropriate mix of basic equipment that sets a rational minimum level of primary care diagnostic equipment. The Doctor Bags and Nurse Bags provided by the partnership to area doctors and nurses moving to the medicine model contain the simplest and most essential items necessary for basic physical examinations. Unfortunately, the bags soon lose some of their value as area nurses and doctors are not able to procure replacements for their glucometers because of the $70 per vial cost. Although this raises questions sustainability, in the words of one area nurse, “even if we only received the first supply of strips, and never received any again, there are people who are alive today that would not have been.” A potential value of the dearth of strips may be that it will serve as a partial catalyst for further examination of financial reform within the system. 4 The focus on Pap Smears is of comparatively low value: According to local Ob/gyn specialists, the same results can be obtained during a more complete colposcopy, which does not have the recurrent costs of reagents. Given the very small level of mortality due to cervical cancer, the considerable effort to introduce this new technique should be re￾evaluated. 4 There is a need for skills outside of the US Partner areas of expertise: The example of glucometer sustainability points to the need for some level of outside collaborative technical assistance in the areas of structural reform and cost recovery. FINDINGS ON THE SUCCESS FACTORS OF THE AIHA MODEL AIHA is in a unique position among USAID funded projects in its fundamentally different approach to achieving substantial and sustained health improvements. An examination of these differences provides a deep understanding of the mechanisms of behavior change and motivation that are active in health systems development and reform efforts. 4 Changes are demand driven by the partners functioning as peers in a voluntary helping relationship: Work plans and technical priorities are developed jointly in a give and take exploratory process where neither party has moral or financial primacy over the other. 4 The process works through the established medical hierarchy: A system that is as protocol bound as that of the NIS is the result of long-term adaptation to the environment. Development Associates, Inc. Assessment of the American vi December 2003 International Health Alliance (AIHA) Program in Azerbaijan It is important to remember that much of the present dysfunction of the system is the result of radical change in the environment, and that in the past, the health infrastructure had considerable capabilities and beneficial characteristics. 4 Exchanges of medical professionals between the US and Baku are the sine qua none of expanding the world view of participants that makes change possible: The Baku partners have been able to conceive of changes to the system because of their increased understanding and belief in what is possible. 4 The existing medical/technical hierarchy acting under its own volition implements system changes: AIHA Partnerships were conceived as primary care efforts, but they approach the process through the traditional medical power structure rather than bypassing the hospital centric system. 4 Empowerment of the primary care sector has occurred because the enrichment of area doctors and nurses was done with the guidance, direct support, and blessing of the medical/technical elite: Training and technical support of providers is of little use if the recipients are unable to utilize their increased knowledge because of structural or environmental constraints. 4 Opinion change and adoption of new processes through the medical/technical hierarchy is more durable and effective than the equivalent process occurring in the MOH structure: Although there are many network ties and common actors between the two structures, the MOH’s ability to affect change is subject to political ebb, flow, and redirection. Change through the medical/technical hierarchy carries considerable authority and certitude. 4 Systemic change moving through the medical/technical hierarchy requires a lower level of effort than change creation localized in the lower tiers of the medical/technical infrastructure: Change efforts localized at the primary care level of the medical/technical infrastructure require considerable upwards effort to “buck the system.” Upper level “believers” pave the way for change to occur. 4 The partnership model is effectively self-replicating: The Baku-Houston Partnership has provided an excellent example of the phenomenon of successful strategy imitation by the Mir Kasimov Republican (i.e. tertiary) Hospital establishing a relationship with the Saberabad Rayon Medical District. 4 The attraction principle is conducive to institutionalized change: The Baku-Portland Partnership primary care physician skills building activity has achieved improvements to patient outcomes and physician productivity that have stimulated high level interest in the program resulting in the creation of an MOH Curriculum and Department of Family Medicine. 4 Changes to the power structure stimulates and improves bi-directional communication between health workers and administrative authorities: Communication has become more intense and constructive as physicians and nurses feel that relationships have been Development Associates, Inc. Assessment of the American vii December 2003 International Health Alliance (AIHA) Program in Azerbaijan established with higher level authorities because of increased interest in the progress and problems of care givers. 4 Interest is generated and spread through the Network Hub phenomenon: The term “viral marketing” is used to describe interest in a product or service generated through the observation of early adopters and word-of-mouth promotion. Sociologists have recognized the key role of influential individuals in the speed and ultimate success of idea or technological spread. 4 AIHA programs attract participation because of the link to health worker needs characteristics: People entering (or staying) in the health professions tend to have comparatively high tendencies towards altruistic behaviors and needs. Enabling them to satisfy these needs (i.e. cure a patient) provides powerful motivation. 4 The AIHA administrative support structure in Baku and Tbilisi is effective and cost efficient: The existing staff are the glue that keep the programs running. RECOMMENDATIONS Qualitative and limited quantitative analyses indicate that the disparity between the IDP/Refugee population and the general population may have largely disappeared. Unfortunately, this realignment seems due primarily to a decline in welfare of important segments of the general population rather than to major improvements in the welfare of the IDP/Rs. The prominent disparity now appears to be within the IDP/R population as income levels greatly diverge for individual families much as is occurring in the general population. USAID is encouraged to carefully monitor this situation and continue adjusting program targeting accordingly. 4 If it becomes possible in the future, the Partnership Program should be continued and expanded. Few programs provide this level of benefit for the size of the investment. 4 The AIHA Model should be strengthened through incorporation of outside technical assistance from one of the USAID Flagship Programs: Cost Recovery and financial reforms are greatly needed in Azerbaijan, and PHRplus or possibly the POLICY Project would be appropriate sources for this type of assistance. Technical assistance in cost recovery on a local basis (theoretical legal authority is already present in the rayons) would be the essential minimum involvement. If in the future USAID is able to re-invest in the AIHA model, an integrated three pronged approach of AIHA model, policy reform, and primary sector intervention is highly recommended. 4 Cardiovascular disease should receive increased attention: The severity of the problem demands that rapid impact measures be sought in addition to long-term life-style interventions and promotion. The present level of mortality is at a crisis stage and must be met with primary care level therapeutic interventions. Consideration should be given to the idea of developing “primary care cardiologists” equipped with simple ECGs for screening and diagnosis and the development of a self-sustaining (i.e. revolving fund) medication mechanism. Development Associates, Inc. Assessment of the American viii December 2003 International Health Alliance (AIHA) Program in Azerbaijan 4 Women’s Wellness Center focus should be concentrated on the high priority need to reduce morbidity and mortality related to child birth: The present level of maternal mortality demands that efforts not be diluted by other activities associated with wellness centers. Other “wellness” activities such as the programs for female staff of Mir Kasimov should be spun-off as an independent business (possibly not-for-profit). 4 Neonatal Resuscitation Program should be extended and complemented by topics addressing common conditions of newborn mortality: The alarming rate of neonatal mortality and the faulty status of the referral system for high risk pregnant women and newborns indicates that improvements in basic clinical knowledge of medical personnel at the delivery settings thought country would be of great benefit. 4 Male mortality in the productive age groups is at a crisis level and, therefore, should receive increased attention: Estimates of death in men aged 15 to 60 are very likely low by a considerable margin2 . USAID’s commitment to prioritize on the basis of epidemiological need must take priority over gender preference. AIHA partnerships should be encouraged to examine this crisis with the medical/technical weight and skill of its US and Azeri Partners. 4 Partners should increasingly focus on the wider dissemination of improvements: The imitation of partnerships with peripheral health providers as is being done with Saberabad should be supported and become a priority for all activities. 4 To the extent possible given available funding, the Model Primary Care Clinic being developed in the Narimanov Rayon should be fully assisted by the Baku-Portland Partnership, and receive input from the Baku-Richmond Partnership: The dedication and level of effort exhibited by the physicians developing the breast health, asthma, mental health, and other model primary care programs are commendable. The expanded profile of Area Physician and Nurse capabilities and areas of practice will result in major nationwide improvements in primary care as they are disseminated. Expansion of patient and community outreach practices piloted at the clinic are ready for wide scale dissemination. Full and continued support should be given to the implementation of the Family Medicine Post-Graduate Training Institute at the Model Clinic. 4 Greater communication and joint actions between the Partnerships is encouraged: AIHA and USAID should link the efforts of the Baku-Portland and Baku-Richmond Partnerships. Such a linkage at both the Partnership level and at the local Baku Rayon level would result in significant efficiency gains and increased dissemination of advances. Duplication of efforts would be minimized and each partner could focus on different specialty areas. One area for essential collaboration is in the development of the Family Medicine Curriculum. 4 Trauma and cardiac emergencies are likely the leading causes of death in Azerbaijan: Creating an Emergency Medical Services Training Center on the AIHA model will result 2 Demographic analysis of the Max Planc Institute provides convincing evidence that large inaccuracies in total population numbers caused by undocumented out-migration from the Caucasus leads to erroneously low estimates of mortality rates. Development Associates, Inc. Assessment of the American ix December 2003 International Health Alliance (AIHA) Program in Azerbaijan in major reductions in morbidity and mortality. Funding for this activity should be a priority for future USAID investment. 4 Clinical Practice Guidelines, Evidence Based Medicine, and Learning Resource Centers support improved primary care practice: If possible, USAID should reinstitute AIHA “Cross-Partnership” programs of guideline development and inter-Caucasus collaboration. 4 USAID/Caucasus should combine the AIHA model with policy reform and traditional primary care sectoral interventions: USAID/Caucasus (as well as USAID programs in many other parts of the world) would significantly increase the effectiveness, efficiency, sustainability, and depth of impact if a three part strategy was adapted in future programming. Assessment of the American 1 December 2003 International Health Alliance (AIHA) Program in Azerbaijan ASSESSMENT OF THE AMERICAN INTERNATIONAL HEALTH ALLIANCE (AIHA) PROGRAM IN AZERBAIJAN 1. BACKGROUND The countries of the Former Soviet Union have each faced great challenges in the transition from the old coherent system to a resource poor, adaptive system facing severe new challenges. There has been much inherent conflict as participants at all levels seek new strategies all the while clinging to the legacies of the past. Within this context, these new countries have made varying levels of progress because of differing levels of resources and stresses. Azerbaijan had one of the most stressful beginnings with a pre-independence violent “crack-down” by Soviet troops in Baku, and a post￾independence armed conflict over Nagorno-Karabakh. This ethnic-based struggle resulted in the displacement of 650,000 people from their homes and the influx of additional 200,000 Azerbaijani refugees from Armenia. This IDP/Refugee population is only now coming to a state of near parity with the general population. Unfortunately, parity is being reached more through a decline in the welfare of the general population rather than through gains made by the IDP/R population. Equipped with an economy based on obsolete industrial processes and a highly inefficient petroleum exploitation infrastructure, general standards of living have declined greatly. With western improvements to the extraction infrastructure, the economy is eventually going to turn around, but the effects of this investment are not yet widely shared. Azerbaijan is something of a paradox in its development. Post-independence strife was a period of considerable humanitarian need which has slowly calmed. In the meantime, the general level of economic and social health has declined. Thus, while the humanitarian emergency can be said to have largely ended, the general population is worse off than during the crisis. USAID has developed a program of assistance geared to meeting the critical challenges of economic and democratic transition, and provided humanitarian assistance to the most vulnerable groups, especially IDP/Rs. The major vehicle for providing social/health assistance is the Azerbaijan Humanitarian Assistance Program (AHAP), implemented through an umbrella cooperative agreement sponsoring programs of seven major US NGOs. Outside the AHAP framework, three US-Azeri health partnerships within the AIHA program work to upgrade medical skills and provide Azeri health professionals with exposure to the U.S. health care system. In September, 1998, a region-wide agreement, and separate agreements for the Caucasus, Central Asia, West/NIS, and Russia were awarded to the American International Health Alliance (AIHA). USAID concurrently awarded a three-year sub-agreement for the Caucasus, covering AIHA programs in Georgia, Azerbaijan, and Armenia. Due to U.S. Foreign Assistance Act Section 907 limitations on direct assistance to the Azeri Government, the AIHA partnerships faced difficult circumstances in establishing their activities. Development Associates, Inc. Assessment of the American 2 December 2003 International Health Alliance (AIHA) Program in Azerbaijan In addition, the greatest challenge to partnership success and sustainability has been the severe shortage of public and private resources. The fiscal crisis in Azerbaijan has not abated, and is worsening with the passage of time. AIHA has three established partnerships in Azerbaijan with a forth in the early stages of development. This assessment has focused on the following three established partnerships, as defined in the Scope of Work (the full SOW can be found in Annex A). Baku-Houston: (Established January 2000). Overall goal: to improve medical services to refugees and IDPs in targeted camps and communities served by the Mir Kasimov Hospital in Baku and the Sabirabad Rayon Hospital in Sabirabad. Specific objectives include: establish a neonatal resuscitation center to train health professionals; develop and disseminate practice guidelines according to international standards; and collaborate with other partnerships in Azerbaijan to provide education and training in neonatal resuscitation. Also, the partnership aims at shifting patient-oriented activities to rayon hospitals serving IDP camps in their catchment areas and establishing a Women’s Wellness Training Center to address maternal and child healthcare concerns among refugees/IDPs. Baku-Portland: (Primary Health Care – Established March 2000) Overall Goal: Develop a comprehensive, community-based, primary care system designed to meet the basic health care needs and improve the health status of IDPs and refugees in the Narimanov Health Care District in Baku, Azerbaijan. Specific objectives include: Enhance the ability of the Narimanov Health District to collect, store, organize and analyze basic socio-demographic and health status information on the IDP/refugee population in the district; enhance and adapt the infrastructure of the Health District to provide a solid base for primary care; improve the effectiveness of the nurses serving the IDP/refugee population; and integrate a mental health program into the model primary care clinic. Baku-Richmond: (Primary Health Care-Partnership Established March 2000). The overall goal of the partnership is to improve the health status of the local population including refugees and IDPs accommodated in the Binagadi District of Baku city through socially oriented, public and primary health care programs. Specific objectives include: improve the capacity of the Binagadi Health Department to collect, enter, and analyze data on the health status of the local population including refugees and IDPs; improve the quality of medical care provided by area doctors and nurses to refugees and IDPs through training in clinical assessment skills and sharing evidence based medical guidelines, and; to improve the quality of medical information provided to the population on infectious disease, personal hygiene, sanitation, and general prevention in their home environment. In addition to developing and implementing the above-mentioned partnership programs, AIHA also supports a number of related activities that facilitate and promote inter-partnership communications and synergy, joint partnership initiatives, sharing and dissemination of information, and replication of centers such as Learning Resource Centers, EMS and Nursing Development Associates, Inc. Assessment of the American 3 December 2003 International Health Alliance (AIHA) Program in Azerbaijan 2. METHODOLOGY The state of health information in Azerbaijan appears to be more problematic than the norm for countries of the NIS. Much of the available national data is inaccurate prima facie, and contradictions between data sources are common. This has made quantification of the impact of the AIHA Partnerships difficult at best, and sometimes impossible. Given this constraint, the team placed a great reliance on qualitative methodologies. The evaluation of AIHA’s activities in Azerbaijan was conducted using qualitative research methods of structured interviews, focus groups, document review, and observation. Limited quantitative analysis was conducted using secondary source information such as GoA statistical information, and data from bilateral and multilateral donors and organizations. Interview informants included health professionals from the MoH including staff from APOs, directors and participants from the USPOs, staff of AIHA Caucasus, patients, and IDP/Rs. Observations were made at MoH facilities including those of APOs, and in IDP/R “camps.” Analysis of data and information gathered was conducted using interactive informant feedback and adjustment, health economics and health management frames, clinical medical review, and applied complexity science. 2.1 INTERVIEWS Telephone interviews were conducted with the leadership of the USPOs Baku-Richmond (Virginia Commonwealth University), Baku-Portland (Oregon Health Sciences University), and Baku-Houston (Baylor Medical University). Conducted prior to the team’s arrival in Azerbaijan, The focus was to determine the USPO position on the partnerships, to hear their perspective on achievements and barriers, and to determine the state of institutional commitment. Given the early stage at which these interviews were conducted, the focus was split between gaining a general familiarity with the nature of the partnerships, and with inquiring about the status and evolution of institutional and participant enthusiasm, commitment, environmental adaptation, and learning. General questions and responses were helpful in establishing an increased understanding of the history of these particular partnerships, and in establishing a basic familiarity with the mechanisms, procedures, and guiding principles employed. Discussions then focused on the issues of institutional and participant commitment/fatigue. Questions covered institutional willingness to contribute to the partnerships at different points during the life of the programs. Other questions specifically inquired about the level of concern demonstrated by the institutions’ directors and financial officers over the material, financial, labor, and administrative costs borne by the USPO. Development Associates, Inc. Assessment of the American 4 December 2003 International Health Alliance (AIHA) Program in Azerbaijan Statistics: Total population: 8,095,000 Total IDP/Refugee population: 800,000 GDP per capita (Intl $): 2,676 Life expectancy at birth m/f (years):60.7/66.6 Healthy life expectancy at birth m/f (years):50.3/55.4 Child mortality m/f (per 1000): 100/88 Adult mortality m/f (per 1000):253/132 Total health expenditure per capita (Intl $):57 Total health expenditure as % of GDP:2.1 Total of People with Diabetes: 336981 Total pregnant women receiving antenatal care: 140,000 (95%) Total deliveries in health facilities: 140,000 (95%) Source: World Health Organization, Geneva, 2003 2.2 OBSERVATIONS AND FIELD VISITS Field visits consisted of large and small group meetings with facility and local health authority management and staff. Inspection of facilities included capital and biomedical equipment condition and maintenance. General sanitary and housekeeping was observed in comparison to other Azeri and NIS norms3 . During facility visits, staff, patients, and family members were interviewed on an ad-hoc basis. Brief observations of operations were made in most cases. Visits to hospitals, clinics, and a “Family Practice” serving IDP/R populations involved interviews and observations of physicians, nurses, patients, and patient families. 2.3 FOCUS GROUPS The Evaluation Team conducted three focus groups consisting of 8-12 individuals who received training in the United States and those who have participated in NIS-wide conferences, workshops, or seminars through the AIHA program. Focus groups had mixed participants from all the partnerships and were structured around common interest areas. There was one focus group comprised of nurses while the other two were made up of physicians from the three partnerships. Each focus group lasted approximately two hours. Focus Group 1 was comprised of primary care physicians and examined issues of sustainability and the potential for fee-for-service. Focus Group 2 was composed of area nurses and examined on needs of the profession, the problems of equipment sustainability, and better patient interaction and care skills. Focus Group 3 was composed of area physicians and physicians involved with the specialty training areas. 3. SUMMARY OF KEY FINDINGS The state of health information in Azerbaijan appears to be more problematic than the norm for countries of the NIS. Much data is inaccurate prima facie, and contradictions among data sources are common. To gain useful information, experience and qualitative information was used to evaluate the validity of the various indicators. Greater confidence was placed on data provided directly by the APOs. Although Partnerships are helping local facilities improve data collection and reliability, this is a nation wide problem and 3 “Norms” in this case refers to the common or dominant state of upkeep that exists in the region, and does not refer to “standards” (preferred or minimum acceptable condition) for these services. Figure 1 General health statistics provided by WHO Development Associates, Inc. Assessment of the American 5 December 2003 International Health Alliance (AIHA) Program in Azerbaijan is best addressed through a national program: The CDC has been very successful in the development and delivery of these national programs. For example, in Armenia, the data collection system has been integrated into a highly effective infectious disease surveillance system. In Kazakhstan, the CDC-led effort to improve health data collection found that to the great surprise of the health system, trauma was the leading cause of death. The extent of the trauma mortality rate was masked because of practices in dealing with deaths outside hospitals. When a victim was dead in the opinion of whoever represented official authority, they would not enter the health system through the hospitals or other care facility. Bodies would be taken either to a municipal morgue or to a private entity for burial. Thus, large numbers of deaths due to accident (and very likely other causes where emergency aid was not summoned) were not recorded by the health system. Azerbaijan may have similar surprises in store, but these will remain unknown until such time as the data problem is solved. 3.1 SCOPE OF WORK SPECIFIED TASKS The Scope of Work for the Evaluation Team (see Annex A) specified a number of tasks to be conducted during the evaluation. The findings related to these tasks are summarized below: Describe certain categories or types of partnerships that are more effective and productive within the Azerbaijan context. Describe what characteristics/factors make them more effective and productive. The three US partners are quite similar – major medical education institutions in cooperative relationships with Azeri institutions. On the Azeri side however, the nature of the partnering institution is wider in range. Although the stated goals of the three partnerships are primarily related to primary care, the point of entry ranges from the national tertiary care hospital to a rural rayon health department. USAID/Baku reports some reticence on the part of the MOH to engage in health care reforms. This can be partially attributed to the effect of U.S. Foreign Assistance Act Section 907, but given the pervasiveness of this behavior in other countries, it is likely more related to the general tendency of centric disassociation 4 in health reform efforts. In primary care (PHC), reproductive health (RH), or maternal and child health (MCH) programs, the tendency is often to apply technical assistance and provide external financial and material resources directly at the target. 4 Centric Disassociation is the author’s term for health reform efforts that focus on the formal decentralization of the health care system or other efforts to shift the balance of power, authority, or resources of the established health system without adequate attention given to the central health authorities (usually the ministry of health) and their need for role change and structural evolution. All too frequently, such reform efforts are perceived as loss dominant by the central authority and are thus highly subject to passive or blatant resistance. Development Associates, Inc. Assessment of the American 6 December 2003 International Health Alliance (AIHA) Program in Azerbaijan If impact and system change were linear, these efforts would be more effective. As it is, the recipients of the intervention establish a direct relationship with the donor organization from which it receives resources. This link can be considerably stronger than that which exists with the central authority. This results in a diminishment of ties to the central authority. The central authority not only does not receive any new resources, but it also loses face to some degree because of its comparative impotence. This coupled with the diminished relationship with the target recipient creates further sensitivities. Often this may only be perceived as vague irritation with the reform effort. Because it is difficult to recognize the nature of the irritant (the shifts in power, relationships, and resources), the reform intervention itself may become identified as the problem. The central authority perceives a loss of power, and in some cases, a loss of its raison d’être. The targeted recipient (e.g. the PHC entity) begins to split its allegiances between the traditional center and the new provider of largess. When required by the donor to act in a manner different from accepted norms, it will often do so in hopes of winning the favor of its benefactor. Its behavior from the perspective of the central authority thus becomes more distant from the norm, and attempts to smooth relations with the center may be seen as duplicitous. The Donor sees its role as provider of self-help assistance and desperately needed goods and services. It also requires structural changes as part of the intervention. The intention of the donor is usually beyond reproach. The non-linear impacts of its interventions can, however, be counter productive. Failure of the central authority to respond favorably to the gifts of technical assistance and resources often causes a measure of resentment on the part of the donor. Efforts that go directly to the targeted recipient bypass the central authority, and thus disrupt the established resource provision and control systems. Whether it does so intentionally, or as the result of poorly understood stress, the central authority may begin to resist the intervention. This will manifest through either not committing centrally controlled resources needed for the counterpart contribution, or, through outright obstruction of approvals or blocking other channels still under central control. The application of pressure elsewhere in the system (i.e. Mission pressure on the executive government) is often the only way to slow the self-reinforcement of this “death spiral.” This resistance to assistance phenomenon occurs all too frequently but is usually misdiagnosed or attributed to other causes. A careful analysis of the AIHA Partnership Model however, can provide powerful insight into the behavioral mechanism operational in this paradigm, and, can provide a solution. The best way to arrest this tendency in donor interventions is to avoid starting the cycle in the first place. The USAID/AIHA program has provided an example of how to do this. By going through rather than bypassing the traditional structures of the health system, AIHA enlists the participation of an important segment of the central authority in the development and implementation of the primary care intervention. Although it is a counterintuitive way to address primary health care, the efforts of the Baku-Houston Partnership best illustrate this model. Development Associates, Inc. Assessment of the American 7 December 2003 International Health Alliance (AIHA) Program in Azerbaijan By partnering with the national tertiary care hospital (Mir Kasimov), the Baku-Houston intervention was established on the basis of peer-to-peer relationships developed among the top echelons of the medical establishment. Although not the same as the Ministry of Health hierarchy, these two entities share many common members and very strong network relationships between the traditional power figures. Originating the interventions of the partnership at this level can avoid creating the perception of bypassing the power structure. Working along the principles of “attraction” rather than advising or dictating, the power figures became involved with the interventions out of personal conviction rather than benign or blatant coercion. These willing participants thus opened the path for inputs of goods and services destined for the PHC level. This is not to say that the other partnerships are not good models for intervention. They have also achieved considerable acceptance. This may likely be due to combined effects of their relatively high points of entry (another major hospital, and an evidently politically well￾connected rayon health department), and the synergistic effect of the other partnerships. The conclusion of the Evaluation Team is that the lesson to be taken from this successful integration into the power structure is that some level of effort must be directed at and through the very top of the hierarchy, and that interrelationships among the partnerships can reinforce the benefits of this acceptance and support of the medical establishment. This top entry intervention approach is particularly important when conducting national level activities. The effects of direct institutional intervention may be sufficient to provide local positive outcomes, but the generalization and acceptance of a reform effort must be clearly identified as involving the top of the hierarchy. Analyze the appropriateness of external and internal criteria for establishing a partnership. The criteria presently used by AIHA to identify appropriate partner organizations appear to be effective. The Evaluation Team concludes that the selections are appropriate for disparate interventions. If the AIHA approach is integrated into a macro reform effort (as recommended), then the selection criteria become more critical. Referring to the explanation of “entry point selection” illustrated above, achieving macro or systemic level impact requires close attention to the principle of entering and acting through the established medical hierarchy. Analyze what makes partnership inputs unique and how do these activities enhance other USAID-funded activities in achieving similar results; thus, resulting in a comparative advantage for sustainable impact. At present, AIHA activities are largely carried out in isolation from the rest of USAID projects. This appears to be the result of the common perception that the partnerships are outside of the usual range of USAID activities, and are more concerned with generating good will through the use of volunteer participants. Based solely on the review of project documents prior to undertaking this in-depth assessment, one might assume that bringing US and NIS experts together through the AIHA partnerships model does not provide the appropriate expertise and experience necessary to solve the health Development Associates, Inc. Assessment of the American 8 December 2003 International Health Alliance (AIHA) Program in Azerbaijan system problems that exist in Azerbaijan and the region. And, further, that the participation times are too short for the participants to gain the necessary skills and experience to be effective. While much of this statement has accuracy, the experience of evaluating the partnerships and seeing them close up has shown that the basic assumptions of this perspective miss the point. The value of the partnerships lies less in the transfer of technology and ideas and more in the ability of the partnerships to gain peer entry into the health system and the decision making hierarchy. If one looks at the AIHA partnerships solely from the perspective of technical assistance and knowledge transfer, then the impact is slight. If however, the partnerships are viewed in terms of the establishment of deep relationships based on trust, shared purpose, peer-to-peer exchanges, and self-motivated participation and acceptance of learning, then the AIHA partnerships are without equal. Compared to typical USAID programs, the realistic potential for technical assistance value is moderate. But when compared in terms of integration into the local system and the ability to attract self motivated participation, the partnerships appear to be superior. If AIHA model partnerships were used in combination with more typical macro or systemic level health reform programs, the payoff would be huge. The strengths of system penetration of the AIHA model would greatly improve the acceptance of the more typical interventions, while the more experienced technical assistance and wider targeting of the health reform model would result in the widening of program impact, very possibly on a rapid national basis. Assess the potential for the current programs to be replicated and the extent to which the partnerships have had a broader impact on the community, region, or nation. The question of replication potential is examined later in this section. The Evaluation Team determined that as long as the confidence level of national statistics is low, there is little hope of being able to gain a quantifiable understanding of the larger impact of AIHA programs. Anecdotal information suggests that impact has been wider than the APOs, but the exact extent is impossible to determine. When the team examined the neonatal resuscitation program, one indicator showed a broader impact of this new technology that is interesting to consider for its value in pointing out non￾linear impact. Although the team expected to find a decrease in neonatal mortality, there was a concurrent decline in perinatal mortality as well. It appears that the conceptual change that was brought about by the introduction of neonatal resuscitation is broader than just saving newborns that die during birth. The statistics collected by Mir Kazimov show a 50 percent decline in the number of “stillborn” infants after resuscitation was introduced. Development Associates, Inc. Assessment of the American 9 December 2003 International Health Alliance (AIHA) Program in Azerbaijan Neonatal Mortality: Mir Kasimov Hospital Year Stillbirth Transfers to Peds Hosp Live Births Perinatal Mortality * Neonatal Mortality ** 1999 2000 2001 2002 2003 (6 months) 1439 1673 1646 1808 1045 46 45 40 38 16 Neonatal Resuscitation Program Implemented 2001 10 9 6 7 3 15.87 11.89 8.47 7.71 5.73 9.03 6.58 4.86 3.87 2.87 * Per 1000 total births ** Per 1000 live births The determination of “stillborn” is that the fetus was not viable when born. The ability to attempt resuscitation caused a conceptual shift in defining this viability at birth. That means that physicians have made a considerable mental shift in deciding whether or not a baby is born “alive” or not. Clearly the ability to try to intervene has led to a broader appreciation of when it is possible to intervene. Figure 2 Analyze the extent to which AIHA special initiatives, including Women’s Wellness, Neonatal Resuscitation, EMS Training, Nursing, Infection Control, AIHA central activities such as NIS-wide conferences/workshops facilitate individual partnerships in meeting their agreed￾upon goals and objectives. 5 AIHA Special initiatives fill a role similar to a more typical USAID technical assistance intervention. While the individual partnerships must by nature “reinvent the wheel,” the special or regional initiatives provide a source of greater experience with appropriate technologies and approaches. Because these initiatives are also participant driven, and they are based on locally developed experience and expertise, the technical content is highly appropriate and practical, and the impact value of the learning experience is reported to be very high. In Azerbaijan, the trans-Caucasus initiative to develop practice protocols (such as asthma treatment) was particularly effective in terms of clinical soundness of the products, and because of the motivational effect of participation in the wide area effort. These types of regional efforts are particularly satisfying to participants because of the recognition conveyed with acceptance of their work, and because of the motivational effect of participants knowing that they are having a wider positive impact than they could ordinarily achieve. Describe reported changes in health care delivery that are attributable to lessons learned by health care practitioners through participating in the partnership programs. In short, what are health care professionals and administrators doing differently due to the AIHA program? 5 There is a concern on the part of USAID/Caucasus that there are too many activities in sharing Initiatives, conferences, and workshops and that it takes a great deal of AIHA/Azerbaijan staff efforts and detracts from implementation of the Partnership’s workplans. Development Associates, Inc. Assessment of the American 10 December 2003 International Health Alliance (AIHA) Program in Azerbaijan The most important effect of the primary care focus of the partnerships has been the profound change it has brought about in the attitudes, comportment, and behavior of primary care physicians and nurses. The team was consistently told both by participants and non-participant informants that the change in these providers has been astounding. Prior to the Partnership intervention, the “Visiting Physicians” and “Visiting Nurses” where considered the bottom of the medical hierarchy, and their role was extremely limited. If any pathology was found (or suspected) by these care givers, they would refer the patient to specialists at the polyclinic or hospital. These primary care providers were in fact not so much care givers as minimally skilled sentinels. With the inception of the AIHA primary care program, the teaching of some very basic diagnostic skills, and, the provision of the “doctor bags” and “nurse bags,” the actual performance of these front-line guardians has changed to a degree hard to imagine. Physicians and nurses are now actually conducting skilled physical exams and providing appropriate primary level therapies. These physicians, nurses, and their supervisors reported that the change was remarkable, with both managers and patients showing a level of respect previously unknown. The increased ability of the care givers to actually provide real care not only resulted in decreased morbidity and mortality, but it boosted the motivation of the care givers. Their successful attitudes, as well as their tangible results, have led to increased respect and recognition, which has in turn led to further improved performance. Instead of the self-reinforcing “downward spiral” so common in other interventions, this effort has resulted in a self-reinforcing positive spiral. Propose overall recommendations to enhance and focus the effectiveness and impact of activities performed which demonstrates a comparative advantage and coordinates with other on-going activities aiming to achieve comparable results within USAID Strategic Objective. USAID Caucasus has an opportunity to achieve great increases in program effectiveness in all health areas simultaneously through the integration of the AIHA intervention strategy into both policy/health reform programs, and into traditional primary care sectoral interventions. As this document has stated, the AIHA strategy has several powerful advantages over the more traditional USAID approaches. At the same time, the AIHA method would significantly benefit from the greater technical depth and experience of the other intervention types. Assess the local management and administrative capacity of the AIHA country office staff and structure and make recommendations to increase overall effectiveness and efficiency. The AIHA staff in Azerbaijan are high caliber professionals who provide significant value to the programs. The sole concern of the Evaluation Team is that any decrease in AIHA administrative or managerial funding would significantly jeopardize the value of the programs. 3.2 IMPROVED HEALTH OUTCOMES Based on an assessment of AIHA partnership provided data, informant opinion, direct observation, and a minimal reliance on national and international data, the three AIHA Development Associates, Inc. Assessment of the American 11 December 2003 International Health Alliance (AIHA) Program in Azerbaijan Partnerships in Azerbaijan appear to be effective in achieving significant improvements to the health of the population of Azerbaijan, particularly women and IDP/Refugees: 4 Basic newborn resuscitation training resulted in a major decrease in infant death in targeted facilities. Figure 3 shows the example of the Mir Kasimov Hospital. Similar results can be found at the Saberabad Rayon Hospital. 4 The creation of a neonatal training unit and capability at the Mir Kasimov center for high-risk deliveries provides significantly improved newborn protection for a major portion 6 of the national population of women at this time of greatest peril. 4 The introduction of family medicine practices has resulted in significant measurable improvements in patient health in the areas of hypertension, gastric ulcers, asthma, and diabetes in partner facilities and service areas (i.e., the model family medicine clinic established in the Narimanov Health District and primary health care providers under the Binagadi District Hospital). 4 The development of a program to teach breast health to physicians and nurses, and breast self-examination to women has provided an extremely low cost method of significantly impacting the pathology of this most prevalent cancer in women. In addition to these specific examples of improved health outcomes, there are examples of system improvements that have direct positive health implications for the Azeri population: 4 The success of the family medicine model has resulted in its official adoption by the MoH and the establishment of an Azeri post-graduate training program. 4 Azeri physicians establishing a Baku/Saberabad partnership have copied the AIHA Model. 4 Local demand for training in cardiovascular care resulted in this training being developed and delivered without direct partnership support. 4 The Partnerships have fundamentally changed the power structure of the health system through the empowerment of area physicians and nurses to provide actual care to patients when their traditional roles have been limited to referrals rather than service delivery. 4 Behavioral changes in health workers generated by the Partnerships have been profound and influential. Morale, comportment, competence, and commitment of participants are greatly improved. 6 Presently 2.3% of all pregnancies are delivered at Mir Kasimov Hospital – most referred as high-risk pregnancies. The projection of wide coverage of major segments of the population is based on the protective nature of a tertiary care referral center. When assessing this coverage, an appropriate analogy is a neighborhood ambulance – all residents of the area have a higher level of protection due to the existence of the resource. Measurements of beneficiaries are thus greater than the actual number of patients who use the ambulance. Development Associates, Inc. Assessment of the American 12 December 2003 International Health Alliance (AIHA) Program in Azerbaijan Comparative Life Expectancy Male - Female Source: Gender in Transition 58.0 60.0 62.0 64.0 66.0 68.0 70.0 72.0 74.0 76.0 Male Female 1989 1990 1991 1992 1993 1994 1995 1996 1997 1998 1999 2000 Although the data available in Azerbaijan is particularly problematic, direct observation of provider behavior, environment, comportment, and physician/patient interaction indicates that the self-reported achievements of the partnerships are valid, and that a good level of confidence can be given to the reported data. The conclusions of these direct observations are supported by comparative observations in non-partnership provider institutions. 3.3 COST EFFECTIVENESS AND EFFICIENCY The Evaluators believe that the Partnership Program in Azerbaijan is both cost effective and cost efficient: The widespread nature of the health improvements provided by the AIHA Partnerships and the appropriate level of technical knowledge sharing have achieved actual reductions to morbidity and mortality that are more tangible than most programs.7 The impact of these organic changes to the medical structure of the country reaches more people more effectively than traditional efforts to bypass the medical establishment or through the externally stimulated development of NGOs designed to replace the existing structure. Although they were outside of the scope of this evaluation, calculations of costs and benefits should prove that the Partnership Model achieves tangible lives saved and morbidity averted at a higher rate per dollar invested than alternative models where benefits accrue on a net future value basis. Figure 3. World Bank Comparative Mortality Study 3.4 FOCUS AREAS OF THE PARTNERSHIPS Epidemiological priorities are not always followed in the selection of Partnership activities: Although it is uncertain because of the lack of statistical data, it is likely that cardiovascular disease is the number one scourge of Azerbaijan. However, it is not sufficiently addressed. Despite the risks of childbirth and is not addressed by the AIHA program, the life expectancy of men is significantly lower than that of women.8 (see Figure 3) The Max Planc Institute in Germany conducted a demographic study in Georgia and Armenia to determine if mortality rates were reliable. Their findings were that because of very heavy undocumented out-migration, the official estimates of total population were significantly too high. 7 This is pure speculation at this point by the author. Experience with USAID health programs in 25 countries since 1977 forms the basis of this conclusion. This cost effectiveness question deserves further examination and should be performed soon by USAID as the potential benefits of greater use of this model are believed to be significant. 8 This conclusion is supported by the World Bank’s Gender in Transition Study, the WHO statistical information for Azerbaijan and the Caucasus, and, USAID/Baku’s own publication on development and humanitarian assistance. Development Associates, Inc. Assessment of the American 13 December 2003 International Health Alliance (AIHA) Program in Azerbaijan Although the reasons were slightly different in the two countries, there were sufficient similarities to lead to a reasonable conclusion that rates in Azerbaijan are incorrect as well. A disturbing conclusion of the study is that given these errors in total population, mortality rates, especially those for men between the ages of 15 and 60 were in reality much higher than official estimates. Given that the official estimates are highly worrisome, an increase in this rate as projected by the study is at the magnitude of an epidemiological crisis. Maternal mortality during childbirth is still a grave problem that is being only marginally addressed as a part of more diffuse Women’s Wellness Center activities. The focus on Pap Smears is of comparatively low value. According to local Ob/gyn specialists, the same results can be obtained during a more complete colposcopy, which does not have the recurrent costs of reagents. Given the very small level of mortality due to cervical cancer, the considerable effort to introduce this new technique should be re-evaluated. 3.5 SUSTAINABILITY Most of the structural and health outcome impacts of the Partnership Model are sustainable by nature: The key factors are that these activities are driven by local demand stimulated by an expanded understanding of the possible, and that they are implemented through the medical establishment. Programs that are implemented through this self-powered mechanism are the essence of local ownership. Material resource sustainability is problematic: The Baku-Portland Partnership has developed an appropriate mix of basic equipment that sets a rational minimum level of primary care diagnostic equipment. The Doctor Bags and Nurse Bags provided by the partnership to area doctors and nurses moving to the family medicine model contain the simplest and most essential items necessary for basic physical examinations. Despite great care to keep material support to a sustainable level, total adherence to this principle is impossible if lives are to be saved. There is a point of technical resource poverty below which even the most gifted physician, nurse, or even public health expert cannot provide assistance to those in need. The Baku-Portland Partnership has found an appropriate level for this minimum rational material resource bottom. Anecdotal evidence of the life saving value of these bags is plentiful. Early problems with non￾local procurement of contents has been corrected under AIHA Baku guidance. Unfortunately, the bags soon lose much of their value. At $70 US per vial of fifty test strips, the nurses and doctors are not able to procure replacement strips for their glucometers. Some strips have been purchased out of Rayon funds for a couple of physicians, but the majority are now without. Careful consideration was given to this situation by the Evaluators (including finding and purchasing strips for an IDP/R area nurse) as to whether the inclusion of glucometers is a sustainable item, and if not, should they be discontinued in future bags. The answer to this question is difficult under present circumstances, and leads to larger questions about AIHA’s relationship to the total health reform effort. The immediate answer is that the Development Associates, Inc. Assessment of the American 14 December 2003 International Health Alliance (AIHA) Program in Azerbaijan demonstrated benefit of these machines is sufficiently high as to warrant their appellation as an essential basic device. In the words of one area nurse, “even if we only received the first supply of strips, and never received any again, there are people who are alive today that would not have been.” A potential value of the dearth of strips may be that it will serve as a partial catalyst for further examination of financial reform within the system. 3.6 TECHNICAL ASSISTANCE OUTSIDE OF PARTNERSHIP EXPERTISE There is a need for skills outside of the US Partner areas of expertise: The experience with the glucometer sustainability issue points to the need for considering outside collaborative technical assistance. During one of the focus groups, the question of trying to recover the cost of replacing the glucometer strips from patients was discussed in depth. The idea in general is contrary to the lamented aspects of the Soviet system, and is heavily colored by the perception of this practice as “bribery” or “corruption.” It was interesting that after 30 minutes of discussion, about half of the focus group members were convinced that the situation demands such efforts no matter what one would prefer. The other half however, could not make the emotional leap past “what was good” and “what was right” about the old system. Despite having a clear, first hand understanding of the consequences of being without essential supplies, they were unable to move past “what should be.” The potential for successful cost recovery will remain low while the present perspective of “bribery” is dominant: The lack of financial restructuring poses a significant impediment on the sustainability of improvements based on material resources. More importantly, the lack of financial and structural reform poses a large burden on the morale of health care workers. Extremely low salary levels make the collection of under-the-table payments for health services inevitable. The lack of drugs and materials means that in all but the most desperate of cases, patients must purchase all needed supplies before procedures can be performed. 9 The level of stress and sense of personal failure caused by this monetary problem is debilitating. Patients refer to all such payments (whether for the purchase of essential supplies or for physician income) as “bribery.” The health workers are in a double bind because of the desperation that necessitates actions seen as personally reprehensible, and, because of the vocal condemnation of some patients when faced with such fees. In the worst of circumstances, physicians must demand that supplies be purchased by patient families because absolutely none are available in the system, and the family’s inability to pay results in the death of the patient. Even those health workers who are able to see the necessity of cost recovery in some form perceive it in terms of greater or lesser evil. Given the present framing of the debate, the question is whether or not “bribery” is necessary. In this frame, the health worker is placed in a lose-lose situation. Under the present mental model, even if the government was to actively implement a program of cost recovery, it would be effectively sanctioning a “bad” thing. A change in the framing of the issue is urgently needed. 9 There is much anecdotal information that indicates that there is a true “needs assessment” basis for the collection of payments. Greater efforts are made to find alternative sources of resources for the most at-risk patients. Development Associates, Inc. Assessment of the American 15 December 2003 International Health Alliance (AIHA) Program in Azerbaijan 4. CONCLUSIONS AIHA is in a unique position among USAID funded projects in its fundamentally different approach to achieving substantial and sustained health improvements. An examination of these differences provides a deep understanding of the mechanisms of behavior change and motivation that are active in health systems development and reform efforts. 4.1 COOPERATIVE DEVELOPMENT AND LEARNING Changes are demand driven by the partners functioning as peers in a voluntary helping relationship. Work plans and technical priorities are developed jointly in a give and take exploratory process where neither party has moral or financial primacy over the other. Either party may increase or decrease its level of participation, and neither is fundamentally at serious risk if negotiations fail. Interestingly, it appears that in the case of the three Azeri Partnerships, the initial expectations of the Baku partners were focused on material support. The Azeri Partners entered into the relationships believing that they had little or no need for technical advice from the US partners, but they were very anxious to gain desperately needed equipment, supplies, and drugs. They entered the partnerships through the desire for material gain, and soon were being exposed to medical paradigms that they were completely unaware of. Seeing new structures and processes that they previously did not know of, they became curious. Seeing the outcomes on patients (and providers) of these new paradigms made them interested in learning more. Thus, the entire process was based on attraction rather than imposed mandate or advice. Exchanges of medical professionals between the US and Baku are the sine qua none of expanding the world view of participants that makes change possible: The Baku partners have been able to conceive of changes to the system because of their increased understanding and belief in what is possible. This factor is critical. When one lives in an environment where certain constraints are dominant, perspectives of alternative paradigms will rarely develop. 4.2 WORKING WITH THE SYSTEM The process works through the established medical hierarchy: Although the AIHA Partnerships were conceived as primary care efforts, they approached the process through the traditional medical power structure rather than bypassing the hospital centric system. Fundamental changes may occur in a human system through evolution and learning adaptation, or, through revolution or phase change during punctuated equilibrium (a major change in a population through extinction or other radical change that provides a rapid and fundamental modification to the status quo). In other words, fundamental change can happen comparatively easily with the cooperation of the structure of the system, or, with difficulty, disruption, or not at all when attempted through bypassing the established structures. It is very important to note that there are effectively two major power structures in a national health system (and arguable several more). The visibly dominant system is the official power structure, in the form of the ministry of health. This is the structure that provides the titular leadership of the health care system, and tends to dominate the governance of provider facilities and organizations. Development Associates, Inc. Assessment of the American 16 December 2003 International Health Alliance (AIHA) Program in Azerbaijan The second major power structure in a health system is the medical/technical hierarchy. These are the physician leaders who set the norms of the system and determine what is and is not appropriate activity. In most countries, this hierarchy is dominated by the medical specialists of the tertiary care level and possibly by the academic level. This is particularly true in countries of the NIS where decision making and behavioral control of the entire system is dominated by this elite. The AIHA approach enters into the health system at the higher levels of the medical/technical hierarchy, and relies on attraction and suasion to move changes on through to the primary care levels. Traditional USAID projects often avoid this hospital based hierarchy and proceed either through the titular hierarchy (MOH) or bypass both power structures and intervene directly with the primary care level. Empowering the primary care level is a laudable goal, and it is achieved much more easily with the blessing of the power establishment. 4.3 NETWORK EFFECTS – SPREAD BY ATTRACTION The attraction principle is conducive to institutionalized change: The Baku-Portland Partnership has demonstrated a highly sustainable occurrence of institutionalized reform through the success of the primary care physician skills activity. Improvements to patient outcomes and physician productivity have been sufficient to stimulate high level interest in the program, and to want to implement it on a national basis. The Chief Doctor of the Narimanov Rayon was appointed to head a new MOH department of Family Medicine. This appointment and the beginning of the post-graduate program in Family Medicine were brought about through the interest generated by the success of the practice model and the attractiveness of the benefits of the model to the power structure of the health system. Interest is generated and spread through the Network Hub phenomena: The term “viral marketing” is used to describe interest in a product or service generated through the observation and word-of-mouth of early adapters. Sociologists have recognized the key role of influential individuals in the speed and ultimate success of idea or technological spread. These early adopters will very greatly lead acceptance if they are well connected socially. Rather than simple popularity, these opinion leaders are influential because of widely respect for their opinion or status. Looking at local social systems as networks, these opinion leaders function as hubs with established connections to their peers and associates. Because the AIHA model works with the medical/technical hierarchy, the high correlation between elite status and hub status means that the ideas and technologies that are accepted and adopted will be more likely to be embraced by others in the network. Attractiveness of the AIHA generated behaviors is high because of the link to health worker needs characteristics: People entering (or staying) in the health professions tend to have comparatively higher tendency towards altruistic behaviors and needs. Providing opportunities/mechanisms to satisfy these higher level needs will likely result in reinforcing behavior. 4.4 SYSTEMIC BEHAVIORAL CHANGE The Chief Doctor from the Mir Kasimov Republican Hospital and the Baylor Partners came up with the idea to pilot a Partnership Model relationship between Kasimov Republican and the Development Associates, Inc. Assessment of the American 17 December 2003 International Health Alliance (AIHA) Program in Azerbaijan Saberabad Rayon hospitals. Mir Kasimov has a pre-existing mandate to provide specialty/tertiary services to the entire country. This role has traditionally been performed through regular visits of Kasimov Specialists to the various rayon hospitals throughout the country. These visits were conducted in a manner where the specialists would examine the referred local patients, and then make the decision to treat on the spot or send to Baku for hospitalization. The old modality of this “visiting experts” regime is very much in keeping with the legacy soviet “refer up” process where each level of physician passes along all but the simplest cases. The effect on this model has been devastating over the years of its use. Major flaws of this system include: 4 Concentration of clinical expertise at the top of the system: Rather than tertiary specialists being a resource of last resort, they become the dominant authorities in the delivery of all care of modest to high complexity. 4 Lack of clinical authority at all but the highest levels: Primary care physicians and local specialists from hospitals and polyclinics lose the clinical authority to make decisions. 4 Demoralization/apathy in the primary and secondary care levels: Without the authority to take clinical actions, the tendency is to lose interest and no longer try to effect patient outcomes. 4 Skills deterioration: As skills are not used, they are lost. This is evidenced in the therapists that require remedial training in conducting basic physical exams. It is also evident in secondary specialists that not only lack the technology to The new Partnership Model offers the technical ability and authority to perform. Changes to the power structure stimulates and improves bi-directional communication between health workers and administrative authorities: Communication has become more intense and constructive as physicians and nurses feel that relationships have been established with higher level authorities because of increased interest in the progress and problems of care givers. 4.5 SUSTAINABILITY THROUGH REPLICATION The partnership model is effectively self-replicating: The Baku-Houston Partnership has provided an excellent example of this phenomena of imitation of successful strategies. The Mir Kasimov Republican (i.e. tertiary) Hospital had an established relationship with the Saberabad Rayon Medical District through its visiting specialists role. With the support of the Baylor Partner and the encouragement of AIHA Baku, the Mir Kasimov Hospital decided to form its own partnership with Saberabad along the AIHA model. Successful initiatives such as the Neonatal Resuscitation program and the Women’s Health activity were taken to Saberabad and presented by the Baku Partners. As in the original partnership, local demand soon brought about additional activities that were not planned or financially supported by AIHA. The Saberabad partners have a huge need for increased capabilities in managing cardiovascular disease. Their request, and the agreement of the Mir Kasimov partners resulted in the design and delivery of cardiovascular training and support efforts. Development Associates, Inc. Assessment of the American 18 December 2003 International Health Alliance (AIHA) Program in Azerbaijan There is concern that the Saberabad example is the exception rather than the rule in terms of auto-generated replication of AIHA initiatives, and that the deepening financial crisis makes it difficult to obtain government support for continued or expanded initiatives. This begs a fundamental question: How much behavioral change or program implementation is possible given the absence of resources? Because of the literal life and death nature of health sector activities, health professionals, particularly those in the service of international donors feel a strong moral imperative to keep seeking improvements and changes, no matter what the realistic availability of human, financial, or capital resources. The NIS has the good news/bad news situation of not fundamentally understanding the cost of human resources. If people are employed at an organization, the perceived marginal cost for their labor is zero. Therefore, it is comparatively easy to get staff assigned to new initiatives as long as their absence from their previous duty is not too disruptive. Efforts to replicate AIHA initiatives, or any other health reform initiatives should emphasize activities that rely primarily on human labor and have small material, financial, or capital costs. That said, there are few activities that are truly free from non-labor costs. Initiatives that function mainly on the motivation of participants and are limited in financial requirements will soon find that motivation disappears if material resources are absent. Even the most basic of curative or preventive services and activities have important non-human cost components. Altruistic drive will motivate healthcare workers to keep trying for significant amounts of time, but there is a point where such futile efforts end. Since real financial resources in Azerbaijan are not zero, those that are available are allocated on the basis of perceived return (not necessarily a common good). Two fundamental strategies will encourage the investment of those scarce resources on AIHA or other health activities. The first strategy is showing a demonstrable return for the investment. This is not always framed in social terms but is frequently based on the inherent network power structures of the system. The demonstration of benefit must be evident to the decision makers, and it must be based on their criteria rather than an external metric. The AIHA model of attraction is inherently suited for this as it is dependent on the participants determining need and priority (criteria for determining benefit). The important point is to make sure that to the degree possible, the actors in the system that control resources (both directly and indirectly) are included as participants. Saberabad works because the people who control the resources (at least those needed) are the people who participate and see the value of the activity. As stated previously, the medical hierarchy is not exactly the same as the official government hierarchy, but it is close in terms of common members and social influence. The logical way to increase participation of resource decision makers is to figure out who they are and target them for inclusion. The Family Medicine initiatives in the Narimanov Medical District are able to receive government resources both because of the limited resources controlled directly by the Director, but also because of the influential position held by the Director in the Ministry of Health and the national government. Development Associates, Inc. Assessment of the American 19 December 2003 International Health Alliance (AIHA) Program in Azerbaijan The second principle that is critical to this strategy is to increase the amount of resources available. Cost Recovery and health financing reform is essential. The AIHA model does not have a comparative advantage in this technical area, but, it could provide an entry point into the decision power structure vastly superior to the typical reform project mechanism. When the AIHA model is integrated with the health reform model, there is a realistic possibility of breaking the taboo that holds back this critical reform. 5. RECOMMENDATIONS Qualitative and limited quantitative analyses indicate that the disparity between the IDP/Refugee population and the general population may have largely disappeared. Unfortunately, this realignment seems due primarily to a decline in welfare of important segments of the general population rather than to major improvements in the welfare of the IDP/Rs. The prominent disparity now appears to be within the IDP/R population as income levels greatly diverge for individual families much as is occurring in the general population. USAID is encouraged to carefully monitor this situation and continue adjusting program targeting accordingly. 4 The Partnership Program should be continued and expanded as part of USAID’s future strategy: Few programs provide this level of benefit for the size of the investment. 5.1 INCREASED ATTENTION TO EPIDEMIOLOGICAL PRIORITIES Cardiovascular disease should receive increased attention: The severity of the problem demands that rapid impact measures be sought in addition to long-term life-style interventions and promotion. The present level of mortality is at a crisis stage and must be met with primary care level therapeutic interventions. Consideration should be given to the idea of developing “primary care cardiologists” equipped with simple ECGs for screening and diagnosis and the development of a self-sustaining (i.e. revolving fund) medication mechanism. Women’s Wellness Center focus should be concentrated on the high priority need to reduce morbidity and mortality related to child birth: The present level of maternal mortality demands that efforts not be diluted by other activities associated with wellness centers. Other “wellness” activities such as the programs for female staff of Mir Kasimov should be spun-off as an independent business (possibly not-for-profit). Neonatal Resuscitation Program should be extended and complemented by topics addressing common conditions of newborn mortality: The alarming rate of neonatal mortality and the faulty status of the referral system for high risk pregnant women and newborns indicates that improvements in basic clinical knowledge of medical personnel at the delivery settings thought country would be of great benefit. Male mortality in the productive age groups is at a crisis level and, therefore, should receive increased attention: Estimates of death in men aged 15 to 60 are very likely low by a Development Associates, Inc. Assessment of the American 20 December 2003 International Health Alliance (AIHA) Program in Azerbaijan considerable margin.10 USAID’s commitment to prioritize on the basis of epidemiological need must take priority over gender preference. AIHA partnerships should be encouraged to examine this crisis with the medical/technical weight and skill of its US and Azeri Partners. Cardiac emergencies and trauma are likely the leading causes of death in Azerbaijan: Creating an Emergency Medical Services Training Center on the AIHA model will result in major reductions in morbidity and mortality. 5.2 STRUCTURAL CHANGES Partners should increasingly focus on the wider dissemination of improvements: The imitation of partnerships with peripheral health providers as is being done with Saberabad should be supported and become a priority for all activities. If present funding permits, the Model Primary Care Clinic being developed in the Narimanov Rayon should be fully assisted by the Portland Partnership, and receive input from the Baku￾Richmond Partnership: The dedication and level of effort exhibited by the physicians developing the breast health, asthma, mental health, and other model primary care programs are commendable. The expanded profile of Area Physician and Nurse capabilities and areas of practice will result in major nationwide improvements in primary care as they are disseminated. Expansion of patient and community outreach practices piloted at the clinic are ready for wide scale dissemination. Full and continued support should be given to the implementation of the Family Medicine Post-Graduate Training Institute at the Model Clinic. Greater communication and joint actions between the Partnerships is encouraged: AIHA and USAID should link the efforts of the Baku-Portland and Baku-Richmond Partnerships. Such a linkage at both the Partnership level and at the local Baku Rayon level would result in significant efficiency gains and increased dissemination of advances. Duplication of efforts would be minimized and each partner could focus on different specialty areas. One area for essential collaboration is in the development of the Family Medicine Curriculum. Clinical Practice Guidelines, Evidence Based Medicine, and Learning Resource Centers support improved primary care practice: USAID should reinstitute AIHA “Cross-Partnership” programs of guideline development and inter-Caucasus collaboration. 5.3 INVOLVEMENT WITH OTHER USAID PROGRAMS The AIHA Model should be strengthened through incorporation of outside technical assistance from one of the USAID Flagship Programs: Cost Recovery and financial reforms are greatly needed in Azerbaijan, and PHR Plus or possibly the POLICY Project would be appropriate sources for this type of assistance. Technical assistance in cost recovery on a local basis (theoretical legal authority is already present in the rayons) would be the essential minimum involvement. 10 Demographic analysis of the Max Planc Institute provides convincing evidence that large inaccuracies in total population numbers caused by undocumented out-migration from the Caucasus leads to erroneously low estimates of mortality rates. Development Associates, Inc. Assessment of the American 21 December 2003 International Health Alliance (AIHA) Program in Azerbaijan USAID/Caucasus (as well as USAID programs in many other parts of the world) would significantly increase the effectiveness, efficiency, sustainability, and depth of impact if a three part strategy was adapted in future programming. The AIHA approach has clearly demonstrated that it can achieve a depth of impact and cost efficiency beyond the norm because of the principles of “attraction” and entry through the medical hierarchy. It is not however, a total health sector strategy. Certain policy level issues are clearly needing to be addressed (such as cost recovery) in order for the program to achieve maximum lasting benefit. In addition, a broader arsenal of proven PHC strategies needs to be applied in the target areas. While AIHA has achieved significant impact with the national adaptation of “family practice,” there are highly developed approaches that have been implemented by USAID cooperating agencies in other parts of the world. These strategies would be mutually reinforcing if combined with AIHA methods. AIHA is an effective program, and provides important lessons that are essential for the broader USAID to adopt. The future USAID Health Sector Strategy should be built around a three￾pronged approach based upon the AIHA model and more traditional policy reform and primary health care interventions. Assessment of the American A-1 December 2003 International Health Alliance (AIHA) Program in Azerbaijan ANNEX A SCOPE OF WORK I. PURPOSE The primary purpose of this assessment is to assess the effectiveness, accomplishments and impacts of AIHA’s activities in Azerbaijan. A secondary purpose is to assess how/whether these activities both build upon the network of USAID-funded health activities as well as contribute to the wider accomplishments that other donors and the GOAZ are striving to achieve in the health care sector. This analysis supports the current Strategic Objective 3.1 – Reduced Human Suffering in Conflict-Affected Areas, and Intermediate Result (IR) 3.1.1.3 – Communities have access to better quality services. This analysis will help inform USAID/Caucasus/Baku as it prepares its strategy for the period after FY 2004. II. BACKGROUND Economic trends worsened for most people after Azerbaijan obtained independence from the Soviet Union. This decline was exacerbated by the armed conflict over Nagorno-Karabakh that resulted in the displacement of 650,000 people from their homes and the influx of additional 200,000 Azerbaijani refugees from Armenia. The maternal mortality rate increased four-fold between 1990 and 1998 from 9.3 per 100,000 live births to 41 per 100,000 live births (Statistical Yearbook of Azerbaijan 1999). Unofficially, rates are quoted as 78-80 per 100,000 live births. The overall under-five mortality rate was 38 per 1000 in 1997, which is high compared with western standards of an average of 6 per 1,000. Morbidity and mortality rates appear to have ceased their upward spiral and the infant mortality rate has been stable at 20 per 1000 live births since 1997, according to the State Statistical Committee. This figure is considered unusually low due to Azerbaijan not using the international standard for classifying live births. Since 1993, USAID has assisted Azerbaijan in meeting the critical challenges of economic and democratic transition, and provided humanitarian assistance to the most vulnerable groups, especially IDPs and refugees. Health initiatives in Azerbaijan comprise a small but important component of the Mission’s assistance portfolio. The major vehicle for providing social/health assistance is the Azerbaijan Humanitarian Assistance Program (AHAP), implemented through an umbrella cooperative agreement sponsoring programs of 7 major US NGOs. Outside the AHAP framework, three US-Azeri health partnerships within the AIHA program work to upgrade medical skills and provide Azeri health professionals with exposure to the American health care system. As a result of a free and open competitive process, cooperative agreements (a basic agreement, a regional-wide agreement, and separate agreements for the Caucasus, Central Asia, West/NIS, and Russia) were awarded to AIHA in September 1998. When the basic agreement was awarded, USAID concurrently awarded five separate three-year sub-agreements for activities in individual countries or parts of the NIS. These separate cooperative agreements included a Development Associates, Inc. Assessment of the American A-2 December 2003 International Health Alliance (AIHA) Program in Azerbaijan three-year agreement for the Caucasus, covering AIHA programs in Georgia, Azerbaijan and Armenia. Due to the rigorous socio-political conditions and Section 907 limitations the AIHA partnerships have been implemented under extremely difficult circumstances in Azerbaijan 11. At that time USAID selected the partnership model as one of the components in its program to foster more effective and efficient delivery of health care services in Azerbaijan. The greatest challenge to partnership success and sustainability was and remains to be, the availability of public and private resources. This challenge has been exacerbated by the deepening fiscal crisis in Azerbaijan. The original 3-year cooperative agreement contained a provision for a two year extension. AIHA requested this extension in January 2001 and the cooperative agreement was extended for an additional two years. The Caucasus agreement is currently scheduled to end on September 30, 2003 and has a Total Estimated Cost of $20,537,273 with $4,755,000 comprising the Azerbaijan portion The basic agreement also included a provision for an option for a 5-year extension of all agreements issued under the basic agreement. AIHA has already expressed interest in exercising the extension option. As of May 2003, AIHA had established three partnerships in Azerbaijan: Baku-Houston: (Established January 2000). Overall goal: to improve medical services to refugees and IDPS in targeted camps and communities served by the Mir Kasimov hospital in Baku and the Sabirabad Rayon hospital in Sabirabad. Specific objectives include: establish a neonatal resuscitation center to train health professionals; develop and disseminate practice guidelines according to international standards; and collaborate with other partnerships in Azerbaijan to provide education and training in neonatal resuscitation. Also, the partnership aims at shifting patient-oriented activities to rayon hospitals serving IDP camps in their catchment areas and establishing a women’s wellness training center to address maternal and child healthcare concerns among refugees/IDPs. Baku-Portland: (Primary Health Care – Established March 2000) Overall Goal: Develop a comprehensive, community-based, primary care system designed to meet the basic health care needs and improve the health status of IDPs and refugees in the Narimanov Health Care District in Baku, Azerbaijan. Specific objectives include: Enhance the ability of the Narimanov Health District to collect, store, organize and analyze basic socio-demographic and health status information on the IDP/refugee population in the district; enhance and adapt the infrastructure of the Health District to provide a solid base for primary care; improve the effectiveness of the nurses serving the IDP/refugee population; and integrate a mental health program into the model primary care clinic. 11 Until FY2003, all U.S. Government-funded activities in Azerbaijan were subject to the provisions of Section 907 of the FREEDOM Support Act (FSA). Section 907 states that, “United States assistance under this or any other Act may not be provided to the Government of Azerbaijan until the President determines, and so reports to Congress, that the Government of Azerbaijan is taking demonstrable steps to cease all blockades and other offensive uses of force against Armenia and Nagorno-Karabakh.” In FY 2003, the President exercised the authority provided in the FY 2003 Appropriations act and waived Section 907.. Development Associates, Inc. Assessment of the American A-2 December 2003 International Health Alliance (AIHA) Program in Azerbaijan Baku-Richmond: (Primary Health Care-Partnership Established March 2000). The overall goal of the partnership is to improve the health status of the local population including refugees and IDPs accommodated in the Binagadi District of Baku city through socially oriented, public and primary health care programs. Specific objectives include: improve the capacity of the Binagadi Health Department to collect, enter, and analyze data on the health status of the local population including refugees and IDPs; improve the quality of medical care provided by area doctors and nurses to refugees and IDPs through training in clinical assessment skills and sharing evidence based medical guidelines, and; to improve the quality of medical information provided to the population on infectious disease, personal hygiene, sanitation, and general prevention in their home environment. In addition to developing and implementing the above-mentioned partnership programs, AIHA also supports a number of related activities that facilitate and promote inter-partnership communications and synergy, joint partnership initiatives, sharing and dissemination of information, and replication of centers such as Learning Resource Centers, EMS and Nursing Training Centers. III. GOAL The goal of AIHA/Azerbaijan partnership program assessment is to analyze the general accomplishments of the project--successes, constraints and failures, and to set the stage for future USAID participation in partnership activities in Azerbaijan. The team’s findings and recommendations will be used by USAID/Caucasus/Baku as a tool in decision making and planning of future health partnership strategies. IV. OBJECTIVES AND TASKS As stated above, the independent assessment team will assess the current AIHA partnership portfolio. In addition, the team will review AIHA country office administrative capacities and structure. Focal issues for the assessment are: · Accomplishments of the AIHA Partnership/Volunteer Program. · Contribution to USAID social/health sector goals. · Demonstrable impact in the field. · Value of contribution relative to level of investment. · Sustainability. · Replication. · Effectiveness of AIHA implementation and management. The team will assess partnership activities to identify which activities demonstrate a comparative advantage in its technical field and provide recommendations on how to build upon and focus these activities for future expansion. Under this objective are the following tasks: Development Associates, Inc. Assessment of the American A-3 December 2003 International Health Alliance (AIHA) Program in Azerbaijan ÿ Describe certain categories or types of partnerships that are more effective and productive within the Azerbaijan context. Describe what characteristics/factors make them more effective and productive. ÿ Analyze the appropriateness of external and internal criteria for establishing a partnership. ÿ Analyze what makes partnership inputs unique and how do these activities enhance other USAID-funded activities in achieving similar results; thus, resulting in a comparative advantage for sustainable impact. ÿ Assess the potential for the current programs to be replicated and the extent to which the partnerships have had a broader impact on the community, region, or nation. ÿ Analyze the extent to which AIHA special initiatives, including Women’s Wellness, Neonatal Resuscitation, EMS Training, Nursing, Infection Control, AIHA central activities such as NIS-wide conferences/workshops facilitate individual partnerships in meeting their agreed-upon goals and objectives.12 ÿ Describe reported changes in health care delivery that are attributable to lessons learned by health care practitioners through participating in the partnership programs. In short, what are health care professionals and administrators doing differently due to the AIHA program? ÿ Assess the tool through which AIHA staff monitors and evaluates the adequacy and appropriateness of staff being selected for training. ÿ Propose overall recommendations to enhance and focus the effectiveness and impact of activities performed which demonstrates a comparative advantage and coordinates with other on-going activities aiming to achieve comparable results within USAID Strategic Objective. ÿ Assess the local management and administrative capacity of the AIHA country office staff and structure and make recommendations to increase overall effectiveness and efficiency. Specific issues to be addressed per each partnership entail, but are not limited to: Baku-Houston Partnership: · What progress has been made in establishing a neonatal resuscitation center to train health care professionals? How many health care professionals have been trained? To what extent have the services provided by the trained professionals improved? What are they doing differently since completing the training? What additional training (in terms of content or pedagogical techniques) do they believe they need for improving their services? What has been the impact on the beneficiaries? · Have practice guidelines (according to international standards) been developed and disseminated? How many professionals have received the guidelines? To what extent have they used them to improve the services they provide? Examples? Are the guidelines appropriate for the services they provide? How could they be improved? · To what extent has there been collaboration with other partnerships in Azerbaijan to provide education and training in neonatal resuscitation? 12 There is a concern on the part of USAID/Caucasus that there are too many activities in sharing Initiatives, conferences, and workshops and that it takes a great deal of AIHA/Azerbaijan staff efforts and detracts from implementation of the Partnership’s workplans. Development Associates, Inc. Assessment of the American A-4 December 2003 International Health Alliance (AIHA) Program in Azerbaijan · What progress has been made in shifting patient-oriented activities to rayon hospitals serving IDP camps in their catchment areas and establishing a Women’s Wellness Training Center to address maternal and child healthcare concerns among refugees/IDPs? To what extent have these activities resulted in improving services? What are some of the perceptions of the beneficiaries regarding improvements in services? Baku-Portland Partnership: · What progress has been made in developing a comprehensive, community-based, primary health care system? · To what extent has the new system resulted in meeting the basic health care needs and improving the health status of IDPs and refugees in the Narimanov Health Care District? What are some of the perceptions of the beneficiaries regarding improvements in services? · What progress has been made in enhancing the ability of the Narimanov Health District to collect, store, organize and analyze basic socio-demographic and health status information on the IDP/refugee population in the district? To what extent has the information collected been used for decision-making? Examples? · What progress has been made in enhancing and adapting the infrastructure of the Health District? What changes have been made? How has it made a difference in meeting the basic health care needs and improving the health status of IDPs and refugees in the District? What are some of the perceptions of the beneficiaries regarding improvements in services? · To what extent has the effectiveness of the nurses serving the IDP/refugee population been improved? Examples? What are some of the perceptions of the beneficiaries regarding improvements in services? · What progress has been made in integrating a mental health program into the model primary care clinic? What efforts have been made in establishing a mental health program that is built on the Azerbaijan culture and the mental health needs of the beneficiaries? To what extent has this program been used by beneficiaries? What has been the impact on the beneficiaries? What are some of the perceptions of the beneficiaries regarding the services provided in this program? Baku-Richmond Partnership: · To what extent has the health status of the local population including refugees and IDPs improved since the beginning of the partnership? · What progress has been made in improving the capacity of the Binagadi Health Department to collect, enter, and analyze data on the health status of the local population including refugees and IDPs? To what extent has the information collected been used for decision-making? Examples? · What progress has been made in providing area doctors and nurses training in clinical assessment skills and sharing evidence based medical guidelines? How many doctors and nurses have been trained? How many have received the guidelines? To what extent has the quality of their services improved? Examples? · What progress has been made in improving the quality of medical information provided to the population on infectious disease, personal hygiene, sanitation, and general Development Associates, Inc. Assessment of the American A-5 December 2003 International Health Alliance (AIHA) Program in Azerbaijan prevention in their home environment? What are some of the perceptions of the beneficiaries? To what extent can they understand the information provided on these issues? To what extent has it made a difference in their health practices? AIHA’s Partnership Strengthening Activities: · What activities have been implemented that facilitate and promote inter-partnership communications and synergy? What has been the impact? Examples of changes in communication and synergy? · What activities have been implemented that facilitate and promote joint partnership initiatives? How many new initiatives have begun? What is the quality and appropriateness vis-à-vis health needs of the new initiatives? · What activities have been implemented that facilitate and promote sharing and dissemination of information? To what extent has this information been used? · What progress has been made in replicating centers such as Learning Resource Centers, EMS and Nursing Training Centers? V. METHODOLOGY Information will be collected mainly through personal and/or telephone interviews with key informants from AIHA field representatives, GOAZ officials, Azerbaijan-based AIHA partners, other USAID-funded implementing partners, U.S.-based partners, and USAID/Caucasus staff. Focus group discussions should be held with small groups of returned trainees from the United States and those who have participated in NIS-wide conferences, workshops, or seminars. Interviews and group discussions will collect descriptive data on AIHA’s organizational and management capabilities, and the technical outputs/impacts of partnership exchanges and participation in conferences, workshops, and seminars. Team will also conduct field trips to selected activity sites in Azerbaijan to observe project implementation, verify reported information, and assess performance and progress towards accomplishment of program results and strategic objectives. Team will talk to health professionals who were recipients of trainings, to identify how their performance, attitude and/or provided services have been changed. During field trips, the team will also talk to beneficiaries to find out how services have been improved or not. US partners not present in Azerbaijan at the time of the assessment should be interviewed by phone prior to the teams departing for Azerbaijan. In the even that the consultants are located close to one of the US partners, they may arrange personal interviews before leaving. As additional questions may arise in Azerbaijan in the course of the assessment that require clarification with US partners, the team should remain in e-mail contact with the US partners while in Azerbaijan. USAID/Caucasus staff member(s) may accompany the review team while in Azerbaijan as observer(s) throughout the assessment, as necessary. The Team Members will then assess the outcomes of each set of interviews and group discussions, synthesize the results, and provide/develop a set of findings, lessons learned, and recommendations for future partnership activities. A final briefing near the end of the consultancy will be made to the USAID/Caucasus/Baku Coordinator’s Office management staff. Development Associates, Inc. Assessment of the American A-6 December 2003 International Health Alliance (AIHA) Program in Azerbaijan Interim briefings, as appropriate, may take place in Baku with Mission staff participating on the assessment team. VI. PERSONNEL AND LEVEL OF EFFORT REQUIRED The team will consist of two consultants. All team members will be approved by USAID/Caucasus/Baku. Experience within the former Soviet republics, especially the Caucasus region will be an added advantage. The team will require two specialists. Based upon qualifications, a Team Leader will be named and shall be responsible for the successful management of the assessment and the production of all deliverables. The Team Leader must have USAID assessment experience. Experience with USAID programs and with assessments are important, but not mandatory, qualifications for the other team member. Program Analyst The program specialist will be responsible for assessing the management effectiveness of the AIHA program, as well as of the monitoring and evaluation plan. Recommended Qualifications - Technical expertise and experience in monitoring and evaluation tools and methods, organizational management, and partnership models. Azeri or Russian language capability and experience in analysis in the former Soviet Union is desirable. Health Analyst Recommended Qualifications: Technical expertise and experience in delivering and managing health activities and in international health program planning and management. Knowledge of the health system of the former Soviet Union is also required. An MPH is preferred, as is Azeri or Russian language capability and experience in analysis in the former Soviet Union. VII. DELIVERABLES The activity will be undertaken from 09/23/2003 to 11/21/2003. Prior to the team’s departure to Azerbaijan, preliminary briefings will be scheduled at USAID/Washington and AIHA headquarters in Washington, D.C. The draft report will be presented to USAID/Caucasus - Baku before leaving the country. USAID will provide comments and suggestions within ten days of receiving the draft. The Final Report, incorporating the Mission comments (4 bound copies and an electronic version in Word on a c.d. rom) will be provided to the Mission shortly thereafter but not later than ten days. The report should be concise, not exceeding 25 pages plus appendices, as needed. The team leader will be responsible for completing the report on time and submitting it to the Mission. The final report should include the following sections: Executive summary; Background; Methodology; Description of activities; Summary of key findings; Conclusions; Recommendations; Annexes. Development Associates, Inc. Assessment of the American A-7 December 2003 International Health Alliance (AIHA) Program in Azerbaijan VIII. LEVEL OF EFFORT (TENTATIVE) The level of effort will be approximately as follows: Washington interviews&Background Reading 4 days Travel Days 4 days Interviews/Discussions and report writing in Azerbaijan 12 days Analysis & Final Report Writing 6 days Total number of working days: 26 days IX. ADMINISTRATIVE AND LOGISTICAL ARRANGEMENTS Valerie Ibaan and Gulnara Rahimova, Humanitarian Project Management Specialists will be the official contacts for the team in Azerbaijan. Air Travel to and from Baku, Azerbaijan is authorized. USAID/Baku will provide support in scheduling meetings and site visits. Office space and equipment. Because office space is limited, the team will not be provided with temporary workspace at USAID Baku. The team leader will be required to have a personal computer and should arrange temporary workspace and other assistance with the local administrative/logistical support providers, as needed. No access to USAID Azerbaijan services or facilities will be provided. Transport. Local transport and per diem will be reimbursed at the USAID rate for Azerbaijan on a daily basis. Airline tickets will be reimbursed on a cost basis, upon submission of official receipts. Work week: A six-day workweek is authorized, without any premium pay. X. EXPECTED RESULTS USAID/Caucasus is looking for pragmatic and actionable advice and recommendations based on the team’s assessment in Azerbaijan. The Mission will be the primary user of the information to assist in decision making on future partnership strategies. Given the limited USAID resources in the health assistance area in Azerbaijan, USAID/Caucasus needs to know where the greatest potential for effective and sustainable partnerships exists. The assessment results will be disseminated to AIHA for similar decision making processes. XI. REFERENCES 1. Report of the Continuing Evaluation Panel, “The American International Health Alliance Partnership/Volunteer Program in the Newly Independent States and Central and Eastern Europe”– June 30, 2001 Development Associates, Inc. Assessment of the American A-8 December 2003 International Health Alliance (AIHA) Program in Azerbaijan 2. LTG Associates, Inc. and TvT Associates, Inc., “Social and Health Assessment of Residents, Refugees and Internally Displaced Persons in Azerbaijan” – February, 2002 3. International Medical Corps, “Two Part Baseline Survey on Primary Health Care and Population Health Needs and Utilization of Health Care Services” - August 2000. 4. USAID Azerbaijan Strategic Plan, 2001-2003, May 2000. AIHA Program in Azerbaijan.doc/EvaIQC-35