MID-TERM PERFORMANCE EVALUATION OF USAID/IRAQ PRIMARY HEALTH CARE PROJECT June 6, 2013 This document was produced for review by the United States Agency for International Development. It was prepared on behalf of The QED Group, LLC, under Iraq Personnel Support Services (Manpower II) Task Order number 267-O-00-08-00507-00. MID-TERM PERFORMANCE EVALUATION OF USAID/IRAQ PRIMARY HEALTH CARE PROJECT 1 Iraq Personnel Support Services (Manpower II) Contract Number: 267-O-00-08-00507-00 Mid-term Performance Evaluation of USAID/Iraq Primary Health Care Project June 6, 2013 Submitted by: The QED Group, LLC 1250 Eye Street, NW, Suite 1100 Washington, DC 20005, USA Tel: +1.202.521.1900 Fax: +1.202.521.1901 www.qedgroupllc.com Baghdad Office: Victor 2 Compound Quarter Al Qadisiyah, Section No 604 Street No 57, House No 3 (QED Villa) Tel: +964.780.956.6996 DISCLAIMER -– The author’s views expressed in this publication do not necessarily reflect the views of the United States Agency for International Development or the United States Government. ACRONYMS ANC Antenatal care ARI Acute respiratory infection BCC Behavior change communication BHSP Basic Health Service Package CBO Capacity Building Office CoP Chief of Party COR Contracting Officer’s Representative DO Development Objective DoH Directorates of Health EDL Essential drug list EKG Electrocardiogram EmONC Emergency obstetric and neonatal care FSN Foreign service national GoI Government of Iraq HIS Health information system HSS Health systems strengthening IDP Internally displaced person IEC Information, education, and communication IMCI Integrated management of childhood illness IR Intermediate Result KAB Knowledge, attitude, behavior (survey) LHC Local Health Committee MA Medical Assistant MCH Maternal and child health MoH Ministry of Health MSI Management Systems International NCD Non-communicable diseases NGO Non-governmental organization PHC Primary health care PHCC Primary health care center PHCPI Primary Health Care Project Iraq PMP Performance Management Plan PNC Postnatal care PPP Public-private partnerships QI Quality improvement RTI Reproductive tract infection SOW Scope of work STI Sexually transmitted infection TAG Technical Advisory Group TOR Terms of reference TOT Training of trainers TWG Technical working group URC University Research Company, LLC USAID United States Agency for International Development UTI Urinary tract infection WHO World Health Organization Table of Contents EXECUTIVE SUMMARY .................................................................................................................................. 1 1. INTRODUCTION ............................................................................................................................................ 2 1.1 Purpose .................................................................................................................................................. 2 1.2 Audience ................................................................................................................................................ 2 1.3 Outline of Evaluation ............................................................................................................................. 2 2. BACKGROUND .............................................................................................................................................. 3 3. METHODOLOGY ........................................................................................................................................... 3 4. FINDINGS ........................................................................................................................................................ 3 5. CONCLUSIONS ........................................................................................................................................... 14 6. RECOMMENDATIONS ............................................................................................................................... 19 7. ISSUES ........................................................................................................................................................... 24 7.1 Disclosure of Conflicts of Interest ....................................................................................................... 24 7.2 Statement of Differences ..................................................................................................................... 24 8. REFERENCES ............................................................................................................................................... 24 Annex 1. Evaluation Statement of Work ............................................................................................................ 25 Annex 2. Glossary of Terms ............................................................................................................................... 32 Annex 3. Methodology ....................................................................................................................................... 34 Annex 4. References ........................................................................................................................................... 41 Annex 5. Persons Met ......................................................................................................................................... 44 Annex 6. Documents Produced and Training Provided ...................................................................................... 46 Annex 7. Behavior Change and Communication ................................................................................................ 51 Annex 8. Additional Evidence from the Evaluation to Support Conclusions ..................................................... 54 Annex 9. Local Health Committee Members Interview Analysis ...................................................................... 61 Annex 10. Suggestions on Contract Deliverables ............................................................................................... 63 Annex 11. Suggestions on PMP Indicators and BCC Activities (M&E Strategy).............................................. 71 Annex 12. Outreach Activities for Vulnerable Groups ....................................................................................... 73 Annex 13. Iraqi Patients’ Rights Statement ........................................................................................................ 74 Annex 14. Verification of Technical Deliverables ............................................................................................. 80 Annex 15. Checklist for Strategic Decision Making (to be presented to TAG) ................................................. 82 Annex 16. Strategy for Focusing PHCPI Activities ........................................................................................... 85 Annex 17. Comments on PHCPI Clinical Guidelines ........................................................................................ 92 Annex 18. Data Collection Tools ........................................................................................................................ 96 MID-TERM PERFORMANCE EVALUATION OF USAID/IRAQ PRIMARY HEALTH CARE PROJECT EXECUTIVE SUMMARY The United States Agency for International Development (USAID) awarded the four-year Primary Health Care Project in Iraq (PHCPI), valued at approximately $75 million, to University Research Co., LLC, in March 2011. PHCPI was developed in response to a long-term trend in the decline of the health status of Iraqis and has the goal of improving the quality of primary health care (PHC) services. As part of its commitment to PHCPI, the Government of Iraq (GoI) has committed $70 million as cost-share for a five-year period. The project works in 360 clinics in all 18 Iraqi provinces and is scheduled to continue through March 2015. PHCPI consists of three components: 1) strengthening health management systems; 2) improving the quality of clinical services; and 3) encouraging community involvement to increase the demand for and use of PHC services. While the project has a number of underlying guiding principles, the one most relevant to this mid-term evaluation is that successful project implementation must result in rapid, tangible, measurable improvements in the quality of health care services delivered to the Iraqi people. The purpose of the PHCPI Mid-Term Evaluation is to determine whether progress is being made toward achieving its Development Objective of “Primary Health Care Services Improved” by examining the findings from eight targeted questions, drawing conclusions from the findings, and making recommendations. This approach should allow senior management of USAID to make informed decisions on how to proceed during the remaining life of the project. Thus, the primary audience for this mid-term evaluation is USAID management, implementing partners, and the Ministry of Health (MoH) of the Government of Iraq. The report presents many suggestions as part of the Recommendations section as well as additional details in several annexes. The themes of all of the recommendations can be summarized as follows: PHCPI should continue for its full life until March 2015. While there have been some significant gaps in how the project has been implemented, it has nevertheless laid a foundation for what it is intended to achieve and has made progress toward the Development Objective, as well as fulfilling the terms of its contract. The project will need significant reshaping and rethinking; even so, there are a number of objectives that can be achieved during its remaining time. This will require the full additional two years. The project must start by clearly defining which technical areas will be supported by PHCPI, even if this does not align with all of the MoH’s requests. Starting with a review of the Basic Health Services Package (BHSP) and its principles, the project needs to lay an evidence-based foundation for its strategic decisions and then ground-truth those suppositions to ensure their applicability. PHCPI and relevant stakeholders (primarily the GoI and USAID) must reach a precise agreement on the focus for the final two years that will produce rapid, tangible, measurable, and realistic improvements in the quality of health care services. PCHPI must ensure that all of the project’s various activities are interconnected by thoroughly reviewing them within the framework of a greater strategic vision and what is MID-TERM PERFORMANCE EVALUATION OF USAID/IRAQ PRIMARY HEALTH CARE PROJECT 1 currently possible within Iraq’s primary health care system structure. Many activities were undertaken without ensuring that the necessary supporting physical and human infrastructure was in place, resulting in poor utilization of resources. All stakeholders must be willing to dedicate sufficient human resources for the project’s implementation and oversight. The project’s high visibility comes with high reputational risk. The Technical Advisory Group (TAG) needs to start fulfilling its intended function by serving primarily as a forum for technical discussions while a tighter system for approving strategic decisions and technical deliverables is instituted. 1. INTRODUCTION 1.1 Purpose The purpose of this mid-term evaluation is to determine whether progress is being made toward achieving the Primary Health Care Project in Iraq’s (PHCPI) Development Objective of “Primary Health Care Services Improved” by examining findings from eight targeted questions, drawing conclusions from the findings, and then making recommendations. This will allow senior management of the United States Agency for International Development (USAID) to make informed decisions about the remaining life of the project. The eight questions that form the basis of this evaluation are: 1. To what extent has the project achieved or made progress towards the planned results to improve primary health care (PHC) services in Iraq? 2. How realistic and appropriate is the project intervention logic? 3. To what extent has the project management structure been effective and how has it been affecting the program outcomes to date? 4. To what extent has the project reached out to vulnerable groups including women, persons with disabilities, and the internally displaced persons (IDPs)? 5. Is the project’s approach appropriate to respond to the needs of vulnerable groups, and if not, how can its approach be improved? 6. How is the training provided by the project being used in the clinics? 7. How is project-purchased equipment (if any) being used in the clinics? 8. How is the Government of Iraq cost-share being utilized? 1.2 Audience The primary audience for this mid-term evaluation is USAID management, implementing partners, and the Ministry of Health (MoH) of the Government of Iraq (GoI). Additional audiences include members of PHCPI’s Technical Advisory Group (TAG) and staff of participating Primary Health Care Centers (PHCC) and their corresponding GoI management units. 1.3 Outline of Evaluation After brief sections on this evaluation’s background and methodology, the findings for each of the eight evaluation questions are presented followed by conclusions for those findings. Specific and action-oriented recommendations for the remaining life of the project are included. Issues surrounding conflicts of interest and differences of opinions MID-TERM PERFORMANCE EVALUATION OF USAID/IRAQ PRIMARY HEALTH CARE PROJECT 2 on the part of funders, implementing partners, and evaluation team members follow. The final section of the main body of the report is the references. Several supporting annexes follow the main body of the report. 2. BACKGROUND Both the GoI and USAID have committed to the goal of strengthening, improving, and modernizing the quality of primary health care services by introducing international standards of care into Iraq. To address this issue, USAID awarded University Research Co., LLC (URC) the four-year Primary Health Care Project in Iraq valued at approximately $75 million. PHCPI was launched in March 2011 to assist the Iraqi MoH in achieving its goal of better quality PHC services with a specific focus on supporting the delivery of quality services at the community and facility levels. The project works in 360 clinics in all 18 Iraqi provinces and is scheduled to continue through March 2015. PHCPI consists of three components, namely, 1) strengthening health management systems; 2) improving the quality of clinical services; and 3) encouraging community involvement to increase the demand for and use of PHC services. While the project has a number of guiding principles, the one most relevant to this mid-term evaluation is that successful project implementation must result in rapid, tangible, measurable improvements in the quality of health care services delivered to the Iraqi people. Patients and communities should be able to clearly discern a positive change in the primary care they receive, and objective measurements of performance indicators should demonstrate improvements in the outcomes of management and clinical processes. The GoI has demonstrated its commitment to PHCPI by allocating $14 million to the MoH in 2012 to improve the quality of care at the 360 PHCCs under PHCPI. The same level of annual GoI funding is expected to continue for the life of the project. 3. METHODOLOGY The methodology for the mid-term evaluation was designed to answer the eight primary questions identified in the scope of work (see Annex 1) and consisted of both qualitative (key stakeholder interviews) and quantitative activities (five survey tools used at 24 PHCCs chosen via stratified selection). Additional details on the methodology and tools are found in annexes 3 and 18. 4. FINDINGS 4.1 Findings for Question 1 4.1.1 Project Achievements to Promote Knowledge and Improved Practices for PHC PHCPI has addressed the management component of PHC by focusing on activities to improve knowledge in management theory and practices. A Primary Health Care Management Manual was developed that covers management and leadership theories as well as providing overviews of various sub-components of management for PHC services. Four separate guidelines and scopes of work have also been developed for these subcomponents to address specific systems to support quality services. The project has addressed delivery of PHC services by updating and, when needed, developing guidelines MID-TERM PERFORMANCE EVALUATION OF USAID/IRAQ PRIMARY HEALTH CARE PROJECT 3 for PHCCs for services that are components of the Basic Health Service Package (BHSP). The PHCPI reported that the topics selected for guideline revision/development had been identified by the MoH. Thus far, guidelines for 12 services have been developed and training commenced for 6 of these. The project has addressed the community component by focusing on community partnership, behavior change communication (BCC), advocating for defining patients’ rights, and public-private partnerships (PPP). A manual for community partnerships was developed and training conducted on the contents. To support these activities the MoH issued a letter for all PHCCs to form Local Health Committees (LHCs). In addition, the PHCPI produced a BCC Strategy in 2013 and developed a National BCC Strategy to Promote Primary Health Care through a national campaign. A statement on patients’ rights was produced with ownership of the MoH and incorporated in the Public Health Law. Finally, the PHCPI launched the first PPP in Iraq for healthy dental care. As per project documents, the various manuals and guidelines were developed through a collaborative process between the project and key training and technical persons from the MoH and relevant stakeholders. Technical working groups (TWGs) were formed to agree on the content for the guidelines, which were then submitted to the relevant MoH sectors for approval. Upon MoH approval, PHCPI has been developing training guidelines and curricula, conducting training-of-trainers (ToT) sessions, and, together with MoH, rolling out the training on the district and PHCC level. Revisions to the documents have been incorporated based on input from TOT and training courses and follow-up from some facilities. The BCC documents were developed based on situational and/or needs assessments conducted with relevant departments at various levels of the MoH as well as with community key informants and program beneficiaries, including IDPs. In addition to developing documents and training, PHCPI conducted three studies and produced reports with the expectation that the information would be used for program improvement. In consultation with the MoH, PHCPI is in the process of hiring two long￾term technical staff, one to harmonize and streamline the health information system (HIS), the other to improve the pharmaceutical logistic management system. 4.1.2 Project Achievements to Promote Service Quality The project worked with the MoH to define standards for management, clinical services, laboratory, and nursing care for PHC along with measurable criteria for compliance with the standards. The project plans to initiate assessments of PHC management against standards beginning this year using an external organization for project monitoring and integrating a process for assessing management and clinical practices against standards. Based on the assessment, supportive supervision and quality improvement activities will be used to rectify problems identified, and then support will be provided to sustain improvements. To support these activities, the MoH has issued a letter for all PHCCs to form a Quality Improvement (QI) Committee. In addition, a contract revision was put into place for PHCPI to select 36 clinics to develop as model clinics. 4.1.3 Evaluation Findings on Changes in Service Conditions and Quality The evaluation team was unable to access information on service statistics or client mix (which might indicate improved services) despite requests to the MoH and PHCPI. In addition, no objective evidence on actual quality of services appeared to be available and MID-TERM PERFORMANCE EVALUATION OF USAID/IRAQ PRIMARY HEALTH CARE PROJECT 4 it was beyond the scope of this evaluation to measure this. The project plans to begin assessments of indicators of quality this year. Many of the PHCPI management system revisions are refinements to address identified problems with existing MoH systems. The MoH currently has supervisory checklists for PHC assessment that are used to different degrees of consistency (43 of the 111 interviewed providers commented positively on the value of supervisory checklists). The evaluation team found evidence of routine supervision. In addition to the previously referred-to checklists, 20 of 24 visited facilities reported receiving external supervision (mostly DoH supervision of PHCCs), with 19 reporting that such supervision had occurred during the prior month. The project’s plan is to improve the supportive aspect of supervision and to link this with QI. The MoH also has referral forms and reports on referrals that are used. Twenty of 24 (83%) sample facilities had the referral forms and collated reports on referrals, although only seven facilities reported always receiving feedback on referrals (and 13 reported that they never receive feedback). The project’s plan is to improve feedback and appropriateness of referrals. The project stated that it plans to pilot-test the revised referral system this year in 20 PHCCs and six associated hospitals. In general, staff in the sampled PHCCs believed that service conditions have improved over the past two years. Among the 24 sampled PHCCs, improvements were perceived in supervision/support (23), availability of medicines (20), service management and organization (18), availability and repair of equipment (17), client satisfaction (18), and building repair and equipment repair (9). Key informants from 5 of 18 sample districts cited PHCPI training on waste management and 14 of 18 cited the new medical records and registers as PHCPI activities that have been (or are expected to be) implemented and will be useful. The introduction of the new medical records and register changes was to pilot-test the new documents and processes in 46 facilities, revise the documents based on the pilot experience, and then train PHCC staff in their use. The new medical records/registers are to be provided through an MoH cost-share agreement and have not yet been distributed. The evaluation team found no evidence that practices at the clinical level have improved. However, findings from the 24 sample PHCCs showed that “readiness”1 to provide quality services is weak. As shown in Figure 1, the majority of sample PHCCs report that staff have been trained in management, and to a lesser extent, in clinical services. Among the PHCCs with staff trained in a topic, on average three in four had the guidelines on site. 1 “Readiness” is defined as the facility having a functional infrastructure and systems to support quality services including trained staff and elements necessary for adherence to service guidelines. MID-TERM PERFORMANCE EVALUATION OF USAID/IRAQ PRIMARY HEALTH CARE PROJECT 5 PHCPI has supported training for infection prevention and for treatment guidelines for integrated management of childhood illness (IMCI), asthma, hypertension, diabetes, trauma, and communicable diseases. An assessment of selected medicines, equipment, and diagnostics that are components of PHCC essential drugs and equipment lists and are required to follow the guidelines showed that key items were not present. Despite these findings, 21 of 60 (35%) interviewed clinical service providers believe they are performing better client examination and diagnosis. 4.1.4 Evaluation Findings on Planning and Organizing Services to Improve Results Issues related to possible inefficient service and staff organization were also identified. Among the services offered by the sample PHCCs, services that are internationally acknowledged as critical for maternal and child health (MCH) were often not available. These include family planning/birth spacing (FP/BS) available in only nine of 24 PHCCs and delivery services available in two of 24 PHCCs. There may not be a need for all PHCCs to offer delivery services; however, there was no indication that there had been an assessment of whether the populations served by the target PHCCs that did not offer delivery services have reasonable access to skilled delivery services. Lack of trained staff—attributed in part to a shortage of clinically trained personnel within the system but also due to staff who receive training and transfer to work in hospitals—was cited by 9 of the 18 sample districts as reasons why they cannot implement PHC guidelines supported by the PHCPI. Among PHCPI target facilities, 87 have no physician and an additional 30 have one physician. At the same time, 27% (21 of 77) of the interviewed doctors, nurses, and medical assistants (MAs) reported they have no clinical service responsibility. Two of the four district staff with whom the evaluation team met noted that non-technical personnel are available for clerical and administrative work that does not require technical health knowledge; such work is currently the responsibility of clinically trained staff, who lack the authority to hire additional workers to free up trained staff for clinical services. Among the interviewed PHCC staff, 9 of the 39 interviewed physicians reported they had been in the PHCC for less than one year, while most other staff have worked in the PHCC for two or more years. This finding supports the complaint of rapid turnover among trained staff. Through meetings with the MoH, PHCPI developed a draft staffing pattern for PHCCs with position descriptions including qualifications. The draft is based on staffing patterns 0 4 8 12 16 20 24 PHC Management Infection Prevention and waste… Maintenance and Repair Communicable Diseases Non-communicable Diseases IMCI Handbook of Quality Guidelines Number of sample facilities Figure 1: Training and Guidelines Verified Training reported and guidelines observed Training reported but guideline not observed No training on subject MID-TERM PERFORMANCE EVALUATION OF USAID/IRAQ PRIMARY HEALTH CARE PROJECT 6 required to meet the BHSP plan for different levels of PHCs and fails to address the need for more efficient use of staff based on the personnel and qualifications currently available. The proposed tasks for paramedical and medical staff include those that could be conducted by non-clinical personnel if organized differently. The specific criteria for achieving Model Clinic status have not been defined. The criteria for payment to PHCPI for Model Clinic activities are based solely on equipment procurement and provision, with an equipment maintenance component. Two pieces of equipment (intrauterine device kits and ELISA laboratory test equipment for diagnosing Helicobacter pylori infection) have been ordered for which no one has been trained, nor is there any indication of a plan for training on their use. According to PHCPI reports, the MoH has decided to fund a study on maternal mortality as a result of the PHCPI report on maternal death records. Duplication of the health visitor program documented for Maysan, along with the use of electronic client records, is currently under discussion. No follow-up from the IDP study was evident. 4.1.5 Evaluation Findings on Linking Community and PHC Components Community-facility partnerships are part of the organogram for PHCCs; most had not been functional prior to PHCPI. Along with developing operational guidelines to define the roles and responsibilities of LHC members, PHCPI has also developed training curricula and conducted training to build LHC member skills in communication, leadership, team building, and problem solving. Although 23 of 24 sample facilities reported that staff had been trained in community partnerships, 51.5% (34/66) of interviewed LHC members reported having been trained on the Community Health Partnership Handbook; 25.8% (17/66) reported receiving training on operational guidelines or on skill-building topics. This may be due to the high turnover among LHC members, which was evident from the interview process. Only 68.2% (45/66) of interviewed LHC members reported that their LHC was meeting regularly, with 8 of the 24 LHCs (33.3%) not functional. Four of these eight LHCs had been formed for three months or less. Reports from LHC interviews confirmed low representation among squatter settlements, a vulnerable population. Interviewed LHC members reported intersectoral public health activities in the community that include raising awareness regarding health and health services, reaching out to vulnerable groups to increase service usage, and making demands on the MoH to improve the quality of care. LHC members expressed a desire to contribute to QI of services and a desire for moral and financial support. In addition, they suggested that having local council members on LHCs would give committees the authority needed to implement decisions. The strategy for the national BCC campaign is to use national media to “advocate for desired behaviors for immunization and other MCH and reproductive health issues.”2 As the campaign uses national media, messages that advertise the availability of quality services will reach communities with PHCCs nationwide, even though the project implements QI in just 360 PHCCs. There is no indication in the strategic plan that 2 USAID/Iraq PHCPI, August 2012: National BCC Strategy to Promote Primary Health Care in Iraq. MID-TERM PERFORMANCE EVALUATION OF USAID/IRAQ PRIMARY HEALTH CARE PROJECT 7 messages should be focused on populations served by PHCCs that have already achieved the needed levels of quality and service availability. Sixteen of 18 sample districts (90%) report disseminating information on patient rights via posters and brochures and through seminars for PHCC staff. A total of 46 of 65 (70%) interviewed LHC members, 35 of 116 (30%) interviewed PHCC clients, and 26 of 112 (23%) interviewed health workers reported they had heard about patient rights. Although mostly satisfied with services received from PHCCs, among the 116 interviewed clients asked about large or moderate problems, 29 (25%) cited lack of visual and auditory privacy, 14 (12%) cited inadequate information exchange between client and provider, and 9 (8%) cited poor treatment by PHCC staff. 4.2 Findings for Question 2 The project logic, based on the revised performance management plan (PMP) of February 2013, consists of a Development Objective (DO), three Intermediate Results (IRs), and five sub-IRs. The DO is “Primary Health Care Service Improved” and is to be achieved via IR 1 “PHC Management Systems Strengthened,” IR 2 “PHC Service Providers Performance Improved,” and IR 3 “Community Participation in PHC Service Delivery Increased.” The DO and IRs are appropriate for the project as they are in line with international best practices for health systems strengthening (HSS): for example, the $75 million USAID-funded Health Systems Strengthening Project II in Jordan uses a similar framework. The project logic and its development mostly comply with the World Health Organization’s (WHO) definition of HSS, namely; (i) the process of identifying and implementing the changes in policy and practice in a country’s health system, so that the country can respond better to its health and health system challenges; (ii) any array of initiatives and strategies that improves one or more of the functions of the health system and that leads to better health through improvements in access, coverage, quality, or efficiency. The missing piece in development of the project logic and subsequent project that the evaluation team identified was the intermediate step of ensuring supportive MoH policies for IR implementation. This appears to have been done only partially prior to the project’s start. Necessary supportive policies include: a) delegating managerial and budgetary authority to the district level for IR 1; b) defining clinical scopes of work (SOWs) and legal mandates allowing task-shifting within PHCCs and expanding PHCCs roles for IR 2; and 3) defining LHC authorities for IR 3. The BHSP, which underlies the project logic, is used by PHCPI and MoH for defining PHCCs and their services. Even so, the levels of services described in the BHSP are not in line with current MoH policy, nor do they correspond to common practice and understanding among district and PHCC staff. There is no indication that PHCPI uses a planned level of care based on an assessment of available human and physical capital and the achievement of the best value in health. In addition, there is no indication that PHCPI has a strategy for influencing MoH policy to support this approach. More problematic is the fact that the MoH’s vision for PHCCs is inconsistent with the BHSP and is more expansive. MID-TERM PERFORMANCE EVALUATION OF USAID/IRAQ PRIMARY HEALTH CARE PROJECT 8 The lack of a shared vision among stakeholders on what should comprise PHC services in Iraq has led to the implementation of irrelevant activities at the PHCCs. A package of internationally accepted PHC services was not available at PHCCs assessed by the evaluation team. Moreover, PHCPI is not properly prioritizing the PHC services it is strengthening to ensure that internationally accepted critical services are included. Further, there is a weak linkage among training provided by the project under IR 2, efforts to ensure readiness to provide services where trainees work, and BCC delivered to improve service provider and community attitudes and acceptance as part of IR 3. This fact is underscored by PHCC staff reports that patients want medicines and access to doctors in the absence of strategies to address community and service provider attitudes. Another issue is that equipment and supplies to provide services in accordance with guidelines developed by the project were not consistently available at the PHCCs surveyed. Given that the two higher-level indicators within the PMP focus on PHCC target beneficiaries, these gaps are of particular concern. 4.3 Findings for Question 3 All stakeholders interviewed (PHCPI field staff, implementing partners’ headquarters staff, five past and present TAG members, USAID staff, and MoH officials) cited a sense of partnership and collaboration that has emerged during PHCPI implementation and stressed the importance of reforming the health care system to strengthen PHC services. The TAG was singled out as an important forum for information sharing and coordination. The GoI has demonstrated its commitment as a partner in project leadership by its willingness to cost-share. Despite this goodwill, limited resources have been devoted to oversight of the project to ensure its success. Specifically: 1. PHCPI staff typically defers to MoH leadership and its requests, which may not always be aligned with developing a PHC system according to best practices. For example, the decision to include menopause and obesity as priority issues for PHCPI originated with requests from the MoH. Additionally, in the project’s first two years, per annual reports, it experienced significant recruitment and retention problems, including the dismissal of the Chief of Party (COP) and replacement with an acting COP while awaiting a permanent replacement. It should be noted, though, that at the time of the evaluation team’s mission, approximately 90% of staff positions had been filled. An additional 18 staff were to be hired to oversee implementation of the model clinics, and a new COP was scheduled to arrive in-country in May 2013. 2. URC, the prime contractor, delegated most decision-making to the field office. Most day-to-day technical and managerial oversight is performed by mid- to junior-level staff. Usually headquarters staff visit the field office every two to three months, except during the interim period between COPs, when visits have been more frequent. 3. Management Systems International (MSI), a subcontractor, delegated most technical decision-making to its field staff with fairly limited home office involvement, although there is low-level daily contact for project management. Visits by headquarters staff are usually made quarterly, with ad hoc visits conducted when a specific need arises. 4. USAID has had three Contracting Officer’s Representatives (CORs) for PHCPI since the project’s inception. Technical backstopping was originally carried out on an intermittent basis. A foreign service national (FSN) was hired in May 2012; since MID-TERM PERFORMANCE EVALUATION OF USAID/IRAQ PRIMARY HEALTH CARE PROJECT 9 September 2012, additional technical backstopping has been provided by a third country national on an intermittent basis. USAID staff are severely constrained in their ability to provide field oversight due to security concerns and must rely on field monitors. While the field monitors have been provided checklists for verifying activities, their ability to actively monitor the project and its activities is limited due to their lack of technical background. USAID/Iraq is currently on a “glide path,” with plans to further downsize its staff. 5. All key informants interviewed regarding the TAG noted that it has allowed stakeholders to coordinate projects and activities and learn more about PHCPI. However, there was limited mention of its functioning in a technical advisory capacity; indeed, several TAG members noted that this was one of its key weaknesses, despite the establishment of TWGs as part of the TAG. This was further verified by the evaluation team in a review of meeting minutes from the first five TAG meetings and attendance at the sixth TAG meeting, held on April 8, 2013. 6. Interviewed MoH officials all expressed an interest in reforming Iraq’s PHC system. That said, PHCPI is a small part of the MoH’s large portfolio and may fail to receive sufficient attention from overstretched MoH staff. Further, there is a fairly frequent turnover in MoH leadership, diminishing the ministry’s institutional memory about the project and leading to varying capacity among staff overseeing PHCPI for the GoI. Finally, as previously noted in this report, MoH appears to have an unclear vision of what a primary health care system should include. The PHCPI contract was modified in November 2012, primarily to include the upgrading of 36 PHCs to model clinic status. Currently, contractor fee is tied to quantities of deliverables to be produced and other outputs, although the project is based on technical assistance service provision. Per a review of the Year 3 work plan, approximately 73% (37/51) of the required deliverables are on track. 4.4 Findings for Question 4 Within the Iraqi context, inhabitants of informal settlements or squatter areas are a vulnerable group. Inhabitants of squatter areas include IDPs, who cannot afford to pay rent, along with other very poor families, including widows and orphans. One quarter (6 out of 24) of sampled facilities reported that some groups of people may not be receiving PHCC services; of these, half are IDPs, who mainly reside in poor squatter areas. Other vulnerable groups include tribal minorities, people with disabilities, widows and orphans, and poor people in general. Of the six PHCCs reporting that some vulnerable groups may not be receiving services, three (or 12.5% of all sampled facilities) do not reach out to vulnerable groups reported as not accessing PHCC services. All 18 district managers interviewed reported that all groups have equal access to PHCC services, including IDPs, meaning that there are no systemic barriers to accessing services. Even so, more than 61.1% (11/18) conduct activities to increase service access among vulnerable groups. The evaluation team visited the Karkh District Health Office, which mentioned the existence of the MoH National Program for Registering the Disabled at the PHC level, which is designed to provide disability medical aide devices and physiotherapy for free. However, LHC community members reported that the disabled has difficulty reaching PHCCs due to lack of transportation, especially in rural areas. LHC members’ reports confirmed district reports that the disabled receive services for free and that a few non- MID-TERM PERFORMANCE EVALUATION OF USAID/IRAQ PRIMARY HEALTH CARE PROJECT 10 governmental organizations (NGOs) have distributed wheelchairs. One PHCC reported that there is a specialized clinic nearby to serve the disabled’s needs. PHCPI Component 2 guidelines and training make no note of the need for outreach to vulnerable and marginalized groups. In addition, project quality guidelines lack guidance on introducing ramps in PHCCs to facilitate the movement of the disabled in wheelchairs. However, the Component 3 Community Partnership Handbook has a section on “Identifying Impediments to Engaging Marginalized Groups.” In fact, LHC members expressed awareness of the needs of vulnerable groups and PHCC strategies to address those needs, especially the disabled and the poor, with 36.9% (24.65) of LHC members reporting strategies to enhance the access of vulnerable groups. In one PHCC, a special lane in the PHCC has been dedicated to facilitate disabled entry to the PHCC as part of the LHC’s work. Different programs were reported by individual LHC members (e.g., special health cards, check-ups, home visits for the disabled by health visitors, disabled committee at the PHC). As for addressing women’s health needs, two-thirds (44/66) of LHC members reported the need for female physicians; 12% (8/66) for gynecological services, drugs, and equipment, and 8% for gynecologists. Other reported needs were fetal sonar (12%), delivery rooms (9%), and more female nurses (9%). In response to a question on activities to address discriminatory treatment of women in PHCCs identified by the assessment, 3 two evaluation team interviews highlighted that domestic violence against women is a problem that needs to be addressed in PHCCs, even if only as part of the MCH services. 4.5 Findings for Question 5 In July 2012, PHCPI conducted an IDP health and social needs assessment for the MoH to identify the health needs of Iraq’s IDP population. The assessment reported far less satisfaction of IDPs with accessed health services than the general public. Assessment recommendations included: 1) training health providers of health facilities near IDP settlements on proper patient-provider interaction; 2) “community health workers should be trained and utilized to support services to IDPs”; and 3) “health promotion campaigns need to be developed that specifically target IDP populations.” Despite these recommendations, the PHCPI Year 3 work plan contains no activities to address IDPs living in squatter areas. In addition, a mini-assessment recently conducted by PHCPI did not present data related to IDPs, although the tools had a dedicated section. Furthermore, although the BCC strategy included IDPs as a primary target audience, the campaign strategy identifies the IDPs only as a secondary target group. The primary target group for the campaign is “18–44 year old married women residing in urban, peri￾urban, and rural areas of Iraq.” 4.6 Findings for Question 6 4.6.1 The Project’s Use of Training Much training has been carried out with PHCPI support, with the overall objective of improving PHC service availability, utilization, and quality. Interviewed PHCC staff who 3 PHCPI, December 2011: Baseline Assessment Report. MID-TERM PERFORMANCE EVALUATION OF USAID/IRAQ PRIMARY HEALTH CARE PROJECT 11 received management-related training were generally positive in their assessment of training. Reports that the courses were excellent ranged from 16%–30%, and good from 61%–80%, depending on the topic. However, there is limited objective evidence of training resulting in change, since there are no reports on post-training follow-up for trainees. One example where training has resulted in change is the introduction of screening systems for hypertension and diabetes that are linked with reporting systems on case detection, with the introduction of the systems following training on clinical guidelines for hypertension and diabetes. 4.6.2 Evaluation Findings on Use of Training Statistics on case detection for hypertension and diabetes were not available from PHCPI or from the MoH. The project has not instituted systematic follow-up to assess how training is being used. Indeed, the evidence suggests that training has not been consistently used to improve practices. Although MoH guidelines for hypertension and diabetes developed with PHCPI support provide treatment protocols, MoH policy is that PHCC physicians can only prescribe medicines for 10 days. With such a short time length impractical for managing long-term chronic illnesses, this policy reinforces the referral of uncomplicated hypertensive and diabetic cases to hospitals or to MoH evening clinics. The treatment protocols provide guidance for revising treatment if the first-line strategy is ineffective, yet not all of the medicines referred to in the protocols are available at the PHCC level. Among sampled PHCCs, there were reports of improved medical and sharps waste management, with facilities visited by the evaluation team indicating they were making arrangements with their local government for assistance in removal of contaminated waste and disposal offsite. Despite this, 6 of 24 (25%) sampled facilities had inadequate waste disposal practices (for example, open burning in an unprotected environment). There was some evidence of job descriptions (a recommendation from management training) being developed. Seven of 24 (29%) sampled facilities had job descriptions reported for all staff; job descriptions for nurses were validated for six of these. 4.6.3 Issues Related to Training Identified During the Evaluation Although a project objective is to build capacity within the MoH for carrying out PHC￾related training, PHCPI remains instrumental in developing training materials, organizing ToT, and rolling out training. There was no evident strategy for shifting responsibility to the MoH for implementing the training process. Respondents in five PHCCs and four districts visited by the evaluation team volunteered that while they appreciated the opportunity to learn, the frequency and length of trainings has interfered with client services. Some respondents reported that training is frequently planned with one day of notice. They noted that the lack of lead time does not allow staff enough time to arrange service coverage, so client services were disrupted in some cases and target trainees (usually doctors) were unable to attend the training in other cases. A further consequence of this lack of lead time was that in some cases the participants selected for the training were not from the target audience and so the training was irrelevant to their work. PHCPI-supplied information on trainees supports these comments, with the problem applying to TOT as well as to roll-out training. MID-TERM PERFORMANCE EVALUATION OF USAID/IRAQ PRIMARY HEALTH CARE PROJECT 12 For example, the team’s analysis of training data showed that 10 of the persons selected for TOT for IMCI intended for nurses had no clinical background and had clerical or administration responsibilities; likewise, three persons chosen for the TOT for polytrauma were ill-suited for that training. In terms of training roll-out, there were several instances of non-qualified persons receiving or persons for whom the training was not relevant: 15 for polytrauma, 27 for non-communicable diseases, 64 for IMCI for nurses, and 29 for communicable diseases. Although personnel from 11 of 18 sample districts provided positive comments about the training and learning they have received from PHCPI, nine also indicated some dissatisfaction with training quality and the training process. Problems identified were a crowded training venue, “lack of modern equipment” for the training, and “training is theoretical and what is needed is practical” (five districts). Respondents at the MoH level and 2 of 18 sample districts spontaneously mentioned that they felt some trainers were unqualified. Key informants at the central MoH level commented that the sheer number of trainings being carried out so frequently did not allow time to absorb the new content and apply the training 4.7 Findings for Question 7 Project-funded equipment has only recently been procured and has not been distributed to participating PHCCs. As a result, there are no findings, conclusions, or recommendations for this question. 4.8 Findings for Question 8 The total amount of funding for the Partnership Fund cost-share, per MoH officials, is $14 million per year for five years, or $70 million. Allocation of funds began in 2012; the total amount currently available is nearly $28 million. To date, the level of disbursement has been low, with most of the spent funds used for production and distribution of printed materials. At the time of the evaluation team’s mission, the MoH was awaiting ratification by governorate officials of the cost-share budget that will primarily be used for equipment and furniture. GoI interviews during the evaluation team’s visit revealed that how the cost-share funds will flow to the governorates is unclear, although discussions were on-going to resolve this issue. According to interviews with MoH officials, each governorate will receive approximately $4–5 million in cost-share funding, with implementation to start first in those governorates and districts with higher poverty levels. The MoH official primarily responsible for the cost-share stated that its purpose was to establish the participating PHCCs as model clinics and use these clinics as demonstration sites for a new model of health service provision. Once the value of the PHCCs and corresponding system is validated via the establishment of the model clinics, those results could be used to advocate for a greater proportion of MoH funds to be spent for high￾tech, high-cost equipment (approximately an additional $5–10 million per governorate). This would lead to PHCCs’ further drawing patients away from tertiary care centers. Per the MoH official, the cost-share funds would be used for: 1) refurbishment and rehabilitation of PHCCs, with the Directorates of Health (DoH) taking primary responsibility for ensuring completion; 2) equipment and furniture, with the governorates taking primary responsibility for procurement; 3) publications and information, MID-TERM PERFORMANCE EVALUATION OF USAID/IRAQ PRIMARY HEALTH CARE PROJECT 13 communication, and education (IEC) materials for PHCCs and LHCs; 4) registry books; and, 5) capacity building through study tours and educational courses. Cost-share funds have been expended for printed materials, including project-related publications, advocacy communications, and registry books. Per the interviewed MoH official, all materials had been printed and distributed, except for those to be used in Kurdistan as they were awaiting translation. However, on-the-ground site visits conducted by the evaluation team’s data collectors did not substantiate this, as most non￾Kurdish facilities were missing an average of nearly four of the documents. The evaluation team conducted a review of the list of the equipment that will be used for project purchases to supply the model clinics. Though the team was also provided the equipment list that the MoH will use for its equipment purchases as part of the cost-share, the team focused its review on the PHCPI equipment list, as the MoH purchases are outside the project’s manageable interests. In general, the PHCPI equipment list appeared reasonable, although there were some items outside the project’s technical areas (e.g., intrauterine device insertion sets for FP/reproductive health) and some items for which less-expensive alternatives are available (e.g., ELISA for diagnosing H. pylori infection). 5. CONCLUSIONS 5.1 Conclusions for Question 1 5.1.1 Overall Results Several PHCPI staff expressed the opinion that “if the equipment is provided and the training is provided, then services will be improved…services will be delivered.” International experience has consistently shown this to be a flawed conclusion for sustained change. Through training courses PHCPI has raised awareness in MoH personnel at all levels about the principles of good management and leadership and related issues, as well as generally acknowledged criteria for providing specific services. However, no specific actions to improve management at the PHCC level or to ensure adherence to standards in clinical practice were identified. The PHCPI Year 3 work plan focuses on implementing supportive supervision, a QI process, and assessment of compliance with management and clinical standards. If implemented successfully using a process that results in institutionalization, these methods have been proven to achieve sustained improvement in health services. The plan to measure compliance with management and clinical standards is a step toward establishing benchmarks against which change and need for change can be assessed. There has been no prioritization of standards, which are numerous and cover many different components of varying levels of importance. For example, 93 standards with 291 measurable criteria were defined for assessing management practices. Without prioritization there is a risk that the focus may be placed on achieving improvements in less important areas at the expense of items more critical for achieving service quality. Although sub-clinics make up 23% of PHCPI target facilities, a subset of standards for them has not been identified. Interviewed staff cited improvements in PHCC working conditions, which can reasonably be attributed to the MoH’s policy to strengthen PHC and to the additional MID-TERM PERFORMANCE EVALUATION OF USAID/IRAQ PRIMARY HEALTH CARE PROJECT 14 attention to PHCCs provided through the continuous training supported by PHCPI. PHCPI is only now focusing activities on the quality of supervision. MoH cost-share equipment purchases have not yet occurred, and PHCPI activities for improved pharmaceutical management have not yet been introduced. 5.1.2 Critical Issues for Strengthening PHC Services Despite a BHSP document that describes the level of services and resources for the PHC system, there is no uniform agreement among stakeholders—including the MoH and the PHCPI—on how the BHSP should be implemented. If current MoH policies and agreement on a priority package of services and level of care for PHCs are not addressed, the probability of achieving the objective of reducing hospital caseloads and improving service access is low. Examples of MoH policies related to the BHSP that deviate from the project’s stated objectives include the following: • Guidelines that reinforce a doctor- and even specialist-centered system • Policies that result in referring of non-complicated cases (e.g., hypertension, diabetes, severe pneumonia in children) rather than developing capacity at the PHCC level • Lack of coordination among stakeholders that support PHC services to improve the availability of services that have been internationally shown to improve maternal and child health Although attitudinal barriers among the population and service providers to changing the PHC service delivery model were mentioned often, there was no strategy identified for addressing and changing these attitudes. For example, key informants from PHCPI, PHCCs, MoH, and USAID noted that clients demand physician services; the informants provided examples of clients visiting PHCCs solely to get a referral to a hospital specialist. Major PHC management system issues cited as barriers to improving the PHC system include the following: • Frequent physician turnover in PHCCs and doctors’ preference to work in hospitals instead of PHCCs • Public-private conflicts of interest related to MoH-supported evening clinics (where physicians can earn extra money from client fees) and private practices • Reluctance for change in the system, including examples of lack of delegation to districts for resolving problems such as in procurement, when shortages occur in the availability of authorized drugs or for replacement of broken medical equipment • Shortages of clinically trained staff, while available clinically trained staff are assigned to non-clinical work 5.1.3 Improving Availability and Quality of Services Most of the clinical guidelines developed with PHCPI support are long documents that provide general education on the prevention, diagnosis, and treatment of the condition in question, similar to the level of information provided in medical school. These documents contain treatment guidelines that include medicines not on essential drug lists (EDLs) for PHCCs and refer to levels of treatment that are not allowed at the PHCC under current MID-TERM PERFORMANCE EVALUATION OF USAID/IRAQ PRIMARY HEALTH CARE PROJECT 15 MoH policy. In some cases (e.g., cholera and childhood pneumonia) they do not specify the antibiotic of choice, but generically refer to using an “appropriate” antibiotic. For health systems where facilities are resourced based on policy decisions that define medicines and diagnostics to be made available at the various service levels, specific guidelines are preferable to generic educational guidelines that leave providers to make their own choices from among the many options for treatment. The risk in not having specific guidelines is that some providers will make wrong decisions because they lack the experience to assess the fine points of treatment selection; alternatively, they may prescribe medicines and tests that are in the guidelines but are not available through the facility. Specific guidelines support service providers in withstanding client pressure for unnecessary or inappropriate treatments. Providers can use their judgment in deciding not to follow treatment and referral guidelines in specific instances, but during supervision and service quality checks they should have a solid rationale for deviating from standard guidelines. There is a subset of medicines that are prescribed in sub-centers that lack physicians. PHCPI has not updated or developed any guidelines for use of these medicines in these clinics, despite the fact that among PHCPI’s 360 target facilities, 87 sub-centers have no physician; 20 sub-centers/PHCCs have one physician and may depend on a non￾physician when the one physician is not present. The plan for the model clinics is not specific enough to ensure that these will function as true models for the PHC system. There is no strategy in place for making service and system changes in the selected model clinics beyond the training provided for all 360 facilities and a note in the original documents that there will be more intensive supervision. 5.1.4 Linking Communities with PHC Community participation through the work of the LHCs is likely to produce positive effects on increasing the access of vulnerable groups to needed health care, increasing the use of preventive services by the community at large, improving the quality of care in PHCCs, as well as sustaining a cleaner and safer living environment for the catchment population. The LHCs show promise of fulfilling their objective of strengthening PHC services if they are activated, closely mentored, have financing, and systems are put into place to sustain them. The lack of training and lack of specific terms of reference (TORs) for LHCs to guide their work limit the potential LHC effectiveness. Annex 7 provides a full discussion of issues related to the national BCC strategy and the planned methods for reaching target populations. 5.1.5 Project Implementation The evaluation team agrees with opinion expressed by PHCPI staff that the deliverables related to document production and training are driving PHCPI time and resource allocation at the expense of activities needed to institute and institutionalize support systems to maintain changes at PHCCs to improve service quality. 5.2 Conclusions for Question 2 The project logic and its underlying components are appropriate, but not entirely realistic. Per the terms of the original contract “Technical assistance must result in realistic, MID-TERM PERFORMANCE EVALUATION OF USAID/IRAQ PRIMARY HEALTH CARE PROJECT 16 practical systems, procedures, and tools which can be effectively applied in all primary health care clinics, not only in higher functioning ‘model’ sites.” To date, technical assistance provided as part of IRs 1 and 2 has not fulfilled this objective. Further, the weak linkage that exists between improved PHCCs/provider skills and more aware communities has probably resulted in a limited impact on service delivery. Finally, of greatest concern is the fact that the project objectives as per the contract are based on a BHSP and PHC model that does not appear to align with the vision of the MoH. 5.3 Conclusions for Question 3 As documented in this section and elsewhere in this report, technical accountability is insufficient and the process for oversight is not working adequately. This has led to gaps at both the strategic level (decisions regarding which health issues to address within PHCPI) and the micro level (technical deliverables which are sub-optimal). In particular, per its terms, the TAG has not reached its potential as outlined in the document “Technical Advisory Group TAG Roles and Responsibilities.” “At the same time, the TAG will provide oversight and guidance for the development and execution of project activities with the objective of strengthening MoH capacity to oversee long term improvements in the health sector, including making the best possible use of all health resources and programs available in the country.” The evaluation team believes that much of this lack of technical oversight particular to the TAG is linked to the agenda that is developed for the TAG meetings, which emphasizes project updates rather than the discussion of technical issues. This belief was reinforced by interviews with TAG members, a review of meeting minutes, and attendance at a TAG. Additionally, stakeholders including implementing partners, USAID, and MoH have had issues with dedicating sufficient human resources to ensure that a strategic and technical vision for the project would be articulated and, once developed, that the vision would be adhered to during implementation. It is notable that, although a CoP was in place during the project’s first two years and should have provided needed technical leadership in collaboration with the MoH, it appears that the project has been primarily reactive to MoH requests rather than proactive in establishing its vision. Though the project is mostly on track in terms of fulfilling its contract, this has come at the expense of emphasizing quantity over quality and depth of change. Thus, project results as per the terms of the contract have not been greatly impeded. However, if the intent of PHCPI is to provide longer-term, substantive change, tying the contract’s fee to quantities of deliverables produced (outputs) rather than to any higher-level objective, such as strengthened capacity within the MoH to sustain activities post-project, has and will continue to undermine that desired change. 5.4 Conclusions for Question 4 There is no evidence of PHCPI activities to enhance MOH activities to reach out to vulnerable groups in a systematic way. This is of particular concern since MoH activities to reach the various vulnerable groups may be limited and not uniform across PHCCs. This could lead to a gap in outreach to these populations, which may already be MID-TERM PERFORMANCE EVALUATION OF USAID/IRAQ PRIMARY HEALTH CARE PROJECT 17 underserved, especially in the case of poorer population and those living in squatter and remote areas. 5.5 Conclusions for Question 5 The PHCPI approach to reaching out to vulnerable groups is lacking, with nothing planned to address the needs identified as part of the IDP health needs assessment. Therefore, the project’s approach for addressing the needs of vulnerable groups has been neither appropriate nor sufficient. 5.6 Conclusions for Question 6 Candidate selection for TOT does not appear to be based on developing quality subject￾matter trainers, but rather on having large numbers of persons to rapidly roll out training. Selection of trainees is often shaped by need to mobilize trainees at the last minute rather than the selection of persons for whom the training is relevant. PHCPI staff noted that they are under pressure to respond to deliverables, many of which are related to number of trainers trained and number of target facilities with staff trained across the many topics for which guidelines have been developed. This appears to be the driving force behind the numerous trainings planned at the last minute, which have resulted in inappropriate persons receiving training. This contributes to the project’s tendency to lead training rather than working with the MoH to strengthen its capacity to manage and implement the total training process. It is likely that working with the MoH system and shifting responsibility for implementing training to the MoH—including integrating the PHCPI training into master training plans for the MoH—would be slower and might not result in achieving the required numbers. Therefore, it can be concluded that the existing training systems and skills within the MoH do not appear to be utilized and the current PHCPI process is not contributing to further developing the ministry’s training capacity. 5.7 Conclusions for Question 7 (N/A) 5.8 Conclusions for Question 8 Cost-share funds will be used for the previously agreed upon items.4 However, as has been shown, the MoH has limited capacity to track and manage the printed materials for which the cost-share has already been used, as, despite repeated assurances that all had been printed and distributed, a significant portion of materials were not found at the sites visited. Since the MoH has delegated the procurement of PHCC equipment, furniture, and supplies to the governorate level, it is probable that this larger use of funds will be even more difficult to track. Further, it is unclear if procurement carried out at the governorate level will result in a more efficient or cost-effective use of cost-share funds, as it is possible that economies of scale will be lost through decentralization. Finally, while it is important that the utilization of the cost-share funds for equipment, furniture, and supplies be responsive to on-the-ground needs, until a clearer vision of which 4 Minutes of the Meeting between USAID Capacity Building Office (CBO) and Public Health Directorate/ Ministry of Health. MID-TERM PERFORMANCE EVALUATION OF USAID/IRAQ PRIMARY HEALTH CARE PROJECT 18 services will be offered by PHCCs is determined, it is likely that some items will be procured that are inappropriate. Whether the MoH will be able to disburse funds in a timely manner is also uncertain and may lead to programmatic gaps (i.e., training provided on services for which there is no equipment). The cost-share funding is already back-logged in its disbursement, a problem compounded by the historical inability of the MoH to expend its full budget. In addition, there is currently no guarantee that once the Partnership Fund’s cost-share is spent that the MoH will allocate via its budget additional resources to PHCCs. The idea of increasing the MoH budget for primary health care is only in the planning stages at this point in time. Most worrisome, though, is that even if the MoH decides to increase its budget for the PHCCs, the intended use of those funds is to purchase high-tech, high-cost equipment. In short, the PHCCs may come to resemble mini-tertiary care centers rather than primary health care centers. This obviously runs counter to the intent of primary health care and PHCPI. It also provides further evidence that there may be either an unclear vision on the part of the MoH of the functions of primary health care or a misunderstanding of how a primary-secondary-tertiary care system should operate. 6. RECOMMENDATIONS After careful deliberation, the evaluation team made the determination that all recommendations presented below would be for the full life of the project (i.e., through March 2015). Ending the project prior to its agreed-upon end date would produce very poor results in terms of its Intermediate Results and Development Objective, as well as produce significant social and public health consequences. 6.1 Recommendations for Question 1 6.1.1 PHCPI can improve the system for supporting the quality of clinical services by: a. Developing job-aids for treatment guidelines and referrals that include all priority PHC services. These should be specific to level of service and qualification of service provider and be aligned with MoH policy. b. Reworking the draft Handbook of Quality Standards and Operational Guidelines for Clinical Services Delivery in Primary Health Care to be a manual that includes job-aids for PHCs with and without physicians. c. Identifying gaps and ensuring the availability of all resources necessary to adhere to the job-aid protocols for each level of service. 6.1.2 PHCPI should use the model clinics for program research to provide evidence for change including: a. Defining “model clinic status achieved” based on an agreed-upon minimum package of critical PHC services, staffing levels, and service levels that are feasible within the next few months. Clinical guidelines and referral guidelines may be different from those for all PHCCs if the MoH agrees to test different models for service delivery. b. Developing a strategic plan for achieving model clinic status. MID-TERM PERFORMANCE EVALUATION OF USAID/IRAQ PRIMARY HEALTH CARE PROJECT 19 c. Conducting program research in model clinics to test service model strategies and provide evidence for whether international experiences are applicable in the Iraqi context. 6.1.3 PHCPI should advocate for QI activities at the PHCC level that are focused on improving adherence to guidelines or on client satisfaction within existing constraints. “Lack of staff,” “lack of space,” and “lack of privacy for female clients” can potentially be addressed through creative thinking and reorganizing services. 6.1.4 PHCPI needs to advocate and shape BCC in conjunction with improved service activities so that community and provider attitudes that may not support change are addressed along with along the service changes and improvements. Potential topics include: demand for doctors (when unnecessary); demand for referrals (when unnecessary); changes in the service delivery model; demand for medicines (when unneeded). 6.1.5 When PHCPI is implementing BCC activities, it needs to concurrently ensure that advocated services are available at the quality promoted and plan for changes in service utilization. It should consider: a. Introducing a branding logo that indicates a facility has met quality standards and the Patients’ Rights Charter, along with a system to periodically recertify the facility b. Agreeing on standards that will be highlighted during the BCC campaign c. Implementing the campaign in stages by initially certifying quality on critical services where quality standards can readily be achieved under current resource constraints (e.g., immunization and IMCI) 6.1.6 The PPP for dental health is expected to increase demand for dental services. The PHCPI and the TAG should advise and assist the MoH in planning resources to meet the increased demand. 6.1.7 The PHCPI should provide continuous mentoring and close supervision to help mitigate challenges facing LHCs, especially for start-ups: a. The experience of highly functioning LHCs should be examined to identify factors that enable success and achievement in order to transfer the know-how to other LHCs. The planned compliance survey should aim to provide such data on lessons learned. b. Study the community committees that existed prior to the project without external financing to identify factors that contributed to their survival. c. Identify and test innovative, sustainable methods that districts can use to motivate LHCs. d. Advocate institutionalizing a MoH funding mechanism for LHC work. e. Work to institutionalize the LHC role in the planned national campaign for promotion of PHC services among local communities. MID-TERM PERFORMANCE EVALUATION OF USAID/IRAQ PRIMARY HEALTH CARE PROJECT 20 6.1.8 PHCPI should revise the language in documents on patient rights, as well as add the important missing standard rights. This should be done prior to production of IEC materials for any future campaign. Patients’ rights information should be integrated into management and clinical guidelines and training. Annex 13 provides further suggestions regarding this recommendation. 6.2 Recommendations for Question 2 6.2.1. PHCPI should advocate improving MoH PHC policy to promote a minimum standard package and service delivery model that will best develop services and health in the PHCC setting under current constraints. Annex 16 provides some examples that might be considered. This effort should be carried out using the following methods: a. PHCPI in conjunction with the TAG should develop a tool (see Annex 15 for an example) that serves as a checklist for decision making and then can clearly define what technical areas will be supported via PHCPI. Factors to consider include: 1) epidemiologic profile; 2) cost-effectiveness for high impact; 3) realistic, current human and physical capital at PHCCs; 4) services that address the most vulnerable; 5) GoI policies; and 6) the USAID contract. b. The TAG should serve as the primary forum for developing an advocacy strategy, as well as for presenting evidence and best practices from other countries, including service delivery models that international experience has proven are safe and can expand access and reduce referrals to higher levels of care within existing personnel and resource constraints. The TAG should be structured to allow feedback from field-level (DoHs and PHCCs) staff as well as Kurdistan Regional Government MoH officials. c. As the activities within the model clinics are implemented. They should serve as hands-on demonstration sites via field visits, with the results examined through operations research. d. PHCPI should consider providing refresher training and/or study tours for the MoH on PHCCs in lower-middle income countries (e.g., Albania, Armenia, Egypt, Georgia, Indonesia, Moldova, Morocco, Philippines). 6.2.2. PHCPI and USAID should consider re-wording two of its higher-level beneficiary PMP indicators and also examine how other indicators will be measured (see Annex 11). 6.3 Recommendations for Question 3 6.3.1. USAID should consider modifying the current PHCPI contract to remove the focus on document production and training numbers to a stress on achieving change within the primary health care system and building capacity within the MoH. This will require agreement on a contract modification between URC and USAID. Specific suggestions are given in Annex 10. 6.3.2. The TAG must place more emphasis in fulfilling its higher-level role of providing technical advice to both PHCPI and the Ministry of Health. The immediate step would be to revise the agenda for the TAG meetings such that approximately 75% of meeting time is focused on presentations and discussions of PHC policies, strategies, and technical MID-TERM PERFORMANCE EVALUATION OF USAID/IRAQ PRIMARY HEALTH CARE PROJECT 21 direction, and 25% is spent on general information-sharing, coordination, and PHCPI achievements. 6.3.3 The TAG should draft terms of reference for the establishment of an Oversight Committee, which would be responsible for reviewing PHCPI activities through site visits and document review and then providing feedback to the TAG. 6.3.4 Almost all stakeholders need to strengthen project oversight and technical support by increasing their dedicated human resources: a. PHCPI should consider instituting a document by which project stakeholders verify their technical review of deliverables (see Annex 14). b. URC should work with the COP to ensure that s/he has the authority to negotiate with the MoH with regard to strategic planning and that PHCPI staff are empowered to advocate with stakeholders for rational decision making based on an established set of criteria (see Annex 15). c. URC and MSI should consider increasing the amount of technical review provided by their headquarters to ensure that deliverables are of high quality. d. USAID should continue its current pattern of having at least one FSN with continuous on-site technical oversight and additional intermittent technical assistance, as needed. It should also establish stronger linkages with USAID/Washington for technical assistance to be provided by the Health Systems Division (GH/HIDN/HS) and the Office of Country Support (GH/OCS). 6.4 Recommendations for Question 4 6.4.1 PHCPI needs to pay more attention to establishing a woman-friendly environment within PHCCs to reduce barriers to women accessing PHC services. Options include: a) reviewing and revising policy, if necessary, to reduce barriers to entry by Iraqi women wanting to train and work as physicians; b) increasing visual privacy during consultative services; c) focusing on providing women-related services in PHCCs; d) consider the feasibility of providing delivery services in PHCCs; e) providing incentives for female service providers to work in locations most in need of female health care workers; f) having mobile female service delivery providers where feasible and most practical; and, g) task-shifting so that female health workers are trained to provide the examinations and counseling currently provided by physicians and would work under these new protocols under a physician’s supervision. 6.4.2 Due to the project’s limited time-frame and the need to focus its efforts, USAID should advocate to other donors to address the need for services for victims of violence against women. 6.5 Recommendations for Question 5 6.5.1 URC should raise awareness among PHCPI staff of the different groups making up marginalized populations, international best practices in serving the marginalized, and advocacy strategies to raise awareness and concern among stakeholders, including the MoH. MID-TERM PERFORMANCE EVALUATION OF USAID/IRAQ PRIMARY HEALTH CARE PROJECT 22 6.5.2 The compliance survey should check whether all non-formal settlements and squatter areas are enumerated within the catchment areas of PHCCs. 6.5.3 PHCPI should study, strengthen, and scale up the MoH paper Health Visitor Program in all PHCCs with squatter areas and IDP settlements in their catchment areas. 6.5.4 PHCPI should consider conducting community-based needs assessments for other vulnerable groups such as the disabled, elderly, widows, and orphans, as well as women in general. This study can be carried out as part of the community-based survey to be implemented before the planned BCC campaign. 6.5.5 PHCPI-implemented campaigns should ensure the design of special BCC materials, spots, and community events that are addressed to reach squatters, IDP settlements, and remote rural areas to promote their knowledge and utilization of promoted services. Although PHCPI realizes that IDPs and other married women 18–44 living in squatter areas are a primary audience, this fact should always be explicitly highlighted so the campaign does not accidently miss an opportunity to design BCC materials, spots, and community events that are relevant to this vulnerable group. 6.6 Recommendations for Question 6 PHCPI activities should support institutionalization of the training and material development process within the MoH by shifting responsibility for organization and implementation of document development and training to the MoH. Suggestions are provided in Annex 10. 6.7 Recommendations for Question 7 (N/A) 6.8 Recommendations for Question 8 6.8.1 If requested, PHCPI can offer technical assistance for a technical review to support more effective allocation of future MoH funds for PHC. 6.8.2 PHCPI (and USAID) should monitor and closely coordinate with GoI procurements to ensure that no programmatic gaps exist (i.e., training, guidelines, and supervision match with provided equipment and supplies). This may involve PHCPI development of a tracking tool for all stakeholders. 6.8.3 PHCPI should continue to provide technical assistance to governorates in developing systems to procure equipment and supplies with cost-sharing funds. 6.8.4 If requested, USAID could provide short-term technical assistance to the MoH on health care financing to diagnose and remedy issues related to flow of funds and disbursements. 6.8.5 If necessary, USAID should request an audit of government cost-share expenditures per its previous agreements with the GoI. MID-TERM PERFORMANCE EVALUATION OF USAID/IRAQ PRIMARY HEALTH CARE PROJECT 23 7. ISSUES 7.1 Disclosure of Conflicts of Interest The three international evaluation team members have no known conflicts of interest. All three team members signed the “Disclosure of Real or Potential Conflict of Interest for USAID Evaluation” forms in which each individual has certified that he or she had no real or potential conflicts of interest related to this evaluation. 7.2 Statement of Differences There are no differences among the team members in the findings, conclusions, or recommendations related to this evaluation. 8. REFERENCES Below is a partial listing of references from this evaluation. A full and complete listing can be found in Annex 4. Kurdistan Regional Government, September 2012: Memorandum of Understanding (for the implementation of PHCPI). Ministry of Health/Government of Iraq, September 2012: Memorandum of Understanding (for the implementation of PHCPI) Ministry of Health, January 2009: A Basic Health Services Package for Iraq. Ministry of Health, various dates 2011-2012: Health Management Information System Repots Ministry of Health, various dates 2011-2012: HMIS Monthly Client Utilization Reports. Ministry of Health, 2013: New HMIS Service Registers. Ministry of Health/PHCPI, January 2012: Handbook of Quality Standards and Operational Guidelines for Management of Primary Care Clinics. Ministry of Health/PHCPI, January 2012: Protocols Regarding Emergency Obstetric Care and Newborn Care. Ministry of Health/PHCPI, February 2012: Behavior Change Communication Strategy for Primary Health Care Services. Ministry of Health/PHCPI, February 2012: National Statement of Patients’ Rights in Primary Health Care. Ministry of Health/PHCPI, April 2012: Communicable Diseases Control Guidelines. Ministry of Health/PHCPI, April 2012: IMCI Guidelines for Nurses. Ministry of Health/PHCPI, June 2012: Guideline for Diabetes Mellitus and Metabolic Syndrome Management. MID-TERM PERFORMANCE EVALUATION OF USAID/IRAQ PRIMARY HEALTH CARE PROJECT 24 Annex 1. Evaluation Statement of Work Mid-Term Performance Evaluation for Primary Health Care Project in Iraq STATEMENT OF WORK I. Project Description USAID/Iraq’s Primary Health Care Project (PHCP) was awarded to the University Research Corporate, LLC (URC) in March 2011, under the Capacity Building Office (CBO) Program of Iraq. The $74.9 million PHCP project is helping the Ministry of Health (MoH) to strengthen Iraq’s primary health care sector and achieve their strategic goal of providing improved primary health care (PHC) services to their citizens. The project is designed to modernize PHC by introducing international standards for a) quality of care, b) administration and management of health services and c) provider-client relationships and community involvement. USAID/PHCP works closely with the Iraqi government counterparts at central and regional levels to achieve these goals. The project works in 360 target PHC clinics throughout Iraq’s 18 provinces, with a central project office in Baghdad, and two regional offices in Maysan and Erbil provinces. PHCP’s work is organized around three components: 1) strengthen health management systems, 2) improve the quality of clinical services, and 3) encourage community involvement to increase the demand for and use of PHC services. The project is planned to continue through March 2015. II. Background Iraq faces many challenges in the health sector. The health status of the Iraqi people has significantly declined over the past two decades. The country’s under-five mortality rate is among the highest in the region, at 44 per 1,000 live births, with the majority of child￾age deaths occurring from preventable conditions such as pneumonia, diarrheal disease, and premature birth.5 Child malnutrition has increased steadily, with incidence of low birth weight exceeding 10%. Maternal mortality rates have increased to 84 per 100,000 live births, indicating a serious problem in providing perinatal health care and safe delivery services to pregnant women.6 As the country moves forward with stabilization and reform, it is evident that ensuring access to routine, high quality, and equitable healthcare is critical. As a result the Government of Iraq (GoI) is seriously engaged to respond to citizens’ needs, by strengthening its commitment to improve the quality of and access to PHC services. The ultimate goal of the project, as depicted in the results framework below, is to improve the quality of primary health care services in Iraq. The project achieves its goal through the three components noted above, which are intended to lead to three intermediate results. 5 WHO. Iraq Health Profile, 2009. http://www.who.int/gho/countries/irq.pdf 6 The above indicators were taken from the Iraqi Ministry of Health Annual Report, 2010 and MoH Statistics records 2010. MID-TERM PERFORMANCE EVALUATION OF USAID/IRAQ PRIMARY HEALTH CARE PROJECT 25 USAID is working in close partnership with the Iraqi Ministry of Health in this initiative. The Ministry’s inputs include leadership, strong commitment, allocation of resources (personnel, clinical facilities, health commodities, equipment, health infrastructure, etc). USAID contributes with technical expertise and support to help build sustainable institutional capacities to improve service delivery at the primary care level. This includes development of model clinics and health Centers of Excellence for nationwide replication. III. Purpose The purpose of this mid-term evaluation is to determine whether progress is being made in each of the project’s components, and to inform management decisions on how to further proceed during the remaining life of the project. The evaluation explores the overall performance of the project towards the planned objectives, the relevance of the development hypothesis and of the project’s design, effectiveness of current project management structure and its shortfalls. This evaluation will also assess the project’s effectiveness in reaching vulnerable groups. The findings of the evaluation shall guide the Mission and GoI to define the best way forward in the sector. IV. Existing Data USAID will provide the evaluation team, prior to their arrival, with a set of key documents listed below. Given the security and logistical challenges of operating in Iraq, the evaluation team should be aware that they may not be able to travel around as much as they would like. As a result, the team will need to review these documents in depth prior to their deployment to Iraq, to ensure that the time they spend on the ground is used MID-TERM PERFORMANCE EVALUATION OF USAID/IRAQ PRIMARY HEALTH CARE PROJECT 26 in an effective manner. The following information is available for desk review prior to field work: • Contract and any related amendments to the contract; • The project baseline survey reports; • PHCP quarterly reports and annual performance reports; • The Performance Management Plan (PMP) and data table; • Project annual work plans; • The project’s “Internally Displaced Persons Assessment Report.” V. Key Evaluation Questions The key questions to be addressed during the evaluation are the following: 1. To what extent has the project achieved or made progress towards the planned results to improve primary health care services in Iraq? 2. How realistic and appropriate is the project intervention logic? 3. To what extent has the project management structure been effective and how has it been affecting the program outcomes to date? 4. To what extent has the project reached out to vulnerable groups including women, persons with disabilities, and the internally displaced persons (IDPs)? 5. Is the project’s approach appropriate to respond to the needs of vulnerable groups and if not how can its approach be improved? 6. How is the training provided by the project being used in the clinics? 7. How is project-purchased equipment (if any) being used in the clinics? 8. How is the Government of Iraq cost-share being utilized? As requested below in the methodology section, the evaluation team is required to provide further detail in how it will respond to these questions in its evaluation design matrix. Illustrative sub-questions relevant to each category are included here as an annex for the team’s consideration. VI. Suggested Methodology In addition to desk review of relevant documents, the evaluation will use a non￾experimental design and employ a range of quantitative and qualitative data collection methods in answering the evaluation questions. The field work will consist primarily of meetings and key informant interviews with various stakeholders such as government, non-government, other donors, etc,, and will use interviews, questionnaires, surveys, and direct observation methods as appropriate for acquiring data from respondents. For each evaluation question/sub-question, the team will include explanations of measures or indicators, targets, baseline data (if any for normative questions), data sources, sample sizes, data collection instruments and data analysis plans. The evaluation team should provide to USAID the evaluation design matrix along with the evaluation implementation plan for approval. The team will be based in Baghdad with travel to the two project’s regional offices in Erbil and Maysan, as needed. MID-TERM PERFORMANCE EVALUATION OF USAID/IRAQ PRIMARY HEALTH CARE PROJECT 27 Upon arrival, the team is required to meet with USAID/Iraq to discuss the scope of work and clarify expectations. In addition to this evaluation kick-off meeting, the evaluation team is expected to have interim discussion of the preliminary findings with USAID and an exit briefing, including a presentation of an outline of the major findings and recommendations that will be included in the evaluation report. VII. Deliverables The following are major deliverables of this evaluation 1. Proposed evaluation implementation plan covering evaluation methodology, including data collection instruments along with the evaluation design matrix described above; 2. Evaluation kick-off meeting in which the team presents its understanding of this SoW and discusses the design matrix; 3. A mid-evaluation presentation of preliminary results. The presentation will summarize the key preliminary findings and discuss recommended actions; 4. Exit brief and presentation of major findings and recommendations, with outline of evaluation report; 5. Draft evaluation report; 6. Final report, no longer than 25 pages, excluding annexes, The report must follow USAID’s Evaluation Policy guidelines, noted below: • Executive Summary: should be able to sufficiently detailed, yet brief, to serve a stand-alone product; • Table of Contents: list section headings and page numbers, as well as any figures or tables; Introduction: state the purpose, audience, and outline of the evaluation; • Background: provide a brief overview of project, USAID project strategy and activities implemented in response to the problem; • Methodology: provide an evaluation design methods used , including constraints and gaps, greater detail should be included in the appendices; • Findings/Conclusions/Recommendations—explicitly answer each question. The report should distinguish between findings (the facts), conclusions (interpretation of the facts), and recommendations (specific and action oriented measures for the rest of the project life) • Issues: provide a list of key technical and/or administrative issues, if any; may include Disclosure of Conflict of Interest: statement to the effect that, as external evaluators, all evaluation team members provided a signed statement attesting to a lack of conflict of interest, or describing an existing conflict of interest relative to the project being evaluated, and Statement of Differences: When applicable, evaluation reports should include statements regarding any significant unresolved differences of opinion on the part of funders, implementers and/or members of the evaluation team; • References – including bibliographical documentation, meetings, interviews and focus group discussions; and • Annexes—annexes should include this statement of work, a glossary of terms, and a clear documentation of evaluation methods, schedules, interview lists and MID-TERM PERFORMANCE EVALUATION OF USAID/IRAQ PRIMARY HEALTH CARE PROJECT 28 tables, and any focus group scripts or questionnaires used; the presentation should be succinct, pertinent and readable. The report format should be presented in Microsoft Word and use 12-point type throughout the body of the report, using single-spacing and page margins (top/bottom and left/right) of one inch. The report will be posted on the Agency’s Development Experience Clearinghouse website for use by the public along with any rebuttal report from the implementing agency or USAID as appendix. VIII. Team Composition The team should consist of at least five members, should have a team leader, two other technical experts, and two to three support members for logistics and translation/ interpretation services. Technical members should have prior experience in the primary health care field and program evaluation. At least one team member should have experience in Iraq and one member should read and speak fluent Arabic. The suggested team composition and the required qualification criteria are the following: • Team Leader / Evaluation Expert (International) – This person should have a strong background in international development or development program evaluation in the health sector. S/he should have extensive experience in managing public and private health sector programs. Strong experience in strategic planning, surveillance, operations research, and/or monitoring and evaluation of global and national health programs is required. S/he should have a good understanding of project administration, financing and management skills, including an understanding of USAID projects. The person must be able to lead a diverse team of technical and management experts and to interface with stakeholders ranging from government to non-government organizations to donors and beneficiaries. S/he should have substantial demonstrated expertise in evaluation techniques, in design, management and evaluation of the health sector programs as a requirement. S/he should have excellent English language writing, editing and communication skills. An advanced degree in Public Health, Social Sciences, Business Administration, or other relevant field is required. Familiarity with the Middle East is desirable but not required. Knowledge of Arabic is a plus, but not required, as long as one team member is fluent (written and spoken). • Health Sector Experts (two, International) – These persons should possess substantial international working experience from the field of health system management change, primary health care, planning and health systems strengthening, USAID monitoring, and evaluation processes. A Master’s degree in public health or related field relevant to the broad areas of health systems strengthening is required. They should have a good understanding of developing and implementing assessments and impact studies. They should have demonstrated competence in assessing priorities and in managing a variety of activities in a time-sensitive environment, and in meeting deadlines with attention to detail and quality, as well as being strategic thinkers with interpersonal skills and managerial, coordination, and organizational skills. Professional English-language proficiency (both written and MID-TERM PERFORMANCE EVALUATION OF USAID/IRAQ PRIMARY HEALTH CARE PROJECT 29 oral) is required. They should have knowledge and comprehensive understanding of the public health system sector. Previous experience in evaluating USAID-supported programs is an added advantage. Familiarity with the Middle East and Iraq in particular, is desirable but not required. Knowledge of Arabic is a plus, but not required, as long as one team member is fluent (written and spoken). • Admin/logistical support person (local) – Assist with travel, meeting arrangements, accommodation and any other administrative and office support the team requires. This individual should be based in Baghdad and possess fluent Arabic language skills. • Translation/Interpreter (English, Arabic, and Kurdish) – The interpreter/translator should have substantial experience in interpretation of oral communications and translation of written documents, and should possess the following skills and traits: H/she should be an Iraqi national with knowledge of the country geographically; should be fluent in Arabic, Kurdish and English. H/she should have familiarity with Iraq’s Arab and Kurdish cultures and should be able to translate documents. S/he should possess extensive vocabulary in the three languages; have ability to express thoughts clearly and concisely in all the above languages. IX. Schedule, Timeline and the Estimated Level of Efforts Activity Period of Performance LOE for TL and Health Sector Experts LOE for T/I and DCs Document review and Implementation Plan QED requests USAID approval for personnel January 16, 2013 USAID approval for personnel January 17-22, 2013 Negotiate NPSC agreements; obtain background. documents January 23-28, 2013 Background review Jan 29-February 4, 2013 5 Preparation and submission of Implementation Plan February 5-11, 2013 5 USAID review of Implementation Plan February 12-18, 2013 Revision and submission of Implementation Plan February 19-20, 2013 2 Travel to Baghdad, Iraq February 21-22, 2013 2 Field Work Team planning meeting in Baghdad February 23, 2013 1 Kick-off meeting February 24, 2013 1 Data collection Feb 25-March 13, 2013 15 15 Presentation of Preliminary Results March 14, 2013 1 1 Continued data collection and analysis March 15-30, 2013 14 14 MID-TERM PERFORMANCE EVALUATION OF USAID/IRAQ PRIMARY HEALTH CARE PROJECT 30 Activity Period of Performance LOE for TL and Health Sector Experts LOE for T/I and DCs Exit brief and submission of outline of report March 31, 2013 1 Travel out of Baghdad April 1-3, 2013 2 Report Writing Evaluation Report draft preparation and submission April 4-17, 2013 10 USAID review and comments on Evaluation Report draft April 18-28, 2013 Report revisions and submission of final report April 29-May 5, 2013 5 MID-TERM PERFORMANCE EVALUATION OF USAID/IRAQ PRIMARY HEALTH CARE PROJECT 31 Annex 2. Glossary of Terms Basic Health Services Package: A BHSP in a low-income country consists of a limited list of public health and clinical services that will be provided at the primary and/or secondary care level. BHSPs include different interventions in different countries reflecting variations in economic, epidemiological, and social conditions. They are intended to be a guaranteed minimum (i.e., some clients will have needs that cannot be met by the BHSP). With a BHSP, the human skills, drugs, equipment, and other resources required to deal with interventions within the package should be available. A BHSP is generally developed using some combination of cost-effectiveness analysis and other technical, political, and social considerations. The aim is to concentrate scarce resources on the services that provide the best “value for money.” Behavior change communications: BCC is a research-based, consultative process of addressing knowledge, attitudes, and practices through identifying, analyzing, and segmenting audiences and participants in programs and providing them with relevant information and motivation through well-defined strategies, using an appropriate mix of interpersonal, group, and mass media channels, including participatory methods. Cost-sharing: Cost-sharing is a multiparty arrangement under which the costs of a program or project are shared by the involved parties, according to an agreed-upon formula. These parties may include the host government, private foundations, businesses, or individuals. Cost-sharing can be financial or in-kind. In-kind contributions include volunteer time, donated supplies, equipment and other property, and unrecovered indirect costs. Cost-sharing can be flexible and case-specific, and can be used to support or contribute to the achievement of results. Essential drug list: This is a list of medicines that health managers define for procurement and for availability at different levels of health services. The objectives are rational drug use and cost control. Service providers are usually expected to prescribe only from these lists or else to refer clients. Integrated management of childhood illnesses: IMCI is a systematic approach developed by UNICEF and WHO for assessing a child’s health and providing health services. The objective is to ensure that when a child presents to health workers with symptoms of illness, a full assessment is conducted to identify underlying and other causes of illness, rather than treating the initial presenting symptoms. IMCI also requires that when a child comes to a health facility, he or she is assessed for additional needs for preventive services such as growth monitoring to identify growth faltering or malnutrition and routine immunizations, thereby preventing “missed opportunities” for the child to receive important preventive services. Finally, IMCI promotes integrated records that provide the health history of a child and allow follow up over time. Local health committees: LHCs are joint committees that include people from the PHCC and their counterparts from the local community. Community members may be elected officials, community leaders, NGO representatives, mosque sheikhs, tribal leaders, etc. LHCs meet periodically to discuss issues and plan for service provision to improve quality. They can also conduct awareness raising in the community regarding health issues and services. MID-TERM PERFORMANCE EVALUATION OF USAID/IRAQ PRIMARY HEALTH CARE PROJECT 32 Quality improvement process: QIP for health services is a defined process that is implemented by personnel in the service delivery setting. The process includes forming an interdisciplinary committee, meeting, maintaining records on meetings, identifying and prioritizing perceived problems related to quality services, and identifying options for resolving these problems. As per PHCPI documents, “Quality improvement is guided by principles of teamwork, a focus on the client, systems and processes, and measurement of results.” The focus on teamwork recognizes that team members bring valuable insights regarding the process to be improved because of their knowledge of and experience with it, and are more likely to implement improvements they helped to develop. The focus on systems and processes recognizes that providers must understand the service system and its key service processes to improve them; resolving the problem of unclear, redundant, or incomplete processes or systems yields better results than placing blame on individuals. A focus on the client emphasizes that services should be designed to meet the needs and expectations of clients and communities. A focus on measurement means that data are needed to analyze processes, identify problems, and measure performance. This focus promotes taking action based on facts rather than on assumptions. Task shifting: Task shifting is a term that refers to reorganizing tasks and responsibilities of health workers, most often implying training less highly trained personnel to carry out tasks previously carried out by higher skilled health workers. When accompanied by training, clear guidelines, and supervision, task shifting allows services that otherwise are not available because of a shortage of skilled service providers to be provided safely and with good quality. Tertiary care: Tertiary care is specialized consultative health care, usually for inpatients and on referral from a primary or secondary health professional, in a facility that has personnel and facilities for advanced medical investigation and treatment, such as a tertiary referral hospital. Examples of tertiary care services are cancer management, neurosurgery, cardiac surgery, plastic surgery, treatment for severe burns, advanced neonatology services, palliative, and other complex medical and surgical interventions. MID-TERM PERFORMANCE EVALUATION OF USAID/IRAQ PRIMARY HEALTH CARE PROJECT 33 Annex 3. Methodology The methodology for the mid-term evaluation was finalized based on several factors: 1) The project has had a staggered roll-out and short implementation time (two years). 2) Logistic and security considerations limited the site visits and data that the international consultants could collect firsthand. 3) QED was able to identify eight experienced data collectors for fieldwork. It was logistically feasible for the teams of two persons each to visit a total of 24 different sites within the allocated timeframe. The methodology was selected to provide information from different sources that could be triangulated to achieve the following objectives: 1) Externally validate information on project activities from project reports 2) Provide a picture of the current status of health services at the PHCCs 3) Gather information from different sources on project focus, activities, working methods, achievements to date, and potential for achieving objectives 4) Identify issues and receive recommendations for improving the project’s probability of success Data Collection Methods and Tools The evaluation team developed a structured facility audit, as well as structured question guides for health workers, clients, district managers, and LHCs, which team members shared with USAID for comments and input. The tools were translated into Arabic and Kurdish. After training, all tools except for the district manager structured interview were pretested in Arabic by data collectors in two facilities in Baghdad, which resulted in minor revisions prior to finalization. Structured interview guides were developed separately for key informants at the central government and project office levels. These were tailored to cover specific topics where the project and key informant responsibilities overlapped. The final English language version of the data collection tools are provided in Annex 18, A-I. Table 3-1: Data Collection Tools and Objectives Data Collection Method Objective Review of documents See Annex 4 for a list of documents reviewed. See Annex 4 for a list of documents reviewed. Provide objective evidence of service conditions relevant to the guidelines that have been introduced and for which training has been provided. Health worker interview Provide objective evidence of training as well as the qualifications of and services provided by the trained health workers. Provide qualitative reports from health workers on changes associated with the training. MID-TERM PERFORMANCE EVALUATION OF USAID/IRAQ PRIMARY HEALTH CARE PROJECT 34 Data Collection Method Objective Client exit interview Provide quantitative evidence of knowledge of PHC services, client rights issues, and client satisfaction. Provide qualitative evidence of changes in services or service conditions over the past two years as well as suggestions for service improvement. LHC member interview Provide quantitative and qualitative information on the functioning and activities of the LHCs and impressions on their effectiveness. District management interview Provide quantitative and qualitative information on interactions with project activities and impressions of strengths and weaknesses related to the different components. Other key informant interviews Project, central MoH, and other key informants were identified to provide more in-depth feedback on the project’s focus, working methods, and effectiveness/perceptions of potential for achieving objectives. Ethical considerations All interviewees at the district level and below were read a consent form and verbal agreement received. Confidentiality of individual respondents was guaranteed. Sample selection Facility: To ensure that the sample included facilities and districts where there has been the most project activity, as well as facilities and districts that were part of routine project activities, it was decided to purposely sample from the following groups: • Facilities that have been identified to be upgraded to Model Clinics • Facilities where the new medical record system was pilot tested • Facilities from among all remaining facilities The sample was selected from each group using a list stratified by province and district; the sample facility was then systematically selected using the following methodology: • All eligible facilities for the group were listed by province and district. • The total number of eligible facilities for each group was divided by the desired sample size to provide the sampling interval. • A random number generator was used to identify a starting point within the list. • The sample was selected starting from the random number, with every xth facility (according to the sampling interval) then selected After the Model Clinic and medical records pilot facility samples were selected, the remaining facilities within each group were returned to the master list of facilities for potential selection for the general facilities sample. While the evaluation team believed that the evaluation’s objective would be best served by looking at main PHCCs since they were more likely to have staff and resources impacted by project activities, there was also MID-TERM PERFORMANCE EVALUATION OF USAID/IRAQ PRIMARY HEALTH CARE PROJECT 35 a desire to not exclude sub-centers, as they could be impacted by some project activities. As a result, sub-clinics and PHCCs were equally considered for the general facilities sample. Tables 2 and 3 provide information on the sampling frame and the final sample. Table 3-2: Sampling Frame and Numbers Selected Group Eligible Number Selected from Group Medical record pilot (all main PHCCs) 26 5 Model Clinic (all main PHCC) 35* 5 Remaining facilities: Main PHCCs 254* 11 Remaining facilities: Sub-centers 94 3 *Two main PHCCs, of which one was a Model Clinic, were ineligible because they were used for tool pretesting. In other words, a total of 358 PHCCs were available for sampling. After the 10 Model and medical record clinics were chosen, there remained 348 in the population to be sampled. Health Worker Interviews: The interviewed health workers were selected purposefully and opportunistically. Interviewers were instructed to use the following criteria to select five health workers to interview: a) Facility in-charge b) If possible, one male and one female client service provider who received training through the project; if trained staff were unavailable, other health workers who provide client services were selected c) One staff who works with the Local Health Committee d) One staff who works with maintenance and repair When there were no health workers present who met the criteria, the interviewers selected other staff who had received training, such as the deputy in-charge or laboratory staff. Client Exit Interviews: Clients were selected purposely and opportunistically. Interviewers were instructed to select clients who had completed their visit to the clinic and were ready to leave. They were to try to select at least three females out of five clients. Local Health Committee interviews: LHC members affiliated with the sample facilities were selected for interviews. Evaluation team members interviewed LHC member from the facility, with attempt made to interview an additional two community members from each LHC. In most cases, community LHC members came to the health facility but were interviewed privately. District Managers: The district managers affiliated with each sample facility were interviewed. The teams identified managers responsible for the PHCCs, and they were asked to call other district staff to participate in the interview if they did not know the answer to some of the questions. Some of the districts covered more than one of the sample facilities. In total, 20 district-level managers were interviewed. MID-TERM PERFORMANCE EVALUATION OF USAID/IRAQ PRIMARY HEALTH CARE PROJECT 36 Other Key Informants: USAID, the project, and the evaluation team identified other informants for interviews. Annex 5 provides a list of persons interviewed. Data collection Eight data collectors were identified, most of whom had previously worked with QED on evaluations. As none of the data collectors had substantial experience with health services or systems, the training and tools were tailored to capture the level of quality information deemed feasible for persons with limited subject-matter background. The teams conducted two days of classroom training with job-aids including pictures of items included in the facility survey and instruction guides distributed for completing the various instruments and structured interviews. On the third day the data collection teams were divided between two Baghdad clinics (selected by the project prior to sample selection) for practical experience and to pretest facility, health worker, client, and LHC data collection tools. QED staff and the international consultants were present during the training and the pretest to provide guidance and clarify issues prior to fieldwork. Data collectors were divided into groups of two and assigned regions and facilities where they would collect information from facilities, health workers, clients, LHC members, and district managers. The total process for data collection for each facility and the affiliated interviews took two to three days, depending on how busy the facility’s schedule and informant availability. A Kurdish speaker was identified to collect data in Erbil and Sulamaniyah. As she was unable to participate in the training, she received on-the-job training with a team of data collectors during their first two days of data collection. An international consultant and a QED staff participated in visits with her for her first assigned facility. QED staff was present during all of her data collection to provide support. International consultants accompanied teams when security allowed. The objective was to observe the data collectors, ensure data quality, and to clarify issues when necessary. During these joint visits, the consultants conducted targeted, probing interviews with the informants to gain further insights into questions and issues related to the project and the evaluation. The international consultants visited a total of five sample sites and four district management offices. Data processing and analysis Each international consultant managed the data for the evaluation section for which she was responsible. QED staff developed Excel spreadsheets for data entry in collaboration with the international consultants. Each data collector was responsible for translating their notes into English and completing the relevant spreadsheets. These were submitted to the QED office, along with the original data collection tools and notes. QED staff reviewed the entered data for accuracy. Upon completion of the fieldwork, the data collectors worked at the QED offices two days to clarify any issues and to share their findings and perceptions with the international consultants. Limitations The evaluation’s time frame and the lack of data collectors with knowledge of health services and systems limited the sample size and the subject matter depth of the data collected by third parties. Security issues limited data that could be collected by the MID-TERM PERFORMANCE EVALUATION OF USAID/IRAQ PRIMARY HEALTH CARE PROJECT 37 international consultants. As a result, the bulk of the evaluation depended on key informant interviews by the international consultants and review of documents, with the collected data used to provide greater depth and perspective to add to the impressions gathered by the international consultants. As a result, the data was used to provide illustrative examples of issues, but not to draw statistically significant comparisons or conduct analyses. A baseline assessment across the PHC system was conducted in September 2011, and a mini-assessment of the project’s 360 target facilities was completed the last quarter of 2012 (with the sub-set of the results available to the evaluation team on April 27, 2012, the last week of data collection). Information on objectively measured change between the baseline and midterm should be available through these surveys; however, the timing did not permit inclusion of valid comparisons for this evaluation. The evaluation does not provide any information on actual quality of services, but rather provides information on “readiness to provide quality services” (availability of trained staff, guidelines, equipment, medicines, and infrastructure elements for adhering to guidelines). MID-TERM PERFORMANCE EVALUATION OF USAID/IRAQ PRIMARY HEALTH CARE PROJECT 38 Table 3-3: Final Sample with Basic Characteristics # Province Directorate of Health (DoH) District PHCPI Site ID Name of the PHC Center Type Health Center Catchment Area (Population) Managed by # of Physicians Comment Baghdad Rusafa Rusafa BR07 Hay Babil Main 25781 Physician 4 ** Pretest, model Baghdad Rusafa Baghdad Al￾Jadedah BR17 Al-Dhubat Main 19676 Physician 7 ** Pretest, model, training and family medicine, medical record pilot 1 Baghdad Karkh Karkh BK02 Al-Dakhelia Main 34543 Physician 25 2 Baghdad Karkh Karkh BK08 Al-Mansour (Baghdad) Main￾Model 21808 Physician 21 Model, medical record pilot 3 Babil Babil Al-Hila 2 BB06 Babil Training Center Main 39622 Physician 6 4 Babil Babil Al-Hila 2 BB10 Nader Main￾Model 46448 Physician 4 Model, medical record pilot 5 Karbala Karbala Al-Husseinia KR18 Al-Wand Sub 10100 Paramedic None 6 Najaf Najaf Najaf Al￾Shimaly NF04 Al-Naser Main 70000 Physician 2 7 Diwaniyah Diwaniyah Al-Diwaniyah 2 DI04 Al-Eskan Al￾Qadeem Main 38775 Physician 3 Model, medical record pilot 8 Diwaniyah Diwaniyah Affak DI10 Efak Alawal Main 38500 Physician 5 9 Wasit Wasit Al-Kout 2 WA06 Al-Jihad Main￾Model 58176 Physician 3 Model 10 Basrah Basrah Sector 2 BA10 Al-Qadissiya Main 20911 Physician 2 11 Maysan Maysan Al-Awal MN01 Dijlah Main￾Model 11208 Physician 3 Model 12 Maysan Maysan Al-Awal MN14 Al-Eskan Main 39685 Physician 3 Medical record pilot 13 Dhi-Qar Dhi-Qar Al-Rifaey TQ09 Al-Rifaey Main 43440 Physician 4 Model, medical record pilot 14 Muthanna Muthanna Al-Warka'a MU15 Al-Warka'a Main￾Model 54045 Physician 2 Model, medical record pilot 15 Diyala Diyala Muqdadia DY13 Al-Umrania Sub 11400 Paramedic None 16 Kirkuk Kirkuk Debis KE02 Alton Kobry Main 17814 Pharmacist 6 Model, medical record pilot MID-TERM PERFORMANCE EVALUATION OF USAID/IRAQ PRIMARY HEALTH CARE PROJECT 39 # Province Directorate of Health (DoH) District PHCPI Site ID Name of the PHC Center Type Health Center Catchment Area (Population) Managed by # of Physicians Comment 17 Anbar Anbar Al-Rimady 2 AN09 Al-Omal Main 13000 Physician 2 Medical record pilot 18 Anbar Anbar Al-Rimady 2 AN10 Al-Habania Main 9000 Physician 2 Replacement for Albu-Ubaid Anbar Anbar Al-Rimady 2 AN13 Albu Ubaid Sub 4000 Paramedic None Excluded due to insecurity 19 Ninawa Ninawa Al-Ayman NA01 Bab Albeedh Main 31835 Physician 4 Model 20 Ninawa Ninawa Al-Ayman NA06 Al-Yarmouk Main 57148 Physician 7 21 Erbil Erbil Erbil AR09 Shahedan Main 31951 Physician 5 22 Duhok Duhok Smeel DO2 DK06 Duban Main 15884 Physician 4 Medical record pilot 23 Sulaymaniyah Sulaymaniyah Sulaymaniyah SU04 Sarchanar Main 24793 Physician 7 Model 24 Sulaymaniyah Sulaymaniyah Sulaymaniyah SU07 Shakrakah Main 1200 Physician 1 ** Visited by international consultant. Medical record pilot: Selected from medical record pilot group Model: Selected from Model Clinic group MID-TERM PERFORMANCE EVALUATION OF USAID/IRAQ PRIMARY HEALTH CARE PROJECT 40 Annex 4. References Kurdistan Regional Government, September 2012: Memorandum of Understanding (for the implementation of PHCPI). Ministry of Health/Government of Iraq, September 2012: Memorandum of Understanding (for the implementation of PHCPI). Ministry of Health, January 2009: A Basic Health Services Package for Iraq. Ministry of Health, various dates 2011–2012: Health Management Information System Reports. Ministry of Health, various dates 2011–2012: HMIS Monthly Client Utilization Reports. Ministry of Health, 2013: New HMIS Service Registers. Ministry of Health/PHCPI, January 2012: Handbook of Quality Standards and Operational Guidelines for Management of Primary Care Clinics. Ministry of Health/PHCPI, January 2012: Protocols Regarding Emergency Obstetric Care and Newborn Care. Ministry of Health/PHCPI, February 2012: Behavior Change Communication Strategy for Primary Health Care Services. Ministry of Health/PHCPI, February 2012: National Statement of Patients’ Rights in Primary Health Care. Ministry of Health/PHCPI, April 2012: Communicable Diseases Control Guidelines. Ministry of Health/PHCPI, April 2012: IMCI Guidelines for Nurses Ministry of Health/PHCPI, June 2012: Guideline for Diabetes Mellitus and Metabolic Syndrome Management. Ministry of Health/PHCPI, June 2012: Guideline for Early Management and Life Support of Trauma in PHC Centers. Ministry of Health/PHCPI, June 2012: Infection Prevention and Waste Disposal Guidelines for Primary Health Care Centers. Ministry of Health/PHCPI, June 2012: National Guidelines for Primary Health Care Physicians: Hypertension: Prevention, Diagnosis, and Treatment (Directorate of Public Health, Non-Communicable Diseases Section). Ministry of Health/PHCPI, July 2012: Guidelines for the Diagnosis and Management of Asthma. Ministry of Health/PHCPI, July 2012: Infection Prevention and Waste Management Training Curriculum. Ministry of Health/PHCPI, August 2012: Nursing Standards for PHC Facilities in Iraq. Ministry of Health/PHCPI, October 2012: Operational Research on Recording and Reporting of Maternal Deaths in Iraq. MID-TERM PERFORMANCE EVALUATION OF USAID/IRAQ PRIMARY HEALTH CARE PROJECT 41 Ministry of Health/PHCPI, October 2012: Participant’s Guide (Workshop) to Evidence￾based Intrapartum and Newborn Care for PHC Center Staff. Ministry of Health/PHCPI, November 2012: Training Curriculum of Trauma Guideline for Primary Health Care in Iraq. Ministry of Health/PHCPI, December 2012: Training Curriculum: Leadership, Communication and Team Building for Local Health Committees (Arabic). Ministry of Health/PHCPI, December 2012: Guideline for Management of Overweight/Obesity for Primary Health Care Workers. Ministry of Health/PHCPI, December 2012: Operational Research on Maysan Health Visitor and e-Health Program Evaluation. Ministry of Health/PHCPI, December 2012: Training curriculum on Communicable Diseases Control Guidelines. Ministry of Health/PHCPI, 2012: All IMCI individual training guidelines and records. Ministry of Health/PHCPI, 2012 All IMCI individual training guidelines and records for physicians. Ministry of Health/PHCPI, 2012: Handbook for Community Partnerships for Primary Health Care Developed. Ministry of Health/PHCPI, 2012: Laboratory Standards Guideline for Primary Health Care Clinics in Iraq. Ministry of Health/PHCPI, 2012: Strengthening Primary Health Care Services to Internally Displaced Populations in Iraq, Recommendations Based on Field Assessment. Ministry of Health/PHCPI, 2012: Training Curriculum for Nurses and Paramedics in Family Health. Ministry of Health/PHCPI, 2012: Training Guide for Physicians NCD Clinical Guidelines. Ministry of Health/PHCPI, 2012: Training Guide for Physician Trainers NCD Clinical Guidelines. Ministry of Health/PHCPI, January 2013: Training Curriculum on Management of Overweight/Obesity. Ministry of Health/PHCPI, February 2013: Guidelines for Early Detection and Periodic Screening of Breast and Cervical Cancers in Primary Health Care Settings in Iraq. Ministry of Health/PHCPI, February 2013: Maternal and Child Nutrition Guidelines. Ministry of Health/PHCPI, February 2013: Premarital Counseling Clinical Services Guidelines. Ministry of Health/PHCPI, March 2013: Training curriculum on Maternal and Child Nutrition for the Primary Health Care Worker. Ministry of Health/PHCPI, April 2013: Handbook of Quality Standards and Operational Guidelines for Clinical Services Delivery in Primary Health Care. MID-TERM PERFORMANCE EVALUATION OF USAID/IRAQ PRIMARY HEALTH CARE PROJECT 42 Ministry of Health/PHCPI, 2013: National Guidelines on Menopause. Ministry of Health/PHCPI, 2013: Referral system orientation guidelines. Ministry of Health/PHCPI, 2013: Referral system training curriculum. Ministry of Health/PHCPI, 2013: Standard Operation Procedure for Facility and Equipment Maintenance Management in Primary Health Care Centers. Ministry of Health/PHCPI: Community Health Partnerships: Operational Guidelines for Local Health Committees for Primary Health Care (Arabic). PHCPI, June 2011: Quarterly Performance and Financial Report, April–June 2011. PHCPI, September 2011: Quarterly Performance and Financial Report, July–Sept 2011. PHCPI, December 2011: Baseline Assessment Report. PHCPI, December 2011: Quarterly Performance and Financial Report, October– December 2011. PCHPI, 2011: Technical Advisory Group Roles and Responsibilities. PHCPI, January 2012: Year 2 Work Plan. PHCPI, March 2012: Annual Report – Year 1. PHCPI, August 2012: National BCC Strategy to Promote Primary Health Care in Iraq. PHCPI, September 2012: Annual Report – FY 2012. PHCPI, September 2012: Quarterly Performance and Financial Report, July–September 2012. PHCPI, December 2012: Quarterly Report, October–December 2012. PHCPI, January 2013: Assessment Report: Health and Social Needs among Internally Displaced Persons (IDPs) in Iraq. PHCPI, February 2013: Performance Management Plan. PHCPI, March 2013: Organizational Chart. PHCPI, March 2013: Work Plan FY 2013. PHCPI, 2013: Equipment to be Procured (Model Clinics). QED, January 2013: Limited Data Quality Assessment of PHCP. QED, June 2012: Data Quality Assessment of PHCP. Technical Advisory Group, March 2012: From Competition to Partnerships. Technical Advisory Group, 2011–2012: Meeting Notes for the First Five TAG Meetings. USAID, December 2009: Health Activity Concept Paper. USAID, March 2011: PHCPI Contract. USAID, November 2012: Modification of PHCPI Contract. USAID and Ministry of Health, September 2012: Meeting Notes on Cost-share. MID-TERM PERFORMANCE EVALUATION OF USAID/IRAQ PRIMARY HEALTH CARE PROJECT 43 Annex 5. Persons Met Government of Iraq/Ministry of Health Muna Atanah Ali, Director Non-communicable Diseases Section Hasan Hadi Baqer, Director General, Directorate of Public Health Muthana Aziz Belal, Manager, Al-Salam PHCC Mohammed Jaber Huwail, Deputy Director General, Directorate of Public Health Ikhlass Younis Muhammed, Family Medicine Specialist, PHC and Family Medicine Department Raghdaa Dh. Sadeq, Specialist Physician, Head of Training and Research Division Ammar Sebahy, PHC Clinic Unit Manager, Karkh District, Baghdad Ali Awad Zedan, Deputy Manager, Al-Salam PHCC Government of Iraq/Ministry of Health (KRG) Janeed Al Jazrawee, Coordinator (former), MoH for PHCP Dlovan Sheik Omar, Director, Primary Health Care and Family Medicine Government of Iraq/State Ministry of Women’s Affairs Jwan Akram Ameen, Project Director Local Community of Internally Displaced Persons Abdel Hussein Beraissam Abbas, IDP Community Representative, El-Tobsh Compound Management Sciences International Brad Favor, Director of Field Operations Andrew Griminger, Senior Vice President for Business Development Primary Health Care Project in Iraq (Baghdad) Sarah Abdulrazzaf, Quality Improvement Specialist Mohammed Adel, IDP Specialist Mezaal Alhatim, Communication and Outreach Advisor Thamer Al Hilfi, Acting Chief of Party Mohammed Adel Al-Kayat, Quality Improvement/Quality Assurance Specialist Einas Bassim, BCC Advisor Robert Hagan, M&E Director Asma’a Hussein, M&E Manager Saad Jassim, Facilities and Equipment Maintenance Management Advisor Lorraine Porteus, Director of Community Health Partnerships Hani Riad, Deputy Director for Health Systems Management Youssef Tawfik, Director for Quality Clinical Care Saliena K. Murad, Community Health Partnership Specialist MID-TERM PERFORMANCE EVALUATION OF USAID/IRAQ PRIMARY HEALTH CARE PROJECT 44 Primary Health Care Project in Iraq (Erbil) Thanaa Alladin Aqiqi, Senior Regional Director Tatweer Project (closed) Ali Kazan, Senior Advisor for Project Management (former) UNFPA Ammar Abdul-Qahar, Program Coordinator Georges M. Georgi, Representative United States Agency for International Development (USAID) Lana Al-Shimmery, Health Specialist Pavel Basiladze, Program Development Specialist Rodeina Fattah, Director, Capacity Building Office Stanley Golooba, Performance Management Specialist, Capacity Building Office David Harden, Deputy Mission Director Zhaneta Shatri, Deputy General Development Officer (Health) Sheila Young, Director, Program Office Lisa Whitley, General Development Officer, Capacity Building Office University Research Corporation Hala Jassim, Senior Associate, Program Support Team Neeraj Kak, Senior Vice President World Health Organization Ezechiel Bisalinkumi, Technical Officer Syed Jaffer Hussain, Head of Mission MID-TERM PERFORMANCE EVALUATION OF USAID/IRAQ PRIMARY HEALTH CARE PROJECT 45 Annex 6. Documents Produced and Training Provided Project Documents Produced for Training and Research I Management 1 Deliverable 1.2a. Handbook of Quality Standards and Operational Guidelines for Management of Primary Care Clinics (Ministry of Health, January 2012) 1a Leadership and Management Participants Guide (Arabic) Leadership and Management for Primary Health Care Managers and District Managers. Participant Guide (PHCPI/MoH, December 2011) 1b Training Curriculum for Leadership and Management Managers of Primary Health Care (PHCPI/MoH, December 2011) Management Systems 2 Standard Operating Procedures for Facility and Equipment Maintenance Management in Primary Health Care Centers (2013) 3 Infection Prevention and Waste Disposal Guidelines for Primary Health Care Centers (PHCPI/MoH, June 2012) 3a Infection Prevention and Waste Management Training Curriculum (PHCPI/MoH, July 2012) 4 Deliverable 2.4.d . Current supervision system updated/revised (January 2012) 5 Referral system orientation guidelines (2013) (Arabic) 5a Referral system training curriculum (March 2013) II Standards 1 Handbook of Quality Standards and Operational Guidelines for Clinical Services Delivery in Primary Health Care (April 2013) (Draft) 2 Laboratory Standards Guideline for Primary Health Care Clinics in Iraq (PHCPI/MoH, 2012) (Draft) 3 Nursing Standards for PHC Facilities in Iraq (PHCPI/MoH, August 2012) 3a Training Curriculum for Nurses and Paramedics in Family Health (PHCPI/MoH, 2012) 4 Referral system training curriculum (March 2013) III Clinical services 1 Protocols regarding Emergency Obstetric Care and Newborn Care (PHCPI/MoH, January 2013) 1a Participant’s Guide 5-Day Workshop: Evidence-based Intrapartum and Newborn Care for PHC Center Staff (PHCPI/MoH, October 2012) 1b Trainers Guide 5-Day Workshop: Evidence-based Intrapartum and Newborn Care for PHC Center Staff (PHCPI/MoH, October 2012) 2 IMCI Guidelines for Nurses (PHCPI/MoH, April 2012) 2a All IMCI individual training guidelines and records (PHCPI/MoH, 2012) 2b All IMCI individual training guidelines and records for physicians (PHCPI/MoH, 2012) 3 Maternal and Child Nutrition Guidelines (PHCPI/MoH, February 2013) 3a Training Curriculum on Maternal and Child Nutrition for the Primary Health Care Worker (PHCPI/MoH, March 2013) 4 National Guidelines on Menopause (PHCPI/MoH, 2013) 5 Guideline for Management of Overweight/Obesity for Primary Health Care Workers (PHCPI/MoH, December 2012) 5a Training Curriculum on Management of Overweight/Obesity (PHCPI/MoH, January 2013) 6 Premarital Counseling Clinical Services Guidelines (PHCPI/MoH, February 2013) 7 Communicable Diseases Control Guidelines (PHCPI/MoH, April 2012) 7a Training curriculum on Communicable Diseases Control Guidelines (PHCPI/MoH, December 2012) MID-TERM PERFORMANCE EVALUATION OF USAID/IRAQ PRIMARY HEALTH CARE PROJECT 46 Project Documents Produced for Training and Research 8 National Guidelines for Primary Health Care Physicians: Hypertension: Prevention, Diagnosis, and Treatment (PHCPI/MoH, Directorate of Public Health, Non-Communicable Diseases Section. June 2012) 9 Guidelines for the Diagnosis and Management of Asthma (PHCPI/MoH, Directorate of Public Health, Non-Communicable Diseases Section, July 2012) 10 Guideline for Diabetes Mellitus and Metabolic Syndrome Management (PHCPI/MoH, June 2012) 10a Training Guide for Physicians NCD Clinical Guidelines (PHCPI/MoH, 2012) 10b Training Guide for Physician Trainers NCD Clinical Guidelines (PHCPI/MoH 2012) 11 Guideline for Early Management and Life Support of Trauma in PHC Centers (PHCPI/MoH, June 2012) 11a Training Curriculum of Trauma Guideline for Primary Health Care in Iraq (PHCPI/MoH, November 2012) 12 Guidelines for Early Detection and Periodic Screening of Breast and Cervical Cancers in Primary Health Care Settings in Iraq (PHCPI/MoH, February 2013) IV Community 1 Deliverable 3.2a . Handbook for Community Partnerships for Primary Health Care Developed (July 2012) 2 Deliverable 3.1.a. National Statement of Patients’ Rights in Primary Health Care Developed (February 2012) PHCPI Training as of March 31, 2013 (Information Provided by PHCPI) Event Name Number of Participants from PHCPI's 360 Target Clinics Number of Participants from Outside PHCPI's 360 Target Clinics7 Total Number of Participants Number of PHCPI target clinics reached Management Roll-out of Facility and Equipment Maintenance Management - SOP 841 335 1,176 327 Roll-out of Infection Prevention and Waste Management 879 124 1,003 322 Roll-out of Leadership and Management Program (5-day training) 488 198 686 300 Roll-out of Medical Records System 1,426 293 1,719 279 Roll-out of PHC Management Handbook 628 168 796 319 Totals for main training 4,262 1,118 5,380 TOT on Facility and Equipment Maintenance Management - SOP 15 60 75 13 TOT on Infection Prevention and Waste Management 37 107 144 27 TOT on Leadership and Management Program 35 60 95 34 7 Includes district, DOH, and central-level MOH staff. MID-TERM PERFORMANCE EVALUATION OF USAID/IRAQ PRIMARY HEALTH CARE PROJECT 47 Project Documents Produced for Training and Research TOT on Medical Records System 67 83 150 40 TOT on PHC Management Handbook 17 53 70 16 ToT on Referral System 3 32 35 3 Totals for training of trainers 174 395 569 Clinical Roll-out of CDC 126 34 160 118 Roll-out of IMCI for Nurses 395 72 467 258 Roll-out of NCD (5-day training) 313 70 383 224 Roll-out of Poly Trauma 196 41 237 139 Totals for main training 1,030 217 1,247 TOT on CDC 3 14 17 3 TOT on Emergency Obstetric and Newborn Care (EmONC) 1 14 15 1 TOT on Emergency Obstetrics Care 0 13 13 0 TOT on IMCI for Nurses 32 55 87 23 TOT on IMCI for Physicians 3 23 26 3 TOT on NCD 2 47 49 2 TOT on Newborn Care 1 13 14 1 TOT on Poly Trauma 1 18 19 1 Community Roll-out of Community Health Partnership (CHP) 1,390 333 1,723 356 TOT on CHP 9 175 184 8 PHCPI Training as of March 31, 2013 (Information Provided by PHCPI) Event Name Number of Participants from PHCPI's 360 Target Clinics Number of Participants from Outside PHCPI's 360 Target Clinics8 Total Number of Participants Number of PHCPI target clinics reached Management Roll-out of Facility and Equipment Maintenance Management - SOP 841 335 1,176 327 Roll-out of Infection Prevention and Waste Management 879 124 1,003 322 Roll-out of Leadership and Management Program (5 day training) 488 198 686 300 Roll-out of Medical Records System 1,426 293 1,719 279 Roll-out of PHC Management Handbook 628 168 796 319 Totals for main training 4,262 1,118 5,380 TOT on Facility and Equipment 15 60 75 13 8 Includes district, DOH, and central-level MOH staff. MID-TERM PERFORMANCE EVALUATION OF USAID/IRAQ PRIMARY HEALTH CARE PROJECT 48 Project Documents Produced for Training and Research Maintenance Management - SOP TOT on Infection Prevention and Waste Management 37 107 144 27 TOT on Leadership and Management Program 35 60 95 34 TOT on Medical Records System 67 83 150 40 TOT on PHC Management Handbook 17 53 70 16 ToT on Referral System 3 32 35 3 Totals for training of trainers 174 395 569 Clinical Roll-out of CDC 126 34 160 118 Roll-out of IMCI for Nurses 395 72 467 258 Roll-out of NCD (5-day training) 313 70 383 224 Roll-out of Poly Trauma 196 41 237 139 Totals for main training 1,030 217 1,247 ToT on CDC 3 14 17 3 TOT on Emergency Obstetric and Newborn Care (EmONC) 1 14 15 1 TOT on Emergency Obstetrics Care 0 13 13 0 TOT on IMCI for Nurses 32 55 87 23 TOT on IMCI for Physicians 3 23 26 3 TOT on NCD 2 47 49 2 TOT on Newborn Care 1 13 14 1 TOT on Poly Trauma 1 18 19 1 Totals for training of trainers 43 197 240 Community Roll-out of Community Health Partnership (CHP) 1,390 333 1,723 356 TOT on CHP 9 175 184 8 Summary of training supported by PHCPI, by category of training and trainee Training category Trainee category Target clinics TOT involved * Target PHCC staff District/Province/ MoH Management 569 4,262 1,118 279–327 Clinical services 240 1,030 217 118–258 Community 284 1,390 333 356 *Number varies depending on topic MID-TERM PERFORMANCE EVALUATION OF USAID/IRAQ PRIMARY HEALTH CARE PROJECT 49 27% 17% 26% 16% 30% 63% 78% 74% 80% 61% 10% 4% 4% 9% 0% 25% 50% 75% 100% Leadership & Management (n=41) PHC Management (n=23) Maintenance Management at PHC (n=19) Community Partnerships (n=25) Infection prevention (n=33) Percent of interviewed staff who received training Perception of training course Excellent Good Fair MID-TERM PERFORMANCE EVALUATION OF USAID/IRAQ PRIMARY HEALTH CARE PROJECT 50 Annex 7. Behavior Change Communications PHCPI produced a Behavioral Change Communications (BCC) Strategy in 2013 as a guideline. The project also produced a National BCC Strategy to Promote Primary Health Care in Iraq, referred to in this report as the “national campaign.” The focus chosen for the national campaign is to “advocate for desired behaviors for immunization and other MCH and reproductive health issues.” Both documents were developed based on situational and needs assessment conducted with both MoH relevant departments at various levels as well as with community key informants and program beneficiaries, including IDPs. Both strategies use a variety of communication approaches and channel mix including interpersonal communication, counseling, community mobilization, and national and local media. The BCC Strategy defined its target audiences based on groups identified by MoH, WHO, and USAID as having particularly urgent health burdens and barriers to health care access. For the BCC Strategy, these primary target audiences included IDPs. Under the campaign strategy, contact with various media—e.g., TV, radio, and mobile phones—should be analyzed, access of the urban population should be reported, and coverage of remote and rural areas should be noted. However, the strategy fails to address the issue of identifying accessibility to such media channels among IDP communities. Furthermore, the campaign strategy identifies the IDPs only as a secondary target group, as “females within internally displaced populations.” The primary target group for the Phase 1 campaign, “18-44 year old married women residing in urban, peri-urban, and rural areas of Iraq,” should include women living in IDP settlements as well. Although, PHCPI indicated that it realizes that IDP married women 18-44 are a primary audience, this fact should always be explicitly highlighted so the campaign does not accidently miss an opportunity to design BCC materials, spots, and community events that are relevant to this vulnerable group. Another concern is that part of the campaign is designed to use national media, which will reach communities that frequent all of Iraq’s PHCCs nationwide, while the project implements quality improvement in only 360 PHCCs. Since PHCC utilization for immunization and MCH preventive services would be promoted under the campaign, every effort should be made to ensure that beneficiaries will find that all services promoted are actually available and offered according to the advertised quality. This is vital; otherwise, instead of the BCC campaign having a positive effect on promoting service use, it may have negative implications in terms of the image of both the Iraqi MoH and USAID. For this reason, two things should be observed in planning and producing materials for the campaign. First, for PHCPI PHCCs to participate in the campaign, they need to comply with quality of care standards and observe the Patients’ Rights Charter. Second, the project should create a branding and logo for PHCPI PHCCs that conforms to quality standards and the Patients’ Rights Charter. Participating PHCCs and campaign spots and materials should use the same branding and logo that has been created to indicate good quality. This branding will also serve the purpose of distinguishing quality PHCPI PHCCs from other non-PHCPI PHCCs. This is vital, or otherwise beneficiaries using MID-TERM PERFORMANCE EVALUATION OF USAID/IRAQ PRIMARY HEALTH CARE PROJECT 51 MCH services at non-PHCPI PHCCs who find that the promised quality is unavailable, would view the MoH and USAID as creating false expectations. Therefore, the quality of care provided by PHCCs participating in the campaign must be ensured, especially for MCH services. Hence, campaign activities should be coordinated and synchronized with PHCPI Component 2 activities. Based on current evaluation findings with respect to quality of MCH services offered at PHCPI PHCCs, it would be best to implement the campaign in stages, focusing on meeting quality standards for priority services (e.g., immunization and IMCI, since both target child health and mortality). For the campaign to attain its objective, Component 2 and Component 3 staff must work together to develop a system for identifying PHCCs that meet the campaign’s branding and logo criteria and periodically recertify their status; in addition, they need to agree on the key quality elements to be communicated through campaign channels. Maternal services are to be addressed in the campaign’s second stage. This stage should not start until revised guidelines for antenatal and postnatal care have been produced and implemented in all PHCCs; in addition, required equipment and medication as well as trained providers must be available in participating PHCPI PHCCs. In preparation for the campaign’s first planned stage, PHCPI has conducted two workshops in Baghdad and Erbil to strengthen capacity for conducting BCC campaigns, including campaign planning, writing contracts, and determining roles and responsibilities of different parties. Part of these workshops was dedicated to building MoH capacity in interpersonal communication skills and production of BCC materials. In Erbil and Baghdad, PHCPI recently launched the first public-private partnership campaign in Iraq with Unilever. The campaign promotes healthy behavior for dental care to women of reproductive age and children aged 6–12. The campaign takes place in all primary schools and PHCCs in seven provinces in the north and seven provinces in the south. Such a campaign will bring many more children and their mothers to PHCC dental clinics. This will probably result in increased use of dental services and therefore an increased PHCC need for dental consumables and medication. PHCPI should advise MoH ahead of time regarding this expected rise in needed resources at the PHCCs. This would enable the campaign to be more effective in promoting the dental health of the target group. Both the BCC Strategy and the national campaign articulate a monitoring and evaluation (M&E) strategy. A national-level knowledge, attitude and behavior (KAB) survey is planned, which will be multi-staged so that behavioral changes and outcome indicators from the campaign can be assessed. Behaviors are to be measured through intentions as well as through increased rates of pre- and post-natal screening, immunizations and other preventive behavior. The campaign M&E strategy is strong in that it will measure outcomes through community-based surveys conducted pre- and post-campaign. However, measurement should not be carried out through a national survey, even though the campaign would use mass media with a nationwide reach. This is because local campaign activities and quality supply services would only be for PHCCs participating in PHCPI. Thus the pre- and post-KAB surveys should be conducted with a representative sample of participating local communities. MID-TERM PERFORMANCE EVALUATION OF USAID/IRAQ PRIMARY HEALTH CARE PROJECT 52 The BCC Strategy impact evaluation strategy states that it would take place through “facility-based surveys: To link exposure to BCC campaigns and messages to service utilization. To assess knowledge, attitudes, and behavior regarding PHC; change in access to PHC services and to measure client satisfaction.” This would provide a biased measure of outcomes of the BCC campaigns, as health facility service users are more likely than community non-users of services to know about services and to have positive attitudes toward such services. If for security reasons, community-based surveys are impossible to conduct in Iraq, then the minimum outcome indicators for BCC activities that the project could track would relate to changes in utilization statistics per capita for the promoted services. Records should be set pre-campaign to make a baseline measure, as well as at the end of the campaign. It would be a good practice for PHCPI and USAID to engage the USAID/Iraq Performance Management Specialist to review and agree, ahead of time, on the M&E plan for any BCC activities as well as for survey instruments before their implementation. MID-TERM PERFORMANCE EVALUATION OF USAID/IRAQ PRIMARY HEALTH CARE PROJECT 53 Annex 8. Additional Evidence from the Evaluation to Support Conclusions Following is selected information provided by PHCPI and gathered for this evaluation from a sample survey of 26 PHCCs and 18 affiliated districts. The sample of facilities and health workers was biased to favor those who had received additional attention during PHCPI implementation (see Methods section). While the results may not representative, they do provide evidence of current service conditions and perceptions among staff and district managers, which can be interpreted as being biased toward more positive findings. 1. Information supporting need to consider alternate service delivery models if expanding access to services and decreasing the caseload on hospitals is an objective. Table 8-1: Physician Staffing Levels for PHCPI Target Facilities9 Type of Facility Number of PHCPI Facilities Physician Staff Assigned No Physician Staff One Physician Assigned Main PHCC 266 266 0 23 Sub-clinic 94 7 87 7 A consistent complaint from all levels of MoH and PHCCs assessed was a lack of staff, and of physicians in particular. With almost all sub-clinics (26% of facilities) having no physician and an additional 9% of facilities having only one physician, it is evident that improving access to qualified staff and improving the level of PHC service provision over the next few years will require expanding the services that can be provided by non￾physicians. Relying on referrals may not be practical, particularly for routine follow-up for chronic illnesses. Anecdotal reports consistently provided examples of the difficulties of a dependence on referrals. These include not only financial and logistics constraints, but the major problem of security. International evidence exists to show that within limits and strict guidelines, non￾physicians can safely provide services normally provided by physicians, to improve critical service access and use, with an overall improved population outcome. Advocacy to expand the level of services provided by non-physicians should be undertaken, since the alternative is the population not receiving needed services. A. Inefficient use of trained providers Given the reported shortages of physicians, medical assistants, and nurses, ensuring that these trained personnel are used efficiently and avoid tasks that do not require their training (particularly clerical or other routine administration activities that could be done by less qualified persons who are more widely available) is of critical importance. Among the 77 interviewed doctors/medical assistants/nurses, 21 (27%) reported they provide no clinical services. Those providing clinical services may have also had administration and management responsibilities, but this information was not consistently captured. 9 Information on numbers of physicians per facility provided by PHCPI. MID-TERM PERFORMANCE EVALUATION OF USAID/IRAQ PRIMARY HEALTH CARE PROJECT 54 This provides evidence of areas for which PHCPI research could be useful in testing alternative staff management and service organization systems based on job and patient-flow analyses, with the objective of demonstrating different models that result in more effective service delivery within existing constraints. There were also numerous anecdotal complaints about trained staff, particularly physicians, moving from PHCCs back to hospitals. Among interviewed staff, 9 of the 39 physicians (23%) had worked in the PHCC for less than one year. The majority of doctors, paramedics, and other staff had worked in the PHCC for more than two years, confirming that doctors are more likely to be short-term at the facility. This is an issue to consider when trying to strengthen PHCC services. Interviewees from three of the districts visited by the evaluation team discussed the need for incentives for doctors to remain in the PHC system. Suggestions included salary supplements (agreed in the past for some specialty doctors—for example, forensics, anesthesiology, psychiatry). Another potential initiative would be to develop the specialty of family medicine, raising the prestige of doctors working in PHCCs. Other non-monetary incentives might include access to training and an increase in the level at which they are allowed to function in PHCCs (currently physicians are required to refer cases such as severe child pneumonia, uncomplicated hypertension, and diabetes that may require more than a first-line treatment) commonly treated by general practitioners in other countries. Absence of a Basic Package of Key PHC Services at Facilities 33 23 1 1 5 16 5 6 2 8 7 1 0 20 40 60 Doctor Paramedic lab/pharmacy qualification non-technical Health Information… Administration Maintenance Manager qualification… Number of interviewed staff Interviewed Clinical Service Providers Yes clinical services No clinical services 9 3 1 4 3 1 0 1 26 32 5 6 2 7 7 2 2 0 20 40 60 Doctor Paramedic lab/pharmacy… non-technical Health Information… Administration Maintenance Manager… Dentist Interviewed PHCC staff Qualification/Job of Interviewed Staff and How Long Worked in PHCC < 1 year 12-23m 2+ years MID-TERM PERFORMANCE EVALUATION OF USAID/IRAQ PRIMARY HEALTH CARE PROJECT 55 Among the 24 sample facilities, immunization was universally offered both in the facility and through outreach. The facilities offer curative child and adult services through outreach as well as in the facility. Outreach is used periodically for limited services for very underserved areas, or in one case for hepatitis and tuberculosis cases. Key services that are linked to maternal and child health were not routinely offered with the PHCC package of services. These included family planning (available in 9 of 24 sample facilities) and deliveries (2 of 24). The facilities that did not offer antenatal care (ANC) reported they did not have a female service provider. Feedback from district and PHCC staff on PHCPI supported trainings, system interventions, and changes over the past two years. Interviewed staff who had received different management trainings thought the courses were good and the training practical, for the most part, with those thinking the training was difficult to implement ranging from 12– 39%, depending on the course. During interviews, district-level staff indicated that the guidelines and planned interventions for the new record system and waste management were important to PHCC management. The most common comment on the new record systems was that time would be saved using fewer forms and that the information will be more readily available for use when the new forms are used. 18 13 9 9 20 1 0 6 11 9 2 1 24 15 2 22 0 0 4 8 12 16 20 24 Curative_adult Curative_child Emergency Trauma Family Planning Antenatal care Delivery services Child immunization Number of sample facilities Services Offered Facility Both facility and outreach Not offered MID-TERM PERFORMANCE EVALUATION OF USAID/IRAQ PRIMARY HEALTH CARE PROJECT 56 Comments on waste management were that they got supplies (colored plastic bags for waste, sharps boxes) which helped them make changes to adhere to the protocols for contaminated/sharps waste management in the facility. Among interviewed staff in the 24 sample facilities, almost all reported that service conditions and supervision support have improved over the past two years. These are topics that have been addressed in PHCPI management training; however, the systems for improving these areas have only recently been addressed by PHCPI. Systems already existed within the MoH. The evaluation team’s assessment is that the improvements in MoH support to PHCCs may be indirectly attributed to PHCPI. 2. Readiness to provide services Training in management topics was shown to be widespread, with manuals and guidelines related to the training present in 14 out of 24 facilities. Findings were similar for clinical service guidelines and training. 27% 17% 26% 16% 30% 63% 78% 74% 80% 61% 10% 4% 4% 9% 0% 25% 50% 75%100% Leadership & Management (n=41) PHC Management (n=23) Maintenance Management at… Community Partnerships (n=25) Infection prevention (n=33) Percent of interviewed staff who received training Perception of Training Course Excellent Good Fair 20 17 18 23 9 9 0 4 8 12 16 20 24 Medicines Equipment Facility/service… Supervision/support Equipment repair Building repair Number of sample facilities Faciliites Where Interviewed Staff Report Improved Management Practices 14 5 3 4 3 0 3 6 9 12 15 18 New records Waste management Community partnerships Management Maintenance Number of sample districts Districts Reporting Indicated Training Is Being Put into Practice/Is Valuable MID-TERM PERFORMANCE EVALUATION OF USAID/IRAQ PRIMARY HEALTH CARE PROJECT 57 21 19 24 21 23 19 13 0 4 8 12 16 20 24 Running water Soap or Hand… Disposable latex gloves Medical waste container Sharps container Environmental… All items Number of sample facilities Infection Prevention in Service Area 23 21 21 16 23 14 0 4 8 12 16 20 24 Hemoglobin/hematocrit Blood glucose urine glucose Blood chemistries General microscopy All tests Number of sample facilities Basic Diagnostic Capacity 6 1 11 1 5 1 0 10 20 Incinerator Open burning Remove off site Number of sample facilities Medical and Sharps Waste Disposal Practices Protected Not protected 14 16 12 6 5 4 4 3 7 0 10 20 PHC Management Infection Prevention and waste management Maintenance and Repair Number of sample facilities Training and Guidelines Verified Training reported and guidelines observed Training reported, guideline not observed No training on subject 16 16 16 3 4 0 5 4 8 0 Communicable Diseases Non￾communicable Diseases IMCI Number of sample facilities Training and Guidelines Verified Training reported and guidelines observed Training reported, guideline not observed No training on subject MID-TERM PERFORMANCE EVALUATION OF USAID/IRAQ PRIMARY HEALTH CARE PROJECT 58 Districts reported that there were changes in medical and sharp waste disposal practices after the training. Among the facilities without an incinerator, anecdotal reports were received of new contracts for material removal offsite, awareness of the problem of open burning (including information from one visited facility), and discussions with local authorities for material removal. Facilities reported that the sharps and medical waste containers at the service sites were new. Although most items for infection prevention at the service sites were available, the complete package of needed items was only available in a little more than half (13 of 24) of the facilities. Availability of select elements that are currently part of the PHC service package and that are required for adherence to clinical guidelines were assessed for an indication of whether or not providers could follow the guidelines. The availability of indicated items on the day of the survey was assessed for each of the following services: • General client examination: adult and infant scales, thermometer, light source for examining throat • General client treatment: intravenous for plasma expansion (dextrose 5% normal saline, normal saline, Ringers lactate) and for administering medicines (dextrose 5% and water), paracetamol In addition, the following items were assessed for specific services and guidelines: • ANC: ability to measure hemoglobin, urine glucose, blood pressure apparatus, adult scale, folic acid • IMCI: infant scales, thermometer, otoscope, general microscopy capacity, oxygen, 1st line treatment for pneumonia (cotrim or amoxicillin), the Iraq IMCI 2nd line antibiotic for childhood pneumonia (cefotaxime is not on the essential drug list for PHCCs) vitamin A, treatment for diarrhea (ORS and zinc), albendazole or mebendazole • Hypertension: blood pressure apparatus, ophthalmoscope, test blood chemistries, hydrochlorothiazide, and atenolol or captopril • Diabetes: ability to measure blood glucose, adult weight, glibenclamide or metformin • Asthma: oxygen and delivery apparatus, spacer for inhaler, salbutamol It can be seen that across the sample PHCCs, different items are lacking to provide quality services and to adhere to guidelines for which refresher training was provided 24 22 23 24 24 21 14 13 11 21 5 0 4 8 12 16 20 24 Adult scale Infant scale Thermometer Stethoscope Blood pressure apparatus Light source Otoscope Opthalmoscope Oxygen tank and… Spacer for inhaler All equipment Number of sample facilities Basic Client Exam and Treatment Equipment MID-TERM PERFORMANCE EVALUATION OF USAID/IRAQ PRIMARY HEALTH CARE PROJECT 59 1 14 5 13 0 4 8 12 16 20 24 All meds All lab diagnostics All equipment All infection prevention Number of sample facilities All Items for Basic Diagnosis, Treatment for IMCI, Hypertension, Diabetis, ANC 24 24 4 20 21 23 16 13 15 19 24 17 16 1 0 4 8 12 16 20 24 1st line pneumonia (child) Oral rehydration salts Zinc Tablets Vitamin-A capsules Albendazole/mebendazole Folic acid Betablocker for hypertension (atenolol/captopril) Diuretic for hypertension (hydrochlorthiazide) 1st line diabetes treatment (Glibenclamide,… Salbutamol for inhaler (asthma) Paracetamol Intravenous plasma expander (NS or RL) Intravenous for administering medicines (D5W) All meds (13) Number of sample facilities Basic Medicines for PHC Treatment Guidelines MID-TERM PERFORMANCE EVALUATION OF USAID/IRAQ PRIMARY HEALTH CARE PROJECT 60 Annex 9. Local Health Committee Members Interview Analysis PHCPI reported that it had formed Local Health Committees (LHCs) in almost all 360 PHCs. The evaluation team interviewed LHC members of 24 PHCs. The evaluation aimed at interviewing one committee member representing the PHC and two members representing the community. In fact, 66 LHC members were interviewed: 26 (39%) from the PHCs and 40 (61%) from the community. Only 17% of the interviewees were female. The LHCs were recently formed, with the median period since formation being 4 to 5 months. Eighty-three percent of LHCs had government clerks, including teachers, as members, 23 % of LHCs had membership from NGO and civil society organizations, 21% had an elected Local Council member, 12% a mokhtar, and 8% a mosque sheikh. Three LHCs had only teachers as community members, which implied that the committee was concerned with school health and not the community at large, as is expected of an LHC. The most common number of LHC members was seven members (47%). Only 50% of LHC members reported having at least one female community committee member. Only 68% of LHC members reported that the LHCs met regularly. Specifically 8 out of the 24 LHCs (33.3%) reported not meeting (5) or not active (3). Four of these eight LHCs had been formed for more than three months. Members of these committees still reported not understanding the role of the LHC. In an attempt to overcome this, PHCPI produced operational guidelines in addition to the previously produced Community Health Partnership handbook and training manuals. However, there are no specific TORs that can guide the work of the LHCs. Interviewers asking for LHC members by name according to PHCPI-scanned letters of LHC formation indicated that a few had re-formed with different membership. As a result, interviews with LHC members showed that only half were trained in the Community Health Partnership Handbook; one-quarter received training on other topics. The training was found to be relevant and 88% reported using the concepts. Some expressed that the training was useful and practical. A few viewed the training as short and needed a longer training period to better understand concepts. They understood how to communicate with people and pass information through the use of posters. A few reported that communication and cooperation had improved as a result. Community member communication and cooperation is now good or excellent in 84% of LHCs. PHC member communication and cooperation is good or excellent in 95% of LHCs. The opinions of LHC community members are taken seriously: always 60% and only sometimes 28%. This is because community LHC members are close to the community so their views represent community needs. Also, because they are respected by the community, they are more likely to convince the community on health-related issues, facilitating the work of PHCs. Challenges facing LHC members from the PHC sector are varied and include: lack of transportation, especially to reach remote areas (reported by 12 or 19%); lack of time (reported by 8); shortage of PHC staff; difficulty with meetings including arranging meetings (reported by 4); lack of attendance at meetings attended only by community MID-TERM PERFORMANCE EVALUATION OF USAID/IRAQ PRIMARY HEALTH CARE PROJECT 61 members; lack of a system or structure for conducting meetings; lack of financial and moral support (reported by 4); lack of evening security guards; and lack of training. Challenges facing LHC members from the community are varied and include: lack of time (13 or 20%); difficulty with meetings including arranging meetings and attendance, especially for government staff; lack of financial and moral support (reported by 12 or 19%); and lack of transportation, especially to reach remote areas where there is a lack of roads (reported by 5). Despite these challenges, LHCs have discussed various aspects related to the health of the community. For example: 38% quality of services; 26% planning of service, especially immunization campaigns; 15% health related infra-structure, e.g., water purification, waste disposal, and pest control; and 11% community outreach activities. Topics Discussed Frequency Percent Quality of services 25 37.9 Times services are offered 16 24.2 Planning of services 17 25.8 Implementing services 8 12.1 Community outreach activities 7 10.6 Facilitating PHC requests from government 3 4.5 Inter-sectoral coordination 6 9.1 Feedback from community on services 2 3.0 Health-related infrastructure 10 15.2 66 100.0 In particular, functioning LHCs reported discussing and implementing various important activities mostly related to routine immunization; epidemics; removal of garbage and medical waste; water supply and chlorination; raising community awareness on various issues, including production of IEC materials about violence against women; patients’ right to good, dignified treatment; expansion of the PHCC by adding a delivery room; provision of ambulances; and need for more doctors. Some examples of achievements worthy of special notice include: coordination with local police to control sale of expired food; medical teams of 2 PHCCs sent to provide remote areas with curative and preventive care; meeting with IDPs to identify their needs; announcement of the national immunization campaign on the radio; provision of continuous electricity at 1 PHCC through use of a neighbor’s generator; provision of an ambulance; recruitment of additional physicians; increase in supply of medication based on use. MID-TERM PERFORMANCE EVALUATION OF USAID/IRAQ PRIMARY HEALTH CARE PROJECT 62 Annex 10. Suggestions on Contract Deliverables As currently written, the PHCPI Contract focuses on training and document development. This focus is perceived by the evaluation team, at least two MoH key informants, and other interviewed stakeholders as being at the expense of achieving on-the-ground change. The contract also separates different activities that are related in implementation and should not be carried out vertically (e.g., supportive supervision, monitoring for compliance against standards, quality improvement activities). The following table presents suggestions for revising the deliverables and measures for management and clinical services to shift the project focus to achievements rather than processes. MID-TERM PERFORMANCE EVALUATION OF USAID/IRAQ PRIMARY HEALTH CARE PROJECT 63 Annex 10 Suggestions on Contract Deliverables Recommendation Deliverable Verifiable Indicator Comments/Suggestions TO REPLACE ALL DELIVERABLES SHADED BELOW NEW DELIVERABLE 1 Institutional capacity within the MoH to implement training to improve PHC services and service management. Either there is evidence of MoH carrying out the following training functions without PHCPI inputs or there is evidence of an assessment by PHCPI of further need for capacity building within the MoH and of strategies showing strengthening of MoH capacity to effectively carry out the following training functions: 1) Develop a master training plan for an effective PHC system and quality services 2) Develop training modules where these do not exist or revise current training modules where needed 3) Assess number of trainers needed and qualifications; conduct TOT where needed 4) Implement training as per master training plan using methods supported during Y1 and Y2 during PHCPI training 5) Implement follow-up for district and PHCC trainees in their service sites Comment: This will shift the PHCPI focus from training to developing strategies for the MoH to assume good training practices that will support quality PHC services. The PHCPI role would shift to working with MoH staff who are responsible for planning and implementing training related to management issues. If the MoH does not follow through, the report can document the steps the project took to help the MoH assume more responsibility (e.g., working with them on master training plans, training follow-up, assessing the needs for trainers, etc.) Drop Deliverable 1.2b Institutional capacity within the MoH to implement the Management Handbook developed. Report on number of MoH trainers trained by end of Year 4. This will push the project to continue focusing on TOT courses at the expense of supporting change at PHCC and district level. To date, 70 MoH trainers have been trained on the PHC Management Handbook. Either accept these as sufficient for meeting the definition for having built institutional capacity or roll deliverable into new deliverable 1. Drop Deliverable 1.2c Provide training (directly or through MoH TOTs) on the Number of participating clinics from which at least one staff person has been trained on the Management This will push the project to continue focusing on training PHCC staff at the expense of supporting change at PHCC level. To date, 628 PHCC staff from 319 PHCPI clinics MID-TERM PERFORMANCE EVALUATION OF USAID/IRAQ PRIMARY HEALTH CARE PROJECT 64 Recommendation Deliverable Verifiable Indicator Comments/Suggestions Management Handbook for personnel from a minimum of 360 participating clinics. Handbook. plus 168 MoH staff have been trained on the PHC Management Handbook. Either these should be accepted as sufficient numbers for training to meet the definition, or roll deliverable into new deliverable 1. Drop Deliverable 1.2c1 Technical assistance and training on standard operating procedures (SOPs) for 7 key management functions delineated in the Management Handbook. (35% or 126 participating clinics in Year 2, 55% in Year 3 and 75% in Year 4). Number of participating clinics from which relevant staff (at least two) has been trained on the facility and equipment management SOPs. This will push the project to continue focusing on training PHCC staff at the expense of supporting change at the PHCC level. To date, 628 PHCC staff from 319 PHCPI clinics plus 168 MoH staff have been trained on the PHC Management Handbook. Either these should be accepted as sufficient numbers for training to meet the definition or roll deliverable into new deliverable 1. Another option is to recommend removing as deliverable and make the indicator focus on achieving 1 or 2 key activities, e.g., Percent of target facilities with the following key maintenance and management indicators: • Good waste management for sharps and medical waste in all 360 facilities • Preventive maintenance for select equipment (generators, others TBD) • Facility cleaning schedules and system for monitoring quality Revise verifiable indicator Deliverable 1.2d Put in place effective process/system to achieve and measure compliance with quality standards for 7 key management standards (baseline measure of compliance in Year 2, 35% compliance in Year 3, 75%. Report on number of participating clinics in compliance with standards Recommend replace with “Report on number of PHCCs with evidence of carrying out self-assessment of compliance measurements for management with written evidence of identifying priority areas for improvement.” Or Report on PHCPI districts conducting an assessment of compliance measurements for management, with the report indicating the percent of PHCCs in the district that were assessed, the frequency of the The system to measure management practices is important for long-term sustained change. Achieving the standards (in any manner where they can be reasonably expected to be continued past the project) is a long-term process and may be beyond the ability of the project. Suggest revising the evidence of effective process. Instead of percentage achievement of compliance, the deliverable could \ be “change from baseline score” using all compliance measures, or better, with the MoH/TAG input, prioritizing the management quality standards from among the 292 measurable indicators to a smaller number of the most important ones. MID-TERM PERFORMANCE EVALUATION OF USAID/IRAQ PRIMARY HEALTH CARE PROJECT 65 Recommendation Deliverable Verifiable Indicator Comments/Suggestions assessments and any evidence of results and follow up. Drop Deliverable 1.3b Institutional capacity within the MoH to implement the PHC Leadership and Management Training Program developed. Report on number of MoH trainers trained by end of Year 4. This will push the project to continue focusing on training PHCC staff at the expense of supporting change at the PHCC level. To date, 628 PHCC staff from 319 PHCPI clinics plus 168 MoH staff have been trained on the PHC Management Handbook. Either these should be accepted as sufficient numbers for training to meet the definition, or roll deliverable into new deliverable 1. Drop Deliverable 1.3c Provide training (directly or through MoH TOTs) to at least two leaders/managers from each of the 360 participating clinics and to at least 5 provincial level MoH leaders/managers from each of the 18 provinces. At least 2 managers from participating clinics, districts, and provinces successfully completing the PHC Leadership and Management Training Program. This will push the project to continue focusing on training PHCC staff at the expense of supporting change at PHCC level. To date, 628 PHCC staff from 319 PHCPI clinics plus 168 MoH staff have been trained on the PHC Management Handbook. Either these should be accepted as sufficient numbers for training to meet the definition, or roll deliverable into new deliverable 1. Drop Institutional capacity within the MoH to implement the Policies and Procedures for Establishing National PHC Standards of Care developed. Policy formulation and clinical standard setting working group formalized, trained and functional to regularly disseminate, monitor, and enforce compliance with the policy and quality standards for 20 clinical guidelines in the assisted clinics by the end of Year 4. Change implementation responsibility Deliverable 2.1c Twenty Primary Health Care Clinical Standards/ Protocols developed/updated and tested (7 in Year 1, 8 in Year 2, and 5 in Year 3). Seven clinical protocols/standards updated/revised. MID-TERM PERFORMANCE EVALUATION OF USAID/IRAQ PRIMARY HEALTH CARE PROJECT 66 Recommendation Deliverable Verifiable Indicator Comments/Suggestions Revise handbook so that it meets stated objective Deliverable 2.2a Handbook of Quality Standards and Operational Guidelines for Clinical Services Delivery in Primary Care Clinics developed. Draft handbook completed. Recommendation: Revise draft as per comments/suggestions. The evaluation team believes the current draft document does not achieve the objective: 1) It does not cover key PHC services including, ANC, PNC, FP, IMCI, communicable diseases. It only covers some of the topics that PHCPI trained on and still includes items in the guidelines that are neither available nor planned for the PHC level. There are errors in the equipment processing information. Initial decontamination of equipment is not included; instead it refers to soaking in soapy water to facilitate cleaning. Recommendation: 1. Rework this document so that it includes all priority PHC services. (Priority PHC services may be identified in consultation with the TAG and advocated for MoH concurrence.) 2. Make the treatment guidelines in this document job aids that refer only to items that are expected to be in all PHCs within the next 1-2 years, that provide clear indications for referral, 3. Remove all but the most key background/educational information about a condition. This is already in the guidelines for those conditions and does not belong in a manual meant to “Support a streamlined and effective system which provides up-to-date, clear, useful clinical practice guidelines to primary health care providers.” 4. If possible for each condition provide a job aid for the non-physician who is managing the case. 5. Remove Chapters 2-4 or revise to “key points to remember.” Drop Deliverable 2.2a.1 Training modules covering Handbook of Quality Standards and Operational Guidelines for Clinical Services Delivery in Primary Care Clinics developed. First set of training modules and first draft of handbook completed. MID-TERM PERFORMANCE EVALUATION OF USAID/IRAQ PRIMARY HEALTH CARE PROJECT 67 Recommendation Deliverable Verifiable Indicator Comments/Suggestions This document should be a functional job-aid for topics in which PHC staff have already been trained. An orientation when the manual is distributed may be helpful but training seems redundant. Revise verifiable indicator as per 1.2d Deliverable 2.2d In partnership with the MoH, put an effective process/system in place to achieve and measure 75% compliance among participating clinics with quality standards for 7 key clinical services in the MoH’s basic health service package for primary health care. Report on number of participating clinics in compliance with quality standards related to the 7 key clinical service guidelines developed in Year 1. See suggestions for Deliverable 1.2d. Change verifiable indicator Deliverable 2.2e In partnership with the MoH, put an effective provincial and clinical level supervision process/system in place for 75% of participating clinics according to quality standards in the Clinical Service Delivery Handbook. Report on number of facilities implementing supportive supervision. Change: Report on PHCCs implementing the QI process and supportive supervision activities and report in districts, conducting supportive supervision that includes assessing quality improvement activities of the PHCCs in their districts. It is important that the supervision and assessments against standards also be linked with the QI process and that these not only be facility-level activities, but that the facility￾level process also be supported by district supervision process. Drop Deliverable 2.3b Institutional capacity developed within the MoH to implement the PHC QI Program for management, clinical and community participation issues. PHC department within the MoH equipped with trained capacity/skilled professionals to develop QI standards, QI plans and strategies, QI tools and compliance measurement tools by the end of Year 4. Roll this into Deliverable 2.2e Drop Deliverable 2.4b Institutional capacity within the MoH to implement the Report on number of MoH trainers trained. This seems duplicative of earlier deliverables. MID-TERM PERFORMANCE EVALUATION OF USAID/IRAQ PRIMARY HEALTH CARE PROJECT 68 Recommendation Deliverable Verifiable Indicator Comments/Suggestions PHC In-Service Training Program developed. Drop Deliverable 2.4c Provide training in quality standards and clinical protocols (directly or through MoH ToTs) in 5 or more of the 7 key clinical services for a minimum of 75% of relevant clinical staff. Report on number of relevant participating clinics with at least one staff member trained in IMCI, IPC, EMONC, NCD management, trauma, and other clinical services guidelines developed under Deliverable 2.1c. This will push the project to continue focusing on training PHCC staff at the expense of supporting change at PHCC level. To date, 1,030 contacts from an average of 185 PHCCs plus 217 MoH staff have been trained on different clinical guidelines. Either these should be accepted as sufficient numbers for training to meet the definition, or roll deliverable into new deliverable 1. Drop Deliverable 2.4d Provide training in supportive supervision (directly or through MoH ToTs) for a minimum of 75% of clinical and provincial level MoH staff who have supervisory duties. Percent clinics with at least 1 staff member trained in supervision. Roll this into Deliverable 2.2e. Drop Deliverable 2.4e Provide training in referrals (directly or through MoH ToTs) for relevant staff from a minimum of 360 participating clinics. Percent clinics with at least 1 staff member trained in referrals. This will push the project to continue focusing on training PHCC staff at the expense of supporting change at PHCC level. TOT for referral system has just begun Y3. Roll deliverable into new deliverable 1. Drop Deliverable 2.4f Provide training in quality improvement (directly or through MoH TOTs) for the QI team at a minimum of 360 participating clinics. Number of clinics with at least 1 QI team trained. This will push the project to continue focusing on training PHCC staff at the expense of supporting change at PHCC level. TOT for quality improvement system has just begun Y3. Roll deliverable into new deliverable 1 and PHCP to focus on supporting the actual implementation (Deliverable 2.2e). Focus studies as per recommendations Deliverable 2.5b One study evaluating effectiveness of innovative models for primary care service delivery in Iraq completed and disseminated (total of 3 for life of project). One of 3 studies completed (Maysan health visitor and e-health program). Note: The report is described as Operations Research. The evaluation team finds this is a study, not operations research. Recommend two studies focused on improving PHC services within existing constraints within the model clinics. Topics might be improvements to patient flow, more efficient use of trained service providers, and effective case detection, treatment, and long-term monitoring of chronic conditions of uncomplicated diabetes and hypertension. MID-TERM PERFORMANCE EVALUATION OF USAID/IRAQ PRIMARY HEALTH CARE PROJECT 69 Recommendation Deliverable Verifiable Indicator Comments/Suggestions Focus studies as per recommendations Deliverable 2.5c One study evaluating effectiveness of quality improvement activities in Iraq completed and disseminated (total 3 for life of project). One of 3 studies completed (recording and reporting on maternal deaths). Note: The report is described as operations research. The evaluation team finds this is a study, not operations research. Recommend two studies demonstrating the QI process being effectively followed, including identification of a problem, steps taken to resolve the problem, and outcome attributable to the QI process. See comment Deliverable 2.6 Equip model clinics Steps toward equipping model clinics There is no deliverable of model service provision. It is rather simply equipment supply and maintenance. If feasible, integrate components from the recommendations for model clinic program implementation. MID-TERM PERFORMANCE EVALUATION OF USAID/IRAQ PRIMARY HEALTH CARE PROJECT 70 Annex 11. Suggestions on PMP Indicators and BCC Activities (M&E Strategy) PMP Indicators Indicator Comment Recommendation (1) Percentage of target beneficiaries who receive Public Health Care Clinic Services The PMP describes relying on the PHCC registration book, family files (if applicable), or digital records. The elements required to eliminate double counting from repeated visits do not exist in most (if any) of the PHCCs. A more accurate indicator for data available without a population-based survey or medical record abstract (assuming individual records are in place in all PHCCs) is “The average outpatient visits per capita per catchment population.” This assumes that most visits are from persons from the catchment area, which seems a reasonable assumption. (2) Percentage of target beneficiaries satisfied with PHCC service The PMP describes using client exit interviews to collect this information. This will not measure target beneficiaries satisfied with PHCC services, but rather will measure satisfaction of current users. Opinions of target beneficiaries require a population-based survey. A more accurate indicator for the proposed methodology is the “Percent of users who are satisfied with PHCC services.” (3) Percentage of interview respondents who recall seeing or hearing a specific PHCPI-supported message The PMP describes the PHCPI will use BCC techniques to promote PHCCs to target communities. The PMP describes the interview respondents as PHCC clients. This will provide a biased measure of BCC messages reaching and being recalled by the target community, as clients are more likely to receive and remember PHCC-related messages than other community members. Measurement of this indicator has to be community-based. (4) Percentage of interviewed respondents who can mention two types of services offered by PHCCs The PMP describes the interview respondents as PHCC clients. This will provide a biased measure of BCC messages reaching and being recalled by the target community, as clients are more likely to receive/remember PHCC related messages than other community members. As users of the services, they are likely to know the services without receiving BCC messages. Measurement of this indicator has to be community-based. MID-TERM PERFORMANCE EVALUATION OF USAID/IRAQ PRIMARY HEALTH CARE PROJECT 71 BCC Activities - M&E Issues Issue Comment Recommendation (1) The national campaign strategy identifies that it would conduct a national level knowledge, attitude, and behavior (KAB) survey, which will be multi-staged so that behavioral changes and outcome indicators of the campaign can be assessed. Behaviors are to be measured through intentions as well as through increased rates of pre￾and post-natal screening, immunizations and other preventive behavior. The campaign M&E strategy is correct in that its outcome measures would be through community-based surveys conducted pre- and post￾campaign. However, measurement should not be through a national survey, even though the campaign would use mass media, which has national reach. This is because local campaign activities as well as quality supply services would only be for PHCPI-participating PHCCs. The pre- and post-KAB surveys should be conducted with a representative sample of participating local communities. (2) The BCC strategy impact evaluation strategy states that it would use “Facility-based surveys: To link exposure to BCC campaigns and messages to service utilization. To assess knowledge, attitudes, and behavior regarding PHC; change in access to PHC services and to measure client satisfaction.” This would provide a biased measure of outcomes of the BCC campaigns, as health facility users of services are more likely than community non-users of services to know about services and to have positive attitudes towards such services. BCC impact evaluation should be conducted through a community￾based survey. If for security reasons, community-based surveys are impossible to conduct in Iraq, then the minimum outcome indicators for BCC activities that the project could track relate to changes in utilization statistics per capita for the promoted services. Records should be set pre￾campaign to enable making a baseline measurement as well as at the end of the campaign. The USAID/Iraq Performance Management Specialist should review and agree, ahead of time, on the M&E plan for any BCC activities as well as the survey instruments. MID-TERM PERFORMANCE EVALUATION OF USAID/IRAQ PRIMARY HEALTH CARE PROJECT 72 Annex 12. Outreach Activities for Vulnerable Groups Several district managers noted their outreach activities to vulnerable groups. In Mugdadia District, which is rural, there are no female physicians in the PHCCs; women go to the district hospital. From time to time, the district hospital female physician reaches out to the local community. In Al-Warka’a and Al-Kout 2 districts, they conduct outreach to rural, squatter areas where IDPs live, as well as to remote areas, to conduct therapeutic services in conjunction with immunization campaigns as well as to distribute chlorine tablets for water purification. In Al￾Ramady 2 District, the managers asked the MoH to rehabilitate an old building to make a sub￾PHCC (Health House) serving a remote area. A donated caravan was placed in another remote area, and a medical assistant from the nearest PHCC has begun going there twice a week with medication and vaccines to serve the community. In Affak District, the health promotion program stresses that they should go to these areas where marginalized groups live and provide services and immunization as well as awareness raising. In Karkh District in Baghdad and in Al-Kout 2 in Wasit, a paper Health Visitor Program (HVP) is designed to identify immunization defaulters at their homes. Through this program, all households in a catchment area are enumerated, including in informal settlements and squatter areas. For informal settlements, a health committee from the PHCC regularly visits these areas to check on the water source, harmful insects, rodents or pests, and solid waste. Chlorine tablets and insecticides are provided for the community and the respective government entity is notified of water or environmental problems. Every month a form is filled out by the PHCC committee that visits the squatter areas; the completed form is sent to the district. Until May 2012, this form included information on number of people and families, number of under-5 children, water source and chlorine test, presence of bathrooms, solid waste and harmful insects, and rodents and pests. In June 2012, this form was upgraded to include information on accessibility to and use of health services, e.g., availability of health facilities, medication, ambulance, delivery, and minor surgical services. This information is filled out for the informal settlements exactly as for the rest of the catchment area. An important addition to the form is the inclusion of questions on common diseases such as diarrhea and respiratory problems, which may signify dangerous communicable diseases. These cases are either treated on the spot or referred to the PHCC or a hospital as needed. Another important addition is a question on non-communicable diseases and adequacy of the supply of medication to treat them. However, it is doubtful that the committee would physically check for non-communicable diseases. Theoretically, where the Health Visitor Program is implemented, inhabitants of informal settlements, including IDPs, have the same access to PHCCs as the general population. Their environment is checked by the PHCC committee to remove public health threats. However, an interview with an informal settlement representative has shown that inhabitants suffer from lack of adequate sanitation, a situation that the government is not addressing. MID-TERM PERFORMANCE EVALUATION OF USAID/IRAQ PRIMARY HEALTH CARE PROJECT 73 Annex 13. Iraqi Patients’ Rights Statement 1. General Achievements The Iraqi Patients’ Rights Statement10 produced by USAID/PHCPI with MoH senior officials and the assistance of Ministry of Human Rights, the Medical Syndicate, and the Council of Representatives Environment and Health Committee, represents significant forward progress toward implementing a rights-based approach to health care in Iraq. Patients’ rights and responsibilities have been incorporated into the Public Health Law as Chapter 5, articles 90 and 91, to be submitted to the Council of Representatives for deliberation. The Iraqi Patients’ Rights Statement has been launched through a national conference and is now being disseminated to provinces through workshops for relevant DoH staff as well as district and PHCC managers. These conferences and workshops attracted media coverage, which helps disseminate patients’ rights concepts to the Iraqi population at large. In fact, 16 out of the 18 interviewed district managers reported that they are disseminating information on patients’ rights using posters and brochures as well as holding seminars for PHCC staff. The Patients’ Rights Statement includes four main rights (See Endnote 1), including the right to: 1) information, 2) treatment and care, 3) make decisions, and 4) privacy. Many standard patients’ rights are missing. However, under these four main rights, other rights are articulated, either explicitly or implicitly. 2. Content Analysis The Patients’ Rights Statement was written with great detail relating to permitted and prohibited procedures. Before raising awareness of providers, patients, and communities regarding patients’ rights, expression of these rights should be simplified and made more applicable to the providers’ behavior and patients’ understanding. In the Patients’ Rights document, sometimes two rights are phrased under one right and categorized under a somewhat inappropriate category. For example the right to be referred is stated with the right to information regarding reasons for referral and is categorized under the right to information. In addition, the right to a second opinion is categorized under the “right to information,” although this could have been better categorized under the “right to treatment and care.” A very basic human right guaranteed by the Iraqi constitution, “access to health care,” is not articulated in the Patients’ Rights document. The only two rights under the “right to treatment and care” are the “right to choice of provider” and the “right to receive family support during treatment.” Since the MoH with support from PHCPI is strengthening the PHCC’s ability to provide good quality care, a fundamental right should be “the right, irrespective of age, sex, religion, ethnicity, or socio-economic background, to access health care that is of good quality and safe.” PHCPI Arabic language training handouts and wall posters include “the right of the patient to receive treatment of high quality that is appropriate to the condition without discrimination among patients.” 10 USAID/PHCPI, February 2012. Deliverable Number: 3.1a, National Statement of Patients’ Rights in Primary Health Care Developed. MID-TERM PERFORMANCE EVALUATION OF USAID/IRAQ PRIMARY HEALTH CARE PROJECT 74 Some rights should be articulated in a better way. For example: “The right to know the procedures for lodging a complaint without fear of consequences,” which is categorized under the right to information, is better stated as the “right to file a complaint without fear of consequences.” In articulating the “right to informed consent,” the phrasing instead expresses only the special case of getting consent from the legal representative of the patient, if the patient is unconscious or unable to express his/her will. From the individual rights expressed under the “right to privacy,” it is evident that this right refers to the “right to privacy and confidentiality of information” only. In addition, the “right to access health care that is of good quality and safe” is missing from the Iraqi Patient’s Rights Statement. Other basic rights are missing. For example, the “right to auditory and visual privacy” and the “right to treatment with respect and dignity” are missing. This is despite the fact that the ultimate aim of the Patients’ Rights document, as defined in the document, is to “protect the dignity and integrity of the patient” and to “promote an interactive, respectful relationship between patients and providers.” The PHCPI Management Handbook11 provides the patients’ rights in simple, appropriate language for providers. Both PHCPI and MoH senior officials have indicated that these two missing rights are implicitly included. However, they have to be explicitly articulated in the Patients’ Rights document and in Public Health Law. 3. Information Education and Communication (IEC) Materials The MoH is committed to adopting patients’ rights at PHCCs. PHCPI is cooperating with the MoH in raising awareness of PHCC providers on patients’ rights. Similar awareness-raising is planned for patients and communities. As cost-share, the MoH has already printed 500,000 brochures and two out of four designed posters, which are being distributed to the provinces. For the brochure, the text of the Patients’ Rights document was directly copied without any attempt at making the statements more appropriate and interesting to either the provider or the patient as a target audience. For patients, the posters do a better job than the brochures, as they have selected one statement to explain each of the four main rights. One of the posters provides the rights as well as the responsibilities of the patient; it would have been better to develop a separate poster for each. This would also make the text under patients’ responsibilities more legible. In order to produce patient- and community-friendly IEC materials on Iraqi patients’ rights, the statements should be rephrased to adapt it to the laypersons’ understanding. Endnote 2 at the end of this annex shows Iraqi patients’ rights statements rephrased in a more concise and articulate way and in a different order and grouping. Added text and essential rights, which should have been part of the statement, are inserted in square brackets. The proposed list shows the wide variety of rights included in the Iraqi Patient Rights Statement. This list makes a first attempt at addressing the community as an audience. 4. Patients’ Rights as Guidelines for Providers Patients’ rights should be distilled into a guide to raise provider awareness. Such a guide will instruct the provider on the actions needed to observe patients’ rights and how PHCC management should rearrange service provision. The Handbook of Quality Standards and 11 USAID/PHCPI, January 2012. Deliverable Number: 1.2a, Handbook of Quality Standards and Operational Guidelines for Management of Primary care Clinics. MID-TERM PERFORMANCE EVALUATION OF USAID/IRAQ PRIMARY HEALTH CARE PROJECT 75 Operational Guidelines for Clinical Services Delivery12 includes examples of ways in which to talk to patients/clients in providing counseling. Unfortunately, the guide only directly addresses patients’ rights in an annexed supervisory checklist in the form of questions to patients and provider observations on treating patients with respect, ensuring quality information exchanges between patient and provider, and demonstrating respect for patient privacy. 5. Findings of Evaluation Interviews Client exit interviews conducted as part of this evaluation show that 25% of clients complained of a lack of visual and auditory privacy, 12% complained of inadequate information exchange, and 8% complained of poor treatment by staff. Therefore, providers would benefit from training on patient-provider interaction during consultations as well as sensitization to patients’ rights. Only 23% of health workers and 30% of clients were found to be knowledgeable about patients’ rights. Such low levels of knowledge are indicative of the stage of implementation of awareness￾raising for these two groups, which needs further work. Of those clients aware of patients’ rights, 73% had become familiar with the concept from TV or radio; 46% had seen posters and brochures in the PHCC. Although distribution of posters was not yet complete at the time of the evaluation team’s visit, posters were already found in 50% of visited PHCs. In contrast, there is a high level of knowledge about patients’ rights among district managers (90%) and LHC members (71%). For LHC members who knew about rights, 61% have seen posters and brochures, 46% learnt about it through LHC training, 35% have been told by PHCC staff, and 26% knew through TV and radio. This indicates a good level of project achievement. Of LHC members who knew about patients’ rights, knowledge of specific rights varied widely, from 94% knowing the right to treatment and care, 48% the right to know, 48% the right to referral, 44% the right to privacy of information, and less than 20% knowing about the right to treatment with respect and the right to auditory and visual privacy. This is probably because these latter rights are not explicitly articulated in statements of patients’ rights. Endnote 1: Iraq Patients’ Rights Statement Right to Information: I. The patient has the right to receive information recorded in any of his/her medical records and to be fully informed about his/her health status. II. The patient has the right to direct access to his/her own health records, ask questions about their contents, and request copies of part or whole of their medical file. III. The patient has the right to confidentiality. IV. Information should be given to the patient taking into account the religious, ethnic, or linguistic specificities of the patient. V. The patient has the right to choose who receives information related to his/her health care. VI. The patient has the right to ask for the opinion of another physician at any stage of his/her care. VII. The patient has the right to be referred to other health service providers that provide diagnostic, curative, and teaching facilities. In the event of transferring the patient from a treatment center to another facility, the patient must be given adequate 12 USAID/PHCPI, April 2013. Handbook of Quality Standards and Operational Guidelines for Clinical Services Delivery. MID-TERM PERFORMANCE EVALUATION OF USAID/IRAQ PRIMARY HEALTH CARE PROJECT 76 explanation of the reasons for the transfer and a suitable place must be secured to continue treatment in another hospital or at home if possible. VIII. The patient has the right to know the procedures for lodging a complaint without fear of consequences. Right to Treatment and Care: I. The patient has the right to choose and change the service provider in accordance with the health system of the country. II. The patient has the right to receive support from family and friends during treatment. Right to Make Decisions: I. The patient has the right to make free decisions. The physician will inform the patient of the possible consequences of his/her decision. II. If the patient is unconscious or unable to express his/her will, informed consent must be obtained whenever possible from a legal representative. III. The patient has the right to refer to legal authorities in the event of decisions made by the service provider that were harmful to the patient. Right to Privacy: I. All information related to the patient must be kept confidential even in the event of his/her death. Under the law, it is permitted to disclose some information with the request of a legal authority. II. The patient has the right to request information and copies of medical documents concerning his/her health care. The health center or hospital is not permitted to release any personal information, reports, or documents concerning the patient’s health to family members or other parties for further diagnosis and treatment without his/her written approval. III. The patient has the right to privacy and confidentiality of information and socio￾medical data and can only be seen by those who have a direct relation to his/her treatment and for the purposes of medical visits, improvement of treatment, performance, and quality. IV. Publication of information related to the patient requires written consent unless his/her life is subject to imminent risk according to the legal measures in force or infected with a reported communicable disease. Endnote 2: Iraq Patients’ Rights Statement Rephrased to Show the Wide Variety of Rights Included The text below is extracted from the Iraq Patient Rights Statement, using more concise and articulate phrasing and a different order and grouping. Added text and rights are inserted in square brackets. Right to Information: 1. The right to be fully informed about his/her health status. 2. The right to direct access to his/her own health records. 3. The right to an adequate explanation of reasons for referral, in the event of transferring the patient from s treatment center to another facility. MID-TERM PERFORMANCE EVALUATION OF USAID/IRAQ PRIMARY HEALTH CARE PROJECT 77 4. The right to receive information, taking into account his/her religious, ethnic, or linguistic specificities, Right to Treatment and Care: 5. [The right to treatment with respect and dignity] 6. [The right to auditory and visual privacy during consultation] 7. [The right, irrespective of age, sex, religion, ethnicity, or socio-economic background, to access good quality health care that is safe] 8. The patient has the right to choose and change the service provider in accordance with the health system of the country. 9. The right to ask for [a second] opinion [by] another physician at any stage of his/her care. 10. The right to be referred to other health service [organization if needed services are not available in current health care organization] 11. The right to a suitable place to continue treatment in another hospital or at home if possible, in case the patient is transferred. 12. The right to receive support from family and friends during treatment. 13. The right to [file or make] a complaint without fear of consequences. 14. The right to refer to legal authorities in the event of decisions made by the service provider that were harmful to the patient [other than his/her medical condition prognosis] Right to Make Decisions: 15. The right to make free decisions. The [treating] physician will inform the patient of the possible consequences [(pros and cons) of all alternatives; the patient then has the right to make] his/her decision. 16. The right to informed consent: Even when a patient is unconscious or unable to express his/her will, informed consent must be obtained whenever possible from a legal representative. Right to Privacy and Confidentiality of Information: 17. The right to confidentiality [of information] 18. The right to choose who receives information related to his/her health care [and health status]13 19. The right to confidentiality of information even in the event of his/her death. Under the law, it is permitted to disclose information [only] upon request of a legal authority. Publication of information related to the patient requires written consent unless his/her life is subject to imminent risk according to the legal measures in force or if the patient is infected with a reportable communicable disease. Endnote 3: Additional Patients’ Rights that PHCPI and MoH Should Consider Right to Information: 1. The patient has the right to receive discharge summary and medical reconciliation. 2. The patient has the right to be given information taking into account his/her religious, ethnic, or linguistic specificities. 13 The patient has the right to privacy and confidentiality of information and socio-medical data and can only be seen by those who have a direct relation to his/her treatment and for the purposes of medical visits, improvement of treatment, performance and quality. MID-TERM PERFORMANCE EVALUATION OF USAID/IRAQ PRIMARY HEALTH CARE PROJECT 78 3. The patient has right to know the name of the physician who has primary responsibility for coordinating his/her care and the names of other staff that will care for him/her. 4. The patient has the right to receive information about his/her health status, course of treatment, prospects for recovery, and outcomes of care (including unanticipated outcomes) in terms he/she can understand (layman language, not sophisticated medical jargon). 5. The patient has the right to effective communication and to participate in the development and implementation of his/her plan of medical care. 6. The patient has the right to receive information about any proposed treatment or procedure in order to give informed consent or to refuse a course of treatment. Except in emergencies, this information shall include a description of the procedure or treatment, the medically significant risks involved, alternate courses of treatment or non-treatment and the risks involved in each, and the name of the person who will carry out the procedure or treatment. 7. The patient has the right to access information about health services and outcomes of health service organizations. Right to Treatment and Care: 1. The patient has the right to have his/her cultural, psychosocial, spiritual, and personal values, beliefs, and preferences respected. 2. The patient has the right to get his/her pain assessed and managed and to participate in pain management decisions, 3. The patient has the right to receive care in a safe setting, free from mental, physical, sexual, or verbal abuse and neglect, exploitation, or harassment. 4. The patient has the right to access protective and advocacy services including notifying government agencies of neglect or abuse. Right to Make Decisions: 1. The patient has the right to participate in ethical questions that might arise during the course of his/her care, including issues of withholding resuscitative services, and forgoing or withdrawing life-sustaining treatment. 2. The patient has the right to request or refuse treatment, to the extent permitted by law. However, he/she does not have the right to demand inappropriate or medically unnecessary treatment or services. 3. The patient has the right to leave the hospital even against the advice of physicians, to the extent permitted by law. 4. The patient has the right to refuse to participate in research projects. MID-TERM PERFORMANCE EVALUATION OF USAID/IRAQ PRIMARY HEALTH CARE PROJECT 79 Annex 14. Verification of Technical Deliverables Deliverable: (to be completed by PHCPI staff) By signing below, I acknowledge that I reviewed the deliverable to the best of my abilities and verify that it will support and strengthen the goals of PHCPI and the delivery of PHC services in Iraq. Stakeholder Representation Name Title Organization and Department Date Signature Ministry of Health PHCPI (Chief of Party) USAID/Iraq representative TAG member (1) TAG member (2) PHCPI Component Team Leader MID-TERM PERFORMANCE EVALUATION OF USAID/IRAQ PRIMARY HEALTH CARE PROJECT 80 Stakeholder Representation Name Title Organization and Department Date Signature USAID/W representative URC headquarters representative MSI headquarters representative MID-TERM PERFORMANCE EVALUATION OF USAID/IRAQ PRIMARY HEALTH CARE PROJECT 81 Annex 15. Checklist for Strategic Decision Making (to be presented to TAG) Checklist for Strategic Decision Making (to be presented to TAG) Please provide a narrative description of the proposed activity or technical area of intervention: Please provide the most recent epidemiological data (prevalence, incidence, or number of persons affected) and a narrative description for the health issue that will be addressed. Please relate the information to the top causes of mortality and morbidity in Iraq: Please provide information on the “readiness” of PHCCs to undertake the proposed activity and/or health issues (i.e., are the necessary equipment and supplies available? do the PHCC health care providers have the legal mandate to address the issue? are there the needed skills at the PHCC level?): MID-TERM PERFORMANCE EVALUATION OF USAID/IRAQ PRIMARY HEALTH CARE PROJECT 82 Checklist for Strategic Decision Making (to be presented to TAG) Please provide evidence of the cost-effectiveness of the proposed activity or health issue to be addressed in terms of it having a high impact on mortality and morbidity within Iraq. If possible, please provide two published refereed references which demonstrate cost-effectiveness of the proposed activity(ies): Please provide information, if known, on how the proposed activity will reach vulnerable groups: Are there any GoI laws or policies which would be violated by this activity? Yes/No If yes, please describe: Please provide information on any other similar programs (both government and internationally sponsored): MID-TERM PERFORMANCE EVALUATION OF USAID/IRAQ PRIMARY HEALTH CARE PROJECT 83 Checklist for Strategic Decision Making (to be presented to TAG) Please list the indicators within the PMP which could be (positively) affected by implementing the proposed activity or addressing the proposed health issue: Please describe the transparent and documented process used to engage relevant stakeholders in the development of this proposed activity. Stakeholder Representation Name Title Organization and Department Date Signature MoH PHCPI (CoP) TAG member (1) TAG member (2) PHCPI Component Team Leader MID-TERM PERFORMANCE EVALUATION OF USAID/IRAQ PRIMARY HEALTH CARE PROJECT 84 Annex 16. Strategy for Focusing PHCPI Activities A. Illustrative Example of Prioritization of PHCPI Focus for the Remaining Life of the Project Following is an illustrative example of a selection of priority services and activities for the remaining two years of the PHCPI. This is meant only as an illustrative example of topics and recommendations on service delivery strategies that are likely to achieve the most health benefit with the least risk within the current framework of the Iraqi disease profiles and health system constraints and international proven success in different service delivery models. The BHSP is an inclusive package of primary health care services that the MoH has agreed-upon and is striving to achieve and should be provided in the PHC setting. For the BHSP to be implemented as written under the existing system, physician staffing numbers must be rapidly increased and positioned in PHCCs. If the MoH and GoI plan for a physician-based system for prescriptions and interventions, much of the population will remain without needed services and levels of care provided through the public sector for many years. The PHCPI, however, has different objectives. On one level, it has the global objective of strengthening PHC services, which it has addressed by developing manuals and guidelines that: 1) serve as a management framework for PHC services; and 2) provide guidance on topics relevant to the PHCC service setting. This approach results in an overall framework for a system that will deliver a total package of PHC services, allowing the system to function over time as the needed resources become available. The project also has more immediate objectives per its contract, namely: • Technical assistance must result in realistic, practical systems, procedures, and tools that can be effectively applied in all primary health care clinics, not only in higher functioning model sites. • Successful project implementation must result in rapid, tangible, measurable improvements in the quality of health care services delivered to the Iraqi people. • Sustainable institutional capacity within the MoH must be developed. The evaluation team recommends that these immediate objectives be the focus throughout the remaining life of the project. These objectives can be achieved through the following actions: 1) Selecting a package of services for focus, using the following criteria: • Interventions internationally proven to improve maternal and child health indicators • Most important contributors to mortality and morbidity and the most common conditions presenting for health services • Can reasonably be provided within current staffing and facility infrastructure constraints or with inputs that are feasibly available in a short time frame 2) Focusing systems for supporting quality services on the components of this package. MID-TERM PERFORMANCE EVALUATION OF USAID/IRAQ PRIMARY HEALTH CARE PROJECT 85 The specific systems are those PHCPI has planned for strengthening during Year 3 of the project. This means that supervision strategies, quality improvement activities, and monitoring against standards for clinical services should focus on priority services, not on the total BHSP package. Developing a focused strategy for PHCPI does not imply a revision of the BHSP, but it does clearly identify where PHCPI will put its energy and resources. Activities and services outside of the focused package may continue to be addressed by the MoH and other stakeholders outside of the PHCPI context. Following is a table that provides the list of suggested priority services and suggestions for how access to the service can be expanded within existing constraints, including lack of physician service providers, minimizing referrals, and the lack of services at the nearest facility. The focus on minimizing referrals is particularly relevant in view of security risks, but also in view of common experience where clients may not follow through on referrals due to time, distance, and financial constraints. Overall, the project should: • Advocate for routine follow up (examination, diagnostic testing, assessment for complications, and routine refill of prescriptions if no complications for chronic illnesses); • Focus strengthening of the referral system on: o Promoting information sharing for diagnosis, initial prescription at higher level services, and follow-up at facility nearest to client home o Promoting information sharing so ANC and postnatal care (PNC)/newborn care are promoted at facility nearest to client home even if delivery is elsewhere o Sub-clinic recognizing complications in routine follow-up that require referral • Ensure routine preventive services for maternal child health are available (ANC, PNC, birth spacing/family planning, growth monitoring, and immunization) MID-TERM PERFORMANCE EVALUATION OF USAID/IRAQ PRIMARY HEALTH CARE PROJECT 86 Service Recommendations for Service Components (to be provided by non-physicians to increase access at sub￾clinics and clinics with one physician, to reduce workload for physicians and decrease unneeded referrals) Additional Comments A Maternal and newborn health/reproductive health 1 Antenatal care 1. Testing for blood sugar, urine protein, hemoglobin for risk screening. 2. Refresher training on identifying probable UTI/STI/RTI in maternity cases and either expansion of treatment capacity or referral. 3. Conditions for referral clearly written and referral site clearly identified. 4. Training on counseling; system for improving client follow￾through on referrals of risk conditions, facility deliveries, and returning to nearest facility for PNC/newborn care. 5. BCC activities to promote the above as “good quality services.” 1. Clear guidelines for diagnostic results indicating risk and need for referral. 2. Formal linkages with nearest facility able to manage risk conditions (this may be a larger PHCC, not a hospital). 3. Focus implementation of referral system on follow-up of referred ANC cases with the objective to improve PNC/newborn care services at clinic where ANC services were provided. 4. Needs assessment for job aids and refresher training for diagnosis and treatment of UTI, STI/RTI for maternity cases. 2 Delivery services 1. Needs assessment for delivery services in rural and suburban PHCs. 2. Strategy developed by relevant stakeholders for introducing needed services. 1. Coordination with stakeholder focusing on delivery services, training, and on-the-job supervision and follow￾up. This should not be the focus for PHCPI but left to other stakeholders given the time and resource constraints and the need for intensive follow-up to improve EmONC service quality. 2. Address referral as described with #3, ANC services. 3 Postnatal care 1. Needs assessment for guidelines/refresher training for improving quality of PNC services—areas for focus: • Postpartum examination. • Clear guidelines for symptoms requiring referral. • Counseling on birth spacing/family planning. • Counseling on infant and young-child feeding—focus on exclusive breastfeeding. 1. Address referral as described with #3, ANC section. 2. BCC strategy to reinforce importance of PNC/newborn care and availability of quality services in nearby PHC [branding for ANC/PNC/newborn care] 3. Coordination with stakeholders focusing on BS/FP to provide training and on-the-job supervision and follow￾up. MID-TERM PERFORMANCE EVALUATION OF USAID/IRAQ PRIMARY HEALTH CARE PROJECT 87 Service Recommendations for Service Components (to be provided by non-physicians to increase access at sub￾clinics and clinics with one physician, to reduce workload for physicians and decrease unneeded referrals) Additional Comments 4 Newborn care Integration with PNC and IMCI/growth monitoring/immunization. 5 BS/FP Coordination with stakeholder focusing on BS/FP to introduce service. 1. “Healthy Timing and Spacing of Pregnancy” public health/educational strategy to increase MoH and client acceptance. 2. Information sharing to strengthen linkages with facilities offering long-term and permanent methods. 6 STI/RTI 1. Needs assessment for guidelines and refresher training on most common STI/RTI encountered during ANC and FP services. • Syndromic management if diagnostic tests not available. Focus on the most common STI/RTI that present during BS/FP and maternity services. B Child health 7 Growth monitoring 1. Needs assessment for refresher training to improve counseling for infant and young child feeding and exclusive breastfeeding. 8 Immunization 1. Tetanus toxoid vaccine for ANC where sub-clinic stores vaccines. 9 IMCI 1. Refresher training on IMCI with focus on treatments allowed at sub-clinic, differential diagnosis, developing referral linkages. 2. Specific job-aids and referral criteria for services provided by non-physician. 1. Zinc available in facilities. 2. Advocacy to increase conditions non-physician can treat without referral (e.g., child dysentery) and ensure needed treatment drug on EDL for any changes. 3. Pre-referral injectable antibiotic available in the PHCCs. MID-TERM PERFORMANCE EVALUATION OF USAID/IRAQ PRIMARY HEALTH CARE PROJECT 88 Service Recommendations for Service Components (to be provided by non-physicians to increase access at sub￾clinics and clinics with one physician, to reduce workload for physicians and decrease unneeded referrals) Additional Comments C Other illnesses (greater than 5 years of age) Acute infectious illnesses/communicable diseases 10 Acute respiratory infections (ARI) 1. Needs assessment for job-aids and training on ARI: • Differential diagnosis (bronchitis, pneumonia, strep throat, viral ARI) • Specific job-aids for treatment by non-physicians and referral criteria 11 Urinary tract infection (UTI) 1. Needs assessment for job-aids and refresher training for diagnosis and treatment of UTI. 12 Gastroenteritis 1. Needs assessment for job-aids and training on gastro￾intestinal conditions: • Differential diagnosis (common diarrhea, blood dysentery, bacterial dysentery. • Specific job-aids for treatment by non-physicians and referral criteria. 13 Severe infectious illnesses 1. Needs assessment for job-aids and training on severe infectious illnesses (presenting with fever or other symptoms of septicemia) • Differential diagnosis and referral criteria. • Guidelines and training on pre-referral drugs and counseling to improve referral follow-up. 14 Skin conditions 1. Needs assessment for job-aids and training on common skin infections • Differential diagnosis and referral criteria. 1. Expansion of conditions non-physician can treat at sub￾clinic and at PHC level to reduce caseload for physician. 15 Conjunctivitis 1. Needs assessment for job-aids and training on common eye infections Chronic (non-communicable) illnesses MID-TERM PERFORMANCE EVALUATION OF USAID/IRAQ PRIMARY HEALTH CARE PROJECT 89 Service Recommendations for Service Components (to be provided by non-physicians to increase access at sub￾clinics and clinics with one physician, to reduce workload for physicians and decrease unneeded referrals) Additional Comments 16 Asthma No changes 17 Hypertension 1. As per BHSP, initial treatment and drug regime adjustment by PHC doctor. Follow-up treatment by paramedic—renewal of prescription if no complaints and blood pressure within defined range. 2. Job-aids to specify drugs that are available at the PHC level. 1. Advocacy for long-term follow-up of stable case by PHC physician and routine monitoring and renewal of prescription by non-physician. 2. Focus on referral and information sharing between sub￾clinic and PHCC and higher level services to promote routine follow-up and prescription refill of stable clients by nearest level of facility, with protocol clarifying conditions and frequency for reassessment by PHC physician. 18 Diabetes 1. As per BHSP, initial treatment and drug regime adjustment by PHC doctor. Follow-up treatment by paramedic—renewal of prescription if no complaints and blood sugar within defined range. 2. Job-aids to specify drugs that are available at the PHC level. See comments for hypertension. 19 Cardio￾vascular illness 1. Needs assessment for job-aids and training on common cardiovascular illnesses. • Differential diagnosis and referral criteria. • Guidelines and training for long-term follow-up at PHC level and renewal of prescriptions by specialist, if no complications. 1. As per BHSP equipment lists, EKG machines at the PHC level. 2. Training on reading EKG—identifying key abnormalities requiring referral. 3. Testing of model clinics’ service delivery strategies, referral processes, and long-term follow-up by PHCC. C Other conditions 20 Accidents First aid and emergency pre-referral care. Public health approach (BCC for prevention) 21 Epidemic 1. Needs assessment for differential diagnoses, job-aids, epidemic responses for typhoid, cholera, H5N1, other epidemic flu conditions. BCC for prevention, awareness of health risks, and links with illnesses. 22 Long-term health risks Obesity 1. Encouragement of MoH to consider public health approach (BCC—public information campaigns for MID-TERM PERFORMANCE EVALUATION OF USAID/IRAQ PRIMARY HEALTH CARE PROJECT 90 Service Recommendations for Service Components (to be provided by non-physicians to increase access at sub￾clinics and clinics with one physician, to reduce workload for physicians and decrease unneeded referrals) Additional Comments Breast cancer Uterine cancer Hepatitis prevention, seeking screening services) 2. Availability of screening tests and follow-up on positives prior to BCC campaign. MID-TERM PERFORMANCE EVALUATION OF USAID/IRAQ PRIMARY HEALTH CARE PROJECT 91 Annex 17. Comments on PHCPI Clinical Guidelines Guidelines traditionally are subsets of generic instructions for managing an illness that provide a country-specific first line and second line intervention or treatment. Guidelines use the drugs and diagnostics that are available at the level where the service is being offered, or diagnostics where a specimen can be sent out and results returned for follow￾up. Referral guidelines usually indicate the specific point at which a specified level of provider should refer. These types of guidelines improve the quality of services by setting an approved treatment protocol that providers are expected to follow, using diagnostics (or a system for receiving diagnostic results back for follow-up) and drugs that the MoH procures and supplies to the level of service. Almost all the guidelines developed with PHCPI support are generic instructions. The evaluation team is not recommending that these be changed, given that they are already approved by the MoH; however, it is recommended that job-aids (see Year 3 work plan) be developed that are more likely to result in improved quality of diagnosis and treatment across PHC services. Revising the current draft of the Handbook of Quality Standards and Operational Guidelines for Clinical Services so that it serves the purpose of a job-aid for the priority services described in Annex 16 would provide a tool for improved quality of diagnosis and treatment that reduces differences in service quality due to differences in skills and knowledge among service providers. The job-aids should: 1) Refer to drugs (by name) that are on the EDL at the level on which the job-aid is focusing 2) Provide specific referral criteria for relevant conditions 3) Cover all priority services The comments are not meant to be all-inclusive but rather to provide examples of issues that the evaluation team recommends be addressed when developing job-aids for priority PHC areas of focus. MID-TERM PERFORMANCE EVALUATION OF USAID/IRAQ PRIMARY HEALTH CARE PROJECT 92 Observations on Clinical Guidelines Produced with PHCPI Support Note: the objective of this table is to demonstrate the lack of alignment of MoH policy, the EDL for PHCCs and the guidelines produced with PHCPI support. This is not meant to be an all-inclusive list of issues. Main Sub￾guideline Comments on Drug Treatments Not Aligned with EDL or Policy Comments on the Guideline Document IMCI Pneumonia 1st line treatment (amoxicillin) is on EDL for sub-centers. Clear treatment and referral guidelines needed. 2nd line treatment cotrim not in sub-center; referral criteria needed. Pre-referral injectable antibiotic (cefotaxime) not on PHC EDL. Physician in PHCC should be able to treat and only refer if child does not respond or requires hospitalization. Pre-referral injectable antibiotic should be available in PHCC for physician. Severe: 2nd line injectable refers to gentamicin/ampicillin together. These are available in the PHCCs but IMCI guideline provides no dosages. Watery diarrhea: zinc is on EDL for all levels of PHCCs but not found in clinics. Blood dysentery: metronidazole not available in sub-center. Physician guidelines are generic; many pictures are from Africa. Consider protocol for sub-centers classified as “isolated” allowing 2nd line oral antibiotic (cotrim) and expanding the treatment level allowed to reduce unnecessary referrals, which risk not being followed; this could occur with help from private providers, potentially not IMCI trained. Communicable diseases The document focuses on generic background on what communicable diseases are and provides a brief description of reportable diseases. The main value may be as a reference document for different reportable illnesses. Common communicable illnesses in adults include acute respiratory illnesses, gastrointestinal illnesses, skin infections. This is not a document that the evaluation team expects will result in improved diagnostic and treatment practices at the PHCC. A job-aid for common communicable illnesses will be useful if needs assessment shows problems with differential diagnoses and treatment of common communicable illnesses. Cholera Antibiotics recommended in IMCI guidelines. 1st line: ciprofloxacin (on EDL as ciprofloxacin) adult doses. 1st line: e4rythromycin not on EDL. 2nd line oral ampicillin not on EDL. Non-physician can safely provide emergency treatment for cholera while waiting for help. Communicable diseases guidelines only state “appropriate antibiotic.” This should be specified so where MID-TERM PERFORMANCE EVALUATION OF USAID/IRAQ PRIMARY HEALTH CARE PROJECT 93 Observations on Clinical Guidelines Produced with PHCPI Support culture and sensitivity are not available, rational treatment results. Non￾communicable diseases Asthma Refers to corticosteroid inhalers (not on EDL). Specific drugs mentioned in guideline that are not on EDL include cromolyn (Intal) or nedocromil (Tilade) inhaler. Zafirlukast (Accolate) or montelucast (Singulair). Generic guidelines. Hypertension Guidelines refer to ARB, CCBs (non dihydropyridines), angiotensin II antagonists. None of these category drugs are on EDL. A limited number of drugs within other categories of drugs for hypertension are on the EDL. The job aids should refer to these by name. Although some doctors were using the 1st line treatments described, within the same clinics doctors were prescribing methyldopa and furosemide (an older but appropriate treatment for hypertension that is not mentioned in the guidelines). This was found in 3 of the 4 PHCCs visited by the evaluation team. The problem with this is that where furosemide is used, blood chemistries need to be followed particularly to ensure that potassium levels remain safe and that replacement potassium is not required. Replacement potassium is not in the EDL for PHCCs. Generic guidelines. Diabetes Insulin is on the EDL but was found in only sample PHCCs; the evaluation team was informed by PHCC and MoH key informants that insulin is not prescribed at the PHC level. Draft handbook of quality for PHC refers only to the oral treatments with no mention of insulin. This is in line with PHC services but not with the PHC guidelines for diabetes. No indication at which point referral is indicated for sub-clinic or PHCC. Generic guidelines. Emergency Obstetric Care (EmONC) Management of hypertensive disorder of pregnancy and convulsions: the guidelines recommend injectable phenobarbital and injectable hydralazine or labetolet, which are not on the EDL for PHCCs of any level. The alternative (nefedipine sub-lingual) is available. Guidelines for infant resuscitation with naloxone or epinephrine are provided along with instructions that these are not to be used at the PHC level. These are not on the EDL for PHCCs. The guidelines are good and cover key “best practices,” e.g., neonatal resuscitation (helping babies breathe— HBB, active management of third stage labor—AMTSL, partographs, bimanual compression uterus, manual removal of placenta, kangaroo care. However, the emergency interventions do not address activities or medicines MID-TERM PERFORMANCE EVALUATION OF USAID/IRAQ PRIMARY HEALTH CARE PROJECT 94 Observations on Clinical Guidelines Produced with PHCPI Support Guidelines for neonatal sepsis include cloxacillin (not on EDL). Alternatives (gentamicin and amoxicillin) are available. Injectable phenobarbital for neonatal convulsions is not on EDL. available at the PHCC level. A physician at PHC level should be able resuscitate using these medicines. MID-TERM PERFORMANCE EVALUATION OF USAID/IRAQ PRIMARY HEALTH CARE PROJECT 95 Annex 18. Data Collection Tools Annex 18A. Health Facility Survey SECTION 1.0 FACILITY IDENTIFICATION AND CONSENT 100 Name of facility (INCLUDE OTHER NAMES FACILITY IS KNOWN BY) _____________________________________ DISTRICT CODE FACILITY CODE 101 DATE OF DATA COLLECTION (WRITE DATE HERE) ________________________________________ DAY MONTH YEAR 102a 102b INTERVIEWER NAMES ________________________________________________ INTERVIEWER 1 CODE INTERVIEWER 2 CODE 103 Contact information for person in charge of facility NAME: _________________________ TITLE: _________________________ QUALIFICATION: _________________ PHONE (facility)_______________________ PHONE (cell) _________________________ EMAIL ______________________________ FIND THE MANAGER, THE PERSON IN CHARGE OF THE FACILITY OR MOST SENIOR HEALTH WORKER RESPONSIBLE FOR OUTPATIENT SERVICES WHO IS PRESENT AT THE FACILITY. READ THE FOLLOWING GREETING: Good day! My name is _____________________. I am part of an independent team assessing the Primary Health Care Project being implemented by the University Research Corporation and the Ministry of Health, and funded by the Ministry of Health and USAID. We are conducting an evaluation of the progress thus far in implementing the project and in improving primary health care services. As part of this evaluation we are conducting a survey of health facilities to assist the government and donors in knowing more about the current status of primary health care services. Now I will read a statement explaining the evaluation. Your facility was selected to participate in this study. We will be asking you questions about various health services and will be asking to see service sites, equipment, supplies, and pharmaceuticals, and will be asking to interview staff about training they have received and their experiences with the PHC system. Neither your name nor that of any other health worker respondents participating in this study will be included in the dataset or in any report. You may refuse to answer any question or choose to stop the interview at any time. However, we hope you will answer the questions, which will benefit the services you provide and the nation. MID-TERM PERFORMANCE EVALUATION OF USAID/IRAQ PRIMARY HEALTH CARE PROJECT 96 If there are questions for which someone else is the most appropriate person to provide the information, we would appreciate if you introduce us to that person to help us collect that information. At this point, do you have any questions about the study? Do I have your agreement to proceed? _________________________________________ 2 0 1 INTERVIEWER'S SIGNATURE INDICATING CONSENT OBTAINED DAY MONTH YEAR 104 May I begin the interview? YES ........................................................ 1 NO .......................................................... 2 STOP SECTION 1 POPULATION SERVED 1.1 CATCHMENT POPULATION First, I want to better understand how this facility links with communities, other facilities, and more about the client population served by this facility. 110 Does this facility have a specified catchment area— that is, a geographic area where the facility has direct responsibility? YES......................................................... 1 NO .......................................................... 2 DON'T’ KNOW...................................... 8 120  120 111 How many people live in the catchment area, that is, the geographic areas where this facility has direct responsibility? CATCHMENT POPULATION DON’T KNOW .............................. 99998 1.2 VULNERABLE POPULATIONS 120 Are there any groups of people living within the borders of the catchment area who may not be receiving services that they need or that you think they need? YES......................................................... 1 NO .......................................................... 2 DON’T KNOW ...................................... 8  130  130 121 Do these people belong to specific group? For example, are they from a minority tribe or ethnic group, are they poorer or more conservative, are they internally displaced? RECORD THE GROUP CHARACTERISTIC THAT MAKES THEM AT RISK. YES......................................................... 1 Specify: ______________________ DON’T KNOW ...................................... 8 MID-TERM PERFORMANCE EVALUATION OF USAID/IRAQ PRIMARY HEALTH CARE PROJECT 97 122 Does this facility have any specific practices or strategies that will improve utilization of services by these vulnerable and marginalized groups? YES......................................................... 1 Specify: ______________________ NO .......................................................... 2 123 Does this facility have any specific practices or strategies to increase staff awareness of the importance of listening to female patients, treating them with respect, and making sure that they feel their dignity is maintained? YES......................................................... 1 Specify: ______________________ NO .......................................................... 2 1.3 REFERRALS 130 Are there written guidelines for which patients should be referred to another facility and for when? YES.........................................................1 NO........................................................... 2 NEVER REFER PATIENTS .................. 3 200 131 How do referred patients most often go to the referral site? FACILITY AMBULANCE ....................1 FACILITY CALLS AMBULANCE FROM OTHER SITE TO PROVIDE TRANSPORTATION............................. 2 FAMILY RENTS VEHICLE/PRIVATE VEHICLE............................................... 3 OTHER (SPECIFY)................................ 6 DON’T KNOW...................................... 8 132 Is there a record maintained for all patients referred outside to another facility? IF YES, ASK IF THE INFORMATION IS COMPILED FOR INTERNAL OR EXTERNAL USE. YES, COMPILED...................................1 YES, IN SERVICE REGISTERS, NOT COMPILED ........................................... 2 NO........................................................... 3 NEVER REFER PATIENTS .................. 5 200 133 Is a printed referral form used? IF YES, ASK TO SEE A BLANK COPY. YES, OBSERVED ..................................1 YES, FORM NOT SEEN........................ 2 NO........................................................... 3 134 Does this facility receive information back from facilities when patients are referred? YES, AWAYS ........................................1 YES, SOMETIMES ................................ 2 NO........................................................... 3 SECTION 2 GOVERNANCE AND MANAGEMENT Now I want to ask you questions related to governance and routine systems implemented by the facility. If someone else in the facility is more familiar with the topic, please tell me so that we can arrange for me to talk with them. 2.0 LOCAL HEALTH COMMITTEE MID-TERM PERFORMANCE EVALUATION OF USAID/IRAQ PRIMARY HEALTH CARE PROJECT 98 200 Does this facility interact with a Local Health Committee (LHC)? YES......................................................... 1 NO .......................................................... 2  210 201 How long has the LHC been functional? LESS THAN 1MONTHS ....................... 1 2-3 MONTHS ......................................... 2 4 OR MORE MONTHS ......................... 3 DON’T KNOW ...................................... 8 202 When was the most recent time the Local Health Committee met? WITHIN THE PAST 1 MONTH............ 1 WITHIN THE PAST 2-3 MONTHS ...... 2 WITHIN PAST 4-6 MONTHS............... 3 MORE THAN 6 MONTHS AGO .......... 4 HAS NOT MET YET............................. 0 203 Does this facility have a copy of the Community Health Partnerships Handbook? IF YES, ASK TO SEE THE HANDBOOK. YES, OBSERVED.................................. 1 YES, REPORTED, NOT SEEN ............. 2 NO .......................................................... 3 204 Have you or any staff in this facility been trained on community partnership or the community partnership handbook? YES......................................................... 1 NO .......................................................... 2 DON’T KNOW ...................................... 8 SECTION 2.1 MANAGEMENT SYSTEMS 210 Does this facility have a management team? YES ........................................................ 1 NO .......................................................... 2 212 211 Does this facility have routine staff meetings? YES ........................................................ 1 NO .......................................................... 2 213 212 When was the most recent management team/routine staff meeting? WITHIN THE PAST 1 MONTH............ 1 WITHIN THE PAST 2-3 MONTHS ...... 2 WITHIN PAST 4-6 MONTHS............... 3 MORE THAN 6 MONTHS AGO .......... 4 213 Have any staff in the facility received the Primary Health Care Project Leadership and Management training in the past 2 years? YES......................................................... 1 NO .......................................................... 2 DON’T KNOW ...................................... 8 214 Have any staff in the facility received the Primary Health Care (PHC) Management Training in the past 2 years? YES......................................................... 1 NO .......................................................... 2 DON’T KNOW ...................................... 8 215 Does this facility have a copy of the Primary Health Care Management Handbook? IF YES, ASK TO SEE THE COPY. YES, OBSERVED.................................. 1 YES, REPORTED, NOT SEEN ............. 2 NO .......................................................... 3 MID-TERM PERFORMANCE EVALUATION OF USAID/IRAQ PRIMARY HEALTH CARE PROJECT 99 216 Does this facility receive any external supervision, e.g., from district, regional, or national offices? YES......................................................... 1 NO .......................................................... 2  220 217 When was the last time a supervisor from outside this facility came here on a supervisory visit? Was it within the past 6 months or more than 6 months ago? DO NOT INCLUDE VISITS WHERE GUESTS WERE BROUGHT OR THAT WERE FOR SUPPLIES ONLY.. WITHIN THE PAST 1 MONTH............ 1 WITHIN THE PAST 2-3 MONTHS ...... 2 WITHIN PAST 4-6 MONTHS............... 3 MORE THAN 6 MONTHS AGO .......... 4 220 220 218 Did the supervisor leave any written feedback? IF YES, ASK TO SEE THE WRITTEN FEEDBACK AND CIRCLE ALL THAT APPLY. YES, OBSERVED NOTE IN SUPERVISION REGISTER A COPY OF COMPLETED CHECKLIST ............................................................... B OTHER TYPE OF NOTE .................... C NO ......................................................... X 2.2 PERSONNEL MANAGEMENT 220 Does the facility have a written management structure that details reporting relationships? YES......................................................... 1 NO .......................................................... 2 221 Does the facility have written job descriptions? IF YES, CLARIFY IF JOB DESCRIPTIONS EXIST FOR ALL POSITIONS. ALL POSITIONS ................................... 1 SOME, NOT ALL POSITIONS............. 2 NO .......................................................... 3  223 222 May I see the job description(s) for staff nurses? YES, OBSERVED.................................. 1 REPORTED, NOT SEEN ...................... 2 NOT AVAILABLE ................................ 3 223 Does this facility maintain a written or computerized record for staff training? YES......................................................... 1 NO .......................................................... 2  230 2.3 QUALITY ASSURANCE/IMPROVEMENT SYSTEMS 230 Does this facility have guidelines for the continuous quality improvement process? IF YES, ASK TO SEE THE GUIDELINES. YES, OBSERVED.................................. 1 YES, REPORTED, NOT SEEN ............. 2 NO .......................................................... 3 231 Does this facility have a Quality Improvement Team that has been trained in continuous quality improvement? YES......................................................... 1 NO .......................................................... 2  240 232 When was the most recent QI Team meeting? WITHIN THE PAST 1 MONTH............ 1 WITHIN THE PAST 2-3 MONTHS ...... 2 WITHIN PAST 4-6 MONTHS............... 3 MORE THAN 6 MONTHS AGO .......... 4 2.4 CLIENT OPINION MID-TERM PERFORMANCE EVALUATION OF USAID/IRAQ PRIMARY HEALTH CARE PROJECT 100 240 Does this facility have any system for determining clients’ opinions or receiving feedback about the health facility or its services? YES......................................................... 1 NO .......................................................... 2  300 241 What methods are used to determine client opinions or receiving feedback READ ALL OPTIONS AND MARK YES OR NO YES NO 01 SUGGESTION BOX ............................................................................................... 1 2 02 CLIENT SURVEY FORM ...................................................................................... 1 2 03 CLIENT INTERVIEW FORM................................................................................ 1 2 04 LOCAL HEALTH COUNCIL MEETINGS............................................................ 1 2 05 OFFICIAL MEETING WITH COMMUNITY LEADERS OTHER THAN LOCAL HEALTH COUNCIL................................................................................. 1 2 06 INFORMAL DISCUSSIONS WITH CLIENT OR COMMUNITY ....................... 1 2 96 OTHER (SPECIFY)................................................................................................. 1 2 242 Is there a routine procedure for reviewing feedback from any of the above methods and reporting on clients’ opinions? YES......................................................... 1 NO .......................................................... 2 Numb er Question Result Skip SECTION 3.0 STAFFING I would like to know about staff assigned to this facility. ASK TO SEE THE PERSON WITH THE MOST INFORMATION ON NUMBERS AND CADRE OF STAFF ASSIGNED. Please tell me how many of each cadre of staff I mention who are assigned to this facility, how many are male, and how many are female. 300 CADRE/QUALIFICATION A) MALE B) FEMALE 01 Generalist (non-specialist) medical doctors 02 Specialist medical doctors 03 Non-physician clinicians/paramedical professionals 04 Nursing/midwifery professionals 05 Nursing/midwifery associate professionals 06 Pharmacist/pharmacy assistant 07 Laboratory scientists/technologists/assistants 08 All health workers not elsewhere classified 09 All health management and support workers not elsewhere classified (e.g., cleaning, maintenance, unit managers, records keepers, etc.) SECTION 3.1 FACILITY INFRASTRUCTURE Now I have some questions about the basic infrastructure of the facility, and I will ask to see some of the items I ask about. MID-TERM PERFORMANCE EVALUATION OF USAID/IRAQ PRIMARY HEALTH CARE PROJECT 101 310 Does this facility have a means for communicating outside the facility? IF YES, PLEASE CIRCLE ALL THAT EXIST AND ARE FUNCTIONAL TODAY. LAND-LINE ..........................................A CELL PHONE........................................ B SHORT-WAVE RADIO........................ C COMPUTER.......................................... D INTERNET ............................................ E 311 Is electricity available for this facility, either from the central power grid or through a generator or other source? IF YES, CLARIFY IF IT IS MOSTLY ALWAYS AVAILABLE DURING NORMAL WORKING HOURS OR IF IT IS OFTEN NOT AVAILABLE FOR MORE THAN 2 HOURS AT A TIME. YES, ROUTINELY AVAILABLE.........1 YES, AVAILABLE BUT OFTEN NOT FUNCTIONING......................................2 NO...........................................................3 312 Does the facility have a routine source of water? IF YES, CLARIFY HOW THE WATER IS SUPPLIED. YES, PIPED INTO FACILITY...............1 YES, WELL OR PUMP ON GROUNDS2 YES, DELIVERED.................................3 NO...........................................................4 3.2 HEALTH CARE WASTE MANAGEMENT 320 Does this facility have a copy of the Guidelines for PHC Infection Prevention and Waste Management? YES, OBSERVED..................................1 YES, REPORTED NOT SEEN .............. 2 NO........................................................... 3 321 Have you or any of the staff received training in PHC Infection Prevention and waste management in the past 2 years? YES.........................................................1 NO........................................................... 2 DON’T KNOW....................................... 8 322 Now I would like to ask you a few questions about waste management practices for sharps waste, such as needles or blades. How does this facility finally dispose of sharps waste (e.g., filled sharps boxes)? BURN INCINERATOR..........................1 OPEN BURNING................................... 2 DUMP WITHOUT BURNING .............. 3 REMOVE OFFSITE............................... 4 NEVER HAS SHARP WASTE.............. 5  324 323 ASK TO SEE THE PLACE USED BY THE FACILITY FOR DISPOSAL OF SHARPS WASTE AND INDICATE IF THE SITE IS PROTECTED (ANIMALS AND UNAUTHORIZED PERSONS CANNOT GAIN ACCESS) OR NOT PROTECTED. PROTECTED SITE, ..............................1 SITE NOT PROTECTED....................... 2 324 Now I would like to ask you a few questions about waste management practices for medical waste other than sharps, such as used bandages. How does this facility finally dispose of medical waste other than sharps boxes? BURN INCINERATOR..........................1 OPEN BURNING................................... 2 DUMP WITHOUT BURNING .............. 3 REMOVE OFFSITE............................... 4 NEVER HAS MEDICAL WASTE ........ 5  326 325 ASK TO SEE THE PLACE USED BY THE FACILITY FOR DISPOSAL OF MEDICAL WASTE AND INDICATE IF THE SITE IS PROTECTED (ANIMALS AND UNAUTHORIZED PERSONS CANNOT GAIN ACCESS) OR NOT PROTECTED. PROTECTED SITE, ..............................1 SITE NOT PROTECTED....................... 2 MID-TERM PERFORMANCE EVALUATION OF USAID/IRAQ PRIMARY HEALTH CARE PROJECT 102 3.3 GROUNDS AND BUILDING MAINTENANCE Now I would like to know if you or any of the staff have received the following training during the past 2 years. 330 Have you or any staff received training in PHC maintenance and management? YES.........................................................1 NO........................................................... 2 DON’T KNOW....................................... 8 331 Does this facility have a copy of the Standard Operating Procedures for PHC maintenance management? IF YES, ASK TO SEE THE COPY. YES, OBSERVED..................................1 YES, REPORTED, NOT SEEN ............. 2 NO........................................................... 3 332 Is there a budget line item for building and/or grounds maintenance? YES.........................................................1 NO........................................................... 2 333 Does the facility have an organized grounds and/or building maintenance service that has staff and a designated in-charge? YES, GROUNDS MAINTENANCE ONLY 1 YES, BUILDING MAINTENANCE ONLY 2 YES, BOTH GROUNDS AND BUILDING MAINTENANCE ................................... 3 NO........................................................... 4 335  335  335 334 Does this facility have designated maintenance personnel? YES.........................................................1 NO........................................................... 2 335 Does the facility have a routine maintenance plan for any of the building infrastructure such as water systems, electric systems, sewerage, or building repair? YES.........................................................1 NO........................................................... 2 336 Who routinely takes care of grounds and building maintenance? READ ALL RESPONSES AND CIRCLE ALL THAT APPLY. FACILITY STAFF ................................A SECTOR/DISTRICT PROVIDE STAFF.................................................... B CONTRACT WITH PRIVATE COMPANY FROM OUTSIDE .................................. C HIRE FROM OUTSIDE AS NEEDED . D OTHER (SPECIFY).............................. W 3.4 EQUIPMENT MAINTENANCE 340 Is there a budget line item for routine equipment maintenance and repair? YES.........................................................1 NO........................................................... 2 MID-TERM PERFORMANCE EVALUATION OF USAID/IRAQ PRIMARY HEALTH CARE PROJECT 103 341 Is there a schedule for inspection, testing and preventive maintenance for any major piece of equipment as guided by the manufacturer’s recommendations? IF YES, ASK “Who conducts the preventive maintenance?” READ ALL RESPONSES AND CIRCLE ALL THAT APPLY. YES, COMPANY SUPPLYING MACHINE ............................................................... A YES, CONTRACT WITH OUTSIDE PERSON/ COMPANY .......................... B FACILITY STAFF TRAINED IN MACHINE MAINTENANCE.............. C NO.......................................................... X 342 Does this facility have a process for repairing and/or replacing small equipment such as stethoscopes or blood pressure machines? YES.........................................................1 NO........................................................... 2  400 343 Does this process function well, that is, is broken equipment repaired or replaced in a timely manner? ALMOST ALWAYS ..............................1 SOMETIMES BUT NOT ALMOST ALWAYS ............................................... 2 RARELY................................................. 3 SECTION 4 CLIENT SERVICES ASK TO GO TO WHERE AMBULATORY CURATIVE CARE SERVICES ARE PROVIDED. ASK TO SPEAK WITH THE PERSON MOST FAMILIAR WITH CURATIVE CARE SERVICES AND HOW THEY ARE ORGANIZED. EXPLAIN THE SURVEY AND THAT YOU WILL BE ASKING FOR INFORMATION ABOUT THE SERVICES AND ASKING TO SEE SERVICE SITES, EQUIPMENT, AND SUPPLIES. 4.0 SERVICES OFFERED 400 Please tell me which of the following services are offered by staff in this facility, and if the service is routinely offered in the facility, in the community, or both in the facility and the community. IN FACILITY IN COMMUNITY BOTH FACILITY AND COMMUNITY NOT OFFERED 01 Curative care for adults 1 2 3 4 02 Curative care for children 1 2 3 4 03 Emergency trauma care 1 2 3 4 04 Family planning 1 2 3 4 05 Antenatal care 1 2 3 4 06 Delivery services 1 2 3 4 07 Child immunization 1 2 3 4 MID-TERM PERFORMANCE EVALUATION OF USAID/IRAQ PRIMARY HEALTH CARE PROJECT 104 401 Do you have the Handbook of Quality Standards and Operational Guidelines for Clinical Service Delivery in Primary Care Clinics available in this facility today? IF YES, ASK TO SEE THE GUIDELINES. YES, OBSERVED..................................1 YES, REPORTED NOT SEEN .............. 2 NO........................................................... 3 402 Do you have the clinical guidelines for communicable diseases in PHC available in this facility today? IF YES, ASK TO SEE THE GUIDELINES. YES, OBSERVED..................................1 YES, REPORTED NOT SEEN .............. 2 NO........................................................... 3 403 Do you have the new guidelines for non-communicable diseases? IF YES, ASK TO SEE THE GUIDELINES AND CIRCLE ALL TOPICS FOR WHICH GUIDELINES WERE OBSERVED. OBSERVED REPORTED, NOT SEEN NOT AVAILABLE 01 Asthma .............................................................................. 1 2 3 02 Diabetes ............................................................................ 1 2 3 03 Hypertension ................................... 1 2 3 404 Do you have the integrated management for childhood illness (IMCI) guidelines for the diagnosis and management of childhood illnesses available in this facility today? IF YES, ASK TO SEE THE GUIDELINES. YES, OBSERVED..................................1 YES, REPORTED NOT SEEN .............. 2 NO........................................................... 3 405 Is there a copy of the poster for patient rights in this service area? IF YES, ASK TO SEE THE POSTER. YES, OBSERVED..................................1 YES, REPORTED NOT SEEN .............. 2 NO........................................................... 3 406 Now I would like to know if you or any of the staff have received the following training during the past 2 years. YES NO DON’T KNOW 01 Quality standards and operational guidelines for clinical service delivery in primary care clinics ............... 1 2 8 02 Clinical guidelines for communicable diseases in primary care clinics .......................................................... 1 2 8 03 Clinical guidelines for asthma........................................... 1 2 8 04 Clinical guidelines for diabetes......................................... 1 2 8 05 Clinical guidelines for hypertension ................................. 1 2 8 06 Integrated management of childhood illnesses (IMCI) 1 2 8 4.1 SERVICE DELIVERY CONDITIONS Now I will be asking to see where clinical services for curative care are provided and will be asking about the service environment, available examination and treatment equipment, diagnostic tests available, and essential drugs that are available. 410 ASK TO SEE WHERE CLIENT EXAMINATIONS TAKE PLACE AND MARK IF THE FOLLOWING ARE AVAILABLE IN THE SERVICE AREA OBSERVED REPORTED NOT SEEN NOT AVAILABLE 01 Clean running water (piped, bucket with tap, or pour pitcher) 1 2 3 MID-TERM PERFORMANCE EVALUATION OF USAID/IRAQ PRIMARY HEALTH CARE PROJECT 105 02 Hand-washing soap/liquid soap 1 2 3 03 Alcohol based hand rub 1 2 3 04 Disposable latex gloves 1 2 3 05 Waste receptacle (pedal bin) with lid and plastic bin liner 1 2 3 06 Sharps container ("safety box") 1 2 3 07 Environmental disinfectant (e.g., chlorine, alcohol) 1 2 3 SECTION 4.2 EQUIPMENT AND SUPPLIES I am interested in knowing if the following basic equipment and supplies used in the provision of client services are available in the outpatient area of this facility. 420 For each equipment or item for client movement and examinations, please tell me if it is available today and functioning. ASK TO SEE THE ITEMS. AVAILABLE AND FUNCTIONING AVAILABLE NOT FUNCTIONING NOT AVAILABLE 01 Adult weighing scale 1 2 3 02 Child/infant weighing scale—100 gram gradation 1 2 3 03 Thermometer 1 2 3 04 Stethoscope 1 2 3 05 Blood pressure apparatus 1 2 3 06 Light source (flashlight acceptable) that can be aimed for looking at throat, pelvic examination, etc. 1 2 3 07 Otoscope (for looking in ears and throat) 1 2 3 08 Opthalmoscope (for looking in eye) 1 2 3 09 Oxygen delivery apparatus (tubes and masks/nasal prongs) 1 2 3 10 Filled oxygen cylinder 1 2 3 11 Spacers for inhalers (for treating asthma) 1 2 3 12 Disposable syringes with disposable needles 1 3 13 Sterilizer (functional) (autoclave or dry heat sterilizer) 1 2 3 421 Are any of the following diagnostic tests available in this facility? IF YES, CLARIFY IF THE TEST CAN BE CONDUCTED TODAY. TEST AVAILABLE TODAY TEST USUALLY AVAILABLE BUT NOT TODAY NOT AVAILABLE 01 Hemoglobin or haematocrit 1 2 3 02 Blood glucose testing 1 2 3 03 Urine glucose (dipstix) 1 2 3 04 Blood chemistries 1 2 3 05 General microscopy (microscope with slides) 1 2 3 4.3 MEDICINES AND COMMODITIES I would like to go to where medicines are stored to find out about the availability of medicines and basic drug management practices. ASK THE PHARMACIST TO CHECK IF THE FOLLOWING MEDICINES ARE AVAILABLE. ACCEPT THE RESPONSE OF THE PHARMACIST. 430 Are any of the following medicines available in this facility? IF YES, CLARIFY IF THE MEDICINE IS AVAILABLE TODAY. AVAILABLE TODAY USUALLY AVAILABLE BUT NOT TODAY NOT AVAILABLE 01 1st line treatment for child pneumonia (cotrim or amoxacillin) 1 2 3 02 2nd line antibiotic (ceftriaxone, ciprofloxacin) 1 2 3 03 Oral rehydration salts 1 2 3 MID-TERM PERFORMANCE EVALUATION OF USAID/IRAQ PRIMARY HEALTH CARE PROJECT 106 04 Zinc tablets 1 2 3 05 Vitamin-A capsules 1 2 3 06 Albendazole or mebendazole cap/tab 1 2 3 07 Folic acid tablet 1 2 3 09 Betablocker (e.g., Atenolol, captopril) 1 2 3 10 Diuretic (e.g., hydrochlorthiazide) 1 2 3 11 1st line diabetes treatment (Glibenclamide), metformin capsules,) 1 2 3 12 Insulin injectable 1 2 3 13 Glucose injection 1 2 3 14 1st line asthma treatment: (Salbutamol 0.1 mg/dose inhaler) 1 2 3 15 Paracetamol tablets 1 2 3 16 Normal saline or ringers lactate IV solution 1 2 3 17 5% dextrose IV solution 1 2 3 DRUG MANAGEMENT Now I want to know about your drug management system. 431 Are there written instructions for procurement and drug management practices in this facility? IF YES, ASK TO SEE THE WRITTEN INSTRUCTIONS. YES, OBSERVED ................................. 1 YES, REPORTED, NOT SEEN............. 2 NO.......................................................... 3 432 What type of stock keeping logistics forms do you use to manage the amount of pharmaceutical commodities received, the quantities issued, and the quantity present today? 433 ASK ABUT EACH OF THE FOLLOWING AND ASK TO SEE AN EXAMPLE WHERE THE FORM/SYSTEM IS USED. YES, OBSERVED YES, REPORTED, NOT SEEN NOT USED 01 Stock cards/bin card/ inventory control card 1 2 5 02 Stock ledger 1 2 5 03 Computer system 1 2 5 96 Other (Specify)____________ 1 2 5 95 No system at all 1 2 5 434 Is there a line item for purchasing medicines that are not available elsewhere? YES ........................................................ 1 NO.......................................................... 2 435 During the past 6 months, have you always, not always, but often, or almost never received the amount of each medicine that you ordered (or that you are supposed to routinely receive)? ALWAYS............................................... 1 OFTEN................................................... 2 ALMOST NEVER ................................. 3 NEVER .................................................. 4 SECTION 4.4 PHARMACEUTICAL STORAGE CONDITIONS 440 STORAGE CONDITIONS FOR MEDICINES: PRIMARY PHARMACY FOR OUTPATIENT MEDICINES: OBSERVE THE PLACE WHERE MEDICINES ARE STORED AND INDICATE THE PRESENCE (OR ABSENCE) OR EACH OF THE FOLLOWING CONDITIONS. YES NO MID-TERM PERFORMANCE EVALUATION OF USAID/IRAQ PRIMARY HEALTH CARE PROJECT 107 01 ARE THE MEDICINES STORED ON SHELVES OR IN CABINETS (NOT ON THE FLOOR)?............................................................................... 1 2 02 ARE THE MEDICINES PROTECTED FROM WATER FROM LEAKS? .. 1 2 03 ARE THE MEDICINES PROTECTED FROM THE SUN?.......................... 1 2 04 IS THE ROOM CLEAN OF EVIDENCE OF RODENTS (BATS, RATS) OR PESTS (ROACHES, ETC.). ..................................................................... 1 2 441 LOOK AT THE STORAGE AREA AND INDICATE YES OR NO FOR EACH ITEM BELOW YES NO 01 STORAGE AREA CAN BE LOCKED.......................................................... 1 2 02 THERE IS LIMITED ACCESS...................................................................... 1 2 03 DOORS SOLID .............................................................................................. 1 2 04 WINDOWS HAVE BARS OR SHUTTERS THAT CAN BE LOCKED...... 1 2 THANK YOUR RESPONDENT AND MOVE TO YOUR NEXT DATA COLLECTION POINT IF DIFFERENT FROM CURRENT LOCATION. SECTION 5 IMMUNIZATION SERVICES 500 Does this facility offer any immunization services? YES.........................................................1 NO........................................................... 2 501 Does this facility store any vaccines, or are all its vaccines either picked up from another facility or delivered when services are being provided? YES, STORE VACCINES...................... 1 NO STORAGE OF VACCINES............. 2  600 502 Does this facility have a refrigerator for the storage of vaccines? IF YES, AS TO SEE THE REFRIGERATOR AND CHECK IF IT IS FUNCTIONING. YES, FUNCTIONING............................1 YES, NOT FUNCTIONING................... 2 NO........................................................... 3 SECTION 6 ROUTINE COMMUNITY OUTREACH SERVICES 600 Do staff from this facility provide any routine outreach services, that is, mobile services or static services provided at village level, outside of a standing facility? YES .........................................................1 NO........................................................... 2  700 ASK TO SPEAK WITH THE PERSON MOST KNOWLEDGEABLE ABOUT COMMUNITY OUTREACH SERVICES PROVIDED BY FACILITY STAFF. 601 What type of routine outreach services are provided by this facility? READ ALL OPTIONS AND MARK YES OR NO. YES NO 01 Maternal health............................................................................... 1 2 02 Well baby/child care....................................................................... 1 2 03 Immunization.................................................................................. 1 2 04 School health services 1 2 602 Do community members routinely participate during any of the above routine outreach or community-based activities? YES, ROUTINELY ................................1 SOMETIMES, NOT ROUTINELY........ 2 NO........................................................... 3 603 Does this facility ever provide outreach services as special events, such as campaigns? YES .........................................................1 NO........................................................... 2 MID-TERM PERFORMANCE EVALUATION OF USAID/IRAQ PRIMARY HEALTH CARE PROJECT 108 604 Do community members routinely participate during any of the special campaign activities? YES, ROUTINELY ................................1 SOMETIMES, NOT ROUTINELY........ 2 NO........................................................... 3 SECTION 7 HEALTH INFORMATION RECORDS AND REPORTING Number QUESTION RESPONSE SKIP Now I want to know about service statistics that are routinely submitted to an authority outside of the facility. By service statistics I mean the routine Ministry report on numbers of clients receiving different services. 700 Does this facility have a designated person, such as an M&E officer, data manager, who is responsible for preparing the routine report on service statistics that is sent to authorities outside the facility? YES ......................................................1 NO DEDICATED PERSON................ 2 701 Who is responsible for compiling this report? NOTE: IF EVERY UNIT COMPILES THEIR OWN STATISTICS WE WANT TO KNOW WHO COMPILES DIFFERENT UNIT REPORTS INTO THIS ONE OFFICIAL REPORT. DATA MANAGER/HMIS PERSON .....1 OTHER TECHNICAL STAFF NO SPECIAL DATA TRAINING ................................2 OTHER NON-TECHNICAL STAFF NO SPECIAL DATA TRAINING ...............3 OTHER _______________________.....6 Now I would like to talk with the person or persons most familiar with maintaining client records for this facility. ASK TO SPEAK WITH THE PERSON AND EXPLAIN THE ASSESSMENT TO THE PERSON. 702 What types of data and records are maintained for routine reporting for this facility? READ EACH RESPONSE. YES NO 01 Paper copies of routine monthly or quarterly health information system (HIS) reports. 1 2 02 Paper copies of facility service or department reports. 1 2 03 Computer files for some types of facility service information. 1 2 04 Computerized monthly service statistics database for monthly HIS reports. 1 2 05 Are there paper client charts? 1 2 06 Are there computerized client charts? 1 2 07 Are there service registers with daily client information? 1 2 08 Has the new paper client record been introduced? 1 2 09 Have the new patient registers been introduced? 1 2 ASK TO SEE WHERE HMIS REPORTS AND DOCUMENTS ARE STORED. 703 DESCRIBE THE SITE/LOCATION WHERE HMIS REPORTS AND COMPILED DATA ARE STORED. SEPARATE ROOM(S)........................ 1 CABINET(S) ....................................... 2 SHARED ROOM(S) NO CABINET... 3 NO SPECIFIC SITE ............................ 4 710 MID-TERM PERFORMANCE EVALUATION OF USAID/IRAQ PRIMARY HEALTH CARE PROJECT 109 704 Is access to the HMIS reports/data records files storage area(s) limited? That is, only authorized persons can gain ready access to the records? YES, ALL ...............................................1 YES, SOME, NOT ALL......................... 2 NO........................................................... 3 705 Can the HMIS reports and data records/files area(s) be locked? YES, ALL ...............................................1 YES, SOME, NOT ALL......................... 2 NO........................................................... 3 SECTION 7.1 DATA QUALITY 710 Is there any routine and systematic process within the facility for checking the quality of data compiled for routine monthly or quarterly reports on services? YES.........................................................1 NO........................................................... 2 END 711 Is there a written policy for data quality checking or written guideline for how to carry out data quality checking? YES.........................................................1 NO........................................................... 2 713 712 ASK TO SEE A COPY OF THE POLICY OR METHODOLOGY GUIDELINES. OBSERVED ...........................................1 REPORTED, NOT SEEN....................... 2 713 Is there any written documentation of the data quality system being implemented? YES.........................................................1 NO........................................................... 2 Annex 18B. Health Worker Survey FACILITY IDENTIFICATION NAME OF FACILITY__________________________________________ DISTRICT CODE.................................................................................. FACILITY CODE.................................................................................. PROVIDER INTERVIEW NUMBER PHC STAFF INTERVIEW IINTRODUCE YOURSELF, BRIEFLY EXPLAIN WHY YOU WANT TO INTERVIEW THE PERSON AND THEN READ THE FOLLOWING: My name is ___________________ and I am part of an independent team assessing the Primary Health Care Project being implemented by the University Research Corporation and the Ministry of Health and funded by the Ministry of Health and USAID. I would like to ask you some questions about your work and training you have received to support your work. I will also be asking a few questions to better understand how project activities have affected your work and will ask for your thoughts on ways to improve the effectiveness of activities to improve primary health care services. We would like to interview a series of various staff members over the course of the day. The results of this interview will remain confidential. You do not have to participate; however, your participation will help us to better understand more about the extent to which the activities of the project have reached this level in the health system, and how they have affected your work. MID-TERM PERFORMANCE EVALUATION OF USAID/IRAQ PRIMARY HEALTH CARE PROJECT 110 Do you agree to participate? Signature of Interview Indicates Consent Statement was read and the Provider agreed to the interview. 01 WHAT IS THE GENDER OF THE RESPONDENT? MALE ....................................................................... 1 FEMALE................................................................... 2 02 What is your current qualification or professional cadre? PHYSICIAN .......................................................... 01 NURSE ................................................................... 02 MIDWIFE .............................................................. 03 LABORATORY SCIENCE.................................... 04 PHARMACIST....................................................... 05 NON-TECHNICAL HEALTH PROFESSIONAL . 06 HEALTH INFORMATION SPECIALIST............. 07 OTHER (SPECIFY)................................................ 96 03 Either as a part of the above education or in addition to the above education, do you have a higher education professional health degree? IF YES, IN WHAT AREA? CIRCLE ALL THAT APPLY. MASTERS PUBLIC HEALTH ............................... A MASTERS OTHER HEALTH SCIENCE................ B DIPLOMA HEALTH SCIENCE.............................. C PHD HEALTH SCIENCE ....................................... D OTHER ................................................................... W NO............................................................................. Z 04m 04Y How long have you worked in this facility? THE MONTHS ARE NOT SO IMORTANT IF THE PERSON HAS WORKED HERE MORE THAN ONE YEAR SO IF THEY ARE NOT CERTAIN, ESTIMATE. MONTHS YEARS Now I would like to ask you about specific training courses that have been provided during the past few years for improving Primary Heath Care management and services. 10 Have you received the Primary Heath Care Project training in Leadership and Management? IF YES, ASK When did you first receive the training? YES, 2013 ................................................................. 1 YES, 2012 ................................................................. 2 YES, 2011 ................................................................. 3 NO............................................................................. 5 15 11 Have you received refresher training on the same subject since the first training? YES........................................................................... 1 NO............................................................................. 2 MID-TERM PERFORMANCE EVALUATION OF USAID/IRAQ PRIMARY HEALTH CARE PROJECT 111 12 Which of the following best describes your opinion of the leadership and management training? READ ALL RESPONSES AND CIRCLE MOST APPLICABLE. EXCELLENT............................................................ 1 GOOD ....................................................................... 2 FAIR ......................................................................... 3 POOR........................................................................ 4 13 Which of these other descriptions of the leadership and management training content best describes your opinion? READ ALL RESPONSES AND CIRCLE MOST APPLICABLE. PRACTICAL, EASY TO APPLY TRAINING TO CURRENT WORK............................................ 1 PRACTICAL, DIFFICULT TO APPLY TRAINING TO CURRENT WORK ....................... 2 NOT PRACTICAL ................................................... 3 OTHER (SPECIFY).................................................. 6 14 Have you been able to apply any of the leadership and management training to your current work? YES........................................................................... 1 NO............................................................................. 2 YES, 2012 ................................................................. 2 YES, 2011 ................................................................. 3 NO............................................................................. 5 15 Have you received the Primary Heath Care Project training in Primary Health Care Clinic Management? IF YES, ASK: When did you first receive the training? YES, 2013 ................................................................. 1 YES, 2012 ................................................................. 2 YES, 2011 ................................................................. 3 NO............................................................................. 5 21 16 Have you received refresher training on the same subject since the first training? YES........................................................................... 1 NO............................................................................. 2 17 Which of the following best describes your opinion of the Primary Heath Care Project management training? READ ALL RESPONSES AND CIRCLE MOST APPLICABLE. EXCELLENT............................................................ 1 GOOD ....................................................................... 2 FAIR ......................................................................... 3 POOR........................................................................ 4 18 Which of these other descriptions of the Primary Health Care management training content best describes your opinion? READ ALL RESPONSES AND CIRCLE MOST PRACTICAL, EASY TO APPLY TRAINING TO CURRENT WORK ........................ 1 MID-TERM PERFORMANCE EVALUATION OF USAID/IRAQ PRIMARY HEALTH CARE PROJECT 112 APPLICABLE. PRACTICAL, DIFFICULT TO APPLY TRAINING TO CURRENT WORK ........................ 2 NOT PRACTICAL ................................................... 3 OTHER (SPECIFY).................................................. 6 19 Have you been able to apply any of the Primary Health Care management training to your current work? YES........................................................................... 1 NO............................................................................. 2 20 Can you provide any examples of changes in the Primary Health Care management that have taken place over the past 2 years that you feel improved your working situation or the functioning of the facility? PROBE FOR CHANGES RELATED TO: Availability of medicines? Availability of functional equipment? Organization of services? Support from supervisors or managers? Client satisfaction? 21 Have you received the Primary Heath Care Project training in facility and equipment maintenance management? IF YES, ASK: When did you first receive the training? YES, 2013 ................................................................. 1 27 22 Have you received refresher training on the same subject since the first training? YES........................................................................... 1 NO............................................................................. 2 23 Which of the following best describes your opinion of the facility and equipment maintenance management? READ ALL RESPONSES AND CIRCLE MOST APPLICABLE. EXCELLENT............................................................ 1 GOOD ....................................................................... 2 FAIR ......................................................................... 3 POOR........................................................................ 4 24 Which of these other descriptions of the facility and equipment maintenance management training content best describes your opinion? READ ALL RESPONSES AND CIRCLE MOST APPLICABLE. PRACTICAL, EASY TO APPLY TRAINING TO CURRENT WORK............................................. 1 PRACTICAL, DIFFICULT TO APPLY TRAINING TO CURRENT WORK ........................ 2 NOT PRACTICAL ................................................... 3 OTHER (SPECIFY).................................................. 6 MID-TERM PERFORMANCE EVALUATION OF USAID/IRAQ PRIMARY HEALTH CARE PROJECT 113 25 Have you been able to apply any of the Facility and equipment maintenance management training to your current work? YES........................................................................... 1 NO............................................................................. 2 26 Can you provide any examples of changes in the maintenance and repair within the facility that have taken place over the past 2 years that you feel improved your working situation or the functioning of the facility? PROBE FOR CHANGES RELATED TO: Efficiency of system for repairing equipment. Efficiency of system for repairing building problems. 27 Have you received the Primary Heath Care Project training in community partnerships? IF YES, ASK: When did you first receive the training? YES, 2013 ................................................................. 1 YES, 2012 ................................................................. 2 YES, 2011 ................................................................. 3 NO............................................................................. 5 33 28 Have you received refresher training on the same subject since the first training? YES........................................................................... 1 NO............................................................................. 2 29 Which of the following best describes your opinion of the community partnership training? READ ALL RESPONSES AND CIRCLE MOST APPLICABLE. EXCELLENT............................................................ 1 GOOD ....................................................................... 2 FAIR ......................................................................... 3 POOR........................................................................ 4 30 Which of these other descriptions of the community partnership training content best describes your opinion? READ ALL RESPONSES AND CIRCLE MOST APPLICABLE. PRACTICAL, EASY TO APPLY TRAINING TO CURRENT WORK............................................. 1 PRACTICAL, DIFFICULT TO APPLY TRAINING TO CURRENT WORK ........................ 2 NOT PRACTICAL ................................................... 3 OTHER (SPECIFY).................................................. 6 31 Have you been able to apply any of the community partnership training to your current work? YES........................................................................... 1 NO............................................................................. 2 32 Can you provide any examples of changes in the attitudes of community members in the past 2 years? PROBE FOR CHANGES RELATED TO: Community participation with community health activities supported by the facility Client satisfaction MID-TERM PERFORMANCE EVALUATION OF USAID/IRAQ PRIMARY HEALTH CARE PROJECT 114 Community or client feedback resulting in changes in the facility infrastructure or heath service systems 33 Have you received the Primary Heath Care Project training in infection prevention and waste management? IF YES, ASK: When did you first receive the training? YES, 2013 ................................................................. 1 YES, 2012 ................................................................. 2 YES, 2011 ................................................................. 3 NO............................................................................. 5 40 34 Have you received refresher training on the same subject since the first training? YES........................................................................... 1 NO............................................................................. 2 35 Which of the following best describes your opinion of the PHC infection prevention and waste management training? READ ALL RESPONSES AND CIRCLE MOST APPLICABLE. EXCELLENT............................................................ 1 GOOD ....................................................................... 2 FAIR ......................................................................... 3 POOR........................................................................ 4 36 Which of these other descriptions of the PHC infection prevention and waste management training content best describes your opinion? READ ALL RESPONSES AND CIRCLE MOST APPLICABLE. PRACTICAL, EASY TO APPLY TRAINING TO CURRENT WORK............................................. 1 PRACTICAL, DIFFICULT TO APPLY TRAINING TO CURRENT WORK ........................ 2 NOT PRACTICAL ................................................... 3 OTHER (SPECIFY).................................................. 6 37 Have you been able to apply any of the PHC infection prevention and waste management training to your current work? YES........................................................................... 1 NO............................................................................. 2 38 Can you provide any examples of changes in the infection prevention or waste management situation for the facility that have taken place over the past 2 years that you feel improved your working situation or the functioning of the facility? PROBE FOR CHANGES RELATED TO: Routine availability of infection prevention supplies (hand washing soap; water; disinfectant) Use of sharps boxes for needle disposal Final disposal of waste Individual practices for infection prevention 40 Now I want to ask you about services that you personally provide and aspects for which you personally are responsible. Then I will ask you about related training. Do you personally provide any clinical services? YES............................................... 1 NO................................................. 2 42 41 01 Do you personally provide antenatal care? YES...............................................1 NO................................................. 2 02 Do you personally provide delivery services? YES............................................... 1 MID-TERM PERFORMANCE EVALUATION OF USAID/IRAQ PRIMARY HEALTH CARE PROJECT 115 NO.................................................2 03 Do you personally provide family planning services? YES...............................................1 NO................................................. 2 04 Do you personally provide curative care for adults? YES...............................................1 NO................................................. 2 05 Do you personally provide curative care for children? YES...............................................1 NO................................................. 2 06 Do you personally provide emergency and/or trauma care? YES...............................................1 NO................................................. 2 07 Do you personally provide clinical services in the community through outreach? YES...............................................1 NO................................................. 2 42 Have you received any training in clinical services in the last two years? YES............................................... 1 NO................................................. 2 47 01 Have you received the PHCPI training in Primary Health Care clinical service guidelines in the last two years? YES...............................................1 NO................................................. 2 02 Have you received the PHCPI training in integrated management of childhood illness in the last two years? YES............................................... 1 NO................................................. 2 43 03 Did you receive a post-training follow-up visit for Integrated Management of Childhood Illness in your clinic? YES...............................................1 NO................................................. 2 43 NON-COMMUNICABLE DISEASES 01 Have you received any training in the diagnosis and management of diabetes in the last two years? YES...............................................1 NO................................................. 2 02 Have you received any training in the diagnosis and management of hypertension in the last two years? YES...............................................1 NO................................................. 2 03 Have you received any training in the diagnosis and management of asthma in the last two years? YES...............................................1 NO................................................. 2 44 Have you received the PHCPI training in clinical guidelines for communicable diseases in the last two years? YES...............................................1 NO................................................. 2 45 Have you received the PHCPI training in the early management and life support for trauma patients in the last two years? YES............................................... 1 NO................................................. 2 46 IF ANY TRAINING ON CLINICAL SERVICES HAS BEEN PROVIDED, ASK THE FOLLOWING: Do you feel you have been able to fully implement the new guidelines for clinical services? YES, MOSTLY ............................1 YES, SOMEWHAT...................... 2 NO................................................. 3 47 What do you think are the main barriers to implementing the clinical guidelines? CIRCLE ALL THAT APPLY. LACK OF MEDICINES.......................................... A LACK OF EQUIPMENT.......................................... B LACK OF OTHER SUPPLIES................................................................ C LACK OF SPACE ................................................... D LACK OF SUPPORTIVE SUPERVISION OR MENTORING ................................................... E INSUFFICIENT SKILL............................................ F MID-TERM PERFORMANCE EVALUATION OF USAID/IRAQ PRIMARY HEALTH CARE PROJECT 116 D AND G (LACK OF SPACE) ARE DUPLICATE LACK OF SPACE ................................................... G LACK OF TIME...................................................... H OTHER (SPECIFY)................................................. W 48 Can you provide any examples of changes in the quality of clinical services that have taken place over the past 2 years that you feel improved your working situation or the functioning of the facility? PROBE FOR CHANGES RELATED TO: 1) Availability of medicines 2) Availability of functional equipment 3) Organization of services 4) Support from supervisors or managers 5) Client satisfaction 6) Client examinations are more thorough 7) Skill level for diagnosis and treatment is improved 50 Do you personally supervise any clinical staff? YES...............................................1 NO................................................. 2 53 51 Have you received training in supportive supervision, using a checklist? IF YES, ASK WHEN. YES, 2013.....................................1 YES, 2012..................................... 2 YES, 2011..................................... 3 NO................................................. 5 52 Do you use a checklist when you supervise the other clinical staff? IF YES, ASK TO SEE THE CHECKLIST. Yes, observed…………………1 Yes, not observed…………… 2 No………………………………3 53 Do you have any responsibilities for drug or commodity management or distribution? YES...............................................1 NO................................................. 2 60 01 Have you received any training in pharmaceutical supply management? IF YES, ASK: When did you most recently receive the training? YES, 2013.....................................1 YES, 2012..................................... 2 YES 2011...................................... 3 NO................................................. 5 02 Have you received other training on pharmaceutical systems (how to request medicines, good storage practices, how to maintain records) in the past 2 years? YES...............................................1 NO................................................. 2 60 Now I want to ask you about other training related to quality of services. If you have received the training, please tell me when you most recently received the training. TRAINING TO STRENGTHEN SERVICES AND MANAGEMENT YES 2013 YES 2012 YES 2011 NO 01 Training in continuous quality improvement 1 2 3 5 02 Training in coaching the quality improvement process 1 2 3 5 03 Training on the Family Health Approach 1 2 3 5 04 Did you receive training for health promotion and social mobilization or behavior change communication? 1 2 3 5 05 Did you receive training in completing health records and registers or health reports? 1 2 3 5 MID-TERM PERFORMANCE EVALUATION OF USAID/IRAQ PRIMARY HEALTH CARE PROJECT 117 06 Did you receive training in data quality? 1 2 3 5 07 Did you receive training in using health information for decision making? 1 2 3 5 70 Now I would like to ask you some questions about supervision you have personally received. Do you receive technical support or supervision from a supervisor in this facility? IF YES, ASK: When was the most recent time YES, IN THE PAST 3 MONTHS. ............................1 YES, IN THE PAST 4-6 MONTHS. ........................ 2 YES, IN THE PAST 7-12 MONTHS........................ 3 YES, MORE THAN 12 MONTHS AGO.................. 4 NO. ........................................................................... 5 71 Do you receive technical support or supervision from outside the facility? This might be someone from district, provincial, or central level? IF YES, ASK: When was the most recent time? YES, IN THE PAST 3 MONTHS. ............................1 YES, IN THE PAST 4-6 MONTHS. ........................ 2 YES, IN THE PAST 7-12 MONTHS........................ 3 YES, MORE THAN 12 MONTHS AGO.................. 4 NO. ........................................................................... 5 72 IS ‘1’ “SUPERVISED IN THE PAST 3 MONTHS” CIRCLED FOR EITHER Q71 OR Q72? YES ...........................................................................1 NO ............................................................................. 2 80 73 The last time you were personally supervised, did your supervisor do any of the following: YES NO 01 Check your records or reports? 1 2 02 Observe your work? 1 2 03 Use a checklist when observing your work? 1 2 04 Provide any feedback (either positive or negative) on your performance? 1 2 05 Give you verbal or written feedback that you were doing your work well? 1 2 06 Provide updates on administrative or technical issues related to your work? 1 2 07 Discuss problems you have encountered? 1 2 74 Can you provide any examples of changes in the supervision that you have provided or received over the past 2 years that you feel improved your working situation or the functioning of the facility? PROBE FOR CHANGES RELATED TO: 1) Supervision being more organized, systematic 2) Advantages of using checklist 3) Problems with using checklist 4) Feedback being provided to health workers 80 Do you have a written job description for your current job or position in this facility? YES ...........................................................................1 NO ............................................................................. 2 81 Have you heard about patient rights? IF YES, ASK: How did you learn about patient rights? DO NOT READ RESPONSES, BUT CIRCLE ALL THAT ARE SPONTANEOUSLY MENTIONED TRAINING...............................................................A SAW POSTER ......................................................... B SUPERVISOR DISCUSSED ................................... C JUST HEARD INFORMALLY ............................... D MID-TERM PERFORMANCE EVALUATION OF USAID/IRAQ PRIMARY HEALTH CARE PROJECT 118 OTHER (SPECIFY) ................................................W THANK YOUR RESPONDENT. MID-TERM PERFORMANCE EVALUATION OF USAID/IRAQ PRIMARY HEALTH CARE PROJECT 119 Annex 18C. Client Exit Interview 01 FACILITY NAME _______________________________________ 02 FACILITY NUMBER ..................................................................... 03 DISTRICT CODE ........................................................................... 04 INTERVIEWER NAME________________________________ 05 INTERVIEWER NUMBER ........................................................... 06 CLIENT INTERVIEW NUMBER ................................................. 07 DATE (a) (b) (c) 2013 DAY MONTH YEAR READ TO CLIENT: Hello, I am___________. We are implementing a survey for the Ministry of Health which supports health services. We would like to interview you to better understand the experiences of people who receive care in this facility and your opinion of the services received. The information you provide will be used to improve services to your community. All information you provide will remain confidential. Please know that whether you decide to allow this interview or not is completely voluntary and will not affect services you receive during any future visit. You may refuse to answer any question, and you may stop the interview at any time. MID-TERM PERFORMANCE EVALUATION OF USAID/IRAQ PRIMARY HEALTH CARE PROJECT 120 Do you have any questions for me? Do I have your permission to continue with the interview? Signature of interviewer indicates that the consent form was read and the client agreed to participate. Now I am going to ask you some questions about the services you received today. I would like to have your honest opinion about the things that we will talk about. 7 Which service did you receive? Antenatal/postnatal care ......................... 1 Delivery services.................................... 2 Family planning...................................... 3 Child immunization ............................... 4 Maternal and child health ....................... 5 Dentistry ................................................. 6 Ophthalmology ....................................... 7 Consultation for child curative care........ 8 Consultation for adult curative care........ 9 Breast cancer ........................................ 10 Communicable diseases........................ 11 Chronic diseases................................... 12 Continuous availability of drugs/popular clinic that distributes drug supply.................. 13 Laboratory ............................................ 14 X-ray..................................................... 15 Diagnostic (lab & x-ray)....................... 16 Ambulance............................................ 17 Minor surgery ....................................... 18 Referral services................................... 19 Availability of inspection ..................... 20 Pediatric specialist................................ 21 Availability of specialists ..................... 22 Modernization & development............. 23 MID-TERM PERFORMANCE EVALUATION OF USAID/IRAQ PRIMARY HEALTH CARE PROJECT 121 Injections.............................................. 24 OTHER (SPECIFY) ............................. 96 DON’T KNOW .................................... 98 8 Did you receive any medicines or other items from the facility to take at home? YES, ALL MEDS................................... 1 YES, SOME MEDS ONLY ................... 2 NO SAID TO BUY FROM OUTSIDE.. 3 NO PRESCRIPTION/ TESTS ONLY.... 4 10 Did a doctor or nurse at the facility explain to you how to give/take these medicines at home? YES ........................................................ 1 NO .......................................................... 2 PHARMACIST EXPLAINED ............... 3 DON’T KNOW ...................................... 8 11 Did you receive any diagnostic tests today such as blood, urine, or x-ray? YES ........................................................ 1 NO .......................................................... 2 13 12 Did a health worker discuss the results of the tests with you? YES ........................................................ 1 NO .......................................................... 2 RESULTS NOT OUT YET.................... 3 13 Do you have a health card where the services you received are documented? YES ........................................................ 1 NO .......................................................... 2 CLIENT SATISFACTION 14 How long did you wait between the time you arrived at this facility and the time you were able to see a provider for the consultation? MINUTES SAW PROVIDER IMMEDIATELY..000 DON’T KNOW...................................998 Now I am going to ask about some common problems clients have at health facilities. As I mention each one, please tell me whether any of these were problems or not problems for you today, and if so, whether they were major or minor problems for you. 15 COMMON PROBLEMS TYPE OF PROBLEM NO PROBLEM DON’T KNOW MAJOR MINOR 01 Time you waited to see a provider. ………………………………………………… 1 2 3 8 02 Ability to discuss problems or concerns about your health or condition. ………………………………………………… 1 2 3 8 03 Amount of explanation you received about any questions or problems you had or the care or treatment provided. ………………………………………………… 1 2 3 8 04 The care or treatment provided. 1 2 3 8 MID-TERM PERFORMANCE EVALUATION OF USAID/IRAQ PRIMARY HEALTH CARE PROJECT 122 ………………………………………………… 05 Privacy from having others see the examination. ………………………………………………… 1 2 3 8 06 Privacy from having others hear your consultation discussion. ………………………………………………… 1 2 3 8 07 Availability of medicines at this facility. ………………………………………………… 1 2 3 8 08 Availability of diagnostic tests such as blood, urine, or x-ray. ………………………………………………… 1 2 3 8 09 The cleanliness of the facility. ………………………………………………… 1 2 3 8 10 State of repair/condition of the facility. ………………………………………………… 1 2 3 8 11 How the doctor or nurse treated you during examination. ………………………………………………… 1 2 3 8 12 How the staff treated and spoke with you during the visit. ………………………………………………… 1 2 3 8 16 Have you visited this facility before? YES.........................................................1 NO........................................................... 2 19A 17 Have you noticed any difference in the services or the way that health workers treat you over the past year? YES.........................................................1 NO........................................................... 2 19A 18 For each of the following, please tell me if you have noticed any change in this facility, or if others in the community have discussed changes in the facility over the past year. (Instruction: After reading each statement repeat the words better /no change /worse.) BETTER NO CHANGE WORSE DON’T KNOW 01 Time you waited to see a provider 1 2 3 8 02 Ability to discuss problems or concerns about your health or condition. 1 2 3 8 03 Amount of explanation you received about any questions or problems you had or the care or treatment provided. 1 2 3 8 04 The care or treatment provided. 1 2 3 8 05 Privacy from having others see the examination. 1 2 3 8 06 Privacy from having others hear your consultation discussion. 1 2 3 8 MID-TERM PERFORMANCE EVALUATION OF USAID/IRAQ PRIMARY HEALTH CARE PROJECT 123 07 Availability of medicines at this facility. 1 2 3 8 08 Availability of diagnostic tests such as blood, urine, or x-ray. 1 2 3 8 09 The cleanliness of the facility. 1 2 3 8 10 State of repair/condition of the facility. 1 2 3 8 11 How the doctor or nurse treated you during examination. 1 2 3 8 12 How the staff treated and spoke with you during the visit. 1 2 3 8 19A Is this the closest health facility to your home? YES.........................................................1 NO........................................................... 2 DON’T KNOW.....................................98 20 20 19B What was the main reason you did not go to the facility nearest to your home? IF CLIENT MENTIONS SEVERAL REASONS, PROBE FOR THE MOST IMPORTANT, OR MAIN REASON. INCONVENIENT OPERATING HOURS................................................... 1 BAD MEDICAL REPUTATION........... 2 DON’T LIKE PERSONNEL.................. 3 NO MEDICINE...................................... 4 PREFERS TO REMAIN ANONYMOUS ...................................... 5 IT IS MORE EXPENSIVE..................... 6 NO DOCTOR......................................... 7 NO FEMALE DOCTOR........................ 8 TREAT MY COMMUNITY BAD......... 9 TREAT WOMEN BAD ....................... 10 WAS REFERRED................................ 11 OTHER (SPECIFY) ............................. 96 DON’T KNOW .................................... 98 20 In general, which of the following statements best describes your opinion of the services you received at this facility today READ ALL STATEMENTS, CIRCLE ONLY ONE 1) I AM VERY SATISFIED WITH THE SERVICES I RECEIVED IN FACILITY…... 1 2) I AM MORE OR LESS SATISFIED WITH THE SERVICES I RECEIVED………..2 3) I AM NOT SATISFIED WITH THE SERVICED I RECEIVED ……………………...3 Why are you not satisfied with the services you received?……………………………………… …………………………………………………………………………………………………… MID-TERM PERFORMANCE EVALUATION OF USAID/IRAQ PRIMARY HEALTH CARE PROJECT 124 21 Will you recommend this health facility to a friend or family member? WHY NOT ……………………………………… …………………………………………………… YES.........................................................1 NO .......................................................... 2 NO CHOICE........................................... 3 DON’T KNOW....................................... 8 Patient Rights 22 Did you hear of patient rights? YES.........................................................1 NO .......................................................... 2 25 23 How did you come to know about patient’s rights? MARK ALL MENTIONED SEEN POSTERS /PAMPHLET .............1 HEALTH WORKER TOLD ME............ 2 HEALTH PROVIDER TOLD ME......... 3 NGO/CIVIL SOCIETY .......................... 4 COMMUNITY LEADER....................... 5 TV / RADIO ........................................... 6 NEWSPAPER......................................... 7 OTHER (SPECIFY) ............................. 96 DON’T KNOW..................................... 98 24 What are the various patients’ rights? MARK ALL MENTIONED RIGHT TO KNOW ................................1 TREATMENT AND CARE................... 2 MAKE OWN DECISIONS ON TREATMENT ........................................ 3 PRIVACY OF PERSONAL INFO......... 4 REFERAL............................................... 5 SECOND OPINION............................... 6 VISUAL PRIVACY ............................... 7 AUDITORY PRIVACY ......................... 8 CHOICE OF PROVIDER....................... 9 OTHER (SPECIFY) ............................. 96 DON’T KNOW..................................... 98 25 Do you know what services are provided in this health facility? Antenatal/postnatal care..........................1 Delivery services..................................... 2 MID-TERM PERFORMANCE EVALUATION OF USAID/IRAQ PRIMARY HEALTH CARE PROJECT 125 MARK ALL MENTIONED PROBE TO IDENTIFY ALL SERVICES THE CLIENT KNOWS (AND WHAT ELSE) Family planning ......................................3 Child immunization ............................... 4 Maternal and child health........................ 5 Dentistry ................................................. 6 Ophthalmology ....................................... 7 Consultation for child curative care........ 8 Consultation for adult curative care........ 9 Breast cancer......................................... 10 Communicable diseases........................ 11 Chronic diseases ................................... 12 Continuous availability of drugs / popular clinic that distributes drug supply................................................... 13 Laboratory............................................. 14 X-ray..................................................... 15 Diagnostic (lab & x-ray)....................... 16 Ambulance............................................ 17 Minor surgery ....................................... 18 Referral services ................................... 19 Availability of inspection...................... 20 Pediatric specialist ................................ 21 Availability of specialists...................... 22 Modernization & development ............. 23 Injections............................................... 24 OTHER (SPECIFY) ............................. 96 DON’T KNOW..................................... 98 26 Do these services cover the health needs of your family and community? YES.........................................................1 NO .......................................................... 2 29 27 If no, specify what health services need to be added to the facility services? Antenatal/postnatal care..........................1 MID-TERM PERFORMANCE EVALUATION OF USAID/IRAQ PRIMARY HEALTH CARE PROJECT 126 MARK ALL MENTIONED AND SPECIFY ALL NOT MARKED. SPECIFY …………………………………………………… …………………………………………………… …………………………………………………… …………………………………………………… …………………………………………………… …………………………………………………… …………………………………………………… Delivery services.....................................2 Family planning ...................................... 3 Child immunization ............................... 4 Maternal and child health........................ 5 Dentistry ................................................. 6 Ophthalmology ....................................... 7 Consultation for child curative care........ 8 Consultation for adult curative care........ 9 Breast cancer......................................... 10 Communicable diseases........................ 11 Chronic diseases ................................... 12 Continuous availability of drugs/popular clinic that distributes drug supply ........ 13 Laboratory............................................. 14 X-ray..................................................... 15 Diagnostic (lab & x-ray)....................... 16 Ambulance............................................ 17 Minor surgery ....................................... 18 Referral services ................................... 19 Availability of inspection...................... 20 Pediatric specialist ................................ 21 Availability of specialists...................... 22 Modernization & development ............. 23 Injections............................................... 24 OTHER (SPECIFY) ............................. 96 DON’T KNOW..................................... 98 28 If no, why are these health services needed? MARK ALL MENTIONED AND SPECIFY ALL NOT MARKED. Medication cost is high ...........................1 Reason 2.................................................. 2 ................................................................ 3 MID-TERM PERFORMANCE EVALUATION OF USAID/IRAQ PRIMARY HEALTH CARE PROJECT 127 SPECIFY …………………………………………………… …………………………………………………… …………………………………………………… …………………………………………………… ................................................................4 ................................................................ 5 OTHER (SPECIFY) ............................. 96 DON’T KNOW..................................... 98 29 Is the manner in which the available services are provided appropriate to you, and your family? YES.........................................................1 NO .......................................................... 2 32 30 Why is it not appropriate? MARK ALL MENTIONED AND SPECIFY ALL NOT MARKED. SPECIFY …………………………………………………… …………………………………………………… …………………………………………………… …………………………………………………… …………………………………………………… Bad physician treatment..........................1 Bad nurse treatment ................................ 2 Bad physician & nurse treatment............ 3 Bad staff treatment.................................. 4 Need to smile/treatment should be better 5 Immunization time and place not appropriate .............................................. 6 Medication not available......................... 7 ................................................................ 8 OTHER (SPECIFY) ............................. 96 DON’T KNOW..................................... 98 31 How could services become more appropriate? MARK ALL MENTIONED AND SPECIFY ALL. NOT MARKED SPECIFY …………………………………………………… …………………………………………………… …………………………………………………… …………………………………………………… Availability of chronic disease medication............................................... 1 Make medication available ..................... 2 Specialist physicians............................... 3 Pediatricians............................................ 4 Obstetrician/gynecologist ....................... 5 Redesign the reception and waiting procedures............................................... 6 Better treatment....................................... 7 More counseling, explanation & advice 8 OTHER (SPECIFY) ............................. 96 DON’T KNOW..................................... 98 32 Do you have any suggestions on making the services better meet your needs and those of your Availability of chronic disease medication...............................................1 MID-TERM PERFORMANCE EVALUATION OF USAID/IRAQ PRIMARY HEALTH CARE PROJECT 128 family and community and delivered in an appropriate manner? MARK ALL MENTIONED AND SPECIFY ALL NOT MARKED SPECIFY …………………………………………………… …………………………………………………… Make medication available .....................2 Specialist physicians............................... 3 Pediatricians............................................ 4 Obstetrician/gynecologist ....................... 5 Redesign the reception and waiting procedures............................................... 6 Better treatment....................................... 7 More counseling, explanation & advice 8 OTHER (SPECIFY) ............................. 96 DON’T KNOW..................................... 98 33 Do you have any suggestions on making the services more appropriate for women and girls and better meet their needs? MARK ALL MENTIONED AND SPECIFY ALL NOT MARKED SPECIFY …………………………………………………… …………………………………………………… Female physician ....................................1 Method 2................................................. 2 ................................................................ 3 ................................................................ 4 ................................................................ 5 ................................................................ 6 OTHER (SPECIFY) ............................. 96 DON’T KNOW..................................... 98 Now I am going to ask you some questions about yourself. I would like to have your honest responses as this information will help to improve services in general. 34 Sex of respondent MALE.....................................................1 FEMALE ................................................ 2 35 Do not read but record if you already know the identity of the interviewee. (Need to document if the interviewee is from a vulnerable group in Iraqi context.) IDP..........................................................1 HANDICAPED ...................................... 2 RELIGIOUS MINORITY ...................... 3 ETHNIC MINORITY WITHIN THIS COMMUNITY ....................................... 4 ................................................................ 5 ................................................................ 6 OTHER (SPECIFY) ............................. 96 36 How old were you at your last birthday? (respondent) AGE LAST BIRTHDAY MID-TERM PERFORMANCE EVALUATION OF USAID/IRAQ PRIMARY HEALTH CARE PROJECT 129 37 Have you ever attended school? (respondent) YES.........................................................1 NO .......................................................... 2 39 38 What is the highest level of school you attended? PRIMARY ..............................................1 POST-PRIMARY/VOCATIONAL........ 2 SECONDARY/A-LEVEL...................... 3 COLLEGE (MIDDLE LEVEL) ............. 4 UNIVERSITY ........................................ 5 END END END END 39 Do you know how to read or how to write? YES.........................................................1 NO .......................................................... 2 END Thank you very much for taking the time to answer my questions. Once again, any information you have given will be kept completely confidential. Have a good day! MID-TERM PERFORMANCE EVALUATION OF USAID/IRAQ PRIMARY HEALTH CARE PROJECT 130 Annex 18D. District Department of Health Interview Tool Instructions for the District Department of Health Interviews: Meet with the District (Department of Health) Supervisor and read the following: “We are here on behalf of the Ministry of Health and the Primary Health Care Project in Iraq (PHCPI) funded by the United States Agency for International Development. We are conducting a survey of health managers and health facilities to assist the government and donors in knowing more about the current status of primary health care services. Your district and at least one facility in your district were selected to represent this region for how primary health care strengthening activities supported by PHCPI are being implemented. We would like to have a discussion with you and district staff who are responsible for supervising the PHCs, including their community activities. Any responses from you or your staff and from the PHC will remain confidential. We are visiting a total of 24 facilities so the results will only be used in aggregate, for example, 10 of 24 District Supervisors reported this activity. The discussion with you and your staff should take a maximum of 1 hour. Can we arrange a convenient time to meet with you and the relevant district-level staff to discuss some of the Primary Health Care strengthening activities?” Then ask the District Manager the best time to meet. If a group of 4-5 managers and supervisors who have received training can participate in one meeting, this is ideal. If this is not possible, identify the most relevant persons and interview at a minimum the District Manager, at least one other manager who has received management training, and at least one clinical supervisor who has received training in PHC standards. Then arrange a schedule for meeting with these persons. District Level (DoH Supervisor) Questionnaire Question Response 01 Name (s) and Title(s): Date: District: 02 Have you heard about the PHCPI project? YES ............................................. 1 NO ................................................ 2 05 03 If yes, would you say you are well-informed, somewhat informed, or slightly informed about the project? Well-informed .............................. 1 Somewhat informed ..................... 2 Slightly informed.......................... 3 04 How would you describe your involvement in the PHCPI project or the activities supported by the project? MID-TERM PERFORMANCE EVALUATION OF USAID/IRAQ PRIMARY HEALTH CARE PROJECT 131 Question Response 05 Have you or other district-level staff participated in developing any of the management guidelines that have been produced during these past 2 years? By these I mean the PHC Management, PHC Maintenance, the Infection Prevention, or the Community Partnership guidelines. YES ............................................. 1 N0 ................................................. 2 07 06 If yes, which ones? 07 Have you or other district-level staff participated in developing any of the clinical guidelines that have been produced during these past 2 years? YES ............................................. 1 N0 ................................................. 2 09a 08 If yes, which ones? 09a Have you or other district-level staff participated in any of the PHCPI management trainings? YES ..............................................1 N0.................................................. 2 10a 09b If yes, please indicate approximately the total number of trainings your staff has received. Have you received any reports (either written or verbal) from your staff or PHC facilities that indicate that: 09c The management training content is being used in their work? YES ..............................................1 N0.................................................. 2 09d There have been difficulties with implementing the recommendations from the training? YES ..............................................1 N0.................................................. 2 09e Please provide some additional details about your responses to the questions on using the training. MID-TERM PERFORMANCE EVALUATION OF USAID/IRAQ PRIMARY HEALTH CARE PROJECT 132 Question Response 10a Have you or any other district-level staff participated in any of the PHCPI clinical guideline trainings? YES ..............................................1 N0.................................................. 2 11a 10b If yes, please indicate approximately how many staff have been trained. Have you received any reports (either written or verbal) from your staff or PHC facilities that indicate that: 10c The training content from clinical guidelines is being used in their work? YES ..............................................1 N0.................................................. 2 10d There have been difficulties with implementing the recommendations from the clinical guideline training? YES ..............................................1 N0.................................................. 2 10e Please provide some additional details about your responses to the questions on using the training. 11a Have you or any other district-level staff received any training on supportive supervision? YES ............................................. 1 N0 ................................................. 2 12a 11b When did district staff provide supportive supervision most recently to PHC facilities? WITHIN PAST MONTH ............. 1 WITHIN PAST 2-3 MONTHS..... 2 WITHIN PAST 4-5 MONTHS..... 3 6 OR MORE MONTHS ............... 4 11c Please describe any changes you or other district-level staff have made in supervision as a result of project-supported training. 12a Do you or other district-level staff use checklists to assess PHCs compliance with management standards or with clinical standards? Yes ............................................... 1 No ................................................. 2 12c 12b Have you heard, or have there been any discussions about the district staff using checklists to assess PHC management or clinical practices against standards? YES ............................................. 1 N0 ................................................. 2 13a 12c Please discuss any barriers or issues that have resulted in delays in implementing monitoring against standards in the PHCs. MID-TERM PERFORMANCE EVALUATION OF USAID/IRAQ PRIMARY HEALTH CARE PROJECT 133 Question Response 13a Have you heard anything about the new medical record system that has been developed and is being introduced? YES ............................................. 1 N0 ................................................. 2 14a 13b Can you provide an example of how the new medical record system may improve client care? Yes ............................................... 1 No ................................................. 2 NOT SURE................................... 3 14a 14a 13c Please explain your response: 14a Does the district management receive any project-specific reports from PHCPI? IF YES, CLARIFY IF THE REPORTS ARE RECEIVED ROUTINELY OR IF THEY ARE SPECIAL REPORTS ONLY. Yes, routinely .............................. 1 Yes, special reports only............... 2 No ................................................. 3 15 14b If you receive PHCPI project reports, is the information received used to provide feedback to PHC staff? Yes ............................................... 1 No ................................................. 2 NOT SURE................................... 3 15 15 14c If yes, please provide details as to how this is done. 15 Can you provide any examples you haven’t already mentioned of how the project or project activities have supported or improved your work? 16 Can you mention any particular strong points in how the project is implemented, or how the staff work with the district? MID-TERM PERFORMANCE EVALUATION OF USAID/IRAQ PRIMARY HEALTH CARE PROJECT 134 Question Response 17 Are you aware of any major challenges faced by the PHCPI project that haven’t yet been mentioned so far? What, if any, major challenges do you anticipate in the future? 18 How satisfied are you overall with the interventions the PHCPI project has implemented to date? Very satisfied ................................1 Somewhat satisfied,....................... 2 Slightly satisfied............................ 3 Not satisfied .................................. 4 19b Do they reflect the needs of the district and PHC staff? Yes ...............................................1 No.................................................. 2 19c PLEASE PROVIDE A BRIEF EXPLANATON IF THERE WERE SOME NEGATIVE RESPONSES ABOUT THE PHCPI PROJECT AND PRIORITIES. 20a Do you think that the Local Health Committees (LHCs) s are important for improving the quality of PHC services or service utilization? YES , VERY IMPORTANT .............................. 1 YES, SOMEWHAT IMPORTANT .............................. 2 NO................................................. 3 20b Can you recommend any changes that could improve community participation or community representativeness from vulnerable or marginalized groups in LHCs? YES ..............................................1 NO................................................. 2 20d 20c Please discuss your recommendations to improve community participation, representation of vulnerable or marginalized groups, and the effectiveness of the LHC. 20d Can you recommend any changes that could improve community use of the PHC services?? YES ..............................................1 NO................................................. 2 20f 20e Please discuss your recommendations to improve community utilization of PHC services. 20f Is there a need to take action to increase access and utilization of vulnerable and marginalized groups, e.g., IDPs, religious or ethnic minorities, or disabled? YES ..............................................1 NO.................................................2 20g MID-TERM PERFORMANCE EVALUATION OF USAID/IRAQ PRIMARY HEALTH CARE PROJECT 135 Question Response 20g If you responded that there is no need to take action to increase access and utilization of vulnerable and marginalized groups, please explain your response. 20h Have there been any activities or decisions at the district, the PHC, or LHC to increase access and utilization of vulnerable and marginalized groups? YES ..............................................1 NO.................................................2 21a 20i What has been done and with what effect? 21a Are you familiar with the concepts of patient rights? YES ..............................................1 HEARD OF IT ONLY ..................2 NO.................................................2 21d 21d 21b What actions have been taken to educate the PHC staff or communities about patient rights? 21c What are the challenges faced in implementing the patient rights and what can be done to overcome them? 21d Has the LHC implemented any activities or taken any decisions regarding reducing differential treatment of women at PHC facilities? Yes ............................................... 1 No ................................................. 2 22a What are some ways PHCPI project could be improved? MID-TERM PERFORMANCE EVALUATION OF USAID/IRAQ PRIMARY HEALTH CARE PROJECT 136 Question Response 22b Do you believe the new management and quality activities promoted by the PHCPI Project are being implemented in a way that they will be sustained when project support is no longer available? Yes ............................................... 1 No ……………………..2 22d 22c If yes, please provide some examples of activities that have been instituted and that you think are becoming institutionalized and will be sustained. Provide any explanations for systemic changes that will support the activities when the project is no longer present. 22d Do you have any additional comments about the PHCPI Project that we haven’t already discussed? MID-TERM PERFORMANCE EVALUATION OF USAID/IRAQ PRIMARY HEALTH CARE PROJECT 137 Annex 18E. Key Informant Interview Guide with Local Health Committee Members 01 FACILITY NAME _______________________________________ 02 FACILITY NUMBER ..................................................................... 03 DISTRICT CODE ........................................................................... 04 INTERVIEWER NAME________________________________ 05 INTERVIEWER NUMBER ........................................................... 06 LHC MEMBER INTERVIEW NUMBER .................................... 07 DATE (a) (b) (c) 2013 DAY MONTH YEAR MID-TERM PERFORMANCE EVALUATION OF USAID/IRAQ PRIMARY HEALTH CARE PROJECT 138 READ TO REPRESENTATIVE: Hello, I am___________. We are implementing a survey for the Ministry of Health, which supports health services. We would like to interview you to better understand the work of the LHC. The information you provide will be used to improve the work of the LHC and community participation. All information you provide will remain confidential. Please know that whether you decide to allow this interview or not is completely voluntary. You may refuse to answer any question, and you may stop the interview at any time. Do you have any questions for me? Do I have your permission to continue with the interview? Signature of interviewer indicates that the consent form was read and the LHC member agreed to participate. NAME ____________________________ GROUP REPRESENTED___________ INTERVIEWER DOCUMENT THE FOLLOWING INFORMATION ABOUT RESPONDENT BACKGROUND Sex of Respondent MALE..................................................... 1 FEMALE................................................ 2 Could you give us a brief background about yourself and how and why you joined the Local Health Committee (LHC)? ………………………………………………………………………………………… ………………………………………………………………………………………… ………………………………………………………………………………………… LHC member represents: MARK ALL THAT ARE RELEVANT PHC STAFF ARE RECORDED ONLY ONCE AS “PHC” PHC........................................................ 1 COMMUNITY....................................... 2 IDP COMMUNITY................................ 3 RELIGIOUS MINORITY...................... 4 ETHNIC MINORITY ............................ 5 CIVIL SOCIETY ORGANIZATION .... 6 NGO ....................................................... 7 GOVERNMENT CLERK ...................... 8 ELECTED LOCAL COUNCIL ............. 9 DISABLED .......................................... 10 OTHER (SPECIFY) ............................. 96 DON’T KNOW .................................... 98 MID-TERM PERFORMANCE EVALUATION OF USAID/IRAQ PRIMARY HEALTH CARE PROJECT 139 Sex of Respondent MALE..................................................... 1 FEMALE................................................ 2 Could you give us a brief background about the community of the catchment area of the PHC and its composition? ………………………………………………………………………………………… ………………………………………………………………………………………… ………………………………………………………………………………………… Composition of the community: MARK ALL THAT ARE RELEVANT HOMOGENOUS COMMUNITY.......... 1 IDP COMMUNITY................................ 2 RELIGIOUS MINORITY...................... 3 ETHNIC MINORITY ............................ 4 DISABLED ............................................ 5 OTHER (SPECIFY) ............................. 96 DON’T KNOW .................................... 98 Since when has the LHC been formed? Number of months ……………………….. Number of years …………………………. 1 Have you been trained on the community health partnership handbook? YES ........................................................ 1 NO .......................................................... 2 7 2 What are your views about the training? PROBE FOR THE MOST ACCURATE RESPONSE. VERY RELEVANT............................... 1 SOMEWHAT RELEVANT................... 2 NOT SO RELEVANT............................ 3 3 Comments: ………………………………………………………………………………………… ………………………………………………………………………………………… ………………………………………………………………………………………… 4 Did you use any of the concepts or activities outlined in the handbook and/or training? YES.........................................................1 NO........................................................... 2 6 5 Describe what you have used and what happened as a result. ……………………………………………………………………………………… ……………………………………………………………………………………… ……………………………………………………………………………………… 7 6 Why have you not used any of the concepts or activities outlined in the handbook and/or training? ……………………………………………………………………………………… MID-TERM PERFORMANCE EVALUATION OF USAID/IRAQ PRIMARY HEALTH CARE PROJECT 140 Sex of Respondent MALE..................................................... 1 FEMALE................................................ 2 ……………………………………………………………………………………… ……………………………………………………………………………………… 7 Have you received any other training? YES.........................................................1 NO...........................................................2 9 8 What were the training topics? MARK ALL MENTIONED. PATIENT RIGHTS AND RESPONSIBILITIES..............................1 INTERPERSONAL COMMUNICATION..............................2 BEHAVIOURAL CHANGE COMMUNICATION..............................3 TRAINING .............................................4 OTHER (SPECIFY) ..............................96 DON’T KNOW.....................................98 9 How many members are there in the LHC, including PHC staff? 10 How many members of the LHC represent the community side? 11 How many members of the LHC representing the community are female? 12 Please specify the affiliations of all members of the LHC. MARK ALL THAT ARE RELEVANT. PHC.........................................................1 COMMUNITY........................................ 2 IDP COMMUNITY ................................ 3 RELIGIOUS MINORITY....................... 4 ETHNIC MINORITY............................. 5 CIVIL SOCIETY ORGANIZATION..... 6 NGO........................................................ 7 GOVERNMENT CLERK....................... 8 ELECTED LOCAL COUNCIL.............. 9 DISABLED...........................................10 WOMEN...............................................11 OTHER (SPECIFY) ..............................96 DON’T KNOW.....................................98 13 Is the LHC meeting regularly? YES.........................................................1 NO........................................................... 2 MID-TERM PERFORMANCE EVALUATION OF USAID/IRAQ PRIMARY HEALTH CARE PROJECT 141 Sex of Respondent MALE..................................................... 1 FEMALE................................................ 2 14 When was the most recent meeting? PAST WEEK ..........................................1 PAST 2 WEEKS ..................................... 2 PAST MONTH ....................................... 3 PAST 2 MONTHS.................................. 4 MORE THAN 2 MONTHS AGO........... 5 NO MEETINGS YET............................. 0 15 How many members attended the most recent meeting, including the PHC members? 16 Please tell me about topics that were discussed during your meetings. PROBE, BUT DO NOT MENTION ANY SPECIFIC TOPIC. CIRCLE ALL THAT APPLY. ……………………………………………………… ……………………………………………………… ……………………………………………………… ……………………………………………………… ……………………………………………………… ……………………………………………………… ……………………………………………………… ……………………………………………………… ……………………………………………………… ……………………………………………………… PHC BUDGET........................................1 QUALITY OF SERVICES ..................... 2 TIMES SERVICES ARE OFFERED............................................... 3 PLANNING OF SERVICES .................. 4 IMPLEMENTING SERVICES............... 5 COMMUNITY OUTREACH ACTIVITIES .......................................... 6 FEEDBACK COMMUNITY VIEWS ON SERVICES.............................................. 7 IMPROVING TREATMENT OF WOMEN AT PHC.................................. 8 IMPROVING ACCESS AND UTILIZATION BY IDP GROUPS......... 9 HEALTH RELATED INFRA￾STRUCTURE .......................................10 FACILITATING PHC REQUESTS FROM GOVERNMENT ......................11 INTER-SECTORAL COORDINATION ..............................................................12 OTHER (SPECIFY) ..............................96 DON’T KNOW.....................................98 MID-TERM PERFORMANCE EVALUATION OF USAID/IRAQ PRIMARY HEALTH CARE PROJECT 142 Sex of Respondent MALE..................................................... 1 FEMALE................................................ 2 17 Have you discussed any of these topics during your meetings READ ALL TOPICS NOT MENTIONED. SPONTANEOUSLY IN THE ABOVE QUESTION CIRCLE ALL THAT APPLY. MARK ALL DISCUSSE.D ……………………………………………………… ……………………………………………………… ……………………………………………………… ……………………………………………………… ……………………………………………………… ……………………………………………………… ……………………………………………………… ……………………………………………………… ……………………………………………………… ……………………………………………………… PHC BUDGET........................................1 QUALITY OF SERVICES ..................... 2 TIMES SERVICES ARE OFFERED............................................... 3 PLANNING OF SERVICES .................. 4 IMPLEMENTING SERVICES............... 5 COMMUNITY OUTREACH ACTIVITIES .......................................... 6 FEEDBACK COMMUNITY VIEWS ON SERVICES.............................................. 7 IMPROVING TREATMENT OF WOMEN AT PHC.................................. 8 IMPROVING ACCESS AND UTILIZATION BY IDP GROUPS......... 9 HEALTH RELATED INFRA￾STRUCTURE .......................................10 FACILITATING PHC REQUESTS FROM GOVERNMENT ......................11 INTER-SECTORAL COORDINATION ..............................................................12 OTHER (SPECIFY) .............................96 DON’T KNOW....................................98 18 Give examples of some of the decisions taken and their effects as well as activities that took place as a result of these meetings in the PHC facility and with what effect. …………………………………………………………………………………………….. …………………………………………………………………………………………….. …………………………………………………………………………………………….. …………………………………………………………………………………………….. 19 Give examples of some of the decisions taken and their effects as well as activities that took place as a result of these meetings at the community level and with what effect. …………………………………………………………………………………………….. MID-TERM PERFORMANCE EVALUATION OF USAID/IRAQ PRIMARY HEALTH CARE PROJECT 143 Sex of Respondent MALE..................................................... 1 FEMALE................................................ 2 ……………………………………………………………………………………………..………… ………………………………………………………………………………….. …………………………………………………………………………………………….. 20 What are your views about the level and type of cooperation and communication of the community members of the LHC with you? IF POOR, WHY/WHAT COULD BE DONE? ………………………………………………… ………………………………………………… EXCELLENT..........................................1 GOOD..................................................... 2 FAIR ....................................................... 3 POOR...................................................... 4 21 What are your views about the level and type of cooperation and communication of the PHC members of the LHC with you? IF POOR, WHY/WHAT COULD BE DONE? ………………………………………………… ………………………………………………… EXCELLENT..........................................1 GOOD..................................................... 2 FAIR ....................................................... 3 POOR...................................................... 4 22 Do you feel that the opinions of the community members of the LHC are taken seriously in decisions related to the PHC services and functioning? ALWAYS.................................................. SOMETIMES.......................................... 2 RARELY................................................. 3 NEVER ................................................... 4 23 COMMENTS ON INFLUENCE OF COMMUNITY REPRESENTATIVE VIEWS IN LHC DECISIONS: …………………………………………………………………………………………….. …………………………………………………………………………………………….. …………………………………………………………………………………………….. 24 What are the challenges faced by PHC LHC members in their work? …………………………………………………………………………………………….. …………………………………………………………………………………………….. …………………………………………………………………………………………….. 25 What are the challenges faced by community LHC members in their work? …………………………………………………………………………………………….. …………………………………………………………………………………………….. …………………………………………………………………………………………….. MID-TERM PERFORMANCE EVALUATION OF USAID/IRAQ PRIMARY HEALTH CARE PROJECT 144 Sex of Respondent MALE..................................................... 1 FEMALE................................................ 2 28 How can community health partnerships and the work of LHC be made more effective in raising awareness about healthy behavior and patient rights? …………………………………………………………………………………………….. …………………………………………………………………………………………….. …………………………………………………………………………………………….. 29 Do you have any thoughts about the type of treatment women face at the PHC? Please comment. ………………………………………………………………………………… ………….. …………………………………………………………………………………………….. …………………………………………………………………………………………….. 30 Do you know of any specific practices or strategies to increase PHC staff awareness of the importance of listening to female patients, treating them with respect, and making sure that they feel their dignity is maintained? YES.........................................................1 NO........................................................... 2 3 2 31 What are these practices, strategies, and decisions? What were the challenges faced? Have they been overcome? What is the outcome? …………………………………………………………………………………………….. …………………………………………………………………………………………….. …………………………………………………………………………………………….. 32 Do you know of any activities that have been implemented in the PHC to try to reach out for vulnerable groups such as internally displaced persons (IDPs), the disabled, or religious or ethnic minority to increase their use of PHC services? YES.........................................................1 NO........................................................... 2 3 4 33 What are these activities? What were the challenges faced? Have they been overcome? What is the outcome? Disabled …………………………………………………………………………………………………….. IDP…………………………………………….…………………………………………………………….. Religious minority………………………………………………………………………………………….. Ethnic minority .…………………………………………………………………………………………….. Other (specify)….…………………………………………………………………………………………….. 3 5 34 Why not? …………………………………………………………………………………………………………………. …………………………………………………………………………………………………………………. MID-TERM PERFORMANCE EVALUATION OF USAID/IRAQ PRIMARY HEALTH CARE PROJECT 145 Sex of Respondent MALE..................................................... 1 FEMALE................................................ 2 …………………………………………………………………………………………………………………. 35 Do you think these vulnerable groups face difficulties in utilizing services? Disabled …………………………………………………………………………………………………….. IDP…………………………………………….…………………………………………………………….. Religious minority………………………………………………………………………………………….. Ethnic minority .…………………………………………………………………………………………….. Other (specify)….…………………………………………………………………………………………….. 36 In general do you think that the work of the LHC will lead to improvements in the quality of the health services provided at the PHC? YES DEFINITELY.................................1 YES SOMEWHAT................................. 2 NOT AT ALL ......................................... 3 3 9 3 8 37 Why not? ………………………………………………………………………………………………………….. …………………………………………….…………………………………………………………….. ………….….…………………………………………………………………………………………….. 38 What can be done so that the LHC work can lead to improvements in the quality of the health services provided at the PHC? ………………………………………………………………………………………………………….. …………………………………………….…………………………………………………………….. ………….….…………………………………………………………………………………………….. 39 In general do you think that the work of the LHC will lead to increased utilization of PHC services by the community, especially among vulnerable groups and women? YES DEFINITELY.................................1 YES SOMEWHAT................................. 2 NOT AT ALL ......................................... 3 4 2 4 1 40 Why not? ………………………………………………………………………………………………………….. …………………………………………….…………………………………………………………….. ………….….…………………………………………………………………………………………….. 41 What can be done so that the LHC work can lead to increased utilization of PHC services by the community, especially vulnerable groups and women? ………………………………………………………………………………………………………….. …………………………………………….…………………………………………………………….. MID-TERM PERFORMANCE EVALUATION OF USAID/IRAQ PRIMARY HEALTH CARE PROJECT 146 Sex of Respondent MALE..................................................... 1 FEMALE................................................ 2 ………….….…………………………………………………………………………………………….. Patient Rights 42 Do you know of patient rights? YES .........................................................1 NO........................................................... 2 45 43 How did you come to know about patient’s rights? MARK ALL MENTIONED. SEEN POSTERS / PAMFLET................1 DURING TRAINING............................. 2 PHC STAFF TOLD LHC ....................... 3 NGO/CIVIL SOCIETY........................... 4 COMMUNITY LEADER....................... 5 TV/RADIO.............................................. 6 NEWSPAPER......................................... 7 OTHER (SPECIFY) ..............................96 DON’T KNOW.....................................98 44 What are the various patients’ rights? MARK ALL MENTIONED. RIGHT TO KNOW.................................1 TREATMENT AND CARE ................... 2 MAKE OWN DECISIONS ON TREATMENT......................................... 3 PRIVACY OF PERSONAL INFO ......... 4 REFERAL............................................... 5 SECOND OPINION ............................... 6 VISUAL PRIVACY................................ 7 AUDITORY PRIVACY ......................... 8 CHOICE OF PROVIDER....................... 9 OTHER (SPECIFY) ..............................96 DON’T KNOW.....................................98 45 In general, do you know what services are provided in this health facility? Antenatal/postnatal care ..........................1 Delivery services..................................... 2 MID-TERM PERFORMANCE EVALUATION OF USAID/IRAQ PRIMARY HEALTH CARE PROJECT 147 Sex of Respondent MALE..................................................... 1 FEMALE................................................ 2 MARK ALL MENTIONED. PROBE TO IDENTIFY ALL SERVICES THE LHC MEMBER KNOWS. Family planning ......................................3 Child immunization ................................ 4 Maternal and child health........................ 5 Dentistry.................................................. 6 Ophthalmology........................................ 7 Consultation for child curative care ........ 8 Consultation for adult curative care ........ 9 Breast cancer.........................................10 Communicable diseases ........................11 Chronic diseases....................................12 Continuous availability of drugs/popular clinic that distributes drug supply......... 13 Laboratory.............................................14 X-ray .....................................................15 Diagnostic (lab & x-ray) .......................16 Ambulance ............................................17 Minor surgery........................................18 Referral services....................................19 Availability of inspection ......................20 Pediatric specialist................................. 21 Availability of specialists......................22 Modernization & development..............23 Injections...............................................24 OTHER (SPECIFY) ..............................96 DON’T KNOW.....................................98 46 Do these services cover the health needs of the community? YES .........................................................1 NO........................................................... 2 49 MID-TERM PERFORMANCE EVALUATION OF USAID/IRAQ PRIMARY HEALTH CARE PROJECT 148 Sex of Respondent MALE..................................................... 1 FEMALE................................................ 2 47 What health services need to be added to the facility services? MARK ALL MENTIONED AND SPECIFY ALL NOT MARKED. SPECIFY …………………………………………………… …………………………………………………… …………………………………………………… …………………………………………………… …………………………………………………… …………………………………………………… …………………………………………………… Antenatal/postnatal care ..........................1 Delivery services..................................... 2 Family planning ...................................... 3 Child immunization ................................ 4 Maternal and child health........................ 5 Dentistry.................................................. 6 Ophthalmology........................................ 7 Consultation for child curative care ........ 8 Consultation for adult curative care ........ 9 Breast cancer.........................................10 Communicable diseases ........................11 Chronic diseases....................................12 Continuous availability of drugs/popular clinic that distributes drug supply......... 13 Laboratory.............................................14 X-ray .....................................................15 Diagnostic (lab & x-ray) .......................16 Ambulance ............................................17 Minor surgery........................................18 Referral Services...................................19 Availability of auditing (رقابة (...............20 Pediatric specialist................................. 21 Availability of specialists......................22 ............................................23تحديث وتطوير Injections...............................................24 OTHER (SPECIFY) ..............................96 DON’T KNOW.....................................98 MID-TERM PERFORMANCE EVALUATION OF USAID/IRAQ PRIMARY HEALTH CARE PROJECT 149 Sex of Respondent MALE..................................................... 1 FEMALE................................................ 2 48 Why are these health services needed? MARK ALL MENTIONED AND SPECIFY ALL NOT MARKED. SPECIFY …………………………………………………… …………………………………………………… …………………………………………………… …………………………………………………… …………………………………………………… …………………………………………………… Medication cost is high ...........................1 Reason 2 .................................................. 2 ................................................................. 3 ................................................................. 4 ................................................................. 5 ................................................................. 6 ................................................................. 7 ................................................................. 8 OTHER (SPECIFY) ..............................96 DON’T KNOW.....................................98 49 Is the manner in which services are available and provided appropriate to the community? YES .........................................................1 NO........................................................... 2 52 50 Why is it not appropriate? MARK ALL MENTIONED AND SPECIFY ALL NOT MARKED. SPECIFY …………………………………………………… …………………………………………………… …………………………………………………… …………………………………………………… …………………………………………………… Bad physician treatment ..........................1 Bad nurse treatment................................. 2 Bad physician & nurse treatment ............ 3 Bad staff treatment .................................. 4 Need to smile/treatment should be better 5 Immunization time and place not appropriate............................................... 6 Medication not available ......................... 7 ................................................................. 8 OTHER (SPECIFY) ..............................96 DON’T KNOW.....................................98 51 How could services become more appropriate? MARK ALL MENTIONED AND SPECIFY ALL NOT MARKED. SPECIFY Availability of chronic disease medication ............................................... 1 Make medication available...................... 2 Specialist physicians ............................... 3 Pediatricians............................................ 4 MID-TERM PERFORMANCE EVALUATION OF USAID/IRAQ PRIMARY HEALTH CARE PROJECT 150 Sex of Respondent MALE..................................................... 1 FEMALE................................................ 2 …………………………………………………… …………………………………………………… …………………………………………………… …………………………………………………… Obstetrician/gynecologist........................5 Redesign the reception and waiting procedures............................................... 6 Better treatment....................................... 7 More counseling, explanation & advice . 8 OTHER (SPECIFY) ..............................96 DON’T KNOW.....................................98 52 Do you have any suggestions on making the services better meet the needs of the community and delivered in an appropriate manner? MARK ALL MENTIONED AND SPECIFY ALL NOT MARKED. SPECIFY …………………………………………………… …………………………………………………… Availability of chronic disease medication ............................................... 1 Make medication available...................... 2 Specialist physicians ............................... 3 Pediatricians............................................ 4 Obstetrician/gynecologist........................ 5 Redesign the reception and waiting procedures............................................... 6 Better treatment....................................... 7 More counseling, explanation & advice . 8 OTHER (SPECIFY) ..............................96 DON’T KNOW.....................................98 53 Do you have any suggestions on making the services more appropriate for women and girls and better meet their needs? MARK ALL MENTIONED AND SPECIFY ALL NOT MARKED. SPECIFY …………………………………………………… …………………………………………………… Female physician.....................................1 Method 2 ................................................. 2 ................................................................. 3 ................................................................. 4 ................................................................. 5 ................................................................. 6 ................................................................. 7 ................................................................. 8 OTHER (SPECIFY) ..............................96 MID-TERM PERFORMANCE EVALUATION OF USAID/IRAQ PRIMARY HEALTH CARE PROJECT 151 Sex of Respondent MALE..................................................... 1 FEMALE................................................ 2 DON’T KNOW.....................................98 54 Do you have any other comments? ……………………………………………………………………………………………………………. …………………………………………………………………………………………………………… …………………………………………………………………………………………………………… Thank you very much for taking the time to answer my questions. Once again, any information you have given will be kept completely confidential. Have a good day! Annex 18F. Key Informant Interviews with Technical Advisory Group Members Question Expected Response What do you think the main roles and responsibilities of the TAG should be? Open-ended question Does the current TAG’s roles and responsibilities match that you previously described? Yes or No Is the TAG functioning optimally? Yes or No If not, what could be done to improve this? Open-ended question Are PHCPI results shared sufficiently with the TAG? Yes or No If not, what could be done to improve this? Open-ended question What information that is provided through the TAG meetings is of most use? Open-ended question Is the TAG sufficiently involved in the PHCPI planning process? Yes or No If not, how would you change the process to make it more effective? Open-ended question What recommendations do you have to improve the implementation of PHCPI activities? Open-ended question Has your organization participated in any PHCPI trainings or other activities? Yes or No MID-TERM PERFORMANCE EVALUATION OF USAID/IRAQ PRIMARY HEALTH CARE PROJECT 152 Question Expected Response If so, please describe. Open-ended question Has PHCPI provided any materials or information to your organization? Yes or No If yes, how have you used these materials? Open-ended question Have you or your organization participated in any PHCPI field visits with your organization to review activities? Yes or No What are the most important contributions the TAG has made for the PHCPI’s implementation? Open-ended question What else would you like to see the TAG accomplish? Open-ended question Annex 18G. Questionnaire for Implementing Partners Question Expected Response Can you give me examples in which USAID has provided your organization with support to tackle challenges the project has faced? Yes or No If yes, please provide a few examples. Open-ended question Has the USAID mission provided feedback on your work plans? Yes or No If yes, can you provide an example of that feedback? Open-ended question Has USAID provided clear direction? Yes or No Are there clear channels for reporting to USAID? Yes or No If no, what might be done to improve the situation? Open-ended question How often does the project communicate with USAID/Iraq, USAID/W, and its own headquarters? For each – weekly, monthly, quarterly, annually Have there been any USAID administrative delays? Yes or No If so, what have these delays been and how have they effected implementation? Open-ended question Does the current USAID funding mechanism provided enough flexibility to respond to changing needs? Yes or No If no, please give an example in which the project was unable to respond to an evolving need because of the contract mechanism? Open-ended question How did URC determine staffing needs for PHCPI implementation? Open-ended question Please describe how URC’s M&E system captures reporting data, especially for vulnerable groups. Open-ended question What challenges have you encountered to recruit/retain staff? Open-ended question How do plan to address these challenges? Open-ended question Where does your organization currently require additional capacity? Open-ended question How does your organization identify capacity needs/gaps at the organizational/personnel level? Open-ended question How does you organization build staff skills? Open-ended question What challenges or barriers has your organization encountered to implementing your staff capacity development projects? Open-ended question MID-TERM PERFORMANCE EVALUATION OF USAID/IRAQ PRIMARY HEALTH CARE PROJECT 153 MID-TERM PERFORMANCE EVALUATION OF USAID/IRAQ PRIMARY HEALTH CARE PROJECT 154 Annex 18H. Questionnaire for USAID Question Expected Response What was the rationale for USAID’s decision to focus on primary health care and specifically the three technical components? Open-ended question What alternatives were considered? Open-ended question How many mission staff are working on the project? Numeric What are their technical backgrounds? Open-ended question Has there been any turnover either in CORs or technical advisors? Yes or No Have mission staff participated in any PHCPI activities? Yes or No If yes, what activities have the mission staff participated in? Open-ended question Have any site visits been conducted? Yes or No If so, how frequently are they done? Monthly, quarterly, annually Does mission staff feel sufficiently informed about project implementation and results? Yes or No If no, what else can be done to improve this? Open-ended question What strengths does URC bring to PHCPI? Open-ended question What have been the main challenges? Open-ended question If the mid-term evaluation or other assessment revealed grave problems with PHCPI, what would the strategy be in dealing with the particular issues and the project in general? Open-ended question What will a successful PHCPI project look like? Open-ended question Has a project exit strategy been developed? Yes or No If yes, what does it involve and how does it ensure sustainability? Open-ended question MID-TERM PERFORMANCE EVALUATION OF USAID/IRAQ PRIMARY HEALTH CARE PROJECT 155 Annex 18I. Questionnaire for Ministry of Health Officials Question Response Name: Date: Title: Province or District: Have you heard about the PHCPI project? Yes No (Circle one) If yes, would you say you are well￾informed, somewhat informed, or slightly informed about the project? Well-informed Somewhat informed (circle one) Slightly informed How would you describe your involvement in the PHCPI project? Can you describe for me the inputs provided to PHCPI as part of the GoI cost-share? What percentage of the total GoI cost-share do you believe has been utilized to date? Yes No (circle one) Have you or your institution participated in developing any of the management guidelines that have been produced? MID-TERM PERFORMANCE EVALUATION OF USAID/IRAQ PRIMARY HEALTH CARE PROJECT 156 Question Response If yes, which ones? Have you or any of your staff participated in any of the PHCPI management and administration trainings? Yes No (circle one) If yes, please indicate approximately how many trainings. In what way, if at all, does the PHCPI project differ from other approaches used to strengthen primary health care facilities? Who receives data and reports from the PHCPI project? Please provide the person’s position and title. How often is information received? Weekly, monthly, quarterly, annually or other (circle one) Is feedback given on data and results generated by the PHCPI project to PHC and project staff? Yes No (circle one) If yes, how is this done? MID-TERM PERFORMANCE EVALUATION OF USAID/IRAQ PRIMARY HEALTH CARE PROJECT 157 Question Response Do you have an example of how project results and data have been used for decision￾making? Yes No (circle one) If yes, please describe. What are the program’s strengths? What have been the major challenges faced by the PHCPI project so far? What, if any, major challenges do you anticipate in the future? How satisfied are you overall with the interventions the PHCPI project has implemented to date? Very satisfied, somewhat satisfied, slightly satisfied, not satisfied (circle one) Do the interventions target what is important? Yes No (circle one) What has been done so far concerning patient rights and public health law at different levels of MoH and PHC facilities? MID-TERM PERFORMANCE EVALUATION OF USAID/IRAQ PRIMARY HEALTH CARE PROJECT 158 Question Response What are the challenges faced in implementing patients rights and what can be done to overcome them? Please discuss any changes that have been implemented within this training institution with regards to processes for updating or changing guidelines that you can attribute in some part to working with the project. Discuss changes you think are beneficial, and those you are not sure are beneficial. Among the changes you have mentioned, please discuss those which you think now can continue without project support. Among the changes you have mentioned, please discuss which you think need continued project support to be maintained, and discuss why. Over the next two years, what type of project support can help institutionalize the changes that you think are beneficial? Please discuss any changes that have been implemented within this training institution in planning for training and training methods used that you can attribute in some part to working with the project. Discuss changes you think are beneficial, and those you are not sure are beneficial. Among the changes you have mentioned, please discuss which you think can now continue without project support MID-TERM PERFORMANCE EVALUATION OF USAID/IRAQ PRIMARY HEALTH CARE PROJECT 159 Question Response Among the changes you have mentioned, please discuss which you think need continued project support to be maintained, and discuss why. Over the next two years what type of project support can help institutionalize the changes that you think are beneficial? How responsive do you feel that the PHCPI has been to your needs? Very responsive, somewhat responsive, slightly responsive, not responsive (circle one) What are some ways PHCPI project could be improved? If USAID funding ended, would the activities continue in-country? Yes No (circle one) What alternative funding sources might replace USAID resources? MID-TERM PERFORMANCE EVALUATION OF USAID/IRAQ PRIMARY HEALTH CARE PROJECT 160 Question Response Do you have any additional comments about the PHCPI Project that we haven’t already discussed? MID-TERM PERFORMANCE EVALUATION OF USAID/IRAQ PRIMARY HEALTH CARE PROJECT 161