Final Evaluation of the Pilot Health Project in the West Bank and Gaza - February 2002 - Marc Debay, MD, PhD, MPH Matthew Tayback, ScD Johns Hopkins University Bloomberg School of Public Health This evaluation was funded by the U.S. Agency for International Development (USAID) under the Purchase Order No 294-O-00-02-00205-00. PHP Final Evaluation Report i Table of content Executive Summary........................................................................................................... iv I. Introduction................................................................................................................. 1 II. Project design.............................................................................................................. 2 A. Problem statement................................................................................................... 3 B. Objectives ............................................................................................................... 3 C. Interventions ........................................................................................................... 4 D. Management............................................................................................................ 6 E. Strengths and weaknesses of the project design ..................................................... 8 III. Project implementation ......................................................................................... 11 A. Timeline ................................................................................................................ 11 B. Human resources................................................................................................... 13 C. Work plans and management................................................................................ 16 D. Funding ................................................................................................................. 17 E. Reporting............................................................................................................... 21 IV. Project results........................................................................................................ 23 A. Coordination and capacity building...................................................................... 23 B. Renovation and equipment of clinics.................................................................... 26 C. Training and behavior change............................................................................... 30 D. Management Information System......................................................................... 35 E. Operations Research ............................................................................................. 39 V. Conclusions............................................................................................................... 42 A. Accomplishments.................................................................................................. 42 B. Shortfalls............................................................................................................... 45 C. Recommendations................................................................................................. 47 D. Concluding statement............................................................................................ 49 PHP Final Evaluation Report ii APPENDICES .................................................................................................................. 50 Appendix 1 Schedule of the PHP final evaluation............................................................ 51 Appendix 2 List of documents reviewed .......................................................................... 54 Appendix 3 Organizational and management structure of the Pilot Health Project ......... 57 Appendix 4 Timeline of USAID/WBG general and health programs.............................. 58 Appendix 5 Timeline of the PHP implementation............................................................ 59 Appendix 6 Population Council West Bank and Gaza Obligations vs. Expenditures...... 61 Appendix 7 CARE West Bank/Gaza Obligations vs. Expenditures................................. 62 Appendix 8 Vision statement for the Coordinating Council ............................................ 63 Appendix 9 List of the 27 clinics included in the PHP..................................................... 65 Appendix 10 List of equipment ........................................................................................ 66 Appendix 11 Reviews of Operations Research protocols................................................. 69 Appendix 12 Photographs of renovated clinics ................................................................ 75 List of Tables Table 1 Summary of Obligations to the PHP and Expenditures by CARE and Population Council.................................................................................................................. 18 Table 2 Budget allocations by CARE for the three sub grants to the PHP Service Delivery partners.................................................................................................................. 19 Table 3 Scope of work, total budget and effective dates of sub awards and contracts by the Population Council to its local partners in the West Bank and Gaza ............. 20 Table 4 Distribution of the Pilot Health Project and the Primary Health Care clinics by district and NGO in the West Bank and Gaza ...................................................... 27 Table 5 Distribution of Trainers and Health Care Providers trainees by institutional affiliation and professional categories .................................................................. 33 PHP Final Evaluation Report iii Acronyms CARE Cooperative for Assistance and Relief Everywhere CDPHC Center for Development in Primary Health Care FRONTIERS Frontiers in Reproductive Health HDIP Health, Development, Information and Policy IVCHS Integrated Village and Community Health Services MIS Management Information System MOH Ministry of Health NGO Non-Governmental Organization OR Operations Research PHC Primary Health Care PFS Patients Friend's Society PHP Pilot Health Project RH Reproductive Health RTA Research, Training, and Analysis SD Service Delivery TOT Training of trainers UHWC Union of Health Workers Committees UNFPA United Nations Population Fund UNRWA United Nations Relief and Works Agency UPMRC Union of Palestinian Medical Relief Committees USAID United States Agency for International Development USAID/ANE USAID Bureau for Asia and Near East USAID/WBG USAID Mission in West Bank and Gaza USAID/W USAID Washington UDDH US Direct Hire USPSC US Personal Service Contract WBG West Bank and Gaza PHP Final Evaluation Report iv Executive Summary The goal of the Pilot Health Project (PHP) was to improve the health status of Palestinian mothers and their children. It was the first major initiative in support of the “Healthier Palestinian Families” Strategic Objective later adopted by the USAID/WBG Mission. Given the history of its development, its main technical reference (the Concept Paper), and the course of its implementation, the evaluators summarize the purposes of the PHP as follows. 1. To develop and implement a pilot activity that could serve as a basis for a national scale-up to improve the health status of Palestinian women and their children. This aspect of the PHP explains part of the investments made and the challenges faced by the Mission. 2. To offer and test the effectiveness of a basic package of health services in approximately 27 health clinics in 3 pilot areas. This package, initially based on integrated antenatal, postpartum, and related outreach services, slightly changed during the course of the project implementation. 3. To test 3 experimental service delivery interventions to improve health practices and health seeking behavior related to antenatal and postpartum care and to birth spacing: • Involvement of male motivators in promotion of reproductive health. • Enhanced postpartum services to low parity mothers. • Coordinated hospital and clinic follow-up post delivery. 4. To design a Management Information System for reproductive and child health within primary health care settings. This activity, initially aimed at monitoring service enhancements within the scope and timeframe of the PHP, embraced a broader goal in the course of project implementation. PHP Final Evaluation Report v Historical Considerations Prior to 1997, there was no active health objective pursued by the Mission. During 1997, the USAID Bureau for Asia and Near East organized an assessment of the feasibility of including a pilot health activity as a prelude to possible future investments in the West Bank and Gaza. A team of 3 health specialists from USAID/W, following extensive consultation with local representatives of Non-Governmental Organizations (NGO) and international organizations in the West Bank and Gaza, and considering the current capability of the Ministry of Health (MOH) and the private sector, recommended a limited health services enhancement program. The Mission evaluated the report of the assessment team and decided to launch a health program in 1998. Administrative Considerations At the time that the Mission approved an initial health project, it had no full time health professional staff, or prior experience in health planning, implementation and operations research. A management structure, proposed by the 1997 assessment team, thus included two USAID/W/G/PHN Cooperating Agencies (CA): CARE International for service delivery related tasks through the CARE-MoRR Cooperative Agreement (HRN-A-00-98- 00023-00), and the Population Council for operations research, training, and improvements in existing management information systems through the FRONTIERS Cooperative Agreement (HRN-A-00-98-00012-00). The two CAs sole-sourced five Palestinian NGOs designated as partners in their Cooperative Agreements. With respect to the task of service delivery enhancement, CARE International sole-sourced (1) Patients Friends Society (PFS) in Jenin, (2) Union of Palestinian Medical Relief Committees (UPMRC) in Hebron and Jenin, and (3) Union PHP Final Evaluation Report vi of Health Work Committees (UHWC) in Gaza. For the operations research, training and management information systems tasks, the Population Council sole-sourced (1) Health, Development, Information and Policy Institute (HDIP) and (2) Center for Development of Primary Health Care (CDPHC) in Ramallah. HDIP, initially considered for all operations research, declined to assume responsibility for two studies that the Population Council then assigned to CDPHC and Alpha International, respectively. For the purpose of promoting coordination of activities among the two Cooperating Agencies, the Palestinian NGOs, and the MOH, USAID/WBG created Coordinating Council with a secretariat by CARE International. Financial Considerations An estimated total of $5,268,852 has been expended on the Pilot Health Project, of which USAID/WBG obligated $2,155,000 to Population Council and $2,353,852 to CARE International. These expenditures were distributed among the two Cooperating Agencies and their Palestinian partner NGOs as follows: CARE International $2,583,852 Sub grants to Palestinian NGOs UHWC $219,084 UPMRC $216,228 PFS $215,429 Medical equipment and supplies to 3 NGOs $765,370 Population Council $2,685,000 Sub grants to Palestinian NGOs HDIP $510,536 CDPHC $332,394 Alpha $58,832 Total expended $5,268,852 PHP Final Evaluation Report vii Accomplishments 1. Health infrastructure improvements initiated and completed include: (1) Significant physical upgrade of 25 clinics (2) Significant equipment upgrade of 27 clinics (3) Training of 58 clinic workers in 27 clinics 2. Development of a cadre of 13 trainers and a training manual for health workers engaged in prenatal, postnatal and reproductive health services. 3. Successful organization and conduct of a Coordinating Council 4. Baseline and end of project surveys of prenatal, postnatal and reproductive health care in 27 clinics completed 5. Design, and field execution of 3 operations research projects 6. Design and completion of a series of studies and analyses on selective subjects of reproductive health 7. Review of existing management information system operative in 3 NGO primary care clinics 8. On-going pilot test of a computerized women and child patient record system as a basis for a national-level health and management information system. 9. Partnership with five Palestinian NGOs have strengthened their capacity to implement reproductive health programs. 10. The lessons learned under the PHP have been included in the design and early implementation of the MARAM project. 11. The Mission has a health and population Strategic Objective and is an important player in the health sector in the West Bank and Gaza PHP Final Evaluation Report viii Shortfalls 1. Target dates for tasks completion were not met, due in part to the following: - Late obligation of funds - Difficulty with negotiating sub agreements in a non-competitive situation - Complicated procurement procedures - Closures, travel restrictions and insecurity since the beginning of the Intifada 2. The PHP clinics do not constitute a representative sample of the primary care system in the West Bank and Gaza 3. There are no explicit standards for antenatal, postpartum and reproductive health services adopted and implemented in the 27 PHP clinics 4. The management information systems piloted in a few PHP clinics did not reach the point of effective use. 5. None of the operations research studies had reached conclusion by the end of the project. 6. Some of the specific objectives of the operations research studies may not be met because of design issues 7. The MOH did not have any opportunity to participate in the PHP beyond that of attending the Coordinating Council meetings. 8. The Cooperative Agencies did not always provide adequate level of in-country technical assistance. 9. The implementation of the project has been more difficult in Gaza than in the West Bank 10. The Mission initially, did not fully assert authority and assume responsibility regarding the implementation of the PHP by the two centrally administered Cooperating Agencies PHP Final Evaluation Report ix Recommendations The Mission should: 1. Develop strategies and means to support the MOH as a partner in primary health care, and particularly in the development of a national health and management information system 2. Recognize the advanced expertise of centrally funded Cooperating Agencies while retaining policy responsibility for in-country activities that it supports. 3. Take advantage of the experience gained by the PHP partners through their successful clinic renovation and equipment program. The MARAM project should: 4. Assist the MOH and NGOs in ensuring quality of care, including performance according to standards. Accountability should be clearly assigned at the clinic level as well as for the oversight of group of clinics within the MOH or NGOs. 5. Review the operations research findings of the PHP with the objective of determining how to use them to improve maternal and child health services. 6. Ensure (1) joint determination of operations research questions with the MOH, universities, NGOs, and communities; and (2) peer review of the study designs. 7. Assist the MOH in reviewing the recently developed protocols and guidelines for reproductive health to determine priorities for further investments. 8. Coordinate activities relating to management information systems with the current initiatives by the MOH. 9. Assist the MOH in the definition of a minimum set of indicators for primary health care clinics 10. Assist the MOH in evaluating the Unified Patient Medical Records and related software tested by HDIP before testing the system in the MOH and other clinics. PHP Final Evaluation Report x Concluding statement The PHP has provided a useful platform for the development of a national level program of enhanced reproductive and child health services by the Mission. Issues for service delivery consistent with universally recognized standards have been identified. The capacity to conduct operations research pertinent to the problems of reproductive health has been enhanced as a result of the PHP experience. A modality for coordinated planning among major NGOs, international organizations and the MOH was created and serves as a prototype for future major health project. Of critical significance, as a product of the PHP, has been the recognition of the necessity for a final point of project accountability to be assigned to the Mission even in the context of delegation of operational responsibility to Cooperating Agencies. The investment in the PHP appears to be one that has resulted in a set of benefits that equates with the costs. PHP Final evaluation report 1 I. Introduction The purpose of this evaluation was to provide USAID/WBG Mission (thereafter, the Mission) with a documented statement of impact, results achieved and lessons learned from the Pilot Health Project (PHP). The evaluation team consisted of Drs. Marc Debay and Matthew Tayback (thereafter, the evaluators). The evaluators adopted the schedule proposed by USAID/WBG (Appendix 1), starting with in-person and phone interviews in Washington. This was followed by a series of in-country meetings and site visits between February 4 and 22, 2002. Specifically, the evaluators have (1) reviewed a wide range of documents, which include early assessment studies, periodic work plans and progress reports, sub grant agreements and research protocols, and end of task reports (Appendix 2); (2) interviewed project stakeholders1 in the West Bank, Gaza, Tel Aviv, and Washington; (3) visited selected clinics in the West Bank and Gaza. This report commences with a brief history of events leading to the approval of the Pilot Health Project and a review of the project design at its inception. An overview of the project implementation is provided where the timeline, human resources, funding, and reporting mechanisms are described. The report then reviews the implementation of the major project components with specific reference to the stated objectives, efforts made, results achieved, and lessons learned. The report concludes with a review of the main accomplishments, shortfalls, and recommendations. The evaluation team was fully cognizant of the extraordinary burden which all involved in PHP faced as a result of the Intifada, and have considered this in reaching conclusions in respect to appropriateness, effectiveness, and efficiency of the various components of PHP. 1 In this report, the major stakeholders of the PHP are USAID/WGB and Washington, CARE International, the Population Council, the five Palestinian implementing partners, the Ministry of Health, and other donors such as UNWRA and UNFPA. PHP Final evaluation report 2 II. Project design Following the 1997 Results Report and Resources Request for the West Bank and Gaza, the Mission and a Technical Working Group in USAID/Washington conducted various analyses to address the rapid population growth in the West Bank and Gaza. They developed three options for a possible Family Health Initiative. In December 1997, a three-person team (consisting of Keys MacManus, Zynia L. Rionda, and Amita Barbey) conducted a rapid assessment in the West Bank and Gaza to further explore these options. The final recommendation of this assessment process was to implement a comprehensive postpartum service program of integrated maternal and child health services in partnership with the NGO sector, and to employ an operations research-based demonstration approach. In 1998, the Mission decided to commit its resources to finance and coordinate a health program that would begin with a short-term pilot activity consistent with the recommendations of the 1997 assessment team, but would eventually become a longer￾term investment in the health sector. In November 1998, the Mission hired a Health Advisor under a US Personal Service Contract (USPSC). The Health Advisor’s tasks were to further develop the design and manage the new pilot health activity, and to concurrently oversee the development of the longer term USAID/WBG strategy in the health sector. In January 1999, the Mission approved a Concept Paper developed with assistance of a team from USAID/W. This document outlined a 28-month activity called the Pilot Health Project (PHP). The following month, the Mission decided to obligate funds to support this work. As the 1999 Concept Paper served as the principal substantive document that guided the implementation of the PHP, its main features are reviewed below to provide for a reference against which to evaluate the PHP. A summary of its strengths and weaknesses is provided at the end of this review. PHP Final evaluation report 3 A. Problem statement The Concept Paper highlights the following problems to be addressed by the PHP in the West Bank and Gaza: 1. High fertility with short birth intervals in the West Bank and Gaza is one of the most serious health problems for both mothers and children. 2. Contraceptive use remains very low in some isolated areas among young, low parity women, while the desired family size has fallen, particularly among younger women. 3. There is a serious lack of care to mothers and newborns during the perinatal period. Only 17% of mothers who received prenatal care return for postpartum checks. 4. While the availability to Primary Health Care (PHC) services is satisfactory, with more than 90% of the population living less than 5 km from a PHC center, there are problems of equity in access to quality care. Thus, only 50% of the population is covered by health insurance, and NGOs often offer services of higher quality then the MOH but on a fee basis. In the course of project implementation, the Mission and the various implementing partners also recognized that breast and cervical cancer were important problems to be addressed by the PHP. With the eruption of violence and travel restrictions in September 2000, they also began to address the problems of accessibility to emergency and general primary health care. B. Objectives The goal of the PHP was to improve the health status of Palestinian mothers and their children. The overall strategy to achieve this goal was “to improve the quality of antenatal and postpartum care by focusing on approaches for reaching the mother and the newborn as a pair.” In addition to upgrading the related services provided by NGO clinics in selected underserved areas, the PHP was to provide the opportunity to PHP Final evaluation report 4 determine the most effective approaches to reach mothers during the antenatal and postnatal period through operations research. The series of specific objectives outlined in the Concept Paper was not completely addressed by the proposed interventions described later on. For instance, three out of the seven objectives related to education of family, community leaders and the general public about the importance of antenatal and postnatal care, while only counseling of pregnant and postpartum women was proposed as an intervention to support these objectives. Another specific objective related to the promotion of the concept of “birth preparedness,” but this was not referred to anywhere else in the Concept Paper. The expected results listed in another section of the Concept Paper were more specifically related to the proposed interventions. They included processes (postpartum home visits institutionalized practice among NGO partners; MIS standardized and strengthened; etc.) and outcomes (increase in knowledge of infant and maternal health among fathers; increase in rates of postpartum visits and acceptance of family planning). A series of health outcome indicators and some quantified targets were proposed for inclusion in the baseline and post-test surveys and in the MIS, but needed to be further developed. C. Interventions The Concept Paper limits the PHP in time (28 months of project implementation) and place (areas served by about 30 NGO clinics in Jenin, Hebron, and Gaza). It also specified the Palestinian NGOs that operate the clinics in the selected areas. Within that framework, the selected NGOs are to implement in their clinics a basic package of services to improve the quality of antenatal and postpartum care. This package is described in the Concept Paper to include: 1. Improved antenatal care; 2. A single home visit community health worker 2-3 days after discharge from the hospital, or after delivery in the home, to assess the health condition of PHP Final evaluation report 5 both the mother and the newborn and to provide health education about breastfeeding, nutrition, immunization and care of the newborn; 3. Counseling the mother on birth spacing and on effective methods for postpartum contraception; and 4. Prenatal encouragement to bring the 10-day-old infant to the clinic or hospital for a check up by a physician. Special encouragement for the mother to return to the clinic with the newborn on the 40th day after delivery for postpartum assessment of the baby and mother, including counseling about the health benefits of optimal birth spacing. One of the main purposes of the Pilot Health Project was to test this basic package of services to determine the potential for scale-up. Thus the various elements of this package had to be further defined at the beginning of the project in order to plan the activities to support their implementation: renovation and equipment of clinics, training of health providers, development and distribution of behavioral change materials, and development of a management information system, etc. In the course of project implementation, the PHP implementing partners introduced several changes in the basic package of interventions. One early addition was the promotion of self-breast examination and Pap smear tests to address the problem of breast and cervical cancer. The decision to add these interventions within the scope and timeframe of the PHP as described in the Concept Paper reflects the perception by the NGO partners of the urgency of this problem in the West Bank Gaza. In addition to the basic package of services, the Concept Paper proposed to test three innovative interventions to increase the use of postpartum services and birth spacing: 1. Provision of a second home visit to low parity women prior to the Day 40 postpartum visit at the clinic. PHP Final evaluation report 6 2. Involvement of male village health workers to motivate fathers, grandfathers, and male village leaders in promotion of reproductive health. 3. Provision of a post delivery visit in hospitals and an early postpartum home visit to women who did not have prenatal visits, to encourage them to go to the clinic for postpartum care. The Concept Paper merely lists all the critical tasks to be conducted to implement the above interventions in the selected areas: initial needs assessment of the clinics (renovation and equipment) and health providers (training needs), renovation and equipment of clinics, development of training curriculum and training of service providers, development of a management information system, etc. The list of topics for training clinical and outreach workers does not specifically orient towards the implementation of the basic package of services. No estimate of the amount of time needed and number of trainees involved is provided. In a one-year schedule of activities in appendix, the training of community health workers and that of physicians anticipates completion within 5 and 9 months after the beginning of the project. Reference to a Management Information System includes the development of standardized forms and reporting mechanisms to be used by the implementing partners, with no further detail. A series of studies was proposed to support the PHP and the development of future USAID programs in the health sector: national level formative research; a national study on maternal and perinatal morbidity mortality; a national demographic and health survey (DHS); etc. The DHS was not implemented. Other studies were conducted outside the PHP. They are not reviewed in the present final evaluation (see completed studies in the list of references). D. Management The Concept Paper proposes the following management arrangements for the PHP: 1. Within the Mission, management oversight was to be assigned to a USPSC Health Advisor (already hired at the time), assisted by a Foreign Service National public PHP Final evaluation report 7 health professional (to be hired in July 1999 as a Program Specialist). Provision was made for close involvement of the Mission in the further development of the design and launching of the PHP. 2. To provide technical and managerial support, USAID was to fund two Cooperating Agencies through centrally administered cooperative agreements: • CARE International to oversee Service Delivery (SD) infrastructure enhancement through their Management of Reproductive Risk (CARE-MoRR) cooperative agreement (HRN-A-00-98-00023-00). • Population Council to oversee Research, Training, and Analysis (RTA) through their FRONTIERS cooperative agreement (HRN-A-00-98-00012-00). 3. CARE International was to provide a part-time, senior, medically trained coordinator and two area coordinators in the West Bank and in Gaza. FRONTIERS was to provide a senior national manager and a coordinator in the West Bank and Gaza. 4. CARE International was to negotiate sub agreements with three established Palestinian NGOs running the clinics designated for the PHP: Patients Friends Society (PFS), Union of Palestinian Medical Relief Committees (UPMRC), and Union of Health Work Committees (UHWC)--thereafter, the PHP SD partners. CARE-MoRR to also procure vehicles, equipment and supplies to ensure the quality of services delivered by its sub grantees. 5. FRONTIERS was to negotiate sub agreements with a Palestinian research institution, Health, Development, Information and Policy Institute (HDIP), and a local training center, Center for Development of Primary Health Care (CDPHC)--thereafter, the PHP RTA partners. PHP Final evaluation report 8 6. Coordination was to be achieved by (1) a working relationship between CARE and the Population Council and (2) the establishment of a Coordinating Council, membership in which to include USAID, CARE, the Population Council, the PHP partners, the MOH, and UNRWA. Appendix 3 shows the organizational and management structure of the Pilot Health Project proposed in the Concept Paper. This structure was maintained throughout the implementation of the PHP, except that CDPHC also undertook one of the main operations research studies, and that another partner, Alpha International, joined the PHP through a sub agreement to conduct the post-test survey of the basic package of services. Both Cooperating Agencies contributed to the project design, and in to preparation of the Concept Paper in particular. CARE, with a long-term presence in the West Bank and Gaza and UNFPA-funded reproductive health project in 15 clinics in Jenin coming to a close, provided invaluable information on field operations, logistics arrangements, and meetings with partners. Population Council provided the services of a staff member from the Regional Office in Cairo to join the team that prepared the Concept Paper. This participation enabled both CARE and Population Council to be fully aware of the PHP when awarded funds for its implementation. E. Strengths and weaknesses of the project design The structure of PHP reflects circumstances at the end of 1997 when no health planning capability existed in the Mission. The MOH was in an early stage of its development, having been established no earlier than 1994, and the NGOs were providing most of the health services. Only a limited number of Palestinian organizations had a history of management in health delivery systems or in the conduct of operations research in the health sector. A wide range of international organizations were active in the health sector including UNICEF, WHO, UNRWA, UNFPA, EU, IPPF, and it was therefore important to establish a strong coordination mechanism for the project that also involved the MOH and other donors. PHP Final evaluation report 9 Given this context, the strength of PHP design was that: • The focus on the mother and infant dyad, and on postpartum care in particular, addressed a well-established gap in the health system in the West Bank and Gaza. • The development and documentation of effective interventions through operations research was a good way to demonstrate USAID’s comparative advantage as a donor in the health sector. • The limited scope and timeframe was an adequate way for the Mission to begin activities in the sector while preparing for scale up. Although the overall design of the PHP was sound, the Concept Paper did not provide a detailed plan of activities, a monitoring plan, and a budget. Although this was not expected for a concept paper, it has been a weakness since this document was used as the main substantive reference for the PHP, and a detailed and concerted implementation plan was absent. The following are other shortcomings of the project design as outlined in the Concept Paper: • No participation of the beneficiary population and of the Palestinian partner NGOs in the design of the project and in the definition of the research topics. • No provision for the participation of the MOH, except for a consultative role in the Coordinating Councils or technical committees. • No provision for community involvement in the design and management of services in the clinics. • No community education and mass media program (as discussed above, three specific objectives are related to the education of the general population but there is no description of interventions with this respect). • No account for administrative and cultural differences and communication difficulties between the West Bank and Gaza. • No organizational linkage between the RTA and the SD components of the PHP, particularly with respect to the implementation of the basic package of interventions in the clinics. PHP Final evaluation report 10 • No specification of the type of technical assistance needed for project oversight by the two Cooperating Agencies. • Limited generalization from results of operations research because of the exceptional focus on Jenin (20/27 clinics) and an absence of MOH clinics. PHP Final evaluation report 11 III. Project implementation A. Timeline Appendix 4 shows the timeline of the development and the implementation of the PHP in the context of the general and health programs of the Mission from 1997 through 2002. The initiation of the PHP in 1997 occurred at a time when the Mission had no prior health programs since its creation in 1994, and therefore no established presence or working relationships in the health sector in the West Bank and Gaza. Less than three years after launching the PHP in early 1999, and despite the challenge of the Intifada starting in September 2000, and the related travel restrictions and insecurity, the Mission had brought this new project to completion, adopted a health and population Strategic Objective, approved two other larger health sector projects (MARAM and EMAP), and established a USDH Health Officer position. Appendix 5 presents the timeline of the main PHP activities from August 1999 through March 2002. It is important to understand this sequence of events to identify and assess the results and constraints of the implementation of the project, further discussed in section IV. Although the Concept Paper was approved and the Mission decided to obligate funds in February 1999, the beginning of PHP activities was delayed until August 1999 when the Cooperating Agencies received their respective funding. Thus, with an expected date of completion of September 2001, the duration of the project was initially 26 months. As the date of completion changed to December 31, 2001, then to March 31, 2002, the duration of the project implementation extended to 32 months. General capacity building activities such as facilitation of the Coordinating Council meetings and overall grants management and monitoring were conducted throughout the duration of the project. Specific capacity building activities such as financial and management assessment and training occurred during the first semester of 2000, that is, PHP Final evaluation report 12 when various sub grant agreements had already been signed with each partner, and when substantial activities had already been undertaken. The confirmation of the clinics that each partner would include in the PHP was completed in October 1999. The bidding and contracting process for the renovation work began in February 2000. A total of 15 clinics were renovated by June 2000, but only by February 2001 was the final number of 25 renovated clinics achieved. The needs assessment and the development of a standardized list of medical and non￾medical equipment and supplies for the clinics took a total of five months, starting in October 1999. The tender of the equipment and supplies took another eight months, and the distribution was completed at the end of 2001. As a result of these various delays, the 27 clinics of the PHP were renovated and fully equipped in December 2001. The training program began with a rapid assessment of the standards of health care conducted in August 1999. The development of the training curriculum and the preparation of the training program then extended through March 2000. The delivery of training started in July 2000 with the Training of Trainers, nearly a year after the beginning of the project. The training of the CHWs could only be offered in December 2000, and that of physicians and nurses in April 2001. CDPHC then organized refresher￾training workshops in June and July 2001, and conducted supervisory visits until October 2001. The development of the MIS began with an extensive assessment of the current systems used in the three PHP SD organizations, and the preparation of a proposal for the development of the new system. This first phase lasted five months and ended in March 2000. The HDIP proposal generated many discussions between the Palestinian NGOs, the CAs, and the Mission health team. The negotiations were eventually conducted directly with the Mission, and the sub grant was only signed in October 2000. Pilot testing of the new system began in August 2001 and was still on-going at the time of the evaluation. PHP Final evaluation report 13 The evaluation of the basic package of services consisted of two phases, a baseline survey and a post-test survey. The collection of baseline data began in May 2000, and preliminary data were available in August 2000. The data collection for the post-test survey, conducted by a new partner organization, was completed in December 2001. No preliminary data was available in February 2002 for the present evaluation. Although originally intended for a relatively early start and of short duration, the three operations research studies were delayed by the development of the protocols and sub agreements with the partner organizations. The implementation began in early 2001 and was severely affected by the travel restrictions and insecurity since the Intifada. Despite these problems, the training and data collection for all of the studies were completed, and preliminary analyses and draft reports were available at the Population Council regional office, in March 2002. The eruption of violence on and following September 28, 2000, adversely affected all project activities because of the frequent and often unpredictable closures, the danger associated with traveling between towns, and the cancellation of travel permits. This affected particularly the gathering of partners for the Coordinating Councils, the organization of training events, the supervision of renovation and other activities in the clinics, the field implementation of the operations research studies, and the provision of health care in the clinics. Another consequence of the travel restrictions has been an increase in attendance for emergency but also for primary health care services in many clinics. B. Human resources Mission The Mission hired a USPSC Health Advisor in November 1998, before launching the PHP. The Health Advisor played an important role in finalizing the Concept Paper, in establishing positive relationships with the various implementing partners and other PHP Final evaluation report 14 donors, in orienting the FSN Health Advisor in charge of the PHP, and in developing the health portfolio for the Mission. Cooperating Agencies As soon as funds were available, the Population Council hired a Program Associate to manage the PHP. Initially based in Cairo to provide for close supervision by a senior staff member in the Regional Office, the Program Associate was relocated to Ramallah after a year, that is, after sub agreements were negotiated with the partner NGOs and the project activities began. The Population Council also hired a Palestinian public health physician as Project Manager at the beginning of the PHP to manage day-to-day activities from the Council’s office in Ramallah. Together, these two full-time professionals had good public health background but not the specific experience required to efficiently lead the development and implementation of the operations research and training activities of the PHP. However, the Population Council Regional Program Associate based in Cairo, and the Associate Director for Asia and Near East based in New Delhi, provided oversight to project implementation on a part-time basis (approximately 50% and 25%, respectively). This oversight included frequent trips to the West Bank and Gaza. At the beginning of the project, the FRONTIERS Director also visited the Mission and the PHP partners in the West Bank and Gaza to assist in the development of the work plan for the RTA component of the project. All the Population Council staff remained on the project throughout the end of the year 2001, and the regional staff members continued supporting the project until the end of the extension in March 2002. At the time of the evaluation, the Population Council regional staff had plans to continue working on the PHP until July 2002 to complete the operations research studies and disseminate the findings through a workshop and other activities in the West Bank and Gaza. CARE hired a full-time Palestinian public health physician as Project Manager as soon as PHP funds were available. The Project Manager was instrumental in facilitating the definition of technical standards for the renovation and equipment of the clinics through the Coordinating Council meetings, and in overseeing all the PHP activities. The Project Manager’s departure in May 2001 left CARE without sufficient senior public health PHP Final evaluation report 15 management capacity until the end of the PHP. The Country Director also had substantial involvement in the management of the PHP, including a pivotal role in the Coordinating Council meetings. The Country Director was able to build positive relationships with the local partner organizations, including constructive negotiation processes. CARE also maintained one field coordinator in Jenin and one in Gaza, and several support staff members. Finally, the CARE/Headquarters Senior Reproductive Health Advisor made three trips at the beginning of the project to help develop a strategic plan for the Service Delivery component of the PHP, and define working relationships with Population Council. Sub Grantees The initial five, and later six, PHP partner NGOs showed a clear commitment to the project through the regular and active participation of the highest level executives in the Coordinating Council meetings, and the assignment of staff to the various project-funded positions and activities. The sub grants to the three PHP SD partners included about a third allocated to personnel (see section below). This often included salary of the physicians, community health workers, or other staff in the clinics. Similarly, the series of sub grants to the PHP RTA partners all included significant percentage allocation to personnel. Undoubtedly, the PHP provided opportunities to many health professionals to gain invaluable experience in service delivery, training, and operations research in primary and reproductive health care. Presumably, this experience gain during the PHP enabled these health professionals and the organizations employing them to provide better primary and reproductive health services and better compete for other grants in the future. One PHP SD partner, PFS, plans to use the income from the fees collected by the clinics to pay the physicians and community health workers. This is now possible because of the increase in attendance in many of clinics. PHP Final evaluation report 16 C. Work plans and management As indicated in section II, the Concept Paper constituted the technical description of the activities to be conducted by the Population Council and CARE under the PHP. Upon receipt of their funding, CARE and the Population Council developed operational details for achievement of the goals stated in the Concept Paper, and submitted work plans to the Mission. In the context of centrally administered cooperative agreements, however, the Mission does not have a formal mechanism to approve CAs’ work plans. This situation may have created a sense of uncertainty and lack of confidence among the staff of the CAs, and may also have urged the Mission health team to closely monitor or even get involved in operational aspects of project implementation. Both CARE and the Population Council implemented the PHP through sub grants with their respective Palestinian partners, but in a somewhat different way. After submitting its work plan to the Mission in August 1999, the Population Council began developing various contracts and sub agreements with its partners. The Population Council sub agreement documents demonstrated careful discussion and negotiation of the technical aspects of the work to be done by the partner organizations. They included the rationale and purpose of the grants, the activities to be conducted, the timeline, the roles and responsibilities, and the budget. This represented a very good investment in terms of building the capacity of these organizations. The Population Council appeared to have difficulty in negotiating the budgets proposed by its partners, however, and this often led to delays in obtaining final approval by the Mission. Using a standard contract and the Concept Paper as scope of work, CARE was able to sign sub agreements with its three partners in December 1999. This was soon followed by the payment of a first advance. The management of these sub agreements was based on a strong partnership among the three partners organizations built through the coordination and decision-making role of the Coordination Council (see section IV.A). Also, the PHP SD partners were required to submit detailed quarterly progress and financial reports that CARE would approve before making a new payment. The PHP SD partners otherwise had the latitude of managing their own grants. This included, for PHP Final evaluation report 17 instance, selecting contractors for the renovation of their clinics. Both the CARE Country Director and the PHP Project Manager were instrumental in building confidence and competence among their partners for all programmatic and financial reporting requirements. In the course of this grant management process, CARE was in general able to approve financial reports from its partners without any problem. Some issues arose with UHWC in relation with payment of personnel, buying property for the clinics instead of paying for renovations, and, more recently, the political affiliation of the organization. D. Funding Cooperating Agencies Table 1 presents a summary of the total USAID obligations to the Population Council and CARE, and the related expenditures by five main budget categories. These estimates were compiled for the purpose and at the time of the final evaluation, and confirmed by the administrators of cooperative agreement (FRONTIERS and MoRR). Appendix 6 and Appendix 7 present a more detailed breakdown of these obligations and expenditures. An estimated total of $5,268,852 has been expended on the Pilot Health Project, of which USAID/WBG obligated $2,155,000 to Population Council and $2,353,852 to CARE International. Overall, 36% of the total expenditures represent sub grants to the PHP partner NGOs, This percentage is 43% for the Population Council and 28% for CARE. The sub grants and the total amount for procurements given to the PHP partners represented 59% of the expenditures by CARE. The total amount spent on personnel was equivalent for both CAs (27% for the Population Council and 24% for CARE). The Population Council used more expatriate personnel than CARE and incurred expenses through the frequent travel of its staff between Cairo and the West Bank and Gaza, but the cost of the Cairo office was not charged to the PHP. The indirect costs from the Population Council were several times higher than those of CARE. On the other hand, the Population Council will have contributed about $467,000 to the PHP from population core funds when all the planned activities are completed. PHP Final evaluation report 18 Table 1 Summary of Obligations to the PHP and Expenditures by CARE and Population Council Population Council CARE Total Obligations 2,685,000 2,583,852 5,268,852 % % % Expenditures 2,408,249 100.0 2,377,267 100.0 4,785,616 100.0 Personnel 589,670 27.0 575,579 24.2 1,227,023 25.6 Procurement 0 0.0 765,370 32.2 765,370 16.0 Sub grants 1,046,132 43.4 656,978 27.6 1,703,153 35.6 Other direct costs 168,265 4.4 208,927 8.8 315,449 6.6 Indirect costs 604,182 25.1 170,413 7.2 774,620 16.2 Balance 276,751 206,584 483,236 Note: The USAID/WBG Mission obligated $2,155,000 to Population Council and $2,353,852 to CARE International. Source: Estimates compiled by the evaluators and confirmed by FRONTIERS and CARE-MoRR administrators in March 2002. The breakdowns of the obligations to and the expenditures by each CA are provided in Appendix 6 (Population Council) and Appendix 7 (CARE). Sub Grantees The three PHP SD sub grantees to CARE each received equivalent funding. For analytical purpose, Table 2 presents the breakdown of the budget of the sub agreement of each PHP SD partners at the time of their signature in December 1999. Overall, more 37% was budgeted for personnel and 30% for the renovation of clinics. PFS, with 15 out the 27 clinics participating in the PHP, budgeted 40% of its funding on renovation, and less than 5% on operations and 0% administration. PHP Final evaluation report 19 Table 2 Budget allocations by CARE for the three sub grants to the PHP Service Delivery partners UHWC UPMRC PFS Total $ % $ % $ % $ % Personnel 104,354 36.7 106,100 39.1 97,700 36.2 308,154 37.3 Renovations of clinics 84,400 29.6 56,240 20.7 107,239 39.7 247,879 30.0 Outreach 22,300 7.8 44,100 16.2 45,111 16.7 111,511 13.5 Operations 34,200 12.0 30,000 11.0 12,350 4.6 76,550 9.3 Administration 33,577 11.8 20,115 7.4 0.0 53,692 6.5 Staff Training 4,643 1.6 10,000 3.7 5,000 1.9 19,643 2.4 Miscellaneous 1,200 0.4 5,000 1.8 2,600 1.0 8,800 1.1 Total 284,674 100.0 271,555 100.0 270,000 100.0 826,229 100.0 Note: Budgets from the December 1999 sub agreements. As of December 31, 2001, UHWC had only spent $41,640 on renovations of clinics while UPMRC and PFS had exhausted their budget. As shown in Table 1, the Population Council spent 43% of its funding on a series of sub grants and contracts. Table 3 lists the scope of work, total budgets and effective dates of these sub agreements with the three PHP RTA partners (HDIP, CDPHC, Alpha International). As mentioned in section III.C, these sub agreements are all based on detailed proposals jointly developed by the Population Council and the respective partners. Table 3 also lists various contracts with HDIP and CDPHC that were set up as part of an “In-house Technical Assistance Project” to provide funding for preliminary tasks while waiting for the Mission's initial obligation to be formally received. Indeed, the Population Council could only offer formal sub agreements for the activities to be implemented by the PHP RTA partners after it received the obligated funds. PHP Final evaluation report 20 Table 3 Scope of work, total budget and effective dates of sub awards and contracts by the Population Council to its local partners in the West Bank and Gaza Scope of work Total Budget Effective Dates CDPHC Assessment of Standards of Health Care Services (External consultants) IH-TA Aug 99 – Sept 99 Design of Health Education/Behavioral Change Messages for the PHP IH-TA Sept 99 – Oct 99 Development of Training Manual and Curriculum, and of Proposal for Training IH-TA Sept 99 – Dec 99 Develop Behavioral Change Materials Prototypes IH-TA Oct 99 – Dec 99 Training Program for the Pilot Health Project $262,000 Jun 00 – Dec 01 Improving Postpartum Care Among Low Parity Mothers in Palestine $70,394 Mar 01 – Mar 02 HDIP Development of Baseline Survey Research Proposal IH-TA Aug 99 - Oct 99 Assessment of Existing Management Information Systems among PHP Service Delivery Partners IH-TA Aug 99 - Oct 99 Development of MIS Proposal IH-TA Sept 99 – Nov 99 Baseline Survey for the West Bank / Gaza Pilot Health Project 68,322 Mar 00 - Aug 00 Institutional Support for a Program of Activities with the West Bank/Gaza Pilot Health Project. This incorporated: 1. Institutional support to build research capacity 2. Outreach linkages study with public hospitals 3. Involving men study $274,699 Mar 00 – Feb 02 Creation of a Unified Management Information System for three NGOs in the West Bank/Gaza $235,837 Nov 00 – Apr 02 Alpha International Post-test Survey for the West Bank / Gaza Pilot Health Project $58,832 Aug 01 – Feb 02 Special Studies $31,000 Total In-House Technical Assistance Project, IH-TA $112,660 Notes: Total budgets and effective dates include end-of-project extensions. IH-TA: In-House Technical Assistance project (see text). PHP Final evaluation report 21 E. Reporting In March 2000, CARE adopted a quarterly work plan and reporting format that included financial information. Until then, CARE had provided short narrative reports to the Mission. All these reports were regularly sent to the Mission until April 2001. As secretariat of the Coordinating Council, CARE also prepared and distributed minutes of all its meetings. These reports are a very valuable source of factual information and provide insight into the dynamics of the relationships among the various partners and stakeholders of the PHP. The three PHP SD partners sent quarterly reports to CARE, who reviewed them before disbursing additional funds. These reports did not include data on activities in the clinics because it was expected that the MIS would provide this information. There was no final report from the PHP SD partners available at the time of the evaluation. The Population Council did not systematically provide financial reports to the Mission because the FRONTIERS cooperative agreement does not require reporting by line item within sources of funds (core funds, field support, and MAARD agreements), and therefore the Population Council accounting department does not provide this information. The Population Council only reports by line item against one simple source into FRONTIERS, that is, USAID. FRONTIERS program staff does collect financial information by individual country on an informal basis, but this information is only compiled at the end of each fiscal year to estimate program costs and is not suitable for financial reporting to the Mission. This absence of current financial information on Population Council activities in the West Bank and Gaza provided limited opportunities for oversight and monitoring by the Mission. On the other hand, the Population Council and the PHP RTA partners produced numerous technical reports on their specific activities, including a series of four Updates written for a larger audience (see Appendix 2 for a list of these reports). At the request of the Mission Health Advisor, the Program Associate provided bi-weekly progress PHP Final evaluation report 22 reports on project activities and constraints, and maintained regular personal contacts. The regional staff in Cairo and Delhi also had telephone conversations with the Mission between their visits in the West Bank and Gaza. PHP Final evaluation report 23 IV. Project results A. Coordination and capacity building Coordination Among the initiatives launched through the Pilot Health Project was the creation of a Coordinating Council to guide the management of the project, harmonize the functions of its two major components (Service Delivery--SD and Research, Training and Analysis-- RTA), and ensure coordination with the MOH and other donors in the health sector in the West Bank and Gaza. The Coordinating Council was first convened under the chairmanship of the Mission Health Advisor in March 1999 in Ramallah. During that meeting, the Ministry of Health expressed its support for the funding of NGO’s through the PHP, but emphasized that NGO activities must be consistent with MOH policies, protocols and standards. The MOH also offered to participate in the operational committees assigned by the Coordinating Council. The first two areas for coordination were identified as (1) assessment of service facilities and service providers; and (2) planning for a Management Information System. During this first meeting, the Mission Health Advisor proposed the development of a Vision Statement for the Coordinating Council, which was subsequently discussed during several meetings before its adoption (see Appendix 8). During this early period of the project, the Coordinating Council members also discussed the Concept Paper, translated into Arabic for that purpose, and began discussing project activities in detail. Meetings of the Coordinating Council were held nearly monthly from September 1999, when the actual project activities started, until July 2000, two months before the Intifada began. The content of these meetings covered operational details of the two major components (SD and RTA). Frank discussion covered the duties and charge of the Coordinating Council, and the relationship in decision-making between local partners and the two Cooperating Agencies. PHP Final evaluation report 24 During the second year of project implementation, the Coordinating Council only met once in April and May 2001. This was at the time of major change at the Mission, when the new USDH Health Officer succeeded the PSC Health Advisor, and the CARE Health Project Manager was recruited as an FSN Health Advisor. No other meeting was organized until the final meeting of the Coordinating Council held on January 3, 2002. The Mission Director and the health team, the Deputy Minister of Health and the MOH staff, and representatives of all PHP partners and of other Mission health projects were present at this meeting. The minutes of the meeting summarize the accomplishments of the Pilot Health Project as perceived by the principal partner organizations. A primary objective of the Council was to avoid redundancy in respect to the programs of the MOH, other NGOs and donors, and to utilize fully the materials and resources already designed and tested. This objective was at least partially achieved through the regular participation of the MOH, and the open exchange of information among the Coordinating Council members. Examples of PHP activities discussed at the Coordinating Council level, and that the MOH and other donors were also supporting, are the development of clinical protocols and standards, the development of an MIS, and the adoption of a list of equipment for maternal and child health services. CARE played a leadership role in assuming the secretariat of the Coordinating Council. This responsibility included consultation with the elected chairman prior to the meetings, logistical support (including requests and follow-up on travel permits for PHP partners), preparation of the agenda, facilitation of meetings, keeping the meeting minutes and distributing them to the various stakeholders. This role has been widely appreciated by the PHP partners and the Mission. The roles and responsibilities of the Coordinating Council have not always been clear. Some decisions by the Coordinating Council, for instance, were not in line with project objectives (development of a MIS for use at the national level; addition of cervical and breast cancer screening in the basic package of interventions). Also, some lengthy discussions and decision-making processes delayed project implementation. However, PHP Final evaluation report 25 the regular and well-organized Coordinating Council meetings fostered a strong collaboration and ownership among the PHP partners. A further opportunity for improvement of the process and promise of succeeding Coordinating Councils in follow￾on projects could result from a clear statement of the duties and policy authority of the Council. The continued use of venues in the West Bank for meetings of the Coordinating Council restricted participation by the senior staff of the Ministry of Health in Gaza. CARE was fully aware of the communication and coordination difficulties between the West Bank and Gaza. For that reason, CARE insisted on having a part time employee in Gaza to liaise with the Gaza partners. CARE Health Project Manager and Country Director also traveled on a regular basis to Gaza to visits PHP clinics and meet the PHP partners. Recent establishment of video-conferencing capability in the offices of the MOH in Gaza should remedy this difficulty, as demonstrated in the Health Sector MIS Consultative Meeting of October 2001. Capacity Building The management decision to implement the project through large sub-agreements with local partners was one of the strengths of the PHP. Both CARE and the Population Council placed a strong emphasis on building the capacity of their respective partners when developing, negotiating and monitoring their sub agreements. Their respective approaches to sub grant development and management are discussed in section III.C. The fact that the implementing partners were predetermined rather than selected on a competitive basis, however, sometimes put the CAs in difficult negotiating positions. In some instances, this has led to protracted negotiations and caused unwarranted delays in project implementation. Early in the implementation of the PHP, CARE contracted with a local branch of an international auditing firm to conduct systematic assessments of its partners’ management systems. This firm also provided tailored training in USAID regulations and procedures and other managerial areas as needed by each partner organization. Regular audits and PHP Final evaluation report 26 financial monitoring were conducted throughout the project. CARE found the quarterly audit to be costly but warranted. Given the success of this activity, CDPHC, one of the Population Council's partners, also requested this assistance and was later included in the program. The PHP provided multiple other opportunities for its local partners to participate in activities and events that strengthened their organizational capacity, mission, and exposure: • In November 1999, Hillary Clinton and Suha Arafat presided over the signing of the MOU between the PHP partners and officially launched the PHP. • In December 1999, one member of each of the PHP partners participated in a study tour to Jordan to examine the USAID-funded CCP project. • In March 2000, SEATS organized a technical workshop in Jericho to review the PHP objectives and preliminary accomplishments with all the partners, representatives from the Mission and USAID/Washington, and representatives from CARE/HQ and Population Council/Cairo. • In 1999 and 2000, the Population Council sponsored two Palestinian participants from the MOH and the Faculty of Nursing to attend the workshop that it organizes with the Cairo Demographic Center on “Operations Research in Family Planning and Reproductive Health.” • One sub grant to HDIP (March 2000 - February 2002) was specifically geared to promote its institutional capacity for conducting operations research. B. Renovation and equipment of clinics In September and October 1999, the PHP SD partners confirmed the 27 clinics to be included in the project according to the recommendations of the 1997 Assessment and the 1999 Concept Paper teams. In the course of project implementation, two PFS clinics were replaced and two UHWC clinics were not renovated but received other support from the PHP (supplies and equipment, human resources support and training). The list of the 27 clinics included in the PHP is in Appendix 9. PHP Final evaluation report 27 Table 4 shows the numbers of PHP clinics by district and NGO, and the total number of Primary Health Care (PHC) clinics in the same districts. Overall, the number of PHP clinics represents 4.5% of the PHC clinics in the West Bank and Gaza. They also represent 15% of the total NGO clinics and 15% of the total FP clinics in the West Bank and Gaza (The status of health in Palestine 2000). In Jenin, the PHP clinics represent 26% of the total PHC clinics. According to the Concept Paper, the PHP clinics served about 35,000, 20,500, and 20,000 women of reproductive age in Jenin, Hebron, and Gaza, respectively, that is, about 11% of the WRA. Table 4 Distribution of the Pilot Health Project and the Primary Health Care clinics by district and NGO in the West Bank and Gaza District NGO PHP Clinics PHC Clinics1 % Jenin PFS 16 UPMRC 4 78 25.6 Hebron UPMRC 2 155 1.3 Gaza UHWC 5 100 5.0 Total 27 333 8.1 Total West Bank Gaza 27 595 4.5 1 The status of health in Palestine, 2000. CARE selected the Palestine Hydrology Group (PHG) to conduct the physical assessments of the 27 PHP clinics. The PHG experts used tools based a WHO form and followed recommendations from the Coordinating Council to conduct these assessments in August and September 1999 in the West Bank, and in January and February 2000 in Gaza. CARE also assisted its partners in hiring engineers to develop the tenders for the renovations No major difficulties were encountered in the selection of vendors and contractors for the renovations. At their request, each partner took the lead on tendering and selecting contractors for their own clinics, and thereby built ownership in the process and results. CARE was able to provide assistance to each partner, however, facilitating the process PHP Final evaluation report 28 and ensuring transparency. For instance, a CARE representative was present on all bid opening boards. CARE also assisted the partners in the monitoring of the contractors to ensure the timeliness and quality of the renovations. The main difficulty in the renovation program has been a series of lengthy negotiations with UHWC, which found it a better use of project funds to buy property rather than renovating some of the clinics that it currently rents. As this proposition was incompatible with USAID policy, the Mission health team spent substantial efforts to finally be able to approve this special request, but UHWC found unacceptable some of the conditions for this approval. As a result, only three of the five UHWC clinics included in the PHP were renovated, and only part of the budget for renovation in the UHWC sub agreement was used. The negotiation difficulties also delayed the beginning of the renovation in these three clinics until September 2000. Overall, 15 of the 27 PHP clinics were renovated in July 2000, and 25 by February 2001. CARE did not impose any standard for the renovation of the clinics because all the clinics were very different. However, general public health and quality of care principles were taken into consideration during the process of clinic renovation. Thus, at the end of the renovation program, all the PHP clinics had: • Hand washing facilities in each examination room, washable walls, a washroom, and clean sanitary facilities easily accessible to clinic clients and health workers. (Before renovation, some clinics did not have any running water, which obviously limits the basic hygiene and infection prevention measures that can be implemented) • Electricity supply for lighting, TV/VCR, refrigerators, autoclaves, laboratory devices, x-ray viewers, etc. • Waiting, examination and counseling rooms accessible (proper pathways, stairs, doors, etc) and attractive (cleanliness, painting, pictures, windows, curtains, chairs) • Adequate privacy through a minimum of two examination/counseling rooms per clinic and a separate waiting room, all with proper doors, curtains, etc. • Beneficiary communities that had been involved in the renovation process. PHP Final evaluation report 29 An average of $5,000 per clinic was originally budgeted for renovations and equipment. In the course of implementation of this program, the actual cost of renovation alone turned out to be $8,000 per clinic on average, raising the total cost of this activity to $205,119. This budgetary constraint delayed the implementation of the renovation program. In October 1999, CARE began facilitating the preparation and adoption of a standard list of medical equipment, furniture, and disposable supplies for the PHP clinics. The Coordinating Council formed a sub committee on equipment that reviewed the requests from the three PHP SD partners and, using reference lists from the WHO and the MOH, adopted a limited list of items for which they developed the technical specifications. The final list was approved by all partners in February 2000, and formed the basis for a request for quotations. Appendix 10 presents the amount of medical equipment and disposable items distributed to each by PHP SD partner. The medical equipment is proportionate to the number of clinics, while the amount of disposables is the same for each partner. In addition to the medical equipment and supplies, the procurement included non-medical furniture such as desks and chairs, filing cabinets, chairs for the waiting rooms, refrigerators, televisions, videos, etc. (not in Appendix 10) Selecting vendors for the equipment was a very lengthy process, in part because of the USAID source and origin requirement that it be made and purchased in the US and the difficulty to obtain the related waiver when necessary. As a consequence, only half of the non-medical equipment was delivered in the clinics by January 2001, and none of the medical equipment had been delivered by April 2001 except for the 3 ultrasounds and 6 autoclaves. All the medical equipment ordered by the PHP was eventually distributed in the clinics in December 2001. PHP Final evaluation report 30 Overall, the total direct cost of the equipment provided to the 27 PHP clinics was $405,315, that is, about $15,000 per clinic. This component of the project was delayed but successful in terms of the adequacy of the equipment provided at to the clinics. Under the current Mission’s portfolio, the renovation of primary health care clinics is to be conducted under the Community Services Project, with support from MARAM for purchasing the equipment. Although this is appropriate for various reasons, including community participation and ownership, the Community Services Project should take advantage of the experience gained by the PHP partners in this successful component of the project. With the resumption of violence on September 28th, 2000, and the shortage in medical supplies, equipment and drugs that resulted from the increase in casualties and travel restrictions, the Mission allocated an additional $200,000, approximately, to CARE￾MoRR in support of a Rapid Emergency Response component to be added to the PHP. This enabled CARE to provide the Palestinian Ministry of Health, hospitals, clinics and NGOs with first aid and emergency supplies such as medicines, bandages, vaccines, stretchers, surgical instruments and other disposable supplies. This equipment and supply was distributed in the clinics throughout August 2001. The Rapid Emergency Response was successful but delayed the procurement of the other goods planned for the PHP clinics. C. Training and behavior change The two PHP specific objectives of (1) improvement of antenatal and postnatal care, and (2) of testing a basic package of interventions, assume the definition of standards of care. Improving the quality of care implies a change from one level of care - explicit or implicit - to a new, well-defined level. The basic package of interventions, outlined in the Concept Paper, presumably constitute the new standards to be introduced in the antenatal and postnatal services provided by the 27 clinics of the three PHP SD partners. PHP Final evaluation report 31 From June to August 1999, the Population Council contracted a team of consultants from Egypt who worked with CDPHC to assess the quality of care in the 27 PHP clinics. This assessment identified the areas of deficiency in skills and knowledge to be addressed by training, using observation of providers during their clinical practice, interviews of providers and managers, and client exit and home visit interviews. As there were no explicit standards of care implemented in the PHP clinics, the consultants developed a series of 9 assessment tools using WHO guidelines and standards. These tools are included in the assessment report, but not the standards themselves. Most indicators presented percentages of respondents satisfying 75% of a series of criteria characterizing a particular skill or area of knowledge. The actual items included in these unweighted, additive scales are not provided. Also, many of the estimates of these indicators are based in each instance on samples not exceeding 10 respondents. This makes it difficult to generalize the findings, unless these small numbers of respondents represent the majority of the staff in the clinics, but this is not clearly indicated in the report. Nevertheless, the strength and contribution of this assessment, conducted at a very early stage of the PHP, is its timeliness and the introduction of the concepts of standards of care and data based decision-making. The training needs identified by the assessment seemed broad but valid: essential antenatal and postnatal care; communication skills; infection prevention; quality assurance and management. The assessment also identified critical issues, such as the fact that very few postpartum home visits were made, usually by social workers as a reminder of a missed appointment rather than to provide care or education. Also, the assessment identified a lack of clearly identified management responsibility in the clinics, and a lack of referral system between primary health care units and higher level of care. Based on this assessment of standards of care, CDPHC and the Population Council agreed upon the topics for the training to conduct under the PHP, and began developing the related training curriculum and materials. This work was conducted with oversight from a Technical Review Committee appointed by the Coordinating Council that included members from all PHP partners and the MOH. CDPHC and the Population PHP Final evaluation report 32 Council also developed a proposal for the implementation of the training. As soon as this proposal was approved and the sub-agreement signed, CDPHC finalized the training curriculum and materials. Just before the beginning of the training program, CDPHC conducted a rapid assessment of the general background, the prior training in the proposed areas, and the expectations of the future trainers and of the health providers in the PHP clinics. This assessment was done using a self-administered questionnaire. At the same time, CDPHC contacted the senior staff of all the SD partners and MOH, and the managers of the PHP clinics, to ensure their full support of the training program. These consultations were also used to involve all the persons contributing to decisions regarding the organization and logistical arrangements for the various types of training events. The training of trainers was conducted from July to August 2000, and reached a total of 13 trainers from the three PHD SD partners and the MOH. This 20-day training program was provided to 6 physicians, 4 nurses, and 3 community health workers at the same time. A total of 43 community health workers were trained in December 2000 and January 2001, and 7 physicians and 8 nurses were trained in August 2001. Table 5 presents the distribution of trainers and trainees by institutional affiliation and professional categories. PHP Final evaluation report 33 Table 5 Distribution of Trainers and Health Care Providers trainees by institutional affiliation and professional categories Trainees UPMRC UHWC PFS MOH Total Trainers Physician 3 2 1 6 Nurse/Midwife 2 1 1 4 Health Worker 1 1 1 3 Total 3 4 4 2 13 Health care providers Physician 3 3 1 7 Nurse 1 5 2 8 Health Worker 11 17 15 43 Total 15 25 18 58 Source: List of participants, CDPHC The number of providers trained by the PHP probably represents the majority of those working in the 27 clinics. However, the rate of turnover among health providers is high (>20% per year), and this should be taken into account when assessing the potential impact of the training on the quality of services. After completion of the training program described above, CDPHC organized a series of follow-up activities. For instance, one post-training assessment provided information on the status of skills and knowledge among the health workers who participated in the training. Deficiencies were identified that formed the basis for two 4-day follow-up workshops provided in the West Bank and Gaza (4 workshops total) in June and July 2001. CDPHC also conducted supervision visits. After the training program was completed, CDPHC appointed a Manual Development Committee to publish the training materials developed so far. This manual was recently published and distributed to the three PHP SD partners. The training manual is considered by CDPHC as a reference to be adapted by the trainers to the particular audience of interest (CHW, nurses, physicians). It includes a series of modules on reproductive health written by selected trainers or other professionals. Each module PHP Final evaluation report 34 begins with training objectives. The text employs a fairly high-level language and medical terminology that may not be appropriate for CHWs. It does not refer to specific standards agreed upon for implementation and monitoring at the clinic level. The evaluators did not examine the actual training materials used for the various training events organized by the PHP, and therefore were not able to assess the emphasis placed on the specific aspects of the basic package of interventions described in the Concept Paper. From the training needs assessment to the publication of the training manual, CDPHC established participatory working relationships with the staff and the health providers from the three PHP SD partners. Overall, the different steps of the training program conducted by the PHP were all very much appreciated by the various persons involved: managers of the SD partners, master trainers, trainers, and trainees. The health providers recognize that they otherwise have very little opportunities for updating their knowledge and skills. A limitation of the PHP training program was that it was not accompanied by the definition of explicit standards to be applied at the clinic level and during home visits. One of the reasons may have been that an extensive, national level review, definition, and adoption of clinical guidelines and protocols in reproductive health was under way at the same time, and that the PHP partners wanted to avoid duplication. In fact, several Coordinating Council members participated in this process organized under the sponsorship of UNFPA. Given the short timeframe and the operations research emphasis of PHP, however, the clear definition and adoption of at least a few key elements of the basic package of interventions was necessary. These standards could then have been used to define the competencies to be achieved by each type of provider through training, to develop job aids and clinical guidelines, and to adopt performance criteria. This would have helped the staff in the PHP clinics to implement the basic package of interventions. This would have also provided clear indicators to monitor the compliance with these standards and measure the effectiveness of the package through the pre and post surveys. PHP Final evaluation report 35 One of the most successful and appreciated components of the training may have been the emphasis on counseling. This finding pertains to the community health workers, the nurses and the physicians. The counseling training was supported by the development of some educational materials for use during home visits, like leaflets on newborn care, on the danger signs during pregnancy, and on gestational diabetes. These leaflets are given to the women after the education talk is completed. The PHP also developed two posters on infection prevention for use in the clinics. Several aspects of the renovation and equipment of clinics support counseling and group education: dedicated rooms with appropriate privacy; posters and other educational materials displayed in various places; TV/VCR equipment in the waiting rooms. Other PHP strategies for behavior change listed as specific objectives in the Concept Paper were not addressed. For instance, no significant activity was conducted to educate families, community leaders, or the general public. CDPHC proposed to conduct such activities in their initial proposal for training, but this was abandoned during the negotiation process with the Population Council and the Mission. D. Management Information System The Pilot Health Project was designed to strengthen the capacity of 3 NGOs to deliver defined elements of prenatal, postpartum, early infant and reproductive health services through a group of 27 clinics. A key instrument for achieving this objective, and in determining the levels of performance achieved, was considered to be a practical management information system. The service activities under PHP were intended to serve as a base of knowledge and experience to guide an anticipated scale-up project, later identified as MARAM. In the 1997-1998-assessment report, reference to MIS is incorporated in the statement “Design and add-on to an existing MIS which will input the additional data essential for monitoring and evaluation of the changes in clinic activities.” In the Concept Paper, reference is made to a MIS with standardized forms and reporting mechanisms and to a time frame for this task of 3 months subsequent to contract signing with the 2 CA agencies. PHP Final evaluation report 36 The responsibility for accomplishing the MIS task was assigned to the Population Council with the understanding that HDIP would be the sub grantee to produce the product sought. The Population Council reached an agreement with HDIP on or about September 1999 to proceed with the MIS task. During a period of 7 months terminating March 2000, HDIP conducted an in-depth assessment of existing MIS procedures employed by PHP SD partners, examined the MIS of the MOH and of UNRWA and consulted with UNFPA. A report “Management Information System Assessment” was completed and proposed a substantial upgrade to existing information systems that could serve as a national information system in the future. The first full proposal submitted by HDIP to Population Council in March 2000 was rejected on the grounds of its content and cost. An alternative based on the adaptation of a software developed for a similar USAID-funded project in Jordan was thoroughly discussed during a meeting of the Coordinating Council, but rejected by HDIP and all the PHP partners primarily due to the difficulty of adapting the Jordanian system to the PHP context, and in the expectation that a HDIP-designed system would be more appropriate. This finding by the Coordinating Council was not in accordance with the Population Council staff advice. Given the delays on this component of the project, and the risk that this would also delay other components of the project, the Mission eventually recommended that HDIP pursue its proposition and allocated additional funds to this activity. The Population Council then finalized a proposal with HDIP, who then resumed activities on the MIS in October 2000. The HDIP proposal entitled “Creation of a Unified Management Information System for 3 NGOs in the West Bank and Gaza,” was funded at a cost of $235,837. It carried a time line of 15 months commencing September 2000 with reporting to commence on or about Feb. 2001. It specified a list of 42 illustrative indicators that the MIS should produce, but no other detail on their exact calculation, frequency and format of reporting and use. The indicators are categorized into outcomes, service/health, and socioeconomic classes. Some of these indicators are simple and typically constitute the basis of a management information system for PHC clinics, like caseload for type of service. Others may not be PHP Final evaluation report 37 meaningful if pertaining to the clients of the clinic as opposed to a particular target group of the general population. The system proposed is entirely based on the outpatient clinical records for women and their children. The first draft of a set of patient records was described in a document titled "Unified Patient Medical Records" and distributed in June 2001 (English translation made available). This set includes separate but integrated records with the following scope: Family Information Sheet, Maternal Health File, Antenatal Care Record, Risk Assessment, Pregnant home visits, Post-Natal Care at Home (day 3-5), Post-Natal Care at Clinic (week 4-6), Breast Feeding Sheet (at home or the clinic), Family Planning Sheet, Gynecology Sheet, Breast Cancer Preventive Test Card, Child follow-up visits, and Community Activities Sheet. This document does not include a description and justification of the variables included in the various forms, or of the indicators and reports expected. Some variables may indeed be necessary for the health provider for clinical purpose, but not necessarily for the clinic manager or at any other level of the health system. It can be argued, however, that in a "paper less clinic" that information should be available to the health provider consulting the electronic files of its clients. Another document called "Manual for Using the Unified Reproductive Health Medical Records" (English translation not available) includes the list of indicators from the MIS proposal in the appendices, indicating that the system was built on that list. Again, the exact definition (numerator and denominator) of these indicators and the way they will be calculated should have been described somewhere before writing the software. One positive finding in that last document is that several protocols adopted at the national level by the large committee of local experts working under the sponsorship of the MOH and UNFPA have been reproduced in the appendix, presumably to indicate that these standards had been taken into account in the design of the MIS system and related forms and indicators. This is at variance with the criticism that HDIP did not fully take into account similar work on MIS done by the MOH. PHP Final evaluation report 38 The training of selected CHWs in the use of the new Unified Patient Medical Record began in June 2001. The actual field test began in August 2001. It continues in 7 clinics, in which, either a CHW enters the data using the new software installed on a computer given to the clinic, or a HDIP staff member visits the clinic and enters the data in a portable laptop computer. This HDIP staff member also provides assistance to all clinics where the new system is tested, and identifies the problems encountered. At the time of the evaluation, there were still important developments to be made in the structure of the database and in the data entry and analysis software. Critical operational aspects of the systems still need to be assessed: time needed by clinical workers to fill in the forms and enter the data; importance and actual use of the data generated at the various level of decision-making (patient, clinic, NGO, district, national). At the request of the Minister of Health, HDIP and the Population Council organized a formal review of the Health Sector MIS during a Consultative Meeting (October 10, 2001). The Minister called for the production of one Palestinian HMIS that can be used by UNRWA and NGOs. He recognized that patient records developed in hospitals and primary care facilities have to be different and yet a common referral form was required. The Deputy Minister reported that the MOH was establishing a national Health Information Center with offices in Nablus and Gaza City. A concluding action was that the MOH would take the lead in organizing and formulating a technical working group for HMIS. During the 28-month term of PHP, an operational MIS to provide current information for performance analysis and management decision-making among the 27 targeted clinics was not realized. An ambitious thorough revision of clinic records, extraction of data and report formulation was configured. The sustainability of this revised system and its place within a national MIS are questions that remain to be clarified. There is a need to understand the relationship of MIS and HIS. In general MIS is concerned with the relationship between resources used (inputs) and services rendered or performance units (outputs). When one is concerned with outcomes e.g. mortality, PHP Final evaluation report 39 morbidity, and fertility, data is obtained on a population basis and involves information systems such as vital event registration, sample population surveys and case control studies. Only in very special circumstances, i.e. for members of populations who are covered for total care by a single provider, is it possible to combine MIS and HIS. E. Operations Research Two related specific objectives were set forth for the task of operations research: (1) to test the effectiveness of four interventions with the purpose of increasing use of antenatal, postpartum and family planning services in 3 pilot areas and (2) to determine lessons learned from the implementation of the four interventions for later scale-up of services to a much wider universe. It was planned that the operations research projects would be implemented jointly by the Population Council and two local partners, i.e. Health, Development, Information and Policy Institute (HDIP) and the Center for Development of Primary Health Care (CDPHC). The four interventions would include: (1) a Basic Model consisting of a defined set of prenatal, postpartum and family planning services, including a clinic visit on the 40th day postpartum; (2) an extension of the Basic Model to include an outreach to low parity mothers (with one or two births) involving extensive counseling, and which would occur immediately prior to the Basic Model day 40 visit; (3) a study of male participation in prenatal and postpartum care; and (4) outreach linkage with public hospitals to promote postpartum care. The Population Council and its partners developed and implemented detailed research protocols to test these interventions. Critical reviews of these study protocols are provided in Appendix 11. The lack of a complete definition of the basic package of interventions and related standards and protocols was discussed in the section on Training (IV.C). This limitation made difficult the choice of indicators of the expected effectiveness of the package. In addition, the beginning of the implementation of the basic package is not clearly defined. PHP Final evaluation report 40 In principle, the PHP basic package included quality antenatal and postpartum care, usually provided by a physician in the clinic; counseling provided by the physician, nurse, or the CHW; and at least one home visit by the CHW. The PHP basic package was therefore fully implemented when the appropriate combination of trained health providers work in renovated clinics with appropriate equipment and supplies, that is, during the last quarter of year 2001. The posttest survey, conducted at that time, may therefore have been done too early to detect a real increase in the use of services as a result of the implementation of the basic package. One of the key indicators of effectiveness of the PHP, i.e. the percent of women returning for postpartum visits among those who made one or more prenatal visits, was not clearly cited in the baseline report of clinic activities. Instead, the indicator related to postpartum visits presented in the baseline report was the percent of women visiting the clinic for postpartum care among all women visiting the clinic for any reason, which is not a reliable measure of the percent of postpartum return visits because it depends on the number of visits for any other reasons. The delays in the initiation and implementation of the three other OR studies are identified in the Timeline section (III.A). These delays may not have in themselves constrained the potential successful implementation. For the low parity study, for instance, which tests the marginal utility of having an extra visit by a trained CHW, and, presumably, is not affected by the quality of services in the clinics, the intervention is considered to begin after the training of the CHW, that is, in January 2001. The design of the Low Parity study has one potential source of error in that the control and intervention groups of mothers, determined by random allocation of the clinics, receive a 2-3 day postpartum visit that is carried out by different Community Health Workers. The investigators have considered, however, that the two groups of clinics were similar, and that the CHWs had similar background and received the same training and supervision. This potential source of imbalance could have been avoided by randomization of mothers within clinics and CHWs, but the investigators did not consider PHP Final evaluation report 41 this option as operationally feasible. Another potential source of error is that the end of study evaluators may not be blind. None of the 3 operations research studies had reached conclusion by February 28, 2002, i.e., no final report of findings was available. However, the Population Council estimated at that time that final reports for all studies would be available by the end of July 2002. Given the delays in implementation and the absence of results at the end of the project, there has not been any lesson learned directly from the operations research studies yet. Somewhat related to the Operations Research component of the PHP, the Population Council funded a set of 6 Special Studies with the purpose of generating capacity for operations research and contributing to further understanding of problems of maternal health. These studies, conducted by graduate students and also practicing health professionals, covered the following subject areas: (1) male involvement in family planning (2 studies); (2) client satisfaction with family planning programs at UNRWA and MOH clinics (Gaza); (3) early marriage and premature birth; (4) compliance with iron supplementation among anemic women; (5) diabetes among pregnant women in Gaza. Based upon convenience samples, these studies were completed over a relatively short interval, under careful mentorship, and provide useful evidence on selective determinants of maternal and reproductive health. PHP Final evaluation report 42 V. Conclusions A. Accomplishments 1. Health infrastructure improvements initiated and completed include: (1) Significant physical upgrade of 25 clinics (2) Significant equipment upgrade of 27 clinics (3) Training of 58 clinic workers in 27 clinics These improvements can be expected to increase the quality of prenatal, postnatal and reproductive health care and to increase the attendance of the targeted population in these clinics. 2. Development of a cadre of 13 trainers and a training manual for health workers engaged in prenatal, postnatal and reproductive health services. These trainers can be expected to continue training and supervise the physicians, nurses, and community health workers in NGOs and MOH clinics. 3. Successful organization and conduct of a Coordinating Council In a complex delivery system including the Ministry of Health, NGO’s, international organizations and the private sector, the initiation and continuation of the Coordinating Council as an administrative element of the Pilot Health Project was an unparalleled success. It has contributed to keeping other organizations, including the MOH, informed of the activities of the PHP, avoided duplication of efforts, fostered ownership by the local partners, and created prototype for future projects. 4. Baseline and end of project surveys of prenatal, postnatal and reproductive health care in 27 clinics completed PHP Final evaluation report 43 The pre / post test design of the PHP has focused the implementing partners on collecting data specific to the selected clinics to make managerial decisions and document progress. The results of the two surveys can be expected to demonstrate changes in key process and outcome indicators. 5. Design, and field execution of 3 operations research projects Detailed protocols for testing three interventions to complement the basic package of services have been developed, and the training and data collection were completed at the end of the project. The results of these investigations are expected to contribute to future improvements in prenatal, postnatal and reproductive health care provided in NGOs and MOH clinics. 6. Design and completion of a series of studies and analyses on selective subjects of reproductive health Several rapid situation analyses and reviews conducted at the beginning of the PHP provided critical information for the implementation of the project. A series of six more formal but small studies were completed by investigators external to the project. The results of all these studies and analyses are of interest to health professionals and decision-makers in the health sector in the West Bank and Gaza. 7. Review of existing management information systems operative in 3 NGO primary care clinics This in-depth review identified critical gaps in the current health and management information systems in the West Bank and Gaza and generated important policy level discussions and decisions among various organizations involved in the health sector. 8. On-going pilot test of a computerized women and child patient record system as a basis for a national-level health and management information system. PHP Final evaluation report 44 The Unified Patient Record System and related software being tested 7 PHP clinics can be expected to provide answers to important issues that the MOH currently faces in the design and development of a health and management information system in the West Bank and Gaza. 9. Partnership with five Palestinian NGOs have strengthened their capacity to implement reproductive health programs. Most of the in-country funding for the PHP was expanded through sub agreements to HDIP, CDPHC, UPMRC, UHWC, and PFS involving collaborative relationships with CARE and the Population Council. Four of these organizations benefited from financial and managerial assessments and training by an international audit firm. 10. The lessons learned under the PHP have been included in the design and early implementation of the MARAM project. Among lessons learned from the PHP and incorporated in the MARAM project are the importance of a strong partnership with the MOH; the various opportunities for scaling-up activities started in the PHP; the need to provide for strong project implementation mechanisms in Gaza as well as in the West Bank; the critical role of a coordinating structure like the Coordinating Council of the PHP; and the need to specify accountability for performance according to standards at the clinic level. 11. The Mission has a health and population Strategic Objective and is an important player in the health sector in the West Bank and Gaza With no prior health programs since its creation in 1994, the Mission launched and brought to completion the PHP in less than three years (August 1999-March 2002). At the same time, the Mission adopted a health and population Strategic Objective, approved two other larger health sector projects (MARAM and EMAP), and PHP Final evaluation report 45 established a USDH Health Officer position. The Mission is at the forefront among the other partners of the MOH in assisting in the development of national health policies. B. Shortfalls 1. Target dates for tasks completion were not met, due in part to the following: - Late obligation of funds - Difficulty with negotiating sub agreements in a non-competitive situation - Complicated procurement procedures - Closures, travel restrictions and insecurity since the beginning of the Intifada 2. The PHP clinics do not constitute a representative sample of the primary care system in the West Bank and Gaza The 27 PHP clinics are managed by NGO’s, 20 clinics are located in the Jenin district, and 15 clinics are managed by PFS and located in the Jenin district. Therefore the results of the evaluation of the effectiveness of the basic package and the various pilot interventions cannot be easily generalized to other primary health care clinics in the West Bank and Gaza. 3. There are no explicit standards for antenatal, postpartum and reproductive health services adopted and implemented in the 27 PHP clinics Although elements constituting an acceptable standard of antenatal, postpartum and reproductive health services were delineated through the training program, there is no procedure to ensure compliance with these standards and quality of care. Neither the CAs nor any of the local partners devoted resources to the tasks of oversight of the clinic operations relating to maternal and infant health services. The absence of PHP Final evaluation report 46 explicit standards of care made it difficult to specify the basic package of interventions to be measured by the pre / post surveys. 4. The management information systems piloted in a few PHP clinics did not reach the point of effective use. The system being pilot tested with objectives broader than those envisioned in the project design may provide results that can be generalized outside the PHP. In meantime, however, at the end of the PHP the 3 PHP SD partners still do not produce and use service statistics for basic management purposes. 5. None of the operations research studies had reached conclusion by the end of the project. There was no report of findings available at the time of the final evaluation for any of the 3 operations research studies and for the posttest survey for the evaluation of the basic package of services. The Population Council regional office had draft reports in April and planned to disseminate the final findings and reports by July 2002. 6. Some of the specific objectives of the operations research studies may not be met because of design issues • In the Low Parity study, the random allocation of interventions by clinic may fail to isolate the effect of a special 30-40 day intensive home care visit. • In the pre / post surveys, the percent of women returning for postpartum visits among those who made one or more prenatal visits, a critical indicator of the effectiveness of the PHP basic package of services, is not clearly cited. 7. The MOH did not have any opportunity to participate in the PHP beyond that of attending the Coordinating Council meetings. PHP Final evaluation report 47 8. The Cooperative Agencies did not always provide adequate level of in-country technical assistance. Additional technical assistance was needed in: • Health systems research applied to primary health care and reproductive health. • Health worker training and performance improvement • Senior level public health project management (intermittently) 9. The implementation of the project has been more difficult in Gaza than in the West Bank These difficulties resulted at least partially from the limitations in communication and movement. 10. The Mission initially, did not fully assert authority and assume responsibility regarding the implementation of the PHP by the two centrally administered Cooperating Agencies The Mission did not clearly approve definitive work plans and budgets for the CAs. This responsibility was a higher priority for the PHP than consideration of operational details. C. Recommendations To the Mission 1. The Mission should develop the strategies and means to support the growing role of the MOH as a partner in primary health care, and particularly in the development of a PHP Final evaluation report 48 national health and management information system with an emphasis on maternal, child and reproductive health. 2. The Mission should recognize, when it exists, and take full advantage of the advanced expertise of centrally funded Cooperating Agencies, while retaining policy responsibility for in-country activities that it supports. 3. The Mission’s Community Services Project should take advantage of the experience gained by the PHP partners through their successful clinic renovation and equipment program. To the MARAM project 4. The MARAM project should assist the MOH and health services delivery NGOs in ensuring quality of care, including performance according to accepted standards. Accountability should be clearly assigned at the clinic level to the physician or a senior community health worker. A second level of accountability should be specified for the oversight of groups of clinics within administrative units of the MOH or within NGOs. 5. The MARAM project should review the operations research findings of the PHP with the objective of determining how they can be used in planning improvements in maternal and infant health services. 6. The MARAM project should include in its operations research component: (1) joint determination of the research questions with the MOH, representatives from the universities, NGOs, and communities; and (2) peer review of the design of the studies by research experts in the subject areas. PHP Final evaluation report 49 7. The MARAM project should assist the MOH in reviewing the recently developed protocols and guidelines for reproductive health to determine priorities for further investments in the health sector. 8. The MARAM Project should coordinate activities relating to management information systems with the current initiatives by the MOH, funded by the World Bank and other donors. 9. The MARAM project should assist the MOH in the definition of a minimum set of indicators for primary health care clinics, and assist the clinics that it supports in the collection, analysis, and reporting of these indicators with an emphasis on short-term analysis. 10. The MARAM project should assist the MOH in evaluating the findings of the test of the Unified Patient Medical Records and related software in 7 PHP clinics to be made available by HDIP, and in making the necessary adaptation before a larger-scale introduction and test in MOH and other clinics. D. Concluding statement The PHP has provided a useful platform for the development of a national level program of enhanced reproductive and child health services by the Mission. Issues for service delivery consistent with universally recognized standards have been identified. The capacity to conduct operations research pertinent to the problems of reproductive health has been enhanced as a result of the PHP experience. A modality for coordinated planning among major NGOs, international organizations and the MOH was created and serves as a prototype for future major health project. Of critical significance, as a product of the PHP, has been the recognition of the necessity for a final point of project accountability to be assigned to the Mission even in the context of delegation of operational responsibility to Cooperating Agencies. The investment in the PHP appears to be one that has resulted in a set of benefits that equates with the costs. PHP Final evaluation report 50 APPENDICES PHP Final evaluation report 51 Appendix 1 Schedule of the PHP final evaluation DATE ACTIVITY ATTENDEES Thursday, January 31 Phone interview Jack Thomas, former PSC Health Advisor Phone interview Nancy Ali, former Program Associate, Population Council Meeting at USAID/G/PHN Lisa Childs, CARE-MoRR Technical Advisor and USAID/WBG country coordinator Meeting at Population Council FRONTIERS John Townsend, FRONTIERS Director Joanne Gleason, FRONTIERS Administrator Friday, February 1 Phone interview Zynia Rionda, former RH Advisor, USAID/ANE Meeting with SO8 Team Leader & General Development Officer Thomas H. Staal, GDO Officer Gaby Ab Boud, CTO CSP Meeting with Bassam Kort Bassam Kort Meeting with SO7 Core Team Sherry Carlin, Health Officer Dr. Taroub Faramand, former FSN Advisor Dr. Suzy Srouji, FSN Advisor Monday, February 4 CARE International, Jerusalem Office Earl Wall, Director Meeting with Minister of Health, Gaza Dr. Riad Za’noun, Minister of Health Dr Maged Abu Ramadan, DG-IC; Dr Dina Abu Sha’ ban, WHD Visit to Beit Hanoun clinic. Dr Abdel Jabar Tibi, Medical Director, UHWC Tuesday, February 5 Meeting at MARAM, Gaza Dr Yehia Abed, MARAM, Deputy COP Meeting at USAID/WBG, Tel Aviv Dr. Taroub Faramand, former FSN Advisor Wednesday, February 6 Meeting at CDPHC, Ramallah Dr. Mohammad Shaheen, CDPHC Director Ms Enas Dhaher; Ms Haleana Al Sabbah Meeting at MARAM, Ramallah Dr. Umaiyah Khammash, Chief of Party; Dr. Salwa Najjab, RH Advisor; Dr. Rand Salman, Medical Advisor; Ms Anne Roberts, BCC TL; Hammonda Bellamine, Performance Improvement TL; Walid Nammour, Capacity Building TL Thursday, February 7 Meeting at HDIP, Ramallah Dr. Mustafa Barghouthi, Director Ms. Tamara Tamimi, Program Coordinator PHP Final evaluation report 52 DATE ACTIVITY ATTENDEES Meeting with USAID/WBG Mission Director Larry Garber, Mission Director Meeting with former PHP Health Advisor, CARE International Dr. Suzy Srouji, FSN Advisor; former CARE Program Manager Saturday, February 09 Meeting with Alpha International Dr Faycal Awartani, Chief Operations Office Meeting with Population Council Representative, Jerusalem Dale Huntington, Population Council Associate Director, ANE Saturday, February 09 Meeting with USAID/DGO Martha Myers, USAID/DGO Meeting Deputy Minister of Health, Ramallah Dr. Munther Al Sharif, Deputy Minister of Health; Ms. Wijdan Siam, Director General of Women’s Health Site visits to Ras El Jora clinic in Hebron Dr Mahmoud Breghar, Director of Clinic, UPMRC; Mr Teddy Rhalajab, Director of MIS, UPMRC Sunday, February 10 Meeting with former Director of MIS, UPMRC Dr Khaled Saifi, Al Mezan Hospital, General Manager Meeting w/ Director of UNRWA Dr. Hussam Siam Dr. May Keileh Monday, February 11 Meeting with former Population Council National Project Manager Dr. Mahmoud Shaheen, former Population Council Project Manager Meeting at Consulate General of Italy Dr. Marco Barone, Health Project Coordinator Meeting at UPMRC, Ramallah Dr. Jihad Masha’l, Director Tuesday, February 12 Phone interview Dr. Laila Nawar, Population Council Regional Program Associate Meeting at Patients Friends Society, Jenin Dr. Bilal Al Taher, Director And Staff Site visit to Rummaneh clinic - PFS, Jenin Dr En’am Jarar; Ms Nihayah, Health Worker; Ms Mohad Al Ahmad, President of Development Society of Rummaneh; Clinic clients Wednesday, February 13 Phone interview Dr. Laila Nawar, Population Council Regional Program Associate Meeting at CARE office in Jenin Othman Abbas, Field Coordinator Site visit to Al Sileh clinic - UPMRC, Jenin Dr Montaha Hamarshe; Dr Waddah Jabarim Ms Haifa Dubes, staff nurse; Two Health Workers ; Municipality President Site visit to Qabatia clinic - PFS, Jenin Ms Radjah Nazzal, Health Worker Ms Haifa Dubes, PFS HW Supervisor Meeting at CARE International Earl Wall, Country Director Thursday, February 14 Meeting at USAID/WBG, Tel Aviv Sherry Carlin, Health Officer Saturday, February 16 Phone interview Susan Ross, CARE MoRR Director Sunday, February 17 Site visit to UPMRC Ithna clinic, Hebron Dr Khadidja Jarar, UPMRC Monday, February 18 PHP Final evaluation report 53 DATE ACTIVITY ATTENDEES Site visit UNWRA Amary clinic Dr. May Keileh Meeting at UNFPA, Jerusalem Dr. Mohammad Abdel Ahad and Mrs. Laila Baker Tuesday, February 19 Phone interview with Dr. Rabah Muhanna, UHWC, Gaza Wednesday, Brief MARAM team - Ramallah CANCELED due to security February 20 Brief Coordinating Council CANCELED due to security Thursday, February 21 Brief MARAM team - Ramallah Umaiyah Khammash, Anne Roberts, Hammonda Bellamine, Walid Nammour, Ziad Abdeen, Earl Wall, Greg Greenough, Friday, February 22 Briefing for USAID Mission Director, Program Officer and SO7 Core Team Larry Garber, Rick Scott, Thomas H. Staal, Sherry Carlin, Dr. Suzy Srouji, Veerle Sterling and Peter Malnak PHP Final evaluation report 54 Appendix 2 List of documents reviewed By organization and chronological order. USAID West Bank and Gaza Family Health Assessment Report. A report by the USAID/W Assessment Team. Keys MacManus, Zynia L. Rionda, and Amita Barbey. January 1998. Concept Paper. Pilot Health Project. Agency for International Development, West Bank & Gaza. March, 1999. The burden of disease in the West Bank and Gaza. An assessment report. Zynia L. Rionda and Andrew Clements. Submitted by LTG and TvT to USAID/WBG, February 2000. HIV/AIDS and Sexually Transmitted Infections (STIs) in the West Bank and Gaza. Kai Spratt, ANE HIV/AID and STI technical advisor. USAID/WBG, February 2000. Nutrition Assessment In The West Bank And Gaza. Mellen Tanamly. Submitted by LTG and TVT to USAID/WBG, February 2000. Assessment of behavior change communication capacity in the West Bank and Gaza. Elizabeth Fox. Submitted by LTG and TVT to USAID/WBG, June 2000. Transition Plan. USAID/West Bank & Gaza. 2001. CARE Combined CARE / Population Council Timeline, August 2000. CARE Workplan, Revised November 2000. Financial and narrative quarterly reports. Midterm progress report, December 8, 2000. Coordinating Council minutes. Standard list of equipment for PHP clinics. PHP Final evaluation report 55 Population Council First Year Workplan for Research, Training and Analysis. Pilot Health Project. Frontiers in Reproductive Health Program. The Population Council Cairo. August 1999. Assessment of Standards of Health Care Services. Consultancy report, Pilot Health Project in West Bank and Gaza. Frontiers in Reproductive Health, Population Council, Cairo, Egypt, 1999. A manual for training in maternal and child health in the context of reproductive health. Center for Development in Primary Health Care, Al Quds University, in collaboration with the Frontiers in Reproductive Health, Population Council, 2000. Rapid assessment of providers expectations. A specific study done as a part of the USAID Pilot Health Project. Center for Development in Primary Health Care, Al-Quds University, July 2000. Palestinian Maternal and Child Health: a Qualitative National Study (Full Study Report). By Green Edward C., Khalid Abu-Khalid, and Mohammed Omari, 2000. In-House Sub-project: Technical Assistance to the WBG Pilot Health Project. Nancy Ali and Laila Nawar. Frontiers in Reproductive Health, Population Council, October 2000. Review of Formative Research in the Area of Reproductive Health. Ayesha Rifai, CDPHC, 1999. Special Studies Program of the Pilot Health Project in West Bank and Gaza. Editors: Nancy Ali. Frontiers in Reproductive Health, Population Council, December 2001. Pilot Health Project Follow-up Training Assessment in the West Bank & Gaza. Prepared by Haleama As Sabbah. Center for Development in Primary Health Care, Al Quds University. –DRAFT– Management Information System Assessment – Final Report. The Health, Development, Information and Policy Institute. Unified Reproductive Health Patient Medical Records for Pilot Health Project (PHP) Management Information System. Frontiers in Reproductive Health, Population Council, May 2001. Manual for Using the Unified Reproductive Health Patient Medical Records (in Arabic). Meeting Outcome Report. Consultative Meeting: Health Sector Management Information Systems (MIS). October 10, 2001. PHP Final evaluation report 56 Project Update 1: PHP A move to improve the health status of Palestinian women and their children Project Update 2: Upgrading the Capacity and Skills of Service Providers of the Pilot Health Project Project Update 3: Special Studies Program of the Pilot Health Project Project Update 4: Development of Management Information System for the Pilot Health Project in West Bank & Gaza Pilot Health Project Baseline Survey Report. West Bank and Gaza. Frontiers in Reproductive Health, Population Council, May 2001. Others relevant documents National Strategic Health Plan 1999- 2003. Palestinian National Authority, Ministry of Health. The Status of Health in Palestine. 2000 Annual Report. Palestine National Authority, Ministry of Health, HMIS, July 2001. Annual Report of the Department of Health 2000. United Nations Relief and Works Agency for Palestine Refugees in the Near East (UNRWA), 2000. Protocols and standards for Reproductive Health in the West Bank and Gaza. MOH, UNFPA, 2001. Counseling in reproductive health, MOH, UNFPA, 2001. PHP Final evaluation report 57 Appendix 3 Organizational and management structure of the Pilot Health Project Source: Concept Paper, 1999 PHP Final evaluation report 58 Appendix 4 Timeline of USAID/WBG general and health programs ACTIVITY 1997 1998 1999 2000 2001 2002 I II III IV I II III IV I II III IV I II III IV I II III IV I II III IV USAID/WBG general programs Strategic Plan 1996-2000 Transition Plan (Jan 01 - Jul 03) USAID/WBG health programs Staff Personal Services Contractor Foreign Services National US Direct Hire Projects Pilot Health Project RA CP MARAM Dgn RFP Emergency Medical Assist. Project RFP PHP implementation Capacity building Renovation and equipment of clinics Assessment Renovation Training Assessment Implementation MIS Assessment Implementations Research Baseline / post test survey Baseline Post-test Low parity Proposal Implementation Male involvement Proposal Implementation Hospital based Proposal Implementation Notes: RA: Rapid Assessment mission; CP: Concept Paper approved; Dgn: Design of the MARAM project; RFP: Request for Proposal PHP Final evaluation report 59 Appendix 5 Timeline of the PHP implementation ACTIVITY 1999 2000 2001 2002 A S O N D J F M A M J J A S O N D J F M A M J J A S O N D J F M Service Delivery CAPACITY BUILDING Coordin. Council meetings (serial #) 4 5 6 7 8 9 10 12 13 14 15 Quarterly and Midterm reports Q Q Q Q Q Q QM Q Q Q Various events Inau gural Jor dan FAST Jeri cho MIS MIS Sub grants/workplans w/SD partners Financial and mngt assessment UP MRC UH WC PFS CD PHC RENOVATION Sites identified Clinic physical assessments Bidding and contracting Renovations (cumul. # of clinics) 3 15 17 25 EQUIPMENT Needs assessment Tender of equipment Op en Distributes Equipment Non- Med. US auto Med Rapid Emergency Response PHP Final evaluation report 60 ACTIVITY 1999 2000 2001 2002 A S O N D J F M A M J J A S O N D J F M A M J J A S O N D J F M Research, Analysis and Training TRAINING AND B.C.C. Assessment of clinical standards Curriculum development Preparation & delivery of training ToT CHW P/N Monit/Superv/Refresher training Refresh Finalization of training manual Devlpt/prod/distrib of BCC materials Distrib. M.I.S. Assessment and Implementation prop CC $ Trai Pilot ? RESEARCH Review of formative research Baseline / post-test survey Baseline Proposal Data coll. Dissem. Post-test a. Low Parity Field b. Husbands Involvement c. Facility based Special Studies Program RFP sel Implementation rep Updates 1 2/3 4 PHP Final evaluation report 61 Appendix 6 Population Council West Bank and Gaza Obligations vs. Expenditures Obligations 2,685,000 ANE Bureau 63,000 West Bank/Gaza MAARD 1,755,000 West Bank/Gaza MAARD 200,000 West Bank/Gaza Field Support 200,000 FRONTIERS population core funds (projected) 467,000 Expenditures Personnel 589,670 includes Dale Huntington's time at 20% FY99, 30% FY00, 25% FY01 includes Laila Nawar's time at 50% FY99, FY00, FY01 includes Nancy Ali's time at 100% FY00, FY01 includes all local staff at Population Council Ramallah Subawards total budget 1 1,046,132 Training-CDPHC 262,000 MIS-HDIP 235,837 Low parity-CDPHC 70,394 Institutional Support/Program-HDIP 274,699 Post-Test Survey-Alpha 58,832 Includes In-house projects2 TA to Pilot Health Project - 13024 112,660 Special Studies Program - 13047 31,710 Other direct costs 168,265 Office in Ramallah 106,518 Travel 61,747 Indirect costs at 33.49% 604,182 Total Expenditures 2,408,249 Balance 276,751 Note: Obligations include FRONTIERS contribution from population core funds, which are not strictly speaking “obligations.” Expenditures include sub grants commitments through 2/28/02; all other expenditures through 9/30/01. PHP Final evaluation report 62 Appendix 7 CARE West Bank/Gaza Obligations vs. Expenditures Total Obligations 2583852 West Bank and Gaza MAARD 2,153,852 West Bank and Gaza Field Support (Emergency Procurement) 200,000 Expenditures Personnel 575,579 Country Director at 10% FY99-FY01 Project Manager at 50% FY99, 100% FY01, 50% FY02 All local staff at Population Council Ramallah Sub-awards total budget 656,978 Includes following expenditures as of Dec 31, 2001: UHWC (including clinic remodeling: $41,640) 219,084 UPMRC (including clinic remodeling: $56,240) 216,228 PFS (including clinic remodeling: $107,239) 215,429 Procurement 765,370 Includes the following expenditures as of Dec 31, 2001: Equipment for clinics 405,315 Emergency procurement 202,472 Other direct costs Office in Ramallah 208,927 Indirect costs at 7.722% 170,413 Total Expenditures 2,377,267 Balance 206,585 Note: Totals include expenditures as of end of January, estimates for February, and commitments made under contracts until March, 2002. PHP Final evaluation report 63 Appendix 8 Vision statement for the Coordinating Council VISION STATEMENT AND STATEMENT OF PRINCIPLES PILOT HEALTH PROJECT COORDINATING COUNCIL VISION The Coordinating, Council Of the USAID Pilot Health Project is established to coordinate the joint efforts of the participant partners in improving the health of Palestinian mothers and children through improved antenatal and postpartum care, increased use of modern contraception for birth spacing and enhanced knowledge of Palestinian parents on when and how to have healthier children. THE COUNCIL AND ITS COMPOSITION The Council is the entity that maintains the integrity of the vision, guides the management of the project and harmonizes the functions of the two major components – research and service delivery. Each organization shall maintain its authority over its own operations but will be held equally responsible for its part in implementing the project. However, the Council’s recommendations shall be considered the guiding principles and guidelines which all shall endeavor to carry out. The Coordinating Council will comprise representatives of all the partner organizations: CARE International Center for Development of Primary Health Care (CDPHC) Health, Development, Information and Policy Institute (HDIP) Patients’ Friends Society (PFS) The Population Council (PC) The Women’s Health Division, Ministry of Health (WHD) Union of Palestinian Medical Relief Committees (UHWC) U. S. Agency for International Development (USAID) One of the Council’s primary roles will be to ensure that the Project does not duplicate the efforts of the Ministry of Health, other NGOs or donors, as well as protocols, policies, training and logistical materials and curricula that already have been researched, designed, and tested. The Council will decide on matters of public relations, representation and promotion of the pilot project. PHP Final evaluation report 64 MEETINGS The Coordinating Council will meet regularly to review both joint activities and to be apprised of activities and progress of other members, as well as to make technical reviews of protocols, curricula, research results, recommendations for alternative courses of action and to guide the joint efforts of all members. The Chair of the Coordinating Council will be shared for equal amounts of time (14 weeks each) by the member organizations. USAID will serve as the first chair until the Council decides how to determine the order for the other member organizations to be the chair. The Chair will call meetings of the Council, at least once per month for the first six months of the Project, and will circulate and synthesize each meeting's agenda prior to the meeting. Thereafter, meetings will be convened as needed. VOTING The Council will strive for consensus, but will vote on critical issues if necessary. A simple majority will carry motions. SECRETARIAT CARE was accepted as a volunteer to be the Secretariat for the Coordinating Council. It will keep records of correspondence, act as a central point for communications, and assist whichever member is the current chair. PHP Final evaluation report 65 Appendix 9 List of the 27 clinics included in the PHP Village / Clinic District NGO 1 Qabatia Jenin PFS 2 Dier Abu –D’ief Jenin PFS 3 Rumaneh Jenin PFS 4 Kufradan Jenin PFS 5 Dier Ghazaleh (REPLACED Al-Almanieh) Jenin PFS 6 Misielieh Jenin PFS 7 Araneh Jenin PFS 8 Keferet (REPLACED Zabdeh) Jenin PFS 9 Nazlet Al –Shiekh Hussain Jenin PFS 10 Al-Tarem Jenin PFS 11 Al-Fundoukomieh Jenin PFS 12 Mashro’ Beit –Qad Jenin PFS 13 Te’enak Jenin PFS 14 Sier Jenin PFS 15 Al-Shuhada Jenin PFS 16 Sielet Alharthieh Jenin PFS 17 Miethaloon Jenin UPMRC 18 Al-Zababdeh Jenin UPMRC 19 Toubas Jenin UPMRC 20 Al-Nassarieh Jenin UPMRC 21 Ithna Hebron UPMRC 22 Ras El Joura Hebron UPMRC 23 Biet- Hanoun/Alquds Clinic Gaza UHWC 24 Jabalia Camp/Al-Assrieh Clinic Gaza UHWC 25 Al-Nusseirat Camp/Alkhirieh clinic Gaza UHWC 26 Rafah/Rafah Medical Center (NOT RENOVATED) Gaza UHWC 27 Al-Awda Hospital Gaza UHWC Source: CARE International PHP Final evaluation report 66 Appendix 10 List of equipment Description PFS UPMRC UHWC Total MEDICAL EQUIPMENT AND FURNITURE Scales/Health O-Meter 15 7 5 27 Doppler/Medasonics 15 7 5 27 a. Airway Black (2 boxes) 30 14 10 54 b. Airway Red (2 Boxes) 30 14 10 54 Manual Breast Pump/Omron 15 7 5 27 Binkman 1810 Trolleys w/ Solution Bowls 15 7 5 27 Mobile Lights, Medine w/Casters/Halogen 15 7 5 27 Wheel Chair/Medine, Model MDS806100DOE 15 7 5 27 Trolleys Blinkman 15 7 5 27 Otoscope set 15 7 5 27 Torech Halogen Light 15 7 5 27 IV Stand 15 7 5 27 Vacu-Aide Suction Unit 15 7 5 27 Gynecology beds & examination couch 5 7 5 17 X-Ray Viewer 15 7 5 27 Examination Table 5 7 5 17 Oxygen Tank and Mask Watch 15 7 5 27 IUD SET: Graves Vaginal Speculum (Floor Graded), 300 140 100 135* Stainless steel, Medium, 4"x1 1/4" Blade, 4ea Graves Vaginal Speculum (Floor Graded ), 300 140 100 135* Stainless steel, Large, 4"x1 1/4" Blade, 4ea Uterine Sound Graduated Malleable 300 140 100 540 Iris Scissors (Floor Grade), 4 1/2", Curved, 12/box 300 140 100 45 Volsellum Forceps 300 140 100 540 Kidney Basin 300 140 100 540 Iodine Bowl 300 140 100 540 IUD REMOVAL SET: Forceps Jar, Stainless Steel Utensils, 4 1/2" 6/cs 300 140 100 135 Kelly Haemostatic Forceps, Straight, 5 1/2 (14cm) 300 140 100 135 Graves Vaginal Speculum (Floor Graded), 300 140 100 135 stainless steel, Large, 4 1/2"x1 1/4" Blade, 4/box Forester Sponge Forceps, Serrated, Straight, 7", 300 140 100 135 12/bx Kelly Haemostatic Forceps, Straight, 5 1/2 (14cm) 300 140 100 135 Amino Hook, Plastic, Sterile, 100pcs/cs 300 140 100 135 Description PAP SMEAR SET: Pap Smear Complete Tray, 20cs 15 7 5 27 PHP Final evaluation report 67 Description PFS UPMRC UHWC Total Universal Bandage Scissors, 18cm 15 7 5 27 HOME DELIVERY SET: Kelly Haemostatic Forceps, Straight, 5 1/2 (14cm) 75 35 25 135 Operating Iris Scissors, Straight, 11.5 cm, 4.5" 15 7 5 27 Umbilical Cord Clamp, Plastic, Sterile, 100pc/cs 15 7 5 27 Gauze, Economy, 4x4, 8ply, sterile, 2's, 600pc/cs 15 7 5 27 ABD/Combine Pads, 5"x9", Non-Sterile, Latex Free 15 7 5 27 576pc/cs 15 7 5 Sterile Powder-Free Latex Examination Gloves, 15 7 5 27 Med. Pairs 100pr 15 7 5 Rubbermaid Step on containers, 8-Gallon Capacity 15 7 5 27 Suture Chromic Cutting Needle, Straight 30" 12/pk 15 7 5 27 (2-0),12/pk Stainless Steel Tray 15 7 5 27 HOME VISIT KIT: Sphegnommanometer & Stethoscope Set 15 7 5 27 (Pro Combo III Pocket Set) Thermometer (mother and baby) 30 14 10 54 Measuring Tape (Disposable Pediatric),1000pcs/cs 15 7 5 27 Timer 15 7 5 27 Stethoscope 15 7 5 27 SURGICAL AND DRESSING SET Needle Holder 30 14 10 54 Hemostatic Forceps 30 14 10 54 Tissue Forceps 30 14 10 54 Dissecting Forceps 30 14 10 54 Scapel Handle 30 14 10 54 Surgical Scissors (Sharp) 30 14 10 54 Surgical Scissors (Blount) 30 14 10 54 DISPOSABLES Gauze Bandages 7 cm 4000 4000 4000 12000 Gauze-non sterile 5*5= 8pl 10000 10000 10000 30000 Gauze-non sterile 7.5*7.5 = 8PL 50000 50000 50000 150000 Gauze- non Sterile 10*10= 8pl 20000 20000 20000 60000 Gauze- Sterile5*5 3000 4000 3000 10000 Gauze- Sterile 7.5*7.5 10000 10000 10000 30000 Gauze- Sterile 10*10 10000 10000 1000 30000 Latex Gloves- Sterile 7.5 1650 1700 1650 5000 Latex Gloves- Sterile 8 1300 1400 1300 4000 Latex Gloves-Non Sterile, medium 10000 10000 10000 30000 Cotton 45 50 45 140KG Sanitary Napkins 2000 2000 2000 6000 Paper Rolls for examination Tables (53cm*100m) 150 150 150 450 roll Sheets (cloth) 200 200 200 600 PHP Final evaluation report 68 Description PFS UPMRC UHWC Total Autoclave Bags 36*46cm 4000 4000 4000 12000 Autoclave Tapes Uni Size 115 120 115 350 Ultrasound Jelly 440 450 440 1330 L. Disinfectant solutions: chlorine 420 430 420 1270 L. Sharp Edge Disposable Bin 115 120 115 350 Scalp Vein (IV Kits) G21 400 400 400 1200 Scalp Vein (IV Kits) G23 400 400 400 1200 Dressing Pads 1150 1200 1150 3500 Suture Chromic 3/0 30 40 30 100dozen Suture Chromic 4/0 30 40 30 100dozen Suture Chromic 5/0 30 40 30 100dozen Suture Silk 3/0 60 80 60 200dozen Suture Silk 4/0 60 80 60 200dozen Suture Silk 5/0 60 80 60 200dozen Folley Catheters 16 250 300 250 800 Folley Catheters 18 350 400 350 1100 Source: CARE International. PHP Final evaluation report 69 Appendix 11 Reviews of Operations Research protocols 1. Improving Postpartum Care Among Low Parity Mothers in Palestine Agency: CDPHC Objective: Examine effectiveness of an additional home visit to low parity (1 or 2) mothers Intervention: Home visit to low parity mothers during perinatal period and 30-40 days postpartum Endpoint: Increase in Day 40 postpartum clinic visit frequency Increase in contraceptive use at 6 months postpartum Design: Included are women who present for antenatal care to PHP clinics, low parity (1 or 2), and give consent for study participation. Stratification by NGO; within stratum random allocation of clinics to control or intervention class; sample size of 22-23 women per clinic; anticipated total per arm:300. Of women in study, 77% are located in the Jenin area. Comment: No justification for sample size Design has an inherent source of error: Control and Intervention have a 2- 3 day postpartum visit which is carried out by different Community Health Workers. This source of imbalance would have been avoided by randomization within clinics. Not clear if end of study evaluators are blind. Strengths: Evidence of constructive collaboration between Pop Council and CDPHC. Subject area relevant to attainment of birth spacing objective Evidence of informed consent and of quality assurance in respect to intervention. Protocol is attentive to detail required in experimental design. PHP Final evaluation report 70 2. Involving Men in their Wives’ Postpartum Care in Palestine Agency: HDIP Objective: 1. To describe cultural and social beliefs among women and men regarding male role in reproductive health. 2. To identify practical interventions to involve husbands and influential males in reproductive health Research Questions: What proportion of women, receiving antenatal and/or postpartum services are interested in special counseling of their husbands in respect to birth spacing? What proportion of husbands are willing to receive counseling at a clinic or at home regarding health care needs of their wives and infants? What characteristics of outreach workers are acceptable to women and men? Design: The sample locations are in Hebron and Jenin. Data is acquired through focus group discussion and in-depth interview. Focus group discussion involves 6-8 participants of similar socio￾demographic characteristics. Four focus group discussions involve women; two discussion groups involve men. Interviews involve available staff (physicians and nurses) at 2 clinics in Hebron and 8 clinics in Jenin. A total of 30 interviews involve men (3 at each of 10 clinics). Data collectors to receive six days of training. All instruments to be pretested. Analysis planned to use standard procedures for qualitative research. Technical guidance to be provided by Population Council staff. Comment: The subject is relevant to the reproductive health issue of birth spacing. No results are available from this study to guide clinic or home visit education by clinic staff during the Pilot Health Project operational period. This protocol is concerned exclusively with the West Bank, and is relatively limited in its geographic coverage. The two special studies in Gaza on the same subject provide an opportunity for broader generalization of study findings. There is a need to assign to a specified technical resource the task of determining what direction behavioral modification might take given the West Bank-Gaza combined findings on male attitude relating to birth spacing. It may be expedient to assign this task to IVCHS staff. PHP Final evaluation report 71 3. Outreach Linkages With Public Hospitals to Promote Postpartum Care Agency: HDIP Significance: High proportion of women deliver in hospitals (85%), particularly in public hospitals (43%), but few women visit PHP clinics to receive postpartum care and/or family planning services. It is reasonable to suppose that closer coordination between hospital and clinic providers of perinatal care could result in a higher use of postpartum care/family planning services. Research Questions: Is it feasible for staff at hospitals providing delivery services to provide counseling on postpartum care? Will women receiving postpartum counseling in hospital demonstrate an increased prevalence of postpartum visiting to NGO clinics? Is it feasible to establish a viable referral system between hospital and NGO clinic to increase postpartum care and Day 40 counseling? Design: Study sites include 2 major hospitals – Al Shifaa Hospital in Gaza and a hospital in West Bank located in Jenin. The NGO partner in Gaza is UHWC and in West Bank, both UPMRC and PFS. In depth interviews with service providers and key management staff at each of the 2 sites. Purpose of interviews is to (1) determine extent of services provided to women before, during, and after delivery, and to (2) assess feasibility of linkage with clinics for postpartum service delivery. Home interviews will be made to women in the consenting sample. These interviews will have the purpose of determining (1) extent of information re: postpartum care acquired (2) extent of postpartum practices accepted. The sample size for the home interviews will be 40 (20 per study site) and will involve women who agree to receive intervention activities including a home interview. Comment: The subject area, linkage of hospital of delivery with a follow-up clinic facility for postpartum care, is an important issue for system development in maternal care. The sample scope, i.e., 2 hospitals and 20 women per hospital, is limited for purposes of generalization and for standardization of process. PHP Final evaluation report 72 The cost of the study seems somewhat excessive for the anticipated results. 4. Baseline and End of Project Surveys A. Baseline Survey Agency: HDIP Significance: The global objective of improving the health status of Palestinian women and children was initially addressed by the Pilot Health Project. Among the tasks pursued in the project was a specification and the provision of a basic package of quality antenatal, postpartum, and reproductive health services. Purpose: To determine the state of practice and utilization patterns at the start of the project, a baseline survey was undertaken by HDIP during the period May30-Aug. 2000, and an end of project survey was fielded by Alpha Inc. during the period October, 2001 – January, 2002. Design: The baseline survey included (1) review of routinely collected service statistics, standardized interviews with health care providers, and standardized exit interviews with antenatal, postpartum and reproductive health charts. The sampling frame included 27 clinics, 9 physicians, 3 nurses, 42 community health workers, and 792 clinic visitors (patients). Findings: Among clients interviewed (n=792), 27, or 3.3%, were visiting for post partum care (not included among the 27 were 109 clients visiting for family planning). Among those women who made a post partum visit, over 70% visited only once. A principal purpose of the postpartum visit was to initiate a contraceptive method. Analysis of service statistics for the 3 participating NGO’s covering the 6 month period prior to the Baseline Survey (11/99-4/00) proved to be difficult due to variance in the classification used to denote reason for maternal health related visits. However it is noteworthy that significant volume of service is reported in the categories of family planning clinic visits and in the category of home visits for antenatal and post partum care. PHP Final evaluation report 73 Among community health workers, 60% reported having diplomas and/or training; the remainder had no type of educational degree. Among physicians, most reported receiving training in women’s health, and expressed a desire for additional training. Comment: No data was secured from examination of individual clinic records. The one time interview of a current cohort of visiting women does not validly determine a key index, i.e percent of women, registered for antenatal care, who are seen one or more times during the postpartum period. The small sample of postpartum women interviewed (n=27) does not provide a database sufficient to provide reliable estimates relating to reasons for visit, prevalence of home visits, frequency of postpartum visits, etc. Information secured from the entire cohort of 792 women clinic visitors relating to knowledge and use (current, past) of family planning methods and relating to reproductive health knowledge is useful for subsequent planning of a quality basic package of reproductive health services. B. End of Project Survey Agency: Alpha International Purpose: A baseline survey completed by HDIP provided data on several indicators selected to measure the effect of the PHP. An end of project survey was intended to measure the impact of the project on key indicators. Method: The procedures employed replicated those used in the Baseline Survey, namely: (1) An average of 43 clients per clinic would be targeted for exit interview; (2) All consenting health providers present during the period when the data collection teams are in the facility; (3) monthly clinic caseload by types of services would be abstracted retrospectively for 6 months prior to the end of project survey. Findings: At the time of the field phase of the current assessment (February 1- February 15, 2002), no findings were available. Population Council reports that analysis of baseline and end of project data is in progress as of March 1. PHP Final evaluation report 74 Comment: The effort to determine the effectiveness of the basic package as delivered by the three participating NGOs is appropriate for a pilot health project. The design of the baseline and end of project surveys provided for examinations of a wide spectrum of questions relating to provider capacity and performance. An important source of information, namely the clinic record, to examine frequency and related characteristics of antenatal clinics, home visiting, and postpartum follow-up was not used. Perhaps, the clinic records did not meet a reasonable standard of completeness. The substitution of exit interviews does provide useful data on knowledge, attitude, and practice on the part of the interviewee mother, but depends upon recall relative to actual service. Such recall can be a source of disturbing error. This basic package evaluation is based upon a sample that cannot be considered generalizable to the national delivery system for antenatal, postpartum reproductive health services. PHP Final evaluation report 75 Appendix 12 Photographs of renovated clinics 1. External view, Rummaneh clinic, Jenin 2. Former warehouse turned into a three-room women’s health clinic, Ithna, Hebron 3. Waiting room, Rummaneh clinic, Jeninc 4. Community Heath Worker and Supervisor in waiting room, Qabatia clinic, Jenin 5. Reception area, Ras El Jora clinic, Hebron 6. Washing hands to prevent infection, Qabatia clinic, Jenin 7. Appropriate equipment and privacy, Sileh clinic, Jenin 8. Appropriate equipment and privacy, Rummaneh clinic, Jenin 9. Individual counseling, Rummaneh clinic, Jenin 10. Preparing for small group education, Rummaneh clinic, Jenin 11. Audiovisual equipment for health education, Qabatia clinic, Jenin 12. Health worker extracting from Unified Patient Records, Qabatia clinic, Jenin External view, Rummaneh clinic, Jenin Former warehouse turned into a three-room women’s health clinic, Ithna, Hebron Waiting room, Rummaneh clinic, Jenin Community Heath Worker and Supervisor in waiting room, Qabatia clinic, Jenin Reception area, Ras El Jora clinic, Hebron Washing hands to prevent infection, Qabatia clinic, Jenin Appropriate equipment and privacy, Sileh clinic, Jenin Appropriate equipment and privacy, Rummaneh clinic, Jenin and privacy, Rummaneh clinic, Jenin 1. Individual counseling, Rummaneh clinic, Jenin 2. Preparing for small group education, Rummaneh clinic, Jenin Audiovisual equipment for health education, Qabatia clinic, Jenin Health worker extracting from Unified Patient Records, Qabatia clinic, Jenin