Nepal Family Health Program (NFHP) II Evaluation photo by Deborah Armbruster Joy Riggs-Perla, MPH Murari Adhikari, MS Ramesh Kant Adhikari, MBBS, MD Deborah Armbruster, CNM, MPH, FACNM Deepak Paudel, MPA, MPH, PhD (cand) Devi Prasad Prasai, PhD Sudip Pokhrel, MS July 29, 2011 Nepal Family Health Program (NFHP) II Evaluation DISCLAIMER The author’s views expressed in this report do not necessarily reflect the views of the United States Agency for International Development or the United States Government. ii Table of Contents Page List of Acronyms iv Executive Summary vii I. Background 1 A. Evaluation Objectives and Rationale 1 B. Context of the USAID Health Program in Nepal 1 II. Scope of Work and Methodology 2 A. Summary of SOW 2 B. Evaluation methodology 2 III. Summary NFHP II Description 3 A. Brief Summary of Project Goals, Objectives, Components 3 B. NFHP II’s Performance Monitoring Plan 4 C. Project Alignment with Global Health Initiative (GHI) 5 and Best Action Plan IV. Findings: Progress and Issues 6 A. Component 1: Systems, Policy and Leadership 6 A. 1. Logistics Systems 6 A. 2. Information Management 9 A. 3. Policy Support 10 A. 4. National and District Leadership/ 13 Management Capacity B. Component 2: Health Service Delivery 14 B. 1. Family Planning Services 15 B. 2. Maternal Health Services 19 B. 3. Child Health Services 23 B. 4. Newborn Health Services 25 C. Component 3: Community and Household Level Interventions 26 C. 1. Strengthening Local Governance in Health 26 C. 2. Literacy and Life Skills 27 C. 3. Behavior Change Communication 28 C. 4. GESI 29 D. NFHP II Impact 29 D. 1. Performance Monitoring 29 iii D. 2. Output, Outcome and Program Management Indicators 30 V. Planning, Implementation, Monitoring and Reporting 31 A. Planning, Monitoring and Reporting Processes 31 B. Effectiveness of Capacity Development Work At 32 All Levels VI. Management Systems 33 A. Systems for Program Management with Government of Nepal 33 B. Management of Internal Partners 34 C. Compliance to USAID Policies and Requirements 34 VII. Conclusions and Recommendations 35 A. Technical and Programmatic Recommendations 35 B. Program Management Recommendations 37 VIII. Considerations/Recommendations for USAID Future Programs 38 Annexes 1. List of Documents Reviewed 2. List of People Interviewed and Sites Visited 3. Full Scope of Work 4. PMP Targets and Current Status 5. Table of selected findings from facility observation checklist iv Acronyms AHW Auxiliary Health Worker AMTSL Active Management of Third Stage of Labor ANC Antenatal Care ANM Auxiliary Nurse Midwife ARI Acute Respiratory Infection BBC WST BBC World Service Trust BCC Behavior Change Communication BEmOC Basic Emergency Obstetric Care BPP Birth Preparedness Package CB-IMCI Community-based Integrated Management of Childhood Illness CBLP Central Bidding Local Purchasing CB-NCP Community-based Neonatal Care Package CEDPA Center for Development and Population Activities CEmOC Comprehensive Emergency Obstetric Care CFWC Chhetrapati Family Welfare Center CHD Child Health Division CHW Community Health Worker CHX Chlorhexidine CMA Community Medical Assistant CPD Core Program District CSHGP Child Survival and Health Grants Program CTS Clinical Training Skills DDC District Development Committee DFID Department for International Development (U.K.) DoHS Department of Health Services DHO District Health Office DPHO District Public Health Office EDP External Development Partner EPI Expanded Program on Immunization FCHV Female Community Health Volunteer FHD Family Health Division FP Family Planning FPAN Family Planning Association of Nepal FY Fiscal Year GATE Girls' Access to Education GESI Gender Equity and Social Inclusion GIS Geographic Information System GIZ German International Cooperation GoN Government of Nepal HA Health Assistant HEAL Health Education and Adult Literacy HF Health Facility HFMSP Health Facility Management Strengthening Program HFOMC Health Facility Operational Management Committee HIDN Health Infectious Disease and Nutrition HMIS Health Management Information System HP Health Post IEC Information, Education, Communication I/NGO International/Non-Governmental Organization IP Infection Prevention IUCD Intra-Uterine Contraceptive Device v IYCF Infant and Young Child Feeding KfW German Development Bank LC Learning Circle LHGSP Local Health Governance Strengthening Program LMD Logistics Management Division LMIS Logistics Management Information System M&E Monitoring and Evaluation MCH Maternal and Child Health MCHIP Maternal and Child Health Integrated Program MCHW Maternal and Child Health Workers MDG Millennium Development Goal MgSO4 Magnesium Sulfate MINI Morang Innovative Newborn Intervention MNCH Maternal, Newborn and Child Health MNH Maternal and Newborn Health MoHP Ministry of Health and Population MoLD Ministry of Local Development MSC Maatri Surakchya Chakki (misoprostol) NDHS Nepal Demographic Health Survey NFHP Nepal Family Health Program II NGO Non-governmental Organization NHEICC National Health Education, Information & Communication Centre NHSP Nepal Health Sector Programme NHSSP Nepal Health Sector Support Programme NHTC National Health Training Center NRCS Nepal Red Cross Society NSV No-Scalpel Vasectomy NTAG Nepal Technical Assistance Group ORC Outreach Clinic ORS Oral Rehydration Solution PHA Public Health Analytics PHCC Primary Health Care Center PI Performance Improvement PNC Postnatal Care PPH Postpartum Hemorrhage PVO Private Voluntary Organization QA Quality Assurance QAWG Quality Assurance Working Group QI Quality Improvement RH Reproductive Health RHCC Reproductive Health Coordination Committee RMS Regional Medical Store SBA Skilled Birth Attendant SDC Swiss Development Cooperation SHP Sub-health Post SWAp Sector Wide Approach programme TA Technical Assistance TOT Training of Trainers TSV Technical Support Visit TT Tetanus Toxoid TWG Technical Working Group UMN United Mission to Nepal UNDP United Nations Development Programme UNFPA United Nations Population Fund UNICEF United Nations Children’s Fund USAID United States Agency for International Development vi USG United States Government VAC Vitamin A Capsule VDC Village Development Committee VHW Village Health Worker VSC Voluntary Surgical Contraception WHO World Health Organization vii Executive Summary In June of 2011, USAID/Nepal commissioned an external evaluation of the Nepal Family Health Program II (NFHP II) to assess progress, identify challenges and make recommendations for any changes needed during the final year of program implementation. The rationale for the evaluation was also to learn lessons from NFHP II in order to inform the development of a new program to begin as NFHP II is phasing out in 2012. The evaluation team, consisting of seven experts from Nepal and the U.S., reviewed documents, interviewed a wide range of stakeholders at the national, regional, district and facility levels, traveled to view programs in six of the 22 NFHP II core program districts and reviewed data from national surveys and the project’s monitoring and evaluation reports. Because NFHP II builds on the work of a previous program (NFHP I), which spanned the period from 2002-2007, the evaluation team did its best to distinguish and assess the progress since the current NFHP II began in late 2007. NFHP II was designed as a health systems strengthening program but with the greatest emphasis on strengthening the health service delivery aspects of the Ministry of Health and Population (MoHP) system. The program is based on the goals and objectives of the MoHP’s Nepal Health Sector Programme (NHSP II) strategic plan for the period 2010-2015 and works entirely by supporting the staff and various government programs at the national, district, facility and community levels. The services targeted are family planning and maternal, neonatal, and child health, with most of the emphasis on community level programs to achieve the greatest impact. NFHP II was also built on the success of NFHP I by continuing to generate evidence at the field level to support the scale up of new approaches to reducing maternal, neonatal and child mortality and expanding the use of family planning and reproductive health services, and by placing a strong emphasis on ensuring the availability of essential drugs and commodities at the community and health facility levels. This evaluation looked at the progress made during the past four years, examined the challenges and issues facing the program and recommended actions that will help maximize the program’s contribution, keeping in mind that it has only another year of implementation. It also provides some general guidance to USAID on approaches to consider for a follow on program in similar areas of work. It is clear that NFHP II has been a highly successful program with substantial accomplishment in support of the Government of Nepal and USAID’s health sector objectives. NFHP II benefited from the lessons learned and relationships built during the preceding NFHP I program. While the two programs were not exactly the same in content or geographic coverage areas, there was enough overlap in areas of health system capacity development to allow sufficient time to institutionalize important program contributions. This is particularly striking in service delivery at the community level where Female Community Health Volunteers (FCHVs) have been mobilized to provide key life-saving interventions such as preventing postpartum hemorrhage during home deliveries, treating childhood pneumonia and diarrhea, making non-clinical family planning methods available at the household level and teaching simple techniques for newborn care. The quality and availability of maternal, newborn and child health and family planning services in primary care facilities has also been strengthened with support from NFHP II. The Government of Nepal (GoN), with support from all of its partners, continues to make strong progress toward achieving its Millennium viii Development Goals (MDGs) in maternal and child health and family planning. NFHP II’s contribution to developing other areas of health systems is mixed; strong on logistics management but slower and more challenging in the areas of systems to support quality improvement and supervision. Exciting and potentially far-reaching progress is evident with improving local governance of health services but future work progress will depend to some extent on political decisions made by the central government on decentralization and on the upcoming federal form of governance. The principal challenge for NFHP II in its final year is to ensure the institutionalization and sustainability of the technical and programmatic approaches and systems supported in some cases since 2002 (through NFHP I) and for others since late 2007. Human resource deficiencies, frequent transfers and insufficient skills among staff continue to hamper the ability of NFHP II to develop lasting capacity and to disengage from important areas of work. In some cases, new challenges have emerged, for example, the rapid growth in facility deliveries in recent years, driven in part by the incentives provided by the government to both clients and providers. There is now an urgent need for improved monitoring and ensuring the quality of obstetric and newborn care at the most peripheral facilities. The expansion of safe abortion services by the Nepal government also provides a strong need to ensure that clients receive a full range of family planning counseling and services to reduce the need for future abortions. New policies are needed to protect the FCHVs, an important national asset for Nepal, from becoming overloaded from requirements and demands that are unreasonable for a cadre of volunteer workers. The evaluation team attempted to provide recommendations that will help consolidate and maximize the impact of NFHP II and guide USAID/Nepal’s work in the future. Among the most important recommendations for the final year of the program are to:  Selectively pull back during the last year from a direct operational role within the District Health Offices (DHOs) to determine how well the various management and capacity development activities will endure without such intensive NFHP II support.  Maximize the sustainability of the work on supervision and quality improvement by engaging with Management Division of the Department of Health Services on modifications to the national systems to ensure that NFHP II’s contributions are institutionalized.  Place increased emphasis on healthy timing and spacing of pregnancies to reduce pregnancies too early or late in life, increase the spacing of pregnancies to at least 24 months and promote the longer acting methods especially for postpartum and post abortion clients.  Improve the clinical supervision of Auxiliary Nurse Midwives (ANMs) at Health Posts (HPs) and Sub Health Posts (SHPs) to ensure the quality of obstetric and newborn services among facilities with new birthing rooms, and help apply the minimum standards of care that must be met before new birthing services are opened.  Continue to support the MoHP’s efforts to rationalize and take a life-cycle approach to the program of work for FCHVs.  Document and disseminate the important lessons learned from working with the Health Facility Operations Managements Committees on local governance of health services. ix The evaluation team also recommends to USAID/Nepal that a new program after NFHP II focus strongly on health systems strengthening (HSS) including requiring indicators and benchmarks that measure HSS progress. Further work is recommended in health governance, supporting the analysis and use of data, and strengthening data systems for decision-making, logistics management, some investment in improving pre-service training for ANMs, and support for the establishment of improved systems for supervision and quality improvement. Continued involvement in service delivery for family planning (FP), maternal, neonatal and child health (MNCH) and nutrition is important to strengthen the quality of services. An additional five years of targeted work in these areas, closely coordinated with the support from other development partners, will yield substantial and lasting progress toward meeting Nepal’s health sector goals and targets and reduce the need for as much support in the future. 1 I. Background A. Evaluation Objectives and Rationale USAID’s NFHP II is in its fourth year of implementation and the first time it has been evaluated by an external team.1 The evaluation was conducted from June 20 to July 8, 2011 with the overall objective to “assess the effectiveness of NFHP II, document how NFHP II is making a difference in the health status of the people of Nepal with regard to family planning/reproductive health (FP/RH) and maternal, newborn and child health (MNCH), document best practices, and recommend future longer term FP/RH and MNCH programming directions for USAID/Nepal”2 . An evaluation at this juncture is timely as it will help capture successes, challenges and lessons useful for guiding USAID/Nepal’s discussions with the Government of Nepal about the future program in this sector. B. Context of the USAID Health Program in Nepal The table above, taken from the NHSP II (2010-2015), provides an overall view of the substantial progress Nepal has made on health indicators during the past two decades. Nepal has received international attention and acclaim for its progress on reducing maternal and child mortality, and it is on track to meet those MDGs for 2015. The Interim Constitution of Nepal, for the first time in the country’s history, has stipulated health as a fundamental right of the citizens and the forthcoming constitution is also expected to 1 In 2009 NFHP II itself commissioned a mid-term assessment and conducted a population-based survey to determine progress on key indicators. 2 See Annex 3: Evaluation Scope of Work 2 uphold this right. In spite of this, the country faces substantial challenges given its topography, levels of poverty, political instability and vulnerability to disasters. New approaches are needed to address high levels of under nutrition among young children and women, and to increase service utilization among those who live in remote rural areas or belong to castes or ethnic minorities that have had limited access to modern health care. The health care system suffers from human resource deficiencies and other problems that constrain achieving and sustaining better health care outcomes. The MoHP aspires to lead and steer the health sector but its stewardship is undermined by current political, bureaucratic, and economic factors. As Nepal gets close to achieving the mortality and fertility rate goals it set for itself in 2015, progress may be increasingly difficult unless aggressive efforts are made to achieve key maternal, newborn and child health as well as family planning targets. NFHP II is USAID/Nepal’s primary vehicle for supporting the government’s efforts in MNCH and FP/RH. NFHP II functions within the overall program of cooperation between USAID/Nepal and the MoHP in support of the NHSP II (2010-2015). As part of a Sector￾Wide Approach (SWAp) model, the MoHP works closely with all external development partners (EDPs) in Nepal guided by a single national health strategy and work plan. The MoHP and EDPs meet regularly to discuss all aspects of the program including a Joint Consultative Meeting four times a year and annual Joint Appraisal Reviews. The EDP group meets fortnightly and has developed a mechanism where the chair and the co￾chair of the group are rotated among different EDP agencies on an annual basis. Technical Working Groups meet regularly to discuss, plan and monitor family planning, maternal health, child health, nutrition, HIV, and other activities. NFHP II also works in collaboration with other USAID supported health programs including those focused on FP/RH, MNCH, HIV/AIDS, Child Survival and Health Grants Program (CSHGP) PVO grantees, and other partners. II. Scope of Work and Methodology A. Summary of SOW The evaluation scope of work (Annex 3) calls for what is essentially a performance evaluation as defined by the new USAID Evaluation Policy document.3 The scope of work included an assessment of progress toward achieving the outcomes as well as the issues and challenges that pose constraints. It also called for an assessment of lessons learned in order to inform the design of a follow-on program. B. Evaluation methodology USAID/Nepal assembled a team of seven experts to undertake the evaluation. Five of the team members were Nepali experts, two from USAID/Nepal, two consultants (a senior pediatrician and a health systems expert), and one senior health economist from the DFID-supported Nepal Health Sector Support Programme (NHSSP). Two U.S.- based members joined the team, the Senior Maternal Health Advisor from the Office of Health, Infectious Disease and Nutrition (HIDN) from USAID/Washington and a retired USAID Senior Foreign Service Officer who served as Evaluation Team Leader. 3 USAID Evaluation Policy, January 11, 2011 3 The team reviewed a large number of MoHP planning and strategy documents, various survey research and other studies related to NFHP II program areas, NFHP II reports, and USAID documents. Six4 of the 22 core program districts were visited, which included consultations with district health office leaders and managers, health facility staff and service providers, Female Community Health Volunteers (FCHVs), members of Health Facility Operations Management Committees and health facility clients. Stakeholders interviewed in Kathmandu included senior officials from the MoHP, EDP representatives, partner organizations managing other USAID health programs, and other partners of the NFHP II. (See Annex 2) The team had extensive briefings by the NFHP II staff as well as interaction with the internal consortium partners who help implement the program. The USAID members of the Evaluation Team did not participate in the Kathmandu level stakeholder consultations to facilitate frank and open dialogue about the program. The Evaluation Team met for a day to plan the evaluation and develop the evaluation work plan. The work plan consisted of: an analytic framework, a set of questions for various stakeholders aimed at collecting key information, a facility and community checklist to be used by each of the two sub-teams that visited three districts each, a draft report outline and a timetable for the evaluation process. When the evaluation was nearing completion, oral debriefings were conducted for both the USAID/Nepal and NFHP II staff to present and discuss the findings, after which the team completed writing this document. III. Summary NFHP II Description A. Brief summary of project goals, objectives, components The NFHP II, a $30 million five-year program of support to the MoHP, began in December of 2007 and is implemented through a Cooperative Agreement between USAID/Nepal and John Snow Inc. (JSI) Research and Training Institute and its partners. It followed the NFHP I that spanned the period 2002-2007, building on many of the same program elements with some changes in the focus districts and with new program activities and partners. In NFHP II, while the partners contribute various staff based on their areas of expertise, JSI manages the program and activity budgets. NFHP II was designed as a direct support mechanism to the MoHP and therefore provides technical and other inputs as directed by the MoHP at the national and district levels in specific areas described in the NHSP II. Some technical and program support is provided directly by USAID/Nepal through NFHP II. This includes funding for what is termed a “supplemental budget” for a set of activities mutually planned and documented in annual work plans that are signed by all parties, including any other EDPs contributing to the same programs to avoid any overlap. In addition, USAID/Nepal provides “Redbook” funding which are funds disbursed directly from USAID/Nepal to the MoHP’s budget and reflected in the annual work plans developed with each of the relevant divisions within the Department of Health Services (DoHS). The following is NFHP II strategic framework that defines the results to be achieved: 4 Mid-Western Region: Dailekh, Surkhet and Bangke, Central Region: Sindhuli and Sarlahi, Eastern Region: Siraha 4 Strategic Framework of NFHP II In the Cooperative Agreement the key results are summarized as follows:  Public health impact at scale in 20 intensive focus districts5 and nationally  Strengthened capacity of MoHP systems and staff at central, district and sub￾district levels  Expanded service coverage, especially among marginalized and disadvantaged groups  Increased community participation in decision-making regarding management of local health services  Innovative FP/MNCH best practices tested, refined and introduced into the Nepali health system. Activities are organized into three principal components: (1) Systems, Leadership and Policy – the health systems strengthening activities, (2) Health Service Delivery – the health service improvements at the community and facility levels and (3) Community and Household – a set of behavior change communication activities, a life skills and literacy program, the local governance and health facility operations management committee work and other Gender Equity and Social Inclusion (GESI) efforts. B. NFHP II’s Performance Monitoring Plan NFHP II’s Monitoring and Evaluation (M&E) Plan6 outlines the impact level indicators as  Neonatal, infant and under-5 mortality 5 Two remote districts were subsequently added to make a total of 22 districts. 6 NFHP II Monitoring and Evaluation Plan, November 2007 Objective: Improve provision & use of public sector family planning, maternal, neonatal, child health and related social services in order to reduce fertility and mortality POR 2: Basic Health & Social Services Delivered at Scale through Improved Systems & Development of Scalable Programs Sub‐Result 2.1: Services Delivered at Scale through Improved Systems Sub‐Result 2.2: Scalable Program Models Developed POR 3: Community Capacity to Access & Manage Basic Health & Social Services Improved SR 3.1: Community Participation in Health Services Improved SR 3.2: Community Efficacy for Healthy behavior Strengthened POR 1: Government Leadership & Management Capacity for Basic Health Services Strengthened SR 1.1: National Level Leadership and Management Capacity Strengthened SR 1.2: District Level Leadership and Management Capacity Strengthened 5  Maternal mortality  Contraceptive prevalence rate, total fertility rate  Child anthropometrics, especially stunting rates While the impact indicators will only be measured at the beginning and end of the program, the plan also defines a set of output and outcome level indicators that the program will report on regularly to USAID. (See Annex 4 for a complete list of M&E indicators reported to USAID/Nepal.) C. Project Alignment with Global Health Initiative (GHI) and Best Action Plan for Nepal While NFHP II was designed and began work before the advent of the GHI Initiative, it is completely consistent with its seven major principles and in fact serves a mechanism to operationalize the GHI strategy in Nepal. As a “whole of US government” effort, the GHI strategy focuses most strongly on three of the major principles as follows: 1. Increase Government of Nepal ownership and capacity to govern, manage and improve decision-making in the health sector, 2. Build public, private and not-for-profit partnerships that integrate services and facilitate exchange of innovative approaches, and 3. Improve health care and opportunities for women, children, and marginalized populations in the context of extending services to all. NFHP II contributes most directly to #1 and #3 of these strategies. The entire NFHP II program is aimed at increasing the capacity of the health care system, which is led by the government. It also has an explicit GESI strategy to reach disadvantaged ethnic minorities, castes, women and residents of remote rural areas. It collects gender￾disaggregated data on training and on the proportion of health facility clients that are dalits (a disadvantaged caste group). The Best Action Plan for Nepal7 outlines the various program areas and best practices implemented through the USAID/Nepal’s health program and the ways in which it works with the MoHP and other partners to support achievement of health sector objectives. The activities included in the NFHP II feature heavily in this strategy as it is one of the key vehicles for contributing best practices in the areas of FP and MNCH. IV. Findings: Progress and Challenges A. Component One: Systems, Policy and Leadership A. 1. Logistics System Progress towards achieving results 7 “Best Practices at Scale in Home, Communities and Facilities: Five Year Action Plan for Family Planning, Maternal, Neonatal and Child Health, and Nutrition”, March 2011-September 2015. 6 NFHP II works with MoHP’s central, regional, and district level authorities to strengthen the logistics system, as a continuation of support USAID/Nepal has provided to this system for several decades. This support is in the form of technical assistance to Logistics Management Division (LMD) across a broad range of logistics functions. The USAID Deliver Project also partners with NFHP II to provide support to logistics systems. NFHP II and Deliver Project both provide seconded staff to LMD in Kathmandu and four out of five Regional Medical Stores (RMS) of the country. In addition, Deliver Project has contracted assistant level staff in all 75-district health offices to support government personnel in Web-based LMIS and storekeeping. The evaluation team noted that the government partners do not generally distinguish between the support provided by NFHP II or Deliver Project; nevertheless, the Annual Report of DoHS 2009/10 explicitly mentions both the agencies as development partners of LMD. Most government and external development partners the evaluation team interviewed considered logistics a mainstay of NFHP II’s support in health systems strengthening. NFHP II’s continuous support to logistics has, among other things, contributed to reduce the stock-outs of health commodities at government health facilities of the country. In the current fiscal year, national stock-out of FP commodities stands at 2.4% compared to 4.1% in 2007/08. Similarly, stock-out of MCH commodities this year is 2.8% compared to 9.4% in 2007/08. Furthermore, according to the Annual Report of DoHS 2009/10, physical achievement of planned activities of LMD was 99% and financial achievement was 80%. Following are some concrete areas within Logistics Systems where NFHP II provides technical assistance: 1. Logistics Management Information System (LMIS) 2. Procurement of Quality Health Commodities 3. Inventory Management and Pull System 4. Human Resource Development 5. Strengthening of Storage Capacity LMIS was first developed and tested in four districts in 1994 with the technical support from JSI and USAID. In 1997, it was expanded nationwide and JSI continued to provide support to further strengthen the system. NFHP I eventually became the key partner to support and improve the system. NFHP II continues with this legacy and is considered the main partner by MoHP to provide technical support to LMIS. LMIS is widely considered as one of the better-established health sector information systems in Nepal with reporting coverage exceeding 95% nationwide. Currently, LMIS tracks all health commodities, including essential drugs. LMIS remains an indispensable tool for inventory management, forecasting, and monitoring of health commodities and increasing number of logistics decisions are based on it. LMIS produces quarterly reports, and this reporting frequency has certain limitations (these are discussed under issues and challenges below). In 2008, LMD with the support of NFHPII, UNFPA and DFID decided to capitalize on continuously advancing and expanding Internet technologies in Nepal to launch Web-based LMIS with the main objective of increasing the LMIS reporting frequency by making it monthly. The Web-based LMIS is now expanded to all 75 districts of Nepal. From FY 2008/09 onwards, LMD has begun to allocate resources from its own budget to provide annual refresher training to its district staff on web-based LMIS and inventory management system. 7 Although NFHP II does not procure health commodities, it supports LMD and DHOs with various technical and managerial aspects of procurement. These include regularly supporting LMD for long-term forecasting of health commodities and providing technical inputs to develop a multi-year procurement plan. Better quantification and forecasting have enabled the government to allocate more and more of its own resources for commodity procurement. For FY 2011/12, LMD plans to purchase 100% of FP commodities from its budget compared to 74% it procured in 2007/08. NFHP II was also instrumental in providing technical inputs to develop a ‘central bidding local purchasing’ mechanism. In 2011, NFHP II recruited an external short-term consultant to support LMD to evaluate expression of interest and bidding documents for purchasing health commodities. To ensure better availability of health commodities at health facilities, MoHP introduced the ‘Pull’ system (demand-based supply system) in 2004 with support from NFHP II and KfW (German Development Bank). Initially, the system was implemented in six districts as a pilot; with the support of NFHP II and Deliver Project, it is now scaled up to all 75 districts. The Pull system mainly functions within districts, i.e. peripheral health facilities ‘pull’ commodities from respective district stores. On the other hand, the center and regional stores still push the commodities to their subsidiaries. Alongside LMIS, the pull system is widely recognized by both the government and development partners as the key initiative of logistics management, which has improved availability of health commodities and reduced stock-outs. Some policy makers see the pull system as reflective of the spirit of decentralization where subordinate authorities of different levels (district, facility, community) are able to take logistics decisions. NFHP II has also provided support to develop a computerized inventory management system. All district health offices and regional stores currently use this system but it has not yet been rolled out to the peripheral health facilities – perhaps owing to technological constraints. Apart from improving inventory management at districts and regions, the computerized system also leverages the hybrid push/pull system of the government by allowing authorities to make better decisions and monitor stocks. Efforts are underway to capacitate LMD to handle HIV and AIDS commodities and NFHP II and Deliver are supporting it to integrate these commodities in the main logistics system. NFHP II has significantly invested in human resource development for improved logistics management in the health sector. Partnering both with the National Health Training Centre (NHTC) and LMD, it has provided various training packages. These range from basic logistics training for district storekeepers to orientation on logistics to FCHVs. To date, more than 9000 government employees have received logistics related training with the support of NFHP II. NFHP II has also played a crucial role in bolstering the government’s logistics training capacity. Currently, within the logistics training pool, 70% trainers belong to the government and the other 30% are supported through NFHP II. NFHP II also supports in developing training curricula and guidelines for the government. Furthermore, seconded staff from NFHP II and Deliver Project are also expected to provide on-the-job coaching and transfer knowledge to their government counterparts. Strengthening storage capacity is crucial for maintaining an effective supply-chain of health commodities. One of the constraints for the government in implementing a nationwide pull system is the lack of storage capacity, especially at center and regions. Although NFHP II’s support does not include direct construction or erecting infrastructure, JSI has partnered with KfW to provide management support in constructing district medical stores. This places NFHP II in a position of comparative 8 advantage to provide technical inputs in strengthening storage systems. For example, NFHP II supported the government in developing detailed architectural schema for new regional medical stores. NFHP II also advocated successfully to the government to include the construction of new medical stores in MoHP’s work plan. Issues and challenges Persistent lack, and inadequate management, of human resources continue to plague the health sector at large, and logistics system is also adversely affected by it. There is a shortage of adequately trained staff to handle the logistics functions. Frequent transfer of trained personnel – often outside of the health sector – makes developing and retaining capacity a daunting task. There remains a concern that seconded staff from NFHP II/Deliver Project may create dependency for the government and may also deter it from taking proactive steps to fulfill vacant positions. The government and NFHP II have yet to come up with a concrete plan to institutionalize NFHP II's support and build on its accomplishments in logistics after the program ends in 2012. The current quarterly reporting system of LMIS has a limitation on monitoring stock-out of health commodities that occur in between the two reporting periods of three months. As LMIS reports from all health facilities of the country are sent and analyzed at LMD in Kathmandu, increasing the frequency of reporting (e.g. monthly) would place an enormous strain on already stretched workforce of LMD. There are mechanisms in place, such as filing the ‘Emergency Order Point’ when stocks drop below a certain level, to reduce stock-outs. Nevertheless, effective utilization of these mechanisms is largely dependent on attitude and management capacity of health facility in-charges and respective DHOs. The still-young web-based LMIS system provides a better picture for stock-outs, but it is only implemented at DHOs. Once the glitches on current Web-based system are reduced, and if the system is eventually rolled out to various health facilities, monitoring of the stock-out situation may improve drastically and may even provide real￾time information to decision makers. The attitude and perception of health workers towards stock-outs and proper logistics management is also an important aspect to consider. For example, the evaluation team observed that a particular health facility was maintaining a minimum prescribed stock of ORS (for the sake of maintaining the stock) without handing it out to patients while it awaited supplies from the district store. The evaluation team also came across an anecdote of low drug stocks resulting in under dosing of patients. However, there is no evidence to determine whether these are serious concerns or one-off sporadic instances. Further study may be required to scrutinize current logistics practices and rational use of drugs. Even though NFHP II does not conduct direct procurement, and limits its support in this area to management and technical aspects of operations, a senior government official informed the evaluation team that as other agencies like DFID have stepped up to provide technical assistance in procurement, NFHP II should instead be focusing on other areas of the logistics system. The evaluation team found mixed feelings among government and external development partners when it came to the implementation of the pull system. Even though most agreed that the pull system is an effective mechanism to ensure availability of commodities in the health facilities, some thought the implementation could be better. 9 One constraining factor for effectively implementing the pull system across all four tiers of governance (center, region, districts, and periphery) is inadequate storage capacity of central and regional medical stores. This forces central and regional authorities to push the commodities downward, often resulting in wastage and expiry. The planned construction of central warehouses and regional medical stores is expected to alleviate this situation. The evaluation team observed issues at regional and district stores and health facilities regarding oxytocin: 1) it is being procured in 5 IU ampoules when 10 IU is needed – likely doubling the cost of the drug, and 2) no one from the regional storehouse down to the sub-health post was aware that oxytocin should be kept cool. The storehouses should definitely keep it refrigerated and clinics need to keep it at 25-30 degrees C. Innovative ways must be found to keep the oxytocin cool or supplying birthing centers with regular, smaller quantities to ensure rapid use, especially in the Terai (southern plains) during the summer months when indoor temperature can reach up to 40°C. Labor and delivery rooms in the facilities with birthing centers must keep it available at all times – refrigerator or not. But it is of concern that national level storekeepers have apparently not acted on this issue and not provided guidance to regions and districts. A. 2. Information Management Progress towards achieving results In addition to LMIS, as part of health systems strengthening, NFHP II also provides some technical support to Health Management Information System (HMIS) and to government’s overall endeavor in improving health information management. At the districts NFHP II ’s field staff engage closely with the government staff to improve the quality and use of HMIS data. They provide technical support to district health staff in HMIS data verification and during ilaka (sub-district level) meetings, NFHP II provides support to analyze and interpret HMIS data, which improves the quality of these meetings. As part of its routine support to DHOs, NFHP II continues to support them to prepare for district reviews and in preparing annual district health reports. Similarly, NFHP II also supports the Regional Health Directorates to conduct performance review meetings. NFHP II has piloted the Public Health Analytical Course – a mixture of training and workshop intended to improve analysis and promote HMIS data use – in two districts. After an initial five-day training, the participants identified gaps in data analysis and areas for improvement. An action plan was subsequently drafted and linked with district level Quality Assurance Working Group (QAWG) for further action. As an example, the DHO team of Surkhet district identified an underserved community of Dalits using this exercise and planned a Community Based (CB)-MNCH intervention to address their need. The staff prepared an action plan to offer the CB-MNCH care and monitor the progress regularly. The course contributed to an increase in the coverage of the MNCH services. Moreover, the course brought considerable improvements in recording and reporting system of the districts. Both district and central government partners the evaluation team spoke to have appreciated the results brought about by this course and said it should be scaled-up in other districts. Reporting of private sector data to HMIS remains a nationwide concern. In 2011 NFHP II organized an orientation to 21 private health institutions of Dhanusha and Banke 10 districts. It is too early to assess its results and whether such orientations would prove fruitful to enable the private sector to better report to HMIS. However, NFHP II staff say that some progress has been observed. At the central level, NFHP II placed a seconded staff at HMIS Unit/Management Division to support it in analyzing health data. NFHP II has advocated and supported the incorporation of relevant indicators in the HMIS once a program is scaled-up to majority districts of the country, e.g. Community Based Integrated Management of Childhood Illness (CB-IMCI) indicators. NFHP II continues to be part of national level workshops and working groups to develop an integrated approach to information management in the health sector. While other development partners support the Management Division to organize Annual National Review meetings and in developing Annual Report of DoHS, NFHP II was been most active in regional review meetings, especially in the Mid- and Far West regions. Issues and challenges Even though NFHP II provides a seconded staff to the HMIS Unit, it does not have a structured assistance plan for the unit. The evaluation team also encountered a concern that NFHP II is not very proactive in sharing its learning and lessons gathered from the field to the central HMIS unit. For example, even though personnel from HMIS unit were engaged as the resource persons in training sessions of Public Health Analytical Course, the unit was not actively involved during conceptualization and initial planning of the course. Engaging the HMIS unit is paramount for scaling-up the course effectively in the future. NFHP II’s potential contribution to developing an integrated and comprehensive health information system has not been fully realized due to the complexity of the task and problems within the government. Despite being one of the eight core outputs of NHSP-I, and in spite of Health Sector Information Strategy (HSIS) being in place, the government was not able to provide sufficient impetus to make meaningful inroads in integrating health information systems. Recently, under the auspices of NHSP-II, there has been some effort to revitalize this process, and NFHP II may still have a potential contribution to make in this area. A. 3. Policy Support Progress towards achieving results NFHP II’s support for policy stems from its considerable experience and wide recognition that it is one of the foremost TA agencies engaged in strengthening health service delivery. A long history of piloting different products and scaling them up, such as CB￾IMCI, also places it in an advantageous position to influence health sector strategies. Mainly working at the implementation level, NFHP II generates evidence and lessons that the government and other development partners utilize to formulate policies and set strategic directions. For example, technical working groups drafting the NHSP II extensively utilized the preliminary findings from a mid-term survey conducted by NFHP II. In fact, the survey findings were used as de-facto baseline information for developing the strategy. NFHP II’s continuous system-wide support to logistics also made it an important actor to contribute in the development of Central Bidding Local Purchasing (CBLP) mechanism of MoHP. NFHP II was also able to bring in its rich field-level 11 implementation experience to contribute to the development of facility level Quality Assurance Guidelines in 2008/09. The project also offered evidence, advice and field experience in the process of supporting the development of national family planning policies and strategies. Following the creation of FCHV endowment funds intended to motivate FCHVs, and its subsequent expansion to 21 districts, NFHP II conducted a study to assess its merits. The study findings raised doubts on the ability of the fund to motivate FCHVs. The government has acknowledged the study findings and is currently in a process of revising the fund mechanism to address the need of FCHVs. NFHP I and II supported pilot programs such as the Morang Innovative Neonatal Intervention (MINI) and the MSC (Misoprostol) pilot that generated evidence and informed policy makers on the effectiveness of community based neonatal care and prevention of post-partum hemorrhage. Other on-going initiatives such as the use of Chlorhexidine for umbilical cord care and the Gentamicin in Uniject for neonatal sepsis are also expected to inform the policy makers accordingly. In keeping with the Paris Declaration, working through a common strategic framework is a growing culture in Nepal’s health sector, as illustrated by the commitment of the MoHP and EDPs to the SWAp process since 2004. Within this approach, programs and activities are being steered and coordinated through various Technical/Thematic Working Groups (TWGs) to which both state and non-state actors engaged in the health sector participate. The TWGs at central level often play a significant role in policy and strategy formulation at the central level, while at the district level, they contribute to strengthen the district health system and provide policy feedback to the center. NFHP II continues to help coordinate and participate actively in several of these TWGs both at the center and district level. For example, at the district level, NFHP II is a core member of Quality Assurance Working Group (QAWG) and Reproductive Health Coordination Committee (RHCC), among others. Similarly, at the central level, NFHP II is a core member of CB-IMCI TWG, Training TWG, a special MSC (Misoprostol) Working Group under DG/DoHS, etc. One of NFHP II’s support areas is Local Health Governance Strengthening Program (LHGSP). Local health governance operates under the guidance of the Local Self Governance Act of 1999, which in itself is part of the overarching political strategy of decentralization. Therefore, success of this program very much depends on developing effective linkages between the policy and implementation level. In light of forthcoming federal form of governance, connecting these levels becomes even more crucial. NFHP II was instrumental in providing conceptual insights to the government in developing local health governance program. NFHP II is a part of the Local Health Governance Task Force within MoHP where it provides technical inputs on strengthening local health governance, devolution of health programs, local resource mobilization, fostering good governance, etc. NFHP II is also able to bring field level perspective from its Health Facility Operational Management Committee (HFOMC) strengthening programs to central level governance forums. NFHP II regularly supports LHGSP Technical Coordination Team and Steering Committee by facilitating their meetings and providing technical inputs to develop or refine ideas and concepts. In 2010, NFHP II supported the government to organize a workshop to finalize Health Sector Devolution Framework and implementation guidelines with orientation and advocacy packages. NFHP II is also part of the process led by the MoHP and the Ministry of Local Development (MoLD), which is the principal line agency for promoting local governance. 12 NFHP II was also closely involved in developing NHSP-II. It contributed to the human resource chapter of NHSP-II document by adding information and evidence regarding FCHVs. As part of a thematic working group to develop the chapter on logistics management, NFHP II was instrumental in framing results in this area. Furthermore, during the development process of NHSP-II, the government and development partners regularly consulted with NFHP to obtain evidence and information on MNCH. NFHP II continues to participate in national forums such as Joint Annual Reviews and National Review of DoHS where it has ample scope to advocate for the smooth implementation of key health programs. Issues and challenges Many EDPs interviewed by the evaluation team view NFHP II as well positioned to work in the policy arena. Some said NFHP II is not engaging effectively enough at the policy level. They felt that although NFHP II is indirectly influencing policies through its implementation experience, it does not have direct influence on policy matters, especially with regards to sector reform. Regular participation of NFHP II in the Health Policy Advisory Committee meeting and the fortnightly EDP meeting is limited. USAID/Nepal, however, regularly participates in these forums. Some EDPs also said that internal arrangement and specific roles of USAID/Nepal and NFHP II when it comes to operating in the field of policy support, is not clear to them. With a multitude of TA actors and components engaged in the health sector, the issue of role clarification is often a broader one that encompasses other donors and the government as well. Efforts are currently underway to develop a Joint Technical Assistance Agreement, which is expected to coordinate TA better in the health sector and provide operational clarity to all partners. In many areas where NFHP II was actually instrumental in influencing policies, an information gap is evident. As compared to NFHP II’s excellent system of documenting results and evidence, there is much to be desired when it comes to documenting policy support. For example, NFHP II has updated and published 21 different technical briefs highlighting its contribution to the health sector; however, a separate technical brief that highlights its support to the policy level is not there. NFHP II’s own field-monitoring system based on the Technical Support Visit (TSV) is well recognized as an innovative product in program supervision and performance improvement. However, NFHP II has not always been able to ensure that the relevant DHO staff accompany them on the visits, and has yet to contribute substantial efforts to institutionalize this at the national level. The evaluation team did notice that in Surkhet district a ‘hybrid’ supervision instrument based on the TSV and the government’s integrated supervision checklist is being used effectively. NFHP II can make a better effort to share such experiences at the central level to improve the national system for supervision. A. 4. National and District Level Leadership/Management Capacity Progress towards achieving results The Cooperative Agreement between USAID and JSI outlines developing national and district level leadership/management capacities as separate strategic areas with their own sets of objectives. However, in practice, most of NFHP II’s efforts in building 13 leadership and management capacity are woven into its activities across the three components. The close relationship and continuous engagement of NFHP II with the government units at both the central and district level have enhanced the leadership and management capacity of the government officials. There are examples to support this: senior managers of LMD have been able to negotiate with the government to allocate more budget for the Division over the years. An indication of this is LMD’s ability to procure a progressively higher percentage of FP commodities from its own resources. Tight integration of NFHP II within the DHOs has created an environment for learning by doing together. Such an environment allows for innovations to foster from within, under the leadership of DHOs, rather than changes being instigated from outside. Furthermore, it encourages the district health leadership to assume ownership over innovative concepts and products. A case in point here is the QAWG, which is a mechanism being institutionalized within the district health system. Contributions to the QA District Fund are beginning to trickle from district health budget and other local resources. In addition to being an active participant in TWGs related to its core program areas like MNCH and logistics, NFHP II supports strengthening the leadership and management capacity of these groups. For example, in 2010, encouraged by results produced by quarterly meeting of QAWG, NFHP II successfully advocated to increase the frequency of QAWG meetings by making them monthly. Similarly, at the central level, NFHP II advocated to set up two TWGs focusing on training under the leadership of NHTC. As part of its support to HFOMCs, NFHP II regularly provides the committees with management and leadership training. The result is very encouraging. To date, 422 HFOMCs supported by NFHP II have been able to generate the cumulative total of Nepalese Rs. 45,048,810 (USD 625,678) in cash or kind from their Village Development Committees (VDCs) and other local contributions. There are numerous other examples of HFOMCs exhibiting their leadership and management capacity. These include encouraging the establishment of birthing centers in their health facilities (HFs). A HFOMC that the evaluation team met in Dailekh District was constructing a new building mainly through the funds it generated locally. The members were also busy lobbying with different organizations to obtain an ambulance for the facility. In addition, there are examples where HFOMCs are reconstituted to include persons from marginalized and disadvantaged groups in their membership. At the community level, NFHP II’s continuous support to FCHVs on various fronts has enhanced the FCHV’s leadership capacity and made them better advocates for demanding quality healthcare services. In one particular instance, the FCHVs alerted the evaluation team members about a health facility facing the stock-out of oral rehydration solution (ORS). NFHP II has conducted some training to enhance the leadership and management capacity of the government staff. It conducted Appreciative Inquiry training for the district health staff in seven districts. Similarly, in 2009 both central and district health staff participated in the Leadership Development Program conducted with the support of Management Sciences for Health. Issues and challenges 14 There is a concern that the tight integration of NFHP II staff within the government’s structures may create dependency, which in turn may actually undermine the government’s leadership and management capacity. As one EDP official told the evaluation team, “NFHP II is not here for direct service delivery, it is here to hand-hold the government in its effort to provide better healthcare services.” There seems to be a very thin line between strengthening the government’s service capacity and substituting for its day-to-day functions, which could create problems for the sustainability of NFHP II’s activities. It is also not clear what extra qualifications or expertise NFHP II’s staff, who are embedded within government structures, possess in order to complement the existing capacities of the government’s own team. In spite of NFHP II’s explicitly stated vision of supporting the MoHP’s Health Sector Reform Unit to develop its leadership and management capacity, no concrete efforts have been made so far. The unit is still under-capacitated and lacks adequate resources to undertake its responsibilities effectively. B. Component Two: Health Service Delivery In a country with the difficult geographical terrain and roughly 80% of the population living in rural areas, USAID and its bilateral project, NFHP II, focused on both community and facility-based health services with strong accomplishments and well-documented achievements. NFHP II has a comparative advantage and long history of work in the community with the FCHVs and continues to strengthen and support the services provided by this cadre. It is also clear from many stakeholder discussions and field visits that NFHP II has made significant contributions to the district-level facility-based services in the PHCCs, HPs and SHPs. UNICEF and WHO staff that the evaluation team met with pointed out that NFHP II has done a “great job in supporting service delivery” and added that NFHP II balances it with work on health systems. NFHP II has uses mechanisms such as the QAWG, improvement in commodity management, and TSVs to strengthen the links and “enabling environment” of the various district level facilities and the district health system. This connects the direct service provision with the needed infrastructure and support systems to allow clients to receive services in a timely and efficient manner. Additionally, the fieldwork of NFHP II could best be described as “flexible, filling gaps and responsive” to the DHOs’ needs. The combined lessons learned and experience from NFHP I and II’s are seen in the support and strengthening provided to the district. The accomplishments of the program are apparent when listening to the DHO and some health facility staff discussing their use of data, seeing the graphs of data on walls, and seeing a full cupboard of essential drugs, needed equipment, pits/running water and more. Many EDPs and partners of NFHP II identified the strong technical capacity of NFHP II and extensive field experience as its strengths. The program has been instrumental in both technical areas of service delivery (e.g., misoprostol) and some areas of health systems strengthening that have been supported and scaled-up. AusAid noted that NFHP II’s pilots help define strategies and mainstreaming of best practices and that this was a strength of the program. The collaborative and strong relationships developed with the government officials at both national and district levels are also key to NFHP II’s successes. NFHP II has also supported the development of numerous technical and training guidelines and standards at the request of the Nepal government. As the Director General of Health Services pointed out, NFHP II was developed in close 15 collaboration with the Nepal government and therefore, he is quite satisfied with the work of NFHP II. B.1 Family Planning Services USAID/Nepal has a long history of leadership and strengthening of family planning services in Nepal. NFHP II continues this legacy as identified by a number of EDPs and partners. UNFPA pointed to NFHP II’s strong contributions in the national FP program. Additionally, a number of partners and EDPs pointed to NFHP II’s important role in coordination through Technical Working Groups and other forums, as well as bringing partners together, to strengthen the FP program of Nepal. It is difficult to get a full picture of all that NFHP II is doing in FP as they cover some national level activities and specific interventions in 22 districts, based on the needs and gaps found in each district. Progress toward achieving results Nepal has shown consistently declining fertility rates from the 1996, 2001 and 2006 moving from 4.6 to 3.1 (NDHS 2006). Modern contraceptive prevalence among currently married women has increased from 26% to 44.2% in that same time frame. The NFHP II mid-term survey shows an additional increase to 45.1%. In meetings with a key stakeholder, the Family Health Division (FHD) of the DoHS, their staff stated that USAID/Nepal has contributed substantially to the success of FP in Nepal, through NFHP II. The success of Nepal’s FP efforts is clearly multi-factorial but NFHP II’s makes a significant contribution on supporting FP policy development and revision, logistics management to ensure availability of commodities, increased access of rural and marginalized populations to FP services, quality service provision and data collection, monitoring and supervision. Additionally, NFHP II is thorough and innovative in its quest to increase access, availability and coverage of FP services. It supports government’s multiple community-based strategies through FCHVs; outreach services through Village Health Workers (VHW) and Maternal and Child Health Workers (MCHWs) as well as voluntary surgical contraception (VSC) through mobile clinics throughout the country and the provision of long-term methods through satellite clinics from PHCCs down to health centers and rural areas. Method choice, quality and consistent services are a focus at health facilities. NFHP II is also conducting and supporting important studies and work to determine if there is a “plateau” in FP services or whether there are reasons such as lack of contraceptive use due to migrant husbands that explain the plateau in the contraceptive prevalence rate (CPR). This will inform the government in its future interventions and activities in FP. NFHP II is supporting the FHD in the revision and updating of its FP policy with a main objective to address the FP needs of remote, rural and marginalized communities. NFHP II is also helping put this policy into action in its support for community-based services. The need for appropriate FP messages and targeting of the appropriate audiences is also key to a successful strategy. Four in five women received messages from radio, two in five mentioned television and billboards were also important as nearly one in two (49%) saw messages here. (NDHS 2006) To achieve progress toward achieving results, NFHP II assisted in the development of the FP/RH Communication Strategy 2011-2015, which is currently being reviewed for final approval by the National Health Education, Information and Communications Centre (NHEICC). Lastly, trained health providers and FCHVs are necessary to provide quality services and messages to women and families. 16 NFHP II supports the Nepal government with FP training at the national level through the NHTC, FHD, LMD and others; regional training institutions; and district level organizations to provide training in FP. NFHP II staff indicated that their focus is to work through the government system by either developing government trainers or using trainers or training centers run by government. The current focus is on developing district level training capacity. NFHP II staff stated that they support three government training centers: Banke, Kathmandu and Morang. Because institutional capacity is now sufficiently strong, USAID/Nepal turned over the management of two training centers previously supported directly by NFHP II through sub-grants to the government in June 2011 (Chhetrapati Family Welfare Center – CFWC - and the Institutionalized Clinical Training Center - ICTC). Training remains an important way that service providers are updated in knowledge and skills to provide quality services to clients; NFHP II has supported:  orientation workshops on postpartum FP  revision of the Comprehensive FP/Counseling Training Package  FP and management of training data  FP (implants, non-surgical vasectomy – NSV- and mini-laps) at 6 training centers  FP clinical training skills. The total number of persons trained in FP to date through NFHP II is 1,388 (from TraiNet data – June 28, 2011). The NFHP II mid-term survey shows a significant decline in the government sector as the prime source of contraception. The non-governmental sector has more than doubled in the last 3 years with the Family Planning Association of Nepal (FPAN) and Marie Stopes carrying out increased female sterilization at 7% and 8%, respectively of the total. While the private sector has remained the same, Sangini service centers (7%) and pharmacies (16%) are increasing their provision of injectables. Community-based services: The mid-term survey shows an increase in the role of FCHV to 5% and the PHC outreach to 4% as sources of contraception. The field visits found all FCHVs well supplied with FP commodities – pills and condoms, multiple records and health education materials. FCHVs bring their reports and get resupplied monthly from the closest health facility, although most facilities do not appear to use this opportunity for training or technical updating. FCHVs get periodic, focused refresher training supported by NFHP II, e.g. one-day orientation on VSC and need to reach marginalized populations. The total number of FCHVs trained, which includes FP training, is 26,343 (from TraiNet data – June 28, 2011) While the role of the mobile clinics has decreased slightly, these clinics are still the source of FP for 22% of the population. PHC outreach sessions also present another opportunity to take services and messages to the people when they are conducted regularly. The percentage of FP users that are receiving their services from outreach has more than doubled in the NFHP II core program districts (CPDs) from 1.9% in 2006 to 4.4% in 20098 . VHWs and MCHWs, the health providers who support outreach services 8 NFHP II Mid-Term Survey, 2009 17 in the district, received FP refresher training in two new districts and follow-up in 12 districts during the last 6 months. This way to strengthen their ability to reach the marginalized people and communities, including adding FP counseling and services to immunization clinics to reduce missed opportunities. Analysis of HMIS data from three districts shows an increase in current users of condoms, pills and injectables in the year following this training. Knowledge of some methods of contraception is nearly universal but messages to communities to actually stimulate the increased use of FP or to increase use of long term methods is more challenging, requiring innovation and creativity. NFHP II is supporting a number of activities that specifically increase the reach of FP services and address the needs of the rural and marginalized including the addition of two new partners, Center for Development and Population Activities (CEDPA) and FPAN to increase its reach to these populations. CEDPA work includes reaching out to Muslim communities, which are underserved minority populations in Nepal. NFHP II supports the VSC services as they remain an important component, even though decreasing as a proportion of Nepal’s total FP service provision as other methods become more popular. NFHP II orients FCHVs up to two weeks ahead of VSC mobile clinics in their areas and assisted all 20 CPDs with pre-planning, planning, and implementation of VSC mobile outreach services. A number of DHOs and facility staff found these sessions very helpful. NFHP II also coordinated with NHEICC for airing FP messages in the area. Additionally, NFHP II staff supported VSC services in all CPDs, including the monitoring of the quality of services. A total of 53,657 clients received VSC services through static, seasonal and mobile sites during this past year. NFHP II staff stated that these figures have been decreasing each year over the life of NFHP II – from a high of 90,000 and are analyzing possible reasons for this steady decrease. Facility-based services: District level facilities (PHCCs, HPs and SHPs) have decreased as the source of contraception from 17.2% to 12.2% in NFHP CPDs. The government’s policy, assisted by NFHP II, to take the services directly to the communities is likely having an effect. NFHP II has strengthened the district facilities through a variety of trainings and provision of a new FP counseling box with client pamphlets. These “boxes” were found in many facilities during field visits. Recognizing the need to increase use of the long-term methods and provide communities other options to receive services, select facility staff have been trained to provide IUCD and implants. Three of the ten sites visited by the evaluation team provided IUCDs – all were PHCCs; two PHCCs provided implants (one PHCC was not asked and one PHCC stated no training for implant); one health post and two sub-health posts reported receiving satellite clinics for IUCD and implants. Some health posts reported referring clients requesting IUCDs, including to local Marie Stopes clinics. There were multiple requests from providers (primarily ANMs) for training in IUCDs and implants. Additionally, NFHP II also supports the PHCC staff in 17 CPDs to 18 conduct two to three satellite clinics each by visiting health centers two to three times per month to offer comprehensive FP services, including long-term methods – IUCD and implants. Recognizing abortion as a leading cause of maternal mortality, activities were implemented by NFHP II to improve FP counseling and services at safe abortion sites. Other ways that NFHP II improves quality (and ensures compliance with US policies) includes Tiahrt posters9 prominently displayed in all facilities, with information on informed choice and all FP methods. There were no stock-outs of FP commodities seen at any of the 10 field sites visited or with the FCHVs (all FCHVs’ commodities were not checked so verbal answers were accepted). The guidelines on storage supply and FP standards were identified as valuable by the district staff. Privacy is important and separate rooms are available in health facilities to see clients and screens were often used to hide an examining bed. However, it appeared that the lack of seats elsewhere in the facility has encouraged multiple women/ clients sitting in the same room as the person providing FP (or antenatal care - ANC) services. Other mechanisms identified by the government, as ways to increase local coordination and quality, are the RH Committees and QAWGs. These are important working groups within the districts and from the district health officers’ perspective, both serve important functions. QAWG meets monthly and focuses on problem solving, particularly around quality (that seems primarily defined around infection prevention and waste disposal). The RH Committees meet every three to four months. Both FP and MNH issues are discussed. NFHP II plays an important coordinating role in both of these groups. Issues and Constraints NFHP II staff stated that during the first three to four years, they focused on “promoting methods” but that they have also focused on healthy timing and spacing for the past two to three years. They have now conducted activities with nearly all FCHVs, with the messages: 1) delay marriage until 20 years; 2) wait 24 months before another pregnancy and 3) wait 6 months after an abortion. Guidance is provided in training to talk with each client about their particular situation, including areas on health timing and spacing of pregnancies. With 78% of girls married before age 20, these messages are of critical importance and need more emphasis. Seventeen percent of girls 15-19 years of age have begun childbearing and more than half have given birth before age 20. Among women of reproductive age, 21.3 % have birth intervals of less than 24 months. (NDHS 2006) Clearly, NFHP II’s focus on health timing and spacing is appropriate but likely needs to be reviewed and strengthened. As birthing rooms and medical abortion services expand, IUCD and implant availability at those sites need to keep pace. B. 2. Maternal Health Services Safe motherhood (SM) is a national priority program in Nepal and has expanded significantly over the past 15 years. The global attention to Nepal’s impressive declines in maternal mortality and Nepal’s focus on achieving MDG targets by 2015 keep maternal health in the spotlight. The maternal mortality survey in 1998 provided data and the first significant impetus for Nepal to rethink and make changes to its national policies 9 Helping to fulfill a USAID FP policy requirement for informing clients about the full range of contraceptives and services. 19 in maternal health. Important changes included the skilled birth attendant policies, which created 15 (now 18) SBA training sites and moved skills that had been the purview of physicians to less highly trained personnel to staff of the rural facilities. Programs to increase the awareness of women, their families and communities on danger signs, timely decision making and emergency funds for transport were instituted. Comprehensive emergency obstetric care – CEmOC (13), Basic emergency obstetric care – BEmOC (20) and birthing centers (87) were developed and these numbers have increased since then. The Safe Delivery Incentive Program (SDIP) was initiated in 2005 using Pool Funds to encourage women to give birth in a health facility by providing a lump sum payment (based on geographic accessibility) at the time of discharge. Currently, funds for receiving 4 ANC visits are also provided to the woman and both FCHVs and providers receive an incentive. The SDIP, modified in 2009 as the Aama Program, also removes fees from facility births, obstetric complication management and c-sections. This provided a solid platform upon which NFHP I and II could build and assist the government to strengthen its SM program. A misoprostol pilot was initiated by NFHP I in 2005 to decrease postpartum hemorrhage deaths from women delivering at home and its success has led to a national program on misoprostol. Progress toward achieving results In maternal health, the progress seen in Nepal over the past 15 years is impressive. Nepal’s maternal mortality ratio has decreased from 539 per 100,000 live births in 1996 to 281 in 2006. The trend was confirmed by the eight district maternal mortality and morbidity study done jointly by FHD, USAID and DFID in 2009 with an estimated 229 deaths per 100,000 live births. NFHP II has continued to make important contributions as did its predecessor program, NFHP I. Its support for the GoN has facilitated the development of a number of policies and guidelines, including the National Medical Standards, Vol. III (2nd edit.), training programs with materials and job-aids/ posters. The Director General (DG) of DoHS was particularly pleased with the maternal and child health components of NFHP II. The DG spoke of the successful move of NFHP II’s misoprostol pilot activity into national policy and program. NFHP II has actively coordinated efforts, worked through TWGs and linked with partners and donors to assist the government to move its agenda forward. A clear sign of the NFHP II’s impact is seen with multiple organizations/ partners and donors currently initiating or supporting misoprostol expansion in 16 districts. Additionally, a consistent message of stakeholders interviewed was that this coordinating role played by NFHP II in many sectors has a huge impact on the successes of partner collaboration around the government’s goals, including those in maternal and newborn health. NFHP II’s collection and use of data is one key element to its success. Well-planned and systematic use of data is seen in its pilots, the building and support to the LMIS and HMIS, the use of TSVs for supervision, and the influential survey FP/ MCH Health Situation in Rural Nepal: A Mid-term Survey for NFHPII. The use of this data has allowed NFHP II to identify program gaps and creatively support government and other partners to address them. A good example in maternal health is their recent partnering with two non-governmental organizations (NGOs): United Missions in Nepal to increase NFHP’s reach in Mugu and Bajhang, two very remote districts of Nepal. NFHP II also supported Swiss Development Agency’s Rural Health Development Program to expand the program in Ramechhap and Okhaldhunga districts. 20 Community-based services: FCHVs are the pillar of community activities and the backbone of health services and health education that provides results in rural areas. A strength of NFHP II is its thorough review of systems and linkages between the various levels of the health system. From national policy revision to FCHV support, NFHP II connects it all in an effective and systematic way, including the commodity management program described in Section IV A.1, which includes the supplies used by FCHVs. The FCHV is the link to the women in communities and to the increase in ANC #4 and the decrease in women having no ANC visits. They also play a role in the increased use of SBAs due to NFHP II’s support for refresher training for the Birth Preparedness Program (BPP). During the evaluation team’s field visits, FCHVs made minor requests and had minor complaints but they are pleased with their volunteer status and will continue. The coverage data of MNH services for 10 districts at the community level are impressive. FCHVs are identifying one half or more of the expected pregnancies (though two districts are far behind). Seven of the districts are providing 88 – 93% of women with iron tablets and 93% of recently delivered women were protected from postpartum hemorrhage. Facility-based maternal health services: In maternal health, the mid-term survey shows a significant rise in women using facility-based services. This includes a significant rise in women receiving four ANC visits as compared with the 2006 DHS and there is a significant decline in those receiving no ANC. Skilled birth attendant (SBA) deliveries have increased from 17.4 to 28.8% and there were sharp rises in use of iron tabs, de-worming and a small increase in TT (64% to 81%, 26% to 60%, 67% to 72%, respectively). These statistics show important gains in use of services to the benefit of women but hides some inequities and concerns. The maternal mortality study of 2009 found the strongest links to maternal mortality to be Gross Domestic Product and literacy. Discussions with ANMs in PHCCs, HPs, and SHPs during the evaluation team’s field visits found appropriate knowledge and skills (though not directly observed) in critical obstetric and newborn care functions. NFHP II’s development and support for the “MNH Update” three-day course to address the unmet need for SBA training was identified as a key reason for the ANM’s knowledge indicating that these updates are serving an important function. Again, NFHP II identified a gap – SBA training could not keep up with the demand so NFHP II developed a shorter course focused on the major killers and essential skills needed to save lives. NFHP II, working with the ACCESS Project, also provided limited support to ensure the quality of training in the 18 SBA training sites managed by the MoHP. NFHP II’s support and successful advocacy are definitely assisting Nepal to make progress toward an improved SM program in Nepal and the achievement of MDG 4 & 5 goals. To support this statement, all ANMs queried could give the steps in ‘active management of third stage labor’ (AMTSL); if they didn’t know 21 how to use magnesium sulfate – they knew where the job-aid was to help them; they knew Essential Newborn Care steps (though drying before wrapping was not always mentioned); and steps to resuscitation. The use of partograph is the one tool that is being increasingly used at the facilities. NFHP II has printed partographs, trained in their use and printed job-aids to assist providers to use them. QAWGs, often facilitated by NFHP II, play a very important role in the infection prevention and waste management for facilities. The findings from the field visit were impressive with running water found in 8 of 10 facilities (either spicket or bucket with spicket), toilets were found in 8 of 10 (one was a new building from the VDC and the toilet was not yet built), and placental pits and pits for burning were found at 8 facilities and incinerators at 3 facilities. Issues and Constraints The facility staff and HFOMC took ownership of IP and waste management seriously but there were a few gaps that could improve the quality and safe use, especially over time. No more than half of the placenta pits had covers; the “sharps” – needles - were usually boxed but were often just thrown into the burn pits. Some of the burn pits were quite shallow; and upon inspection, some pits had unburned trash, which is a hazard to the community if children or animals have contact with the trash. In the facilities, all birthing centers either had an autoclave or momo pot for sterilization. Evidence of use was seen in most clinics but the staff were not taught to remove the instruments, especially wrapped instruments, and store them. The condensation that sits inside the top of the containers gets the instruments wet again and spoils the sterilization. Six of ten facilities had chlorine- based disinfectant (one had phenol), primarily Virex, but few could explain how to mix it up and use it. No job-aids were seen on either how to mix chlorine solution or the steps to disinfect instruments. Of note during the field visits by the evaluation team was the lack of attention to breastfeeding, particularly exclusive breastfeeding. There were no posters, no mention by FCHVs or ANMs unless asked specifically. Given the critical importance exclusive breastfeeding has for the infant (and the mother if she uses the Lactational Amenorrhea Method - LAM), exclusive breastfeeding should be given much more prominence in health education both by FCHV and ANMs. On maternal mortality, the data show mortality highest among Muslims, Terai/Madhesi and dalits10. This data strongly supports NFHP II’s focus on reaching the rural, remote and marginalized as the groups listed above are among the poorest and most illiterate. The data highlights an unnoticed but significant concern which found that the MMR for women over 35 increases to 962 per 100,000 live births (compared to the national average of 229). Additionally, a study on stillbirths by Lee, A., et. al found that older women (there are actually significant increases after age 30) and women having their first baby are at significantly higher risk for stillbirth and early neonatal death. It will be important to increase the awareness of providers and communities on the dangers 10 All disadvantaged and underserved religious groups and castes 22 associated with childbearing at older ages, particularly after 35 and the increase in stillbirths for first births. Nepal’s stillbirth rate is high at 35.4% (a study of six low and middle income countries found a range of 9-34% stillbirth rate). Additionally, 60% of the stillbirths occur after 37 weeks which means the infant is viable and these deaths could likely have been prevented with improved recognition of complications in pregnancy and labor. While there is significant focus on maternal mortality (MM), there appears to be much less attention to perinatal, still birth and neonatal mortality. The Nepal Stillbirth Study and the identification of stillbirths in all but one of the facilities visited in the field by the evaluation team highlights the seriousness of this problem and the need to pay more attention to the fetus during labor and delivery. The use of the partograph will ensure better care in labor and with increased monitoring the fetus and prompt and effective use of emergency obstetric and neonatal care, these neonatal/stillbirth deaths could be reduced. One concern for the evaluation team is the lack of clinical skills practice in the MNH Update. The concern about clinical practice is compounded because the TSVs and DHO supervision visits do not include an assessment of clinical competence. On the essential drugs, oxytocin was found in all facilities with birthing centers and magnesium sulfate was found in 5 of the birthing centers. (See IV. A.1. for additional information on oxytocin storage.) Health providers may have knowledge and skills but a common problem is that they lack the equipment, supplies and systems to allow them to use their skills effectively. NFHP II works through the QAWGs and RH committees at the district level and has minimized this problem in NFHP II CPDs. The need to adequately equip the birthing centers was addressed at the national level and funding for equipment was provided by the Pool Funds, other donor funds and NFHP II. In selected sites, NFHP II provides delivery beds, autoclaves, delivery sets, trolleys, vacuum delivery sets, neonatal resuscitation equipment, and necessary supplies on an as needed basis. Eight of the ten facilities visited by the evaluation team had well-equipped and supplied birthing centers. However, in Dailekh, the DHO cannot provide all birthing centers with minimal equipment, even with NFHP II’s help, due to the increase to 48 birthing centers out of 60 facilities. This points to a significant challenge that is likely to get worse with time. The demand for birthing centers may soon outstrip the ability of the GoN’s ability to provide quality services, including training clinically competent ANM’s, equipping the birthing centers, providing IP and waste management. The MoHP is trying to balance the demand and supply side of increasing facility births but the incentive systems combined with radio and other messages and the push by FCHVs for women to have facility births are all creating huge demands for additional birthing centers. The Government should not allow the push to meet the MDG 5 goal to jeopardize the quality of maternity care, as the lack of quality will lead to poor outcomes. Addressing the quality issues now will save money, save lives and prevent substantial problems later. B. 3. Child Health Services Progress toward achieving results Nepal has experienced substantial reductions in neonatal, infant, child and under five mortality in the past 15 years as shown in the chart below, taken from the 2006 NDHS. 23 NFHP II is involved in supporting child health services both at the facility and community levels, although the work has been more intensive at the community level. In general, data from the program’s M&E reports shows that it is achieving outputs beyond the set targets for indicators such as numbers of cases of child pneumonias treated with antibiotics by facility staff, and numbers of children who received Vitamin A capsules and so on. (See Annex 4). The notable progress in early childhood mortality reduction may be due in part to Nepal’s willingness to allow village level workers such as the FCHVs to provide treatment for childhood illness such as diarrhea and pneumonia at the community level. FCHVs also contribute to the high Vitamin A and immunization coverage rates and the increases in spacing of pregnancies with the expansion of the family planning program. Facility-based child health services: NFHP II has been involved in strengthening child health services at the health facility level, building on but expanding the work of the previous NFHP I. The health facilities visited by the evaluation team consistently showed that they were well prepared to care for sick children according to CB-IMCI11 guidelines both in terms of technical competence of the staff and the availability of drugs. All ten facilities visited were actively using the CB-IMCI patient registers and a quick review of them revealed that the children were being assessed, classified and treated according to the IMCI guidelines. The health workers were proud of their ability to treat children with pneumonia and diarrhea in early stages and they expressed satisfaction with the drugs supplied. However, in some health facilities, staff had not received refresher training, and a few said that they had never been trained to use CB-IMCI manual, and they had learned to treat children from their colleagues. NFHP II has provided CB-IMCI refresher training to AHWs, ANMs, VHWs and MCHWs in four districts. Community level interventions: NFHP II’s contribution to strengthening CB-IMCI has been substantial. It has supported district and community level CB-IMCI review meetings in 7 districts, training of new FCHVs in 15 districts, expanded CB-IMCI in four additional districts and provided technical assistance to the expansion of the zinc program in 18 of its 22 districts. From the NFHP II mid-term survey, a significantly higher percentage of children in NFHP II supported districts were given zinc during diarrheal episodes (5.8%) as compared to the 2006 baseline in the same districts (0.7%) in 2006. The levels, however, are still low. The use of ORS also increased from 42.9% to 45.9% in NFHP II districts but further improvements in these levels is also needed. The levels of treatment of children with ARI symptoms also increased only slightly in NFHP II supported districts (21.7% to 23.1%) and the percentage that sought treatment from a health facility also increased. The FCHVs visited by the evaluation team had adequate supplies of Cotrim, were proficient in the use of timers to measure respiration rates and responded 11 In Nepal, CB-IMCI is used to describe both facility and community based IMCI. 24 competently to questions about treatment and referral. They also had ORS and zinc as well as child health educational materials. Some stakeholders at the national level are concerned, however, that communities are increasingly seeking treatment for ill children directly from untrained private drug sellers and pharmacies, which may be either ineffective or dangerous. The national Vitamin A supplementation program for children 6 months to 59 months of age has achieved and maintained high coverage rates throughout the country. Nepal is also gradually moving toward assuming a greater proportion of the costs of the capsule importation for that program. Issues and Challenges for Child Health At the facility level, a review of CB-IMCI registers by the evaluation team showed lack of recognition of severe malnutrition in children. The register requires them to record the weight and classify the child’s nutritional status as “very low weight” where that applies. There were many children who had a weight less than 80% of expected weight for the age of the child but were recorded as of normal nutritional status. Some additional efforts may be needed to include awareness of under nutrition as a problem, and more intensive counseling/educational services provided in the health facilities when children are brought in with infections. In some facilities, the registers were not up to date. At the community level, the FCHVs appear to be more knowledgeable and enthusiastic about the treatment of diarrhea and ARI than they are with the elements of CB-IMCI that require counseling such as infant and young child feeding (IYCF). Children who are small or underweight for their age go un-noticed as national data on stunting shows that roughly half of the population of children under-five years of age is stunted and therefore, both health providers and parents see children who are small for their age as a norm. Additional attention to this issue is clearly a challenge for the country and for NFHP II in the coming year. Given the current low coverage rate for children with diarrhea treated with ORS and zinc, there is also a need for increased emphasis on that element of CB-IMCI as well. B. 4. Newborn Health Services Progress toward achieving results Health of the newborn is addressed by NFHP II through MNH and Community Based Newborn Care Programme (CB-NCP) related activities. The essential newborn care under the MNH program is aimed at preventing hypothermia and providing appropriate care to low birth weight babies. Activities under CB-NCP, a pilot program of the MoHP, are more comprehensive and include resuscitation of newborn suffering from birth asphyxia. The knowledge of FCHVs was found to be adequate regarding resuscitation of newborn and in the four NFHP II supported CB-NCP districts, the FCHVs are well 25 equipped with the bag and mask as well as disposable tubes for aspirating the nose and mouth of newborns. Some duplication in program effort was observed in districts with both MNH program and CB-NCP. Application of chlorhexidine on the umbilical stump to prevent umbilical infection or sepsis has been piloted in Banke district and Maternity Hospital in Kathmandu, and implemented in four districts, but the program is awaiting a review of program performance results before expanding it further. For both of these pilot newborn activities, NFHP II is playing a key role to collect evidence about how these programs work at the field level in order to inform national policy on health service delivery. Issues and Challenges As noted in the section IV B.2 on maternal health services, the primary emphasis has been on interventions aimed at obstetrical care for the woman rather than on the fetus and newborn. Given the increasing facility births and the high still birth/newborn death rate, improving the skills for newborn care including newborn assessment, resuscitation, drying and wrapping, proper core care, and immediate breastfeeding is important for ANMs and others who are conducting deliveries. ANMs also need to be able to assess when prolonged labor is endangering the fetus and refer to higher levels of care if feasible. A review of the CB-NCP program is ongoing and its findings are expected to inform the policy about adopting it a nationwide program for improving newborn health. There are several key issues that NFHP II has brought to the forefront on the CB-NCP program. (1) FCHVs are not likely to have the opportunity to use their newborn resuscitation skills or equipment very often. FCHVs in many areas of the country are unlikely to actually be present during deliveries that may happen at night and the distances over difficult terrain make it unlikely that a female FCHV would venture out alone. In the Terai where physical barriers are less, access to health facilities is also better and an increasing number of women are choosing facility births because of the Amma program incentives. (2) If FCHVs are not using the newborn resuscitation skills very often, is there a danger that they will not retain their skills after training? Furthermore, is it cost￾effective to equip and train all of them if their opportunity to use these skills is relatively rare? (3) It appears that the FCHVs may actually have received better training (though their actual skills are untested) than health workers at the sub-HP and HP facilities to provide newborn resuscitation where more and more births are occurring. As noted in the maternal health section of this report, the attention to date for training ANMs/SBAs seems to be more focused on interventions to save the mother rather than the newborn. It makes good sense to ensure that the ANMs/SBAs are better trained and coached in the area of newborn resuscitation. Furthermore, currently the FCHVs have separate reporting forms and training sessions that overlap in content. NFHP II and the Child Health Division are considering outlining a more life-cycle approach to defining the program of work for FCHVs once the review of 26 the CB-NCP program is completed. This is a highly important policy level review and discussion where NFHP II can make a significant and lasting contribution because of its ability to bring experience and evidence from the pilot activities to support the MoHP’s decision making process. C. Component Three: Community and Households In NFHP II, there are several specific initiatives at the community level that are included under Component 3. This is in addition to (and linked with) the community level service MNCH and FP delivery activities described under the earlier sections. C. 1. Strengthening Local Governance in Health Progress toward achieving project results NFHP II is involved in local health governance in two specific ways. (1) Two districts are supported by NFHP II under the Local Health Governance Strengthening Program (LHGSP), a district-level pilot activity involving GIZ, NHSSP, Plan and WHO with the MoHP and Ministry of Local Development. (2) NFHP II is also supporting the development of Health Facility Operations and Management Committees (HFOMCs), to date in 590 VDCs of 13 districts. (1) LHGSP12: NFHP II is building local government’s capacity to manage health facilities including using the GON provided block grants for improving local resource mobilization. NFHP II worked with District Development Committees (DDCs) and VDC secretaries as well as officials at the regional and national level to understand how the HFOMCs can help both oversee/monitor the provision of health care but also engage in local resource mobilization activities to support the work of health facilities. The future of this effort will depend entirely on the decisions made at the national level about the new federal system and the exact nature of decentralization with respect to roles and authorities of local government. (2) HFOMCs: In a much broader geographic areas, NFHP II is also working to strengthen HFOMCs by supporting assessments of health facility needs, funding training and review meetings for HFOMCs and generally monitoring and encouraging their activities. All the health facilities visited by the evaluation team had a HFOMCs constituted according to the guidelines developed by the government. NFHP II assisted the National Health Training Center (NHTC) to develop the guidelines for the operation of the management committees. Majority of the HFOMC members had received training and in some cases a follow up refresher training also had been conducted. Thus an active body has been created in the community to facilitate community participation in governance of local health services. The HFOMC guidelines require that they have representation from members of dalit and marginalized community and from among FCHVs. The evaluation team found that all the HFOMCs with whom it met had sizable numbers of female members as well as dalits, marginalized communities and FCHVs. NFHP II staff need to continue to support the women and disadvantaged groups to be more vocal and to provide more leadership as that was not always apparent among those visited. 12 More fully described in Section IV. A.3 27 The evaluation team was pleased to see that both the community and facility staff found the HFOMC an effective mechanism, with the NFHP II training being an essential starting point. The HFOMCs are bringing energy and enthusiasm to overseeing the work of the health facilities. They are monitoring whether the facility staff are available and reporting to work and whether drugs are available for the patients. Members reported that when problems with either staff or supplies occur, they demand action from the district health offices. They also seem to be involved in fund raising activities to make improvements to the health facilities or surroundings. There is enormous potential for these committees to greatly improve the sense of ownership and participation by communities and accountability of local officials in health services if they continue to grow and expand. Issues and Constraints The benefits of the HFOMC activities are clear and unambiguous as an important foundation for improving local governance of health facilities in the future. The long term benefits of the LHGSP pilot work, however, is highly dependent on what happens at the national level with decisions related to decentralization of roles and responsibilities for health care and the role of local governments. Whatever happens in that regard, NFHP II and the other EDPs in these pilot programs should make sure to document the lessons learned, including constraints encountered, to ensure that the national government is provided with the benefit of learning from the pilot work as it plans for scale up. C. 2. Literacy and Life Skills Progress toward achieving project results This element of Component 3 of NFHP II consists of three principal activities; Learning Circles (LCs), the Health Education and Adult Literacy (HEAL) program and the Girls Access to Education (GATE) program all managed through a JSI sub-agreement with World Education. The results under each of these activities is impressive and without a doubt, highly beneficial for those involved. 1,619 FCHVs have been trained in how to use the “Learning Circle” approach to effectively disseminate health information working with Mother’s Groups. Over 31,280 women have participated in the LC program in eight districts. Another 10,814 women have participated in HEAL, two thirds of which have been from disadvantaged castes. NFHP II reports an increase in contraceptive prevalence among HEAL participants before and after the classes. The HEAL and LC participants have been encouraged to participate in income generating activities; the program had helped them to access other organizations which work to alleviate poverty and promote agriculture productivity, savings and credit. The team was impressed with the value women must place in the HEAL program to spend two hours a day, six days a week for nine months to acquire the literacy and health information. The purpose of the GATE program is to bring out-of-school adolescent girls into the mainstream schooling system. The program seems to have achieved that purpose by bringing a large proportion of the participants to the formal schools. Of the 2824 girls that have been enrolled in formal schools, only about 5% have dropped out. Issues and Constraints 28 The LC program appears to offer a method for energizing Women’s Groups and giving FCHVs better tools for expanding the impact of Mother’s Groups more widely. While the benefits of the HEAL and GATE programs for the women and girls involved is clear, it is not clear to the evaluation team how these programs can be sustained over time unless they are somehow institutionalized within a government ministry or that more permanent funding can be found for NGOs who work in this area. While improving literacy is a strong determinant of improved health for women and children, the MoHP does not have this mandate and therefore it is difficult for the evaluation team to see how this institutionalization could occur. C.3. Behavior Change Communication Progress toward achieving project results NFHP II has invested considerable efforts to creating awareness both for need and availability of services in the community through mass media, health exhibitions and development of relevant materials for communication and behavior change as mentioned in earlier sections of this report. Radio programs have been developed by involving the experts in the field (BBC WST, Equal Access and others) and NFHP II has provided funds to air them. Support to FM stations in local languages and national radio channels have been provided. NFHP II has worked at the national level to support activities such as the development of a National Family Planning Communications Strategy with NHEICC and developed information education and communication (IEC) and behavior change communication (BCC) print materials for use through out the country. At the district level, they have supported ‘health exhibitions’ and sponsored folk media events. In general, knowledge in the general population about topics such as family planning and newborn care, as well as utilization of ANC and delivery services, has increased between 2006 and the mid-term survey in 2009. Issues and Constraints The majority of the materials used by FCHVs and health facilities are still is in the Nepali language and use images and illustrations that may not be recognized by the various local communities as their own. There is a need to investigate whether this is an important issue for the effectiveness of the materials in communities where Nepali is not well understood. There may, however, be challenges to standardizing the messages in local contexts and languages. C.4. Gender Equity and Social Inclusion (GESI) In general, NFHP II has made concerted effort to ensure that USAID/Nepal and the MoHP’s GESI strategy objectives are addressed. The main avenues for doing so have been by ensuring that:  HFOMCs have female members as well as those from disadvantaged and minority groups,  the Literacy and Life Skills program includes a majority of women from disadvantaged castes,  through the FCHVs, outreach and satellite services, disadvantaged groups who would otherwise hesitate to seek services from health facilities have access to care, and 29  the proportion of health facility clients from among dalits, the most disadvantaged caste in the country, is monitored in selected health facilities. Considering NFHP II’s mandate and scope of work, the evaluation team concluded that these were very reasonable and effective steps to address GESI issues. D. NFHP II Impact D.1 Performance Monitoring The NFHP II Monitoring and Evaluation Framework13 list six impact level indicators to be monitored during the life of the project: neonatal, infant and under-five mortality, maternal mortality, stunting rates (age for height) for children under five years of age, and the contraceptive prevalence rate. The baseline values for these indicators are calculated from the 2006 NDHS dataset and they will be measured again at the end of the project by the 2011 NDHS dataset for NFHP II program districts. Unfortunately the preliminary report for the NDHS 2011 will not be available until August 2011 and the full report and raw dataset until the end of the year, and therefore these data were not available to the evaluation team. Because NFHP II supports the MoHP’s program, its targets for impact indicators are generally the same as those of the MoHP’s NHSP II as well as the country’s MDGs 4 and 5. The NFHP II mid-term survey, covering 20 NFHP II-supported districts as well as 20 other (comparison) districts, provided the trends between 2006 and 2009. Drawing any conclusions about impact based on the NFHP II’s mid-term survey, however, will not give a valid picture of impact, given that the data collection occurred in February and March of 2009 after about 15 months of NFHP II’s initiation. Taken from NFHP II Mid-Term Survey Summary Report 13 NFHP II Monitoring and Evaluation Framework, 27 November 2007 30 For the impact level indicators, the 2011 NDHS trends in mortality reduction and contraceptive prevalence in the regions where NFHP II works may reflect a better composite picture of progress. NFHP II’s contribution, however, cannot be disaggregated from those of the MoHP and its other partners. NFHP II is not likely to contribute substantially to the goal of reducing rates of stunting among under-five children because its primary nutrition activities have been in the areas of micronutrients and some nutrition education through the FCHVs. Nepal’s MDG #1 focus is to reduce prevalence of underweight children under-five years of age to 29% by 2015. D. 2. Output, Outcome and Program Management Indicators NFHP II reports annually to USAID on two sets of indicators. The first set of 14 are USAID’s “Operational Plan (OP)” indicators that are primarily numbers of trainings conducted, clients reached, children treated, etc. which may be considered output level indicators. (See Annex 4 for details) The table in Annex 4 shows that the program is performing very well in terms of meeting expectations on these indicators, in some cases greatly exceeding them. NFHP II also reports on a second set of 12 indicators that are more outcome-oriented. They include population coverage data for key services, as well as health systems achievements such as percentage of facilities receiving supervision visits and the availability of key commodities. Not all of these are reported every year, depending on the source of the data, but in general progress is evident in all areas. Achievements are reported through the end of Year 3 in Annex 4 as the Year 4 data is not yet available. Interestingly, for the important indicator of “Percentage of PHCCs and HPs in core program districts that receive a quarterly supervision visit by DHO staff” has declined from 70% in Year 1 to about 67% in Years 2 and 3 which is disturbing as it may reflect the DHO staff feeling that NFHP II staff visits can simply replace their own need to make the visit. This suggests that a significant effort needs to be made to increase this level during the remaining period in the project for reasons cited earlier in Section IV of this report. In addition to the M&E data reported to USAID, NFHP II collects and uses data for program management purposes from a much larger set of indicators, some of which are reflected in the semi-annual reports. Some of this information comes from the TSV forms filled out on a monthly basis by district level NFHP II staff and analyzed by regional NFHP II staff for purposes of monitoring and supervising activities at the district level. The TSV is a highly useful mechanism for NFHP II to monitor the status of its activities. The data quality assessment conducted by USAID in June 2011 did not reveal any serious deficiencies or problems in NFHP II’s reporting. The report concluded that “JSI and its partners work closely with the government in a collaborative partnership model that covers a wide range of topics and activities. NFHP II places strong value in the collection and use of data for program development and makes ambitious use of information for program management. NFHP II’s MIS appears strong for most indicators although data mining revealed some quality-related issues.”14 14 Data Quality Assessment Summary Report, 2011 31 V. Planning, Implementation, Monitoring and Reporting A. Planning, monitoring and reporting processes A joint planning process in place and NFHP II plans are aligned with GoN priorities as outlined in NHSP I and II. MoHP stakeholders interviewed at the national level appeared to be satisfied that the collaborative process used for planning was reflecting their priorities and needs. The program seeks GoN feedback to the work plan through regular meetings and briefings. NFHP II staff are also highly active in various technical working groups of MoHP and uses these forums for inputs to their planning process as well as a mechanism to develop consensus on key programmatic issues and directions. At the District level, the planning for areas of technical or managerial support is driven by the actual needs as identified through TSVs that should be conducted jointly, whenever feasible, with DHO officials. District NFHP II staff reported that they visit facilities that are not performing as well more frequently than others. The TSV checklists provide a good basis for prioritizing work as well as tracking progress over time. Deficiencies encountered during TSV visits are also brought to the district level QAWG for discussion and decisions about how to help. The M&E Plan for the program focuses on documenting the outputs required by the OP indicators for USAID/Nepal (numbers of trainings, numbers of children receiving selected interventions, etc.) as well as tracking coverage of health services, community participation and so on. M&E staff both at the central and regional offices provide backstopping to support district level staff. The data collected at the facilities from the TSV forms are discussed at each facility and also in the district level to analyze how well they are achieving targets and what kinds of issues they need to focus on. In general NFHP II’s internal monitoring and reporting systems are well developed and very useful for guiding their work and assessing progress. Program monitoring includes staff members from regional and central offices visiting the project sites for tracking the progress; sometimes such visits also include GoN officials from the national or district levels. The program targets are monitored on a regular basis based on the data reported from the field and measures to ensure timely achievement of targets and results are discussed at monthly meetings at the regional and field levels. Program performance is also reviewed using a combination of the GoN system, primarily the HMIS, and other information collected by the field staff through their TSVs and their interactions with HFOMC and HF staff members. NFHP II staff members work closely with DHO staff both at the district and health facility levels to improve the project performance monitoring by supporting in timely collection and reporting of HMIS data and also using this information in their meetings. The practice of analyzing and using the monitoring data is not as common at health facilities as at the districts. The Public Health Analytical Course was helpful in developing DHO staff members’ skills for processing and presenting the HMIS and other data and using them for monitoring the progress of the program. Regarding reporting, NFHP II’s semiannual reports are somewhat fragmented, reflecting accomplishments of a long list of activities rather than a more strategic overview of progress and constraints. While the evaluation team is not advocating a substantial change in the semiannual and annual reports during the last year of the program, a final 32 report on NFHP II would provide an important opportunity to capture and share widely that kind of more thoughtful analysis with a wide range of stakeholders. B. Effectiveness of capacity development work at all levels As noted in Section IV A.4 of this report, NFHP II staff face considerable challenges in the realm of their capacity development work. The unfulfilled government positions, frequent transfer of staff, and inadequately trained personnel have forced NFHP II, as well as other EDPs, to place seconded staff within the government offices to undertake important functions. Furthermore, in order to help strengthen the managerial capacity of district level program managers, NFHP II district staff are fully embedded with the DHO structure which gives them excellent access to a full range of district health activities but also poses a danger for the DHOs to become somewhat dependent on their additional inputs. The evaluation team questioned officials at all levels about whether NFHP II’s approach to capacity development was such that at the end of the NFHP II next year, the capacities developed would enable district health offices to continue without any loss of efficiency or effectiveness of their work. The picture seems to be mixed. Many officials, especially at the central level explained to the evaluation team that, in the absence of needed positions in the MoHP, pulling out NFHP II seconded staff would be highly detrimental. Others said that most programs would continue but perhaps at the reduced pace or with less attention to quality. Clearly new skills and tools have been developed and being applied to benefit FP and MNCH programs as well as program management. In some cases, systems introduced by NFHP II have been better institutionalized (e.g., logistics management) while in others, the systems used by NFHP II have not resulted in significant improvements or changes to the existing MoHP system (e.g., supervision and quality improvement) such that at the end of NFHP II, the danger is higher that the program’s inputs are likely to gradually dissipate if support is not continued. Building robust health systems is a time-consuming and complex process that must continue for years with a gradual decrease of external support especially in more fragile health care systems. NFHP II’s work in logistics management has benefited from that long duration, while in other areas have not. The evaluation team concluded that the most important lesson from this experience is that while USAID/Nepal’s support to health systems development should certainly continue, future interventions must seek to establish clear indicators for measuring progress toward institutionalizing these systems. Section VI: Management Systems A. Systems for Program Management with Government of Nepal NFHP II’s systems for working with the Nepali MoHP have been honed over time and are highly productive. At the central level, there are frequent meetings at various levels to discuss NFHP II support to the DoHS and to develop work plans on an annual basis. DoHS officials interviewed were in general satisfied with the responsiveness and collaborative arrangements with NFHP II, with the exception of those Divisions where the program is not working intensively, and therefore feel somewhat neglected. On an annual basis, USAID/Nepal, NFHP II and the relevant DoHS Divisions develop a work plan for the “supplemental” funds managed through NFHP II with administrative support from Management Support Services Pvt. Ltd. (MASS) (a private contractor) and Redbook funding (provided directly by USAID to the MoHP budget). In some cases, 33 where other donors are involved in the same program, they also indicate their budget support levels and co-sign the work plans. USAID committed about $1 million to Redbook funding with the MoHP this year. At the central level, there is also intensive consultation about the activities to be supported by NFHP II including decisions about which districts will be involved in national pilots and where various NFHP II activities will be focused. Once these decisions have been reached annually, they are passed along to the Regional NFHP II offices and down to the district teams, which discuss them with their DHO counterparts to develop the schedule of activities and events. District health official interviewed by the evaluation team generally felt adequately involved although some had complaints about why they were left out of particular pilot programs (e.g., CB-NCP), a decision usually tied to national DoHS directives to NFHP II. NFHP II’s annual work plans – apart from the supplemental portion - are not shared in draft with the DoHS, a step that could increase transparency. The NFHP II district offices do not have independent budgets as such to support the work plans. They depend on the NFHP II Regional or the Central Office to procure commodities or equipment as needed or to authorize them to do so, when that is more efficient. The NFHP II regional offices provide the technical inputs needed during planned events such as new types of training and so on. Training events are often paid for through either the ‘supplemental’ funding managed by MASS or through Redbook support from USAID. USAID has been trying to shift funds for the more routine activities such as training to the Redbook although there are problems with delayed financial reports and government not being able to fully utilize the funds. Regional Training Center staff complained to the evaluation team that the Redbook funding mechanism is sometimes problematic because the quality of the teaching materials is poor or because the budget for snacks for trainees is so low that participants often leave early, compromising the quality of the training. In general, the management and financial systems used to carry out work with the government functions well for NFHP II and has enabled it to support a large number of small activities even in very remote rural areas simultaneously and without an undue amount of financial risk. B. Management of Internal Partners JSI as the prime recipient for the NFHP II Cooperative Agreement works with 13 other internal partners to accomplish project activities. The role of the organizations and nature of the relationship is quite diverse. Many but not all of the partners are co-located in the NFHP II office building. Some partners carry out discrete activities that do not require them to co-locate (e.g. CEDPA or Equal Access). Others contribute key senior staff positions such as EngenderHealth and Jhpiego, and are fully part of a broad range of NFHP II activities even though they may have corporate offices with staff that are located elsewhere. In addition, two central projects operate within or alongside NFHP II: DELIVER – also implemented by JSI – has separate field support funding from USAID as does MCHIP, which has a very specific scope of work closely tied with the NFHP II program. 34 Discussions with representatives of the internal partners revealed that their staff who are co-located in the NFHP II office feel a part of a cohesive NFHP II team despite being from different ‘home’ institutions. They pointed out that in NFHP I, each organization was also allocated program budgets, which complicated coordination and oversight. In NFHP II, JSI manages all program budgets while the other partners manage the staff salaries and costs. There are a uniform set of administrative procedures governing travel and per diem, leave days and so on that are followed by all NFHP II partners which reduces friction among them. A Senior Management Team (SMT) has been established to provide a strategic guidance for the program. The SMT meets every month and also meets separately with USAID on a monthly basis to discuss progress and issues. This structure also provides a productive mechanism for the interface with USAID/Nepal. In addition, a Senior Technical Team (STT) composed of the Team Leaders of each program area, also meets monthly to discuss technical and strategic issues. The STT includes the heads of the two NFHP II Regional Offices to ensure that the field perspective is represented in key decisions. C. Compliance to USAID Policies and Requirements The evaluation team did not independently scrutinize whether NFHP II was managing USAID assets, either financial or equipment, as required. A meeting with USAID/Nepal Financial Management and Contracts staff did not reveal any specific problems in this regard. The team noted the following: 1. The provisions requiring USAID/Nepal written approval of annual work plans and any changes of key personnel are being followed. 2. The NFHP II staff appear to be fully cognizant of the family planning policies regarding Tiahrt and the Helms provision and have passed that information along to counterparts. USAID employs a full time Nepali “Compliance Monitoring Officer” who looks after these issues with all of the USAID/Nepal programs. The team observed during the field visits that the “Tiahrt posters” describing the full range of family planning methods were found on the walls of all facilities visited. 3. NFHP II also appears to be following the branding and marking requirements of USAID. The team noted equipment and furniture purchased by NFHP II was marked with the USAID logo. 4. The Cooperative Agreement requires a cost-share of $4,511,465, of which $3,335,416 has been reported to date by JSI. 5. No other problems or issues related to the provisions of the Cooperative Agreement were identified. VII. Conclusions and Recommendations From looking at the findings of this evaluation, it is clear that NFHP II has been a highly successful program with substantial accomplishment in support of the Government of Nepal and USAID’s health sector objectives. NFHP II benefited from the lessons learned and relationships built during the preceding NFHP I program. Apart from some specific recommendations based on the findings outlined in Sections IV, V, and VI, the principal challenge for NFHP II in its final year is ensuring the institutionalization and sustainability 35 of the technical approaches and systems supported in some cases since 2002 and for others since late 2007. The evaluation team attempted to provide recommendations that are practical and actionable during the final year of NFHP II and also some that have broader implications for USAID’s work in the future, described in Section VIII. A. Technical and Programmatic Recommendations Component One: Systems, Policy and Leadership 1) Use the TSVs and the TSV forms to advise the Department of Health Services on the modification and improvement of the MoHP’s Integrated Supervision Checklist and system. 2) Place increased emphasis on capturing the lessons learned and experience from the pilot programs through technical briefs and other mechanisms to inform national policy on service delivery approaches and tools. 3) Begin to selectively pull back on the direct operational role of district level NFHP II staff to test whether the DHO program managers can continue to manage without NFHP II inputs. 4) Work with the District QAWG to think beyond just infection control as a mechanism to improve quality of services at the facility level. At the district level, discuss methods for assessing compliance to standards of care for each program area. 5) Consider ways to further improve and institutionalize the public health analytic course as a means for strengthening HMIS data analysis and use at the district level. 6) Review and revise procurement and storage of oxytocin through the Nepal system. National level storekeepers and procurement in-charge must address the issues with oxytocin. NFHP II (or Deliver) should provide technical expertise and guidance. Component Two: Health Service Delivery Family Planning 1) Continue the successful FP program activities that are meeting the government’s needs and are addressing the identified needs, gaps found in Nepal, especially among special population e.g. migrant workers, unreached populations. 2) Increase focus and activities on healthy timing and spacing of pregnancies: to include delayed marriage, spacing of at least 24 months and education of women/ men and providers on the dangers of pregnancy over 35 – with its enormous increase in risk of maternal death. 3) Continue the strong emphasis and activities to reach the rural, remote and marginalized populations but consider strengthening messages and services for adolescent friendly services, wives of migrant laborers and the needs of urban populations in NFHP II’s last year. 4) Develop a postpartum FP strategy for the district facilities where birthing centers are rapidly expanding. This should include a stronger focus on exclusive breastfeeding counseling and use of LAM as well as making sure that ANMs in those sites are trained in IUCD and implant insertions, particularly for post￾abortion clients. 36 5) A series of focus groups with women and with providers or a small study is recommended to determine how best to increase privacy for individual clients in facilities, particularly for FP. It will likely be important to collect data from the various ethnic groups and geographical areas to ensure that the findings are representative. 6) Given the popularity of depo and the need to reach remote, rural and marginalized populations, it is recommended that all VHWs and MCHWs receive training and support to provide depo to their populations. Training should be given first to those in the most remote and difficult geographical areas. 7) Continue to institutionalize FP training programs with the government, as possible and pursue strengthening both national and district level training capacities. 8) Continue research to better understand issues related to the “plateau” of FP, including migrants, decreasing VSC services and other important topics in FP. Maternal Health 1) FCHVs, with the HFOMC, need to participate in discussions and decisions on any requests for additions to the FCHV scope of work, e.g. additional tasks. The risk/benefit ratio of adding more to the effective and motivated FCHV should be determined. 2) Continue to support national scale-up of misoprostol, particularly to the remote areas, recognizing the procurement issue. Continue to work with global partners to address the difficulty in procuring the drug. UNFPA has agreed to purchase it for Liberia so discussions might be fruitful with them. 3) FCHVs need to share and promote the message that women older than 35 and women having their first baby should be strongly encouraged to have a facility birth. Health workers, stakeholders, women and communities need to be provided this data. Messages need to be created and used to get this message to communities. (See data on this topic in facility-based services.) 4) Provide clinical supervision for the rapidly expanding cadre of ANMs and birth attendants. Two suggested possibilities include: (1) add a clinical supervisor to the DHO or use the Public Health Nurse to supervise the ANMs, (2) identify more facilities that have a high volume of births as sites for all ANMs to visit once or twice a year for skills practice and drills on rarely used skills (resuscitation, use of magnesium sulfate). 5) Review and revise the Infection Prevention and Waste Management standards and guidelines to ensure that development and management of the sites address the above-mentioned issues. 6) Review criteria for establishment of a birthing center. This guidance must be shared and enforced at all levels to ensure quality of care at all birthing centers. 7) Additional review, practice sessions, guidelines written and disseminated, job aids for the walls are needed to improve infection prevention practices on making chlorine solution and disinfecting instruments. Child and Newborn Health 1) More focused attention should be given to the fetus during labor and delivery to identify complications and provide prompt referrals and appropriate care at the referral sites. FCHVs need additional content in refreshers to understand this issue and how they can assist. ANMs and other birth attendants must use 37 partographs routinely and correctly. Stillbirths and early neonatal deaths should be reviewed to determine what could be done better at future deliveries. 2) Following the review of the CB-NCP pilot experience, work with the Child Health Division to develop a more life-cycle approach to defining the program of work for FCHVs. Revise the FCHV record keeping forms to reflect this new integrated program of work. 3) New FCHV and refresher training should emphasize recognition of under nutrition, especially for children under 24 months of age and increasing the amount of IYCF counseling. Help FCHVs to monitor children at risk because of under nutrition to ensure that they are gaining weight and that they receive the full range of other services. 4) Strengthen FCHV counseling skills. Component Three: Community and Household 1) Expand use of Learning Circles by FCHVs as a way to reenergize Mother’s Groups and expand their influence to larger numbers of women in the communities where NFHP II works. 2) Document and disseminate the lessons learned from working with the HFOMCs and the pilot LHGSP program to help inform national policy. 3) Complete the planned activities under the HEAL and GATE programs but USAID/Nepal should find another mechanisms to fund these literacy programs in the future to ensure sustainability. B. Program Management Recommendations 1) Work aggressively at the district level to encourage DHO staff to undertake supervision visits more regularly, focusing on facilities that need additional attention based on analysis of HMIS data and Integrated Supervision Checklist forms. 2) As feasible, USAID should continue to increase its funding of some activities through the Redbook to ensure MoHP management responsibility for routine activities 3) NFHP II should strengthen communications with internal and external partners during the final year to share NFHP II experience and lessons learned. 4) NFHP II should share its full annual work plan (not just the supplemental) with the DoHS and seek inputs. 5) The final report for NFHP II should analyze and synthesize the accomplishments, as well as the challenges and constraints, to help all partners benefit from the lessons. VIII. Considerations/Recommendations for USAID Future Programs 1. Family Planning: USAID/Nepal’s future support to the MoHP for family planning (along with other EDPs) should emphasize repositioning FP to focus on healthy timing and spacing of pregnancies (HTSP) by expanding the availability of long term methods (IUCDs and implants) especially postpartum and post abortion; spacing births at least 24 months; reaching adolescents and youth with messages about delaying marriage and childbearing; and discouraging births among women over 35. Efforts to ensure a full range of FP services in hard to reach rural areas and among marginalized populations is critical. 38 2. Maternal and newborn care: NFHP II has contributed to a highly successful and expanding program of community level maternal and newborn care. Involvement with community level MNH should continue into the future to ensure that a sensible and sustainable program of work for FCHVs is agreed to and supported by government and EDPs. The follow-on program should also focus on ways to ensure quality obstetric and newborn health services at the peripheral health facilities (sub-HPs and HPs) where birthing centers are expanding rapidly. 3. Logistics: USAID/Nepal should continue to support logistics management in the future but focus on the newer challenges facing LMD such as integrating the HIV/AIDS drug distribution system including supplying ARV drugs to patients using a unique patient identifying number to ensure an uninterrupted supply to each patient needing the drugs. This can be done in collaboration with USAID/Nepal’s new HIV/AIDS program (Saath￾Saath). Continuation of support to the LMIS will also be needed. The follow-on program should provide for technical assistance for problem solving, monitoring and addressing unanticipated problems to protect the investment but play a less operational role where possible. Specific indicators should be developed to assess progress with this as well as other health system strengthening activities. 4. Quality Improvement: NFHP II played a key role to establish some components of key systems strengthening activities at the national and district levels such a service quality improvement. Additional work is needed to transform NFHP II district level experience and national standards of care into a more comprehensive national QI program. 5. Management and Supervision Capacity Development: USAID/Nepal’s follow-on program needs to continue to work on management capacity development at the district level but move beyond skills development to focus more on “structures, systems and roles”15. The Integrated Supervision checklist and systems of supervision especially designed for remote rural and inaccessible areas need to be further developed, institutionalized and linked with the QI program, based on practical field experience. 6. Health Governance: USAID/Nepal should remain significantly involved in the future in strengthening the capacity of local government and district health offices to expand health facility management committees and help expand the national pilots for strengthening local management of health services, once there is greater clarity on ‘federal’ system. Future work should be guided by lessons learned from NFHP II’s involvement in the pilot efforts in the two districts. 7. Health Metrics: The USAID/Nepal follow-on program should invest more heavily in building capacity for using evidence to plan and manage government health services. Emphasis should be on using HMIS data to identify geographic areas and facilities within districts that are under-performing to focus additional resources and oversight (including EPI, FP, MNCH, nutrition, etc.) Strengthening the capacity within the MoHP to analyze data and utilize research findings as well as survey research should be included. 8. Training: The NHSSP will be taking on the broader human resources reform issues with the MoHP. USAID/Nepal, working with other partners, could take on strengthening 15 Potter and Brough capacity pyramid 39 the pre-service training of ANMs. More of the routine, in-service training activities previously funded by NFHP II should be gradually shifted to Redbook support as feasible. 1 Annex 1: List of Documents Reviewed NFHP II Documents Annual Work Plan – January 2009 to July 2010 Annual Work Plan – July 2010 to June 2011 Annual Work Plan (draft) – July 2011 to June 2012 Cooperative Agreement between USAID and JSI, dated December 19, 2007 District Specific Activities Matrix (2008-2011) “Family Planning, Maternal, Newborn and Child Health Situation in Rural Nepal, A Mid￾Term Survey for NFHP II”, New ERA, March 2010, Monitoring and Evaluation Plan, NFHP II, dated November, 2007 NFHP II 360 Client Satisfaction Report, not dated, Hurdec Semiannual Report Supplement: April 2010 – September 2011 Semiannual Report Supplement: October 2010 – March 2011 Summary program description and various program fliers for NFHP II Multiple NFHP II power point presentations on various program elements Journal Articles “Community-based stillbirth rates and risk factors in rural Sarlahi, Nepal”, Anne C. Lee, Luke C. Mullany, James M. Tielsch, Joanne Katz, Subarna K. Khatry, Steven C. LeClerq, Ramesh K. Adhikari, Gary L. Darmstadt, International Journal of Gynecology and Obstetrics 113 (2011) 199–204. “Expanding uterotonic protection following childbirth through community-based distribution of misoprostol: Operations research study in Nepal”, International Journal of Gynecology and Obstetrics, Swaraj Rajbhandari a, Stephen Hodgins b, , Harshad Sanghvi c, Robert McPherson d, Yasho V. Pradhan e, Abdullah H. Baqu, 108 (2010) 282–288. “Lessons from the field: From research to national expansion: 20 years’ experience of community-based management of childhood pneumonia in Nepal”, P Dawson, YV Pradhan, R Houston, S Karki,D Poudela & S Hodgins, Bulletin of the World Health Organization, May 2008, 86 (5). “Should we care what policy makers think? A response to Maes, Kohrt and Closser”, Claire Glenton, Inger B. Scheel, Sabina Pradhan, Simon Lewin, Stephen Hodgins, Social Science & Medicine 71 (2010) 1379e1380. “Testing a scalable community-based approach to improve maternal and neonatal health in rural Nepal”, S Hodgins, R McPherson, BK Suvedi, RB Shrestha, RC Silwal, B Ban, S Neupane, and AH Baqui, Journal of Perinatology (2009), 1 – 8. “The female community health volunteer programme in Nepal: Decision makers’ perceptions of volunteerism, payment and other incentives”, Claire Glenton, Inger 2 B. Scheel, Sabina Pradhan, Simon Lewin, Stephen Hodgins, Vijaya Shrestha, Social Science & Medicine xxx (2010) 1e8. Other Documents “Best Practices at Scale in Home, Communities and Facilities: Five Year Action Plan for Family Planning, Maternal, Neonatal and Child Health, and Nutrition”, March 2011- September 2015. Data Quality Assessment, Summary Report, USAID, 2011 Evaluation of DFID support to the NHSP-1: An assessment of the maternal mortality decline and SSMP, Nepal, Ipac and University of Aberdeen, 2010 “Measuring the Quality of Rural Based Government Mid-Level Health Care Workers: A Clinical Skills Assessment”, Nick Simons Institute, August 2007. Maternal Mortality and Morbidity Study 2008-2009, Family Health Division, MOHP, May 2010. Ministry of Health and Population Annual Report 2009-2010 Nepal Global Health Initiative Strategy, November 2010. Nepal Health Sector Programme (NHSP I) Implementation Plan 2004-2009 Nepal Health Sector Programme (NHSP II) Implementation Plan 2010-2015 “Post Training Follow-up for Skilled Birth Attendants: Review of Implementation Experiences: Nepal Ministry of Health and Population, September 2009 “Repositioning Family Planning: Strategic Review of Nepal National Family Planning Program”, MOHP, USAID, UNFPA, 2006. Work Plan for Safe Motherhood and Neonatal Health, (Combined Redbook and Supplemental Work Plan) USAID and MOHP, Fiscal Year 2010-2011 2006 Nepal Demographic and Health Survey 1 Annex 2: List of Stakeholders Interviewed Ministry of Health and Population Dr. YV Pradhan, Director General, Department of Health Services Dr. Baburam Marasini, Chief Health Sector Reform Unit Dr. Shyam Raj Upreti, Director, Child Health Division Dr. Naresh Pratap KC, Director, Family Health Division Mr. Arjun Bdr Singh, Director, National Health Training Center Mr. Badri Bahardur Khadka, NHEICC Dr. Bikash Lamichhane, Director, Logistics and Management Division Mr. Paban Ghimire, Chief, HMIS Section, Management Division Ms. Mangala Manandhar, Family Health Division Mr. Rajkumar Pokharel, Nutrition, Child Health Division Mr. Bhanu Yengden, LMD DPHOs and DHO staff of Surkhet, Dailekh, Sindhuli, Sarlahi, Siraha UN Agencies and Other Donors Dr. Amit Bhandari, Health Advisor, DFID Mr. Jhabhindra Bhandari, Health Systems Strengthening Officer, UNFPA Mr. Dinesh Bista, Field Officer, UNDP Ms. Kristna Castell, UNFPA Dr. Susanne Grimm, Deputy Programme Manager, GIZ Mr. Lok Nath Kandel, Programme Officer, UNDP Ms. Latika Maskey, AusAID Ms. Pushpa Lata Pandey, Adolescent Reproductive Health Coordinator, GFA/GIZ Dr. Pankaj Mehta, Chief, Health and Nutrition, UNICEF Mr. Anju Pun, Programme Officer, UNDP Dr. Geeta Rana, UNFPA Mr. Dep Narayan Sapkoka, Programme Officer, UNDP Dr. Nastu Sharma, AusAID Dr. Bert A. Voetberg, World Bank Dr. Sudhir Khanal, UNICEF Dr Kishori Mahat, WHO Donor Supported Projects Dr. Nancy Gerein, International Lead, NHSSP Dr. Ganga Shakya, Maternal Health Specialist, NHSSP Dr. Maureen Dariang, NHSSP Mr. Peter Oyloe, COP, GGMS/AED Mr KB Rayamaji, COP, GGMS/CRS Company Dr. Satish Pandey, COP, ASHA Project/FHI Ms Dale Davis, DCOP, ASHA Project/FHI Mr Deepak Dhungel, ASHA Project/FHI 2 Other Partners Mr. Chandra Rai, Health Rights International Mr. Bhagawan Das Shrestha, Plan International Ms. Nirmala Sharma, CARE Nepal Ms Pooja Pandey, Helen Keller International USAID/Nepal Naramaya Limbu, HFP Cliff Lubitz, former HFP Linda Kentro, HFP Pradeep Neupane, CON Anne Peniston, HFP DP Raman, former AOTR Rajeeb Shakya, OC NFHP II Team Bharat Ban Rajendra Bhadra Dr. Janardan Lamichhane Gagan Gurung Dr. Steve Hodgins, former COP Dr. Robin Houston Dilip Poudel Sabina Pradhan Jaganath Sharma Ashoke Shrestha Dirgha Raj Shrestha Shyam Shrestha Ram Silwal Dhurba Thapa Bishwa Nath Poudyal (Nepalgunj regional office) Rajendra Chaudhard, Field Officer, Dailekh District Internal NFHP II Partners Mr. Upendra Aryal, Equal Access Dr. Rajendra Prasad Bhadra, Jhpiego Mr. Gagan Gurung, Save the Children Dr. Rajendra Gurung, FPAN Mr. Brian Hunter, Country Director, Save the Children Ms. Ramrajya Joshi, Program Director, CEDPA Mr. Udaya Manandhar, Deputy Director, Save the Children Mr. Ty Prasad Pant, World Education Mr. Pushparaj Paudel, Nepal Red Cross Society Mr. Om Rajbhandari, MASS Mr. Santosh Ghimire, BBC/WST 3 Mr. Dirgha Raj Shrestha, EngenderHealth Dr. Mahendra Pd Shrestha, Deputy Executive Director, NFCC Ms. Shanta Laxmi Shrestha, Associate Country Director, World Education Ms. Stephanie Suhowatsky, Jhpeigo Dr. Navin NP Thapa, Director, FPAN Mr. Anu Upaddhayay, Program Manager, Equal Access Dr. Neena Khadka, Save the Children 1 Annex 3 Evaluation of Nepal Family Health Program (NFHP) II 1. PURPOSE This is a Statement of Work (SOW) for an evaluation of USAID/Nepal’s Nepal Family Health Program (NFHP) II. The evaluation will focus on best practices and lessons learned in advancing survival and quality of life of Nepali mothers and children through the implementation of NFHP II. The results of this evaluation will inform USAID/Nepal Health and Family Planning Office of NFHP II’s successes and lessons learned and provide the basis for the needs and direction of the next iteration of the USAID/Nepal FPRH/MNCH program. 2. BACKGROUND Health in Nepal Nepal continues to struggle to make progress towards democracy, peace and prosperity after the decade-long conflict that nominally ended in 2006. While politically fragile, Nepal is making progress towards improving the health of its citizens and is on-track to achieve MDG 4 and 5, but it is a poor, agriculturally based country with an increasing population size and limited arable land. A third of the country consists of some of the most rugged mountainous areas on earth that make health care delivery extremely difficult. There are hundreds of caste and ethnic groups, some of which have been traditionally excluded from health and social services by design or by default for decades. Mortality and morbidity rates among women and children are alarmingly high. Acute preventable childhood diseases, complications of childbirth, nutritional disorders and endemic diseases such as tuberculosis, sexually-transmitted infections, rabies, and water, food and vector-borne diseases are prevalent at high rates. Such conditions are associated with pervasive poverty, low education and literacy rates, low levels of hygiene and sanitation, poor access to safe drinking water, formidable terrain, geographic isolation and difficult communications. These problems are further exacerbated by under-utilization of resources; shortages of adequately trained personnel; underdeveloped infrastructure; and weak public sector management. NFHP II In December 2007, USAID/Nepal awarded Cooperative Agreement No. 367-A-00-08- 00001-00 designating John Snow, Inc. (JSI) as the prime award recipient for NFHP II. The period of performance is January 2008 until September 2012. NFHP II provides ongoing support to system strengthening within the MOHP to ensure high quality health services at community and health facility levels. This includes technical and financial assistance (“on-budget” and supplemental) to national programs including the 50,000 Female Community Health Volunteers (FCHV), Vitamin A and De￾worming Supplementation, Community-Based Integrated Management of Childhood Illness (CB-IMCI), and Family Planning and Safe Motherhood programs. In addition, USAID works with the GON through NFHP II to test scalable program models for implementation of safe motherhood and newborn care at the household level including preventing post-partum hemorrhage, infection prevention and treatment, low-birth weight 2 care, vitamin A dosing, and use of chlorhexadine applied to the newborn umbilical stump to prevent sepsis. All technical assistance delivered through non-governmental organizations is designed to support GON-led programs. NFHP II’s goal is to improve the delivery and use of basic public sector family planning, maternal, newborn, child health, and literacy/life skills services, in a manner that builds local capacity and encourages stakeholder collaboration. Working with government and NGO partners at national and district levels, NFHP II works to improve policy and strengthen leadership and management capacity; improve service delivery; spearhead innovative approaches in community-based maternal and neonatal care, including nutrition; and increase community participation in health decisions and activities. Building on a previous, five-year program, following are the planned outcomes of NFHP II:  Strengthen GON systems, policy, and leadership;  Enhance public health service delivery;  Increase access to and utilization of health services especially by marginalized populations;  Increase community participation in health service management; and  Advance global best practices in family planning, maternal and child health services (FP/MNCH). Other USAID projects in FP/RH and MNCH NFHP II works closely with other USAID funded FP/MNCH programs and provides technical support to the activities implemented by those agencies, such as Child Survival Projects implemented by Plan, CARE, Helen Keller International (HKI), HealthRight International (HRI); GGMS implemented by Nepal Contraceptive Retail Sales (CRS) Company and AED; and MCHIP implemented by Jhpiego. Local Innovation for Better Outcomes for Neonates (LIBON) through Plan The LIBON project is part of the four-year (September 2007 – September 2011) Community-Based Neonatal Care Program (CB-NCP) pilot that USAID and the Nepal MOHP are sponsoring in ten Districts in Nepal; Plan Nepal is implementing this pilot in two Districts—Sunsari and Parsa; and is conducting a “learning lab” in a third District, Bara. Plan Nepal implemented two consecutive USAID Child Survival projects in Bara District1; Bara is now monitored for sustainability after hand-over to the MOHP, and does not implement the CB-NCP pilot components. In the three Districts it serves, LIBON targets women of reproductive age and children under five, focusing specifically on pregnant and post partum women and neonates. Community Responsive Antenatal Delivery and Life Essential (CRADLE) Support for Mothers and Newborns through CARE CARE Nepal implemented the four year CRADLE Support (MANASHI) Project (October 2007 – September 2011), in partnership with Ministry of Health and Population (MOHP), for improving health of mothers and newborns in Doti and Kailali districts of Far West region. The project supported MOHP in the piloting, implementation and strengthening of Community-Based Newborn Care Program (CB-NCP) and Birth Preparedness Package (BPP) in the program districts. This also has supported follow-up of CB-IMCI and 3 integration of HIV and AIDS prevention intervention. The goal of the project was “sustained improvements in maternal & neonatal (MN) health in the districts of Doti and Kailali”. Action Against Malnutrition through Agriculture (AAMA) through HKI The AAMA program, an existing four-year program, which began in October 2008, integrates agriculture and health interventions for addressing malnutrition among children aged 0 – 23 months and women of reproductive age as well as household food insecurity in targeted districts. USAID/Nepal funded the expansion of this project to Bajura district and a new program component; enhancing governance capacity in Baitadi, Kailali, Kanchanpur and Bajura districts to sustain and strengthen intersectoral activities in health, nutrition, and agriculture. Partnership for Maternal and Newborn Health (PMNH) through HRI The four-year PMNH project aims to build an integrated continuum of maternal and neonatal care (MNC) from the household level throughout the health system. To achieve this, the project increases the quality, access, availability, demand, knowledge, and enabling environment for MNC services and infrastructure in the community and health system. PMNH is based in Kapilvastu and Arghakhachi districts of Nepal’s Western Development Region. Kapilvastu is situated in the southern terai (plains) region, and Arghakhachi is located in the central hills. The primary beneficiary population includes neonates and women of reproductive age; however, children under five, Female Community Health Volunteers (FCHVs), Health Facility Operation and Management Committees (HFOMCs), and facility-based health workers will also receive direct benefit from the project. Ghar Ghar Maa Swasthya (GGMS) through CRS Company and AED USAID/Nepal’s social marketing program, the Ghar Ghar Maa Swasthya program, is comprised of two components. A social marketing implementation component is operated through the CRS Company focused on achieving self-sustainability for performance of high quality social marketing activities in Nepal. The technical services to assist CRS in achieving self-sustainability and to provide behavior change communication expertise for both CRS and other USAID projects are provided through AED. GGMS assists the GON in expanding the depth, reach, and impact of the private sector in social marketing, by providing a low-cost supply of maternal/child health, family planning, and HIV-prevention products and services. GGMS focuses program efforts on underserved and most-at-risk populations, scaling up promising rural and community￾based marketing initiatives, and engaging non-governmental organizations and commercial distributors to increase product accessibility in hard-to-reach areas. Maternal and Child Health Integrated Project (MCHIP) through Jhpiego MCHIP in Nepal builds on 10 years of maternal and newborn health technical assistance supported by USAID to the Government of Nepal to bring evidence-based practices to scale. Having successfully developed strategies to address the leading cause of maternal mortality—postpartum hemorrhage—in the past decade, the Government of Nepal is now focused on addressing pre-eclampsia/eclampsia (PE/E). MCHIP employs a three-pronged strategy for preventing, detecting and managing PE/E: prevention, screening and diagnosis, and treatment and management. MCHIP is partnering with the USAID-funded bilateral Nepal Family Health Program II (NFHP II) under the leadership of the Family Health Division (FHD) of the Ministry of 4 Health and Population (MOHP) to test the acceptability of calcium in two forms (tablets and powder) among pregnant women for three months in two village development committees (VDCs) of Banke district in southwestern Nepal. Based on this study, FHD plans to scale up the findings through community-based volunteers in several districts. MCHIP with NFHP II will implement calcium supplementation district-wide in one district and other districts supported by different development partners. 3. EVALUATION WORK TO BE PERFORMED AND METHODOLOGY The overall objective of this evaluation is to assess the effectiveness of NFHP II, document how and if NFHP II is making a difference in the health status of the people of Nepal with regard to FP/RH and MNCH, document best practices, and recommend future longer term FP/RH and MNCH programming directions for USAID/Nepal. In addition to illustrative questions below, the consultant evaluation team will lead a group exercise using the Lives Saved Tool (LiST) to estimate the level of impact of NFHP II. The evaluation team should consider the following illustrative questions: 1. Overall result areas a. Describe NFHP II’s progress in achieving identified results. In what areas is NFHP II on track or not to meet targets? b. Are NFHP II’s programs being deployed in the areas of greatest need? c. What are the major activities and achievements with regard to health systems strengthening, service delivery approaches, improving quality of care and improving social inclusion in health? What are these specific result areas, and what illustrative measurable indicators would we use to monitor and evaluate impact? d. Are there new, emerging result areas that USAID should consider adding in future programs? If so, what is USAID’s comparative advantage in these result areas, and with what other organizations would USAID collaborate to address them? e. How are NFHP II’s achievements perceived and valued by GON and other stakeholders? 2. Planning, implementation, monitoring and reporting f. How is NFHP II working with regional directorates, district health offices (DHOs), and local health facilities (HFs) to plan and implement activities? Describe NFHP II’s level of technical support visits to local HFs, health workers, and volunteers; i.e. is the level of support adequate, well received, and linked to DHOs. g. Are the monitoring and evaluation framework and indicators relevant to the program objectives and in-line with USAID standard indicators and MOHP Nepal Health Sector Plans? What specific recommendations are there to improve them? h. Are all of the annual NFHP II workplans, including the current one, relevant to the approved NFHP II program description? Are workplan activities sufficiently reported in the semi-annual and annual technical reports? Are key activities missing or under-reported? Is there sufficient monitoring and reporting of compliance with USG regulations concerning family planning? 5 i. Are NFHP II’s monitoring systems sufficiently robust to adequately monitor the quality of activities conducted by sub-recipients and their compliance with USG regulations? How could they be strengthened without compromising the reach of the program? 3. Management systems j. What are the strengths and weaknesses in the project management structure and systems? What could be done to make them stronger and build capacity of Nepali staff, both within NFHP II and sub-grantee NGOs, to manage technical and administrative functions? k. Are the structure and management systems of the technical units and field offices relevant to the overall program objectives and results? How could they be improved? l. Does the current management structure allow for constructive relationships with mutual benefit between consortium members? What are the advantages of working in a consortium? What are the challenges? m. Is the management of USAID assets strong and compliant with USG regulations? Is there adequate understanding and compliance with branding and marking regulations? n. In terms of financial management, are internal controls strong, transparent and accountable? o. Is the management and oversight of awards to sub-recipients strong, accountable and compliant with USAID regulations? Is the management of sub￾recipients strong, accountable and effective? The evaluators will use a range of methods to collect and analyze information related to the evaluation objectives and questions to be answered, including but not limited to: Document Review The evaluation team will review background documents (preliminary list provided in Annex 1). Documents to be reviewed include NFHP II project documents: work plans; monitoring and evaluation framework and plan; semi-annual and annual technical reports; and other NFHP II related technical documents including NFHP Mid Term Survey, Semi Annual Reports, Technical Briefs, as well as other country specific documents as listed, i.e. national strategies, national action plans, and country reports (Annex 1). Team Planning Meeting It is anticipated that the evaluation team leader will facilitate and conduct a one-day team planning meeting at the beginning of the evaluation process in Nepal, and before starting the in-country portion of the evaluation. USAID/Nepal’s focal person and other USAID staff will participate in the team planning meeting. The agenda may include the following items:  Clarify team members’ roles and responsibilities;  Establish a team atmosphere, share individual working styles, and agree on procedures for resolving differences of opinion;  Finalize a work plan for the evaluation;  Review and develop final evaluation questions;  Review and finalize the assignment timeline and share with USAID;  Finalize data collection plans and tools; 6  Review and clarify any logistical and administrative procedures for the assignment;  Develop a preliminary draft outline of the team’s report; and  Assign drafting responsibilities for the final report. USAID/Nepal will review and approve the documents noted above before further work on the evaluation. Key Informant Interviews and Site Visits The evaluation team will conduct key informant interviews with (preliminary list or key informants provided in Annex 2) selected NFHP II staff, USAID NFHP II program managers, and key stakeholders including donors, government counterparts, selected implementing agencies, networks, other program beneficiaries and stakeholders. 4. TEAM COMPOSITION, SKILLS AND LEVEL OF EFFORT The evaluation team of experts, selected by USAID/Nepal, is led by a Team Leader with strong experience working with host-government led sector plans with multiple stakeholders contributing to coordinated planning processes and a variety of financing arrangements including SWAps. Team members consist of USAID staff from Nepal and Washington having expertise in the fields of maternal, newborn, and child health and family planning and reproductive health, health system in Nepal for the design part. However, up to three Nepali experts (child health, health systems, family planning) will be involved in the evaluation part only. The evaluation work is anticipated to begin o/a June 17, 2011, and will be completed by July 29, 2011. Timing and Level of Effort (LOE): Background reading 1 day (June 17) Travel to Nepal 2 days (June 18-19) Team planning meeting 1 day (June 20) NFHP II briefing sessions 1 day (June 21) Meetings/interviews (NFHP II) 5 days (June 22-25 and June 27) Travel to field sites (NFHP II) 4 days (June 28-July 1) Evaluation team meeting to synthesize findings 1 day (July 2) (Other evaluation team consultants will write their assigned sections of the report during the week of July 4) Meetings/consultations 11 days (July 4-15) (design work) 7 Writing (July 16) and return travel to US* 2 days (July 16-17) Compile/draft NFHP II evaluation report 4 days (July 18-21) and submit July 21 pm Revise Evaluation Report (based on USAID 2 days (July 27-28) inputs -1 week review period) 5. LOGISTICS The logistic manager will make all necessary arrangements of logistics in conjunction with USAID/Nepal NFHP II point person in Nepal. 6. MEETINGS AND BRIEFINGS The team leader will work closely with the USAID/Nepal NFHP II point person and other USAID staff as necessary. Entry and exit and regular update meetings of the team leader with USAID/Nepal will be arranged by the NFHP II AOTR/contact person. 7. DELIVERABLES  A completed report outlining the findings of the evaluation of NFHP II and recommendations for possible future follow-on program directions, approaches, and activities for USAID/Nepal. 9. MISSION CONTACT PEOPLE Deepak Paudel Program Management Specialist, AOTR NFHP II Office of Health and Family Planning. USAID/Nepal Tel: 977-1-400-7200 Ext. 4532 dpaudel@usaid.gov Naramaya Limbu Division Chief, FP/MNCH/Nutrition Office of Health and Family Planning. USAID/Nepal Tel: 977-1-400-7200 Ext. 4147 nlimbu@usaid.gov Han Kang, Deputy Director Office of Health and Family Planning. USAID/Nepal Tel: 977-1-400-7200 Ext. 4456 hkang@usaid.gov Annex 4 Table 1 –Part A NFHP-II Operational Plan (OP) Indicators (14) – FY 2009/10 No Indicator Definition Data Source Reporting Frequency Baseline 2063/64 (2006/07) Year 1 2064/65 (2007/08) Year 2 2065/66 (2008/09) Year 3 2066/67 (2009/10) Year 4 2067/68 (2010/11) Year 5 2068/69 (2011/12) Expected Actual Expected Actual Expected Actual Expected Actual Expected Actual Family Planning 1 Annual protection against pregnancy afforded by contraceptives distributed. (National)1 HMIS Annual 1,564,819 (National) 1,611,763 1,477,174 1,656,895 1,540,2202 1,704944 1,536,728 1,754,388 1,805,265 (National) 2 Annual protection against pregnancy afforded by contraceptives2 distributed. (CPDs) HMIS Annual 746,960 (20 CPDs) 768,622 695,889 (20 CPDs) 790,912 733,489 813,844 733,695 837,450 861,736 (CPDs) 3 Number of people trained in FP/RH with USG funds. (National) TIMS & NFHP records Annual 596 (F) 213 (M) 104 (F) 84 (M) 95 (F) 79 (M) 156 (F) 208 (M) 291 (F) 255 (M) 200 (F) 272 (M) 269 (F) 236(M) 110(F) 152 (M) 17 (F) 25 (M) 4 Number of USG-assisted service delivery points (PHCs, HPs, SHPs) experiencing stock-outs of any contraceptives commodity. (National) LMIS Annual 1,444 1,324 1,222 1,204 1,053 1,083 916 (Jul ‘09 - Jul ’10) 963 802 5 Number of USG-assisted service delivery points (PHCs, HPs, SHPs) experiencing stock-outs (average quarterly stockouts of number of service delivery points) of any contraceptives commodity4. (National) LMIS Annual 270 240 (by mistake OP reported 300) 163 210 149 175 135 (Jul ‘09 – Jul ’10) 150 125 FP5 Number of people that have seen or heard a specific USG-supported FP/RH message 1,455,145 1,455,145 8,153,479 Maternal, Neonatal and Child Health 1 CYP is calculated as: 120 Condom = 1 CYP; 15 pills cycles = 1 CYP; 4 doses Depo = 1 CYP; IUCD = 3.5 CYPs; Norplant = 3.5 CYPs; VSC = 10 CYPs. 2 This is from preliminary HMIS report, final figures will be available after completion of Regional Review Meetings towards end of October No Indicator Definition Data Source Reporting Frequency Baseline 2063/64 (2006/07) Year 1 2064/65 (2007/08) Year 2 2065/66 (2008/09) Year 3 2066/67 (2009/10) Year 4 2067/68 (2010/11) Year 5 2068/69 (2011/12) Expected Actual Expected Actual Expected Actual Expected Actual Expected Actual 6 Number of cases of child pneumonia treated with antibiotics by health facility or trained community health workers at national level in USG supported programs. (National) HMIS Annual 439,187 461,000 720,078 484,204 875,567 508,441 850,904 533,835 560,526 7 Number of cases of child pneumonia treated with antibiotics by health facility or trained community health workers in CB-IMCI districts. (CB-IMCI Districts) HMIS Annual 399,196 100,000 567,648 (44 dists)3 356,155 727,975 (60 dists) 875,567 850,904 (all 72 districts) 533,835 560,526 8 Number of newborns receiving antibiotic (cotrim or/and gentamycin) treatment for infection from appropriate health workers through USG supported programs. GON records Annual 734 1,200 1,660 1,395 1,389 1,425 904 (Morang only) 1,456 1,488 9 Number of postpartum visit within 3 days of birth in CB-MNC districts. CB-MNC Reg Annual 28,724 11,000 22,833 20,287 20,342 20,340 15,046 (Shrawan 66 – Asar 67,) 27,384 27,384 10 Number of children (6-59 months) nationwide who received Vitamin A capsule during the preceding round of supplementation. Vitamin A Supplement ation survey and DHS Annual 3,315,661 3,200,000 3,277,533 3,086,431 3,351,623 (Calculated based on Mini Survey, Oct 2008) 3,108,786 3,352,681 (old projection) 2,955,422 (from new HMIS pop projection) (Calculated based on Mini Survey, April 2009) 3,130,690 3,143,213 11 Number of people trained in child health and nutrition through USG-supported health area programs. (National) GON records Annual 10,974 (F) 3,371 (M) 2,000 (F) 1,000 (M) 3,411 (F) 2,933 (M) 1,496 (F) 446 (M) 3,483 (F) 602 (M) 2,295 (F) 425 (M) 3084 (F) 522 (M) (July 09 - July 10) 1,700 (F) 425 (M) 850(F) 425(F) 3 These are CB-IMCI districts, data are collected and submitted up to community level No Indicator Definition Data Source Reporting Frequency Baseline 2063/64 (2006/07) Year 1 2064/65 (2007/08) Year 2 2065/66 (2008/09) Year 3 2066/67 (2009/10) Year 4 2067/68 (2010/11) Year 5 2068/69 (2011/12) Expected Actual Expected Actual Expected Actual Expected Actual Expected Actual 12 Number of people trained in maternal/newborn health through USG￾supported programs. NFHP-II records Annual 2,528 (F) 445 (M) 2,000 (F) 500 (M) 2,815 (F) 377 (M) 1,812 (F) 273 (M) 118 (F) 274 (M) 2,951 (F) 524 (M) 6,311 (F) 1,097 (M) 2,951(F) 524(M) 2,951(F) 524(M) 13 Number of USG-assisted service delivery points (PHCs, HPs, SHPs) experiencing stock-outs of specific tracer drugs. (National) LMIS Annual 2,608 2,487 2,342 2,367 1,935 2,247 1,726 (Jul ‘09 – Jul ’10) 2,126 2,006 14 Number of USG-assisted service delivery points (PHCs, HPs, SHPs) experiencing stock-outs (average quarterly stockouts of number of service delivery points) of specific tracer drugs. (CPDs) LMIS Annual 108 (17 CPDs) 50 (based on 17 CPDs baseline) 37 135 (20 CPDs) 36 (20 CPDs) 118 (20 CPDs) 34 (Jul ‘09 – Jul ’10) 101 84 Table 1 – Part B NFHP-II Program Monitoring Indicators (12) – FY 2009/10 No Indicator Definition Data Source Reporting Frequency Baseline 2063/64 (2006/07) Year 1 2064/65 (2007/08) Year 2 2065/66 (2008/09) Year 3 2066/67 (2009/10) Year 4 2067/68 (2010/11) Year 5 2068/69 (2011/12) Expected Actual Expected Actual Expected Actual Expected Actual Expected Actual Monitoring/Supervision 1 Percentage of PHCs and HPs in CPDs that receive a quarterly supervision visit by D(P)HO staff. NFHP-II TSV records Annual NA 45% 70% 50% 67% (N-213) 55% 67.4% (N-236) 60% 65% 2 Percentage of health institutions in CPDs participating in monthly Ilaka meeting and reviewing monthly monitoring worksheet. Meeting observati on Annual NA 10% 18% 15% 28% (N-1,476) 20% 35% (N=2,139) 25% 30% Logistics 3 Percentage of FCHVs in CPDs who have all key commodities4 available at the time of survey, as appropriate for the programs in their district. (USAID reporting indicator) FCHV Survey Every two years 20% 25% 36.8% NA NA 38% 57% (N=2797) (From TSV data) NA 50% Service Delivery/Utilization/Social Inclusion 4 Percentage of children age 2-59 months with diarrhea who were treated with both ORS and Zinc. (National i.e Zinc intervention districts) DHS Mid￾Term End of project 20% NA NA NA NA NA NA NA 25% 5 Percentage of births that are attended by a SBA (doctor, nurse or ANM) in CPDs. HMIS, NFHP-II Mid-term survey and DHS Annual 12.2% (DHS) 15% (HMIS) 17% 18% (HMIS) 25.8% (Mid-term) 22% 24% (HMIS￾incomplete report) 26% 30% 4 Condom, OC, ORS, cotrimoxazole and Iron folate. No Indicator Definition Data Source Reporting Frequency Baseline 2063/64 (2006/07) Year 1 2064/65 (2007/08) Year 2 2065/66 (2008/09) Year 3 2066/67 (2009/10) Year 4 2067/68 (2010/11) Year 5 2068/69 (2011/12) Expected Actual Expected Actual Expected Actual Expected Actual Expected Actual 6 Percentage of Recently Delivered Women (RDW) protected from PPH. (CBMNC-MSC district) CB-MNC Register Annual 90% >= 75% 96% >= 75% 96% (Total 7,478 PW received MSC tablets) >= 75% >93% (July 09 – June10) Shrawan 66 to Asar 67 (Total >= 75% >= 75% 7 Percentage point increase in current use of contraceptives among newly literate married women of reproductive age (MWRA). Pre-Post Survey (test) Annual NA TBD 15% point increase from pre￾test TBD 15% point increase from pre￾test 53.9% (baseline 38.9 % + 15% increase) 70.8% (HEAL First cycle￾post test CPR) 58.7% TBD Community Support/Participation 8 Percentage of FCHVs who report community level support for their activities in the past one year in CPDs. FCHV Survey Every two years 25% 30% 39% NA NA 40% 77% (N=2,797) (From TSV data) NA 50% 9 Percentage of HFOMCs that conducted meeting with meeting minutes every month in CHFP districts. HFOMC meeting minutes Annual 15% 20% NA 30% 86% (N-370) 35% 86% (N-497) 40% 50% 10 Among the HFOMCs that conducted meeting with meeting minutes, percentage that conducted effective meeting in the last month in focused CHFP VDCs. HFOMC meeting minutes Annual 15% 25% NA 40% 38% (N-317) 50% 55% (N=426) 60% >= 75% 11 Ratio of dalit proportion among HF clients vs dalit proportion among catchment population in focused VDCs of CHFP districts. HMIS Annual NA 0.70 NA 0.75 1.41 >=0.80 1.44 >=0.80 >=0.80 Adolescent Education 12 Percentage of GATE graduates enrolled in formal education. WEI records Annual 40% 40% NA 40% 61.7% (N-590) 40% 72.9% 40% 40% Annex 5: Observations from field visits ‐ facility checklist Indicator Total observed Denominator (facilities visited that should have) Number with at least 4 of the 5 basic registers 10 10 Number of facilities with wall charts, graphs or evidence of use of HMIS data 9 9 Facilities with at least 1 DHO supervision visit last quarter 10 10 Number with running water and soap available and evidence of use 7 10 Number of facilities with toilets for clients 8 10 Number with functioning sterilization equipment 8 9 Number of facilities using appropriate medical waste disposal systems 8 placenta pits; 8 burn pits; 3 incinerators 10 Number using IMCI patient records 10 10 Number of facilities with ANC/ MH cards for services: TT, iron, deworming 8 8 Number of facilities having birthing rooms that have oxytocin and MgSO4 available in birthing room Oxytocin 8 Mg SO4 5 8 8 Number of facilities having birthing centers that are using partographs 8 9 Number of birthing centers that are also inserting IUCDs 3 8 Number of facilities with chlorine‐based disinfectant 6* 10 Number of facilities with staff who have attended SBA training or MNH updates SBA 4 MNH 4 8 8 Number of maternal deaths over last year 2 8 Number of stillbirths or perinatal (# stillbirth or newborn deaths within 1 week) deaths 0‐5** 8 Number of facilities providing short and long‐term methods (condoms, pills, depo, IUCD, implant) 3 10 Number of facilities that are fully stocked with drugs and supplies on observation list 10 10 Number of facilities with majority of equipment on observation list 10 10 * one clinic used phenol and most used Virex but ANMs could not explain how to use it ** all facilities , but one, had stillbirths