Contact No.: AID-614-A-14-00006 August 5, 2015 This publication was produced for review by the United States Agency for International Development. It was prepared by Banyan Global for the USAID Saving Maternity Homes in Ghana, Innovate for Health Project. USAID SAVING MATERNITY HOMES IN GHANA, INNOVATE FOR HEALTH MONITORING AND EVALUATION PLAN YEAR I USAID SAVING MATERNITY HOMES IN GHANA, INNOVATE FOR HEALTH PROJECT MONITORING AND EVALUATION PLAN, YEAR 1, DRAFT V.3 Contract No.: AID-614-A-14-00006 Submitted to: Salamatu Futa AOR USAID Ghana DISCLAIMER The author’s views expressed in this publication do not necessarily reflect the views of the United States Agency for International Development or the United States Government. Monitoring and Evaluation Plan, Year 1 i PROJECT OVERVIEW Name USAID Saving Maternity Homes in Ghana, Innovate for Health Contract Number AID-614-A-14-00006 Start Date September 9, 2014 End Date September 29, 2017 Sub-grantee Ghana Registered Midwives Association (GRMA) Geographic Coverage Brong Ahafo, Central, Eastern, Northern, Volta, Western Reporting Period October 1, 2015 – September 30, 2016 Monitoring and Evaluation Plan, Year 1 Acronyms ACRONYMS BFS Business Formalization Score CDCS Country Development Cooperation Strategy DEA Data Envelopment Analysis DQA Data Quality Assessment FP Family Planning GHS Ghana Health Services GoG Government of Ghana GPS Global Positioning System GRMA Ghana Registered Midwives Association KII Key Informant Interview LOP Life of Project LQAS Lot Quality Assurance Sampling M&E Monitoring and Evaluation MCH Maternal and Child Health MEP Monitoring and Evaluation Plan MNCH Maternal, Newborn and Child Healthcare MOH Ministry of Health NHIA National Health Insurance Authority NHIS National Health Insurance Scheme PMHs Private Maternity Homes PMP Performance Management Plan PSV Peer Support Visits QAT Quality Assessment Tool RH Reproductive Health SMH USAID Saving Maternity Homes in Ghana, Innovate for Health Project USAID US Agency for International Development Monitoring and Evaluation Plan, Year 1 TOC CONTENTS Project Overview...................................................................................................................................................................... i Acronyms.................................................................................................................................................................................. iii 1. Introduction..........................................................................................................................................................................1 1.1 Project Overview..................................................................................................................................................1 1.2 Levels of Project Assistance................................................................................................................................2 1.3 Purposes of this Document................................................................................................................................3 2. Project Objective, Theory of Change and Results Framework...............................................................................5 2.1 Project Objectives.................................................................................................................................................5 2.2 Theory of Change/Development Hypothesis.................................................................................................5 2.3 SMH Project Results Framework ......................................................................................................................8 2.3.2 Results Framework: Summary Diagram..................................................................................................8 2.3.2 Performance Indicators Per Results Level .............................................................................................9 3. Monitoring Plan..................................................................................................................................................................10 3.1 Sources of Data and Data Collection Tools.................................................................................................10 3.2 Analysis and Reporting.......................................................................................................................................11 3.3 Data Quality Assurance.....................................................................................................................................12 3.4 Gender Considerations.....................................................................................................................................13 3.5 Critical Assumptions...........................................................................................................................................14 4. Baseline Assessment Plan................................................................................................................................................16 4. 1. Purpose of the Baseline Assessment..........................................................................................................16 4. 2. Methods............................................................................................................................................................16 4. 3. Data Collection Plan ......................................................................................................................................16 4. 4. Analysis and Reporting ..................................................................................................................................18 5. Operations Research Plan...............................................................................................................................................21 5. 1. Overall Approach ...........................................................................................................................................21 5. 2. Background.......................................................................................................................................................21 5. 3. Research Hypothesis and Questions.........................................................................................................21 5. 4. Methodology ....................................................................................................................................................22 5. 5. Operations Research Results and Dissemination ...................................................................................22 6. Evaluation Plan ...................................................................................................................................................................23 6. 1. Purpose of Evaluation ....................................................................................................................................23 6. 2. Key Evaluation Question...............................................................................................................................23 6. 3. Evaluation Methodology................................................................................................................................23 6. 4. Data Collection And Analysis......................................................................................................................25 Monitoring and Evaluation Plan, Year 1 TOC 6. 5. Anticipated Constraints ................................................................................................................................25 7. MEP Responsibilities.........................................................................................................................................................26 Annex 1: Business Formalization Score (BFS) ................................................................................................................27 Annex 2: Indicators Table....................................................................................................................................................32 Annex 3: Monitoring and Evaluation Workplan: Year 1 ..............................................................................................46 Annex 4: Indicator Reference Sheets................................................................................................................................47 Annex 5: PMH Survey...........................................................................................................................................................71 Monitoring and Evaluation Plan, Year 1 1 1. INTRODUCTION 1.1 PROJECT OVERVIEW The Saving Maternity Homes (SMH) project is a three-year, $1.78 million USAID-funded project under the Innovate for Health Annual Program Statement (APS). The project, implemented by Banyan Global in partnership with the Ghana Registered Midwives Association (GRMA), offers a creative approach to revitalizing the private maternity sector in Ghana by transferring the ownership/management of declining or closed facilities to motivated, often younger midwives, who can make investments that will increase capacity, improve quality and expand important, maternal child health and family planning services in rural areas. Operations research will enable the project to not only share lessons about revitalizing maternity homes but also to test business models that stimulate private sector investment in health in rural areas more broadly, bringing non-traditional partners to address this long-standing development challenge. Ghana is a country that is experiencing rapid economic growth, yet a development gap exists between rural and urban areas, higher and lower income quintiles and in key development indicators that raises concerns about equity and the long term sustainability of Ghana’s growth. Nowhere is this gap more evident than in important health indicators, such as maternal, infant and child mortality, which remain troublesomely high. The Government of Ghana (GoG) has made a major effort to address the development gap and expand equity in access to healthcare services with the launch of the Ghana National Health Insurance Scheme (NHIS). While the rollout of the NHIS has increased access to essential health services, it is not without its challenges, and access in rural areas remains an issue due to the lack of accredited facilities. In the past, private maternity homes (PMHs) were an important provider of maternal child health care, especially in rural areas, but over the past decade, this sector has been in steady decline as midwives have aged and facilities have closed and struggled under the NHIS. The Ghana Registered Midwives Association (GRMA) estimates that in the past five years, 100 private maternity homes have closed, and many more are operating at reduced capacity as midwives reach retirement age without any succession plans. The SMH project aims to revitalize the private maternity sector by creating stronger, more modern maternity homes in rural Ghana run by younger midwives that replace the ageing generation of current owners and operators. The project will: • Build the capacity of private maternity homes in succession and investment planning; • Broker the sale of declining or closed maternity homes to younger midwives, testing different business transfer models, including outright sale, leasing, partnerships, and management contracts; • Leverage financing and investment in the PMHs to improve quality and expand services. The project will bring non-traditional partners to the sector, exploring financing from a variety of sources, including commercial banks, angel investors, investment clubs and crowdfunding. This will be complemented by a small amount of seed funding from the project to reduce risk for local investors; provide younger midwives with additional capital; and ensure that investments are used to improve quality; • Provide business strengthening and clinical capacity building of new owners to ensure success and maximize health outcomes; Monitoring and Evaluation Plan, Year 1 2 • Build the capacity of the GRMA to continue revitalizing private maternity homes in the future; and, • Conduct operations research to provide lessons learned for future expansion to other cadres of health providers in rural areas. 1.2 LEVELS OF PROJECT ASSISTANCE SMH aims to provide four main tiers of intervention to PMHs in the target regions: 1. Tier 1: Peer Support Visits – The lowest level of project assistance is to facilitate only Peer Support Visits (PSVs), provided by GRMA. These visits will enable GRMA to review the facility, identify areas of improvement, and provide recommendations. Facilities at this level may or may not participate in any further capacity building activities or transformation assistance provided by the project. PMHs that receive only PSVs, and do not attend other SMH project trainings or receive transformation assistance, will be regarded as unassisted by the project. Life of project (LOP) target and rationale: The project has set an LOP target of 108 facilities for Tier 1. This target is set equal to the actual achievement as of July, 2015. While the initial list of PMH facilities in the target regions included 170 facilities, many of those facilities were either closed, or the key personnel were not available to receive the PSVs when the SMH/GRMA team arrived. Thus, GRMA was able to deliver PSVs to a total of 108 facilities. 2. Tier 2: Capacity Building Support – At this level of project assistance, PMHs will receive capacity building support in the form of the various SMH project trainings, including the Business Continuity and Succession Planning (BCSP) training, the Clinical Capacity Building (CCB) training, and the Business Strengthening Support (BSS) training. PMHs receiving this support will be regarded as assisted by the project. LOP target and rationale: The project has set an LOP target of 146 facilities for Tier 2. At the time of submission, 146 discrete PMH facilities have attended the BCSP trainings, and all 146 of those facilities also attended the CCB trainings, administered by GRMA. The BSS training has not yet occurred, though the project anticipates that approximately 146 facilities will attend, similar to the other trainings. We believe that all of these facilities will have attended one of the two previous trainings. Thus, the overall target for Tier 2 support is 146 facilities. 3. Tier 3: Transformation Support – This tier of project support is focused on PMHs who submit Expressions of Interest in pursuing a transformation, and are accepted by the project to receive specialized transformation support. At this level of project assistance PMHs will receive all of the assistance included in Tiers 1 and 2, plus individual technical assistance focused on deal structuring, needs analysis, business plan support, and other pre-transformation assistance. PMHs in this tier are also eligible to apply for small grants from the project, and to be linked to external sources of financing. PMHs receiving this level of support will be regarded as assisted by the project. LOP target and rationale: The project has set an LOP target of 33 facilities for Tier 3. In focusing on the four transformation options, the SMH project is presenting PMHs with opportunities that are largely unfamiliar to them. The baseline assessment found that 14% of facilities had a manager who was not also the owner, making a management contract the most common or well-known form of transformation. Only 7.6% of facilities had more than one owner (partnership), while only 1.7% of facilities has been acquired in a sale. Given this low level of exposure to ownership/management transformations at baseline, the project anticipates Monitoring and Evaluation Plan, Year 1 3 substantial effort will be required to sensitize PMHs to the idea of transformation, and even then only a minority percentage of facilities will be interested in seriously exploring transformations as an option. These baseline findings have been reinforced through discussions from project staff at capacity development events that have been conducted to date. Thus, the project believes that 33 PMHs selected for transformation support is a reasonable LOP target. 4. Tier 4: Post-Transformation Support – This tier of project support focuses on facilities that actually achieve an ownership and/or management transformation. The project will provide tailored technical assistance to enhance the likelihood of business and financial sustainability in the new arrangement. PMHs receiving this level of support will be regarded as assisted by the project. LOP target and rationale: The project has set an LOP target of 20 facilities for Tier 4. Extending the rationale for the Tier 3 target, the project believe that while 33 facilities be selected for pre￾transformation technical assistance, not all of these facilities will successfully complete an ownership/management transformation. Given the information from the baseline and the informal learning acquired through trainings to date, the project believes that 20 PMHs is a realistic target for number of facilities that transform, and receive post-transformation support. 1.3 PURPOSES OF THIS DOCUMENT Activities described in the performance management plan (PMP) have three main purposes: 1. To measure the extent to which the Saving Maternity Homes project is meeting its objectives (accountability); 2. To provide Banyan Global, USAID, and project partners with information needed to evaluate the effectiveness of program implementation (project learning); and 3. To provide the project, its partners, and the broader development community with practical information related to future interventions (operations research). This document will serve as the main monitoring and evaluation plan (MEP) to track and report the progress of project implementation. It will also be an important resource to engage the private sector and support the participating providers in performance management. The primary users of monitoring and evaluation learning and written products – for accountability, project learning, and operations research purposes – will be: 1. Project participants and members of participating communities 2. Project partners, including USAID, the GRMA, the Ghana Ministry of Health (MOH), and other government agencies and civil society organizations 3. Banyan Global staff, including headquarters staff and Ghana staff 4. Other organizations interested in private health sector development in Ghana. Additionally, the M&E process will be used to build capacity among partner organizations, primarily GRMA. The Banyan Global M&E team will involve GRMA in a discussion of appropriate intermediate results and indicators and will solicit their input in the development of all data gathering tools. Additionally, Banyan Global will provide input into GRMA’s own on-going data collection processes and methods, with the objective of identifying areas in which these methods could be made more efficient and result in more useful outcomes for GRMA and its membership. SMH will continue to seek additional Monitoring and Evaluation Plan, Year 1 4 ways to include partner organizations in the monitoring, evaluation, and learning activities, as appropriate. The project will perform ongoing validation of the MEP indicators by conducting formal and informal data quality assessments over the life of the project in addition to annual assessments. The project will review the MEP on an annual basis together with the Mission. Monitoring and Evaluation Plan, Year 1 5 2. PROJECT OBJECTIVE, THEORY OF CHANGE AND RESULTS FRAMEWORK 2.1 PROJECT OBJECTIVES The assistance provided by this project will enable revitalized maternity homes to capitalize on the opportunity presented by the NHIS and its revenue stream, while responding to the demand for quality health services in rural areas. Ultimately, this program will increase access, improve efficiencies and expand quality maternal and child healthcare and family planning services in rural areas. This work directly supports the USAID/Ghana Country Development Cooperation Strategy (CDCS) results framework by supporting Development Objective 3: Equitable Improvements in Health Status. Specifically, this work supports IR 3.1: Increased access to integrated health services, and IR 3.3: Strengthened and responsive health systems. Additionally, these improvements support the Government of Ghana’s strategic policy objectives, as well as the Ghana Global Health Initiative’s IR 1.1: Increased access to quality reproductive health and HIV care. 2.2 THEORY OF CHANGE/DEVELOPMENT HYPOTHESIS The SMH project theory of change (see Figure 1) is focused on the operational and management efficiency and quality improvements that are expected to result from ownership transformations, through the arrival of new investors, new management, or both. This theory of change posits that the project can take specific steps to increase the awareness among both private maternity homes and the investor community of the potential benefits of succession planning and ownership and management transformation, as well as to improve owners’ ability to carry out ownership and management transformation by assisting them to cultivate the required knowledge base, skills, and attitude. Similarly, the project believes it can sensitize potential investors and funders regarding the business opportunities available through investing in private maternity homes, leading to market-driven investments in revitalizing these facilities. Toward this end, the project will provide trainings and other sensitization activities to all PMHs in the target states who are interested in participating, and will reach out to potential sources of capital regarding investment opportunities in private maternity homes. The result of these activities will be increased sustainability and expanded operations of PMHs in the rural areas. It is hypothesized that this awareness will result in all of the participating maternity homes having a better sense of the potential benefits of effecting an ownership or management transformation (selling the facility, leasing the facility, bringing on a partner, or bringing on a manager) in order to ensure the future sustainability of the practice. In a subset of these participants, this awareness will lead to a desire to pursue one of these forms of transformation. The project will identify these facilities and assist with the successful completion of their chosen form of ownership/management transformation. Monitoring and Evaluation Plan, Year 1 6 Awareness of PMHs of the benefits of operational efficiency and financial sustainability increased Ownership transformations, including the arrival of new management Operational efficiency of private maternity homes increased and quality improved Number of sustainable PMHs increased and expanded, and access to quality care increased It is also hypothesized that these new owners/managers will then work to improve the operational efficiency of their newly acquired PMHs by enacting facility-wide improvements in the areas of business and financial management, as well as the quality of care. This hypothesis is based on the assumption that these new owners/managers will have purchased or taken over management of the facilities in order to make a profit (in addition to the motivation to provide health services), and will therefore work to increase the efficiency of the facility and improve the quality of the health products and services available in order to ensure the financial success of their investments and efforts. It is further hypothesized that the improvements in operational efficiency and quality of care will result in an overall increase in the number of sustainable PMHs operating in target regions, which will in turn result in increased access to quality health care (MNCH, FP, malaria treatment, and other basic health care services) for the people in these regions. Finally, as operational efficiency and quality of care increase, it is hypothesized that facilities will expand the number of services and amount of care they are able to offer. In order to support the long term sustainability of these project interventions, the SMH project will work directly with the GRMA, building its capacity to deliver this assistance after the project has ended. This assistance will include linkages to sources of funding available in Ghana. The project’s M&E plan will gather data to track the hypothesized cause-and-effect linkages in the SMH theory of change. The plan will enable the project to assess whether project activities are indeed enabling participating PMHs to improve their business and financial practices, as well as to increase the efficiency and the quality of healthcare they are providing. It will also enable the project to assess whether there may be unintended (and likely temporary) decreases in efficiency and quality of care resulting from less experienced midwives or other owners taking over ownership and/or management of maternity homes. Figure 1. SMH Theory of Change To have a clear understanding of the theory of change, there are several key concepts that must be defined: Transformation: Transformation is defined as occurring when; 1) the PMH facility is legally sold to a new owner or investor; 2) the existing owner brings on a partner who has both an ownership stake and an active management role; 3) the existing owner retains ownership, but appoints a manager with a job Monitoring and Evaluation Plan, Year 1 7 description that gives this person final authority over all operational aspects of facility management; or 4) the existing owner retains ownership and leases the facility to a new manager/operator. Business and Financial Sustainability: Levels of business and financial sustainability will be determined through a “business formalization score (BFS),” which is a composite of facility-level performance against key indicators demonstrating the capacity of a PMH to sustain its business operations in the future. This score includes clearly verifiable considerations, such as whether the business is legally incorporated, whether it is in compliance with regulations and licensing requirements, whether it is currently accredited with NHIA, whether the business has written contracts with partners and suppliers, and whether it has a bank account (personal or corporate). The BFS of each facility receiving project assistance will be assessed at the baseline and then again at the endline to measure changes in business and financial sustainability following the intervention. Expansion: A sustained or transformed facility has expanded when following the intervention, a maternity home has added new health services (e.g., neo-natal services, family planning services, malaria, diarrhea, inpatient, etc.), increased capacity (e.g., number of beds, increased staff, additional consultation rooms), increased partnerships (e.g., health franchise with MSI, DKT, IPAS etc.) and, ultimately, increased the number of clients. Monitoring and Evaluation Plan, Year 1 8 USAID DO 3: Equitable Improvements in Health Status IR 3.1: Increased access to integrated health services IR 3.3: Strengthened and responsive health system PROGRAM GOAL: Increase private maternity homes' ability to sustain and grow quality of health services in the underserved areas of Ghana STRATEGIC OBJECTIVE: PMHs sustained, MCH services expanded, and efficiency and quality improved IR 3: SUSTAINABILITY Business and programmatic capacity of PMHs and GRMA increased Sub-IR 3.1: Capacity of PMHs to sustain, increase and improve service delivery strengthened Sub-IR 3.2: Increased capacity of PMHs to manage businesses and offer health services Sub-IR 3.3 Management and programmatic capacity of GRMA increased IR 2: TRANSFORMATION PMHs under new ownership / management transformed and improved/expanded health service delivery Sub-IR 2.2: Funding Provided and Leveraged by the Project Sub-IR 2.1: PMH ownership / management transformations brokered IR 1: PRE￾TRANSFORMATION PMH owners and managers ready for ownership and/or management change Sub-IR 1.1: PMHs interested in and ready for transformation identified Sub-IR 1.2: Capacity of PMHs for ownership / management transformation strengthened 2. 3. SMH PROJECT RESULTS FRAMEWORK 2.3.1 RESULTS FRAMEWORK: SUMMARY DIAGRAM The following summarizes the SMH results framework that captures the overall long term goal, strategic objective and intermediate results as they relate to the three key project components: pre￾transformation activities, transformation activities and sustainability activities. Monitoring and Evaluation Plan, Year 1 9 2.3.2 PERFORMANCE INDICATORS PER RESULTS LEVEL Below is a summary of the performance indicators for each results level. Further explanation can be found in Annex 2: Indicator’s Table. Sub-IR 3.3 Management and programmatic capacity of GRMA increased • Indicator 26: Number of GRMA staff and volunteers who receive capacity building assistance. Sub-IR 3.2: Increased capacity of PMHs to manage their businesses efficiently • Indicator 23: Number of business training events conducted • Indicator 24: Percentage of assisted PMHs maintaining core financial documents (cash log, profit and loss statement, balance sheet) • Indicator 25: Percentage of participating PMHs that have registered their businesses with the Ghana Registrar General Department Sub-IR 3.1 Capacity of PMHs to sustain, increase, and improve service delivery strengthened • Indicator 21: Percentage of assisted PMHs applying for NHIA accreditation who (a) receive a passing grade or (b) improve their grade. • Indicator 22: Percentage of assisted PMHs that receive or maintain NHIA accreditation after transformation. Sub-IR 2.2: Funding provided and leveraged by the project • Indicator 17: Number of assisted PMHs that receive seed funding in the form of SMH small grants • Indicator 18: Amount of seed funding provided by the SMH project in the form of small grants • Indicator 19: Number of assisted PMHs that receive funds from non-project sources • Indicated 20: Amount of funding that assisted PMHs receive from non-project sources Sub-IR 2.1: PMH ownership/management transformations brokered • Indicator 11: Number of assisted active PMHs who complete an ownership/management transformation • Indicator 12: Number of inactive PMHs re-opened with new owners or managers • Indicator 13: Number of new PMHs opened • Indicator 14: Number of investment proposals for PMH transformations prepared • Indicator 15: Number of new/younger midwives interested in ownership and/or management of PMHs • Indicator 16: Percentage of PMHs that have a neutral or positive attitude towards the idea of ownership/management transformation. Sub-IR 1.2: Capacity of PMHs for ownership/management transformation strengthened • Indicator 7: Number of PMH owners who have received pre-transformation technical assistance • Indicator 8: Percentage of training participants who attain a score of 75% or higher on training post-tests • Indicator 9: Number of unique PMH facilities trained by the SMH project • Indicator 10: Number of participants in SMH training courses Sub-IR 1.1: PMHs interested in and ready for transformation identified • Indicator 6: Number of assisted PMHs expressing interest in ownership/management transformation Strategic Objective: PMHs sustained, MCH services expanded, and efficiency and quality improved • Indicator 1: Percentage of assisted PMHs with increased financial sustainability • Indicator 2: Number of assisted PMHs with improved quality of services • Indicator 3: Number of assisted PMHs with improved technical efficiency • Indicator 4: Number of assisted PMHs that add new health services • Indicator 5: Number of patient or client visits received by assisted PMHs that undergo an ownership/management transformation Long-Term Goal: Increase private maternity homes’ ability to sustain and grow services in the underserved areas of Ghana IR1: PMH owners and management ready for ownership and/or management change IR2: PMHs under new ownership/ management continue health service delivery IR3: Business and programmatic capacity of PMHs and GRMA increased Monitoring and Evaluation Plan, Year 1 10 3. Monitoring Plan 3.1 SOURCES OF DATA AND DATA COLLECTION TOOLS The SMH project will use four different information sources and data collection tools to collect the data needed to inform its M&E and operations research. These information sources/data collection tools are described below. Secondary Data In order to build on the knowledge and best practices that have already been established by USAID and other development actors, and to provide context and background to its specific data collection and measurement activities, SMH will draw on data produced by the MOH, GHS and NHIA. Nationally, the MOH/GHS receives data about the private health facilities through the district offices per the usual channel of data submission (monthly data returns). The project will also be collecting data from NHIA on number and geographic distribution of accredited PMHs, accreditation grades, and results of audit visits. The SMH M&E team will input this data into the SMH database and compile other written reports from these two sources as per the reporting schedule. Source data acquired through these channels will be stored at the district health office and the USAID/Ghana SMH Project. It will be electronically kept in the GRMA/ SMH database while ensuring confidentiality. PMH Survey1 SMH will design a customized PMH survey that will provide baseline and endline data on key PMH indicators. The survey will focus on the health services and the business aspects of the PMHs and will be developed taking into account the requirements of GHS and NHIA supplemented with the best practices in managing maternity homes. In addition to the survey and the GHS data, the baseline and endline reports will also draw on key informant interviews. During the baseline and endline, the project will attempt to administer the survey to all 170 PMHs in the target regions using the formhub.org mobile data collection platform. The PMH survey is included in Annex 6. Quality Assessment Tool SMH will assist the GRMA in developing a quality assessment tool (QAT) that will capture the various aspects of healthcare quality provided by a well-run PMH. The QAT will be developed taking into account the regulatory requirements of the Health Facilities Regulatory Agency (HEFRA), GHS and NHIA, as well as the principles of the best practices in midwifery. The QAT will be constructed to cover all aspects of PMH quality that will allow SMH to create an index of service quality for all PMHs. The index will be used to benchmark all PMHs together with other data collected at the baseline. During the baseline and endline, the project will attempt to administer the QAT to all 170 PMHs in the target regions. The QAT is included in Annex 7. Once the QAT has been developed, GRMA will revive its practice of conducting ‘peer support visits’ (PSV). These visits are intended to monitor and improve quality of service and were a practice of GRMA during the past few years before being discontinued due to lack of funding. PSVs consist of administering the QAT, immediately tallying the score and grade and then providing feedback directly to the PMH 1 The Reviewer had suggested renaming this “PMH Baseline Survey.” Our preference would be to continue to refer to this as the “PMH Survey.” By PMH Survey, we mean the questionnaire that is administered to the PMHs at both the baseline and the endline, to generate quantitative survey data. We feel that renaming it PMH Baseline Survey might be misleading, since our plan is to use the same instrument at the endline. Monitoring and Evaluation Plan, Year 1 11 regarding areas where they did well and areas in which they could improve. During the baseline, GRMA will travel along with SMH enumerators and administer the QAT during the same visit at which enumerators administer the PMH survey. All 170 PMHs surveyed for the baseline will also receive the PSV, and will be evaluated using the QAT. PSVs will be led by the regional GRMA executive who will be accompanied by a member of Greater Accra Chapter of GRMA, which is the chapter working directly with the SMH project. During the visit, the PMHs will receive their QAT score along with feedback from the PSV team. In-person PSVs will be administered to all PMHs during the baseline and endline and will ideally also be repeated annually subject to budget constraints. In lieu of annual in-person PSVs, SMH is currently developing a telephone-based version of the PSV in case annual in-person administration proves infeasible. Additional Project Records and Project Documents The project will develop and maintain records related to its technical assistance and capacity building activities, which document the technical assistance to PMHs and GRMA, and in particular will document the transformation process of the PMHs. These documents will provide additional data for the project and will include, among other things, the following forms, which will be developed in due course of the project. • Trainee Registration Forms – PMHs will complete these forms at the beginning of each training course conducted by SMH. These forms are related to Tier 2 of project assistance. • Training Pre- and Post-Test Forms – PMHs will complete these forms at the beginning and end of each SMH training course. These forms relate to Tier 2 of project assistance. • Expression of Interest Forms – PMHs will submit these forms to the project to officially declare their interest in receiving project assistance to pursue an ownership and/or management transformation. These forms will be submitted to PMHs receiving Tier 2 of project assistance who want to be considered for participation in Tier 3 of project assistance. • Technical Assistance Record Forms – SMH will use these forms to record the topics discussed and action items decided during each one-on-one technical assistance visit to a PMH. These forms relate to Tier 3 of project assistance. 3.2 ANALYSIS AND REPORTING Some data acquisition and analysis activities will occur at specific points in time during the project cycle in order to establish benchmark values against which to judge project performance. The project will also engage in ongoing data acquisition and analysis in order to inform project implementation. In terms of the focus of the analysis, the project will be primarily interested in the relationship between various levels of project support and business and financial sustainability (as measured by the BFS), and to changes in the efficiency and quality of care offered by PMHs from baseline to endline. A summary of the various data analysis and reporting activities is provided below. Baseline Assessment Report The Baseline Assessment Report will summarize the baseline survey methodology and implementation and the results of the research data, including information about the business performance and health services provided by the PMHs. The report will be compiled after the baseline survey has been completed and data analyzed. Endline Report Monitoring and Evaluation Plan, Year 1 12 The Endline Report will have a similar structure to the Baseline Assessment Report except that it will be based on project endline survey conducted during Year 3. In addition to the results of the PMH survey and QAT, the Endline Report will include qualitative data in order to gain a deeper understanding of the factors driving quantitative data collected in the PMH survey and QAT. Operations Research Final Report The Operations Research Final Report will be prepared in Year 3 as a summary of the relative situation of PMHs before and after the project interventions and will reflect on the changes that occurred as a result of project activities. The two main research questions addressed in this report include the following: • Does ownership and/or management transformation improve the quality of health services delivered by private maternity homes? • Does ownership and/or management transformation improve the efficiency of private maternity homes? For more on the operations research plan, see the section below titled: Operations Research Plan. Ongoing Monitoring, and Annual and Quarterly Reports The project will produce Annual and Quarterly Reports (as required by USAID) in addition to ad hoc monitoring reports in order to inform project implementation and decision-making. The project does intend to contribute to Innovate for Health’s overall evaluation plans through the demonstration effect of targeted business management assistance to improve the quality and quantity of healthcare in the underserved areas in the country, and in particular the importance of business success and business continuity management for the continued provision of healthcare services. 3.3 DATA QUALITY ASSURANCE The USAID/Ghana Saving Maternity Homes Project shall work towards complete, accurate and reliable data. This will be done through mentoring and coaching of data collectors and enumerators, and through carrying out data quality assessments. Review of data collected shall be done through the support supervision visits and data quality assessments (DQAs). In addition, the project will do limited quality assurance sampling by the project staff participating in randomly selected survey visits to evaluate the quality of work of the enumerators. The project will monitor data quality through a Lot Quality Assurance Sampling (LQAS) approach. LQAS was originally designed for use in manufacturing, where it provided a way to perform statistically valid quality-assurance testing at minimum cost. In the context of international development, LQAS has become an accepted sampling method in the fields of public health and other areas. With LQAS, a relatively small random sample of 19 observations is drawn to determine whether outcomes satisfy pre￾determined quality standards. LQAS is applied to dichotomous (e.g., yes/no or pass/fail), quality standards questions. As such, it requires advance determination of which characteristics must be met for a sample to be considered ‘acceptable.’ In the context of DQAs, the LQAS approach will be used to determine whether data being collected and reported for selected PMP indictors is accurate. It will do this by comparing data collected during DQAs at 19 randomly selected PMHs per supervision area for selected indicators with the data collected and reported for that indicator by the project performance monitoring system. If the data collected during the DQA corresponds to the data reported by the performance monitoring system for Monitoring and Evaluation Plan, Year 1 13 that indicator at a pre-determined percentage of PMHs, then the data for that indicator will be considered validated, and vice versa. In developing the specific tools and processes for conducing DQA visits and data collection, the project will draw on USAID DQA best practices,2 as well as tools such as the USAID “Data Quality Assessment Checklist and Recommended Procedures.”3 3.4 GENDER CONSIDERATIONS The project is committed to a thorough and thoughtful integration of gender considerations into the implementation of project activates. The primary unit of analysis and intervention of the SMH project is the PMH facility. In terms of gender, the project has identified the gender of the PMH owner as the most appropriate point of analysis for integrating gender considerations, though we do intend to incorporate a focus on client-level gender considerations as appropriate. The project will collect gender￾disaggregated data based on the owner of the PMH facilities that operate in the six target regions, and will implement a Gender Analysis in Year 2 designed to gain an understanding of how gender dynamics affect the project, and how the project might affect gender dynamics. Gender disaggregated data and the Year 2 Gender Analysis will focus on two framing issues. First, the project will use gender disaggregated data to make sure project activities and participation are not inappropriately skewed to male or female PMH owners. Preliminary baseline survey data indicates that 95% of the PMHs in the target regions are owned by women. This finding suggests the possibility of systemic or social constraints to male ownership of PMHs. These constraints could affect levels of interest of potential new investors/owners of declining PMHs if there are deterrents to male involvement, an aspect that has direct relevance to overall project success.4 The second framing issue will focus on a well-recognized gender disparity in opportunities for women business owners and entrepreneurs. A recent IFC report on the financing options for women-owned SMEs noted that a variety of obstacles to business success: “Financial barriers include local financial conditions – such as lack of collateral, inadequate financial infrastructure, or the high cost of funding – while non-financial barriers include the social and cultural norms underlying gender biases, as well as the tendency for women SMEs to be smaller in size, and the limited access to business education opportunities and networks.”5 In addition to the two framing issues mentioned above, the gender analysis would include the five domains of gender analysis, as stipulated by USAID: 1) status of women and men and their differential access to/control over assets, resources, opportunities, and services relevant to the project; 2) male and female roles, responsibilities, and time use that could prevent or facilitate participation in the project; 3) relevant laws, policies, and institutional practices that may contain implicit or explicit gender biases; 4) gender norms that may affect women’s ability to participate and assume leadership roles and decision￾making in the project; and, 5) potential impacts of the project on women and men, including unintended or negative consequences. As mentioned above, this gender analysis will first look at gender at the facility level, but will also incorporate an investigation of gender at the client level as well. We plan to design and implement this gender analysis during the first quarter of Year 2. 2 The project will utilize practices as described in the following website: http://usaidprojectstarter.org/content/conducting-data-quality-assessments 3 http://usaidlearninglab.org/sites/default/files/resource/files/Data%20Quality%20Assessment%20Checklist.pdf 4 In addition to a gender disparity, the baseline found that current PMH owners tend to be homogeneous in terms of age (they are older) and professional background (they are trained as midwives). The project will look for opportunities for efficiencies in terms of investigating these issues, as well as the homogeneity of the gender of PMH owners. 5 “Women-Owned SMEs: A Business Opportunity for Financial Institutions,” IFC, 2014. Monitoring and Evaluation Plan, Year 1 14 Recognizing the factors outlined above, the project will pay particular attention to gender-disaggregated data related to the following areas: • Gender of prospective new owners/managers: One of the SMH project’s main objectives is to facilitate ownership/management transformations, often involving new owners and managers. In recognition of the gender disparity of PMH ownership, the project will deliberately monitor the gender of prospective new owners/manager, with a focus on the potential impact of systemic constraints that could be affecting male interest in becoming a PMH owner/manager. • Business formalization: Business formalization, measured through the BFS, is one of the project’s primary analytical tools for evaluating the financial sustainability of a PMH. The project will investigate whether gender correlates to differing levels of formalization, and if so, what actions are available to the project help address the disparity. • Profitability: Profitability is, of course, a critical factor in the financial sustainability of any business. The project will apply a gender lens to levels of profitability in order to identify any potential disparities, and explore potential actions to address these disparities. • Optimism about the future: The project will likewise investigate whether there are consistent difference between men and women in terms of optimism about the future. If differences are discovered, we will investigate the explanations behind those differences, and explore possible activities to address them. • Interest in ownership/management transformation: The project will also investigate whether there are substantial differences between the genders in terms of levels of interest in the various forms of ownership or management transformation. This analysis will help the project uncover potentially powerful obstacles, and more effectively pursue the project’s primary objective of using these transformations to boost the availability of PMH services for local communities. The project will revisit its approach to addressing relevant gender considerations regularly, and will update and refine its approach as needed. One anticipated point of revision will be after the conclusion of the gender analysis mentioned above. 3.5 CRITICAL ASSUMPTIONS The M&E plan as described in this document makes several critical assumptions. These include: • Political stability and security in the target regions and in Ghana broadly: Continued implementation of project activities depends on a political and security environment that allows for these activities. Should unexpected events arise that raise concerns about the political stability of target regions or of Ghana broadly, or if the security of project beneficiaries, partners, or staff is in any way uncertain, the project will curtail project activities to an appropriate extent until the situation has been resolved. Should that occur, the M&E strategy and activities as described in this document will be revised. • Availability of funds for project activities: In order to carry out planned project and M&E activities, it is assumed that sufficient funds will be available. Should anticipated funding be reduced, the project will revise its implementation strategy, and M&E plan. • Data collection from other organizations will be regular and reliable and will be sufficiently comprehensive to serve as a source data for the relevant requirements: In an effort to maximize efficiency, the project plans to use GHS and NHIA data as an ongoing source of M&E data, informing ongoing monitoring, operations research, and the proposed evaluation. Should this data become unreliable, data collection, analysis, and reporting for the relevant indicators may become infeasible. In such a Monitoring and Evaluation Plan, Year 1 15 situation, the aspects of project implementation and of this M&E plan that relate to those indicators will be revisited. • Availability of personnel from GRMA and partner organizations to conduct data gathering and analysis activities as required: The project design assumes that GRMA will be available to participate in data gathering activities for the baseline and endline, conduct analysis on findings from data gathered from the quality assessment tool, and participate in M&E activities in other ways. Additionally, the project anticipates participation from other project partners and potential key informants in terms of contributing insights and expertise, and playing other support roles. Should the participation and support of these organizations and actors be less than anticipated, this M&E plan will be revised accordingly. Monitoring and Evaluation Plan, Year 1 16 4. Baseline Assessment Plan 4. 1. PURPOSE OF THE BASELINE ASSESSMENT The USAID/Ghana Saving Maternity Homes Project will carry out a baseline assessment to establish benchmark values of the PMP indicators, and to gain an understanding of important considerations related to SMH project implementation. This assessment will include data collection and reporting for various PMH health and business performance indicators. The baseline data collected will ground the indicators and targets with evidence and allow the project to carry out pre- and post-intervention comparisons to measure project effectiveness and impact. The baseline assessment will draw on both secondary data from a variety of sources and stakeholders and primary data collected from PMHs using the PMH survey. The results of the baseline will be shared with USAID, regional and national stakeholders, supported private health facilities, partners and GRMA for planning and project implementation purposes. 4. 2. METHODS In order to complete the baseline assessment, the project will engage in the following activities: • Literature review: The project will review the relevant literature in order to obtain background quantitative and qualitative information on the challenges facing private maternity homes in underserved areas of Ghana. • PMH Survey: The project will design a survey tool to be used to capture baseline data that cannot be acquired through existing data sets from the GRMA or the GoG. The questionnaire will be designed by the project and tested with a select number of PMHs in the Greater Accra region before launching the full data collection effort. • Key Informant Interviews: These interviews will be used to gather in-depth qualitative data on the challenges facing private maternity homes in Ghana and the considerations related to revitalization of these nursing homes through investment and management transformations. 4. 3. DATA COLLECTION PLAN PMH Survey The PMH survey will be administered to all PMHs that have a current registration with GHS located in the target regions (Brong Ahafo, Central, Eastern, Northern, Volta and Western). At the time of writing, there were 170 PMHs in the target regions registered with GHS. The survey will target the managing directors of the PMHs, as these individuals are likely to be the only ones employed by the PMH who will be able to provide the required information. The PMH survey will be designed and tested in the first quarter of FY15 (October-December 2014), and administered during the second quarter of FY15 (January-March, 2015). The SMH M&E team will recruit, hire and train experienced enumerators to administer the questionnaire and record the resulting data. The survey will be built onto the formhub.org mobile data collection platform, and enumerators will utilize affordable Android tablets to collect data. This form of data collection increases the quality of the data by reducing the range of decision-making available to the enumerator and minimizing input and Monitoring and Evaluation Plan, Year 1 17 coding errors. The data uploads automatically online to a centralized database once the tablet is connected to the internet thereby reducing the chances of raw data being lost or damaged. This strategy also facilitates ongoing data analysis by the M&E team, who will have access to the data online. The formhub.org platform also enables easy collection of GPS coordinates, allowing the project to incorporate geospatial characteristics into the data analysis as well as project planning and implementation. The survey will be combined with a peer support visit conducted by GRMA that will collect qualitative data about the facility’s emergency response capabilities, quality assurance measures, safe motherhood services, and advocacy efforts. This approach will incentivize the PMH owners and/or managers to participate in the data collection in that they are likely to be more forthcoming in providing information if the enumerators are legitimized by the presence of GRMA, and if they feel that they benefit from the time spent with the team. While partnering with GRMA offers valuable data-gathering benefits in terms of gaining access to, and the trust of, the PMHs in the target regions, this strategy also opens the possibility that GRMA’s presence might lead respondents to skew or misrepresent their business operations. Initial testing of the survey tool in Greater Accra suggested that the risk of this type of bias is minimal; however, the project still intends to take specific steps to reduce the likelihood of GRMA’s presence adversely affecting the quality of the data collected. Both the PMH survey and the QAT involve the sighting of specific elements (documents in the case of the questionnaire, protocols and equipment in the case of the QAT). The project has designed the interview process so that, whenever possible, either GRMA is touring the facility and sighting the required elements while the PMH survey is being administered, or the survey enumerators are working with assistants and/or administrative staff to sight formalization documents while GRMA is administering the QAT and providing feedback. This approach allows each team to leave the room when appropriate and has the added benefit of reducing the overall time that the visit requires. The project believes that this approach will minimize any potential information bias that might have resulted from having GRMA present when the PMH survey is being administered. Qualitative Data Collection In addition to the PMH survey and QAT, the SMH M&E team will conduct a series of key informant interviews (KIIs) during the baseline and endline data collection rounds with persons belonging to key stakeholder groups within the country. KIIs are in-depth and semi-structured interviews conducted with individual stakeholders or small groups of stakeholders. KIIs will take approximately 60 minutes to complete and will be conducted face-to-face, by phone or by Skype. All KIIs will use a pre-prepared discussion guide, although interviewers will also be expected to follow-up with in-depth probing questions and to deviate from the discussion guide where appropriate. KIIs will be conducted with a wide range of stakeholder groups. KIIs will be used to generate information and insights not available through the largely quantitative and closed-ended nature of the PMH survey and QAT as well as to provide context and depth to the quantitative data generated by the survey and QAT via its use of in￾depth probing. Key informants will be selected for their experience and expertise in some area that has substantial bearing on midwifery practice in Ghana. The list will be drawn from midwifery policy-makers and regulators; the health insurance industry; health service managers; industry bodies of midwives in Ghana, and independent non-governmental agencies providing training to midwives, among others. For the baseline, the project anticipates conducting approximately ten key informant interviews. Stakeholders potentially participating in the KIIs include those listed below. Monitoring and Evaluation Plan, Year 1 18 • Deputy Chief Executive, Operations: National Health Insurance Authority • Director, Quality Assurance: National Health Insurance Authority • Director, Family Health Division: Ghana Health Service • Director, Institutional Care Division: Ghana Health Service • Director, Nursing & Midwifery: Ministry of Health • Registrar: Nursing and Midwifery Council • Head: Health Facility Regulatory Agency • President: Ghana Registered Midwives Association • Country Director: Jphiego (or other organizations providing training to midwives) Focus Group Discussions (endline) While they will not be conducted as part of the baseline report, as part of the endline research, the M&E team will conduct a series of focus group discussions (FGDs) with clients of transformed PMHs. FGDs are in-depth and semi-structured interviews conducted face-to-face with PMH clients. FGDs will typically involve a small group of approximately 6-10 clients selected for their experience, with or knowledge of, the topics of interest and take anywhere from 1-1.5 hours to complete. All FGDs will use a pre-prepared discussion guide, although discussion facilitators will also be expected to follow￾up with in-depth probing questions and to deviate from the discussion guide where appropriate. The purpose of the FGDs will be to probe into the clients’ experiences with the PMHs, their perceptions of the PMHs’ services (e.g., quality and range) and the PMH service providers (e.g., knowledge, skill, concern) and the health outcomes of their visits to the PMHs. Our intention is to include clients in the FGDs who have used the PMH services both prior to and after the transformation so that we can gain an understanding about whether and the extent the quality, range, etc. of care have changed from the pre-transformation to the post-transformation. If sufficient numbers of these clients are not available, we will conduct separate FGDs with both pre and post-clients and with post-clients only. Overall, we anticipate conducting in the range of 6-8 FGDs, depending on the number of PMHs that have successfully completed the transformation process. PMHs selected for the FGDs will be selected purposively to represent different geographic locations and/or transformation models. Data from Other Sources During the months that the project is collecting survey data, the project will also be working with GRMA and other stakeholders to access the required data from other sources. (See the Indicators Table in Annex 3 for information on the data source for each indicator.) Additionally, the project will be conducting key informant interviews to gather in-depth qualitative data (as described above). 4. 4. ANALYSIS AND REPORTING Quantitative Data Analysis Data obtained from the PMH survey will be reviewed, cleaned and analyzed to provide the information about the state of PMHs in selected regions of Ghana at the beginning of the project. Analytical conclusions will be summarized across all PMHs and according to selected sub-categories, including regions, so as to allow the project to garner insights into the business operations and quality of service aspects of PMHs overall and broken down by various sub-categories. Qualitative Data Analysis We will use a common approach to analyze the qualitative data gathered through KIIs and FGDs. This approach includes the five steps described below: Monitoring and Evaluation Plan, Year 1 19 • Step 1 – Extract and Summarize the Data: In this step, we read through all of the data collected and extract from it information that is relevant for understanding relevant considerations. We next take this information to create a summary document describing the ‘main evaluation findings.’ • Step 2 – Analyze the Data: In this step, we perform a ‘content analysis’ of the data found in the summary document created in Step 1 in which we identify themes and patterns in the data (e.g., ideas, concepts, behaviors, interactions, incidents, or terminology and phrases used) and then organize them into coherent categories and sub-categories that summarize and bring meaning to the text. • Step 3 – Identify Patterns and Connections: In this step, we look for and identify patterns and connections both within and across categories, while simultaneously assessing the relative importance of different themes or highlighting subtle variations that may be important to our analysis. We also look for information that can help us understand the effectiveness and impact of project activities, the significance of those results, and the process by which they occur. • Step 4 – Review, Reflect, and Discuss: In this step, we conduct a thorough review of our analysis and findings. This step involves all M&E team members in a process of review, reflection, and discussion as we move collaboratively toward a consensus on what the main findings are and what they mean. • Step 5 – Draw Conclusions: In this final step, we take the results of our analysis in Steps 1-4 and draw our conclusions, lessons learned and recommendations. Measuring Efficiency: Data Envelopment Analysis (DEA) In order to measure the efficiency of each individual PMH in the target regions, and of the PMHs in the target regions as a whole, the project will use a methodology known as data envelopment analysis (DEA). This methodology is designed to provide an understanding of how efficiently a health provider is using its resources to deliver health services. The DEA methodology quantifies various key inputs, including labor (clinical and non-clinical personnel), materials (pharmaceutical supplies, non￾pharmaceutical supplies, other goods and commodities) and capital (building, medical equipment, vehicles, beds), together with outputs, in the form of the health services provided and the number of client visits the facility receives during a specific period of time (such as a typical calendar month). After these quantities have been collected (in the case of the SMH project, through the survey questionnaire), the facility that is able to deliver the greatest amount of health services (outputs) given the resources at its disposal (inputs) is declared to be the most efficient, and that facility’s performance serves as the best practices frontier, or data envelop, for the sample. That facility receives a score of 1, and each other facility receives a score of between zero and one, based on how efficient it is in comparison to the frontier. This methodology will enable the project to benchmark each PMH in the target region according to their relative efficiency, and analyze changes in that efficiency over time, by comparing the scores obtained during the baseline and endline. 6 Dissemination of Findings and Analysis The Baseline Assessment Report will summarize the baseline survey results and will be shared with GRMA, GHS, NHIA as well as other stakeholders, following USAID review. In addition, each PMH participating in the survey will receive feedback from the PSVs and a short summary of the performance of the PMH in relation to others that will allow the owners to benchmark themselves to other PMHs. SMH will organize a dissemination event to present the findings of the baseline survey results. Raw data 6 For an example of how DEA has been applied in Ghana, please see Jehu-Appiah, “Ownership and technical efficiency of hospitals: evidence from Ghana using data envelopment analysis”, Cost Effectiveness and Resource Allocation 2014, 12:9. Monitoring and Evaluation Plan, Year 1 20 will be provided upon request, although all specific identifying information will be kept strictly confidential and will only be available to project staff and to USAID. In order to ensure that privacy is maintained, dissemination will be limited to information and findings derived from summary statistical analysis. The Baseline Assessment Report will be submitted to USAID’s Development Experience Clearinghouse (DEC), and the dataset will be submitted to USAID’s Development Data Library (DDL) in a compliant machine-readable, non-proprietary format. Monitoring and Evaluation Plan, Year 1 21 5. Operations Research Plan 5. 1. OVERALL APPROACH The project will conduct a pre- and post-transformation assessment of PMHs to determine the impact of ownership and/or management transformation on the quantity, quality and efficiency of services provided. The data for this assessment will be drawn from the baseline and endline surveys and will be enhanced by other findings as described below. Specific attention will be paid to the potential improvements in the quality, expansion and outreach of services as they relate to the various models of PMH transformation under consideration (outright sale, co-ownership/partnership, leasing, and management contract). 5. 2. BACKGROUND As shown by the research sponsored by GHS-affiliated research institutes, health facilities in Ghana are underutilized. These facilities have excess capacity and lower than optimal efficiency, leading to higher unit costs of operations7. These challenges extend beyond the public sector, and are also obstacles faced by private health providers in Ghana, including private maternity homes. The reasons why these facilities are underutilized are many, ranging from issues with government support and the broader health system, to challenges that individual clients face in accessing health facilities and paying for services. There are also demand constraints resulting in low usage of professional health facilities. SMH is interested in the potential impact that ownership and/or management transformation could play in increasing the quality and efficiency of private maternity homes. 5. 3. RESEARCH HYPOTHESIS AND QUESTIONS The SMH Operations Research will test the following research hypothesis: Private maternity homes that execute an ownership and/or management transformation will deliver measurable improvements in quality and efficiency over private maternity homes that do not execute such a transformation. In testing the above hypothesis, the SMH Operations Research will further seek to answer the following research questions: • Does ownership and/or management transformation improve the quality of health services delivered by private maternity homes? • Does ownership and/or management transformation improve the efficiency of private maternity homes? 7 Dalaba et al. Cost of maternal health services in selected primary care centres in Ghana: a step down allocation approach. BMC Health Services Research 2013, 13:287 http://www.biomedcentral.com/1472-6963/13/287. Navrongo Health Research Centre, Navrongo, Ghana Monitoring and Evaluation Plan, Year 1 22 5. 4. METHODOLOGY The project will use operational research methods to identify and solve problems related to the operations of PMHs, in particular related to the efficiency and quality of services. All data required for the operations research will be drawn from the data collection activities described in the project’s PMP. No additional tools, personnel, or resources will be required. The ways in which the operations research methodology will differ from the standard M&E methodology described above include is in the analytical methods applied. Those methods include the following: • Technical Efficiency: Using data from the baseline and endline PMH surveys, SMH will develop and calculate a measure of technical efficiency for PMHs. Inputs and outputs for PMHs will be analyzed using data envelopment analysis (DEA)8 that will allow benchmarking of all PMHs according to their technical efficiency score. • Quality Assessment Tool Index: Peer Support Visits will result in a comprehensive assessment of the PMH service quality using the QAT. As you can see in Annex 7, each quality assessment results in a QAT score. For the operations research, the project will normalize the data to a 0-1 scale, in order to enable easy comparisons between the quality index and the technical efficiency score resulting from the DEA analysis described above. 5. 5. OPERATIONS RESEARCH RESULTS AND DISSEMINATION The project will summarize the results of the operations research in the Operations Research Final Report and, together with GRMA, will organize a dissemination event to present the findings to the relevant stakeholders. Local research institutes in Ghana which have done similar work will be invited to enrich the discussion and compare results and methods applied. 8 Data envelopment analysis (DEA) is a non-parametric method in operations research used to empirically measure productive efficiency of decision-making units (DMUs). The methodology is also used for benchmarking in operations management, where a set of measures is selected to benchmark the performance of manufacturing and service operations. Monitoring and Evaluation Plan, Year 1 23 6. Evaluation Plan 6. 1. PURPOSE OF EVALUATION The most reliable approach to determining and attributing impact is an impact evaluation. The USAID Evaluation Policy defines an impact evaluation as an evaluation that seeks to measure the change in project outcomes that are attributable to (caused by) the project using methods that a construct a “credible and rigorously defined counterfactual” so as “to control for factors other than the intervention that might account for the observed change.” Creating a credible and rigorously defined counterfactual in turn requires methods for comparing a treatment group of project beneficiaries and a control group of non-beneficiaries who are as similar as possible to project beneficiaries. Unfortunately, the SMH project does not have the resources to conduct an impact evaluation using the methods necessary to create a credible or rigorously defined counterfactual. Additionally, the small anticipated sample size of PMH facilities that will undergo ownership/management transformations does not lend itself to a “large n evaluation,” which would be appropriate for this type of rigorous impact evaluation.9 However, given the innovative nature of the project design and the potential learning to be derived, the SMH project does plan to conduct a limited “small n evaluation” using less rigorous, but, nonetheless, sound methods. The purpose of the evaluation will be to generate findings related to the effectiveness and impact of project activities along with lessons learned for the purpose of improving, replicating, or scaling-up aspects of the intervention, or to justify a more rigorous evaluation of future programming that takes a similar approach. 6. 2. KEY EVALUATION QUESTION The SMH project evaluation will seek to answer the following key question: 1. What was the effect of SMH project activities on the financial sustainability (as measured by the BFS) of PMHs in the target regions? 6. 3. EVALUATION METHODOLOGY Given the limited budgetary resources and the inherently small sample size involved with the SMH survey, the project will employ the General Elimination Methodology (GEM), a type of “small n evaluation” methodology that focuses on systematically identifying and eliminating alternative causal factors that could explain an observed result.10 While it does not provide the level of rigor of a large n evaluation, this type of small n evaluation will enable the project to gain some understanding of project effectiveness and the impact that can be attributed to project activities. An outline of this evaluation methodology is as follows: 1. Define the sample to be evaluated: In our case, the sample will be all PMHs that post an improvement in their Business Formalization Score at endline, as compared to their scores at 9 In statistical notation “n” refers to the size of the sample on which statistical analysis is conducted, while “N” refers to the size of the full population from which the sample is drawn. 10 For an overview discussion of the GEM methodology, as well as other small n evaluation methodologies, see Howard White and Daniel Phillips, “Addressing attribution of cause and effect in small n impact evaluations: towards an integrated framework,” International Initiative for Impact Evaluation, Working Paper 15, 2012. Monitoring and Evaluation Plan, Year 1 24 baseline. Given current LOP projections, the project expects this sample to comprise approximately 29 facilities.11 2. Investigate the extent to which the outcomes of interest have occurred: In the case of the SMH project, the outcomes of interest are improvements in profitability and financial sustainability of the PMH facilities in the sample. (In the operations research, we focus specifically on efficiency and quality.) The project will identify the extent to which these outcomes have occurred for the facilities comprising the sample through the collection and analysis of PMH survey data. The primary outcome of interest would be that a PMH achieves a greater BFS at endline than it did at baseline. 3. Draw up a list of possible causes or competing explanations for the outcomes of interest: The project will rely on project expertise as well as key informant input to draw up a list of alternative explanations for why PMH facilities in the sample may have shown improvements in profitability or financial sustainability. This list will represent the alternative hypothetical explanations for the outcomes of interest, and therefore the possible alternative causes to which the outcomes can be attributed. For example, a competing explanation for a higher BFS would be that the PMH has started keeping financial statements and written contracts with suppliers (two factors in the BFS) because it received business management training from an NGO that is not affiliated with the project. 4. Outline the Modus Operandi (MO) for each possible cause: The project will identify the Modus Operandi – or the set of “footprints” – for each possible cause identified in the previous step. Michael Scriven, the author of the GEM methodology, describes these footprints as, “a sequence of intermediate or concurrent events, a set of conditions or a chain of events that has to be present when the cause is effective.”12 For example, the MO of the PMH receiving separate business management training could be that 1) an NGO is offering business management training, 2) this business management training included a specific focus on financial statement and keeping contracts, 3) the PMH in question attended this business management training, 3) the PMH began keeping the financial records recommended by the training and keeping written contracts as recommended by the training. 5. Establish the “facts of the case” to determine if the MO of each competing explanation is present or not: The project will conduct interviews and other forms of investigation to determine whether the MO of each competing cause is present. If the MO of a competing explanation is present, that would suggest that the competing explanation plays some causal role in explaining the change. If the MO of the competing explanation is not present, that suggests the competing explanation either did not happen, or did not play a causal role in explaining the change, and can be eliminated. For each PMH in the sample, the project will systematically investigate each competing explanation in order to gain a full sense of the extent to which the event of interest is attributable to the project intervention. While this methodology does not provide the same level of statistical rigor and confidence that is achievable in a large n evaluation, it does provide a systematic approach to identifying and attributing causation to various project outcomes. The project will seek to incorporate USAID and other 11 Given the intensiveness of the GEM methodology, and the project’s time and resource constraints, this number may need to be reduced in the final evaluation design. In that case, the project will apply a reasoned approach to selecting facilities to be evaluated, given the objectives of the evaluation. 12 Scriven, M. (2008) A summative evaluation of RCT methodology and an alternative approach to causal research. Journal of Multidisciplinary Evaluation, 5 (9): 11-24. Monitoring and Evaluation Plan, Year 1 25 stakeholders in the design of the GEM evaluation, specifically in the critical areas of drawing up a list of competing causes and establishing the MO of each competing cause. We believe that this approach will enable us to gain some useful understanding of the extent to which project activities caused increases in profitability and financial sustainability, while recognizing budget and time constraints. 6. 4. DATA COLLECTION AND ANALYSIS The project will not know which facilities posted improvements in the BFS until after the endline assessment data has been collected. At that point, the project will prioritize the calculation of BFS scores, so that this evaluation can be undertaken while the full endline assessment report is being drafted. For this evaluation, the project will develop the needed data gathering tools, including interview guide to gather the necessary data to establish the “facts of the case” as described above. Once the evaluation data has been gathered, the project will conduct analysis to identify the extent to which each competing explanation is present and the extent to which each competing explanation played a causal role in the observed outcome (increased profitability, or increased financial sustainability). The project will then summarize this analysis in an evaluation report that describes the extent to which project activities played a causal role in the observed outcomes, and the extent to which competing explanations played a causal role in the observed outcomes. This report will be designed to indicate clearly the extent to which project activities achieved the desired impact, and highlight useful learning that could benefit future interventions. 6. 5. ANTICIPATED CONSTRAINTS The main constraint that we anticipate in the above evaluation design is bias that could affect the validity of the evaluation data and findings. On the one hand, there is a concern about respondent bias, particularly in the form of respondents seeking to provide project personnel (or representatives of project personnel) with positive answers out of a desire to maintain good relationships, and appreciation for the relationship that has been built. On the other hand, there is potential evaluator bias, in that the project itself will read the data to reflect positive outcomes in order to validate the project model or activities. The project will take various steps to address these biases. We will employ an independent consultant to conduct the bulk of the data collection, and devise a detailed research design that includes triangulation of the “facts of the case” through multiple sources (for example, the PMH owner, the local GRMA representative, and local business owners familiar with the health services available in the area). Additionally, we will employ data analysis practices that ensure systemic analysis of the qualitative data, such as the data analysis process outlined above. Such practices mitigate against the risk of seeing positive results in raw qualitative data, and they force the analyst to see true consistencies and valid insights. Monitoring and Evaluation Plan, Year 1 26 7. MEP Responsibilities Banyan Global M&E Specialist and project manager (Matthew Griffith) has the overall responsibility of providing technical clarity and guidance for M&E data collection and reporting processes. Mr. Griffith will be supported by Dr. Gary Woller as a methodology advisor. Banyan Global will improve the M&E capacity of the project team and the GRMA through training, mentoring and coaching. Additionally, Banyan Global will report on performance indicators as outlined in the PMP. Survey enumerators and the Banyan Global project team will be responsible for data collection, with assistance from GRMA. The scope of data collected will cover all the IR areas, including the program areas to be supported. The SMH project team in Ghana, with support from GRMA and oversight by the project M&E team, will have the responsibility of ensuring regular and continuous data collection, dissemination, and use in a timely manner. It will also be responsible for ensuring data quality, providing data and regular feedback and for complying with agreed-on reporting requirements. Performance Monitoring Plan (Draft August 5, 2015) 27 Annex 1: Business Formalization Score (BFS) Overview and Purpose One of the primary goals of the Saving Maternity Homes project is to increase the ability of a PMH to sustain its operations as a going concern. The ability of any business to sustain its operations are determined by a number of factors. Several of these factors are external to the business itself, such as the level of community awareness of the benefits of health services and the level of market competition of PMHs, chemical shops, or other actors. Several of these factors are internal, but difficult to measure, such as the managerial capacity of the owner and/or manager, and the quality of customer service of the staff. In an effort to identify a measurable approach to understanding of the ability of a PMH to sustain its operations as a going concern, the SMH project has chosen to focus formalization. The relevant literature indicates that formalization has a direct impact on business sustainability. One paper focused on SMEs in Vietnam presents a variety of findings that emphasize the importance of formalization, including increase in profits and investments and better access to credit: “There is causal evidence that becoming more formalized leads to an increase in profits and investments, and a decrease in the use of casual labor thereby improving contract conditions for workers. Moreover, formal firms have better access to credit.”13 A recent World Bank review of the literature focused on formalization also emphasizes the correlation between informality and lower levels of growth and productivity.14 At the point of the baseline, the project will calculate a score for each PMH that will allow measuring change over time, and therefore quantifying the effect of the project’s business support activities. The project expects this measurement to be particularly useful in respect to PMHs that have undergone full transformation. Additionally, the BFS will assist the project to identify the areas of business and financial management that are the weakest for each PMH, which will inform future project assistance. Defining Formalization In the process of designing the BFS, the project reviewed the factors commonly used in defining informality and formalization. The World Bank review lists registration with a government agency, size of the business (or activity), whether personal capital is separate from business capital, and the keeping of regular accounts as common factors in defining informality. In one prominent paper, the authors argue for the advantages of defining formality and informality along a continuum, listing the size of the business, registration, the existence of financial statements, mobility of workplace, and access to bank credit as important criteria.15 Finally, as part of the development of the project’s approach to understanding both internal and external formalization, the project reviewed The Five Stages of Small Business Growth, by Neil C. Churchill and 13 The benefits of formalization: evidence from Vietnamese SMEs: John Rand and Nina Torm (draft 2010) 14 Informal Economy and the World Bank, by Nancy Benjamin with Kathleen Beegle, Francesca Recanatini, and Massimiliano Santini, Policy Research Working Paper 6888, May 2014 15 Informality, productivity, and enforcement in West Africa: A firm level analysis (World International Policy Center Working Paper Series Number 100, June 2010), Nancy Benjamin and Ahmadou Aly Mbaye Monitoring and Evaluation Plan, Year 1 28 Virginia L. Lewis. 16 This often-cited paper outlines typical progressive stages of business development, and informed the project’s thinking on how internal formalization increases as businesses move through the stages. The design of the BFS score builds on this research, as well as the SMH team’s own expertise in the areas of enterprise development and business strengthening for private health providers in Ghana. BFS Design The BFS is designed to incorporate factors that set the foundation for survival and growth of a business, including those related to: 1) External formalization of the business such as legal organization of a business, registration with the tax authority, etc. and 2) Internal formalization of business processes and operations such as reliability of business records and information, level of professionalization of business functions, and capacity to understand and manage the business. External formalization relates to the degree of participation of a business in the formal mainstream economy while the internal formalization describes the extent to which a business has developed internal organizational processes and procedures that allow it to function efficiently. There is a relation between external and internal formalization: businesses that start off in an informal way and remain so for some time reach a certain level of operations that necessitates internal formalization to grow the business. Both types are equally important and will be assumed to have equal weights in the construction of the Business Formalization Score. BFS includes clearly verifiable factors that reflect the two aspects of formalization, and will offer scores for each as well as the cumulative score for the business. BFS Factors The following factors will be considered for the BFS. Factors Related to External Formalization: • Health Facility Regulatory Agency (HEFRA) certification is issued to the private maternity home after the Agency is satisfied with all inspection reports. Without this certification, a PMH is in breach of the law and its operating deemed illegal. • Nurses and Midwives Council (NMC) certification ensures that the primary midwife’s qualification to practice midwifery is current, as defined by NMC guidelines. Without a current renewal of practice license from NMC, a PMH cannot secure either HEFRA or NHIA certifications. • National Health Insurance Authority (NHIA) certification is necessary if the home is to benefit from the opportunities presented by the NHIS and its revenue streams. The underserved communities, which mostly constitute the target market for the PMHs, overwhelmingly rely on the NHIS to seek health care. With the current competition facing private maternity homes from public health facilities, private hospitals and clinics, PMHs without the NHIA certification will often be less competitive, and vulnerable to financial challenges and closure. • Registration with Ghana Revenue Authority is required for any business in Ghana, and is key if one is to avoid legal penalties deriving from the non-payment of taxes. These penalties include fines, and potentially closure of facility. • Registration status as a business with Registrar General Department is a key step toward formalization, without which PMHs will be unable to progress with other certifications or engaging with professional services, such as the opening of a bank account. 16 Harvard Business Review, May 1983. Monitoring and Evaluation Plan, Year 1 29 Factors Related to Internal Formalization: • The practice of maintaining written contracts with partners and suppliers is an indicator of strong business relationships. Written contracts enable a PMH to make accurate financial plans based on agreed prices, availability and quality of needed supplies. • The practice of separating personal finances from business finances is an indicator of good financial governance, and evidence of existence of an arm’s length relationship between business and personal. PMHs that do not separate personal and business finances often demonstrate poor financial management, and a low level of awareness of the use of funds. • The practice of keeping funds in a bank account is a generally efficient financial management practice, and is especially important to facilities that receive out-of–pocket payments from clients. Additionally many PMHs receive direct debit/check payments through NHIA. • Size of the business (as defined by number of employees) is a common indicator of informality (most prominently by the International Labor Organization), with size relating to a business’s ability to incorporate systems, structures and functions appropriate to the stage of growth or maturity of the business, which requires employees. Most definitions of informality note that size itself is not sufficient to define informality, but should be incorporated as one factor among several.17 • The availability of financial records indicates the ability of the firm to track and review financial performance. These allow the business to make informed business decisions based on evidence. Data for BFS Factors Data for the BFS factors will be obtained through the baseline and endline surveys. For each factor, the PMH will be awarded a score of between 0 and 1, depending on their answer to the relevant question. The relevant survey questions, and the points available for each answer, that will gather the necessary data, are the following: Factors Related to External Formalization (5 points): 1. Is the facility’s certification with the Health Facilities Regulatory Agency (HEFRA) up to date? a. Not certified = score 0 b. Certified but certificate not sighted = score 0.5 c. Certified and certificate sighted but not current = score 0.5 d. Certified and certificate sighted and current = score 1 2. Is the facility’s certification of MIDWIFE with the Nurses and Midwives Council up to date? a. Not certified = score 0 b. Certified but certificate not sighted = score 0.5 c. Certified and certificate sighted but not current = score 0.5 d. Certified and certificate sighted and current = score 1 3. Is the facility currently accredited with NHIA? a. Not accredited = score 0 b. Accreditation outdated or withdrawn, or accreditation certificate sighted but not current = score 0.25 c. Claims to be accredited, but accreditation certificate not sighted = score 0.5 d. Accredited and accreditation certificate sighted and current = score 1 17 For a further discussion as size as a criterion in defining informality, see “Informality and the World Bank”, p8. Monitoring and Evaluation Plan, Year 1 30 4. Is the facility registered as a business with the Registrar General Department? a. Not registered = score 0 b. Registered but registration document not sighted = score 0.5 c. Registered and registration document sighted but not current = score 0.5 d. Registered and registration document sighted and current = score 1 5. Is the facility registered with the Ghana Revenue Authority (GRA)? a. Not registered = score 0 b. Registered but no evidence of tax registration sighted = score 0.5 c. Registered and evidence of tax registration sighted but not current = score 0.5 d. Registered and evidence of tax registration sighted and current = score 1 Factors Related to Internal Formalization (5 points): 6. Does the facility maintain written contracts or agreements with partners and suppliers? a. Does not maintain written contracts = score 0 b. Maintains written contracts, but example not sighted = score 0.5 c. Maintains written contracts, and example sighted = score 1 7. Does the business keep personal funds separate from business funds? a. No = score 0 b. Yes = score 1 8. Does the facility keep funds in a bank account (personal or business)? a. No bank account = score 0 b. Personal bank account = score 0.5 c. Business bank account = score 1 9. Does the facility keep financial statements (cash log, income statement, balance sheet)? a. Facility reports keeping none of the three = score 0 b. Facility reports keeping one of the three = score 0.33 c. Facility reports keeping two of the three = score 0.66 d. Facility reports keeping all three = score 1 10. Size of the business defined by number of full-time employees (clinical and non–clinical)? a. Number range:1-5 = score 0.25 b. Number range:6-10 = score 0.50 c. Number range11-20 = score 0.75 d. Number range 21 or above = score 1 Calculation and Interpretation of BFS Data on the factors related to business formalization will allow the SMH project to calculate several measures of formalization: Total BFS, Partial BFS and Relative BFS. Total BFS The Total BFS (BFST) is calculated as a sum of all the factor scores obtained through the ten survey questions listed above. The BFS score is a number between 0 and 10, with 0 denoting no formalization and 10 denoting full formalization. PMHs are divided into groups of low, moderate, high and very high overall levels of formalization according to the following value of BFST: Monitoring and Evaluation Plan, Year 1 31 Table I-2: Levels of Overall Formalization Levels of Formalization Total BFS (BFST ) Low ≤ 5.9 Moderate = 6.0 to 7.9 High = 8.0 to 8.9 Very High ≥ 9 Partial BFS Partial BFS (BFSP) is calculated separately for the external formalization (BFSE) and internal formalization (BFSI). Each partial BFS is calculated as the sum of the factors related to the particular aspect of business formalization. The BFSP score is a number between 0 and 5, with 0 meaning no formalization and 5 denoting full formalization in a particular category. PMHs are divided into groups of low, moderate, high and very high overall levels of formalization according to the following value of BFSP: Table I-3: Levels of Partial Formalization Levels of Partial Formalization Partial BFS (BFSP) Low ≤ 2.0 Moderate = 2.0 .2.9 High = 3.0 to 3.9 Very High ≥ 4 Relative BFS The relation of external factors and internal factors determine the type of formalization exhibited by the PMH. The Relative BFS (BFSR) is calculated as a ratio of BFSE and BFSI. The BFSR score is 1 when the external and internal formalization is at the same level; it is more than 1 if external formalization of a business is relatively more important, and less than 1 when, despite weaker external formalization, the business’s internal policies and procedures are formalized to a relatively higher degree. Table I-4: Relative Formalization Type of Formalization Relative BFS (BFSR) Externally Oriented > 1 Balanced = 1 Internally Oriented < 1 Performance Monitoring Plan (Draft August 5, 2015) 32 Annex 2: Indicators Table The following tables summarize the key indicators that will be used to measure progress against SMH Project results, including the data sources and timing/frequency of data collection. For more information on each indicator, please see the indicator reference sheets included in Annex 5. # Performance indicator Indicator definition Data Source Data Acquisition Analysis and Reporting Schedule/ Frequency By Whom Schedule/ Frequency By Whom Long-Term Goal: Increase in private maternity homes’ ability to sustain and grow health services in the underserved areas in Ghana Strategic Objective: PMHs sustained, MCH services expanded, and efficiency and quality improved. 1 Percentage of assisted PMHs with increased financial sustainability The percentage of assisted PMHs that demonstrate an increase in financial sustainability. Numerator: Number of PMHs receiving project assistance that improve their sustainability score on the BFS at the endline as compared to their score at the baseline. Denominator: Number of PMHs receiving project assistance. This indicator will be disaggregated based on sex of the owner and region. PMH Survey Baseline and endline SMH End of project SMH Monitoring and Evaluation Plan, Year 1 33 # Performance indicator Indicator definition Data Source Data Acquisition Analysis and Reporting Schedule/ Frequency By Whom Schedule/ Frequency By Whom 2 Percentage of assisted PMHs with improved quality of services18 Percentage of assisted PMHs showing improvements in their QAT score. Numerator: Number of PMHs receiving project assistance that improve their QAT score at the endline as compared to their score at the baseline. Denominator: Number of PMHs receiving project assistance. This indicator will be disaggregated based on sex of the owner and region. QAT Baseline and endline GRMA / SMH End of project GRMA / SMH 3 Number of assisted PMHs with improved technical efficiency Number of assisted PMHs showing improvements in technical efficiency. Efficiency will be determined using Data Envelopment Analysis (DEA). Data envelopment analysis is a non-parametric method in operations research used to empirically measure productive efficiency of decision-making units (DMUs). This indicator will be disaggregated based on sex of the owner and region. PMH Survey Baseline and endline SMH End of project SMH 18 Note: In previous versions of the MEP, this indicator was phrased as a number. We have changed it to a percentage for consistency with Indicator 1. Monitoring and Evaluation Plan, Year 1 34 # Performance indicator Indicator definition Data Source Data Acquisition Analysis and Reporting Schedule/ Frequency By Whom Schedule/ Frequency By Whom 4 Number of assisted PMHs that add new health services Number of assisted PMHs that add new health services. This indicator will be disaggregated based on sex of the owner, type of health service and region. QAT Baseline and endline SMH End of project SMH 5 Number of outpatient visits19 received by assisted PMHs that undergo an ownership/ management transformation Average number of outpatient visits per month over a period of 6 months, received by those assisted facilities that undergo an ownership and/or management transformation. This indicator will be disaggregated by sex of the owner, by type of business transfer model, by region, and by type of health service (MNCH, FP, other basic health services). PMH Survey, GHS returns Baseline and endline SMH End of project SMH IR1: PMH owners and managers ready for ownership and/or management change Sub IR 1.1: PMHs interested in and ready for transformation identified 6 Number of assisted PMHs expressing interest in ownership / management transformation Number of assisted PMHs who submit an Expression of Interest Form, which formally indicates an interest in exploring a management or ownership Expression of Interest Forms Quarterly SMH/ GRMA Quarterly SMH 19 Note: In previous versions of this document this indicator was phrased as “number of patient visits.” It is changed here to “number of outpatient visits” in order to increase clarity regarding what the project actually intends to measure. Monitoring and Evaluation Plan, Year 1 35 # Performance indicator Indicator definition Data Source Data Acquisition Analysis and Reporting Schedule/ Frequency By Whom Schedule/ Frequency By Whom transformation. This indicator will be disaggregated based on sex of the owner, region, and age of the registered midwife who submits the Form. Sub IR 1.2: Capacity of PMHs for ownership/management transformation strengthened. 7 Number of PMH owners who have received pre￾transformation technical assistance Number of PMH owners who receive at least one visit from the project during which the owner received TA focused on required preparations for investment. All PMHs counted against this indicator will have expressed an interest in ownership/management transformation (See Indicator 7.) This indicator will be disaggregated based on sex of the owner and region. Technical Assistance Record Forms Routinely SMH Routinely SMH 8 Percentage of training participants who attain a score of 75% or higher on training post￾tests This indicator measures the quality and effectiveness of the trainings offered by SMH to PMHs. Numerator: Number of training participants who attain a score of 75% or higher on the training Project records Routinely SMH Routinely SMH Monitoring and Evaluation Plan, Year 1 36 # Performance indicator Indicator definition Data Source Data Acquisition Analysis and Reporting Schedule/ Frequency By Whom Schedule/ Frequency By Whom post-test. Denominator: Total number of training participants. This indicator will be disaggregated based on type of training (Business Continuity and Succession Planning (BCIP), Business Strengthening Support (BSS), or Clinical Capacity Building (CCB)), and sex of the training participant. 9 Number of unique PMH facilities trained by the SMH project Number PMH facilities who send at least one representative to at least one of the project’s training courses. This indicator will be disaggregated based on type of training (Business Continuity and Succession Planning (BCSP), Business Strengthening Support (BSS), or Clinical Capacity Building (CCB)), sex of the owner, and region. Trainee registration forms Routinely SMH Routinely SMH 10 Number of participants in SMH training courses Number of participants trained, where participant is defined as an individual participating in a training. One individual would Trainee registration forms Routinely SMH Routinely SMH Monitoring and Evaluation Plan, Year 1 37 # Performance indicator Indicator definition Data Source Data Acquisition Analysis and Reporting Schedule/ Frequency By Whom Schedule/ Frequency By Whom count as two participants if that person participated in two separate trainings. This indicator will be disaggregated based on type of training (Business Continuity and Succession Planning (BCIP), Business Strengthening Support (BSS), or Clinical Capacity Building (CCB)), sex of the owner, and geographic location. IR2: PMHs under new ownership / management continue health service delivery Sub-IR 2.1: PMH ownership/management transformations brokered 11 Number of assisted active PMHs who complete an ownership / management transformation Number of active PMH facilities who receive project assistance, that complete the process of an ownership and/or management transformation. This indicator will be disaggregated based on sex of the owner, region, and type of transformation. Project records Annually SMH Annually SMH Monitoring and Evaluation Plan, Year 1 38 # Performance indicator Indicator definition Data Source Data Acquisition Analysis and Reporting Schedule/ Frequency By Whom Schedule/ Frequency By Whom 1220 Number of inactive PMHs re-opened with new owners or managers Number of inactive PMH facilities that are re-opened and begin delivering health care services after obtaining new ownership/ management. The new owners/managers of these facilities will have received assistance from the SMH project. This indicator will be disaggregated based on sex of the owner and region. Project records Annually SMH Annually SMH 13 Number of investment proposals for PMH transformations prepared Number of transformative investment proposals prepared as a result of project assistance. These proposals could be prepared by existing PMHs seeking new investors / managers, by investors seeking to purchase PMH facilities, or by new / younger midwives seeking to open their own facilities. This indicator will be disaggregated based on sex of Project records Routinely SMH Routinely SMH 20 Note: In previous versions of this document, there was an indicator following this one, focused on “Number of PMHs opened.” After the conclusion of the baseline assessment and the initial BCSP trainings, the project believes that the opening of brand new PMHs (as opposed to the re-opening of currently close PMHs) is beyond the scope of transformations (sales, leasing, new partnerships, new management) that are the focus of this project. We look forward to USAID’s thoughts on this topic. Monitoring and Evaluation Plan, Year 1 39 # Performance indicator Indicator definition Data Source Data Acquisition Analysis and Reporting Schedule/ Frequency By Whom Schedule/ Frequency By Whom the owner and region. 14 Number of new / younger midwives interested in ownership and/or management of PMHs. 21 A new / younger midwife is defined as an individual who is below the age of 55. This indicator refers to the number of these individuals who complete an “Expression of Interest” form that formally declares their interest in receiving project assistance to take on the ownership/management duties of PMHs in target regions. This indicator will be disaggregated based on sex of the owner and region. Expression of Interest Forms Quarterly SMH Quarterly SMH 15 Percentage of PMHs that have a neutral or positive attitude towards the idea of ownership/management transformation. This indicator refers to the overall attitudes of PMH owners towards the idea of an ownership/ management transformation. Numerator: Number of current PMH owners indicating a neutral PMH Survey Baseline and endline SMH Baseline and endline SMH 21 In a recent USAID review, the reviewer inquired as to whether this indicator is necessary. The SMH team is of the opinion that this indicator is a useful way to measure the project’s prospecting and business development efforts, as well as the interest in transformation from this younger group of midwives. Therefore, our preference is that we retain this indicator. Monitoring and Evaluation Plan, Year 1 40 # Performance indicator Indicator definition Data Source Data Acquisition Analysis and Reporting Schedule/ Frequency By Whom Schedule/ Frequency By Whom or positive attitude on a Likert scale to a statement about specific types of ownership/ management transformation. Denominator: Number of PMH facilities responding to the PMH survey. This indicator will be disaggregated based on sex of the owner, region, and type of ownership/management transformation (selling the facility, leasing the facility, bringing on a partner, or bringing on a manager). Sub-IR 2.2: Funding Provided and Leveraged by the Project 16 Number of assisted PMHs that receive seed funding in the form of SMH small grants The total number of PMHs that receive seed funding from the SMH project in a given year. This indicator will be disaggregated based on sex of the owner and region. Project records Annually SMH Annually SMH 17 Amount of seed funding provided by the SMH Amount of grant funding in US$ provided to the PMHs by the Project records Annually SMH Annually SMH Monitoring and Evaluation Plan, Year 1 41 # Performance indicator Indicator definition Data Source Data Acquisition Analysis and Reporting Schedule/ Frequency By Whom Schedule/ Frequency By Whom project in the form of small grants22 SMH project for transformation in a given year. This indicator will be disaggregated based on sex of the owner and geographic location. 18 Number of assisted PMHs that receive funds from non-project sources Number of PMHs receiving project assistance that receive funds from non-project sources for transformation in a given year. This indicator will be disaggregated based on sex of the owner and region. Project records Annually SMH Annually SMH 19 Amount of funding that assisted PMHs receive from non-project sources Amount of funding from non￾project sources received by project-assisted PMHs for the transformation of PMHs resulting from the project interventions (total amount, average amount per PMH, leverage ratio between seed funding and additional mobilized Project records Annually SMH Annually SMH 22 In a recent review from USAID, the reviewer inquired as to whether this indicator is necessary. The project is still deliberating on this point, and has not yet reached a conclusion. Monitoring and Evaluation Plan, Year 1 42 # Performance indicator Indicator definition Data Source Data Acquisition Analysis and Reporting Schedule/ Frequency By Whom Schedule/ Frequency By Whom funding). This indicator will be disaggregated based on sex of the owner and region. IR3: Business and programmatic capacity of PMHs and GRMA increased Sub-IR 3.1: Capacity of PMHs to sustain, increase and improve service delivery strengthened 20 Percentage of assisted, non-accredited PMHs applying for NHIA accreditation that successfully become accredited23 This indicator provides information on the rate at which project-assisted PMHs are successfully participating in Ghana’s National Health Insurance Scheme. Numerator: Number of non￾accredited project-assisted PMHs that receive a grade that is sufficient to receive NHIA accreditation. Denominator: Non-accredited project-assisted PMHs that apply to NHIA to become accredited. PMH Survey and NHIA Baseline and endline SMH Endline SMH 23 Note: In previous versions of this document, this indicator was structured to include both unaccredited facilities that achieve accreditation, and accredited facilities that improve their score. It is here changed to focus only on unaccredited facilities achieving accreditation for the sake of simplicity, and feasibility of tracking and analysis. Additionally, the frequency of collecting and reporting on this indicator was changed from annually to baseline/endline, in recognition of the changing NHIA accreditation schedule, as well as the relatively short project duration. Monitoring and Evaluation Plan, Year 1 43 # Performance indicator Indicator definition Data Source Data Acquisition Analysis and Reporting Schedule/ Frequency By Whom Schedule/ Frequency By Whom This indicator will be disaggregated based on sex of the owner and region. 21 Percentage of assisted PMHs that receive or maintain NHIA accreditation after transformation This indicator measures the effect of transformation on NHIA accreditation among PMHs. Numerator: Number of assisted PMHs that complete transformation AND that receive or maintain NHIA accreditation after transformation is complete. Denominator: Number of assisted PMHs that complete transformation. This indicator will be disaggregated based on sex of the owner and region. NHIA, Project records Baseline and endline NHIA and SMH Baseline and endline SMH Sub-IR 3.2: Increased capacity of PMHs to manage their businesses efficiently Monitoring and Evaluation Plan, Year 1 44 # Performance indicator Indicator definition Data Source Data Acquisition Analysis and Reporting Schedule/ Frequency By Whom Schedule/ Frequency By Whom 22 Number of one-on-one business training events conducted24 Number of one-on-one training and one-on-one capacity building events provided by the project to participating PMHs. Project records Quarterly SMH Quarterly SMH 2325 Percentage of assisted PMHs maintaining core financial documents (cash log, profit and loss statement, or income statement) This indicator refers to the practice of keeping up to date fundamental financial documents. Numerator: Project-assisted PMHs that report having one of these three documents in place, and up to date. Denominator: All project￾assisted PMHs. This indicator will be disaggregated based on sex of the owner and region. PMH Survey Baseline and endline SMH Baseline and endline SMH Sub-IR 3.3: Management and programmatic capacity of GRMA increased. 24 Note: The words “one-on-one” were added to the phrasing of this indicator, in order to make it more clear that the indicator focuses on one-on-one training events, rather than the group training events (such as BCSP and CCB). 25 Note: In previous versions of this document there was an indicator, immediately following this indicator, that has been removed. That indicator was: Percentage of participating PMHs that have registered their businesses with the Ghana Registrar General Department. In conducting the baseline, we found that 93% of PMHs in the target regions were already registered. Thus, the project came to the conclusion that focusing on an indicator that was already 93% achieved was not the best use of project resources and effort. If USAID would prefer the indicator be included, we are happy to do that. Monitoring and Evaluation Plan, Year 1 45 # Performance indicator Indicator definition Data Source Data Acquisition Analysis and Reporting Schedule/ Frequency By Whom Schedule/ Frequency By Whom 24 Number of GRMA staff and volunteers who receive capacity building assistance Number of individuals who are working for GRMA, either as paid staff or as volunteers, who receive capacity building assistance. Project records Annually SMH Annually SMH Performance Monitoring Plan (Draft August 5, 2015) 46 Annex 3: Monitoring and Evaluation Workplan: Year 1 Task Year 1 O N D J F M A M J J A S Finalize M&E Plan Baseline Design Survey Pilot Hire Enumerators Train enumerators Conduct surveys Review and tabulate data Analyze data Summarize the outcomes Conduct OR Prepare reports Organize a dissemination event Collect periodic data Monitoring and Evaluation Plan, Year 1 47 Annex 4: Indicator Reference Sheets USAID Performance Indicator Reference Sheet Name of Result Measured (Goal, DO, IR, sub-IR, Project Purpose, Project Output, etc.): Goal: Increase private maternity homes’ ability to sustain and grow health services in the underserved areas of Ghana. Name of Indicator: 1. Percentage of assisted PMHs with increased financial sustainability Is this a Performance Plan and Report indicator? No _X__ Yes ____, for Reporting Year(s) _________ If yes, link to foreign assistance framework: DESCRIPTION Precise Definition(s): Private maternity homes (PMHs); Business Formalization Score (BFS) = An index defined by the project that is comprised of specific indicators that relate to business formalization, which the project is using as a proxy for business and financial sustainability (See previous PIRS). Unit of Measure: The percentage of assisted PMHs that demonstrate an increase in business and financial sustainability, as measured by the BFS (See previous PIRS). Numerator: Number of PMHs receiving project assistance that improve their sustainability score on the BFS on the endline as compared to their score at the baseline. Denominator: Number of PMHs that receive project assistance. Disaggregated by: This indicator will be disaggregated based on sex of the owner and geographic location of the facility. Rationale or Justification for indicator (optional): PLAN FOR DATA COLLECTION Data Source: PMH Survey Method of data collection and construction: The project will interview PMH owners and managers using the PMH survey questions. The questions have been built into the formhub.org mobile data collection platform. Reporting Frequency: Baseline (beginning of project) and endline (end of project) Individual(s) responsible: Bernard Akotey, SMH Project Team Lead; Matthew Griffith, Banyan Global Program Manager DATA QUALITY ISSUES Dates of Previous Data Quality Assessments and name of reviewer: N/A Date of Future Data Quality Assessments (optional): Known Data Limitations: There is a risk that respondents will not have the documentation to back up their reported answers regarding the different factors that go into the BFS. TARGETS AND BASELINE Baseline timeframe (optional): Baseline findings submitted in June, 2015 Rationale for Targets (optional): LOP target: 20%. Rationale: Recognizing the mixed record that business training interventions have had in improving business practices, the project still believes it can have an impact. At baseline, facilities displayed significant room for improvement in the area of internal formalization, specifically in terms of keeping written contracts (only 8% of PMHs were found to be doing this at baseline), and keeping financial documents (at baseline, 58% kept an income statement; 47% kept a cash log; 39% kept a balance sheet). Considering both the difficulty in effecting change in these areas, and the substantial room for improvement, the project expects that 20% of assisted facilities will improve over the life of project. CHANGES TO INDICATOR Changes to indicator: July, 2015: Added an LOP target and rationale. Other Notes (optional): THIS SHEET LAST UPDATED ON: July 31, 2015 Monitoring and Evaluation Plan, Year 1 48 USAID Performance Indicator Reference Sheet Name of Result Measured (Goal, DO, IR, sub-IR, Project Purpose, Project Output, etc.): Goal: Increase private maternity homes’ ability to sustain and grow health services in the underserved areas of Ghana. Name of Indicator: 2. Percentage of assisted PMHs with improved quality of services. Is this a Performance Plan and Report indicator? No _X__ Yes ____, for Reporting Year(s) _________ If yes, link to foreign assistance framework: DESCRIPTION Precise Definition(s): Private maternity homes (PMHs); Quality Assistance Tool (QAT) = A tool developed jointly between the SMH project and GRMA that captures the level of quality at which a specific PMH is delivering the various healthcare services. The QAT focuses on four areas: Emergency response; Quality assurance; Safe motherhood; Advocacy/Health Promotion Numerator: Number of PMHs receiving project assistance that improve their QAT score at the endline as compared to their score at baseline. Denominator: Number of PMHs receiving project assistance. Unit of Measure: Number of assisted PMHs showing improvements in their QAT score. Disaggregated by: This indicator will be disaggregated based on sex of the owner and geographic location of the facility. Rationale or Justification for indicator (optional): PLAN FOR DATA COLLECTION Data Source: Quality Assessment Tool Method of data collection and construction: GRMA will interview PMH owners and managers using the QAT. GRMA will then record that data, and the SMH team will also take record of the outcomes of the QAT for storage in the project’s M&E database. Reporting Frequency: Baseline (beginning of project) and endline (end of project) Individual(s) responsible: GRMA; Bernard Akotey, SMH Project Team Lead; Matthew Griffith, Banyan Global Program Manager DATA QUALITY ISSUES Dates of Previous Data Quality Assessments and name of reviewer: N/A Date of Future Data Quality Assessments (optional): Known Data Limitations: TARGETS AND BASELINE Baseline timeframe (optional): Baseline findings submitted in June, 2015 Rationale for Targets (optional): LOP target: 33%: There are two types of assistance available through the SMH project designed to increase quality of services: the peer support visits, and the Clinical Capacity Building (CCB) training. The total number of PMHs that will receive the training is 146 (with the substantial majority of these also receiving the PSV). While the project regards these as effective interventions, they are occurring at the start of the project. Thus, while the project believes some facilities will apply and maintain improved practices over the life of the project, others will revert back to their old quality practices. Thus, the project proposes a target of 33% of assisted facilities. CHANGES TO INDICATOR Changes to indicator: July, 2015: Added an LOP target and rationale. Other Notes (optional): THIS SHEET LAST UPDATED ON: July 31, 2015 Monitoring and Evaluation Plan, Year 1 49 USAID Performance Indicator Reference Sheet Name of Result Measured (Goal, DO, IR, sub-IR, Project Purpose, Project Output, etc.): Goal: Increase private maternity homes’ ability to sustain and grow health services in the underserved areas of Ghana. Name of Indicator: 3. Number of assisted PMHs with improved technical efficiency. Is this a Performance Plan and Report indicator? No _X__ Yes ____, for Reporting Year(s) _________ If yes, link to foreign assistance framework: DESCRIPTION Precise Definition(s): Private maternity homes (PMHs); Efficiency will be determined using Data Envelopment Analysis (DEA). Data envelopment analysis is a non-parametric method in operations research used to empirically measure productive efficiency of decision-making units (DMUs). The methodology is also used for benchmarking in operations management, where a set of measures is selected to benchmark the performance of manufacturing and service operations. Unit of Measure: Number of assisted PMHs showing improvements in technical efficiency. Disaggregated by: This indicator will be disaggregated based on sex of the owner and geographic location of the facility. Rationale or Justification for indicator (optional): PLAN FOR DATA COLLECTION Data Source: PMH Survey Method of data collection and construction: The project will interview PMH owners and managers using the PMH survey questions. The questions have been built into the formhub.org mobile data collection platform. Reporting Frequency: Baseline (beginning of project) and endline (end of project) Individual(s) responsible: GRMA; Bernard Akotey, SMH Project Team Lead; Matthew Griffith, Banyan Global Program Manager DATA QUALITY ISSUES Dates of Previous Data Quality Assessments and name of reviewer: N/A Date of Future Data Quality Assessments (optional): Known Data Limitations: TARGETS AND BASELINE Baseline timeframe (optional): Baseline findings submitted in June, 2015 Rationale for Targets (optional): LOP target: 25. Rationale: At baseline 27 facilities were technically efficient, meaning they scored a 100% in terms of technical efficiency, and therefore cannot score higher. While the project does believe that the project’s interventions in terms of business strengthening, quality of services improvements, and transformations will have an impact on efficiency, it is difficult to predict how much of an impact. To be conservative, the project proposes a target of 25 facilities improving their technical efficiency from baseline to endline. CHANGES TO INDICATOR Changes to indicator: July, 2015: Added an LOP target and rationale. Other Notes (optional): THIS SHEET LAST UPDATED ON: July 31, 2015 Monitoring and Evaluation Plan, Year 1 50 USAID Performance Indicator Reference Sheet Name of Result Measured (Goal, DO, IR, sub-IR, Project Purpose, Project Output, etc.): Goal: Increase private maternity homes’ ability to sustain and grow health services in the underserved areas of Ghana. Name of Indicator: 4. Number of assisted PMHs that add new health services Is this a Performance Plan and Report indicator? No _X__ Yes ____, for Reporting Year(s) _________ If yes, link to foreign assistance framework: DESCRIPTION Precise Definition(s): Private maternity homes (PMHs); Health services include services that fall within a PMH’s legal and ethical mandate to provide. The list of health services under consideration in the project’s data gathering can be found in the QAT. Unit of Measure: Number of assisted PMHs that offer add new health services Disaggregated by: This indicator will be disaggregated based on sex of the owner and geographic location of the facility. Rationale or Justification for indicator (optional): PLAN FOR DATA COLLECTION Data Source: QAT Method of data collection and construction: GRMA will interview PMH owners and managers using the QAT. GRMA will then record that data, and the SMH team will also take record of the outcomes of the QAT for storage in the project’s M&E database. Reporting Frequency: Baseline (beginning of project) and endline (end of project) Individual(s) responsible: GRMA; Bernard Akotey, SMH Project Team Lead; Matthew Griffith, Banyan Global Program Manager DATA QUALITY ISSUES Dates of Previous Data Quality Assessments and name of reviewer: N/A Date of Future Data Quality Assessments (optional): Known Data Limitations: TARGETS AND BASELINE Baseline timeframe (optional): Baseline findings submitted in June, 2015 Rationale for Targets (optional): LOP target: 25. Rationale: At the Clinical Capacity Building (CCB) trainings, participants expressed interest in adding services that will increase patient flow and yield a least a modest financial benefit. Additionally, NHIA capitation may require facilities to add on new services in order for NHIS clients to select their facilities as preferred primary provider. Much of the achievement against this indicator is expected to come from the 20 PMHs to be transformed, which are expected to add new services given their desired to be profitable at the earliest opportunity. However, since the project doesn’t expect these factors to affect all PMHs in the target regions, the project proposes a conservative LOP target of just over 17% of assisted facilities, or 25. CHANGES TO INDICATOR Changes to indicator: July, 2015: Added an LOP target and rationale. Other Notes (optional): THIS SHEET LAST UPDATED ON: July 31, 2015 Monitoring and Evaluation Plan, Year 1 51 USAID Performance Indicator Reference Sheet Name of Result Measured (Goal, DO, IR, sub-IR, Project Purpose, Project Output, etc.): Goal: Increase private maternity homes’ ability to sustain and grow health services in the underserved areas of Ghana. Name of Indicator: 5. Number of outpatient visits received by assisted PMHs that undergo an ownership/ management transformation Is this a Performance Plan and Report indicator? No _X__ Yes ____, for Reporting Year(s) _________ If yes, link to foreign assistance framework: DESCRIPTION Precise Definition(s): Private maternity homes (PMHs); Outpatient visits refers to discrete visits by an individual seeking outpatient care at the facility. One individual can make multiple visits within the period under review. Transformation is defined as occurring when; 1) the PMH facility is legally sold to a new owner or investor; 2) the existing owner brings on a partner who has both an ownership stake and an active management role; 3) the existing owner retains ownership, but appoints a manager with a job description that gives this person final authority over all operational aspects of facility management; or 4) the existing owner retains ownership and leases the facility to a new manager/operator. Unit of Measure: Average number of outpatient visits per month over a period of 6 months, received by those assisted facilities that undergo an ownership and/or management transformation. Disaggregated by: This indicator will be disaggregated by sex of the owner, by type of business transfer model, by geographic location of the facility, and by type of health service (MNCH, FP, other basic health services). Rationale or Justification for indicator (optional): PLAN FOR DATA COLLECTION Data Source: PMH Survey, GHS returns Method of data collection and construction: The project will interview PMH owners and managers using the PMH survey questions. The questions have been built into the formhub.org mobile data collection platform. The project will also use GHS returns to triangulate data from the PMH survey. Reporting Frequency: Baseline (beginning of project) and endline (end of project) Individual(s) responsible: GRMA; Bernard Akotey, SMH Project Team Lead; Matthew Griffith, Banyan Global Program Manager DATA QUALITY ISSUES Dates of Previous Data Quality Assessments and name of reviewer: N/A Date of Future Data Quality Assessments (optional): Known Data Limitations: TARGETS AND BASELINE Baseline timeframe (optional): Baseline findings submitted in June, 2015 Rationale for Targets (optional): LOP target: 550. Rationale: At baseline, the average number of monthly outpatient visits for all facilities was 505. In setting this target, the project assumes incremental growth of about 10% above the baseline average of 505. The project believes that facilities that undergo transformation will add new services, as well improve service quality. Thus, one will expect the PMHs to be attractive to patients who hitherto will not have accessed the facility, and their average monthly outpatient client visits will go up. CHANGES TO INDICATOR Changes to indicator: July, 2015: Added an LOP target and rationale. Other Notes (optional): THIS SHEET LAST UPDATED ON: July 31, 2015 Monitoring and Evaluation Plan, Year 1 52 USAID Performance Indicator Reference Sheet Name of Result Measured (Goal, DO, IR, sub-IR, Project Purpose, Project Output, etc.): Sub IR 1.1: PMHs interested in and ready for transformation identified Name of Indicator: 6. Number of assisted PMHs expressing interest in ownership / management transformation Is this a Performance Plan and Report indicator? No _X__ Yes ____, for Reporting Year(s) _________ If yes, link to foreign assistance framework: DESCRIPTION Precise Definition(s): Private maternity homes (PMHs); Transformation is defined as occurring when; 1) the PMH facility is legally sold to a new owner or investor; 2) the existing owner brings on a partner who has both an ownership stake and an active management role; 3) the existing owner retains ownership, but appoints a manager with a job description that gives this person final authority over all operational aspects of facility management; or 4) the existing owner retains ownership and leases the facility to a new manager/operator. Unit of Measure: Number of assisted PMHs who submit an Expression of Interest Form, which formally indicates an interest in exploring a management or ownership transformation. Disaggregated by: This indicator will be disaggregated based on sex of the owner, geographic location, and age of the registered midwife who submits the Form. Rationale or Justification for indicator (optional): PLAN FOR DATA COLLECTION Data Source: Expression of Interest Forms Method of data collection and construction: The project and GRMA will collect Expression of Interest Forms as they are submitted, and feed summary data from these forms into the project’s central database. Reporting Frequency: Quarterly Individual(s) responsible: GRMA; Bernard Akotey, SMH Project Team Lead; Matthew Griffith, Banyan Global Program Manager DATA QUALITY ISSUES Dates of Previous Data Quality Assessments and name of reviewer: N/A Date of Future Data Quality Assessments (optional): Known Data Limitations: TARGETS AND BASELINE Baseline timeframe (optional): Baseline findings submitted in June, 2015 Rationale for Targets (optional): LOP target: 46. Rationale: The baseline assessment found varying levels of interest for the four different types of transformations. Seventy-six percent of respondents indicated interest in bringing on a manager, while 69% of respondents indicated interest in bringing on a partner. Forty percent of respondents were interested in the possibility of leasing out the facility, while only 15% were interested in the idea of one day selling the facility. These findings indicate that, while only a few facilities might be interested in sales or leasing, there is greater interest in management contracts and partnerships. However, expressing interest and expending the effort to initiate the process (submitting the EOI) are different steps. Thus, we set the target for this indicator at just under one-third of assisted facilities, at 46. CHANGES TO INDICATOR Changes to indicator: July, 2015: Added an LOP target and rationale. Other Notes (optional): THIS SHEET LAST UPDATED ON: July 31, 2015 Monitoring and Evaluation Plan, Year 1 53 USAID Performance Indicator Reference Sheet Name of Result Measured (Goal, DO, IR, sub-IR, Project Purpose, Project Output, etc.): Sub IR 1.2: Capacity of PMHs for ownership/management transformation strengthened. Name of Indicator: 7. Number of PMH owners who receive pre-transformation technical assistance Is this a Performance Plan and Report indicator? No _X__ Yes ____, for Reporting Year(s) _________ If yes, link to foreign assistance framework: DESCRIPTION Precise Definition(s): Private Maternity Home (PMH); Transformation is defined as occurring when; 1) the PMH facility is legally sold to a new owner or investor; 2) the existing owner brings on a partner who has both an ownership stake and an active management role; 3) the existing owner retains ownership, but appoints a manager with a job description that gives this person final authority over all operational aspects of facility management; or 4) the existing owner retains ownership and leases the facility to a new manager/operator. Technical assistance as used includes individual technical assistance focused on deal structuring, needs analysis, business plan support, and other pre-transformation assistance. Unit of Measure: Number of PMH owners who receive at least one visit from the project during which the owner received TA focused on required preparations for investment. All PMHs counted against this indicator will have expressed an interest in ownership/management transformation. Disaggregated by: This indicator will be disaggregated based on sex of the owner and geographic location of the facility. Rationale or Justification for indicator (optional): PLAN FOR DATA COLLECTION Data Source: Technical Assistance Record Forms Method of data collection and construction: The project will collect Technical Assistance Record Forms as the technical assistance moves forward, and feed summary data from these Forms into the project’s central database. Reporting Frequency: Routinely Individual(s) responsible: Bernard Akotey, SMH Project Team Lead; Matthew Griffith, Banyan Global Program Manager DATA QUALITY ISSUES Dates of Previous Data Quality Assessments and name of reviewer: N/A Date of Future Data Quality Assessments (optional): Known Data Limitations: TARGETS AND BASELINE Baseline timeframe (optional): Baseline findings submitted in June, 2015 Rationale for Targets (optional): LOP target: 33. Rationale: This indicator is a restatement of the number of facilities that ascend to tier 3 of project assistance. Thus, we set the target for this indicator equal to the target for tier 3 support, at 33 facilities. CHANGES TO INDICATOR Changes to indicator: July, 2015: Added an LOP target and rationale. Other Notes (optional): THIS SHEET LAST UPDATED ON: July 31, 2015 Monitoring and Evaluation Plan, Year 1 54 USAID Performance Indicator Reference Sheet Name of Result Measured (Goal, DO, IR, sub-IR, Project Purpose, Project Output, etc.): Sub IR 1.2: Capacity of PMHs for ownership/management transformation strengthened. Name of Indicator: 8. Percentage of training participants who attain a score of 75% or higher on training post-tests Is this a Performance Plan and Report indicator? No _X__ Yes ____, for Reporting Year(s) _________ If yes, link to foreign assistance framework: DESCRIPTION Precise Definition(s): “Participant” is defined as an individual participating in a training. One individual would count as two participants if that person participated in two separate trainings. Training post-tests are evaluations administered to participants upon completion of the training to gauge their level of knowledge acquisition. Unit of Measure: Numerator: Number of training participants who attain a score of 75% or higher on the training post￾test. Denominator: Total number of training participants. Disaggregated by: This indicator will be disaggregated based on type of training (Business Continuity and Succession Planning (BCIP), Business Strengthening Support (BSS), or Clinical Capacity Building (CCB)), sex of the owner, and geographic location. Rationale or Justification for indicator (optional): This indicator measures the quality and effectiveness of the trainings offered by SMH to PMHs. PLAN FOR DATA COLLECTION Data Source: Project records Method of data collection and construction: The project will record post-test scores for each training administered by the project. Reporting Frequency: Routinely Individual(s) responsible: Bernard Akotey, SMH Project Team Lead; Matthew Griffith, Banyan Global Program Manager DATA QUALITY ISSUES Dates of Previous Data Quality Assessments and name of reviewer: N/A Date of Future Data Quality Assessments (optional): Known Data Limitations: TARGETS AND BASELINE Baseline timeframe (optional): Baseline findings submitted in June, 2015 Rationale for Targets (optional): LOP target for BCSP training: 81%. Rationale: This activity is concluded, and so this target is set equal to the achievement. LOP target for BSS training: 80%. Rationale: The project anticipates a similar level of achievement to that of the BCSP training. LOP target for CCB training: 27%. Rationale: This activity is concluded, and so the target is set equal to the achievement. CHANGES TO INDICATOR Changes to indicator: July, 2015: Added an LOP target and rationale. Other Notes (optional): THIS SHEET LAST UPDATED ON: July 31, 2015 Monitoring and Evaluation Plan, Year 1 55 USAID Performance Indicator Reference Sheet Name of Result Measured (Goal, DO, IR, sub-IR, Project Purpose, Project Output, etc.): Sub IR 1.2: Capacity of PMHs for ownership/management transformation strengthened. Name of Indicator: 9. Number of unique PMH facilities trained by the SMH project Is this a Performance Plan and Report indicator? No _X__ Yes ____, for Reporting Year(s) _________ If yes, link to foreign assistance framework: DESCRIPTION Precise Definition(s): Private maternity homes (PMHs); “Trained” here refers to receiving training from the SMH project. Unit of Measure: Number PMH facilities who send at least one representative to at least one of the project’s training courses. Disaggregated by: This indicator will be disaggregated based on type of training (Business Continuity and Succession Planning (BCIP), Business Strengthening Support (BSS), or Clinical Capacity Building (CCB)), sex of the owner, and geographic location. Rationale or Justification for indicator (optional): PLAN FOR DATA COLLECTION Data Source: Trainee Registration Forms Method of data collection and construction: The project will collect Trainee Registration Forms as the trainings move forward, and feed summary data from these forms into the project’s central database. Reporting Frequency: Routinely Individual(s) responsible: Bernard Akotey, SMH Project Team Lead; Matthew Griffith, Banyan Global Program Manager DATA QUALITY ISSUES Dates of Previous Data Quality Assessments and name of reviewer: N/A Date of Future Data Quality Assessments (optional): Known Data Limitations: TARGETS AND BASELINE Baseline timeframe (optional): Baseline findings submitted in June, 2015 Rationale for Targets (optional): LOP target: 146. Rationale: The project has set a target of 146 assisted facilities, determined by the number of facilities that receive tier 2 support. This tier represents the broadest level of assistance, designed to affect the largest number of facilities. Additional support (tier 3 and tier 4) is anticipated to involve facilities that already participated in tier 2. Thus, the project sets the LOP target for this indicator equal to the target for tier 2 support, at 146. CHANGES TO INDICATOR Changes to indicator: July, 2015: Added an LOP target and rationale. Other Notes (optional): THIS SHEET LAST UPDATED ON: July 31, 2015 Monitoring and Evaluation Plan, Year 1 56 USAID Performance Indicator Reference Sheet Name of Result Measured (Goal, DO, IR, sub-IR, Project Purpose, Project Output, etc.): Sub IR 1.2: Capacity of PMHs for ownership/management transformation strengthened. Name of Indicator: 10. Number of participants in SMH training courses Is this a Performance Plan and Report indicator? No _X__ Yes ____, for Reporting Year(s) _________ If yes, link to foreign assistance framework: DESCRIPTION Precise Definition(s): Private maternity homes (PMHs); “Trained” here refers to receiving training from the SMH project. “Participant” is defined as an individual participating in a training. One individual would count as two participants if that person participated in two separate trainings. Unit of Measure: Number of participants trained. Disaggregated by: This indicator will be disaggregated based on type of training (Business Continuity and Succession Planning (BCIP), Business Strengthening Support (BSS), or Clinical Capacity Building (CCB)), sex of the owner, and geographic location. Rationale or Justification for indicator (optional): PLAN FOR DATA COLLECTION Data Source: Trainee Registration Forms Method of data collection and construction: The project will collect Trainee Registration Forms as the trainings move forward, and feed summary data from these forms into the project’s central database. Reporting Frequency: Routinely Individual(s) responsible: Bernard Akotey, SMH Project Team Lead; Matthew Griffith, Banyan Global Program Manager DATA QUALITY ISSUES Dates of Previous Data Quality Assessments and name of reviewer: N/A Date of Future Data Quality Assessments (optional): Known Data Limitations: TARGETS AND BASELINE Baseline timeframe (optional): Baseline findings submitted in June, 2015 Rationale for Targets (optional): LOP target: 409. Rationale: For the initial BCSP and CCB trainings, the project had 153 participants (representing 146 facilities) attend the trainings, leading to a total of 306 participants in these trainings. Given that the project has one more training, at which it expects similar levels of participation, the project arrives at a final figure of 409 total participants in project trainings. CHANGES TO INDICATOR Changes to indicator: July, 2015: Added an LOP target and rationale. Other Notes (optional): THIS SHEET LAST UPDATED ON: July 31, 2015 Monitoring and Evaluation Plan, Year 1 57 USAID Performance Indicator Reference Sheet Name of Result Measured (Goal, DO, IR, sub-IR, Project Purpose, Project Output, etc.): Sub-IR 2.1: PMH ownership/management transformations brokered Name of Indicator: 11. Number of assisted active PMHs who complete an ownership / management transformation Is this a Performance Plan and Report indicator? No _X__ Yes ____, for Reporting Year(s) _________ If yes, link to foreign assistance framework: DESCRIPTION Precise Definition(s): Private Maternity Home (PMH); Transformation is defined as occurring when; 1) the PMH facility is legally sold to a new owner or investor; 2) the existing owner brings on a partner who has both an ownership stake and an active management role; 3) the existing owner retains ownership, but appoints a manager with a job description that gives this person final authority over all operational aspects of facility management; or 4) the existing owner retains ownership and leases the facility to a new manager/operator. Technical assistance as used includes individual technical assistance focused on deal structuring, needs analysis, business plan support, and other pre-transformation assistance. Unit of Measure: Number of active PMH facilities who receive project assistance, that complete the process of an ownership and/or management transformation. Disaggregated by: This indicator will be disaggregated based on sex of the owner, geographic location of the facility, and type of transformation. Rationale or Justification for indicator (optional): PLAN FOR DATA COLLECTION Data Source: Project records Method of data collection and construction: The project will track the required data to report against this indicator. Reporting Frequency: Annually Individual(s) responsible: Bernard Akotey, SMH Project Team Lead; Matthew Griffith, Banyan Global Program Manager DATA QUALITY ISSUES Dates of Previous Data Quality Assessments and name of reviewer: N/A Date of Future Data Quality Assessments (optional): Known Data Limitations: TARGETS AND BASELINE Baseline timeframe (optional): Baseline findings submitted in June, 2015 Rationale for Targets (optional): LOP target: 20. Rationale: This indicator is a restatement of the number of facilities that ascend to tier 4 of project assistance. Thus, we set the target for this indicator equal to the target for tier 4 support, at 20 facilities Though the EOI funnel may be wider, the screen criteria to be adopted may preclude SMH from moving forward with these initial EOIs. CHANGES TO INDICATOR Changes to indicator: July, 2015: Added an LOP target and rationale. Other Notes (optional): THIS SHEET LAST UPDATED ON: July 31, 2015 Monitoring and Evaluation Plan, Year 1 58 USAID Performance Indicator Reference Sheet Name of Result Measured (Goal, DO, IR, sub-IR, Project Purpose, Project Output, etc.): Sub-IR 2.1: PMH ownership/management transformations brokered Name of Indicator: 12. Number of inactive PMHs re-opened with new owners or managers Is this a Performance Plan and Report indicator? No _X__ Yes ____, for Reporting Year(s) _________ If yes, link to foreign assistance framework: DESCRIPTION Precise Definition(s): An inactive PMH is defined as a facility that is not currently delivering health services. Unit of Measure: Number of inactive PMH facilities that are re-opened and begin delivering health care services after obtaining new ownership/ management. The new owners/managers of these facilities will have received assistance from the SMH project. Disaggregated by: This indicator will be disaggregated based on sex of the owner and geographic location of the facility. Rationale or Justification for indicator (optional): PLAN FOR DATA COLLECTION Data Source: Project records Method of data collection and construction: The project will track the required data to report against this indicator. Reporting Frequency: Annually Individual(s) responsible: Bernard Akotey, SMH Project Team Lead; Matthew Griffith, Banyan Global Program Manager DATA QUALITY ISSUES Dates of Previous Data Quality Assessments and name of reviewer: N/A Date of Future Data Quality Assessments (optional): Known Data Limitations: TARGETS AND BASELINE Baseline timeframe (optional): Baseline findings submitted in June, 2015 Rationale for Targets (optional): LOP target: 3. Rationale: The baseline discovered 30 closed facilities (18% of the sample). The survey team was not able to get a reason for the closure of nine facilities. Six facilities were newly created and awaiting legal registration, while fifteen facilities were either temporarily or permanently closed due to the death, disability, or absence of the owner or primary midwife. Even though there are probably more closed facilities than the ones we discovered, these numbers suggest the total number of closed facilities that could be re-opened is low. From discussions during the PMHs during the baseline and initial BCSP trainings, as well as input from GRMA and other key informants, the project believes that the state of most of these inactive facilities will require more resources to reopen than the project originally anticipated. Almost all of these defunct facilities serve as homes for the owners and their families, hence the possibility of re-opening may require either the relocation of either the facility or the family. Thus, the project arrives at a modest LOP target for this indicator of three facilities. CHANGES TO INDICATOR Changes to indicator: July, 2015: Added an LOP target and rationale. Other Notes (optional): THIS SHEET LAST UPDATED ON: July 31, 2015 Monitoring and Evaluation Plan, Year 1 59 USAID Performance Indicator Reference Sheet Name of Result Measured (Goal, DO, IR, sub-IR, Project Purpose, Project Output, etc.): Sub-IR 2.1: PMH ownership/management transformations brokered Name of Indicator: 13. Number of investment proposals for PMH transformations prepared Is this a Performance Plan and Report indicator? No _X__ Yes ____, for Reporting Year(s) _________ If yes, link to foreign assistance framework: DESCRIPTION Precise Definition(s): Investment proposals are effectively plans to effect transformation, prepared through assistance from the project. These proposals could be prepared by existing PMHs seeking new investors / managers, by investors seeking to purchase PMH facilities, or by new / younger midwives seeking to open their own facilities. Unit of Measure: Number of transformative investment proposals prepared as a result of project assistance Disaggregated by: This indicator will be disaggregated based on sex of the owner and geographic location of the facility. Rationale or Justification for indicator (optional): PLAN FOR DATA COLLECTION Data Source: Project records Method of data collection and construction: The project will take note when any investment proposals on which it is assisting are completed. Reporting Frequency: Routinely Individual(s) responsible: Bernard Akotey, SMH Project Team Lead; Matthew Griffith, Banyan Global Program Manager DATA QUALITY ISSUES Dates of Previous Data Quality Assessments and name of reviewer: N/A Date of Future Data Quality Assessments (optional): Known Data Limitations: TARGETS AND BASELINE Baseline timeframe (optional): Baseline findings submitted in June, 2015 Rationale for Targets (optional): LOP target: 20. Rationale: By setting an LOP target of 20 facilities, equal to the total number of transformations we are targeting, we recognize that each transformation is likely to have a legitimate need for seed funding in order to undertake needed initial investments that will boost the likelihood of success of the new arrangement. CHANGES TO INDICATOR Changes to indicator: July, 2015: Added an LOP target and rationale. Other Notes (optional): THIS SHEET LAST UPDATED ON: July 31, 2015 Monitoring and Evaluation Plan, Year 1 60 USAID Performance Indicator Reference Sheet Name of Result Measured (Goal, DO, IR, sub-IR, Project Purpose, Project Output, etc.): Sub-IR 2.1: PMH ownership/management transformations brokered Name of Indicator: 14. Number of new / younger midwives interested in PMHs Is this a Performance Plan and Report indicator? No _X__ Yes ____, for Reporting Year(s) _________ If yes, link to foreign assistance framework: DESCRIPTION Precise Definition(s): A new / younger midwife is defined as an individual midwife who is below the age of 55. These individuals will express interest through the completion and submission of “Expression of Interest” forms that formally declare their interest in receiving project assistance to take on the ownership/management duties of PMHs in target regions. Unit of Measure: Number of individual midwives below the age of 55 who complete an “Expression of Interest” form that formally declares their interest in receiving project assistance to take on the ownership/management duties of PMHs in target regions. Disaggregated by: This indicator will be disaggregated based on sex of the owner, and geographic location of the facility. Rationale or Justification for indicator (optional): This indicator refers to the overall attitudes of PMH owners towards the idea of potentially selling their facilities, which is one of the project’s proposed forms of ownership transformation. PLAN FOR DATA COLLECTION Data Source: PMH Survey Method of data collection and construction: The project and GRMA will collect Expression of Interest Forms as they are submitted, and feed summary data from these forms into the project’s central database. Reporting Frequency: Quarterly Individual(s) responsible: Bernard Akotey, SMH Project Team Lead; Matthew Griffith, Banyan Global Program Manager DATA QUALITY ISSUES Dates of Previous Data Quality Assessments and name of reviewer: N/A Date of Future Data Quality Assessments (optional): Known Data Limitations: TARGETS AND BASELINE Baseline timeframe (optional): Baseline findings submitted in June, 2015 Rationale for Targets (optional): LOP target: 33. Rationale: The project has set an LOP target of 46 for total EIOs. Of these, we believe the bulk will be coming from midwives below the age of 55, due to the way the project will market these opportunities, and based on initial feedbacks from our the project’s encounter with public sector midwives in the Western region. Thus, we arrive at an LOP target of 33. CHANGES TO INDICATOR Changes to indicator: July, 2015: Added an LOP target and rationale. Other Notes (optional): THIS SHEET LAST UPDATED ON: July 31, 2015 Monitoring and Evaluation Plan, Year 1 61 USAID Performance Indicator Reference Sheet Name of Result Measured (Goal, DO, IR, sub-IR, Project Purpose, Project Output, etc.): Sub-IR 2.1: PMH ownership/management transformations brokered Name of Indicator: 15. Percentage of PMHs that have a neutral or positive attitude towards the idea of ownership/ management transformation. Is this a Performance Plan and Report indicator? No _X__ Yes ____, for Reporting Year(s) _________ If yes, link to foreign assistance framework: DESCRIPTION Precise Definition(s): A PMH will be deemed to have a neutral or positive if they provide an answer of “Neutral”, “Somewhat interested” or “Very interested” to questions related to various forms of ownership/management transformation (selling, leasing, bringing on a partner, bringing on a manager). Unit of Measure: Numerator: Number of current PMH owners indicating a neutral or positive attitude on a Likert scale to a statement about an ownership/management transformation. Denominator: Number of PMH facilities responding to the PMH survey. Disaggregated by: This indicator will be disaggregated based on sex of the owner, geographic location of the facility and type of ownership/management transformation. Rationale or Justification for indicator (optional): PLAN FOR DATA COLLECTION Data Source: PMH Survey Method of data collection and construction: The project will interview PMH owners and managers using the PMH survey questions. The questions have been built into the formhub.org mobile data collection platform. Reporting Frequency: Baseline (beginning of project) and endline (end of project) Individual(s) responsible: Bernard Akotey, SMH Project Team Lead; Matthew Griffith, Banyan Global Program Manager DATA QUALITY ISSUES Dates of Previous Data Quality Assessments and name of reviewer: N/A Date of Future Data Quality Assessments (optional): Known Data Limitations: TARGETS AND BASELINE Baseline timeframe (optional): Baseline findings submitted in June, 2015 Rationale for Targets (optional): LOP target for bringing on a manager: 90%. Rationale: At baseline, 88% of facilities had a neutral or positive attitude towards this form of transformation. We believe that, as the project sensitizes facilities about the advantages of this option and showcases success stories, the percentage of facilities interested will improve. However, we do believe that a sizable proportion of the 12% who were not interested might have specific reasons for their opinion, and so would not be very easily swayable, thus the modest goal of a 2% increase from baseline to endline. LOP target for bringing on a partner: 88%. Rationale: At baseline, 81% of facilities had a neutral or positive attitude towards this form of transformation. We believe that, as the project sensitizes facilities about the advantages of this option and showcases success stories, the percentage of facilities interested will improve, leading to an increase of 7% from baseline to endline. LOP target for leasing out the facility: 67%. Rationale: At baseline, 55% of facilities had a neutral or positive attitude towards this form of transformation. We believe that, as the project sensitizes facilities about the advantages of this option and showcases success stories, the percentage of facilities interested will improve. We also believe the low level of interest at baseline may reflect lack of understanding, and that these sensitization efforts may have a greater impact than it will on the transformations listed above. Thus, we set a target of an increase in around 12% from baseline to endline. LOP target for selling the facility: 38%. Rationale: At baseline, 28% of facilities had a neutral or positive attitude towards this form of transformation. We believe that, as the project sensitizes facilities about the advantages of this option and showcases success stories, the percentage of facilities interested will improve. And while we also believe the low level of interest at baseline may reflect lack of understanding, key informants indicated that there may be a visceral opposition to this particular form of transformation, making opinions somewhat difficult to change. Thus, we set a target of an increase in around 10% from baseline to endline. CHANGES TO INDICATOR Changes to indicator: July, 2015: Added an LOP target and rationale. Other Notes (optional): THIS SHEET LAST UPDATED ON: July 31, 2015 Monitoring and Evaluation Plan, Year 1 62 USAID Performance Indicator Reference Sheet Name of Result Measured (Goal, DO, IR, sub-IR, Project Purpose, Project Output, etc.): Sub-IR 2.2: Funding Provided and Leveraged by the Project Name of Indicator: 16. Number of assisted PMHs that receive seed funding in the form of SMH small grants Is this a Performance Plan and Report indicator? No _X__ Yes ____, for Reporting Year(s) _________ If yes, link to foreign assistance framework: DESCRIPTION Precise Definition(s): Seed funding refers to relatively small amounts of funding that the project plans to use to reduce risk for local investors to invest in PMHs; and provide younger midwives with additional capital. Small grants refers to the mechanism through which the project will provide this funding. Unit of Measure: The total number of PMHs that receive seed funding from the SMH project in a given year. Disaggregated by: This indicator will be disaggregated based on sex of the owner and geographic location of the facility. Rationale or Justification for indicator (optional): The project will bring non-traditional financial partners to the sector, exploring financing from a variety of sources, including commercial banks, angel investors, investment clubs and crowdfunding. This financing will be complemented by a small amount of seed funding from the project. PLAN FOR DATA COLLECTION Data Source: Project records Method of data collection and construction: Through its grants management systems, the project will track the required data to report against this indicator. Reporting Frequency: Annually Individual(s) responsible: Bernard Akotey, SMH Project Team Lead; Matthew Griffith, Banyan Global Program Manager DATA QUALITY ISSUES Dates of Previous Data Quality Assessments and name of reviewer: N/A Date of Future Data Quality Assessments (optional): Known Data Limitations: TARGETS AND BASELINE Baseline timeframe (optional): Baseline findings submitted in June, 2015 Rationale for Targets (optional): LOP target: 20. Rationale: We have set the LOP target equal to the number of anticipated transformations. The assumption here is that, even if some transformation do not require seed funding to finalize the transformation arrangement (mainly, in the case of management contracts), all transformed facilities could benefit from seed funding to maximize the gains from the transformation. The baseline assessment found that access to finance was the most frequently cited constraint to growth (mentioned by 96% of respondents), and helping to address this constraint in the case of transformed facilities is an area in which the project can add clear value. CHANGES TO INDICATOR Changes to indicator: July, 2015: Added an LOP target and rationale. Other Notes (optional): THIS SHEET LAST UPDATED ON: July 31, 2015 Monitoring and Evaluation Plan, Year 1 63 USAID Performance Indicator Reference Sheet Name of Result Measured (Goal, DO, IR, sub-IR, Project Purpose, Project Output, etc.): Sub-IR 2.2: Funding Provided and Leveraged by the Project Name of Indicator: 17. Amount of seed funding provided by the SMH project in the form of small grants Is this a Performance Plan and Report indicator? No _X__ Yes ____, for Reporting Year(s) _________ If yes, link to foreign assistance framework: DESCRIPTION Precise Definition(s): Seed funding refers to relatively small amounts of funding that the project plans to use to reduce risk for local investors to invest in PMHs; and provide younger midwives with additional capital. Small grants refers to the mechanism through which the project will provide this funding. Unit of Measure: Amount of grant funding in US$ provided to the PMHs by the SMH project for transformation in a given year. Disaggregated by: This indicator will be disaggregated based on sex of the owner and geographic location of the facility. Rationale or Justification for indicator (optional): The project will bring non-traditional financial partners to the sector, exploring financing from a variety of sources, including commercial banks, angel investors, investment clubs and crowdfunding. This financing will be complemented by a small amount of seed funding from the project. PLAN FOR DATA COLLECTION Data Source: Project records Method of data collection and construction: Through its grants management systems, the project will track the required data to report against this indicator. Reporting Frequency: Annually Individual(s) responsible: Bernard Akotey, SMH Project Team Lead; Matthew Griffith, Banyan Global Program Manager DATA QUALITY ISSUES Dates of Previous Data Quality Assessments and name of reviewer: N/A Date of Future Data Quality Assessments (optional): Known Data Limitations: TARGETS AND BASELINE Baseline timeframe (optional): Baseline findings submitted in June, 2015 Rationale for Targets (optional): LOP target: $100,000. The baseline found that while facilities viewed access to finance as the single most significant constraint (cited by 96% of respondents), only a small number of facilities had used outside capital to facilitate the founding or expanding of their facilities. It was much more common for facilities to us their own savings, or to borrow from family and friend. Thus, we believe that access to SMH project grant funds will be welcome, but that the initial uses of these funds might be modest, with investments averaging $5,000 per transformed facility. Multiplying $5,000 by the projected number of transformed facilities (20), we arrive at an LOP target of $100,000. CHANGES TO INDICATOR Changes to indicator: July, 2015: Added an LOP target and rationale. Other Notes (optional): THIS SHEET LAST UPDATED ON: July 31, 2015 Monitoring and Evaluation Plan, Year 1 64 USAID Performance Indicator Reference Sheet Name of Result Measured (Goal, DO, IR, sub-IR, Project Purpose, Project Output, etc.): Sub-IR 2.2: Funding Provided and Leveraged by the Project Name of Indicator: 18. Number of assisted PMHs that receive funds from non-project sources Is this a Performance Plan and Report indicator? No _X__ Yes ____, for Reporting Year(s) _________ If yes, link to foreign assistance framework: DESCRIPTION Precise Definition(s): Funds refers to investment, grant, or loan financing. Non-project sources referrers to third-party sources of financing, such as commercial banks, angel investors, investment clubs, or crowdfunding donors. Unit of Measure: Number of PMHs receiving project assistance that receive funds from non-project sources for transformation in a given year. Disaggregated by: This indicator will be disaggregated based on sex of the owner and geographic location of the facility. Rationale or Justification for indicator (optional): PLAN FOR DATA COLLECTION Data Source: Project records Method of data collection and construction: The project will track the required data to report against this indicator. Reporting Frequency: Annually Individual(s) responsible: Bernard Akotey, SMH Project Team Lead; Matthew Griffith, Banyan Global Program Manager DATA QUALITY ISSUES Dates of Previous Data Quality Assessments and name of reviewer: N/A Date of Future Data Quality Assessments (optional): Known Data Limitations: TARGETS AND BASELINE Baseline timeframe (optional): Baseline findings submitted in June, 2015 Rationale for Targets (optional): LOP target: 14. Rationale: The project plans to strongly encourage facilities completing an ownership/management transformation to seek counter-party funding from non-project sources, and will provide assistance in helping them access this funding. However, we do not expect all transformed facilities to succeed in acquiring this external funding. Thus, we set this target at two-thirds of the projected number of transformed facilities (20), arriving at a target of 13. CHANGES TO INDICATOR Changes to indicator: July, 2015: Added an LOP target and rationale. Other Notes (optional): THIS SHEET LAST UPDATED ON: July 31, 2015 Monitoring and Evaluation Plan, Year 1 65 USAID Performance Indicator Reference Sheet Name of Result Measured (Goal, DO, IR, sub-IR, Project Purpose, Project Output, etc.): Sub-IR 2.2: Funding Provided and Leveraged by the Project Name of Indicator: 19. Amount of funding that assisted PMHs receive from non-project sources Is this a Performance Plan and Report indicator? No _X__ Yes ____, for Reporting Year(s) _________ If yes, link to foreign assistance framework: DESCRIPTION Precise Definition(s): Funding refers to amount of investment, grant, or loan financing. Non-project sources referrers to third-party sources of financing, such as commercial banks, angel investors, investment clubs, or crowdfunding donors. Unit of Measure: Amount of funding from non-project sources received by project-assisted PMHs for the transformation of PMHs resulting from the project interventions. Disaggregated by: This indicator will be disaggregated based on sex of the owner and geographic location of the facility. Rationale or Justification for indicator (optional): PLAN FOR DATA COLLECTION Data Source: Project records Method of data collection and construction: The project will track the required data to report against this indicator. Reporting Frequency: Annually Individual(s) responsible: Bernard Akotey, SMH Project Team Lead; Matthew Griffith, Banyan Global Program Manager DATA QUALITY ISSUES Dates of Previous Data Quality Assessments and name of reviewer: N/A Date of Future Data Quality Assessments (optional): Known Data Limitations: TARGETS AND BASELINE Baseline timeframe (optional): Baseline findings submitted in June, 2015 Rationale for Targets (optional): LOP target: $70,000. The project believes that the average amount of third-party funding, when it is acquired, will be roughly equal to the average amount of SMH project grant funding: $5,000. If we multiply this projected average by the project number of facilities that will secure third party funding (14), we come to a projection of $70,000. CHANGES TO INDICATOR Changes to indicator: July, 2015: Added an LOP target and rationale. Other Notes (optional): THIS SHEET LAST UPDATED ON: July 31, 2015 Monitoring and Evaluation Plan, Year 1 66 USAID Performance Indicator Reference Sheet Name of Result Measured (Goal, DO, IR, sub-IR, Project Purpose, Project Output, etc.): Sub-IR 3.1: Capacity of PMHs to sustain, increase and improve service delivery strengthened Name of Indicator: 20. Percentage of assisted, non-accredited PMHs applying for NHIA accreditation that successfully become accredited. Is this a Performance Plan and Report indicator? No _X__ Yes ____, for Reporting Year(s) _________ If yes, link to foreign assistance framework: DESCRIPTION Precise Definition(s): NHIA is the National Health Insurance Authority; Accreditation is NHIA’s process for deciding if a health facility is able to serve NHIA enrollees, and receive reimbursement for those services. Unit of Measure: Numerator: Number of (a) non-accredited project-assisted PMHs that receive a grade that is sufficient to receive NHIA accreditation. Denominator: Non-accredited, project-assisted PMHs that apply to NHIA to become accredited. Disaggregated by: This indicator will be disaggregated based on sex of the owner and geographic location of the facility. Rationale or Justification for indicator (optional): This indicator provides information on the rate at which project￾assisted PMHs are successfully participating in Ghana’s national health insurance scheme. PLAN FOR DATA COLLECTION Data Source: PMH survey and NHIA Method of data collection and construction: The project will collect the necessary data through the PMH survey, and from NHIA. Reporting Frequency: Baseline and endline Individual(s) responsible: Bernard Akotey, SMH Project Team Lead; Matthew Griffith, Banyan Global Program Manager DATA QUALITY ISSUES Dates of Previous Data Quality Assessments and name of reviewer: N/A Date of Future Data Quality Assessments (optional): Known Data Limitations: Any difficulty or delay in accessing the required data from NHIA could affect the project’s ability to report against this indicator. TARGETS AND BASELINE Baseline timeframe (optional): Baseline findings submitted in June, 2015 Rationale for Targets (optional): LOP target: 5% Rationale: At baseline, a total of 81% of facilities were registered with NHIA. If we extend that percentage to all assisted PMHs (146), we could project that around 118 facilities are registered with NHIA, while around 28 facilities are not. The SMH interventions will increase recognition of the financial advantages of NHIA accreditation in terms of generating revenue and accessing clients in these underserved areas on which the project is focusing, almost all of whom rely on NHIS). We believe that, through these efforts, we can effect a 5% increase in NHIA accreditation, effectively resulting in 7 assisted facilities (5% of the total 146 assisted facilities) registering with NHIA that were not registered before. CHANGES TO INDICATOR Changes to indicator: July, 2015: Added an LOP target and rationale. Other Notes (optional): There may be limitations based on the NHIA accreditation timeline. Some transformed PMHs may be in the process of becoming evaluated, or may be waiting on the next round of accreditation evaluations. In this case, there may be limitations on the ability to provide a full account of the project’s achievement against this indicator. THIS SHEET LAST UPDATED ON: July 31, 2015 Monitoring and Evaluation Plan, Year 1 67 USAID Performance Indicator Reference Sheet Name of Result Measured (Goal, DO, IR, sub-IR, Project Purpose, Project Output, etc.): Sub-IR 3.1: Capacity of PMHs to sustain, increase and improve service delivery strengthened Name of Indicator: 21. Percentage of assisted PMHs that receive or maintain NHIA accreditation after transformation Is this a Performance Plan and Report indicator? No _X__ Yes ____, for Reporting Year(s) _________ If yes, link to foreign assistance framework: DESCRIPTION Precise Definition(s): NHIA is the National Health Insurance Authority; Accreditation is NHIA’s process for deciding if a health facility is able to serve NHIA enrollees, and receive reimbursement for those services. Unit of Measure: Numerator: Number of assisted PMHs that complete transformation AND that receive or maintain NHIA accreditation after transformation is complete. Denominator: Number of assisted PMHs that complete transformation. Disaggregated by: This indicator will be disaggregated based on sex of the owner and geographic location of the facility. Rationale or Justification for indicator (optional): This indicator measures the effect of transformation on NHIA accreditation among PMHs. PLAN FOR DATA COLLECTION Data Source: NHIA, Project Records Method of data collection and construction: The project will collect the necessary data from NHIA, and will also collect data from PMHs that complete transformation to inform this indicator. Reporting Frequency: Baseline and endline Individual(s) responsible: Bernard Akotey, SMH Project Team Lead; Matthew Griffith, Banyan Global Program Manager DATA QUALITY ISSUES Dates of Previous Data Quality Assessments and name of reviewer: N/A Date of Future Data Quality Assessments (optional): Known Data Limitations: Any difficulty or delay in accessing the required data from NHIA could affect the project’s ability to report against this indicator. TARGETS AND BASELINE Baseline timeframe (optional): Baseline findings submitted in June, 2015 Rationale for Targets (optional): LOP target: 100%. PMHs who complete an ownership/management transformation will have benefitted from the projects broad sensitization efforts regarding the benefits of accreditation, as well as pre- and post-transformation support, which will also reinforce these benefits. Thus, we believe transformed facilities will recognize the value of catering to NHIS cardholders, who form a sizeable portion of the potential clients available, and that recognition will lead all transformed facilities to maintain or achieve accreditation after transformation. CHANGES TO INDICATOR Changes to indicator: July, 2015: Added an LOP target and rationale. Other Notes (optional): There may be limitations based on the NHIA accreditation timeline. Some transformed PMHs may be in the process of becoming evaluated, or may be waiting on the next round of accreditation evaluations. In this case, there may be limitations on the ability to provide a full account of the project’s achievement against this indicator. THIS SHEET LAST UPDATED ON: July 31, 2015 Monitoring and Evaluation Plan, Year 1 68 USAID Performance Indicator Reference Sheet Name of Result Measured (Goal, DO, IR, sub-IR, Project Purpose, Project Output, etc.): Sub-IR 3.2: Increased capacity of PMHs to manage their businesses efficiently Name of Indicator: 22. Number of business training events conducted Is this a Performance Plan and Report indicator? No _X__ Yes ____, for Reporting Year(s) _________ If yes, link to foreign assistance framework: DESCRIPTION Precise Definition(s): A business training event refers to the production and delivery of one of the SMH project’s one-on￾one business training sessions. Unit of Measure: Number of one-on-one training and one-on-one capacity building events provided by the project to participating PMHs. Disaggregated by: Rationale or Justification for indicator (optional): PLAN FOR DATA COLLECTION Data Source: Project Records Method of data collection and construction: The project will collect the necessary data to report against this indicator Reporting Frequency: Quarterly Individual(s) responsible: Bernard Akotey, SMH Project Team Lead; Matthew Griffith, Banyan Global Program Manager DATA QUALITY ISSUES Dates of Previous Data Quality Assessments and name of reviewer: N/A Date of Future Data Quality Assessments (optional): Known Data Limitations: TARGETS AND BASELINE Baseline timeframe (optional): Baseline findings submitted in June, 2015 Rationale for Targets (optional): LOP target: 55. Rationale: This indicator referrers to the one-on-one pre￾transformation and post-transformation technical assistance. We are projecting that 25 facilities will receive pre￾transformation technical assistance, in the form of one structured TA visit (in the form of a business counseling visit), resulting in 25 events. (The facilities who prove to be serious about proceeding with a transformation will receive additional, unstructured support in that process.) In terms of post-transformation assistance, we are projecting that 15 PMHs will achieve transformation, and receive two structure post-transformation technical assistance visits, resulting in 30 events. The total LOP projection is thus 55 events. CHANGES TO INDICATOR Changes to indicator: July, 2015: Added an LOP target and rationale, and updated this indicator to focus on one-on-one business training efforts. Other Notes (optional): THIS SHEET LAST UPDATED ON: July 31, 2015 Monitoring and Evaluation Plan, Year 1 69 USAID Performance Indicator Reference Sheet Name of Result Measured (Goal, DO, IR, sub-IR, Project Purpose, Project Output, etc.): Sub-IR 3.2: Increased capacity of PMHs to manage their businesses efficiently Name of Indicator: 23. Percentage of assisted PMHs maintaining core financial documents (cash log, profit and loss statements or income statements) Is this a Performance Plan and Report indicator? No _X__ Yes ____, for Reporting Year(s) _________ If yes, link to foreign assistance framework: DESCRIPTION Precise Definition(s): Core financial documents include: cash log, profit and loss statement and income statement Unit of Measure: Numerator: Project-assisted PMHs that report having one of these three documents in place, and up to date. Denominator: All project-assisted PMHs. Disaggregated by: This indicator will be disaggregated based on sex of the owner and geographic location of the facility. Rationale or Justification for indicator (optional): This indicator refers to the practice of keeping up to date fundamental financial documents. PLAN FOR DATA COLLECTION Data Source: PMH Survey Method of data collection and construction: The project will interview PMH owners and managers using the PMH survey questions. The questions have been built into the formhub.org mobile data collection platform. Reporting Frequency: Baseline and endline Individual(s) responsible: Bernard Akotey, SMH Project Team Lead; Matthew Griffith, Banyan Global Program Manager DATA QUALITY ISSUES Dates of Previous Data Quality Assessments and name of reviewer: N/A Date of Future Data Quality Assessments (optional): Known Data Limitations: TARGETS AND BASELINE Baseline timeframe (optional): Baseline findings submitted in June, 2015 Rationale for Targets (optional): LOP target: 75%. Rationale: At baseline, 67% of facilities kept at least one of the core financial documents described in this indicator. The project believes that, through its broad training support (primarily the BSS) and the more tailored support given to Tier 3 and Tier 4 PMHs, it can positively influence this percentage. That said, the project recognizes the challenges that business management training interventions have traditionally had in producing substantial behavior change in terms of financial record keeping. Thus, we set an indicator of an 8% increase over baseline, resulting in 75%. CHANGES TO INDICATOR Changes to indicator: July, 2015: Added an LOP target and rationale. Other Notes (optional): THIS SHEET LAST UPDATED ON: July 31, 2015 Monitoring and Evaluation Plan, Year 1 70 USAID Performance Indicator Reference Sheet Name of Result Measured (Goal, DO, IR, sub-IR, Project Purpose, Project Output, etc.): Sub-IR 3.3: Management and programmatic capacity of GRMA increased. Name of Indicator: 24. Number of GRMA staff and volunteers who receive capacity building assistance Is this a Performance Plan and Report indicator? No _X__ Yes ____, for Reporting Year(s) _________ If yes, link to foreign assistance framework: DESCRIPTION Precise Definition(s): GRMA staff include paid staff and volunteers. Unit of Measure: Number of individuals who are working for GRMA who receive capacity building assistance Disaggregated by: Rationale or Justification for indicator (optional): PLAN FOR DATA COLLECTION Data Source: Project records Method of data collection and construction: The project will collect the necessary data to report against this indicator Reporting Frequency: Annually Individual(s) responsible: Bernard Akotey, SMH Project Team Lead; Matthew Griffith, Banyan Global Program Manager DATA QUALITY ISSUES Dates of Previous Data Quality Assessments and name of reviewer: N/A Date of Future Data Quality Assessments (optional): Known Data Limitations: TARGETS AND BASELINE Baseline timeframe (optional): Baseline findings submitted in June, 2015 Rationale for Targets (optional): LOP target: 14. Rationale: The total population of GRMA personnel that could be trained includes the 20 member Leadership Council, and three secretariat staff. The project does not believe it will have trouble securing time from the secretariat staff, but to date there have been challenges securing the time of some members of the Leadership Council. Thus, the project is aiming to train eleven members of the Leadership Council and all three member of the secretariat staff, for a total LOP target of 14. CHANGES TO INDICATOR Changes to indicator: July, 2015: Added an LOP target and rationale. Other Notes (optional): THIS SHEET LAST UPDATED ON: July 31, 2015 Monitoring and Evaluation Plan, Year 1 71 Annex 5: PMH Survey SMH PROJECT PMH SURVEY Part 1: Pre-interview data: The interviewer should complete the following questions before beginning the interview. # Category Answers (and Coding) Instructions 1.1 Interviewer Name 1.2 Date 1 5 Day Month Year 1.3 Region 1.4 District 1.5 Town 1.6 GPS Coordinates Please record the GPS coordinates of the facility. 1.7 Photograph of facility sign Please take a photograph of the facility sign or building. Part 2: Introduction and informed consent: GRMA will provide an introduction. The introduction will include the following: • Description of the SMH project, and the roles of GRMA and Banyan Global • Explanation of the two parts of this visit: 1. The SMH survey questionnaire, for information gathering purposes 2. The GRMA Peer Support Visit (PSV), the purposes of which is to gather information, and to provide feedback to the PMH abut areas in which they might improve • Estimate of the total time required – about 2 hours • An understanding of the sequencing 1. We will ask some general questions about your facility, that are relevant to both GRMA and Banyan Global 2. Banyan Global will ask some questions, while GRMA tours the facility and reviews whether specific equipment and other items are in place 3. GRMA will ask some questions, while Banyan Global works with assistants (if possible) to sight some specific documents related to business operations 4. GRMA will give specific feedback (Banyan Global will not be in the room) 5. We will thank you for your time, and conclude the visit Once GRMA has concluded the introduction, and left the room to tour the facility and sight the equipment and other items, the enumerator should read the following to the respondent: Monitoring and Evaluation Plan, Year 1 72 Hello. My name is ________________. As you just heard from GRMA, we are conducting a survey of Private Maternity Homes as part of a USAID funded project called Saving Maternity Homes (SMH), which seeks to revitalize declining or closed private maternity homes. The purpose of the survey is to learn more about your facility and the services you offer. Participation in the survey is voluntary and you are free to decline to answer any or all questions. The results will be kept confidential and will only be used to help the project do better work in this area. This survey usually takes 60 minutes to complete. Will you participate in this survey? Yes…………………………………………………………………………….1 No……………………………………………………………………………...2 Part 3: Facility information: The interviewer should read aloud: I’d like to start by learning a little bit about your facility. # Question Answers (and Coding) Instructions N/A Could you tell us a bit more about the history of this facility? N/A The purpose of this question is to give the respondent a chance to talk about the facility a bit, and to put her at ease. Do not worry about recording any information at this point (though some of what is said might be relevant for later questions). But do pay attention to the time, and do not let the respondent speak for more than 1-2 minutes. 3.1 What is your name? 3.2 Respondent sex Male………………………………………………1 Female…………………………………..………...2 3.3 What is your position or title? If the respondent is not the owner, manager, or another person with significant responsibility, confer with GRMA about the possibility of postponing the interview. 3.4 How long have you been working with this facility? 3.5 What is the name of the facility? 3.6 What is the location address of the facility? Get the address, the district, and the region. 3.7 What is the facility’s landline telephone number (if any)? Monitoring and Evaluation Plan, Year 1 73 3.8 What is the best mobile number for the facility? 3.9 What is the best email address for the facility? 3.10 What is the facility’s website (if any)? 3.11 How many years has the facility been in operation? ___________ years Don’t know……………………………………..89 Refused……………...……………..……………99 Part 4: Ownership and management: Read aloud: Now I would like to ask some questions about the ownership and management of the facility. 4.1 How many individuals have an ownership stake in this facility? ___________ individual(s) If 1, go to question 4.2. 4.1.1 If there are multiple owners, please indicate the ownership stake (in percentage terms) for each owner. Collect the relationship of each partner (family, friend, business associate, etc.), and ownership stake of each individual. 4.2 What is the name of the primary owner of the facility? Same as respondent…………………….………...1 ______________ [name]………………………2 Refused………………………………...………...99 The primary owner is the one who has the largest ownership stake. 4.3 What is the sex of the primary owner? Male…………………………………....………….1 Female…………….…………………..…………..2 At this point, GRMA can take their leave to begin the tour of the facility. The Banyan Global enumerators will then continue with the questioning. 4.4 How long has this person been the owner of the facility? ___________ years Don’t know……………………………………..89 Refused……………………………..…………...99 4.4.1 What is the owner’s professional background? Midwife………………………………………….1 Other medical professional……………..………2 Non-medical health professional……...………...3 Other (Specify)………………………………….4 4.5 How did this person come to be the owner of the facility? Founded the facility…………………….……….1 Purchased the facility…………………..………..2 Inherited from parents/family….…….....……….3 Other (Specify)………………………………….4 Monitoring and Evaluation Plan, Year 1 74 4.5.1 If this person purchased the facility, what funds were used? Personal savings………………………...………..1 Loan from family/friends………………...……….2 Loan from financial institution…………..………3 Third party investment………………….………4 Grant from government……………….………..5 Grant from foundation or NGO……….……….6 Other (Specify)………………………….………7 The respondent should check all that apply. 4.6 Is the owner of the facility also the manager of the facility? Yes…………………………………….………….1 No………………………………………………...2 If yes, skip to question 4.10. 4.7 What is the sex of the manager? Male……………………………………………….1 Female…………….……………………..………..2 4.8 How long has this person been the manager of the facility? ___________ years Don’t know……………………………………..89 Refused…………………………………..……...99 4.9 What is the manager’s professional background? Midwife………………………………….……….1 Other medical professional……………...………2 Non-medical health professional………………...3 Other (Specify)………………………….……….4 4.10 To what extent do you agree with the following statement: “The financial future of private maternity homes like this one is promising.” Strongly agree……………………………..……..1 Agree………………………...……….…………..2 Neutral……………………..…………………….3 Disagree…………………………….……………4 Strongly disagree……………………...………….5 4.11 To what extent do you agree with the following statement: “Expanding or growing my facility would be a wise decision.” Strongly agree……………………..……………..1 Agree…………………….…...…………………..2 Neutral……………………..…………………….3 Disagree………………….………………………4 Strongly disagree………...……………………….5 4.12 If you were to pursue expansion or growth, where might you see opportunities for expansion or growth? New building or addition………..……………….1 Additional employees……………..……………..2 Additional equipment or new technologies……..3 Other (specify)…………………………………..4 Respondent should check all that apply. 4.13 What are the constraints to the growth or expansion Limitations to the range of services you can offer………………………………………………1 Not willing to invest at this time……..………….2 Respondent should check all that apply. Monitoring and Evaluation Plan, Year 1 75 opportunities mentioned above? Insufficient funds………………………...………..3 Delays in payments from NHIA………...……….4 Owner/Manager does not have the time/energy to take the necessary steps………………..…….…5 Owner/Manager does not have the business management skills that would be necessary……..6 Owner/Manager does not have the clinical skills that would be necessary……………….………...7 Competition from other health facilities…..……8 Low client visits……………………….…………9 Other (specify)………………………...………..10 4.14 What is the average reimbursement delay that you have experienced from NHIA (in months)? _________ months 4.15 To what extent would you be willing to borrow funds to improve or expand this facility? Very willing……..……………………….………..1 Somewhat willing…...…………………...………..2 Neutral……………………..…………………….3 Not willing…..……………………....……………4 Would never consider this.……………..……….5 4.16 How interested are you in the idea of selling your facility someday? Very interested..………………..………………..1 Somewhat interested…………...………………..2 Neutral……………………..…………………….3 Not interested……………….……..……………4 Would never consider this.…………..………….5 Emphasize to the respondent that we are only looking for her thoughts on this idea, and that her answers will not commit her to any course of action whatsoever. 4.17 How interested are you in the idea of leasing out your facility someday? Very interested..………………...………………..1 Somewhat interested…………...………………..2 Neutral……………………..…………………….3 Not interested……………….……..……………4 Would never consider this.……..……………….5 Emphasize to the respondent that we are only looking for her thoughts on this idea, and that her answers will not commit her to any course of action whatsoever. 4.18 How interested are you in the idea of bringing on an additional business partner someday? Very interested..………………..………………..1 Somewhat interested……………...……………..2 Neutral……………………..…………………….3 Not interested……………….……..……………4 Would never consider this.…………..………….5 Emphasize to the respondent that we are only looking for her thoughts on this idea, and that her answers will not commit her to any course of action whatsoever. 4.19 How interested are you in the idea of bringing on a manager, or a new manager if your current manager leaves, for the Very interested..………………..………………..1 Somewhat interested……………...……………..2 Neutral……………………..…………………….3 Not interested……………….……..……………4 Would never consider this.…………..………….5 Emphasize to the respondent that we are only looking for her thoughts on this idea, and that her answers will not commit her to any course of action whatsoever. Monitoring and Evaluation Plan, Year 1 76 facility someday? Part 5: Formalization and business practices: Read: Now I would like to ask you some questions related to different aspects of formal business activities. These questions are only for our information, and it is okay if the facility is not engaging in some of these practices. Please answer honestly, and remember that your answers will remain confidential. 5.1 Is the facility’s certification of MIDWIFE with the Nurses and Midwives Council up to date? Not registered…………………………………...1 Registration is expired………………..………….2 Registration is current………………..………….3 5.2 Is the facility’s certification with the Health Facilities Regulatory Agency (HEFRA) up to date? Not registered…………………………………...1 Registration is expired………………..………….2 Registration is current………………..………….3 HEFRA was previously known as the Private Maternity Hospital Board. 5.3 Is the facility registered as a business with the Registrar General Department? Yes………………………………….…………….1 No………………………………………………...2 5.3.1 What is the incorporation model of the business? Sole Proprietor……………..……….….………..1 Limited Liability Company…………....….…….....2 Partnership………………………………………3 5.4 Is the facility registered with the Ghana Revenue Authority (GRA)? Not registered…………………………………...1 Registered but don’t pay tax…….…...………….2 Registered and pay tax…………..……...……….3 5.5 Is the facility currently accredited with NHIA? Accredited……………………………………….1 Not accredited…………………….…………….2 Accreditation withdrawn..………..………..…….3 If no, go to question 5.6. 5.5.1 What was the score the facility received on its most recent NHIA evaluation? A+ A B C D Provisional Failed 5.6 Does the facility maintain written contracts or agreements with partners and suppliers? No written contracts…………………………….1 Some written contracts.………….……...……….2 Maintains full written contracts………...………...3 Monitoring and Evaluation Plan, Year 1 77 5.7 Does the facility have an up-to-date organogram, or a chart that shows who works at the facility and who everyone reports to? Yes……………………………………………….1 No………………………………………………...2 Yes, but not up-to-date……………….…………3 5.8 Is there a plan in place that clearly indicates who runs the facility if the owner/ manager is no longer able to run the facility? Yes……………………………………………….1 No………………………………………………...2 5.9 Can you name the person who is going to take over? Yes……………………………………………….1 No………………………………………………...2 Respondent provides the name, though you don’t need to record it. 5.10 To what extent would you agree with the following statement: “If the owner were to die or retire, the facility would likely continue to operate.” Strongly agree…………………………………..1 Agree………………...…………………………..2 Neutral……………………..…………………….3 Disagree…………………………………………4 Strongly disagree……………………………….5 5.11 Does the facility maintain an annual business plan? Yes……………………………………………….1 No………………………………………………...2 Is done informally, not documented…………..3 5.12 Does the facility maintain a regular budget? Yes……………………………………………….1 No………………………………………………...2 If no, go to question 5.14. 5.12.1 How often does the facility create a budget? Annual budget…………………………………..1 Quarterly budget………………………………..2 Monthly budget………………………………….3 5.12.2 How often does the facility review the budget (for example, to manage spending and guide decision￾making)? Daily…………………….………………………..1 Weekly……………….…………………………..2 Monthly……………...…………………………...3 Quarterly…………………………………………4 Annually……..……..…………………………….5 5.13 Does the facility keep personal funds separate from business funds? No…………………...……………………………1 Yes……………………..………………………...2 Monitoring and Evaluation Plan, Year 1 78 5.14 Does the facility keep its funds in a bank account? No bank account…..……………………………1 Personal bank account………………………...2 Business bank account.………………………..3 If no, go to question 5.16. 5.15 Does the facility keep an up-to-date daily cash log? Yes……………………………………………….1 No………………………………………………...2 5.16 Does the facility keep an up-to-date record of all income generated and expenses incurred (income statement)? Yes……………………………………………….1 No………………………………………………...2 5.17 Does the facility keep an up-to-date balance sheet? Yes……………………………………………….1 No………………………………………………...2 5.18 To what extent does your business earn more money (revenues) than it spends (costs)? The business makes much more than it spends…………………………………………...1 The business makes slightly more than it spends…………………………………………...2 What the business makes and what it spends are about equal.………………………………...3 The business makes slightly lower than it spends…….....................................................4 The business makes much less than it spends…………………………………………...5 Don’t know……………………………………..89 Refused………………………………………...99 5.19 What is the total income that your business earns during a typical calendar month after paying all expenses including salaries of employees? That is, what are the profits of your business during a typical calendar month (in cedis)? Provides an amount……………….…………...1 Don’t know……………………………………..89 Refused………………………………………...99 If they provide an amount, go to question 5.19.1. If they don’t know, or refuse, go to Part 6. 5.19.1 [Enter amount provided] __________ cedis Part 6: Services: Read: Now I would like to ask you a few additional questions about your facility. 6.1 In the dry season, how many patient Facility does not provide that service..……….1 Provides an amount…………….……….……..2 Please stress the importance of getting at least the Monitoring and Evaluation Plan, Year 1 79 or client visits does your facility receive in a typical month? Don’t know……………………………………..89 Refused………………………………………...99 respondent’s best guess. If they don’t provide the service, don’t know, or refuse, go to question 6.1.1. [Enter amount provided] ___________ visits 6.1.1 In the rainy season, how many patient or client visits does your facility receive in a typical month? Facility does not provide that service..……….1 Provides an amount…………….……….……..2 Don’t know……………………………………..89 Refused………………………………………...99 Please stress the importance of getting at least the respondent’s best guess. If they don’t provide the service, don’t know, or refuse, go to question 6.2. [Enter amount provided] ___________ visits 6.2 In the dry season, how many visits does your facility receive in a typical month for maternal, newborn, and child health services? Facility does not provide that service..……….1 Provides an amount…………….……….……..2 Don’t know……………………………………..89 Refused………………………………………...99 Please stress the importance of getting at least the respondent’s best guess. If they don’t provide the service, don’t know, or refuse, go to question 6.2.1. [Enter amount provided.] ___________ visits 6.2.1 In the rainy season, how many visits does your facility receive in a typical month for maternal, newborn, and child health services? Facility does not provide that service..……….1 Provides an amount…………….……….……..2 Don’t know……………………………………..89 Refused………………………………………...99 Please stress the importance of getting at least the respondent’s best guess. If they don’t provide the service, don’t know, or refuse, go to question 6.3. [Enter amount provided.] ___________ visits 6.3 In the dry season, how many deliveries does your facility perform in a typical month? Facility does not provide that service..……….1 Provides an amount…………….……….……..2 Don’t know……………………………………..89 Refused………………………………………...99 Please stress the importance of getting at least the respondent’s best guess. If they don’t provide the service, don’t know, or refuse, go to question 6.3.1. [Enter amount provided.] ___________ deliveries 6.3.1 In the rainy season, how many deliveries does your facility perform in a typical month? Facility does not provide that service..……….1 Provides an amount…………….……….……..2 Don’t know……………………………………..89 Refused………………………………………...99 Please stress the importance of getting at least the respondent’s best guess. If they don’t provide the service, don’t know, or refuse, go to question 6.4. Monitoring and Evaluation Plan, Year 1 80 [Enter amount provided.] ___________ deliveries 6.4 In the dry season, how many visits does your facility receive in a typical month for family planning or reproductive health services? Facility does not provide that service..……….1 Provides an amount…………….……….……..2 Don’t know……………………………………..89 Refused………………………………………...99 Please stress the importance of getting at least the respondent’s best guess. If they don’t provide the service, don’t know, or refuse, go to question 6.4.1. [Enter amount provided.] ___________ visits 6.4.1 In the rainy season, how many visits does your facility receive in a typical month for family planning or reproductive health services? Facility does not provide that service..……….1 Provides an amount…………….……….……..2 Don’t know……………………………………..89 Refused………………………………………...99 Please stress the importance of getting at least the respondent’s best guess. If they don’t provide the service, don’t know, or refuse, go to question 6.5. [Enter amount provided.] ___________ visits 6.5 In the dry season, how many visits does your facility receive in a typical month for treatment of malaria? Facility does not provide that service..……….1 Provides an amount…………….……….……..2 Don’t know……………………………………..89 Refused………………………………………...99 Please stress the importance of getting at least the respondent’s best guess. If they don’t provide the service, don’t know, or refuse, go to question 6.5.1. [Enter amount provided.] ___________ visits 6.5.1 In the rainy season, how many visits does your facility receive in a typical month for treatment of malaria? Facility does not provide that service..……….1 Provides an amount…………….……….……..2 Don’t know……………………………………..89 Refused………………………………………...99 Please stress the importance of getting at least the respondent’s best guess. If they don’t provide the service, don’t know, or refuse, go to question 6.6. [Enter amount provided.] ___________ visits 6.6 In the dry season, how many visits does your facility receive in a typical month for treatment of diarrhea? Facility does not provide that service..……….1 Provides an amount…………….……….……..2 Don’t know……………………………………..89 Refused………………………………………...99 Please stress the importance of getting at least the respondent’s best guess. If they don’t provide the service, don’t know, or refuse, go to question 6.6.1. Monitoring and Evaluation Plan, Year 1 81 [Enter amount provided.] ___________ visits 6.6.1 In the rainy season, how many visits does your facility receive in a typical month for treatment of diarrhea? Facility does not provide that service..……….1 Provides an amount…………….……….……..2 Don’t know……………………………………..89 Refused………………………………………...99 Please stress the importance of getting at least the respondent’s best guess. If they don’t provide the service, don’t know, or refuse, go to question 6.7. [Enter amount provided.] ___________ visits 6.7 In the dry season, how many visits does your facility receive in a typical month for treatment of minor cuts and injuries? Facility does not provide that service..……….1 Provides an amount…………….……….……..2 Don’t know……………………………………..89 Refused………………………………………...99 Please stress the importance of getting at least the respondent’s best guess. If they don’t provide the service, don’t know, or refuse, go to question 6.7.1. [Enter amount provided.] ___________ visits 6.7.1 In the rainy season, how many visits does your facility receive in a typical month for treatment of minor cuts and injuries? Facility does not provide that service..……….1 Provides an amount…………….……….……..2 Don’t know……………………………………..89 Refused………………………………………...99 Please stress the importance of getting at least the respondent’s best guess. If they don’t provide the service, don’t know, or refuse, go to question 6.8. [Enter amount provided.] ___________ visits 6.8 In the dry season, how many visits does your facility receive in a typical month for the administering of vaccinations? Facility does not provide that service..……….1 Provides an amount…………….……….……..2 Don’t know……………………………………..89 Refused………………………………………...99 Please stress the importance of getting at least the respondent’s best guess. If they don’t provide the service, don’t know, or refuse, go to question 6.8.1. [Enter amount provided.] ___________ visits 6.8.1 In the rainy season, how many visits does your facility receive in a typical month for the administering of vaccinations? Facility does not provide that service..……….1 Provides an amount…………….……….……..2 Don’t know……………………………………..89 Refused………………………………………...99 Please stress the importance of getting at least the respondent’s best guess. If they don’t provide the service, don’t know, or refuse, go to question 6.9. [Enter amount provided.] ___________ visits Monitoring and Evaluation Plan, Year 1 82 6.9 How many beds does your facility have? Facility does not provide that service..……….1 Provides an amount…………….……….……..2 Don’t know……………………………………..89 Refused………………………………………...99 Please stress the importance of getting at least the respondent’s best guess. If they don’t provide the service, don’t know, or refuse, go to question 6.10. [Enter amount provided.] ___________ beds 6.10 In the dry season, how many inpatient days does your facility provide in a typical month? Facility does not provide that service..……….1 Provides an amount…………….……….……..2 Don’t know……………………………………..89 Refused………………………………………...99 Please stress the importance of getting at least the respondent’s best guess. If they don’t provide the service, don’t know, or refuse, go to question 6.11. [Enter amount provided.] ___________ inpatient days 6.10.1 In the rainy season, how many inpatient days does your facility provide in a typical month? Facility does not provide that service..……….1 Provides an amount…………….……….……..2 Don’t know……………………………………..89 Refused………………………………………...99 Please stress the importance of getting at least the respondent’s best guess. If they don’t provide the service, don’t know, or refuse, go to question 6.11. [Enter amount provided.] ___________ inpatient days 6.11 How many full-time clinical staff are working at this facility? Facility does not provide that service..……….1 Provides an amount…………….……….……..2 Don’t know……………………………………..89 Refused………………………………………...99 Clinical staff refers to: doctors, nurses, pharmacists, medical assistants, physiotherapists, etc. Please stress the importance of getting at least the respondent’s best guess. If they don’t provide the service, don’t know, or refuse, go to question 6.12. [Enter amount provided.] ___________ full-time clinical staff 6.12 How many full-time non-clinical staff are working at this facility? Facility does not provide that service..……….1 Provides an amount…………….……….……..2 Don’t know……………………………………..89 Refused………………………………………...99 Non-clinical staff refers to administrators, orderlies, accountants, nutrition officers, etc. Please stress the importance of getting at least the respondent’s best guess. If they don’t provide the service, don’t know, or refuse, go to question 6.13. Monitoring and Evaluation Plan, Year 1 83 [Enter amount provided.] ___________ full-time non-clinical staff 6.13 How many part￾time clinical staff are working at this facility? Facility does not provide that service..……….1 Provides an amount…………….……….……..2 Don’t know……………………………………..89 Refused………………………………………...99 Clinical staff refers to: doctors, nurses, pharmacists, medical assistants, physiotherapists, etc. If they don’t provide the service, don’t know, or refuse, go to question 6.14. [Enter amount provided.] ___________ part-time clinical staff 6.14 How many part￾time non-clinical staff are working at this facility? Facility does not provide that service..……….1 Provides an amount…………….……….……..2 Don’t know……………………………………..89 Refused………………………………………...99 Non-clinical staff refers to administrators, orderlies, accountants, nutrition officers, etc. If they don’t provide the service, don’t know, or refuse, go to Part 7. [Enter amount provided.] ___________ part-time non-clinical staff 7. Sighting documents: This section focuses on the sighting and confirming of specific documents. If possible, this should be done with the assistance of the facility administration staff, while the owner (respondent) is working with GRMA on finishing the QAT, and discussing the outcomes. Say to the owner: “Thank you very much for all the information that you have provided so far. It has been very helpful, and we are very appreciative. At this point, we would like to review some of the documents discussed above. May we work with your administrative staff to view the relevant documents?” 7.1 May I view your certification of MIDWIFE with the Nurses and Midwives Council? Yes……………………………………………….1 No………………………………………………...2 If no, go to question 7.2. 7.1.1 [Certification sighted] Yes……………………………………………….1 No………………………………………………...2 7.1.2 [Certification current] Yes……………………………………………….1 No………………………………………………...2 7.2 May I view the facility’s certification with the Health Facilities Regulatory Agency (HEFRA)? Yes……………………………………………….1 No………………………………………………...2 If the certification is not current, ask to view the most recent certification. If no, go to question 7.3. 7.2.1 [Certification sighted] Yes……………………………………………….1 No………………………………………………...2 7.2.2 [Certification current] Yes……………………………………………….1 No………………………………………………...2 Monitoring and Evaluation Plan, Year 1 84 7.3 May I view the facility’s registration document with the Registrar General Department? Yes……………………………………………….1 No………………………………………………...2 If no, go to question 7.4. 7.3.1 [Registration sighted] Yes……………………………………………….1 No………………………………………………...2 For a limited liability company, the document is a Certificate of Incorporation and Commencement; For a sole proprietor, the document is a Certificate of Registration 7.3.2 [Registration current] Yes……………………………………………….1 No………………………………………………...2 7.4 May I view evidence of your tax registration? Yes……………………………………………….1 No………………………………………………...2 Sufficient documents include GRA Certificate of Registration, tax receipts, or tax clearance certification If no, go to question 7.5. 7.4.1 [Tax document sighted] Yes……………………………………………….1 No………………………………………………...2 7.4.2 [Tax document current] Yes……………………………………………….1 No………………………………………………...2 7.5 May I see your NHIA accreditation certificate? Yes……………………………………………….1 No………………………………………………...2 If no, go to question 7.6. 7.5.1 [Accreditation certificate sighted] Yes……………………………………………….1 No………………………………………………...2 7.5.2 [Accreditation certificate current] Yes……………………………………………….1 No………………………………………………...2 7.6 May I view an example of a written contract or agreement with a supplier? Yes……………………………………………….1 No………………………………………………...2 If no, go to question 7.7. 7.6.1 [Document sighted] Yes……………………………………………….1 No………………………………………………...2 7.7 May see an example of a bank statement? Yes……………………………………………….1 No………………………………………………...2 If no, conclude the document sighting exercise. 7.7.1 [Document sighted] Yes……………………………………………….1 No………………………………………………...2 Monitoring and Evaluation Plan, Year 1 85 At this point, you are finished sighting documents. Thank the staff member for his/her time and assistance in reviewing these documents, and ask if there is a place where you can wait for the GRMA staff to finish their meeting with the facility owner. (NOTE: You should NOT re-enter the area in which GRMA is meeting with the owner.) 8. Data from QAT: After both the survey questionnaire and the QAT sessions are concluded, there is some information to collect from GRMA. Request that your GRMA colleague provide you with the following information. This can be done in the car as you travel to the next place, or elsewhere. But it is important that you record this information accurately. 8.1 Emergency services (24 hours) Yes……………………………………………….1 No………………………………………………...2 8.2 Outpatient services Yes……………………………………………….1 No………………………………………………...2 8.3 In-patient services Yes……………………………………………….1 No………………………………………………...2 8.4 Maternity services Yes……………………………………………….1 No………………………………………………...2 If no, go to question 8.5. 8.4.1 ANC Yes……………………………………………….1 No………………………………………………...2 8.4.2 Delivery Yes……………………………………………….1 No………………………………………………...2 8.4.3 PNC Yes……………………………………………….1 No………………………………………………...2 8.5 Family planning counseling Yes……………………………………………….1 No………………………………………………...2 8.6 Family planning methods Yes……………………………………………….1 No………………………………………………...2 If no, go to question 8.7. 8.6.1 Condoms Yes……………………………………………….1 No………………………………………………...2 If no, go to question 8.6.2. 8.6.1.1 Female condoms Yes……………………………………………….1 No………………………………………………...2 8.6.1.2 Male condoms Yes……………………………………………….1 No………………………………………………...2 8.6.2 Combined Oral Contraceptives Yes……………………………………………….1 No………………………………………………...2 8.6.3 Combined Injectable Contraceptives Yes……………………………………………….1 No………………………………………………...2 8.6.4 Progestin-Only Pills Yes……………………………………………….1 No………………………………………………...2 8.6.5 Projestin-Only Yes……………………………………………….1 Monitoring and Evaluation Plan, Year 1 86 Injectable No………………………………………………...2 8.6.6 Implants Yes……………………………………………….1 No………………………………………………...2 8.6.7 IUDs Yes……………………………………………….1 No………………………………………………...2 8.7 Child welfare clinic Yes……………………………………………….1 No………………………………………………...2 8.8 Under five immunization Yes……………………………………………….1 No………………………………………………...2 8.9 Pharmaceutical services Yes……………………………………………….1 No………………………………………………...2 8.10 Breast cancer screening Yes……………………………………………….1 No………………………………………………...2 8.11 Cervical cancer screening Yes……………………………………………….1 No………………………………………………...2 8.12 Diagnostic service Yes……………………………………………….1 No………………………………………………...2 If no, go to question 8.13. 8.12.1 Laboratory Yes……………………………………………….1 No………………………………………………...2 8.12.2 X-ray Yes……………………………………………….1 No………………………………………………...2 8.12.3 Ultrasound scan Yes……………………………………………….1 No………………………………………………...2 8.12.4 ECG Yes……………………………………………….1 No………………………………………………...2 8.12.5 Other diagnostic services [Please specify] 8.13 Catering services Yes……………………………………………….1 No………………………………………………...2 8.14 Specialist services [Please specify] Score in Emergency Response __________ points Score in Quality Assurance __________ points Score in Safe Motherhood __________ points Monitoring and Evaluation Plan, Year 1 87 Score in Advocacy/Health Promotion __________ points Total score __________ points Grade A+ Excellent……………………………………..1 A Very Good…………………………………….2 B+ Good…………………………………………3 B Fairly Good……………………………………4 C Fair…………………………………………….5 D Pass…...………………………………………6 E Fail……………………………………………..7 Monitoring and Evaluation Plan, Year 1 88 QUALITY ASSESSMENT TOOL GHANA REGISTERED MIDWIVES ASSOCIATION QUALITY ASSESSMENT TOOL Ghana Saving Private Maternity Homes Project GRMA 1/1/2015 Monitoring and Evaluation Plan, Year 1 89 PREAMBLE PURPOSE This instrument reflects facilities and equipment available and the skills/activities to be performed by service providers at all levels (within the maternity home) as described in the national Safe Motherhood (SM) Protocols Document. The use of this questionnaire helps to (among other reasons) perform quality assessment and support supervision for multiple purposes including: • Determine the baseline for the SMH project • Identify skills and service gaps of the PMH / comprehensive needs assessment • Quality of services provided and continuous performance improvement WHO ARE THE POTENTIAL USERS? The checklist should be used by qualified supervisors and trained enumerators with required technical and clinical skills in SM areas. Managers at various levels should utilized data collected to bring improvement into their facilities. WHAT ARE THE KEY STEPS Trainers/supervisors and enumerators team planning • What are the team objectives for the visit? • Who will visit whom, when and with what? • What are the expected output/outcomes of the visits? • Are tools /checklist ready and are enumerators trained? • Is Regional Chairperson of GRMA and person to be visited aware of date and purpose of visit? VISIT IMPLEMENTATION • Courtesy visit to person in charge of facility (purpose of visit, how the visit will be implemented, who are the persons to be involved, end of visit discussions, etc.) • Proceed with concerned person (explain purpose of visit and that you are not for fault finding, invite to open/honest discussion, discuss the process, proceed, give feedback). VISIT WRAP UP • Discuss findings both positive and those needing improvement from the visit. • Discuss next actions to be taken. TEAM EVALUATION MEETING • What happened? • What are the major outcomes? • What should be the next steps (next visit objectives) • Document findings of the visit. • What are the lessons learnt? • What are the recommendations? Monitoring and Evaluation Plan, Year 1 90 FACILITY INFORMATION Name of Facility Location Address District Region Postal Address Telephone (Landline, if any) Mobile Phone Number (s) E-mail Website Facility Licensed by License Number Date of Expiry Name of Head of Facility Contact number (s) of Head E-mail of Head/Website Name of Owner of Facility Contact Number of Owner Gender of Owner E-mail of Owner (if different from facility head) RANGE OF SERVICES Please tick services provided in the facility SERVICE DETAILS/REMARKS Monitoring and Evaluation Plan, Year 1 91 1. Emergency services (24 hours) YES ( ) NO ( ) 2. Out-patient services YES ( ) NO ( ) 3. In-patient services YES ( ) NO ( ) 4. Maternity services (Tick details) YES ( ) NO ( ) ANC Delivery PNC 5. Family planning counselling YES ( ) NO ( ) 6. Provision of Family planning methods YES ( ) NO ( ) Condoms (tick details) YES ( ) NO ( ) Female Male Combined Oral Contraceptives YES ( ) NO ( ) Combined Injectable Contraceptives YES ( ) NO ( ) Projestin-Only pills YES ( ) NO ( ) Projestin-Only Injectable YES ( ) NO ( ) Implants YES ( ) NO ( ) IUDs YES ( ) NO ( ) 7. Child welfare Clinic YES ( ) NO ( ) 8. Under Five immunization YES ( ) NO ( ) 9. Pharmaceutical services YES ( ) NO ( ) 10. Breast cancer screening YES ( ) NO ( ) 11. Cervical Cancer Screening YES ( ) NO ( ) 12. Diagnostic service YES ( ) NO ( ) Laboratory YES ( ) NO ( ) X-ray YES ( ) NO ( ) Ultrasound scan YES ( ) NO ( ) ECG YES ( ) NO ( ) Other diagnostic services (specify) YES ( ) NO ( ) ………………………………… 13. Catering services YES ( ) NO ( ) 14. Specialist services (specify below) YES ( ) NO ( ) Monitoring and Evaluation Plan, Year 1 92 GENERAL INFORMATION Year maternity home was opened: …………………………..………………………………………...…… Qualification of Head /Manager: A. Nursing: Certificate No: …………………………………………………………………………..………….. Date of qualification: …………………………….…………………………………….…………… Name /place of training school: ………………………………………………….………………… B. Midwifery: Certificate No: ……………………………………………....…………………..………………….. Date of qualification: ……………………………………………………………..………………… Name/place of training school: ……………………………………………………..………………. Any other Date of qualification: …………………………………………………………………… C. Any other qualification: (professional) ………………………………………….……………………… D. GRMA Registration of midwife who owns/heads the facility………….……… Date: ………………... E. GRMA Registration of all midwives who work in facility i. …………………………… Date: …………………. ii. ……………………………. Date: …………….…… iii. …………………………..... Date: …………………. iv. ……………………………. Date: ………….……… v. …………………………….. Date: …………………. vi. ……………………………. Date: ………...……….. PURPOSE OF VISIT Follow up: …………………………………………………………………………………………………… Needs assessment: ……………………………………………………………..…………………………… Quality care: ………………………..………………………..…………………………..…………………... Performance: ……………………………………………………………………………...………………… Team = Specific Objectives: ………………………………………………………………………………… Ask /discuss any success: concerns/problems and address them as appropriate during the visit How is your work progressing? What successes have you seen? What concerns or problems do you have? Is there anything else you would like to ask or tell me? Monitoring and Evaluation Plan, Year 1 93 EMERGENCY RESPONSE (54 MARKS) 24 hrs Services 1. a. Do you have a joint duty roster? (Check for the roster for the past three months) Yes ( ) No ( ) 2 marks b. Is co-ordination of duty roster done by MW/Facility head? (Check for evidence) Yes ( ) No ( ) 3 marks 2. Has emergency team been trained for the past year? (Check for evidence) Yes ( ) No ( ) 5 marks 3. Have you made arrangement with any hospital in case of referral (Check for evidence) Yes ( ) No ( ) 5 marks 4. Does the PMH have links with a hospital in any of the following? - Clinical meeting Yes ( ) No ( ) 5 marks Check for evidence - Training Yes ( ) No ( ) 5 marks (Check for evidence) 5. Are the following protocols available and in use /displayed? (Select a midwife to interview and give full marks if displayed and correctly demonstrated by midwife) a. Eclampsia Yes ( ) No ( ) 2 marks b. PPH Yes ( ) No ( ) 2 marks c. APH Yes ( ) No ( ) 2 marks d. Diarrhoea Yes ( ) No ( ) 2 marks e. Convulsion Yes ( ) No ( ) 2 marks f. Anemia Yes ( ) No ( ) 2 marks g. URTI Yes ( ) No ( ) 2 marks h. Malaria Yes ( ) No ( ) 2 marks i. Shock Yes ( ) No ( ) 2 marks 6. Tracer Drugs availability (95 – 100 % availability) 2 marks 7. Do you have Resuscitation sets and functioning? - Sucker Yes ( ) No ( ) 3 marks - Oxygen Yes ( ) No ( ) 3 marks - Ambubag Yes ( ) No ( ) 3 marks (Check for evidence) QUALITY ASSURANCE (83 MARKS) 1. Do you have a functional Quality Assurance Team Yes ( ) No ( ) 3 marks (Check for minutes and copies of latest report) 2. Have you carried out the following surveys? a. Patient Satisfaction Yes ( ) No ( ) 3 marks b. Rational Use of Medicines Yes ( ) No ( ) 3 marks 3. Do you have Action Plans to address QA problems identified (Check for evidence) Yes ( ) No ( ) 3 marks 4. Is there privacy, confidentiality in place for patients Yes ( ) No ( ) 3 marks Monitoring and Evaluation Plan, Year 1 94 (Observe consulting room/ward for evidence i.e. screen) INFECTION PREVENTION 5. Infection Prevention Practices (observe the following) a. Neat environment (Landscaping) Yes ( ) No ( ) 2 marks b. Puncture proof container waste disposal (Check)Yes ( ) No ( ) 2 marks c. Waste bin lined with polythene bag(check) Yes ( ) No ( ) 2 marks d. Waste bins boldly labelled (check) Yes ( ) No ( ) 2 marks e. Segregation of waste carried out (observe) Yes ( ) No ( ) 2 marks f. Site for Disposing Placenta (Observe) Yes ( ) No ( ) 2 marks g. Veronica Bucket with water/running water Yes ( ) No ( ) 2 marks 6. Is the following done at this level? a. Sterilization of articles (instrument/material) Yes ( ) No ( ) 3 marks b. Correct dilution of chorine (strength) Yes ( ) No ( ) 3 marks 7. Do you use partographs to monitor labour ` Yes ( ) No ( ) 4 marks (Check used partographs) 8. a. Do you have delivery sets? Yes ( ) No ( ) 1 marks (cord scissors, cord clamps) episiotomy scissors, sutures, speculum, kidney dishes) b. If or equals 2 set or more are sited. (check) Yes ( ) No ( ) 2 marks c. Is it sterile and packed Yes ( ) No ( ) 1 mark 9. Do you have a complaint desk or suggestion box Yes ( ) No ( ) 2 marks Check for evidence 10. Display of services available Yes ( ) No ( ) 2 marks a. Directional signs (Observe) Yes ( ) No ( ) 2 marks b. Prenatal / Perinatal NEONATAL SURVIVAL 11. Do you have all your midwives trained in neonatal resuscitation? (Select a midwife to demonstrate) Yes ( ) No ( ) 3 marks 12. Do you have equipment for resuscitation (air way or suction machine)? Check for evidence. Yes ( ) No ( ) 3 marks 13. Do you undertake PNC for mothers/caregivers of children to educate them on the following? 14. (Check register / note book for the following) • Bed net usage Yes ( ) No ( ) 2 marks • Education on hygiene and sanitation Yes ( ) No ( ) 2 marks • Family Planning Yes ( ) No ( ) 2 marks INFANT SURVIVAL 15. Has your facility undertaken any nutrition training program? (Check for evidence) Monitoring and Evaluation Plan, Year 1 95 Yes ( ) No ( ) 2 marks 16. Is there any functional Mother support groups? (Check for evidence) Yes ( ) No ( ) 2 marks 17. Do you provide education on Infant feeding practices Check for evidence Yes ( ) No ( ) 2 marks 18. Is there a functional ORT corner in this facility? (Check for evidence) Yes ( ) No ( ) 2 marks UNDER FIVE SURVIVALS: 19. a. Are health workers trained in malaria case management? Yes ( ) No ( ) 2 marks b. Are prescribers using ACT? Yes ( ) No ( ) 2 mark c. Does your facility confirm suspected malaria cases? (Microscopy/RDT)? (Evidence from OPD Cards or folder) Yes ( ) No ( ) 1mark 20. a. Do you have the STG Document? Yes ( ) No ( ) 2 marks b. Are prescribers using the Standard Treatment Guideline (STG?) (Check for evidence) Yes ( ) No ( ) 2 marks 21. Are there laminated protocols of top 5 diseases? (Malaria, ART, diarrhoea, anaemia, malnutrition) Yes ( ) No ( ) 5 marks C. SAFE MOTHERHOOD (25 MARKS) ANTENATAL CARE 1. Is this facility practicing focus antenatal? Yes ( ) No ( ) 2 marks (Observe) 2. Maternal health records properly filled (BP recordings, comment on presence or absence of pedal oedema, temperature recordings, weight, urine test results especially for sugar and protein, hemoglobin measurements, malaria parasites) Yes ( ) No ( ) 2marks MATERNITY SERVICES 3. Diagnostic tools for mothers available at each location i.e. consulting room, labour ward (Thermometers, sphygmomanometers, stethoscope, adult weighing scale, tape measure, spatula) Check for evidence Yes ( ) No ( ) 2marks 4. Diagnostic tools for infants available (Tape measure, baby weighing scale, foetal stethoscope/sonicaid) Check for evidence Yes ( ) No ( ) 2marks 5. Basic equipment available (Wheel chair(s), patient trolley(s), drip stands, wall clock, delivery bed,) Check for evidence Yes ( ) No ( ) 2marks 6. Emergency medicines available (Oxytocin, cold chain for oxytocin, magnesium sulphate, ergometrine) Check for evidence Yes ( ) No ( ) 2marks 7. Emergency packs for PPH (gloves, syntocinon, IV Fluids, wide bore cannula, elbow length glove, giving set, urinary catheter, urine bags) Yes ( ) No ( ) 2marks 8. Emergency Pack for eclampsia (magnesium sulphate, IV fluids, gloves, Infusion, Delivery tray, General Infusion Tray. Yes ( ) No ( ) 2marks Monitoring and Evaluation Plan, Year 1 96 OBSTETRIC CARE 9. Are SM protocols laminated and displayed Yes ( ) No ( ) 1mark 10. Is a referral system in place? (Check for referral book) Yes ( ) No ( ) 1 mark 11. Community transport system pre-arranged? (Check ) Yes ( ) No ( ) 2 marks POST NATAL 12. Postnatal process correctly followed and documented Yes ( ) No ( ) 2marks 1. Mother: (BP, examination of perineum, weight of mother, Lactation, family planning counselling) 2. Baby: ( temperature of baby, weight of baby, , breast feeding, Observe / check for evidence. FAMILY PLANNING 1. Is counselling protocol available and in use? Yes ( ) No ( ) 1 mark 2. Availability of stock out of FP logistics for the past 3 months (Check for Family Planning Reports) Yes ( ) No ( ) 2 marks ADVOCACY/HEALTH PROMOTION (8 MARKS) Advocacy/Health Promotion (Check for reports/notebook) 1) Is Health Education plan implemented in the following areas? - HIV/AIDS / STI Yes ( ) No ( ) I mark - Adolescent Health Yes ( ) No ( ) I mark - Regenerative Health & Nutrition Yes ( ) No ( ) I mark - Hypertension Yes ( ) No ( ) I mark - Diabetes Yes ( ) No ( ) I mark - Malaria Yes ( ) No ( ) I mark - Child Health Yes ( ) No ( ) I mark - Reproductive Health Yes ( ) No ( ) I mark Any other comment Monitoring and Evaluation Plan, Year 1 97 SCORING No. ITEM EXPECTED SCORE FACILITY SCORE 1 Emergency Response 54 marks 2 Quality Assurance 83 marks 3 Safe Motherhood 25 marks 4 Advocacy/Health Promotion 8 marks Grand Total 170marks GRADING No Score range Grade 1 160 - 170 A+ Excellent 2 145 - 159 A Very good 3 130 - 144 B+ Good 4 115 - 129 B Fairly Good 5 100 - 114 C Fair 6 85 - 99 D Pass 7 Below 85 E Fail DATE: ………………………………………………………………………………………………………………….. SUBDISTRICT: …………………………………………………………………………………………….…………… RESPONDENTS: ………………………………………………………………….……………………..……………... NAME OF ASSESSOR: ………………………………………………………...……………………...……………..….. TIME STARTED: ……………………….……………………. TIME ENDED: ……..……….………………………… NAME OF REVIEWER OF ASSESSOR……………………………………………………………….………………… SIGNATURE: …………………………………………………………………………………………………………... Monitoring and Evaluation Plan, Year 1 98 GHANA REGISTERED MIDWIVES ASSOCIATION (GRMA) 1st Circular Road, H/No. 11B, Cantonments, Accra, Tel: + 233-(0) 302-779348/772971, email￾ghanamidwives@yahoo.com PEER SUPPORT FEEDBACK FORM No. ITEM EXPECTED SCORE FACILITY SCORE 1 Emergency Response 54 marks Strengths Gaps 2 Quality Assurance 83 marks Strengths Gaps 3 Safe Motherhood 25 marks Strengths Gaps 4 Advocacy/Health Promotion 8 marks Strengths Gaps Grand Total 170marks GRADING No Score range Grade 1 160 - 170 A+ Excellent 2 145 - 159 A Very good 3 130 - 144 B+ Good 4 115 - 129 B Fairly Good 5 100 - 114 C Fair 6 85 - 99 D Pass 7 Below 85 E Fail DATE: ………………………………………………………………………………………………………………….. SUBDISTRICT: ………………………………………………………………………………………………………… RESPONDENTS: ………………………………………………………………….…………………………………... NAME OF ASSESSOR: ………………………………………………………...…………………………………..….. TIME STARTED: ……………………….……………………. TIME ENDED: ……..……………………………… 99 Monitoring and Evaluation Plan, Year 1 99