Monitoring and Evaluation Plan, Year 1 i Contact No.: AID-614-A-14-00006 5 August 2015 (Updated September 1, 2017) This document was produced for review by the United States Agency for International Development. It was prepared by Banyan Global for the Saving Maternity Homes in Ghana, Innovate for Health Project. SAVING MATERNITY HOMES IN GHANA, INNOVATE FOR HEALTH Monitoring and Evaluation Plan, Year 4 F SAVING MATERNITY HOMES IN GHANA, INNOVATE FOR HEALTH MONITORING AND EVALUATION PLAN, YEAR 4 Contract No.: AID-614-A-14-00006 Submitted to: Juliana Pwamang AOR USAID Ghana DISCLAIMER The authors’ views expressed in this publication do not necessarily reflect the views of the United States Agency for International Development (USAID) or the United States Government. Saving Maternity Homes in Ghana Project, Y4 Monitoring and Evaluation Plan i PROJECT OVERVIEW Name USAID Saving Maternity Homes in Ghana, Innovate for Health Contract Number AID-614-A-14-00006 Start Date September 9, 2014 End Date December 31, 2017 Sub-grantee Ghana Registered Midwives Association (GRMA) Geographic Coverage Brong Ahafo, Central, Eastern, Northern, Volta, Western Reporting Period October 1, 2017 – December 31, 2017 Saving Maternity Homes in Ghana Project, Y4 Monitoring and Evaluation Plan ii Table of Contents PROJECT OVERVIEW....................................................................................................................... i ACRONYMS...................................................................................................................................... iii 1. INTRODUCTION ....................................................................................................................... 1 1.1 Project Overview..................................................................................................................................................1 1.2 Levels of Project Assistance................................................................................................................................1 1.3 Purposes of this Document................................................................................................................................3 2. PROJECT OBJECTIVES, THEORY OF CHANGE AND RESULTS FRAMEWORK............ 4 2.1 Project Objectives.................................................................................................................................................4 2.2 Theory of Change .................................................................................................................................................4 2.3 Smh Project Results Framework .......................................................................................................................7 3. MONITORING PLAN................................................................................................................. 9 3.1 Sources of Data and Data Collection Tools...................................................................................................9 3.2 Analysis, Reporting, and Dissemination .........................................................................................................11 3.3 Data Quality Assurance.....................................................................................................................................12 3.4 Ethical Considerations........................................................................................................................................12 3.5 Gender and Disability Considerations...........................................................................................................13 3.6 Critical Assumptions...........................................................................................................................................13 4. ENDLINE ASSESSMENT PLAN ............................................................................................. 14 4. 1. Purpose of the Endline Assessment................................................................................................................14 4. 2. Methods.................................................................................................................................................................14 4. 3. Data Collection Plan...........................................................................................................................................14 4. 4. Analysis And Reporting......................................................................................................................................16 5. OPERATIONS RESEARCH PLAN.......................................................................................... 18 5. 1. Purpose of Operations Research ....................................................................................................................18 5. 2. Focus Areas and Operations Research Events ............................................................................................18 6. EVALUATION PLAN............................................................................................................... 20 6. 1. Purpose of Evaluation and Design Updates ..................................................................................................20 6. 2. Evaluation Questions..........................................................................................................................................20 6. 3. Evaluation Methodology.....................................................................................................................................21 6. 4. Data Collection And Analysis...........................................................................................................................22 6. 5. Anticipated Constraints.....................................................................................................................................22 7. MEP RESPONSIBILITIES......................................................................................................... 23 ANNEX 1: BUSINESS FORMALIZATION SCORE (BFS).......................................................... 24 ANNEX 2: INDICATORS TABLE.................................................................................................. 28 ANNEX 3: INDICATOR REFERENCE SHEETS .......................................................................... 41 ANNEX 4: PMH SURVEY............................................................................................................... 68 ANNEX 5: QUALITY ASSESSMENT TOOL ............................................................................... 85 ANNEX 6: DRAFT PMH EXTENSION SURVEY......................................................................... 97 ANNEX 7: QAT ADDENDUM..................................................................................................... 102 ANNEX 8: DRAFT EVALUATION IN-DEPTH INTERVIEW GUIDE................................... 1023 Saving Maternity Homes in Ghana Project, Y4 Monitoring and Evaluation Plan iii ACRONYMS BCSP Business Continuity and Succession Planning BFS Business Formalization Score BSS Business Strengthening Support CCB Clinical Capacity Building CDCS Country Development Cooperation Strategy C-TIP Countering Trafficking in Person DDL USAID’s Development Data Library DEA Data Envelopment Analysis DEC USAID’s Development Experience Clearinghouse DQA Data Quality Assessment EOI Expression of Interest ERC Ethics Review Committee FP Family Planning FY Fiscal Year GEM General Elimination Methodology GHS Ghana Health Services GoG Government of Ghana GPS Global Positioning System GRA Ghana Revenue Authority GRMA Ghana Registered Midwives Association HEFRA Health Facilities Regulatory Agency HIV Human Immunodeficiency Virus IR Intermediate Result KII Key Informant Interview LOP Life of Project M&E Monitoring and Evaluation MCH Maternal and Child Health MEP Monitoring and Evaluation Plan MNCH Maternal, Newborn and Child Healthcare MOH Ministry of Health NHIA National Health Insurance Authority NHIS National Health Insurance Scheme NMC Nurses and Midwives Council PMHs Private Maternity Homes PPP Preferred Primary Provider PSV Peer Support Visits QAT Quality Assessment Tool SME Small and Medium Enterprises SMH USAID Saving Maternity Homes in Ghana, Innovate for Health Project TA Technical Assistance USAID US Agency for International Development Saving Maternity Homes in Ghana Project, Y4 Monitoring and Evaluation Plan 1 1. INTRODUCTION 1.1 PROJECT OVERVIEW The USAID-funded, $1.78 million Saving Maternity Homes (SMH) in Ghana, Innovate for Health project, implemented by Banyan Global in partnership with the Ghana Registered Midwives Association (GRMA), focuses on revitalizing private maternity homes (PMHs) in Ghana to expand access to quality maternal and child health services. The project aims to facilitate ownership and/or management transformations that will result in increased capacity, improved quality, and expanded services. The project is targeting six regions in Ghana: Brong Ahafo, Western, Central, Eastern, Volta, and Northern. Ghana is experiencing rapid and sustained economic growth, but large challenges remain, particularly in the area of health. The Government of Ghana (GoG) has taken steps to address these challenges and improve access to healthcare services, most notably through the launch of the National Health Insurance Scheme (NHIS). However, access to healthcare, particularly in rural areas, remains limited. Private maternity homes are important providers of health services, especially in rural areas; however, in recent years the sector has experienced a steady decline as midwives have aged and facilities have closed. GRMA estimates that in the five years leading up to the start of the project, 100 PMHs closed. The SMH project, which began in September of 2014, aims to help address these challenges through increasing awareness of ownership and management transformations, and facilitating these transformations directly in the target regions. Specifically, the project: develops the capacity of PMHs in business continuity planning; brokers the transfer of declining PMHs to interested investors; leverages financing and investment – both in the form of small funds disbursements directly by the project, and third￾party investment – to improve quality and expand service delivery; provides business strengthening and clinical capacity building to new owners/managers; develops the capacity of GRMA; and conducts operations research to enhance the understanding of important issues and considerations related to the project’s design. Participating PMHs receive various “tiers” of project support, ranging from basic peer support visits (PSVs) led by GRMA, to training in business continuity and clinical services strengthening, to support for effectively implementing an ownership/management transformation. The SMH project theory of change posits that the project can take specific steps to facilitate ownership/management transformations, and that these activities will result in increased sustainability and expanded operations of PMHs in rural areas of Ghana, thereby improving access to maternal, neonatal and child health services. 1.2 LEVELS OF PROJECT ASSISTANCE The SMH project provides four main tiers of intervention to PMHs in the target regions: 1. Tier 1: Peer Support Visits – The lowest level of project assistance is for GRMA to provide PSVs. These visits involve GRMA reviewing a facility, identifying areas of improvement, and providing recommendations. Some facilities at this level participate in further capacity building activities or transformation assistance provided by the project, and some do not. PMHs that receive only PSVs, and do not attend other SMH project trainings or receive transformation assistance, are regarded as unassisted by the project. Saving Maternity Homes in Ghana Project, Y4 Monitoring and Evaluation Plan 2 Life of project (LOP) target and rationale: The project has set an LOP target of 108 facilities for Tier 1. This target is set equal to the actual achievement as of July 2015, when PSVs were administered in conjunction with the baseline assessment. While the initial list of PMHs in the target regions included 170 facilities, many facilities were either closed, or the key personnel were not available to receive the PSVs when the SMH/GRMA team arrived. Thus, GRMA was able to deliver PSVs to a total of 108 facilities. 2. Tier 2: Capacity Building Support – At this level of project assistance, PMHs receive capacity building support in the form of various SMH project trainings, including the Business Continuity and Succession Planning (BCSP) training, the Clinical Capacity Building (CCB) training, and the Business Strengthening Support (BSS) training. PMHs receiving this support are regarded as assisted by the project. LOP target and rationale: The project has set an LOP target of 146 facilities for Tier 2. During Year 1, 146 discrete PMH facilities attended the BCSP trainings, and all 146 of those facilities also attended the CCB trainings administered by GRMA. This number informed the setting of this target. 3. Tier 3: Transformation Support – This tier of project support is focused on PMHs who submit Expressions of Interest in pursuing a transformation, and are accepted by the project to receive specialized transformation support. At this level of project assistance PMHs receive all of the assistance included in Tiers 1 and 2, plus individual technical assistance (TA) focused on deal structuring, needs analysis, business plan support, and other pre-transformation assistance. PMHs in this tier are also eligible to apply for small grants from the project if funds are needed to complete the transformation, and to be linked to external sources of financing. (SMH project small grants are only approved for facilities that complete ownership/management transformations.) PMHs receiving this level of support are regarded as assisted by the project. LOP target and rationale: The project set an LOP target of 33 facilities for Tier 3. In focusing on the five transformation options, the SMH project is presenting PMHs with opportunities that are largely unfamiliar to them. The baseline assessment found that 14% of facilities had a manager who was not also the owner (management contract). Only 7.6% of facilities had more than one owner (partnership), while only 1.7% of facilities had been acquired in a sale. Given this low level of exposure to ownership/management transformations at baseline, the project anticipated that substantial effort would be required to sensitize PMHs to the idea of transformation, and even then only a small percentage of facilities would be interested in seriously pursuing transformations. These baseline findings have been reinforced through discussions with PMHs during project activities to date. Thus, the project set a target of 33 PMHs selected for transformation support. 4. Tier 4: Post-Transformation Support – This tier of project support focuses on facilities that actually complete a transformation. The project will provide tailored technical assistance to enhance the likelihood of business and financial sustainability in the new arrangement, as well as clinical technical assistance to ensure that transformed facilities provide high-quality health services. PMHs receiving this level of support are regarded as assisted by the project. LOP target and rationale: The project set an LOP target of 20 facilities for Tier 4. Extending the rationale for the Tier 3 target, the project believes that, while a total of 33 facilities will be selected for pre-transformation technical assistance, not all of these facilities will successfully complete a transformation. Given the information from the baseline and the learning acquired through Saving Maternity Homes in Ghana Project, Y4 Monitoring and Evaluation Plan 3 subsequent trainings and interactions, the project believes that 20 PMHs is a realistic target for number of facilities that transform, and receive post-transformation support. 1.3 PURPOSES OF THIS DOCUMENT Activities described in the Year 4 Monitoring and Evaluation Plan have three main purposes: 1. Accountability. To measure the extent to which the Saving Maternity Homes Project is meeting its objectives; 2. Project Learning. To provide Banyan Global, USAID, and project partners with information needed to evaluate the effectiveness of program implementation; and, 3. Operations Research. To provide the project, its partners, and the broader development community with practical information related to future interventions. This document will serve as the main monitoring and evaluation plan (MEP) to track and report the progress of project implementation. It will also be an important resource to engage the private sector providers and support the participating providers in performance management. The primary users of monitoring and evaluation learning and written products – for accountability, project learning, and operations research purposes – will be: 1. Project participants and members of participating communities; 2. Project partners, including USAID, GRMA, the Ghana Ministry of Health (MOH), and other government agencies and civil society organizations; 3. Banyan Global staff, including home office staff and Ghana field office staff; 4. Other organizations interested in private health sector development in Ghana. Additionally, the M&E process will continue to be used to build capacity among partner organizations, primarily GRMA. The Banyan Global M&E team will continue to involve GRMA in a discussion of appropriate intermediate results and indicators and will solicit their input in the development of all data gathering tools. Additionally, Banyan Global will continue to provide input into GRMA’s own on-going data collection processes and methods, with the objective of identifying areas in which these methods could be made more efficient and result in more useful outcomes for GRMA and its membership. SMH will continue to seek additional ways to include partner organizations in the monitoring, evaluation, and learning activities, as appropriate. The project’s MEP will gather data to track the hypothesized cause-and-effect linkages in the SMH theory of change. The plan will enable the project to assess whether project activities are indeed enabling participating PMHs to improve their business and financial practices, as well as to increase the efficiency and the quality of healthcare they are providing. It will also enable the project to assess whether there may be unintended (and likely temporary) decreases in efficiency and quality of care resulting from less experienced midwives or other owners taking over ownership and/or management of maternity homes. The project will perform ongoing validation of the MEP indicators by ensuring necessary data quality checks are in place. The SMH project monitors indicators by conducting ongoing reviews in efforts to deal with challenges and overcome them in a proactive and pragmatic way. Saving Maternity Homes in Ghana Project, Y3 Monitoring and Evaluation Plan 4 2. PROJECT OBJECTIVES, THEORY OF CHANGE AND RESULTS FRAMEWORK 2.1 PROJECT OBJECTIVES The assistance provided by this project enables revitalized maternity homes to capitalize on the opportunity presented by the NHIS, while responding to the demand for quality health services in rural areas. Ultimately, this program will increase access, improve efficiencies and expand quality maternal and child healthcare and family planning (FP) services in rural areas. This work directly supports the USAID/Ghana Country Development Cooperation Strategy (CDCS) results framework in supporting Development Objective 3: Equitable Improvements in Health Status. Specifically, this work supports IR 3.1: Increased access to integrated health services, and IR 3.3: Strengthened and responsive health systems. Additionally, these improvements support the Government of Ghana’s strategic policy objectives, as well as the Ghana Global Health Initiative’s IR 1.1: Increased access to quality reproductive health and HIV care. 2.2 THEORY OF CHANGE The SMH project theory of change (see Figure 1) is focused on the operational and management efficiency, business and financial sustainability and quality improvements that are expected to result from ownership/management transformations. Figure 1. SMH Theory of Change1 1 During a review of the project’s M&E plan in FY17, the SMH project refined step 1 of the theory of change. Originally, step 1 captured the “PMH awareness of the beneficial impacts of operational efficiency and financial sustainability. The project revised the language to state “PMH awareness of business continuity and financial 1. PMH awareness of the beneficial impacts of business continuity and financial sustainability increased 2. Ownership and management transformations completed 3. Operational efficiency, business and financial sustainability, and quality of clinical services in transformed facilities improved 4. Transformed facilities expanded Saving Maternity Homes in Ghana Project, Y4 Monitoring and Evaluation Plan 5 DEFINITIONS To have a clear understanding of the theory of change, there are several key concepts that must be defined: Transformation: Transformation is defined as occurring when; 1) the PMH facility is legally sold to a new owner or investor (sale); 2) the existing owner brings on a partner who has both an ownership stake and an active management role (partnership); 3) the existing owner retains ownership, but appoints a manager with a job description that gives this person substantial authority over operational aspects of facility management (management contract); 4) the existing owner retains ownership and leases the facility to a new manager/operator (lease); or 5) the existing owner retains ownership and management, but hires a new individual who brings a specific skill set that enables the facility to offer additional health services (service diversification contract).2 Business and Financial Sustainability: Levels of business and financial sustainability will be measured by the “business formalization score” (BFS), a composite indicator of key facility-level performance indicators demonstrating the capacity of a PMH to sustain its business operations in the future. This score includes clearly verifiable considerations, such as whether the business is legally incorporated, whether it is in compliance with regulations and licensing requirements, whether it is currently accredited with National Health Insurance Association (NHIA), whether the business has written contracts with partners and suppliers, and whether it has a bank account (personal or corporate). The BFS of each facility receiving project assistance will be assessed at the baseline and then again at the endline to measure changes in business and financial sustainability following the intervention. Expansion: A transformed facility has expanded when, following the intervention, it has added new health services (e.g., neo-natal services, family planning services, malaria, diarrhea, inpatient, etc.), increased capacity (e.g., number of beds, increased staff, additional consultation rooms), or increased partnerships (e.g., health franchise with MSI, DKT, IPAS etc.). Such expansion should ultimately increase the number of clients. HYPOTHESES This theory of change rests on four hypotheses, as follows: 1. It is hypothesized that project trainings, presentations, direct outreach and other sensitization efforts will result in assisted maternity homes having a better sense of the potential benefits of business continuity and financial sustainability. Anticipated outcome: PMH awareness of the benefits of business continuity and financial sustainability increased. 2. It is further hypothesized that, in a subset of the project’s beneficiaries, an increased awareness of business continuity and financial sustainability will lead to a desire to complete a transformation. The project will assist these facilities to prepare for transformation, and will assist in identifying potential new owners, partners, leasees, and employees (managers and technicians). Anticipated outcome: Ownership and management transformations completed. sustainability” because the project believes that business continuity more precisely describes the objectives of the steps in the theory of change. 2 During the second quarter of Year 2, the SMH project added a new type of transformation, the “service diversification contract”, to the existing transformation types. The “service diversification contract” occurs when a facility hires a new employee who enables it to diversify and expand the clinical services it offers. While not included in the original transformation types, the service diversification contract was derived from an operational research finding that, with the pending arrival of capitation, an additional skill set and the ability to offer additional health services are critical to continuity and sustainability. Saving Maternity Homes in Ghana Project, Y4 Monitoring and Evaluation Plan 6 3. It is further hypothesized that transformed facilities will engage in targeted efforts to improve the operational efficiency, business and financial management, as well as the quality of care of their newly acquired PMHs. This hypothesis is based on the assumption that these new owners/managers, or facilities that have diversified their clinical services, will have taken this action in order to improve financial performance (as well as clinical service delivery), and will therefore work to increase the efficiency of the facility and improve the quality of the health products and services available in order to ensure the financial success of their investments and efforts. Anticipated outcome: Operational efficiency, business and financial sustainability and quality of clinical services in transformed facilities improved. 4. It is further hypothesized that improvements in operational efficiency, business and financial sustainability, and quality of care will result the expansion of transformed facilities, and the increased capacity of PMHs to administer these services in the target regions. As described in the definition above, this expansion may manifest in adding new health services, increasing capacity, and/or increasing partnerships. Anticipated outcome: Transformed facilities expanded. In order to support the long-term sustainability of these project interventions, the SMH project will work directly with the GRMA, building its capacity to deliver this assistance after the project has ended. This assistance will include efforts to link GRMA to non-United States government sources of funding in Ghana. Saving Maternity Homes in Ghana Project, Y4 Monitoring and Evaluation Plan 7 2.3 SMH PROJECT RESULTS FRAMEWORK RESULTS FRAMEWORK: SUMMARY DIAGRAM The following summarizes the SMH results framework that captures the overall long term goal, strategic objective and intermediate results as they relate to the three key project components: pre-transformation activities, transformation activities, and sustainability activities. USAID DO 3: Equitable Improvements in Health Status IR 3.1: Increased access to integrated health services IR 3.3: Strengthened and responsive health system PROGRAM GOAL: Increase private maternity homes' ability to sustain and grow quality of health services in the underserved areas of Ghana STRATEGIC OBJECTIVE: PMHs sustained, MCH services expanded, and efficiency and quality improved IR 3: SUSTAINABILITY Business and programmatic capacity of PMHs and GRMA increased Sub-IR 3.1: Capacity of PMHs to sustain, increase and improve service delivery strengthened Sub-IR 3.2: Increased capacity of PMHs to manage businesses and offer health services Sub-IR 3.3 Management and programmatic capacity of GRMA increased IR 2: TRANSFORMATION PMHs under new ownership / management transformed and improved/expanded health service delivery Sub-IR 2.2: Funding provided and leveraged by the project Sub-IR 2.1: PMH ownership / management transformations brokered IR 1: PRE￾TRANSFORMATION PMH owners and managers ready for ownership / management transformation Sub-IR 1.1: PMHs interested in and ready for transformation identified Sub-IR 1.2: Capacity of PMHs for ownership / management transformation strengthened Saving Maternity Homes in Ghana Project, Y3 Monitoring and Evaluation Plan 8 2.3.2 PERFORMANCE INDICATORS PER RESULTS LEVEL Below is a summary of the performance indicators for each results level. Further explanation can be found in Annex 2: Indicator’s Table. Please see the Year 1 MEP which details the rationale of changes made to the original indicators as captured in the Baseline Assessment Report. Sub-IR 3.3 Management and programmatic capacity of GRMA increased  Indicator 24: Number of GRMA staff and volunteers who receive capacity building assistance. Sub-IR 3.2: Increased capacity of PMHs to manage their businesses efficiently  Indicator 22: Number of one-on-one business training events conducted  Indicator 23: Percentage of assisted PMHs maintaining core financial documents (cash log, profit and loss statement, balance sheet) Sub-IR 3.1 Capacity of PMHs to sustain, increase, and improve service delivery strengthened  Indicator 20: Percentage of assisted, non￾accredited PMHs applying for NHIA accreditation that successfully become accredited  Indicator 21: Percentage of assisted PMHs that receive or maintain NHIA accreditation after transformation. Sub-IR 2.2: Funding provided and leveraged by the project  Indicator 16: Number of assisted PMHs that receive seed funding in the form of SMH small grants  Indicator 17: Amount of seed funding provided by the SMH project in the form of small grants  Indicator 18: Number of assisted PMHs that receive funds from non-project sources  Indicated 19: Amount of funding that assisted PMHs receive from non-project sources Sub-IR 2.1: PMH ownership/management transformations brokered  Indicator 11: Number of assisted active PMHs who complete an ownership/management transformation  Indicator 12: Number of inactive PMHs re-opened with new owners or managers  Indicator 13: Number of investment proposals for PMH transformations prepared  Indicator 14: Number of new/younger midwives interested in ownership and/or management of PMHs  Indicator 15: Percentage of PMHs that have a neutral or positive attitude towards the idea of ownership/management transformation. Sub-IR 1.2: Capacity of PMHs for ownership/management transformation strengthened  Indicator 7: Number of PMH owners who have received pre-transformation technical assistance  Indicator 8: Percentage of training participants who attain a score of 75% or higher on training post￾tests  Indicator 9: Number of unique PMH facilities trained by the SMH project  Indicator 10: Number of participants in SMH training courses Sub-IR 1.1: PMHs interested in and ready for transformation identified  Indicator 6: Number of assisted PMHs expressing interest in ownership/management transformation Strategic Objective: PMHs sustained, MCH services expanded, and efficiency and quality improved  Indicator 1: Percentage of assisted PMHs with increased financial sustainability  Indicator 2: Percentage of assisted PMHs with improved quality of services  Indicator 3: Number of assisted PMHs with improved technical efficiency  Indicator 4: Number of assisted PMHs that add new health services  Indicator 5: Number of outpatient visits received by assisted PMHs that undergo an ownership/management transformation Long-Term Goal: Increase private maternity homes’ ability to sustain and grow services in the underserved areas of Ghana IR1: PMH owners and management ready for ownership and/or management change IR2: PMHs under new ownership/ management continue health service delivery IR3: Business and programmatic capacity of PMHs and GRMA increased Saving Maternity Homes in Ghana Project, Y4 Monitoring and Evaluation Plan 9 3. MONITORING PLAN 3.1 SOURCES OF DATA AND DATA COLLECTION TOOLS In Year 4, the SMH project will utilize a series of additional sources and data collection tools developed in Year 3 as well as the same information sources and data collection tools articulated in the Year 1 MEP to collect the data needed to inform its M&E and operations research. These information sources/data collection tools are described below. Secondary Data. In order to continue building on the knowledge and best practices that have already been established by USAID and other development actors, and to provide context and background to its data collection and measurement activities, SMH has drawn on data produced by the MOH, GHS and NHIA. Nationally, the MOH/GHS receives data about the private health facilities through the district offices per the usual channel of data submission (monthly data returns). The project collected data from NHIA on number and geographic distribution of accredited PMHs during the baseline assessment in Year 1. For the endline assessment, the project plans to obtain the up-to-date number and geographic distribution of accredited PMHs from NHIA. Source data acquired through these channels will be stored by the SMH project, which will ensure its security and confidentiality. PMH Survey. SMH designed a customized PMH survey during Year 1 that was used in the baseline assessment to provide baseline data on key PMH indicators. The survey focuses on the health services and the business aspects of the PMHs, and was developed taking into account the requirements of GHS and NHIA. In Year 4 Q1, the SMH project will conduct the endline assessment using the same PMH survey to collect endline data on key PMH indicators. During the endline, the project will attempt to administer the survey to all PMHs in the target regions, including the 118 PMHs who participated in the baseline, using the ona.oi mobile data collection platform.3 The PMH survey is included in Annex 4. PMH Extension Survey. In preparation for the endline assessment, the SMH project developed a customized extension to the PMH survey, to capture additional data required to inform the project’s performance indicators and build upon the project’s operations research. The survey focuses on a facility’s change in awareness, attitudes and motivation towards improving the health services and the business aspects of the PMHs, level of participation in ownership/management transformation activities, and efforts to obtain external financing. During the endline, the project will administer the survey to PMHs immediately following the administration of the PMH survey. The draft PMH extension survey is included in Annex 6. Quality Assessment Tool. GRMA developed a quality assessment tool (QAT) in Year 1 of the SMH project. The QAT was designed to capture the various aspects of quality of service delivery provided by a well-run PMH. The QAT was developed taking into account the regulatory requirements of the Health Facilities Regulatory Agency (HEFRA), GHS, and NHIA, as well as the principles of the best practices in midwifery. The QAT was constructed to cover all aspects of PMH quality, enabling the SMH project to create an index of service quality for all PMHs. The index has been used to benchmark all PMHs together with other data collected at the baseline. During the endline, the project will attempt to administer the QAT to all PMHs in the target regions, including the 118 who participated in the baseline. The QAT is included in Annex 5. 3 During the baseline assessment, the SMH project used formhub.org, a mobile data collection platform, to gather and record survey data. During the life of the project, formhub.org ceased to be maintained and updated by the platform host. In light of this, the SMH project transferred it survey tools to the ona.oi platform (https://ona.io/home/), which is similar to formhub.org. Saving Maternity Homes in Ghana Project, Y3 Monitoring and Evaluation Plan 10 QAT Addendum. In Year 3, GRMA expanded the QAT in efforts to incorporate gender, domestic violence, persons with disabilities, and countering trafficking in persons (C-TIP) considerations. The QAT addendum will gather information on the facility’s participation and completion of gender, persons with disabilities and trafficking in persons training to provide appropriate care to their patients as well as their abilities to recognize cases of abuse and trafficking. The expansion of the QAT also gathered information around the facility’s awareness of available resources and referral mechanisms. The QAT addendum is included in Annex 7. GRMA revived its practice of conducting ‘peer support visits’ (PSV) during Year 1 of the SMH project. These visits are intended to monitor and improve quality of service, and were a practice of GRMA in years prior to the SMH project but had been discontinued due to lack of funding. PSVs consist of administering the QAT, immediately tallying the score and grade, and then providing feedback directly to the PMH regarding areas where they did well and areas in which they could improve. During the baseline, GRMA traveled along with SMH enumerators, and administered the QAT and PSV during the same visit at which enumerators administered the PMH survey. This process will be repeated during the endline with the addition of the administration of the PMH extension survey and QAT addendum. As completed in Year 1 during the baseline, PSVs will be led by the regional GRMA executive who will be accompanied by a member of the Greater Accra Chapter of GRMA, which is the chapter working directly with the SMH project. Additional Project Records and Project Documents. The project will continue to maintain records related to its technical assistance and capacity building activities. These documents provide additional data for the project and include, among other things, the following forms.  Trainee Registration Forms – PMHs completed these forms at the beginning of each training course conducted by SMH. These forms are related to Tier 2 of project assistance.  Training Pre- and Post-Test Forms – PMHs completed these forms at the beginning and end of each SMH training course. These forms relate to Tier 2 of project assistance.  Supply-Side4 Expression of Interest (EOI) Forms – PMHs submit these forms to the project to indicate officially their interest in receiving project assistance to pursue an ownership/management transformation. These forms are submitted by PMHs receiving Tier 2 of project assistance who want to be considered for participation in Tier 3 of project assistance.  Demand-Side5 Expression of Interest Forms – Public sector midwives, non-midwife medical professions, and non-medical sector potential investors submit these forms to the project to indicate officially their interest in receiving project assistance to pursue an ownership/management transformation.  Technical Assistance Record Forms – The SMH project uses these forms to record the topics discussed and action items decided during each one-on-one technical assistance visit to a PMH. These forms relate to Tier 3 of project assistance. 4 “Supply-side” is a term that refers to one side of the SMH project’s “transformation equation.” A supply-side party is defined as a PMH interested in undergoing transformation. 5 “Demand-side” is also a term that refers to one side of the project’s transformation equation. A demand-side party is defined as a prospective new owner/partner/lease/manager/employee of a PMH. Saving Maternity Homes in Ghana Project, Y3 Monitoring and Evaluation Plan 11  GRMA Organizational Assessment Forms – The SMH project developed a capacity assessment form designed for the purpose of identifying the association’s organizational strengths and weaknesses, as well as the priority capacity areas, which should be retained for improvement. These forms relate to supporting the capacity development of sub grantee, GRMA and specifically the project’s Intermediary Result 3: Business and Programmatic Capacity of PMHs and GRMA increased. 3.2 ANALYSIS, REPORTING, AND DISSEMINATION Some data acquisition and analysis activities occur at specific points during the project cycle in order to establish benchmark values against which to judge project performance. The project also engages in ongoing data acquisition and analysis in order to inform project implementation. Additionally, in Year 4 dissemination of project learning will also be a priority. Following is a summary of planned analysis, reporting, and dissemination. ANALYSIS AND REPORTING Baseline Assessment Report. The Baseline Assessment Report summarizes the baseline survey methodology and implementation and the results of the research data, including information about the business performance and health services provided by the PMHs. The report was originally submitted to USAID on June 23, 2015 with a final revised version incorporating USAID comments submitted on October 21, 2015. Endline Assessment Report. The Endline Report will have a similar structure to the Baseline Assessment Report and will be based on the project’s endline survey conducted in Year 4 Q1. In addition to the results of the PMH survey, PMH Extension, QAT and QAT Addendum, the Endline Report will include qualitative data in order to gain a deeper understanding of the findings from the quantitative data collected from these tools. For more on the Endline Assessment, see the section below titled: Endline Assessment Plan. Project Evaluation Report. The Project Evaluation Report will summarize the outcomes of an end of project evaluation. This evaluation is designed to provide some understanding of the effectiveness of project activities. For more on the project evaluation, see the section below titled: Evaluation Plan. Ongoing Monitoring, and Annual and Quarterly Reports. The project will continue to produce Annual and Quarterly Reports (as required by USAID). The project will also continue to conduct ongoing monitoring in order to inform project implementation and decision-making. DISSEMINATION Operations Research Dissemination Events. The project has generated a variety of learning through project operations. Towards the end of Year 4 Q1, the project will summarize and disseminate that learning during the end of project technical meeting. The meeting will highlight selected findings from operations research, including: key preparations for the arrival of NHIA capitation, access to finance for PMHs, and quality and efficiency outcomes of the SMH project. For more on the operations research dissemination events, see the section below titled: Operations Research Plan. End of Project Event: As Year 4 is the final year of the project, the SMH team will also organize and host an end of project technical meeting in December 2017. This event will highlight learning acquired during the project, as well as findings from the endline assessment and the project evaluation. Saving Maternity Homes in Ghana Project, Y3 Monitoring and Evaluation Plan 12 3.3 DATA QUALITY ASSURANCE The SMH project will continue to work towards complete, accurate and reliable data. The SMH project effectively collects two types of data: large amounts of survey data connected to the baseline assessment and the endline assessment, and smaller amounts of ongoing monitoring data, much of which is done directly by the project staff through the course of implementing activities (such as collecting EOIs from PMHs interested in transformation). DQA for each type of data is described below. DQA FOR SURVEY DATA During the baseline, the project followed best practice data collection, data entry, and data cleaning protocols. These protocols included the following:  Pilot testing of survey questionnaires.  Rigorous training of survey enumerators.  Data collection using mobile data collection devices that automatically sync with an online database when connected to the internet, leaving no need for manual data entry.  Consistent participation in data collection efforts by the Team Lead and other core project staff, enabling the project to address any data quality issues in an ongoing way.  Regular monitoring of data submitted by Banyan Global home office staff, to ensure consistency, and completeness.  Data cleaning by a qualified statistician.  Discrepancies resolved through checking with the data collection team, and through callbacks to respondent when necessary. The project also intends to collect survey data in relation to the endline assessment. For the endline, the project will utilize all the same data collection protocols employed during the baseline. DQA FOR ONGOING MONITORING DATA Given the small number of participants in the SMH project, and the small number of facilities considering or completing transformation, the data requirements for ongoing monitoring are modest. The indicators that do not rely on survey data rely on sources of data collected directly by project staff. These sources of data include expression of interest forms, TA record forms, or trainee registration forms. Other indicators rely on project records, such as records of the number of PMHs that complete a transformation (Indicator 11), or the number of PMHs that receive seed funding in the form of small grants (Indicator 16). For these data collection efforts, the project monitors the accuracy of records as they are submitted, and ensures that all original documentation (such as EOIs) is retained in hard copy in the project office. Banyan Global home office staff perform a limited audit of this original documentation during each management visit. The limited amount of data required, combined with the direct participation of core project staff in collection and the supportive oversight from home office staff, ensure that data required to inform ongoing monitoring and reporting is of sufficient quality. 3.4 ETHICAL CONSIDERATIONS The SMH project is committed to ensuring the protection of human subjects. The SMH project has performed periodic reviews of the research design and implementation procedures inclusive of ethical considerations and the protection of human subjects. Furthermore, after receiving guidance from E4H, the SMH project submitted the protocol and data collection tools to inform the endline assessment and evaluation to Ghana Health Services (GHS) Ethics Review Committee (ERC) to obtain ethical clearance and approval. E4H assisted the SMH project in reviewing the protocol and data collection tools and Saving Maternity Homes in Ghana Project, Y3 Monitoring and Evaluation Plan 13 supported the submission of the application to the ERC in July of 2017. The project expects to receive feedback from the ERC in early September after the ERC August general meeting has concluded. The project will adhere to the rules and regulations outlined in the project’s agreement and the GHS ERC to ensure the protection of human subjects. These rules and regulations include obtaining written informed consent from respondents, communicating that participation is voluntary and ensuring the confidentiality of respondents by performing a de-identification process of data. 3.5 GENDER AND DISABILITY CONSIDERATIONS The SMH project is committed to thoughtful integration of gender and disability considerations into the implementation of project activities. The primary unit of analysis and intervention of the SMH project is the PMH facility. In terms of gender, the project has identified the gender of the PMH owner as the most appropriate point of analysis for integrating gender considerations, though a focus on client-level gender considerations has been incorporated as appropriate, primarily through supporting GRMA to maintain gender awareness in its activities. The project will continue to disaggregate data by sex, and make adjustments to project activities when feasible and appropriate. 3.6 CRITICAL ASSUMPTIONS The MEP as described in this document makes several critical assumptions. These include:  Political stability and security in the target regions and in Ghana broadly. Continued implementation of project activities depends on a political and security environment that allows for these activities. Should unexpected events arise that raise concerns about the political stability of target regions or of Ghana broadly, or if the security of project beneficiaries, partners, or staff is in any way uncertain, the project will curtail project activities to an appropriate extent until the situation has been resolved. Should that occur, the M&E strategy and activities as described in this document will be revisited.  Availability of funds for project activities. In order to carry out planned project and M&E activities, it is assumed that sufficient funds will be available. Should anticipated funding be reduced, the project will revise its implementation strategy, and MEP.  Availability of personnel from GRMA and partner organizations to conduct data gathering and analysis activities as required. The project design assumes that GRMA will be available to participate in data gathering activities for the baseline and endline, conduct analysis on findings from data gathered from the quality assessment tool, and participate in M&E activities in other ways. Additionally, the project anticipates participation from other project partners and potential key informants in terms of contributing insights and expertise, and playing other support roles. Should the participation and support of these organizations and actors be less than anticipated, this MEP will be revised accordingly. Saving Maternity Homes in Ghana Project, Y4 Monitoring and Evaluation Plan 14 4. ENDLINE ASSESSMENT PLAN 4. 1. PURPOSE OF THE ENDLINE ASSESSMENT The Saving Maternity Homes Project will carry out an endline assessment to ascertain the extent to which the SMH project met the established targets, and to document lessons learned for future activities. This assessment will include data collection and reporting for various PMH health and business performance indicators. The endline data collected will be assessed against the project’s indicators and targets, to inform pre- and post-intervention comparisons to measure project effectiveness. The endline assessment will draw on both secondary data from a variety of sources and stakeholders and primary data collected from PMHs using the PMH survey, PMH Extension Survey, QAT and QAT addendum. The results of the endline will be shared with USAID, regional and national stakeholders, partners and GRMA. 4. 2. METHODS In order to complete the endline assessment, the project will engage in the following activities:  Literature review. The project will review additional relevant literature published since the completion of the baseline assessment in order to build upon the background of quantitative and qualitative information on the challenges facing private maternity homes in underserved areas of Ghana.  Surveys. The project will deploy the same PMH Survey and QAT that was used for the baseline, in order to ensure comparable data. Upon reflection on the project’s implementation activities, the SMH project developed two additional customized surveys, the PMH Extension Survey and QAT Addendum, which will be administered during the endline assessment to generate useful additional learning and to contribute to the end of project evaluation.  Key Informant Interviews (KIIs). These interviews will be used to gather in-depth qualitative data on the challenges facing PMHs in Ghana and the considerations related to revitalization of these facilities through ownership/management transformations. 4. 3. DATA COLLECTION PLAN Surveys The PMH and PMH Extension surveys will be administered to all available PMHs in the target regions, including the 118 PMHs who participated in the baseline. The survey will again target the managing directors of the PMHs, as these individuals are likely to be the best able to provide the required information. The endline is currently envisioned to involve three-person assessment teams comprised of a SMH staff member to administer the PMH and PMH Extension Surveys, a GRMA SMH consultant to administer the QAT, QAT Addendum and deliver the PSV, and a GRMA Regional Representative to coordinate the meeting and provide additional support. The surveys will be built onto the ona.oi mobile data collection platform, and the team will utilize Android tablets to collect data. This form of data collection increases the quality of the data by reducing the range of decision-making available to the enumerator and minimizing input and coding errors. The data uploads Saving Maternity Homes in Ghana Project, Y3 Monitoring and Evaluation Plan 15 automatically online to a centralized database once the tablet is connected to the internet, thereby reducing the chances of raw data being lost or damaged. This strategy also facilitates ongoing data analysis by the M&E team, which will have access to the data online. The ona.oi platform also enables easy collection of GPS coordinates, allowing the project to incorporate geospatial characteristics into the data analysis as well as project planning and implementation. As implemented during the baseline, the data collection will be combined with PSVs conducted by GRMA. This approach helps PMH owners and/or managers to feel at ease, and thus increase the likelihood that they would provide full, accurate information. While partnering with GRMA offers valuable data-gathering benefits in terms of gaining access to, and the trust of, the PMHs in the target regions, this strategy also opens the possibility that GRMA’s presence might lead respondents to skew or misrepresent their business operations. The project will take specific steps to reduce the likelihood of GRMA’s presence adversely affecting the quality of the data collected, as described below. Both the PMH survey and the QAT involve the sighting of specific elements (documents in the case of the questionnaire, protocols and equipment in the case of the QAT). The project has designed the interview process so that, whenever possible, either GRMA is touring the facility and sighting the required elements while the PMH survey is being administered, or the survey enumerators are working with assistants and/or administrative staff to sight documents while GRMA is administering the QAT and providing feedback. This approach allows each team to leave the room when appropriate and has the added benefit of reducing the overall time that the visit requires. The project believes that this approach minimizes any potential information bias that might have resulted from having GRMA present when the PMH survey is being administered. Qualitative Data Collection In addition to the PMH survey and QAT, the SMH team will conduct a series of key informant interviews with representatives of key stakeholder groups. KIIs are in-depth and semi-structured interviews conducted with individual stakeholders. KIIs will take approximately 60 minutes to complete and will be conducted face-to-face, by phone, or by Skype. All KIIs will use a pre-prepared discussion guide, although interviewers will also be expected to follow-up with in-depth probing questions and to deviate from the discussion guide where appropriate. KIIs will be used to generate information and insights not available through the largely quantitative and closed-ended nature of the PMH survey and QAT, as well as to provide context and depth to the quantitative data gathered. Key informants will be selected for their experience and expertise in some area that has substantial bearing on midwifery practice in Ghana. The list will be drawn from midwifery policy-makers and regulators; the health insurance industry; health service managers; industry bodies of midwives in Ghana, and independent non-governmental agencies providing training to midwives, among others. During baseline, the project encountered some resistance in securing KIIs. Despite persistent attempts to secure input from key informants through in-person interviews, telephone interviews, or email, several individuals with whom the project had intended to speak were not available. In attempts to address the constraints encountered during Year 1, the project has expanded the list of respondents to account for the potential lack of availability of respondents during the endline. The list of potential respondents includes:  Deputy Chief Executive, Operations: National Health Insurance Authority  Director, Quality Assurance: National Health Insurance Authority  Director, Family Health Division: Ghana Health Service  Director Program Manager, Maternal Health, Family Health Division: Ghana Health Services  Director, Institutional Care Division: Ghana Health Service Saving Maternity Homes in Ghana Project, Y3 Monitoring and Evaluation Plan 16  Director, Nursing & Midwifery: Ministry of Health  Acting Deputy Director, Nursing & Midwifery: Ministry of Health  Registrar, Nursing and Midwifery Council  Head, Health Facility Regulatory Agency  Officer, Health Facility Regulatory Agency  President, Ghana Registered Midwives Association  Country Director, Jphiego (or other organizations providing training to midwives) Data from Other Sources During the months that the project is collecting survey data, the project will also be working with GRMA and other stakeholders to access the required data from other sources. (See the Indicators Table in Annex 2 for information on the data source for each indicator.) Informed Consent For all primary data collection, informed consent will be acquired before proceeding with any data collection activities. Respondents will be limited to persons able to give personal consent. Subjects not able to give personal consent (e.g. for cultural reasons, children or adolescents less than the legal age of consent, subjects with mental illness) should not be subjects of this study, and will not be interviewed. If the respondent is incapable of reading and signing the written consent form, the enumerators will read the form to them in order to acquire informed consent. 4. 4. ANALYSIS AND REPORTING Quantitative Data Analysis As was done for the baseline, data obtained from the PMH survey and the QAT as well as the PMH extension survey and QAT addendum will be reviewed, cleaned and analyzed to provide the information about the state of PMHs in selected regions of Ghana at the end of the project. Data will undergo a de￾identification process to ensure the protection of respondents. No personally identifiable information or PMH-specific information will be reported. Analytical conclusions will be summarized across all PMHs and according to selected sub-categories so as to allow the project to garner insights into the business operations and quality of service delivery of PMHs. Qualitative Data Analysis We will use a common approach to analyze the qualitative data gathered through KIIs. This approach includes the five steps described below: 1. Gather and Record the Data. In this step, the SMH team will transcribe and document the content discussed during each of the KIIs. As a best practice, the interviewers and moderators will seek to capture the raw data using an audio recording, with the permission of the respondents. 2. Summarize the Data. In this step, the SMH M&E team will read through all of the data collected and extract from it information that is relevant for understanding relevant considerations. We next use this information to create a summary document describing the ‘main evaluation findings.’ 3. Analyze the Data. In this step, we will perform a ‘content analysis’ of the data found in the summary document created in Step 2 in which we identify themes and patterns in the data (e.g., ideas, concepts, behaviors, interactions, incidents, or terminology and phrases used) and then organize them into coherent categories and sub-categories that summarize and bring meaning to the text. Saving Maternity Homes in Ghana Project, Y3 Monitoring and Evaluation Plan 17 4. Identify Patterns and Connections. In this step, we look for and identify patterns and connections both within and across categories, while simultaneously assessing the relative importance of different themes or highlighting subtle variations that may be important to our analysis. We also look for information that can help us understand the effectiveness and impact of project activities, the significance of those results, and the process by which they occur. 5. Review, Reflect, and Discuss. In this step, we conduct a thorough review of our analysis and findings. This step involves all team members in a process of review, reflection, and discussion as we move collaboratively toward a consensus on what the main findings are and what they mean. 6. Draw Conclusions. In this final step, we take the results of our analysis in Steps 1-5 and draw our conclusions, lessons learned and recommendations. Measuring Efficiency: Data Envelopment Analysis (DEA) In order to measure the efficiency of the PMHs in the target regions, the project will use a methodology known as data envelopment analysis (DEA). This methodology is designed to provide an understanding of how efficiently a health provider is using its resources to deliver health services. The DEA methodology quantifies various key inputs, including labor (clinical and non-clinical personnel), materials (pharmaceutical supplies, non-pharmaceutical supplies, other goods and commodities) and capital (building, medical equipment, vehicles, beds), together with outputs, in the form of the health services provided and the number of client visits the facility receives during a specific period of time (such as a typical calendar month). After these quantities have been collected (in the case of the SMH project, through the PMH Survey), the facility that is able to deliver the greatest amount of health services (outputs) given the resources at its disposal (inputs) is declared to be the most efficient, and that facility’s performance serves as the best practices frontier, or data envelop, for the sample. That facility receives a score of 1, and each other facility receives a score of between zero and one, based on how efficient it is in comparison to the frontier. This methodology will enable the project to benchmark each PMH in the target region according to their relative efficiency, and analyze changes in that efficiency over time, by comparing the scores obtained during the baseline and endline. 6 Dissemination of Findings and Analysis The Endline Assessment Report will summarize the endline survey results and will be shared with GRMA, GHS, NHIA as well as other stakeholders, following USAID review. SMH will organize an end of project technical meeting to present the findings of the endline survey results and lessons learned. Raw data will be provided upon request, although all specific identifying information will be kept strictly confidential and will only be available to project staff and to USAID. In order to ensure that privacy is maintained, dissemination will be limited to information and findings derived from summary analysis. The Endline Assessment Report will be submitted to USAID’s Development Experience Clearinghouse (DEC), and the dataset will be submitted to USAID’s Development Data Library (DDL) in a compliant machine-readable, non-proprietary format. 6 For an example of how DEA has been applied in Ghana, please see Jehu-Appiah, “Ownership and technical efficiency of hospitals: evidence from Ghana using data envelopment analysis”, Cost Effectiveness and Resource Allocation 2014, 12:9. Saving Maternity Homes in Ghana Project, Y4 Monitoring and Evaluation Plan 18 5. OPERATIONS RESEARCH PLAN 5. 1. PURPOSE OF OPERATIONS RESEARCH The purpose of operations research in the SMH project is to identify useful lessons and learning generated over the course of project operations that can be used to continue to revitalize and strengthen the Ghanaian private maternity home sector. Initially the project had planned to focus operations research only on the effect that transformation has on the efficiency of PMHs and the quality of health services that PMHs offer. However, over the first two years of implementation the project has generated a variety of learning from operations. In light of this broader learning, the project plans to expand the focus areas of the operations research to include other topics, including the arrival of NHIA capitation and challenges around access to finance. 5. 2. FOCUS AREAS AND OPERATIONS RESEARCH EVENTS Throughout the project, the SMH team has identified constraints and opportunities affecting the sustainability and expansion of PMH facilities. These efforts have resulted in three focus areas emerging as particularly important: the arrival of NHIA capitation, access to finance for PMHs, and the relationship between transformations and efficiency and quality. Each topic, as well as the operations research activities planned, are described below. The SMH project will present the learnings for each operations research topic during the end of project technical meeting and summarize the findings, learning and results in the end of project report. The presentation will be designed to showcase SMH project operations research, as well as the research and insights of other stakeholders and organizations able to contribute to the topic. OPERATIONS RESEARCH TOPIC: KEY PREPARATIONS FOR THE ARRIVAL OF NHIA CAPITATION The SMH project has generated valuable learning to date about how PMHs can best prepare for the arrival of capitation. In order to be selected as Preferred Primary Providers (PPPs), many PMHs will have to expand the range of services they offer. As the project worked with PMH facilities interested in transformations, many indicated that, in order for them to expand their range of services, they needed to hire additional personnel with the needed skill set. In response to this learning, the project added a fifth transformation type, called a “service diversification contract.” A service diversification contract is a type of employment contract that occurs when a new individual is hired who brings a specific skill set that enables the facility to offer additional health services. Furthermore, the project has incorporated issues of cash flow and other considerations related to the arrival of capitation into the Business Strengthening Support trainings. However, regarding the recent pronouncement by senior government officials that NHIS capitation being piloted in the Ashanti Region will be cancelled, the project will identify broader applications of this learning. For example, the project will explore how adding an additional clinician through a service diversification contract can generate improved health services and additional revenue, in addition to helping a facility prepare for capitation. The SMH project will summarize the findings, learning and results in the end of project report. OPERATIONS RESEARCH TOPIC: ACCESS TO FINANCE FOR PRIVATE MATERNITY HOMES Saving Maternity Homes in Ghana Project, Y4 Monitoring and Evaluation Plan 19 Access to finance for small and medium enterprises (SMEs) continues to prove to be a persistent challenge in Ghana generally, and the private health sector is no exception. High interest rates from commercial banks combined with devaluations in the Cedi and generally challenging macroeconomic conditions have led to difficulty in accessing financing in all sectors. For private health providers, the challenge is compounded by reimbursement delays from NHIA. These delays have meant that private health providers have difficulty accessing loans from any source that requires repayment in regular installments – not just commercial and microfinance banks, but also informal sources such as savings groups. The project has explored a variety of access to finance options, including large commercial banks, microfinance banks, local rural banks, and credit unions, and continues to monitor the available access to finance options described above to identify access points for PMHs to secure additional sources of financing. However, none of these options has proven to be a clear or reliable source of financing for PMHs. During Q1 of FY18, the project will present the findings of the structured investigation conducted in FY17 as part of its operations research mandate. The project worked with several transforming facilities to gain a detailed understanding of their current financial resources, their investment goals (purchase of equipment, hiring of trained staff, etc.), the potential health impact of realizing those goals, and the access to finance obstacles that they face. The SMH project will summarize the findings, learning and results of this investigation in the end of project report. OPERATIONS RESEARCH TOPIC: QUALITY AND EFFICIENCY OUTCOMES OF THE SMH PROJECT As shown by the research sponsored by GHS-affiliated research institutes, health facilities in Ghana are underutilized. These facilities have excess capacity and lower than optimal efficiency, leading to higher unit costs of operations. 7 These challenges extend beyond the public sector, and are also obstacles faced by private health providers in Ghana, including PMHs. The reasons why these facilities are underutilized are many, ranging from issues with government support and the broader health system, to challenges that individual clients face in accessing health facilities and paying for services. There are also demand constraints resulting in low usage of professional health facilities. SMH is interested in the potential impact that ownership and/or management transformations have on increasing the quality and efficiency of PMHs. Consistent with the original Operations Research Plan submitted within the MEP in August of 2015, the project plans to conduct a systematic investigation of the results of the SMH project model and activities on the quality of clinical care, and the efficiency of PMHs. Using data from the baseline and endline PMH surveys, SMH will develop and calculate a measure of technical efficiency for PMHs. Inputs and outputs for PMHs will be analyzed using data envelopment analysis (DEA) that will allow benchmarking of all PMHs according to their technical efficiency score.8 The SMH project will summarize the findings, learning and results in the end of project report. 7 Dalaba et al. Cost of maternal health services in selected primary care centres in Ghana: a step down allocation approach. BMC Health Services Research 2013, 13:287 http://www.biomedcentral.com/1472-6963/13/287. Navrongo Health Research Centre, Navrongo, Ghana 8 Data envelopment analysis (DEA) is a non-parametric method in operations research used to empirically measure productive efficiency of decision-making units (DMUs). The methodology is also used for benchmarking in operations management, where a set of measures is selected to benchmark the performance of manufacturing and service operations. Saving Maternity Homes in Ghana Project, Y4 Monitoring and Evaluation Plan 20 6. EVALUATION PLAN 6. 1. PURPOSE OF EVALUATION AND DESIGN UPDATES PURPOSE OF THE EVALUATION The most reliable approach to determining and attributing impact is an impact evaluation. The USAID Evaluation Policy defines an impact evaluation as an evaluation that seeks to measure the change in project outcomes that are attributable to (caused by) the project using methods that a construct a “credible and rigorously defined counterfactual” so as “to control for factors other than the intervention that might account for the observed change.”9 Creating a credible and rigorously defined counterfactual requires methods for comparing a treatment group of project participants and a control group of non-participants who are as similar as possible to project participants. Unfortunately, the SMH project does not have the resources to conduct an impact evaluation using the methods necessary to create a credible or rigorously defined counterfactual. However, given the innovative nature of the project design and the potential learning to be derived, the SMH project does plan to conduct a limited evaluation using less rigorous, but nonetheless sound, methods. The purpose of the evaluation will be to generate findings related to the effectiveness and impact of project activities along with lessons learned for the purpose of improving, replicating, or scaling-up aspects of the intervention, or to justify a more rigorous evaluation of future programming that takes a similar approach. ADJUSTMENTS IN EVALUATION DESIGN In the evaluation design that was included in the originally submitted MEP, the SMH project planned to focus on a single research question: What was the effect of SMH project activities on the financial sustainability (as measured by the BFS) of PMHs in the target regions? Upon further reflection, and consultations with external M&E experts, the project has determined that focusing only on financial sustainability as measured by the BFS would not provide sufficient findings regarding the effectiveness of project activities. Thus, the project has expanded the number of evaluation questions so as to evaluate each hypothesis in the theory of change. It should be noted that, while the number of evaluation questions has grown, the project believes that this adjustment will not increase the time and financial resources required as compared to the original design. In the revised design, the majority of the required data can be collected largely through adding questions to the PMH survey, PMH Extension Survey and KIIs, thus creating efficiencies with the data collection activities required for the endline assessment. 6. 2. EVALUATION QUESTIONS The SMH project evaluation will seek to answer the following four evaluation questions: 1. Evaluation Question for Step 1 of the Theory of Change. To what extent, if any, have SMH project activities increased the awareness of the benefits of business continuity and financial sustainability of PMHs in the target regions? 9 USAID, 2011. USAID Evaluation Policy. U.S. Agency for International Development, Washington, DC Saving Maternity Homes in Ghana Project, Y4 Monitoring and Evaluation Plan 21 2. Evaluation Question for Step 2 of the Theory of Change. To what extent, if any, have SMH project activities led to the successful completion of ownership/management transformations for PMHs in the target regions? 3. Evaluation Question for Step 3 of the Theory of Change. To what extent, if any, have SMH project activities led to the improved financial sustainability, operational efficiency and the quality of clinical services of transformed PMHs in the target regions? 4. Evaluation Question for Step 4 of the Theory of Change. To what extent, if any, have SMH project activities led to the expansion of transformed PMHs in the target regions? 6. 3. EVALUATION METHODOLOGY The project will employ a modified General Elimination Methodology (GEM),10 a non-experimental methodology that focuses on systematically identifying and eliminating alternative causal factors that could explain an observed result. While a non-experimental evaluation does not provide the level of rigor of an impact evaluation, it will enable the project to gain some understanding of project effectiveness and the impact that can be attributed to project activities. An outline of the planned evaluation methodology is as follows: 1. Define the group to be evaluated: This step is effectively completed. The group to be evaluated consists of private maternity homes in the target regions. 2. Define the outcomes of interest: This step is also effectively completed. In the case of the SMH project, the outcomes of interest are articulated in the theory of change, and can be measured through survey questions. (In the case of outcomes for which the existing tools are insufficient, the project will add questions to the PMH Survey and other tools as necessary.) 3. Draw up a list of competing explanations for the outcomes of interest: The project will rely on the expertise of project staff as well as key informant input to draw up a list of alternative explanations for why the outcomes of interest may have occurred. This list will represent the alternative hypothetical explanations for the outcomes of interest, and therefore the possible alternative causes to which the outcomes can be attributed. 4. Outline the “footprints” of each competing explanation: The project will identify the set of “footprints” for each competing explanation identified in the previous step. Michael Scriven, the author of the GEM methodology, describes these footprints as, “a sequence of intermediate or concurrent events, a set of conditions or a chain of events that has to be present when the cause is effective.”11 This step will occur prior to data collection, so that any additional questions or interviews can be incorporated into the data collection process. 5. Collect data regarding the extent to which the outcomes of interest have occurred and the extent to which the footprints of each competing explanation are present: The project will identify the extent to which the outcomes of interest have occurred through the collection and analysis of quantitative and 10 For an overview of GEM, see Howard White and Daniel Phillips, “Addressing attribution of cause and effect in small n impact evaluations: towards an integrated framework,” International Initiative for Impact Evaluation, Working Paper 15, 2012. 11 Scriven, M. (2008) A summative evaluation of RCT methodology and an alternative approach to causal research. Journal of Multidisciplinary Evaluation, 5 (9): 11-24. Saving Maternity Homes in Ghana Project, Y4 Monitoring and Evaluation Plan 22 qualitative data. Concurrently, the project will identify the extent to which the footprints of each competing explanation are present. The project will add any necessary questions to the existing research tools and collect qualitative data as needed. 6. Analyze data to determine the relative attribution of project interventions and of competing explanations. Once the data has been collected, the project will analyze data to gain some idea of the extent to which the outcome of interest can be attributed to project interventions, and the extent to which it can be attributed to competing explanations. While this methodology does not provide the same level of statistical rigor and confidence that is achievable in an impact evaluation, it does provide a systematic approach to identifying and attributing causation to various project outcomes. We believe that this approach will enable us to gain some useful understanding of the extent to which project activities caused the expected outcomes and desired results, while recognizing budget and time constraints. 6. 4. DATA COLLECTION AND ANALYSIS The evaluation will draw largely on the data collected during the endline assessment to identify the extent to which the outcomes of interest have occurred and are attributed to the project’s interventions. Building on the data collected during the endline, the project will conduct in-depth interviews with specific facilities, primarily transformed facilities, to gain a clearer understanding of the events and the causes. Please see Annex 8 for a draft evaluation in-depth interview guide. Once the evaluation data has been gathered, the project will conduct analysis to identify the extent to which each competing explanation is present and the extent to which each competing explanation played a causal role in the observed outcomes. The project will then summarize this analysis in an evaluation report, indicating clearly the extent to which project activities achieved the desired impact, and highlighting useful learning that could benefit future interventions. 6. 5. ANTICIPATED CONSTRAINTS The main constraint anticipated in the above evaluation design is bias that could affect the validity of the evaluation data and findings. On the one hand, there is a concern about respondent bias in respondents seeking to provide project personnel with positive answers out of a desire to maintain good relationships, and appreciation for the relationship that has been built. On the other hand, there is potential evaluator bias, in that the project itself will read the data to reflect positive outcomes in order to validate the project model or activities. The project will take various steps to address these biases. We will devise a detailed research design that includes triangulation of findings through multiple sources (for example, the PMH owner, the local GRMA representative, and local business owners familiar with the health services available in the area). Additionally, we will employ data analysis practices that ensure systemic analysis of the qualitative data, such as the data analysis process outlined in Section 4. Such practices mitigate the risk of seeing positive results in raw qualitative data, and they force the analyst to see true consistencies and valid insights. Saving Maternity Homes in Ghana Project, Y4 Monitoring and Evaluation Plan 23 7. MEP RESPONSIBILITIES Banyan Global M&E Officer (Tanya Hurst), who will be supported by the SMH Program Manager (Matthew Griffith), has the overall responsibility of providing technical clarity and guidance for M&E data collection and reporting processes. Banyan Global will improve the M&E capacity of the project team and the GRMA through training, mentoring and coaching. Additionally, Banyan Global will report on performance indicators as outlined in the performance management plan. Survey enumerators and the Banyan Global project team will be responsible for data collection, with assistance from GRMA. The scope of data collected will cover all the IR areas, including the program areas to be supported. The SMH project team in Ghana, with support from GRMA and oversight by the project M&E team, will have the responsibility of ensuring regular and continuous data collection, dissemination, and use in a timely manner. It will also be responsible for ensuring data quality, providing data and regular feedback and for complying with agreed-on reporting requirements. Performance Monitoring Plan (Draft August 5, 2015) 24 ANNEX 1: BUSINESS FORMALIZATION SCORE (BFS) OVERVIEW AND PURPOSE One of the primary goals of the Saving Maternity Homes project is to increase the ability of a PMH to sustain its operations as a going concern. The ability of any business to sustain its operations is determined by a number of factors. Several of these factors are external to the business itself, such as the level of community awareness of the benefits of health services and the level of market competition of PMHs, chemical shops, or other actors. Several of these factors are internal, but difficult to measure, such as the managerial capacity of the owner and/or manager, and the quality of customer service of the staff. In an effort to identify a measurable approach to understanding of the ability of a PMH to sustain its operations as a going concern, the SMH project has chosen to focus formalization. The relevant literature indicates that formalization has a direct impact on business sustainability. One paper focused on SMEs in Vietnam presents a variety of findings that emphasize the importance of formalization, including increase in profits and investments and better access to credit: “There is causal evidence that becoming more formalized leads to an increase in profits and investments, and a decrease in the use of casual labor thereby improving contract conditions for workers. Moreover, formal firms have better access to credit.”12 A recent World Bank review of the literature focused on formalization also emphasizes the correlation between informality and lower levels of growth and productivity.13 At the point of the baseline, the project will calculate a score for each PMH that will allow measuring change over time, and therefore quantifying the effect of the project’s business support activities. The project expects this measurement to be particularly useful in respect to PMHs that have undergone full transformation. Additionally, the BFS will assist the project to identify the areas of business and financial management that are the weakest for each PMH, which will inform future project assistance. DEFINING FORMALIZATION In the process of designing the BFS, the project reviewed the factors commonly used in defining informality and formalization. The World Bank review lists registration with a government agency, size of the business (or activity), whether personal capital is separate from business capital, and the keeping of regular accounts as common factors in defining informality. In one prominent paper, the authors argue for the advantages of defining formality and informality along a continuum, listing the size of the business, registration, the existence of financial statements, mobility of workplace, and access to bank credit as important criteria.14 Finally, as part of the development of the project’s approach to understanding both internal and external formalization, the project reviewed The Five Stages of Small Business Growth, by Neil C. Churchill and Virginia L. Lewis. 15 This often-cited paper outlines typical progressive stages of business development, and informed the project’s thinking on how internal formalization increases as businesses move through the 12 The benefits of formalization: evidence from Vietnamese SMEs: John Rand and Nina Torm (draft 2010) 13 Informal Economy and the World Bank, by Nancy Benjamin with Kathleen Beegle, Francesca Recanatini, and Massimiliano Santini, Policy Research Working Paper 6888, May 2014 14 Informality, productivity, and enforcement in West Africa: A firm level analysis (World International Policy Center Working Paper Series Number 100, June 2010), Nancy Benjamin and Ahmadou Aly Mbaye 15 Harvard Business Review, May 1983. Saving Maternity Homes in Ghana Project, Y4 Monitoring and Evaluation Plan 25 stages. The design of the BFS score builds on this research, as well as the SMH team’s own expertise in the areas of enterprise development and business strengthening for private health providers in Ghana. BFS DESIGN The BFS is designed to incorporate factors that set the foundation for survival and growth of a business, including those related to: 1) External formalization of the business such as legal organization of a business, registration with the tax authority, etc. and 2) Internal formalization of business processes and operations such as reliability of business records and information, level of professionalization of business functions, and capacity to understand and manage the business. External formalization relates to the degree of participation of a business in the formal mainstream economy while the internal formalization describes the extent to which a business has developed internal organizational processes and procedures that allow it to function efficiently. There is a relation between external and internal formalization: businesses that start off in an informal way and remain so for some time reach a certain level of operations that necessitates internal formalization to grow the business. Both types are equally important and will be assumed to have equal weights in the construction of the Business Formalization Score. BFS includes clearly verifiable factors that reflect the two aspects of formalization, and will offer scores for each as well as the cumulative score for the business. BFS FACTORS The following factors will be considered for the BFS. Factors Related to External Formalization:  Health Facility Regulatory Agency (HEFRA) certification is issued to the private maternity home after the Agency is satisfied with all inspection reports. Without this certification, a PMH is in breach of the law and its operating deemed illegal.  Nurses and Midwives Council (NMC) certification ensures that the primary midwife’s qualification to practice midwifery is current, as defined by NMC guidelines. Without a current renewal of practice license from NMC, a PMH cannot secure either HEFRA or NHIA certifications.  National Health Insurance Authority (NHIA) certification is necessary if the home is to benefit from the opportunities presented by the NHIS and its revenue streams. The underserved communities, which mostly constitute the target market for the PMHs, overwhelmingly rely on the NHIS to seek health care. With the current competition facing private maternity homes from public health facilities, private hospitals and clinics, PMHs without the NHIA certification will often be less competitive, and vulnerable to financial challenges and closure.  Registration with Ghana Revenue Authority is required for any business in Ghana, and is key if one is to avoid legal penalties deriving from the non-payment of taxes. These penalties include fines, and potentially closure of facility.  Registration status as a business with Registrar General Department is a key step toward formalization, without which PMHs will be unable to progress with other certifications or engaging with professional services, such as the opening of a bank account. Factors Related to Internal Formalization: Saving Maternity Homes in Ghana Project, Y4 Monitoring and Evaluation Plan 26  The practice of maintaining written contracts with partners and suppliers is an indicator of strong business relationships. Written contracts enable a PMH to make accurate financial plans based on agreed prices, availability and quality of needed supplies.  The practice of separating personal finances from business finances is an indicator of good financial governance, and evidence of existence of an arm’s length relationship between business and personal. PMHs that do not separate personal and business finances often demonstrate poor financial management, and a low level of awareness of the use of funds.  The practice of keeping funds in a bank account is a generally efficient financial management practice, and is especially important to facilities that receive out-of-pocket payments from clients. Additionally many PMHs receive direct debit/check payments through NHIA.  Size of the business (as defined by number of employees) is a common indicator of informality (most prominently by the International Labor Organization), with size relating to a business’s ability to incorporate systems, structures and functions appropriate to the stage of growth or maturity of the business, which requires employees. Most definitions of informality note that size itself is not sufficient to define informality, but should be incorporated as one factor among several.16  The availability of financial records indicates the ability of the firm to track and review financial performance. These allow the business to make informed business decisions based on evidence. DATA FOR BFS FACTORS Data for the BFS factors will be obtained through the baseline and endline surveys. For each factor, the PMH will be awarded a score of between 0 and 1, depending on their answer to the relevant question. The relevant survey questions, and the points available for each answer, that will gather the necessary data, are the following: Factors Related to External Formalization (5 points): 1. Is the facility’s certification with the Health Facilities Regulatory Agency (HEFRA) up to date? a. Not certified = score 0 b. Certified but certificate not sighted = score 0.5 c. Certified and certificate sighted but not current = score 0.5 d. Certified and certificate sighted and current = score 1 2. Is the facility’s certification of MIDWIFE with the Nurses and Midwives Council up to date? a. Not certified = score 0 b. Certified but certificate not sighted = score 0.5 c. Certified and certificate sighted but not current = score 0.5 d. Certified and certificate sighted and current = score 1 3. Is the facility currently accredited with NHIA? a. Not accredited = score 0 b. Accreditation outdated or withdrawn, or accreditation certificate sighted but not current = score 0.25 c. Claims to be accredited, but accreditation certificate not sighted = score 0.5 d. Accredited and accreditation certificate sighted and current = score 1 4. Is the facility registered as a business with the Registrar General Department? 16 For a further discussion as size as a criterion in defining informality, see “Informality and the World Bank”, p8. Saving Maternity Homes in Ghana Project, Y4 Monitoring and Evaluation Plan 27 a. Not registered = score 0 b. Registered but registration document not sighted = score 0.5 c. Registered and registration document sighted but not current = score 0.5 d. Registered and registration document sighted and current = score 1 5. Is the facility registered with the Ghana Revenue Authority (GRA)? a. Not registered = score 0 b. Registered but no evidence of tax registration sighted = score 0.5 c. Registered and evidence of tax registration sighted but not current = score 0.5 d. Registered and evidence of tax registration sighted and current = score 1 Factors Related to Internal Formalization (5 points): 6. Does the facility maintain written contracts or agreements with partners and suppliers? a. Does not maintain written contracts = score 0 b. Maintains written contracts, but example not sighted = score 0.5 c. Maintains written contracts, and example sighted = score 1 7. Does the business keep personal funds separate from business funds? a. No = score 0 b. Yes = score 1 8. Does the facility keep funds in a bank account (personal or business)? a. No bank account = score 0 b. Personal bank account = score 0.5 c. Business bank account = score 1 9. Does the facility keep financial statements (cash log, income statement, balance sheet)? a. Facility reports keeping none of the three = score 0 b. Facility reports keeping one of the three = score 0.33 c. Facility reports keeping two of the three = score 0.66 d. Facility reports keeping all three = score 1 10. Size of the business defined by number of full-time employees (clinical and non–clinical)? a. Number range:1-5 = score 0.25 b. Number range:6-10 = score 0.50 c. Number range11-20 = score 0.75 d. Number range 21 or above = score 1 CALCULATION AND INTERPRETATION OF BFS The BFS is a composite indicator, comprised of external and internal factors. Based on their answers to the relevant survey questions, PMHs are assigned a score between 0 – 1 for each of the ten factors described above. Maximum of 10 points is possible, including an external score between 0 – 5 and an internal score between 0 – 5. The higher the score, the greater degree of formalization. Performance Monitoring Plan (Draft August 5, 2015) 28 ANNEX 2: INDICATORS TABLE The following tables summarize the key indicators that will be used to measure progress against SMH Project results, including the data sources and timing/frequency of data collection. For more information on each indicator, please see the indicator reference sheets included in Annex 5. # Performance indicator Indicator definition Data Source Data Acquisition Analysis and Reporting Schedule/ Frequency By Whom Schedule/ Frequency By Whom Long-Term Goal: Increase in private maternity homes’ ability to sustain and grow health services in the underserved areas in Ghana Strategic Objective: PMHs sustained, MCH services expanded, and efficiency and quality improved. 1 Percentage of assisted PMHs with increased financial sustainability The percentage of assisted PMHs that demonstrate an increase in financial sustainability. Numerator: Number of PMHs receiving project assistance that improve their sustainability score on the BFS at the endline as compared to their score at the baseline. Denominator: Number of PMHs receiving project assistance. This indicator will be disaggregated based on sex of the owner and region. PMH Survey Baseline and endline SMH End of project SMH 2 Percentage of assisted PMHs with improved quality of services17 Percentage of assisted PMHs showing improvements in their QAT score. QAT Baseline and endline GRMA / SMH End of project GRMA / SMH 17 Note: In previous versions of the MEP, this indicator was phrased as a number. We have changed it to a percentage for consistency with Indicator 1. Saving Maternity Homes in Ghana Project, Y4 Monitoring and Evaluation Plan 29 # Performance indicator Indicator definition Data Source Data Acquisition Analysis and Reporting Schedule/ Frequency By Whom Schedule/ Frequency By Whom Numerator: Number of PMHs receiving project assistance that improve their QAT score at the endline as compared to their score at the baseline. Denominator: Number of PMHs receiving project assistance. This indicator will be disaggregated based on sex of the owner and region. 3 Number of assisted PMHs with improved technical efficiency Number of assisted PMHs showing improvements in technical efficiency. Efficiency will be determined using Data Envelopment Analysis (DEA). Data envelopment analysis is a non-parametric method in operations research used to empirically measure productive efficiency of decision-making units (DMUs). This indicator will be disaggregated based on sex of the owner and region. PMH Survey Baseline and endline SMH End of project SMH 4 Number of assisted PMHs that add new health services Number of assisted PMHs that add new health services. This indicator will be disaggregated based on sex of QAT Baseline and endline SMH End of project SMH Saving Maternity Homes in Ghana Project, Y4 Monitoring and Evaluation Plan 30 # Performance indicator Indicator definition Data Source Data Acquisition Analysis and Reporting Schedule/ Frequency By Whom Schedule/ Frequency By Whom the owner, type of health service and region. 5 Number of outpatient visits18 received by assisted PMHs that undergo an ownership/ management transformation Average number of outpatient visits per month over a period of 6 months, received by those assisted facilities that undergo an ownership and/or management transformation. This indicator will be disaggregated by sex of the owner, by type of business transfer model, by region, and by type of health service (MNCH, FP, other basic health services). PMH Survey, GHS returns Baseline and endline SMH End of project SMH IR1: PMH owners and managers ready for ownership and/or management change Sub IR 1.1: PMHs interested in and ready for transformation identified 6 Number of assisted PMHs expressing interest in ownership / management transformation Number of assisted PMHs who submit an Expression of Interest Form, which formally indicates an interest in exploring a management or ownership transformation. This indicator will be disaggregated based on sex of the owner, region, and age of the Expression of Interest Forms Quarterly SMH/ GRMA Quarterly SMH 18 Note: In previous versions of this document this indicator was phrased as “number of patient visits.” It is changed here to “number of outpatient visits” in order to increase clarity regarding what the project actually intends to measure. Saving Maternity Homes in Ghana Project, Y4 Monitoring and Evaluation Plan 31 # Performance indicator Indicator definition Data Source Data Acquisition Analysis and Reporting Schedule/ Frequency By Whom Schedule/ Frequency By Whom registered midwife who submits the Form. Sub IR 1.2: Capacity of PMHs for ownership/management transformation strengthened. 7 Number of PMH owners who have received pre￾transformation technical assistance Number of PMH owners who receive at least one visit from the project during which the owner received TA focused on required preparations for investment. All PMHs counted against this indicator will have expressed an interest in ownership/management transformation (See Indicator 7.) This indicator will be disaggregated based on sex of the owner and region. Technical Assistance Record Forms Routinely SMH Routinely SMH 8 Percentage of training participants who attain a score of 75% or higher on training post￾tests This indicator measures the quality and effectiveness of the trainings offered by SMH to PMHs. Numerator: Number of training participants who attain a score of 75% or higher on the training post-test. Project records Routinely SMH Routinely SMH Saving Maternity Homes in Ghana Project, Y4 Monitoring and Evaluation Plan 32 # Performance indicator Indicator definition Data Source Data Acquisition Analysis and Reporting Schedule/ Frequency By Whom Schedule/ Frequency By Whom Denominator: Total number of training participants. This indicator will be disaggregated based on type of training (Business Continuity and Succession Planning (BCSP), Business Strengthening Support (BSS), or Clinical Capacity Building (CCB)), and sex of the training participant. 9 Number of unique PMH facilities trained by the SMH project Number PMH facilities who send at least one representative to at least one of the project’s training courses. This indicator will be disaggregated based on type of training (Business Continuity and Succession Planning (BCSP), Business Strengthening Support (BSS), or Clinical Capacity Building (CCB)), sex of the owner, and region. Trainee registration forms Routinely SMH Routinely SMH 10 Number of participants in SMH training courses Number of participants trained, where participant is defined as an individual participating in a training. One individual would count as two participants if that Trainee registration forms Routinely SMH Routinely SMH Saving Maternity Homes in Ghana Project, Y4 Monitoring and Evaluation Plan 33 # Performance indicator Indicator definition Data Source Data Acquisition Analysis and Reporting Schedule/ Frequency By Whom Schedule/ Frequency By Whom person participated in two separate trainings. This indicator will be disaggregated based on type of training (Business Continuity and Succession Planning (BCSP), Business Strengthening Support (BSS), or Clinical Capacity Building (CCB)), sex of the owner, and geographic location. IR2: PMHs under new ownership / management continue health service delivery Sub-IR 2.1: PMH ownership/management transformations brokered 11 Number of assisted active PMHs who complete an ownership / management transformation Number of active PMH facilities who receive project assistance, that complete the process of an ownership and/or management transformation. This indicator will be disaggregated based on sex of the owner, region, and type of transformation. Project records Annually SMH Annually SMH 1219 Number of inactive PMHs re-opened with Number of inactive PMH facilities that are re-opened and begin delivering health care Project records Annually SMH Annually SMH 19 Note: In previous versions of this document, there was an indicator following this one, focused on “Number of PMHs opened.” After the conclusion of the baseline assessment and the initial BCSP trainings, the project believes that the opening of brand new PMHs (as opposed to the re-opening of currently close PMHs) is beyond the scope of transformations (sales, leasing, new partnerships, new management) that are the focus of this project. We look forward to USAID’s thoughts on this topic. Saving Maternity Homes in Ghana Project, Y4 Monitoring and Evaluation Plan 34 # Performance indicator Indicator definition Data Source Data Acquisition Analysis and Reporting Schedule/ Frequency By Whom Schedule/ Frequency By Whom new owners or managers services after obtaining new ownership/ management. The new owners/managers of these facilities will have received assistance from the SMH project. This indicator will be disaggregated based on sex of the owner and region. 13 Number of investment proposals for PMH transformations prepared Number of transformative investment proposals prepared as a result of project assistance. These proposals could be prepared by existing PMHs seeking new investors / managers, by investors seeking to purchase PMH facilities, or by new / younger midwives seeking to open their own facilities. This indicator will be disaggregated based on sex of the owner and region. Project records Routinely SMH Routinely SMH 14 Number of new / younger midwives interested in ownership A new / younger midwife is defined as an individual who is below the age of 55. This indicator refers to the number of these individuals who complete an “Expression of Interest” form Expression of Interest Forms Quarterly SMH Quarterly SMH Saving Maternity Homes in Ghana Project, Y4 Monitoring and Evaluation Plan 35 # Performance indicator Indicator definition Data Source Data Acquisition Analysis and Reporting Schedule/ Frequency By Whom Schedule/ Frequency By Whom and/or management of PMHs.20 that formally declares their interest in receiving project assistance to take on the ownership/management duties of PMHs in target regions. This indicator will be disaggregated based on sex of the owner and region. 15 Percentage of PMHs that have a neutral or positive attitude towards the idea of ownership/management transformation. This indicator refers to the overall attitudes of PMH owners towards the idea of an ownership/ management transformation. Numerator: Number of current PMH owners indicating a neutral or positive attitude on a Likert scale to a statement about specific types of ownership/ management transformation. Denominator: Number of PMH facilities responding to the PMH survey. This indicator will be disaggregated based on sex of PMH Survey Baseline and endline SMH Baseline and endline SMH 20 In a recent USAID review, the reviewer inquired as to whether this indicator is necessary. The SMH team is of the opinion that this indicator is a useful way to measure the project’s prospecting and business development efforts, as well as the interest in transformation from this younger group of midwives. Therefore, our preference is that we retain this indicator. Saving Maternity Homes in Ghana Project, Y4 Monitoring and Evaluation Plan 36 # Performance indicator Indicator definition Data Source Data Acquisition Analysis and Reporting Schedule/ Frequency By Whom Schedule/ Frequency By Whom the owner, region, and type of ownership/management transformation (selling the facility, leasing the facility, bringing on a partner, or bringing on a manager). Sub-IR 2.2: Funding Provided and Leveraged by the Project 16 Number of assisted PMHs that receive seed funding in the form of SMH small grants The total number of PMHs that receive seed funding from the SMH project in a given year. This indicator will be disaggregated based on sex of the owner and region. Project records Annually SMH Annually SMH 17 Amount of seed funding provided by the SMH project in the form of small grants21 Amount of grant funding in US$ provided to the PMHs by the SMH project for transformation in a given year. This indicator will be disaggregated based on sex of the owner and geographic location. Project records Annually SMH Annually SMH 18 Number of assisted PMHs that receive funds from non-project sources Number of PMHs receiving project assistance that receive funds from non-project sources Project records Annually SMH Annually SMH 21 In a recent review from USAID, the reviewer inquired as to whether this indicator is necessary. The project is still deliberating on this point, and has not yet reached a conclusion. Saving Maternity Homes in Ghana Project, Y4 Monitoring and Evaluation Plan 37 # Performance indicator Indicator definition Data Source Data Acquisition Analysis and Reporting Schedule/ Frequency By Whom Schedule/ Frequency By Whom for transformation in a given year. This indicator will be disaggregated based on sex of the owner and region. 19 Amount of funding that assisted PMHs receive from non-project sources Amount of funding from non￾project sources received by project-assisted PMHs for the transformation of PMHs resulting from the project interventions (total amount, average amount per PMH, leverage ratio between seed funding and additional mobilized funding). This indicator will be disaggregated based on sex of the owner and region. Project records Annually SMH Annually SMH IR3: Business and programmatic capacity of PMHs and GRMA increased Sub-IR 3.1: Capacity of PMHs to sustain, increase and improve service delivery strengthened 20 Percentage of assisted, non-accredited PMHs applying for NHIA accreditation that This indicator provides information on the rate at which project-assisted PMHs are successfully participating in Ghana’s National Health Insurance Scheme. PMH Survey and NHIA Baseline and endline SMH Endline SMH Saving Maternity Homes in Ghana Project, Y4 Monitoring and Evaluation Plan 38 # Performance indicator Indicator definition Data Source Data Acquisition Analysis and Reporting Schedule/ Frequency By Whom Schedule/ Frequency By Whom successfully become accredited22 Numerator: Number of non￾accredited project-assisted PMHs that receive a grade that is sufficient to receive NHIA accreditation. Denominator: Non-accredited project-assisted PMHs that apply to NHIA to become accredited. This indicator will be disaggregated based on sex of the owner and region. 21 Percentage of assisted PMHs that receive or maintain NHIA accreditation after transformation This indicator measures the effect of transformation on NHIA accreditation among PMHs. Numerator: Number of assisted PMHs that complete transformation AND that receive or maintain NHIA accreditation after transformation is complete. NHIA, Project records Baseline and endline NHIA and SMH Baseline and endline SMH 22 Note: In previous versions of this document, this indicator was structured to include both unaccredited facilities that achieve accreditation, and accredited facilities that improve their score. It is here changed to focus only on unaccredited facilities achieving accreditation for the sake of simplicity, and feasibility of tracking and analysis. Additionally, the frequency of collecting and reporting on this indicator was changed from annually to baseline/endline, in recognition of the changing NHIA accreditation schedule, as well as the relatively short project duration. Saving Maternity Homes in Ghana Project, Y4 Monitoring and Evaluation Plan 39 # Performance indicator Indicator definition Data Source Data Acquisition Analysis and Reporting Schedule/ Frequency By Whom Schedule/ Frequency By Whom Denominator: Number of assisted PMHs that complete transformation. This indicator will be disaggregated based on sex of the owner and region. Sub-IR 3.2: Increased capacity of PMHs to manage their businesses efficiently 22 Number of one-on-one business training events conducted23 Number of one-on-one training and one-on-one capacity building events provided by the project to participating PMHs. Project records Quarterly SMH Quarterly SMH 2324 Percentage of assisted PMHs maintaining core financial documents (cash log, profit and loss statement, or income statement) This indicator refers to the practice of keeping up to date fundamental financial documents. Numerator: Project-assisted PMHs that report having one of these three documents in place, and up to date. Denominator: All project￾assisted PMHs. PMH Survey Baseline and endline SMH Baseline and endline SMH 23 Note: The words “one-on-one” were added to the phrasing of this indicator, in order to make it more clear that the indicator focuses on one-on-one training events, rather than the group training events (such as BCSP and CCB). 24 Note: In previous versions of this document there was an indicator, immediately following this indicator, that has been removed. That indicator was: Percentage of participating PMHs that have registered their businesses with the Ghana Registrar General Department. In conducting the baseline, we found that 93% of PMHs in the target regions were already registered. Thus, the project came to the conclusion that focusing on an indicator that was already 93% achieved was not the best use of project resources and effort. If USAID would prefer the indicator be included, we are happy to do that. Saving Maternity Homes in Ghana Project, Y4 Monitoring and Evaluation Plan 40 # Performance indicator Indicator definition Data Source Data Acquisition Analysis and Reporting Schedule/ Frequency By Whom Schedule/ Frequency By Whom This indicator will be disaggregated based on sex of the owner and region. Sub-IR 3.3: Management and programmatic capacity of GRMA increased. 24 Number of GRMA staff and volunteers who receive capacity building assistance Number of individuals who are working for GRMA, either as paid staff or as volunteers, who receive capacity building assistance. Project records Annually SMH Annually SMH Saving Maternity Homes in Ghana Project, Y4 Monitoring and Evaluation Plan 41 ANNEX 3: INDICATOR REFERENCE SHEETS USAID Performance Indicator Reference Sheet Name of Result Measured (Goal, DO, IR, sub-IR, Project Purpose, Project Output, etc.): Goal: Increase private maternity homes’ ability to sustain and grow health services in the underserved areas of Ghana. Name of Indicator: 1. Percentage of assisted PMHs with increased financial sustainability Is this a Performance Plan and Report indicator? No _X__ Yes ____, for Reporting Year(s) _________ If yes, link to foreign assistance framework: DESCRIPTION Precise Definition(s): Private maternity homes (PMHs); Business Formalization Score (BFS) = An index defined by the project that is comprised of specific indicators that relate to business formalization, which the project is using as a proxy for business and financial sustainability (See previous PIRS). Unit of Measure: The percentage of assisted PMHs that demonstrate an increase in business and financial sustainability, as measured by the BFS (See previous PIRS). Numerator: Number of PMHs receiving project assistance that improve their sustainability score on the BFS on the endline as compared to their score at the baseline. Denominator: Number of PMHs that receive project assistance. Disaggregated by: This indicator will be disaggregated based on sex of the owner and geographic location of the facility. Rationale or Justification for indicator (optional): PLAN FOR DATA COLLECTION Data Source: PMH Survey Method of data collection and construction: The project will interview PMH owners and managers using the PMH survey questions. The questions have been built into the formhub.org mobile data collection platform. Reporting Frequency: Baseline (beginning of project) and endline (end of project) Individual(s) responsible: Bernard Akotey, SMH Project Team Lead; Matthew Griffith, Banyan Global Program Manager DATA QUALITY ISSUES Dates of Previous Data Quality Assessments and name of reviewer: N/A Date of Future Data Quality Assessments (optional): Known Data Limitations: There is a risk that respondents will not have the documentation to back up their reported answers regarding the different factors that go into the BFS. TARGETS AND BASELINE Baseline timeframe (optional): Baseline findings submitted in June, 2015 Saving Maternity Homes in Ghana Project, Y4 Monitoring and Evaluation Plan 42 Rationale for Targets (optional): LOP target: 20%. Rationale: Recognizing the mixed record that business training interventions have had in improving business practices, the project still believes it can have an impact. At baseline, facilities displayed significant room for improvement in the area of internal formalization, specifically in terms of keeping written contracts (only 8% of PMHs were found to be doing this at baseline), and keeping financial documents (at baseline, 58% kept an income statement; 47% kept a cash log; 39% kept a balance sheet). Considering both the difficulty in effecting change in these areas, and the substantial room for improvement, the project expects that 20% of assisted facilities will improve over the life of project. CHANGES TO INDICATOR Changes to indicator: July, 2015: Added an LOP target and rationale. Other Notes (optional): THIS SHEET LAST UPDATED ON: July 31, 2015 Saving Maternity Homes in Ghana Project, Y4 Monitoring and Evaluation Plan 43 USAID Performance Indicator Reference Sheet Name of Result Measured (Goal, DO, IR, sub-IR, Project Purpose, Project Output, etc.): Goal: Increase private maternity homes’ ability to sustain and grow health services in the underserved areas of Ghana. Name of Indicator: 2. Percentage of assisted PMHs with improved quality of services. Is this a Performance Plan and Report indicator? No _X__ Yes ____, for Reporting Year(s) _________ If yes, link to foreign assistance framework: DESCRIPTION Precise Definition(s): Private maternity homes (PMHs); Quality Assistance Tool (QAT) = A tool developed jointly between the SMH project and GRMA that captures the level of quality at which a specific PMH is delivering the various healthcare services. The QAT focuses on four areas: Emergency response; Quality assurance; Safe motherhood; Advocacy/Health Promotion Numerator: Number of PMHs receiving project assistance that improve their QAT score at the endline as compared to their score at baseline. Denominator: Number of PMHs receiving project assistance. Unit of Measure: Number of assisted PMHs showing improvements in their QAT score. Disaggregated by: This indicator will be disaggregated based on sex of the owner and geographic location of the facility. Rationale or Justification for indicator (optional): PLAN FOR DATA COLLECTION Data Source: Quality Assessment Tool Method of data collection and construction: GRMA will interview PMH owners and managers using the QAT. GRMA will then record that data, and the SMH team will also take record of the outcomes of the QAT for storage in the project’s M&E database. Reporting Frequency: Baseline (beginning of project) and endline (end of project) Individual(s) responsible: GRMA; Bernard Akotey, SMH Project Team Lead; Matthew Griffith, Banyan Global Program Manager DATA QUALITY ISSUES Dates of Previous Data Quality Assessments and name of reviewer: N/A Date of Future Data Quality Assessments (optional): Known Data Limitations: TARGETS AND BASELINE Baseline timeframe (optional): Baseline findings submitted in June, 2015 Rationale for Targets (optional): LOP target: 33%: There are two types of assistance available through the SMH project designed to increase quality of services: the peer support visits, and the Clinical Capacity Building (CCB) training. The total number of PMHs that will receive the training is 146 (with the substantial majority of these also receiving the PSV). While the project regards these as effective interventions, they are occurring at the start of the project. Thus, while the project believes some facilities will apply and maintain improved practices over the life of the project, others will revert back to their old quality practices. Thus, the project proposes a target of 33% of assisted facilities. CHANGES TO INDICATOR Changes to indicator: July, 2015: Added an LOP target and rationale. Other Notes (optional): THIS SHEET LAST UPDATED ON: July 31, 2015 Saving Maternity Homes in Ghana Project, Y4 Monitoring and Evaluation Plan 44 USAID Performance Indicator Reference Sheet Name of Result Measured (Goal, DO, IR, sub-IR, Project Purpose, Project Output, etc.): Goal: Increase private maternity homes’ ability to sustain and grow health services in the underserved areas of Ghana. Name of Indicator: 3. Number of assisted PMHs with improved technical efficiency. Is this a Performance Plan and Report indicator? No _X__ Yes ____, for Reporting Year(s) _________ If yes, link to foreign assistance framework: DESCRIPTION Precise Definition(s): Private maternity homes (PMHs); Efficiency will be determined using Data Envelopment Analysis (DEA). Data envelopment analysis is a non-parametric method in operations research used to empirically measure productive efficiency of decision-making units (DMUs). The methodology is also used for benchmarking in operations management, where a set of measures is selected to benchmark the performance of manufacturing and service operations. Unit of Measure: Number of assisted PMHs showing improvements in technical efficiency. Disaggregated by: This indicator will be disaggregated based on sex of the owner and geographic location of the facility. Rationale or Justification for indicator (optional): PLAN FOR DATA COLLECTION Data Source: PMH Survey Method of data collection and construction: The project will interview PMH owners and managers using the PMH survey questions. The questions have been built into the formhub.org mobile data collection platform. Reporting Frequency: Baseline (beginning of project) and endline (end of project) Individual(s) responsible: GRMA; Bernard Akotey, SMH Project Team Lead; Matthew Griffith, Banyan Global Program Manager DATA QUALITY ISSUES Dates of Previous Data Quality Assessments and name of reviewer: N/A Date of Future Data Quality Assessments (optional): Known Data Limitations: TARGETS AND BASELINE Baseline timeframe (optional): Baseline findings submitted in June, 2015 Rationale for Targets (optional): LOP target: 25. Rationale: At baseline 27 facilities were technically efficient, meaning they scored a 100% in terms of technical efficiency, and therefore cannot score higher. While the project does believe that the project’s interventions in terms of business strengthening, quality of services improvements, and transformations will have an impact on efficiency, it is difficult to predict how much of an impact. To be conservative, the project proposes a target of 25 facilities improving their technical efficiency from baseline to endline. CHANGES TO INDICATOR Changes to indicator: July, 2015: Added an LOP target and rationale. Other Notes (optional): THIS SHEET LAST UPDATED ON: July 31, 2015 Saving Maternity Homes in Ghana Project, Y4 Monitoring and Evaluation Plan 45 USAID Performance Indicator Reference Sheet Name of Result Measured (Goal, DO, IR, sub-IR, Project Purpose, Project Output, etc.): Goal: Increase private maternity homes’ ability to sustain and grow health services in the underserved areas of Ghana. Name of Indicator: 4. Number of assisted PMHs that add new health services Is this a Performance Plan and Report indicator? No _X__ Yes ____, for Reporting Year(s) _________ If yes, link to foreign assistance framework: DESCRIPTION Precise Definition(s): Private maternity homes (PMHs); Health services include services that fall within a PMH’s legal and ethical mandate to provide. The list of health services under consideration in the project’s data gathering can be found in the QAT. Unit of Measure: Number of assisted PMHs that offer add new health services Disaggregated by: This indicator will be disaggregated based on sex of the owner and geographic location of the facility. Rationale or Justification for indicator (optional): PLAN FOR DATA COLLECTION Data Source: QAT Method of data collection and construction: GRMA will interview PMH owners and managers using the QAT. GRMA will then record that data, and the SMH team will also take record of the outcomes of the QAT for storage in the project’s M&E database. Reporting Frequency: Baseline (beginning of project) and endline (end of project) Individual(s) responsible: GRMA; Bernard Akotey, SMH Project Team Lead; Matthew Griffith, Banyan Global Program Manager DATA QUALITY ISSUES Dates of Previous Data Quality Assessments and name of reviewer: N/A Date of Future Data Quality Assessments (optional): Known Data Limitations: TARGETS AND BASELINE Baseline timeframe (optional): Baseline findings submitted in June, 2015 Rationale for Targets (optional): LOP target: 25. Rationale: At the Clinical Capacity Building (CCB) trainings, participants expressed interest in adding services that will increase patient flow and yield a least a modest financial benefit. Additionally, NHIA capitation may require facilities to add on new services in order for NHIS clients to select their facilities as preferred primary provider. Much of the achievement against this indicator is expected to come from the 20 PMHs to be transformed, which are expected to add new services given their desired to be profitable at the earliest opportunity. However, since the project doesn’t expect these factors to affect all PMHs in the target regions, the project proposes a conservative LOP target of just over 17% of assisted facilities, or 25. CHANGES TO INDICATOR Changes to indicator: July, 2015: Added an LOP target and rationale. Other Notes (optional): THIS SHEET LAST UPDATED ON: July 31, 2015 Saving Maternity Homes in Ghana Project, Y4 Monitoring and Evaluation Plan 46 USAID Performance Indicator Reference Sheet Name of Result Measured (Goal, DO, IR, sub-IR, Project Purpose, Project Output, etc.): Goal: Increase private maternity homes’ ability to sustain and grow health services in the underserved areas of Ghana. Name of Indicator: 5. Number of outpatient visits received by assisted PMHs that undergo an ownership/ management transformation Is this a Performance Plan and Report indicator? No _X__ Yes ____, for Reporting Year(s) _________ If yes, link to foreign assistance framework: DESCRIPTION Precise Definition(s): Private maternity homes (PMHs); Outpatient visits refers to discrete visits by an individual seeking outpatient care at the facility. One individual can make multiple visits within the period under review. Transformation is defined as occurring when; 1) the PMH facility is legally sold to a new owner or investor; 2) the existing owner brings on a partner who has both an ownership stake and an active management role; 3) the existing owner retains ownership, but appoints a manager with a job description that gives this person final authority over all operational aspects of facility management; or 4) the existing owner retains ownership and leases the facility to a new manager/operator. Unit of Measure: Average number of outpatient visits per month over a period of 6 months, received by those assisted facilities that undergo an ownership and/or management transformation. Disaggregated by: This indicator will be disaggregated by sex of the owner, by type of business transfer model, by geographic location of the facility, and by type of health service (MNCH, FP, other basic health services). Rationale or Justification for indicator (optional): PLAN FOR DATA COLLECTION Data Source: PMH Survey, GHS returns Method of data collection and construction: The project will interview PMH owners and managers using the PMH survey questions. The questions have been built into the formhub.org mobile data collection platform. The project will also use GHS returns to triangulate data from the PMH survey. Reporting Frequency: Baseline (beginning of project) and endline (end of project) Individual(s) responsible: GRMA; Bernard Akotey, SMH Project Team Lead; Matthew Griffith, Banyan Global Program Manager DATA QUALITY ISSUES Dates of Previous Data Quality Assessments and name of reviewer: N/A Date of Future Data Quality Assessments (optional): Known Data Limitations: TARGETS AND BASELINE Baseline timeframe (optional): Baseline findings submitted in June, 2015 Rationale for Targets (optional): LOP target: 550. Rationale: At baseline, the average number of monthly outpatient visits for all facilities was 505. In setting this target, the project assumes incremental growth of about 10% above the baseline average of 505. The project believes that facilities that undergo transformation will add new services, as well improve service quality. Thus, one will expect the PMHs to be attractive to patients who hitherto will not have accessed the facility, and their average monthly outpatient client visits will go up. CHANGES TO INDICATOR Changes to indicator: July, 2015: Added an LOP target and rationale. Other Notes (optional): THIS SHEET LAST UPDATED ON: July 31, 2015 Saving Maternity Homes in Ghana Project, Y4 Monitoring and Evaluation Plan 47 USAID Performance Indicator Reference Sheet Name of Result Measured (Goal, DO, IR, sub-IR, Project Purpose, Project Output, etc.): Sub IR 1.1: PMHs interested in and ready for transformation identified Name of Indicator: 6. Number of assisted PMHs expressing interest in ownership / management transformation Is this a Performance Plan and Report indicator? No _X__ Yes ____, for Reporting Year(s) _________ If yes, link to foreign assistance framework: DESCRIPTION Precise Definition(s): Private maternity homes (PMHs); Transformation is defined as occurring when; 1) the PMH facility is legally sold to a new owner or investor; 2) the existing owner brings on a partner who has both an ownership stake and an active management role; 3) the existing owner retains ownership, but appoints a manager with a job description that gives this person final authority over all operational aspects of facility management; or 4) the existing owner retains ownership and leases the facility to a new manager/operator. Unit of Measure: Number of assisted PMHs who submit an Expression of Interest Form, which formally indicates an interest in exploring a management or ownership transformation. Disaggregated by: This indicator will be disaggregated based on sex of the owner, geographic location, and age of the registered midwife who submits the Form. Rationale or Justification for indicator (optional): PLAN FOR DATA COLLECTION Data Source: Expression of Interest Forms Method of data collection and construction: The project and GRMA will collect Expression of Interest Forms as they are submitted, and feed summary data from these forms into the project’s central database. Reporting Frequency: Quarterly Individual(s) responsible: GRMA; Bernard Akotey, SMH Project Team Lead; Matthew Griffith, Banyan Global Program Manager DATA QUALITY ISSUES Dates of Previous Data Quality Assessments and name of reviewer: N/A Date of Future Data Quality Assessments (optional): Known Data Limitations: TARGETS AND BASELINE Baseline timeframe (optional): Baseline findings submitted in June, 2015 Rationale for Targets (optional): LOP target: 46. Rationale: The baseline assessment found varying levels of interest for the four different types of transformations. Seventy-six percent of respondents indicated interest in bringing on a manager, while 69% of respondents indicated interest in bringing on a partner. Forty percent of respondents were interested in the possibility of leasing out the facility, while only 15% were interested in the idea of one day selling the facility. These findings indicate that, while only a few facilities might be interested in sales or leasing, there is greater interest in management contracts and partnerships. However, expressing interest and expending the effort to initiate the process (submitting the EOI) are different steps. Thus, we set the target for this indicator at just under one-third of assisted facilities, at 46. CHANGES TO INDICATOR Changes to indicator: July, 2015: Added an LOP target and rationale. Other Notes (optional): THIS SHEET LAST UPDATED ON: July 31, 2015 Saving Maternity Homes in Ghana Project, Y4 Monitoring and Evaluation Plan 48 USAID Performance Indicator Reference Sheet Name of Result Measured (Goal, DO, IR, sub-IR, Project Purpose, Project Output, etc.): Sub IR 1.2: Capacity of PMHs for ownership/management transformation strengthened. Name of Indicator: 7. Number of PMH owners who receive pre-transformation technical assistance Is this a Performance Plan and Report indicator? No _X__ Yes ____, for Reporting Year(s) _________ If yes, link to foreign assistance framework: DESCRIPTION Precise Definition(s): Private Maternity Home (PMH); Transformation is defined as occurring when; 1) the PMH facility is legally sold to a new owner or investor; 2) the existing owner brings on a partner who has both an ownership stake and an active management role; 3) the existing owner retains ownership, but appoints a manager with a job description that gives this person final authority over all operational aspects of facility management; or 4) the existing owner retains ownership and leases the facility to a new manager/operator. Technical assistance as used includes individual technical assistance focused on deal structuring, needs analysis, business plan support, and other pre-transformation assistance. Unit of Measure: Number of PMH owners who receive at least one visit from the project during which the owner received TA focused on required preparations for investment. All PMHs counted against this indicator will have expressed an interest in ownership/management transformation. Disaggregated by: This indicator will be disaggregated based on sex of the owner and geographic location of the facility. Rationale or Justification for indicator (optional): PLAN FOR DATA COLLECTION Data Source: Technical Assistance Record Forms Method of data collection and construction: The project will collect Technical Assistance Record Forms as the technical assistance moves forward, and feed summary data from these Forms into the project’s central database. Reporting Frequency: Routinely Individual(s) responsible: Bernard Akotey, SMH Project Team Lead; Matthew Griffith, Banyan Global Program Manager DATA QUALITY ISSUES Dates of Previous Data Quality Assessments and name of reviewer: N/A Date of Future Data Quality Assessments (optional): Known Data Limitations: TARGETS AND BASELINE Baseline timeframe (optional): Baseline findings submitted in June, 2015 Rationale for Targets (optional): LOP target: 33. Rationale: This indicator is a restatement of the number of facilities that ascend to tier 3 of project assistance. Thus, we set the target for this indicator equal to the target for tier 3 support, at 33 facilities. CHANGES TO INDICATOR Changes to indicator: July, 2015: Added an LOP target and rationale. Other Notes (optional): THIS SHEET LAST UPDATED ON: July 31, 2015 Saving Maternity Homes in Ghana Project, Y4 Monitoring and Evaluation Plan 49 USAID Performance Indicator Reference Sheet Name of Result Measured (Goal, DO, IR, sub-IR, Project Purpose, Project Output, etc.): Sub IR 1.2: Capacity of PMHs for ownership/management transformation strengthened. Name of Indicator: 8. Percentage of training participants who attain a score of 75% or higher on training post-tests Is this a Performance Plan and Report indicator? No _X__ Yes ____, for Reporting Year(s) _________ If yes, link to foreign assistance framework: DESCRIPTION Precise Definition(s): “Participant” is defined as an individual participating in a training. One individual would count as two participants if that person participated in two separate trainings. Training post-tests are evaluations administered to participants upon completion of the training to gauge their level of knowledge acquisition. Unit of Measure: Numerator: Number of training participants who attain a score of 75% or higher on the training post-test. Denominator: Total number of training participants. Disaggregated by: This indicator will be disaggregated based on type of training (Business Continuity and Succession Planning (BCSP), Business Strengthening Support (BSS), or Clinical Capacity Building (CCB)), sex of the owner, and geographic location. Rationale or Justification for indicator (optional): This indicator measures the quality and effectiveness of the trainings offered by SMH to PMHs. PLAN FOR DATA COLLECTION Data Source: Project records Method of data collection and construction: The project will record post-test scores for each training administered by the project. Reporting Frequency: Routinely Individual(s) responsible: Bernard Akotey, SMH Project Team Lead; Matthew Griffith, Banyan Global Program Manager DATA QUALITY ISSUES Dates of Previous Data Quality Assessments and name of reviewer: N/A Date of Future Data Quality Assessments (optional): Known Data Limitations: TARGETS AND BASELINE Baseline timeframe (optional): Baseline findings submitted in June, 2015 Rationale for Targets (optional): LOP target for BCSP training: 81%. Rationale: This activity is concluded, and so this target is set equal to the achievement. LOP target for BSS training: 80%. Rationale: The project anticipates a similar level of achievement to that of the BCSP training. LOP target for CCB training: 27%. Rationale: This activity is concluded, and so the target is set equal to the achievement. CHANGES TO INDICATOR Changes to indicator: July, 2015: Added an LOP target and rationale. Other Notes (optional): THIS SHEET LAST UPDATED ON: July 31, 2015 Saving Maternity Homes in Ghana Project, Y4 Monitoring and Evaluation Plan 50 USAID Performance Indicator Reference Sheet Name of Result Measured (Goal, DO, IR, sub-IR, Project Purpose, Project Output, etc.): Sub IR 1.2: Capacity of PMHs for ownership/management transformation strengthened. Name of Indicator: 9. Number of unique PMH facilities trained by the SMH project Is this a Performance Plan and Report indicator? No _X__ Yes ____, for Reporting Year(s) _________ If yes, link to foreign assistance framework: DESCRIPTION Precise Definition(s): Private maternity homes (PMHs); “Trained” here refers to receiving training from the SMH project. Unit of Measure: Number PMH facilities who send at least one representative to at least one of the project’s training courses. Disaggregated by: This indicator will be disaggregated based on type of training (Business Continuity and Succession Planning (BCSP), Business Strengthening Support (BSS), or Clinical Capacity Building (CCB)), sex of the owner, and geographic location. Rationale or Justification for indicator (optional): PLAN FOR DATA COLLECTION Data Source: Trainee Registration Forms Method of data collection and construction: The project will collect Trainee Registration Forms as the trainings move forward, and feed summary data from these forms into the project’s central database. Reporting Frequency: Routinely Individual(s) responsible: Bernard Akotey, SMH Project Team Lead; Matthew Griffith, Banyan Global Program Manager DATA QUALITY ISSUES Dates of Previous Data Quality Assessments and name of reviewer: N/A Date of Future Data Quality Assessments (optional): Known Data Limitations: TARGETS AND BASELINE Baseline timeframe (optional): Baseline findings submitted in June, 2015 Rationale for Targets (optional): LOP target: 146. Rationale: The project has set a target of 146 assisted facilities, determined by the number of facilities that receive tier 2 support. This tier represents the broadest level of assistance, designed to affect the largest number of facilities. Additional support (tier 3 and tier 4) is anticipated to involve facilities that already participated in tier 2. Thus, the project sets the LOP target for this indicator equal to the target for tier 2 support, at 146. CHANGES TO INDICATOR Changes to indicator: July, 2015: Added an LOP target and rationale. Other Notes (optional): THIS SHEET LAST UPDATED ON: July 31, 2015 Saving Maternity Homes in Ghana Project, Y4 Monitoring and Evaluation Plan 51 USAID Performance Indicator Reference Sheet Name of Result Measured (Goal, DO, IR, sub-IR, Project Purpose, Project Output, etc.): Sub IR 1.2: Capacity of PMHs for ownership/management transformation strengthened. Name of Indicator: 10. Number of participants in SMH training courses Is this a Performance Plan and Report indicator? No _X__ Yes ____, for Reporting Year(s) _________ If yes, link to foreign assistance framework: DESCRIPTION Precise Definition(s): Private maternity homes (PMHs); “Trained” here refers to receiving training from the SMH project. “Participant” is defined as an individual participating in a training. One individual would count as two participants if that person participated in two separate trainings. Unit of Measure: Number of participants trained. Disaggregated by: This indicator will be disaggregated based on type of training (Business Continuity and Succession Planning (BCSP), Business Strengthening Support (BSS), or Clinical Capacity Building (CCB)), sex of the owner, and geographic location. Rationale or Justification for indicator (optional): PLAN FOR DATA COLLECTION Data Source: Trainee Registration Forms Method of data collection and construction: The project will collect Trainee Registration Forms as the trainings move forward, and feed summary data from these forms into the project’s central database. Reporting Frequency: Routinely Individual(s) responsible: Bernard Akotey, SMH Project Team Lead; Matthew Griffith, Banyan Global Program Manager DATA QUALITY ISSUES Dates of Previous Data Quality Assessments and name of reviewer: N/A Date of Future Data Quality Assessments (optional): Known Data Limitations: TARGETS AND BASELINE Baseline timeframe (optional): Baseline findings submitted in June, 2015 Rationale for Targets (optional): LOP target: 409. Rationale: For the initial BCSP and CCB trainings, the project had 153 participants (representing 146 facilities) attend the trainings, leading to a total of 306 participants in these trainings. Given that the project has one more training, at which it expects similar levels of participation, the project arrives at a final figure of 409 total participants in project trainings. CHANGES TO INDICATOR Changes to indicator: July, 2015: Added an LOP target and rationale. Other Notes (optional): THIS SHEET LAST UPDATED ON: July 31, 2015 Saving Maternity Homes in Ghana Project, Y4 Monitoring and Evaluation Plan 52 USAID Performance Indicator Reference Sheet Name of Result Measured (Goal, DO, IR, sub-IR, Project Purpose, Project Output, etc.): Sub-IR 2.1: PMH ownership/management transformations brokered Name of Indicator: 11. Number of assisted active PMHs who complete an ownership / management transformation Is this a Performance Plan and Report indicator? No _X__ Yes ____, for Reporting Year(s) _________ If yes, link to foreign assistance framework: DESCRIPTION Precise Definition(s): Private Maternity Home (PMH); Transformation is defined as occurring when; 1) the PMH facility is legally sold to a new owner or investor; 2) the existing owner brings on a partner who has both an ownership stake and an active management role; 3) the existing owner retains ownership, but appoints a manager with a job description that gives this person final authority over all operational aspects of facility management; or 4) the existing owner retains ownership and leases the facility to a new manager/operator. Technical assistance as used includes individual technical assistance focused on deal structuring, needs analysis, business plan support, and other pre-transformation assistance. Unit of Measure: Number of active PMH facilities who receive project assistance, that complete the process of an ownership and/or management transformation. Disaggregated by: This indicator will be disaggregated based on sex of the owner, geographic location of the facility, and type of transformation. Rationale or Justification for indicator (optional): PLAN FOR DATA COLLECTION Data Source: Project records Method of data collection and construction: The project will track the required data to report against this indicator. Reporting Frequency: Annually Individual(s) responsible: Bernard Akotey, SMH Project Team Lead; Matthew Griffith, Banyan Global Program Manager DATA QUALITY ISSUES Dates of Previous Data Quality Assessments and name of reviewer: N/A Date of Future Data Quality Assessments (optional): Known Data Limitations: TARGETS AND BASELINE Baseline timeframe (optional): Baseline findings submitted in June, 2015 Rationale for Targets (optional): LOP target: 20. Rationale: This indicator is a restatement of the number of facilities that ascend to tier 4 of project assistance. Thus, we set the target for this indicator equal to the target for tier 4 support, at 20 facilities Though the EOI funnel may be wider, the screen criteria to be adopted may preclude SMH from moving forward with these initial EOIs. CHANGES TO INDICATOR Changes to indicator: July, 2015: Added an LOP target and rationale. Other Notes (optional): THIS SHEET LAST UPDATED ON: July 31, 2015 Saving Maternity Homes in Ghana Project, Y4 Monitoring and Evaluation Plan 53 USAID Performance Indicator Reference Sheet Name of Result Measured (Goal, DO, IR, sub-IR, Project Purpose, Project Output, etc.): Sub-IR 2.1: PMH ownership/management transformations brokered Name of Indicator: 12. Number of inactive PMHs re-opened with new owners or managers Is this a Performance Plan and Report indicator? No _X__ Yes ____, for Reporting Year(s) _________ If yes, link to foreign assistance framework: DESCRIPTION Precise Definition(s): An inactive PMH is defined as a facility that is not currently delivering health services. Unit of Measure: Number of inactive PMH facilities that are re-opened and begin delivering health care services after obtaining new ownership/ management. The new owners/managers of these facilities will have received assistance from the SMH project. Disaggregated by: This indicator will be disaggregated based on sex of the owner and geographic location of the facility. Rationale or Justification for indicator (optional): PLAN FOR DATA COLLECTION Data Source: Project records Method of data collection and construction: The project will track the required data to report against this indicator. Reporting Frequency: Annually Individual(s) responsible: Bernard Akotey, SMH Project Team Lead; Matthew Griffith, Banyan Global Program Manager DATA QUALITY ISSUES Dates of Previous Data Quality Assessments and name of reviewer: N/A Date of Future Data Quality Assessments (optional): Known Data Limitations: TARGETS AND BASELINE Baseline timeframe (optional): Baseline findings submitted in June, 2015 Rationale for Targets (optional): LOP target: 3. Rationale: The baseline discovered 30 closed facilities (18% of the sample). The survey team was not able to get a reason for the closure of nine facilities. Six facilities were newly created and awaiting legal registration, while fifteen facilities were either temporarily or permanently closed due to the death, disability, or absence of the owner or primary midwife. Even though there are probably more closed facilities than the ones we discovered, these numbers suggest the total number of closed facilities that could be re-opened is low. From discussions during the PMHs during the baseline and initial BCSP trainings, as well as input from GRMA and other key informants, the project believes that the state of most of these inactive facilities will require more resources to reopen than the project originally anticipated. Almost all of these defunct facilities serve as homes for the owners and their families, hence the possibility of re-opening may require either the relocation of either the facility or the family. Thus, the project arrives at a modest LOP target for this indicator of three facilities. CHANGES TO INDICATOR Changes to indicator: July, 2015: Added an LOP target and rationale. Other Notes (optional): THIS SHEET LAST UPDATED ON: July 31, 2015 Saving Maternity Homes in Ghana Project, Y4 Monitoring and Evaluation Plan 54 USAID Performance Indicator Reference Sheet Name of Result Measured (Goal, DO, IR, sub-IR, Project Purpose, Project Output, etc.): Sub-IR 2.1: PMH ownership/management transformations brokered Name of Indicator: 13. Number of investment proposals for PMH transformations prepared Is this a Performance Plan and Report indicator? No _X__ Yes ____, for Reporting Year(s) _________ If yes, link to foreign assistance framework: DESCRIPTION Precise Definition(s): Investment proposals are effectively plans to effect transformation, prepared through assistance from the project. These proposals could be prepared by existing PMHs seeking new investors / managers, by investors seeking to purchase PMH facilities, or by new / younger midwives seeking to open their own facilities. Unit of Measure: Number of transformative investment proposals prepared as a result of project assistance Disaggregated by: This indicator will be disaggregated based on sex of the owner and geographic location of the facility. Rationale or Justification for indicator (optional): PLAN FOR DATA COLLECTION Data Source: Project records Method of data collection and construction: The project will take note when any investment proposals on which it is assisting are completed. Reporting Frequency: Routinely Individual(s) responsible: Bernard Akotey, SMH Project Team Lead; Matthew Griffith, Banyan Global Program Manager DATA QUALITY ISSUES Dates of Previous Data Quality Assessments and name of reviewer: N/A Date of Future Data Quality Assessments (optional): Known Data Limitations: TARGETS AND BASELINE Baseline timeframe (optional): Baseline findings submitted in June, 2015 Rationale for Targets (optional): LOP target: 20. Rationale: By setting an LOP target of 20 facilities, equal to the total number of transformations we are targeting, we recognize that each transformation is likely to have a legitimate need for seed funding in order to undertake needed initial investments that will boost the likelihood of success of the new arrangement. CHANGES TO INDICATOR Changes to indicator: July, 2015: Added an LOP target and rationale. Other Notes (optional): THIS SHEET LAST UPDATED ON: July 31, 2015 Saving Maternity Homes in Ghana Project, Y4 Monitoring and Evaluation Plan 55 USAID Performance Indicator Reference Sheet Name of Result Measured (Goal, DO, IR, sub-IR, Project Purpose, Project Output, etc.): Sub-IR 2.1: PMH ownership/management transformations brokered Name of Indicator: 14. Number of new / younger midwives interested in PMHs Is this a Performance Plan and Report indicator? No _X__ Yes ____, for Reporting Year(s) _________ If yes, link to foreign assistance framework: DESCRIPTION Precise Definition(s): A new / younger midwife is defined as an individual midwife who is below the age of 55. These individuals will express interest through the completion and submission of “Expression of Interest” forms that formally declare their interest in receiving project assistance to take on the ownership/management duties of PMHs in target regions. Unit of Measure: Number of individual midwives below the age of 55 who complete an “Expression of Interest” form that formally declares their interest in receiving project assistance to take on the ownership/management duties of PMHs in target regions. Disaggregated by: This indicator will be disaggregated based on sex of the owner, and geographic location of the facility. Rationale or Justification for indicator (optional): This indicator refers to the overall attitudes of PMH owners towards the idea of potentially selling their facilities, which is one of the project’s proposed forms of ownership transformation. PLAN FOR DATA COLLECTION Data Source: PMH Survey Method of data collection and construction: The project and GRMA will collect Expression of Interest Forms as they are submitted, and feed summary data from these forms into the project’s central database. Reporting Frequency: Quarterly Individual(s) responsible: Bernard Akotey, SMH Project Team Lead; Matthew Griffith, Banyan Global Program Manager DATA QUALITY ISSUES Dates of Previous Data Quality Assessments and name of reviewer: N/A Date of Future Data Quality Assessments (optional): Known Data Limitations: TARGETS AND BASELINE Baseline timeframe (optional): Baseline findings submitted in June, 2015 Rationale for Targets (optional): LOP target: 33. Rationale: The project has set an LOP target of 46 for total EOIs. Of these, we believe the bulk will be coming from midwives below the age of 55, due to the way the project will market these opportunities, and based on initial feedbacks from our the project’s encounter with public sector midwives in the Western region. Thus, we arrive at an LOP target of 33. CHANGES TO INDICATOR Changes to indicator: July, 2015: Added an LOP target and rationale. Other Notes (optional): THIS SHEET LAST UPDATED ON: July 31, 2015 Saving Maternity Homes in Ghana Project, Y4 Monitoring and Evaluation Plan 56 USAID Performance Indicator Reference Sheet Name of Result Measured (Goal, DO, IR, sub-IR, Project Purpose, Project Output, etc.): Sub-IR 2.1: PMH ownership/management transformations brokered Name of Indicator: 15. Percentage of PMHs that have a neutral or positive attitude towards the idea of ownership/ management transformation. Is this a Performance Plan and Report indicator? No _X__ Yes ____, for Reporting Year(s) _________ If yes, link to foreign assistance framework: DESCRIPTION Precise Definition(s): A PMH will be deemed to have a neutral or positive if they provide an answer of “Neutral”, “Somewhat interested” or “Very interested” to questions related to various forms of ownership/management transformation (selling, leasing, bringing on a partner, bringing on a manager). Unit of Measure: Numerator: Number of current PMH owners indicating a neutral or positive attitude on a Likert scale to a statement about an ownership/management transformation. Denominator: Number of PMH facilities responding to the PMH survey. Disaggregated by: This indicator will be disaggregated based on sex of the owner, geographic location of the facility and type of ownership/management transformation. Rationale or Justification for indicator (optional): PLAN FOR DATA COLLECTION Data Source: PMH Survey Method of data collection and construction: The project will interview PMH owners and managers using the PMH survey questions. The questions have been built into the formhub.org mobile data collection platform. Reporting Frequency: Baseline (beginning of project) and endline (end of project) Individual(s) responsible: Bernard Akotey, SMH Project Team Lead; Matthew Griffith, Banyan Global Program Manager DATA QUALITY ISSUES Dates of Previous Data Quality Assessments and name of reviewer: N/A Date of Future Data Quality Assessments (optional): Known Data Limitations: TARGETS AND BASELINE Baseline timeframe (optional): Baseline findings submitted in June, 2015 Saving Maternity Homes in Ghana Project, Y4 Monitoring and Evaluation Plan 57 Rationale for Targets (optional): LOP target for bringing on a manager: 90%. Rationale: At baseline, 88% of facilities had a neutral or positive attitude towards this form of transformation. We believe that, as the project sensitizes facilities about the advantages of this option and showcases success stories, the percentage of facilities interested will improve. However, we do believe that a sizable proportion of the 12% who were not interested might have specific reasons for their opinion, and so would not be very easily swayable, thus the modest goal of a 2% increase from baseline to endline. LOP target for bringing on a partner: 88%. Rationale: At baseline, 81% of facilities had a neutral or positive attitude towards this form of transformation. We believe that, as the project sensitizes facilities about the advantages of this option and showcases success stories, the percentage of facilities interested will improve, leading to an increase of 7% from baseline to endline. LOP target for leasing out the facility: 67%. Rationale: At baseline, 55% of facilities had a neutral or positive attitude towards this form of transformation. We believe that, as the project sensitizes facilities about the advantages of this option and showcases success stories, the percentage of facilities interested will improve. We also believe the low level of interest at baseline may reflect lack of understanding, and that these sensitization efforts may have a greater impact than it will on the transformations listed above. Thus, we set a target of an increase in around 12% from baseline to endline. LOP target for selling the facility: 38%. Rationale: At baseline, 28% of facilities had a neutral or positive attitude towards this form of transformation. We believe that, as the project sensitizes facilities about the advantages of this option and showcases success stories, the percentage of facilities interested will improve. And while we also believe the low level of interest at baseline may reflect lack of understanding, key informants indicated that there may be a visceral opposition to this particular form of transformation, making opinions somewhat difficult to change. Thus, we set a target of an increase in around 10% from baseline to endline. CHANGES TO INDICATOR Changes to indicator: July, 2015: Added an LOP target and rationale. Other Notes (optional): THIS SHEET LAST UPDATED ON: July 31, 2015 Saving Maternity Homes in Ghana Project, Y4 Monitoring and Evaluation Plan 58 USAID Performance Indicator Reference Sheet Name of Result Measured (Goal, DO, IR, sub-IR, Project Purpose, Project Output, etc.): Sub-IR 2.2: Funding Provided and Leveraged by the Project Name of Indicator: 16. Number of assisted PMHs that receive seed funding in the form of SMH small grants Is this a Performance Plan and Report indicator? No _X__ Yes ____, for Reporting Year(s) _________ If yes, link to foreign assistance framework: DESCRIPTION Precise Definition(s): Seed funding refers to relatively small amounts of funding that the project plans to use to reduce risk for local investors to invest in PMHs; and provide younger midwives with additional capital. Small grants refers to the mechanism through which the project will provide this funding. Unit of Measure: The total number of PMHs that receive seed funding from the SMH project in a given year. Disaggregated by: This indicator will be disaggregated based on sex of the owner and geographic location of the facility. Rationale or Justification for indicator (optional): The project will bring non-traditional financial partners to the sector, exploring financing from a variety of sources, including commercial banks, angel investors, investment clubs and crowdfunding. This financing will be complemented by a small amount of seed funding from the project. PLAN FOR DATA COLLECTION Data Source: Project records Method of data collection and construction: Through its grants management systems, the project will track the required data to report against this indicator. Reporting Frequency: Annually Individual(s) responsible: Bernard Akotey, SMH Project Team Lead; Matthew Griffith, Banyan Global Program Manager DATA QUALITY ISSUES Dates of Previous Data Quality Assessments and name of reviewer: N/A Date of Future Data Quality Assessments (optional): Known Data Limitations: TARGETS AND BASELINE Baseline timeframe (optional): Baseline findings submitted in June, 2015 Rationale for Targets (optional): LOP target: 20. Rationale: We have set the LOP target equal to the number of anticipated transformations. The assumption here is that, even if some transformation do not require seed funding to finalize the transformation arrangement (mainly, in the case of management contracts), all transformed facilities could benefit from seed funding to maximize the gains from the transformation. The baseline assessment found that access to finance was the most frequently cited constraint to growth (mentioned by 96% of respondents), and helping to address this constraint in the case of transformed facilities is an area in which the project can add clear value. CHANGES TO INDICATOR Changes to indicator: July, 2015: Added an LOP target and rationale. Other Notes (optional): THIS SHEET LAST UPDATED ON: July 31, 2015 Saving Maternity Homes in Ghana Project, Y4 Monitoring and Evaluation Plan 59 USAID Performance Indicator Reference Sheet Name of Result Measured (Goal, DO, IR, sub-IR, Project Purpose, Project Output, etc.): Sub-IR 2.2: Funding Provided and Leveraged by the Project Name of Indicator: 17. Amount of seed funding provided by the SMH project in the form of small grants Is this a Performance Plan and Report indicator? No _X__ Yes ____, for Reporting Year(s) _________ If yes, link to foreign assistance framework: DESCRIPTION Precise Definition(s): Seed funding refers to relatively small amounts of funding that the project plans to use to reduce risk for local investors to invest in PMHs; and provide younger midwives with additional capital. Small grants refers to the mechanism through which the project will provide this funding. Unit of Measure: Amount of grant funding in US$ provided to the PMHs by the SMH project for transformation in a given year. Disaggregated by: This indicator will be disaggregated based on sex of the owner and geographic location of the facility. Rationale or Justification for indicator (optional): The project will bring non-traditional financial partners to the sector, exploring financing from a variety of sources, including commercial banks, angel investors, investment clubs and crowdfunding. This financing will be complemented by a small amount of seed funding from the project. PLAN FOR DATA COLLECTION Data Source: Project records Method of data collection and construction: Through its grants management systems, the project will track the required data to report against this indicator. Reporting Frequency: Annually Individual(s) responsible: Bernard Akotey, SMH Project Team Lead; Matthew Griffith, Banyan Global Program Manager DATA QUALITY ISSUES Dates of Previous Data Quality Assessments and name of reviewer: N/A Date of Future Data Quality Assessments (optional): Known Data Limitations: TARGETS AND BASELINE Baseline timeframe (optional): Baseline findings submitted in June, 2015 Rationale for Targets (optional): LOP target: $100,000. The baseline found that while facilities viewed access to finance as the single most significant constraint (cited by 96% of respondents), only a small number of facilities had used outside capital to facilitate the founding or expanding of their facilities. It was much more common for facilities to us their own savings, or to borrow from family and friend. Thus, we believe that access to SMH project grant funds will be welcome, but that the initial uses of these funds might be modest, with investments averaging $5,000 per transformed facility. Multiplying $5,000 by the projected number of transformed facilities (20), we arrive at an LOP target of $100,000. CHANGES TO INDICATOR Changes to indicator: July, 2015: Added an LOP target and rationale. Other Notes (optional): THIS SHEET LAST UPDATED ON: July 31, 2015 Saving Maternity Homes in Ghana Project, Y4 Monitoring and Evaluation Plan 60 USAID Performance Indicator Reference Sheet Name of Result Measured (Goal, DO, IR, sub-IR, Project Purpose, Project Output, etc.): Sub-IR 2.2: Funding Provided and Leveraged by the Project Name of Indicator: 18. Number of assisted PMHs that receive funds from non-project sources Is this a Performance Plan and Report indicator? No _X__ Yes ____, for Reporting Year(s) _________ If yes, link to foreign assistance framework: DESCRIPTION Precise Definition(s): Funds refers to investment, grant, or loan financing. Non-project sources referrers to third-party sources of financing, such as commercial banks, angel investors, investment clubs, or crowdfunding donors. Unit of Measure: Number of PMHs receiving project assistance that receive funds from non-project sources for transformation in a given year. Disaggregated by: This indicator will be disaggregated based on sex of the owner and geographic location of the facility. Rationale or Justification for indicator (optional): PLAN FOR DATA COLLECTION Data Source: Project records Method of data collection and construction: The project will track the required data to report against this indicator. Reporting Frequency: Annually Individual(s) responsible: Bernard Akotey, SMH Project Team Lead; Matthew Griffith, Banyan Global Program Manager DATA QUALITY ISSUES Dates of Previous Data Quality Assessments and name of reviewer: N/A Date of Future Data Quality Assessments (optional): Known Data Limitations: TARGETS AND BASELINE Baseline timeframe (optional): Baseline findings submitted in June, 2015 Rationale for Targets (optional): LOP target: 13. Rationale: The project plans to strongly encourage facilities completing an ownership/management transformation to seek counter-party funding from non￾project sources, and will provide assistance in helping them access this funding. However, we do not expect all transformed facilities to succeed in acquiring this external funding. Thus, we set this target at two-thirds of the projected number of transformed facilities (20), arriving at a target of 13. CHANGES TO INDICATOR Changes to indicator: July, 2015: Added an LOP target and rationale. Other Notes (optional): THIS SHEET LAST UPDATED ON: July 31, 2015 Saving Maternity Homes in Ghana Project, Y4 Monitoring and Evaluation Plan 61 USAID Performance Indicator Reference Sheet Name of Result Measured (Goal, DO, IR, sub-IR, Project Purpose, Project Output, etc.): Sub-IR 2.2: Funding Provided and Leveraged by the Project Name of Indicator: 19. Amount of funding that assisted PMHs receive from non-project sources Is this a Performance Plan and Report indicator? No _X__ Yes ____, for Reporting Year(s) _________ If yes, link to foreign assistance framework: DESCRIPTION Precise Definition(s): Funding refers to amount of investment, grant, or loan financing. Non-project sources referrers to third-party sources of financing, such as commercial banks, angel investors, investment clubs, or crowdfunding donors. Unit of Measure: Amount of funding from non-project sources received by project-assisted PMHs for the transformation of PMHs resulting from the project interventions. Disaggregated by: This indicator will be disaggregated based on sex of the owner and geographic location of the facility. Rationale or Justification for indicator (optional): PLAN FOR DATA COLLECTION Data Source: Project records Method of data collection and construction: The project will track the required data to report against this indicator. Reporting Frequency: Annually Individual(s) responsible: Bernard Akotey, SMH Project Team Lead; Matthew Griffith, Banyan Global Program Manager DATA QUALITY ISSUES Dates of Previous Data Quality Assessments and name of reviewer: N/A Date of Future Data Quality Assessments (optional): Known Data Limitations: TARGETS AND BASELINE Baseline timeframe (optional): Baseline findings submitted in June, 2015 Rationale for Targets (optional): LOP target: $70,000. The project believes that the average amount of third-party funding, when it is acquired, will be roughly equal to the average amount of SMH project grant funding: $5,000. If we multiply this projected average by the project number of facilities that will secure third party funding (14), we come to a projection of $70,000. CHANGES TO INDICATOR Changes to indicator: July, 2015: Added an LOP target and rationale. Other Notes (optional): THIS SHEET LAST UPDATED ON: July 31, 2015 Saving Maternity Homes in Ghana Project, Y4 Monitoring and Evaluation Plan 62 USAID Performance Indicator Reference Sheet Name of Result Measured (Goal, DO, IR, sub-IR, Project Purpose, Project Output, etc.): Sub-IR 3.1: Capacity of PMHs to sustain, increase and improve service delivery strengthened Name of Indicator: 20. Percentage of assisted, non-accredited PMHs applying for NHIA accreditation that successfully become accredited. Is this a Performance Plan and Report indicator? No _X__ Yes ____, for Reporting Year(s) _________ If yes, link to foreign assistance framework: DESCRIPTION Precise Definition(s): NHIA is the National Health Insurance Authority; Accreditation is NHIA’s process for deciding if a health facility is able to serve NHIA enrollees, and receive reimbursement for those services. Unit of Measure: Numerator: Number of (a) non-accredited project-assisted PMHs that receive a grade that is sufficient to receive NHIA accreditation. Denominator: Non-accredited, project-assisted PMHs that apply to NHIA to become accredited. Disaggregated by: This indicator will be disaggregated based on sex of the owner and geographic location of the facility. Rationale or Justification for indicator (optional): This indicator provides information on the rate at which project-assisted PMHs are successfully participating in Ghana’s national health insurance scheme. PLAN FOR DATA COLLECTION Data Source: PMH survey and NHIA Method of data collection and construction: The project will collect the necessary data through the PMH survey, and from NHIA. Reporting Frequency: Baseline and endline Individual(s) responsible: Bernard Akotey, SMH Project Team Lead; Matthew Griffith, Banyan Global Program Manager DATA QUALITY ISSUES Dates of Previous Data Quality Assessments and name of reviewer: N/A Date of Future Data Quality Assessments (optional): Known Data Limitations: Any difficulty or delay in accessing the required data from NHIA could affect the project’s ability to report against this indicator. TARGETS AND BASELINE Baseline timeframe (optional): Baseline findings submitted in June, 2015 Rationale for Targets (optional): LOP target: 5% Rationale: At baseline, a total of 81% of facilities were registered with NHIA. If we extend that percentage to all assisted PMHs (146), we could project that around 118 facilities are registered with NHIA, while around 28 facilities are not. The SMH interventions will increase recognition of the financial advantages of NHIA accreditation in terms of generating revenue and accessing clients in these underserved areas on which the project is focusing, almost all of whom rely on NHIS). We believe that, through these efforts, we can effect a 5% increase in NHIA accreditation, effectively resulting in 7 assisted facilities (5% of the total 146 assisted facilities) registering with NHIA that were not registered before. CHANGES TO INDICATOR Changes to indicator: July, 2015: Added an LOP target and rationale. Other Notes (optional): There may be limitations based on the NHIA accreditation timeline. Some transformed PMHs may be in the process of becoming evaluated, or may be waiting on the next round of accreditation evaluations. In this case, there may be limitations on the ability to provide a full account of the project’s achievement against this indicator. Saving Maternity Homes in Ghana Project, Y4 Monitoring and Evaluation Plan 63 THIS SHEET LAST UPDATED ON: July 31, 2015 Saving Maternity Homes in Ghana Project, Y4 Monitoring and Evaluation Plan 64 USAID Performance Indicator Reference Sheet Name of Result Measured (Goal, DO, IR, sub-IR, Project Purpose, Project Output, etc.): Sub-IR 3.1: Capacity of PMHs to sustain, increase and improve service delivery strengthened Name of Indicator: 21. Percentage of assisted PMHs that receive or maintain NHIA accreditation after transformation Is this a Performance Plan and Report indicator? No _X__ Yes ____, for Reporting Year(s) _________ If yes, link to foreign assistance framework: DESCRIPTION Precise Definition(s): NHIA is the National Health Insurance Authority; Accreditation is NHIA’s process for deciding if a health facility is able to serve NHIA enrollees, and receive reimbursement for those services. Unit of Measure: Numerator: Number of assisted PMHs that complete transformation AND that receive or maintain NHIA accreditation after transformation is complete. Denominator: Number of assisted PMHs that complete transformation. Disaggregated by: This indicator will be disaggregated based on sex of the owner and geographic location of the facility. Rationale or Justification for indicator (optional): This indicator measures the effect of transformation on NHIA accreditation among PMHs. PLAN FOR DATA COLLECTION Data Source: NHIA, Project Records Method of data collection and construction: The project will collect the necessary data from NHIA, and will also collect data from PMHs that complete transformation to inform this indicator. Reporting Frequency: Baseline and endline Individual(s) responsible: Bernard Akotey, SMH Project Team Lead; Matthew Griffith, Banyan Global Program Manager DATA QUALITY ISSUES Dates of Previous Data Quality Assessments and name of reviewer: N/A Date of Future Data Quality Assessments (optional): Known Data Limitations: Any difficulty or delay in accessing the required data from NHIA could affect the project’s ability to report against this indicator. TARGETS AND BASELINE Baseline timeframe (optional): Baseline findings submitted in June, 2015 Rationale for Targets (optional): LOP target: 100%. PMHs who complete an ownership/management transformation will have benefitted from the projects broad sensitization efforts regarding the benefits of accreditation, as well as pre- and post-transformation support, which will also reinforce these benefits. Thus, we believe transformed facilities will recognize the value of catering to NHIS cardholders, who form a sizeable portion of the potential clients available, and that recognition will lead all transformed facilities to maintain or achieve accreditation after transformation. CHANGES TO INDICATOR Changes to indicator: July, 2015: Added an LOP target and rationale. Other Notes (optional): There may be limitations based on the NHIA accreditation timeline. Some transformed PMHs may be in the process of becoming evaluated, or may be waiting on the next round of accreditation evaluations. In this case, there may be limitations on the ability to provide a full account of the project’s achievement against this indicator. THIS SHEET LAST UPDATED ON: July 31, 2015 Saving Maternity Homes in Ghana Project, Y4 Monitoring and Evaluation Plan 65 USAID Performance Indicator Reference Sheet Name of Result Measured (Goal, DO, IR, sub-IR, Project Purpose, Project Output, etc.): Sub-IR 3.2: Increased capacity of PMHs to manage their businesses efficiently Name of Indicator: 22. Number of business training events conducted Is this a Performance Plan and Report indicator? No _X__ Yes ____, for Reporting Year(s) _________ If yes, link to foreign assistance framework: DESCRIPTION Precise Definition(s): A business training event refers to the production and delivery of one of the SMH project’s one-on-one business training sessions. Unit of Measure: Number of one-on-one training and one-on-one capacity building events provided by the project to participating PMHs. Disaggregated by: Rationale or Justification for indicator (optional): PLAN FOR DATA COLLECTION Data Source: Project Records Method of data collection and construction: The project will collect the necessary data to report against this indicator Reporting Frequency: Quarterly Individual(s) responsible: Bernard Akotey, SMH Project Team Lead; Matthew Griffith, Banyan Global Program Manager DATA QUALITY ISSUES Dates of Previous Data Quality Assessments and name of reviewer: N/A Date of Future Data Quality Assessments (optional): Known Data Limitations: TARGETS AND BASELINE Baseline timeframe (optional): Baseline findings submitted in June, 2015 Rationale for Targets (optional): LOP target: 73. Rationale: This indicator referrers to the one-on￾one pre-transformation and post-transformation technical assistance. We are projecting that 33 facilities will receive pre-transformation technical assistance, in the form of one structured TA visit (in the form of a business counseling visit), resulting in 33 events. (The facilities who prove to be serious about proceeding with a transformation will receive additional, unstructured support in that process.) In terms of post-transformation assistance, we are projecting that 20 PMHs will achieve transformation, and receive two structure post-transformation technical assistance visits, resulting in 40 events. The total LOP projection is thus 73 events. CHANGES TO INDICATOR Changes to indicator: July, 2015: Added an LOP target and rationale, and updated this indicator to focus on one-on-one business training efforts. Other Notes (optional): THIS SHEET LAST UPDATED ON: February 8, 2016 Saving Maternity Homes in Ghana Project, Y4 Monitoring and Evaluation Plan 66 USAID Performance Indicator Reference Sheet Name of Result Measured (Goal, DO, IR, sub-IR, Project Purpose, Project Output, etc.): Sub-IR 3.2: Increased capacity of PMHs to manage their businesses efficiently Name of Indicator: 23. Percentage of assisted PMHs maintaining core financial documents (cash log, profit and loss statements or income statements) Is this a Performance Plan and Report indicator? No _X__ Yes ____, for Reporting Year(s) _________ If yes, link to foreign assistance framework: DESCRIPTION Precise Definition(s): Core financial documents include: cash log, profit and loss statement and income statement Unit of Measure: Numerator: Project-assisted PMHs that report having one of these three documents in place, and up to date. Denominator: All project-assisted PMHs. Disaggregated by: This indicator will be disaggregated based on sex of the owner and geographic location of the facility. Rationale or Justification for indicator (optional): This indicator refers to the practice of keeping up to date fundamental financial documents. PLAN FOR DATA COLLECTION Data Source: PMH Survey Method of data collection and construction: The project will interview PMH owners and managers using the PMH survey questions. The questions have been built into the formhub.org mobile data collection platform. Reporting Frequency: Baseline and endline Individual(s) responsible: Bernard Akotey, SMH Project Team Lead; Matthew Griffith, Banyan Global Program Manager DATA QUALITY ISSUES Dates of Previous Data Quality Assessments and name of reviewer: N/A Date of Future Data Quality Assessments (optional): Known Data Limitations: TARGETS AND BASELINE Baseline timeframe (optional): Baseline findings submitted in June, 2015 Rationale for Targets (optional): LOP target: 75%. Rationale: At baseline, 67% of facilities kept at least one of the core financial documents described in this indicator. The project believes that, through its broad training support (primarily the BSS) and the more tailored support given to Tier 3 and Tier 4 PMHs, it can positively influence this percentage. That said, the project recognizes the challenges that business management training interventions have traditionally had in producing substantial behavior change in terms of financial record keeping. Thus, we set an indicator of an 8% increase over baseline, resulting in 75%. CHANGES TO INDICATOR Changes to indicator: July, 2015: Added an LOP target and rationale. Other Notes (optional): THIS SHEET LAST UPDATED ON: July 31, 2015 Saving Maternity Homes in Ghana Project, Y4 Monitoring and Evaluation Plan 67 USAID Performance Indicator Reference Sheet Name of Result Measured (Goal, DO, IR, sub-IR, Project Purpose, Project Output, etc.): Sub-IR 3.3: Management and programmatic capacity of GRMA increased. Name of Indicator: 24. Number of GRMA staff and volunteers who receive capacity building assistance Is this a Performance Plan and Report indicator? No _X__ Yes ____, for Reporting Year(s) _________ If yes, link to foreign assistance framework: DESCRIPTION Precise Definition(s): GRMA staff include paid staff and volunteers. Unit of Measure: Number of individuals who are working for GRMA who receive capacity building assistance Disaggregated by: Rationale or Justification for indicator (optional): PLAN FOR DATA COLLECTION Data Source: Project records Method of data collection and construction: The project will collect the necessary data to report against this indicator Reporting Frequency: Annually Individual(s) responsible: Bernard Akotey, SMH Project Team Lead; Matthew Griffith, Banyan Global Program Manager DATA QUALITY ISSUES Dates of Previous Data Quality Assessments and name of reviewer: N/A Date of Future Data Quality Assessments (optional): Known Data Limitations: TARGETS AND BASELINE Baseline timeframe (optional): Baseline findings submitted in June, 2015 Rationale for Targets (optional): LOP target: 14. Rationale: The total population of GRMA personnel that could be trained includes the 20 member Leadership Council, and three secretariat staff. The project does not believe it will have trouble securing time from the secretariat staff, but to date there have been challenges securing the time of some members of the Leadership Council. Thus, the project is aiming to train eleven members of the Leadership Council and all three member of the secretariat staff, for a total LOP target of 14. CHANGES TO INDICATOR Changes to indicator: July, 2015: Added an LOP target and rationale. Other Notes (optional): THIS SHEET LAST UPDATED ON: July 31, 2015 Saving Maternity Homes in Ghana Project, Y4 Monitoring and Evaluation Plan 68 ANNEX 4: PMH SURVEY SMH PROJECT PMH SURVEY PART 1: PRE-INTERVIEW DATA The interviewer should complete the following questions before beginning the interview. # Category Answers (and Coding) Instructions 1.1 Interviewer Name 1.2 Date 1 5 Day Month Year 1.3 Region 1.4 District 1.5 Town 1.6 GPS Coordinates Please record the GPS coordinates of the facility. 1.7 Photograph of facility sign Please take a photograph of the facility sign or building. PART 2: INTRODUCTION AND INFORMED CONSENT GRMA will provide an introduction. The introduction will include the following:  Description of the SMH project, and the roles of GRMA and Banyan Global  Explanation of the two parts of this visit: 1. The SMH survey questionnaire, for information gathering purposes 2. The GRMA Peer Monitoring Visit (PMV), the purposes of which is to gather information, and to provide feedback to the PMH abut areas in which they might improve  Estimate of the total time required – about 2 hours  An understanding of the sequencing 1. We will ask some general questions about your facility, that are relevant to both GRMA and Banyan Global 2. Banyan Global will ask some questions, while GRMA tours the facility and reviews whether specific equipment and other items are in place 3. GRMA will ask some questions, while Banyan Global works with assistants (if possible) to sight some specific documents related to business operations 4. GRMA will give specific feedback (Banyan Global will not be in the room) 5. We will thank you for your time, and conclude the visit Saving Maternity Homes in Ghana Project, Y4 Monitoring and Evaluation Plan 69 Once GRMA has concluded the introduction, provide the respondent with the Informed Consent Form, and allow them time to read through it thoroughly. Answer any questions that they ask, and if they do not volunteer any questions, ask them specifically if they have any questions. Once all questions have been answered, and they wish to proceed, ask them to sign the form. If they do not wish to proceed or do not which to sign the form, assure them that there will be no adverse repercussions for not participating, thank them for their time, and depart. Note: [Respondents will be limited to persons able to give personal consent. Subjects not able to give personal consent (e.g. for cultural reasons, children or adolescents less than the legal age of consent, subjects with mental illness) should not be subjects of this study, and will not be interviewed. If the respondent is incapable of reading and signing the written consent form, the enumerators will read the form to them in order to acquire informed consent.] PART 3: FACILITY INFORMATION The interviewer should read aloud: I’d like to start by learning a little bit about your facility. # Question Answers (and Coding) Instructions N/A Could you tell us a bit more about the history of this facility? N/A The purpose of this question is to give the respondent a chance to talk about the facility a bit, and to put her at ease. Do not worry about recording any information at this point (though some of what is said might be relevant for later questions). But do pay attention to the time, and do not let the respondent speak for more than 1-2 minutes. 3.1 What is your name? 3.2 Respondent sex Male……………………………………………..…1 Female………………………………..……..............2 3.3 What is your position or title? If the respondent is not the owner, manager, or another person with significant responsibility, confer with GRMA about the possibility of postponing the interview. 3.4 How long have you been working with this facility? 3.5 What is the name of the facility? Saving Maternity Homes in Ghana Project, Y4 Monitoring and Evaluation Plan 70 # Question Answers (and Coding) Instructions 3.6 What is the location address of the facility? Get the address, the district, and the region. 3.7 What is the facility’s landline telephone number (if any)? 3.8 What is the best mobile number for the facility? 3.9 What is the best email address for the facility? 3.10 What is the facility’s website (if any)? 3.11 How many years has the facility been in operation? ___________ years Don’t know………………………………..……..89 Refused……………...………..…………………..99 PART 4: OWNERSHIP AND MANAGEMENT Read aloud: Now I would like to ask some questions about the ownership and management of the facility. # Question Answers (and Coding) Instructions 4.1 How many individuals have an ownership stake in this facility? ___________ individual(s) If 1, go to question 4.2. 4.1.1 If there are multiple owners, please indicate the ownership stake (in percentage terms) for each owner. Collect the relationship of each partner (family, friend, business associate, etc.), and ownership stake of each individual. 4.2 What is the name of the primary owner of the facility? Same as respondent………………………..…........1 ______________ [name]………………………..2 Refused………………………………...………....99 The primary owner is the one who has the largest ownership stake. 4.3 What is the sex of the primary owner? Male……………………………..………............….1 Female…………….…………………….…..............2 4.3.1 What is the age of the primary owner? __________ years At this point, GRMA can take their leave to begin the tour of the facility. The Banyan Global enumerators will then continue with the questioning. 4.4 How long has this person been the owner of the facility? ___________ years Don’t know………………………………….........89 Refused………………………………….................99 4.4.1 What is the owner’s professional background? Midwife………………………………..............…….1 Other medical professional……………………..…2 Non-medical health professional……….…..............3 Other (Specify)………………..…………………...4 4.5 How did this person come to be the owner of the facility? Founded the facility………………………………..1 Purchased the facility…………………….………...2 Inherited from parents/family….……..…..………..3 Other (Specify)………………………...…………..4 Saving Maternity Homes in Ghana Project, Y4 Monitoring and Evaluation Plan 71 # Question Answers (and Coding) Instructions 4.5.1 If this person purchased the facility, what funds were used? Personal savings…………………………….….…..1 Loan from family/friends……………………….….2 Loan from financial institution…………………….3 Third party investment…………………...……….4 Grant from government…………………………..5 Grant from foundation or NGO………………….6 Other (Specify)……………………………..............7 The respondent should check all that apply. 4.6 Is the owner of the facility also the manager of the facility? Yes………………………………………….............1 No…………………………………………..............2 If yes, skip to question 4.10. 4.7 What is the sex of the manager? Male…………………………………………….….1 Female…………….…………………...…………...2 4.8 How long has this person been the manager of the facility? ___________ years Don’t know………………………….……….…..89 Refused……………………………...…….............99 4.9 What is the manager’s professional background? Midwife………………………………...…………..1 Other medical professional……………………….2 Non-medical health professional…………..............3 Other (Specify)…………………………….............4 4.10 To what extent do you agree with the following statement: “The financial future of private maternity homes like this one is promising.” Strongly agree…………………………..............…..1 Agree………………………...…………...................2 Neutral……………………..……………….......….3 Disagree……………………………………...……4 Strongly disagree………………………...………...5 4.11 To what extent do you agree with the following statement: that “Expanding or growing my facility would be a wise decision.” Strongly agree…………………………..............…..1 Agree………………………...…………...................2 Neutral……………………..……………….......….3 Disagree……………………………………...……4 Strongly disagree………………………...………...5 4.12 If you were to pursue expansion or growth, where might you see opportunities for expansion or growth? New building or addition………………………….1 Additional employees……………………………...2 Additional equipment or new technologies……….3 Other (specify)…………………………………….4 Respondent should check all that apply. 4.13 What are the constraints to the growth or expansion opportunities mentioned above? Limitations to the range of services you can offer…1 Not willing to invest at this time……………….….2 Insufficient funds…………………………………...3 Delays in payments from NHIA…………………...4 Owner/Manager does not have the time/energy to take the necessary steps………………………...…5 Owner/Manager does not have the business management skills that would be necessary……….6 Owner/Manager does not have the clinical skills that would be necessary………………………....……...7 Competition from other health facilities……..……8 Low client visits……………………………………9 Other (specify)………………………..……....…..10 Respondent should check all that apply. 4.14 What is the average reimbursement delay that you have _________ months Saving Maternity Homes in Ghana Project, Y4 Monitoring and Evaluation Plan 72 # Question Answers (and Coding) Instructions experienced from NHIA (in months)? 4.15 To what extent would you be willing to borrow funds to improve or expand this facility? Very willing……..………………...………………...1 Somewhat willing…...……………………………...2 Neutral……………………..…………………...….3 Not willing…..…………………………………..…4 Would never consider this.……………………….5 4.16 How interested are you in the idea of selling your facility someday? Very interested..…………………………………...1 Somewhat interested………………………….…...2 Neutral……………………..……………………....3 Not interested……………….……..……………...4 Would never consider this.…………………….….5 Emphasize to the respondent that we are only looking for her thoughts on this idea, and that her answers will not commit her to any course of action whatsoever. 4.17 How interested are you in the idea of leasing out your facility someday? Very interested..…………………………………...1 Somewhat interested………………………….…...2 Neutral……………………..……………………....3 Not interested……………….……..……………...4 Would never consider this.…………………….….5 Emphasize to the respondent that we are only looking for her thoughts on this idea, and that her answers will not commit her to any course of action whatsoever. 4.18 How interested are you in the idea of bringing on an additional business partner someday? Very interested..…………………………………...1 Somewhat interested………………………….…...2 Neutral……………………..……………………....3 Not interested……………….……..……………...4 Would never consider this.…………………….….5 Emphasize to the respondent that we are only looking for her thoughts on this idea, and that her answers will not commit her to any course of action whatsoever. 4.19 How interested are you in the idea of bringing on a manager, or a new manager if your current manager leaves, for the facility someday? Very interested..…………………………………...1 Somewhat interested………………………….…...2 Neutral……………………..……………………....3 Not interested……………….……..……………...4 Would never consider this.…………………….….5 Emphasize to the respondent that we are only looking for her thoughts on this idea, and that her answers will not commit her to any course of action whatsoever. 4.20 How interested are you in the idea of bringing on a new clinical staff member who brings a new set of skills to the practice? Very interested..…………………………………...1 Somewhat interested………………………….…...2 Neutral……………………..……………………....3 Not interested……………….……..……………...4 Would never consider this.…………………….….5 Emphasize to the respondent that we are only looking for her thoughts on this idea, and that her answers will not commit her to any course of action whatsoever. PART 5: FORMALIZATION AND BUSINESS PRACTICES Read: Now I would like to ask you some questions related to different aspects of formal business activities. These questions are only for our information, and it is okay if the facility is not engaging in some of these practices. Please answer honestly, and remember that your answers will remain confidential. Saving Maternity Homes in Ghana Project, Y4 Monitoring and Evaluation Plan 73 # Question Answers (and Coding) Instructions 5.1 Is the facility’s certification of MIDWIFE with the Nurses and Midwives Council up to date? Not registered………………………………...…...1 Registration is expired…………………………….2 Registration is current…………………………….3 5.2 Is the facility’s certification with the Health Facilities Regulatory Agency (HEFRA) up to date? Not registered…………………………………......1 Registration is expired…………………………….2 Registration is current…………………………….3 HEFRA was previously known as the Private Maternity Hospital Board. 5.3 Is the facility registered as a business with the Registrar General Department? Yes…………………………………………………1 No……………………………………………….....2 5.3.1 What is the incorporation model of the business? Sole Proprietor……………..………….…………..1 Limited Liability Company…………….……............2 Partnership………………………………………...3 5.4 Is the facility registered with the Ghana Revenue Authority (GRA)? Not registered…………………………………......1 Registered but don’t pay tax…….…………………2 Registered and pay tax…………..…………………3 5.5 Is the facility currently accredited with NHIA? Accredited…………………………………………1 Not accredited…………………….………………2 Accreditation withdrawn..………..………………..3 If no, go to question 5.6. 5.5.1 What was the score the facility received on its most recent NHIA evaluation? A+ A B C D Provisional Failed 5.6 Does the facility maintain written contracts or agreements with partners and suppliers? No written contracts…………………………...….1 Some written contracts.………….………………..2 Maintains full written contracts……………….........3 5.7 Does the facility have an up-to-date organogram, or a chart that shows who works at the facility and who everyone reports to? Yes…………………………………………………1 No……………………………………………….....2 Yes, but not up-to-date……………………………3 5.8 Is there a plan in place that clearly indicates who runs the facility if the owner/ manager is no longer able to run the facility? Yes…………………………………………………1 No……………………………………………….....2 5.9 Can you name the person who is going to take over? Yes…………………………………………………1 No……………………………………………….....2 Respondent provides the name, though you don’t need to record it. 5.10 To what extent would you agree with the following statement: “If the owner were to die or retire, the facility Strongly agree…………………………..............…..1 Agree………………………...…………...................2 Neutral……………………..……………….......….3 Disagree……………………………………...……4 Strongly disagree………………………...………...5 Saving Maternity Homes in Ghana Project, Y4 Monitoring and Evaluation Plan 74 # Question Answers (and Coding) Instructions would likely continue to operate.” 5.11 Does the facility maintain an annual business plan? Yes…………………………………………………1 No……………………………………………….....2 Is done informally, not documented…………...…..3 5.12 Does the facility maintain a regular budget? Yes…………………………………………………1 No……………………………………………….....2 If no, go to question 5.14. 5.12.1 How often does the facility create a budget? Annual budget……………………………….……..1 Quarterly budget…………………………………..2 Monthly budget………………………………….....3 5.12.2 How often does the facility review the budget (for example, to manage spending and guide decision-making)? Daily…………………….………………...………..1 Weekly……………….……………………...……..2 Monthly……………...………………………...…...3 Quarterly………………………………………….4 Annually……..……..………………………………5 5.13 Does the facility keep personal funds separate from business funds? No separation exists………………………………1 Some separation exists………………………….....2 Full separation……………………………………..3 5.14 Does the facility keep its funds in a bank account? No bank account…..……………………………1 Personal bank account………………………...2 Business bank account.………………………..3 If no, go to question 5.16. 5.15 Does the facility keep an up-to-date daily cash log? Yes…………………………………………………1 No……………………………………………….....2 5.16 Does the facility keep an up-to-date record of all income generated and expenses incurred (income statement)? Yes…………………………………………………1 No……………………………………………….....2 5.17 Does the facility keep an up-to-date record of your assets and liabilities/debts (balance sheet)? Yes…………………………………………………1 No……………………………………………….....2 5.18 To what extent does your business earn more money (revenues) than it spends (costs)? The business makes much more than it spends…....1 The business makes slightly more than it spends......2 What the business makes and what it spends are about equal………………………………………...3 The business makes slightly lower than it spends....4 The business makes much less than it spends……..5 Don’t know………………………………………89 Refused……………………………………….......99 5.19 What is the total income that your business earns during a typical calendar month after paying all expenses including salaries of employees? That is, what are the profits of your business during a typical calendar month (in cedis)? Provides an amount……………….………..……...1 Don’t know……………………………………....89 Refused……………………………………….......99 If they provide an amount, go to question 5.19.1. If they don’t know, or refuse, go to Part 6. 5.19.1 [Enter amount provided] __________ cedis Saving Maternity Homes in Ghana Project, Y4 Monitoring and Evaluation Plan 75 PART 6: SERVICES Read: Now I would like to ask you a few additional questions about your facility. # Question Answers (and Coding) Instructions 6.1 In a typical month in the dry season, what is the total number of outpatient visits your facility receives? Facility does not provide that service..…………….1 Provides an amount…………….……….………....2 Don’t know…..….………………………………..89 Refused…………………………………………...99 Please ask the respondent to answer based on records if those are easily available. If not, a best guess will be sufficient. If they don’t provide the service, don’t know, or refuse, go to question 6.1.1. [Enter amount provided] ___________ visits [Answered based on] Records………………………………………...….1 Best guess………………………………….………2 6.1.1 In a typical month in the rainy season, what is the total number of outpatient visits your facility receives? Facility does not provide that service..…………….1 Provides an amount…………….……….………....2 Don’t know…………………………………….....89 Refused…………………………………………...99 Please ask the respondent to answer based on records if those are easily available. If not, a best guess will be sufficient. If they don’t provide the service, don’t know, or refuse, go to question 6.2. [Enter amount provided] ___________ visits [Answered based on] Records………………………………………….....1 Best guess……………………………………….…2 6.2 In a typical month in the dry season, how many visits does your facility receive for maternal, newborn, and child health services? Facility does not provide that service..…………….1 Provides an amount…………….……….………....2 Don’t know………………………………………89 Refused…………………………………………...99 Please ask the respondent to answer based on records if those are easily available. If not, a best guess will be sufficient. If they don’t provide the service, don’t know, or refuse, go to question 6.2.1. [Enter amount provided.] ___________ visits [Answered based on] Records……………………………….……...…….1 Best guess………………………………….………2 6.2.1 In a typical month in the rainy season, how many visits does your facility receive for maternal, newborn, and child health services? Facility does not provide that service..…………….1 Provides an amount…………….……….………....2 Don’t know………………………………………89 Refused…………………………………………...99 Please ask the respondent to answer based on records if those are easily available. If not, a best guess will be sufficient. If they don’t provide the service, don’t know, or refuse, go to question 6.3. Saving Maternity Homes in Ghana Project, Y4 Monitoring and Evaluation Plan 76 # Question Answers (and Coding) Instructions [Enter amount provided.] ___________ visits [Answered based on] Records……………………………….……...…….1 Best guess………………………………….………2 6.3 In a typical month in the dry season, how many deliveries does your facility perform? Facility does not provide that service..…………….1 Provides an amount…………….……….………....2 Don’t know………………………………………89 Refused…………………………………………...99 Please ask the respondent to answer based on records if those are easily available. If not, a best guess will be sufficient. If they don’t provide the service, don’t know, or refuse, go to question 6.3.1. [Enter amount provided.] ___________ deliveries [Answered based on] Records……………………………….……...…….1 Best guess………………………………….………2 6.3.1 In a typical month in the rainy season, how many deliveries does your facility perform? Facility does not provide that service..…………….1 Provides an amount…………….……….………....2 Don’t know………………………………………89 Refused…………………………………………...99 Please ask the respondent to answer based on records if those are easily available. If not, a best guess will be sufficient. If they don’t provide the service, don’t know, or refuse, go to question 6.4. [Enter amount provided.] ___________ deliveries [Answered based on] Records……………………………….……...…….1 Best guess………………………………….………2 6.4 In a typical month in the dry season, how many counseling visits for family planning or reproductive health does your facility provide? Facility does not provide that service..…………….1 Provides an amount…………….……….………....2 Don’t know………………………………………89 Refused…………………………………………...99 Please ask the respondent to answer based on records if those are easily available. If not, a best guess will be sufficient. If they don’t provide the service, don’t know, or refuse, go to question 6.4.1. [Enter amount provided.] ___________ visits [Answered based on] Records……………………………….……...…….1 Best guess………………………………….………2 6.4.1 In a typical month in the rainy season, how many counseling visits for family planning or reproductive health does your facility provide? Facility does not provide that service..…………….1 Provides an amount…………….……….………....2 Don’t know………………………………………89 Refused…………………………………………...99 Please ask the respondent to answer based on records if those are easily available. If not, a best guess will be sufficient. If they don’t provide the Saving Maternity Homes in Ghana Project, Y4 Monitoring and Evaluation Plan 77 # Question Answers (and Coding) Instructions service, don’t know, or refuse, go to question 6.5. [Enter amount provided.] ___________ visits [Answered based on] Records……………………………….……...…….1 Best guess………………………………….………2 6.5 In a typical month in the dry season, how many cases of malaria does your facility treat? Facility does not provide that service..…………….1 Provides an amount…………….……….………....2 Don’t know………………………………………89 Refused…………………………………………...99 Please ask the respondent to answer based on records if those are easily available. If not, a best guess will be sufficient. If they don’t provide the service, don’t know, or refuse, go to question 6.5.1. [Enter amount provided.] ___________ cases [Answered based on] Records……………………………….……...…….1 Best guess………………………………….………2 6.5.1 In a typical month in the rainy season, how many cases of malaria does your facility treat? Facility does not provide that service..…………….1 Provides an amount…………….……….………....2 Don’t know………………………………………89 Refused…………………………………………...99 Please ask the respondent to answer based on records if those are easily available. If not, a best guess will be sufficient. If they don’t provide the service, don’t know, or refuse, go to question 6.6. [Enter amount provided.] ___________ cases [Answered based on] Records……………………………….……...…….1 Best guess………………………………….………2 6.6 In a typical month in the dry season, how many cases of diarrhea does your facility treat? Facility does not provide that service..…………….1 Provides an amount…………….……….………....2 Don’t know………………………………………89 Refused…………………………………………...99 Please ask the respondent to answer based on records if those are easily available. If not, a best guess will be sufficient. If they don’t provide the service, don’t know, or refuse, go to question 6.6.1. [Enter amount provided.] ___________ cases [Answered based on] Records……………………………….……...…….1 Best guess………………………………….………2 6.6.1 In a typical month in the rainy season, how many cases of diarrhea does your facility treat? Facility does not provide that service..…………….1 Provides an amount…………….……….………....2 Don’t know………………………………………89 Refused…………………………………………...99 Please ask the respondent to answer based on records if those are easily Saving Maternity Homes in Ghana Project, Y4 Monitoring and Evaluation Plan 78 # Question Answers (and Coding) Instructions available. If not, a best guess will be sufficient. If they don’t provide the service, don’t know, or refuse, go to question 6.7. [Enter amount provided.] ___________ cases [Answered based on] Records……………………………….……...…….1 Best guess………………………………….………2 6.7 In a typical month in the dry season, how many cases of minor cuts and injuries does your facility treat? Facility does not provide that service..…………….1 Provides an amount…………….……….………....2 Don’t know………………………………………89 Refused…………………………………………...99 Please ask the respondent to answer based on records if those are easily available. If not, a best guess will be sufficient. If they don’t provide the service, don’t know, or refuse, go to question 6.7.1. [Enter amount provided.] ___________ cases [Answered based on] Records……………………………….……...…….1 Best guess………………………………….………2 6.7.1 In a typical month in the rainy season, how many cases of minor cuts and injuries does your facility treat? Facility does not provide that service..…………….1 Provides an amount…………….……….………....2 Don’t know………………………………………89 Refused…………………………………………...99 Please ask the respondent to answer based on records if those are easily available. If not, a best guess will be sufficient. If they don’t provide the service, don’t know, or refuse, go to question 6.8. [Enter amount provided.] ___________ cases [Answered based on] Records……………………………….……...…….1 Best guess………………………………….………2 6.8 In a typical month in the dry season, how many visits does your facility receive for the administering of vaccinations? Facility does not provide that service..…………….1 Provides an amount…………….……….………....2 Don’t know………………………………………89 Refused…………………………………………...99 Please ask the respondent to answer based on records if those are easily available. If not, a best guess will be sufficient. If they don’t provide the service, don’t know, or refuse, go to question 6.8.1. [Enter amount provided.] ___________ visits [Answered based on] Records……………………………….……...…….1 Best guess………………………………….………2 Saving Maternity Homes in Ghana Project, Y4 Monitoring and Evaluation Plan 79 # Question Answers (and Coding) Instructions 6.8.1 In a typical month in the rainy season, how many visits does your facility receive for the administering of vaccinations? Facility does not provide that service..…………….1 Provides an amount…………….……….………....2 Don’t know………………………………………89 Refused…………………………………………...99 Please ask the respondent to answer based on records if those are easily available. If not, a best guess will be sufficient. If they don’t provide the service, don’t know, or refuse, go to question 6.9. [Enter amount provided.] ___________ visits [Answered based on] Records……………………………….……...…….1 Best guess………………………………….………2 6.9 How many beds does your facility have? Facility does not provide that service..…………….1 Provides an amount…………….……….………....2 Don’t know………………………………………89 Refused…………………………………………...99 Please ask the respondent to answer based on records if those are easily available. If not, a best guess will be sufficient. If they don’t provide the service, don’t know, or refuse, go to question 6.10. [Enter amount provided.] ___________ beds [Answered based on] Records……………………………….……...…….1 Best guess………………………………….………2 6.10 In the dry season, how many inpatient days does your facility provide in a typical month? Facility does not provide that service..…………….1 Provides an amount…………….……….………....2 Don’t know………………………………………89 Refused…………………………………………...99 Please ask the respondent to answer based on records if those are easily available. If not, a best guess will be sufficient. If they don’t provide the service, don’t know, or refuse, go to question 6.11. [Enter amount provided.] ___________ inpatient days [Answered based on] Records……………………………….……...…….1 Best guess………………………………….………2 6.10.1 In the rainy season, how many inpatient days does your facility provide in a typical month? Facility does not provide that service..…………….1 Provides an amount…………….……….………....2 Don’t know………………………………………89 Refused…………………………………………...99 Please ask the respondent to answer based on records if those are easily available. If not, a best guess will be sufficient. If they don’t provide the service, don’t know, or refuse, go to question 6.11. Saving Maternity Homes in Ghana Project, Y4 Monitoring and Evaluation Plan 80 # Question Answers (and Coding) Instructions [Enter amount provided.] ___________ inpatient days [Answered based on] Records……………………………….……...…….1 Best guess………………………………….………2 6.11 How many full-time clinical staff are working at this facility? Facility does not provide that service..…………….1 Provides an amount…………….……….………....2 Don’t know………………………………………89 Refused…………………………………………...99 Clinical staff refers to: doctors, nurses, pharmacists, medical assistants, physiotherapists, etc. Please stress the importance of getting at least the respondent’s best guess. If they don’t provide the service, don’t know, or refuse, go to question 6.12. [Enter amount provided.] ___________ full-time clinical staff 6.12 How many full-time non￾clinical staff are working at this facility? Facility does not provide that service..…………….1 Provides an amount…………….……….………....2 Don’t know………………………………………89 Refused…………………………………………...99 Non-clinical staff refers to administrators, orderlies, accountants, nutrition officers, etc. Please stress the importance of getting at least the respondent’s best guess. If they don’t provide the service, don’t know, or refuse, go to question 6.13. [Enter amount provided.] ___________ full-time non-clinical staff 6.13 How many part-time clinical staff are working at this facility? Facility does not provide that service..…………….1 Provides an amount…………….……….………....2 Don’t know………………………………………89 Refused…………………………………………...99 Clinical staff refers to: doctors, nurses, pharmacists, medical assistants, physiotherapists, etc. If they don’t provide the service, don’t know, or refuse, go to question 6.14. [Enter amount provided.] ___________ part-time clinical staff 6.14 How many part-time non-clinical staff are working at this facility? Facility does not provide that service..…………….1 Provides an amount…………….……….………....2 Don’t know………………………………………89 Refused…………………………………………...99 Non-clinical staff refers to administrators, orderlies, accountants, nutrition officers, etc. If they don’t provide the service, don’t know, or refuse, go to Part 7. Saving Maternity Homes in Ghana Project, Y4 Monitoring and Evaluation Plan 81 # Question Answers (and Coding) Instructions [Enter amount provided.] ___________ part-time non-clinical staff 7. SIGHTING DOCUMENTS This section focuses on the sighting and confirming of specific documents. If possible, this should be done with the assistance of the facility administration staff, while the owner (respondent) is working with GRMA on finishing the QAT, and discussing the outcomes. Say to the owner: “Thank you very much for all the information that you have provided so far. It has been very helpful, and we are very appreciative. At this point, we would like to review some of the documents discussed above. May we work with your administrative staff to view the relevant documents?” # Question Answers (and Coding) Instructions 7.1 May I view your certification of MIDWIFE with the Nurses and Midwives Council? Yes…………………………………………………1 No……………………………………………….....2 If no, go to question 7.2. 7.1.1 [Certification sighted] Yes…………………………………………………1 No……………………………………………….....2 7.1.2 [Certification current] Yes…………………………………………………1 No……………………………………………….....2 7.2 May I view the facility’s certification with the Health Facilities Regulatory Agency (HEFRA)? Yes…………………………………………………1 No……………………………………………….....2 If the certification is not current, ask to view the most recent certification. If no, go to question 7.3. 7.2.1 [Certification sighted] Yes…………………………………………………1 No……………………………………………….....2 7.2.2 [Certification current] Yes…………………………………………………1 No……………………………………………….....2 7.3 May I view the facility’s registration document with the Registrar General Department? Yes…………………………………………………1 No……………………………………………….....2 If no, go to question 7.4. 7.3.1 [Registration sighted] Yes…………………………………………………1 No……………………………………………….....2 For a limited liability company, the document is a Certificate of Incorporation and Commencement; For a sole proprietor, the document is a Certificate of Registration 7.3.2 [Registration current] Yes…………………………………………………1 No……………………………………………….....2 7.4 May I view evidence of your tax registration? Yes…………………………………………………1 No……………………………………………….....2 Sufficient documents include GRA Certificate of Registration, tax Saving Maternity Homes in Ghana Project, Y4 Monitoring and Evaluation Plan 82 # Question Answers (and Coding) Instructions receipts, or tax clearance certification If no, go to question 7.5. 7.4.1 [Tax document sighted] Yes…………………………………………………1 No……………………………………………….....2 7.4.2 [Tax document current] Yes…………………………………………………1 No……………………………………………….....2 7.5 May I see your NHIA accreditation certificate? Yes…………………………………………………1 No……………………………………………….....2 If no, go to question 7.6. 7.5.1 [Accreditation certificate sighted] Yes…………………………………………………1 No……………………………………………….....2 7.5.2 [Accreditation certificate current] Yes…………………………………………………1 No……………………………………………….....2 7.6 May I view an example of a written contract or agreement with a supplier? Yes…………………………………………………1 No……………………………………………….....2 If no, go to question 7.7. 7.6.1 [Document sighted] Yes…………………………………………………1 No……………………………………………….....2 7.7 May see an example of a bank statement? Yes…………………………………………………1 No……………………………………………….....2 If no, conclude the document sighting exercise. 7.7.1 [Document sighted] Yes…………………………………………………1 No……………………………………………….....2 At this point, you are finished sighting documents. Thank the staff member for his/her time and assistance in reviewing these documents, and ask if there is a place where you can wait for the GRMA staff to finish their meeting with the facility owner. (NOTE: You should NOT re-enter the area in which GRMA is meeting with the owner.) 8. DATA FROM QAT After both the survey questionnaire and the QAT sessions are concluded, there is some information to collect from GRMA. Request that your GRMA colleague provide you with the following information. This can be done in the car as you travel to the next place, or elsewhere. But it is important that you record this information accurately. # Question Answers (and Coding) Instructions 8.1 Emergency services (24 hours) Yes…………………………………………………1 No……………………………………………….....2 8.2 Outpatient services Yes…………………………………………………1 No……………………………………………….....2 8.3 In-patient services Yes…………………………………………………1 No……………………………………………….....2 8.4 Maternity services Yes…………………………………………………1 No……………………………………………….....2 If no, go to question 8.5. 8.4.1 ANC Yes…………………………………………………1 No……………………………………………….....2 8.4.2 Delivery Yes…………………………………………………1 No……………………………………………….....2 8.4.3 PNC Yes…………………………………………………1 No……………………………………………….....2 Saving Maternity Homes in Ghana Project, Y4 Monitoring and Evaluation Plan 83 # Question Answers (and Coding) Instructions 8.5 Family planning counseling Yes…………………………………………………1 No……………………………………………….....2 8.6 Family planning methods Yes…………………………………………………1 No……………………………………………….....2 If no, go to question 8.7. 8.6.1 Condoms Yes…………………………………………………1 No……………………………………………….....2 If no, go to question 8.6.2. 8.6.1.1 Female condoms Yes…………………………………………………1 No……………………………………………….....2 8.6.1.2 Male condoms Yes…………………………………………………1 No……………………………………………….....2 8.6.2 Combined Oral Contraceptives Yes…………………………………………………1 No……………………………………………….....2 8.6.3 Combined Injectable Contraceptives Yes…………………………………………………1 No……………………………………………….....2 8.6.4 Projestin-Only Pills Yes…………………………………………………1 No……………………………………………….....2 8.6.5 Projestin-Only Injectable Yes…………………………………………………1 No……………………………………………….....2 8.6.6 Implants Yes…………………………………………………1 No……………………………………………….....2 8.6.7 IUDs Yes…………………………………………………1 No……………………………………………….....2 8.7 Child welfare clinic Yes…………………………………………………1 No……………………………………………….....2 8.8 Under five immunization Yes…………………………………………………1 No……………………………………………….....2 8.9 Pharmaceutical services Yes…………………………………………………1 No……………………………………………….....2 8.10 Breast cancer screening Yes…………………………………………………1 No……………………………………………….....2 8.11 Cervical cancer screening Yes…………………………………………………1 No……………………………………………….....2 8.12 Diagnostic service Yes…………………………………………………1 No……………………………………………….....2 If no, go to question 8.13. 8.12.1 Laboratory Yes…………………………………………………1 No……………………………………………….....2 8.12.2 X-ray Yes…………………………………………………1 No……………………………………………….....2 8.12.3 Ultrasound scan Yes…………………………………………………1 No……………………………………………….....2 8.12.4 ECG Yes…………………………………………………1 No……………………………………………….....2 8.12.5 Other diagnostic services [Please specify] 8.13 Catering services Yes…………………………………………………1 No……………………………………………….....2 8.14 Specialist services [Please specify] Score in Emergency Response __________ points Score in Quality Assurance __________ points Saving Maternity Homes in Ghana Project, Y4 Monitoring and Evaluation Plan 84 # Question Answers (and Coding) Instructions Score in Safe Motherhood __________ points Score in Advocacy/Health Promotion __________ points Total score __________ points Grade A+ Excellent………………………...….…………..1 A Very Good………………………...….………….2 B+ Good……………………………...……………3 B Fairly Good……………………………...………4 C Fair………………………………………...…….5 D Pass…...…………………………………………6 E Fail……………………………………………….7 Saving Maternity Homes in Ghana Project, Y4 Monitoring and Evaluation Plan 85 ANNEX 5: QUALITY ASSESSMENT TOOL QUALITY ASSESSMENT TOOL Ghana Saving Private Maternity Homes Project GRMA 1/1/2015 Saving Maternity Homes in Ghana Project, Y4 Monitoring and Evaluation Plan 86 PREAMBLE PURPOSE This instrument reflects facilities and equipment available and the skills/activities to be performed by service providers at all levels (within the maternity home) as described in the national Safe Motherhood (SM) Protocols Document. The use of this questionnaire helps to (among other reasons) perform quality assessment and support supervision for multiple purposes including:  Determine the endline for the SMH project  Identify skills and service gaps of the PMH / comprehensive needs assessment  Quality of services provided and continuous performance improvement WHO ARE THE POTENTIAL USERS? The checklist should be used by qualified supervisors and trained enumerators with required technical and clinical skills in SM areas. Managers at various levels should utilized data collected to bring improvement into their facilities. WHAT ARE THE KEY STEPS Trainers/supervisors and enumerators team planning  What are the team objectives for the visit?  Who will visit whom, when and with what?  What are the expected output/outcomes of the visits?  Are tools /checklist ready and are enumerators trained?  Is Regional Chairperson of GRMA and person to be visited aware of date and purpose of visit? VISIT IMPLEMENTATION  Courtesy visit to person in charge of facility (purpose of visit, how the visit will be implemented, who are the persons to be involved, end of visit discussions, etc.)  Proceed with concerned person (explain purpose of visit and that you are not for fault finding, invite to open/honest discussion, discuss the process, proceed, give feedback). VISIT WRAP UP  Discuss findings both positive and those needing improvement from the visit.  Discuss next actions to be taken. TEAM EVALUATION MEETING  What happened?  What are the major outcomes?  What should be the next steps (next visit objectives)  Document findings of the visit.  What are the lessons learnt?  What are the recommendations? Saving Maternity Homes in Ghana Project, Y4 Monitoring and Evaluation Plan 87 FACILITY INFORMATION Name of Facility Location Address District Region Postal Address Telephone (Landline, if any) Mobile Phone Number (s) E-mail Website Facility Licensed by License Number Date of Expiry Name of Head of Facility Contact number (s) of Head E-mail of Head/Website Name of Owner of Facility Contact Number of Owner Gender of Owner E-mail of Owner (if different from facility head) Saving Maternity Homes in Ghana Project, Y4 Monitoring and Evaluation Plan 88 RANGE OF SERVICES Please tick services provided in the facility SERVICE DETAILS/REMARKS 1. Emergency services (24 hours) YES ( ) NO ( ) 2. Out-patient services YES ( ) NO ( ) 3. In-patient services YES ( ) NO ( ) 4. Maternity services (Tick details) YES ( ) NO ( ) ANC Delivery PNC 5. Family planning counselling YES ( ) NO ( ) 6. Provision of Family planning methods YES ( ) NO ( ) Condoms (tick details) YES ( ) NO ( ) Female Male Combined Oral Contraceptives YES ( ) NO ( ) Combined Injectable Contraceptives YES ( ) NO ( ) Projestin-Only pills YES ( ) NO ( ) Projestin-Only Injectable YES ( ) NO ( ) Implants YES ( ) NO ( ) IUDs YES ( ) NO ( ) 7. Child welfare Clinic YES ( ) NO ( ) 8. Under Five immunization YES ( ) NO ( ) 9. Pharmaceutical services YES ( ) NO ( ) 10. Breast cancer screening YES ( ) NO ( ) 11. Cervical Cancer Screening YES ( ) NO ( ) 12. Diagnostic service YES ( ) NO ( ) Laboratory YES ( ) NO ( ) X-ray YES ( ) NO ( ) Ultrasound scan YES ( ) NO ( ) ECG YES ( ) NO ( ) Other diagnostic services (specify) YES ( ) NO ( ) ………………………………… 13. Catering services YES ( ) NO ( ) 14. Specialist services (specify below) YES ( ) NO ( ) Saving Maternity Homes in Ghana Project, Y4 Monitoring and Evaluation Plan 89 GENERAL INFORMATION Year maternity home was opened: …………………………..………………………………………...…… Qualification of Head /Manager: A. Nursing: Certificate No: …………………………………………………………………………..………….. Date of qualification: …………………………….…………………………………….…………… Name /place of training school: ………………………………………………….………………… B. Midwifery: Certificate No: ……………………………………………....…………………..………………….. Date of qualification: ……………………………………………………………..………………… Name/place of training school: ……………………………………………………..………………. Any other Date of qualification: …………………………………………………………………… C. Any other qualification: (professional) ………………………………………….……………………… D. GRMA Registration of midwife who owns/heads the facility………….……… Date: ………………... E. GRMA Registration of all midwives who work in facility i. …………………………… Date: …………………. ii. ……………………………. Date: …………….…… iii. …………………………..... Date: …………………. iv. ……………………………. Date: ………….……… v. …………………………….. Date: …………………. vi. ……………………………. Date: ………...……….. PURPOSE OF VISIT Follow up: …………………………………………………………………………………………………… Needs assessment: ……………………………………………………………..…………………………… Quality care: ………………………..………………………..…………………………..…………………... Performance: ……………………………………………………………………………...………………… Team = Specific Objectives: ………………………………………………………………………………… Ask /discuss any success: concerns/problems and address them as appropriate during the visit How is your work progressing? What successes have you seen? What concerns or problems do you have? Saving Maternity Homes in Ghana Project, Y4 Monitoring and Evaluation Plan 90 Is there anything else you would like to ask or tell me? EMERGENCY RESPONSE (54 MARKS) 24 hrs Services 1. a. Do you have a joint duty roster? (Check for the roster for the past three months) Yes ( ) No ( ) 2 marks b. Is co-ordination of duty roster done by MW/Facility head? (Check for evidence) Yes ( ) No ( ) 3 marks 2. Has emergency team been trained for the past year? (Check for evidence) Yes ( ) No ( ) 5 marks 3. Have you made arrangement with any hospital in case of referral (Check for evidence) Yes ( ) No ( ) 5 marks 4. Does the PMH have links with a hospital in any of the following? - Clinical meeting Yes ( ) No ( ) 5 marks Check for evidence - Training Yes ( ) No ( ) 5 marks (Check for evidence) 5. Are the following protocols available and in use /displayed? (Select a midwife to interview and give full marks if displayed and correctly demonstrated by midwife) a. Eclampsia Yes ( ) No ( ) 2 marks b. PPH Yes ( ) No ( ) 2 marks c. APH Yes ( ) No ( ) 2 marks d. Diarrhoea Yes ( ) No ( ) 2 marks e. Convulsion Yes ( ) No ( ) 2 marks f. Anemia Yes ( ) No ( ) 2 marks g. URTI Yes ( ) No ( ) 2 marks h. Malaria Yes ( ) No ( ) 2 marks i. Shock Yes ( ) No ( ) 2 marks 6. Tracer Drugs availability (95 – 100 % availability) 2 marks 7. Do you have Resuscitation sets and functioning? - Sucker Yes ( ) No ( ) 3 marks - Oxygen Yes ( ) No ( ) 3 marks - Ambubag Yes ( ) No ( ) 3 marks (Check for evidence) Saving Maternity Homes in Ghana Project, Y4 Monitoring and Evaluation Plan 91 QUALITY ASSURANCE (83 MARKS) 1. Do you have a functional Quality Assurance Team Yes ( ) No ( ) 3 marks (Check for minutes and copies of latest report) 2. Have you carried out the following surveys? a. Patient Satisfaction Yes ( ) No ( ) 3 marks b. Rational Use of Medicines Yes ( ) No ( ) 3 marks 3. Do you have Action Plans to address QA problems identified (Check for evidence) Yes ( ) No ( ) 3 marks 4. Is there privacy, confidentiality in place for patients Yes ( ) No ( ) 3 marks (Observe consulting room/ward for evidence i.e. screen) INFECTION PREVENTION 5. Infection Prevention Practices (observe the following) a. Neat environment (Landscaping) Yes ( ) No ( ) 2 marks b. Puncture proof container waste disposal (Check)Yes ( ) No ( ) 2 marks c. Waste bin lined with polythene bag(check) Yes ( ) No ( ) 2 marks d. Waste bins boldly labelled (check) Yes ( ) No ( ) 2 marks e. Segregation of waste carried out (observe) Yes ( ) No ( ) 2 marks f. Site for Disposing Placenta (Observe) Yes ( ) No ( ) 2 marks g. Veronica Bucket with water/running water Yes ( ) No ( ) 2 marks 6. Is the following done at this level? a. Sterilization of articles (instrument/material) Yes ( ) No ( ) 3 marks b. Correct dilution of chorine (strength) Yes ( ) No ( ) 3 marks 7. Do you use partographs to monitor labour ` Yes ( ) No ( ) 4 marks (Check used partographs) 8. a. Do you have delivery sets? Yes ( ) No ( ) 1 marks (cord scissors, cord clamps) episiotomy scissors, sutures, speculum, kidney dishes) b. If or equals 2 set or more are sited. (check) Yes ( ) No ( ) 2 marks c. Is it sterile and packed Yes ( ) No ( ) 1 mark 9. Do you have a complaint desk or suggestion box Yes ( ) No ( ) 2 marks Check for evidence 10. Display of services available Yes ( ) No ( ) 2 marks a. Directional signs (Observe) Yes ( ) No ( ) 2 marks b. Prenatal / Perinatal Saving Maternity Homes in Ghana Project, Y4 Monitoring and Evaluation Plan 92 NEONATAL SURVIVAL 11. Do you have all your midwives trained in neonatal resuscitation? (Select a midwife to demonstrate) Yes ( ) No ( ) 3 marks 12. Do you have equipment for resuscitation (air way or suction machine)? Check for evidence. Yes ( ) No ( ) 3 marks 13. Do you undertake PNC for mothers/caregivers of children to educate them on the following? 14. (Check register / note book for the following)  Bed net usage Yes ( ) No ( ) 2 marks  Education on hygiene and sanitation Yes ( ) No ( ) 2 marks  Family Planning Yes ( ) No ( ) 2 marks INFANT SURVIVAL 15. Has your facility undertaken any nutrition training program? (Check for evidence) Yes ( ) No ( ) 2 marks 16. Is there any functional Mother support groups? (Check for evidence) Yes ( ) No ( ) 2 marks 17. Do you provide education on Infant feeding practices Check for evidence Yes ( ) No ( ) 2 marks 18. Is there a functional ORT corner in this facility? (Check for evidence) Yes ( ) No ( ) 2 marks UNDER FIVE SURVIVALS 19. a. Are health workers trained in malaria case management? Yes ( ) No ( ) 2 marks b. Are prescribers using ACT? Yes ( ) No ( ) 2 marks c. Does your facility confirm suspected malaria cases? (Microscopy/RDT)? (Evidence from OPD Cards or folder) Yes ( ) No ( ) 1 mark 20. a. Do you have the STG Document? Yes ( ) No ( ) 2 marks b. Are prescribers using the Standard Treatment Guideline (STG?) (Check for evidence) Yes ( ) No ( ) 2 marks 21. Are there laminated protocols of top 5 diseases? (Malaria, ART, diarrhoea, anaemia, malnutrition) Yes ( ) No ( ) 5 marks SAFE MOTHERHOOD (25 MARKS) ANTENATAL CARE Saving Maternity Homes in Ghana Project, Y4 Monitoring and Evaluation Plan 93 1. Is this facility practicing focus antenatal? Yes ( ) No ( ) 2 marks (Observe) 2. Maternal health records properly filled (BP recordings, comment on presence or absence of pedal oedema, temperature recordings, weight, urine test results especially for sugar and protein, hemoglobin measurements, malaria parasites) Yes ( ) No ( ) 2 marks MATERNITY SERVICES 3. Diagnostic tools for mothers available at each location i.e. consulting room, labour ward (Thermometers, sphygmomanometers, stethoscope, adult weighing scale, tape measure, spatula) Check for evidence Yes ( ) No ( ) 2 marks 4. Diagnostic tools for infants available (Tape measure, baby weighing scale, foetal stethoscope/sonicaid) Check for evidence Yes ( ) No ( ) 2 marks 5. Basic equipment available (Wheel chair(s), patient trolley(s), drip stands, wall clock, delivery bed,) Check for evidence Yes ( ) No ( ) 2 marks 6. Emergency medicines available (Oxytocin, cold chain for oxytocin, magnesium sulphate, ergometrine) Check for evidence Yes ( ) No ( ) 2 marks 7. Emergency packs for PPH (gloves, syntocinon, IV Fluids, wide bore cannula, elbow length glove, giving set, urinary catheter, urine bags) Yes ( ) No ( ) 2 marks 8. Emergency Pack for eclampsia (magnesium sulphate, IV fluids, gloves, Infusion, Delivery tray, General Infusion Tray. Yes ( ) No ( ) 2 marks OBSTETRIC CARE 9. Are SM protocols laminated and displayed Yes ( ) No ( ) 1 mark 10. Is a referral system in place? (Check for referral book) Yes ( ) No ( ) 1 mark 11. Community transport system pre-arranged? (Check ) Yes ( ) No ( ) 2 marks POST NATAL 12. Postnatal process correctly followed and documented Yes ( ) No ( ) 2 marks 1. Mother: (BP, examination of perineum, weight of mother, Lactation, family planning counselling) 2. Baby: ( temperature of baby, weight of baby, , breast feeding, Observe / check for evidence. FAMILY PLANNING 1. Is counselling protocol available and in use? Yes ( ) No ( ) 1 mark Saving Maternity Homes in Ghana Project, Y4 Monitoring and Evaluation Plan 94 2. Availability of stock out of FP logistics for the past 3 months (Check for Family Planning Reports) Yes ( ) No ( ) 2 marks ADVOCACY/HEALTH PROMOTION (8 MARKS) Advocacy/Health Promotion (Check for reports/notebook) 1) Is Health Education plan implemented in the following areas? - HIV/AIDS / STI Yes ( ) No ( ) I mark - Adolescent Health Yes ( ) No ( ) I mark - Regenerative Health & Nutrition Yes ( ) No ( ) I mark - Hypertension Yes ( ) No ( ) I mark - Diabetes Yes ( ) No ( ) I mark - Malaria Yes ( ) No ( ) I mark - Child Health Yes ( ) No ( ) I mark - Reproductive Health Yes ( ) No ( ) I mark Any other comment Saving Maternity Homes in Ghana Project, Y4 Monitoring and Evaluation Plan 95 SCORING No. ITEM EXPECTED SCORE FACILITY SCORE 1 Emergency Response 54 marks 2 Quality Assurance 83 marks 3 Safe Motherhood 25 marks 4 Advocacy/Health Promotion 8 marks Grand Total 170marks GRADING No Score range Grade 1 160 - 170 A+ Excellent 2 145 - 159 A Very good 3 130 - 144 B+ Good 4 115 - 129 B Fairly Good 5 100 - 114 C Fair 6 85 - 99 D Pass 7 Below 85 E Fail DATE: ……………………………………………………………………………………………………….. SUBDISTRICT: ……………………………………………………………………………...….…………… RESPONDENTS:……………………………………………………….……………………..……………... NAME OF ASSESSOR:………………………………………………………...…………...……………..….. TIME STARTED: ……………………….………………. TIME ENDED:……..……….…………………… NAME OF REVIEWER OF ASSESSOR……………...…………………………………….………………… SIGNATURE: ………………………………………………………………………………………………... Saving Maternity Homes in Ghana Project, Y4 Monitoring and Evaluation Plan 96 GHANA REGISTERED MIDWIVES ASSOCIATION (GRMA) 1st Circular Road, H/No. 11B, Cantonments, Accra, Tel: + 233-(0) 302-779348/772971, email￾ghanamidwives@yahoo.com PEER SUPPORT FEEDBACK FORM No. ITEM EXPECTED SCORE FACILITY SCORE 1 Emergency Response 54 marks Strengths Gaps 2 Quality Assurance 83 marks Strengths Gaps 3 Safe Motherhood 25 marks Strengths Gaps 4 Advocacy/Health Promotion 8 marks Strengths Gaps Grand Total 170marks GRADING No Score range Grade 1 160 - 170 A+ Excellent 2 145 - 159 A Very good 3 130 - 144 B+ Good 4 115 - 129 B Fairly Good 5 100 - 114 C Fair 6 85 - 99 D Pass 7 Below 85 E Fail DATE: ……………………………………………………………………………………………………….. SUBDISTRICT: ……………………………………………………………………………………………… RESPONDENTS: ……………………………………………………….…………………………………... NAME OF ASSESSOR: …………………………………………………………………………………..….. TIME STARTED: ……………………….…………. TIME ENDED: ……..……………………………… Saving Maternity Homes in Ghana Project, Y4 Monitoring and Evaluation Plan 97 ANNEX 6: DRAFT PMH EXTENSION SURVEY SMH PROJECT DRAFT PMH EXTENSION SURVEY PART 1: PRE-INTERVIEW DATA The interviewer should complete the following questions before beginning the interview. # Category Answers (and Coding) Instructions 1.1 Interviewer Name 1.2 Date 1 5 Day Month Year 1.3 Region 1.4 District 1.5 Town 1.6 GPS Coordinates Please record the GPS coordinates of the facility. 1.7 Photograph of facility sign Please take a photograph of the facility sign or building. PART 2: INTRODUCTION AND INFORMED CONSENT Immediately following the delivery of the PMH survey, the team will introduction the PMH extension survey as follows:  Description of the purpose of the endline assessment and evaluation  Estimate of the total time required – 30 minutes PART 3: FACILITY INFORMATION The interviewer should read aloud: I’d like to start by learning a little bit about your facility. # Question Answers (and Coding) Instructions 3.1 What is your name? Saving Maternity Homes in Ghana Project, Y4 Monitoring and Evaluation Plan 98 # Question Answers (and Coding) Instructions 3.2 Respondent sex Male……………………………………………..…1 Female………………………………..……..............2 3.3 What is your position or title? If the respondent is not the owner, manager, or another person with significant responsibility, confer with GRMA about the possibility of postponing the interview. PART 4: FINANCIAL SUSTAINABILITY, OPERATIONAL EFFICIENCY AND QUALITY OF CLINICAL SERVICES Read aloud: Now I would like to ask some questions about the ownership and management of the facility. # Question Answers (and Coding) Instructions 4.1 To what extent do you agree with the following statement: “I have a better awareness of the benefits of business continuity and financial sustainability than you did three years ago”? Strongly agree…………………………..............…..1 Agree………………………...…………...................2 Neutral……………………..……………….......….3 Disagree……………………………………...……4 Strongly disagree………………………...………...5 4.2 Have you change the way in which you have managed or conducted your facility’s business and financial practices in the last three years? Yes………………………………………….............1 No…………………………………………..............2 If yes, proceed to question 4.3. If no, skip to question 4.4. 4.3 If so, how? 4.4 Did you or a representative of the facility attend the regional meetings and quality circles? Yes………………………………………….............1 No…………………………………………..............2 If yes, proceed to question 4.3. If no, skip to question 4.4. 4.5 If so, how often? Monthly…………………...…………..............…….1 Quarterly……………….…...…………………..…2 Annually………………………...……….…..............3 Less often…….………………..…………………...4 4.6 Have you in the past three years sought or hired outside council to improve your business and financial practices? Yes………………………………………….............1 No…………………………………………..............2 4.7 Have you in the past three years received training in business continuity and/or financial sustainability Yes………………………………………….............1 No…………………………………………..............2 If yes, proceed to question 4.8. If no, skip to question 4.9. Saving Maternity Homes in Ghana Project, Y4 Monitoring and Evaluation Plan 99 # Question Answers (and Coding) Instructions outside of the SMH project? 4.8 If yes, from where? 4.9 Have you in the past three years applied for a bank loan? Yes………………………………………….............1 No…………………………………………..............2 If yes, proceed to question 4.10. If no, skip to question 4.14. 4.10 If yes, were you successful in obtaining the loan? Yes………………………………………….............1 No…………………………………………..............2 4.11 If yes, when did was the loan approved? 4.12 What was the value of the approved loan? 4.13 What was the purpose of the loan? 4.14 Have you in the past three years invested personal funds into you facility? Yes………………………………………….............1 No…………………………………………..............2 If yes, proceed to question 4.15. If no, skip to question 4.16. 4.15 If so, what was the investment used for? 4.16 Prior to the SMH project, how motivated were you to expand your facility. Strongly motivated….…………………..............…..1 Somewhat motivated………...…………...................2 Neutral……………………..……………….......….3 Not very motivated...………………………...……4 Not motivated…………………………...………...5 4.17 Has the owner/manager of the PMH experienced a decline in their health over the past three years? Yes………………………………………….............1 No…………………………………………..............2 4.18 Is the current owner/manager of the PMH in good health? Yes………………………………………….............1 No…………………………………………..............2 4.19 Has the facility’s owner/manager passed away within the past three years? Yes………………………………………….............1 No…………………………………………..............2 4.20 Has the owner/manager of the PMH retired over the past three years? Yes………………………………………….............1 No…………………………………………..............2 4.21 Does the owner/manager of the PMH plan to retire in the next five years? Yes………………………………………….............1 No…………………………………………..............2 Saving Maternity Homes in Ghana Project, Y4 Monitoring and Evaluation Plan 100 # Question Answers (and Coding) Instructions 5.1 Have you in the last three years brought on a new manager? Yes………………………………………….............1 No…………………………………………..............2 If yes, continue to 5.1.1 If no, skip to 5.2 5.1.1 If yes, can we see the contract/employment agreement outlining the recently completed transformation? Yes………………………………………….............1 No…………………………………………..............2 5.1.2 [Contract sighted] Yes…………………………………………………1 No……………………………………………….....2 5.1.3 Is it signed by both parties? Yes………………………………………….............1 No…………………………………………..............2 5.1.4 When was it signed? Date__________________________________ 5.2 Have you in the last three years hired an additional clinical staff member to diversify the services offered at your facility? Yes………………………………………….............1 No…………………………………………..............2 If yes, continue to 5.2.1 If no, skip to 5.3 5.2.1 If yes, can we see the contract/employment agreement outlining the recently completed transformation? Yes………………………………………….............1 No…………………………………………..............2 5.2.2 [Contract sighted] Yes…………………………………………………1 No……………………………………………….....2 5.2.3 Is it signed by both parties? Yes………………………………………….............1 No…………………………………………..............2 5.2.4 When was it signed? Date__________________________________ 5.3 Have you in the last three years sold or purchased your facility? Yes………………………………………….............1 No…………………………………………..............2 If yes, continue to 5.3.1 If no, skip to 5.4 5.3.1 If yes, can we see the contract outlining the recently completed transformation? Yes………………………………………….............1 No…………………………………………..............2 5.3.2 [Contract sighted] Yes…………………………………………………1 No……………………………………………….....2 5.3.3 Is it signed by both parties? Yes………………………………………….............1 No…………………………………………..............2 5.3.4 When was it signed? Date__________________________________ 5.4 Have you in the last three years leased out or leased your facility? Yes………………………………………….............1 No…………………………………………..............2 If yes, continue to 5.4.1 If no, skip to 5.5 5.4.1 If yes, can we see the contract outlining the recently completed transformation? Yes………………………………………….............1 No…………………………………………..............2 5.4.2 [Contract sighted] Yes…………………………………………………1 No……………………………………………….....2 5.4.3 Is it signed by both parties? Yes………………………………………….............1 No…………………………………………..............2 Saving Maternity Homes in Ghana Project, Y4 Monitoring and Evaluation Plan 101 # Question Answers (and Coding) Instructions 5.4.4 When was it signed? Date__________________________________ 5.5 Have you in the last three years brought on a new partner to share in the ownership of your facility? Yes………………………………………….............1 No…………………………………………..............2 If yes, continue to 5.5.1 If no, skip to 6.1. 5.5.1 If yes, can we see the contract outlining the recently completed transformation? Yes………………………………………….............1 No…………………………………………..............2 5.5.2 [Contract sighted] Yes…………………………………………………1 No……………………………………………….....2 5.5.3 Is it signed by both parties? Yes………………………………………….............1 No…………………………………………..............2 5.5.4 When was it signed? Date__________________________________ 6.1 Have you contacted any financial institutions to learn about available loan product? Yes………………………………………….............1 No…………………………………………..............2 6.2 Have you in the past three years received clinical support or technical assistance outside of the SMH project? Yes………………………………………….............1 No…………………………………………..............2 If yes, proceed to question 5.3. If no, skip to question 5.4. 6.3 If yes, from where? Saving Maternity Homes in Ghana Project, Y4 Monitoring and Evaluation Plan 102 ANNEX 7: QAT ADDENDUM DISABILITY, TRAFFICKING OF PERSONS, GENDER AND DOMESTIC VIOLENCE (10 MARKS) DISABILITY 1. Has anyone in the facility received training to provide care for the needs of persons with disabilities? (observe a demonstration) Yes ( ) No ( ) 1 marks 2. Is the facility accessible to persons with disabilities? (e.g., railings, slope etc. inspect) Yes ( ) No ( ) 1 marks 3. Does the facility keep records of types of disability receiving services? Yes ( ) No ( ) 1 marks GENDER BASED AND DOMESTIC VIOLENCE 4. Has anyone in the facility received training or materials to adequately support victims of Gender based and domestic violence? Yes ( ) No ( ) 1 marks 5. Does the facility have links with other agencies and or institutions dealing with issues of gender based and domestic violence? (contact details) Yes ( ) No ( ) 1 marks 6. Does the facility keep records of advocacy and sensitization community sessions on gender based and domestic violence? (notes/reports) Yes ( ) No ( ) 1 marks 7. Does the facility keep records of gender based and domestic violence cases reported and referrals made? (notes/reports) Yes ( ) No ( ) 1 marks TRAFFICKING IN PERSONS 8. Has anyone in the facility received training or materials to adequately support victims of trafficking? Yes ( ) No ( ) 1 marks 9. Does the facility display/provide/make available resources or referral mechanisms to support victims of trafficking? (e.g. Referrals to domestic violence service providers.) Yes ( ) No ( ) 1 marks 10. Does the facility keep records of advocacy, sensitization and community sessions on countering trafficking of persons? (notes/reports) Yes ( ) No ( ) 1 marks Saving Maternity Homes in Ghana Project, Y4 Monitoring and Evaluation Plan 103 ANNEX 8: DRAFT EVALUATION IN￾DEPTH INTERVIEW GUIDE SMH PROJECT DRAFT EVALUATION IN-DEPTH INTERVIEW GUIDE Discussion Guide – Key Informants Starting the Session Interviewer provides a brief introduction of the SMH project, and explains the objectives of the project as well as the interview / discussion points. Interviewer thanks the respondent for his/her time, and stresses that his/her contribution is valuable and appreciated. Informed Consent. Provide the respondent with the Informed Consent Form, and allow them time to read through it thoroughly. Answer any questions that they ask, and if they do not volunteer any questions, ask them specifically if they have any questions. Once all questions have been answered, and they wish to proceed, ask them to sign the form. If they do not wish to proceed or do not which to sign the form, assure them that there will be no adverse repercussions for not participating, thank them for their time, and depart. Note: Respondents will be limited to persons able to give personal consent. Subjects not able to give personal consent (e.g. for cultural reasons, children or adolescents less than the legal age of consent, subjects with mental illness) should not be subjects of this study, and will not be interviewed. If the respondent is incapable of reading and signing the written consent form, the enumerators will read the form to them in order to acquire informed consent. Inform the respondent that this survey usually takes 60 minutes to complete. Will you participate in this interview? Respondent information Name (first name and surname) Facility Title Date of interview Telephone number Permission to contact with follow-up questions? Assessment questions Response Ownership/ Management Transformations 1 To what extent do you agree with the following statement: “I have been able to meet the demands and needs of my existing clients over the past three years when I take into consideration staffing, facility space and services provided”? 2 Have you seen an increase in client demand over the past three years? Financial sustainability, operational efficiency and quality of clinical services Saving Maternity Homes in Ghana Project, Y4 Monitoring and Evaluation Plan 104 3 If you hired a new manager or staff member with a diverse skillset, did the facility qualify for an accreditation since the hiring of the new staff member which is was not previously qualified for? If so, which one? Were you successful in achieving this certification or accreditation? 4 If you hired a new manager or staff member with a diverse skillset, does the new staff members’ responsibilities and expertise relate to business management and financial practices? 5 Are you aware of when capitation is scheduled to come to your region? 6 Do you believe capitation will arrive? 7 Have you in the last three years participated in continuing medical education training and events to improve the quality of care provided at your facility? Expansion of Clinical services 8 If you hired a new manager or staff member with a diverse skillset, does the new staff members’ responsibilities include expanding the PMHs available clinical services? 9 If you hired a new manager or staff member with a diverse skillset, does the new staff member have experience or qualifications which were not previously covered by the existing staff at your facility? Proportional Piling Activity At this point in the interview, the interviewer will introduce the proportional piling activity, a participatory research method, used to determine the relative attribution of the project’s interventions as compared to the competing explanations for the outcomes observed. Proportional piling allowed respondents from transformed facilities to rank the causal factors that were present, determining the levels of attribution across the SMH project interventions and the competing explanations. The interviewer will present each competing explanation for the observed outcomes of interest, and provided a set of 100 stones to the interviewee. Interviewees will then be prompted to allocate the relative number of stones per causal factor to identify the proportional significance of that causal factor in contributing to the observed outcome of interest. Following the allocation of the stones, the interviewer will record the number of stones allocated to each causal factor and ask some probing Saving Maternity Homes in Ghana Project, Y4 Monitoring and Evaluation Plan 105 questions to further understand why the facility owner ranked the causal factors in the order that they chose. 10 Please attribute the relative number of stones, demonstrating significance, across the various causal factors present that have contributed to the increased awareness of the benefits of business continuity and financial sustainability within your facility. 11 Please attribute the relative number of stones, demonstrating significance, across the various causal factors present that have contributed to the successful completion of your facility’s ownership/management transformations. 12 Please attribute the relative number of stones, demonstrating significance, across the various causal factors present that have contributed to the improvements in financial sustainability, operational efficiency and quality of clinical services of your facility. 13 Please attribute the relative number of stones, demonstrating significance, across the various causal factors present that have supported the expansion of your facility.