CSHGP COOPERATIVE AGREEMENT NUMBER: AID-OAA-A-12-00093 Samreen Baloch, a Community Midwife from Quetta District, provides antenatal care to a client. FINAL EVALUATION OF THE SAVING MOTHERS AND NEWBORNS IN COMMUNITIES PROJECT A FOCUSED STRATEGIC ASSESSMENT OF A COMMUNITY MIDWIVES PROGRAM IN THREE DISTRICTS OF BALOCHISTAN PROVINCE, PAKISTAN November 2016 This publication was produced at the request of the United States Agency for International Development. It was prepared independently by Kathy Tilford, External Consultant, with assistance from Dr. Sohail Amjad, National Consultant. ACKNOWLEDGEMENTS First of all, my sincere thanks to the many individuals in Quetta, Gwadar and Kech who participated in the focus group discussions and the key informant interviews – CMWs, DoH personnel, beneficiaries, members of the policy forums, SMNC project staff and Mercy Corps senior managers. Your perspectives and suggestions enriched the content of this report and helped to validate findings from the stakeholders’ point of view. Ahmed Ullah, SMNC Program Manager, was tireless in responding to my many requests for additional documents, more information and clarifications and I am very grateful to him. He and his colleagues, including Dr. Saeedullah Khan (Team Leader South), Dr. Shaihak Riaz and Ghulam Haider (Security Officer), helped the field team organize their program, made logistical arrangements for them and ensured that the team was able to conduct the qualitative research in a safe environment. I sincerely appreciate the time and effort that Ahmed Ullah, Dr. Farah Naureen and Jennifer Norman devoted to reviewing the first draft of this document. Their observations and suggestions were very helpful and made for a more accurate report. Dr. Saima Hamid, the consultant who prepared the Learning Agenda report, kindly answered my queries about the studies described in the report. My thanks to her. My deepest appreciation goes to Dr. Sohail Amjad, the national consultant for the Focused Strategic Assessment, who worked with me from beginning to end. His in-depth knowledge of the Pakistani health system and the Community Midwives program were invaluable in helping me understand the context of the project. In addition to contributing to the methodology and the development of the qualitative research instruments, he led the team that conducted the field work, ensured that transcripts were well-prepared and provided comments and input for the report. Thanks also to the two senior research associates, Dr. Muslim Abbas and Ms. Saima Zeb Faredi, who conducted the qualitative research in the field with Dr. Amjad and prepared detailed, high-quality transcripts with pertinent observations and reflections. Sincerely, Kathy Tilford External Consultant FINAL EVALUATION OF THE SAVING MOTHERS AND NEWBORNS IN COMMUNITIES PROJECT: TESTING INTERVENTIONS TO STRENGTHEN A PRIVATE-SECTOR COMMUNITY MIDWIVES PROGRAM TO IMPROVE MATERNAL AND NEWBORN HEALTH STATUS IN UNDERSERVED AREAS OF BALOCHISTAN PROVINCE, PAKISTAN DISCLAIMER: The author’s views expressed in this publication do not necessarily reflect the views of the United States Agency for International Development or the United States Government. CONTENTS Acronyms......................................................................................................................................................... i Executive Summary .......................................................................................................................................... 1 Purpose of the Focused Strategic Assessment....................................................................................................... 4 Questions for the Focused Strategic Assessment ............................................................................................ 4 Project Background........................................................................................................................................... 5 Situation of Maternal and Neonatal Mortality in Balochistan ............................................................................ 5 Project Summary........................................................................................................................................ 5 Partnerships and Collaboration .................................................................................................................... 7 The Learning Agenda ................................................................................................................................. 7 Assessment Methodology and Limitations ........................................................................................................... 8 Principal Findings...........................................................................................................................................11 Progress Toward Project Goal and Objectives .............................................................................................11 Status of the Midcourse Corrections...........................................................................................................19 Key Strategies and Factors Affecting Results................................................................................................19 Sustainability and Potential for Scaling Up...................................................................................................22 Conclusions and Recommendations..................................................................................................................23 Recommendations....................................................................................................................................24 Annexes Annex I. Focused Strategic Assessment Terms of Reference Annex II. Assessment Methodology and Limitations Annex III. Data Collection Instruments and Work Plan Annex IV. Sources of Information Annex V. Disclosure of Any Conflicts of Interest Annex VI. Assessment Team Members, Roles and Titles Annex VII. Stakeholder Debrief Powerpoint Presentation Annex VIII. Learning Agenda Final Report Annex IX. Project Data Form October 2016 i ACRONYMS Acronym Definition ANC Antenatal Care CMW Community Midwife CSHGP Child Survival and Health Grants Program DG Director General DHF District Health Forum DHO District Health Officer DoH Department of Health FGD Focus Group Discussion FSA Focused Strategic Assessment GoB Government of Balochistan IEC Information, Education and Communication IR Intermediate Result KII Key Informant Interview LHS Lady Health Supervisor LHV Lady Health Visitor LHW Lady Health Worker LQAS Lot Quality Assurance Sampling MCH Maternal and Child Health Program MCSP Maternal and Child Survival Program MIS Management Information System MNCH Maternal, Neonatal and Child Health MoU Memorandum of Understanding PHS Public Health Specialist October 2016 ii PKR Pakistani Rupees PNC Pakistan Nursing Council PSC Provincial Steering Committee SMNC Saving Mothers and Newborns in Communities SMS Texting TBA Traditional Birth Attendant ToR Terms of Reference TWG Technical Working Group USAID United States Agency for International Development WSG Women Support Group October 2016 1 EXECUTIVE SUMMARY Purpose of the Focused Strategic Assessment and Questions This Focused Strategic Assessment (FSA), conducted in September-October 2016, serves as the final evaluation of the Saving Mothers and Newborns in Communities (SMNC) project implemented in Balochistan Province, Pakistan. The FSA provided an opportunity to take stock of accomplishments to date and to listen to the stakeholders at all levels. The FSA report will be used by the following audiences as a source of evidence to help inform decisions about future program designs and policies: in-country partners at the national, provincial and local levels; USAID (Child Survival and Health Grants Program [CSHGP], Global Health Bureau, USAID Missions), the Maternal and Child Survival Program (MCSP) and other CSHGP grantees; and the international global health community. The five overarching questions addressed in the FSA are: 1. To what extent did the project accomplish and/or contribute to the results (goals/objectives) stated in the Strategic Work Plan? 2. What were the key strategies and factors, including management issues and policy environment, that contributed to what worked or did not work? 3. Which elements of the project have been or are likely to be sustained or expanded? 4. What are stakeholder perspectives on the overall project implementation, the policy forums, and the Learning Agenda implementation? 5. Working around strengthening community-based maternal and newborn healthcare provision, to what extent has the project been successful? In addition to answering these five questions, the FSA also focused on four Learning Agenda themes, which were selected jointly with the Balochistan Department of Health (DoH) to provide information needed to improve Community Midwives (CMW) policies and programs. These themes were improving the selection process to more effectively recruit and deploy CMWs; promoting financial self-sustainability for CMWs; ensuring that CMWs provide quality care; and streamlining reporting using mobile phones. Project Background Compared to the other three provinces, Balochistan has the worst maternal, neonatal and child health indicators in the country according to recent data provided by the Balochistan DoH: a maternal mortality rate of 785 per 100,000 live births; a neonatal mortality rate of 63 per 1,000; and a skilled birth attendance rate of only 17.8 percent. Eighty-three percent of women deliver at home and 55.7 percent receive no antenatal care (ANC). To address Balochistan’s sustained high rates of maternal and neonatal mortality and to ensure skilled birth attendance, the Government of Balochistan (GoB) and the Balochistan provincial DoH have given top priority to training a cadre of private-sector CMWs to provide much-needed maternal and neonatal services in underserved areas. The DoH requested Mercy Corps’ assistance to address some of the underlying issues in the CMW program; the result was a joint project with a dual purpose: 1) to demonstrate a health impact within three target districts (Quetta, Gwadar and Kech) and 2) to test interventions and provide key lessons October 2016 2 for developing an improved CMW program model that the DoH could replicate across the province. Working closely with the DoH, Mercy Corps is implementing a maternal, newborn and child health (MNCH) project in Quetta, Gwadar, and Kech Districts of Balochistan Province, Pakistan with support from USAID CSHGP (September 2012-September 2016) and the Scottish Government (through March 2017). The Saving Mothers and Newborns in Communities (SMNC) project seeks to improve maternal and newborn health status, especially for poor and marginalized women of Balochistan (Goal), through increased use of quality essential maternal and newborn care provided by private-sector community midwives (Strategic Objective). The project’s Intermediate Results are: 1. Increased availability of quality maternal and newborn care in communities 2. Improved knowledge and demand for essential maternal and newborn care 3. Improved access to emergency transport in remote communities 4. Improved policy environment for improved maternal, newborn and child health care based on evidence from the Operations Research (Operations Research later replaced with Learning Agenda activity) SMNC is an innovative model designed to reach 382,515 beneficiaries; it has been tested with 95 CMWs and includes the following main components: clinical refresher training; provision of standard equipment and business skills training; mHealth, especially the use of mobile phones for tracking data; reinvigorating Women Support Groups (WSGs) for behavior change; improving the emergency transportation systems; policy initiatives including two provincial level forums and the development of a five-year Balochistan MNCH Strategy; a voucher scheme to enable the poorest women to access CMW services; and strengthening the referral mechanism between CMWs and other health facilities. Assessment Methodology and Limitations Mercy Corps hired an external consultant to lead the FSA remotely from the U.S. She worked closely with a well-qualified national consultant hired by Mercy Corps Pakistan who served as the field team leader, working with two assistants experienced in qualitative data collection. The methodology consisted of a participatory mixed-methods approach that included two principal components: a comprehensive desk review of secondary quantitative and qualitative data sources and the collection of qualitative data. Additional information was acquired through a series of Skype calls and e-mail exchanges with project staff. The main limitation was that security concerns meant that the evaluation field team could not access remote sites. Project Findings The SMNC project convincingly demonstrated that with appropriate selection, training and continued support, the CMW can acquire the necessary skills, confidence and community status to be a lifesaving provider of MNCH services, especially in rural areas with widely￾scattered populations where few if any other health services exist. The project achieved its main objectives: it increased access to quality maternal and neonatal health services for families in underserved areas and provided tested CMW program interventions for replication throughout Balochistan. In doing so, it left lasting achievements such as: • 95 midwives trained and equipped, the majority of whom continue to provide essential services to women with few other options for skilled birth attendance • A clinical skills refresher training course endorsed by the Pakistan Nursing Council (PNC) October 2016 3 • A business skills course to be included in the 18-month midwifery training • Increased demand for MNCH services at the community level • A three-module set of materials for WSGs, available in four languages • Two institutionalized provincial policy platforms, the Technical Working Group (TWG) and the Provincial Steering Committee (PSC) • A new five-year MNCH strategy for Balochistan with a major component for CMWs • An mHealth application for improved data collection and reporting That the project has been able to achieve this in only four years is a testament to the DoH’s commitment, the strong partnership between Mercy Corps and the Government of Balochistan and a sound project design that addressed high priorities for the stakeholders. While the successes are evident, challenges remain, including: identifying enough qualified and interested women from rural areas who meet the minimum requirements to become a CMW; ensuring that the CMWs have sufficient monetary incentives to continue providing services; providing adequate support through regular supportive supervision; improving linkages between the MNCH program and the CMW program; strengthening the referral mechanism between CMWs and secondary facilities; and overcoming GoB budgetary constraints. Among the principal recommendations from the FSA are the following: 1. Prepare a briefing paper that summarizes the main findings and recommendations from the Learning Agenda exercise; ensure wide-spread dissemination and follow-up. 2. Commit resources to continue supportive monitoring and supervision of CMWs and invest in regular refresher training on technical themes and business skills for CMWs. 3. Continue with the plan to integrate the mHealth application into the MNCH MIS. 4. Continue the policy forums (Provincial Steering Committee and Technical Working Group), especially for overseeing the implementation of the Balochistan MNCH Strategy. 5. Mobilize resources and partners for implementing the Balochistan MNCH Strategy. 6. Create a more collaborative working relationship between CMWs and Lady Health Workers (LHWs). 7. Make quality staffing of midwifery schools a priority. The Saving Mothers and Newborns in Communities project (SMNC) in Balochistan Province, Pakistan is supported by the American people through the United States Agency for International Development (USAID) through its Child Survival and Health Grants Program. The SMNC project is managed by Mercy Corps under Cooperative Agreement No. AID-OAA-A-12-00093. The views expressed in this material do not necessarily reflect the views of USAID or the United States Government. October 2016 4 PURPOSE OF THE FOCUSED STRATEGIC ASSESSMENT This Focused Strategic Assessment serves as the final evaluation of the Saving Mothers and Newborns in Communities (SMNC) project. USAID approved Mercy Corps’ choice of an external evaluator who was hired with project funds; the external evaluator is an independent consultant and had no previous connection with the project. USAID/CSHGP also reviewed and approved the draft Terms of Reference (ToR). The ToR and proposed report outline were modified from the standard guidelines to allow a more in-depth focus on key issues designed to assist the Balochistan Department of Health (DoH) to improve and expand its Community Midwives program. In addition to providing a review of the overall project, the purpose of the FSA is to focus on four Learning Agenda themes, designed jointly with the Balochistan DoH, to provide them with the information they need to refine Community Midwives policies and programs: 1. How can the Balochistan Department of Health improve its selection process to effectively recruit and deploy CMWs in underserved areas? 2. How can CMWs become financially self-sustaining while serving the needs of the poorest of the poor? 3. Do CMWs offer quality care? How? 4. How can the Balochistan DoH streamline CMW reporting using mobile phone technology and expand mHealth in the province? The FSA provides an opportunity for all project stakeholders to take stock of accomplishments to date and to listen to the beneficiaries at all levels, including mothers and caregivers, community members and opinion leaders, health workers, health system administrators, local partners, other organizations, and donors. The FSA report will be used by the following audiences as a source of evidence to help inform decisions about future program designs and policies: • In-country partners at the national, provincial and local levels (e.g., DoH and other relevant ministries, provincial and district health teams, local organizations, communities in project areas). • USAID (CSHGP, Global Health Bureau, USAID Missions), the Maternal and Child Survival Program (MCSP) and other CSHGP grantees. • The international global health community. The FSA report will be posted for public use at http://www.mchipngo.net and the USAID Development Experience Clearinghouse at https://dec.usaid.gov. QUESTIONS FOR THE FOCUSED STRATEGIC ASSESSMENT In addition to the focus on the four Learning Agenda themes, the ToR outlined five overarching questions to be addressed in the FSA: 1. To what extent did the project accomplish and/or contribute to the results (goals/objectives) stated in the Strategic Work Plan, keeping in view the revisions in the midcourse correction document? 2. What were the key strategies and factors, including management issues and policy environment, that contributed to what worked or did not work? October 2016 5 3. Which elements of the project have been or are likely to be sustained or expanded (e.g., through institutionalization or policies)? 4. What are stakeholder perspectives on the overall project implementation, the policy forums, and the Learning Agenda implementation, and how could the Learning Agenda affect capacity, practices, and policy? 5. Working around strengthening community-based maternal and newborn healthcare provision, to what extent has the project been successful? For the more detailed list of questions, please see the ToR in Annex I. PROJECT BACKGROUND SITUATION OF MATERNAL AND NEONATAL MORTALITY IN BALOCHISTAN Compared to the other three provinces, Balochistan has the worst maternal, neonatal and child health indicators in the country as the following table (prepared by the Balochistan Department of Health for a recent donor conference) shows: Indicator Balochistan Khyber Pakhtunkhwa Punjab Sindh Maternal mortality ratio (per 100,000 live births)* 785 275 227 314 Neonatal mortality (per 1,000) 63 41 63 54 Infant mortality (per 1,000) 97 58 88 74 Under five mortality (per 1,000) 111 70 105 93 Proportion of home deliveries 83.1 59.3 51.3 41.4 No ANC 55.7 37.5 19.5 20.9 Skilled birth attendance 17.8 48.3 52.5 60.5 Contraceptive prevalence rate 19.5 28.1 40.7 29.5 Fertility rate 4.2 3.9 3.8 3.9 All basic vaccinations 16.4 52.7 65.6 29.1 *Maternal mortality ratio is taken from Demographic and Health Survey (DHS) 2006-07. Remaining data is from DHS 2012-13. A number of factors contribute to the poor health outcomes for women and children in Balochistan. For pregnant women and newborns, a major constraint is the lack of skilled birth attendants at the community level. As the table shows, less than one in five women in Balochistan used a skilled birth attendant at her last delivery. Accessing care outside the community presents its own set of obstacles: a poor transportation infrastructure, long distances to a health facility, cost and societal norms that restrict women’s mobility. A pregnant woman may need a male relative escort to go to a distant health facility, increasing costs and impinging on the time of the relative, especially if the woman needs to stay for several days to receive care. PROJECT SUMMARY To address Balochistan’s sustained high rates of maternal and neonatal mortality and to ensure skilled birth attendance, the Government of Balochistan and the Balochistan provincial DoH have given top priority to training a cadre of private-sector Community Midwives to provide much- October 2016 6 needed maternal and neonatal services in underserved areas. However, training alone has not been sufficient to improve rates of skilled birth attendance and overall health indicators as most of the CMWs have not been able to establish their clinics and attract clients. The DoH requested Mercy Corps’ assistance to address some of the underlying issues in the CMW program and the result was a joint project with two purposes: 1) to demonstrate a health impact within three target districts (Quetta, Gwadar and Kech) and 2) to test interventions and provide key lessons for developing an improved CMW program model that the DoH could replicate across the province. As a result, Mercy Corps is implementing a maternal, newborn and child health (MNCH) project in Balochistan, Pakistan with support from USAID CSHGP (September 2012-October 2016) and the Scottish Government. The Scottish Government funding also covers a six-month extension phase from October 2016 through March 2017. The Saving Mothers and Newborns in Communities (SMNC) project seeks to improve maternal and newborn health status, especially for poor and marginalized women of Balochistan (Goal), through increased use of quality essential maternal and newborn care provided by private-sector community midwives (Strategic Objective). The project’s Intermediate Results are: 1. Increased availability of quality maternal and newborn care in communities 2. Improved knowledge and demand for essential maternal and newborn care 3. Improved access to emergency transport in remote communities 4. Improved policy environment for improved maternal, newborn and child health care based on evidence from the Operations Research (Operations Research later replaced by the Learning Agenda activity as a result of midcourse corrections) SMNC is an innovative model designed to enable CMWs to become self-sustaining, private MNCH service providers. The model, which has been tested with 95 CMWs, contains the following main components: 1. To ensure quality, Mercy Corps has provided CMWs with clinical refresher training, supported CMW registration with the Pakistan Nursing Council (PNC) for those who were not already registered and conducted joint supervision visits with the DoH. 2. To enable CMWs to set up home-based clinics, Mercy Corps has provided standard equipment and business skills training for the CMWs. 3. Through Mercy Corps’ partnership with PakVista Technologies, CMWs have been using their mobile phones to track patient data, send automatic reminders to clients and offer mass SMS (texting) for awareness raising. Through automatic data transfer, the DoH is now able to track uptake of CMWs’ services in real time. (Note that the automatic reminders and mass SMS activities were discontinued in April 2016 based on midcourse correction recommendations.) 4. For behavior change and demand creation, Mercy Corps reinvigorated the Women Support Groups conducted by CMWs and Lady Health Workers. These groups also generate support to facilitate access to emergency transport. 5. For timely referrals, Women Support Groups and CMWs have been linked with not-for￾profit ambulance services. 6. At the policy level, Mercy Corps has assisted the provincial DoH in developing a five-year strategic MNCH plan and in establishing two policy forums, the Technical Working Group October 2016 7 (TWG) and the Provincial Steering Committee (PSC). 7. A voucher scheme has been introduced to support the DoH in operationalizing plans that address the needs of poorest women to access maternal and newborn health services. 8. Although it is not yet fully functional, Mercy Corps has also prepared and oriented DoH staff on a referral mechanism between CMWs and secondary health facilities. Over the life of the project, an estimated 382,515 beneficiaries will be reached: Beneficiaries (National MNCH Program and provincial DoH estimates) Total Total Population 2,689,838 Total Neonates 11,093 Infants aged 0–11 Months 13,388 Children aged <5 Years 65,028 Women of Reproductive Age (15–49 years) 84,153 Total Beneficiaries 382,515 Expected Pregnancies 13,006 Community Health Workers or Volunteers (CHWs), Disaggregated by Sex 95 CMWs, 272 LHWs/Community Educators (Female) Community-based Structures 272 Women Support Groups PARTNERSHIPS AND COLLABORATION The principal partners are Mercy Corps and the Government of Balochistan, represented through the DoH with which Mercy Corps has a long and productive working relationship. Within the DoH key relations have been established with provincial-level senior managers and government administrators; at the district level, the District Health Officers (DHOs) and his/her team are the principal partners. At the community level, the project works closely with the private-sector CMWs, the Women Support Groups, Community Educators and community leaders. The main private-sector parties participating in the project are PakVista Technologies, which provides technical expertise in mHealth, and two private not-for-profit ambulance companies with which Mercy Corps has signed Memorandums of Understanding (MoUs). THE LEARNING AGENDA Following discussions with USAID and the DoH, Mercy Corps proposed in 2015 to replace the project’s Operation Research activity with a revised Learning Agenda as part of the overall midcourse correction plan. It was anticipated that the Learning Agenda would enable Mercy Corps and the DoH to measure the health impact of the project and to provide the DoH with the information they need to improve their CMW policies and programs, contributing to Intermediate Result (IR) 4: Improved policy environment for improved maternal, newborn and child healthcare based on evidence from the Operations Research. Mercy Corps and the DoH discussed the DoH’s priorities, which were to determine i) how to improve selection and deployment of CMWs in a way that they reach underserved populations and ii) how to keep CMWs engaged with the program. As a result, Mercy Corps and the DoH jointly agreed on the four key Learning Agenda topics listed on page 4. October 2016 8 The four sub-studies (one for each question) contributing to the Learning Agenda started in February 2016 and were completed by the time the FSA began. Each study had a unique methodology combining quantitative and qualitative instruments. The Learning Agenda consultant prepared a comprehensive report which describes the methodology, presents the results and proposes recommendations. (See Annex IX.) This report was one of the principal secondary data sources for the FSA. ASSESSMENT METHODOLOGY AND LIMITATIONS Mercy Corps hired an external consultant, Kathy Tilford, to lead the Focused Strategic Assessment remotely from the U.S. She worked closely with a well-qualified national consultant, Dr. Sohail Amjad, hired by Mercy Corps/Pakistan. The national consultant had extensive experience in evaluation, including CSHGP projects, and in-depth knowledge of the Pakistani health system, the Community Midwives program and the local context. He served as the field team leader, working with two assistants experienced in qualitative data collection: Dr. Muslim Abbas and Ms. Saima Zeb Faredi. The methodology consisted of a participatory mixed-methods approach that included two principal components: 1. Comprehensive desk review of secondary data sources: Prior to developing the instruments for the field work, the external consultant conducted an extensive desk review of quantitative and qualitative data sources. These included project documents such as the proposal, the Strategic Work Plan, surveys and assessments, routine monitoring updates and the Case for Midcourse Corrections; background documents on maternal and child health and the Pakistan health system; the Learning Agenda report; and DoH publications such as the Balochistan MNCH Strategy for 2016-2020. (See Annex IV for a complete list of documents consulted.) 2. Collection of qualitative data: Using semi-structured Key Informant Interviews (KIIs), Focus Group Discussions (FGDs) and informal discussions with project stakeholders, the external consultant and the three-person field team carried out qualitative data collection over a three-week period in September 2016. (See Annex III for the data collection instruments and work plan.) Selection of data collection methods: In addition to KIIs and FGDs, the team considered other data collection methods including observations and exit interviews. However, it would have been difficult to organize observations of CMWs and exit interviews with clients for several reasons: • a field team member would need to be present at the exact time a client came for a consultation; • for cultural reasons only a female team member would be able to conduct these data collection activities; • time constraints would mean a limited number of observations and/or exit interviews; and • the data collection carried out for the Learning Agenda already included a large number of observations of CMWs by trained health care providers. Choice of groups to interview: In deciding which groups to include for the FGDs, the team discussed the possibility of a FGD with male community leaders. However, the Program Manager October 2016 9 suggested that this might not be as useful since men had not been very involved at the community level and would likely not have a lot to share. Standard themes: To facilitate the triangulation of data, the team selected a number of common themes to include in all data collection instruments: support for CMWs, including monitoring and supervision; role of the Women Support Groups; potential for sustainability and replicability; status of the referral system and emergency transportation; contribution of the project to improving MNCH; and challenges encountered. Qualitative Data Collection Field work: The national consultant conducted qualitative work in the field with his two assistants over ten days (September 19-28, 2016). In each of the three districts (Kech, Quetta and Gwadar) they executed three FGDs, with each group having eight to ten participants: 1. Community Midwives: To identify participants for this FGD, the field team leader selected every third name on the list of project-supported CMWs and asked the Program Manager to contact 11-12 women in each district in order to have 7-9 participants. The team leader attempted to keep a balance between urban and rural CMWs but ultimately CMWs closest to the district capital were invited so that the women could return home the same day. 2. Lady Health Workers, Lady Health Visitors (LHVs) and Lady Health Supervisors (LHSs): Participants from the same geographic areas as the CMW participants and who had remained active in monitoring and coordination comprised this FGD group. 3. Female community members: Participants for this FGD included Lead Mothers from the Women Support Groups and women who had accessed services from the CMWs who were in the CMW Focus Group Discussion. Using semi-structured interview guides, the field team also conducted in-depth KIIs with four DoH stakeholders in each district: • the District Health Officer • the Medical Superintendent of the District Headquarters Hospital • the Public Health Specialist (PHS) for the Maternal, Newborn and Child Health (MNCH) program who oversees the MNCH program at the district level • the District Coordinator who oversees the LHW program at the district level At the provincial level, the national consultant held KIIs with the following stakeholders: the Director General Health Services for Balochistan Province; the Provincial MNCH Coordinator; the Provincial LHW Program Coordinator; the Chairperson of the Technical Working Group (also a member of the Provincial Steering Committee); and two other stakeholders, one each from the Technical Working Group and the Provincial Steering Committee. Interviews with Mercy Corps staff: The external consultant collected qualitative data through Skype interviews with project staff (Program Manager, Project Officer, Security Officer and the Monitoring, Evaluation and Learning Manager) and with two Mercy Corps Pakistan senior managers, the Team Leader/South who has been very involved in the project since the beginning and the Senior Director of Programs. It was not possible for her to conduct Skype interviews with other stakeholders due to connectivity issues outside Quetta and Islamabad. Limitations October 2016 10 Since the project had recently completed several qualitative and quantitative community-level surveys in the target area, the team ensured that engagement with stakeholders focused primarily on information gaps. The field team could not visit remote areas for FGDs due to security issues and therefore CMWs and the LHWs/LHVs/LHSs were invited to the project’s district field offices. The FGDs with female community members were held in homes of Lead Mothers. In Kech the security situation meant that male members of the team could not travel there; only the female team member conducted interviews in this district. However, this did not appear to affect the quality of data collected. Ethical Considerations The field team made it clear to all FGD participants that they were under no obligation to participate but if they did participate, anonymity and confidentiality were assured. Verbal informed consent from the participants was obtained. Where necessary, an interpreter assisted the team members. For each encounter, the team obtained permission for taking photographs for reports and presentations. Quality of Evidence for Results The SMNC project was well-documented and during the FSA the project staff provided a number of additional documents requested by the external consultant. The three Annual Reports prepared per USAID requirements were comprehensive with extensive annexes. Each of the major interventions such as mHealth had accompanying explanations and periodic assessments that provided a chronological description of how the intervention evolved. The assessment team compared findings from the qualitative research with project documents, external assessments from firms such as PakVista Technologies, the Learning Agenda report produced by an independent consultant and documents produced during the project’s lifetime such as the Balochistan MNCH Strategy 2016-20. The external consultant also checked facts with SMNC project staff when preparing this report. The cross-checking of data and the triangulation of findings both ensured that the evidence used for drawing conclusions was valid and reliable. However, it should be noted that for information presented in project documents such as the quantitative data in the three Annual Reports and routine monitoring documents, the external consultant relied on the accuracy of the information at the source as there was no way to independently verify the information. Data Analysis The information collected from key informants was compiled and tabulated using MS Office software for each question and inputs were organized by themes and dimensions of program intervention. Important quotes and observations were identified and used to build the analysis. Data emerging from interviews was validated internally through triangulation with information from project documents, routine monitoring, communications with project staff and other sources gathered prior to and during the field work. The interpretations of triangulated thematic data were discussed with Mercy Corps district and country office teams for further modification and amendment. Information was synthesized by creating matrices around identified themes and the findings organized accordingly. For additional details on the methodology, see Annexes II, III, IV and VII. October 2016 11 PRINCIPAL FINDINGS This section of the report presents the principal findings for the questions posed in the ToR. The findings are organized under four main themes: Progress toward Project Goal and Objectives; Status of Midcourse Corrections; Key Strategies and Factors Affecting Results; and Sustainability and Potential for Scaling Up. Under each of the four themes additional findings are presented for the sub-questions from the ToR. The analysis is based primarily on four sources: the qualitative research (the FGDs and KIIs carried out during the FSA period); SMNC project reports; the Learning Agenda report; and other key documents such as the Balochistan MNCH Strategy 2016-20 and the mHealth assessment reports from PakVista Technologies. PROGRESS TOWARD PROJECT GOAL AND OBJECTIVES The project has largely achieved three of the four Intermediate Results (IRs) and made significant progress toward reaching the overall goal of “improving maternal and newborn health status, especially for poor and marginalized women”. The following discussion describes progress made for the components within each IR. IR1. Increased availability of quality maternal and newborn care in communities This IR included the largest number of components and activities: recruitment and deployment of CMWs; provision of standard equipment and business skills training to promote financial self￾sustainability of CMWs; development of a clinical skills refresher course; mHealth for improved reporting and tracking of client data; supportive monitoring and supervision; and the introduction of a voucher scheme to enable the poorest women to access CMW services. By September 2016 the project had achieved its major targets for all the IR1 components except for the voucher scheme, which was introduced only at the beginning of Year 4 as part of the mid-course corrections. Improving recruitment and deployment of CMWs: The project has helped the DoH improve its selection, deployment and retention of CMWs in two ways: through actions undertaken during project implementation and through the Learning Agenda exercise. Evidence gathered through the project’s baseline survey, the LQAS survey (Lot Quality Assurance Sampling) and the CMW mapping exercise conducted in April-May 2015 convincingly demonstrated that there is a shortage of CMWs practicing in rural communities, especially in the more remote, underserved areas. This evidence helped Mercy Corps and the DoH recognize that both the selection and the deployment processes needed to be improved if underserved areas were to benefit from the CMW program. As a result, the DoH is working closely with Mercy Corps to improve both processes. As a first step, the MNCH Program has decided to use mapping exercises and assessments in all districts to determine where CMWs are needed most so that underserved areas are a priority for selection of new CMWs. A second important step is the revision of the selection committees: they will now be smaller and include representatives from the underserved communities to reduce nepotism and to streamline the process. A third step for increasing the number of CMWs practicing in rural areas will be to explore how to overcome the fact that low literacy rates for women in rural areas results in fewer CMW candidates with the requisite education level. One suggestion is to institute a continuing education program so that young women who have completed some schooling To what extent did the project accomplish and/or contribute to the results (goals/objectives) stated in the Strategic Work Plan, keeping in view the revisions in the mid-course correction document? October 2016 12 can obtain the required 10 years of schooling. While such an initiative would require an inter-sectoral approach, additional resources and time to develop, it is encouraging that the DoH is interested in exploring such options. Another important contribution of the project was to work with the DoH to improve the deployment process so that the CMWs could begin to serve their communities more quickly. Examples of actions that accelerated the deployment process include: • Mercy Corps worked with the DoH to facilitate the PNC midwifery registration process for the CMWs, shortening the time considerably so that they could begin providing services sooner. • The DoH allowed the CMWs to practice while waiting for their registration approval. • At the request of the TWG, the PSC recommended that the CMWs be allowed to provide services in their homes (work stations) rather than always going to a client’s home. This recommendation was approved and the establishment of work stations was a positive innovation mentioned often during the qualitative research, especially by provincial and district health administrators. In addition to these project-supported actions for accelerating the provision of quality and accessible CMW services, the Learning Agenda included a sub-study on the question: How can the DoH improve its selection process to effectively recruit and deploy CMWs in underserved areas? This sub-study resulted in a solid analysis of some of the reasons for the lack of CMWs in underserved areas; a good discussion of a range of incentive packages for retaining CMWs; and concrete recommendations for better selection, deployment and retention processes for CMWs in rural areas. The discussion of different incentive packages and the most important recommendations for increasing the number of qualified CMWs providing services in underserved areas are summarized below: 1. Establish an accelerated education program for women and girls to increase the available pool of CMW candidates. 2. Improve family (and community) support for CMW candidates through better orientation. 3. Ensure community involvement in the selection process. 4. Institute an inter-departmental coordination team to include the LHW and the MNCH programs as well as development partners. This team would provide oversight for improved recruitment, training and monitoring of CMWs. 5. Introduce an accelerated process for midwifery registration with the PNC. 6. Determine which incentives are feasible for CMW retention, including incentives for DoH personnel conducting supportive monitoring and supervision. 7. Ensure that the federal government transfers the entire allocated budget for the MNCH program to the provincial program in a timely manner. Promoting financial self-sustainability for CMWs: Finding ways to help the CMWs become financially self-sustaining as private health care providers was an important IR1 component; this included providing standard equipment to the CMWs and developing a business training course for them. Both initiatives were very much appreciated by the CMWs and the business training course was a project activity recommended for replication by a number of respondents who participated in the qualitative research. October 2016 13 The FSA highlighted some of the constraints that hinder the CMWs’ financial sustainability: • Competition, even in rural areas, from other providers such as traditional birth attendants (TBAs) • People’s inability to pay coupled with the perception that the CMW is a salaried employee and should not charge fees • Sometimes there is a relatively low demand due to the size of the catchment area • Problematic transportation, limiting access to clients • Lack of understanding of entrepreneurship, even among those CMWs who participated in the project-sponsored business skills training course Given the importance of this component for long-term sustainability of the CMW program, the DoH and Mercy Corps included a second sub-study in the Learning Agenda to address the question: How can CMWs become financially self-sustaining while serving the needs of the poorest of the poor? The expenditure and investment assessment conducted as part of this sub-study showed that the monthly salary for the CMWs participating in the study (both SMNC-supported and other CMWs) ranged from a low of Pakistani rupees (PKR) 1000 to a high of 10,000. Among the SMNC￾supported CMWs, forty percent earned from PKR 1000 – 3000, well below the standard minimum wage of PKR 13,000. Although the project did not actually demonstrate how CMWs can achieve financial sustainability, there were achievements for this component nonetheless that the DoH and its partners can build on. For example, the Learning Agenda report highlighted the importance of the monthly stipend of PKR 5,000 that CMWs receive from the DoH for two years following deployment. The report also provided two major recommendations on financial sustainability: 1. Program support from the MNCH program is needed to provide supervision, stipends and other support to CMWs for at least five years post-deployment. 2. Integrate training on business skills into the pre-service training curriculum for all CMWs. The Learning Agenda report makes a compelling case for eventually integrating the CMWs into the health system in the same way that LHWs became salaried employees. This was also a strong recommendation made by a number of stakeholders (CMWs, project staff, two of the three DHOs and DoH administrators) interviewed during the qualitative research. Recognizing that the long-term success of the CMW program rests largely on the ability of CMWs to earn a decent wage, there is heightened interest in exploring the possibility of integrating them into the DoH system in Balochistan; this topic was a major point of interest at the recent Donor Conference organized by the GoB to discuss the new Balochistan MNCH Strategy 2016-20. A provincial-level committee has been formed and includes Mercy Corps’ Team Leader for the South, providing another opportunity for the DoH and Mercy Corps to work together on this issue. Enhancing quality of care through a clinical skills refresher course: This four-week course was one of the biggest successes of the project. According to those interviewed during the qualitative research, it clearly improved the quality of care the CMWs offer and bolstered their confidence in their skills. The PNC has endorsed the course and it is slated to become standard throughout Pakistan, a lasting contribution from the project. Improving reporting and tracking of client data through mHealth: Mercy Corps, in conjunction with its technology partner PakVista Technologies and the DoH, developed a mobile October 2016 14 phone application for CMWs to use for reporting, including acquisition, storage and processing of client records. The introduction of this mHealth application for streamlining data collection was one of the most successful and appreciated project components, mentioned consistently during the qualitative research by CMWs, their supervisors and the provincial and district health administrators. FGD participants and key informants noted that the use of mobile phones for collecting client data and preparing reports was particularly appropriate for CMWs in Balochistan as distances to district offices and limited transportation options were obstacles to delivering written reports in a timely fashion. Data transmission via mobile phones allowed for real time record keeping. The qualitative research findings from the FSA respondents – that mobile phone technology has streamlined CMW reporting – mirror the findings in the Learning Agenda report. However, the DoH respondents noted that this innovation will only be useful in the long run if it is integrated into the MNCH MIS. A typical comment comes from the Kech District PHS: “I really appreciate the mHealth intervention for registering cases and increasing the validity of the findings in real time but it has to be integrated with our MNCH database.” (Note: This integration is planned for the extension phase, which goes through March 2017.) During the KIIs, DoH managers at both the district and provincial level consistently cited the use of mobile phones for reporting when asked about project successes and which interventions showed potential for scaling up. This interest in scaling up is echoed in the Learning Agenda report and in a comment from PakVista Technologies: “In the long term, the provincial health department is clear on moving towards a mobile-based system as they can see the process improvements in the three districts covered in the pilot program.” Responding to this interest, Mercy Corps contracted with PakVista Technologies to prepare additional assessments. The September 2016 mHealth Assessment Final Report went into detail on the possibilities for integration of the technology with the MNCH Program’s Management Information system (MIS), covering technical and financial implications of several different options. The project has laid a solid foundation for reporting through mobile technology and documented the various options for the DoH to consider; given the DoH’s positive reaction, the potential for eventually scaling up this initiative throughout the province is high. Further steps are on hold for now: the DoH decided to discontinue its existing MNCH Program MIS and has contracted with a local firm to develop a new, more comprehensive MIS. PakVista and the project staff thoroughly documented the mHealth component at key points, describing lessons learned, obstacles encountered and subsequent actions taken; this documentation illustrated that the SMNC project is a learning project and provided a good example of how a component was introduced, tested, evaluated and adapted. Providing supportive monitoring and supervision: The project also invested resources in monitoring and supervision, another component designed to improve quality of care. Resources included transportation; incentives for the LHSs who provided administrative monitoring and for the LHVs who ensured technical supervision; and project staff time for visits to CMWs, including joint visits with DoH personnel. In addition to helping CMWs improve their skills, this monitoring and supervision also provided much-needed moral support for the CMWs. Health administrators observed that the project’s initiative to provide regular monitoring and supervision also improved coordination between the LHWs and the CMWs at the community level. Introducing a voucher scheme: To provide financial assistance for women too poor to pay for CMW services, the project introduced a voucher scheme in 2016. Since this component is relatively October 2016 15 new, a project-wide assessment was not available and few FGD and KII respondents were familiar with this activity. Those who did know about it expressed concern that the eligibility criteria were not always respected and that improvements needed to be made to ensure that the vouchers actually reached those most in need. A strong assessment and verification system has been put in place to verify beneficiary eligibility and satisfaction with services provided by the CMW. A September 2016 monitoring visit to Kech indicated that the five randomly-selected voucher beneficiaries were eligible; the beneficiairies also declared that they were satisfied with the quality of care provided. However, with an extension phase of only six months, it seems doubtful that the project will be able to thoroughly test this mechanism. Nevertheless, if the project staff documents the activity well, it will provide additional evidence for the government, which is interested in an improved voucher scheme. IR2. Improved knowledge and demand for essential maternal and newborn care The project registered only moderate success with this Intermediate Result: the uptake of the CMWs’ services indicates that demand was increased and the Learning Agenda report states that the project-supported CMWs were conducting “significantly more deliveries” compared to CMWs in non-intervention areas. However, since there was no final measurement of “improved knowledge”, assessing the success of this IR also required examining the two approaches used. Re-invigorating Women Support Groups: WSGs already existed in many of the project communities. With the help of the TWG, the project refined three modules focusing on maternal and child health and retrained the Lead Mothers, LHWs and Community Educators. During the FGDs with CMWs, the participants mentioned that the WSGs had been very helpful in introducing them to the community, generating demand for their services and carrying out health promotion activities. However, the consensus from CMWs and LHWs seemed to be that the groups were no longer meeting regularly and that incentives for the Lead Mothers were needed to encourage them to continue their activities. Using mHealth for health promotion: Also included as part of the mHealth component of the project was the dissemination of Voice Over Internet Protocol (VOIP) messages to clients. This included 12 behavior change messages for improved maternal and child health and automatic appointment reminders through SMS. In spite of the messages being available in four languages and tailored to the CMWs’ clients, this activity was not as successful as the use of mHealth for data collection. In Year 3 for example, only eight percent of the CMW clients registered for this service and of that small number, only 17 percent actually answered the calls to receive the messages. The following table prepared by PakVista Technologies for a May 2016 report provides an indication of the slow uptake of messages: Quetta Kech Gwadar TOTAL Total SMS 571 341 140 1,052 Total voice calls generated 2,146 1,356 540 4,042 Confirmed/accepted voice calls 84 24 7 115 Some of the reasons proposed for the low uptake on messages included the following: clients were reluctant to provide their phone numbers; not all women had access to the phone on a regular basis; October 2016 16 and since there was no caller name displayed, incoming messages were often ignored. In the end, this mHealth activity was discontinued in April 2016. IR3. Improved access to emergency transport in remote communities Compared to the other three IRs, the SMNC project made the least amount of progress on this IR due largely to circumstances beyond the project’s control. As of September 2016 the two components – improving emergency transportation and strengthening the referral system – had met with only limited success. Improving emergency transportation for timely referrals: In Balochistan a major obstacle to obtaining advanced medical care is the lack of accessible transportation to a secondary facility. To address this obstacle, the project undertook two activities. First, the CMWs were encouraged to work within their communities to identify drivers who could be hired to transport women to a secondary facility and to publicize the name(s) and contact information. Second, the project established Memorandums of Understanding with two private, not-for-profit ambulance companies, Edhi Foundation in Quetta and Gwadar Districts and Al-Falah in Kech District. Routine monitoring data from Years 3 and 4 indicate that on average 20-50 percent of those referred use the CMW￾linked drivers. The ambulance services are less frequently used due to how far away the ambulances are stationed. In Year 4, for example, only 7 out of 1,056 referrals used them. While these were positive steps, most respondents in the FGDs and KIIs noted that it is usually the family that arranges transportation through its own means. Reinforcing the referral mechanism between CMWs and secondary health facilities: In July 2016 the project conducted referral mechanism workshops in all three districts. DoH participants included relevant staff (gynecologists, hospital Medical Superintendent, transport heads, labor room managers and pediatricians). The not-for-profit ambulance service providers also participated. The workshop participants drew up a list of suggestions, which were presented to the TWG for discussion. The TWG recommendations are on the agenda for the October 2016 PSC meeting. Although these steps are promising, it is unlikely that a viable, sustainable referral system will be fully developed by the end of the project (March 2017). While the orientations and recommendations are a good first step and may provide a point of departure for the DoH, establishing a permanent referral system will require additional resources (especially adequate staff), reliable transportation and sustained oversight by senior managers, systemic issues within the DoH that are beyond the project’s capacity to address in the time remaining. IR4. Improved policy environment for improved maternal, newborn and child healthcare This IR, like IR1, was quite successful and there are strong indications that the achievements will continue to positively influence the policy environment in the future. According to those interviewed for the FSA, both of the components for improving the policy environment – the establishment of forums and the development of a provincial five-year MNCH strategy – resulted in long-lasting policy changes and a renewed commitment to CMWs. Establishing forums at the provincial and district levels: One of the most well-regarded project initiatives was the establishment of two provincial level forums, the Technical Working Group (TWG) and the Provincial Steering Committee (PSC). The TWG met frequently and provided guidance to the SMNC project and three other health initiatives. The members made recommendations to the PSC and if the PSC agreed with the recommendations, they were passed on to the DoH for action. Both forums have been institutionalized, receiving official government approbation. October 2016 17 Everyone interviewed about these two forums was highly complimentary and pointed out ways the forum members had facilitated project implementation and made permanent improvements in policies and procedures for the health sector. Concrete examples they cited include: • Provided support on the mHealth design for reporting and made suggestions on how it can be integrated with the MNCH Program MIS • Played a vital role in designing the five-year Balochistan MNCH Strategy, including the development of a communication strategy and a strong CMW component • Advocated successfully for the establishment of CMW work stations, allowing CMWs to provide services in their homes • Successfully lobbied for an increase in the CMW stipend from PKR 2,000 to PKR 5,000 to promote sustainability and retention and lobbied successfully for the value of the voucher to be increased • Successfully lobbied for the inclusion of chlorhexidine and misoprostol in the Essentials Drug List • Assisted with the development of training materials and IEC brochures for the WSGs , including translations into local languages As for the district health forums (DHFs), whether they were successful or not seemed to depend to a great extent on the leadership of the DHO. In one district, the DHO stated that he was too busy to attend the DHF meetings. This was in stark contrast to another district where the DHO organizes monthly meetings with his team (e.g., the Public Health Specialist for MNCH, the Medical Supervisor for the hospital and the District Coordinator for the LHW program). The team members interviewed all mentioned how useful these meetings were for discussing policies, solving problems and improving coordination. Developing and costing a MNCH strategy: A high priority for the DoH was to develop a five￾year provincial MNCH strategy for the 2016-2020 period and the project provided invaluable assistance in helping the DoH achieve this milestone. Stakeholders characterized the process as highly participatory, stating that it “involved all relevant stakeholders, especially the TWG” (KII with forum members). The GoB approved the strategy and organized a donor conference on September 28, 2016 to present the strategy and solicit resources. According to Mercy Corps staff, the conference was well-attended by 10-12 donors and the strategy was well-received. The GoB is reviewing its current budget to see what components of the strategy the provincial government can fund and what activities donors may be interested in supporting. Increasing Access to Quality Care Achievements for each Intermediate Result, especially IR1 and IR4, demonstrate that the project made significant progress toward improving access to quality health care for women and children. The findings from the FSA provide a definite YES to an important question in the ToR: Do CMWs offer quality care? The evidence from a number of sources used for the FSA – project reports, the Learning Agenda report and the qualitative research – shows that the CMWs supported by the SMNC project provide quality care and that their clients appreciate not only their technical competence but also their interpersonal skills. Four sources of evidence are summarized below. First, the project Performance Monitoring Indicator Table included approximately 15 quality of care indicators covering the spectrum of maternal and newborn care from antenatal to postnatal services. October 2016 18 CMWs recorded activities in their registers and brought a summary to the monthly meetings. During monitoring visits supervisors would select a number of CMW registers to verify the quality of care data. According to the registers, the CMWs routinely provided 90-100 percent of the actions that comprise quality care. A notable exception was that the majority of their clients did not complete at least four ANC visits. However, this indicator depends on factors beyond the capacity of a CMW herself to control. A second verification of quality care reflected in project monitoring reports was provided by the LHVs who conducted the technical supervision of the CMWs using a standardized checklist. One of the project indicators is “Proportion of CMWs who scored at least 80% on the Technical Supervisory Checklist”, which indicates that the standards for quality care are met. The last Annual Report (October 2014 – September 2015) showed that in the final quarter of Year 3 a total of 77 CMWs were supervised and all scored at least 80 percent. The data available for the period October 2015 through June 2016 corroborates this positive outcome as all of the CMWs supervised during this nine-month period scored at least 80 percent. A third source of information showing that CMWs provide quality care is provided in the Learning Agenda report, which devoted one of the four sub-studies to exploring the issue of quality of care. One of the sub-study methods used was to have independent LHVs observe the CMWs – both project-supported and other CMWs – as they provided services to clients. The following table shows how many CMWs were observed for each category of services: Practices Project-Supported CMWs (N=50) Other CMWs (N=79) Antenatal care 30 64 Natal care 37 62 Postnatal care 31 65 General care 48 76 According to the report, the CMWs supported by the project scored significantly higher with 96.6% providing services with “high competency” compared to 34.5% of CMWs from non￾project areas. The Learning Agenda sub-study also measured client satisfaction, asking clients if the CMWs were “acceptable health care providers at the community level”. Although both groups of CMWs were rated positively by the clients, those from project areas had a higher level of acceptability, due perhaps to the fact that they had more equipment, supplies and facilities for receiving clients. For both groups, clients commented that they appreciated the CMWs not only for their technical skills but also because they were always available, treated them with dignity and took a personal interest in them. The qualitative research conducted during the FSA provided a fourth source of information on quality of care and client satisfaction. This information corroborated the findings in the Learning Agenda report. FGD participants and key informants were all asked to rate the care provided by the CMWs and the vast majority of respondents (CMWs’ clients, supervisors and other stakeholders) rated the quality of services from Satisfactory to Excellent. In addition, women who had accessed services from the project-supported CMWs were asked to comment on their interactions. Clients October 2016 19 “The provincial forums have played a pivotal role in directing and suggesting objectives and strategies at the conception of the project and throughout its implementation. They will remain very useful to provide direction and further refinement of interventions and we will continue to support them." (Balochistan DG/Health Services during a KII.) gave high marks to the CMWs and noted the personal touch and approachability of the CMWs, mirroring the Learning Agenda results for this topic. STATUS OF THE MIDCOURSE CORRECTIONS The majority of the proposed actions described in the Case for Midcourse Corrections have been completed. The following table summarizes the status of each major action. Action Status Observations Finalize the five-year Balochistan MNCH strategy Completed The strategy was finalized, costed and shared at a donor conference held in September 2016. mHealth application: Transition the server to the DoH and fully synchronize the application with the revised MNCH MIS. In progress The DoH is in the process of revising its MNCH MIS. The plan is for the server to be housed in the DoH by the end of the extension period (March 2017). Recruit, deploy and support 10 additional CMWs in rural areas. Completed The new CMWs began providing services as of December 2015. Pilot the use of vouchers to reach the poorest women. In progress The pilot began only in 2016 and will be documented and evaluated during the six-month extension period. Orient DoH staff and CMWs on a referral mechanism between CMWs and secondary health facilities. Completed The project organized referral mechanism orientation workshops in all three project districts in July 2016. Starting July 2015 the DoH will provide monthly stipends of PKR 10,000 to all CMWs. Not done Although mentioned in the Case for Midcourse Corrections, this action is beyond the project’s control; ultimately it will depend on whether the MNCH department allocates resources. However, the stipends have been increased from PKR 2,000 to PKR 5,000. Carry out four sub-studies through the Learning Agenda to provide information for the DoH to improve policies and procedures for the CMW program. Completed A detailed final report with findings and recommendations is available and the project staff is in the process of disseminating it. KEY STRATEGIES AND FACTORS AFFECTING RESULTS One of the most important factors underlying the project’s achievements was the close working relationship between the GoB, represented primarily by the DoH, and Mercy Corps. Project documents, including periodic reports, illustrate the degree to which the DoH was involved not only in the design of the project but in every important decision made during implementation, including the midcourse corrections, the different interventions selected for testing and the themes for the Learning Agenda. It was also evident in the KIIs conducted for the FSA that the two parties worked well together. Informants included SMNC project staff, senior program managers from Mercy Corps, key DoH personnel at What were the key strategies and factors, including management issues and policy environment, that contributed to what worked or did not work? October 2016 20 the district and provincial level and members of the PSC and TWG. In their remarks they gave the same impression – that Mercy Corps was a responsive partner, testing approaches and interventions to provide evidence for decision-making on the part of the government and facilitating activities (such as the development of the Balochistan MNCH Strategy 2016-20) that were high priorities for the DoH. The three members of the PSC and TWG, for example, attributed project success to “…excellent coordination with government and strong working relationships with stakeholders”. According to Mercy Corps staff and DoH interviewees, the active engagement of the two policy forums was the most important factor ensuring a close partnership with the DoH; both forums made significant contributions to project achievements and positively influenced policies and procedures for current and future programming. Forum members include senior government officials, key decision-makers in the DoH, Mercy Corps senior managers and other health sector stakeholders such as the WHO representative for the province. Given their technical expertise, their status and their knowledge of the context, they were able to facilitate project implementation and advocate for changes that will have a lasting impact on the CMW program in Balochistan. This effective partnership was likely strengthened by the fact that Mercy Corps has worked in the health sector in Balochistan since 1986 and that there was little turnover in the senior Mercy Corps personnel involved in managing and backstopping SMNC during its four years of implementation. The fact that the key technical person for the project (Project Officer) was a government official on loan to the project may also have been a contributing factor in building a strong partnership and facilitating implementation. Another factor contributing to the project’s success was that the SMNC design was multi-faceted and took into account almost all the aspects that are critical to ensuring that the CMWs can provide quality services. Project components addressed recruitment, deployment, financial sustainability, quality of care, supportive supervision, community support, referrals and creating demand for CMW services. Although not all components achieved their targets, all needed to be included for achieving the objective of demonstrating a workable model. Contextual Factors Affecting Implementation and Outcomes A number of contextual factors affected project implementation and outcomes, including characteristics of the health system in Balochistan Province. Ineffective human resources management of DoH personnel negatively affected certain aspects of the project. Posts not filled and a high rate of absenteeism, especially at the district level, sometimes hindered implementation. This was especially true for midwifery schools which often lacked staff. Conversely, there were also good examples where a dedicated individual with strong leadership qualities (e.g., the DHO for Kech) embraced the CMW program and ensured that his entire team supported the CMWs, stating that “We own and respect them as they share our burden.” Along the same lines, a lack of government financial resources for items such as vehicles and incentives for monitoring CMWs also negatively affected some outcomes and will likely also affect sustainability. For example, Mercy Corps provided many of the resources, including transportation and funds, for the LHVs to conduct CMW supervision. Without adequate resources, this support for CMWs will not continue. Demographics also influenced project implementation. Balochistan Province has approximately 44 percent of the country’s land but only 5-6 percent of the population. As a result, there are over 25,000 widely-scattered settlements, most of which have a relatively small population size. The lack of transportation and difficult terrain hinders mobility in the rural areas both for the CMWs and for October 2016 21 their clients. The mobility limitations and the relatively small catchment area for most rural CMWs can mean that they may not have enough clients to make a living wage. On the positive side, these same characteristics of the three districts also highlight the value of a program where CMWs live in the community and can provide immediate and affordable quality services to people who cannot easily access other health care options. The nature of the patriarchal society in the project area sometimes imposed constraints on women, especially their mobility. For example, women may require a male family member to escort them to a meeting or a training. Families may also be unwilling for their female family members to live in a hostel for an extended period to participate in training. As the Quetta CMWs expressed it during their FGD: “The strong patriarchal mindset of the Balochistan culture discourages females working independently and being mobile.” Participants in the FGDs and KIIs cited a number of other factors that presented challenges for project implementation. Illustrative examples include: • Low literacy rate among rural girls and women means a limited pool of candidates for CMW training. • A lack of coordination between the MNCH program and the LHW program can result in a fragmented approach to maternal and newborn care. This situation can result in conflict between LHWs and CMWs in the same catchment area and negatively impacts the level of support the CMWs need to feel valued and to deliver quality services. • Poverty levels were such that not everyone is able or willing to pay for CMW services. Low household income levels can also foster competition between LHWs, TBAs and CMWs, all of whom are vying to bring earnings home. • According to the CMWs in the FGDs for all three districts, some families do not consider midwives to have the same status as a profession as being a teacher and/or consider it an inappropriate vocation for an unmarried woman, limiting further the pool of prospective CMW candidates. • During the first two years of the project, security concerns preoccupied managers, leading to a series of measures to prevent and mitigate security-related incidents. While the situation has much improved, a number of those measures are still in place. Mercy Corps staff, for example, cannot travel by road between the three districts and periodically call on local government counterparts in Kech District to monitor eight CMWs because security issues prevent Mercy Corps staff from visiting them. Capacity Building Capacity-building was another strategy to reinforce the potential for sustainability. The most important contribution was reinforcing the technical skills and building the entrepreneurship capacities of the 95 CMWs. Not only were capacities for this group strengthened but the courses developed through the project have received official approval and will be adopted in future CMW trainings in Balochistan and potentially throughout Pakistan. It should be noted, however, that continuing technical skills education for CMWs is essential if they are to maintain their skills. It would also be useful to provide refresher training on business skills since many of the concepts were new and these skills are essential for financial self-sustainability. October 2016 22 The project-supported CMWs were not the only groups to benefit from capacity-building. Other examples include: • The DoH gained valuable experience in crafting and costing a five-year MNCH strategy. • The LHWs, Community Educators and Lead Mothers benefitted from capacity building in health promotion techniques. • At the provincial level, DoH technicians and managers as well as their colleagues from other departments and organizations gained practical experience in advocacy and influencing policy through their participation in the TWG and the PSC. SUSTAINABILITY AND POTENTIAL FOR SCALING UP The table below summarizes the potential for sustainability for key project elements; the assessment is based primarily on the qualitative research and the desk review and presents the potential for sustainability as of September 2016. Project Element Potential for Sustainability, Scaling Up Observations Four-week clinical refresher course Achieved PNC approval for scaling up. Business skills course Achieved Approved for incorporation into standard midwifery training course. Work stations Achieved Policy approved to allow CMWs to provide services from their own work stations, usually in own home. Balochistan MNCH Strategy High Strategy developed, costed and approved. Resources required for implementation. mHealth application for streamlined reporting and client data retention High DoH committed to adapting application to their system. Provincial forums (TWG, PSC) High Have been institutionalized; keen interest in sustaining. District health forums Medium Sustainability likely to depend on each DHO. Regular monitoring and supervision for CMWs by LHVs and LHSs Low to medium Will depend on political will of GoB, DoH and resources. Emergency transportation system established Low to medium Is a greater awareness of options, especially local drivers, but obstacles remain for establishing a reliable, affordable system. Improved referral system Low to medium Will require resources and commitment beyond project’s capacity, including improved management of government ambulances. WSGs continue health promotion, including periodic meetings Low May occur on individual Lead Mother level. Voucher scheme Low Too little time to adequately test a model; may result in useful lessons learned for DoH to follow up on Which elements of the project have been or are likely to be sustained or expanded (e.g., through institutionalization or policies)? October 2016 23 Working around strengthening community-based maternal and newborn healthcare provision, to what extent has the project been successful? “I would definitely recommend the model to be replicated in other districts of Balochistan, particularly the ones which are the most remote. We have included the deployment of CMWs in our new five-year MNCH strategy and the DoH is committed to supporting them. We are having a donor conference next week [September 28, 2016] and will highlight this issue.” (Balochistan DG/Health Services during a KII.) CONCLUSIONS AND RECOMMENDATIONS The information collected during this FSA provided a resounding “YES” to one of the questions posed in the ToR: “Is the community midwife a suitable solution for contributing to reductions in maternal and newborn mortality considering the quality of care of her services and in-line with cultural and economic constraints?” The SMNC project convincingly demonstrated that with appropriate selection, training and continued support, the CMW can acquire the necessary skills, confidence and community status to be a lifesaving provider of MNCH services, especially in rural areas with widely-scattered populations where few if any other health services exist. This was the unanimous sentiment expressed during the qualitative research and is substantiated by the secondary data sources such as the recently-developed Balochistan MNCH Strategy. The Strategy contains numerous references to the DoH’s commitment to improve the province-wide CMW program by incorporating successful components of the SMNC project such as the clinical skills refresher course, the business training and the mHealth application. The project achieved its main objectives: it increased access to quality maternal and neonatal health services for families in underserved areas and it provided tested CMW program interventions for replication throughout Balochistan. In doing so, it left lasting achievements such as: • 95 midwives trained and equipped, the majority of whom continue to provide essential services to women with few other options for skilled birth attendance • A clinical skills refresher training course endorsed by the PNC, to be incorporated into all midwifery training schools in Balochistan and eventually in Pakistan • A business skills course to be included in the 18-month midwifery training • Increased demand for MNCH services at the community level • A three-module set of materials for WSGs, available in four languages • Two institutionalized provincial policy platforms, the TWG and the PSC • A new approved and costed five-year MNCH strategy for Balochistan with a major component for CMWs • An mHealth application for improved data collection and reporting ready to be integrated with the DoH system Perhaps one of the most useful contributions moving forward will be the Learning Agenda report. The use of qualitative and quantitative methods for the research and the fact that the activity compared SMNC-supported CMWs to CMWs in non-project areas result in a body of evidence that provides convincing reasons how interventions such as those piloted by SMNC can strengthen a CMW program. The findings documented through the four sub-studies, especially the studies on October 2016 24 CMW recruitment and retention and on options for financial self-sufficiency, and the extensive set of recommendations should promote continued dialogue and action for enhancing the CMW program not only in Balochistan but in other underserved areas where access to skilled birth attendance is limited. That the project has been able to achieve this in only four years is a testament to the DoH’s commitment, the strong partnership between Mercy Corps and the Government of Balochistan and a sound project design that addressed high priorities for the stakeholders. Challenges While the successes are evident, challenges remain. The following list of major challenges comes primarily from KII interviews with DoH stakeholders at the provincial and district levels: 1. Identifying enough qualified and interested women from rural areas who meet the minimum requirements to become a CMW 2. Improving linkages between the MNCH program and the CMW program 3. Creating more harmonious working relationships between TBAs, CMWs and LHWs 4. Providing adequate support through regular supportive supervision 5. Ensuring that the CMWs have sufficient monetary incentives to continue providing services 6. Improving the referral mechanism between CMWs and secondary facilities 7. Establishing a reliable transportation system for referrals 8. Overcoming budgetary constraints within the GoB To sum up the overarching challenge, one of the provincial senior managers stated: “It will be difficult to sustain this model without political commitment, government ownership and adequate resource allocation.” RECOMMENDATIONS If the following recommendations are carried out, the potential for improving the Balochistan CMW program will be increased and the lessons learned from the close collaboration between the GoB and Mercy Corps will contribute to sustaining and scaling up SMNC achievements. Learning Agenda Follow-up 1. Prepare a briefing paper that summarizes the main findings and recommendations from the Learning Agenda; ensure wide-spread dissemination and follow-up conferences and working commissions for key recommendations, especially those concerning i) improved selection, deployment and retention and ii) adequate remuneration and incentives for CMWs. (DoH, Mercy Corps) Support for CMWs 2. Create a more collaborative and harmonious working relationship between CMWs and LHWs. Methods could include regular meetings, joint community initiatives and strong leadership and team building from senior DoH managers and community leaders. (DoH – PHS and MNCH Coordinators at the district level) 3. Reinforce orientations for family members of CMWs and for their communities to create sustained support for the CMWs. Examples could include i) community elders such as October 2016 25 religious leaders exchanging experiences for creating a supportive environment for CMWs and ii) calling on WSGs where they are active. The Quetta CMWs, for example, credited the WSGs with paving the way for them to provide services. (MNCH District Coordinators, District PHS, LHWs, community leaders, active WSG) 4. Consider integrating CMWs into the DoH to ensure adequate financial compensation, supervision and support. (DoH) 5. Investigate creative options for accommodating married women with young children who qualify for CMW training. (DoH) Ensuring CMWs Provide Quality Care 6. Commit resources to continue supportive monitoring and supervision of CMWs. (GoB and partners)1 7. Conduct refresher training for LHSs on how to provide effective administrative supervision and on-the-job mentoring for CMWs. (DoH) 8. Invest in regular refresher training on technical themes and business skills for CMWs. This could include on-the-job training and mentoring during routine monitoring and supervision. (DoH) 9. Make quality staffing of midwifery schools a priority. (DoH) Sustaining and Scaling Up SMNC Achievements 10. Continue with the plan to integrate the mHealth application into the MNCH MIS. (DoH, Mercy Corps) 11. Disseminate the voice messages in four languages developed under SMNC. CMWs and LHWs can use these messages for health promotion and demand generation. (DoH) 12. Continue the policy forums (Provincial Steering Committee and Technical Working Group), especially for overseeing the implementation of the Balochistan MNCH Strategy 2016-20. (DoH and partners) 13. Document the voucher scheme activity at the end of the extension period and share findings with DoH colleagues. (Mercy Corps) 14. Mobilize resources and partners for the implementation of the Balochistan MNCH Strategy 2016-20. (DoH) 15. Longer term: Consider integrating the MCH, MNCH and LHW programs. This would enhance coordination within the DoH and contribute to improved supervision and support for CMWs. (DoH) Dissemination of FSA Findings 16. Consider publishing the FSA findings in the Global Health: Science and Practice Journal. (Mercy Corps with the DoH) 1 The Balochistan MNCH Strategy 2016-20 and MNCH PC-1 provide clear methodology for supportive monitoring and supervision of CMWs complemented by tools for monitoring and supervision in the CMWs Deployment Guidelines. According to the methodology, the LHV from the nearest health facility is responsible for technical supervision while the LHS is responsible for administrative supervision of CMWs. October 2016 U.S. Agency for International Development 1300 Pennsylvania Avenue NW Washington, DC 20523 October 2016 ANNEX I. FOCUSED STRATEGIC ASSESSMENT TERMS OF REFERENCE Terms of Reference for External Consultant for the Focused Strategic Assessment of the Saving Mother and Newborns in Communities Project in districts Quetta, Kech and Gwadar, Province of Balochistan, Pakistan August 5, 2016 I. Introduction Mercy Corps will hire an independent international consultant to lead a Focused Strategic Assessment (FSA) for the Saving Mother and Newborns in Communities (SMNC) project funded by USAID’s Child Survival and Health Grants Program (CSHGP), COOPERATIVE AGREEMENT NUMBER USAID CA No. AID-OAA-A-12-00093, dated September 30, 2012, of US$ 2,322,520 ($1,741,836 USAID contribution) in Quetta, Kech and Gwadar Provinces of Balochistan Pakistan. As it currently not possible to obtain a visa for foreign nationals, the lead consultant will remotely lead the FSA, working closely with an external national consultant identified by Mercy Corps. USAID’s CSHGP supports community-oriented projects implemented by U.S. private voluntary organizations (PVOs) and nongovernmental organizations (NGOs) and their local partners. The purpose of this program is to contribute to sustained improvements in child survival and health outcomes by supporting the innovations of PVOs/NGOs and their in-country partners in reaching vulnerable populations. This document describes the FSA international consultant’s Scope of Work (SOW) for the Saving Mother and Newborns in Communities FSA. II. Background Mercy Corps is implementing a four-year maternal, newborn and child health (MNCH) Program in Quetta, Gwadar, and Kech Districts of Balochistan, Pakistan with support from USAID CSHGP and the Scottish Government. To address Pakistan’s sustained high rates of maternal and neonatal mortality and to ensure skilled birth attendance, the Government of Pakistan (GOP) and the provincial Department of Health (DoH) have given top priority to reach out to pregnant mothers in remote communities by training a cadre of community midwives (CMW). However, training alone has not been sufficient as most of the CMWs have not been able to establish their clinics and attract clients. Saving Mothers and Newborns in Communities (SMNC) seeks to improve maternal and newborn health status, especially for poor and marginalized women of Balochistan (Goal), through increased use of quality essential maternal and newborn care, through private-sector October 2016 community midwives (Strategic Objective). The project’s intermediate Results are: 1. Increased availability of quality maternal and newborn care in communities 2. Improved knowledge and demand for essential maternal and newborn care 3. Improved access to emergency transport in remote communities 4. Improved policy environment for improved maternal, newborn and child healthcare based on evidence from the Operations Research SMNC is an innovative model that was designed to enable CMWs to become self-sustaining, private MNCH service providers. The program was designed jointly with the Balochistan DoH, upon their request, to offer evidence for how to scale up high impact MNCH interventions in Balochistan through the CMW. The model has been tested with 95 CMWs in Quetta, Gwadar, and Kech districts of Balochistan. The project contains the following main components: 1. To ensure quality, Mercy Corps has provided CMWs with clinical refresher training, supported CMW registration with the Pakistan Nursing Council (PNC) for those who were not already registered, and conducted joint supervision visits with the DoH. 2. To enable CMWs to set up home-based clinics, Mercy Corps has provided standard equipment and business skills training for the CMWs. 3. Through Mercy Corps’ partnership with Pak Vista Shared Technologies, CMWs have been using their mobile phones to track patient data, send automatic reminders to clients, and offer mass SMS’ for awareness raising. Through automatic data transfer, the DoH is now able to track uptake of CMWs’ services in real time. 4. For behavior change and demand creation, Mercy Corps reinvigorated the Women Support Groups conducted by CMWs and Lady Health Workers. These groups also generate support to facilitate access to emergency transport. 5. For timely referrals, Women Support Groups and CMWs have been linked with not-for￾profit ambulance services. 6. At the policy level, Mercy Corps has assisted the provincial DoH in developing a five￾year strategic MNCH plan. The plan will be revised and updated based on findings from the Learning Agenda which is exploring whether CMWs can become self-sustaining private providers, while increasing access and utilization of high impact, quality MNCH interventions. 7. To support the DoH in operationalizing their plans to address the needs of poorest of the poor women in accessing maternal and newborn health services, a voucher scheme has been introduced in the project. 8. Mercy Corps has also prepared and oriented DoH staff on a referral mechanism between CMWs and secondary health facilities, which is not fully functional at the moment. Through funding from USAID’s CSHGP and the Scottish Government, SMNC and its partners will become key players within a global community of researchers supported by USAID seeking to identify innovative solutions to scale up high impact MNCH interventions. October 2016 III. Project Population Beneficiaries* Total Total Population 2,689,838 Total Neonates 11,093 Infants aged 0–11 Months 13,388 Children aged <5 Years 65,028 Women of Reproductive Age (15–49 years) 84,153 Total Beneficiaries 382,515 Expected Pregnancies 13,006 Community Health Workers or Volunteers (CHWs), Disaggregated by Sex 95 CMWs, 272 LHWs/CEs (Female) Health Facilities (Hospital to Sub Health Post) N/A Community-Based Structures (e.g., Village Development Committees [VDCs]) 272 Lead Mothers Groups *Source: * As per guidelines of National MNCH Program and provincial DOH estimates IV. Partners Department of Health, Government of Balochistan V. Key Activities Intermediate Result 1: Increased availability of quality maternal and newborn care in communities 1.1 Selection and registration of Community Midwives (CMWs) 1.2 CMW refresher training 1.3 Financial and structural support to CMWs 1.4 CMW deployment 1.5 Supervision of CMWs in the field 1.6 Development of mobile phone application for CMWs to track clients, send reminders and BCC messages Intermediate Result 2: Improved knowledge and demand for essential maternal and newborn care 2.1 Mobile phone mass SMS 2.2 Formation of Women Support Groups Intermediate Result 3: Improved access to emergency transport in remote communities 3.1 Emergency Transport Services Intermediate Result 4: Improved policy environment for improved maternal, newborn and child healthcare based on evidence from the Operations Research October 2016 4.1 Provincial MNCH Steering Committee formation and quarterly meetings 4.2 Provincial Technical Working Group formation and quarterly meetings 4.3 District Health Forum formation and quarterly meetings 4.4 Development of a 5 Year MNCH Strategy for the DoH 4.5 Implementation of learning agenda with the following four questions: • How can the DoH improve its selection process to effectively recruit and deploy CMWs in underserved areas? • How can CMWs become financially self-sustaining while serving the needs of the poorest of the poor? This will probe into issues of: establishment of workstations (what the government should provide and what CMWs should source on their own) monthly stipend from government; fee for services; and vouchers for the poorest of the poor. How do the factors differ in rural versus urban areas? • Do CMWs offer quality care? If so, how? • How can the MOH streamline CMW reporting using cell phone technology and expand mHealth in the province. VI. Purpose of the Focused Strategic Assessment The purpose of USAID’s CSHGP is to contribute to advancing the health system strengthening goals of Ministries of Health toward achieving sustained improvements in child survival and health outcomes, particularly among vulnerable populations, by supporting the innovative, integrated community-oriented programming of PVOs/NGOs and their in-country partners. CSHGP cooperative agreements offer unique opportunities to demonstrate the links between specific delivery strategies and measured outcomes. The FSA is intended to focus on four learning agenda questions (described above), which were designed jointly with the Balochistan DoH to provide them with the information they need to improve their CMW policies and programs, but will also cover the scope of the overall project as it will be broadly accessible to various audiences including Ministries of Health (MOHs). It is important that the lead FSA consultant consider the audiences listed below, when conducting the FSA and writing the report. The FSA provides an opportunity for all project stakeholders to take stock of accomplishments to date and to listen to the beneficiaries at all levels, including mothers and caregivers, other community members and opinion leaders, health workers, health system administrators, local partners, other organizations, and donors. The FSA Report will be used by the following audiences as a source of evidence to help inform decisions about future program designs and policies: • In-country partners at the national, regional, and local levels (e.g., MOH and other relevant ministries, district health team, local organizations, communities in project areas). • USAID (CSHGP, Global Health Bureau, USAID Missions), MCSP and other CSHGP grantees. October 2016 • The international global health community. The FSA report will be posted for public use at http://www.mchipngo.net and the USAID Development Experience Clearinghouse at https://dec.usaid.gov. VII. Methodology The FSA methodology consists of a mixed-methods approach comprising a desk review of secondary data sources and the collection of qualitative data to complement existing data. The written design of the FSA must be further defined and specified by the FSA lead consultant (e.g., number of key informant interviews, focus groups discussions, observations, and locations) and must be shared with project stakeholders and implementing partners for comment before the FSA commences. Mercy Corps will facilitate this sharing and feedback. Secondary Data: The FSA lead consultant will review documents mentioned in Annex A of the TORs. These include; project reports, key MoH policy and strategy documents and, U.S. Government/USAID strategic documents at the global and national levels relevant to the content of project. As outlined in Annex A, one of the key documents that the FSA lead consultant will review is the learning agenda report. The FSA will primarily focus on the four studies conducted under the learning agenda. The four studies contributing to the learning agenda started in February 2016 and will be completed by July 31, 2016. The reports from these studies will be available for the final evaluator to review for the purpose of the FSA. Each study has a unique methodology; the final evaluator will review the methodology, findings and conclusions of each of the four studies contributing to the learning agenda as part of the FSA. The Learning Agenda methodology is attached as Annex B. The following are the expected outcomes of the learning agenda: 1. Recommended incentive package for CMWs and recruitment procedures for CMWs 2. Proposed financially self-sustainable model for a CMW 3. Documentation of best practices demonstrating the quality of care of services offered by the CMWs 4. Identification of gaps in the existing reporting and monitoring system and recommendations for improvement Qualitative Data: As part of the FSA, in-depth qualitative interviews or focus group discussions may be conducted with stakeholders, including project staff, DoH staff, local NGOs and community-based organizations (that are the part of policy forums established by DoH and MC), district health teams, community and facility based health workers, community members, community leaders, and mothers (exit interviews). In addition, review of policy forum meeting minutes may be conducted. If possible, the assessment will also include observations of activities supported by the project. This will involve site visits to one or more implementation area. The team lead will develop the methodology and tools for the qualitative data collection, and will remotely guide the national consultant to randomly select communities to visit from a list provided by Mercy Corps. However, purposive sampling may be warranted in addition to explore certain areas in October 2016 more depth to investigate particular results (e.g., high or low performance or unexpected results). Team Lead will analyze the qualitative data collected by the national consultant and will be responsible for the overall report writing. Limitations: The FSA report must include a discussion of the methodological limitations of the FSA. Please refer to section XIII (proposed outline of the FSA) for guidance on the report template. VIII. Focused Strategic Assessment Questions The FSA lead consultant and the assessment team will use existing data collected or compiled during the life of the project, as well as additional data collected during the FSA to answer the following questions: 1. To what extent did the project accomplish and/or contribute to the results (goals/objectives) stated in the Strategic Work Plan, keeping in view the revisions in the mid-course correction document? • Specifically focusing on the four learning agenda questions (described above in Section V. Key Activities) and how the project contributed to these: o How has the project helped the DoH improve its selection process to effectively recruit and deploy CMWs in underserved areas? o Has the project been able to demonstrate how CMWs can become financially self￾sustaining while serving the needs of the poorest of the poor? This will probe into issues of: establishment of workstations; monthly stipend from government; fee for services; and vouchers for the poorest of the poor. The FSA will look into how these components have been implemented and what results have been achieved and will specifically comment on how the factors differ in rural versus urban areas in light of the results of the learning agenda studies. o Do CMWs offer quality care? If so, how? o How has the DoH streamlined CMW reporting using cell phone technology and expanded mHealth in the province? • What is the quality of evidence for project results? • What progress has the project shown regarding the mid-course corrections (explicitly), and what evidence has there been to show this progress? 2. What were the key strategies and factors, including management issues and policy environment, that contributed to what worked or did not work? • What were the contextual factors such as socioeconomic factors, gender, demographic factors, environmental characteristics, baseline health conditions, health services characteristics,* and so forth that affected implementation and outcomes? * See Table 1 in the document here: http://heapol.oxfordjournals.org/content/20/suppl_1/i18.long October 2016 • What capacities were built (with a focus on CMWs, midwifery tutors, Department of Health and other partners), and how? • Specifically asses the policy-level interventions implemented under SMNC, including the development of provincial MNCH strategy, and implementation of policy forums (Provincial Steering Committee and Technical Working Group)? 3. Which elements of the project have been or are likely to be sustained or expanded (e.g., through institutionalization or policies)? • Analyze the elements of scaling-up and types of scaling-up that have occurred or could likely occur (including mHealth, voucher scheme, refresher training, policy forums). 4. What are stakeholder perspectives on the overall project implementation, the policy forums, and the Learning Agenda implementation, and how could the Learning Agenda affect capacity, practices, and policy? • Analyze the notifications and recommendations put forward by the policy forums (Provincial Steering Committee and Technical Working Group), as well as their meeting minutes. 5. Working around strengthening community-based maternal and newborn healthcare provision, to what extent has the project been successful? • Is the community midwife a suitable solution for contributing to reductions in maternal and newborn mortality considering the quality of care of her services and in-line with the cultural and economic constraints? IX. FSA Consultant Characteristics and Expected Timeline The international consultant will serve as the FSA team leader and is welcome to propose additional assessment team members to round out the assessment team’s skill set in order to ensure adequate representation of evaluation, technical, geographic, cultural and language skills. Team members, their affiliations, and disclosure of conflicts of interest must be listed in an annex to the FSA report. The lead consultant will coordinate closely with the Mercy Corps team regarding tool finalization, FSA methodology, timeline, and draft report finalization. Requirements: The lead consultant must be approved by USAID CSHGP and should meet the following minimum requirements: • Proven expertise and leadership in − integrated community-oriented reproductive, maternal, newborn, and child health projects − conduct of evaluations (baseline, endline) using mixed methods • Experience with design, collection, and analysis using applied research methods in a program implementation context October 2016 • Familiarity with public health system in Pakistan and should be from Pakistan due to visa constraints • Demonstrated ability to communicate with and lead a team of stakeholders, staff, and national experts in participatory evaluation • Familiarity with USAID programming • Skill or familiarity with methods for program assessments • Excellent analytical and writing skills (English) • Signed statement explaining any conflict of interest† Key Tasks of the FSA Team Leader: • Review project documents and resources to understand the project • Review Learning Agenda sub-studies and their findings and assess if they address DoH needs‡ • Refine the evaluation objectives and key questions based on the CSHGP guidelines in coordination with Mercy Corps team and its partners • Develop detailed ToR for the national consultant and work with him or her to plan, implement and complete the FSA. • Develop the field FSA schedule and assessment tools • Remotely train the national consultant (and other evaluation team members if applicable) on objective and process of the FSA including evaluation tools • Lead the development and methodology of data collection, as well as the data collection tools, and work remotely with the national consultant to complete the collection, analysis, and synthesis of supplemental information regarding the program performance • Interpret qualitative results and draw conclusions, lessons learned, and recommendations regarding project outcome • Prepare presentation (a PowerPoint slideshow deliverable, no longer than 20 slides) for an in-country debriefing meeting with key stakeholders, (with USAID/Washington, DC, participation remotely, as able) • Prepare draft report in line with the agreed upon FSA guidelines and submit to Mercy Corps for review and feedback • Respond to feedback from Mercy Corps in the Statement of Differences, if applicable, and make any final revisions prior to grantee submission of the final report which is due † CSHGP grantees are required to hire an external evaluator for the final evaluation. That fiduciary relationship creates a conflict of interest that is minimized by the CSHGP requirement of submission of a draft evaluation report directly to the CSHGP. ‡ Mercy Corps and the Balochistan DoH jointly agreed on the key learning agenda topics which would equip the DoH to better utilize the CMW as a resource to address MNCH outcomes. The DoH identified their main needs as determining how: to improve selection and deployment of CMWs in a way that they reach underserved populations and; to keep CMWs engaged with the program. October 2016 to USAID CSHGP GH/HIDN/NUT office on or before 90 days after the end of the project Timeline: 1-MONTH TIMEFRAME FOR A TOTAL LEVEL OF EFFORT OF 24 DAYS. The breakdown of the level of efforts is given in following table. S. No. Activities Number of Lead Consultant’s Days 1 Online meeting with Mercy Corps team to discuss the implementation of the FSA methodology and related logistics 0.5 2 Desk review 4 3 Develop methodology data collection tools for FSA 2 4 Incorporate feedback and finalize tools 1.5 5 Mentoring of national consultant for data collection 2 6 Analysis of the data 5 7 Develop and share preliminary reports of the FSA 6 8 Submit final report after incorporating feedback 3 Total Days 24* X. Focused Strategic Assessment Report The FSA report should follow the agreed upon outline for this FSA. A draft and final report, written by the FSA consultant, must be submitted to Mercy Corps. Mercy Corps is responsible for submission of the final draft to the CSHGP and other required parties as indicated in the guidelines. XI. Budget Total days 24; consultancy fee is $ 500 per day. At the conclusion of the consultancy period, the consultant is expected to complete the following deliverables: October 2016 • Prepare presentation (a PowerPoint slideshow deliverable, no longer than 20 slides) for an in-country debriefing meeting with key stakeholders, (with USAID/Washington, DC, participation remotely, as able) • Prepare a draft report in line with the CSHGP guidelines and submit to Mercy Corps for review and feedback. • Prepare the final report in time for formal submission by Mercy Corps. The final report with all annexes is due at the USAID CSHGP GH/HIDN/NUT office on or before 90 days after the end of the project. XIII. Proposed outline of the Focused Strategic Assessment I. Cover page II. Executive summary III. Project background a. Situation of maternal and neonatal mortality in the province b. Project summary including mid-course corrections c. Partnerships and collaboration d. Learning agenda IV. Purpose of the Focused Strategic Assessment V. Evaluation methodology VI. Main results 1. To what extent did the project accomplish and/or contribute to the results (goals/objectives) stated in the Strategic Work Plan, keeping in view the revisions in the mid-course correction document? • Specifically focusing on the four learning agenda questions: o How has the project helped the DoH improve its selection process to effectively recruit and deploy CMWs in underserved areas? o Has the project been able to demonstrate how the CMWs can become financially self-sustaining while serving the needs of the poorest of the poor? This will probe into issues of: establishment of workstations) monthly stipend from government; fee for services; and vouchers for the poorest of the poor. The FSA will look into how these components have been implemented and what results have been achieved and will specifically comment on how the factors differ in rural versus urban areas in light of the results of the learning agenda studies. o Do CMWs offer quality care? If so, how? o How has the DoH streamlined CMW reporting using cell phone technology and expanded mHealth in the province? • What is the quality of evidence for project results? • What progress the project has shown regarding the mid-course corrections (explicitly), and what evidence has there been to show this progress? October 2016 2. What were the key strategies and factors, including management issues and policy environment, that contributed to what worked or did not work? • What were the contextual factors such as socioeconomic factors, gender, demographic factors, environmental characteristics, baseline health conditions, health services characteristics,§ and so forth that affected implementation and outcomes? • What capacities were built (with a focus on CMWs, midwifery tutors, Department of Health and other partners), and how? Specifically assess the policy level interventions implemented under SMNC including the development of provincial MNCH strategy, and implementation of policy forums (Provincial Steering Committee and technical working group) 3. Which elements of the project have been or are likely to be sustained or expanded (e.g., through institutionalization or policies)? • Analyze the elements of scaling-up and types of scaling-up that have occurred or could likely occur (including mHealth, voucher scheme, refresher training, policy forums) 4. What are stakeholder perspectives on the overall project implementation, the policy forums, and the Learning Agenda implementation, and how could the Learning Agenda affect capacity, practices, and policy? • Analyze the notifications and recommendations put forward by the policy forums (Provincial Steering Committee and Technical Working Group), as well as their meeting minutes. 5. Working around strengthening community-based maternal and newborn healthcare provision, to what extent has the project been successful? • Is community midwife a suitable solution for contributing to reductions in maternal and newborn mortality considering the quality of care of her services and in-line with the cultural and economic constraints? VII. Conclusion VIII. Recommendation § See Table 1 in the document here: http://heapol.oxfordjournals.org/content/20/suppl_1/i18.long Focused Strategic Assessment: SMNC Project October 2016 ANNEX II. ASSESSMENT METHODOLOGY AND LIMITATIONS Mercy Corps hired an external consultant, Kathy Tilford, to lead the Focused Strategic Assessment (FSA) remotely from the U.S. She worked closely with a well-qualified local Pakistani consultant, Dr. Sohail Amjad, hired by Mercy Corps/Pakistan. The local consultant had extensive experience in evaluation, including Child Survival and Health Grants Program (CSHGP) projects, and in-depth knowledge of the Pakistani health system, the Community Midwives (CMW) program and the local context. He served as the field team leader, working with two assistants experienced in qualitative data collection: Dr. Muslim Abbas and Ms. Saima Zeb Faredi. (See Annex VII for list of team members, titles and roles.) The methodology for the Focused Strategic Assessment was designed to answer five key questions: 1. To what extent did the project accomplish and/or contribute to the results (goals/objectives) stated in the Strategic Work Plan, keeping in view the revisions in the midcourse correction document? 2. What were the key strategies and factors, including management issues and policy environment, that contributed to what worked or did not work? 3. Which elements of the project have been or are likely to be sustained or expanded (e.g., through institutionalization or policies)? 4. What are stakeholder perspectives on the overall project implementation, the policy forums, and the Learning Agenda implementation, and how could the Learning Agenda affect capacity, practices, and policy? 5. Working around strengthening community-based maternal and newborn healthcare provision, to what extent has the project been successful? The methodology consisted of a participatory mixed-methods approach that included two principal components: 1. Comprehensive desk review of secondary data sources: Prior to developing the instruments for the field work, the team conducted an extensive desk review of quantitative and qualitative data sources. These included project documents such as the proposal, the Strategic Work Plan, surveys and assessments, routine monitoring updates and the “Case for Midcourse Corrections”; background documents on maternal and child health and the Pakistan health system; the Learning Agenda report; and Department of Health (DoH) publications such as the Balochistan MNCH Strategy for 2016-2020. (See Annex IV for a complete list of documents consulted.) 2. Collection of qualitative data: Using semi-structured Key Informant Interviews (KIIs), Focus Group Discussions (FGDs) and informal discussions with project stakeholders, the external consultant and the three-person field team carried out qualitative data collection over a two-week period. (See Annex III for the data collection instruments and work plan.) Focused Strategic Assessment: SMNC Project October 2016 Selection of data collection methods: In addition to KIIs and FGDs, the team considered other data collection methods including Observations and Exit Interviews. However, it would have been difficult to organize Observations of CMWs and Exit Interviews with clients for several reasons: - a field team member would need to be present at the exact time a client came for a consultation; - for cultural reasons only a female team member would be able to conduct such a data collection activity; - time constraints would mean a limited number of observations and/or exit interviews; and - the data collection carried out for the Learning Agenda already included a large number of observations of CMWs by trained health care providers. Choice of groups to interview: In deciding which groups to include for the FGDs, the team discussed the possibility of a discussion with male community leaders. However, the Project Manager suggested that this might not be as useful since men had not been very involved at the community level and would likely not have a lot to share. Standard themes: To facilitate the triangulation of data, the team selected a number of common themes to include in all data collection instruments: support for CMWs, including monitoring and supervision and the Women Support Groups; sustainability and replicability; referral system and emergency transportation; contribution of the project to improving MNCH; and challenges encountered. Qualitative Data Collection Field work: The local consultant conducted qualitative work in the field with his two assistants over ten days (September 19 – 28, 2016). In each of the three districts they conducted three FGDs, with each group having eight to ten participants: 1. Community Midwives (CMWs): To identify participants for this FGD, the local consultant selected every third name on the list of project-supported CMWs and asked the SMNC Project Manager to contact 11-12 women in each district in order to have 7-9 participants. The local consultant attempted to keep a balance between urban and rural CMWs but in the end, CMWs closest to the district capital were invited so that the women could return home the same day. 2. Lady Health Workers (LHWs), Lady Health Visitors (LHVs) and Lady Health Supervisors (LHS): Participants from the same geographic areas as the CMW participants and who had remained active in monitoring and coordination comprised this FGD group. 3. Female community members: Participants for this FGD included Lead Mothers from the Women Support Groups (WSGs) and women who had accessed services from the CMWs included in the CMW group. Focused Strategic Assessment: SMNC Project October 2016 Using semi-structured interview guides, the field team also conducted in-depth KIIs with four Department of Health (DoH) stakeholders in each district: - the District Health Officer (DHO) - the Medical Supervisor of the District Headquarters Hospital - the District Coordinator for the Maternal, Newborn and Child Health (MNCH) program - the Public Health Specialist who oversees the LHW program At the provincial level, the local consultant held KIIs with the following stakeholders: the Director General/Health Services for Balochistan Province; the Provincial MNCH Coordinator; the Provincial LHW Program Coordinator; the Chairperson of the Technical Working Group (also a member of the Provincial Steering Committee); and two other stakeholders, one each from the Technical Working Group and the Provincial Steering Committee. Interviews with Mercy Corps staff: The external consultant collected qualitative data through Skype interviews with project staff (Project Manager, Project Officer, Security Officer and the Monitoring, Evaluation and Learning Manager) and with two Mercy Corps/Pakistan senior managers, the Team Leader/South who has been very involved in the project and the Senior Director/Programs. It was not possible for her to conduct Skype interviews with other stakeholders due to connectivity issues outside Quetta and Islamabad. Limitations Since the project had recently completed several qualitative and quantitative community-level surveys in the target area, the team ensured that engagement with stakeholders focused primarily on information gaps. The field team could not visit remote areas for FGDs due to security issues and therefore CMWs and the LHWs/LHVs/LHSs were invited to the project’s district field offices. The FGDs with female community members were held in homes of Lead Mothers. In Kech the security situation and cultural sensitivities meant that male members of the team could not travel there; therefore, the female team member conducted interviews in this district. However, this did not appear to affect the quality and completeness of the data collected. Ethical Considerations The field team made it clear to all FGD participants that they were under no obligation to participate but if they did participate, anonymity and confidentiality were assured. Verbal informed consent from the participants was obtained. Where necessary, an interpreter assisted the team members. For each encounter, the team obtained permission for taking photographs for reports and presentations. Quality of Evidence for Results The SMNC project was well-documented and during the FSA the project staff provided a number of additional documents requested by the external consultant. The three Annual Reports prepared per USAID requirements were comprehensive with extensive annexes. Each of the major interventions such as mHealth had accompanying explanations and periodic assessments that provided a chronological description of how the intervention evolved. Focused Strategic Assessment: SMNC Project October 2016 The assessment team compared findings from the qualitative research with project documents, external assessments from firms such as PakVista Technologies, the Learning Agenda report produced by an independent consultant and documents produced during the project’s lifetime such as the Balochistan MNCH Strategy 2016-20. The external consultant also checked facts periodically with SMNC project staff when preparing this report. The cross-checking of data and the triangulation of findings both helped to ensure that the evidence used for drawing conclusions was valid and reliable. However, it should be noted that for information presented in project documents such as the quantitative data in the three Annual Reports and routine monitoring documents, the external consultant relied on the accuracy of the information at the source as there was no way to independently verify the information. Data Analysis The information collected from key informants was compiled and tabulated using MS Office software for each question and inputs were organized by themes and dimensions of program intervention. Important quotes and observations were identified and used to build the analysis. Data emerging from interviews was validated internally through triangulation with information from project documents, routine monitoring, and other sources gathered prior to and during the field work. The interpretations of triangulated thematic data were discussed with Mercy Corps district and country office teams for further modification and amendment. Information was synthesized by creating matrices around identified themes and the findings organized accordingly. Focused Strategic Assessment: SMNC Project October 2016 ANNEX III. DATA COLLECTION INSTRUMENTS AND WORK PLAN This Annex contains the following documents: A. Detailed Work Plan for Field Work B. Data Collection Instruments 1. FGD: CMWs 2. FGD: LHWs, LHVs, LHSs 3. FGD: Female Community Members 4. Semi-structured Guide for KII: Director General/Health Services (provincial level) 5. Semi-structured Guide for KII: Provincial Coordinator for LHW Program and Provincial Coordinator for MNCH Program (provincial level) 6. Semi-structured Guide for KII: For forum representatives - PSC members and TWG members (provincial level) 7. Semi-structured Guide for KII: District Health Officer (district level) 8. Semi-structured Guide for KII: District Coordinator for LHW Program and Public Health Specialist for MNCH (district level) 9. Semi-structured Guide for KII: Medical Superintendent for District HQ Hospital (district level) 10. Semi-structured Guide for KII: SMNC Project Staff and Mercy Corps/Pakistan Senior Managers Focused Strategic Assessment: SMNC Project October 2016 Detailed Work Plan for FSA Field Mission Date Name of Consultant Place Activities Travel from Travel to/ Arrived at Remarks Sept 18, 2016 Dr. Sohail Amjad Dr. Muslim Abbas Ms. Saima Zeb Faredi Islamabad Travel by air to Baluchistan Islamabad (11:05 am) Quetta (12:25 pm) Arrival at Quetta. Meeting with SMNC Team, discussion and finalization of detailed work plan, printing instruments, training of field team, skype call with international consultant, clarification of questionnaires and team orientation Sept 19, 2016 Dr. Muslim Abbas Ms. Saima Zeb Faredi Quetta (SMNC Office) FGD with CMWs Support required for language and interpretation Morning (9 am-12 noon) Dr. Muslim Abbas Ms. Saima Zeb Faredi FGD with LHWs, LHSs & LHVs Afternoon (3 pm -5 pm) Dr. Sohail Amjad Local note taker (Kamal Khan) Quetta Provincial Coordinator LHW Program (Dr. Noor Qazi) Support of note taker Morning (11:30 am-1 pm) Quetta Provincial Coordinator MNCH Program (Dr. Rafiq Mengal) Morning (9:30 am-11 am) Sept 20, 2016 Ms. Saima Zeb Faredi Quetta (SMNC Office) FGD with female community members Support required for language and interpretation Morning (9 am-12 noon) Focused Strategic Assessment: SMNC Project October 2016 Date Name of Consultant Place Activities Travel from Travel to/ Arrived at Remarks Sept 20, 2016 Dr. Sohail Amjad Dr. Muslim Abbas Quetta KI with 2 members PSC (Dr. Asfand WHO, Farooq Mengal and separately with 2 members of TWG (Prof Aisha Sadiqa, Dr. Tahira Kamal, Dr. Mukthiar Zheri) Support of note taker Morning (9 am-11 am) Dr. Sohail Amjad Dr. Muslim Abbas Quetta Morning (11:30 am-1 pm) Sept 21, 2016 Dr. Sohail Amjad Dr. Muslim Abbas Ms. Saima Zeb Faredi Quetta KI with DHO Quetta (Dr. Shah Jahan) Morning (9 am-11 am) Dr. Muslim Abbas Ms. Saima Zeb Faredi Quetta KI with District Coordinator LHW Program (Asst. Dist. Coordinator) Morning (11:30 am-1 pm) Dr. Sohail Amjad Local note taker (Kamal Khan) Quetta KI with PHS MNCH Program (Dr. Zaheer Kakar) Morning (11:30 am-1 pm) Sept 22, 2016 Dr. Muslim Abbas Ms. Saima Zeb Faredi Quetta KI with MS SPH Hospital, Quetta (Dr. Abdur Rehman) Morning (9:30 am-11 am) Dr. Sohail Amjad Local note taker (Kamal Khan) Quetta DGHS Baluchistan (Dr. Masood Qadir Nowsherwani) Morning (9:30 am-11 pm) Focused Strategic Assessment: SMNC Project October 2016 Date Name of Consultant Place Activities Travel from Travel to/ Arrived at Remarks Sept 23, 2016 Dr. Sohail Amjad Dr. Muslim Abbas Ms. Saima Zeb Faredi Preparation and refinement of transcripts with data editing and cleaning Sept 24, 2016 Dr. Sohail Amjad Dr. Muslim Abbas Quetta Travel by air Quetta (11:45 am) Gwadar (1:55 pm) Morning-Afternoon Ms. Saima Zeb Faredi Quetta Quetta (5:20 pm) Karachi (6:45 pm) Afternoon-Evening Dr. Sohail Amjad Dr. Muslim Abbas Gwadar FGD with CMWs, Gwadar Afternoon (3-5 pm) Sept 25, 2016 Dr. Sohail Amjad Gwadar FGD with female community members, Gwadar Morning Dr. Muslim Abbas Gwadar FGD with LHWs, LHSs and LHVs Gwadar Morning Ms. Saima Zeb Faredi Karachi Travel by air Karachi (6:00 am) Turbat/Kech (7:40 am) Morning Focused Strategic Assessment: SMNC Project October 2016 Date Name of Consultant Place Activities Travel from Travel to/ Arrived at Remarks Sept 25, 2016 Ms. Saima Zeb Faredi Kech FGD with CMWs, Kech Morning (11 am to 1 pm) Sept 26, 2016 Ms. Saima Zeb Faredi Kech KI with MS DHQ Hospital, Kech Morning (9:00 am to 10:30 am) Ms. Saima Zeb Faredi Kech FGD with female community members, Kech Morning (11 am to 1 pm) Ms. Saima Zeb Faredi Kech FGD with LHWs, LHSs & LHVs, Kech Afternoon (3 pm to 5 pm) Dr. Sohail Amjad Dr. Muslim Abbas Gwadar KI with DHO, Gwadar Morning (9 am-11 am) Dr. Sohail Amjad Dr. Muslim Abbas Gwadar KI with District PHS MNCH, Gwadar Morning (11:30 am-1 pm) Sept 27, 2016 Dr. Sohail Amjad Dr. Muslim Abbas Gwadar KI with District Coordinator LHW Program, Gwadar Morning Dr. Sohail Amjad Dr. Muslim Abbas Gwadar KI with MS DHQ Hospital, Gwadar Morning Focused Strategic Assessment: SMNC Project October 2016 Date Name of Consultant Place Activities Travel from Travel to/ Arrived at Remarks Sept 27, 2016 Ms. Saima Zeb Faredi Kech KI with DHO, Kech Morning (9 am-11 am) Ms. Saima Zeb Faredi Kech KI with District Coordinator LHW Program, Kech Morning (11:30 am-1 pm Sept 28, 2016 Dr. Sohail Amjad Dr. Muslim Abbas Gwadar Travel by air Gwadar (8:45 am) Karachi (10:25 am) Morning Ms. Saima Zeb Faredi Kech KI with District Public Health Specialist MNCH, Kech Morning (10 am-12 pm) Ms. Saima Zeb Faredi Turbat/ Kech Travel by air Turbat/ Kech (2:30 pm) Karachi (4:00 pm) Afternoon Sept 28, 2016 Dr. Sohail Amjad Dr. Muslim Abbas Ms. Saima Zeb Faredi Karachi Preparation and refinement of transcript for Kech and Gwadar Debriefing meeting with Ahmed Ullah, SMNC Program Manager Sept 29, 2016 Dr. Sohail Amjad Dr. Muslim Abbas Ms. Saima Zeb Faredi Karachi Travel to Islamabad from Karachi (3 pm flight) Focused Strategic Assessment: SMNC Project October 2016 Focused Strategic Assessment of Saving Mothers and Newborns in Communities (SMNC) Project FOCUS GROUP DISCUSSION WITH COMMUNITY MIDWIVES (At Community Level) FGD Study ID Number _________________ Date of FGD: ____/_____/____ Address: ___________________________________ Start time: ________ ___________________________________ Finish time: ________ District: ___________________________________ Language(s) of interview: _____________________________ Place of FGD: Health facility, CMW Station, Other (specify):______________________ FGD Facilitator’s Name: __________________________________ (Suggested participants of FGD: CMWs from rural SMNC locations (ensure 6-8 participants) (for office use only) Transcript prepared by: ______________________ Date prepared: _____/_____/_____ Transcript reviewed by: ______________________ Date reviewed: ____/____/_____ FACILITATOR NOTE: The discussion will take about two hours. If you don’t understand a question, please tell me. If you don’t know the answer to a question, tell me and we will go on to the next one. If you don’t want to answer a question, we will skip it. Is it OK to begin now? Please confirm your consent to participate in this interview/discussion. Focused Strategic Assessment: SMNC Project October 2016 Attach List of Participants: Yes  No  OVERALL QUESTIONS TO ANSWER IN FOCUS GROUP DISCUSSION: Reminder to facilitator: The purpose of this focus group is to determine the following: 1. Are CMW services available, accessible and acceptable? 2. What type and quality of support do the CMWs receive and how could it be improved? 3. Have the CMWs’ skills and capacities been strengthened? PART ONE: INTRODUCTION 1. Good morning. My name is ________ and I am a member of the study team to guide this discussion. First, I want to thank you all for taking the time to be with us today. 2. We will be discussing your thoughts and ideas about maternal and child health in general and perception about MNCH services in your community. We are learning about CMWs and other service providers’ role in the provision of these services. Our discussion will provide us with information that will help us improve these services. 3. Before we begin, I’d like to explain what a focus group is and then give you some information about this specific focus group. A focus group is like a discussion group. It’s a way of listening to people and learning from them. In a focus group, people are asked to talk with others about their thoughts and ideas about a subject. We are interested in hearing what you think and feel about each topic. There is no right or wrong answer. We expect that many of you will have different points of view. 4. Our discussion today will be about two hours. We’ll take a ten-minute break about halfway through. I’d like the discussion to be informal, so there’s no need to wait for me to call on you to respond. In fact, I encourage you to respond directly to the comments other people make. If you don’t understand a question, please let me know. I am here to ask questions, listen and make sure everyone has a chance to share. 5. We are interested in hearing from each of you, so if we seem to be stuck on a topic, I may interrupt you. If I do, please don’t feel bad about it, it’s just our way of making sure we get through all of the questions and everyone has a chance to talk. Helping me is my associate ____________. He/She will be taking notes and be here to assist me. Let’s begin. I want to find out some more about each of you, so let’s introduce ourselves and tell us your favorite food and sports/games. I’ll start. Focused Strategic Assessment: SMNC Project October 2016 Note to Facilitator: Do not correct misinformation about maternal and child health during the focus group. Tell participants that they will have the opportunity to have all of their questions answered at the end of our discussion. PART TWO: DISCUSSION QUESTIONS 1. Accessibility, Availability and Acceptability of CMW Services [Approx. 30 min] Q.1. What are the challenges you face in setting up your services in the community? Please elaborate. ⇒ PROBE: What are major difficulties encountered in encouraging women to access your services? What difficulties do you encounter in attending deliveries? Q. 2. Do you receive referrals from local MNCH service providers? ⇒ PROBE: Do LHWs or local TBAs (dais) refer cases? If not: why? ⇒ PROBE: Do CMWs get regular support and supervision from District Health Department to strengthen coordination with local health workers or hospitals to ensure service delivery? Examples? Q. 3. In case of emergency, how do you transfer maternity case or sick child to a larger health facility (hospital)? ⇒ PROBE: Availability of ambulance/transportation service ⇒ PROBE: Challenges and obstacles in transportation ⇒ PROBE: Are local Women Support Groups helpful to facilitate on financial support? Q. 4. Are you familiar with the health voucher scheme for clients who may have financial difficulties? ⇒PROBE: What is your experience with the voucher scheme? Who are the beneficiaries of the vouchers? ⇒PROBE: How many times have clients used the voucher system? ⇒PROBE: Do these vouchers compensate you for the services you provide? ⇒PROBE: Do you have suggestions for improving the voucher system? Focused Strategic Assessment: SMNC Project October 2016 2. Support for CMWs [Approx. 20 min.] Q. 5. Do you get regular supervision and monitoring? How often? ⇒ PROBE: Who provides Technical and Administrative Monitoring? ⇒ PROBE: When was the last supervision? Q. 6. Do you receive any equipment or other supplies from the SMNC project? How often? What equipment and/or supplies were given to you? Delivery Table, Autoclave, Delivery Kit, BP Apparatus, Others (specify): ____________________________________________ Q. 7. Do you have Women Support Groups in your community? What is their role? ⇒ PROBE: Do they work with you? How? ⇒ PROBE: What motivates the Women Support Groups to work with you and the community? ⇒ PROBE: Do you have any recommendations to help to improve collaboration with Women Support Groups? ⇒ PROBE: Do you think the Women Support Groups will continue after the project ends? Why/why not? Q. 8. Do you work with other community health service providers? Who? ⇒ PROBE: How do you work together? ⇒ PROBE: Who refers maternity cases to you? Q. 9. Who in your family and/or in your community supports you in your work? ⇒ PROBE: How? ⇒ PROBE: What additional support would you like to have from your community? (Be specific about type of support needed, who could provide.) 3. Capacity and Skill Development [Approx. 20 min.] Q. 10. Do CMWs receive further refresher training or skill development opportunity after deployment? Focused Strategic Assessment: SMNC Project October 2016 ⇒ PROBE: What were the topics of refresher courses or skill development? Do these refresher courses or skill development help improve your daily work? How? ⇒ PROBE: Do you think you need other trainings or skill development particular to the local community needs? What specific types? Q. 11. In addition to being able to provide maternal and child health services, what other abilities or capacities have you acquired as a result of becoming a CMW? ⇒ PROBE: What are you able to do now that you did not do before? Q.12. Has becoming a CMW changed your status within your family? In the community? How? Q. 13. What is the biggest benefit to you of becoming a CMW? Acknowledgements Thank you very much for coming here today. We appreciate your thoughts and ideas. They will be very helpful. Name Occupation Address Contact Signature Focused Strategic Assessment: SMNC Project October 2016 Focused Strategic Assessment of Saving Mothers and Newborns in Communities (SMNC) Project FOCUS GROUP DISCUSSION WITH LHWs, LHVs, and LHS (At Community Level) FGD Study ID Number _________________ Date of FGD: ____/_____/____ Address: ___________________________________ Start time: ________ ___________________________________ Finish time: ________ District: ___________________________________ Language(s) of interview: _____________________________ Place of FGD: Health facility, Other (specify)_____________ FGD Facilitator’s Name: __________________________________ (Suggested participants of FGD: LHS, LHVs, and especially LHWs from rural areas. (Ensure 7-8 participants) (for office use only) Transcript prepared by: ______________________ Date prepared: _____/_____/_____ Transcript reviewed by: ______________________ Date reviewed: ____/____/_____ FACILITATOR NOTE: The discussion will take about two hours. If you don’t understand a question, please tell me. If you don’t know the answer to a question, tell me and we will go on to the next one. If you don’t want to answer a question, we will skip it. Is it OK to begin now? Please confirm your consent to participate in this interview. Focused Strategic Assessment: SMNC Project October 2016 Attach List of Participants: Yes  No  OVERALL QUESTIONS TO ANSWER IN FOCUS GROUP DISCUSSION: Reminder to facilitator: The purpose of this focus group is to determine the following: • Are CMW services available, accessible and acceptable? • Are the beneficiaries satisfied with the quality of healthcare services provided by the CHWs? • What were the factors (social contexts, perceptions, socio-cultural barriers and obstacles, challenges, etc.) that impacted CMW uptake and performance in the target communities? • What is the role of LHSs, LHVs and LHWs in managing and supporting the CMWs? • What type of community support do the CHWs receive? Who collaborates with them? • What is the contribution of SMNC interventions in terms of service delivery and utilization of MNCH services by the community? • What is the potential of sustainability (defined as the majority of CMWs continuing their work)? PART ONE: INTRODUCTION Good morning. My name is ________ and I am a member of the study team to guide this discussion. First, I want to thank you all for taking the time to be with us today. We will be discussing your thoughts and ideas about maternal and child health in general and perception about MNCH services in your community. We are learning about CMWs and other service providers’ role in the provision of these services. Our discussion will provide us with information that will help us improve these services. Before we begin, I’d like to explain what a focus group is and then give you some information about this specific focus group. A focus group is like a discussion group. It’s a way of listening to people and learning from them. In a focus group, people are asked to talk with others about their thoughts and ideas about a subject. We are interested in hearing what you think and feel about each topic. There is no right or wrong answer. We expect that many of you will have different points of view. Focused Strategic Assessment: SMNC Project October 2016 Our discussion today will be about two hours. We’ll take a ten-minute break about halfway through. I’d like the discussion to be informal, so there’s no need to wait for me to call on you to respond. In fact, I encourage you to respond directly to the comments other people make. If you don’t understand a question, please let me know. I am here to ask questions, listen and make sure everyone has a chance to share. We are interested in hearing from each of you, so if we seem to be stuck on a topic, I may interrupt you. If I do, please don’t feel bad about it, it’s just my way of making sure we get through all of the questions and everyone has a chance to talk. Helping me is my assistant ____________. He/She will be taking notes and be here to assist me. Let’s begin. I want to find out some more about each of you, so let’s introduce ourselves and tell us your favorite food and sports/games. I’ll start. Note to facilitator: Do not correct misinformation about maternal and child health during the focus group. Tell participants that they will have the opportunity to have all of their questions answered at the end of our discussion. PART TWO: DISCUSSION QUESTIONS Accessibility, Availability and Acceptability of CMW Services [Approx. 30 min.] Q.1. What obstacles have CMWs faced in setting up their practice and attracting clients? Q.2. Who makes referrals to the CMWs? ⇒ PROBE: Do LHWs or local TBAs refer cases? If not: why not? ⇒ PROBE: How do LHWs/TBAs coordinate with CMWs for referral? ⇒ PROBE: Do CMWs provide feedback on outcome of referral? Q.3. Are clients/communities satisfied with the CMW’s services in terms of maternal and child health? ⇒PROBE: How do you know this? ⇒PROBE: What do clients/the community appreciate most about the CMWs? Support for CMWs [Approx. 20 min.] Focused Strategic Assessment: SMNC Project October 2016 Q.4. For LHWs only: What is your role in relation to the CMWs in your communities? ⇒PROBE: Please provide some concrete examples of how you support the CMWs. ⇒PROBE: Do you have suggestions for how LHWs and CMWs can work even more effectively together? Q.5. For LHS and LHVs only: Please describe your role/responsibility in the management of CMWs. ⇒PROBE: On average, how many CMWs do you manage? On average, how often do you see each CMW? ⇒PROBE: What are some of the reasons that LHSs and LHVs might not visit the CMWs as often as they should? [If the participants do not mention stipends or other incentives, ask: - What incentives do you receive for managing CMWs? - Are these incentives provided on a regular basis and in a timely fashion? ⇒PROBE: Please provide recommendations for improving the management of CHWs. Q.6. What is the role of Women Support Groups in relation to CMWs? Would you be able to give some recommendations to reinforce the collaboration between lead mothers in Women Support Groups and CMWs? Q.7. Are there other individuals or groups in the community that support the CMW in her work? Who? How? ⇒ PROBE: Can you provide some suggestions for how the community can better support the CMW? Project Impact and CMW Sustainability [Approx. 20 min.] Q.8. Have the CMWs made a positive impact on the health of mothers and newborns? PROBE: How do you know this? PROBE: Please provide some concrete examples of how a CMW has helped her community. Q.9. Has your participation in SMNC improved or added to your own technical and/or managerial skills? PROBE: What new skills have you acquired? PROBE: How have you benefitted from participating in SMNC? Q.10. Please describe any innovations SMNC has introduced and the benefits of these innovations. PROBE: Emergency transport? Health vouchers? Mobile technology for record-keeping? Business training form CMWs? Refresher training for CMWs? Q.11. Which of these innovations has the greatest possibility of being replicated in other areas? Why? Focused Strategic Assessment: SMNC Project October 2016 Q.12. What is the greatest barrier to sustainability (defined as the CMWs continuing to provide services after SMNC ends)? Q.13. Please provide a concrete recommendation for ensuring sustainability. Acknowledgements Thank you very much for coming here today. We appreciate your thoughts and ideas. They will be very helpful. Name Occupation Address Contact Signature Focused Strategic Assessment: SMNC Project October 2016 Focused Strategic Assessment of Saving Mothers and Newborns in Communities (SMNC) Project FOCUS GROUP DISCUSSION WITH FEMALE COMMUNITY MEMBERS AND SERVICE USERS (At Community Level) FGD Study ID Number _________________ Date of FGD: ____/_____/____ Address: ___________________________________ Start time: ________ ___________________________________ Finish time: ________ District: ___________________________________ Language(s) of interview: _____________________________ Place of FGD: Health facility, Other (specify):_____________ FGD Facilitator’s Name: __________________________________ (Suggested participants of FGD: Priority participants are CMW service users and the Women Support Group lead mothers or other members. Other participants may include local female social worker/activist, female school teacher, focal CBO/NGO female representative, female community elders. DO NOT INCLUDE LHW, LHV, etc. (Ensure 6-8 participants) (for office use only) Transcript prepared by: ______________________ Date prepared: _____/_____/_____ Transcript reviewed by: ______________________ Date reviewed: ____/____/_____ FACILITATOR NOTE: The discussion will take about two hours. If you don’t understand a question, please tell me. If you don’t know the answer to a question, tell me and we will go on to the next one. If you don’t want to answer a question, we will skip it. Is it OK to Focused Strategic Assessment: SMNC Project October 2016 begin now? Please confirm your consent to participate in this interview. Attach List of Participants: Yes  No  OVERALL QUESTIONS TO ANSWER IN FOCUS GROUP DISCUSSION: Reminder to facilitator: The purpose of this focus group is to determine the following: • Are CMW services available, accessible and acceptable? • What were the factors (social contexts, perceptions, socio-cultural barriers and obstacles, challenges, etc.) that impacted CMW uptake and performance in the target communities? • What type of community support do the CHWs receive? Who collaborates with them? • Are the beneficiaries satisfied with the quality of healthcare services provided by the CHWs? • What is the contribution of SMNC interventions in terms of service delivery and utilization of MNCH services by the community? PART ONE: INTRODUCTION Good morning. My name is ________ and I am a member of the study team to guide this discussion. First, I want to thank you all for taking the time to be with us today. We will be discussing your thoughts and ideas about maternal and child health in general and perception about MNCH services in your community. We are learning about the CMWs’ role in the provision of these services. Our discussion will provide us with information that will help us improve these services. Before we begin, I’d like to explain what a focus group is and then give you some information about this specific focus group. A focus group is like a discussion group. It’s a way of listening to people and learning from them. In a focus group, people are asked to talk with others about their thoughts and ideas about a subject. We are interested in hearing what you think and feel about each topic. There is no right or wrong answer. We expect that many of you will have different points of view. Our discussion today will be about two hours. We’ll take a ten-minute break about halfway through. I’d like the discussion to be informal, so there’s no need to wait for me to call on you to respond. In fact, I encourage you to respond directly to the comments other people make. If you don’t understand a question, please let me know. I am here to ask questions, listen and make sure everyone has a chance to share. Focused Strategic Assessment: SMNC Project October 2016 We are interested in hearing from each of you, so if we seem to be stuck on a topic, I may interrupt you. If I do, please don’t feel bad about it, it’s just our way of making sure we get through all of the questions and everyone has a chance to talk. Helping me is my assistant ____________. He/She will be taking notes and be here to assist me. Let’s begin. I want to find out some more about each of you, so let’s introduce ourselves and tell us your favorite food and sports/games. I’ll start. Note to Facilitator: Do not correct misinformation about maternal and child health during the focus group. Tell participants that they will have the opportunity to have all of their questions answered at the end of our discussion. PART TWO: DISCUSSION QUESTIONS 1. Perceptions about MNCH Issues and Services & Barriers to Health/Treatment Seeking. [Approx. 40 min.] Q1. What kind of maternal and child health problems are common in this community? ⇒PROBE: What are common maternal and newborn complications in your community? Q2. Where would you go to seek medical help or treatment for maternal and child health problems? ⇒PROBE: Personal physician, CMW, Govt. health facility, private hospital, Hakeem, local TBA? Etc. Q3. Why do you prefer to go to one type of provider as compared to others? ⇒ PROBE: What might prevent you from seeing a CMW or other medical practitioner? ⇒ PROBE: For treatment cost, travel or other barrier. ⇒ PROBE: Are there some women in the community who do not want to go to the CMW? Why? 2. Accessibility, Availability and Acceptability of CMW Services [Approx. 30 min.] Focused Strategic Assessment: SMNC Project October 2016 Q4. Have you or anyone in your family received antenatal, delivery or postnatal care from a CMW? ⇒ PROBE: How did you learn about the CMW and the services she provides? ⇒ PROBE: Is the CMW available whenever you or your family members need to visit her? Q5. What do you think of the services of CMWs? ⇒ PROBE: Acceptability of waiting time ⇒ PROBE: Cost of her services ⇒ PROBE: Quality of care provided by CMW ⇒ PROBE: Confidence in her skills Q6. Have the CMWs made a difference in your community? How? ⇒ PROBE: What concrete contributions have they made? Q7. In case of emergency, how do you transfer maternity or sick child to a larger health facility (hospital)? ⇒ PROBE: Availability of ambulance/transportation service ⇒ PROBE: Challenges and obstacles in transportation ⇒ PROBE: Are Women Support Groups helpful in facilitating financial support for transport? How? ⇒ PROBE: Have you heard of health vouchers? Do you think these vouchers are useful to get timely services? To remove financial barriers? 3. Support for CHWs [Approx 20 min.] Q8. Do you have Women Support Groups in your community? What is their role? ⇒ PROBE: Do they work with CMWs? How? ⇒ PROBE: Do you have any recommendations to help to reinforce collaboration between CMWs and Women Support Groups? ⇒ PROBE: Do you think the Women’s Support Groups will continue when SMNC ends? Why or why not? Focused Strategic Assessment: SMNC Project October 2016 Q9. Are there other individuals or groups that support the CMW in her work? Who? How? ⇒ PROBE: Can you provide some suggestions for how the community can better support the CMW? ⇒ PROBE: Would you be able to give some recommendations to improve the CMW’s work? Acknowledgements Thank you very much for coming here today. We appreciate your thoughts and ideas. They will be very helpful. Name Occupation Address Contact Signature Focused Strategic Assessment: SMNC Project October 2016 Focused Strategic Assessment of Saving Mothers and Newborns in Communities (SMNC) Project SEMI-STRUCTURED QUESTIONNAIRE FOR KEY INFORMANT INTERVIEW WITH DGHS KII Study ID Number _________________ Date of KII: ____/_____/____ Interviewee name: _____________________________ Start time: ________ Interviewee’s title: _____________________________ Finish time: ________ Language(s) of interview: ______________________ KII interviewer’s name: __________________________________ (Suggested participant: Director General/Health Services or his/her representative if DG not available.) (for office use only) Transcript prepared by: ______________________ Date prepared: _____/_____/_____ Transcript reviewed by: ______________________ Date reviewed: ____/____/_____ REASONS FOR INTERVIEW: • Have a better understanding of MNCH services management at the provincial level as it relates to CMWs, including barriers that may prevent the CMWs from working effectively. • Determine the factors (social contexts, perceptions, socio-cultural barriers and obstacles, challenges, etc.) that impacted CMW uptake and performance in the target communities. • Determine the contribution of project interventions in terms of service delivery and utilization of MNCH services by the communities. • Determine whether the Provincial Steering Committee (PSC) and the Technical Working Focused Strategic Assessment: SMNC Project October 2016 Group (TWG) have made contributions to MNCH services at the provincial and district levels. • Discuss the potential for sustaining any positive impacts of the SMNC project and for replication of any successful elements. Instructions for the Interviewer: Before the interview: Make an appointment with the DGHS through SMNCP/MC Managers and explain to him/her the objective of the Study and the reason for doing the interview. At the time of interview: a) Felicitate the DGHS and introduce yourself. Clearly explain to him/her the objective of the Study and the reason for doing the interview with him/her. Explain how he/she was selected for the interview. Also, request the DGHS to allow you enough time for conducting the interview, highlighting the importance of his/her views. Discourage prompting by other people in the room if their presence there is unavoidable. b) Ask the questions one by one and note down the replies clearly. If the DGHS seems not to clearly understand the question, explain further but avoid putting any leading question that suggests the answer in itself. Facilitate discussion, if any, to remain within the context of the interview. If you are not clear about the answer provided to you, request the respondent to repeat his/her view on that particular question. c) Before ending the interview session, reconfirm that all questions have been asked. Thank the respondent at the end of the session. After the interview: Organize the answers according to the questions. Collate all other views expressed by the DGHS that do not fall directly under any question in a separate section. Prepare a summary of the interview session with each respondent. PART ONE: INTERVIEW QUESTIONS A. Management of MNCH Services 1. Coordination: How do you ensure coordination among various provincial Program Service Providers in organizing MNCH healthcare services in your province? a. How do you see the role of the SMNC Project/MC at the district level (Quetta, Kech and Gowader) in improving MNCH outcomes? b. Can a CMW model such as the SMNC project play an important role in reducing maternal morbidity and mortality? How? Focused Strategic Assessment: SMNC Project October 2016 2. Emergency Referral/Transportation of Maternity: Does your provincial health system have an operational referral system in place for complicated maternity referral from primary and community based service outlets? Is it effective? a. Do you have enough resources to manage referral cases? (Probe for transportation mechanisms!) b. How do you see the role of SMNC project strategies and interventions in improving timely referral and transportation of complicated maternity? 3. Monitoring and Supervision: Could you describe the monitoring system for CMWs? a. Who is directly responsible for administrative and technical monitoring of CMWs? b. Are there any barriers that prevent regular monitoring? c. Do you get feedback on the CMWs’ administrative and technical monitoring from the districts? B. Policy and Planning 4. One activity of the SMNC project was to work with the DoH on the development of the provincial MNCH Strategy. Could you describe the process and the results? a. What is the status of the strategy at this point? b. Were you satisfied with the process and the results? Why or why not? c. Is your department preparing or have you already prepared PC 1 for ‘Integrated Primary Healthcare Services’ like Punjab and KP? If so what is its status, approved or still pending? 5. As part of the SMNC project, two forums were established: a Provincial Steering Committee (PSC) and a Technical Working Group (TG). Please describe what contributions, if any, these forums made to the SMNC project in particular and to the province’s MNCH program in general. C. Sustainability and Scaling Up Focused Strategic Assessment: SMNC Project October 2016 6. What are your views on the CMW model used in the SMNC project? a. What are strengths and weaknesses of the SMNC model? b. Would you recommend that the SMNC model be replicated in other districts of Balochistan and Pakistan? Why or why not? c. What are some of the challenges linked with such replication? How can these challenges be minimized? 7. Were there any innovations/new activities introduced by the SMNC project that could be scaled up? (Probe for use of cell phone technology for record keeping, use of a voucher scheme to reach poorer women, business training for CMWs, etc.) Acknowledgements Thank you very much for your time today. We appreciate your thoughts and ideas. Focused Strategic Assessment: SMNC Project October 2016 Focused Strategic Assessment of Saving Mothers and Newborns in Communities (SMNC) Project SEMI-STRUCTURED QUESTIONNAIRE FOR KEY INFORMANT INTERVIEWS WITH PROVINCIAL COORDINATORS FOR LHWs AND MNCH KII Study ID Number _________________ Date of KII: ____/_____/____ Interviewee name: _____________________________ Start time: ________ Interviewee’s title: _____________________________ Finish time: ________ Language(s) of interview: ______________________ KII interviewer’s name: __________________________________ Suggested participant: Provincial Coordinator/LHWs, Provincial Coordinator/MNCH (Separate interviews) (for office use only) Transcript prepared by: ______________________ Date prepared: _____/_____/_____ Transcript reviewed by: ______________________ Date reviewed: ____/____/_____ REASONS FOR INTERVIEW: • Have a better understanding of LHW and MNCH services management at the provincial level as it relates to CMWs, including barriers that may prevent the CMWs from working effectively. • Determine the factors (social contexts, perceptions, socio-cultural barriers and obstacles, challenges, etc.) that impacted CMW uptake and performance in the target communities. • Determine the contribution of project interventions in terms of service delivery and Focused Strategic Assessment: SMNC Project October 2016 utilization of MNCH services by the communities. • Discuss the potential for sustaining any positive impact of the SMNC project and for replication of any successful elements. Instructions for the Interviewer: Before the interview: Make an appointment with the Provincial Coordinators through SMNCP/MC Managers and explain to him/her the objective of the Study and the reason for doing the interview. At the time of interview: a) Felicitate the Provincial Coordinator and introduce yourself. Clearly explain to him/her the objective of the Study and the reason for doing the interview with him/her. Explain how he/she was selected for the interview. Also, request the Provincial Coordinator to allow you enough time for conducting the interview, highlighting the importance of his/her views. Discourage prompting by other people in the room if their presence there is unavoidable. b) Ask the questions one by one and note down the replies clearly. If the Provincial Coordinator seems not to clearly understand the question, explain further but avoid putting any leading question that suggests the answer in itself. Facilitate discussion, if any, to remain within the context of the interview. If you are not clear about the answer provided to you, request the respondent to repeat his/her view on that particular question. c) Before ending the interview session, reconfirm that all questions have been asked. Thank the respondent at the end of the session. After the interview: Organize the answers according to the questions. Collate all other views expressed by the Provincial Coordinator that do not fall directly under any question in a separate section. Prepare a summary of the interview session with each respondent. PART ONE: INTERVIEW QUESTIONS A. MNCH Services Management 1. Coordination of MNCH services: How do you ensure coordination of LHW [or MNCH] program and other private MNCH service providers such as the SMNC project? Focused Strategic Assessment: SMNC Project October 2016 2. In your opinion, what role can CMWs play in reducing maternal morbidity and mortality? 3. Emergency Referral/Transportation of Maternity: Does your health system have an operational referral system in place for complicated maternity referral from primary and community based service outlets? Is it effective? Please explain your responde. a. Do the LHWs/CMWs coordinate to manage maternal referral cases? If yes, ask approximately how many LHWs and CHWs coordinate: MAJORITY ABOUT HALF LESS THAN HALF b. Do you get feedback on the outcome of maternal referral from your staff? If so, what is the source of the information? c. Do CMWs receive feedback on the outcome of referrals they make? If yes, who provides the feedback to them? d. Do you have enough resources to manage referral cases? e. How do you see the role of SMNC project strategies and interventions in improving timely referral and transportation of complicated maternity? (Probe for community-based transportation mechanisms!) 4. Monitoring and Supervision: Do you have regular supervision of outreach staff, especially CMWs? a. Who is directly responsible for administrative and technical monitoring of CMWs? b. How often are CMWs supervised? Who supervises them? c. Do you have resources for CMW monitoring? Are the resources sufficient? d. Do you get feedback on CMW’s administrative and technical monitoring from her respective monitors? e. Do you collect data from the CMWs on numbers of antenatal contacts, deliveries attended, and/or referrals made? Is the data compiled and how do you use it? Can you show me the numbers? f. Overall, how would you rate the quality of care provided by the CMWs: Focused Strategic Assessment: SMNC Project October 2016 VERY GOOD – SATISFACTORY – BELOW AVERAGE B. Policy and Planning 5. One activity of the SMNC project was to work with the DoH on the development of the provincial MNCH Strategy. Could you describe the process and the results? a. What is the status of the strategy at this point? b. Were you satisfied with the results? Why or why not? c. Is your department preparing or have you already prepared PC 1 for ‘Integrated Primary Healthcare Services’ like Punjab and KP? If so, what is its status, approved or still pending? 6. As part of the SMNC project, two forums were established: a Provincial Steering Committee (PSC) and a Technical Working Group (TWG). Please describe what contributions, if any, these forums made to (a) the SMNC project in particular and (b) to the province’s MNCH program in general. For the TWG, you can probe using the following questions: a. Did the TWG help SMNC overcome implementation barriers? If yes: Please provide examples. b. Did the TWG provide technical support in the development of a communication strategy? If yes: Please describe the support. c. Did the TWG contribute to helping the CMWs establish their home￾based private practices? If yes: Please describe how. d. Did the TWG play a role in the development of training materials and IEC materials for the Women Support Groups? If yes: Please describe the support provided. C. Sustainability and Replicability Focused Strategic Assessment: SMNC Project October 2016 8. What are your views on the CMW model used in the SMNC project? 9. What are the strengths and weaknesses of the SMNC model for CMWs? 10. What, if anything, would you change in the model to make it more effective? 11. Would you recommend that the SMNC model be replicated in other districts of Balochistan and Pakistan? Why or why not? 12. What are some of the challenges linked with such replication? How can these challenges be minimized? D. Innovations and Capacity Building 13. Were there any innovations/new activities introduced by the SMNC project? Probe for: - use of cell phone technology for record keeping; - use of a voucher scheme to reach poorer women; - business training for CMWs; - Other? 14. Were any of these innovations effective? Which and why? 15. Were there any innovations/new activities introduced by the SMNC project that could be replicated? Probe for use of cell phone technology for record keeping, use of a voucher scheme to reach poorer women, business training for CMWs, etc. 16. Was any capacity-building done as a result of the SMNC project? If yes, please describe what was done and what the results are for your team. Acknowledgements Thank you very much for your time today. We appreciate your thoughts and ideas. Focused Strategic Assessment: SMNC Project October 2016 Focused Strategic Assessment of Saving Mothers and Newborns in Communities (SMNC) Project SEMI-STRUCTURED QUESTIONNAIRE FOR KEY INFORMANT INTERVIEWS WITH PSC, TWG REPRESENTATIVES KII Study ID Number _________________ Date of KII: ____/_____/____ Interviewee name: _____________________________ Start time: ________ Interviewee’s title: _____________________________ Finish time: ________ Language(s) of interview: ________________ KII interviewer’s name: __________________________________ Suggested participants: Active members of the Provincial Steering Committee (PSC) and the Technical Working Group (TWG) – a separate KII for each forum, ideally with at least 2 representatives per forum (for office use only) Transcript prepared by: ______________________ Date prepared: _____/_____/_____ Transcript reviewed by: ______________________ Date reviewed: ____/____/_____ REASONS FOR INTERVIEW: • Determine the impact of project interventions in terms of service delivery and utilization of MNCH services by the communities. • Learn about the potential for sustaining the positive impact of the SMNC project (e.g., capacity of CMWs trained within SMNC to successfully continue their activities post￾project). • Understand the potential for and barriers to replicating successful elements of the project. Focused Strategic Assessment: SMNC Project October 2016 • Determine whether the Provincial Steering Committee (PSC) and the Technical Working Group (TWG) have been able to facilitate the implementation of the SMNC project. Instructions for the Interviewer: Before the interview: Make an appointment with the PSC and TWG members through SMNCP/MC Managers and explain to him/her the objective of the Study and the reason for doing the interview. At the time of interview: d) Felicitate the interviewee and introduce yourself. Clearly explain to him/her the objective of the Study and the reason for doing the interview with him/her. Explain how he/she was selected for the interview. Also, request the interviewee to allow you enough time for conducting the interview, highlighting the importance of his/her views. Discourage prompting by other people in the room if their presence there is unavoidable. e) Ask the questions one by one and note down the replies clearly. If the interviewee seems not to clearly understand the question, explain further but avoid putting any leading question that suggests the answer in itself. Facilitate discussion, if any, to remain within the context of the interview. If you are not clear about the answer provided to you, request the respondent to repeat his/her view on that particular question. f) Before ending the interview session, reconfirm that all questions have been asked. Thank the respondent at the end of the session. After the interview: Organize the answers according to the questions. Collate all other views expressed by the interviewee that do not fall directly under any question in a separate section. Prepare a summary of the interview session with each respondent. PART ONE: INTERVIEW QUESTIONS Role of the PSC/TWG 1. Briefly explain the role/mandate of the PSC [or TWG] in relation to the SMNC project. a. How long have you served on the PSC [or TWG]? b. On average, how often do you meet? When was the last meeting? Focused Strategic Assessment: SMNC Project October 2016 2. Has the PSC [or TWG] influenced the direction SMNC has taken? If yes: How? Please provide some concrete examples of specific actions or recommendations the PSD [or TWG] made to guide SMNC or facilitate its implementation. Impact of SMNC 3. In your opinion what were the biggest successes, if any, of the SMNC project? 4. What activities/ideas did not succeed? Why? 5. TWG only: What impact has SMNC had on MNCH at the community level? How do you know this? 6. What impact has SMNC had on MNCH at the provincial level? How do you know this? 7. TWG only: Were there any innovations/new activities introduced by the SMNC project that were successful? Probe for use of: - cell phone technology for record keeping; - use of a voucher scheme to reach poorer women; - business training for CMWs; - refresher course; etc. 8. TWG only: Which of these innovations/new activities would you recommend for replication? [TWG only] Sustainability and Replicability 9. What are your views on the CMW model used in the SMNC project? (Probe for strengths, weaknesses, etc.) 10. What, if anything, would you change in the model to make it more effective? 11. Would you recommend that the SMNC model be replicated in other districts of Balochistan and Pakistan? Why or why not? 12. What are some of the challenges linked with such replication? How can these challenges be minimized? Policy and Planning 13. One activity of the SMNC project was to work with the DoH on the development of the provincial MNCH Strategy. Focused Strategic Assessment: SMNC Project October 2016 a. Were you involved with this process? b. Were you satisfied with the results? Why or why not? c. What is the status of the strategy at this point? 14. What do you consider the most important contribution(s) of the PSC [or TWG] to SMNC? And to MNCH in Balochistan Province? 15. Is there a role for the PSC [or TWG] after the project ends? If yes, please describe how you see this role. Acknowledgements Thank you very much for your time today. We appreciate your thoughts and ideas. Focused Strategic Assessment: SMNC Project October 2016 Focused Strategic Assessment of Saving Mothers and Newborns in Communities (SMNC) Project SEMI-STRUCTURED QUESTIONNAIRE FOR KEY INFORMANT INTERVIEWS WITH DHO KII Study ID Number _________________ Date of KII: ____/_____/____ Interviewee name: _____________________________ Start time: ________ Interviewee’s title: _____________________________ Finish time: ________ Language(s) of interview: ______________________ KII interviewer’s name: __________________________________ (Suggested participant: Executive District Officer-Health/District Health Officer (for office use only) Transcript prepared by: ______________________ Date prepared: _____/_____/_____ Transcript reviewed by: ______________________ Date reviewed: ____/____/_____ REASONS FOR INTERVIEW: • Have a better understanding of MNCH services management at the district level as it relates to CMWs, including barriers that may prevent the CMWs from working effectively. • Determine the factors (social contexts, perceptions, socio-cultural barriers and obstacles, challenges, etc.) that impacted CMW uptake and performance in the target communities. • Determine the contribution of project interventions in terms of service delivery and utilization of MNCH services by the communities. • Determine whether the District Health Forum has contributions to MNCH services at the district level. Focused Strategic Assessment: SMNC Project October 2016 • Discuss the potential for sustaining the positive impact of the SMNC project and for replication of successful elements. Instructions for the Interviewer: Before the interview: Make an appointment with the EDOH/DHO through SMNCP/MC Managers and explain to him/her the objective of the Study and the reason for doing the interview. At the time of interview: a. Felicitate the EDOH/DHO and introduce yourself. Clearly explain to him/her the objective of the Study and the reason for doing the interview with him/her. Explain how he/she was selected for the interview. Also, request the EDOH/DHO to allow you enough time for conducting the interview, highlighting the importance of his/her views. Discourage prompting by other people in the room if their presence there is unavoidable. b. Ask the questions one by one and note down the replies clearly. If the EDOH/DHO seems not to clearly understand the question, explain further but avoid putting any leading question that suggests the answer in itself. Facilitate discussion, if any, to remain within the context of the interview. If you are not clear about the answer provided to you, request the respondent to repeat his/her view on that particular question. c. Before ending the interview session, reconfirm that all questions have been asked. Thank the respondent at the end of the session. After the interview: Organize the answers according to the questions. Collate all other views expressed by the EDOH/DHO that do not fall directly under any question in a separate section. Prepare a summary of the interview session with each respondent. PART ONE: INTERVIEW QUESTIONS MNCH Services Management 1. Coordination of MNCH services: How do you ensure coordination of LHW [MNCH] program and other private MNCH service providers such as the SMNC project? Focused Strategic Assessment: SMNC Project October 2016 2. In your opinion, what role can CMWs play in reducing maternal morbidity and mortality? a. Emergency Referral/Transportation of Maternity: Does your district health system have an operational referral system in place for complicated maternity referral from primary and community based service outlets? Is it effective? 3. What is the transportation mechanism in the District Health system? Is it operational and effective? 4. Do you have enough resources to manage referral cases/ transportation? 5. How do you see the role of SMNC project strategies and interventions in improving timely referral and transportation of complicated maternity? Monitoring and Supervision: Do you have regular supervision of outreach staff, especially CMWs? 1. Who is directly responsible for administrative and technical monitoring of CMWs? 2. How often are CMWs supervised? Who supervises them? 3. Do you allocate resources for CMW monitoring? Are the resources sufficient? 4. Do you get feedback on CMW’s administrative and technical monitoring from her respective monitors? 5. Do you collect data from the CMWs on numbers of antenatal contacts, deliveries attended, and/or referrals made? How is this data used? 6. Overall, how would you rate the quality of care provided by the CMWs: VERY GOOD – SATISFACTORY – BELOW AVERAGE Policy and Planning 1. One activity of the SMNC project was to work with the DoH on the development of the provincial MNCH Strategy. Focused Strategic Assessment: SMNC Project October 2016 a. Were you satisfied with the results? Why or why not? b. What is the status of the strategy at this point? c. Is your department preparing or have you already prepared PC 1 for ‘Integrated Primary Healthcare Services’ like Punjab and KP? If so, what is its status, approved or still pending? d. Please describe what contributions, if any, the District Health Forum has made to the SMNC project in particular and to the MNCH program in general. Innovations and Capacity Building 1. Were there any innovations/new activities introduced by the SMNC project that could be scaled up? Probe for use of cell phone technology for record keeping, use of a voucher scheme to reach poorer women, business training for CMWs, etc. 2. Was any capacity-building done as a result of the SMNC project? If yes, please describe what was done and what the results are for your district health team. Sustainability and Replicability 1. What are your views on the CMW model used in the SMNC project? 2. What are Strengths and weaknesses of the SMNC model for CMWs? 3. What, if anything, would you change in the model to make it more effective? 4. Would you recommend that the SMNC model be replicated in other districts of Balochistan and Pakistan? Why or why not? 5. What are some of the challenges linked with such replication? How can these challenges be minimized? Acknowledgements Thank you very much for your time today. We appreciate your thoughts and ideas. Focused Strategic Assessment: SMNC Project October 2016 Focused Strategic Assessment of Saving Mothers and Newborns in Communities (SMNC) Project SEMI-STRUCTURED QUESTIONNAIRE FOR KEY INFORMANT INTERVIEWS WITH DISTRICT COORDINATORS FOR LHW, MNCH KII Study ID Number _________________ Date of KII: ____/_____/____ Interviewee name: _____________________________ Start time: ________ Interviewee’s title: _____________________________ Finish time: ________ Language(s) of interview: ______________________ KII interviewer’s name: __________________________________ Suggested participant: District Coordinator/LHWs, District Coordinator/MNCH (Separate interviews) (for office use only) Transcript prepared by: ______________________ Date prepared: _____/_____/_____ Transcript reviewed by: ______________________ Date reviewed: ____/____/_____ REASONS FOR INTERVIEW: • Have a better understanding of LHW and MNCH services management at the district level as it relates to CMWs, including barriers that may prevent the CMWs from working effectively. • Determine the factors (social contexts, perceptions, socio-cultural barriers and obstacles, challenges, etc.) that impacted CMW uptake and performance in the target communities. • Determine the contribution of project interventions in terms of service delivery and utilization of MNCH services by the communities. Focused Strategic Assessment: SMNC Project October 2016 • Discuss the potential for sustaining any positive impact of the SMNC project and for replication of any successful elements. Instructions for the Interviewer: Before the interview: Make an appointment with each District Coordinator through SMNCP/MC Managers and explain to him/her the objective of the Study and the reason for doing the interview. At the time of interview: a. Felicitate the District Coordinator and introduce yourself. Clearly explain to him/her the objective of the Study and the reason for doing the interview with him/her. Explain how he/she was selected for the interview. Also, request the District Coordinator to allow you enough time for conducting the interview, highlighting the importance of his/her views. Discourage prompting by other people in the room if their presence there is unavoidable. b. Ask the questions one by one and note down the replies clearly. If the District Coordinator seems not to clearly understand the question, explain further but avoid putting any leading question that suggests the answer in itself. Facilitate discussion, if any, to remain within the context of the interview. If you are not clear about the answer provided to you, request the respondent to repeat his/her view on that particular question. c. Before ending the interview session, reconfirm that all questions have been asked. Thank the respondent at the end of the session. After the interview: Organize the answers according to the questions. Collate all other views expressed by the District Coordinator that do not fall directly under any question in a separate section. Prepare a summary of the interview session with each respondent. PART ONE: INTERVIEW QUESTIONS MNCH Services Management 1. Coordination of MNCH services: How do you ensure coordination of LHW [or MNCH] program and other private MNCH service providers such as the SMNC project? Focused Strategic Assessment: SMNC Project October 2016 a. In your opinion, what role can CMWs play in reducing maternal morbidity and mortality? 2. Emergency Referral/Transportation of Maternity: Does your district health system have an operational referral system in place for complicated maternity referral from primary and community based service outlets? Is it effective? a. Do the LHWs/CMWs coordinate to manage maternal referral cases? If yes, ask approximately how many coordinate: MAJORITY ABOUT HALF LESS THAN HALF b. Do you get feedback on the outcome of maternal referral from your staff? If so, what is the source of the information? (1) How do you see the role of SMNC project strategies and interventions in improving timely referral and transportation of complicated maternity? (Probe for community-based transportation mechanisms!) 3. For District Coordinator/LHW only: Monitoring and Supervision: Do you have regular supervision of outreach staff, especially CMWs? a. Who is directly responsible for administrative and technical monitoring of CMWs? b. How often are CMWs supervised? Who supervises them? c. Do you have resources for CMW monitoring? Are the resources sufficient? d. Do you get feedback on CMW’s administrative and technical monitoring from her respective monitors? e. Do you collect data from the CMWs on numbers of antenatal contacts, deliveries attended, and/or referrals made? Is the data compiled and how do you use it? Can you show me the numbers? f. Overall, how would you rate the quality of care provided by the CMWs: Focused Strategic Assessment: SMNC Project October 2016 VERY GOOD – SATISFACTORY – BELOW AVERAGE Innovations and Capacity Building 1. Were there any innovations/new activities introduced by the SMNC project? Probe for: - use of cell phone technology for record keeping; - use of a voucher scheme to reach poorer women; - business training for CMWs; - Other? 2. Were any of these innovations effective? Which and why? 3. Were there any innovations/new activities introduced by the SMNC project that could be scaled up? (Probe for use of cell phone technology for record keeping, use of a voucher scheme to reach poorer women, business training for CMWs, etc.) 4. Was any capacity-building done as a result of the SMNC project? If yes, please describe what was done and what the results are for your team. Sustainability and Replicability 1. What are your views on the CMW model used in the SMNC project? 2. What are strengths and weaknesses of the SMNC model for CMWs? 3. What, if anything, would you change in the model to make it more effective? 4. Would you recommend that the SMNC model be replicated in other districts of Balochistan and Pakistan? Why or why not? 5. What are some of the challenges linked with such replication? How can these challenges be minimized? Acknowledgements Thank you very much for your time today. We appreciate your thoughts and ideas. Focused Strategic Assessment: SMNC Project October 2016 Focused Strategic Assessment of Saving Mothers and Newborns in Communities (SMNC) Project SEMI-STRUCTURED QUESTIONNAIRE FOR KEY INFORMANT INTERVIEW WITH MEDICAL SUPERINTENDENT OF DISTRICT HQ HOSPITAL KII Study ID Number _________________ Date of KII: ____/_____/____ Interviewee name: _____________________________ Start time: ________ Interviewee’s title: _____________________________ Finish time: ________ Language(s) of interview: ______________________ KII interviewer’s name: __________________________________ Suggested participant: Medical Superintendent of the District HQ Hospital (for office use only) Transcript prepared by: ______________________ Date prepared: _____/_____/_____ Transcript reviewed by: ______________________ Date reviewed: ____/____/_____ REASONS FOR INTERVIEW: • Have a better understanding of MNCH services management at the district HQ hospital as it relates to CMWs. • Determine the factors (social contexts, perceptions, socio-cultural barriers and obstacles, challenges, etc.) that impacted CMW uptake and performance in the target communities. • Discuss the potential for sustaining any positive impact of the SMNC project and for replication of any successful elements. Focused Strategic Assessment: SMNC Project October 2016 Instructions for the Interviewer: Before the interview: Make an appointment with the Medical Superintendent through SMNCP/MC Managers and explain to him/her the objective of the Study and the reason for doing the interview. At the time of interview: a. Felicitate the Medical Superintendent and introduce yourself. Clearly explain to him/her the objective of the Study and the reason for doing the interview with him/her. Explain how he/she was selected for the interview. Also, request the Medical Superintendent to allow you enough time for conducting the interview, highlighting the importance of his/her views. Discourage prompting by other people in the room if their presence there is unavoidable. b. Ask the questions one by one and note down the replies clearly. If the Medical Superintendent seems not to clearly understand the question, explain further but avoid putting any leading question that suggests the answer in itself. Facilitate discussion, if any, to remain within the context of the interview. If you are not clear about the answer provided to you, request the respondent to repeat his/her view on that particular question. c. Before ending the interview session, reconfirm that all questions have been asked. Thank the respondent at the end of the session. After the interview: Organize the answers according to the questions. Collate all other views expressed by the Medical Superintendent that do not fall directly under any question in a separate section. Prepare a summary of the interview session with each respondent. PART ONE: INTERVIEW QUESTIONS MNCH Services Management 1. Emergency Referral/Transportation of Maternity: Does your hospital have an operational referral system in place for complicated maternity referral from primary and community based service outlets? Is it effective? a. Do you have enough resources to manage referral cases? (Probe for transportation mechanisms!) Focused Strategic Assessment: SMNC Project October 2016 b. Do you get feedback on the outcome of maternal referral cases from your hospital service providers? If yes, what is the source of information? Do you share the outcome with referee or referral point? c. How do you see the role of SMNC project strategies and interventions in improving timely referral and transportation of complicated maternity? Innovations and Capacity Building 1. Were there any innovations/new activities introduced by the SMNC project that could be scaled up? (Probe for use of cell phone technology for record keeping, use of a voucher scheme to reach poorer women, business training for CMWs, etc.) [Note to Interviewer: MS may not be aware of this.] 2. Was any capacity-building done as a result of the SMNC project? If yes, please describe what was done and what the results are for your hospital team. [Note to Interviewer: MS may not be aware of this.] Sustainability and Replicability 1. What are your views on the CMW model used in the SMNC project? 2. What are the strengths and weaknesses of this CMW model? 3. What, if anything, would you change in the model to make it more effective? 4. Would you recommend that the SMNC model be replicated in other districts of Balochistan and Pakistan? Why or why not? 5. What are some of the challenges linked with such replication? How can these challenges be minimized? Acknowledgements Thank you very much for your time today. We appreciate your thoughts and ideas. Focused Strategic Assessment: SMNC Project October 2016 SEMI-STRUCTURED INTERVIEW GUIDE: THEMES FOR SMNC STAFF AND MERCY CORPS SENIOR MANAGERS 1. Length of time in position? Role in SMNC? 2. Highlights of the September 28 Donor Conference and next steps 3. Clarification on sustainability and exit strategies 4. Are there notable differences across the three Districts in terms of performance? In terms of potential for sustaining activities? 5. Learning Agenda report: What are the next steps for finalizing the report? For dissemination? For using the results? - Do you anticipate any important changes to the MNCH Strategy as a result of the Learning Agenda findings? 6. Have there been any collaboration/coordination/exchanges with other organizations/donors supporting CMW activities in Balochistan Province? - Example: Save the Children supported some CMWs in Gwadar District. 7. Has the security situation and/or the political situation affected implementation? If yes: How? When? Concrete examples? 8. The policy platforms: What have been their contributions? Will they continue? - The DHF seems to have been less successful. Why? What is the future of this platform? - What is the quality of collaboration with the DoH and other government bodies compared to other government departments Mercy Corps works with in other areas? 9. Quality of monitoring during the project: - Do they have records for each CMW: who visited her when? Results? Reports? - Joint monitoring visits? 10. LLs by Mercy Corps to apply to future projects - What worked/didn’t work - Any LLs applied to current projects or future designs? 11. The Road Map: what happened with this document? Any monitoring done? The following questions are directed more toward SMNC staff than senior managers: Focused Strategic Assessment: SMNC Project October 2016 12. Linkages, especially between LHWs and CHWs: Some “tension” comes out in field interviews, FGD transcripts. Cause(s)? Solutions? 13. Creating demand for the CMWs’ services: The emphasis on WSGs started relatively late. Field work indicates they may not be meeting regularly. - Other options for community support? For creating demand? 14. Quality of care: This issue was raised in the Learning Agenda report. Does SMNC have other data about the quality of care the CMWs are providing? - Example: Are there any records of the Technical Supervisory List results during monitoring visits? 15. mHealth: Status of the 2 components; potential for sustainability 16. Voucher scheme: Status 17. Referral mechanism: Update 18. Emergency transport: Status of MoUs with Edhi and Al-Falah? 19. Security concerns and impact, if any, on project 20. Are other organizations providing substantial support to the SMNC CMWs: DKT? 21. Other factors that affected implementation (positively, negatively) October 2016 ANNEX IV. SOURCES OF INFORMATION This annex contains: 1. The list of Focus Group Discussions and Key Informant Interviews 2. The documents consulted during the desk review October 2016 SMNC FSA Data Collection: List of FGDs and Key Informants No. Name Designation Place Provincial Level Key Informants 1 Dr. Masood Nowsherwani DGHS, Baluchistan Quetta 2 Dr. Rafiq Khan Mangal Provincial MNCH Coordinator Quetta 3 Dr. Noor Qazi Provincial LHW Program Coordinator Quetta 4 Prof. Dr. Aisha Sadiqa Chairperson Technical Working Group & member Provincial Steering Committee, Baluchistan Quetta 5 Dr. Asfand Yar Sherani Member Technical Working Group, Baluchistan Quetta 6 Mr. Farooq Mangal Member Provincial Steering Committee, Baluchistan Quetta District Level Key Informants (Quetta) 7 Dr. Shah Jehan Panezai District Health Officer, Quetta Quetta 8 Dr. Sher Ahmed Satahakzai District LHW Program Coordinator, Quetta Quetta 9 Dr Zaheer Ahmed Kakar District Public Health Specialist, MNCH Program, Quetta Quetta 10 Dr. Abdul Rehman Medical Superintendent, Sandman Provincial Hospital, Quetta Quetta Focus Group Discussions (Quetta) 11 CMWs SMNC-supported CMWs Quetta 12 LHWs, LHSs and LHVs CMW Coordinators, Administrative and Technical Monitors Quetta 13 Lead Mothers of WSG and beneficiary mothers Members of WSG and direct beneficiaries Quetta District Level Key Informants (Gwadar) 14 Dr. Sher Dil District Health Officer, Gwadar Gwadar 15 Dr. Abdul Wahid Deputy District Health Officer, Gwadar Gwadar October 2016 16 Dr. Abdul Latif Medical Superintendent, DHQ Hospital, Gwadar/ Current Charge District Coordinator LHW Program, Gwadar Gwadar 17 Dr. Ghulam Nabi Ex-District Coordinator LHW Program, Gwadar Gwadar 18 Dr. Shah Nawaz District Public Health Specialist, MNCH Program, Gwadar Gwadar Focus Group Discussions (Gwadar) 19 CMWs SMNC-supported CMWs Gwadar 20 LHWs, LHSs and LHVs CMW Coordinators, Administrative and Technical Monitors Gwadar 21 Lead Mothers of WSG and beneficiary mothers Members of WSG and direct beneficiaries Gwadar District Level Key Informants (Kech) 22 Dr. Sajjad Ahmed District Health Officer, Kech Kech 23 Dr. Mohammad Ikram District LHW Program Coordinator, Kech Kech 24 Dr. Abdul Hameed District Public Health Specialist, MNCH Program, Kech Kech 25 Dr. Mohammad Aslam Aazar Medical Superintendent, District Headquarters Hospital, Kech Kech Focus Group Discussions (Kech) 26 CMWs SMNC-supported CMWs Kech 27 LHWs, LHSs and LHVs CMW Coordinators, Administrative and Technical Monitors Kech 28 Lead Mothers of WSG and beneficiary mothers Members of WSG and direct beneficiaries Kech Skype Interviews with SMNC Staff and Mercy Corps/Pakistan Senior Managers 29 Ahmed Ullah SMNC Program Manager Skype 30 Dr. Shaihak Riaz SMNC Project Officer Skype 31 Dr. Saeedullah Khan Team Leader South Skype October 2016 LIST OF DOCUMENTS REVIEWED A. SMNC Documents 1. MC Pakistan CSHGP FY12 Final Proposal 2. Revised Performance Management Plan 3. Project Data Form 4. Consultative Workshop for MNCH Strategy – Workshop Report Final 5. Revised Strategic Workplan (December 2013) 6. Knowledge, Practice and Coverage Baseline Survey Report 7. Lot Quality Assurance Sampling Survey Report (April 2015) 8. Routine monitoring reports a. CMW Services Data (Dec. 2013-Dec. 2014) b. CMW Services Data (Jan-Dec 2015) c. Quarterly M&E Report: October-December 2015 d. CMW Services and Indicators Progress Updated Last 3 Quarters e. Field Monitoring Reports (from field visits conducted by Team leader/South, SMNC Program Manager, SMNC Project Officer and MEL Manager) 9. Case for Midcourse Corrections – Revised (July 2015) 10. Annual Reports to USAID: a. Mercy Corps’ First Annual Report for: Saving Mothers and Newborns in Communities b. Mercy Corps’ Second Annual Report for: Saving Mothers and Newborns in Communities c. Mercy Corps’ Third Annual Report for: Saving Mothers and Newborns in Communities 11. SMNC Risk Management Plan Updated September 2015 12. Summary of Security Incidents in SMNC Districts (October 2014-September 2015) 13. Extension proposal - Description of Activities (April 2016 to March 2017) 14. Work Plan (USAID-CSHGP and Scottish Government Extension Grant): April 2016 – March 2017 32 Dr. Farah Naureen Senior Director/Programs Skype 33 Mazhar Iqbal MEL Manager Skype + e-mail 34 Ghulam Haider Security Officer Skype October 2016 15. Voucher Scheme Mechanism Final (with annexes) B. Learning Agenda 16. Learning Agenda Methodology 17. Final Draft Learning Agenda Report (30 September 2016) C. MHealth Component – from PakVista Technologies 18. Saving Mothers and Newborns in Communities: Lessons Learned (May 5, 2016) 19. SMNC Assessment Report (August 22, 2016) 20. mHealth Assessment Final Report (September 26, 2016) 21. Journey of a 1,000 Miles (published in IEEE Pulse) D. Pakistan Ministry of Health 22. National MNCH Program P-1: 2006-2012 23. National MCH Policy and Strategic Framework: 2005-2015 24. Balochistan MNCH Strategy: 2016-20 Final (April 2016) E. Other Background Materials 25. Strategic road Map for Improving SBA Through CMWs: Balochistan (submitted by RH￾AID, 17 December 2013) 26. A Promise Renewed: Pakistan Updates 27. Acting on the Call: Ending Preventable Child and Maternal Deaths Report (2016) 28. Every Newborn: An Action Plan To End Preventable Deaths (WHO 2014) ANNEX V. DISCLOSURE OF ANY CONFLICTS OF INTEREST DISCLOSURE OF ANY CONFLICTS OF INTEREST Name Kathy Tilford Title International Consultant Organization None Evaluation Position Team Leader Evaluation Award Number (Contract or other instrument) N/A USAID Project Evaluated (Include project name(s), implementer name(s) and award number(s), if applicable) Saving Mothers and Newborns in Communities Mercy Corps CA No. AID-OAA-A-12-00093 Ihave real or potential conflicts of interest to disclose. Icertify (1) that Ihave completed this disclosure form fully and to the best of my ability and (2) that Iwill update this disclosure form promptly if relevant circumstances change. Signature Date October 16, 2016 October 2016 ANNEX VI. ASSESSMENT TEAM MEMBERS, ROLES AND TITLES NAME TITLE ROLE Kathy Tilford External Consultant 1. Serve as FSA Team Leader 2. Conduct desk review. 3. Lead the development of the qualitative research methodology and design of tools. 4. Interpret results and draw conclusions. 5. Prepare final FSA report and PowerPoint presentation. Dr. Sohail Amjad Local Consultant and Field Team Leader 1. Assist with development of methodology and preparation of qualitative data collection instruments. 2. Serve as field team leader for the qualitative data collection. 3. Provide input into the Focused Strategic Assessment report. Dr. Muslim Abbas Senior Research Associate Conduct qualitative data collection and prepare transcripts. Ms. Saima Zeb Faredi Senior Research Associate Conduct qualitative data collection and prepare transcripts. FINAL EVALUATION Saving Mothers and Newborns in Communities (SMNC) Results of a Focused Strategic Assessment Conducted in September-October 2016 SMNC Project in Brief  Goal: Improve maternal and newborn health status, especially for poor and marginalized women of Balochistan  Strategic Objective: Increased use of quality essential maternal and newborn care provided through private-sector community midwives (CMWs)  Timeframe: September 2012 – September 2016 + 6-month extension through March 2017 SMNC Project in Brief – cont.  Location: Quetta, Gwadar and Kech District of Balochistan Province  Beneficiaries: 382,515  Principal partners: Government of Balochistan, Department of Health (DoH) and Mercy Corps  Supported by: USAID with additional funding from the Scottish Government Intended Results SMNC aimed to pilot interventions that would result in:  Increased availability of maternal and newborn health services at the community level  Improved knowledge of behaviors and demand for services  Improved access to emergency transport in remote communities  Improved policy environment based on evidence from the Learning Agenda activity Purpose of the Focused Strategic Assessment 5 questions from the Terms of Reference:  Did SMNC accomplish the 4 intended results?  What were the key strategies and factors that influenced outcomes?  Which elements of SMNC are likely to be sustained or scaled up?  What are the stakeholder perspectives on the project?  Has SMNC been successful in strengthening community￾based maternal and newborn care provision? FSA Focus on Learning Agenda Themes In addition to the 5 questions in the Terms of Reference, the FSA focused on the 4 themes examined in the Learning Agenda activity (conducted by an external consultant in 2016):  More effective recruitment and deployment of CMWs to improve retention  Promoting financial self-sustainability for CMWs, who are private sector  Ensuring that CMWs provide quality care  Streamlining reporting via mHealth technology FSA Methodology Two main components:  Comprehensive document review, including the Learning Agenda report  Qualitative data collection via key informant interviews and focus groups: CMWs, DoH personnel at district and provincial levels, Lady Health Workers/Visitors/Supervisors, beneficiaries, SMNC staff, Mercy Corps senior managers, etc.  Used common themes across tools to triangulate qualitative data collection: support for CMWs, monitoring and supervision, sustainability, referral system, challenges, etc.  Continual fact-checking with SMNC staff Findings  Increased access to essential maternal and newborn services for families in underserved areas  Successfully tested and documented interventions to assist the DoH to improve CMW program design, policies and procedures  Largely accomplished the 4 intended results, especially:  Increased availability of maternal and newborn health services at the community level  Improved policy environment through provincial forums (Technical Working Group and Provincial Steering Committee) and and the new five-year MNCH Strategy Potential for Sustainability and Scaling Up  95 midwives trained and equipped, the majority of whom continue to provide essential services to women with few other options for skilled birth attendance  A clinical skills refresher training course endorsed by the Pakistan Nursing Council, to be incorporated into all midwifery training schools in Balochistan and eventually in Pakistan  A business skills course to be included in the standard 18-month midwifery training  An mHealth application for improved data collection and reporting ready for DoH adaptation into the MNCH MIS Sustainability and Scaling Up – cont.  A new approved and costed five-year MNCH strategy (2016- 2020) for Balochistan with a major component for CMWs  Increased demand for MNCH services at the community level  A three-module set of materials for Women Support Groups, available in four languages  Two institutionalized provincial policy platforms, the Technical Working Group and the Provincial Steering Committee Key Factors Influencing Results Positive factors:  Excellent working relationship between the GoB, represented primarily by the DoH, and Mercy Corps  Active engagement of the 2 policy forums: facilitated project implementation and successfully advocated for changes that will have a lasting impact on the CMW program in Balochistan  SMNC responded directly to DoH priorities and requests  The SMNC design was multi-faceted and took into account the key aspects of a successful CMW program: recruitment, deployment, financial sustainability, quality of care, supportive supervision, community support, referrals and creating demand for CMW services. Key Factors Influencing Results – cont. Constraints:  Widely-scattered population, limited mobility and relatively small catchment areas = fewer clients, affecting financial self￾sustainability for CMWs  Ineffective human resource management in some areas (vacancies and absenteeism), affecting support for CMWs  Inadequate GoB financial resources for CMW stipends, incentives for supervisors, vehicles, etc.  Patriarchal society norms can discourage women working independently and women’s mobility. Challenges Remaining  Identifying enough qualified and interested women from rural areas who meet the minimum requirements to become a CMW  Providing adequate support to CMWs through regular supportive supervision and better community support  Ensuring that the CMWs have sufficient monetary incentives to continue providing services  Improving linkages between the MNCH program and the CMW program Challenges Remaining – cont.  Creating more harmonious working relationships between TBAs, CMWs and LHWs  Establishing a reliable transportation system for referrals  Improving the referral mechanism between CMWs and secondary facilities  Overcoming budgetary constraints within the Government of Balochistan, including allocating funding for the new MNCH Strategy Conclusion With appropriate selection, training and continued support from the Department of Health and the community, the CMW can acquire the necessary skills, confidence and community status to provide lifesaving maternal and newborn care, especially in rural areas with widely-scattered populations and limited health services. Principal Recommendations Learning Agenda Follow-up 1. Prepare a briefing paper that summarizes the main findings and recommendations from the Learning Agenda exercise; ensure wide-spread dissemination and follow-up for key recommendations, especially those concerning: i) improved selection, deployment and retention ii) adequate remuneration and incentives for CMWs Principal Recommendations – cont. Support for CMWs 2. Create a more collaborative and harmonious working relationship between CMWs and LHWs. 3. Reinforce orientations for family members of CMWs and for their communities to create sustained support for the CMWs. 4. Consider integrating CMWs into the DoH to ensure adequate financial compensation, supervision and support. 5. Investigate creative options for accommodating married women with young children who qualify for CMW training Principal Recommendations – cont. Ensuring CMWs Provide Quality Care 6. Commit resources to continue supportive monitoring and supervision of CMWs. 7. Conduct refresher training for LHSs on how to provide effective administrative supervision and on-the-job mentoring for CMWs. 8. Invest in regular refresher training on technical themes and business skills for CMWs. This could include on-the-job training and mentoring during routine monitoring and supervision. 9. Make quality staffing of midwifery schools a priority. Principal Recommendations – cont. Balochistan MNCH Strategy 2016-20. Sustaining and Scaling Up SMNC Achievements 10. Continue with the plan to integrate the mHealth application into the MNCH MIS. 11. Disseminate the voice messages in four languages developed under SMNC. CMWs and LHWs can use these messages for health promotion and demand generation. 12. Document the voucher scheme activity and share with DoH colleagues. 13. Continue the policy forums (Provincial Steering Committee and Technical Working Group), especially for overseeing the implementation of the Balochistan MNCH Strategy 2016-20. 14. Mobilize resources and partners for the implementation of the Principal Recommendations – cont. Dissemination of FSA Findings 15. Consider publishing the FSA findings in the Global Health: Science and Practice Journal Saving Mothers and Newborns in Communities LEARNING AGENDA Improving Maternal and Newborn Services: Strengthening Community Midwives in Balochistan Dr. Saima Hamid Report Writer Associate Professor, Health Services Academy, Islamabad, Pakistan This document is made possible by the generous support of the American people through the United States Agency for International Development (USAID). The contents are the responsibility of Mercy Corps and do not necessarily reflect the views of USAID or the United States Government 2 Table of Contents LIST OF TABLES.......................................................................................................3 LIST OF FIGURES.....................................................................................................4 ABBREVIATIONS......................................................................................................5 EXECUTIVE SUMMARY............................................................................................6 INTRODUCTION ........................................................................................................9 SUB STUDY 1..........................................................................................................15 METHODOLOGY ............................................................................................................15 RESULTS........................................................................................................................19 SUB STUDY 2..........................................................................................................42 METHODOLOGY ............................................................................................................42 RESULTS........................................................................................................................43 SUB STUDY 3..........................................................................................................57 METHODOLOGY ............................................................................................................57 RESULTS........................................................................................................................59 SUB STUDY 4..........................................................................................................86 METHODOLOGY ............................................................................................................86 RESULTS........................................................................................................................86 REFERENCES .........................................................................................................93 ANNEXURES ...........................................................................................................95 3 LIST OF TABLES Sub Study 1 Table 1: National, Provincial and District MNCH Data.................................................122 Table 2: List of Respondents in In-depth Interviews ......................................................16 Table 3: Background Characteristics of participants in FGDs........................................18 Table 4: Relation of demographic variables with location of work preference in Non-Intervention Group.............................................................................................................................35 Table 5:: Job Attributes and their significance level.......................................................36 Table 6: Weighted Ranking of Preferences ...................................................................37 Table 7: Potential Retention Packages..........................................................................38 Sub Study 2 Table 8: Demographic Information of Respondents.......................................................44 Table 9: Cash Flow Data - Intervention .........................................................................45 Table 10: Cash Flow Data - Non-Intervention............................................................4747 Table 11: Average costs of Setup and Services (Intervention)......................................48 Table 12: Average costs of Setup and Services (Non-Intervention) ..............................49 Table 13: -Profitable services provided by CMWs .........................................................50 Sub Study 3 Table 14: General characteristics of CMWs ..................................................................59 Table 15: Knowledge of CMWs regarding antenatal care..............................................61 Table 16: Knowledge of CMWs regarding Natal Care .................................................622 Table 17: Knowledge of CMWs regarding new born care............................................633 Table 18: Knowledge of CMWs regarding postnatal care............................................656 Table 19: Knowledge score of CMWs............................................................................68 Table 20: Attitude of CMWs towards Antenatal care .....................................................71 Table 21: Attitude of CMWs towards Natal care ............................................................71 Table 22: Attitude of CMWs towards postnatal care......................................................71 Table 23: General Attitude of CMWs .............................................................................72 Table 24: Attitude score of CMWs .................................................................................73 Table 25: Practices of CMWs towards Antenatal care...................................................74 Table 26: Practices of CMWs towards Natal care..........................................................75 Table 27: Practices of CMWs towards postnatal care ...................................................76 Table 28: General practice of CMWs.............................................................................77 Table 29: Practice score of CMWs ................................................................................77 Sub Study 4 Table 30: Demographic Information of Respondents.....................................................78 4 LIST OF FIGURES Study 1 Figure 1: Methodology Flow Chart.............................................................................................................. 15 Figure 2: Analysis Process from main theme to categories ........................................................................ 20 Figure 3: Analysis process moving from main theme to sub-themes to categories ..................................... 79 5 ABBREVIATIONS ANC ANM Antenatal Care Auxiliary Nurse Midwife BDoH Balochistan Department of Health CMW Community Midwife DoH Department of Health EmOC Emergency Obstetric Care DHF DHIS FPP District Health Forum District Health Information System Family Planning Program IMR Infant Mortality Rate LHS Lady Health Supervisor LHV Lady Health Visitor LHW Lady Health Worker MMR Maternal Mortality Rate MDGs Millennium Development Goals MNCH Maternal, Newborn and Child Health MPW NMR Multi-purpose Worker Newborn Mortality Rate SBA Skilled Birth Attendant SMNC Saving Mothers and Newborns in Communities TBA Traditional Birth Attendant PC-1 Planning Commision-1 PNC PNC PPHI Pakistan Nursing Council Postnatal Care People’s Primary Healthcare Initiative U5MR WB WHO WMO Under 5 Mortality Rate The World Bank World Health Organization Women Medical Officers WSG Women Support Group 6 EXECUTIVE SUMMARY According to the Pakistan Demographic and Health Survey (PDHS), Balochistan province has the highest maternal mortality rate (MMR) in Pakistan. In 2006 the Government of Pakistan set-up the national Maternal, Neonatal and Child Health (MNCH) program. This program implemented a number of initiatives, including the introduction of Community Midwives (CMWs). The MNCH program intended to recruit CMWs from rural and underserved areas of Pakistan, with the aim to overcome the challenge women face in accessing skilled care before, during and after delivery. Recruited CMWs in Balochistan would be trained for 18 months and serve a designated catchment population of 5000 through a private health care model. The Balochistan Department of Health (BDoH), in collaboration with Mercy Corps, have been delivering the Saving Mothers and Newborns in Communities (SMNC) program that aims to strengthen the CMW cadre to enable them to become long-term, financially sustainable, skilled and quality maternal and child healthcare providers. The SMNC program was funded by USAID and the Scottish Government to a total of $2.32m and delivered in three districts in Balochistan. The program provided CMWs with clinical refresher courses, supportive supervision, business skills training and small grants of standardized equipment. To support the program, a mobile phone application for reporting health data was introduced, Women Support Groups (WSG) were activated to deliver peer education on health and a Mamta Fund was distributed to provide emergency transport to pregnant women. The results and learning from the project were to be incorporated into the Government of Balochistan’s Planning Commision-1 (PC-1) for the 2015- 2020 MNCH Program. From March to June 2015, the SMNC program underwent a midcourse correction where the program’s Learning Agenda was revised. The new Learning Agenda initiated research that comprised of four sub-studies that took place in both program intervention and non-intervention areas. The aim of the research was to document the contributions of the program to increasing women’s access to skilled care before, during and after delivery and to gather and share lessons learnt. The four research questions for the Learning Agenda were: 1. How can the BDoH improve its selection process to effectively recruit and deploy CMWs in underserved areas? 2. How can CMWs become financially self-sustaining while serving the needs of the poorest of the poor? 3. Do CMWs offer quality care? How? 7 4. How can the BDoH streamline CMW reporting using cell phone technology and expand mHealth in the province? Four sub-studies were conducted to respond to the four Learning Agenda research questions, as follows: Sub-study 1 aimed to understand how CMWs could be effectively recruited and then retained in the CMW system after deployment. It also aimed to identify CMW’s motivation for working in rural and resource-constrained areas and to propose appropriate, cost effective incentive packages to attract CMWs to austere environments and to ensure their retention. A qualitative study was followed by a Discrete Choice Experiment (DCE) survey. The findings from sub-study 1 identified that poor CMW retention was due to a lack of quality CMW training provision from the MNCH program and the BDoH, an inadequate CMW deployment strategy and harsh environmental conditions, particularly in rural areas. The findings from the DCE survey suggested that there might be a number of effective strategies to attract and retain CMWs in rural and remote areas of Balochistan. While a stipend was important to those interviewed, it was seen that when a combination of other valued interventions was offered, a stipend became less important. The study showed that preferences were given to a transport allowance, support in refresher training, housing facility provision (basic amenities or allowance) and supervision (through government Lady Health Supervisors (LHSs)/Lady Health Visitors (LHVs)/Woman Medical Officer (WMOs)) over the supply of medicines and equipment, and good schooling for their children. The right combination of incentives can likely retain CMWS up to 89%. Sub-study 2 identified the financial successes and failures of CMWs delivering private health care by undertaking an expenditure and investment assessment of CMWs. It was evident that the majority of the CMWs had been unable to sustain themselves financially, particularly those in non￾intervention areas. CMWs needed continued financial support from the program, as well as access to business skills and management training. Furthermore, the study proved that it was essential to integrate CMWs into the public sector health system of Balochistan for their survival as community based maternal healthcare providers. Sub-study 3 assessed the knowledge, attitudes and practices of CMWs, as well as client satisfaction with the care provided by CMWs. In intervention and non-intervention areas, the study showed that the overall knowledge of CMWs was poor. However, the practices of CMWs in 8 intervention areas were significantly better than those in the non-intervention areas. Overall communities considered CMWs acceptable community-based trained health care providers. Sub-study 4 explored the potential gaps in the mHealth reporting system adopted by CMWs in the intervention areas and how the system could be improved, sustained and expanded in the province by the BDoH. Despite the challenges in its use, most CMWs were using the technology to report routine progress in patients’ antenatal check-ups. The BDoH, although supportive of the technology, had limited systems in place to collect, analyse and report on data generated by the CMWs in intervention areas. 9 INTRODUCTION Globally, an estimated 3.6 million newborns and 360,000 mothers die every year. Of these, maternal health complications contribute to 1.5 million neonatal deaths in the first week of life and 1.4 million stillbirths, suggesting that a major gap of intervention exists around childbirth and in the early postnatal period, a time at which mothers and babies are most at risk1 . This situation is more alarming in low-income countries where 1 maternal death occurs in 44 as compared to 1 maternal death in 3300 in high-income countries2, 3. Common reasons for maternal deaths are hemorrhage, sepsis, obstructed labor and unsafe abortion etc. Prevention, detection, and timely management of these complications are a primary health care need of any country. Reviews elicit that acceptance and recognition of the midwifery model of care and delivery by a skilled birth attendant, significantly has direct implications on the maternal mortality ratio. Different initiatives have been taken globally to increase levels of skilled birth attendance for reducing maternal deaths. In countries where home based deliveries are preferred or there is lack of capacity in facilities, the focus has been on deploying community-based skilled birth attendants for providing domiciliary care. Mixed results are depicted by these programs. Sri Lanka introduced public health midwives in the early 1900s, and reduced its maternal mortality rate (MMR) from 2000 to 31/100,000 live births between 1930 and 20114-7. Thailand and Malaysia reduced their MMRs from 425 and 275/100,000 live births respectively in 1960 to <30 in 2010 using the same strategy8 . This showed that attendance at delivery by well-trained public health midwives with the back up of emergency obstetric care (EmOC) services helped in the reduction of maternal mortality, even in resource-poor settings. In contrast, implementation of a community midwifery program in Indonesia over 30 years has had disappointing results, stagnating at 220 deaths per 100,000 live births9 . Afghanistan’s community midwifery program has produced little change in its MMR; currently at 1400/100,000 live births8, 10. As with the other developing countries, India also faces a dearth of skilled birth attendants (SBA). In the rural healthcare system, the auxiliary nurse midwife (ANM) is the frontline (female) health worker and is the central focus of all reproductive and child health programs. With changes in program priorities, the role and the capacity of ANM have changed substantially over the years. The role of ANM has transformed to a multipurpose worker (MPW) who is mostly involved in implementing national health programs in contrast to the ANMs of the 1960s who were providing delivery and basic curative services. The transformation of this role had direct implications for maternal health and provision of maternal health services in India. The quality of nursing and 10 midwifery education has also deteriorated over a period of time, which has partly contributed to the shortage of SBAs at health facilities11. In a multi-country study in India, Malaysia, Sri Lanka, and Indonesia, the dramatic reduction in maternal deaths was reported through a midwifery care model. The model had a universal acceptance and focused on improving communication and collaboration between traditional midwives who provided home care, with midwives with formal training. Support from gynecologists when necessary, and participation of women in their own care was an additional integral part of the intervention12. In Iran, community orientation in medical sciences has been discussed for many years, but community orientation in midwifery needs more consideration due to the importance of maternal and child health. In a survey conducted on all the midwives working in the provinces of Kerman and Shahroud, Iran, in 2010–2011, the community-based midwifery knowledge of the midwives of Kerman was at a low level, and for midwives of Shahroud, it was at a moderate level13, 14. Cambodia is a recent example where maternal mortality declined remarkably from 472 per 100,000 live births in 2000-2005 to 206 in 2006-2010. The maternal mortality rate was reduced by modifying the underlying social and structural factors related to health, for example, improving girls education, improving roads, improving access to health information, and increasing the level of communication and coordination within the health system. Improvements specifically related to health systems included increased skilled birth attendance, investment in the training of midwives and a monetary incentive for facility-based midwives for each live birth conducted15. The Pakistan health system As a federal state, the management of health services in Pakistan is devolved to a provincial level. The federal Ministry of National Health Services, regulations and coordination is responsible for setting national policies, strategies and targets. Provincial authorities interpret these policies, strategies and targets and apply them to the local context, establishing provincial-level policies and ensuring appropriate budgeting, planning and implementation. District health office who report to the Provincial Department of Health manage most of the healthcare service delivery to communities, including community outreach. Community based services are particularly important in areas where access to health services is challenging due to geographical terrain, poverty and cultural practices (e.g. limited travel for women and girls). The national MNCH Program in Pakistan Pakistan is among the few countries in South Asia that continues to have dismal maternal and child health indicators. In Pakistan, the maternal mortality ratio (MMR) is high, ranging from 240 to 11 700 per 100,000 live births. National figures show that only 52% of deliveries are conducted by skilled birth attendants (SBAs)16. Approximately two-thirds of all births (61%) take place at home by traditional birth attendants (TBAs) or un-trained family relatives due to limited access to health facilities. Realizing the need for a community health workforce, the Government of Pakistan launched the national MNCH program in 2006 to help rural women deliver safely. The national MNCH program was designed and implemented as a concerted effort to help achieve the Millennium Development Goals (MDGs). The program required federal funds and some donor funding in the form of grants, budgetary support and technical assistance. The national MNCH program was inspired by the desire of the government to reduce maternal, newborn and child morbidity and mortality, particularly among poor, marginalized and disadvantaged segments of society by strengthening, upgrading and integrating ongoing interventions and introducing new strategies17. The national MNCH program, in order to increase women’s access to skilled care during birth, introduced Community Midwives (CMWs) as a new cadre of community skilled birth attendants. The program aimed to deploy 12,000 rural CMWs over a 5-year period. CMWs were expected to provide domiciliary maternity care through the establishment of private practices in their home villages. Rural women with ten years of education were recruited and provided with 18 months of midwifery training. Their training included a 12-month classroom component followed by a 6-month practical clinical component. After completing their training, CMWs were deployed back to their home villages and expected to establish private practices and provide domiciliary maternity care to a population of 10,000, in geographically defined catchment areas18, 19. Different studies have been conducted in an attempt to map the coverage of CMWs outreach; determine the barriers in CMW acceptance and their ability to provide relevant care and how CMWs interact with other care providers in different parts of Pakistan20, 21. Up until 2011, the country had trained and deployed 4,700 CMWs in all the four provinces. Yet, a survey conducted in two districts of Punjab reported that only 3 - 11.7% births were attended by CMWs22. Recent literature shows that the selection, training and deployment process of CMWs needs further improvement in all parts of the country. Owing to the ‘newness’ of the program, there are many delays in the sequencing of activities, including the selection of candidates, the initiation of training, the setting of examinations, the provision of certification and most importantly, the deployment of CMWs. Communities in rural areas also have low awareness of the presence of 12 CMWs. The linkage of CMWs with other community health providers and relevant health facilities has also not yet been fully established throughout the country23, 24, 25. Various interventions on capacity building, business training and the use of technology in health care reporting have been provided by donor agencies and this support has significantly enhanced CMW performance 15. The most effective part of the training, as verbalized by the CMWs, was the hands on practice opportunity in primary and secondary health care setups26, 27. The Balochistan context Balochistan has the worst maternal, neonatal and child health indicators in Pakistan. Delivering quality healthcare services in Balochistan can be a challenge. Balochistan is made up of 32 districts and although geographically the largest province in Pakistan, Balochistan is the least populated with less than 10 million residents spread across geographically challenging terrain. The poverty of communities and cultural practices (limited travel of women) make reaching communities more difficult for the Government of Balochistan. These challenges are compounded by the security context. The literacy rates of Balochistan are low, with only 39% of men able to read and write, and only 16% of women. The Saving Mothers and Newborns in Communities (SMNC) Program, Balochistan Mercy Corps implemented a four-year (2012-2016) program, Saving Mothers and Newborns in Communities, in Quetta, Kech, and Gwadar districts of Balochistan to improve maternal and newborn health status, especially for poor and marginalized women. The program is primarily funded by USAID and co-financed by the Scottish Government. The strategic objective of the program is to increase the use of quality essential maternal and newborn care, through private￾sector community midwives. According to PDHS 2006-07 and 2012-13, Balochistan has the poorest maternal health indicators of all the provinces of Pakistan (Table 1) 29. Table 1: National, Provincial and District MNCH Data MMR U5MR IMR NMR SBA ANC (skilled provider) PNC (within 2 days after birth) MDG Target 140 89 74 N/A >90 N/A N/A National 276 94 78 54 52.1 73.1 60.3 Balochistan (rural-urban) 785 59 49 30 17.8 (14.2-34.4) 30.6 (24.9-53.8) 37.2 (36.2-42.3) 13 Mercy Corps together with the BDoH, designed and implanted a program to test whether CMWs in Balochistan could become self-sustaining maternal and child healthcare providers and increase the coverage of high impact MNCH services. The project included a four-week clinical refresher course, a business skill-training course, a mobile phone application for reporting health data, and the establishment of WSGs to deliver peer education on health to women. The results of the project were to be incorporated into the Government of Balochistan’s Planning Commision-1 (PC￾1) for the 2015-2020 MNCH Program. Under the MNCH Program, strengthening the capacity of the CMWs is a major priority for the BDoH. While communities deserve access to high quality care from well-qualified workforce, healthcare providers deserve to work in well-supported environments. Therefore, CMWs collaboration with other providers in the health system is both relevant and important. The need to balance the right number of CMWs, with the right level of skills, in the right geographical areas, is a challenge for the BDoH, specifically given the security context found in Balochistan. From March – June 2015, the SMNC program underwent a midcourse correction where the program’s Learning Agenda was revised. The new Learning Agenda initiated research that comprised of four sub-studies carried out in both program intervention and non-intervention areas. The aim of the research was to document the contributions of the program to increasing women’s access to skilled care during delivery and to gather and share lessons learnt, in order to generate evidence to inform MNCH policies and strategies for the province 30. The objectives of the four sub-studies undertaken were: Sub-study 1: Research Question: How can the BDoH improve its selection process to effectively recruit and deploy CMWs in underserved areas? Objectives: 1. To understand how CMWs could be effectively recruited and retained in the system after deployment. 2. Identify preferences of CMWs (qualified and current students) for working in rural and resource-constrained areas and to propose appropriate, cost effective incentive packages for attracting/retaining CMWs. Sub-study 2: 14 Research Question: How can CMWs become financially self-sustaining while serving the needs of the poorest of the poor? Objectives: 1. To identify successes and failures of CMWs by undertaking an expenditure and investment assessment of CMWs. 2. To propose a sustainable model based on the findings of objective 1. Sub-study 3: Research Question: Do CMWs offer quality care? How? Objectives: 1. To assess the knowledge, attitude and practices of CMWs in the intervention and non￾intervention areas. 2. To understand the satisfaction of clients with the services of CMWs. Sub-study 4: Research Question: How can the DoH streamline CMW reporting using cell phone technology and expand mHealth in the province? Objectives: 1. To document experiences of CMWs and gaps in the reporting system in order to potentially expand mHealth services. 15 SUB STUDY 1 RESEARCH QUESTION How can the DoH improve its selection process to effectively recruit and deploy CMWs in underserved areas? METHODOLOGY An exploratory study using a mixed study approach was undertaken, including qualitative methods and a Discrete Choice Experiment (DCE). DCE is an analytical method that can be used to quantify a respondent’s preferences for various attributes of a service or good. DCE is used to systematically identify and evaluate interventions, which are more effective in attracting and retaining the needed human resource. DCE data is being increasingly used in health services to quantify the degree to which healthcare providers in developing countries perceive and accept various incentives and it models the likely impact of different human resource strategies on rural recruitment in resource poor countries. This methodology was used to investigate the motivation of CMWs for working in rural and resource-constrained settings using a systematic analysis technique. Figure 1: Methodology Flow Chart Discrete Choice Experiment Qualitative Study Focus Group Discussion to determine attributes and levels FGDs with CMWs of Intervention and Non￾Intervention areas In-depth Interviews to develop insight into the barriers faced by the already deployed CMWs IDIs with Government officials, CMW tutors and Mercy Corps Program Team Quantitative Study Discrete Choice Experiment DCE was conducted with 50 CMWs of Intervention and 60 Non-Intervention areas 16 Phase I: Qualitative Study Thirty-eight key informant interviews were undertaken with Mercy Corps project staff and relevant district and provincial health officials (Annex: 1) to develop an understanding of the barriers faced by the already deployed CMWs and how to improve CMW recruitment and retention. Interviews with CMW trainers were carried out to investigate their role in recruitment, along with the facilities available for CMWs during training. Interviews focused on the interest of the CMWs to continue to work especially in rural and remote areas after their training. In addition, Focus Group Discussions (FGDs) were conducted with qualified CMWs working in intervention and non-intervention areas to identify factors and incentives that would attract women to take up the profession of a CMW deployed to a rural area and to examine how to retain CMWs in rural areas. Table 2 gives the description of the respondents. Table 2: List of Respondents in In-depth Interviews Respondents Designation Department of Health at Provincial Level Director General Health, Balochistan Provincial Lead, MNCH Program, Balochistan Provincial Epidemiologist, Balochistan Deputy Coordinator, LHWs Program, Balochistan Health Department at District Level District Health Officers (DHO) – Kech & Gwadar District Coordinator MNCH Program - Kech Deputy Program Manager, HIV/Aids Program – Quetta Medical Superintendent District Headquarter Hospital – Gwadar Ex Deputy DHO – Gwadar LHS - Quetta, Kech, Gwadar LHV - Quetta, Kech, Gwadar CMW tutors - Quetta, Kech, Gwadar Mercy Corp Program (SMNC) Staff, Quetta Team Leader Monitoring, Evaluation and Learning Manager Project Officer Community Midwives* Intervention areas: Quetta -14, Gwadar -9, Kech -12 Non-Intervention areas: Quetta -7, Gwadar -3, Kech -19 *CMWs list in Annex 2 The CMWs invited for FGDs were selected with the help of the BDoH and Mercy Corps project staff. Only those CMWs were included who had at least 6 months of post-training experience in the field. Prior to FGDs, permission was taken for recording and photographing the participants. Seven FGDs were conducted in total (two each in Quetta and Gwadar and three in Kech). At the end of 17 each FGD, the participants were requested to list four priority attributes for working in resource constrained/underserved areas. The FGDs were transcribed verbatim. The data was reviewed by the research team and coded independently by two members of the research team who had experience of working in public health research projects. Through deliberations the research team reviewed statement by statement to identify similarities and differences. Graneheim and Lundman’s content analysis method was undertaken. Codes were then categorized followed by identification of three sub￾themes and a main theme. At the end of each FGD, participants were requested to list four priority attributes that they deemed important when working as a CMW in rural areas. Phase II: Discrete Choice Experiment Survey Using the most frequently listed priorities by CMWs, a list of top six attributes was generated for the DCE questionnaire. Each attribute was assigned sub-levels, endorsement for which was taken from the policy makers in the BDoH and Mercy Corps project team before finalization of the DCE survey questionnaire. This survey was then designed in Sawtooth Software and results were analyzed using statistical analysis software Stata. In the survey, respondents were presented with 12 paired job scenarios on the most important attributes identified. The minimum sample required for statistically significant results was 50. Through the assistance of BDoH and Mercy Corps staff, a list of deployed CMWs was generated and they were administered the questionnaire during their monthly reporting period. The demographic information of DCE participants was analyzed using descriptive analytics and choice data was analyzed through inferential statistical analysis. Responses of participants from intervention and non-intervention areas in districts Quetta, Kech and Gwadar of Balochistan were comparatively analyzed. In descriptive analysis, simple frequencies and percentages of background information were calculated. In inferential analysis the choice data was analyzed using mixed logit model, which allowed modeling of repeated choices. Mixlogit regression analysis was undertaken in Stata on job pairs to identify the job preferences of CMWs. This comparative analysis within and across different individuals in the DCE survey, generated the significance of each attribute through p values and coefficients. These were then used to compare the relative importance of attributes. Finally, the result of the mixed logit models was used to predict the effect of different attributes on proportion of CMWs retaining their rural job. The DCE Questionnaire and FGD guide are attached in Annex 4 and 5. 18 Table 3: Background characteristics of participants in FGDs Age Intervention (N=35) f (%) Non-Intervention (N=30) f (%) 20-25 years 16 (46%) 16 (55%) 26-30 years 18 (51%) 4 (14%) 30-35 years 1 (3%) 4 (14%) 36 and above 5 (17%) Marital Status Married 14 (40%) 23 (77%) Unmarried 19 (54%) 7 (23%) Widow 2 (6%) Years of work experience 1-3 years 20 (57%) 9 (30%) 4-6 years 14 (40%) 21 (70%) 6 years and above 1 (3%) Location Urban 24 (69%) 15 (50%) Rural 11 (31%) 12 (40%) Not given (46%) 3 (10%) 19 RESULTS Phase I: Qualitative Analysis All together thirty-eight interviews with DoH (at the provincial and district level) and Mercy Corps project team and seven FGDs with 64 CMWs were conducted in intervention and non-intervention areas of Districts Quetta, Kech and Gwadar (Participant details in Annex 3). Table 3 shows the background information of the FGDs participants. FGDs were conducted with 35 participants from intervention areas and 30 participants from non￾intervention areas. The majority of participants in the intervention areas were aged 26-30 years (51%). 54% were unmarried in the intervention areas whereas in non-intervention areas, 77% were married. The majority of participants (57%) in intervention areas had 1-3 years of experience Table 4: Background characteristics of participants in FGDs Intervention (N=35) f (%) Non-Intervention (N=30) f (%) Age 20-25 years 16 (46%) 16 (55%) 26-30 years 18 (51%) 4 (14%) 30-35years 1 (3%) 4 (14%) 36 and above 5 (17%) Marital Status Married 14 (40%) 23 (77%) Unmarried 19 (54%) 7 (23%) Widow 2 (6%) Years of work experience 1-3 years 20 (57%) 9 (30%) 4-6 years 14 (40%) 21 (70%) 6 years plus 1 (3%) Location Urban 24 (69%) 15 (50%) Rural 11 (31%) 12 (40%) Not given (46%) 3 (10%) 20 as CMWs whereas 70% in non-intervention areas had experience of 4-6 years. 69% of respondents in intervention areas and 50% from non-intervention areas were located in urban areas. Interviews and FGDs data were analyzed using content analysis approach. Figure 2 shows the analysis process moving from main theme to sub-themes and categories. Figure 2: Analysis Process from main theme to categories Poor Retention of CMWs in Balochistan The main theme grounded in the data was “poor retention of CMWs in Balochistan”. Provincial level and project level respondents reported that CMWs were trained healthcare professionals that received 18 months of training to earn certification to provide maternal and neonatal services at a community level. According to the records of BDoH, approximately 800 CMWs had been trained to date with an additional 350 undergoing training in different midwifery schools of the province. All respondents shared that retention of this trained workforce was a challenge for the province. The BDoH respondents were aware of the problems related to the retention of CMWs but felt that due to financial constraints, poor human resource management, political influence and a lack of consistency in government policies, this important workforce was neglected. Respondents reported that even after eight months, less than half of the MNCH program budget had been released from the federal authorities and given to provinces. This added to the complexity of the situation and made retention of CMWs more challenging. Particular challenges included a lack of 21 funds to provide stipends to CMWs and implement adequate supervision. Additional challenges faced by the provincial level policy makers included the lack of quality teaching in midwifery schools and poor ownership of CMWs by the BDoH and district offices. There was also an issue with the deployment strategies initiated by the BDoH due to a lack of coordination and weak linkages with the private sector. For example, some health facilities were run under the public private partnership of the Peoples Primary Healthcare Initiative (PPHI) and the BDoH - CMWs working in the PPHI run facilities were not adequately linked with the solely government-led health facilities. The geographical terrain of Balochistan only added to the magnitude of the challenges. Senior Health Department officials stated: “To date we have received only 45% of the total budget allocated to the MNCH program.” “Unless CMWs get the registration card from Pakistan Nursing Council they are not allowed to touch any patient. And if the government continues to give a stipend of Rs. 5000 per month to each of the registered CMWs, then that amounts to a lot of money……. we don’t have such funds at the moment. So far the CMW work plan has been approved but unfortunately there is no money for deployment” [Government official, Quetta] “There is no proper planning at a district level. The PPHI is an independent body and working at a district level and doesn’t involve the Health Department. They directly hire people on contract and we don’t know where they have hired people and from where they are giving them salaries. They hire people and then come to DHO and ask for all details of vacant seats at BHUs. They approach the finance department directly to release budget for vacant seats. So the budget of the health department is given away without us being in the loop. Also linking up CMWs with their staff is difficult. We don’t even know who is working where.” [Government official, Quetta] “The majority of midwifery schools don’t have qualified tutors. Also tutors have not been given any refresher training for years. Also the midwifery school budget is very little. Going to a tertiary hospital during training is a challenge - even transportation is a basic issue.” [Mercy Corps SMNC program staff] The three sub-themes contributing to the main theme are detailed further in the following sections. 1. Lack of Quality Teaching and Training A lack of quality teaching in midwifery schools was highlighted by the respondents as a challenge to the recruitment and retention of CMWs in Balochistan. In the urban setting of Quetta, midwifery 22 schools were functional and the teaching staff were well trained. However, outside of Quetta the situation was drastically different with poor infrastructure and a lack of qualified staff. There was no available trained teaching staff and some of the midwifery schools were housed in a single room. The learning environment was perceived as being highly inadequate by all respondents. This was particularly pointed out by the staff of the midwifery school of Quetta but applied to all schools providing training to CMWs. In addition to this, practical midwifery training was being delivered at the tertiary hospital in the Obstetrics and Gynecology department in Quetta, resulting in CMW trainees not gaining work experience in community-based health services. There was also no mechanism of coordination between the midwifery schools to facilitate student or faculty exchange to enhance learning. In addition to this, according to the provincial level respondents, most of the midwifery schools lacked the capacity to provide quality training and ad hoc teaching staff posted were not familiar with their roles and responsibilities. Some of the trained teachers who were politically well connected did not want to serve in the remote areas and were never transferred to such locations. School management was also poor. It was reported that even for successful basic administration of the schools, such as procurement of supplies and equipment, linkages with politically higher authorities was required. The midwifery school respondents shared that most schools in the districts of Balochistan were run solely by the Principal. One respondent stated: “As per the recommendation of the Pakistan Nursing Council, tutors should hold a post￾graduate nursing degree. In Balochistan there are very few post-graduate nurses and when we transfer them they don’t go to the outskirts. In rural districts there are schools where only a principal is working.” [Representative BDoH, Quetta] Another participant said: “There are very few trained tutors, we need to train the faculty of the midwifery schools first and then only can we produce a quality product.” [Mercy Corps SMNC program staff] All respondents were of the view that if the teaching staff were strengthened and given improved facilities, the situation could be improved substantially. They agreed that even by improving the working condition of the teaching staff in the periphery, many midwives would be willing to work in the remote districts. 23 Regarding revisiting the content of the curriculum and practical training, many respondents expressed the need to allocate more time to hands on training in a community setting. They believed that this aspect was neglected in the current training. CMWs received training in a tertiary care setting that did not prepare them for their work at a community level. “We can see that theoretically everything is remarkably good but when we talk about practical exposure, we came to know that the majority of CMWs have not attended any deliveries during their 18-month training course. So either they didn’t get a chance or their tutors were not playing their role properly. Even it’s written that during the course they will go to the community and serve, that component is never implemented.” [Mercy Corps SMNC program staff] The lack of quality training stemmed from the poor ownership of the CMWs by the BDoH and poor Human Resource management as detailed hereunder: a. Poor ownership of CMWs by the BDoH By design, CMWs were to work in the communities and after training sustain themselves as trained private health care providers deployed with minimal support from the BDoH. The CMWs by design were to be selected from the communities so that after deployment they would serve in their respective areas. There were many gaps between the design and implementation. When the midwifery schools advertised admissions, applicants fulfilling the educational criteria were offered places. Remote areas lacked applicants that met the minimum education criteria for admission and this resulted in CMWs being recruited and trained from urban settings. Recruitment of rural CMWs was also undermined by the lack of rural community representation during the CMW selection procedure. The CMW selection process led to a concentration of trained CMWs in urban environments. While establishing their health practices in urban areas, CMWs had to compete with higher qualified health care providers, including doctors and Lady Health Visitors in both the private and public sector. This meant that CMWs were not deployed to areas that lacked trained health care providers. Many CMW respondents indicated that at the time of training they were explicitly told that they would be given permanent government jobs on completion of the CMW course. Post-deployment when the CMWs realized the nature of the work and their roles, they lost motivation to establish their own practices. This is reflected in the quote below: “At the time of admission, the applicants should be informed fully about what this job demands and what it actually is. This is one of the biggest reasons for drop-outs because 24 some CMWs have the expectation that they will gain proper government employee status.” [CMW, Quetta] The participants shared that as per the government policy, trainees were required to stay at the hostel during the training period. Since there was no accommodation for married women with children in hostels, most of the CMWs that completed their training were unmarried. The potential for CMWs to set-up clinics after training was not checked at the admission stage and often CMWs left training without the space or resources to establish a clinic. It was also mentioned by BDoH respondents that after training, many CMWs did not turn up to collect delivery kits provided to them by the BDoH. “When we called deployed CMWs to collect supplies and equipment from the department only 30 out of 180 came. The rest of them said that they did not have the space in their homes.” [MNCH program respondent, Quetta] CMWs also cited that they were demotivated after their initial post-deployment two-year stipend was stopped and they were expected to sustain themselves through private enterprise. They perceived this as a lack of ownership by the BDoH. The Government of Balochistan was indecisive about whether CMWs should be made part of the government structure or a private entity. However, it was repeatedly mentioned by all respondents that for the successful retention of CMWs, the government has to take major steps, including providing incentives. “If you have to retain them then incentives should be increased to at least a minimum monthly wage of Rs. 13,000.” [MNCH program respondent, Kech] Another participant said: “The government should provide CMWs a monthly salary package of at least Rs 5000 or 7000. Besides that, CMWs should run their own businesses. In this way it will work. The government should provide training, do monitoring and CMWs should generate reports and should not sit idle.” [Government Official, Quetta] It usually took one year from a CMW to complete their training to being deployed. During that time period CMWs were not provided with any refresher training by the government. Also, no support was given to CMWs to maintain their workstations. CMWs in the program intervention areas highlighted the support provided by Mercy Corps. The respondents confirmed that not only were Mercy Corps providing refresher training to CMWs before deployment, they were also providing support to CMWs in the maintenance of their home set-ups. 25 “The refresher trainings are very beneficial for us. They should be conducted regularly. We learn a lot and get new knowledge during such trainings. Our knowledge gets revised if we forget our lessons.” [CMW, Kech] b. Poor Human Resource Management Efficient and proper utilization of human resource requires financial and technical management of investments. The respondents shared that for the long-term sustainability of the program, the national MNCH program should ensure timely deployment of CMWs after training, regular monitoring and supportive supervision, and an appropriate stipend. They also shared that career advancement was necessary for retention within the system. It was feared that without proper management of CMWs by the health department, this trained workforce would be lost. One respondent said: “The Balochistan Health Department trained thousands of TBAs and now no TBA is seen. Similarly, if we don’t bring CMWs within our system they will also be gone like TBAs.” [BDoH respondent, Quetta] Another issue that respondents raised was that when offering admissions to the CMWs, the same geographical areas were repeatedly considered for the selection of girls. Therefore, the balance between those trained and the need of CMWs in a particular area was not achieved. This was especially evident in urban areas of Balochistan. As a result, a larger number of CMWs were concentrated in urban areas instead of in rural settings where their services were needed the most. “When I visit my village, I find that the residents badly need a CMW. There is no one to attend to the needs of women. When I was visiting for a wedding, I was called to attend to a seven-month pregnant woman who had been in labor for over 48 hours. She had been advised by her relatives to rest as it was too soon for delivery. What I saw was that the baby was almost out. She had been in a lot of pain and no one had taken her to a doctor.” [LHS, Quetta] The provincial and district level health department respondents were of the opinion that mapping of CMWs should be done to identify areas which required CMWs. The tutors of Midwifery schools shared that the BDoH had recently advertised names of the union councils from which applicants were required. Unfortunately, very few of applicants from the specified areas fulfilled the admission criteria. 26 “We normally advertise and do not mention the Union Councils where the applicants are encouraged to apply from. Then when we tried to select girls on the basis of UCs, only eight girls qualified. The rest did not meet the required minimum qualification. It is not cost effective to run a training program for only eight students.” [Midwifery school respondent, Quetta] It was suggested that CMWs should replicate the regularization process previously followed by the LHW cadre. The respondents felt that the two types of workers could work together to provide services. However, many respondents believed that since LHWs were busy with other activities, for example implementing polio campaigns, they could not extend support to CMWs. This gap was evident from the small number of pregnant women being referred to CMWs by LHWs. Traditional birth attendants/ Dais (TBAs) were also in competition with CMWs. Dais have been involved in home based deliveries for generations and people tended to trust them and their skills more. “We are facing problems from Dais. We are in competition with them. They socially boycotted us and our families and proactively stopped people from coming to us for treatment.” [CMW, Gwadar] However, CMWs from intervention areas felt that the communities were gradually appreciating their skills and were willing to accept CMWs as a trained health care provider. This was considered as being the result of CMWs having their own setup and treating their clients with respect. “The community welcomed us when we started our work in our areas. The community is happy with our work because it is much better than Dais.” [CMW, Kech] Another respondent said: “There are some clients who are unable to pay for our services. Some pay later. If we insist on payment they go to the Dai instead. We then have to convince them by showing them the difference in our work…… They like it when we conduct delivery behind a curtain, giving them privacy (sharam parda) whereas Dais do not take care of such things. Whosoever wants enters that room. Mother-in-law is often sitting nearby smoking huqqa and no one cares about the feelings of the patient. Now many clients prefer our setup.” [CMW, Quetta] 2. Neglected Deployment Strategy Upon completion of their 18 months training CMWs had to wait one or two years before deployment. As a result, many trained CMWs moved towards other career options including teaching in schools, working as a midwife in a hospital or as an assistant to a doctor. Many employers hired them at a lesser remuneration than a nurse. A major chunk of CMWs were 27 therefore ‘lost’ to the system and of no value to the health infrastructure. Because of the time gap between training completion and deployment, refresher training of CMWs was needed but no such opportunity was given to them. The delay in deployment was primarily due to the delayed registration of midwives by the Pakistan Nursing Council (PNC). The registration processing time of 6 weeks was often extended to over a year. “Many girls start working as a teacher or substitute for nurses in hospitals. They earn around Rs. 10,000.” [CMW, Quetta] Another respondent said: “Deployment is delayed to such an extent that a CMW forgets everything. Initially the midwifery school registers students with the PNC as students. Later registration became the responsibility of the MNCH program. In PNC the process takes 6 weeks. But that 6 weeks’ registration period was delayed to 1 year or even more. “[Mercy Corps SMNC program Staff] The respondents attributed this neglected deployment strategy to poor coordination between the government stakeholders, including the BDoH, the national MNCH Program and the PNC. Respondents from the Mercy Corps project team shared that according to the deployment guidelines, CMWs were supposed to be introduced to the communities at a public event to confirm their credibility and facilitate acceptance by the community. Mercy Corps arranged community sensitization meetings for CMWs in intervention areas. However, no such measures were undertaken for CMWs in non-intervention areas. Some CMWs, especially those from the intervention areas, introduced themselves and their services to communities: “My parents told the community that their daughter is taking training and she will become a doctor or baji. Otherwise, we introduce ourselves as a CMW in the community. We educate people about health and people respect us and our work. We are successful in conducting safe deliveries and people appreciate our work.” [CMW, Kech] a. Lack of Coordination Between Stakeholders In the design of the CMW program, different stakeholders were supposed to provide technical and administrative support to recruit, train and retain CMWs. However, a lack of coordination between the BDoH MNCH program, the PNC and the National Program for Family Planning and Primary Health (NPFP&PHC) was evident. The PNC, although involved in the registration of CMWs delayed the process. One reason for the lack of coordination mentioned by the respondents was the financial constraints of the BDoH. 28 “We faced many hurdles. For deployment we have registered 102 CMWs who have qualified but we have some financial constraints. This project had Rs. 2600 plus million budget and it was supposed to close in 2012. However, by 2012 no province had achieved the targets so they extended it to 2015. But unluckily no province, except Punjab, got the full budget.” [MNCH respondent, Quetta] LHWs were recognized as a government health cadre and have been regularized by the BDoH. By design, technical and administrative supervision of CMWs was to be provided by LHVs and LHSs respectively, with support in community mobilization provided by LHWs. However, because of a lack of coordination within the BDoH supervision of CMWs was neglected. Respondents said that although the MNCH program and (NPFP&PHC)program, aimed to reduce maternal and child mortality, it worked independently from the CMW program. This lack of coordination resulted in a duplication of activities. This not only led to a waste of resources but also resulted in inefficient service delivery. It was also expressed during the interviews that integration of both programs should be proactively implemented to enhance the planning of activities and achievement of outcomes. The MNCH program had developed a complete checklist for monitoring and supervision of CMWs. However, it was only used for Mercy Corp supported CMWs whereas in non￾intervention areas, CMWs were not evaluated on the basis of the checklist. Also, all the reports with related information were not shared across departments and programs and instead sent to District Health Information System (DHIS) only. “Family planning and MNCH programs personnel sit together but there is gap in information sharing. Their implementation is not aligned. So if the level of coordination is improved between these two departments, things in the field will also be improved.” [Mercy Corp SMNC program staff member] There was a lack of coordination between CMWs, LHWs, LHVs and LHSs in non-intervention areas. Mercy Corps had however facilitated establishing such linkages. LHWs in intervention areas provided great support in community mobilization. “There are some areas where linkages between LHWs and CMWs were facilitated. Actually in a monthly meeting a LHW was called and told that they have a CMW in their area. So a communication link was created between them and it has helped a lot.” [CMW, Quetta] Mercy Corps extended support by expediting the process of PNC registration of CMWs. This reduced the delay in deployment of CMWs and potentially reduced their drop out. In the interim 29 period i.e. between completion of training, PNC registration and deployment, the CMWs were forced to either stay at home or seek gainful employment elsewhere. “Mercy Corps helped us in our registration with the PNC as without it we could not start our work. Also the MNCH program would not take responsibility in case something happened to the clients.” [CMW, Quetta] b. Weak Linkages with Private Sector CMWs were required to eventually sustain themselves as private practitioners. During the interviews respondents expressed the need for the government to take a multi-dimensional approach. They believed that the government and donor organizations could not work in isolation. CMWs expressed that through support of donors, they could establish their own service delivery setups. However, they still preferred to be seen as part of the public sector. They recognized the role of donors in the completion of their training and the continuation of maternal healthcare services in the field. CMWs were given by Mercy Corps, a stipend, provided support in registration, provided with equipment and supplies, supported to set-up of workstations, provided with refresher training and supported to market their services through community mobilization. Such steps led to the CMWs being motivated and committed to serve the communities in the intervention areas. “Mercy Corps is doing our appraisal and they also monitor us and supervise us and give us feedback after evaluation. We are also getting good technical support from LHV/LHSs and field officers. Their staff is very good and helps us if there is some problem in our work and suggestions on ways for improving our services are given.” [CMW, Gwadar] Regarding the support provided to CMWs in the marketing of their services to communities, one respondent said: “Initially we informed people in the community about our work. We conducted meetings and went to their homes so now they are aware of our work. The introductory meetings were conducted by Mercy Corps in the communities. They also supported us in putting sign boards outside our houses for our marketing.” [CMW, Gwadar] Government officials were in favor of taking measures to ensure the induction of young women from the remote areas. One official was of the view that an accelerated learning program for girls from remote areas should be started so that local residents could be enabled to train as CMWs: 30 “The government lacks experience. For retention of CMWs, they should be given some incentive. They should be enrolled in accelerated education programs, or should be properly allocated, or Mercy Corps or UNICEF or some other organization should support them. Regularizing them will become very costly for the government.” [Government official, provincial level] 3. Harsh Environment The terrain of Balochistan added to the challenges of delivering the CMW program in Balochistan. The geographical distances between settlements affected the decisions of young women to opt to train as CMWs and also hindered the pursuit of their careers. Harsh weather conditions and traveling without availability of transport in a patriarchal society made their service delivery difficult and challenging. “It took almost a year for me to gain recognition in the community. I made regular door-to￾door visits for a year or two. But now it’s not needed. Two years were tough but now people come to me themselves.” [CMW, Quetta] “We went from door-to-door initially. The security conditions were not good here, so people would not welcome us in their homes. Once a client came to me and left happy with treatment…. that’s how I gained people’s trust.” [CMW, Gwadar] The make-up of trainee CMWs was influenced by the fact that there was a lack of hostel facilities for married women and because families were often reluctant to send their girls to hostels. After post-training deployment, the cost of travel to provide services within communities often cost more than the income generated from the services provided. CMWs often had to be accompanied by an escort when travelling which obviously incurred additional costs. The majority of CMWs said that they are supported by their parents in their work. However, it was also mentioned that after marriage, mobility of CMWs would be determined by the husband and his family. a. Geographical Terrain Challenges The geographical terrain of Balochistan is very challenging especially for the mobility of young women. With consideration of the scattered settlements in the catchment areas of CMWs in Balochistan, their client coverage was considerable reduced (as stated in the design document of the Balochistan MNCH program). A catchment population of 5000 households for CMWs could not be followed. The dispersed population on one hand reduced the clientele of CMWs while on the other hand made their sustainability a big issue. 31 “There are CMWs who leave their homes at 5am in the morning and change many buses before reaching the training institutes. During training we had to spend on transport from our own pockets. That was a very difficult time as some of us were pregnant, some had small children………. it was a difficult time. In Quetta too there are settlements that are widely spread out.” [CMW, Quetta] Another participant said: “We don’t get matriculate girls from every union council, and we cannot send CMW from one union council to the other. In some areas they are deployed on populations of 4000 whereas in others only 1000 to 2000 households fall in their vicinity. Some have even lesser numbers of households.”[MNCH, Quetta] The clientele was thus reduced. Most families with low income perceived CMWs as government employees and expected free treatment. “There are some clients that can’t afford giving money at all, there are others who say the will give money later. This is because people think that we are attached to the government and we are getting salary from there. They want us to treat them for free.” [CMW, Quetta] b. Patriarchal Culture Respondents expressed that because Balochistan has a male dominated society, most families discourage women from working alone outside their homes. In poor families women are married at an early age and polygamy with multiple children borne by each wife is often witnessed. The needs of a mother were often ignored. In communities that were unaware of the services of CMWs for maternal and child health, the situation was considered worse. The respondents shared that Dais (traditional birth attendants) were preferred by many families. They were respected for their services because of their age and experience - the trained cadre of CMWs were not valued. “People who are educated understand easily but it gets very difficult to communicate with those not educated and to make them understand. They come to you every other day asking you the same thing again and again. A girl came to me and said she is bleeding. So I took her to the hospital and the doctor said that we have to operate. Her relatives took their time deciding. What if she died who would be responsible? You see they do not understand things.” [CMW, Quetta] Respondents expressed the need to counsel families and inform them of the importance of utilizing CMWs’ services. 32 CMWs who had established their clinics had done so with support from their families. They shared that most families accepted young women pursuing teaching because it was considered a respectable profession for women. The community midwifery profession has not yet received such recognition. “CMW service recognition by the society is important so that CMWs can continue to work even after they get married.” [CMW, Kech] Most respondents reported that with time, the acceptance of CMWs is increasing. However, the need for efforts to mobilize communities was highlighted. Respondents were of the opinion that LHWs could support the promotion of CMWs within communities. “This is because the LHWs that are working, they know which house has a pregnant woman and can refer them to a CMW.” [LHV, Kech] Phase II: Quantitative Analysis Based on qualitative results from the interviews and focus groups, a final list of job attributes for rural CMWs and the relevant levels was developed (given in Box 1). For each of the attributes, two to three levels were determined. Each level represented a privilege that the government could potentially offer the CMWs for working in rural areas. Box 1 – Attributes and levels Job Attribute Attribute Level 1 Housing None Housing allowance Housing with basic amenities 2 CMW practice setup Seed money for set up Regular continuous supply of medicines/ delivery kits 3 Transportation None Transport allowance 4 Supportive management No supervision Supervision through program (LHS/LHS/WMO) Refresher courses 5 Stipend PRs. 5000/ month PRs. 7000/ month PRs. 12000/ month 6 Good schooling for children Yes No 33 From the list of attributes, twelve pairs of incentive packages were designed using Sawtooth software. An example of one pair of incentive packages is given in Box 2 (see Annex 4 for the 12 pairs included in the survey). Box 2: Paired Incentive Packages Job Package 1 Job Package 2 Housing None Housing with Basic Amenities CMW Practice Setup Seed money for setup Regular continuous supply medicines/ delivery kits Transportation None Transport Allowance Supportive management No supervision Refresher courses Stipend Rs. 7000/ Month Rs. 5000/ Month Good schooling for children Yes No A total of 110 respondents participated in the DCE survey. The sample included 50 respondents from intervention areas and 60 from non-intervention areas. Five sets of questionnaires were generated through Sawtooth. Each set of questionnaires was administered to a minimum of 10 CMWs each from intervention and non-intervention areas. All the respondents were residents of Balochistan. The respondents profile is given in Table 3. 34 Table 3: Socio-demographic description of CMWs Intervention (n=50) f (%) Non Intervention (n=60) f (%) P value Age 17-20 years 10 (17%) 0.038 21-25 years 23 (46%) 32 (53%) 25-29 years 22 (44%) 15 (25%) 30 years and above 5 (10%) 3 (5%) District Quetta 20 (40%) 8 (13%) <0.001* Gwadar 10 (20%) -** Kech 20 (40%) 13 (22%) Other 39 (65%) Marital Status Married 18 (36%) 29 (48%) 0.134 Unmarried 32 (64%) 31 (52%) Current Work Location Rural 28 (56%) 30 (50%) 0.569 Urban 22 (44%) 30 (50%) Work Experience (Years) No Experience 1 (2%) 6 (10%) 0.012 1-2 years 9 (18%) 10 (17%) 3-4 years 27 (54%) 20 (33%) 5-6 years 10 (20%) 13 (22%) More than 6 years 3 (6%) 7 (12%) Average Monthly Income Not given 1 (2%) 8 (13%) <0.001* Less than 3000/month 21 (42%) 29 (48%) Between 3000-5000/ month 9 (18%) 21 (35%) Between 5000-7000/ month 6 (12%) 1 (2%) Between 7000-10000/month 13 (26%) 1 (2%) Average Monthly Household Income Not given 1 (2%) 0 (0%) 0.111 Less than 10,000/month 10 (20%) 19 (32%) Between 10,000-20,000/ month 18 (36%) 27 (45%) Between 20, 000-30,000/ month 15 (30%) 8 (13%) More than 30,000/month 6 (12%) 6 (10%) Work Preference .007* Preferred working in rural over urban areas 42 (84%) 36 (60%) Didn’t prefer working in rural over urban areas 7 (14%) 24 (40%) * Statistically significant at p<0.05 between intervention versus non-intervention areas**All the CMWs in Gwadar were from intervention areas. Only 13 CMWs were working in Kech in non-intervention areas. CMWs from Nushki and Harnai (neighboring districts to Gwadar and Kech) were also included in the non￾intervention area sample. 35 No significant difference was found in the ages of CMWs in intervention and non-intervention areas. The majority of CMWs from both groups were less than 25 years of age. There were very few CMWs in non-intervention areas so additional CMWs were included from the neighboring districts of Nushki and Harnai. The majority of the participants (64% in intervention areas and 52% in non-intervention areas) were unmarried. Participants from rural and urban settings were included in the sample. The number of years of experience in the two groups was comparable. The majority of participants (54% from intervention and 33% from non-intervention areas) had work experience of 3-4 years. The average monthly income of participants from the intervention areas was higher than those in the non-intervention areas. However, the average monthly household income was similar in both groups. Participants in intervention and non-intervention areas when asked whether they preferred practicing in a rural versus urban setting indicated preference for working in rural areas. 84% of respondents in intervention areas preferred working in rural settings as compared to 60% in non￾intervention areas. The relation between preferred work location with marital status, age and current work location was also explored (). No significance association was seen. Married CMWs preferred working in rural areas. Table 4: Relation of demographic variables with location of work preference in non￾intervention areas Prefer working in rural (N=36) Don’t prefer working in rural (N=24) P Value Intervention Non-intervention Intervention Non-Intervention f % f % f % f % Marital Status Single 16 38% 17 47% 2 29% 12 50% 0.246 Married 26 62% 19 53% 5 71% 12 50% Age 21-25 years 19 45% 23 64% 3 43% 19 79% 0.15 25-30 years 18 43% 10 28% 4 57% 5 21% 31 years and above 5 12% 3 8% - Current Work Location Rural 24 57% 22 61% 3 43% 8 33% 0.568 Urban 18 43% 14 39% 4 57% 16 67% * Statistically significant at p<0.05 between intervention versus non-intervention areas 36 Inferential Analysis of Choice Data The data generated in the DCE job scenario pair section were analyzed using the mixed logit regression function of STATA program. This modeling technique was used to determine the statistical significance of each job attribute. Analysis of job pair data from intervention and non-intervention groups was carried out separately (Annex 5). However, since the results of both groups were similar, a combined analysis is reported. All the attributes yielded statistical significance as factors influencing the choice of a rural job (at the p<0.05 level), except for support with setup and schooling for children (Table 5). Table 5: Job Attributes and their significance level p values Stipend 0.000* Housing Allowance 0.000* Housing Amenities 0.000* Setup 0.366 Transportation 0.000* Supervision by Government 0.008* Refresher Course 0.000* Schooling 0.698 * Statistically significant at p<0.05 The raw output from STATA includes the p-values as well as coefficients of each of the sub levels within the job attributes (Annex 6). This output was then used to determine the weighted preference ranking of each job attribute by the CMWs. Weighted preference ranking provided a priority ranking order of respondents’ preferences for the job attributes or factors surveyed, and showed how much more respondents favor the most preferred attribute to all the others—i.e., the “weight” or value they placed on an attribute as compared to the other factors/attributes. This ranking was determined by comparing the mean coefficients resulting from the mixed logit regression analysis of the 12 job scenario pairs and listing them from the most (highest mean coefficient) to least preferred (lowest mean coefficient) attribute. Table 6 illustrates the weighted ranking of job attributes in order of highest to least mean coefficient value. 37 Stipend was the most important motivation factor for the CMWs. The degree to which this influenced the CMWs’ preferences depended on value of the stipend. Highest preference was for PKR 10,000/month followed by 7,000 and 5,000/month. Refresher courses were valued higher than supervision while housing with basic amenities was valued more than a housing allowance. Good schooling for children was the least preferred job attribute. Retention Packages In order to develop options for job packages, coefficient values of the job attributes were used. Different potential packages of retention incentives were developed to estimate predicted preference impact. For this purpose, a ‘Preference Calculation Worksheet’ developed in MS Excel by Capacity Plus 32 was used (Annex 7). Attributes for developing packages were selected based on their weighted ranking, discussions with policy makers, project team members and CMWs. The preference impact measure estimates what percentage of the CMWs would prefer a job posting that offers the presented package of incentives to other available jobs that do not have those benefits. In other words, the preference impact measure looks at how the probability of selecting a given post changes as the attributes and levels of those attributes change. The preference impact measure can assist stakeholders in determining which incentives and in what specific combination will be the most attractive to health workers and more likely to motivate and retain them to work in rural and remote areas. Table 6: Weighted ranking of preferences Job Attribute Coefficient 1 PRs 10000/month 1.433 2 PRs 7000/month 1.0031 3 PRs 5000/month .7165 4 Refresher courses .5482413 5 Transport allowance .5441802 6 Housing with basic amenities .5319988 7 Housing allowance .35678 8 Supervision through program (LHS/LHV/WMO) .3001255 9 Regular supply of medicines/ delivery kits .0731114 10 Good schooling for children .0290965 38 Only those packages are presented which showed a preference impact of 80% and above (Table 7). Table 7: Potential retention packages Salary Amount PRs 0 PRs 3000 PRs 5000 PRs 10,000 Incentive Package 1 Housing allowance 72% 78% 82% 89% Transport allowance Supervision through program (LHV/LHS) Refresher courses Incentive Package 2 Transport allowance 69% 75% 79% 87% Supervision through program (LHV/LHS) Refresher courses Incentive Package 3 Supervision through program (LHV/LHS) 60% 67% 72% 82% Refresher courses Package1 Package 1 proposes provision of housing allowance, transport allowance, supervision through program (LHV/LHS) and refresher courses. Results showed that if no extra stipend is given to CMWs and just these four incentives are provided, the rate of retention will increase from the current 50% to 72%. Similarly, if the stipend is increased to PKR 3000, retention rate will further increase to 78%. A further increase in the stipend amount will result in retention of up to 89%. Package 2 Package 2 proposed transport allowance, supervision through program (LHV/LHS) and refresher courses. Results showed that in the absence of a financial stipend, retention will increase to 69%. A progressive increase in the stipend from PKR 3000, PKR 5000 and PKR 10,000 will increase retention to 75%, 79% and 87% respectively. Package 3 In Package 3 is proposed - supervision (through program by LHVs or LHS), and refresher courses only. Results showed that provision of these incentives without stipend will increase retention to 60%. Similarly, progressive increase in stipend from PKR 3000 to PKR 5000 and PKR 10,000 per month will potentially increase the retention to 67%, 72% and 82% respectively. 39 Conclusions The study identified that a lack of quality training services, an inadequate deployment strategy and the harsh environmental conditions of Balochistan led to the poor retention of CMWs, particularly in rural areas. • The BDoH saw retention of CMWs as a challenge for the province due to the geographical terrain of Balochistan and the patriarchal culture that limited the mobility of women. Furthermore, financial constraints, poor human resource management, inadequate deployment strategies, and insufficient training provision resulted in CMWs being poorly trained and utilized. • The BDoH was keen to train CMWs from remote areas but no accelerated learning program for girls was in place to support rural girls to train as CMWs. • CMWs trained were concentrated in certain geographical areas where they had to compete with more qualified health care providers and also traditional birth attendants (Dais). • The CMWs expected employment with the government after the completion of their training. The expectation to sustain themselves in the communities through private practice was perceived by them as a lack of ownership by the government. • Deployment guidelines were published but only implemented in intervention areas. • Supportive supervision was in place for the CMWs in the intervention areas. Intervention area CMWs were able to establish linkages with LHWs, LHVs and LHSs. The monitoring and supervision checklist prepared by the MNCH program was being used in the intervention areas but was not utilized in the non-intervention areas. • CMWs in both areas voiced a need for refresher training. They welcomed support by International NGOs and development partners to establish their service delivery but at the same time wanted to be recognized as a public sector workforce. The findings from the DCE survey suggest that there may be a number of effective strategies to improve the recruitment and retention of health workers in rural and remote areas of Balochistan. CMWs expressed a willingness to take jobs in rural areas if the postings were made more attractive. While a stipend was important, it was seen that when a combination of other valued interventions was offered, a stipend became less important. This finding is consistent with World Health Organization (WHO) and World Bank’s (WB) recommendations for Human Resources for Health (HRH) retention, which contend that increasing salary alone is not enough to motivate 40 health workers in rural and remote areas. Appropriate incentive packages such as housing allowances, transport allowances, supportive supervision and refresher courses are required. • The majority (84%) of respondents in the intervention areas preferred working in rural areas as compared to 60% in non-intervention areas. The preferred work location had no statistically significant association with marital status, age and current work location. • Three packages have been proposed that may improve retention of CMWs: o A package including the provision of a housing allowance, transport allowance, supervision through program (LHV/LHS) and refresher courses with no extra stipend to CMWs suggested an increase in the rate of retention from the current 50% to 72%. If, however a monthly stipend was to be increased to PKR 3000 and PKR 10,000, the retention rate would increase to 78% and 89% respectively. o A package including a transport allowance, supervision through program (LHV/LHS) and refresher courses in the absence of a financial stipend would increase retention to 69%. An increase in stipend from PKR 3000, PKR 5000 and PKR 10,000 will increase retention to 75%, 79% and 87% respectively. o A package with supervision (through program by LHVs or LHS), and refresher courses without stipend would increase retention to 60%. Increase in monthly stipend from PKR 3000 to PKR 5000 and PKR 10,000 will potentially increase retention to 67%, 72% and 82% respectively. Recommendations 1. Selection criteria and accelerated education programs: To counter the low retention of CMWs in the rural and remote areas of Balochistan, new selection criteria for CMWs must be introduced. The health department should seek active collaboration to establish accelerated education programs to provide women and girls with the opportunity to complete higher secondary level education through a fast-track course. A higher secondary accelerated education program in chosen rural areas would raise the education attainment levels of local women and girls and therefore increase the number of women and girls meeting the minimum education standards of CMWs. Steps must also be taken to ensure family support for selected candidates through a thorough orientation for candidates and their families. 41 2. Community involvement in the CMW selection process: Community representation in the CMW selection committee must be implemented to ensure fair selection from each community. 3. Inter-Departmental Coordination: The development of a functional coordination team representing stakeholders from the BDoH health worker programs (LHW, MNCH) and development partners should be considered. The purpose of this coordination team should be to provide oversight for improved recruitment, training, monitoring and supervision of CMWs, through information exchange and inter-departmental collaborations. An accelerated process for midwifery registration with the PNC needs to be instigated, followed by the expedited deployment of CMWS to relevant areas. 4. Functional Human Resource Management (HRM) System: The MNCH program should develop an integrated human resource management system to maintain a record of all CMWs under-going training and deployed. GIS mapping of functional CMWs may also be integrated in the HRM system. 5. Incentives for CMW retention: The decision regarding which incentives or interventions to include in a provincial CMW retention strategy needs to be determined by stakeholders based on political and economic feasibility, with a focus on capacity to deliver. For example, to provide supervision to CMWs, there must be an adequate number of qualified supervisors in the department, trained in the use of modern/current monitoring tools with access to transport and other services for ensuring regular supervisory support within the challenging geographical terrain. 6. Transfer of allocated funds: The allocated program budget for the MNCH program must be transferred from the federal level to the provincial program accounts in its entirety without delays to ensure continuity in the implementation of program goals and objectives. 42 SUB STUDY 2 RESEARCH QUESTION How can CMWs become financially self-sustaining while serving the needs of the poorest of the poor? METHODOLOGY An expenditure and investment assessment of CMWs was undertaken. As this was not a conventional costing model (cost effectiveness or cost utility), the analysis for this was based on an average of various costs to determine operational costs incurred by CMWs. A key objective of the CMW business training program offered by Mercy Corps to CMWs in intervention areas was to improve livelihood opportunities for them while providing community based maternity care. In order to evaluate the impact of the business skills training a comparative financial analysis of deployed CMWs in both intervention and non-intervention areas was undertaken. Using the same sampling strategy and size as sub-study 3, a minimum of 50 CMWs from each of the intervention and non￾intervention area were included in the survey. In all 63 CMWs from the intervention areas (who had undergone business skills training) and 72 CMWs from the non-intervention areas were included. Reaching out and identifying CMWs in the non-intervention area was a challenging task. In the absence of a sampling frame, the District Health Office and Mercy Corps staff made multiple field visits to identify practicing CMWs. The CMWs were asked to maintain a daily expenditure log for a month on a structured checklist. Based on this daily log, direct (mobile phone charges, personnel at market rate, medicines, equipment, etc.), indirect (travel, food, chaperones, etc.) and opportunity costs (other best alternative) for delivering the CMW health care services were reported. Additional investments such as establishing their clinics, enhancing networking/marketing, financial management, maintaining quality practices and others investments were recorded as well. The financial data was entered into Microsoft Excel. Income and expenditure statements for each CMW were developed. Income was generated by CMWs through patient fees for services, charges for diagnostic tests and medicines sold to clients. Expenditures included direct and indirect costs of providing care. Direct costs consisted of the cost of drugs, diagnostic kits and other supplies required to provide clinical care. Indirect costs consisted of rent for the clinic facility, monthly costs of procuring drugs and other supplies, and payments made to any hired transport vehicle (if applicable). Opportunity costs, based on market rates, were also added as expenditure (see tools in Annex 8). Net income was calculated by subtracting total expenditures from total 43 income. The average costs of services provided by CMWs were also calculated. This data was then categorized and analyzed in SPSS, along with background information collected from CMWs. The background information included socio-demographics, use of Tameer (or any other) loan and initial investment in establishing the clinical facility. These findings were supplemented by FGDs with CMWs and key informant interviews with the concerned officials of the Health department (at the provincial and district level) and Mercy Corps staff. RESULTS The socio-demographic characteristics of CMWs in intervention and non-intervention areas are outlined in Table 8. A total of 63 CMWs were randomly selected from intervention areas (Quetta, Kech and Gwadar) and 72 from non-intervention areas (Quetta, Kech, Gwadar, Noshki and Harnai). The majority of the CMWs from both areas were between the ages of 21 to 30 years. More than 45% of the participants had 12 years of education (Intermediate). Most of the participants in both areas were unmarried (Intervention 76%, Non-Intervention 54%). The reported monthly income of the CMWs ranged between PKR 1000 to 10,000. Forty % of CMWs from intervention areas reported a monthly income of PKR 1000 to 3000. 44 Table 8: Demographic information of respondents Intervention (N=63) f (%) Non-intervention (N=72) f (%) Age Not given 4(6%) 1(1%) Less than 20 5(8%) 9(13%) 21-25 years 28(44%) 31(43%) 26-30 years 23(37%) 25(35%) More than 30 years 3(5%) 6(8%) Education Secondary 12(19%) 19(27%) Intermediate 29(46%) 34(47%) Graduate and above 21(33%) 19(27%) Madrasah 1(2%) 0(0%) Marital Status Married 15(24%) 33(46%) Unmarried 48(76%) 39(54%) Monthly Income None 1(1%) 0(0%) <1000 3(5%) 3(4%) 1000-3000 25(40%) 12(17%) 3000-5000 10(16%) 17(24%) 5000-7000 4(6%) 1(1%) 7000-10,000 5(8%) 3(4%) Don’t Know 0(0%) 34(47%) Not given 15(24%) 2(3%) Financial Analysis of CMW practices Financial data of only ‘functional’ CMWs was analysed. A functional CMWs was defined as one who had worked for a minimum of 7 days in that month and maintained a complete record of her activities in the logbook. Findings in this section are based on cost data (complete records) from 29 of the 63 CMWs from intervention areas, and 8 of the 72 CMWs from non-intervention areas. One outlier in the intervention area was excluded as she was working at a clinic with a doctor due to which her reported income was considerably higher. Cost data was entered and analysed in Microsoft Excel to make calculations of net income of each CMW and the average cost of various activities was calculated. Net income was calculated by subtracting total expenditures from total income. Averages were calculated for the start-up costs, materials purchased, travel and fees charged for services. All costs are reported in Pakistani Rupees. 45 Cash Flows and Net Income Table 9 shows the income and expenditure statement of CMWs from the intervention districts. Overall the mean net income was PKR 2581 (ranging from negative 4910 to 13450 rupees). Excluding the negative cash flows, average income was calculated to be PKR 4510 per month. Seven of the Mercy Corps supported CMWs had negative cash flows as per the reported data. A negative cash flow is indicative of investment being higher than monthly income i.e. CMWs were earning less than their monthly expenditure on service provision. Only two of the respondents had a net income above PKR 10,000, while only one CMW was earning the standard minimum wage of PKR 13,000. Details of the cash flows are in the table below. Table9: Cash flow data – intervention areas ID Monthly expense (Maintenance, HR, Utilities) Direct Indirect Total Expense Total Income Net Income (per month) 1 0 2180 3300 5480 3450 -2030 5 100 380 0 480 3925 3445 9 100 0 0 100 4600 4500 10 500 1400 1000 2900 4000 1100 11 100 5500 1650 7250 2340 -4910 12 50 2500 500 3050 5310 2260 32 0 100 1500 1600 15050 13450 40 10000 1265 0 11265 16380 5115 41 1600 2550 400 4550 6478 1928 42 0 710 0 710 2540 1830 44 4800 3218 50 8068 11070 3002 45 4300 6305 0 10605 20790 10185 46 4500 6875 480 11855 15700 3845 47 0 1855 0 1855 5410 3555 48 0 1060 0 1060 3560 2500 49 8500 1200 0 9700 6825 -2875 50 6500 4470 600 11570 20930 9360 46 52 100 1300 0 1400 9720 8320 53 0 2591 0 2591 9780 7189 54 4750 1210 0 5960 6320 360 55 4700 1800 0 6500 5580 -920 56 3000 820 970 4790 9500 4710 57 3000 3915 200 7115 2700 -4415 58 1200 3090 0 4290 4555 265 59 5000 3240 320 8560 13945 5385 60 2300 820 0 3120 1230 -1890 47 Table 10 shows income and expenditure statements from 8 CMWs from non-intervention areas. Overall the mean net income was negative 462 (ranging from negative 7015 to 17300 rupees). Excluding the negative cash flows, average income was calculated to be PKR 4510 per month. Four of the CMWs reported negative cash flows in the recorded data. Only one respondent had a net income above the standard minimum wage of PKR 130,00. Details of the cash flows are in the table below. Table 10: Cash flow data - non-intervention areas Respondent ID Monthly expense (Maintenance, HR, Utilities) Direct Indirect Total Expense Total Income Net Income (per month) 21 0 5000 915 1100 0 -7015 22 0 0 840 0 895 55 23 0 0 905 210 1220 105 24 0 0 980 0 1560 580 25 0 5000 65 20 760 -4325 27 0 0 2000 0 19300 17300 28 0 4500 570 80 1270 -3880 31 0 6000 305 210 0 -6515 Average income (all inclusive) -461.875 Average income (only those with positive net income) 4510 The CMWs in the intervention and non-intervention areas were also asked about how they perceived their own business skills. Only 24 of the CMWs from non-intervention responded to this ID Monthly expense (Maintenance, HR, Utilities) Direct Indirect Total Expense Total Income Net Income (per month) 61 2000 0 0 2000 180 -1820 62 0 695 200 895 3300 2405 63 5000 1950 0 6950 5950 -1000 Average income (all inclusive) 2581 Average income (only those with positive net income) 4510 48 section of the survey. More than 90% of CMWs from both areas responded as being well aware of business skills required for the set-up of a maternity care services clinic, including the principles of financial management, investment rules and regulations, and knowledge of business plan development. The use of telephones, fax machines and personal computers for establishing and maintaining a business was identified as being important by almost 70% of the respondents. The findings of the perceptions survey are in stark contrast to the costing analysis as several of the respondents were incurring losses in their clinical set-up. Average costs of setup and services The CMWs were asked to record their initial investment in developing a functional clinic, expenses incurred due to the purchase of medicines and diagnostic kits, costs incurred due to travel, as well as the fees they charged for services offered (Table 11). Using this data, the average of each category was calculated and is reported in PKR. The average initial investment was PKR 51552, ranging from nil to a maximum of 662,000. Costs above PKR 10,000 were explored by the field staff. It was revealed that costs above PKR 10,000 were often due CMWs investing in construction work to establish a separate clinical space. The average fee charged for a delivery, was PKR 1736, ranging from nil to 3000. The lowest fee charged was for family planning services (average charge PKR 46, ranging from nil to 225). Post-natal care was the least popular service provided, based on the reported services. Details of charges for other services are detailed in Table 4. Table 11: Average costs of setup and services (intervention areas) ID Start-up costs Materials Purchase d (medicine s, diagnosti c kits) Travel Services (Charges) Antenata l Care Post￾natal Care Deliver y Family Planning General Healthcar e Medicine s 1 0 390 250 233 0 850 0 0 210 5 455 127 0 0 0 2500 0 83 588 9 2000 0 0 100 0 1250 0 0 2000 10 500 675 500 225 0 2000 0 0 220 11 1000 589 100 50 0 750 200 0 77 12 0 321 150 200 0 1350 0 0 157 32 3000 100 200 100 0 2000 0 0 1390 40 2500 38 0 100 0 2600 113 0 117 41 100000 319 400 163 0 2000 0 0 229 42 10700 355 0 0 0 2000 20 87 240 49 44 37000 402 50 150 0 3050 0 0 388 45 110000 370 30 478 30 2786 98 395 674 46 40000 313 200 125 0 2000 90 103 313 47 60000 368 15 113 0 850 177 50 372 48 10000 177 0 50 0 2000 50 0 177 49 10500 400 0 0 0 3000 0 50 242 50 662000 574 90 579 283 2375 115 0 409 52 60000 650 0 26 0 4000 0 0 650 53 8500 288 0 50 0 2667 30 0 333 54 168750 130 0 117 0 2500 0 0 89 55 31200 360 0 275 0 2250 0 0 446 56 11110 164 194 0 0 1740 0 0 200 57 45000 260 100 125 0 0 225 275 242 58 35000 224 0 317 0 0 80 162 206 59 15000 183 160 58 500 1972 100 77 147 60 33000 410 0 50 50 0 0 515 0 61 10300 111 160 142 680 680 30 45 137 62 16500 232 0 100 88 0 0 500 475 63 11000 279 0 78 0 1167 0 0 250 Average costs of setup and services (PKR) 51552 304 90 138 56 1736 46 81 379 Investment and expenditure records of CMWs from non-intervention areas were also analysed to understand their initial investment in developing a functional clinic, the cost incurred by purchasing medicines and diagnostic kits and the expense of travel, as well the fees charged for services delivered (Table 12). Averages of each category were calculated and reported as PKR. The average initial investment was PKR 14000, ranging from 4000 to a maximum of 25,000. The average fee charged for a delivery was calculated to be PKR 1681, ranging from 750 to 2600, while the lowest fees were charged for family planning services (average PKR 46, ranging from nil to 225). From the data reported, it appears that postnatal care (PNC) were not provided by any of CMWs in non-intervention areas. Details of charges for other services are detailed in Table 12. Table 12: Average costs of setup and services (non-intervention areas) ID Start￾up Materials Purchase Travel Services (Charges) Antenatal Post Delivery Family General Medicines 50 costs d (medicin es, diagnosti c kits) Care - natal Care Planning Healthcar e 21 4000 390 250 233 0 850 0 0 210 22 4000 127 0 0 0 2500 0 83 588 23 25000 0 0 100 0 1250 130 0 2000 24 15000 675 500 225 0 2000 0 98 220 25 9000 589 100 50 0 750 200 50 77 27 15000 321 150 200 0 1500 0 0 157 28 25000 100 200 100 0 2000 0 200 1390 31 15000 38 0 100 0 2600 113 0 117 Average costs of Setup and Services (PKR) 14000 280 150 126 0 1681 55 54 595 The CMWs are required to provide all services associated with maternity care including antenatal care, delivery, post-natal care; family planning services and general health care and advice. As per the program policy, CMWs can charge their clients a fee for these services. Survey respondents were asked to identify services that were the most and least profitable (Table 13). Delivery services were identified as the most profitable activity by CMWs from both intervention (65%) and non-intervention (29%) areas. The other two services deemed profitable were general health care (36.5%) and antenatal care (31.7%). Less than 3% of the CMWs from non-intervention areas identified general health care and family planning services as being profitable. Table 13: Profitable services provided by CMWs Most profitable services provided by CMWs Intervention areas (N=63) f (%) Non-intervention areas (N=72) f (%) p-value Answer not given 5 1 Antenatal Care 20(31.7) 1(1.4) <0.001* Delivery 41(65.1) 21(29.1) <0.001* Postnatal Care 16(25.4) 1(1.4) <0.001* 51 General Health Care 23(36.5) 2(2.7) <0.001* Family Planning 12(19.0) 2(2.7) 0.003* Least profitable services provided by CMWs Intervention (N=63) f (%) Non-intervention (N=72) f (%) p-value Answer not given 5 7 Antenatal Care 22(34.0) 11(15.2) 0.009* Delivery 2(3.1) 3(4.1) 1.000 Postnatal Care 15(23.8) 2(2.7) <0.001* General Health Care 24(38.1) 6(8.3) <0.001* Family Planning 14(22.2) 1(1.4) <0.001* * Statistically significant at p<0.05 between Intervention versus non-intervention areas CMWs in intervention areas categorized general health care (38%), antenatal care (34%) and family planning (22.2%) as being some of the least profitable services. CMWs from non￾intervention areas also identified the same services as being the least profitable. With reference to financial sustainability, CMWs were asked whether they took any loans (Tamer loans, personal loans etc.) to establish themselves as community based maternal care providers. None of the CMWs in the non-intervention areas reported taking loans to set-up their clinics. In intervention areas 32 CMWs took out a personal loan to support the establishment of their clinics. Reasons given for not taking a Tameer loan included the interest rate on the loan (by 25.3% of respondents) and a lack of need for the loan (by 19% of respondents). Some of the CMWs who had taken loans from their family members reported using the loan to set-up a workstation (28.5%), purchasing equipment (20.6%), buying medicines (41.2%) and purchasing diagnostic kits (22.2%). Six of the respondents from intervention areas indicated that there was no demand for services at a community level or there was presence of other competitors providing maternal care. This resulted in CMW respondents not investing in the establishment of their own maternity care set￾ups. Respondents also identified the need for refresher training in business and midwifery skills. More than 50% of the CMWs from intervention areas and 32% from non-intervention areas 52 expressed the need for training on business skills. More than 20% of the respondents identified need for refresher courses on midwifery skills. Challenges to achieving financial sustainability The qualitative study to understand challenges in achieving financial sustainability of CMWs as community based maternal care providers supplemented the quantitative findings. The following three themes were identified from the data (FGDs with CMWs and IDI with the officials of the BDoH and Mercy Corps): i) Barriers to achieving financial sustainability ii) Entrepreneurship – concept and initiatives iii) Health Department and program support for financial viability i) Barriers to achieving financial sustainability In discussions with CMWs in both intervention and non-intervention areas, many challenges in establishing services as maternal health care providers were identified. A major barrier to the establishment of financially sustainable services was a low demand for their particular services due to issues of access because of the challenging geographical terrain. The CMWs highlighted that due to the challenging geographical landscape they found it difficult to visit households in their catchment population and encourage women to utilize their services. In the interviews with the Mercy Corps team it was mentioned that special permission had to be taken from the BDoH to allow CMWs to establish workstations in their homes. This step was taken to encourage CMWs to establish themselves within their communities. CMWs in intervention areas were also trained by Mercy Corps in business skills and management. Most of the participants appreciated the business skills training and found it informative and useful. However, the majority of CMWs trained in business skills were unable to utilize it effectively. Some of the participants from intervention areas shared that they had established clinics following the training but due to a low demand for services, they were forced to work either as school teachers or start home business such as a salon for women. Some of the participants raised the issue of community members being unwilling to pay for CMW services as they considered the CMWs as public sector employees. Communities were of the view that CMWs were salaried government staff and therefore should not demand a fee for services nor 53 should they charge them for medicines or other materials used for purpose of treatment. The majority of the participants shared that this community attitude was primarily due to their experience of interacting with LHWs who provide free health advice and medicines. ii) Entrepreneurship – concept and initiatives The participants were probed to establish their understanding of entrepreneurship and whether they had successfully used this skill for establishing their CMW services. The participants’ views were consistent with the survey findings. The majority of CMW respondents were not aware of the concept of enterprise and had been unsuccessful in utilizing their business skills training for initiating and sustaining their health services. Very few CMWs had taken any initiatives for collaboration with other service providers. Another salient finding was the difference in understanding of the concept of enterprise between participants from intervention and non￾intervention areas. This maybe attributable to the training CMWs in intervention areas were exposed to and the lack of training in non-intervention areas. Some of the participants shared that that they had taken some steps to establish themselves in their communities. These included efforts to work with the local TBAs commonly known as Dai. Some had been successful in developing a working relationship with TBAs whereas most of the participants were unable to foster similar arrangements. Some of the participants had also worked in private health care facilities as support staff to doctors or LHVs. However, this was a temporary working arrangement for most CMWs. Some CMWs took on the initiative of providing free services in the initial days of their deployment. Participants highlighted that they did this to make community members aware of their particular skill set and to establish their worth. This practice had been beneficial to them allowing them to demand a fee for services from their clients, based on a positive reputation. Members of the Mercy Corps SMNC program team were of the view that CMWs should be given regular refresher training on business skills and management as it was a difficult concept for CMWs to understand in a one-off training course, especially considering their educational background. The project team members expressed that CMWs should be given an interest free loan so that they are able to establish their CMW workstation. 54 The qualitative findings are in stark contrast to the survey responses regarding CMW’s own perceptions of their business skills. The majority of CMWs responded that they are well aware of the skills required to set-up and run a small business, that they are able to develop a business plan and that they can manage services according to the need of their respective communities. iii) BDoH and program support for financial viability A significant finding from the discussions and financial survey was the importance of the monthly stipend to ensuring the sustainability and survival of the CMWs. Although the value of this stipend was capped at PKR 5,000 (~USD 50) per month, it was highlighted to be useful in ensuring continuity of maternity care services provided by the CMWs. The stipend release was linked with submission of monthly progress reports by the CMWs. Due to this conditionality, CMWs were compelled to achieve some of the targets set for them. Participants who had been unable to establish their clinics highlighted that this stipend assisted them in managing their expenses such as purchasing medicines, paying for travel costs and other related expenditure for the delivery of their service. Continuity of the monthly stipend was the single most fervent demand of the CMWs from non-intervention areas. In the absence of this, the majority of the CMWs were apprehensive about whether they would be able to sustain their services. Another issue raised by the CMWs was their integration within the BDoH. Many CMW respondents expected to be incorporated into the government structure of health workers. Their expectation was based on the previous assimilation of LHWs into the government health system. The CMWs were of the view that as part of the government system, they would have access to much needed support (such as medicines, equipment, transport etc.) to enable their provision of services to target populations. Participants highlighted that at the time of induction into the CMW program, they were given the impression that upon completion they would become part of the public sector health system. However, the reality turned out to be starkly different. The CMWs were required to establish private healthcare services that would be independently financially sustainable. The BDoH and Mercy Corps SMNC program team members were in agreement with the CMW’s need for long-term support from the health system and/or development partners. It was their view that the government’s support for LHWs resulted in their long-term sustainability and facilitated the achievement of program goals. Similar steps for CMWs would ensure their survival as skilled maternal healthcare providers. 55 Conclusions Considering the results of the survey and discussions with CMWs, it is evident that the majority of the CMWs have been unable to establish financially sustainable services, particularly those in non￾intervention areas. The CMWs in intervention areas were provided with significant support from the Mercy Corps SMNC program team. This support resulted in some CMWs successfully establishing themselves as maternal healthcare providers in their respective communities. However, all CMWs struggled to achieve financial sustainability. This is evident from the fact that the net income of only two CMWs (one from an intervention area and one from a non-intervention area) is above the standard minimum wage of PKR 13000 per month. The average income of CMWs in both areas is approximately PKR 4500 per month. The data also shows that several of the CMWs were failing to get any significant returns on their investment. The majority of the CMW’s clientele is either unable to pay for services due to poverty or they were unwilling to do so. In both scenarios, it is evident that the CMWs need continued support for their survival as community based maternal healthcare providers. Recommendations 1. Program support for CMWs: Support from the MNCH program is direly needed to provide supervision and stipends to CMWs for at least five years post-deployment. This would enhance continued service delivery in rural communities. Program support to improve referrals to CMWs, logistics and travel services for CMWs working in rural areas is especially required. 2. Business Skills and Management trainings: There is the need to integrate training on entrepreneurship and small business management into the pre-service CMW training curriculum. The development of such skills can be beneficial to all midwives regardless of development partner support after deployment. Proposed Sustainability Model: Based on the findings of this survey, it is evident that to be an effective and functional community based maternal healthcare provider the CMWs need continuous support from the MNCH program and the BDoH. The majority of CMWs have been unable to capitalize on the entrepreneurial skills they were taught and establish functional services within the community that made the required profit. Those CMWs who have managed to establish 56 a working set-up, rely very much on the monthly stipend they receive from the MNCH program. The BDoH needs to take the lead in the development of a pathway for the integration of CMWs into the provincial health system. This could safeguard the achievement of program objectives and improve maternal and child health indicators in the province. Integration of the CMWs within the provincial health system may ensure long-term sustainability of this workforce and the continuity of available maternal care services in community settings. 57 SUB STUDY 3 RESEARCH QUESTION Do CMWs offer quality care? How? METHODOLOGY A comparative cross sectional study was undertaken to assess the quality of care offered by CMWs in the intervention areas and the non-intervention areas. A sampling frame was developed for the number of CMWs trained and deployed in the three intervention districts. Non-intervention areas with working conditions similar to those in intervention areas were selected in consultation with the BDoH and Mercy Corps SMNC program staff. All the CMWs in the selected areas were listed to generate a sampling frame and proportionate sample from the three districts (20 each from Quetta and Kech and 10 from Gwadar. Samples were taken from both intervention and non￾intervention areas in target districts, making a sample size of 50 CMWs from intervention and 50 CMWs from non-intervention areas). Through simple random sampling, the selected number of CMWs were invited to participate in the study. While this was the case in the intervention areas, drawing of the sample was a challenge in the non-intervention areas as the ground reality was different. Many CMWs in non-intervention areas on record were no longer providing services. Only those CMWs working and/or reporting to the health department were included in the sample. At a confidence level of 90% (z value 1.645), error 10% (0.1) and with assumption of 20% improvement in practices among intervention areas as compared to the control (n=z2 p(1-p)/e2 =(1.645)2 (.20)(.80)/(.1)2) the sample size required was 43. Including a 10% non-response rate, the minimum sample size = 47 So 50 CMWs in each area (intervention and non intervention) were included in the study. (see tools in Annex 9) Quantitative Survey A structured KAP (knowledge, attitude and practices) questionnaire was used to collect information from 100 CMWs (50 from intervention and 50 non-intervention areas). The pre-tested structured questionnaire and observation check list was based on the scope of work of CMWs covering antenatal, natal and postnatal services in addition to assessing familiarity with cross cutting themes such as communication skills, ethical practice, updating knowledge etc. (Annex 3.1). The feasibility of assessing the practices of CMWs was worked out in consultation with the Mercy 58 Corps SMNC program team. Independent LHVs were hired to visit the CMWs in their clinics and shadow CMWs for 1-2 days to observe their practices. Due to time constraints only those services delivered within the days of observation could be assessed. The quality of care offered by the CMWs in intervention and non-intervention areas was then compared. Although the questionnaire used for assessing KAP consisted of three main domains, including, knowledge, attitude and practices, in order to have a better assessment these domains were further divided into 4 main sections i.e. antenatal care, natal care, post-natal care and general care. The knowledge section of the questionnaire consisted of 20 questions. Each correct answer in the knowledge domain carried 1 mark while wrong or ‘don’t know’ answers carried a 0 mark. This gave a score range of 0 – 20 for the knowledge section. The attitude and practice section of the questionnaire consisted of 41 and 33 questions respectively. In the attitude section, ‘not responded’ carried 0 marks while negative attitude such as strongly disagree and disagree were given “1” and “2”, respectively. Positive attitude such as definitely agree and strongly agree carried a score of “3” and “4” respectively. This gave a score range of 0 to 164 for the attitude section. The practices of CMWs were observed during the data collection period. Due to the time limitation, some practices were not possible to observe. So in the case of practice section, ‘not practiced’ was scored 0 while ‘can’t perform’, ‘not competent’, ‘competent’, ‘very competent’ and ‘not observed ‘were scored as “1”, “2”, “3”,” 4”, and “5” respectively. This gave a score range of 0 to 132 for the practice section. The scores in knowledge, attitude and practice domains were categorized as poor (less than and equal to 60 %), moderate (61 to 70 %) and good (above 80 %). Statistical Analysis: For the analysis of data, Statistical Package for Social Sciences software, version 16.0 (SPSS Inc., Chicago, IL) was used. Initially, all information gathered via questionnaires was coded into variables. Both descriptive and inferential statistics involving Chi square test were used to present results. For each test, a p-value of less than 0.05 was considered statistically significant. Qualitative Assessment FGDs were conducted with clients who had received CMW services in the intervention and non￾intervention areas. FGDs gathered client perspectives on the care they were given by CMWs and their views on how it could be improved. Ten FGDs were conducted altogether; five in intervention 59 and five in non-intervention areas (Annex 3.2-FGD Guide). Altogether 59 female clients from intervention areas and 50 from non-intervention areas participated in the discussions. Information on age, number of children, education, husband’s education and household decision maker were also noted. The FGDs included the perspective of women of reproductive age on the quality of care given by CMWs and their knowledge on MNCH issues (knowledge attained through women groups where applicable, nature of knowledge received and how it affected their decision making regarding the use of services offered by CMWs). Content analysis of the data was undertaken to understand clients’ satisfaction with CMW services. RESULTS Knowledge Attitude and Practices of CMWs The socio-demographic characteristics of CMWs in intervention and non-intervention area are described in Table 14. A total of 50 CMWs were randomly selected from intervention areas and 79 from non-intervention areas (details in Annex 10). 23 (29.1%) CMWs in non- intervention areas and 8 (16.0%) in intervention areas had an educational level of graduation (14 years of education). The majority of CMWs in both areas were married and had 2-5 children. The majority of CMWs had a monthly salary of < 10000 rupees and lived in a home where the total household monthly income from all sources was less than Rs. 20000 – this income supported up to 10 people. The majority (above 60%) of the CMWs in both areas owned a television, refrigerator, motorcycle and washing machine and < 15% of CMWs possessed assets such as an air conditioner and a car. 38% of CMWs in intervention areas owned computers, compared to 20.3% of CMWs in non￾intervention areas. No difference was observed in the experience level of CMWs in both areas. However, the CMWs in intervention areas were conducting significantly more deliveries compared to those in non-intervention areas (p-value< 0.05) and had significantly better income (0.006). Table 14: General characteristics of CMWs Intervention areas (N=50) f (%) Non-intervention areas (N=79) f (%) p-value Educational status Madrasah 0 (0.0) 2 (2.5) 0.029* Secondary 10 (20.0) 24 (30.4) Intermediate 32 (64.0) 30 (38.0) Graduate & above 8 (16.0) 23 (29.1) Marital status Single 16 (32.0) 35 (44.3) 0.164 Married 34 (68.0) 44 (55.7) No of alive children 1 6 (25.0) 6 (19.4) 0.096 2-5 15 (62.5) 25 (80.6) 60 >5 3 (12.5) 0 (0.0) Monthly salary (PRs) < 10000 43 (86.0) 78 (98.7) 0.006* 11000-15000 6 (12.0) 0 (0.0) >15000 1 (2.0) 1 (1.3) Overall household income per month (PRs) < 10000 12 (24.0) 32 (41.0) 0.024* 11000 – 20000 18 (36.0) 31 (39.7) > 20000 20 (40.0) 15 (19.2) No of supported people < 5 14 (29.2) 14 (17.7) 0.320 6-10 19 (39.6) 36 (45.6) >10 15 (31.2) 29 (36.7) Own Television 44 (88.0) 60 (75.9) 0.112 Refrigerator 41 (82.0) 55 (69.6) 0.148 Air Conditioner 6 (12.0) 11 (13.9) 0.797 Motorcycle 34 (68.0) 44 (55.7) 0.197 Washing Machine 34 (68.0) 59 (74.7) 0.427 Computer 19 (38.0) 16 (20.3) 0.041* Car 6 (12.0) 9 (11.4) 0.989 Experience as CMW (years) < 1 4 (8.0) 16 (20.3) 0.226 1.0 -3 23 (46.0) 26 (32.9) 3.01-5 16 (32.0) 26 (32.9) >5 7 (14.0) 11 (13.9) Number of deliveries per month None 6 (13.3) 33 (42.9) 0.003* 1 5 (11.1) 12 (15.6) 2-5 28 (62.2) 26 (33.8) >5 6 (13.3) 6 (7.8) Training was adequate for working in the field 50 (100) 78 (98.7) 0.999 * Statistically significant at p<0.05 between intervention versus non-intervention areas The knowledge required by CMWs was sub-divided into four sections including antenatal, natal, newborn and postnatal care. These four sections covered all services that should be provided by CMWs. The correct responses are given in bold in the following tables (15-18). As shown in Table 15, the knowledge regarding the antenatal care was similar in intervention and non-intervention areas, 84.0 % CMWs in intervention areas and 84.8% of CMWs in non-intervention areas were aware of the total number of antenatal check-ups required during pregnancy while 66% of CMWs in intervention and 57% of CMWs in non-intervention areas were able to explain educational messages required for pregnant women. About 60% of CMWs in both groups correctly mentioned focused antenatal care comprising of the monitoring of women’s blood pressure at every visit. 61 Table 5: Knowledge of CMWs regarding antenatal care Intervention areas (N=50) f (%) Non-intervention areas (N=79) f (%) p-value Antenatal checkups during pregnancies 05 Antenatal checkups 7 (14.0) 5 (6.3) 0.119 04 Antenatal checkups 42 (84.0) 67 (84.8) 03 Antenatal checkups 1 (2.0) 7 (8.9) Pregnant women should receive what educational messages Personal Hygiene, rest and exercise during pregnancy 3 (6.0) 10 (12.7) 0.591 Diet and nutrition during pregnancy 10 (20.0) 16 (20.3) Danger signs during pregnancy 4 (8.0) 8 (10.1) All of the above 33 (66.0) 45 (57.0) Focused antenatal care includes what actions Checking the baby’s position at 28 weeks 15 (30.0) 17 (21.5) 0.643 Checking the woman’s blood pressure at every visit 29 (58.0) 50 (63.3) Assessing ankle edema at 36 weeks 2 (4.0) 6 (7.6) Counseling the women about danger signs only at the last visit 4 (8.0) 6 (7.6) * Statistically significant at p<0.05 between intervention versus non-intervention areas CMW knowledge of natal care was assessed through five questions and similar level of knowledge was observed in both intervention and non-intervention areas. A significant lack of knowledge regarding natal care was observed in both intervention and non-intervention areas. The majority of CMWs (86% in intervention and 77.2% in non-intervention areas) correctly explained that the active management of the third stage of labor was required. However, only 30% of CMWs in intervention and 36.7% in non-intervention areas were able to correctly demonstrate the appropriate order of steps in active management of third stage labor and plot the cervical dilation on the partograph. In addition to this, only 44% of CMWs in intervention and 40.5% in non￾intervention areas were able to correctly interpret the findings. Only 20% of CMWs in intervention and 22.8% in non-intervention areas were aware of measures to decrease the risk of infection during childbirth (table 16). 62 Table 16: Knowledge of CMWs regarding natal care Intervention areas n=50 f (%) Non-intervention areas n=79 f (%) p-value Active management of the 3rd stage of labor should be practiced…. Only for women who have a history of post￾partum hemorrhage 1 (2.0) 5 (6.3) 0.446 Only for primipara 4 (8.0) 11 (13.9) Only for multipara 2 (4.0) 2 (2.5) For all women in labor 43 (86.0) 61 (77.2) Appropriate order of steps in active management of the third stage of labor include Controlled cord traction, fundal massage, and oxytocin 30 (60.0) 40 (50.6) 0.521 Intravenous oxytocin, cord clamping and cutting, and fundal massage 3 (6.0) 3 (3.8) Cord clamping and cutting, controlled cord traction, ergometrine administration, and inspection to be sure the placenta is intact 2 (4.0) 7 (8.9) Intramuscular injection of oxytocin, controlled cord traction with countertraction to the uterus, and uterine massage 15 (30.0) 29 (36.7) Intervention areas n=50 f (%) Non-intervention areas n=79 f (%) p-value If a woman is admitted during the active phase of labor cervical dilation is initially plotted on the partograph To the left of the alert line 22 (44.0) 23 (29.1) 0.058 To the right of the alert line 17 (34.0) 30 (38.0) On the alert line 4 (8.0) 2 (2.5) On the action line 7 (14.0) 24 (30.4) Cervical dilation plotted to the right of the alert lines indicates Satisfactory progress in labor 22 (44.0) 32 (40.5) 0.544 Unsatisfactory progress in labor 10 (20.0) 24 (30.4) The end of the latent phase 7 (14.0) 11 (13.9) The end of the active phase 11 (22.0) 12 (15.2) Risk of infection during childbirth decreases by… Performing frequent vaginal examination 7 (14.0) 12 (15.2) 0.015* Rupturing membranes as soon as possible in the first stage of labor 3 (6.0) 16)0 (20.3) Routine catheterization of the bladder before childbirth 11 (22.0) 22 (27.8) Reducing prolonged labor 19 (38.0) 11 (13.9) All of the above 10 (20.0) 18 (22.8) * Statistically significant at p<0.05 between intervention areas versus non-intervention areas 63 Significant variation in responses was observed in the assessment of CMWs knowledge regarding newborn care in intervention and non-intervention areas (Table 17). The assessment showed that knowledge regarding the natal care was minimal in both areas. However, in comparison, CMWs in intervention areas had better knowledge regarding immediate care for newborns. Only 30% of CMWs in intervention and 30.4% in non-intervention areas were aware of the main causes of hyperthermia in newborns and appropriate care of umbilicus (22% in intervention and 13.9% in non-intervention areas). Only one quarter of CMWs (24% in intervention and 26.6% in non￾intervention areas) correctly specified the best way to determine need for resuscitation. Table 17: Knowledge of CMWs regarding new born care Intervention areas (N=50) f (%) Non-intervention areas (N=79) f (%) p￾value Immediate care for a normal newborn includes Skin to skin contact followed by placing the baby in a warning incubator 5 (10.0) 9 (11.4) 0.043* Drying the baby, removing the wet cloth and covering the baby with a clean, dry cloth 15 (30.0) 36 (45.6) Stimulating the baby by slapping the soles of the baby’s feet 1 (2.0) 2 (2.5) Deep suctioning of the airway to remove mucus 1 (2.0) 8 (10.1) All of the above 28 (56.0) 24 (30.4) Which of the following can contribute to hypothermia in newborns The baby is not dried thoroughly immediately after birth 15 (30.0) 24 (30.4) 0.094 The baby is bathed immediately after birth 18 (36.0) 23 (29.1) The baby is dried and placed in skin to skin contact with the mother 4 (8.0) 14 (17.7) A and B 11 (22.0) 8 (10.1) All of the above 2 (4.0) 10 (12.7) 64 CMWs knowledge of postnatal care was assessed through eight questions and a significant variation in responses was observed in intervention and non-intervention areas. Overall, a noticeable proportion of CMWs reported a lack of knowledge regarding postnatal care. However, CMWs in intervention areas were well aware of postpartum hemorrhage, control of eclampsia convulsions, the signs and symptoms of a ruptured uterus, postpartum examinations and danger signs. However, CMWs in non-intervention areas reported better knowledge of how to palpate the uterus and how to provide immediate care within two hours of delivery compared to CMWs in intervention areas (Table 18). Intervention areas (N=50) f (%) Non-intervention areas (N=79) f (%) p￾value Care of the umbilicus should include Cleaning with Alcohol Covering with sterile compress Cleaning with cooled, boiled water and leaving uncovered 26 (52.0) 11 (22.0) 4 (8.0) 46 (58.2) 11 (13.9) 4 (5.1) 0.543 Applying antibiotic cream 9 (18.0) 18 (22.8) The best way to determine if a new born needs resuscitation is to Wait until one minute after birth and assign the Apgar score 17 (34.0) 13 (16.5) 0.019* Listen to the baby’s heart rate 10 (20.0) 10 (12.7) Observe respirations immediately and begin resuscitation if they are less than 30 per minute 11 (22.0) 27 (34.2) Perform resuscitation only if central cyanosis is present 0 (0.0) 8 (10.1) Apgar score at the time of birth 12 (24.0) 21 (26.6) * Statistically significant at p<0.05 between Intervention versus non-intervention areas 65 Table 18: Knowledge of CMWs regarding postnatal care Intervention areas (N=50) f (%) Non-intervention areas (N=79) f (%) p￾value Immediate postpartum hemorrhage can be due to Uterine atony 8 (16.0) 14 (17.7) 0.023* Genital trauma 5 (10.0) 11 (13.9) Retained placenta 3 (6.0) 19 (24.1) All of the above 34 (68.0) 35 (44.3) The most effective way to immediately control eclamptic convulsions is to Give diazepam 4 (8.0) 17 (21.5) 0.014* Give magnesium sulfate 42 (84.0) 47 (59.5) Deliver the baby as soon as possible 4 (8.0) 8 (10.1) Give nifedipine 0 (0.0) 7 (8.9) A woman with ruptured uterus has which of the following signs and symptoms Rapid maternal pulse 6 (12.0) 8 (10.1) 0.037* Persistent abdominal pain and suprapubic tenderness 11 (22.0) 24 (30.4) Fetal distress 1 (2.0) 12 (15.2) All of the above 32 (64.0) 35 (44.3) 66 Intervention areas (N=50) f (%) Non-intervention areas (N=79) f (%) p￾value During the first 2 hours following birth, the provider should Measure the woman’s blood pressure and pulse once, and insert a catheter to empty her bladder 22 (44.0) 19 (24.1) 0.104 Measure the woman’s blood pressure and pulse, and check the uterine tone every 15 minutes 14 (28.0) 31 (39.2) Not disturb the woman if asleep because her rest is more important than her vital signs 2 (4.0) 7 (8.9) Measure the woman’s temperature and pulse, massage the uterus, and perform a vaginal examination to remove clots 12 (24.0) 22 (27.8) After childbirth, the mother should have a postpartum visit with a skilled provider Once, at 3 weeks postpartum 8 (16.0) 11 (13.9) 0.206 Once, at 6 weeks postpartum 5 (10.0) 7 (8.9) Three visit times 30 (60.0) 37 (46.8) Only if she has danger signs 7 (14.0) 24 (30.4) By the tenth day postpartum, you should be able to palpate the uterus Just below the umbilicus 21 (42.0) 22 (27.8) 0.414 At the level of the umbilicus 8 (16.0) 14 (17.7) Just above the symphysis pubis 9 (18.0) 19 (24.1) Halfway between the symphysis pubis and the umbilicus 12 (24.0) 24 (30.4) 67 Intervention areas (N=50) f (%) Non-intervention areas (N=79) f (%) p￾value Each postpartum examination should include Measurement of blood pressure and… Measurement of blood pressure and temperature, and assessment of conjunctiva, breasts, abdomen, perineum, and legs 15 (30.0) 23 (29.1) 0.318 Observation of breastfeeding 2 (4.0) 11 (13.9) Information about contraception, safer sex, and counseling and testing for HIV 1 (2.0) 2 (2.5) All of the above 32 (64.0) 43 (54.4) At each postpartum visit, mother should be counseled to seek care of the following danger signs Normal lochia, temperature 370 C, or slight breast engorgement 3 (6.0) 6 (7.6) 0.395 Edema of hands and face, severe abdominal pain, or sore, cracked nipples 12 (24.0) 25 (31.6) Severe headache, foul-smelling lochia, or calf tenderness 5 (10.0) 5 (6.3) B and C 11 (22.0) 24 (30.4) All of the above 19 (38.0) 19 (24.1) * Statistically significant at p<0.05 between Intervention versus non-intervention areas 68 Using a chi-square test, knowledge scores were categorized into three categories: good (>80%), fair (61-70) and poor (<60) [1]. The results of this categorization are shown in (Table 19). The data revealed that CMW’s knowledge of all domains (antenatal care, natal care and new born care was minimal in both intervention and non-intervention areas. CMW’s knowledge of postnatal care was better in intervention areas compared to non-intervention areas. Table 19: Knowledge score of CMWs Intervention (N=50) f (%) Non-intervention (N=79) f (%) p￾value Knowledge score regarding antenatal care Poor 28 (56.0) 47 (59.5) 0.739 Moderate 21 (42.0) 29 (36.7) Good 1 (2.0) 3 (3.8) Knowledge score regarding natal care Poor 45 (90.0) 73 (92.4) 0.193 Moderate 3 (6.0) 6 (7.6) Good 2 (4.0) 0 (0.0) Knowledge score regarding new born care Poor 46 (92.0) 72 (91.1) 0.386 Moderate 3 (6.0) 7 (8.9) Good 1 (2.0) 0 (0.0) Knowledge score regarding postnatal care Poor 31 (62.0) 64 (81.0) 0.017* Moderate 19 (38.0) 15 (19.0) Overall score Poor 47 (94.0) 76 (96.2) 0.429 Moderate 3 (6.0) 3(3.8) * Statistically significant at p<0.05 between intervention areas versus non-intervention areas Poor score: <60%, Moderate score: 60-80%, Good Score: > 80 69 Similarly, the attitude domain was divided into four sections including antenatal, natal, postnatal and general care questions. Attitude was measured through the Likert scale ranging from 0 to 4, ‘Never’ was scored as 0 while ‘can’t perform’, ‘not competent’, ‘competent’ and ‘very competent’ were scored as “1”, “2”, “3”, and "4”, respectively. CMWs attitude towards antenatal care varied in intervention and non-intervention areas as indicated in Table 20. The findings are based on how CMWs perceived their own performance in delivering care. While the majority of CMWs in intervention areas felt satisfied with their performance in delivering antenatal care services, CMWs in non-intervention areas reported dissatisfaction with the antenatal services they delivered, indicating that CMWs in non-intervention areas did not feel competent in delivering antenatal care. 70 Table 20: Attitude of CMWs towards antenatal care Average Score^ p-value Intervention areas Non-intervention areas I feel confident that I register pregnant mothers as per the guidelines 94 80 <0.001* I am trained to prepare mothers for examination 96 84 <0.001* I am competent to discuss mothers’ problems individually 95 85 <0.001* I can perform abdominal examination for: Assessing fetal growth Determining the lie of fetus 95 67 0.014* I am confident in: Counting fetal heart sounds 92 79 0.009* Measuring maternal blood pressure 96 84 <0.001* Identifying impending eclampsia 87 74 0.146 Weighing mothers 96 84 <0.001* Recording weight in maternal record 96 85 <0.001* Advising on maternal nutrition 94 87 0.006* Correcting retracted nipple 84 75 0.940 Determining expected date of Delivery 90 76 0.023* I am competent to take appropriate action regarding varicose veins 81 65 0.75 I can diagnose the onset of labor 91 77 0.027* Average Score^ p-value Intervention areas Non-intervention areas I am confident that I give appointments to come to health facilities as per protocol 90 79 0.112 I am competent in instructing how to take the supplementary nutrition 94 84 <0.001* Overall Score 92 79 0.009* ^On 100 point scale, a higher score indicated a higher participant’s agreement with the item tested CMWs attitude towards natal care is significantly different between intervention and non￾intervention areas as described in Table 21. CMWs in intervention areas were satisfied with their 71 performance in delivering natal care, while CMWs in non-intervention areas reported that they were dissatisfied with their natal care service delivery. Table 21: Attitude of CMWs towards natal care Average Score^ p-value Intervention areas Non-intervention areas P Performing a delivery without assistance 84 68 <0.001* Managing a post-partum hemorrhage 88 69 0.001* Action to be taken of retained Placenta 88 69 0.004* Assessment of progress of labor 89 72 0.033* Performing an episiotomy 82 60 0.022* Deciding when to perform an episiotomy in multipara 83 58 0.012* Aseptic severance of umbilical cord 91 70 0.003* Clearing of airway of newborn 93 80 .001* Overall Score 87 68 0.001* ^ On a 100 point scale, higher score indicated a higher participant’s agreement with the item tested * Statistically significant at p<0.05 between intervention versus non-intervention areas Significant diversity was found in the attitudes of CMWs in intervention areas regarding post-natal care as compared to CMWs in non-intervention areas as shown in Table 22. The majority of CMWs in intervention areas were satisfied with the post-natal care services they delivered, CMWs in non-intervention area scored significantly less in this domain. Table 22: Attitude of CMWs towards postnatal care Average Score^ p-value Intervention areas Non￾intervention areas Identification of abnormalities of lochia 92 79 0.039* Identification of involuted uterus 94 76 0.005* Cleaning episiotomy wound 91 76 0.034* Examination of breasts for infection 95 77 0.020* Measuring mothers' temperature 94 84 0.004* 72 Teaching a family member to care for mother 97 85 <0.001* Advising on maternal nutrition 96 87 <0.001* Overall Score 94 80 0.001 ^ On 100 point scale, a higher score indicated a higher participant’s agreement with the item tested * Statistically significant at p<0.05 between Intervention versus non-intervention areas A noticeable difference was found in the general attitude of CMWs in intervention and non￾intervention areas as indicated in Table 23. The majority CMWs in intervention areas were fully confident in the services they delivered in general care i.e. creating awareness regarding breastfeeding, child spacing etc. while CMWs in non-intervention areas reported a relatively poor attitude towards general care. Table 23: General attitude of CMWs Average Score^ p-value Intervention Non-intervention Delivering a health education talk regarding general care of newborn, breastfeeding, immunization, child spacing 96 88 <0.001* Deliver all services in friendly and helpful manner 96 88 <0.001* If you were to recommend a healthcare provider to a friend or a close relative to avail antenatal, neonatal or perinatal services etc. (for example services given by a CMW), you would recommend seeing a CMW 94 84 0.001* CMWs are a preferred choice health care provider because of convenience and providing home-based care 95 82 <0.001* If your sister was to go into labor I would call a CMW 94 83 <0.001* My community is well informed about CMW services 93 81 <0.001* My community’s preferred choice of 92 84 0.001* 73 healthcare provider for antenatal, perinatal and neonatal services is a CMW My community needs to get more information about a CMW 94 87 <0.001* In my opinion CMWs are doing a good job in the field 95 84 <0.001* In my opinion quality of service provided by a CMW is good 95 85 <0.001* Overall Score 94 84 0.001* ^ On 100 point scale, a higher score indicated a higher participant’s agreement with the item tested * Statistically significant at p<0.05 between intervention versus non-intervention areas Attitude scores were categorized into three categories: good (>80%), fair (61-70) and poor (<60) according to the quartile distribution. The results of this categorization revealed (Table 24) that the attitudes of CMWs in all domains i.e. antenatal care, natal care, newborn care and postnatal care were significantly better than the CMWs in non-intervention areas. Table 24: Attitude score of CMWs Average Score^ p-value Intervention areas Non-intervention areas Attitude score regarding antenatal care Poor 0 (0.0) 7 (8.9) <0.001* Moderate 8 (16.0) 34 (43.0) Good 42 (84.0) 38 (48.1) Attitude score regarding natal care Poor 4 (8.0) 34 (43.0) <0.001* Moderate 11 (22.0) 22 (27.8) Good 35 (70.0) 23 (29.1) Attitude score regarding postnatal care Poor 0 (0.0) 5 (6.3) <0.001* Moderate 7 (14.0) 36 (45.6) Good 43 (86.0) 38 (48.1) Attitude score regarding general health care Poor 0 (0.0) 4 (5.1) <0.001* Moderate 7 (14.0) 28 (35.4) Good 43 (86.0) 47 (59.5) Overall score Poor 0 (0.0) 7 (8.9) <0.001* Moderate 9 (18.0) 38 (48.1) Good 41 (82.0) 34 (43.0) ^ On 100 point scale, a higher score indicated a higher participant’s agreement with the item tested * Statistically significant at p<0.05 between Intervention versus non-intervention areas 74 CMWs practices regarding antenatal care were scored on a scale of 100. The average scores for CMW respondents are shown in Tables 25. Significant variation was observed during the assessment of CMW practices in intervention and non-intervention areas. Antenatal practices of CMWs in intervention areas were significantly better than those of CMWs in non-intervention areas. Table 25: Practices of CMWs towards antenatal care Average Score^ p-value Intervention Non￾intervention Confident in registering pregnant mothers as per the guidelines 94 79 <0.001* Trained to prepare mothers for examination 95 81 <0.001* Competent to discuss mothers’ problems individually 95 81 <0.001* Competent to perform abdominal examination for: Assessing fetal growth Determining the lie of fetus 92 72 0.014* Confident in: Counting fetal heart sounds 94 78 0.009* Measuring maternal blood pressure 98 89 <0.001* Identifying impending eclampsia 93 69 0.146 Weighing mothers 99 88 <0.001* Recording weight in maternal record 99 84 <0.001* Advising on maternal nutrition 98 86 0.006* Correcting a retracted nipple 89 68 0.940 Determining expected date of Delivery 94 70 0.023* Competent to take appropriate action regarding varicose veins 86 61 0.75 Trained in diagnosing onset of labor 94 73 0.027* Confident to give appointments to come to health facilities as per protocol 92 77 0.112 Competent in instructing how to take the supplementary nutrition 96 83 <0.001* Overall Score 92 74 0.01* ^ On 100 point scale, a higher score indicated a higher participant’s agreement with the item tested * Statistically significant at p<0.05 between Intervention versus non-intervention areas Significant difference in CMW practices in the delivering of antenatal care was observed in intervention and non-intervention areas as outlined in Table 26. CMWs in intervention areas were more competent in delivering natal services with an average of 94%, while 75 Significant diversity was found in the postnatal care practices of CMWs in intervention areas compared to CMWs in non-intervention areas as shown in Table 27. The majority of CMWs in intervention areas were confident in conducting postnatal checkups i.e. identification of abnormalities of lochia and involuted uterus, cleaning episiotomy wound, examining for breast infection etc. CMWs in non-intervention areas (average score 61%) were not confident in delivering natal care services i.e. performing episiotomy, retaining placenta, managing a post-partum hemorrhage and performing delivery without assistance. Table 26: Practices of CMWs towards natal care Average Score^ p-value Intervention areas Non-intervention areas Competent to: Perform a delivery without assistance 95 61 <0.001* Manage a post-partum hemorrhage 95 61 0.001* Take action for a retained Placenta 95 60 0.004* Assess progress of labor 94 65 0.033* Perform an episiotomy 89 53 0.022* Decide when to perform an episiotomy in multipara 92 49 0.012* Aseptic severance of umbilical cord 96 67 0.003* Clear airway of newborn 97 75 .001* Overall Score 94 61 0.001* ^ On 100 point scale, a higher score indicated a higher participant’s agreement with the item tested * Statistically significant at p<0.05 between Intervention versus non-intervention areas 76 A noticeable difference was observed in general practices of CMWs in intervention and non￾intervention area as indicated in Table 28. The majority CMWs in intervention areas were fully confident about the general services they provided i.e. creating awareness of breast-feeding, child -spacing etc. while CMWs in non-intervention areas were performing low in general care. Table 27: Practices of CMWs towards postnatal care Average Score^ p-value Intervention areas Non-intervention areas Confident in: Identification of abnormalities of lochia 98 77 0.039* Identification of involuted uterus 96 75 0.005* Cleaning episiotomy wound 96 67 0.025* Examination of breasts for infection 93 74 0.020* Measuring mothers' temperature 96 84 0.004* Teaching a family member to care for mother 97 85 <0.001* Advising on maternal nutrition 98 88 <0.001* Overall Score 96 78 0.005* ^ On 100 point scale, a higher score indicated a higher participant’s agreement with the item tested * Statistically significant at p<0.05 between intervention versus non-intervention areas 77 Scores were categorized into three categories: good (>80%), fair (61-70) and poor (<60). The results of this categorization (Table 29) revealed a clear gradient within the different levels of CMW practices in antenatal, natal, and postnatal care in intervention and non-intervention areas. CMW practices were significantly poorer in non-intervention areas as only 34.5% CMWs in non￾intervention areas were competent in delivering services whereas 96.6% of CMWs in intervention areas were performing their services with high competency. Table 6: General practice of CMWs Average Score^ p-value Intervention areas Non-intervention areas Confident in: Delivering a health education talk regarding general care of newborn, breastfeeding, immunization, child spacing 96 86 <0.001* Deliver all services in friendly and helpful manner 98 88 <0.001* Overall Score 97 87 0.001* ^ On 100 point scale, a higher score indicated a higher participant’s agreement with the item tested * Statistically significant at p<0.05 between Intervention versus non-intervention areas Table 7: Practice score of CMWs Average Score^ p-value Intervention areas Non-intervention areas Practice score regarding antenatal care Poor 0 (0.0) 11 (17.2) <0.001* Moderate 0 (0.0) 24 (37.5) Good 30 (100) 29 (45.3) Practice score regarding natal care Poor 1 (2.7) 34 (54.8) <0.001* Moderate 3 (8.1) 17 (27.4) 78 Client Satisfaction Altogether 59 clients from intervention areas and 50 from non-intervention areas were included in the FGDs to explore their satisfaction with CMW services. A description of the respondents is given in Table 30 (Annex 10). Table 8: Demographic Information of Respondents Intervention (n=60) Non-Intervention (n=50) F (%) F (%) Age less than 25 years 11(15%) 11(33%) 26-30 years 10(14%) 9(18%) 31-35 years 18(25%) 16(32%) 36-40 years 28(39%) 11(22%) more than 40 years 5(7%) 3(6%) No. of Children no child 4(7%) 2(4%) 1-4 children 34(57%) 28(56%) 5-8 children 20(34%) 17(34%) 9-12 children 1(2%) 3(6%) Education Illiterate 37(63%) 37(74%) Primary 5(8%) 2(4%) Good 33 (89.2) 11 (17.7) Practice score regarding postnatal care Poor 0 (0.0) 7 (10.8) <0.001* Moderate 2 (6.5) 32 (49.2) Good 29 (93.5) 26 (40.0) Practice score regarding general health care Poor 0 (0.0) 4 (5.3) <0.001* Moderate 3 (6.2) 27 (35.5) Good 45 (93.8) 45 (59.2) Overall score Poor 0 (0.0) 10 (17.2) <0.001* Moderate 1 (3.4) 28 (48.3) Good 28 (96.6) 20 (34.5) ^ On 100 point scale, a higher score indicated a higher participant’s agreement with the item tested * Statistically significant at p<0.05 between intervention versus non-intervention areas 79 Secondary 11(19%) 6(12%) Matric/FSc 4(7%) 5(10%) Graduation 2(3%) Household Decision Maker Husband 55(92%) 39(78%) Mother in law - - Self 5(8%) - Other - 11(22%) Figure 3: Analysis process moving from main theme to sub-themes to categories . CMWs as acceptable health care providers at a community level The main theme grounded in the data was “CMWs as acceptable health care providers at a community level.” Women in intervention and non-intervention areas considered CMWs as acceptable trained healthcare providers. Not only were they satisfied with their availability but expressed the need for the government to support them to be more effective. “We visit the CMW, as she is near and can provide us with satisfactory services. If there is an issue or complication, she refers us to a hospital.” [Client - Intervention Area, Kech] “We prefer visiting the CMW for the first time as she is nearby and can advise us better.” [Client – Non-Intervention Area, Gwadar] 80 The two sub-themes contributing to the main theme pertained to the perceived quality of care provided by CMWs and are described in the following sections. 1. Provision of Care by CMWs The different types of services provided by CMWs was stated as an important reason for community members to access CMW care. This was identified during the FGDs with clients from both intervention and non-intervention areas. The FGD participants regarded CMWs as trained healthcare providers who empathized with them and were accessible. “We visit the CMW for her advice but if she had the required equipment for pregnancy and delivery we would prefer her as she is near.” [Client – Non-Intervention Area, Gwadar] “She informs the client about the complications during ANC checkups and refers her to the hospital. In case of an emergency she even accompanies women to the hospital.” [Client - Intervention Area, Gwadar] a. Trained Health Care Provider Women interviewed trusted CMWs as a trained health service provider who took care of clients and who would not leave in the middle of an emergency. The respondents expressed their intentions to continue to seek care from CMWs and to advise their families to do so too. “She is trained and treats us well, we are satisfied with her services.” [Client -Intervention Area, Gwadar] “For our relatives and family members we would prefer to consult the CMW for her advice and suggestions. When our relatives visit us we take them to the CMW’s workstation for a checkup and advice.” [Client - Intervention Area, Kech] It was also mentioned that CMWs were more knowledgeable than the traditional birth attendants and were better trained to advise women on maternal and child health issues. They were preferred by the women as they treated their patients with respect and dignity, even at the time of delivery, ensuring that their privacy was maintained. Women shared that they felt comfortable discussing and sharing their problems with CMWs. 81 “She is trained and has knowledge and she is better than a Dai. We can easily share our issues with the CMW. The doctors on the other hand don’t give us proper time or listen to us.” [Client - Intervention Area, Quetta] However, respondents from non-intervention areas were of the view that CMWs were potentially a good healthcare provider provided that they had the equipment and supplies to serve community needs. For delivery, clients had to go to the hospitals. However, CMWs helped them in injecting prescribed medicines at home. One of the respondents said; “She has knowledge but she lacks equipment, otherwise for injections and drips she is always available at home.” [Client – Non-Intervention Area, Gwadar] b. Timely Referral The FGD participants in intervention and non-intervention areas expressed that CMWs provided timely referrals when their clients faced complications. CMWs being knowledgeable and easily accessible were able to assess the condition of the women during antenatal visits and refer them to hospitals or nearby health facilities if required. Women in intervention areas shared that in thee case of an emergency CMWs would go out of their way to arrange for transport and at times accompanied clients to hospital. “She is well trained and knows when to refer to a hospital in case of such complications or issues. She refers us with a referral slip and arranges a vehicle in the neighborhood to take us to hospital and at times comes with us to the hospital as well.” [Client - Intervention Area, Kech] Another respondent said; “She advises and refers us to the referral health facilities in case of complications, and we go there ourselves.” [Client – Non-Intervention Area, Quetta] c. Close Proximity/Availability CMWs were within the reach of the community. This was mentioned by all the respondents. They were of the opinion that such a workforce was an asset to the community given the long distance to the health facility, non-availability of transport and restricted mobility of women. 82 “We are satisfied with her and she is near, we don’t need any transportation to visit her.” [Client - Intervention Area, Gwadar] Many women preferred to visit CMWs first. The other facilities being far away and expensive were only utilized in serious cases. It was stated that hospitals were visited in case of an emergency only. The women in the non-intervention areas suggested that CMWs be given equipment so that their services can be utilized more. “To visit hospital, we require transportation as they are at some distance so we don’t go there until there is an emergency or when we are advised to have a blood or sugar test as CMW doesn’t have equipment.” [Client – Non-Intervention Area, Kech] “We have long waiting times at the hospital. Here at the CMW workstation, she takes proper care and being a female we feel comfortable with her. She is nearby and available all the time.” [Client - Intervention Area, Quetta] 2. Experience of Care from CMWs Overall the respondent’s experience with CMWs was good, especially in the intervention areas. CMWs showed personal interest in their clients and this was appreciated. All participants agreed that CMWs were knowledgeable and had facilities to treat their clients. It was also mentioned that CMWs are a source of information for women in the community. This was true for the women in the intervention areas. CMWs provided information about antenatal and postnatal visits, delivery complications, family planning, nutrition and vaccinations. “She prefers her clients over everything and she tries her best to provide services, she never takes her clients for granted.” [Client - Intervention Area, Kech] Participants stated that CMWs were reachable and willing to provide treatment at any time as they worked out of their own homes. The level of services provided however varied greatly between CMWs working in intervention areas and CMWs working in non-intervention areas. Only a few CMWs in non-intervention areas had maintained their set-up to provide services. CMWs would also make home visits if required. “She is always well prepared and her clinic/workstation is ready for client check-ups and delivery as well. She brings a bag with delivery equipment when she conducts delivery at a client’s home.” [Client - Intervention Area, Quetta] 83 a. Availability of Facilities/Use of Appropriate Technology Participants mentioned that CMWs were trained and well prepared to serve their clients especially the ones in the intervention areas. CMWs also visited homes of the clients with portable equipment and were available to their clients. “She has equipment and is knowledgeable, whenever we need her, we visit her or she comes to our homes but she always treats us well.” [Client - Intervention Area, Kech] Those CMWs that lacked medicines, prescribed medicines to clients for safe delivery. It was also mentioned that CMWs charged minimal fees and provided support to the people in the best possible way. Some of the CMWs in the intervention areas would do simple tests at home as well. “We are satisfied with the way she treats us but she doesn’t have equipment herself for deliveries. She writes prescriptions for her clients and once they bring them, she conducts deliveries. Overall she is good at providing services.” [Client – Non-Intervention Area, Kech] b. Contribution in Raising Awareness Respondents in the intervention areas highlighted the Women Support Groups (WSG) initiated by the CMWs. The women found the WSGs very informative. The WSGs had generated awareness amongst the communities about the services provided by the CMWs. They indicated that CMWs were a rich source of information to the communities. Being qualified and knowledgeable, women trusted the opinions of CMWs. This was evident in the respondents from the intervention areas. Information about family planning including healthy birth spacing and the use of contraceptives was also given to the clients by CMWs. “We visit the CMW sometimes for advice, otherwise we visit doctors at hospitals or private clinics as CMW doesn’t have equipment.” [Client – Non-Intervention Area, Quetta] “At least 2-3 years gap between births i.e. birth spacing is essential for health of mother and child. Besides, post-natal care check-up within first day of birth is also a must.” [Client - Intervention Area, Kech] “We learned about the importance of antenatal and postnatal visits and that at least four antenatal checkups are a must. We have also learned about the birth preparedness and birth spacing.” [Client - Intervention Area, Gwadar] 84 It was evident from the FGDs that participants from the communities in intervention areas knew about WSGs. FGD participants in intervention areas knew that CMWs select a group of active women from within the community and these women attend CMW-led sessions about maternal and child health. These women mobilize the community to access CMW services. Women who attended WSGs were given information about antenatal visits, postnatal care, blood pressure, anemia, complications of delivery and vaccinations by CMWs. Information about family planning and birth spacing was also provided. These sessions were considered useful by the clients. “CMW and other educated women in the community have taught us about maternal and child health topics in classroom like sessions.” [Client - Intervention Area, Kech] “These meetings are very useful, we have learned a lot and now we can take care of ourselves during pregnancy and ensure appropriate immunization and nutrition of newborns and children.” [Client - Intervention Area, Kech] Conclusions Sub-study 3 revealed that the knowledge of CMWs regarding midwifery services (antenatal, natal, post-natal and general services) in both intervention and non-intervention areas was poor. The majority of the CMWs in both areas had been deployed in the field for more than a year, having completed their formal training some time before that. Many CMWs achieved a low knowledge score, probably due to poor recall. However, the attitudes and practices of CMWs in intervention areas scored better compared to those from non-intervention areas. This higher score may be attributable to the regular use of learned skills, supportive supervision and the refresher training CMWs were exposed to via the Mercy Corps SMNC Program. Additionally, the acceptability of the CMWs as trusted health care providers was higher in communities from intervention areas compared to non-intervention areas. This may be due to the adherence of CMWs in intervention areas to the deployment guidelines, and in particular, the introduction of CMWs to communities through the existing network of LHWs. The quality of care provided by CMWs was also better in intervention areas because of the availability of resources (equipment and supplies) and Mercy Corp program support in supervision. 85 Recommendations 1. Quality of care: The MNCH program should mobilize resources (financial and technical) through donor support and private organizations to support CMWs in improved service delivery. This could be achieved through the provision of equipment, supplies, medicines, support in the maintenance of workstations and a continued series of refresher trainings opportunities. 2. Capacity development: The midwifery training curriculum must be implemented in its entirety, including hands-on practical training opportunities and adequate field exposure to support the appropriate skills development of midwives. The training of CMW trainers needs to be improved, including improved training on clinical skills, teaching and student assessments. CMW trainers should receive regular re-orientation. CMW training institutions should be integrated into the existing public health school system 3. Community interaction to improve the acceptability of CMWs as trained healthcare providers: Orientation sessions promoting the services of the CMWs must be designed and conducted with communities of interest before the selection and after the deployment of CMWs. This will ensure the selection of suitable candidates and improved community uptake and support after deployment. 86 SUB STUDY 4 RESEARCH QUESTION How can the DoH streamline CMW reporting using cell phone technology and expand mHealth in the province? METHODOLOGY The implementation of a mHealth application implies multiple features. As per the BDoH’s request to Mercy Corps, the digitalization of the health recording and reporting process for all CMWs living in mobile coverage areas was envisaged for Balochistan. Hence this study focused on exploring the status of the digitalization of the health reporting system in Balochistan, including the identification of system gaps. MHealth was introduced in Balochistan in November 2013 and CMWs started reporting data from their work via mobile phone applications from the 1st January 2014. CMWs were invited for a FGD to examine the acceptability of the new system and key barriers to its implementation. FGDs were conducted with CMWs using mHealth application from all three intervention areas of the program. Discussion was directed about advantage of mHealth application, feasibility and gaps. For completeness, IDIs were also conducted with the Mercy Corps SMNC team to gain an understanding of the mHealth application and its relevance (see study tools in annex 11). The data collected was manually analyzed using thematic analysis and the gaps in the online system were reported. RESULTS Interviews and FGDs were conducted with field staff from the Mercy Corps SMNC program and CMWs in the three intervention districts of Quetta, Kech and Gwadar. Participants were asked about the useful features of the mHealth mobile application and the challenges they faced in using it. The mHealth application is a mobile phone application developed for improved monthly reporting and monitoring of CMWs’ activities, including improved acquisition, storage and processing of client data. The application was installed on smart, touch screen phones given to the CMWs working in intervention areas. The Mercy Corps SMNC program team and CMWs were the two main actors interfacing with the system. The BDoH, especially the MNCH program at a provincial and district level were also reviewing the CMW reports on an mHealth dashboard. Core functions of the application for CMWs included registering clients, scheduling follow-up visits and 87 the closing of client cases. As soon as clients were registered and their contact numbers recorded, CMWs could generate and relay tailored behavior change messages and send reminders for follow up visits. A CMW could also close a client’s case following completion of services. The data was accessible to the Mercy Corps SMNC program team and BDoH staff through a dashboard accessible through a web application. The website supported the review of data, the generation of behavior change messages for registered clients and the analysis of data through customized reports (Annex 12). The transcripts of interviews with the Mercy Corps SMNC program team and CMWs were analyzed through content analysis and the results described below. The themes emerging from the data were; how the mHealth application was being used, challenges faced in the use of the application and the potential benefits the application holds for both the program and the BDoH. 1. Utility of the mHealth application The utility of the mobile application for reporting and monitoring was discussed with the participants in interviews and FGDs and all agreed that the application was useful for both the CMWs, program teams and the health departments. In general, all participants considered the mHealth application a useful tool for reporting and monitoring. The use of the application increased after it was made available in the Urdu language. Compared to the previous practice of recording CMW activities on paper registers, the use of the application was convenient and allowed for the rapid syncing of field data. “Previously we used to carry our registers or we would come home and make our notes in the registers. But now with the phone, we can record everything in the form instantly.” (CMW, Quetta) The Mercy Corps SMNC program staff responsible for providing technical support to the CMWs, agreed that the application allowed for the integration of health data from CMWs into program reporting. This facilitated timely intervention and supervisory support. The system had several set￾up issues which included system adaptation, system implementation, the training of CMWs and project teams on application usage, coordination between all stakeholders to utilize the application and the continuous oversight of the mHealth program in intervention districts. Despite these issues, the CMWs and Mercy Corps program staff believed that the system had improved reporting times. Client records were no longer delayed. 88 “They (CMWs) used to enter information in registers but now we stay updated through this e-system about which CMW is going to which patient, nature of the visit and other details. So we also stay updated. The mHealth system is used and owned by Mercy Corps through which the program staff get all the information from CMWs and can keep a track of the CMWs activities.” (Mercy Corps SMNC program field officer, Gwadar) The CMWs were still, however, required to record activity in the registers they were given, which some of the CMWs considered a duplication of work. The Mercy Corps SMNC program staff cross￾checked the hard copies of the registers with the online report submissions. Despite its usefulness, some issues regarding the use of the mHealth application were highlighted. The mHealth application developers trained the Mercy Corps SMNC program team members (program focal person and five other team members) and three members of the MNCH program in the management of the system. This training included data generation, device installation management, the backup of system records and the troubleshooting of any application related issues at a data entry, collection and review level. Understanding how to use smart phones and relevant functions related to the application was a significant problem for some of the CMWs, especially those from rural areas. Although the CMWs were trained for using this technology and were provided with support from the program team, participants expressed that there should be regular refresher trainings as well. Members of the Mercy Corps SMNC program team also highlighted that CMWs from Quetta were more qualified than those from other areas and were therefore better at using smart phone technology. The CMWs approved of the two-day training given to them on the use of the application but also expressed that refresher training at regular intervals was also required. Mercy Corps SMNC program team members shared that several CMWs required regular assistance from them to use the application properly, for which purpose they visited them frequently. The program team members were of the view that trainings could be improved by increasing the number of training days and incorporating more practical work during the training so that those participants not familiar with the functions of a smart phone may improve their ability. “Before usage of this software, a two-day training session was given to CMWs in which they were told about ways of reporting. This training can be made better. Because some CMWs are from rural areas they take time in understanding, so two days is not enough for them. For them 4-5 days are needed and they should be 89 given time to practice during the training so that they understand it completely during the training.” (Mercy Corps SMNC program field officer, Gawadar) Another issue highlighted was a lack of certain data fields in the mobile application for recording observations of clients. This was a cause of confusion and incomplete reports. However, it was mentioned that this issue was resolved by Mercy Corps through the hiring of Pakvista technologies to assess the mHealth application and make the required changes. One of the application features was voice messages and alerts for community members regarding maternal and child health care. However, the CMWs highlighted that either the people in their catchment community did not share their contact numbers or they would not respond to the alerts. Community members often mistook health information messages as advertisements. Considering this, they were restricted to mostly using the reporting feature of the application. Furthermore, voice messages were removed (after April 2016) from the mHealth component by Mercy Corps on the basis of the mid-course review. 2. Challenges in using the mHealth application The recurrent challenges faced by the CMWs and Mercy Corps SNMC program team members in using the application were also discussed. Several issues were highlighted including technical problems with the application, hardware issues, slow network speeds and no cellular network coverage. The participants shared that the mhealth application frequently responded very slowly and sometimes would stop working. This usually happened when there was a new version of the application and the CMWs were required to update it but were unable to do so because of slow network speeds or poor internet connectivity. The participants identified that the application also had a slow response when other applications were open on the smart phone. This was brought to the notice of the Mercy Corps team during the routine monthly meetings. To combat this issue, the CMWs were advised to limit their internet usage and avoid installation of other applications on the phone, which resulted in improvement. However, when such issues arose, it required the CMWs to frequently visit the program office and get the application re-installed along with their client data, as the backup was only available at the program office. “Because the application is not updated it stops working after some time. So they come to us. Sometimes the whole application needs to be uninstalled and then downloaded again. So sometimes it takes two days because of URL code, which is not accepted by software. These are the issues we commonly face. Even if internet 90 is available sometime it isn’t updated anyway.” (Mercy Corps SNMC program field officer, Kech) During the field visits, due to the slow processing speed of the smart phones (with the application installed) participants would sometimes get held up while they were in process of entering a record. This required them to re-start their device and enter the data again. Sometimes the screen would become non-responsive. This was particularly the case with any old handsets the CMWs had been provided with. Participants suggested that the handsets they had be upgraded so that the problems encountered may be reduced. Similar views were also shared by the Mercy Corps SMNC program team members interviewed. They highlighted that the utility of the application is appreciated by the CMWs as well as by development partners but its entire potential is not being utilized. This was not just because the data is not being utilized but also because the system keeps getting jammed and hence the records are not updated regularly, with the result that sometimes reports are delayed by weeks. To counter this problem, the project officer also made the suggestion to give the CMWs handsets with faster processors and having a 4G network adaptability for better internet connectivity. These upgrades would obviously have financial implications. “The mHealth system is better than registers but if the quality of phones is improved it will reduce problems because the phone gets jammed during reporting. Other than that, they should be given 4G system rather than card of Rs. 500 for SIM Internet (which does not provide good service). CMWs do appreciate this system, and even our stakeholders do but the quality of the phone sets has to be improved.” (Mercy Corps SMNC program project officer, Quetta) Another significant issue raised by all participants was poor network coverage in several parts of the intervention areas, especially in rural areas. Due to this, syncing of the records with the main server was often delayed, sometime for weeks at a time. A review of the cellular network coverage in intervention areas shows that Quetta and Gwadar are being provided services by all the telecom networks, whereas Kech has services only by the three major networks. The coverage is concentrated in urban areas and rural areas are not well covered (coverage maps in annex II). The CMWs shared that to overcome this problem, they saved the records and as soon as there was network coverage, they uploaded their reports. In contrast, Mercy Corps SMNC program staff shared that network and coverage issues caused a delay of up to three to four weeks in the syncing of records as records had to be manually synced in monthly review meetings with the 91 main server. The suggested solution for this was to provide the CMWs with a system that allowed for automatic syncing of saved data, as soon as CMWs were in the range of a network. The issue of poor network speed and low coverage was found to be common in most intervention areas, except those that were in the vicinity of a cellular network towers. Improving this, however, was out of the remit of Mercy Corps SMNC program team. 3. Potential benefits for the program and the BDoH The potential of the application and the benefits of such a system for the Mercy Corps SMNC program and the BDoH were also discussed. The CMWs were appreciative of the fact that the system allowed for rapid monitoring of their work as previously their registers were reviewed only whenever a supervisor visited. After implementation of the mHealth system, their reports were managed, evaluated and reviewed through use of a dedicated dashboard managed by the program team. The Mercy Corps SMNC program team members interviewed were of the view that even though there were gaps and flaws in implementation, the information uploaded had potential for effective monitoring and management of the CMW services. This was however, not being fully done by the Mercy Corps program team and BDoH. Currently the online reports were being used to keep track of a CMW’s monthly activity and timely online submission was required for payment of the monthly stipend. Feedback on CMW’s performance was also based on the submitted reports. Mercy Corps also utilized the mhealth generated reports to inform decision-making and is in the process of integrating the mHealth application into the MNCH management information system. “The Mercy Corps program took the initiative of developing an online system of reporting for CMWs to improve reporting practices; the system evolved over time with errors corrected. As a routine, CMWs submit a monthly report and after verification, they are paid their monthly stipend. Similar practice is also followed by the CMWs in other districts, but they submit a paper report.” (Project Manager, SMNC Program, Quetta) The mHealth system is currently not integrated in the district health information system, contributing to its underutilization. The need to work on information sharing between the Mercy Corps SMNC program team and district and provincial health departments was emphasized by the program staff interviewed. It was pointed out that many of the government health system officials were unaware of the existence of this system and the monthly reports generated. This limited 92 awareness amongst government staff of the mhealth data contributed to a lack of productive utilization of the information generated. The Mercy Corps SMNC program staff were of the opinion that using a comprehensive approach to regularly monitor and review the range of outreach services provided by CMWs would enhance the tracking of program performance, successfully monitor strategy implementation and support the achievement of the MNCH program targets. Conclusions The mHealth application has the potential to play a significant role in improving reporting and monitoring of CMW services and of advancing the objectives of the MNCH program. The BDoH however lacks the capacity to take up mHealth technology and to expand its use across the province. Support in terms of infrastructure, capacity building (IT and health professionals) and integration with existing BDoH reporting systems is required. The following are recommendations for improved implementation and utilization of the mHealth system tested by the Mercy Corps SMNC program. 1. Inter-Departmental Integration: Automated reporting system should be integrated into the district health information system and concerned officials briefed on distinctive feature so they can utilities the online reporting. Real time monitoring of service utilization patterns can be initiated, thereby improving outreach of services to underserved areas. Uptake of this system for the monitoring and evaluation of services by the BDoH, as well as development partners will require financial support, technical oversight and effective leadership. 2. Switch over to Personal Digital Assistants: Smart phones given to the CMWs may be replaced with Personal Digital Assistant (PDA) devices. These are touch screen devices that have better processing speeds than a smart phone, innate ability to manage information and wireless networking capability. Use of PDA devices will eliminate the data management problems arising from the use of the smart phones for other purposes such as phone calls, internet browsing and use of other applications. Additionally, cost implications for scaling up will be substantially reduced with the use of PDAs. 3. Refresher trainings: Regular refresher training for CMWs and Mercy Corps SMNC program field officers must be scheduled to ensure continuity in the use of the online reporting system. To tackle the problem of frequent support required from CMWs, instructional videos should be developed and included in the application. The CMWs can use these to guide themselves through the process and troubleshoot any issues arising. 93 REFERENCES 1. Lassi Z, Bhutta Z. Unfolding the universe of newborn health interventions: the role of innovative community-based strategies. BJOG: An International Journal of Obstetrics & Gynaecology. 2011;118:18-21. 2. WHO, UNICEF, UNFPA, World Bank Group and the United Nations Population Division. Trends in maternal mortality: 1990 to 2015 [Internet]. WHO; 2015. Available from: http://data.unicef.org/corecode/uploads/document6/uploaded_pdfs/corecode/Trends-in￾MMR-1990-2015_Full-report_243.pdf 3. Global Health Observatory Data [Internet]. Who.int. 2016 [cited 1 April 2016]. Available from: http://www.who.int/gho/child_health/mortality/neonatal_infant_text/en/ 4. Arulkumaran S. The State of the World's Midwifery, 2011: Organization of Midwifery Services in Sri Lanka. 2011. [http://www.unfpa.org/sowmy/resources/docs/background_papers/01_ArulkumaranS_SriLa nkaServices]. Accessed 20 December 2013. 5. Bureau FH, Ministry of Health, Sri Lanka. Annual Report on Family Health 2010. Columbo: Family Health Bureau; 2012. 6. Haththotuwa R, Senanayake L, Senarath U, Attygalle D. Models of care that have reduced maternal mortality and morbidity in Sri Lanka. Int J Gynecol Obstet. 2012;119:S45–9. [PubMed] 7. Fernando D, Jayatilleka A, Karunaratna V. Pregnancy-reducing maternal deaths and disability in Sri Lanka: national strategies. Br Med Bull. 2003;67:85–98. [PubMed 8. World Health Organization. Trends in Maternal Mortality: 1900 to 2010. WHO, UNICEF, UNFPA and The World Bank Estimates. Geneva, Switzerland: World Health Organization; 2012. UNICEF, UNFPA, The World Bank. 9. Webster PC. Indonesia: the midwife and maternal mortality miasma. Can Med Assoc J. 2013;185:E95–6. 10. Central Statistics Organisation, United Nations Children's Fund. Afghanistan: Monitoring the Situation of Women & Children, Afghanistan Multiple Indicator Cluster Survey 2010/2011. Kabul: Central Statistics Organisation & UNICEF; 2013. 11. WHO. Making pregnancy safer: The critical role of skilled birth attendant- A joint statement by WHO, ICM and FIGO, New Delhi. 2004. p. 30. Back to cited text no. 6 12. Prasad R, Dasgupta R. Missing Midwifery: Relevance for Contemporary Challenges in Maternal Health. Indian J Community Med. 2013;38:9–14. [PMCID: PMC3612303] [PubMed: 23559697] 13. Keramat A, Arab R, Khorasani L. Awareness and Attitude of Midwives and Midwifery Students toward Community Oriented, Midwifery. J Knowl Health. 2012;6:25–3. 14. Foroud A. Midwives’ knowledge, attitude and readiness to practice community oriented midwifery in Kerman. J Ardabil Univ Med Sci. 2003;6:46–52. 15. Liljestrand J, Sambath MR. Socio-economic improvements and health system strengthening of maternity care are contributing to maternal mortality reduction in Cambodia. Reprod Health Matters. 2012;20:62–72. [PubMed: 22789083] 16. Pakistan Demographic and Health Survey 2006–07. Islamabad and Calverton, MA: National Institute of Population Studies and Macro International Inc.; 2008. dhsprogram.com/pubs/pdf/FR200/FR200.pdf. 17. National Maternal New Born and Child Health (MNCH) Program [Internet]. 2005 [cited 3 April 2016]. Available from: http://can-mnch.ca/wp-content/uploads/2013/09/National￾Maternal-Newborn-and-Child-Health-Programme.pdf 94 18. Sarfraz M, Hamid S. Challenges in delivery of skilled maternal care-experiences of community midwives in Pakistan. BMC Pregnancy and Childbirth. 2014; 14(1):59. 19. Mumtaz Z, Levay A, Bhatti A, Salway S. Good on paper: the gap between programme theory and real-world context in Pakistan's Community Midwife programme. BJOG: An International Journal of Obstetrics & Gynaecology. 2014;122(2):249-258. 20. Journal of Asian Midwives (JAM), Vol. 2, Iss. 2 [2015] Published by eCommons@AKU, 2015 from:http://r4d.dfid.gov.uk/pdf/outputs/raf/Are_Community_Midwives_Accessible_RAF_Fina lReport.pdf 21. Rehman S, Ahmed J, Bahadur S, Ferdoos A, Shahab M, Masud N, Exploring operational barriers encountered by community midwives when delivering services in two provinces of Pakistan: A qualitative study. Midwifery. 2015; 31(1), 177-183. 22. USAID, the Community Midwives Program in Pakistan; Research and Development Solutions Policy Briefs Series No. 20, October 2012. Available from: http://www.resdev.org/files/policy_brief/20/20.pdf 23. Mumtaz Z, O’Brien B, Bhatti A, S Jhangri GS. Are community midwives addressing the inequities in access to skilled birth attendance in Punjab, Pakistan? Gender, class and social exclusion. BMC Health Services Research. 2012;12(1):326 23. Jan Y. Community Based Midwives Practice in Patriarchal Social System [Internet]. Ecommons.aku.edu. 2015 [cited 3 April 2016]. Available from: http://ecommons.aku.edu/cgi/viewcontent.cgi?article=1029&context=jam 24. Faisel A, Shoaib S, Ahamad M, Suleman S, Imran M, Are Community Midwives Accessible in Punjab and Sindh. Maternal and New-born Health Programme Research & Advocacy Fund. 2012. Available 25. Khan SA, Zaman T, Shams B, Shehzad M, Yaqoob A, How far can I go? Social mobility of community midwives in Azad Kashmir. Research Advocacy Fund 2012 26. “How far can I go? Social Mobility of CMWs in AJK” [Internet]. RAF; 2012. Available from: http://r4d.dfid.gov.uk/pdf/outputs/raf/SummaryReport_SocialMobility_of_CMWs.pdf 27. Abbas S. Enhancement of knowledge and skills of community midwives in Sindh, Pakistan [Internet]. Ecommons.aku.edu. 2015 [cited 3 April 2016]. Available from: http://ecommons.aku.edu/cgi/viewcontent.cgi?article=1027&context=jam 28. Lalji L. Birth Centre Management and Business Skill Training for Community Midwives of Pakistan. Journal of Asian Midwives [Internet]. 2014 [cited 3 April 2016];. Available from: http://ecommons.aku.edu/cgi/viewcontent.cgi?article=1005&context=jam 29. PDHS 2012-13 [Internet]. 2014. Available from: http://www.nips.org.pk/abstract_files/Priliminary%20Report%20Final.pdf 30. Case for Midcourse Corrections Saving Mothers and Newborns in Communities Revised July 2015. Mercy Corps; 2015. 31. CapacityPlus [Internet]. Capacityplus.org. 2016 [cited 30 June 2016]. Available from: http://www.capacityplus.org/ 32. Rapid Retention Survey Toolkit: Designing Evidence-Based Incentives for Health Workers | CapacityPlus [Internet]. Capacityplus.org. [cited 30 June 2016]. Available from: http://www.capacityplus.org/rapid-retention-survey-toolkit 95 ANNEXURES ANNEX 1: Study 1-List of IDI Participants ANNEX 2: Study 1, 2, 4-List of FGD Participants ANNEX 3: Study 1-IDI & FGD Guide ANNEX 4: Study 1-DCE Questionnaire ANNEX 5: Study 1-Mix Logit separate results of intervention and non-intervention group ANNEX 6: Study 1-Raw output from STATA ANNEX 7: Study 1-Preference calculation worksheet ANNEX 8: Study 2-Study Tools ANNEX 9: Study 3-Study Tools ANNEX 10: Study 3-List of FGD Participants ANNEX 11: Study 4-Study Tools ANNEX 12: Study 4-Pakistan Telecommunication Footprints LIST OF PARTICIPANTS IN IDIs Sub Study 1- Quetta Participants: Mercy Corps’ project staff, concerned district and provincial health officials, LHV/LHS and CMW Tutors Interviewer: Dr. Saima Hamid & Sana Azmat Rana Sr No. Name Designation and Organization Contact Detail 1 Dr. Abdul Qadir Nuashewani Director General Health Services, Balochistan 081-9211356 2 Dr. Abdul Wahid Baloch Provincial lead MNCH program Balochistan 081-9211352 3 Dr. Naqeebullah Provincial Epidemiologist 0300-3869070 4 Dr. Dawood Achakzai Deputy Coordinator LHWs Program 3337828649 5 Aster Noveen Principal Midwifery School 3458323038 6 Azra Joseph Tutor Midwifery School 081-9213020 7 Rukhsana Dost Mohammad Tutor Midwifery School 081-9213020 8 Abida Ashraf Tutor Midwifery School 081-9213020 9 Sajida Saeed Tutor Midwifery School 081-9213020 10 Salma Faiz Non- Intervention group- LHS 3013786702 11 Niamat Bibi Intervention group- LHS 3338899578 12 Gohar Taj (from Hazara town) Intervention group- LHS 3138741192 13 Razia Non-Intervention group- LHV 3342324961 14 Saeeda Intervention group-LHV 3313153331 15 Rubina Amjad Non-Intervention group- LHV 3337814347 16 Shaheen Kousar Intervention group- LHV 3323968242 17 Dr. M. Daood Deputy Program manager HIV/Aids program 18 Dr. Saeedullah Team Leader South, Mercy Corps Pakistan 3008500374 19 Mr. Mazhar Iqbal Monitoring, Evaluation and Learning Manager South 3458347987 20 Dr. Shaihak Riaz Project Officer, Saving Mothers and Newborns in Communities (SMNC) 3003401486 21 Ahmed Ullah Project Manager, Saving Mothers and Newborns in Communities (SMNC 3003819133 Sub Study 1- Kech Participants: Concerned district and provincial health officials, LHV/LHS and CMW Tutors Interviewer: Dr Tahira Sr No. Name Designation and Organization Contact Detail 1 Dr.Sajjad Ahmad District Health Officer ( DHO ) 3212691332 2 Dr. Mohammad Ikram Baloch District Coordinator National Program ( DC-NP ) 3132823255 3 Dr. Abdul Hameed Public Health Specialist (PHS) MNCH Program 3332377026 4 MailaAyaz LHV – BHU Sarikhan 3222641991 5 TohfaNisa LHV MCH Absar 3232073689 6 ZarGul Allah Bukhsh LHS – CD Kallag 3213593312 7 Hozira LHS- CD Shahrak 3361181603 Sub Study 1- Gawadar Participants: Concerned district and provincial health officials, LHV/LHS and CMW Tutors Interviewer: Dr Tahira Sr No. Name Designation and Organization Contact Detail 1 Rozina Asghar LHV DHQ Hospital 3243251861 2 Rahila Abdul Rasool LHV BHU Dor 3248039274 3 Shahida LHS Ormara 3422389604 4 Farida Mohammad LHS Gwadar Town 3352173714 5 Dr. Akhater Ali Buladi DHO Gawadar 3218090878 6 Dr. Abdul Lateef MS DHQ Hospital 3 Years / DC-NP 3218623955 7 Dr. Ghulam Nabi Ex - DC – NP / Deputy DHO 3218081350 8 Dr. Shahnawaz PHS MNCH Program 3218064646 9 Ms Razia Principal Midwifery School 3228192456 10 Ms Farida Tutor Midwifery School 3337957932 LIST OF PARTICIPANTS (Study 1, 2 & 4) Study 1: FGD Quetta FGD (Intervention Group): Participants: CMWs Moderator: Dr. Saima Hamid Sr N o. Name Contact No. No. of Family memb ers Domicile Residence Work Place Distance from home Age Marital Status No. of Kids Experien ce 1 Rehana yaar Muha mmad 0347-1815716 No Local Urban Urban In home 24 Un￾married 0 4 years 2 Rubina Yaqoob 0346-8344558 4 Local Urban Urban In home 38 Un￾married 0 6 years 3 Arifa 0310-8179681 No Local Urban Urban In home 24 Un￾married 0 3 years 4 Sadeeq a 0347-1800330 No Local Rural Urban In home 24 Un￾married 0 3 years 5 Rukhsa na Irshad 0312-8032588 No Local Rural Urban In home 30 Un￾married 0 3 years 6 Fatima 0313-8832873 No Local Urban Urban In home 22 Un￾married 0 3 years 7 Nargis 0347-1815765 No Local Urban Urban In home 24 Un￾married 0 5 years 8 Hasina 0347-8376515 No Local Urban Urban In home 24 Un￾married 0 4 Years 9 Zahra 0335-2527663 3 Local Urban Urban In home 27 Married 3 6 years 10 Dozina 0313-8045086 5 Local Urban Urban In home 29 Married 5 3 years 11 Khursh eed 0347-1803196 1 Local Urban Urban In home 42 Widow 5 10 years 12 Samina 0333-7618876 1 Local Urban Urban In home 28 Married 1 2 years 13 Mah Jabeen 0303-3344064 No Local Urban Urban In home 24 Married 1 2 years 14 Naz bibi 0310-8102287 No Local Urban Urban In home 24 Un￾married 0 2 Years Study 1: FGD Quetta FGD (Intervention Group): Participants: CMWs Moderator: Dr. Saima Hamid Sr N o. Name Contact No. No. of Family memb ers Domicile Residence Work Place Distance from home Age Marital Status No. of Kids Experien ce 1 Sareeta Kumari 0334-2481860 - Local Urban - Kawari Road 32 Married 2 6 years 2 Bushar a Sultana 0332-875339 - Local Urban - Madni Restura nt 40 Married 5 6 years 3 Sabira 0344-1478097 - Local Urban - Kali Paid Khan 36 Married 3 6 years 4 Rukhsa na Gulzar 0344-0236599 - Local Urban - Arbab Karam Road 36 Married 4 6 years 5 Rout Anwar 0333-7847388 - Local Urban - TWFC cant 37 Married 4 3 years 6 Asia Nasir 0312-9957337 - Local Urban - Kali 32 Married 2 2 Years 7 Sarwat 0346-8062144 - Local Urban - Kansi Road 26 Married 2 3 years Study 2 & 4: FGD Quetta (Intervention Group): Participants: CMWs Moderator: Dr. Tahira SrN o. Name Contact No. No. of Family members Domic ile Reside nce Work Place Distance from home Age Marital Status No. of Kids Experience 1 Nargis Parveen 0347- 181776 5 No Local Urban Urban in home 26 Single 0 5 years 2 Bushra Jabeen 0300- 955188 5 No Local Urban Urban in home 24 married 1 5 years 3 Samina Ramzan 0333- 781886 7 No Local Urban Urban in home 27 married 1 2 Years 4 Rukhsan a Arshad 0312- 803250 8 No Local Urban Urban in home 30 married 0 2 Years 5 Fatima 0313- 883287 8 No Local Urban Urban in home 22 married 0 2 Years 6 Mahjabe en 0303- 334406 4 No Local Urban Rural in home 24 married 0 2 Years 7 Farazana 0331- 850900 7 No Local Urban Rural in home 25 Single 1 2 Years 8 Khurshe ed 0347- 180319 6 No Local Urban Urban in home 42 Widow 0 2 Years 9 Ayesha 0344- 105123 7 No Local Urban Rural in home 20 married 0 2 Years 10 Bibi Yaseen 0333- 272967 2 - Local Urban Urban in home 30 married 0 2 Years 11 Bakht bibi 0301- 371020 0 5 Local Urban Rural in home 27 married 3 2 Years 12 Zahra 0335- 252766 3 4 Local Urban Rural in home 27 married 3 5 years Study 2 & 4: FGD Quetta (Non- Intervention Group): Participants: CMWs Moderator: Dr. Tahira SrNo. Name Contact No. No. of Family memb ers Domi cile Reside nce Work Place Distance from home Age Marital Status No. of Kids Experience 1 Wasia Yaqoob 0346- 8344558 4 Local Urban Urban in home married 4 5 years 2 Abida 0331- 8029452 5 Local Urban Urban in home married 4 6 years 3 Norren 0332- 9989677 5 Local Urban Urban in home married 3 6 years 4 Sadiqa 0347- 1800830 0 Local Urban Urban 5 Mint Single 0 3 years 5 Rozina 5 Local Urban Urban 5 Mint married 5 3 years 6 Rehana 0347- 1815716 0 Local Urban Urban 5 Mint Single 0 4 years 7 Sadiqa Rustam 0335- 274613 0 Local Urban Urban 5 Mint Single 0 4 years 8 Hasina 0342- 8326515 0 Local Urban Urban in home Single 0 4 years 9 Zakira 0335- 0233425 1 Local Urban Urban in home married 1 6 years 10 Arza Ackazai 0 Local Urban Urban in home Single 0 3 years Study 1, 2 & 4: FGD Gawadar (Intervention Group): Participants: CMWs Moderator: Dr. Tahira Serial No. Name Contact No. No. of Family members Domicile Residence Work Place Distance from home Age Marital Status No. of Kids Experience 1 Hafeeza Wahid Buhksh - No Local New Town Urban 0 24 Single NA 2 2 Tahira - 4 Local Bal Nagor Rural 0 23 Single NA 2 3 Majida Tahir - No Local Pasni Urban 0 27 Married 2 3 4 Bilqees Faiz - No Local New Abad Urban 0 26 Single NA 3 5 Habeeba Sabzal - No Local Shumby Ismail Urban 0 29 Married 4 3 6 Samina Abid - No Local Dasht Khndaan Rural 10 minutes 26 Married 1 3 7 Shazia - No Local Koh bin Dawood Urban 0 23 Single NA 2 8 Sakeena Ghafoor - No Local Baloch Ward Urban 0 24 Single NA 2 9 Nadia Gul 0335- 2527663 3 Local Suhrabi Rural - 24 Single NA 2 Study 1, 2 & 4: FGD Gawadar (Non-Intervention Group): Participants: CMWs Moderator: Dr. Tahira Serial No. Name Domicile Residence Distance from home Age Marital Status No. of Kids Experience 1 Summaiya Local Koh Ban Ward Kawari Road 20 Single NA 2 years 2 Assia Local New Abad Madni Resturant 21 Single NA 2 years 3 Tasleema Local Chabari Ward Kali Paid Khan 25 Married 0 3 years Study 1, 2 & 4: FGD Kech (Intervention Group): Participants: CMWs Moderator: Dr. Tahira Seria l No. Name Contact No. Domicil e Residence Work Place Distance from home Age Marital Status No. of Kids Experience 1 Dar Gul 0347-9745086 local Kakan Rural 0 25 Married 0 3 years 2 Beebal 0321-3822452 Local Kalatuk Rural 0 25 Single 0 3 years 3 Hameed a 0322-2882679 Local Singani Sar Urban 0 29 Married 3 6 Years 4 Rahat Noor 0321-2677379 Local Fish Market Urban 0 27 Married 0 6 years 5 Shireen 0321-8092005 Local Degari Rural 0 23 Married 0 3 years 6 Zohra 0321-3733695 local MalikAbad Urban 0 28 Single 0 4 years 7 Assia 0323-3293148 Local Koshkalat Rural 0 27 Single 0 6 years 8 Rubina 0323-0225009 Local Nasir Abad Rural 0 25 Married 0 6 years 9 Humaira 0311-1077398 Local Jammuk Rural 0 26 Married 1 6 years 10 Haan Bibi Not given Local Niami Kalag Rural 0 25 Married 0 6 years 11 Gul Afroz 0320- 92866274 local Bahkter Urban 0 27 Married 3 6 years 12 Naheeda 0323-3705784 local Abser Rural 0 22 Single 0 6 years Study 1, 2 & 4: FGD Kech (Non- Intervention Group): Participants: CMWs Moderator: Dr. Tahira Seria l No. Name Contact No. No. of Family memb ers Domicil e Residence Work Place Dista nce from home Ag e Marital Status No. of Kids Experie nce 1 Shaista - - local Jummak Gor Rural 25 25 Married 0 5 Years 2 Shahnaz 0316- 2479395 - Local Jummak Gor Rural 20 24 Married 2 5 Years 3 Meena 0316- 2743402 - Local Jummak Gor Rural 0 26 Married 2 5 Years 4 Sheema Naseem - - Local Absar Urban 0 25 Married 0 5 Years 5 Rakhsha nda 0323- 2160849 - Local Singani Sar Urban 0 25 Married 1 4 Years 6 Zakia 0322- 3611540 - local Singani Sar Urban 0 27 Single 0 4 Years 7 Rajda 0321- 2694918 - Local Jummak Gor Rural 0 23 Married 1 4 Years 8 Humma 0333- 3490643 - local Shahpuk Rural 0 22 Single 0 4 Years 9 Murad Bibi 0322- 051190 - Local Shahpuk Rural 0 23 Married 0 4 Years 10 Sabeeta 0323- 3173942 - Local Pedark Rural 0 23 Single 0 6 Years 11 Yasmeen - - local Perak Rural 0 24 Married 0 6 Years Study 1, 2 & 4: FGD Kech (Non- Intervention Group): Participants: CMWs Moderator: Dr. Tahira Seria l No. Name Domic ile Residence Work Place Distance from home Age Marital Status No. of Kids Experience 1 Ruqia Sabir Ali local Singani Sar Urban 0 30 Married 3 4 Years 2 Zahida local Shahpuk Rural 0 32 Married 0 5 Years 3 Meher Jan local MalikAbad Urban 0 40 Married 2 5 Years 4 Meena local Shahpuk Rural 0 22 Single 3 years 5 Rubina Abdullah local Singani Sar Urban 0 25 Married 0 3 years 6 Zarina Moosa local Shaikhani Bazar Urban 0 27 Married 1 3 years 7 Aangal Akhter local Pedarak Rural 0 25 Married 0 5 Years 8 Naila Sher Mohammad Local Gulshan Abid Urban 0 25 Married 2 5 Years Study 1: FGD guide (CMWs deployed in Intervention and Control group and trainees) After consent and addressing queries, begin as below: I will circulate this sheet for everyone to sign and record your details. We will begin with the introductions once everyone has filled out the sheet. Name Area of residence (rural/urban) Area of Work (rural/urban) Distance from area residence to area of work Age Marital Status Number of Children (if applicable) Experience working as a CMW in months Introductory question 1. Can you all please introduce yourselves one by one? If deployed state where you work and since when have you been working there. Main content 1. What motivated you to become a CMW? a. Financial benefits b. Position in the society, c. Interest/ inspiration, d. Chances to get government job e. Expectations from work f. Support of family g. Security/safety in mobility 2. How were you recruited in the training for CMWs? a. Logistics b. Application process c. Costs incurred d. Positive experiences e.g. any form of facilitation from any authority etc e. Negative experiences 3. What were the eligibility criteria to become a CMW? In your opinion are they relevant or should they be revised? If so, how? 4. What were the important issues you considered when taking up this job? (skip this question with trainees) 5. Were you adequately trained to take up your current roles and responsibilities? (skip this question with trainees) a. Clarity on Roles and Responsibilities • Role in the community for providing maternal health services? • What maternal services/how • Training and skills taught in the school • Importance of their role in improving maternal health • Any reservations/difficulties in providing these services • Marketing of Midwifery services in community • How do the clients come to know about your services? (Source of introduction in community, success factors, communication barriers) 6. Now that you have worked in the field please suggest how the recruitment processes may be improved to better equip you to fulfill your roles and responsibilities? (skip this question with trainees) a. Barriers from supervisors, communities, Health Department i. From supervisors (financial, logistics, lack of support) ii. From community (acceptability, image, respect, age, mobility, access, security, catchment population iii. Competition with local TBAs and other health providers) iv. Mentorship programs v. Refresher trainings, Incentives vi. Linkages with the Health Department (coordination –feedback systems) vii. Appraisal 7. Do you think working in rural areas (urban) allows you to provide the kind and quality of health care you want to provide (e.g., the scope of practice you were trained to provide)? 8. How important is this to you when deciding where you would like to work? 9. What are the factors that you deem essential for work in rural settings? 10. How would you compare the quality of services provided to you (e.g., cleanliness, availability of equipment, access to referral services etc.) in rural areas as compared to urban areas? 11. How do you compare the CMWs working in rural or urban areas? (Satisfaction from work, motivation, willingness to do work) 12. How would additional support from the Health Department to CMWs working in rural areas retain them? a. Salary increase for those working in rural areas? (how much) b. Living conditions (communication, transport, phone, water, electricity etc) c. Social environment (access to social activities) d. Availing leave, substitute worker, opportunities of refresher trainings (How would you compare opportunities for in-service training if you work in a rural area and an urban area? What should the in-service training opportunities be to make working in a rural area more attractive? How important are in-service training opportunities when considering where you will work?) e. Considerations/benefits to children as support for education etc. f. Appraisal g. Timely promotions h. Supportive Supervision i. Security j. business skills workshops k. Mumta fund l. provision of equipment m. Technical/monitoring support from LHV/LHS etc. 13. Are there any other allowances/bonusesthat would be important to motivate CMWs to work in rural areas? What should allowances/bonuses be given for? How much should they be? 14. How would you compare the opportunities for your career development if you work in a rural area versus urban areas? What should the career development opportunities be to make working in a rural area more attractive? How important are these opportunities when considering where you will work? 15. How would you compare community support given to CMWs in rural areas than in urban areas? How important is community support to you when considering where you will work? (any stigma associated with the work) 16. Are there any other factors, which have not yet been mentioned, that are important to you when deciding where you will work (rural /urban)? I thank you for your contributions in the discussion. I will now circulate a list of factors that we have discussed that affect your decision to serve as a CMW in rural area/urban area. Please rank the 4 most important factors that you consider when taking the decision of where to work. These include: • The types of health care you may provide (scope of practice) • The quality of facilities, including availability of equipment, drug supply, etc. • Supportive management • Career mentoring programs • Salary • Housing • Living conditions (electricity, water, social activities) • Transportation • Performance bonuses • Children’s education • Career advancement/promotion opportunities • Opportunities for continued education • Community support • Security • Supportive supervision • (Add any other factors mentioned that seem important). (Facilitator: Distribute a copy of the table below to each participant for them to rank their top four choices. Collect the form from the participants when they are done.) PS. Before adjourning, after consent to include the photo in the final report of the project, take a photograph while sitting in a circle Consent Form Study 1 Introduction Thank you very much for coming to this meeting. I welcome you on behalf of the Department of Health, Mercy Corps and Health Services Academy. This study is being undertaken to learn about Community Midwives’ motivation for working in rural districts of Balochistan. We are interested in understanding the how the health department can better recruit CMWs to retain them in the system. This focus group would approximately take an hour. Your participation will help us to create a survey questionnaire that will be administered to other CMWs as part of this study. If you have any queries I would be happy to address them. If you agree to participate in this study, please sign at the bottom and if not then please state the reason for refusal. Agreed Reused Reason for refusal Phase I: Qualitative Section In-depth Interview Guide (CMW Trainers, LHS, Health Department Officials – district/provincial level, Mercy Corps Project Team) After consent and addressing queries begin the questions as below: Introductory question 1. Can you all please introduce yourself? State where you work and since when have you been working there. Main content 1. How is your current job related to the CMWs? Please explain 2. In your experience how should the CMWs recruited to improve their retention in rural settings? a. Selection Process b. Training Program c. Deployment- When a CMW is deployed in your area, how does community know about her? i. Coordination between LHWs and CMWs (introduction, referral, information sharing) ii. Communication and coordination barriers /solution d. Role and responsibilities e. Post deployment support f. Monitoring g. Technical support i. Service delivery set up (home delivery, workstation, charges) ii. What is your opinion regarding maternal services delivered by CMWs? 1. Skills of CMWs 2. Performance/issues 3. Community perception about CMWs h. Refresher trainings – training needs i. Other incentives, bond j. In your opinion what are the barriers faced by CMWs in delivery of these services? • Age, image, branding • Acceptability • Motivation, family support • Mobility, catchment area/Logistic support • Retention • Competition with Dais, charges • Work load • Social factors • Private practice k. In your opinion how can these issues be addressed? • Quality supportive supervision/training/authority • Refresher trainings of CMWs • Coordination between LHWs and CMWs • Communication skills • Appraisal PS. Before adjourning, after consent to include the photo in the final report of the project Study 1: FGD guide (CMWs deployed in Intervention and Control group and trainees) After consent and addressing queries, begin as below: I will circulate this sheet for everyone to sign and record your details. We will begin with the introductions once everyone has filled out the sheet. Name Area of residence (rural/urban) Area of Work (rural/urban) Distance from area residence to area of work Age Marital Status Number of Children (if applicable) Experience working as a CMW in months Introductory question 1. Can you all please introduce yourselves one by one? If deployed state where you work and since when have you been working there. Main content 1. What motivated you to become a CMW? a. Financial benefits b. Position in the society, c. Interest/ inspiration, d. Chances to get government job e. Expectations from work f. Support of family g. Security/safety in mobility 2. How were you recruited in the training for CMWs? a. Logistics b. Application process c. Costs incurred d. Positive experiences e.g. any form of facilitation from any authority etc e. Negative experiences 3. What were the eligibility criteria to become a CMW? In your opinion are they relevant or should they be revised? If so, how? 4. What were the important issues you considered when taking up this job? (skip this question with trainees) 5. Were you adequately trained to take up your current roles and responsibilities? (skip this question with trainees) a. Clarity on Roles and Responsibilities • Role in the community for providing maternal health services? • What maternal services/how • Training and skills taught in the school • Importance of their role in improving maternal health • Any reservations/difficulties in providing these services • Marketing of Midwifery services in community • How do the clients come to know about your services? (Source of introduction in community, success factors, communication barriers) 6. Now that you have worked in the field please suggest how the recruitment processes may be improved to better equip you to fulfill your roles and responsibilities? (skip this question with trainees) a. Barriers from supervisors, communities, Health Department i. From supervisors (financial, logistics, lack of support) ii. From community (acceptability, image, respect, age, mobility, access, security, catchment population iii. Competition with local TBAs and other health providers) iv. Mentorship programs v. Refresher trainings, Incentives vi. Linkages with the Health Department (coordination –feedback systems) vii. Appraisal 7. Do you think working in rural areas (urban) allows you to provide the kind and quality of health care you want to provide (e.g., the scope of practice you were trained to provide)? 8. How important is this to you when deciding where you would like to work? 9. What are the factors that you deem essential for work in rural settings? 10. How would you compare the quality of services provided to you (e.g., cleanliness, availability of equipment, access to referral services etc.) in rural areas as compared to urban areas? 11. How do you compare the CMWs working in rural or urban areas? (Satisfaction from work, motivation, willingness to do work) 12. How would additional support from the Health Department to CMWs working in rural areas retain them? a. Salary increase for those working in rural areas? (how much) b. Living conditions (communication, transport, phone, water, electricity etc) c. Social environment (access to social activities) d. Availing leave, substitute worker, opportunities of refresher trainings (How would you compare opportunities for in-service training if you work in a rural area and an urban area? What should the in-service training opportunities be to make working in a rural area more attractive? How important are in-service training opportunities when considering where you will work?) e. Considerations/benefits to children as support for education etc. f. Appraisal g. Timely promotions h. Supportive Supervision i. Security j. business skills workshops k. Mumta fund l. provision of equipment m. Technical/monitoring support from LHV/LHS etc. 13. Are there any other allowances/bonusesthat would be important to motivate CMWs to work in rural areas? What should allowances/bonuses be given for? How much should they be? 14. How would you compare the opportunities for your career development if you work in a rural area versus urban areas? What should the career development opportunities be to make working in a rural area more attractive? How important are these opportunities when considering where you will work? 15. How would you compare community support given to CMWs in rural areas than in urban areas? How important is community support to you when considering where you will work? (any stigma associated with the work) 16. Are there any other factors, which have not yet been mentioned, that are important to you when deciding where you will work (rural /urban)? I thank you for your contributions in the discussion. I will now circulate a list of factors that we have discussed that affect your decision to serve as a CMW in rural area/urban area. Please rank the 4 most important factors that you consider when taking the decision of where to work. These include: • The types of health care you may provide (scope of practice) • The quality of facilities, including availability of equipment, drug supply, etc. • Supportive management • Career mentoring programs • Salary • Housing • Living conditions (electricity, water, social activities) • Transportation • Performance bonuses • Children’s education • Career advancement/promotion opportunities • Opportunities for continued education • Community support • Security • Supportive supervision • (Add any other factors mentioned that seem important). (Facilitator: Distribute a copy of the table below to each participant for them to rank their top four choices. Collect the form from the participants when they are done.) PS. Before adjourning, after consent to include the photo in the final report of the project, take a photograph while sitting in a circle Survey 1 Survey 1 Improving Maternal and Newborn Services: Strengthening Community Midwives in Balochistan Introduction Thank you very much for taking out time for this survey. I welcome you on behalf of the Department of Health, Mercy Corps and Health Services Academy. This study is being undertaken to learn about factors that would motivate Community Midwives for working in rural districts of Balochistan. We are interested in understanding how the health department can improve recruitment of CMWs and retain them in the system. Since you have completed your 18 months training and are looking towards the future, we are interested in knowing more about incentives or characteristics that would influence your decision to work in a rural area as a CMW. This survey should take approximately 20-30 minutes. You will be asked questions to obtain demographic and other background information. Then, there will be a series of questions about hypothetical job postings. Your participation will help us in identifying appropriate incentives and characteristics to motivate CMWs to work in rural areas. Please read the question carefully and give your most honest responses throughout the questionnaire. There is no right or wrong answer. We will ensure confidentiality of the information you share.. If you agree to participate in this study, please sign at the bottom and if not then please state the reason for refusal. If you have any questions during the survey, please feel free to ask me. Thank you for your participation! Agreed Reused Reason for refusal Survey 1 Background Information 1. Name 2. Contact Number 3. Domicile: a) Punjab b) Sindh c) Balochistan d) KPK e) AJK 4. Age: a) 14-16 b) 17-20 c) 21-25 d) 25-30 e) 31 and above 5. District: a) Quetta b) Gawadar c) Kech 6. Marital Status: a) Single b) Married c) Divorced 7. Current location of work: a) Rural b) Urban 8. Name of area where you work 9. Your average monthly income a) Less than 3000/month b) Between 3000-5000/ month c) Between 5000-7000/ month d) Between 7000-10000/month 10. Monthly household income a) Less than 10,000/month b) Between 10,000-15,000/ month c) Between 15,000-20,000/ month d) Between 20, 000-30,000/ month Survey 1 e) More than 30,000/month 11. No. of dependents 12. Number of children? 13. Work experience as a CMW (in years) 14. Would you prefer working in a rural area over urban area? Yes/N Survey 1 Scenario Imagine that you have just successfully completed your 18 months CMW training. Through newspaper, radio, and other sources, and you find that there are two work packages that health department offers. For both packages, the location is rural Balochistan. However, each of the two postings provides different characteristics or benefits. Please imagine yourself in this situation and make a decision as to which of the two presented work package you would prefer. For the sake of this survey please assume that you would indeed receive the full benefits described for the package. In making your choice, please carefully read the full list of benefits for each work package and do not imagine any additional features of it. There are 12 different scenarios presented. Please note that while they may look similar at a quick glance, they are indeed each very different. Survey 1 Which of these job postings do you prefer? Select one by marking the circle under the job you prefer. Job package 1 Job Package 2 Housing None Housing with Basic Amenities CMW Practice Setup Seed money for setup Regular continuous supply medicines/ delivery kits Transportation None Transport Allowance Supportive management No supervision Refresher courses Stipend Rs 7000/ Month Rs 5000/ Month Good schooling for children yes No Survey 1 Which of these job postings do you prefer? Select one by marking the circle under the job you prefer. Job package 1 Job Package 2 Housing House Allowance Housing with Basic Amenities CMW Practice Setup Regular continuous supply medicines/ delivery kits Seed money for setup Tranportation Transport Allowance None Supportive management Supervision through program(LHV/LHS/WMO) Supervision through program(LHV/LHS/WMO) Stipend Rs 12000/ Month Rs 12000/ Month Good schooling for children yes No Survey 1 Which of these job postings do you prefer? Select one by marking the circle under the job you prefer. Job package 1 Job Package 2 Housing House Allowance None CMW Practice Setup Seed money for setup Seed money for setup Tranportation Transport Allowance None Supportive management No supervision No supervision Stipend Rs 7000/ Month Rs 5000/ Month Good schooling for children No yes Survey 1 Which of these job postings do you prefer? Select one by marking the circle under the job you prefer. Job package 1 Job Package 2 Housing Housing with Basic Amenities House Allowance CMW Practice Setup Seed money for setup Regular continuous supply medicines/ delivery kits Tranportation Transport Allowance None Supportive management Refresher courses Refresher courses Stipend Rs 12000/ Month Rs 7000/ Month Good schooling for children No yes Survey 1 Which of these job postings do you prefer? Select one by marking the circle under the job you prefer. Job package 1 Job Package 2 Housing None House Allowance CMW Practice Setup Regular continuous supply medicines/ delivery kits Regular continuous supply medicines/ delivery kits Tranportation Transport Allowance None Supportive management Refresher courses Supervision through program(LHV/LHS/WMO) Stipend Rs 7000/ Month Rs 5000/ Month Good schooling for children No yes Survey 1 Which of these job postings do you prefer? Select one by marking the circle under the job you prefer. Job package 1 Job Package 2 Housing None Housing with Basic Amenities CMW Practice Setup Seed money for setup Regular continuous supply medicines/ delivery kits Tranportation None None Supportive management Supervision through program(LHV/LHS/WMO) Supervision through program(LHV/LHS/WMO) Stipend Rs 5000/ Month Rs 7000/ Month Good schooling for children yes No Survey 1 Which of these job postings do you prefer? Select one by marking the circle under the job you prefer. Job package 1 Job Package 2 Housing None House Allowance CMW Practice Setup Regular continuous supply medicines/ delivery kits Seed money for setup Tranportation Transport Allowance None Supportive management No supervision No supervision Stipend Rs 5000/ Month Rs 12000/ Month Good schooling for children yes yes Survey 1 Which of these job postings do you prefer? Select one by marking the circle under the job you prefer. Job package 1 Job Package 2 Housing House Allowance None CMW Practice Setup Seed money for setup Regular continuous supply medicines/ delivery kits Tranportation Transport Allowance Transport Allowance Supportive management Refresher courses Supervision through program(LHV/LHS/WMO) Stipend Rs 7000/ Month Rs 12000/ Month Good schooling for children No No Survey 1 Which of these job postings do you prefer? Select one by marking the circle under the job you prefer. Job package 1 Job Package 2 Housing House Allowance Housing with Basic Amenities CMW Practice Setup Regular continuous supply medicines/ delivery kits Regular continuous supply medicines/ delivery kits Tranportation Transport Allowance None Supportive management No supervision Refresher courses Stipend Rs 12000/ Month Rs 5000/ Month Good schooling for children yes No Survey 1 Which of these job postings do you prefer? Select one by marking the circle under the job you prefer. Job package 1 Job Package 2 Housing None Housing with Basic Amenities CMW Practice Setup Regular continuous supply medicines/ delivery kits Seed money for setup Tranportation Transport Allowance Transport Allowance Supportive management Supervision through program(LHV/LHS) No supervision Stipend Rs 12000/ Month Rs 5000/ Month Good schooling for children yes No Survey 1 Which of these job postings do you prefer? Select one by marking the circle under the job you prefer. Job package 1 Job Package 2 Housing None Housing with Basic Amenities CMW Practice Setup Seed money for setup Seed money for setup Tranportation None None Supportive management Supervision through program(LHV/LHS/WMO) Refresher courses Stipend Rs 7000/ Month Rs 7000/ Month Good schooling for children No yes Survey 1 Which of these job postings do you prefer? Select one by marking the circle under the job you prefer. Job package 1 Job Package 2 Housing Housing with Basic Amenities House Allowance CMW Practice Setup Seed money for setup Regular continuous supply medicines/ delivery kits Tranportation Transport Allowance None Supportive management Supervision through program(LHV/LHS) No supervision Stipend Rs 7000/ Month Rs 12000/ Month Good schooling for children yes No Survey 2 Survey 2 Improving Maternal and Newborn Services: Strengthening Community Midwives in Balochistan Introduction Thank you very much for taking out time for this survey. I welcome you on behalf of the Department of Health, Mercy Corps and Health Services Academy. This study is being undertaken to learn about factors that would motivate Community Midwives for working in rural districts of Balochistan. We are interested in understanding how the health department can improve recruitment of CMWs and retain them in the system. Since you have completed your 18 months training and are looking towards the future, we are interested in knowing more about incentives or characteristics that would influence your decision to work in a rural area as a CMW. This survey should take approximately 20-30 minutes. You will be asked questions to obtain demographic and other background information. Then, there will be a series of questions about hypothetical job postings. Your participation will help us in identifying appropriate incentives and characteristics to motivate CMWs to work in rural areas. Please read the question carefully and give your most honest responses throughout the questionnaire. There is no right or wrong answer. We will ensure confidentiality of the information you share.. If you agree to participate in this study, please sign at the bottom and if not then please state the reason for refusal. If you have any questions during the survey, please feel free to ask me. Thank you for your participation! Agreed Reused Reason for refusal Survey 2 Background Information 1. Name 2. Contact Number 3. Domicile: a) Punjab b) Sindh c) Balochistan d) KPK e) AJK 4. Age: a) 14-16 b) 17-20 c) 21-25 d) 25-30 e) 31 and above 5. District: a) Quetta b) Gawadar c) Kech 6. Marital Status: a) Single b) Married c) Divorced 7. Current location of work: a) Rural b) Urban 8. Name of area where you work 9. Your average monthly income a) Less than 3000/month b) Between 3000-5000/ month c) Between 5000-7000/ month d) Between 7000-10000/month 10. Monthly household income Survey 2 a) Less than 10,000/month b) Between 10,000-15,000/ month c) Between 15,000-20,000/ month d) Between 20, 000-30,000/ month e) More than 30,000/month 11. No. of dependents 12. Number of children? 13. Work experience as a CMW (in years) 14. Would you prefer working in a rural area over urban area? Yes/N Survey 2 Scenario Imagine that you have just successfully completed your 18 months CMW training. Through newspaper, radio, and other sources, and you find that there are two work packages that health department offers. For both packages, the location is rural Balochistan. However, each of the two postings provides different characteristics or benefits. Please imagine yourself in this situation and make a decision as to which of the two presented work package you would prefer. For the sake of this survey please assume that you would indeed receive the full benefits described for the package. In making your choice, please carefully read the full list of benefits for each work package and do not imagine any additional features of it. There are 12 different scenarios presented. Please note that while they may look similar at a quick glance, they are indeed each very different. Survey 2 Which of these job postings do you prefer? Select one by marking the circle under the job you prefer. Job package 1 Job Package 2 Housing House Allowance Housing with Basic Amenities CMW Practice Setup Seed money for setup Regular continuous supply medicines/ delivery kits Tranportation None Transport Allowance Supportive management Supervision through program(LHV/LHS/WMO) No supervision Stipend Rs 12000/ Month Rs 7000/ Month Good schooling for children No yes Survey 2 Which of these job postings do you prefer? Select one by marking the circle under the job you prefer. Job package 1 Job Package 2 Housing None Housing with Basic Amenities CMW Practice Setup Regular continuous supply medicines/ delivery kits Regular continuous supply medicines/ delivery kits Tranportation Transport Allowance None Supportive management Supervision through program(LHV/LHS/WMO) Refresher courses Stipend Rs 12000/ Month Rs 5000/ Month Good schooling for children No No Survey 2 Which of these job postings do you prefer? Select one by marking the circle under the job you prefer. Job package 1 Job Package 2 Housing None House Allowance CMW Practice Setup Seed money for setup Seed money for setup Tranportation Transport Allowance None Supportive management Supervision through program(LHV/LHS/WMO) Refresher courses Stipend Rs 7000/ Month Rs 12000/ Month Good schooling for children No yes Survey 2 Which of these job postings do you prefer? Select one by marking the circle under the job you prefer. Job package 1 Job Package 2 Housing Housing with Basic Amenities House Allowance CMW Practice Setup Seed money for setup Regular continuous supply medicines/ delivery kits Tranportation None Transport Allowance Supportive management No supervision Supervision through program(LHV/LHS/WMO) Stipend Rs 12000/ Month Rs 5000/ Month Good schooling for children No yes Survey 2 Which of these job postings do you prefer? Select one by marking the circle under the job you prefer. Job package 1 Job Package 2 Housing None Housing with Basic Amenities CMW Practice Setup Seed money for setup Regular continuous supply medicines/ delivery kits Tranportation Transport Allowance None Supportive management No supervision No supervision Stipend Rs 7000/ Month Rs 7000/ Month Good schooling for children No yes Survey 2 Which of these job postings do you prefer? Select one by marking the circle under the job you prefer. Job package 1 Job Package 2 Housing House Allowance Housing with Basic Amenities CMW Practice Setup Regular continuous supply medicines/ delivery kits Seed money for setup Tranportation Transport Allowance None Supportive management Supervision through program(LHV/LHS/WMO) Refresher courses Stipend Rs 5000/ Month Rs 5000/ Month Good schooling for children No yes Survey 2 Which of these job postings do you prefer? Select one by marking the circle under the job you prefer. Job package 1 Job Package 2 Housing None None CMW Practice Setup Regular continuous supply medicines/ delivery kits Seed money for setup Tranportation None Transport Allowance Supportive management No supervision No supervision Stipend Rs 5000/ Month Rs 5000/ Month Good schooling for children No yes Survey 2 Which of these job postings do you prefer? Select one by marking the circle under the job you prefer. Job package 1 Job Package 2 Housing None House Allowance CMW Practice Setup Regular continuous supply medicines/ delivery kits Seed money for setup Tranportation Transport Allowance None Supportive management No supervision Supervision through program(LHV/LHS/WMO) Stipend Rs 7000/ Month Rs 7000/ Month Good schooling for children yes No Survey 2 Which of these job postings do you prefer? Select one by marking the circle under the job you prefer. Job package 1 Job Package 2 Housing Housing with Basic Amenities House Allowance CMW Practice Setup Seed money for setup Regular continuous supply medicines/ delivery kits Tranportation None Transport Allowance Supportive management Refresher courses Refresher courses Stipend Rs 12000/ Month Rs 12000/ Month Good schooling for children yes No Survey 2 Which of these job postings do you prefer? Select one by marking the circle under the job you prefer. Job package 1 Job Package 2 Housing Housing with Basic Amenities None CMW Practice Setup Seed money for setup Seed money for setup Tranportation Transport Allowance None Supportive management Supervision through program(LHV/LHS/WMO) Supervision through program(LHV/LHS/WMO) Stipend Rs 7000/ Month Rs 12000/ Month Good schooling for children yes No Survey 2 Which of these job postings do you prefer? Select one by marking the circle under the job you prefer. Job package 1 Job Package 2 Housing House Allowance None CMW Practice Setup Regular continuous supply medicines/ delivery kits Regular continuous supply medicines/ delivery kits Tranportation None None Supportive management No supervision Refresher courses Stipend Rs 7000/ Month Rs 5000/ Month Good schooling for children yes No Survey 2 Which of these job postings do you prefer? Select one by marking the circle under the job you prefer. Job package 1 Job Package 2 Housing House Allowance Housing with Basic Amenities CMW Practice Setup Seed money for setup Regular continuous supply medicines/ delivery kits Tranportation Transport Allowance Transport Allowance Supportive management Refresher courses Refresher courses Stipend Rs 5000/ Month Rs 7000/ Month Good schooling for children No yes Survey 3 Survey 3 Improving Maternal and Newborn Services: Strengthening Community Midwives in Balochistan Introduction Thank you very much for taking out time for this survey. I welcome you on behalf of the Department of Health, Mercy Corps and Health Services Academy. This study is being undertaken to learn about factors that would motivate Community Midwives for working in rural districts of Balochistan. We are interested in understanding how the health department can improve recruitment of CMWs and retain them in the system. Since you have completed your 18 months training and are looking towards the future, we are interested in knowing more about incentives or characteristics that would influence your decision to work in a rural area as a CMW. This survey should take approximately 20-30 minutes. You will be asked questions to obtain demographic and other background information. Then, there will be a series of questions about hypothetical job postings. Your participation will help us in identifying appropriate incentives and characteristics to motivate CMWs to work in rural areas. Please read the question carefully and give your most honest responses throughout the questionnaire. There is no right or wrong answer. We will ensure confidentiality of the information you share.. If you agree to participate in this study, please sign at the bottom and if not then please state the reason for refusal. If you have any questions during the survey, please feel free to ask me. Thank you for your participation! Agreed Reused Reason for refusal Survey 3 Background Information 1. Name 2. Contact Number 3. Domicile: a) Punjab b) Sindh c) Balochistan d) KPK e) AJK 4. Age: a) 14-16 b) 17-20 c) 21-25 d) 25-30 e) 31 and above 5. District: a) Quetta b) Gawadar c) Kech 6. Marital Status: a) Single b) Married c) Divorced 7. Current location of work: a) Rural b) Urban 8. Name of area where you work 9. Your average monthly income a) Less than 3000/month b) Between 3000-5000/ month c) Between 5000-7000/ month d) Between 7000-10000/month 10. Monthly household income Survey 3 a) Less than 10,000/month b) Between 10,000-15,000/ month c) Between 15,000-20,000/ month d) Between 20, 000-30,000/ month e) More than 30,000/month 11. No. of dependents 12. Number of children? 13. Work experience as a CMW (in years) 14. Would you prefer working in a rural area over urban area? Yes/N Survey 3 Scenario Imagine that you have just successfully completed your 18 months CMW training. Through newspaper, radio, and other sources, and you find that there are two work packages that health department offers. For both packages, the location is rural Balochistan. However, each of the two postings provides different characteristics or benefits. Please imagine yourself in this situation and make a decision as to which of the two presented work package you would prefer. For the sake of this survey please assume that you would indeed receive the full benefits described for the package. In making your choice, please carefully read the full list of benefits for each work package and do not imagine any additional features of it. There are 12 different scenarios presented. Please note that while they may look similar at a quick glance, they are indeed each very different. Survey 3 Which of these job postings do you prefer? Select one by marking the circle under the job you prefer. Job package 1 Job Package 2 Housing None House Allowance CMW Practice Setup Seed money for setup Regular continuous supply medicines/ delivery kits Tranportation None Transport Allowance Supportive management No supervision Supervision through program(LHV/LHS/WMO) Stipend Rs 7000/ Month Rs 7000/ Month Good schooling for children No No Survey 3 Which of these job postings do you prefer? Select one by marking the circle under the job you prefer. Job package 1 Job Package 2 Housing Housing with Basic Amenities None CMW Practice Setup Seed money for setup Regular continuous supply medicines/ delivery kits Tranportation Transport Allowance Transport Allowance Supportive management Refresher courses Refresher courses Stipend Rs 12000/ Month Rs 7000/ Month Good schooling for children No yes Survey 3 Which of these job postings do you prefer? Select one by marking the circle under the job you prefer. Job package 1 Job Package 2 Housing Housing with Basic Amenities House Allowance CMW Practice Setup Seed money for setup Seed money for setup Tranportation Transport Allowance None Supportive management Refresher courses No supervision Stipend Rs 5000/ Month Rs 12000/ Month Good schooling for children yes No Page Break Survey 3 Which of these job postings do you prefer? Select one by marking the circle under the job you prefer. Job package 1 Job Package 2 Housing House Allowance None CMW Practice Setup Regular continuous supply medicines/ delivery kits Regular continuous supply medicines/ delivery kits Tranportation None None Supportive management No supervision Supervision through program(LHV/LHS/WMO) Stipend Rs 7000/ Month Rs 5000/ Month Good schooling for children No yes Survey 3 Which of these job postings do you prefer? Select one by marking the circle under the job you prefer. Job package 1 Job Package 2 Housing Housing with Basic Amenities House Allowance CMW Practice Setup Regular continuous supply medicines/ delivery kits Seed money for setup Tranportation None Transport Allowance Supportive management Supervision through program(LHV/LHS/WMO) No supervision Stipend Rs 12000/ Month Rs 5000/ Month Good schooling for children yes yes Survey 3 Which of these job postings do you prefer? Select one by marking the circle under the job you prefer. Job package 1 Job Package 2 Housing House Allowance Housing with Basic Amenities CMW Practice Setup Seed money for setup Regular continuous supply medicines/ delivery kits Tranportation Transport Allowance None Supportive management Refresher courses Supervision through program(LHV/LHS/WMO) Stipend Rs 7000/ Month Rs 7000/ Month Good schooling for children yes yes Survey 3 Which of these job postings do you prefer? Select one by marking the circle under the job you prefer. Job package 1 Job Package 2 Housing Housing with Basic Amenities None CMW Practice Setup Seed money for setup Regular continuous supply medicines/ delivery kits Tranportation Transport Allowance None Supportive management Supervision through program(LHV/LHS) No supervision Stipend Rs 12000/ Month Rs 5000/ Month Good schooling for children yes No Survey 3 Which of these job postings do you prefer? Select one by marking the circle under the job you prefer. Job package 1 Job Package 2 Housing None None CMW Practice Setup Regular continuous supply medicines/ delivery kits Seed money for setup Tranportation Transport Allowance None Supportive management Refresher courses Refresher courses Stipend Rs 12000/ Month Rs 12000/ Month Good schooling for children No No Survey 3 Which of these job postings do you prefer? Select one by marking the circle under the job you prefer. Job package 1 Job Package 2 Housing House Allowance Housing with Basic Amenities CMW Practice Setup Regular continuous supply medicines/ delivery kits Seed money for setup Tranportation None None Supportive management Supervision through program(LHV/LHS/WMO) No supervision Stipend Rs 5000/ Month Rs 7000/ Month Good schooling for children No yes Survey 3 Which of these job postings do you prefer? Select one by marking the circle under the job you prefer. Job package 1 Job Package 2 Housing House Allowance None CMW Practice Setup Regular continuous supply medicines/ delivery kits Regular continuous supply medicines/ delivery kits Tranportation Transport Allowance Transport Allowance Supportive management Refresher courses Supervision through program(LHV/LHS/WMO) Stipend Rs 7000/ Month Rs 12000/ Month Good schooling for children yes No Survey 3 Which of these job postings do you prefer? Select one by marking the circle under the job you prefer. Job package 1 Job Package 2 Housing Housing with Basic Amenities Housing with Basic Amenities CMW Practice Setup Seed money for setup Seed money for setup Tranportation Transport Allowance None Supportive management No supervision No supervision Stipend Rs 7000/ Month Rs 5000/ Month Good schooling for children No yes Survey 3 Which of these job postings do you prefer? Select one by marking the circle under the job you prefer. Job package 1 Job Package 2 Housing None House Allowance CMW Practice Setup Seed money for setup Seed money for setup Tranportation None Transport Allowance Supportive management Refresher courses Supervision through program(LHV/LHS/WMO) Stipend Rs 5000/ Month Rs 12000/ Month Good schooling for children No No Survey 4 Survey 4 Improving Maternal and Newborn Services: Strengthening Community Midwives in Balochistan Introduction Thank you very much for taking out time for this survey. I welcome you on behalf of the Department of Health, Mercy Corps and Health Services Academy. This study is being undertaken to learn about factors that would motivate Community Midwives for working in rural districts of Balochistan. We are interested in understanding how the health department can improve recruitment of CMWs and retain them in the system. Since you have completed your 18 months training and are looking towards the future, we are interested in knowing more about incentives or characteristics that would influence your decision to work in a rural area as a CMW. This survey should take approximately 20-30 minutes. You will be asked questions to obtain demographic and other background information. Then, there will be a series of questions about hypothetical job postings. Your participation will help us in identifying appropriate incentives and characteristics to motivate CMWs to work in rural areas. Please read the question carefully and give your most honest responses throughout the questionnaire. There is no right or wrong answer. We will ensure confidentiality of the information you share.. If you agree to participate in this study, please sign at the bottom and if not then please state the reason for refusal. If you have any questions during the survey, please feel free to ask me. Thank you for your participation! Agreed Reused Reason for refusal Survey 4 Background Information 1. Name 2. Contact Number 3. Domicile: a) Punjab b) Sindh c) Balochistan d) KPK e) AJK 4. Age: a) 14-16 b) 17-20 c) 21-25 d) 25-30 e) 31 and above 5. District: a) Quetta b) Gawadar c) Kech 6. Marital Status: a) Single b) Married c) Divorced 7. Current location of work: a) Rural b) Urban 8. Name of area where you work 9. Your average monthly income a) Less than 3000/month b) Between 3000-5000/ month c) Between 5000-7000/ month d) Between 7000-10000/month Survey 4 10. Monthly household income a) Less than 10,000/month b) Between 10,000-15,000/ month c) Between 15,000-20,000/ month d) Between 20, 000-30,000/ month e) More than 30,000/month 11. No. of dependents 12. Number of children? 13. Work experience as a CMW (in years) 14. Would you prefer working in a rural area over urban area? Yes/N Survey 4 Scenario Imagine that you have just successfully completed your 18 months CMW training. Through newspaper, radio, and other sources, and you find that there are two work packages that health department offers. For both packages, the location is rural Balochistan. However, each of the two postings provides different characteristics or benefits. Please imagine yourself in this situation and make a decision as to which of the two presented work package you would prefer. For the sake of this survey please assume that you would indeed receive the full benefits described for the package. In making your choice, please carefully read the full list of benefits for each work package and do not imagine any additional features of it. There are 12 different scenarios presented. Please note that while they may look similar at a quick glance, they are indeed each very different. Survey 4 Which of these job postings do you prefer? Select one by marking the circle under the job you prefer. Job package 1 Job Package 2 Housing None House Allowance CMW Practice Setup Regular continuous supply medicines/ delivery kits Seed money for setup Tranportation Transport Allowance None Supportive management No supervision Refresher courses Stipend Rs 12000/ Month Rs 5000/ Month Good schooling for children No yes Survey 4 Which of these job postings do you prefer? Select one by marking the circle under the job you prefer. Job package 1 Job Package 2 Housing Housing with Basic Amenities None CMW Practice Setup Regular continuous supply medicines/ delivery kits Regular continuous supply medicines/ delivery kits Tranportation Transport Allowance None Supportive management No supervision Refresher courses Stipend Rs 5000/ Month Rs 12000/ Month Good schooling for children yes yes Survey 4 Which of these job postings do you prefer? Select one by marking the circle under the job you prefer. Job package 1 Job Package 2 Housing Housing with Basic Amenities Housing with Basic Amenities CMW Practice Setup Seed money for setup Seed money for setup Tranportation None Transport Allowance Supportive management Refresher courses Supervision through program(LHV/LHS/WMO) Stipend Rs 12000/ Month Rs 5000/ Month Good schooling for children No yes Survey 4 Which of these job postings do you prefer? Select one by marking the circle under the job you prefer. Job package 1 Job Package 2 Housing Housing with Basic Amenities House Allowance CMW Practice Setup Regular continuous supply medicines/ delivery kits Seed money for setup Tranportation None Transport Allowance Supportive management No supervision Supervision through program(LHV/LHS/WMO) Stipend Rs 7000/ Month Rs 7000/ Month Good schooling for children No No Survey 4 Which of these job postings do you prefer? Select one by marking the circle under the job you prefer. Job package 1 Job Package 2 Housing None House Allowance CMW Practice Setup Seed money for setup Regular continuous supply medicines/ delivery kits Tranportation Transport Allowance None Supportive management No supervision Refresher courses Stipend Rs 5000/ Month Rs 5000/ Month Good schooling for children yes No Survey 4 Which of these job postings do you prefer? Select one by marking the circle under the job you prefer. Job package 1 Job Package 2 Housing Housing with Basic Amenities None CMW Practice Setup Regular continuous supply medicines/ delivery kits Seed money for setup Tranportation None Transport Allowance Supportive management Supervision through program(LHV/LHS/WMO) Refresher courses Stipend Rs 7000/ Month Rs 12000/ Month Good schooling for children yes No Survey 4 Which of these job postings do you prefer? Select one by marking the circle under the job you prefer. Job package 1 Job Package 2 Housing House Allowance House Allowance CMW Practice Setup Regular continuous supply medicines/ delivery kits Seed money for setup Tranportation Transport Allowance None Supportive management No supervision Refresher courses Stipend Rs 5000/ Month Rs 12000/ Month Good schooling for children No yes Survey 4 Which of these job postings do you prefer? Select one by marking the circle under the job you prefer. Job package 1 Job Package 2 Housing House Allowance None CMW Practice Setup Seed money for setup Regular continuous supply medicines/ delivery kits Tranportation Transport Allowance None Supportive management Supervision through program(LHV/LHS/WMO) Refresher courses Stipend Rs 7000/ Month Rs 5000/ Month Good schooling for children yes No Survey 4 Which of these job postings do you prefer? Select one by marking the circle under the job you prefer. Job package 1 Job Package 2 Housing Housing with Basic Amenities None CMW Practice Setup Regular continuous supply medicines/ delivery kits Seed money for setup Tranportation None None Supportive management Supervision through program(LHV/LHS/WMO) No supervision Stipend Rs 12000/ Month Rs 7000/ Month Good schooling for children yes No Survey 4 Which of these job postings do you prefer? Select one by marking the circle under the job you prefer. Job package 1 Job Package 2 Housing None House Allowance CMW Practice Setup Seed money for setup Regular continuous supply medicines/ delivery kits Tranportation Transport Allowance Transport Allowance Supportive management Supervision through program(LHV/LHS/WMO) Refresher courses Stipend Rs 7000/ Month Rs 5000/ Month Good schooling for children No yes Survey 4 Which of these job postings do you prefer? Select one by marking the circle under the job you prefer. Job package 1 Job Package 2 Housing House Allowance Housing with Basic Amenities CMW Practice Setup Seed money for setup Regular continuous supply medicines/ delivery kits Tranportation None Transport Allowance Supportive management Supervision through program(LHV/LHS/WMO) No supervision Stipend Rs 5000/ Month Rs 12000/ Month Good schooling for children No yes Survey 4 Which of these job postings do you prefer? Select one by marking the circle under the job you prefer. Job package 1 Job Package 2 Housing None Housing with Basic Amenities CMW Practice Setup Seed money for setup Regular continuous supply medicines/ delivery kits Tranportation None Transport Allowance Supportive management Refresher courses No supervision Stipend Rs 12000/ Month Rs 5000/ Month Good schooling for children yes No Survey 5 Survey 5 Improving Maternal and Newborn Services: Strengthening Community Midwives in Balochistan Introduction Thank you very much for taking out time for this survey. I welcome you on behalf of the Department of Health, Mercy Corps and Health Services Academy. This study is being undertaken to learn about factors that would motivate Community Midwives for working in rural districts of Balochistan. We are interested in understanding how the health department can improve recruitment of CMWs and retain them in the system. Since you have completed your 18 months training and are looking towards the future, we are interested in knowing more about incentives or characteristics that would influence your decision to work in a rural area as a CMW. This survey should take approximately 20-30 minutes. You will be asked questions to obtain demographic and other background information. Then, there will be a series of questions about hypothetical job postings. Your participation will help us in identifying appropriate incentives and characteristics to motivate CMWs to work in rural areas. Please read the question carefully and give your most honest responses throughout the questionnaire. There is no right or wrong answer. We will ensure confidentiality of the information you share.. If you agree to participate in this study, please sign at the bottom and if not then please state the reason for refusal. If you have any questions during the survey, please feel free to ask me. Thank you for your participation! Agreed Reused Reason for refusal Survey 5 Background Information 1. Name 2. Contact Number 3. Domicile: a) Punjab b) Sindh c) Balochistan d) KPK e) AJK 4. Age: a) 14-16 b) 17-20 c) 21-25 d) 25-30 e) 31 and above 5. District: a) Quetta b) Gawadar c) Kech 6. Marital Status: a) Single b) Married c) Divorced 7. Current location of work: a) Rural b) Urban 8. Name of area where you work 9. Your average monthly income a) Less than 3000/month b) Between 3000-5000/ month c) Between 5000-7000/ month d) Between 7000-10000/month Survey 5 10. Monthly household income a) Less than 10,000/month b) Between 10,000-15,000/ month c) Between 15,000-20,000/ month d) Between 20, 000-30,000/ month e) More than 30,000/month 11. No. of dependents 12. Number of children? 13. Work experience as a CMW (in years) 14. Would you prefer working in a rural area over urban area? Yes/N Survey 5 Scenario Imagine that you have just successfully completed your 18 months CMW training. Through newspaper, radio, and other sources, and you find that there are two work packages that health department offers. For both packages, the location is rural Balochistan. However, each of the two postings provides different characteristics or benefits. Please imagine yourself in this situation and make a decision as to which of the two presented work package you would prefer. For the sake of this survey please assume that you would indeed receive the full benefits described for the package. In making your choice, please carefully read the full list of benefits for each work package and do not imagine any additional features of it. There are 12 different scenarios presented. Please note that while they may look similar at a quick glance, they are indeed each very different. Survey 5 Which of these job postings do you prefer? Select one by marking the circle under the job you prefer. Job package 1 Job Package 2 Housing None House Allowance CMW Practice Setup Regular continuous supply medicines/ delivery kits Regular continuous supply medicines/ delivery kits Tranportation Transport Allowance None Supportive management No supervision No supervision Stipend Rs 7000/ Month Rs 12000/ Month Good schooling for children yes yes Survey 5 Which of these job postings do you prefer? Select one by marking the circle under the job you prefer. Job package 1 Job Package 2 Housing Housing with Basic Amenities Housing with Basic Amenities CMW Practice Setup Seed money for setup Regular continuous supply medicines/ delivery kits Tranportation Transport Allowance None Supportive management Supervision through program(LHV/LHS/WMO) Supervision through program(LHV/LHS/WMO) Stipend Rs 5000/ Month Rs 7000/ Month Good schooling for children yes No Survey 5 Which of these job postings do you prefer? Select one by marking the circle under the job you prefer. Job package 1 Job Package 2 Housing None Housing with Basic Amenities CMW Practice Setup Seed money for setup Regular continuous supply medicines/ delivery kits Tranportation Transport Allowance None Supportive management Refresher courses Refresher courses Stipend Rs 12000/ Month Rs 7000/ Month Good schooling for children yes No Survey 5 Which of these job postings do you prefer? Select one by marking the circle under the job you prefer. Job package 1 Job Package 2 Housing House Allowance None CMW Practice Setup Regular continuous supply medicines/ delivery kits Seed money for setup Tranportation Transport Allowance None Supportive management Refresher courses Supervision through program(LHV/LHS/WMO) Stipend Rs 5000/ Month Rs 7000/ Month Good schooling for children No yes Survey 5 Which of these job postings do you prefer? Select one by marking the circle under the job you prefer. Job package 1 Job Package 2 Housing House Allowance House Allowance CMW Practice Setup Regular continuous supply medicines/ delivery kits Seed money for setup Tranportation None Transport Allowance Supportive management No supervision Supervision through program(LHV/LHS/WMO) Stipend Rs 12000/ Month Rs 5000/ Month Good schooling for children No yes Survey 5 Which of these job postings do you prefer? Select one by marking the circle under the job you prefer. Job package 1 Job Package 2 Housing Housing with Basic Amenities None CMW Practice Setup Seed money for setup Regular continuous supply medicines/ delivery kits Tranportation None None Supportive management Supervision through program(LHV/LHS/WMO) Refresher courses Stipend Rs 5000/ Month Rs 5000/ Month Good schooling for children No yes Survey 5 Which of these job postings do you prefer? Select one by marking the circle under the job you prefer. Job package 1 Job Package 2 Housing None None CMW Practice Setup Regular continuous supply medicines/ delivery kits Seed money for setup Tranportation Transport Allowance None Supportive management No supervision No supervision Stipend Rs 7000/ Month Rs 12000/ Month Good schooling for children No yes Survey 5 Which of these job postings do you prefer? Select one by marking the circle under the job you prefer. Job package 1 Job Package 2 Housing Housing with Basic Amenities House Allowance CMW Practice Setup Regular continuous supply medicines/ delivery kits Seed money for setup Tranportation Transport Allowance Transport Allowance Supportive management Supervision through program(LHV/LHS/WMO) No supervision Stipend Rs 12000/ Month Rs 12000/ Month Good schooling for children No yes Survey 5 Which of these job postings do you prefer? Select one by marking the circle under the job you prefer. Job package 1 Job Package 2 Housing House Allowance Housing with Basic Amenities CMW Practice Setup Seed money for setup Regular continuous supply medicines/ delivery kits Tranportation None Transport Allowance Supportive management No supervision Refresher courses Stipend Rs 5000/ Month Rs 7000/ Month Good schooling for children No yes Survey 5 Which of these job postings do you prefer? Select one by marking the circle under the job you prefer. Job package 1 Job Package 2 Housing House Allowance None CMW Practice Setup Seed money for setup Seed money for setup Tranportation Transport Allowance Transport Allowance Supportive management Refresher courses Supervision through program(LHV/LHS/WMO) Stipend Rs 7000/ Month Rs 5000/ Month Good schooling for children No yes Survey 5 Which of these job postings do you prefer? Select one by marking the circle under the job you prefer. Job package 1 Job Package 2 Housing Housing with Basic Amenities House Allowance CMW Practice Setup Regular continuous supply medicines/ delivery kits Regular continuous supply medicines/ delivery kits Tranportation None None Supportive management No supervision Refresher courses Stipend Rs 12000/ Month Rs 12000/ Month Good schooling for children yes yes Survey 5 Which of these job postings do you prefer? Select one by marking the circle under the job you prefer. Job package 1 Job Package 2 Housing Housing with Basic Amenities None CMW Practice Setup Seed money for setup Regular continuous supply medicines/ delivery kits Tranportation Transport Allowance None Supportive management Refresher courses Refresher courses Stipend Rs 5000/ Month Rs 7000/ Month Good schooling for children No yes ANNEX 5 Mix Logit Regression Analysis Intervention Non Intervention p values p values Stipend 0.000* 0.000* Housing Allowance 0.001* 0.058* Housing Amenities 0.000* 0.025* Setup 0.078 0.747 Transportation 0.000* 0.000* Supervision by Government 0.025 0.106 Supervision Refresher Course 0.002* 0.000* Schooling 0.711 0.853 --------------------------------------------------- name: log: C:\Users\hp\Desktop\New folder\combine > d.log log type: text opened on: 13 Jun 2016, 12:57:27 . insheet using "C:\Users\hp\Desktop\New folder\com > bined.csv", comma (26 vars, 2640 obs) . recode stipend (1=5000) (2=7000) (3=12000) (stipend: 2640 changes made) . gen housing_no = (housing == 1) . gen housing_alwnc = (housing == 2) . gen housing_amen = (housing == 3) . gen supervision_no = (management == 1) . gen supervision_govt = (management == 2) . gen supervision_ref = (management == 3) . mixlogit choice stipend, id( respond_id) group( pair) rand( housing_alwnc housing_amen setup transportation supervision_govt supervision_ref > schooling) nrep(500) Iteration 0: log likelihood = -826.79575 (not concave) Iteration 1: log likelihood = -824.93162 (not concave) Iteration 2: log likelihood = -824.80708 (not concave) Iteration 3: log likelihood = -824.49748 Iteration 4: log likelihood = -824.38453 Iteration 5: log likelihood = -823.67824 Iteration 6: log likelihood = -823.67377 Iteration 7: log likelihood = -823.67376 Mixed logit model Number of obs = 2626 LR chi2(7) = 7.36 Log likelihood = -823.67376 Prob > chi2 = 0.3928 ------------------------------------------------------------------------------ choice | Coef. Std. Err. z P>|z| [95% Conf. Interval] -------------+---------------------------------------------------------------- Mean | stipend | .0001433 .0000158 9.07 0.000 .0001123 .0001743 housing_al~c | .35678 .096239 3.71 0.000 .1681551 .545405 housing_amen | .5319988 .10859 4.90 0.000 .3191663 .7448314 setup | .0731114 .0808991 0.90 0.366 -.085448 .2316707 transporta~n | .5441802 .0782899 6.95 0.000 .3907348 .6976256 supervisio~t | .3001255 .1122887 2.67 0.008 .0800436 .5202074 supervisio~f | .5482413 .121033 4.53 0.000 .3110209 .7854617 schooling | -.0290965 .0748789 -0.39 0.698 -.1758563 .1176634 -------------+---------------------------------------------------------------- SD | housing_al~c | -.0174106 .1976786 -0.09 0.930 -.4048536 .3700324 housing_amen | .3527573 .1938986 1.82 0.069 -.0272769 .7327914 setup | .3245795 .1599245 2.03 0.042 .0111331 .6380258 transporta~n | .065078 .4015999 0.16 0.871 -.7220434 .8521994 supervisio~t | .3426816 .2266149 1.51 0.130 -.1014754 .7868386 supervisio~f | .4124563 .2025801 2.04 0.042 .0154066 .8095059 schooling | .2879158 .142358 2.02 0.043 .0088993 .5669324 ------------------------------------------------------------------------------ The sign of the estimated standard deviations is irrelevant: interpret them as being positive . log close name: log: C:\Users\hp\Desktop\New folder\combined.log log type: text closed on: 13 Jun 2016, 13:02:55 ----------------------------------------------------------------------------------------------------------------------------------------------- FGD-STUDY 2 1. What are your views regarding the field challenges in establishing yourself as a health provider in the community? a. How can sustainability be achieved? 2. Major initiatives you undertook in this regard a. Lessons learnt (positives & negatives) i. (e.g. small loans, process of getting the loans, role of women’s groups to sustain mumta fund, ambulance service, charge for services, etc ) 3. Are you aware of the Entrepreneurship initiatives? Was the Business Skills Training offered (to some of you) useful? 4. For those who attended the training, a. What did you learn and implement? Please elaborate b. What did you learn but was not feasible? c. What was learnt, feasible but not done? d. What was found useful? Other comments. 5. What are your suggestions to the Health Department for facilitating CMWs in becoming self-sustainable Study Two Expenditure and Investment Assessment of CMWs Consent Form Introduction In collaboration with the Department of Health, a study to assess how can CMWs become financially self-sustaining while serving the needs of the poorest of the poor is being undertaken. We are interested in understanding the details regarding the expenditure and the investments you have made to establish yourself as a healthcare provider in the community. There are no harms associated with your participation in the study. However, your participation will have the policy makers to move towards effective implementation of the CMW program. You will be required to fill out one month’s log about the investment and expenditure made. If you have any queries I would be happy to address them or you could contact the focal persons in the Mercy Corps team in Quetta or Department of Health. (Contact: Dr. Shaihak Riaz – phone:03003401486 ) If you agree to participate in this study, please sign at the bottom and if not then please state the reason for refusal. Agreed Reused Reason for refusal Q # 8 Section I – Socio-Demographic Respondent ID No. Name Age UC/Tehsil Site Intervention/Control Q # Question Code Skip pattern Response Q # 1 What is your current status of education? Primary …………. 1 Secondary ……..... 2 Intermediate …...... 3 Graduate & above..4 Madrasah ……..… 5 Can read & write…6 Illiterate ………….7 Other ……………. 0 Single ………...…. 1 If 1, go Q # 2 What is your marital status? Married ………..…2 to Q # 4 Q # 3 How many alive children do you have? Actual (in numbers) Q # 4 What is your place of job? Q # 5 Do you receive a monthly salary? Actual (in Rs.) Q # 6 What is your total monthly income? Actual (in Rs.) Q # 7 What is overall household income? Actual (in Rs.) How many people are supported on this income? Actual (in numbers) Do you own? (in your home) (a) Television Yes………………1 No ………………2 (b) Refrigerator Yes………………1 No ………………2 (c) Air Conditioned Yes………………1 No ………………2 Q # 9 (d) Motor-Cycle Yes………………1 No ………………2 (e) Washing Machine Yes………………1 No ………………2 (f) Computer Yes………………1 No ………………2 (g) Car Yes………………1 No ………………2 Section II – Tameer Loan and Refresher Training Q # Question Response 1 Did you receive a Business Skills Development training Yes/No 2 When (date, Year) 3 Who conducted the training 4 Where was this training arranged 5 Was this training helpful? 6 Have you taken any loan? Yes/No 7 If yes, specify from where and how much and on what terms and conditions. 8 Why have you not taken loan from ‘Tameer Bank’? Please explain 9 Have you used the training to establish maternity health services Yes/No If Yes, how was the fund used (check all that apply) - Establish work station - Purchase equipment - Purchase medicines - Advertising - Any other (list) If No, what were the challenges faced (check all that apply) - Lack skills to manage business - Too many competitors - Community unwilling to use services - Any other (list) 10 In your opinion, how can your services be improved - Refresher training on business skills - Skills development - Any other (list) 11 Among the services you provide - a. Which are the most profitable - b. Which are the least profitable - Section III – Services Cost Assessment Investment in establishing services Record details of initial expenses incurred under the following budget heads No. Budget Head Expense 1 Establishment of clinic 2 Marketing – printing of business cards, flyers/brochures, banner, billboard 3 Materials – Medicines, equipment purchased 4 Routine clinic maintenance – cleaning, sterilization 5 Personnel hired 6 Others: Specify A. Expenditure - Direct Month Instructions: The purpose of the table below is to maintain a record of information about the expenses that you incur while providing antenatal, natal, postnatal and child health services to your communities. Please include expenditures related only the activities related to provision of services according to the list 1 provided on the next page Date Activity Materials Purchased + Cost Additional costs (Phone calls, Personnel) Time taken to complete activity List 1 – CMW services list B. Expenditure – Indirect Instructions: The purpose of the table below is to maintain a record of information about the additional expenses that you incur while providing services. Please include expenditures related to support of other persons accompanying you, travel costs or any other costs that were incurred which were needed for you to deliver your services Date Activity Travel Chaperone Any other expenses – food, fuel Mode (on foot, taxi, Suzuki, bus, rickshaw, own transport, any other) Cost Accompanied by Occupation Travel costs Item Cost C. Income Instructions: The purpose of the table below is to maintain a record of income from various services or materials sold to the client/s. Date Activity* Did you charge the client Payment received – cash or other Materials sold to client Cost Section IV: Please Tick (√) for “YES” and Cross (X) for “NO”. 1. I am aware of my skills and personal qualities. Yes No 2. I can identify the needs for products and services in my community Yes No 3. I can decide on additional business that suits my skills and talents Yes No 4. I can work out the price for my services / products Yes No 5. I know who my patients / customers will be Yes No 6. I know where to sell my services and products Yes No . 7. I know how much money, I need to start my business Yes No 8. I know how much material, I need to start my business Yes No 9. I know about basic principles of financial management Yes No 10. I understand how important it is to keep personal money and business money separate. Yes No 11. I can give out receipts for money which I earn from my business receives Yes No 12. I can keep a cashbook 13. I can talk to new clients easily 14. I know how to follow the debtors 15. I can organize my time well for my personal life and business Yes Yes Yes Yes No No No No 16. I know about my community’s rules and regulations for business Yes No 17. I know where are the possible referral facilities Yes No 18. I know the re-payment plan for my loan Yes No 19. I know how to calculate profit and loss Yes No 20. I have worked out my business plan Yes No 21. I understand how a small business could use a telephone, fax machine and e-mail Yes No 22. I understand how some businesses use computers Yes No ** Please attach a copy of the monthly report of the same period submitted to the Department of Health CMWs 1. What are your views regarding the field challenges in establishing yourself as a health provider in the community? a. How can sustainability be achieved? 2. Major initiatives you undertook in this regard a. Lessons learnt (positives & negatives) i. (e.g. small loans, process of getting the loans, role of women’s groups to sustain mumta fund, ambulance service, charge for services, etc ) 3. Are you aware of the Entrepreneurship initiatives? Was the Business Skills Training offered (to some of you) useful? 4. For those who attended the training, a. What did you learn and implement? Please elaborate b. What did you learn but was not feasible? c. What was learnt, feasible but not done? d. What was found useful? Other comments. 5. What are your suggestions to the Health Department for facilitating CMWs in becoming self sustainable Study 4 Documents required 1. 6 months reports submitted prior to mHealth introduction (completeness and missing information- hard and soft copy) 2. Manually entered data files and digitalized data files during the study period shared as soft copy (excel sheets or specify the software used) to check for accuracies (in built checks for errors). 3. Manual tallying of the records 5-10% by Mercy Corps staff and results submitted to consultant 4. Time of submission of reports before and after the introduction of mHealth project. Note the above documents relevance became redundant after meeting with project staff 5. Relevant reports of the project indicating uptake of mHealth app by the CMWs and the project meetings and outcomes if documented. Unstructured interviews with the Mercy Corps Project staff will be undertaken. Introduction and Consent Qualitative study Focus Group Discussion The Mercy Corps upon request by the Department of Health is interested in improving the quality of the new reporting and monitoring system in Quetta/ Balochistan. In this context we would like your candid opinion regarding your experience with this service and request you to answer a few questions. Findings from the study will help the relevant authorities to take appropriate steps to improve CMWs performance. The information collected will be kept confidential and you have the right to withdraw from the study at any time. Your responses will have no bearing on your current position with the Department of Health. If you have any queries I would be happy to address them. If you agree to participate in this study, please sign at the bottom and if not then please state the reason for refusal. Agreed Refused Reason for refusal After consent and addressing queries begin the questions as below: 1. Before the mHealth system, how did you report your work progress report to the program/Department of Health /Supervisor? 2. How long have you owned a mobile phone? a. Type of phone (manual or touch screen, simple without applications, smart phone etc) 3. What is the most common feature of the phone that you use a. Voice calls b. Text messages c. Internet surfing/browsing 4. Did you receive any training for proper use of the mHealth system? a. If not, will such a training be useful? b. If yes, was it useful, what are your recommendations to improve the training to facilitate the use of the technology by the CMWs. 5. Do you currently use the mHealth system for reporting? a. Are you confident in using it? b. If no – why not c. Are there any gaps in the current mHealth application? i. Issues faced during reporting ii. Technical failures iii. Others 6. If you were to compare, is the mHealth system better than the manual system? a. Why? Which features made that system better? b. Voice messages c. Alerts d. Feasibility e. Uptake by the clients etc 7. How do you think can the mHealth system be made better? Study 3 KAP Survey of CMWs Consent Form Introduction Department of Health and Mercy Corps is conducting a study to assess the knowledge, attitude and practices of the Community Midwives. This study is being undertaken to learn about Community Midwives gaps in knowledge so that appropriate support may be provided through feedback to policy makers. The information collected will be kept confidential and you have the right to withdraw from the study at any time. Your performance will have no bearing on your current position with the Department of Health. If you have any queries I would be happy to address them. If you agree to participate in this study, please sign at the bottom and if not then please state the reason for refusal. Agreed Refused Reason for refusal Background Information Q # Question Code Skip pattern Response Q # 1 What is your current status of education? Primary …………. 1 Secondary ……..... 2 Intermediate…...... 3 Graduate & above..4 Madrasah ……..… 5 Can read & write…6 Illiterate ………….7 Other ……………. 0 Q # 2 What is your marital status? Single ………...…. 1 If 1, go Married ………..…2 to Q # 4 Q # 3 How many alive children do you have? Actual (in numbers) Q # 4 What is your place of job? Q # 5 Do you receive a monthly salary? Actual (in Rs.) Q # 6 What is your total monthly income? Actual (in Rs.) Q # 7 What is overall household income? Actual (in Rs.) Q # 8 How many people are supported on this income? Actual (in numbers) Q # 9 Do you own? (in your home) (a) Television Yes………………1 No ………………2 (b) Refrigerator Yes………………1 No ………………2 (c) Air Conditioned Yes………………1 No ………………2 (d) Motor-Cycle Yes………………1 No ………………2 (e) Washing Machine Yes………………1 No ………………2 (f) Computer Yes………………1 No ………………2 (g) Car Yes………………1 No ………………2 Q#10 How long have you been working in the area as a CMW In months Q#11 Number of deliveries per month Q#12 Do you think that the training you received in midwifery school was adequate for working in the field? Yes………………1 No……………….2 Q13 How do you think the quality of care of offered by you may be improved? Section 01: Knowledge Assessment of CMW Please (√) single best answer. 1.1 During pregnancy a woman should have at least following number of antenatal checkups a. 05 Antenatals b. 04 Antenatals c. 03 Antenatals d. 02 Antenatals 1.2 Pregnant women should receive educational messages about which of the following? a. Personal Hygiene, rest and exercise during pregnancy b. Diet and nutrition during pregnancy c. Danger signs during pregnancy d. All of the above 1.3 Focused antenatal care includes which of the following actions? a. Checking the baby’s position at 28 weeks. b. Checking the woman’s blood pressure at every visit. c. Assessing ankle edema at 36 weeks. d. Counseling the women about danger signs only at the last visit. 1.4 At each postpartum visit, the mother should be counseled to seek care if she has which of the following danger signs a. Normal lochia, temperature 370 C, or slight breast engorgement b. Edema of hands and face, severe abdominal pain, or sore, cracked nipples c. Severe headache, foul-smelling lochia, or calf tenderness d. B and C e. All of the above 1.5 Active management of the 3rd stage of labor should be practiced a. Only for women who have a history of post partum hemorrhage b. Only for primipara c. Only for multipara d. For all women in labor 1.6 The appropriate order of steps in active management of the third stage of labor include a. Controlled cord traction, fundal massage, and oxytocin b. Intravenous oxytocin, cord clamping and cutting, and fundal massage c. Cord clamping and cutting, controlled cord traction, ergometrine administration, and inspection to be sure the placenta is intact d. Intramuscular injection of oxytocin, controlled cord traction with countertraction to the uterus, and uterine massage. 1.7 If a woman is admitted during the active phase of labor cervical dilation is initially plotted on the partograph a. To the left of the alert line b. To the right of the alert line c. On the alert line d. On the action line 1.8 Cervical dilation plotted to the right of the alert lines indicates a. Satisfactory progress in labor b. Unsatisfactory progress in labor c. The end of the latent phase d. The end of the active phase 1.9 Which of the following will help to decrease the risk of infection during childbirth a. Performing frequent vaginal examination b. Rupturing membranes as soon as possible in the first stage of labor c. Routine catheterization of the bladder before childbirth. d. Reducing prolonged labor. e. All of the above 1.10 Immediate care for a normal newborn includes a. Skin to skin contact followed by placing the baby in a warning incubator b. Drying the baby, removing the wet cloth and covering the baby with a clean, dry cloth c. Stimulating the baby by slapping the soles of the baby’s feet d. Deep suctioning of the airway to remove mucus e. All of the above 1.11 Which of the following can contribute to hypothermia in newborns? a. The baby is not dried thoroughly immediately after birth b. The baby is bathed immediately after birth c. The baby is dried and placed in skin to skin contact with the mother d. A and B e. All of the above 1.12 Care of the umbilicus should include a. Cleaning with Alcohol b. Covering with sterile compress c. Cleaning with cooled, boiled water and leaving uncovered d. Applying antibiotic cream. 1.13 The best way to determine if a new born needs resuscitation is to a. Wait until one minute after birth and assign the Apgar score. b. Listen to the baby’s heart rate c. Observe respirations immediately and begin resuscitation if they are less than 30 per minute d. Perform resuscitation only if central cyanosis is present. e. Apgar score at the time of birth 1.14 Immediate postpartum hemorrhage can be due to a. Uterine atony b. Genital trauma c. Retained placenta d. All of the above 1.15 The most effective way to immediately control eclamptic convulsions is to a. Give diazepam b. Give magnesium sulfate c. Deliver the baby as soon as possible d. Give nifedipine. 1.16 A woman with ruptured uterus has which of the following signs and symptoms a. Rapid maternal pulse b. Persistent abdominal pain and suprapubic tenderness c. Fetal distress d. All of the above 1.17 During the first 2 hours following birth, the provider should a. Measure the woman’s blood pressure and pulse once, and insert a catheter to empty her bladder. b. Measure the woman’s blood pressure and pulse, and check the uterine tone every 15 minutes. c. Not disturb the woman if asleep because her rest is more important than her vital signs d. Measure the woman’s temperature and pulse, massage the uterus, and perform a vaginal examination to remove clots. 1.18 After childbirth, the mother should have a postpartum visit with a skilled provider a. Once, at 3 weeks postpartum b. Once, at 6 weeks postpartum c. Three times: at 6 hours, 6 days, and 6 weeks postpartum and any time she has danger signs d. Only if she has danger signs. 1.19 By the tenth day postpartum, you should be able to palpate the uterus a. Just below the umbilicus b. At the level of the umbilicus c. Just above the symphysis pubis d. Halfway between the symphysis pubis and the umbilicus 1.20 Each postpartum examination should include a. Measurement of blood pressure and temperature, and assessment of conjunctiva, breasts, abdomen, perineum, and legs. b. Observation of breastfeeding c. Information about contraception, safer sex, and counseling and testing for HIV d. All of the above Section 02: Attitudes of CMWs regarding women health Please tick under the appropriate box Score 1 - 4, (1=Strongly Disagree, 1=Disagree, 3=Agree, 4=Strongly agree) Antenatal care 1 2 3 4 1. I feel confident that I register pregnant mothers as per the guidelines 2. I am trained to prepare mothers for examination 3. I am competent to discuss mothers’ problems individually 4. I can perform abdominal examination for: • Assessing fetal growth • Determining the lie of fetus I am confident in: 5. Counting fetal heart sounds 6. Measuring maternal blood pressure 7. Identifying impending eclampsia 8. Weighing mothers 9. Recording weight in maternal record 10. Advising on maternal nutrition 11. Correcting retracted nipple 12. Determining expected date of Delivery 13. I am competent to take appropriate action regarding varicose veins 14. I can diagnosis onset of labor 15. I am confident that I give appointments to come to health facilities as per protocol 16. I am competent in instructing how to take the supplementary nutrition Natal Care : I feel that I am competent to attend to the following: 17. Performing a delivery without assistance 18. Managing a post partum Hemorrhage 19. Action to be taken of retained Placenta 20. Assessment of progress of labor 21. Performing an episiotomy 22. Deciding when to perform an episiotomy in multipara 23. Aseptic severance of umbilical cord 24. Clearing of airway of newborn Postnatal care: I feel that I am competent to attend to the following: 25. Identification of abnormalities of lochia 26. Identification of involuted uterus 27. Cleaning episiotomy Wound 28. Examination of breasts for infection 29. Measuring mothers' Temperature 30. Teaching a family member to care for mother 31. Advising on maternal nutrition General: I feel that I am competent to attend to the following: 32. Delivering a health education talk regarding general care of newborn, breastfeeding, immunization, child spacing 33. Deliver all services in friendly and helpful manner 34. If you were to recommend a healthcare provider to a friend or a close relative to avail antenatal, neonatal or perinatal services etc (i.e services given by a CMW) , you would recommend seeing a CMW 35. CMWs is a preferred choice health care provider because of convenience and home based skilled 36. If your sister were to go into labour I would call a CMW 37. My community is well informed of CMW services 38. My community’s preferred choice of healthcare provider for antenatal, peri-natal and neonatal services is a CMW 39. My community needs to get more information about a CMW 40. In my opinion CMWs are doing a good job in the field 41. In my opinion quality of service provided by a CMW is good Section 03: Checklist Practices of CMWs regarding women health List of activities to assess practices of CMWs by monitoring staff. Instructions: During the supervisory visits, the LHS/LHV (preferably not the regular supervisor) will observe the activities conducted by the CMWs and report in appropriate box in the table below: Score 0 - 4, (0=Not responded, 1=can't perform, 2=not competent, 3=competent, 4=very competent) Antenatal care 0 1 2 3 4 5 42. Registration of pregnant mothers 43. Prepared mothers for examination 44. Discussed mothers’ problems individually 45. Abdominal examination for: • Assessing fetal growth • Determining the lie of fetus 46. Counting fetal heart sounds 47. Measuring maternal blood pressure 48. Identifying impending eclampsia 49. Weighing mothers 50. Recording weight in maternal record 51. Advising on maternal nutrition 52. Correcting retracted nipple 53. Determining expected date of Delivery 54. Appropriate action regarding varicose veins 55. Diagnosis of onset of labor 56. Giving appointments to come to health facilities 57. Instructing how to take the supplementary nutrition Natal Care (it should be likert scale i.e whether they are competent enough to manage a delivery or not ?) 58. Performing a delivery without assistance 59. Managing a post partum Hemorrhage 60. Action to be taken of retained Placenta 61. Assessment of progress of labor 62. Performing an episiotomy 63. Deciding when to perform an episiotomy in multipara 64. Aseptic severance of umbilical cord 65. Clearing of airway of newborn Postnatal care (yes/no) 66. Identification of abnormalities of lochia 67. Identification of involuted uterus 68. Cleaning episiotomy Wound 69. Examination of breasts for infection 70. Measuring mothers' Temperature 71. Teaching a family member to care for mother 72. Advising on maternal nutrition General 73. Delivering a health education talk regarding general care of newborn, breastfeeding, immunization, child spacing 74. Friendly and helpful manner Comments: Qualitative Component Community/Clients FGD Guide Introduction and consent Department of Health and Mercy Corps is conducting a study to examine the quality of services provided by the Community Midwives. This study is being undertaken to learn about the quality of care provided by Community Midwives. In this context we would like your candid opinion regarding quality of the care services provided by CMWs and request you to answer a few questions. Findings from the study will help the relevant authorities to take appropriate steps to improve their performance. The information collected will be kept confidential and you have the right to withdraw from the study at any time. Your performance will have no bearing on your current position with the Department of Health. If you have any queries I would be happy to address them. If you agree to participate in this study, please sign at the bottom and if not then please state the reason for refusal. Agreed Refused Reason for refusal Serial # Name Age Number of children Education Husband’s occupation Decision maker in household After consent and addressing queries begin the questions as below: 1. How long have you lived in this area? 2. Who are the health care providers you prefer to seek care from for maternal and child health and for delivery? 3. Where do you get information on maternal and child health? 4. Is there a CMW in your area? If so then, a. Since when b. Have you attended any health education sessions given by her? What is your opinion about the sessions? i. What was learnt? Were they useful, informative? How so? ii. What knowledge was attained through women groups conducted by CMWs, nature of knowledge received and how it affect/affected the decision making regarding use of services offered by CMWs 5. Have you utilized the CMW’s services? a. Which service (Maternity, Family planning, others) was availed? 6. What is your opinion about the services used? Were you satisfied with the quality of care offered by them? Please explain. a. CMW was well prepared and gave necessary information b. Service quality c. Treatment provided (adequate or not) d. Follow up services e. Identification of complication and referral, etc 7. In future would you use her services again – a. If yes, why b. If no, why not 8. Would you recommend your family members/relatives to the CMWs a. If yes, why b. If no, why not 9. How do you think the services of CMWs can be improved? Once the group has no more suggestions, wrap up discussion. Thank the participants for their participation. Study 3 Qualitative Component Community/Clients FGD Guide Introduction and consent Department of Health and Mercy Corps is conducting a study to examine the quality of services provided by the Community Midwives. This study is being undertaken to learn about the quality of care provided by Community Midwives. In this context we would like your candid opinion regarding quality of the care services provided by CMWs and request you to answer a few questions. Findings from the study will help the relevant authorities to take appropriate steps to improve their performance. The information collected will be kept confidential and you have the right to withdraw from the study at any time. Your performance will have no bearing on your current position with the Department of Health. If you have any queries I would be happy to address them. If you agree to participate in this study, please sign at the bottom and if not then please state the reason for refusal. Agreed Refused Reason for refusal Serial # Name Age Number of children Education Husband’s occupation Decision maker in household After consent and addressing queries begin the questions as below: 1. How long have you lived in this area? 2. Who are the health care providers you prefer to seek care from for maternal and child health and for delivery? 3. Where do you get information on maternal and child health? 4. Is there a CMW in your area? If so then, a. Since when b. Have you attended any health education sessions given by her? What is your opinion about the sessions? i. What was learnt? Were they useful, informative? How so? ii. What knowledge was attained through women groups conducted by CMWs, nature of knowledge received and how it affect/affected the decision making regarding use of services offered by CMWs 5. Have you utilized the CMW’s services? a. Which service (Maternity, Family planning, others) was availed? 6. What is your opinion about the services used? Were you satisfied with the quality of care offered by them? Please explain. a. CMW was well prepared and gave necessary information b. Service quality c. Treatment provided (adequate or not) d. Follow up services e. Identification of complication and referral, etc 7. In future would you use her services again – a. If yes, why b. If no, why not 8. Would you recommend your family members/relatives to the CMWs a. If yes, why b. If no, why not 9. How do you think the services of CMWs can be improved? Once the group has no more suggestions, wrap up discussion. Thank the participants for their participation. Study 3 KAP Survey of CMWs Consent Form Introduction Department of Health and Mercy Corps is conducting a study to assess the knowledge, attitude and practices of the Community Midwives. This study is being undertaken to learn about Community Midwives gaps in knowledge so that appropriate support may be provided through feedback to policy makers. The information collected will be kept confidential and you have the right to withdraw from the study at any time. Your performance will have no bearing on your current position with the Department of Health. If you have any queries I would be happy to address them. If you agree to participate in this study, please sign at the bottom and if not then please state the reason for refusal. Agreed Refused Reason for refusal Background Information Q # Question Code Skip pattern Response Q # 1 What is your current status of education? Primary …………. 1 Secondary ……..... 2 Intermediate…...... 3 Graduate & above..4 Madrasah ……..… 5 Can read & write…6 Illiterate ………….7 Other ……………. 0 Q # 2 What is your marital status? Single ………...…. 1 If 1, go Married ………..…2 to Q # 4 Q # 3 How many alive children do you have? Actual (in numbers) Q # 4 What is your place of job? Q # 5 Do you receive a monthly salary? Actual (in Rs.) Q # 6 What is your total monthly income? Actual (in Rs.) Q # 7 What is overall household income? Actual (in Rs.) Q # 8 How many people are supported on this income? Actual (in numbers) Q # 9 Do you own? (in your home) (a) Television Yes………………1 No ………………2 (b) Refrigerator Yes………………1 No ………………2 (c) Air Conditioned Yes………………1 No ………………2 (d) Motor-Cycle Yes………………1 No ………………2 (e) Washing Machine Yes………………1 No ………………2 (f) Computer Yes………………1 No ………………2 (g) Car Yes………………1 No ………………2 Q#10 How long have you been working in the area as a CMW In months Q#11 Number of deliveries per month Q#12 Do you think that the training you received in midwifery school was adequate for working in the field? Yes………………1 No……………….2 Q13 How do you think the quality of care of offered by you may be improved? Section 01: Knowledge Assessment of CMW Please (√) single best answer. 1.1 During pregnancy a woman should have at least following number of antenatal checkups a. 05 Antenatals b. 04 Antenatals c. 03 Antenatals d. 02 Antenatals 1.2 Pregnant women should receive educational messages about which of the following? a. Personal Hygiene, rest and exercise during pregnancy b. Diet and nutrition during pregnancy c. Danger signs during pregnancy d. All of the above 1.3 Focused antenatal care includes which of the following actions? a. Checking the baby’s position at 28 weeks. b. Checking the woman’s blood pressure at every visit. c. Assessing ankle edema at 36 weeks. d. Counseling the women about danger signs only at the last visit. 1.4 At each postpartum visit, the mother should be counseled to seek care if she has which of the following danger signs a. Normal lochia, temperature 370 C, or slight breast engorgement b. Edema of hands and face, severe abdominal pain, or sore, cracked nipples c. Severe headache, foul-smelling lochia, or calf tenderness d. B and C e. All of the above 1.5 Active management of the 3rd stage of labor should be practiced a. Only for women who have a history of post partum hemorrhage b. Only for primipara c. Only for multipara d. For all women in labor 1.6 The appropriate order of steps in active management of the third stage of labor include a. Controlled cord traction, fundal massage, and oxytocin b. Intravenous oxytocin, cord clamping and cutting, and fundal massage c. Cord clamping and cutting, controlled cord traction, ergometrine administration, and inspection to be sure the placenta is intact d. Intramuscular injection of oxytocin, controlled cord traction with countertraction to the uterus, and uterine massage. 1.7 If a woman is admitted during the active phase of labor cervical dilation is initially plotted on the partograph a. To the left of the alert line b. To the right of the alert line c. On the alert line d. On the action line 1.8 Cervical dilation plotted to the right of the alert lines indicates a. Satisfactory progress in labor b. Unsatisfactory progress in labor c. The end of the latent phase d. The end of the active phase 1.9 Which of the following will help to decrease the risk of infection during childbirth a. Performing frequent vaginal examination b. Rupturing membranes as soon as possible in the first stage of labor c. Routine catheterization of the bladder before childbirth. d. Reducing prolonged labor. e. All of the above 1.10 Immediate care for a normal newborn includes a. Skin to skin contact followed by placing the baby in a warning incubator b. Drying the baby, removing the wet cloth and covering the baby with a clean, dry cloth c. Stimulating the baby by slapping the soles of the baby’s feet d. Deep suctioning of the airway to remove mucus e. All of the above 1.11 Which of the following can contribute to hypothermia in newborns? a. The baby is not dried thoroughly immediately after birth b. The baby is bathed immediately after birth c. The baby is dried and placed in skin to skin contact with the mother d. A and B e. All of the above 1.12 Care of the umbilicus should include a. Cleaning with Alcohol b. Covering with sterile compress c. Cleaning with cooled, boiled water and leaving uncovered d. Applying antibiotic cream. 1.13 The best way to determine if a new born needs resuscitation is to a. Wait until one minute after birth and assign the Apgar score. b. Listen to the baby’s heart rate c. Observe respirations immediately and begin resuscitation if they are less than 30 per minute d. Perform resuscitation only if central cyanosis is present. e. Apgar score at the time of birth 1.14 Immediate postpartum hemorrhage can be due to a. Uterine atony b. Genital trauma c. Retained placenta d. All of the above 1.15 The most effective way to immediately control eclamptic convulsions is to a. Give diazepam b. Give magnesium sulfate c. Deliver the baby as soon as possible d. Give nifedipine. 1.16 A woman with ruptured uterus has which of the following signs and symptoms a. Rapid maternal pulse b. Persistent abdominal pain and suprapubic tenderness c. Fetal distress d. All of the above 1.17 During the first 2 hours following birth, the provider should a. Measure the woman’s blood pressure and pulse once, and insert a catheter to empty her bladder. b. Measure the woman’s blood pressure and pulse, and check the uterine tone every 15 minutes. c. Not disturb the woman if asleep because her rest is more important than her vital signs d. Measure the woman’s temperature and pulse, massage the uterus, and perform a vaginal examination to remove clots. 1.18 After childbirth, the mother should have a postpartum visit with a skilled provider a. Once, at 3 weeks postpartum b. Once, at 6 weeks postpartum c. Three times: at 6 hours, 6 days, and 6 weeks postpartum and any time she has danger signs d. Only if she has danger signs. 1.19 By the tenth day postpartum, you should be able to palpate the uterus a. Just below the umbilicus b. At the level of the umbilicus c. Just above the symphysis pubis d. Halfway between the symphysis pubis and the umbilicus 1.20 Each postpartum examination should include a. Measurement of blood pressure and temperature, and assessment of conjunctiva, breasts, abdomen, perineum, and legs. b. Observation of breastfeeding c. Information about contraception, safer sex, and counseling and testing for HIV d. All of the above Section 02: Attitudes of CMWs regarding women health Please tick under the appropriate box Score 1 - 4, (1=Strongly Disagree, 1=Disagree, 3=Agree, 4=Strongly agree) Antenatal care 1 2 3 4 1. I feel confident that I register pregnant mothers as per the guidelines 2. I am trained to prepare mothers for examination 3. I am competent to discuss mothers’ problems individually 4. I can perform abdominal examination for: • Assessing fetal growth • Determining the lie of fetus I am confident in: 5. Counting fetal heart sounds 6. Measuring maternal blood pressure 7. Identifying impending eclampsia 8. Weighing mothers 9. Recording weight in maternal record 10. Advising on maternal nutrition 11. Correcting retracted nipple 12. Determining expected date of Delivery 13. I am competent to take appropriate action regarding varicose veins 14. I can diagnosis onset of labor 15. I am confident that I give appointments to come to health facilities as per protocol 16. I am competent in instructing how to take the supplementary nutrition Natal Care : I feel that I am competent to attend to the following: 17. Performing a delivery without assistance 18. Managing a post partum Hemorrhage 19. Action to be taken of retained Placenta 20. Assessment of progress of labor 21. Performing an episiotomy 22. Deciding when to perform an episiotomy in multipara 23. Aseptic severance of umbilical cord 24. Clearing of airway of newborn Postnatal care: I feel that I am competent to attend to the following: 25. Identification of abnormalities of lochia 26. Identification of involuted uterus 27. Cleaning episiotomy Wound 28. Examination of breasts for infection 29. Measuring mothers' Temperature 30. Teaching a family member to care for mother 31. Advising on maternal nutrition General: I feel that I am competent to attend to the following: 32. Delivering a health education talk regarding general care of newborn, breastfeeding, immunization, child spacing 33. Deliver all services in friendly and helpful manner 34. If you were to recommend a healthcare provider to a friend or a close relative to avail antenatal, neonatal or perinatal services etc (i.e services given by a CMW) , you would recommend seeing a CMW 35. CMWs is a preferred choice health care provider because of convenience and home based skilled 36. If your sister were to go into labour I would call a CMW 37. My community is well informed of CMW services 38. My community’s preferred choice of healthcare provider for antenatal, peri-natal and neonatal services is a CMW 39. My community needs to get more information about a CMW 40. In my opinion CMWs are doing a good job in the field 41. In my opinion quality of service provided by a CMW is good Section 03: Checklist Practices of CMWs regarding women health List of activities to assess practices of CMWs by monitoring staff. Instructions: During the supervisory visits, the LHS/LHV (preferably not the regular supervisor) will observe the activities conducted by the CMWs and report in appropriate box in the table below: Score 0 - 4, (0=Not responded, 1=can't perform, 2=not competent, 3=competent, 4=very competent) Antenatal care 0 1 2 3 4 5 42. Registration of pregnant mothers 43. Prepared mothers for examination 44. Discussed mothers’ problems individually 45. Abdominal examination for: • Assessing fetal growth • Determining the lie of fetus 46. Counting fetal heart sounds 47. Measuring maternal blood pressure 48. Identifying impending eclampsia 49. Weighing mothers 50. Recording weight in maternal record 51. Advising on maternal nutrition 52. Correcting retracted nipple 53. Determining expected date of Delivery 54. Appropriate action regarding varicose veins 55. Diagnosis of onset of labor 56. Giving appointments to come to health facilities 57. Instructing how to take the supplementary nutrition Natal Care (it should be likert scale i.e whether they are competent enough to manage a delivery or not ?) 58. Performing a delivery without assistance 59. Managing a post partum Hemorrhage 60. Action to be taken of retained Placenta 61. Assessment of progress of labor 62. Performing an episiotomy 63. Deciding when to perform an episiotomy in multipara 64. Aseptic severance of umbilical cord 65. Clearing of airway of newborn Postnatal care (yes/no) 66. Identification of abnormalities of lochia 67. Identification of involuted uterus 68. Cleaning episiotomy Wound 69. Examination of breasts for infection 70. Measuring mothers' Temperature 71. Teaching a family member to care for mother 72. Advising on maternal nutrition General 73. Delivering a health education talk regarding general care of newborn, breastfeeding, immunization, child spacing 74. Friendly and helpful manner Comments: DETAILS OF PARTICIPANTS Study 3 –FGDs with CMW’s Clients / Community District: Gwadar CMW: Nadia (SMNC Intervention area) Community: Sohrabi ward, Gwadar S# Name Age # of children Education Husband’s occupation HH decision maker Area/ street P-1 Rehana 30 6 Eight grade Bank Job Husband Sohrabi ward first street P-2 Saeeda 35 7 Never attended Fishing Husband Sohrabi ward first street P-3 Gul nisa 30 5 Never attended Fishing Husband Sohrabi ward first street P-4 Shahnaz 35 4 Eight grade Fishing Husband Sohrabi ward second street P-5 Rukhsana 30 4 Eight grade Bank Job Husband Sohrabi ward second street P-6 Fahmeeda 35 7 Never attended Fishing Husband Sohrabi ward third street P-7 Waseela 60 4 Never attended Died Self Sohrabi ward third street P-8 Shaheena 35 7 Never attended Fishing Husband Sohrabi ward third street P-9 Fazila 30 3 Never attended Daily wager Husband Sohrabi ward third street P-10 Sabira 35 3 Never attended Fishing Husband Sohrabi ward third street P-11 Hani 23 1 Never attended Fishing Husband Sohrabi ward third street District: Kech CMW: Rahat (SMNC- Intervention area) Community: Machli Market S# Name Age # of children Education Husband’s occupation HH decision maker Area/ street P￾1 Amber 25 5 Never Attended Daily wages Husband Machli Market P￾2 Nagina 40 2 Eight Grade Govt. Job Husband Machli Market P￾3 Ashrat 21 3 Never Attended Daily Wages Husband Machli Market P￾4 Shari 29 4 Never Attended Teacher Husband Malik abad P￾5 Waseela 38 6 Never Attended Private job Husband Malik abad P￾6 Naseema 25 7 Never Attended Daily wages Husband Malik abad P￾7 Nusrat 38 01 Graduate Daily wages Husband Koshk P￾8 Rhima 27 5 Eight Grade Daily wages Husband Koshk P￾9 Shah bibi 40 4 Eight Grade Daily wages/ Embroidery Self Koshk P￾10 Shargul 39 6 Never Attended Daily wages Husband Malik abad, sari bazar P￾11 Parihatoon 21 4 Never Attended Clerk Husband Malik abad, sari bazar District: Kech CMW: Asia (SMNC Intervention area) Community: Kahnay Pusht S# Name Age # of children Education Husband’s occupation HH decision maker Area/ street P-1 Sameena 25 3 Never Attended Daily wages Husband Kahnay Pusht P-2 Taj bibi 39 7 Never Attended Daily wages Husband Kahnay Pusht P-3 Hameeda 27 4 Never Attended Daily wages Husband Kahnay Pusht P-4 Sabira 21 3 Never Attended Daily wages Husband Dabuk Bazar P-5 Shaida 40 8 Never Attended Died Self Dabuk Bazar P-6 Waheeda 31 2 Never Attended Daily wages Husband Dabuk Bazar P-7 Granaz 39 3 Never Attended Shop keer Husband Koshkalat P-8 Jan bibi 35 4 Never Attended Driver Husband Koshkalat P-9 Memona 32 3 Never Attended Daily wages Husband Koshkalat P-10 Adeela 20 1 Never Attended Daily wages Husband Mullahani bazar P-11 Gulbano 25 2 Never Attended Daily wages Husband Mullahani bazar District: CMW: Quetta Sadiqa Barat Khan (SMNC Intervention area) Community: Sabzal Raod Quetta S# Name Age # of children Education Husband’s occupation HH decision maker Area/ street P-1 Fatima 40 3 Never attended Died Self Barech street P-2 Shehzadi 35 6 Never attended Daily wager Husband - P-3 Lal Khatoon 40 6 Never attended Nothing Husband - P-4 Hurmat Khatoon 45 8 Never attended Driver Husband - P-5 Haleema 32 1 FA Driver Husband Sasooli Chowk P-6 Rabia 32 5 Two grade Farmer Husband - P-7 Saleema 35 3 Three grade Farmer Husband - P-8 Sajida 40 7 Never attended Farmer Husband Uzbak Chowk P-9 Naimat Bibi 28 0 BA Teacher Husband - P-10 Jannat Bibi 40 11 One grade Driver Husband Muslimabad P-11 Rubina 38 4 Two grade Driver Husband - District: CMW: Quetta Rubina Sajjad (SMNC Intervention area) Community: Municipal Corporation Brewery road, Quetta S# Name Age # of children Education Husband’s occupation HH decision maker Area/ street P-1 Nasreen 45 5 Never attended Died Self Same street P-2 Salma 50 6 Eight grade Works at private hospital Husband Same street P-3 Farzana 32 5 Eight grade Driver Husband Killi Muhammad Hasni P-4 Samina 40 4 Matric Driver Husband Killi Muhammad Hasni P-5 Asia 25 0 Never attended Driver (own car) Husband Killi Muhammad Hasni P-6 Reha Tharia 30 2 Six grade Works at BMC hospital Husband Raisani Town P-7 Muniza 20 0 Never attended Computer Assistant Husband Raisani Town P-8 Kareema 35 4 Matric Govt. Job Husband Raisani Town P-9 Naila 40 3 Primary Govt. Job Husband Raisani Town P￾10 Sonia 35 3 Never attended Govt. Job Husband TB Sanatorium P￾11 Aeren 35 3 Six grade Tailor master Husband TB Sanatorium P￾12 Moniqa 60 2 Never attended Masonry Husband TB Sanatorium P￾13 Sareeta 35 0 Matric Shopkeeper Husband TB Sanatorium P￾14 Maryam 30 2 Eight grade Works at BMC hospital Husband Railway society P￾15 Mah bibi 35 1 Never attended Nothing Husband Railway society District: CMW: Gwadar Sumaiya (Non-intervention area) Community: Kohbun Ward near Mullah Moosa Chowk, Gwadar S# Name Age # of children Education Husband’s occupation HH decision maker Area/ street P-1 Zakia 23 3 Eight grade Shop keeper Husband Factory side street P-2 Razia 35 1 Never attended Fishing Husband Factory side street P-3 Jameela 30 2 Never attended Nothing Husband Factory side street P-4 Shahida 30 7 Never attended Fishing Husband Factory side street P-5 Nazia 23 3 Never attended Fishing Husband Koh bun ward P-6 Gul Bibi 17 0 Never attended - Father Koh bun ward P-7 Basria 23 0 Eight grade - Father Mullah moosa chowk P-8 Noor Nisa 30 4 Never attended Shop keeper Husband Mullah moosa chowk District: CMW: Kech Rukhshenda (Non-intervention area) Community: Singani sar Turbat S# Name Age # of children Education Husband’s occupation HH decision maker Area/ street P-1 Jameela 40 7 Never attended Driver Uncle Riyaz Mohalla P-2 Bilques 35 4 Intermediate Driver Father in Law Riyaz Mohalla P-3 Fahmida 21 2 Never attended Daily wages Uncle Riyaz Mohalla P-4 Buloor 20 2 Never attended Working in Gulf Brother in Law Abdul Sallam Muhallah P-5 Zainab 21 1 Never attended Clerk Husband Abdul Sallam Muhallah P-6 Rehana 29 3 Never attended Daily wages Father in Law Abdul Sallam Muhallah P-7 Ruqia 25 2 Matric Shop keeper Husband Abdul Sallam Muhallah P-8 Shabana 32 3 Never attended Teacher Uncle Singansi sar P-9 Durgul 29 4 Intermediate Daily wages Grand father Singansi sar P-10 Humaira 38 4 Never attended Daily wages Uncle Singansi sar P-11 Hani 36 6 Intermediate Working in Gulf Husband Singansi sar P-12 Faiza 32 3 Never attended Daily wage Husband Singansi sar District: CMW: Kech Muard bibi (Non-intervention area) Community: Absor S# Name Age # of children Education Husband’s occupation HH decision maker Area/ street P-1 Waseela 35 1 Never attended Suzuki driver Husband Absor P-2 Sabira 38 7 Eitht Grade Suzuki driver Husband Saith aslam Bazar Absor P-3 Zargul 35 5 Eight Grade Driver Husband Saith aslam Bazar Absor P-4 Jangul 35 5 Never attended Daily wages Husband Saith aslam Bazar Absor P-5 Haseena 40 8 Never attended Driver Husband Saith Yaseen Bazar P-6 Waheeda 40 6 Never attended Clerk Husband Saith Yaseen Bazar P-7 Hemna 39 5 Never attended Barber Husband Kauda Yousuf bazar Absor P-8 Zarhatoon 40 4 Never attended Daily wages Husband Kauda Yousuf bazar Absor P-9 Gulbbi 31 3 Never attended Hawaker Husband Kauda Yousuf bazar Absor P-10 Zarbano 32 3 Never attended Daily wages Husband Kauda Yousuf bazar Absor P-11 Mehr a hatoon 38 2 Never attended Daily ages Husband Kauda Yousuf bazar Absor District: CMW: Quetta Sareeta Kumari (Non-intervention area) Community: Kawari road, Quetta S# Name Age # of children Education Husband’s occupation HH decision maker Area/ street P-1 Rubina Shaheen 35 5 Never attended Govt. Job (Masonry) Husband Kawari road P-2 Aglia wanti 45 6 Never attended Shopkeeper Husband Kawari road P-3 Sapna 25 2 Never attended Govt. Job Husband Kawari road P-4 Pauki 20 1 Eight grade Unmarried Father Near Golmandi P-5 Hakim bibi 35 6 Six grade Govt. Job Husband Near Golmandi P-6 Rahila 35 5 Matric Nothing Husband Near Golmandi P-7 Raj bibi 40 9 Never attended Nothing Husband Faqeer Mohammad road P-8 Bakht Taj 35 3 Never attended Own business Husband Faqeer Mohammad road P-9 Bakht Nasir 45 10 Never attended Shopkeeper Husband Faqeer Mohammad road District: CMW: Quetta Jameela (Non-intervention area) Community: Gulshan Hassan Colony Brewery road, Quetta S# Name Age # of children Education Husband’s occupation HH decision maker Area/ street P-1 Gul Plari 35 6 Never attended Shopkeeper Husband Gulshan Hassan Colony P-2 Naz bibi 25 3 Five grade Daily wager Husband Gulshan Hassan Colony P-3 Imam Khatoon 45 9 Never attended Daily wager Husband Gulshan Hassan Colony P-4 Lal bibi 30 2 Never attended Govt. Job Husband Gulshan Hassan Colony P-5 Pari Gul 32 5 Never attended Driver Husband Aminabad P-6 Gul Khatoon 28 4 Never attended Driver Husband Aminabad P-7 Taj Bibi 35 5 Never attended Shopkeeper Husband Aminabad P-8 Haleema 28 4 Never attended Shopkeeper Husband Allahabad street P-9 Zareena 38 7 Never attended Daily wager Husband Allahabad street P￾10 Fareeda 28 2 Five grade Daily wager Husband Allahabad street Study 4 Introduction and Consent Qualitative study Focus Group Discussion The Mercy Corps upon request by the Department of Health is interested in improving the quality of the new reporting and monitoring system in Quetta/ Balochistan. In this context we would like your candid opinion regarding your experience with this service and request you to answer a few questions. Findings from the study will help the relevant authorities to take appropriate steps to improve CMWs performance. The information collected will be kept confidential and you have the right to withdraw from the study at any time. Your responses will have no bearing on your current position with the Department of Health. If you have any queries I would be happy to address them. If you agree to participate in this study, please sign at the bottom and if not then please state the reason for refusal. Agreed Refused Reason for refusal After consent and addressing queries begin the questions as below: 1. Before the mHealth system, how did you report your work progress report to the program/Department of Health /Supervisor? 2. How long have you owned a mobile phone? a. Type of phone (manual or touch screen, simple without applications, smart phone etc) 3. What is the most common feature of the phone that you use a. Voice calls b. Text messages c. Internet surfing/browsing 4. Did you receive any training for proper use of the mHealth system? a. If not, will such a training be useful? b. If yes, was it useful, what are your recommendations to improve the training to facilitate the use of the technology by the CMWs. 5. Do you currently use the mHealth system for reporting? a. Are you confident in using it? b. If no – why not c. Are there any gaps in the current mHealth application? i. Issues faced during reporting ii. Technical failures iii. Others 6. If you were to compare, is the mHealth system better than the manual system? a. Why? Which features made that system better? b. Voice messages c. Alerts d. Feasibility e. Uptake by the clients etc 7. How do you think can the mHealth system be made better ANNEX 12 Pakistan Telecommunication Footp1int Figure 1-PTC Coverage Footprint- Baluchistan Pakistan Telecommunication Coverage Footprint Click on a marker to check GSM coverage for a location Choose Serv ice Cellular Service Kech BChoose Technology GSM BChoose Operator All r· B Show Coverage Home l sate lne Zahedan0 ul-">1, B1kaner 0 Bare0 u PR, RAJASTHAN Jodhpur 0 0 AJmer Gwahor 0 G. llorOman u M .. u .. s . c .. a . t Udaipur 0 Figure 2-PTC Coverage Footprint - Quetta Figure 3-PTC Coverage Footprint - Gawadar Figure 4-PTC Coverage Footprint- Kech Pakistan Telecommunication Coverage Footprint Click on a marker to check GSM coverage for a location Choose Service Ccllulor Service BChoose Technology GSM BChoose Operator All B Show Coverage Home Kcch Map Satellite Gtehk Child Survival and Health Grants Program Project Summary Nov-16-2016 Mercy Corps (Pakistan) General Project Information Cooperative Agreement Number: AID-OAA-A-12-00093 MC Headquarters Technical Backstop: Jennifer Norman MC Headquarters Technical Backstop Backup: Field Program Manager: Ahmed Ullah Midterm Evaluator: Final Evaluator: Headquarter Financial Contact: Jamey Pietzold Project Dates: 9/30/2012 - 9/29/2016 (FY2012) Project Type: Scale USAID Mission Contact: Randolph Augustin Project Web Site: Field Program Manager Name: Ahmed Ullah (Project Manager) Address: H.No-10, Dr. Mustafa Khan street, Arbab Karam Khan Road Quetta , Balochistan 87300 Pakistan Phone: +92 81 2470863 Fax: +92 81 2470240 E-mail: alahmed@mercycorps.org Skype Name: ahmedullah34 Alternate Field Contact Name: Arif Noor (Country Director) Address: Pak Palace, Murree Road, Near Rawal Chowk Islamabad 44000 Pakistan Phone: +92 (0)300 838 0169 Fax: +92 51 2878082 E-mail: anoor@mercycorps.org Skype Name: arif.noor65 Grant Funding Information USAID Funding: $1,741,836 PVO Match: $580,684 General Project Description Mercy Corps’ four year (Sep 30 2012-Sep 29 2016) SCALE program in Quetta, Kech, and Gwadar districts seeks to improve maternal and newborn health status, especially for poor and marginalized women of Balochistan (Goal). Saving Mothers and Newborns in Communities’ (SMNC) Strategic Objective to increase use of quality essential maternal and newborn care, through private-sector community midwivesseeks to facilitate uptake of high-impact MNCH interventions, with a focus on maternal and neonatal health outcomes. SMNC directly contributestoward USAID/Pakistan’sstrategic objective of improving MCH in Pakistan and complements USAID’s new MCH Program in Sindh. SMNC is well positioned to directly influence the MNCH sector in Balochistan, as it was designed jointly with the Balochistan Department of Health (DoH) and upon their request. The DoH is keen to test this model to determine whether CMWs can become self-sustaining private providers in Balochistan and increase coverage of high impact MNCH services. The Operations Research was replaced with a revised Learning Agenda in year 3 which was designed to provide the Balochistan DOH with the evidence and information they need to operationalize CMWs to meet their MNH need. The revised learning agenda topics (jointly agreed on by Mercy Corps and DOH) will equip the DOH to better utilize the CMW as a resource to address MNCH outcomes. The DOH is mainly interested in exploring answers to the following four questions: 1. How can the DoH improve its selection process to effectively recruit and deploy CMWs in underserved areas? 2. How can CMWs become financially self-sustaining while serving the needs of the poorest of the poor? 3. Do CMWs offer quality care? If so, how? 4.. How can the MOH streamline CMW reporting using cell phone technology and expand mHealth in the province? This is an ideal time to document and test these innovations, as the DoH is just now preparing its strategic plans and policies within the newly devolved context. Project Location Latitude: 26.16 Longitude: 63.01 Project Location Types: Rural Levels of Intervention: Home Community Province(s): Balochistan District(s): Quetta, Gwadar, and Kech Districts Sub-District(s): -- Operations Research Information There is no Operations Research (OR) component for this Project. Partners Government of Balochistan (Collaborating Partner) $0 Strategies Social and Behavioral Change Strategies: Community Mobilization Group interventions Interpersonal Communication Social Marketing Mass media and small media Health Services Access Strategies: Emergency Transport Planning/Financing Addressing social barriers (i.e. gender, socio-cultural, etc) Community-based health insurance scheme/Community financing mechanisms Implementation in a geographic area that the government has identified as poor and underserved Health Systems Strengthening: Quality Assurance Supportive Supervision Task Shifting Providing feedback on health worker performance Coordinating existing HMIS with community level data Community input on quality improvement Strategiesfor Enabling Environment: Create/Update national guidelines/protocols Advocacy for revisions to national guidelines/protocols Stakeholder engagement and policy dialogue (local/state or national) Advocacy for policy change or resource mobilization Tools/Methodologies: Community-based Monitoring of Vital Events Mobile Devices for Data Collection Capacity Building Local Partners: Business/Private Sector National Ministry of Health (MOH) Dist. Health System Government sanctioned CHWs Private Providers (Other Non-TBA) Interventions & Components Maternal & Newborn Care (90%) - Emergency Obstetric Care - Neonatal Tetanus - Recognition of Danger signs - Newborn Care - Post partum Care - Child Spacing - Integation. with Iron & Folic Acid - Normal Delivery Care - Birth Plans - STI Treat. with Antenat. Visit - Home Based LSS - Control of post-partum bleeding - Emergency Transport - Kangaroo Mother Care (skin to skin care) - Misoprostol - AMTSL - Pre-eclampsia CHW Training Operational Plan Indicators Number of People Trained in Maternal/Newborn Health Gender Year Target Actual Female 2013 200 Female 2013 45 Male 2013 0 Male 2013 0 Female 2015 26570 Female 2015 30110 Male 2015 0 Male 2015 0 Female 2016 31746 Male 2016 24 Number of People Trained in Child Health & Nutrition Gender Year Target Actual Female 2013 200 Female 2013 45 Male 2013 0 Male 2013 0 Female 2015 26570 Female 2015 30110 Male 2015 0 Male 2015 0 Female 2016 31746 Male 2016 24 Number of People Trained in Malaria Treatment or Prevention Gender Year Target Actual Female 2013 0 Female 2013 0 Male 2013 0 Male 2013 0 Female 2015 0 Female 2015 0 Male 2015 0 Male 2015 0 Female 2016 0 Male 2016 0 Locations & Sub-Areas Total Population: 382,515 Target Beneficiaries Pakistan - MC - FY2012 Children 0-59 months 61,202 Women 15-49 years 84,153 Beneficiaries Total 145,355 Rapid Catch Indicators: DIP Submission Sample Type: 30 Cluster Indicator Numerator Denominator Percentage Confidence Interval Percentage of mothers with children age 0-23 months who received at least two Tetanus toxoid vaccinations before the birth of their youngest child 588 1940 30.3% 2.9 Percentage of children age 0-23 months whose births were attended by skilled personnel 1618 1940 83.4% 2.3 Percentage of children age 0-5 months who were exclusively breastfed during the last 24 hours 953 1940 49.1% 3.1 Percentage of mothers of children age 0-23 months who had four or more antenatal visits when they were pregnant with the youngest child 1145 1940 59.0% 3.1 Percentage of mothers of children age 0-23 months who are using a modern contraceptive method 1079 1940 55.6% 3.1 Percentage of children age 0-23 months who received a post-natal visit from an appropriately trained health worker within two days after birth 1154 1940 59.5% 3.1 Rapid Catch Indicators: Mid-term Sample Type: LQAS Indicator Numerator Denominator Percentage Confidence Interval Percentage of mothers with children age 0-23 months who received at least two Tetanus toxoid vaccinations before the birth of their youngest child 137 228 60.1% 6.4 Percentage of children age 0-23 months whose births were attended by skilled personnel 187 228 82.0% 5.0 Percentage of children age 0-5 months who were exclusively breastfed during the last 24 hours 130 228 57.0% 6.4 Percentage of mothers of children age 0-23 months who had four or more antenatal visits when they were pregnant with the youngest child 107 228 46.9% 6.5 Percentage of mothers of children age 0-23 months who are using a modern contraceptive method 89 228 39.0% 6.3 Percentage of children age 0-23 months who received a post-natal visit from an appropriately trained health worker within two days after birth 166 228 72.8% 5.8 Rapid Catch Indicators: Final Evaluation Rapid Catch Indicator Comments The Baseline survey was conducted as part of the original Operations Research by University of Alberta and has been reported under the DIP tab. Data was collected in May/June 2014 and therefore was conducted only with batch 2 CMWs who had been deployed in August 2014 (Batch 1 was deployed in September 2013). Due to security concerns, the majority of the rural project areas were dropped from the survey. A follow on LQAS survey was conducted 9 months after the baseline survey was done; results are reported under the midterm tab. This survey captured batch 1 and batch 2 CMWs (data reported is for all CMWs) A signigicant portion of rural areas were also dropped from this survey due to security challenges. The LQAS survey was re-designed to provide: i) A mid-term assessment of project across key selected indicators; ii) An overall comparison between the results of the LQAS survey and those of the UoA baseline; iii) A comparative analysis of performance of each district and each batch.