FINAL EVALUATION OF THE PARTNERSHIP FOR MATERNAL & NEONATAL HEALTH PLUS (PMNH+) PROJECT IN ELGEYO MARAKWET COUNTY, NORTH RIFT KENYA March 2017 This publication was produced at the request of the United States Agency for International Development. It was prepared independently by: Dorothy Oluoch (MSc Medical Anthropology), Eric Tama (MPH Health Economics) and Peter Nguhiu (MSc Health Economics) DISCLAIMER The author’s views expressed in this publication do not necessarily reflect the views of the United States Agency for International Development or the United States Government. 2 CONTENTS Contents ........................................................................................................................................................2 Acronyms ......................................................................................................................................................3 Executive Summary ......................................................................................................................................4 Project Background.......................................................................................................................................7 Findings.......................................................................................................................................................10 1. Knowledge, practice and coverage of ANC services at the community and household level. ......10 2. Adoption of healthy behaviors and appropriate care seeking........................................................10 2.1 ANC attendance .............................................................................................................................11 2.2 Skilled delivery................................................................................................................................11 2.3 Family Planning ..............................................................................................................................14 2.4 Immunizations.................................................................................................................................15 2.5 Exclusive Breastfeeding .................................................................................................................15 3. Improved evidence, county and national policy and enabling environment for MNH....................16 4. Challenges and opportunities of implementing a task-sharing intervention for maternal and newborn health...............................................................................................................................17 5. SUMMARY OF FINDINGS FROM THE EXIT INTERVIEWS ..........................................................20 Conclusions ..........................................................................................................................................23 Recommendations................................................................................................................................26 References..................................................................................................................................................30 ANNEXES ...................................................................................................................................................31 3 ACRONYMS ANC Antenatal Care BEMoC Basic Emergency Obstetric Care CBOs Community Based Organizations CHA Community Health Assistant (previously Community Health Extension Worker) CHEW Community Health Extension Worker CHV Community Health Volunteer CU Community Unit FP Family Planning KDHS Kenya Demographic Health Survey KPC Knowledge, Practice and Coverage MDG Millennium Development Goals MNC Maternal and Newborn Care MNH Maternal and Newborn Health MNCH Maternal and Neonatal Child Health MoH Ministry of Health NASCOP National AIDS STI Control Program PMNH+ Partnership for Maternal and Neonatal Health Plus RMC Respectful Maternity Care TBA Traditional Birth Attendant WHO World Health Organization 4 PARTNERSHIP FOR MATERNAL & NEONATAL HEALTH PLUS (PMNH+) PROJECT IN ELGEYO MARAKWET COUNTY, NORTH RIFT KENYA - EXECUTIVE SUMMARY This project was funded by the U.S. Agency for International Development through the Child Survival and Health Grants Program. The purpose of this evaluation was to assess the PMNH+ project achievements and challenges over the course of four years in Elgeyo Marakwet County. The evaluation provided an opportunity to engage key stakeholders in providing feedback about the PMNH+ project. It also presented an opportunity to identify promising practices and recommendations for integrated and sustainable community-based programming. The findings of this evaluation will contribute to the larger body of literature and ongoing global health initiatives in the region and in particularly on task sharing through integrated community-based programming. The evaluation sought to answer the following main questions: 1. To what extent did the project accomplish and/or contribute to the following objectives as per the strategic work plan: (1) knowledge, practice and coverage (KPC) of ANC services at the community and household level; (2) increased adoption of healthy behaviors, including appropriate care seeking; and (3) improved evidence, county and national policy and enabling environment for maternal and newborn health (MNH).  Project Impact: Were there differences in the level of MNH KPC between the comparison and the intervention arms?  Project Implementation: How were results achieved? If the project improved coverage of high-impact interventions simultaneously, what types of integration enabled this? 2. What are the challenges and opportunities of implementing a task-sharing intervention for MNH?  What were the contextual factors, such as socioeconomic factors, gender, demographic factors, environmental characteristics, baseline health conditions, health services characteristics that affected implementation and outcomes? Evaluation, Purpose, and Evaluation Questions 5  Was skill capacity strengthened for: community health volunteers (CHVs); county and sub county health management teams; community health committee members; primary health care service providers; and CBO members? How were these capacities strengthened?  Were gender considerations incorporated into the project at the design phase or midway through the project? If so, how? Are there any specific gender-related outcomes? Are there any unintended consequences related to gender?  What are the stakeholders’ perspectives on intervention delivery? What were the elements of scaling-up and types of scaling-up that have occurred or could likely occur?  What are the sustainability prospects of the intervention considering the various forms of identified sustainability frameworks associated with intervention delivery? In Elgeyo Marakwet County, 34% still deliver at home and 75% of the mothers do not go for post-natal care services (Kenya National Bureau of Statistics et al. 2015) The PMNH+ project was implemented in the two subcounties of Marakwet East and West to improve MNH outcomes. The project established 20 Community Units (CUs) -10 Intervention and 10 comparison/nonintervention sites. In the 10 intervention units, task sharing to the CHVs was implemented. CHVs were trained and engaged in the delivery of family planning services in the community. The project would create a continuum of MNC services from health facilities to households to improve MNH outcomes by increasing coverage of essential MNC services. Project Background Design, Methods, and Limitations We used in-depth interviews (IDIs) (n=23) with key stakeholders in order to capture their insights, feelings and perceptions about the project. Informed consent was sought and obtained from respondents before interviews. Respondents were advised that participation was entirely voluntary and that they were free to terminate the interview at any point without consequences These interview questions were semi-structured and open ended, this enabled participants to freely express their views. Interview questions were structured along the key evaluation questions. We also conducted Focus Group Discussions (FGDs) (n=12) with a total of 102 participants (CHVs, men and women of reproductive age) to gather community perceptions about MNC service utilization and views on community and facility level services. Results from the FGDs were used to complement the findings from the IDIs. For the community FGDs, of the 102 who participated and had complete data were 61% females. Median age was 35, 85.7% were married, 42.9% and 37.14% had attained some secondary and primary level of education respectively. The main economic activity was subsistence farming (74%). A total of 90 structured exit interviews were administered to clients as they exited the facilities. These exit interviews were used to capture users’ perceptions of the quality of services. In addition, a desk 6 review of secondary data sources, such as the project’s KPC survey reports, was also conducted. The use of the different data collection methodologies was useful for purposes of triangulation of the findings. One main limitation of this study was translation of the data collection tools and participant responses during data collection and analysis -from English to Kiswahili and vice versa - may have introduced bias to the data collection process and analysis. There was no back-translation carried out. Due to insecurity in Marakwet East we were not able to visit Kaparon and Kabetwa Health centres and hold face to face interviews with the facility staff and instead we conducted phone interviews.A total of 90 structured exit interviews were administered to clients as they exited the facilities. These exit interviews were used to capture users’ perceptions of the quality of services. In addition, a desk review of secondary data sources such as the project’s baseline and KPC survey reports was also conducted in order to contextualize the study. The use of the different data collection methodologies was useful for purposes of triangulation of the findings. Findings and Conclusions 7 Majority of stakeholders reported that the PMNH+ project had resulted in a positive impact. It was reported that the uptake of ANC, FP and immunizations had greatly increased. It was also stated that more mothers were now giving birth in health facilities as opposed to giving birth at home. Stakeholders also gave positive feedback about their engagement in the project, stating that they were involved from the onset of the project and that they felt they were part and parcel of the project. However, respondents indicated that factors such as inadequate supply capacities at health facilities, distance and cultural factors were barriers to increased uptake of services. Overall, the results indicate that the PMNH+ project has been successful in meeting its intended targets. Health seeking behavior for MNC services has greatly improved as a result of increased awareness among households on the benefits of seeking care at health facilities. This is evident from many respondents who acknowledged that compared to the period before the project implementation, there is a marked improvement in the state of MNCH in their communities. A key contributor to this success according to the findings has been stakeholder involvement from the onset of the project, which fostered acceptance and created a support base for the project. However, respondents indicated there is still a need for increased sensitization efforts among men to counter cultural beliefs and myths regarding reproductive health and increase their involvement in improving MNCH in their communities. While the project has increased awareness leading to more women seeking MNCH services at health facilities therefore increasing demand for services, supply side challenges still persist. Health facilities are sparsely located, and in some cases, they lack sufficient capacity to adequately provide services to the increased number of clients, which if not addressed could compromise quality of services. The views expressed in this material do not necessarily reflect the views of USAID or the United States Government. 8 PROJECT BACKGROUND In Kenya, maternal mortality still remains a big health challenge with maternal mortality ratio (for the 7 year period before 2014) estimated at 362 (95% C.I 254 – 471) deaths per 100,000 live births (Kenya National Bureau of Statistics et al. 2015). While 96% of pregnant women attended at least one antenatal care (ANC), only 58% make the recommended four or more ANC visits. Disaggregation of the four ANC visits by place of residence-urban or rural-shows that urban women were more likely than rural women to have had four or more antenatal visits (68 percent and 51 percent, respectively). The proportion of women delivering at a health facility in Kenya is 61% (DHS 2014), up from 43% in 2008-2009 KDHS. In Elgeyo Marakwet County, 34% still deliver at home and 75% of the mothers do not go for post-natal care services (Kenya National Bureau of Statistics et al. 2015). Across the country, there have been concerted efforts to increase access to life saving interventions (such as family planning, ANC and facility delivery) in order to reduce maternal and newborn mortality. Research shows that maximizing coverage with ANC and facility delivery is necessary, but not sufficient to avert perinatal mortality (Souza et al. 2013). Universal coverage with lifesaving interventions needs to be matched with comprehensive emergency care and overall improvements in the quality of maternal care (Tunçalp et al. 2015). Research also has shown that mothers’ negative perceptions about facility delivery are a key barrier to uptake and use of these services (Bohren et al. 2014). These perceptions are contributed to by disrespect and abuse perpetuated by health workers and other facility staff (Abuya et al. 2015). MNH outcomes in Marakwet East and West subcounties were poor due to barriers like low utilization of facility-based MNC services, inadequate staffing levels, lack of provider skills, insufficient health system capacity, and poor access to care, negative perceptions of facility-based care and low community knowledge about MNC services. The PMNH+ project therefore sought to improve MNH outcomes in these two subcounties by creating a continuum of MNC services from households to health care facilities. To this end, the project established 9 model MNC facilities and 20 new CUs. The project, together with local partners would employ interconnected strategies of capacity building and mobilization of communities in order to increase the demand for and utilization of MNC services and at the same time improve quality of care while maximizing the limited human resources. The project would support and empower the partnering County Health Management Team (CHMT) to manage and supervise services. The project planned to upgrade infrastructure and donate essential equipment and supplies. At the selected intervention sites, the project would adopt a task sharing strategy of certain MNC services by CHWs at the facility and community level to increase access to these services. The project was going to 9 achieve its goals by increasing coverage of essential MNC services and this would have the impact of improving quality, availability, acceptability and accessibility of MNC services at facility and community levels, increased adoption of healthy behaviors and a more enabling environment for MNH. Project Population Beneficiaries Total Total Population 82,194 Total Neonates 83 Infants aged 0-11 Months 2713 Children aged <5 years 11919 Women of Reproductive Age (15-49 years) 30,988 Total Beneficiaries 45,703 Expected Pregnancies 3,288 Community Health Workers/Volunteers (CHVs disaggregated by sex 849 (Male: 447 and Female: 398) Health facilities (Hospital to Sub-Health Post) 9 Community-Based structures (Community Health Committees, Community Based Organizations) 20 CUs , 5CBOs Project Sites Marakwet East Marakwet West Intervention Sites Control Sites Intervention Sites Control Sites Chemwoto Endo Lelan Kamanin Chekitu Kaben Kakuchur B. Kondabilet Kapyego Koibirir Kapcherop Cheptongei Segut Mokoro Sengwer Yemit Kaptich Chesongoch Tenden Chebiemit 10 FINDINGS 1. KNOWLEDGE, PRACTICE AND COVERAGE OF ANC SERVICES AT THE COMMUNITY AND HOUSEHOLD LEVEL. Respondents indicated that as a result of the intervention, knowledge, practice and coverage (KPC) of ANC services increased. The reported PMNH+ impact on KPC of ANC services at the community and household level were similar across all the in-depth interviews and focus group discussions in both the intervention and control sites. Mobilization and education efforts of the CHVs at both the households and the community at large were reported to have greatly added to the knowledge levels, especially with regards to the importance and the availability of the MNH services, as well as knowledge about danger signs in pregnancy. “Women have been enlightened about ANC, pregnant women are going for ANC, after delivery they know about exclusive breastfeeding for six months” (30 year old female, FGD intervention site). “They have done a lot, first of all they have been educated the community on health issues, they have taught the women about how to live well and improve their health in pregnancy by attending ANC, issues of skilled delivery, balanced diet, family planning” (Chief, IDI intervention site). IDI respondents (nursing staff and sub-county officials) reported that through the outreach services, coverage especially in marginalized and hard to reach locations increased. They all appreciated that during the outreaches the project offered them transport which facilitated their movement. “HealthRight supported outreaches, and at least most clients were now being reached, they funded the outreaches and the facility staff were able to offer the services at the outreaches” (Nurse in-charge, IDI control site). 2. ADOPTION OF HEALTHY BEHAVIORS AND APPROPRIATE CARE SEEKING According to the study participants, the PMNH+ project led to the adoption of healthy behavior and positive care seeking practices in the intervention as well as in the control sites. Respondents compared health seeking behavior practice prior to the PMNH+ intervention and post implementation and they reported a positive change in the following key areas: 11 2.1 ANTENATAL CARE (ANC) ATTENDANCE FGD participants in both control and intervention sites indicated that uptake and utilization of ANC services increased as a result of the increased awareness of attending ANC services was important in ensuring a successful pregnancy. “And also women have been enlightened about importance of attending clinic, right now women go to the clinic during their pregnancy” (39 year old female, FGD control site). “When a woman observes any of the danger signs during pregnancy, they have been taught about them, like fever, they have understood what they have been taught. They all rush to the hospital to save the child, she can get treatment there” (34 year old female CHV, FGD control site). Reported also in the IDIs with key stakeholders was that ANC attendance by women had increased due to the PMNH+ intervention. “CHVs have done a good job by referring cases for ANC. Some women are now even coming early” (Sub County community focal person). 2.2 SKILLED DELIVERY The majority of the participants in the IDIs and FGDs reported increased uptake of skilled delivery. A practice that they all attributed to the reduced or decline in the number of mothers dying due to childbirth complications. Specific project attributes and strategies that were mentioned to have led to this achievement were: 2.2.1. Sensitization by CHVs Respondents in the community FGDs mentioned that most women were now delivering in health facilities due to the education and the sensitization work done by the CHVs in the community. “as at now, many women prefer to deliver in the hospital than at home, because of having been sensitized, the CHVs visits the homesteads to each women, and since then this issue of home deliveries has greatly reduced” (43 year old male, FGD intervention site). IDI respondents also reported that the sensitization done by the CHVs positively impacted on hospital deliveries. 12 “Our maternity health services in the facility have also improved in number. Because when we came here they could serve only one mother in a month. And after that, after the community strategy, in a month, they could – within only very few months, could see over 11 mothers-15 mothers come to the facility” (Male, CBO official). 2.2.2 Free maternity services Additionally, reported consistently across the FGDs was that the free maternity services also contributed to increased uptake of hospital skilled delivery in the County. Through the free maternity service, delivery was free of charge and funds from this policy were reported to have been allocated to purchase some gifts, which were given to all the women delivering in the health facilities. This was seen as a motivator to the women to go for skilled delivery. “When women were taught, encouraged to go for skilled delivery, they used to say that it costs a lot of money, maybe KES 800, so they were hesitant. But of late, most women have started going to deliver in the hospitals to deliver because it is now free. They are even given khangas [traditional garment], they get a basin and baby shawl” (28 year old female, FGD control site). 2.2.3 Renovation, trainings and supply of equipment Reported mainly in the IDIs with the health facility staff, as well as the sub county and county officials, was the maternity unit renovations, supply of equipment, as well as capacity building that was offered to them by the PMNH+ project. The project renovated maternity units among the intervention facilities, which was said to have made the maternity environment very welcoming and comfortable in terms of cleanliness as well as privacy. Hot water was also reported to have been made available for the mothers. Health workers indicated having received training on Emergency Obstetric Care (EmOC). On the job trainings, as well as supportive supervision for the facility staff, was also mentioned to have taken place regularly. The respondents felt that all these supply side inputs led to an improvement in the quality of health services that they offered. “We have seen a good number of women coming for these skilled services in our health facilities, more so with our partners who came in to assist in terms of equipment that one has improved the quality and the environment where the women deliver. They have also put up showers, and this has really improved the services, the quality in these facilities has improved- we have seen facilities that were not conducting deliveries reporting a good number of deliveries. The neonates 13 are also kept in a warm environment with the heaters, which is a good thing and this one goes a long way in reducing neonatal deaths” (Sub County Official). They offered CHV trainings and also they trained staff on EmOC, offered on job trainings and supportive supervision, they brought equipment. The supported areas were things that had been identified by the facility management committee” (Health staff, intervention site). 2.2.4 Respectful Maternity Care (RMC) The majority of the stakeholders interviewed in the IDIs talked about the respectful maternity care training that the health workers underwent and this was said to have improved the attitudes of the staff. They felt that the PMNH+ project recognized RMC as a key area that required attention and championed change towards this end. The project was reported to have excelled in advocating for RMC in the County. Stakeholders as well as the community reported that following the PMNH+ project implementation, women in the area were now being treated with respect and that the quality of maternity services have generally improved. “During their monthly routine visits, they were reinforcing the aspect of RMC, where they were advocating that when a mother comes they should be offered the best service, and this service begins from the attitude of that staff and part of it is that when the maternity was improved in terms of there were new paintings there were new tiles, then they made sure that there was flowing water” (sub county MoH). “I feel that nowadays, things have improved. I feel the quality is not that bad. Even the nurses these days do not abuse people, they are very nice these days”. (31 year old female, FGD intervention site). 2.2.5 Male Involvement Male involvement was mentioned in the key stakeholder IDIs as a strategy that was employed to increase uptake of MNH services, particularly for facility-based delivery. Key stakeholders interviewed in the IDIs reported that intervention facilities held maternity open days in which men were invited to visit the maternity units. This they said was done in order to motivate the men to encourage their wives to opt for hospital delivery. 14 “Husbands were called or they were invited and then they were taken through what happens in maternity and actually they were taken to the maternity, they were taken… obviously there was no mother at that time, they were taken from each room and the impression was good” (sub county MoH). “We had an open day for the men to come and see the facilities (conducive nice environment) so that they accept for their wives to come and deliver in the hospital, they liked it. At least we have won that now, women are now delivering in the hospital” (IDI health facility staff, intervention site). 2.2.6 Traditional Birth Attendants (TBAs) as birth companions In-depth interviews with key stakeholders drawn from local CBO (Sobon) highlighted that they targeted TBAs and trained them on the importance of skilled delivery. It was reported that after the training and the sensitization of TBAs, there was an observed change in their role. TBAs who previously offered and assisted in home deliveries were not only referring women to hospitals for skilled delivery, but they were now accompanying them to the hospitals for skilled delivery. The TBAs testify that, “after I was taught and understood the danger sign[s], the issue to do with pregnant mothers. So even if a pregnant mother comes to my house, the first thing I tell her, get prepared we are going to the facility. Not here”. I’ve been told and I’m 100% sure. And even calls back, because some have our numbers. “I have a pregnant mother here. And we are coming [to the health facility] with her” (IDI, CBO official). 2.3 FAMILY PLANNING (FP) Uptake of FP services in the control and intervention arms of the study was perceived to have increased. Through education by CHVs, both men and women reported in the FGDs that they are now aware of the importance of FP and spacing child births. This knowledge, they felt, has increased the uptake of family planning methods. “They are now using FP methods because they have seen the benefits of family planning unlike before when they were getting a child every year. Now with family planning even their marriage lives have improved. You find now husbands are not going out to get other women because their wives now have strength to go out and work and earn income, make better meals for their husbands” (36 year old CHV, FGD control site). 15 Training of the CHVs and issuing them with certificates as well as mentorship offered to the CHVs at the intervention facilities with regards to FP services was reported to have greatly contributed to the acceptance of shifting of this role by both the health staff and the community members. It was explained that initially there was some resistance but this changed overtime. These two aspects were linked to high confidence levels in the ability of the CHVs to effectively offer these services. This rigorous training, assessment and preparation of the CHVs to undertake this task as explained by the participants interviewed below. “We did theory and practical for two days and then the best 13 were selected. Then we were told to go for training at the hospital, once or twice a month or longer depending on one’s program and availability and desire to learn. The nurses mentored us in the hospital... So right now we are offering depo [injectable FP] in the community very well” (34 year old CHV, FGD intervention site) 2.4 IMMUNIZATIONS Child immunizations were indicated to have increased in most IDIs and FGDs. The perceived increase was linked to the CHVs efforts of awareness creation, education and outreach. Through community outreach, these services were said to have been brought closer to the households. “Just to add, a child must get immunization, they must get immunized according to their age. For example, there is immunization for six months, for nine months, this is in relation to the health benefits for the child” (31 year old male, FGD control site). “We get immunization in the rural [areas], mostly it is brought to us in the schools. Community health workers come with service providers. They walk together, they come to the homes, they go to schools, they work together, children are fully immunized” (28 year old female, intervention site). “So for children, those who are brought to the clinic, we tell them that it is good for a child to get all the required immunizations. It is not good to default, this is what we teach them, we tell them to bring the children to the hospital for their post-natal clinics” (36 year old CHV, control site). 2.5 EXCLUSIVE BREASTFEEDING Participants in some of the adult female-only community FGDs, as well as in one CHV group, reported that knowledge about the requirement for six months exclusive breastfeeding was high, 16 as this was one of the key information areas touched by the CHVs. However, some challenges with its practice came up during the discussions. Some participants felt that it was difficult to maintain exclusive breastfeeding due to women’s role in the family. Women were said to engage in farming activities to provide for their families, and this kept them away from home for long hours. Secondly, it was mentioned that there was a belief that breast milk alone was inadequate to fully satisfy a child. “Because if you look at the distance as well, the time they start farming during the rainy season. They travel far. They will leave for the farms at eight and get back in the evening at six o’clock. So during this time they are out in the farms they cannot breastfeed because also they cannot go with the child to the farms” (32 year old female, control site). “Exclusive breastfeeding for six months is hard, some say that they child cries due to hunger, milk is not enough, the child must be given other food stuffs” (24 year old female, control site). 3. IMPROVED EVIDENCE, COUNTY AND NATIONAL POLICY AND ENABLING ENVIRONMENT FOR MNH Respondents from the County and sub County indicated in their IDIs that some of the approaches introduced and advocated for the PMNH+ project, such as the RMC work, influenced practice. It was reported that a policy document with regards to RMC had been developed and was awaiting ratification at the County. “RMC idea was brought by HealthRight, it was a brain child of HealthRight, and when they initiated it then the County again took it up well, are you seeing, now we have free maternity funds, so when this money came in like this. And with this idea from HealthRight that you need to serve this mother like this and like this and like this, surely we did it and it is really showing a lot of good results” (IDI Sub-county official). Secondly, IDIs with sub County officials also revealed that increasingly, during annual County public participation forums, a few community units (CUs) have been able to advocate for allocation of funds to support the CUs. “I think there has been a very positive impact because there are even some examples where in dialogue, in community dialogue and even when they have their own meetings they are able to dialogue with the community, they are even able to source for funds even from the County through 17 public participation for some of the interventions in some of the wards…. so it has worked a unit was awarded some money to support the work of the CHVs” (IDI Sub-county official). “Information and progress was shared quarterly in meetings with key stakeholders, results and what we have used in project areas we have used it in areas where HR did not have an intervention” (IDI Sub-county official). 4. CHALLENGES AND OPPORTUNITIES OF IMPLEMENTING A TASK￾SHARING INTERVENTION FOR MNH The project has encountered a number of challenges according to the respondents in the FGDs as well as in the IDIs. The highlighted challenges include: 4.1 Distance A major challenge frequently mentioned in the community FGDs was distance and the difficult terrain of the area that hampered transport and access. Mothers complained that, at times, even though they are aware of the benefits of attending a health facility, sometimes they are unable to afford the cost of transport to get to a health facility. Distance was also mentioned as a challenge for CHVs and CHAs, who pointed out that they had to travel long distances to reach households. “Women and children get problems with regards to the topography of this area, it has hills and valleys. Then the distance from one place to the next, so getting to the health facility is far... is a challenge. So you find that despite the fact the CHVs and CHWs reach the homesteads to educate, it is not easy for women to come to the hospitals as referred. Some feel it is far and not easy to come to the hospitals” (45 year old male, FGD intervention site) . 4.2 Supply Side Challenges Stakeholders, particularly, health staff as well as some sub County officials, highlighted in their interviews that staff shortage and frequent staff turnover was a major challenge. High staff turnover at health facilities was said to have affected consistency in delivery of BEMoC services, despite facility improvement and equipping them with staff training that was conducted by the PMNH+ project. “The challenges we have had in staffing, in most of our facilities we have less staff, most of our dispensaries have only one staff, and… in situations where you find that maybe one staff is not there, and maybe that is the site where an intervention is taking place, you find it can stall, or like when they are going for mentorship or on job training, when someone has come to relieve the other one, so the continuity of what was going on would not be as they planned, what we did 18 is ask the nearest facility to fill that gap for the period that the staff would not around” (Sub county MoH). 4.3 Male Involvement Male involvement was identified in the community FGDs as a key hindrance to the uptake of some of the interventions particularly FP. A main finding from the community FGDs was that due to the low acceptance among men, women were secretly seeking these FP services without their husband’s knowledge. Culture was identified as a major hindrance to engaging men on MNCH issues. Some men pointed out that their culture did not allow them to accompany their wives to clinics and the best they could do was provide financial support or request female relatives or neighbours to accompany their wives to the health facility. The perception among many men is that reproductive health is a woman’s issue and fear of HIV testing also arose. “Sometimes men will not want their wives to attend ANC clinics because they fear that when their wives go to the clinic, they will be asked to bring their husbands for HIV testing to know their status” (28 year old male, control site). “So in our land there’s a culture that the way our forefathers told us, that we should be blessed with children, so we use that excuse. Therefore, this issue of family planning we say is a foreign thing and for us here as Kalenjins, we must be blessed with children and expand our homesteads￾have many children” (40 year old male, intervention site). “We have also held here what we call open maternity day, where we also brought on board all the stakeholders- so remember our culture here does not allow a man to escort a mother to the facility, they say oh okay, a mother must escort another mother. But thing[s] have changed, that open maternity day has made things change” (Sub county PHO). 4.4 Women’s Multiple Roles Some women pointed out in the community FGDs that they are unable to exclusively breastfeed because of their role as bread winners of their families and also because of the belief that breast milk alone cannot satisfy a baby. “Yeah, that is it. So I think that’s another challenge because exclusive breastfeeding in this region, we still have a big problem because it is a bit expensive. It is very expensive though we try, in fact myself I have tried, and it is quite expensive. Because a woman is supposed to stay at home for six months without going out, you must have enough food. So I have tried some of my neighbors 19 have also tried, and because income in this area is low. So exclusive breastfeeding, they are trying, reaching third month it becomes difficult” (35 year old female, control site). “Because if you look at the distance as well, the time they start farming during the rainy season. They travel far. They will leave for the farms at eight and get back in the evening at six o’clock. So during this time they are out in the farms they cannot breastfeed because also they cannot go with the child to the farms” (41 year old female, control site). “Exclusive breastfeeding for six months is hard, some say that they child cries due to hunger, milk is not enough, the child must be given other food stuffs” (29 year old female, control site). 4.5 Health Worker Motivation and Incentives CHVs and CHAs noted that they did not receive adequate resources, especially from the county government, that would enable to carry out their roles more effectively and reach more households. They complained about the distance they had to walk to cover assigned households. They also mentioned the lack of proper pay or motivation. “You know, when we go to households there are many challenges because this is volunteer work, there is no money. You can do it until evening and you have not done anything at home, the whole day and at the end of the month, there is no pay. You only get 50 shillings, now what can 50 shillings do? So it is a big challenge” (35 year old female CHV, Intervention site). “On our side, I would like them to recognize that going to villages is a hard task and they should look at ways of helping us financially so we can be motivated to do this job” (33 year old female CHV Intervention site). This lack of motivation for the CHVs was linked to attrition by enrolled CHVs which eventually affected project outcomes. For example, in one intervention site a facility in-charge linked the lagging behind in the uptake of hospital delivery in his site to poor education and sensitization efforts in the community. “Home delivery is still a big challenge here, most mothers still prefer delivering at home- probably because of lack of knowledge or ignorance. And this is an area that should really be focused on. There is need to strengthen the capacity and motivate the CHVs” (Facility in charge, Intervention site). 20 CHVs, particularly in Marakwet East, mentioned a weakness in the referral mechanism between them and the mission hospital. They reported that their referral notes issued at the community level were not accepted at the mission hospital. “Regarding the connection between the CHVs and the mission hospital, is that when women we refer women to the dispensary here, and by bad luck, she has to go to the mission hospital for delivery. She will not be attended to at the mission hospital. She will be asked to go where she got her clinic card” (24 year old CHV, Control site). 4.6 Insecurity Mentioned in key informant interviews with sub County officials, as well as in FGDs with CHVs, in Marakwet East was that insecurity in the area greatly affected the implementation of the program. Insecurity was reported to have greatly impacted on the planned activities such as monthly CHV meetings. This resulted in discontentment among the CHVs there, and slow uptake of services. “The main challenge that we had was this issue of insecurity” (42 year old male CHV, control site). 5. SUMMARY OF FINDINGS FROM THE EXIT INTERVIEWS Exit interviews were conducted at both control and intervention facilities, targeting women of reproductive age. They revealed that the main services sought were antenatal and postnatal care, and that the average cost of obtaining care was KES 77.00. Intervention sites were on average further (average distance to facility 7.0km), but imposed less costs of care (average cost of service KES 51.30), than control facilities (2.6km and KES 99.60 respectively). There was no difference in perceptions concerning the facility environment and health workers across sites. All interviewed reported that the facility environment was either welcoming or very welcoming, and 99% reported that the staff at facilities were either friendly or very friendly. 99% of respondents reported overall satisfaction with the facility services received and would recommend the hospital to friends and family. 21 Summary Table of Inputs, Activities, and Outputs That Contributed to Key Outcomes Project Objective No. 1: Knowledge, practice and coverage of ANC services at the community and household levels Project Inputs Activities Outputs Outcome  Funding (transport to outreach)  Training curricula for CHVs  M&E tools (monitoring of referrals and monthly records/indicators)  Trained CHAs and CHVs on community￾based strategies  Conducted and facilitated supervision  Provided incentives to support the CHVs monthly meetings.  Supported DHMT partners to conduct MNC outreach clinics  M&E developed for tracking referrals.  Collected and reviewed CB-MNC monthly data from CHWs  Collected PMNH+ specific data according to the ME plan  Conducted a baseline, midterm and final assessments  CHAs and CHVs trained and received refresher training  10 community units supported with non￾financial incentives, job aids and ME tools  CHWs conducted home visits offer visits to their allocated households on a regular basis  Outreach clinics conducted monthly  CHW referral tracking systems established in each facility  Baseline, midterm and final evaluation conducted (including CHW capacity, community perspectives) Increased district capacity for community-based M&E Increased community satisfaction with care Mothers receive timely and appropriate MNC information and services. Sick mothers and newborns receive timely and appropriate care. Greater collection and use of data to monitor and evaluate progress at the community and project level. Project Objective No. 11: Increased adoption of healthy behaviors, including appropriate care seeking Project Inputs Activities Outputs Outcome  Training funding (Trainer fees, materials, venue, transport)  MoH trainers  Training curricula  M&E tools  Project and County staff  Non-financial incentives  Sub-grant funding for CBO partners  Orientation training for CHWs conducted  Sobon Support trained as MNC facilitators  Provide coordination and non-financial incentives support to CHW units  Provided financial support to Sobon to implement social behavior change campaigns  20 community units trained  ID badges, job aids, data collection tools, and training certificates provided to all CHWs  Monthly coordination and CHW data review meetings were held  Social behavior change campaigns conducted  Ongoing data collection and monitoring of CBO Improved knowledge and attitudes about MNC services (and their benefits) among CHWs, CBOs, and community members Increased knowledge about MNH among 22  IEC materials  Developed M&E tools to assess social behavior changes  Collected CB-HIS data monthly  Collect project data according to the ME plan  Conducted baseline, midterm and final assessments at community level impacts, including on community attitudes  Baseline, midterm and final KPC surveys conducted CHWs, CBOs and community members Increased CHW, CBO, and community capacity to improve MNH in Marakwet Greater collection and use of data to monitor and evaluate progress at the community and project level Project Objective No. 1II: Improved evidence, policy and enabling environment for MNH Project Inputs Activities Outputs Outcome  Funding  Project and District staff  National policy guidance  Technical Advisory Group (TAG)  Stakeholder and TAG engagement and coordination meetings (county and national), in all stages of the project.  Research protocol was developed and approved.  Training was offered to facility, CHW, and partner staff in intervention packages  Assessment of the task sharing intervention through ongoing monitoring, process evaluation, and summative evaluation done  Dissemination of OR findings  Project management, TAG and project partners advocate for national level policy change based on OR research findings  Quarterly PMNH+ stakeholders meetings held in Marakwet  Biannual meetings with national MOH partners and TAG  OR protocol received IRB approval in Kenya  Providers/CHWs trained & MNC services delivered  OR analysis completed and results disseminated  Manuscripts submitted for publication in peer￾reviewed journal  Advocacy done at national level for appropriate policy updates based on evidence Increased capacity of staff and partners to implement OR Increased MNC service delivery evidence and knowledge at local, national, and global levels Improved evidence of the effectiveness of task sharing to improve MNH outcomes and to increase coverage of essential MNC services 23 DISCUSSION In this evaluation, we assessed the achievements of the PMNH+ project and the challenges over the course of its implementation in Marakwet East and West sub counties. Based upon the study findings, the PMNH+ project has been successful in meeting its intended targets. Data from this evaluation strongly suggest that health seeking behavior for MNCH services has greatly improved as a result of increased awareness among households on the benefits of seeking care at health facilities. This correlates with the endline report that found a significant increase in knowledge and utilization levels over the course of the project. These reported positive impacts of the PMNH+ project is reflective of the marked improvement of MNH indicators between the baseline and end line surveys in both the intervention and control sites. The PMNH+ project was able to attain a number of achievements. Findings from this final evaluation suggest that knowledge and awareness about MNCH services increased and this positively influenced the health-seeking behavior among the population. More women were reported as attending ANC clinics, giving birth in health facilities and more children were being taken for immunizations. The involvement of CHVs to create awareness about MNCH by going to households was effective and a key factor in the increased uptake of MNCH services. Through education by CHVs, more mothers were reported to be breastfeeding for longer and more families adopted family planning methods because of understanding the benefits of spacing their births. Our findings are similar to that of a recent Cochrane review (Lassi & Bhutta, 2015), they provided evidence that community-based intervention packages reduce morbidity for women, mortality and morbidity for babies, and improves care-related outcomes particularly in low- and middle-income countries. CHVs efforts and work in the community positively impacts on the continuum of care for MNCH at the facility, community and household levels. In their study on the effectiveness of a CHW program in rural Kenya, Adam and colleagues (2014) also made similar observations, they found that the delivery of health messages by CHVs increased knowledge of MNC among women encouraging skilled birth attendance. With regards to sustainability, the PMNH+ intervention invested in key elements of sustainability. Firstly, the project invested in fostering community ownership. The project also developed sound infrastructure by investing in capacity building of both the health workers and CHVs as well as in developing monitoring and evaluation mechanisms. Thirdly, the project worked within the existing structures by embedding itself and working very closely with the sub county health management teams in Marakwet East and West. These key steps undertaken by the project are facilitators to sustainability. Success of any task sharing intervention is dependent on the participation of the key stakeholders (Iwelunmor et al. 24 2017). We found that a key contributor to the success of the task sharing intervention implemented by the PMNH+ project was stakeholder involvement. This fostered acceptance and created a support base for the project. From the onset, the project involved local stakeholders in its implementation and this enhanced a sense of ownership of the project by stakeholders. The key areas of focus of the PMNH+ intervention were in line with the gaps identified at baseline which were also key priority areas for the County as reflected in the county strategic plan. It is these gaps such as high rates of home deliveries, delayed and few ANC attendances, maternal and neonatal deaths that the PMNH+ intervention sought to address. Stakeholders therefore perceived the project to be an important step towards changing the state of MNCH in the community and eventually lead to improved MNH of the community. The use of volunteers selected by the community members also enhanced acceptance of the project because CHVs were locals who were known by the community. It was also important that the CHVs as well as the CHCs were trained and that monthly meetings were conducted with the CHVs to discuss their monthly progress and challenges. All these engagements contributed to the success of the PMNH+ project. Regular key stakeholder engagement enhanced the level of communication of the project goals such that the project was well understood by all stakeholders. The final evaluation as well as review of the project annual reports highlighted that the project attended together with key stakeholders in the sub-County and County held quarterly meetings. Additionally, monthly progress reports of the CHV activities in the community as well as monthly reports of indicators from the facilities were jointly shared and continuously monitored. This provided a good avenue for monitoring the implementation progress and challenges over the course of the project life. Policy formulation ordinarily follows a top down approach with policies developed at national level and then transferred on to the counties for implementation. The PMNH+ project was, however, able to develop a RMC policy document, which was adopted and implemented by the county. This is a key learning point and mechanisms of how this was achieved can be transferable in the context of devolution. The task sharing intervention, particularly the process of introduction of the community based FP component by the CHVs, was well thought out. The project started by conducting a baseline survey that assessed the capacity of the staff, commodity assessment that assessed the utilization of the method mix as well as availability of these commodities. The information derived from this baseline assessment was critical in the design of the program. Measures to address the weaknesses identified were put in place such as staff and CHV training needs and systems for monitoring stocks across the facilities. This inventory system strengthened the availability and supply side factors of FP commodities. The intervention initially 25 faced challenges with its acceptance in the community as well as with service providers. Thus, it was vital that key stakeholders were effectively engaged in the design, training and monitoring of this task shifting intervention. However, the project faced a number of challenges. Distance was identified as a major barrier to access to services for mothers and also for CHVs visiting households. Mothers noted that at times though aware of the benefits of visiting a health facility. They were discouraged by distance and by the cost of transport. Due to the increased awareness about MNCH services, more mothers are now seeking services at health facilities, which has led to health facility capacities being stretched. It was noted that some facilities did not have sufficient infrastructure and staff to handle the increased number of patients. This is a situation that could potentially compromise the quality of health services. Deterioration of the quality of services could further discourage mothers from attending health facilities. Culture was identified by respondents as a major hindrance to the involvement of men on MNCH issues. Culture prohibited men from accompanying their wives to health facilities and they perceived reproductive health as a woman’s issue. There was also a fear of HIV testing among men and a major reason why they did not accompany their wives to the health facilities. This lack of involvement and support from their male partners, could negatively affect the health seeking behavior of women. There is still a need for increased sensitization efforts among men to counter cultural beliefs and myths regarding reproductive health and increase their involvement in improving MNCH in their communities and uptake of FP services. CHV motivation is a key area that should be looked into and effectively addressed. Several studies have observed under performance and increased attrition rates in community based health programs that rely on CHVs, thereby questioning the sustainability of such approaches (Bhattacharyya et al 2001; Nkonki et al 2011; McElroy et al 2001). Findings of this evaluation depict that outcome and success of the community strategy is dependent on the CHVs’ motivation. There were sites where CHVs dropped out. CHVs and CHAs interviewed in the final evaluation noted that they lacked adequate facilitation to be able to effectively carry out their roles of education and creating awareness on MNCH among households. In addition, insecurity has been a major hindrance to community outreach activities and has also restricted movement, thereby interfering with project activities as well as access to services at health facilities. There is need for more research on CHV retention and attrition. More work needs to be conducted to determine the appropriate motivation for CHVs in different settings. Lastly, of importance for future consideration is a look into the supply side factors such as staffing in health facilities. The evaluation found that high staff turnover, as well as staffing levels, was a major challenge. 26 Health facilities are sparsely located, and in some cases, they lack sufficient capacity to adequately provide services to the increased number of clients. This, together with the challenges highlighted above, if not addressed could compromise quality of services as well as outcomes of similar community strategy interventions that are aimed at improving MNCH outcomes. CONCLUSION Data from the three methodologies used in the final evaluation support the fact that the PMNH+ project has been mostly successful in meeting its intended targets. Health seeking behavior for MNCH services has greatly improved as a result of increased awareness among households on the benefits of seeking care at health facilities. This is evident from many respondents who acknowledged that compared to the period before the project implementation, there is a marked improvement in the state of MNCH in their communities. A key contributor to the success of the project has been stakeholder involvement from the onset of the project which fostered acceptance and created a support base for the project. The task sharing intervention was particularly successful in increasing the awareness and uptake of FP services. However, there is still a need for increased sensitization efforts among men to counter cultural beliefs and myths regarding reproductive health and increase their involvement in improving MNCH in their communities. While the project has increased awareness leading to more women seeking MNCH services at health facilities therefore increasing demand for services, supply side challenges still persist. Health facilities are sparsely located and in some cases they lack sufficient capacity to adequately provide services to the increased number of clients which if not addressed could compromise quality of services. Cultural beliefs and practices that reproductive health is purely a woman’s issue have led to low male involvement and support for their female counterparts in seeking MNCH services. RECOMMENDATIONS Based on the results of the final evaluation of the PMNH+ intervention, we make the following four recommendations 1. Set up sustainable mechanisms of supporting and motivation of the CHVs. Motivation of CHVs is key for sustainability and positive project outcomes. Similar to findings on studies that have looked in motivation of CHVs, this evaluation has also found that CHV motivation is a key area that needs to be adequately addressed. Although the PMNH+ project provided CHVs with solar batteries as an income generating activity, they felt this was inadequate. 27 CHVs reported that very few people in the community sought phone charging services from them. They further explained that many people in the community owned solar batteries. Some ways of achieving adequate CHV motivation are through:  Allocation of funds through the county governments: through continued public participation, and sharing of positive achievements as seen in active CUs who have been able to source for funding from the county governments. There is a need for the county government to allocate a budget towards the project to ensure CHVs and CHAs are better facilitated, better trained, continuously supervised and monitored to increase efficiency and effectiveness of the project. The project has proved to have huge benefits and therefore should be taken forward and up scaled.  Continued capacity building: refresher courses for the CHVs to keep them motivated to continue with their services  Setting up income generating activity to support the CHVs’ livelihood. 2. For successful uptake and implementation of task sharing, negotiate and engage with all key stakeholders both locally and at the national level in all stages of the project right from the design, implantation and evaluation. Community-based programming and approaches that involves task sharing between the health professionals and CHVs is a process that needs to be negotiated carefully. Training and supervision is key to the success of such community strategies. For health staff capacity building, it is important to keep in mind high staff turnover. Strengthening of on job training for all and supportive supervision for front line providers is a means of mitigating the effects on high staff turnover for purposes of ensuring sustainability of the intervention. 3. Conduct targeted interventions to demystify myths and misconceptions that hamper the uptake of family planning. There is need for awareness creation among men to demystify reproductive health issues and counter their cultural beliefs that only women should be involved in reproductive health. This will increase their involvement and support which will in turn lead to increased levels of care seeking among the households. 28 4. Address supply side factors that may interfere with project activities and implementation. The County government needs to take a primary role in addressing the supply side challenges and in ensuring health facilities have the necessary capacity to provide quality services to patients. This will safeguard that clients do not lose confidence in the health system and revert to their old ways of not seeking care at health facilities, which will erode the benefits of the project. Table of Recommendations Finding Conclusion Recommendation Action Who Is Responsible Drop out of CHVs. Tokens were provided but CVs struggled with lack of money for transport, walking long distances to cover wide areas, lack of airtime to make follow up phone calls. Some CHVs became inactive due to lack of motivation. Need to set up sustainable mechanisms of supporting and motivation of the CHVs. Set up adequate support systems Project County Task sharing intervention was initially met by some resistance. With stakeholder engagement, the task sharing intervention was eventually accepted For successful uptake and implementation of task sharing, negotiate and engage with all key stakeholders both locally and at the national level in all stages of the project right from Stakeholder engagement Project 29 the design, implantation and evaluation. Uptake of FP was higher among women than men. Cultural belief and misconceptions hindered FP uptake. Cultural beliefs and misconceptions affected FP uptake and acceptance especially among men. Have targeted interventions to demystify myths and misconceptions that hamper the uptake of family planning. Community engagement and advocacy. Project County There was high staff turnover as well as shortage of staff in some areas High staff turnover and staff shortages affected smooth continuity of the intervention. Address supply side factors that may interfere with project activities and implementation. Train more staff to act as Tots in their facilities. Encourage and support on job training and supportive supervision. County 30 REFERENCES 1. Kenya National Bureau of Statistics (KNBS) and ICF Macro (2015). Kenya Demographic and Health survey 2014-15. Calverton, Maryland: KNBS and ICF Macro. 2. Souza, J.P. et al. "Moving Beyond Essential Interventions for Reduction of Maternal Mortality (The WHO Multicountry Survey on Maternal and Newborn Health)". Obstetric Anesthesia Digest 34.3 (2014): 153-154. 3. Tunçalp, Ö et al. "Authors' Reply Re: Quality of Care for Pregnant Women and Newborns-The WHO Vision". BJOG: An International Journal of Obstetrics & Gynaecology 123.1 (2015): 145-145. 4. Bohren, Meghan A et al. "Facilitators and Barriers to Facility-Based Delivery in Low- And Middle￾Income Countries: A Qualitative Evidence Synthesis". Reproductive Health 11.1 (2014) 5. Abuya, Timothy et al. "Exploring the Prevalence of Disrespect and Abuse during Childbirth in Kenya". PLOS ONE 10.4 (2015): e0123606. 6. Lassi ZS, Bhutta ZA. “Community-based intervention packages for reducing maternal and neonatal morbidity and mortality and improving neonatal outcomes”. Cochrane Database of Systematic Reviews 2015, Issue 3. Art. No.: CD007754. DOI: 10.1002/14651858.CD007754.pub3. 7. Adam MB, Dillmann M, Chen M-k, Mbugua S, Ndung’u J, et al. (2014) Improving Maternal and Newborn Health: Effectiveness of a Community Health Worker Program in Rural Kenya. PLoS ONE 9(8): e104027. doi:10.1371/journal.pone.0104027 8. Iwelunmor, Juliet et al. "Exploring Stakeholders’ Perceptions Of A Task-Shifting Strategy For Hypertension Control In Ghana: A Qualitative Study". BMC Public Health 17.1 (2017) 9. Bhattacharyya K, Winch P, LeBan K, et al. Community health worker incentives and disincentives: how they affect motivation, retention and sustainability. Arlington, Virginia: BASICS/USAID, 2001 10. Nkonki L, Cliff J, Sanders D. Lay health worker attrition: important but often ignored. Bull World Health Organ 2011;89:919–23. doi:10.2471/BLT.11.087825 11. McElroy JC, Morrow PC, Rude SN. Turnover and organizational performance: a comparative analysis of the effects of voluntary, involuntary, and reduction-in-force turn￾over. J Appl Psychol 2001;86:1294–9. doi:10.1037/0021-9010.86.6.1294 31 ANNEXES