The RESPOND Project Study Series: Contributions to Global Knowledge Report No. 15 End-of-Project Evaluation of the RESPOND No-Scalpel Vasectomy Initiative in Uttar Pradesh and Jharkhand States, India The RESPOND Project June 2014 The RESPOND Project Study Series: Contributions to Global Knowledge Report No. 15 End-of-Project Evaluation of the RESPOND No-Scalpel Vasectomy Initiative in Uttar Pradesh and Jharkhand States, India The RESPOND Project June 2014 © 2014 EngenderHealth (The RESPOND Project) The RESPOND Project c/o EngenderHealth 440 Ninth Avenue New York, NY 10001 U.S.A. Telephone: 212-561-8000 Fax: 212-561-8067 e-mail: info@respondproject.org www.respondproject.org This publication is made possible by the generous support of the American People through the U.S. Agency for International Development (USAID), under the terms of cooperative agreement GPO-A-000-08-00007-00. The contents are the responsibility of The RESPOND Project/EngenderHealth and do not necessarily reflect the views of USAID or the United States Government. This work is licensed under the Creative Commons Attribution-Noncommercial￾Share Alike 3.0 Unported License. To view a copy of this license, visit http://creativecommons.org/licenses/by-nc-sa/3.0/. Printed in the United States of America. Printed on recycled paper. Suggested citation: The RESPOND Project. 2014. End-of-project evaluation of the RESPOND No-Scalpel Vasectomy Initiative in Uttar Pradesh and Jharkhand states, India. The RESPOND Project Study Series: Contributions to Global Knowledge—Report No. 15. New York: EngenderHealth (The RESPOND Project). The RESPOND Project  Report No. 15 Evaluation of NSV Initiative in Uttar Pradesh and Jharkhand States iii Contents Acronyms and Abbreviations .................................................................................................................. v Acknowledgments ....................................................................................................................................vii Executive Summary ................................................................................................................................ ix Background ................................................................................................................................................ 1 Evaluation Methodology and Implementation ................................................................................... 5 Study Objectives ......................................................................................................................... 5 Study Design ............................................................................................................................... 5 Data Collection ........................................................................................................................... 6 Analysis ........................................................................................................................................ 7 Limitations ................................................................................................................................... 7 Results ......................................................................................................................................................... 9 Characteristics of Facilities and Participants .......................................................................... 9 Supply ......................................................................................................................................... 10 Enabling Environment ........................................................................................................... 15 Demand ..................................................................................................................................... 17 Conclusions ............................................................................................................................................. 23 Lessons Learned ..................................................................................................................................... 25 References ................................................................................................................................................ 27 Appendix 1: RESPOND NSV Initiative Performance Indicators, 2009–2013 ....................... 29 Appendix 2: NSV Procedure Steps and Provider Scores ............................................................. 31 Appendix 3: Supplies and Equipment Required and Recommended for Quality NSV Services ................................................................................................................ 33 Tables Table 1: NSV uptake in intervention and nonintervention districts ......................................... 4 Table 2: Number of facility audits, observations, and interviews conducted ......................... 6 Table 3: Percentage distribution of facilities in the evaluation sample, by type ...................... 9 Table 4: Percentage of NSV clients reporting previous use of FP methods ......................... 10 Table 5: Percentage of facilities with selected essential supplies and equipment on day of visit .................................................................................................................... 11 Table 6: Percentage of facilities with selected IEC materials with providers who performed key tasks ................................................................................................. 13 Table 7: Percentage distribution of clients, by whether the NSV surgeon informed them of different aspects of the NSV procedure ....................................................... 14 Table 8: Percentage distribution of NSV clients, by time required to access and receive NSV services ........................................................................................................ 15 iv Evaluation of NSV Initiative in Uttar Pradesh and Jharkhand States The RESPOND Project  Report No. 15 Table 9: Percentage of state and district officials aware of RESPOND and RESPOND activities ........................................................................................................ 16 Table 10: Percentage of surgeons and in-charges reporting facility-level changes in NSV-related practices and guidelines, and types of changes reported, by state ................................................................................................................................ 17 Table 11: Percentage of stakeholders reporting demand-generation activities led by RESPOND (spontaneous responses) ........................................................................... 18 Table 12: Percentage of clients reporting awareness of each FP method ................................ 18 Table 13: Percentage of clients offering various reasons for choosing NSV, by state .......... 19 Table 14: Percentage of ASHAs/sahiyas providing information on various FP methods and on key elements and benefits of the NSV procedure, and percentage exhibiting essential interpersonal skills, by state ............................. 20 The RESPOND Project  Report No. 15 Evaluation of NSV Initiative in Uttar Pradesh and Jharkhand States v Acronyms and Abbreviations ANM auxiliary nurse-midwife ASHA accredited social health activist BCC behavior change communication CMO chief medical officer FP family planning IEC information, education, and communication IP infection prevention IUD intrauterine device JHUCCP Johns Hopkins University Bloomberg School of Public Health Center for Communication Program NSV no-scalpel vasectomy OT operating theatre SEED supply–enabling environment–demand SRH sexual and reproductive health UP Uttar Pradesh USAID U.S. Agency for International Development The RESPOND Project  Report No. 15 Evaluation of NSV Initiative in Uttar Pradesh and Jharkhand States vii Acknowledgments This evaluation was designed and conducted by GfK Mode. This report was written by Dr. Prem Talwar and Ms. Urmil Dosajh of GfK Mode and Melanie Yahner of The RESPOND Project. We thank Dr. Talwar and Ms. Dosajh of GfK Mode, as well as the team of data collectors, for their work in designing and implementing the evaluation. We are grateful for the contributions of Anna Kaniauskene and Dr. Hari Singh, both formerly of EngenderHealth, and Lynn Van Lith, of the Johns Hopkins Bloomberg School of Public Health Center for Communication Programs, to the design of the evaluation study. We also thank Maureen Clyde, Santiago Plata, Hannah Searing, Caitlin Shannon, and Jane Wickstrom for their review of this report. The authors are grateful to the Institutional Review Board, Centre for Media Studies, New Delhi, and to Western Institutional Review Board, for their approval to conduct the research. This study was made possible by the generous support of the American people through the U.S. Agency for International Development (USAID), under the terms of Leader with Associate Cooperative Agreement No. GPO-A-00-08-00007-00. The contents are the responsibility of the RESPOND Project and do not necessarily reflect the views of USAID or the United States Government. This report was edited by Michael Klitsch and was formatted by Elkin Konuk. The RESPOND Project  Report No. 15 Evaluation of NSV Initiative in Uttar Pradesh and Jharkhand States ix Executive Summary Background In both Uttar Pradesh and Jharkhand states in India, male sterilization comprises just over 1% of all modern family planning use. Female sterilization, on the other hand, represents 44% of modern family planning use in Uttar Pradesh (UP) and 70% in Jharkhand (IIPS & Macro International, 2007). While awareness of vasectomy is high, misinformation about vasectomy is pervasive. Common misconceptions about vasectomy include its causing weakness, both physical and sexual, and requiring long periods of rest following the procedure. RESPOND No-Scalpel Vasectomy Initiative To ensure that men and couples have access to quality no-scalpel vasectomy (NSV) services and accurate information about all methods of family planning, the U.S. Agency for International Development (USAID) India supported a four-year technical assistance project. The RESPOND NSV Initiative in India was managed by EngenderHealth under the global RESPOND Project, in partnership with Johns Hopkins Bloomberg School of Public Health Center for Communication Programs (JHUCCP). The project’s technical assistance followed EngenderHealth’s holistic Supply–Enabling Environment–Demand (SEED) Programming Model to support the governments of Uttar Pradesh and Jharkhand to expand awareness and acceptance of and access to NSV services. Although RESPOND-supported districts comprised just 15.4% of the population of all districts in the state of UP and 37.0% in Jharkhand (Census Organization of India, 2011), by the last year of the project they provided nearly half (47.9%) of all NSV procedures in UP and more than two-thirds (68.3%) in Jharkhand. The Initiative ended in September 2013. Evaluation Methodology and Implementation In September and October 2013, the RESPOND Project conducted an end-of-project evaluation. The primary objective of the evaluation was to assess the extent to which the NSV Initiative built capacity in service provision at the state, district, and facility levels and promoted service delivery among fieldworkers, which are called accredited social health activist (ASHAs) in UP and sahiyas in Jharkhand, to increase availability of and demand for quality NSV services. Study Design The end-of-project evaluation was implemented in six project districts in Uttar Pradesh (UP) and in three project districts in Jharkhand. A stratified sampling scheme was used to select districts providing high, medium, and low numbers of NSVs to ensure representation of high￾and low-performing districts. Sampled districts in UP were Allahabad, Bulandshahr, Ghaziabad, Kanpur Nagar, Kaushambi, and Meerut; in Jharkhand, districts were Bokaro, Dhanbad, and West Singhbhum. Twenty-four facilities were randomly selected, and a total of 23 providers, 22 auxiliary nurse￾midwives, 39 ASHAs or sahiyas, and 25 clients were interviewed. Twenty-two surgeons and 44 x Evaluation of NSV Initiative in Uttar Pradesh and Jharkhand States The RESPOND Project  Report No. 15 ASHAs/sahiyas were also observed by trained medical doctors. Finally, 19 stakeholder interviews were conducted; stakeholders included the state family welfare officers (two), chief medical officers (CMOs) (five), deputy CMOs (eight), and health education officers (four). Results Supply Facilities’ readiness to provide NSV Only 37.5% of all facilities audited (46.7% in UP, 22.2% in Jharkhand) had all necessary supplies and equipment (five types of infection prevention [IP] supplies, four pieces of IP equipment/infrastructure components, six NSV instruments and supplies, and two pain management supplies) in stock on the day of the visit. Just over half (58.3%) of all facilities had all of the five essential IP supplies and four essential IP infrastructure/equipment components. All facilities in UP had gloves, sterile gauze, iodine, soap, running water, an autoclave, and appropriate disposal containers in stock on the day of the visit. However, the only items available at all facilities in Jharkhand were gloves and an autoclave, with many other essential items out of stock on the day of the visit. Only 70.8% of all facilities had each of the six essential medical instruments and supplies on the day of the visit. Most commonly missing supplies were the NSV ringed clamp forceps and the NSV dissecting forceps (available at 83% of facilities in total). Provider counseling skills Data collectors observed that 95.5% of all 22 observed surgeons (92.3% in UP and 100% in Jharkhand) verified that clients consented to NSV before initiating the procedure. Eighty-six percent of all providers discussed each of the four postprocedure care instructions as well as the three warning signs: In Jharkhand, 100% of providers discussed all instructions and warning signs, while in UP, about three-quarters (76.9%) of providers discussed the warning signs. Provider surgical skills Only 50% of providers (46.2% in UP and 55.6% in Jharkhand) performed all essential infection prevention, anesthesia, NSV procedural, and waste disposal steps. Low performance on some steps reflects stock-outs, particularly in Jharkhand. Of all surgeons, 86.4% performed all four anesthesia-related steps, and just over two-thirds of surgeons in both states (68.2%) performed all nine NSV procedural steps. Proper waste disposal was particularly weak in Jharkhand, with just over half (55.6%) following the three waste disposal steps, compared with 84.6% in UP. This primarily reflects the unavailability of essential IP items at facilities in Jharkhand. Client experience Eighty percent of clients reported that they felt that the surgeon fully or adequately explained the NSV procedure, and 88% reported that they felt that the surgeon adequately explained the postoperative precautions. More than half (56%) of clients reported that registration staff were very courteous, and 64% reported that the surgeon was very courteous. The RESPOND Project  Report No. 15 Evaluation of NSV Initiative in Uttar Pradesh and Jharkhand States xi Enabling Environment Availability of NSV services NSV facilities were reported to be more easily accessible in Jharkhand than in UP; in UP, 46% of clients were able to access the facility in 30 minutes or less, compared to 67% in Jharkhand. Once clients had arrived at the facility, registration was completed more quickly in Jharkhand than in UP, with 92% in Jharkhand requiring less than 30 minutes to register, compared with 54% in UP. Waiting time was also much shorter in Jharkhand than in UP, with no clients waiting more than an hour in Jharkhand; in UP, nearly one-quarter waited more than two hours to see the surgeon. Awareness of RESPOND and RESPOND activities In both states, stakeholders identified several initiatives led by RESPOND to create an enabling environment for NSV at the district level and increase awareness of NSV among health staff. These stakeholders were most familiar with demand generation activities strengthened by RESPOND. Most commonly, district health officers (33% in UP and 80% in Jharkhand) knew that RESPOND had helped the district to develop information, education, and communication (IEC activities), and 25% in UP and 60% in Jharkhand knew that RESPOND had trained ASHAs. Changes in NSV-related practices In Jharkhand, slightly more than half (56%) of NSV surgeons and facility in-charges reported that changes had occurred in practices related to NSV at their facility, compared with 64% of surgeons and 45% of in-charges in UP. The most commonly cited change in UP (89%) was that NSV services were now available more regularly. In Jharkhand, the changes most frequently cited were the involvement of other staff for NSV (60%) and the increased number of surgeons trained (60%). Demand Clients’ knowledge of NSV The mean number of methods known by clients was 3.8; the mean number of methods known was slightly lower in UP (3.5) than in Jharkhand (4.2). After NSV, the most commonly known FP methods were female sterilization and the male condom, known by 84% of all clients. Clients’ reasons for choosing NSV The reasons clients gave for choosing NSV included both method characteristics and social factors. The permanent nature of NSV was the most common method-specific reason, provided by 84% of clients. Another common reason related to the method was its simplicity (72%); smaller proportions of clients cited the limited recovery time (28%) and few side effects (12%) associated with the method. Many clients also cited social factors as a reason for their choice of NSV. Just over one-half of clients (52%) stated that they chose NSV because they did not want their wife to undergo female sterilization,1 while just under one-half (44%) reported that they chose NSV because of                                                              1 Formative research conducted by the RESPOND Project found that many community members, men in particular, believed that undergoing female sterilization could make a woman weak and unable to keep up with her household work. xii Evaluation of NSV Initiative in Uttar Pradesh and Jharkhand States The RESPOND Project  Report No. 15 a sense of responsibility to their wife and family. Smaller proportions of men reported being influenced by the NSV publicity (32%), encouraged by their wife (28%), or provided with NSV information by facility staff (24%). ASHA/Sahiya counseling skills ASHAs in UP discussed 3.8 methods on average, compared with 4.2 discussed by sahiyas in Jharkhand. Male and female sterilization were the methods most frequently discussed during counseling sessions, discussed by 93.2% of ASHAs/sahiyas. Fewer than half (43.9%) of ASHAs/sahiyas discussed each of the four benefits of NSV emphasized in ASHA orientation and coaching. While more than 90% discussed how the procedure would not lead to weakness, an important concern for both men and women, just over half explained that men would still have a normal erection and would still ejaculate semen after the procedure.2 Fewer than half (46.3%) of observed ASHAs/sahiyas discussed all eight elements of the surgical procedure. More than three-quarters (81.8%) of observed ASHAs/sahiyas exhibited the four elements of interpersonal communication skills promoted by RESPOND through coaching sessions. Conclusions The RESPOND NSV Initiative was successful in increasing uptake of NSV services in project districts, and the project was well-known by key stakeholders. However, more effort is needed to fully institutionalize the promotion and provision of quality NSV services by UP and Jharkhand states. Lessons Learned  More extensive follow-up and mentoring of surgeons and facility staff is needed. Surgeons require support posttraining to ensure that they properly perform the NSV procedure.  Facilities need support to ensure that stocks of essential supplies and equipment are maintained, so that clients receive the highest possible standard of care.  Further activities to strengthen the enabling environment are needed to ensure sustainability. These activities should consist of building supervisory systems to support NSV surgeons, training auxiliary nurse-midwives and health education officers to train and support ASHAs/sahiyas to discuss the full range of FP methods, strengthening supply chains to reduce stock-outs; and ensuring that funds for reimbursement payments are available to avoid interruption of services.  ASHAs/sahiyas need additional training and follow-up to ensure that their FP knowledge is complete and accurate and to strengthen their counseling skills. Rather than orienting large numbers of ASHAs/sahiyas, future projects should focus on providing intensive support and follow-up to a limited number of fieldworkers.  Future projects should consider training ASHAs’ husbands, since men often have concerns that they may prefer to discuss with other men. This would complement the work that RESPOND had done in coaching satisfied clients of NSV to share their stories with other men.                                                              2 Previous research found that it is important for men and women to understand that semen would still be ejaculated following the NSV procedure. The RESPOND Project  Report No. 15 Evaluation of NSV Initiative in Uttar Pradesh and Jharkhand States 1 Background Female sterilization is the most widely used modern method of contraception. More than 220 million women worldwide rely on it, representing one out of every five women aged 15–49 years who are married or in union, and 34% of all modern method users (Ripley & Salem, 2012). The prevalence of female sterilization among women in Asia, Latin America, and North America ranges from 22% to 26% (Jacobstein, 2013). In India, both male and female sterilization have played important roles in the method mix, with notable changes over time. In the National Family Health Survey (2007) in India, female sterilization prevalence exceeded vasectomy prevalence by a factor of 37 to 1 (IIPS & Macro International, 2007). This was a significant change from earlier decades. From the inception of India’s national family planning (FP) program in the 1950s through the mid-1970s, vasectomy played a dominant role. Out of 32.7 million sterilizations registered by the Indian government’s FP program between 1956 and 1980, 65% were vasectomies (Soni, 1983; Ross & Huber, 1983).By the late 1970s, however, vasectomy use had begun to decline drastically, due to laparoscopic female sterilization’s becoming more widely available and popular and to a public backlash related to the national program’s high-pressure approach to vasectomy (large camps, cash incentives, and reportedly coercive practices) (Trpathy, Ramachandran, & Ramachandran, 1994; EngenderHealth, 2002).In 1970, vasectomy accounted for nearly three out of four sterilization procedures performed annually; by 1980, vasectomy represented only one in four sterilization procedures performed, even though vasectomy is safer, simpler, less expensive, and more effective than female sterilization (Trussell, 2011). Through the 1980s and 1990s, the prevalence of vasectomy continued to decline. In hopes of revitalizing vasectomy, the central government renewed its attention to the method. In 18 of the 29 Indian states, there has been a perceptible shift in focus on male sterilization in the past few years. Nationally, the number of procedures performed in the public sector nearly doubled between 2006–2007 and 2007–2008, and vasectomy’s contribution to the sterilization mix rose from 2.5% to 4.5%. A major factor contributing to vasectomy’s resurgence has been the focus on no-scalpel vasectomy (NSV). Commonly known as the “no suture, no cut” (Bina Tanka-Bina Chira,) operation, NSV is characterized by less pain, fewer complications, and a quicker return to sexual activity than conventional vasectomy—although both methods require the use of back￾up contraception for four months after the procedure. In both Uttar Pradesh (UP) and Jharkhand states of India, however, male sterilization comprises just over 1% of all modern FP use. Female sterilization, on the other hand, represents 44% of modern FP use in UP and 70% in Jharkhand (IIPS & Macro International, 2007). While awareness of vasectomy is high, misinformation about vasectomy is pervasive. In one study in rural UP, while nine out of 10 men and women reported being aware of vasectomy, fewer than half of the men and only one-fourth of the women had correct information on the procedure/method (Khan & Patel, 1997). Misconceptions about vasectomy’s causing weakness, both physically and sexually, and requiring long periods of rest following the procedure are common. Other studies in UP indicate that two out of three men 2 Evaluation of NSV Initiative in Uttar Pradesh and Jharkhand States The RESPOND Project  Report No. 15 believe that female sterilization is easier to perform than vasectomy, and more than half of the men surveyed commented that vasectomy has a greater chance of postoperative complications and a greater risk of failure than female sterilization (Shrestha, Stoeckel & Tuladhar, 1988; CORT, 2000). Likewise, many providers, including auxiliary nurse-midwives (ANMs), have little information on the details of the procedure and are reluctant to counsel on it (Varkey & Nithyanand, 2009). RESPOND NSV Initiative To ensure that men and couples have access to quality NSV services and accurate information about all FP methods, the U.S. Agency for International Development (USAID)/India supported a four-year technical assistance project. The RESPOND NSV Initiative in India was managed by EngenderHealth under the global RESPOND Project, in partnership with Johns Hopkins Bloomberg School of Public Health Center for Communication Programs (JHUCCP). The project’s technical assistance followed EngenderHealth’s holistic Supply– Enabling Environment–Demand (SEED) Programming Model (Figure 1) and provided technical assistance to the governments of UP and Jharkhand to expand awareness and acceptance of and access to NSV services. The RESPOND NSV Initiative started in October 2009 in three districts of UP and gradually expanded to nine districts in UP and six in Jharkhand. The initiative ended in September 2013. Figure 1: Supply–Enabling Environment–Demand (SEED) Model Supply The project provided support to ensure the quality of NSV services at 44 facilities in UP and at 32 facilities in Jharkhand. RESPOND trained and followed up with 142 NSV surgeons, while building the capacity of the states to conduct NSV trainings by strengthening three training centers in each state. The RESPOND Project  Report No. 15 Evaluation of NSV Initiative in Uttar Pradesh and Jharkhand States 3 Enabling Environment An enabling environment for FP requires equitable policies; adequate resources; good governance, management, and accountability; and supportive social norms, including the transformation of inequitable gender norms. If these needs are not addressed, investments in supply and demand interventions may be neither effective in the short term nor sustainable over time. Strong and effective leadership is crucial for creating an enabling environment at all levels of the health care system and within the community to support and advance FP and sexual and reproductive health (SRH). This leadership is needed to promote evidence-based policies, guidelines, and approaches; support the allocation of human and financial resources for FP/SRH; ensure that the health system has the capacity to provide quality services; and challenge social and gender norms that may adversely affect an individual’s SRH. RESPOND worked to create an enabling environment for NSV services by encouraging supportive guidelines and practices and by supporting the states to maximize resources to generate demand while ensuring quality NSV services. Demand In India, fieldworkers in the community create awareness on health and mobilize the community toward increased utilization of existing health services; in UP, these women are called accredited social health activists (ASHAs); in Jharkhand, their equivalents are called sahiyas. To stimulate demand for NSV services, RESPOND oriented 6,875 ASHAs in UP and 2,517 sahiyas in Jharkhand to provide holistic FP counseling and referrals for services, including NSV. The project provided further one-on-one coaching to 338 ASHAs and 344 sahiyas that focused on interpersonal communication skills and counseling on all FP methods and that included a specialized component on how to better inform couples interested in permanent methods about both female sterilization and NSV. RESPOND also developed behavior change communication (BCC) materials, including a movie and leaflets about NSV, as well as information, education, and communication (IEC) materials to help ASHAs/sahiyas explain NSV to clients. Key project indicators are included as Appendix 1. Impact of NSV Initiative on Service Uptake The following section presents NSV service data in intervention and nonintervention districts of UP and Jharkhand states. These data are derived from the states’ health management information systems. NSV Trends in Intervention and Nonintervention Districts Although RESPOND-supported districts comprised just 15.4% of the population of all districts in the state of UP and 37.0% in Jharkhand (Census Organization of India, 2011), by the last year of the project they provided nearly half (47.9%) of all NSV procedures in UP and more than two-thirds (68.3%) in Jharkhand (Table 1). NSV uptake in supported districts of both states increased over the first three years of the project, while declines in NSV uptake were seen in both intervention and nonintervention districts of Jharkhand and in intervention districts of UP during the project’s last year. 4 Evaluation of NSV Initiative in Uttar Pradesh and Jharkhand States The RESPOND Project  Report No. 15 The declines reflect challenges in fostering policy and administrative support for NSV at the state levels; in particular, several interruptions in payment of reimbursements to facilities and clients occurred over the life of the project. In UP, for example, NSV services were not provided at all for nearly five months in 2013 due to the state’s nonpayment of reimbursements for clients. Table 1: NSV uptake in intervention and nonintervention districts Year 1 April 1, 2009– March 31, 2010 (baseline) Year 2 April 1, 2010– March 31, 2011 Year 3 April 1, 2011– March 31, 2012 Year 4 April 1, 2012– March 31, 2013 UP All districts (n=72) 9,289 7,782 8,658 7,176 RESPOND districts (n=9) 1,257 2,877 5,009 3,439 Other districts (n=63) 8,032 4,905 3,649 3,737 % contribution of RESPOND districts 13.5% 37.0% 57.8% 47.9% Jharkhand All districts (n=24) 6,905* 12,442 13,327 9,198 RESPOND districts (n=6) 3,893* 7,505 7,146 6,284 Other districts (n=18) 3,012* 4,937 6,181 2,914 % contribution of RESPOND districts 56.4%* 60.3% 53.6% 68.3% * RESPOND did not begin to work in Jharkhand until Year 2; Year 1 figures are provided only as a benchmark. Source: Management information systems, state departments of family welfare The RESPOND Project  Report No. 15 Evaluation of NSV Initiative in Uttar Pradesh and Jharkhand States 5 Evaluation Methodology and Implementation In September and October 2013, the RESPOND Project conducted an end-of-project evaluation. The primary objective of the evaluation was to assess the extent to which the RESPOND NSV Initiative built capacity in service provision at the state, district, and facility levels and promoted service delivery at the ASHA/sahiyas level so as to increase availability of and demand for quality NSV services. Study Objectives The objectives of the end-of-project evaluation were to answer the following study questions:  What is the access to and quality of NSV services in the intervention districts?  What is the capacity of ASHAs/sahiyas to counsel on all FP methods, including NSV?  What changes in NSV-related policies or guidelines were achieved at the facility, district, and state levels? Study Design The end-of-project evaluation was implemented in six project districts in UP and in three project districts in Jharkhand. A stratified sampling scheme was used to select districts providing high, medium, and low numbers in each state, to ensure representation of high- and low-performing districts. Sampled districts in UP were Allahabad, Bulandshahr, Ghaziabad, Kanpur Nagar, Kaushambi, and Meerut; in Jharkhand, sampled districts were Bokaro, Dhanbad, and West Singhbhum. Within districts, facilities were randomly selected, and at each selected facility, providers, ANMs, ASHAs/sahiyas, and clients were interviewed. Providers and ASHAs/sahiyas were also observed. Observations of providers were conducted by trained medical doctors. Finally, 19 stakeholder interviews were conducted; stakeholders consisted of two state family welfare officers, five chief medical officers (CMOs), eight deputy CMOs, and four health education officers. Table 2 shows the number of individuals interviewed/observed in each state. Eligible facilities were those with a RESPOND-trained surgeon as well as coached ASHAs/sahiyas. Eligible providers had received training from RESPOND and were providing NSV services at a RESPOND-supported facility at the time of the survey. Eligible ASHAs/sahiyas were associated with a supported facility and had received coaching on interpersonal skills; those who only participated in a one-time orientation were excluded. 6 Evaluation of NSV Initiative in Uttar Pradesh and Jharkhand States The RESPOND Project  Report No. 15 Table 2: Number of facility audits, observations, and interviews conducted UP (6 districts) Jharkhand (3 districts) Total (9 districts) Facility audits 15 9 24 Provider observations 13 9 22 Surgeon interviews 14 9 23 Facility in-charge interviews 11 9 20 NSV client exit interviews 13 12 25 ASHA/sahiya observations 27 17 44 ASHA/sahiya interviews 25 14 39 ANM interviews 15 7 22 Stakeholder interviews 13 6 19 CMOs 3 2 5 Deputy CMOs 6 2 8 Health Education Officers 3 1 4 State Officers 1 1 2 Data Collection Data were collected by five teams of two people; one data collector had a medical background and the other a social science background. From September 5–7, 2013, data collectors completed a training covering the project background, study design and objectives of the evaluation, research ethics and informed consent procedures, steps in NSV surgery, and mock practice in data collection. Fieldwork commenced the week following the training. Facility visits were scheduled to coincide with a day when at least one NSV surgery was scheduled to be performed at the sampled health facilities. When possible, tools were developed based on monitoring and supervision tools utilized by the RESPOND NSV Initiative. The study protocol, the study tools, and consent forms were reviewed by the Institutional Review Board, Centre for Media Studies, New Delhi for approval on July 17, 2013. Approval was received on August 19, 2013. Additionally, a quality improvement exemption was requested and received from Western Institutional Review Board on August 6, 2013. In observing 25 ASHA and 14 sahiya counseling sessions with men in the community, data collectors used a checklist to note the methods discussed, the specific aspects of NSV discussed, and the interpersonal communication skills exhibited by the ASHA/sahiya. To protect clients’ right to informed decision making and to ensure that ASHAs/sahiyas did not feel pressured to steer clients toward NSV, clients’ decisions to choose NSV, another FP method, or no method at all were not recorded. The RESPOND Project  Report No. 15 Evaluation of NSV Initiative in Uttar Pradesh and Jharkhand States 7 Analysis Study tools explored the RESPOND NSV Initiative’s achievements in each of the three project objectives, which corresponded to the elements of the SEED model. Results are disaggregated by state as well as presented as totals. All essential equipment and supplies were verified by observation using a facility audit tool. Equipment and supplies included in this section include only those deemed essential by EngenderHealth; the full list of both essential and recommended equipment and supplies is shown in Appendix 3. In tables presenting the availability of essential items for each purpose (infection prevention [IP], NSV procedure, pain management, and emergency management), a composite score is also presented; this composite score notes the percentage of audited facilities that had all essential items in stock on the day of the visit. Similarly, all essential clinical and counseling steps were verified by observation using a checklist. Composite scores are provided for each subcategory of counseling steps, as well as for clinical steps performed by surgeons. These composite scores indicate the percentage of those observed who performed each of the essential counseling or clinical steps in each subcategory. Limitations General  The results may not be representative of all RESPOND-supported districts.  Comparative baseline data are not available.  As the RESPOND NSV Initiative worked to strengthen support for NSV at the state level as well as the district level; therefore, appropriate control districts could not be selected. Client exit interview  Respondents may have given answers that they thought the interviewer wanted to hear, rather than what they really felt (courtesy bias). Observations of ASHAs/sahiyas and surgeons  The presence of an observer during the counseling and clinical sessions may have positively or negatively influenced the ASHA/sahiya/surgeon’s performance. The provider population of interest was RESPOND-trained surgeons providing NSV services in RESPOND-supported facilities. To be eligible for this study, a provider must have received training from RESPOND and must currently provide NSV services. One provider at each facility was randomly selected for interview and for observation by a data collector with a medical background (medical school graduate). A total of 21 provider observations and 22 provider interviews were completed. A total of 20 facility in-charges were also interviewed. The RESPOND Project  Report No. 15 Evaluation of NSV Initiative in Uttar Pradesh and Jharkhand States 9 Results Characteristics of Facilities and Participants Facilities A total of 24 facilities were randomly selected, 15 from UP and nine from Jharkhand; they represented one-third of RESPOND-supported facilities. One-half of sampled facilities (50%) were community health centers, and nearly one-quarter were district hospitals (20.8%) (Table 3). Table 3: Percentage distribution of facilities in the evaluation sample, by type % (n) UP (n=15) Jharkhand (n=9) Total (n=24) Community or primary health center 40.0 (6) 66.7 (6) 50.0 (12) District hospital 26.7 (4) 11.1 (1) 20.8 (5) Other public hospital 6.7 (1) 11.1 (1) 8.3 (2) Private hospital 13.3 (2) 0.0 (0) 8.3 (2) Private clinic 13.3 (2) 11.1 (1) 12.5 (3) Total 100.0 (15) 100.0 (9) 100.0 (24) NSV Surgeons A total of 21 provider observations and 22 provider interviews were completed; not all providers were willing or available to participate. In UP, all of the participating surgeons held the equivalent of a doctor of medicine (MD) degree, and more than half (57%; n=14) had a postgraduate degree in surgery.3 In Jharkhand, only one-third of the surgeons held a postgraduate degree (n=3). The majority of the surgeons (64% in UP, 67% in Jharkhand) knew about NSV before RESPOND, but most were not performing the procedure prior to RESPOND (50% in UP, 22% in Jharkhand). ASHAs/Sahiyas and ANMs Forty-four ASHA/sahiya counseling sessions were observed, and 39 ASHA/sahiya interviews were conducted. A total of 22 interviews were completed with ANMs, who supervise the demand generation activities of ASHAs. The ASHAs/sahiyas had been in their positions for an average of 6.8 years. On average, ASHAs (in UP) had worked as fieldworkers in their communities for longer than sahiyas (in Jharkhand) (7.9 years, compared with 5.6 years). ANMs had been affiliated with their current facility for an average of 11.5 years (12.8 in UP; 8.7 in Jharkhand).                                                              3 The basic medical degree in India is the Bachelor of Medicine and Bachelor of Surgery (MBBS) and is the equivalent of the MD degree in the United States. After this basic degree, a postgraduate degree in general surgery or in general medicine is pursued. 10 Evaluation of NSV Initiative in Uttar Pradesh and Jharkhand States The RESPOND Project  Report No. 15 NSV Clients A total of 25 clients were interviewed (13 in UP; 12 in Jharkhand). The clients’ mean age was 33 years in UP and 36 years in Jharkhand. NSV clients in UP had an average of three living children, compared with five among clients in Jharkhand. Just over half (56%) of NSV clients reported that they had previously used any modern contraceptive method (Table 4). Among previous users, the most frequently reported method was the condom (93%), followed by the intrauterine device (IUD) (21%). A smaller percentage had used the pill or emergency contraception (7% each). Table 4: Percentage of NSV clients reporting previous use of FP methods* UP (n=13) Jharkhand (n=12) Total (n=25) % who previously used any method 53.8 (7) 58.3 (7) 56.0 (14) % who used Condom 85.7 (6) 100.0 (7) 92.9 (13) IUD 42.9 (3) 0.0 (0) 21.4 (3) Pill 14.3 (1) 0.0 (0) 7.1 (1) Emergency contraception 0.0 (0) 14.3 (1) 7.1 (1) * Clients could report previous use of more than one method. Source: Client exit interview (2013) Supply In this section, the findings relate to the supply of quality NSV services at the 24 audited facilities. Included are: availability of key infrastructure, equipment, and supplies; presence of IP supplies and equipment, medical instruments, pain management supplies, and emergency management supplies; provider and counseling skills; and client satisfaction with services. Facility Readiness to Provide NSV Only 37.5% of all facilities audited (46.7% in UP, 22.2% in Jharkhand) had all necessary supplies and equipment (five IP supplies, four IP equipment/infrastructure components, six NSV instruments and supplies, and two pain management supplies) in stock on the day of the visit. IP Supplies and Equipment IP supplies and equipment are essential for the provision of quality NSV services. However, just over half (58.3%) of all facilities had all of the five essential IP supplies and four essential IP infrastructure/equipment components available on the day of the visit (Table 5). This proportion varied widely between the two states, with only 22% of facilities in Jharkhand having all essential elements, compared with 80% in UP. All facilities in UP had gloves, sterile gauze, iodine, soap, running water, an autoclave and appropriate disposal containers in stock on the day of the visit (see Table 5). However, the only items available in all facilities in Jharkhand were gloves and an autoclave, with many other essential items out of stock on the day of the visit. The RESPOND Project  Report No. 15 Evaluation of NSV Initiative in Uttar Pradesh and Jharkhand States 11 Table 5: Percentage of facilities with selected essential supplies and equipment on day of visit UP (n=15) Jharkhand (n=9) Total (n=24) IP supplies Soap or other antiseptic agent 100.0 (15) 77.8 (7) 91.7 (22) Sterile gauze 100.0 (15) 55.6 (5) 83.3 (20) Sterile gloves 100.0 (15) 100.0 (9) 100.0 (24) Decontamination solution 86.7 (13) 77.8 (7) 83.3 (20) Iodine 100.0 (15) 77.8 (7) 91.7 (22) All essential IP supplies 86.7 (13) 55.6 (5) 75.0 (18) IP infrastructure and equipment Running water 100.0 (15) 77.8 (7) 91.7 (22) Leak-proof containers for decontaminating soiled instruments 100.0 (15) 88.9 (8) 95.8 (23) Autoclave 100.0 (15) 100.0 (9) 100.0 (24) Puncture-proof containers for disposal of sharps 93.3 (14) 55.6 (5) 79.2 (19) All essential infrastructure and equipment 93.3 (14) 44.4 (4) 75.0 (18) All essential IP supplies plus infrastructure and equipment 80.0 (12) 22.2 (2) 58.3 (14) Medical and NSV-specific instruments and supplies NSV ringed clamp forceps 80.0 (12) 88.9 (8) 83.3 (20) NSV dissecting forceps 80.0 (12) 88.9 (8) 83.3 (20) Cup/bowl/gallipot 100.0 (15) 88.9 (8) 95.8 (23) Sponge-holding forceps, straight, 5.5 in. 93.3 (14) 77.8(7) 87.5 (21) Scissors: operating, straight 93.3 (14) 77.8 (7) 87.5 (21) Chromic catgut/nonabsorbable silk/cotton suture for ligation 100.0 (15) 100.0 (9) 100.0 (24) All unique NSV and common instruments plus supplies 66.7 (10) 77.8 (7) 70.8 (17) Pain management supplies Lidocaine without epinephrine 93.3 (14) 100.0 (9) 95.8 (23) 10 ml syringe with 1.5 inch, 27-gauge needle 93.3 (14) 100.0 (9) 95.8 (23) Lidocaine plus syringe 86.7 (13) 100.0 (9) 91.7 (22) Emergency management supplies Antihistaminic 100.0 (15) 77.8 (7) 91.7 (22) Steroids 100.0 (15) 77.8 (7) 91.7 (22) Syringe 100.0 (15) 88.9 (8) 95.8 (23) Oxygen with mask 100.0 (15) 88.9 (8) 95.8 (23) Suction apparatus 100.0 (15) 88.9 (8) 95.8 (23) At least one staff person trained in emergency management 93.3 (14) 66.7 (6) 83.3 (20) All essential emergency management supplies plus trained staff person 93.3 (14) 66.7 (6) 83.3 (20) Source: Facility audit (2013) Medical and NSV-Specific Instruments and Supplies Five medical instruments (three types of forceps, operating scissors, and cup/bowl/gallipot), as well as catgut, silk, or suture, are essential to properly performing the NSV procedure. Only 71% of all facilities had each of the six essential medical instruments and supplies on the day of the visit (Table 5). Most commonly missing supplies were the NSV ringed clamp forceps and the NSV dissecting forceps (available at 83% of facilities in total). 12 Evaluation of NSV Initiative in Uttar Pradesh and Jharkhand States The RESPOND Project  Report No. 15 Pain Management Supplies Proper pain management is critical to ensuring client comfort during the NSV procedure, enabling the surgeon to perform the procedure correctly and influencing the likelihood that the client will recommend the procedure and/or facility to others. Data collectors observed the availability of two essential pain management supplies and found that the majority of facilities (92%) had both items. While all facilities in Jharkhand had both lidocaine and a 10ml syringe in stock on the day of the audit, only 87% of facilities in UP had both. Emergency Management Data collectors asked whether facilities had a staff member trained in emergency management and observed the presence of five essential emergency management supplies. Eighty-three percent of facilities had both a trained provider and the five essential supplies. The element most frequently missing was the trained staff member; while 83% of facilities had at least one staff member trained in emergency management, that figure ranged from 93% in UP to 67% in Jharkhand. IEC Materials Table 6 summarizes the availability of IEC materials on the day of the facility audit, confirmed through observation. Shown in the table are both guidelines for providers and information for clients on different methods available. The most commonly available materials, available at almost all facilities (96% overall), were posters on contraceptive methods. The least commonly available materials were IP protocol charts or posters, available at 63% of facilities. Provider Counseling Skills Quality counseling is critical to ensuring that: (1) NSV is a suitable method for the client, (2) clients provide their informed consent to undergo the procedure, and (3) clients understand the postsurgical warning signs and instructions for postoperative care. Data collectors observed counseling sessions conducted by 22 RESPOND-trained surgeons and used a checklist to identify the elements of quality counseling that they observed. One hundred percent of surgeons in both states ensured confidentiality and privacy in counseling sessions, and 86% completed all essential counseling steps (Table 6). Verification of informed consent NSV surgeons are the last individuals to verify that a client understands the nature of the procedure he is about to undergo and that he has given his informed consent to do so. Data collectors observed that 96% of all 22 observed surgeons (93% in UP and 100% in Jharkhand) verified that clients consented to NSV before initiating the procedure. Postoperative instructions and warning signs Data collectors documented the instructions and warning signs discussed in provider counseling sessions. As can be seen in Table 6, 86% of all providers discussed each of the four instructions for postprocedure care, as well as the three warning signs—100% in Jharkhand 100and 77% in UP. The RESPOND Project  Report No. 15 Evaluation of NSV Initiative in Uttar Pradesh and Jharkhand States 13 Table 6: Percentage of facilities with selected IEC materials with providers who performed key tasks UP Jharkhand Total Facilities with essential IEC materials (n=15) (n=9) (n=24) Client posters depicting a variety of contraceptive options 93.3 (14) 100.0 (9) 95.8 (23) Current FP/RH guidelines 86.7 (13) 77.8 (7) 83.3 (20) IP protocol charts/posters 66.7 (10) 55.6 (5) 62.5 (15) Providers discussed postprocedural care instructions and warning signs (n=13) (n=9) (n=22) Avoid strenuous work for two days 100.0 (13) 100.0 (9) 100.0 (22) No cycling for seven days 100.0 (13) 100.0 (9) 100.0 (22) Use condom or other contraceptive for three months 84.6 (11) 100.0 (9) 90.9 (20) Get semen examination after three months 100.0 (13) 100.0 (9) 100.0 (22) Return to facility if this side effect is experienced: Severe pain 76.9 (10) 100.0 (9) 86.4 (19) Swelling 76.9 (10) 100.0 (9) 86.4 (19) Fever 76.9 (10) 100.0 (9) 86.4 (19) Total providing all postoperative instructions 76.9 (10) 100.0 (9) 86.4 (19) Providers performing essential NSV IP, anesthesia, procedural, and waste disposal steps (n=13) (n=9) (n=22) Prepared operative site with iodine 100.0 (13) 66.7 (6) 86.4 (19) All four anesthesia steps 84.6 (11) 88.9 (8) 86.4 (19) All nine NSV procedure steps 69.2 (9) 66.7 (6) 68.2 (15) All three waste disposal steps 84.6 (11) 55.6 (5) 72.7 (16) All essential steps 46.2 (6) 55.6 (5) 50.0 (11) Source: Facility audit and provider observation (2013) Provider Surgical Skills The NSV procedure consists of essential IP, anesthesia-related, NSV procedural, and waste disposal steps. All steps must be performed in the proper order to ensure IP, the client’s comfort, and the procedure’s effectiveness. Data collectors observed NSV procedures performed by 22 RESPOND-trained providers, using a checklist of 13 essential anesthesia￾related, procedural, and waste disposal steps. (The full lists of steps, and provider scores for each step, are included as Appendix 2.) NSV procedure Only 50% of providers (46% in UP and 56% in Jharkhand) performed all 13 essential steps (Table 6). Low performance on some steps reflects stock-outs, particularly in Jharkhand. Only 67% of surgeons in Jharkhand prepared the operative site with iodine; these surgeons’ facilities had stock-outs of iodine. Of all surgeons, 86% performed all four anesthesia-related steps; the most frequently missed step was Step 2: Raise a skin wheal injecting 1 cc of 2% lidocaine (86.4%). Just over two-thirds of surgeons in both states (68.2%) performed all nine NSV procedural steps. The most frequently missed step was Step 9: Gently pull the long suture of the prostatic 14 Evaluation of NSV Initiative in Uttar Pradesh and Jharkhand States The RESPOND Project  Report No. 15 end of the vas to reexpose the cut end of the vas, which will be covered with fascia, which was performed by only 77.3% of surgeons (Appendix 2). Proper disposal of waste was particularly weak in Jharkhand, with just over half (56%) following the three waste disposal steps, compared with 85% in UP. This primarily reflects the lack of availability of essential IP items at facilities in Jharkhand; as shown in Table 5 (page 11), many facilities lacked puncture-proof containers and containers for decontamination of sharps. Client Experience Clients’ perceptions of counseling Provider counseling observations were substantiated by exit interviews with client; data collectors asked clients whether they believed that their surgeon had adequately explained the NSV procedure and informed them of postoperative precautions. Eighty percent believed that the surgeon fully or adequately explained the NSV procedure, and 88% felt that the surgeon adequately explained the postoperative precautions (Table 7). However, 8% said that the procedure was not explained at all, and 4% reported that postoperative precautions were not explained at all. Table 7: Percentage distribution of clients, by whether the NSV surgeon informed them of different aspects of the NSV procedure UP (n=13) Jharkhand (n=12) Total (n=25) Believed surgeon explained NSV procedure Fully 46.2 (6) 25.0 (3) 36.0 (9) Adequately 23.1 (3) 66.7 (8) 44.0 (11) Inadequately 23.1 (3) 8.3 (1) 16.0 (4) Not at all 7.7 (1) 0 (0) 4.0 (1) Believed surgeon explained postoperative precautions Fully 61.5 (8) 25.0 (3) 44.0 (11) Adequately 23.1 (3) 66.7 (8) 44.0 (11) Inadequately 7.7 (1) 8.3 (1) 8.0 (2) Not at all 7.7 (1) 0 (0) 4.0 (1) Behavior of registration staff Very courteous 69.2 (9) 41.6 (5) 56.0 (14) Courteous 30.7 (4) 58.3 (7) 44.0 (11) Somewhat rude 0 (0) 0 (0) 0.0 (0) Very rude 0 (0) 0 (0) 0.0 (0) Behavior of surgeon Very courteous 84.6 (11) 41.6 (5) 64.0 (16) Courteous 15.4 (2) 58.3 (7) 36.0 (9) Somewhat rude 0 (0) 0 (0) 0.0 (0) Very rude 0 (0) 0 (0) 0.0 (0) Source: Client exit interviews (2013) The RESPOND Project  Report No. 15 Evaluation of NSV Initiative in Uttar Pradesh and Jharkhand States 15 Perceived staff behavior toward NSV clients Welcoming attitudes by facility staff attitudes are critical to ensuring that clients feel comfortable before and during their procedure and that they are able to ask questions as needed. Further, the presence of friendly and welcoming staff may encourage clients to recommend the facility to others, while unwelcoming staff could damage the facility’s reputation. During client exit interviews, data collectors asked NSV clients whether they perceived the behavior of registration staff and surgeons toward them to be courteous or rude. No clients in either state reported that registration staff or surgeons were somewhat or very rude toward them (Table 7). More than half (56%) of clients reported that registration staff were very courteous, and 64% reported that the surgeon was very courteous. Enabling Environment Data collected regarding the enabling environment for NSV include the availability of NSV services, stakeholders’ awareness of RESPOND and its activities, and changes in NSV-related policies and guidelines. Availability of NSV Services When interviewed, NSV clients were asked about the amount of time they had to spend to access and receive NSV services. NSV facilities were reported to be more easily accessible in Jharkhand than in UP: In UP, 46% of clients were able to get to the facility in 30 minutes or less, compared with 67% in Jharkhand. Once clients had arrived at the facility, registration was completed more quickly in Jharkhand than in UP, with 92% in Jharkhand needing less than 30 minutes to register, compared with 54% in UP. Waiting time was also much shorter in Jharkhand than in UP, with no clients waiting more than an hour in Jharkhand; in UP, nearly one-quarter waited more than two hours to see the surgeon. Table 8: Percentage distribution of NSV clients, by time required to access and receive NSV services UP (n=13) Jharkhand (n=12) Total (n=25) Time to reach the facility for service <15 minutes 7.7 (1) 0.0 (0) 4.0 (1) 15–30 minutes 38.5 (5) 66.7 (8) 52.0 (13) >30 minutes 53.8 (7) 33.3 (4) 44.0 (11) Time to complete registration <30 minutes 53.8 (7) 91.6 (11) 12.0 (18) 30–60 minutes 30.8 (4) 0.0 (0) 16.0 (4) >one hour 15.4 (2) 8.3 (1) 72.0 (3) Waiting time to see the surgeon <1 hour 69.0 (9) 100.0 (12) 84.0 (21) 1–2 hours 7.7 (1) 0.0 (0) 4.0 (1) >2 hours 23.1 (3) 0.0 (0) 12.0 (3) Source: Client exit interviews (2013) 16 Evaluation of NSV Initiative in Uttar Pradesh and Jharkhand States The RESPOND Project  Report No. 15 Awareness of RESPOND and RESPOND Activities State and district health officials (health education officers, CMOs, and deputy CMOs) in RESPOND-supported districts were asked about RESPOND’s activities to increase access to and demand for NSV at state, district, and other levels. In both states, stakeholders identified several initiatives led by RESPOND to create an enabling environment for NSV at the district level and to increase awareness of NSV among health care staff. These stakeholders were familiar with both supply-side and demand generation activities strengthened by RESPOND. Most commonly, district health officers (84.6% in UP and 66.7% in Jharkhand) knew that RESPOND had facilitated the training of surgeons, and many (53.8% in UP and 83.3% in Jharkhand) also knew that RESPOND had prepared and supplied IEC materials (Table 9). Table 9: Percentage of state and district officials* aware of RESPOND and RESPOND activities UP (n=13) Jharkhand (n=6) Awareness of the project Aware of RESPOND project 100.0 (13) 100.0 (6) Awareness of facility-level activities Training of surgeons 84.6 (11) 66.7 (4) One-on-one coaching of ASHAs/sahiyas 30.8 (4) 66.7 (4) Preparation and supply of IEC materials 53.8 (7) 83.3 (5) Advocacy with district and state program officers 30.8 (4) 100.0 (6) Support of work planning related to NSV services in facilities and camps 61.5 (8) 66.7 (4) Awareness of activities for demand creation Provision of IEC materials to district 30.8 (4) 50.0 (3) Assistance in IEC activities 30.8 (4) 83.3 (5) Orientation of ASHAs/sahiyas 69.2 (9) 83.3 (5) * District officials include health education officers, CMOs, and deputy CMOs. Source: State and district officer interviews (2013) Changes in NSV-Related Practices In Jharkhand, slightly more than half (56%) of NSV surgeons and facility in-charges reported that changes had occurred in practices related to NSV in their facility, compared with 64% of surgeons and 45% of in-charges in UP (Table 10). Most reported changes focused on the increased availability of NSV services, although the actual changes reported differed by state and by stakeholder. The most commonly cited change among surgeons in UP (89%) was that NSV services were now available more regularly, while only 40% in Jharkhand reported the same. In Jharkhand, the changes most frequently cited by surgeons were the involvement of other staff in NSV (60%) and the increased number of surgeons trained (60%). Among facility in-charges in UP, the most frequently cited change was that NSV clients were being attended to more promptly, which was cited by only 43% of in-charges in Jharkhand. The RESPOND Project  Report No. 15 Evaluation of NSV Initiative in Uttar Pradesh and Jharkhand States 17 Table 10: Percentage of surgeons and in-charges reporting facility-level changes in NSV-related practices and guidelines, and types of changes reported, by state Change NSV surgeons Facility in-charges UP (n=14) Jharkhand (n=9) UP (n=11) Jharkhand (n=9) % reporting changes 64.3 (9) 55.6 (5) 45.5 (5) 55.6 (5) Change (among those reporting facility-level changes): (n=9) (n=5) (n=5) (n=5) Services are available more regularly. 88.9 (8) 40.0 (2) 60.0 (3) 20.0 (1) Other staff are involved in NSV. 0.0 (0) 60.0 (3) 40.0 (2) 60.0 (3) More surgeons have been trained. 0.0 (0) 60.0 (3) 20.0 (1) 0.0 (0) NSV clients are attended promptly. 0.0 (0) 0.0 (0) 80.0 (4) 40.0 (2) Monitoring of NSV activities is increased. 0.0 (0) 0.0 (0) 40.0 (2) 0.0 (0) More IEC activities are undertaken. 55.6 (5) 40.0 (2) 20.0 (1) 0.0 (0) Source: NSV surgeon and facility in-charges (2013) Anecdotally, one severe limitation to the enabling environment noted over the course of the project, as well as by the data collectors, was the frequent transfer of trained surgeons. Many trained surgeons were posted to facilities at which they could not use their NSV skills. In other situations, supported facilities no longer had a surgeon to provide NSV services; those facilities were either referring the clients to other facilities or inviting surgeons from other facilities to provide services when clients came. Demand The findings presented in the following section relate to RESPOND’s activities to build the capacity of the states and districts to implement FP and NSV demand-generation activities. Elements included in this section include stakeholders’ awareness of RESPOND’s demand￾generation activities, clients’ knowledge of FP methods and reasons for choosing NSV, and the counseling skills of ASHAs/sahiyas. Awareness of RESPOND Demand-Generation Activities RESPOND stakeholders—including NSV surgeons (23), facility in-charges (20), ASHAs (39), and ANMs (22)—were asked about their awareness of the demand-creation activities undertaken by RESPOND. The different stakeholder groups differed in their knowledge of demand-creation activities (see Table 11). Surgeons, for example, were not aware of most of the materials (posters, banners) developed by the project and used by other stakeholders for demand creation, while all ASHAs in both states knew of the posters and brochures that had been developed. Surgeons had the lowest knowledge of demand creation activities and products, with 7% in UP and 11% in Jharkhand not able to identify any. Awareness of the movie developed by RESPOND was relatively low, with just over half of ASHAs/sahiyas in both states knowing of it; this indicates not only that they had not seen the movie or did not recall seeing it, but also that they were not using it in their own demand creation efforts. It is important to note that all responses were spontaneous (unprompted). 18 Evaluation of NSV Initiative in Uttar Pradesh and Jharkhand States The RESPOND Project  Report No. 15 Table 11: Percentage of stakeholders reporting demand-generation activities led by RESPOND (spontaneous responses) Activity NSV Surgeons Facility In-Charges ASHAs/Sahiyas ANMs UP (n=14) Jharkhand (n=9) UP (n=11) Jharkhand (n=9) UP (n=25) Jharkhand (n=14) UP (n=15) Jharkhand (n=7) Mass media 71.4 (11) 77.8 (7) 0.0 (0) 0.0 (0) 0.0 (0) 0.0 (0) 0.0 (0) 0.0 (0) Group meetings 71.4 (11) 55.6 (5) 0.0 (0) 0.0 (0) 56.0 (14) 92.8 (13) 60.0 (9) 71.4 (5) Movie 7.1 (1) 0.0 (0) 81.8 (9) 44.4 (4) 56.0 (14) 57.1 (8) 60.0 (9) 28.5 (2) Brochures 7.1 (1) 0.0 (0) 81.8 (9) 88.9 (8) 100.0 (25) 100.0 (14) 100.0 (15) 100.0 (7) Posters 0.0 (0) 0.0 (0) 100.0 (11) 88.9 (8) 100.0 (25) 100.0 (25) 0.0 (0) 0.0 (0) Banners 0.0 (0) 0.0 (0) 100.0 (11) 88.9 (8) 0.0 (0) 0.0 (2) 100.0 (15) 100.0 (7) Glow signs 0.0 (0) 0.0 (0) 9.0 (1) 66.7 (6) 0.0 (0) 0.0 (0) 0.0 (0) 0.0 (0) Folk programmes 0.0 (0) 0.0 (0) 0.0 (0) 0.0 (0) 28.0 (7) 57.1 (8) 33.3 (5) 57.1 (4) Don’t know 7.1 (1) 11.1 (1) 0.0 (0) 0.0 (0) 0.0 (0) 0.0 (0) 0.0 (0) 0.0 (0) Source: Interviews with NSV surgeons, facility in-charges, ASHAs/sahiyas, and ANMs (2013) Client s’ Knowledge of NSV Data collectors read a list of methods and asked the clients to indicate which they knew. The mean number of methods known was 3.8; this mean was slightly lower in UP (3.5) than in Jharkhand (4.2). After NSV, the most familiar FP methods were female sterilization and the male condom, known by 84% of all clients. Table 12: Percentage of clients reporting awareness of each FP method Method UP (n=13) Jharkhand (n=12) Total (n=25) NSV 100.0 (13) 100.0 (12) 100.0 (25) Female sterilization 69.2 (9) 100.0 (12) 84.0 (21) Male condom 92.3 (12) 75.0 (9) 84.0 (21) Traditional vasectomy 23.1 (3) 66.7 (8) 44.0 (11) Pill 30.8 (4) 50.0 (6) 40.0 (10) IUD 30.8 (4) 8.3 (1) 20.0 (5) Emergency contraception 0.0 (0) 8.3 (1) 4.0 (1) Injectable* 0.0 (0) 8.3 (1) 4.0 (1) Female condom 0.0 (0) 0.0 (0) 0.0 (0) Mean number of methods known 3.5 4.2 3.8 *The injectable is not available at public facilities in India; this explains the low knowledge of this method. Source: Client exit interview (2013) Clients’ Reasons for Choosing NSV In exit interviews, data collectors explored NSV clients’ reasons for choosing the method. Their reasons consisted of both method characteristics and social factors. NSV’s permanent nature was the most common method-specific reason, given by 84% of clients. Another The RESPOND Project  Report No. 15 Evaluation of NSV Initiative in Uttar Pradesh and Jharkhand States 19 common reasons related to the method was its simplicity (72%); smaller proportions of clients cited the limited recovery time (28%) and low side effects (12%) associated with the method. Many clients also cited social factors as a reason for their choice of NSV. Just over one-half of clients (52%) stated that they chose NSV because they did not want their wife to undergo female sterilization,4 while just under one-half (44%) reported that they chose NSV because of a sense of responsibility to their wife and family. Smaller proportions of men reported being influenced by the NSV publicity (32%), encouraged by their wife (28%), or provided with NSV information by facility staff (24%). Table 13: Percentage of clients offering various reasons for choosing NSV, by state UP (n=13) Jharkhand (n=12) Total (n=25) Method characteristics Permanence of the method 84.6 (11) 83.3 (10) 84.0 (21) Simplicity of the method 69.2 (9) 75.0 (9) 72.0 (18) Limited recovery time 30.8 (4) 25.0 (3) 28.0 (7) Low side effects 7.7 (1) 16.7 (2) 12.0 (3) Social factors Reluctance for wife to go for female sterilization 53.8 (7) 50.0 (6) 52.0 (13) Responsibility toward wife and family 15.4 (2) 75.0 (9) 44.0 (11) Publicity about NSV 38.5 (5) 25.0 (3) 32.0 (8) Encouragement from wife 7.7 (1) 50.0 (6) 28.0 (7) Provision of information on NSV by facility staff 30.8 (4) 16.7 (2) 24.0 (6) Note: Multiple responses were possible. Source: Client exit interview (2013) Counseling Skills of ASHAs/Sahiyas Complete and accurate counseling by ASHAs/sahiyas is crucial to helping men and women determine which method is best suited for their reproductive intentions and family needs. Formative research conducted by the RESPOND Project in UP in 2010 provided insights into the common barriers to NSV uptake, finding that fears of weakness and impaired sexual performance limited the willingness of both men and women to consider the procedure (Scott, Alam, & Raman, 2011). Orientations for and coaching of ASHAs/sahiyas conducted by RESPOND therefore focused on providing accurate information and dispelling misconceptions about NSV and the NSV procedure, in addition to strengthening interpersonal communication skills. FP Methods Discussed Data collectors noted the methods discussed by each ASHA/sahiya. The ASHAs/Sahiyas tailored counseling sessions to meet the needs and interests of the client; for example, they may not have discussed sterilization with a client who indicated that he wanted to have another child.                                                              4 Formative research conducted by the RESPOND Project found that many community members, men in particular, believed that undergoing female sterilization could make a woman weak and unable to keep up with her household work. 20 Evaluation of NSV Initiative in Uttar Pradesh and Jharkhand States The RESPOND Project  Report No. 15 As can be seen in Table 14, the ASHAs in UP discussed 3.8 methods with clients, on average, compared with 4.2 discussed by sahiyas in Jharkhand. Male and female sterilization were the methods most frequently discussed during counseling sessions, discussed by 93.2% of ASHAs/Sahiyas. The injectable was the least discussed method, discussed by only 22.7% of ASHAs/Sahiyas; however, the method is not available in the public sector in India, and is therefore not likely to be known by many ASHAs/Sahiyas. Table 14: Percentage of ASHAs/sahiyas providing information on various FP methods and on key elements and benefits of the NSV procedure, and percentage exhibiting essential interpersonal skills, by state UP (n=27) Jharkhand (n=17) Total (n=44) Provided information on FP methods* during counseling session Female sterilization 92.6 (25) 94.1 (16) 93.2 (41) Male sterilization 92.6 (25) 94.1 (16) 93.2 (41) Male condom 74.1 (20) 100.0 (17) 84.1 (37) IUD 85.2 (23) 64.7 (11) 77.3 (34) Pill 62.9 (17) 76.5 (13) 68.2 (30) Natural method(s) 25.9 (7) 64.7 (11) 40.9 (18) Injectable 18.5 (5) 29.4 (5) 22.7 (10) Mean no. of methods discussed 3.8 4.2 4.0 Provided information on key elements of NSV procedure No incision 100.0 (25) 100.0 (16) 100.0 (41) No stitches 96.0 (24) 100.0 (16) 97.6 (40) Takes 10–12 minutes 88.0 (22) 100.0 (16) 92.7 (38) Can go home same day 88.0 (22) 93.8 (15) 90.2 (37) Painless 92.0 (23) 75.0 (12) 85.4 (35) Use of local anesthesia 52.0 (13) 81.3 (13) 63.4 (26) Vas is blocked after puncturing skin 56.0 (14) 68.8 (11) 61.0 (25) No blood loss 64.0 (16) 56.3 (9) 61.0 (25) All eight elements of NSV procedure 40.0 (10) 56.3 (9) 46.3 (19) Discussed key benefits of NSV procedure No weakness 88.0 (22) 100.0 (16) 92.7 (38) Normal sex life 80.0 (20) 87.5 (14) 82.9 (34) Normal erection 64.0 (16) 43.8 (7) 56.1 (23) Ejaculation with semen 56.0 (14) 50.0 (8) 53.7 (22) All four benefits of NSV 44.0 (11) 43.8 (7) 43.9 (18) Exhibited essential interpersonal skills Encouraged client to make best decision for him and his family 100.0 (27) 100.0 (17) 100.0 (44) Asked whether participant understood 92.6 (25) 100.0 (17) 95.5 (42) Told client to discuss options with spouse 100.0 (27) 88.2 (15) 95.5 (42) Gave opportunity for participant to ask questions 81.5 (22) 100.0 (17) 88.6 (39) All four interpersonal communication skills 77.8 (21) 88.2 (15) 81.8 (36) * The contraceptive implant is not available in India and was therefore not discussed by any ASHA/sahiya. Source: ASHA/sahiya observations (2013) The RESPOND Project  Report No. 15 Evaluation of NSV Initiative in Uttar Pradesh and Jharkhand States 21 Aspects of NSV Procedure Discussed Fewer than half (46%) of observed ASHAs/sahiyas discussed all eight elements of the surgical procedure (Table 14). (Note: These data do not include those who did not discuss NSV at all.) All observed ASHAs/sahiyas (100%) who discussed NSV disclosed the fact that the procedure requires no incision, and more than 90% discussed the absence of stitches, the short duration of the procedure, and the client’s ability to return home on the same day of the procedure. However, fewer than two-thirds discussed the use of local anesthesia, the fact that the vas is blocked via a skin puncture, or the fact that no blood is lost as a result of the procedure. Benefits of NSV Procedure Discussed Fewer than half (44%) of ASHAs/sahiyas discussed each of the four benefits of NSV emphasized in ASHA orientation and coaching sessions (Table 14). (Note: These data exclude those who did not discuss NSV at all.) While more than 90% discussed how the procedure would not lead to weakness, an important concern for both men and women, just over half explained that men would still have a normal erection and would still ejaculate semen after the procedure.5 Interpersonal Communication Skills More than three-quarters (82%) of observed ASHAs/sahiyas exhibited the four elements of interpersonal communication skills promoted by RESPOND through coaching sessions. In both states, all observed ASHAs/sahiyas encouraged the client to make the best decision for him and for his family, and more than 95% asked whether the participant understood the information she had presented and told the client to discuss the information with his spouse. Anecdotally, data collectors noted that ASHAs/sahiyas were very reluctant to hold counseling sessions. While this may reflect shyness to a certain extent, it also likely reflects a lack of confidence in their NSV knowledge and general counseling skills.                                                              5 Previous research found that it is important for men and women to understand that semen would still be ejaculated following the NSV procedure. The RESPOND Project  Report No. 15 Evaluation of NSV Initiative in Uttar Pradesh and Jharkhand States 23 Conclusions The RESPOND NSV Initiative was successful in increasing uptake of NSV services in project districts, and the project was well-known by key stakeholders. However, more effort is needed to fully institutionalize the promotion and provision of quality NSV services by the states of UP and Jharkhand. For example, key counseling and clinical steps were omitted by surgeons, and facilities were not fully equipped with the supplies and equipment required for quality services. Transfers of providers represent an ongoing challenge that limits the ability of the states to provide routine NSV services. Further, knowledge of NSV among ASHAs/sahiyas and ANMs was incomplete, and FP counseling provided by ASHAs/sahiyas needs to be improved to ensure that these fieldworkers are able to accurately and effectively discuss all FP methods. Refresher trainings for surgeons and ASHA/sahiyas, as well as ongoing follow-up, are needed to build capacity and confidence. The RESPOND Project  Report No. 15 Evaluation of NSV Initiative in Uttar Pradesh and Jharkhand States 25 Lessons Learned Supply  More extensive follow-up and mentoring of trained NSV surgeons and facility staff is needed. NSV surgeons require additional posttraining support, in the form of routine monitoring and supervision, as well as refresher trainings, as needed, to ensure that they properly perform each step of the NSV procedure. Enabling Environment  Stock-outs of essential supplies and equipment limited the ability of facilities, particularly those in Jharkhand, to provide quality services, including NSV. Facilities need support to ensure that stocks of essential supplies and equipment are maintained, so that clients receive the highest possible standard of care.  Further activities to strengthen the enabling environment are needed to ensure the sustainability of the NSV Initiative. These activities should consist of: o Building supervisory systems to support NSV surgeons o Training ANMs and health education officers to train and support ASHAs/sahiyas to discuss the full range of FP methods o Strengthening supply chains to reduce stock-outs o Ensuring that funds for reimbursement payments are available, to avoid interruption of services Demand  As ASHAs/sahiyas are a primary source of FP information for men and women, it is critical that they themselves possess complete and accurate information about all methods, to be able to fully address the concerns of men and women in their community. ASHAs/sahiyas need additional training and follow-up to ensure that their FP knowledge is complete and accurate and to strengthen their counseling skills. Rather than orienting large numbers of ASHAs/sahiyas, future projects should focus on providing intensive support and follow-up to a limited number of fieldworkers.  Future projects should consider providing training to ASHAs’ husbands, since men often have concerns that they may prefer to discuss with other men. During interviews, some ASHAs/sahiyas revealed that they had involved their husbands in talking to men about NSV, with success. This would complement the work RESPOND had done in coaching satisfied clients of NSV to share their stories with other men. The RESPOND Project  Report No. 15 Evaluation of NSV Initiative in Uttar Pradesh and Jharkhand States 27 References Census Organization of India. 2011. Census 2011. Accessed April 21, 2014, at http://www.census2011.co.in/. Centers for Operations Research and Training (CORT). 2000. Attitudes toward male and female sterilization in Uttar Pradesh. Vadodara, India. EngenderHealth. 2002. Contraceptive sterilization: Global issues and trends. New York. International Institute for Population Sciences (IIPS) and Macro International. 2007. National Family Health Survey (NFHS-3), 2005/6: India. Mumbai: IIPS. Jacobstein, R, 2013. Lessons from the recent rise in use of female sterilization in Malawi. Studies in Family Planning 44(1):85–95. Khan, M. E., and Patel, B. C. 1997. Male involvement in family planning: A KABP study of Agra District, Uttar Pradesh. New Delhi: Population Council. Ripley, F., and Salem, R. M. 2012. Essential knowledge about female sterilization. Permanent Methods Toolkit. Available at: https://www.k4health.org/sites/default/files/ Essential%20Knowledge_female%20sterilization_Final%20%282%29.pdf. Ross, J. A., and Huber, D. H. 1983. Acceptance and prevalence of vasectomy in developing countries. Studies in Family Planning 14(3):67–73. Scott, B., Alam, D., and Raman, S. 2011. Factors affecting acceptance of vasectomy in Uttar Pradesh: Insights from community-based, participatory qualitative research. The RESPOND Project Study Series: Contributions to Global Knowledge—Report No. 3. New York: EngenderHealth/The RESPOND Project. Shrestha, A., Stoeckel, J., and Tuladhar, J. 1988. Factors related to non-use of contraception among couples with unmet need for family planning in Nepal. Kathmandu, Nepal: New Era. Soni, V. 1983. Thirty years of the Indian family planning program: Past performance, future prospects. International Family Planning Perspectives 9(2):35–45. Trpathy, S. P., Ramachandran, C. R., and Ramachandran, P. 1994. Health consequences of vasectomy in India. Bulletin of the World Health Organization. 72(5):779–782. Trussell, J. 2011. Contraceptive failure in the United States. Contraception 83(5):397–404. The RESPOND Project  Report No. 15 Evaluation of NSV Initiative in Uttar Pradesh and Jharkhand States 29 Appendix 1: RESPOND NSV Initiative Performance Indicators, 2009–2013 UP Jharkhand Increased supply of NSV services No. of facilities supported by the project 44 32 No. of trained NSV surgeons 62 80 No. of NSV surgeons declared performing to standard 46 52 Increased demand for NSV No. of ASHAs/sahiyas oriented on NSV 6,875 2,517 No. of ASHAs/sahiyas coached/provided with on-the-job support 338 344 No. of potential NSV clients referred by coached ASHAs/sahiyas 2,677 7,375 No. of satisfied NSV clients coached 167 102 The RESPOND Project  Report No. 15 Evaluation of NSV Initiative in Uttar Pradesh and Jharkhand States 31 Appendix 2: NSV Procedure Steps and Provider Scores Table 2A: Percentage of NSV providers performing each anesthesia step UP (n=13) Jharkhand (n=9) Total (n=22) 1. Identified, isolated, and fixed vas 100.0 100.0 100.0 2. Raised a skin wheal injecting 1 cc of 2% lidocaine 84.6 88.9 86.4 3. Advanced needle parallel to vas, aspirated, slowly injected 2 cc lidocaine 100.0 88.9 95.5 4. Injected 2 cc of lidocaine into sheath 100.0 88.9 95.5 Total anesthetizing steps 84.6 88.9 86.4 Table 2B: Percentage of NSV providers performing each step of the NSV procedure UP (n=13) Jharkhand (n=9) Total (n=22) 1. Fixes the right vas again with the three-finger technique and applies the ringed forceps to grasp the vas 100.0 88.9 95.5 2. Uses a quick, sharp, single movement to pierce the skin down to the vas lumen with the medial blade of the dissecting forceps, introduced at a 45° angle 100.0 88.9 95.5 3. Withdraws the medial blade of the dissecting forceps; closes the blades of the forceps and inserts both blades of the forceps into the same puncture hole, to the same depth down the vas 100.0 100.0 100.0 4. To deliver and elevate the vas, pierces the wall of the vas with the tip of the lateral blade of the dissecting forceps clockwise 180° so the tips face upward 100.0 100.0 100.0 5. To deliver a loop of the vas, slowly releases the ringed clamp with the left hand, thus allowing the rotated forceps to elevate the vas through the puncture hole 100.0 100.0 100.0 6. Punctures the sheath underlying the loop with one tip of the dissecting forceps blades open on a horizontal plane and bluntly strips the sheath 100.0 77.8 90.9 7. After suturing, cuts one end of the suture about 2–3 mm from the knot, leaving a single uncut end of about 5–7 cm in length 100.0 100.0 100.0 8. Gently pinches and pulls up on the scrotum with the thumb and index finger to allow both ends of the vas to drop back into their original positions in the scrotum 76.9 88.9 81.8 9. Very gently pulls the long suture of the prostatic end of the vas to reexpose the cut end of the vas, which will be covered with fascia 69.2 88.9 77.3 All NSV procedure steps 69.2 66.7 68.2 32 Evaluation of NSV Initiative in Uttar Pradesh and Jharkhand States The RESPOND Project  Report No. 15 Table 2C: Percentage of NSV providers performing each waste disposal step UP (n=13) Jharkhand (n=9) Total (n=22) 1. Decontaminates instruments in 0.5% bleach solution 100.0 55.6 81.8 2. Puts sharps into puncture-proof container 84.6 77.8 81.8 3. Discards blood-soaked swabs in a different container 84.6 55.6 72.7 Total waste disposal steps 84.6 55.6 72.7 The RESPOND Project  Report No. 15 Evaluation of NSV Initiative in Uttar Pradesh and Jharkhand States 33 Appendix 3: Supplies and Equipment Required and Recommended for Quality NSV Services Unique instruments and contraceptive devices NSV ringed clamp (forceps), 4mm* NSV dissecting forceps* Common medical instruments Cup/bowl/gallipot* Sponge-holding forceps, straight, 5.5 in.* Scissors: operating, straight* Other supplies Adhesive tape or Band-aid Chromic catgut or nonabsorbable silk or cotton suture for ligation* Infection prevention supplies Soap or antiseptic agents (for surgical scrub)* Small sterile towel (for hand-drying) Alcohol-based hand rub Sterile gloves* Iodine* Sterile gauze* Sterile surgical drapes Sterile gowns for surgeon and surgeon’s assistant Cap and face mask Drapes to cover surgical cushion table Drapes for packing instruments Decontamination solution* Infection prevention infrastructure Clean running water* Leak-proof containers for decontaminating soiled instruments* Autoclave* If no autoclave: Boiler* Puncture-proof containers for disposal of sharps* Pain management supplies Local anesthetic—lidocaine without epinephrine, 1% or 2%* Distilled water to dilute lidocaine, if 2%* 10 ml syringe with 1.5-inch, 27-gauge needle* Emergency management supplies Antihistaminic* Steroids* Syringe* Oxygen with mask* Suction apparatus* * Elements required for quality delivery of NSV services