PERFORMANCE EVALUATION OF USAID’S ACCELERATING UNIVERSAL ACCESS TO FAMILY PLANNING (SHUKHI JIBON) ACTIVITY IN BANGLADESH APRIL 2023 Performance Evaluation of USAID’s Accelerating Universal Access to Family Planning (Shukhi Jibon) in Bangladesh | 1 PERFORMANCE EVALUATION OF USAID’S ACCELERATING UNIVERSAL ACCESS TO FAMILY PLANNING (SHUKHI JIBON) ACTIVITY IN BANGLADESH Program Title: Accelerating Universal Access to Family Planning (Shukhi Jibon) in Bangladesh Sponsoring USAID Office: USAID Bangladesh Prepared by: Ahmed Al-Sabir, Nahid Kamal, Anadil Alam and Shumona Shafinaz APRIL 2023 Cooperative Agreement No. 72038818CA00004 Cover photo: Rabiul Hasan, icddr,b DISCLAIMER: This publication was produced with the support of the United States Agency for International Development (USAID) under the terms of USAID’s Research for Decision Makers (RDM) Activity cooperative agreement no. AID-388-A-17-00006. Views expressed herein do not necessarily reflect the views of the U.S. Government or USAID. icddr,b is also grateful to the Governments of Bangladesh, Canada, Sweden and the UK for providing unrestricted/institutional support. Performance Evaluation of USAID’s Accelerating Universal Access to Family Planning (Shukhi Jibon) in Bangladesh | 2 TABLE OF CONTENTS ACRONYMS ...................................................................................................................................4 LIST OF TABLES .............................................................................................................................6 LIST OF FIGURES ...........................................................................................................................6 EXECUTIVE SUMMARY ....................................................................................................................7 1. EVALUATION PURPOSE AND EVALUATION QUESTIONS ................................................................15 1.1 Evaluation purpose ......................................................................................................................................... 15 1.2 Evaluation questions....................................................................................................................................... 15 2. BACKGROUND..........................................................................................................................16 2.1 National context............................................................................................................................................. 16 2.1.1 Social, cultural, and structural challenges to progress...................................................................... 16 2.1.2 Specific issues.................................................................................................................................... 17 2.2 Activity overview............................................................................................................................................ 17 3. EVALUATION METHODS AND LIMITATIONS..................................................................................19 3.1 Evaluation methods ........................................................................................................................................ 19 3.2 Limitations...................................................................................................................................................... 20 4. INTERVENTION DESCRIPTION AND FINDINGS .............................................................................21 4.1 Evaluation Area 1: Strengthening Capacity of Training Institutes.................................................................. 21 4.1.1. Intervention packages ...................................................................................................................... 21 4.1.2 Intervention layers............................................................................................................................ 22 4.1.3 Steps of implementation ................................................................................................................... 23 4.1.4 Conclusion of Evaluation Area 1 ...................................................................................................... 33 4.1.5 Recommendations on Evaluation Area 1.......................................................................................... 34 4.2 Intervention Area 2: Mentorship and supportive supervision (M&SS) .......................................................... 34 4.2.1 Intervention Description ............................................................................................................................. 34 4.2.2 M&SS findings.................................................................................................................................... 36 4.2.3 Conclusion of Evaluation area 2 ....................................................................................................... 44 4.2.4 Recommendations on Evaluation Area 2.......................................................................................... 44 4.3 Evaluation Area 3: Interventions for Adolescents.......................................................................................... 45 4.3.1 Intervention description and findings................................................................................................ 45 4.3.2 Interventions at health facilities relating to Sub-IR 2.1 ..................................................................... 46 4.3.3 Findings on the interventions at health facilities............................................................................... 47 4.3.4 Community Interventions for Adolescents and Youth relating to Sub-IR 2.4 ................................. 49 4.3.5 Findings on community level interventions....................................................................................... 52 4.3.6 Interventions at the school level and using digital platforms............................................................ 53 Performance Evaluation of USAID’s Accelerating Universal Access to Family Planning (Shukhi Jibon) in Bangladesh | 3 4.3.7 Findings on Shukhi Jibon school-level interventions......................................................................... 54 4.3.8 Conclusions of Evaluation Area 3 ..................................................................................................... 54 4.3.9 Recommendations on Evaluation Area 3.......................................................................................... 55 5.CONCLUSION AND RECOMMENDATIONS......................................................................................56 ANNEXES ....................................................................................................................................60 Annex I: Result Framework Shukhi Jibon Intervention Districts......................................................................... 60 Annex II: Result Framework ................................................................................................................................ 61 Annex III: Guidelines and tools for evaluation ..................................................................................................... 62 Annex IV: List of documents reviewed ................................................................................................................ 84 Annex V: List of meetings and stakeholders........................................................................................................ 86 Annex VI: List of KIIs and GIs .............................................................................................................................. 89 Annex VII: Locations and catchment areas of 16 NIPORT institutes.................................................................. 91 Annex VIII: List of Trainers Trained on CBT and Adult Learning Methodology ................................................. 92 Annex IX: List of Participants of TMIS Training................................................................................................. 103 Annex X: Annual number of counsellings provided to adolescents .................................................................. 108 Annex XI: Timeline ............................................................................................................................................ 109 Annex XII: Evaluation statement of work .......................................................................................................... 110 Annex XIII: Disclosure of any conflicts of interest ............................................................................................ 130 Annex XIV: Evaluation Team Members ............................................................................................................. 137 Performance Evaluation of USAID’s Accelerating Universal Access to Family Planning (Shukhi Jibon) in Bangladesh | 4 ACRONYMS A2H a2i AFHS AHI AUAFP AYFHS ASRHR BHFS CBT CC CCSDP CHCP CHT CME CPR DDFP DGFP DGHS DGNM DP DSHE ESDO FPAB FP FPCS-QIT FPI FPRH FTP FWA FWTI FWV GOB GI HA HI HMIS HNPSIP HPNSP HRIS HSS HQ icddr,b IP IR KII LARC LoC Advancing Adolescent Health Aspire to Innovate Adolescent Friendly Health Services Assistant Health Inspector Accelerating Universal Access to Family Planning Adolescent and Youth Friendly Health Services Adolescent Sexual Reproductive Health Rights Bangladesh Health Facility Survey Competency-based training Community Clinic Clinical Contraception Service Delivery Program Community Health Care Provider Chittagong Hill Tracts Continuing Medical Education Contraceptive Prevalence Rate Deputy Director Family Planning Directorate General of Family Planning Directorate General of Health Services Directorate General of Nursing and Midwifery Development Partner Directorate of Secondary and Higher Education Environment and Social Development Organization Family Planning Association of Bangladesh Family Planning FP Clinical Supervision-Quality Improvement Team Family Planning Inspectors Family Planning and Reproductive Health First Time Parents Family Welfare Assistant Family Welfare Training Institute Family Welfare Visitor Government of Bangladesh Group Interview Health Assistant Health Inspector Health Management Information System Health, Nutrition and Population Strategic Investment Plan Health, Population and Nutrition Sector program Human Resources Information Systems Health System Strengthening Head Quarter International Centre for Diarrheal Diseases and Research Implementing Partner Intermediate Result Key Informant Interview Long-Acting Reversible Contraceptives Letter of Collaboration Performance Evaluation of USAID’s Accelerating Universal Access to Family Planning (Shukhi Jibon) in Bangladesh | 5 M&SS MCHTI mCPR MCWC MIS MFSTC MNCAH MOMCH&FP MoE MoHFW MoWCA MoU MR NAHS NGO NIPORT OCP OGSB OP PAC PHD PPFP QIS RCA RCTC RPTI RTC SACMO SMART SRH SSN SWOT TA TCBO TFR TMIF TMIS ToT TNA UH&FWC UHC UNICEF USAID WG YPSA Mentorship and Supportive Supervision Maternal and Child Health Training Institute Modern Contraceptive Prevalence Rate Mother and Child Welfare Center Management Information System Mohammadpur Fertility Services and Training Center Maternal, Newborn, Child and Adolescent Health Medical Officer Maternal and Child Health and Family Planning Ministry of Education Ministry of Health and Family Welfare Ministry of Women and Children Affairs Memorandum of Understanding Menstrual Regulation National Adolescent Health Strategy Non-governmental Organizations National Institute of Population Research and Training Oral Contraceptive Pill Obstetrics Gynecological Society of Bangladesh Operation Plan Post-Abortion Care Partners in Health Development Postpartum Family Planning Quality Improvement Secretariat Root Cause Analysis Regional Clinical Training Centers Regional Population Training Institute Regional Clinical Training Centers Sub-Assistant Community Medical Officer Specific, Measurable, Achievable, Realistic and Time-bound Sexual and Reproductive Health Senior Staff Nurse Strength, Weakness, Opportunity, Threat Technical Assistance Training Capacity Building Officer Total Fertility Rate Training Management Improvement Frameworks Training Management Information System Training of Trainers Training Needs Assessment Union Health & Family Welfare Center Upazila Health Complex United Nations Children’s Fund United States Agency for International Development Working Group Young Power in Social Action Performance Evaluation of USAID’s Accelerating Universal Access to Family Planning (Shukhi Jibon) in Bangladesh | 6 LIST OF TABLES Table 1: Selected Sub-Intermediate Results for Evaluation .............................................................................. 19 Table 2: Trained mentors who are currently engaged ....................................................................................... 38 Table 3: Reasons for drop-out of Mentors .......................................................................................................... 38 Table 4: Distribution and engagement of mentees by designation................................................................... 39 Table 5: Reasons for drop-out of Mentees .......................................................................................................... 39 Table 6: Distribution of Supportive Supervision visits by different level supervisors ....................................... 42 Table 7: Facility readiness improved through supervisory visits ....................................................................... 43 Table 8: Primary Activities of the 6 NGO sub-grantees....................................................................................... 51 LIST OF FIGURES Figure 1: Four implementation layers of Shukhi Jibon ...................................................................................... 23 Figure 2: Steps taken to introduce and integrate CBT in MOHFW activities...................................................... 23 Figure 3: Location of Shukhi Jibon selected training institutes......................................................................... 25 Figure 4: Signboard of a Regional FPCS-QIT Office ............................................................................................ 27 Figure 5: Status of divisional workshops and developing action plans............................................................. 27 Figure 6: The Competency-based training manual for the trainers................................................................... 28 Figure 7: Training of master trainers and locations ........................................................................................... 29 Figure 8: Timeline and participants of training of trainers................................................................................. 30 Figure 9: Locations of training of trainers ........................................................................................................... 30 Figure 10: Objectives and focus of Mentorship and Supportive Supervision .................................................... 35 Figure 11: Steps of Mentorship and Supportive Supervision Intervention........................................................ 36 Figure 12: Mentee's Progress in FP Service Delivery by Division........................................................................ 40 Figure 13: Ministries mentioned in the NAHS and Action Plan .......................................................................... 46 Figure 14: Checklist to assess AYFS facility readiness based on WHO and DGFP guidelines: ........................... 47 Figure 15: Rate of increase in counseling provided to adolescents between 2020 and 2022 ........................... 48 Figure 16: Number of adolescent treatment and counselling services in NGO and non-NGO areas................ 50 Figure 17: Breakdown of hard to reach groups reached by NGO sub-grantees ................................................ 52 Performance Evaluation of USAID’s Accelerating Universal Access to Family Planning (Shukhi Jibon) in Bangladesh | 7 EXECUTIVE SUMMARY Evaluation purpose and evaluation questions The purpose of this performance evaluation is to assess three core components of the USAID’s Accelerating Universal Access to Family Planning (AUAFP) - Shukhi Jibon activity namely, the progress made in enhancing capacity of the family planning (FP) training institutes and of FP service providers, and to identify the promising interventions for reaching adolescents and youths with reproductive health/family planning (FP) information and services. The assessment was conducted by a team of independent consultants over December 2022 and March 2023 involving extensive field visits in Dhaka, Mymensingh, Faridpur and Sylhet. The audience for this report is the USAID mission in Bangladesh and the Implementing Partner (IP) consortium to make necessary adjustments to the ongoing program. It is expected that findings from this evaluation will be used to inform the design of future USAID/Bangladesh FP and adolescent reproductive health programs and national policies. Results obtained may also be used to advocate for changes in government policies and to promote further adoption and implementation of Family Planning and Reproductive Health (FPRH) initiatives. It will be made publicly available to researchers, academics and potential bidders. BACKGROUND The plateauing in the progress in FP indicators in the country is largely explained by supply-side limitations, in particular, a lack of qualified FP service providers. This is attributable to both staff shortages and a lack of formal continuing education for the FP providers. Most training are classroom based using didactic presentations that include limited practicum. Nationally, capacity building is seen as a one-time training - continued follow-up, supervisions, monitoring and mentoring are not systematically organized in order to develop a strong FP workforce. Also, there is inadequate coordination among National Institute of Population Research and Training (NIPORT), the Directorate General of Family Planning (DGFP) and the Directorate General of Health Services (DGHS) that are mandated for FP training. Certain populations, including adolescents and youth, face barriers in accessing FP services although the Ministry of Health and Family Welfare (MOHFW) identified adolescents as a priority age group. The National Strategy for Adolescent Health 2017-2030 aims to ensure that, "By 2030 all adolescents will lead a healthy and productive life in a socially secure and supportive environment where they will have easy access to quality and comprehensive information, education, and services." The Bangladesh Health Facility Survey, 2017 (BHFS 2017) reports that only 22% of health facilities meet the readiness criteria for FP service provision for the general population, let alone for sub-groups such as adolescents. Against this backdrop, in 2018, USAID initiated its Shukhi Jibon activity in order to accelerate family planning utilization in four low-performing divisions in Bangladesh. The activity was intended to contribute to the health and wellbeing of Bangladeshis and to accelerate FP utilization by of the MOHFW and its relevant directorates. Pathfinder International, in partnership with lntraHealth International and with strategic partnership from the Obstetrics Gynecological Society of Bangladesh (OGSB) and the University of Dhaka is responsible for implementing Shukhi Jibon over five years, from July 2018 to July 2023. Recently, USAID extended the activity with no cost till January 2024.The total activity funding over the life of five-year period is USD 40,652,622 (including USD 3,695,622 in cost share). Performance Evaluation of USAID’s Accelerating Universal Access to Family Planning (Shukhi Jibon) in Bangladesh | 8 Shukhi Jibon is currently working in 286 health facilities in 32 districts within four focus divisions, including Chattogram, Dhaka, Mymensingh and Sylhet, as well as with 24 training institutes. The target beneficiary groups are the 24 FP training institutes, the trainers and the health workforce within health facilities and the populations with access barriers to FP, including adolescents and youth, first-time parents, newlyweds, and postpartum, post abortion and post menstrual regulation clients. Moreover, Shukhi Jibon’s adolescent interventions are supported by 6 non-governmental organizations (NGOs) namely, Environment and Social Development Organization (ESDO), Family Planning Association of Bangladesh (FPAB), Lighthouse, Partners in Health Development (PHD), SERAC and Young Power in Social Action (YPSA). These NGOs are mainly responsible for demand generation for adolescent health service in selected hard-to-reach and underserved areas of 15 activity districts. The two intermediate results (IR) of Shukhi Jibon are ‘increased qualified FP workforce’ and ‘increased availability of public sector FP outreach contacts and services, particularly for adolescents and youth’. IR1 aims to build the capacity of NIPORT, MOHFW, and the national systems and mechanisms underpinning provider training, supervision and mentorship to ensure that providers are both adequately skilled and more responsive to the needs of the most underserved populations. The IR2 aims to strengthen the health and FP sector’s ability to provide quality information and services to adolescents and youth and connect them to youth friendly facilities. IR2 also includes awareness raising and behavior change of adolescents and their gatekeepers through using digital platforms as well as community mobilization activities. Evaluation questions, design, methods, and limitations The evaluation team identified four evaluation questions covering three broad program areas in consultation with USAID. The evaluation areas are: ▪ Evaluation Area 1: which investments in the FP Training Institutes are increasing the capacity of the FP training institutes for improving FP providers skills ▪ Evaluation Area 2: which, if any, interventions on mentorship and supportive supervision increase service providers’ skills in providing selected FP services ▪ Evaluation Area 3: which, if any, interventions to reach adolescent and youth populations for reproductive health/FP information and services are promising This evaluation is largely a qualitative assessment of selected components of the Shukhi Jibon activity. As such the findings are derived from triangulation of key informant and group interviews, field observations and document/data review. Questionnaires were fine-tuned in close consultation with the key stakeholders. The assessment was limited to mature Shukhi Jibon intervention areas, that is, the locations where the implementations started during the early years of the activity. Although ideal, collection of field data from non￾intervention districts was beyond the scope of this assessment within the given timeframe. Moreover, the assessment was largely limited to qualitative data collection and analysis. The team nevertheless tried to mitigate these limitations by increasing the number of key informant and group interviews and by conducting an expansive desk review along with health management information system (HMIS) data analysis to the extent possible. Performance Evaluation of USAID’s Accelerating Universal Access to Family Planning (Shukhi Jibon) in Bangladesh | 9 FINDINGS, CONCLUSIONS, AND RECOMMENDATIONS 1. Shukhi Jibon interventions on FP training institutes The investments of Shukhi Jibon in the FP training institutes under MOHFW, particularly the DGFP, DGHS and NIPORT were to strengthen the capacity of the training institutes so that they can impart high-quality training to FP service providers and effectively coordinate the process. Under the investment, Shukhi Jibon introduced competency-based training (CBT) in the FP training institutes of MOHFW for developing knowledge (what), skills (how) and attitude (why) of the trainees on a specific service or topic through learning by doing and follow-up. Unlike the traditional training approaches, CBT is for enhancing trainers’ skills to facilitate discussions based on adult learning principles and develop competencies based on practical examples and exposures. Shukhi Jibon also invested in digitizing NIPORT’s Training Management Information system (TMIS) to promote the use of data in training related decision making and creating a platform for FP-related e-learning and certification. In addition, Shukhi Jibon introduced Training Management Improvement Frameworks (TMIFs) in training institutes for improving the quality of training by identifying bottlenecks of the system and adopting context driven solutions to overcome them. In 2019, the Shukhi Jibon successfully made formal agreements with NIPORT, DGFP and DGHS separately for formulating a shared vision and common understanding for implementing the interventions. This helped in gaining access to and acceptance among NIPORT, DGFP and DGHS policy makers and contributed to facilitating the activities. However, a single agreement involving all parties could have been signed for better collaboration, and coordination among all parties. The Shukhi Jibon identified 24 public training institutes from NIPORT (16) and DGFP (8). NIPORT institutes are NIPORT HQ, Regional Population Training Institute (RPTI), and Regional Training Centre (RTC). DGFP institutes are Mohammadpur Fertility Services and Training Center (MFSTC), Maternal and Child Health Training Institute (MCHTI), Family Welfare Training Institute (FWTI) and Regional Clinical Training Centers (RCTCs). However, there were concerns regarding the selection of Clinical Contraception Service Delivery Program (CCSDP) led Family Planning Clinical Supervision-Quality Improvement Team (FPCS-QIT) offices as ‘Regional Clinical Training Centers (RCTCs)’. Greater care in selection of the training institutes of DGFP could have avoided confusion. A rigorous Training Needs Assessment (TNA) was conducted by Shukhi Jibon in 6 districts among 10 institutional heads, 25 trainers and 493 FP service providers. Subsequently, the divisional workshops were organized at Chattogram, Sylhet, Mymensingh and Dhaka involving policy makers, managers, and service providers from NIPORT, DGFP and DGHS to identify gaps, and developed action plans through in-depth discussions. However, there were gaps in coordination for implementing the action plan which resulted into the unavailability of some essential training equipment, such as mannequin. The Shukhi Jibon developed a training package on CBT. The training package included a comprehensive manual for trainers containing 184 pages with several graphical illustrations. It followed globally accepted standards. However, trainers strongly felt the need for a Bangla version of the manual and soft copies of the manual in Bangla/English for their practical use in conducting training sessions. The Bangla version of the manual has recently been approved, printed and is being currently distributed. The Shukhi Jibon organized a workshop in February 2019, for a total of 20 master trainers from NIPORT, DGFP and DGHS, representing 6 districts, to train on developing curricula and training materials, facilitating sessions based on CBT principles. The workshop was well-accepted by most of the master trainers who strongly agreed on the importance, quality and acceptability of the training sessions, trainer’s manual, and training standards, as well as follow-up, and mentoring of trainers. The master trainers trained 44 trainers of NIPORT, 268 of DGFP, 60 of DGHS and 2 of Directorate General of Performance Evaluation of USAID’s Accelerating Universal Access to Family Planning (Shukhi Jibon) in Bangladesh | 10 Nursing and Midwifery (DGNM) from April 2019 to October 2022. Most of the trainers have reported that they are using the CBT principles in imparting FP specific training. The majority of the participants were from outside of the training institutes which raises concerns regarding the sustainability of the training institutes. On the other hand, the wide availability of trainers from outside the institutes could help in rapid roll-out of training when needed. The Training Capacity Building Officers (TCBOs) of Shukhi Jibon provided technical assistance in conducting periodic visits to the trainers (trained in CBT) and in preparing their training materials based on the CBT principles. The TCBOs also participated in the training sessions conducted by the trainers and assessed the integration and use of CBT principles in those sessions. Although there was early resistance by the trainers regarding TCBOs, they eventually appreciated the technical assistance through follow-up support. That said, there are concerns regarding the sustainability of TCBOs as most of the trainers felt the need for continuing this support. The Shukhi Jibon activity took over the TMIS from USAID’s MaMoni Health System Strengthening (HSS) activity to update and operationalize for NIPORT but the technology transfer process took more than 14 months due to administrative and technical challenges. Shukhi Jibon updated the TMIS based on the need of NIPORT, and the needs were identified through consultative workshops involving the training institutes. TMIS is now operational in NIPORT-HQ and its sub-national level centers. But steps need to be taken to provide access of TMIS to DGFP training institutes. As TMIS and DGFP-HRIS are not interoperable, DGFP and NIPORT will not be able to assess the true training needs and develop a comprehensive training plan. Vacant positions and frequent transfer of staff at RTC and RPTI have created obstacles in the effective utilization and functionality of digitalized TMIS. The Shukhi Jibon started TMIF interventions in year 3 of the activity through conducting Strength, Weakness, Opportunity, Threat (SWOT) analysis workshops with officials of NIPORT and DGFP officials. A TMIF report was developed for each of the participating institutes (16 NIPORT institutes and 8 DGFP RCTCs) during the workshops. All 16 training institutes of NIPORT implemented at least 3 recommendations such as, CBT integration in training programs, use of training manual as reference manual and operationalization of TMIS. However, Shukhi Jibon involved only 1-2 participants from each training institute in the SWOT analysis exercise which was not sufficient to understand the bottlenecks and root causes of ineffective integration and implementation of training activities in each institute. There are major gaps in implementation of recommended activities, also the action points were not clearly defined. Staff were not clear about their roles due to lack of follow-up or monitoring plan. Perhaps Shukhi Jibon needs time for implementation of the TMIF recommendations. Some of the key recommendations include having a tri-party agreement involving all partners for greater sensitization and ownership regarding the priority interventions and gaps in coordination among DGFP, NIPORT and DGHS initiatives; investing on approved and functional training institutes; ensuring critical training equipment, such as mannequin; active facilitation with joint involvement of policy makers and senior officials of DGFP and NIPORT in developing action plans and finally linking the TMIS with the Management Information System (MIS) of DGFP. 2. Integrated Mentorship and Supportive Supervision (M&SS) Interventions Integrated M&SS is central to achieving Shukhi Jibon’s overarching activity objectives to increase the use of FP services by growing and strengthening the qualified FP workforce. Objectives of the approach were to introduce mentorship and supportive supervision in the public sector, increase quality of FP service provision and integrate gender transformative supportive supervision approach. While mentorship focused on individual provider’s skill development, supportive supervision focused on the entire health facility and all health workers and resolved Performance Evaluation of USAID’s Accelerating Universal Access to Family Planning (Shukhi Jibon) in Bangladesh | 11 health systems issues. While mentorship and supportive supervision are distinct, they do have some overlapping functions. Shukhi Jibon developed this intervention based on Pathfinder’s experience of implementing M&SS in different countries as well as in NGO service delivery program in Bangladesh. The intervention was designed in 2019 through consultative workshops with the government and other stakeholders and under guidance of the M&SS Working Group (WG) at the MOHFW. The preparatory activities included consultation workshops, M&SS working group formation, development of trainer’s (master mentor’s) and trainee’s (mentor’s) manuals and a set of job aids, and training for master trainers and mentors. At the field level, mentees were selected and paired with the mentors, mentee profile and action plan were to be developed through SWOT analysis and periodic coaching sessions were organized to promote and track mentee’s skill development through structured checklists. In addition, refreshers trainings, mentor’s periodic review meetings and district FP progress review meetings were held to review progress of the intervention. The mentorship intervention was scaled up in 16 districts while supportive supervision is operational in all 32 districts. The evaluation team visited the districts where both Mentorship and Supportive Supervision are operational. The M&SS training manuals are unique resources, but they were found to be too long and complex both in terms of language and content. There was no training manual for the mentees, as a result they were not clear about the graduation process, follow-up mechanisms etc. and they felt the need for a simple, user-friendly operational guideline in Bangla. A lack of ownership was found among DGFP officials as they were not involved with the manual development process and a few of them only worked as reviewers. The trainings on M&SS were highly appreciated by the respondents as the trainers followed a competency-based training approach and made the trainings interesting. Till date, a total of 617 managers and supervisors in 32 districts were trained in M&SS basic training. While mentors were selected based on positions, mentees were selected based on their performance, i.e. “good performers” were mostly selected instead of poor performers who in fact need more mentoring. The proportion of providers who were enrolled as mentees against the total number of staff in position was rather low. The highest proportion was Medical Officer-Maternal, Child Health & Family Planning (MOMCH-FP) (27%) and lowest was first line supervisors of DGHS such as Health Assistant (HA)/Health Inspector (HI)/Assistant Health Inspector (AHI) (0.13%). Several job aids were developed to support the intervention, however, only the Action Plan handbook and observation checklists were used. Most of the job aids were not used resulting in low compliance to the mentorship process described in the training manual. The checklists were highly appreciated by most of the respondents though use of supervisory checklists was limited. On average, 2-5 mentorship/coaching sessions per mentee per year were held, although the initial plan was to hold at least one session per mentee per month. The major challenges were scheduling sessions, due to busy schedules with competing priorities. High drop-out, long travel time for external mentors were also found challenging. The intervention suffered from high attrition of both mentors and mentees. Only 14% of the trained providers are currently active as a mentor and over half of the mentors dropped out after enrollment as a mentor. Similarly, 54% of mentees are currently engaged and the percentage of drop-out widely varied among different positions. The top three reasons for mentor drop-out were unwillingness (40%), transfer (30%) and retirement (12%) and the reasons for mentee drop-out were reluctance (77%), transfer (12%) and retirement (5%). Mentees were assessed on specific skills at the beginning as well as at each mentorship session. Shukhi Jibon data showed that the baseline and latest scores in key FP service delivery areas have improved significantly from Performance Evaluation of USAID’s Accelerating Universal Access to Family Planning (Shukhi Jibon) in Bangladesh | 12 53.5% to 90%. The evaluation team also examined the scores of individual mentees during field visits, and independently scored mentees during service delivery. Both were found consistent with the Shukhi Jibon database. Till date, only 8.7% mentees have graduated. More mentees are in the graduation process and a few will graduate soon. Mentors and mentees had limited ideas of the graduation criteria and process. Eight of them graduated from mentees to mentors. The reasons for low graduation are strict criteria of “consistently attaining =>80% score for 3 consecutive sessions for each service” which means each mentee has to go through enough sessions to be assessed for each service area at least 3 times if not more. But the number of sessions held was not enough to reach atthat point. Another reason was lack of awareness and focus by both mentors and mentees about the graduation criteria and requirements. Use of Mentorship mobile app is low, only one mentor-mentee pair was successfully using the app. The remaining respondents received training and have the app installed on their phone but are unable to use it. However, Shukhi Jibon database shows that about 20% of sessions were conducted through mentorship mobile applications. Despite a government order for use of supportive supervision checklists by the, the number of visits were limited (only 11 visits per district per year). The majority of the visits (4 visits per person per year) were conducted by the Deputy Director Family Plannings (DDFPs)- while the FPCS-QITs made only 0.4 visits per person per year. Facilities that had more visits showed improved facility readiness. M&SS is a unique approach for improving providers skills and thus overall improvement in quality of care. The mentees become more confident and develop better rapport with the mentor leading to improvement in support seeking and referral. Last but not the least, both mentorship sessions and supportive supervisory visits were largely affected by the pandemic due to movement restrictions and lack of in-person sessions/visits. Sustainability was cited as a major concern by all as the process is resource intensive requiring close support by Shukhi Jibon. It would be difficult to sustain these efforts without strong government commitment or linkage with other donor’s programs. Currently, Quality Improvement Secretariat (QIS) of MOHFW initiated development of National Mentorship Strategy and Action Plan with technical support from the United Nations Children’s Fund (UNICEF). The key recommendations include, identifying operational challenges through consultation workshops, operations research, document and disseminate strong evidence (both quantitative and qualitative), collaborate with development partners and link with national strategy on mentorship being developed QIS. Advocacy with DGFP and DGHS required to integrate this with GOB’s existing supervision system and include the integrated M&SS in respective operation plans (OPs) with adequate resource allocation. Moreover, the app needs to be aligned with the DGFP and DGHS information systems. 3. Interventions for Adolescents and Youth The Shukhi Jibon activity has implemented various interventions for adolescents using four platforms: facility, community, school and digital. Of these, ensuring health facility readiness was the predominant one with the activity having upgraded 700 health centers out of the 800 identified. The interventions for ensuring facility readiness included enhancing the capacity of frontline health workers such as Family Welfare Visitors (FWV) and Sub-Assistant Community Medical Officers (SACMO) to provide adolescent and youth friendly services, promoting ‘whole-site approach’, providing training curricula, job aids and register, and promoting long acting and reversible contraceptives (LARC) for young parents. The whole-site approach whereby supports and community staff of the health centers were sensitized on Adolescent Sexual Reproductive Health Rights (ASRHR) Performance Evaluation of USAID’s Accelerating Universal Access to Family Planning (Shukhi Jibon) in Bangladesh | 13 is to be adopted in the next Operational Plans of DGFP and DGHS. One of the six partner NGOs SERAC supported the adolescent health corners in its catchment area with a youth peer volunteer. This is another intervention that is to be expanded in the next health sector program. Despite these investments on health facilities to make them adolescent friendly, the use rate is low. Staff shortages and the disruption created by COVID-19 were implementational challenges no doubt. One of the findings from the assessment of the earlier iteration of USAID funded adolescent program, Advancing Adolescent Health (A2H, 2016-2019), was that investing in making existing health facilities adolescent friendly is not necessarily associated with increased use of services. Behavioral change takes time and effort – the use rate of health facilities among adolescents continues to be low largely due to socio-cultural reasons. In addition to aligning its activities to government priorities, the Shukhi Jibon could have adopted innovative ways of supply and demand creation like other development partners do, for example, engaging adolescents through sports or activities in schools. Shukhi Jibon’s community level activities include providing orientation session to DGFP and DGHS frontline works with 3,140 field staff having been oriented so far; bolstering community mobilization and engagement through courtyard sessions (Uthan Boithok); developing guidelines for the courtyard sessions; connecting gatekeepers and community leaders; promoting the DGFP’s pilot ‘newlywed’ intervention to delay the first pregnancy. For the outreach and community level interventions, the activity was supported by 6 local NGO partners in 15 out of the total 32 intervention districts. These 6 NGOs have supported the activity with innovative interventions to reach close to 46,000 adolescent and youth members of underserved communities such as transgender, adolescents living in Chattogram Hill Tracts, tea-gardens or climate-vulnerable areas. The courtyard guidelines developed by Shukhi Jibon recommended holding separate sessions with boys, girls, married and unmarried adolescents when the relevant topics could be discussed. In reality school going adolescents hardly attend these sessions. While the courtyard sessions are helpful for mobilizing gatekeepers of adolescents, community leaders and married adolescents, they are not the most effective means for reaching unmarried adolescents, as witnessed during field visits and verified by community workers in KIIs and GIs conducted by the evaluation team. The Shukhi Jibon has had relatively few activities at the school level or using digital platforms although the National Strategy on Adolescents and the Action Plan have outlined 11 school, 19 facility and 20 community level activities. Perhaps the missed opportunity was not having more school level interventions when the secondary school enrolment rate in the country is over 75% (World Bank, 2021). During the initial design and planning phase of Shukhi Jibon, the activity would have benefited from including more school-based initiatives rather than relying solely on community interventions for demand generation. The activity supported DGFP by strengthening capacity of the service providers assigned for school health support- SACMO; to date the activity has provided close to 1,300 training sessions on AFHS to SACMOs. However, the activity started monitoring the school sessions held by the SACMO’s only this year, as a result there is no documentation or information on SACMOs conducting school sessions. The SACMOs are also mostly male, and as such, involving them in school health programs in girls’ schools may not achieve maximum effectiveness. Finally, it can be said with confidence that adolescents’ health knowledge has increased. This is largely due to greater access to digital sources. Their care-seeking continues to be low – according to the Bangladesh Adolescent Health and Well-Being Survey 2019-20, less than ten percent of female adolescents sought reproductive health services with the vast majority going to a private provider. That said, analysis of HMIS data reveals that compared to levels recorded in 2020, more adolescent girls sought counselling services in 2022 in all divisions of the country. However, the rate of increase over that period was relatively higher in non-intervention Performance Evaluation of USAID’s Accelerating Universal Access to Family Planning (Shukhi Jibon) in Bangladesh | 14 areas. A nuanced understanding of the barriers to accessing healthcare is needed for effectively reaching adolescents in the low-performing areas where Shukhi Jibon operates. Some of the key recommendations for adolescent-specific interventions include having separate and targeted interventions for specific adolescent groups - unmarried and married, girls and boys, and the hard to reach, using a combination of platforms.; placing greater emphasis on school-based interventions such as a full-time counselor or peer educator as there is ample evidence from India and Africa that peer-led programs in schools and health facilities improve knowledge on SHR. Moving forward, a stand-alone adolescent SRH program as opposed to one integrated in a bigger activity, coupled with a multi-sectoral approach involving other Ministries such as Ministry of Education (MoE) and Ministry of Women and Children Affairs (MOWCA) may be more effective. Performance Evaluation of USAID’s Accelerating Universal Access to Family Planning (Shukhi Jibon) in Bangladesh | 15 1. EVALUATION PURPOSE AND EVALUATION QUESTIONS 1.1 Evaluation purpose The main purpose of this evaluation is to learn about what worked, what did not and why. More specifically, the scope of this evaluation involves determining if the interventions on increasing the capacity of the FP training institutes worked, whether the mentorship and supportive supervision intervention could improve FP provider’s skills, and interventions to reach the adolescent and youths were useful and promising. The audience for this report is the USAID mission in Bangladesh and the Implementing Partner (IP) consortium to make necessary adjustments to the ongoing program. It is expected that findings from this evaluation will be used to inform future USAID/Bangladesh FP and adolescent reproductive health program designs, and national policies and programming on adolescents. Results obtained may also be used to advocate for changes in the government training and capacity building policies and to promote further adoption and implementation of Family Planning and Reproductive Health (FPRH) initiatives that originate from the Shukhi Jibon investment. Additionally, it will be made publicly available to researchers, academics and potential bidders. 1.2 Evaluation questions Given time and data constraints, the evaluation team identified following evaluation questions covering three broad program areas in consultation with USAID. These areas included strengthening the capacity of training institutions, use of M&SS for strengthening the skills of FP service providers, and finally, effectiveness of the interventions geared towards adolescents and youth. The specific evaluation questions are: ▪ Which investments in the FP Training Institutes are increasing the capacity of the FP training institutes for improving FP provider’s skills ▪ Which, if any, interventions on mentorship and supportive supervision increase service providers’ skills in providing selected FP services ▪ Which, if any, interventions to reach adolescent and youth populations for reproductive health/FP information and services are promising Performance Evaluation of USAID’s Accelerating Universal Access to Family Planning (Shukhi Jibon) in Bangladesh | 16 2. BACKGROUND 2.1 National context Bangladesh has made remarkable progress in improving citizens' health over the past 25 years, including reducing infant mortality and maternal mortality. The country has also seen reductions in the total fertility rate (TFR) from 4.3 in 1991 to 2.3 in 2017 children per women and increases in the modern contraceptive prevalence rate (mCPR) from 36.6% to 52% during this period1 . These gains are largely due to the commitment of the Government of Bangladesh (GOB) in advancing universal health coverage as outlined in the 2011 National Health Policy and the 2016 - 2021 Health, Nutrition, and Population Strategic Investment Plan (HNPSIP). Going forward, the GOB has made ambitious commitments as part of FP2020, including a further reduction in TFR to 2.0, increasing CPR from 62% to 75%, increasing the share of long-acting and permanent methods (LAPM) from 8.1 % to 20%, and reducing unmet need and method discontinuation. Despite this remarkable progress, the country faces key challenges in achieving further progress, including inequities across geographic areas and among age cohorts. For example, while the mCPR among married women is 52% at the national level, it is much lower in Chattogram and Sylhet, two of the four focused divisions of Shukhi Jibon activity. These inequities also exist between age cohorts - modern method use rates among married women aged 15 to 19, and 20 to 24 are only 43.7% and 50.9%, respectively, versus 62.7% among women between 30 and 342 . Furthermore, nearly half of the users rely on short-acting methods, primarily oral contraceptive pills (OCPs). 2.1.1 Social, cultural, and structural challenges to progress In order to overcome the current plateau in mCPR, it is crucial to address the social, cultural, and structural factors that underpin these inequities. Early marriage, socio-cultural pressure to prove fertility, and subsequent early childbearing result in almost half of women giving birth by age 18. FP decisions are driven primarily by male partners and gatekeepers (e.g., mothers-in-law), leading to discrepancies between women's desired and actual fertility levels. These factors also contribute to close birth spacing, especially among young women. Socio￾economic and cultural characteristics of certain sub-populations present further challenges. The public sector is the main provider of modern FP methods, and faces many structural challenges related to its health workforce and facilities. Health facilities at all levels should be prepared to provide FP services - from community-based health workers and community clinics (CCs), which provide OCPs, male condoms, injectables, and emergency contraceptive pills, to facilities at the upazila level and above, which should be able to provide a full range of modern methods. However, on average only 76.8% of union-level public facilities offer FP services every day - with significant disparity across divisions (e.g. 63.2% in Mymensingh, compared with Dhaka (77.5%), Chattogram (88.3%), and Sylhet (89.5%)).3 1 DHS, 2017-18 2 DHS, 2017-18 3 Bangladesh Health Facility Survey [BHFS], 2017 Performance Evaluation of USAID’s Accelerating Universal Access to Family Planning (Shukhi Jibon) in Bangladesh | 17 2.1.2 Specific issues i. Weak system relating to continued human resource development A key factor limiting service availability is a shortage of trained staff. Nationally, the Bangladeshi public health sector is understaffed, "with twice as many doctors as nurses, clustered disproportionately in urban areas, while rural facilities are overburdened, understaffed, and insufficiently equipped".4 There are no formal continuing medical education (CME) requirements for medical personnel, including FP providers. Additionally, the FP training mandate is held by three entities - the NIPORT, the DGFP and DGHS. Due to the lack of a coordinated approach, training is often conducted on an ad hoc basis, and it is seen that few providers receive further training, while others never receive any training. There is also a concern regarding quality of training as most existing FP training are classroom based, use didactic presentations, and include limited practicum and/or refresher training. Weak supervision and monitoring are also chronic issues within the health system both in DGHS and DGFP. Although the government is committed to improving the situation, no operational models have been put in place so far. Nationally, capacity building is seen as a one-time training, continued follow-up, supervisions, monitoring or mentoring are not systematically organized in order to develop a strong FP workforce. ii. Certain populations, including adolescents and youth, face barriers in accessing FP The MOHFW’s progressive National Strategy for Adolescent Health 2017-2030 recognizes that adolescents lack the structural and social support to "develop their full potential" due to harmful social norms, poverty, and lack of education. The strategy aims to ensure that, "By 2030 all adolescents will lead a healthy and productive life in a socially secure and supportive environment where they will have easy access to quality and comprehensive information, education, and services." The BHFS 2017 reports that only 22% of the health facilities (including community clinics) meet the readiness criteria for FP service provision. If community clinics are excluded, then 51% of the health facilities meet the criteria. However, of those who are classified as FP ready, few can meet the special needs of the adolescents.5 While postpartum FP, post-abortion care FP, and post-menstrual regulation FP are offered in Bangladesh's public health system, much remains to be done to ensure that potential recipients of services are fully informed about their options, and that service provision is tailored to meet the needs of all types of clients, especially adolescents and youth. Against this backdrop, in 2018, USAID initiated its Accelerating Universal Access to Family Planning (AUAFP) activity, commonly known as Shukhi Jibon activity, in order to accelerate family planning utilization in 4 divisions in Bangladesh. 2.2 Activity overview The AUAFP activity was intended to contribute to the health and wellbeing of Bangladeshis and to accelerate family planning (FP) utilization by strengthening capacity of the Ministry of Health and Family Welfare (MOHFW), particularly the Directorate General of Family Planning (DGFP), the Directorate General of Health Services (DGHS), and the National Institute of Population Research and Training (NIPORT). AUAFP’s key strategies include accelerating progress towards increased FP utilization in Chittagong, Dhaka, Mymensingh, and Sylhet divisions by providing adaptive, needs-driven technical assistance (TA) and systems strengthening at national, divisional, district, and upazila levels. The activity aims to achieve breadth of coverage, depth of impact, and consolidation and amplification of efforts by GOB partners while serving those who most need quality FP services and information—districts and divisions with lower mCPR and populations facing the greatest barriers (unmarried adolescents; newlyweds; first-time parents [FTP]; and postpartum [PP], post-abortion care [PAC], and post- 4 Bangladesh Health System Review, 2015 5 National Strategy for Adolescent Health 2017-2030 Performance Evaluation of USAID’s Accelerating Universal Access to Family Planning (Shukhi Jibon) in Bangladesh | 18 menstrual regulation [MR] clients). The activity partnered with NIPORT to build the quality and quantity of human capital in the health system, and with DGFP and DGHS to integrate robust adolescent and youth friendly health services (AYFHS) into facilities and communities in target areas. Pathfinder International, in partnership with lntraHealth International and with strategic partnership from the Obstetrics Gynecological Society of Bangladesh (OGSB) and the University of Dhaka is responsible for implementing Shukhi Jibon over five years, from July 2018 to July 2023, in four divisions of Bangladesh. Recently the activity was extended with no cost till January 2024. The total activity funding over the five-year period is USD 40,652,622 (including USD 3,695,622 in cost share). AUAFP is currently working in 286 health facilities in 32 districts within four focus divisions, including Chattogram, Dhaka, Mymensingh and Sylhet, as well as with 24 training institutes. The list of Shukhi Jibon intervention districts is attached as Annex I. The target beneficiary groups are the 24 FP training institutes, the trainers and the health workforce within health facilities and the populations with access barriers to FP, including adolescents and youth, first-time parents, newlyweds, and postpartum, post abortion and post menstrual regulation clients. Moreover, Shukhi Jibon’s adolescent interventions are supported by 6 NGOs namely, ESDO, FPAB, Lighthouse, PHD, SERAC and YPSA. These NGOs are mainly responsible for demand generation for adolescent health service in selected hard-to￾reach and underserved areas of 15 activity districts. The two intermediate results (IR) of AUAFP are IR-1: increased qualified FP workforce’ and ‘IR-2: increased availability of public sector FP outreach contacts and services, particularly for adolescents and youth’, as shown in the results framework (Annex II) IR1 aims to build the capacity of NIPORT, MOHFW, and the national systems and mechanisms underpinning provider training and mentorship to ensure that providers are both adequately skilled, and more responsive to the needs of the most underserved populations. IR2 aims to strengthen the health and FP sector’s ability to provide quality information and services to adolescents and youth and connect them to youth friendly facilities. IR2 also includes awareness raising and behavior change of adolescents and their gatekeepers through using digital platforms as well as community mobilization activities. Due to limited time and scope, USAID decided to examine selected components (Sub-IRs) of the activity that are highlighted in Table 1. The project document envisions that: “Five years of our joint efforts will result in: NIPORT as a preeminent training leader that partners with DGFP and DGHS to train and mentor capable FP providers using state-of-the-art curricula and training methodologies, certification programs that drive provider commitment to quality, increased availability of FP through integration of services at both pre- and postpartum touchpoints, and a responsive health system—ultimately advancing the GOB’s commitment to the right to health for adolescents, youth, and all Bangladeshis.” Performance Evaluation of USAID’s Accelerating Universal Access to Family Planning (Shukhi Jibon) in Bangladesh | 19 Table 1: Selected Sub-Intermediate Results for Evaluation Goal: Improved Health and Human Capital Objective: To increase utilization of Family Planning Services through Universal Health Coverage IR1: Increased qualified FP workforce IR2: Increased availability of public sector FP outreach contacts and services, particularly for adolescents and youth 1.1 Improved supportive supervision of, and coordination among, public and private sector FP workforce 2.1: Increased availability of facilities meeting adolescent and youth friendly health services (AYFHS) criteria at district level and below 1.2 Improved quality of training curriculum for FP workforce 2.2: Strengthened PPFP, PAC FP and post-MR FP services in public sector 1.3 Sub-IR 1.3: Improved classroom and clinical teaching/training skills of public sector FP trainers 2.3: Expanded use of age-appropriate communication technology by public sector FP workforce 1.4: Initiative MOHFW Certification Program for FP workforce 2.4: Increased community mobilization and support for adolescent and youth friendly FP services 1.5: Improved management, planning, administrative and technical capacity of public sector FP Training Institutes Each evaluation question covers an intervention area and is linked to one or more sub-IRs. The description of the intervention/evaluation areas are elaborated in the findings section. 3. EVALUATION METHODS AND LIMITATIONS 3.1 Evaluation methods The evaluation of Shukhi Jibon activity was conducted by a core team of three independent consultants over December 2022-March 2023. It is a qualitative assessment of selected components of the Shukhi Jibon program. As such the findings were derived from triangulation of key informant interviews (KIIs) and Group Interviews (GIs), field observations and document/data review. Questionnaires were fine-tuned in close consultation with the key stakeholders including icddr,b key professionals. The guidelines and tools are attaches as Annex III. Furthermore, recognizing the fact that the Shukhi Jibon interventions were rolled out in phases, the team focused on the mature districts where the activities had been initiated earlier in the life of the activity. The analysis process commenced following the desk review, qualitative and quantitative (from AUAFP, NIPORT, DGFP database) data collection. At the first phase of the analysis, the evaluation team analyzed the desk review Performance Evaluation of USAID’s Accelerating Universal Access to Family Planning (Shukhi Jibon) in Bangladesh | 20 of program documents to pinpoint the evaluation questions and variables to address research questions. This provided the background information on the activity /program. A complete list of documents reviewed for this assessment is attached as Annex-V. Qualitative information collected through field visits for KIIs and GI6 with stakeholders and FP service providers, and field observation of service delivery at the facility and community levels were used to map the progress of Shukhi Jibon’s interventions targeted to strengthen the capacity of training institutions, mentorship, and supportive supervision and adolescent friendly health services. List of meetings and stakeholders are presented for each evaluation question in Annex IV and list of KIIs and GIs are attached as Annex V. The completeness, accuracy, availability, quality, and usability of the Shukhi Jibon interventions and approaches to strengthen service provision will be assessed to examine accessibility of skills in providing selected FP services. Simultaneously the evaluation team conducted secondary analysis of existing data, which includes Shukhi Jibon and training institutions’ MIS and facility service statistics of DGFP. Following the feedback of stakeholders and logical connections with the government policies and program documents, an assessment of lessons learned and sustainability of the interventions were conducted. Areas of best practice were identified and replicated, while solutions to persisting challenges were discussed. Quantitative data available with either the Shukhi Jibon activity or at the national level (HMIS) were often inadequate for verifying findings. Where data was found to be inadequate, the team used alternate sources of information or verification. The evaluation was conducted mainly through KIIs and GIs, and where possible, through observations and field visits. Appropriate figures/tables were used for data visualization. Averages of the five-points Likert scale were used to assess the few components of the investment packages. The qualitative data collected from the stakeholders (mostly through KIIs and GIs) by the evaluation team is used to assess the adoption, intention, and action in terms of changes/differences (now and before), acceptability, feasibility and sustainability, barriers, and enablers of the selected interventions. The operational definition of sustainability for this evaluation is the extent to which the benefits of the program are likely to continue. Findings were presented in a way to help identify whether progress is on track or not on three Shukhi Jibon investments FP Training Institutes, mentorship, and supportive supervision and adolescent friendly health services separately. 3.2 Limitations The evaluation team decided to focus on three core components of the activity, in consultation with USAID Bangladesh in order to best utilize the limited time and resources. The assessment was limited to the mature Shukhi Jibon intervention areas, that is, the locations where the implementation started during the early years of the activity, mostly between 2018 and 2020. The fieldwork was conducted by the team over the month of February 2023 when they visited four sites in Dhaka, Mymensingh, and Sylhet Divisions. Although ideal, collection of field data from non-intervention districts was beyond the scope of this assessment within the timeframe. Moreover, the assessment was largely limited to qualitative data collection and analysis, for the same reasons. The team nevertheless tried to mitigate these limitations by increasing the number of key informant and group interviews and by conducting an expansive desk review along with HMIS data analysis as much as possible. However, availability of quantitative data on certain interventions were inadequate. 6 In a group interview, key informants were interviewed in small groups of similar respondents, as long as all participants were free to express their own opinions. Performance Evaluation of USAID’s Accelerating Universal Access to Family Planning (Shukhi Jibon) in Bangladesh | 21 4. INTERVENTION DESCRIPTION AND FINDINGS 4.1 Evaluation Area 1: Strengthening Capacity of Training Institutes 4.1.1. Intervention packages One of the major interventions of the Shukhi Jibon was to strengthen the capacity of the training institutes of the MOHFW, particularly the DGFP, DGHS and NIPORT, so that they can impart high-quality training to FP service providers and effectively coordinate the process. Under this intervention, the following components were prioritized for the evaluation: ▪ Introduce competency-based training) and follow-up throughout MOHFW training programs. ▪ Strengthen the training management-information system managed by NIPORT. ▪ Develop Training Management Improvement Frameworks to be implemented in participating training institutes. i. Competency-based training (CBT) CBT is an evidence-based and widely used learning approach, which focuses on developing knowledge (what), skills (how) and attitude (why) of trainees on a specific service or topic through learning by doing. The process requires significant investments in enhancing trainers’ skills to facilitate discussions based on adult learning principles and develop competencies based on practical examples and exposures. It differs from the traditional training approaches, which are mostly classroom based and use unidirectional lectures/presentations with limited scope of discussion and participatory learning. Most of the training programs for FP service providers in Bangladesh adopt the traditional training approach with minimum follow-up support. The Shukhi Jibon activity aimed to integrate CBT as the primary method of imparting all training to FP service providers in Bangladesh. Specific objectives of the intervention package included developing capacities of NIPORT, DGFP and DGHS trainers on training planning and training techniques as well as follow-up support to use these skills while training FP services providers adequately and appropriately. Shukhi Jibon built the capacity of trainers with the specific components of CBT. These are: Planning of Training, Adult Learning Principles, Facilitation/ training Skills, Communication and Feedback Skills in training, Introduction to Competency-based Training, Competency-based Training methods and application, Gender Integration in FP training, Techniques of Managing Training, Identification and preparation of practicum sites, and Follow-up of Training. Trainers were also trained on components of Adult Learning Principles based on training relevance to job or life or what they do. It used two ways dialogue for interaction and participation in discussions, groupwork, brain storming, learning from each other, making presentation etc. with a proximity to apply learning immediately after completing the training, respect, and equality during the training as well as cognitive, affective, and psychomotor interaction for thinking, expression, and practicability. ii. Training management information system (TMIS) TMIS is a systematic training management information system to support the monitoring of training system. Shukhi Jibon operationalized the TMIS for decision-making by NIPORT management teams. In Bangladesh NIPORT is responsible for the training of FP service providers but the training management information system of NIPORT was paper based without any direct link to the personnel/human resource database. The training management system (TMS) of NIPORT has been developed, with support from the USAID’s MaMoni Health Performance Evaluation of USAID’s Accelerating Universal Access to Family Planning (Shukhi Jibon) in Bangladesh | 22 Systems Strengthening (HSS) project, as a web-based application software for administration, documentation, tracking, and reporting of the instructor-led-training programs of NIPORT HQ and its institutes. The Shukhi Jibon assisted in operationalizing this system in NIPORT with the aim of ensuring management of training calendars, training record keeping, participant’s record keeping, and generation of management reports. In short, the Shukhi Jibon activity aimed to operationalizing the training management-information system (TMIS) by digitizing NIPORT’s TMS, promoting the use of data in training related planning and decision-making. The objectives of TMIS are to: a) replace paper-based system and b) strengthen NIPORT’s digitalized TMIS iii. Training management improvement frameworks (TMIFs) The TMIFs are a central component of Shukhi Jibon’s approach to quality improvement and strengthening training institutes of NIPORT and DGFP to provide a platform to work together on various tools and approaches such as CBT, In-service Training standards, Optimizing Performance and Quality (OPQ), Improvement Collaboratives (IC) and TMIS to improve quality throughout the training ecosystem. It intended to enhance the management and leadership capacity at Training Institutes to facilitate optimal utilization of trainers who have undergone the CBT trainings and application of tools such as the a) Training Standards, b) Trainers Manual on CBT, c) Evidence based management, d) In-service training to strengthen the system, e) TMIS utilization for training data management, and f) Data driven decision making using TMIS-dashboard. The interventions included to identifying the strength, weakness, opportunity and threat in training management of respective training institutes, identifying the gaps and needs to improve the existing training management in respective training institutes, developing management strategy with recommendations based on the respective context, gaps, needs and SWOT analysis findings. It also aims at capacity building of Principals and Training Officers of respective institutes of NIPORT and relevant officials of DGFP for implementing the recommendations of the respective TMIFs utilizing CBT, OPQ &IC and TMIS for improving quality of training for FP service providers. The TMIF component contains; a) background information of the respective training Institute, b) catchment area, c) current human resource, d) training management practices, e) administration, f) Technical Assistance of Shukhi Jibon, g) Process of TMIF development h) findings of SWOT analysis, i) recommendations for strengthening the training institute following 9 dimensions, and j) photos of consultative workshops of developing the TMIF. The 9 dimensions of recommendations were: a) Leadership and Governance, b) Strategic planning, c) External relations (coordination with DGFP, DGHS, DGNM and other stakeholders), d) Financial resources, e) Personnel (trainers and resource persons), f) Trainees (Service providers), g) Equipment and materials, h) Facilities and infrastructure, and i) Evaluation and knowledge management. 4.1.2 Intervention layers Shukhi Jibon worked with four layers of stakeholders for implementing the priority interventions (Figure 1) with district focus. The layers are as follows: ▪ Organizations: for providing leadership and coordination (NIPORT, DGFP) ▪ Training institutions: for implementation of the planned activities (NIPORT HQ, MFSTC, MCHTI, FWTI, RPTIs, RCTCs) ▪ Master Trainers: for adopting the new approach of training and impart training of trainers (recognized trainers at NIPORT and DGFP Training institutes in six pilot districts of Shukhi Jibon) ▪ Trained Trainers: Creating a resource pool trainer for receiving the training and using the skills in providing relevant training institutions (A list of trained trainers on CBT by districts enclosed in Annexure) Performance Evaluation of USAID’s Accelerating Universal Access to Family Planning (Shukhi Jibon) in Bangladesh | 23 Figure 1: Four implementation layers of Shukhi Jibon 4.1.3 Steps of implementation i. Competency-based training (CBT) As shown in Figure-2, the Shukhi Jibon took the following steps to introduce and integrate CBT in MOHFW training activities. Figure 2: Steps taken to introduce and integrate CBT in MOHFW activities Organizations Leadership and coordination Training Institutions Implementation Master Trainers Adopt and imparting new approach Trained Trainers Receiving the training and using the skills for imparting training and its quality Onboarding the organization Selection of institutes Identifying the training needs and bottlenecks Developing the training package Organizing divisional workshops and developing action plans Training of master trainers Training of trainers Performance Evaluation of USAID’s Accelerating Universal Access to Family Planning (Shukhi Jibon) in Bangladesh | 24 a) Step 1-onboarding the organization The Shukhi Jibon activity made formal agreements with NIPORT, DGFP, DGHS for formulating a shared vision and common understanding for implementing the interventions. The memorandum of understanding (MoU) with DGFP was signed on 13 January 2019, and the MoU with DGHS was signed on 02 May 2019. The MoUs outlined the scope of collaboration for supporting reproductive health related activities under the 4th Health, Population and Nutrition Sector program (4th HPNSP) and relevant operation plans (OPs). Shukhi Jibon also signed a letter of collaboration (LoC) with NIPORT on 31 March 2019. The LoC outlined specific areas of collaboration, including supportive supervision, improving training curriculum for workforce, improving teaching/training skills, improving management, planning, administrative and technical capacity of public sector training institutes. b) Step 2-Selecting training institutes The Shukhi Jibon activity identified 24 public training institutes from NIPORT (16) and DGFP (8)(3). NIPORT institutes are NIPORT HQ, Regional Population Training Institute (RPTI), and Regional Training Centre (RTC). DGFP institutes are Mohammadpur Fertility Services and Training Center (MFSTC), Maternal and Child Health Training Institute (MCHTI), Family Welfare Training Institute (FWTI) and Regional Clinical Training Centers (RCTCs). In some districts, both NIPORT and DGFP institutes were selected, such as Sylhet and Faridpur. A list of all training institutes with location and catchment are is attached as Annex VII. The Shukhi Jibon activity has successfully established formal relationships with NIPORT, DGFP and DGHS through MoUs and LoC. The activity helped in gaining access and acceptance to NIPORT, DGFP and DGHS policy makers and contributed to facilitating the activities in the selected facilities. Achievements • One LoC/MoU involving all parties could have been signed for better clarification, collaboration, and coordination among all parties. According to the Shukhi Jibon activity leadership team, the organizations asked for bilateral MoUs and LoC based on common practice and procedure. Also, these formal agreements helped the Shukhi Jibon activity to avoid startup delays and difficulties of the activity. • The interventions mentioned in the LoC and MoUs were mostly generic, not specific to CBT, TMIS and TMIF etc. According to the Shukhi Jibon leadership, the lack of specificity was deliberate as they wanted to cover a broader agenda under the scope of work. Areas of improvement Performance Evaluation of USAID’s Accelerating Universal Access to Family Planning (Shukhi Jibon) in Bangladesh | 25 Figure 3: Location of Shukhi Jibon selected training institutes • The Shukhi Jibon on boarded all selected training institutions according to the plan. • The NIPORT HQ, RPTIs, RTCs of NIPORT and MFSTC, MCHTI and FWTI of DGFP were found more responsive in imparting training under organizational management structure. There are concerns regarding the selection of Clinical Contraception Service Delivery Program (CCSDP) led FPCS-QIT offices as ‘Regional Clinical Training Centers (RCTCs)’. There are 10 Regional and 54 District FPCS-QIT offices in 64 Districts of the country under the Line Director, CCSDP of DGFP. The regional FPCS-QITs are not renamed as ‘RCTCs’ and are not included in the organogram of DGFP. Figure 4 shows the signboard of a regional FPCS-QIT office, Training Room and Conference Room. A regional FPCS-QIT has only one Regional Consultant and one Senior Staff Nurse and could be difficult to consider as a functioning training institute. The Shukhi Jibon training need assessment (described later) reported ‘non-existence’ of these offices as independent training institutes and excluded them from the assessment sampling frame. However, CCSDP operational plan under the 4th HPNSPJanuary 2017-June 2022 mentioned these regional FPCS-QIT offices as ‘LARC & PM training centers’. Based on this rationale, the Shukhi Jibon activity decided to consider them as training institutes and continued their investment as planned. Corrective measures e.g., proper triparty consultation among USAID, DGFP, and Shukhi Jibon) may be taken to avoid confusions to use these as RCTCs. The Evaluation Team also recommends to facilitate collaboration between NIPORT and DGFP for strengthening the NIPORT facilities for both clinical and non-clinical trainings and active engagement of FPCS-QITs in coordinating and organizing the clinical trainings. This will avoid duplication as well as coordinated effort of NIPORT and DGFP will help improved quality of trainings. Achievements Areas of Improvement Performance Evaluation of USAID’s Accelerating Universal Access to Family Planning (Shukhi Jibon) in Bangladesh | 26 Figure 4: Signboard of a Regional FPCS-QIT Office c) Step 3-identifying the training needs and bottlenecks The Shukhi Jibon activity conducted Training Needs Assessment in 6 districts among 10 institutional heads, 25 trainers and 493 FP service providers in 2019. Achievements The Shukhi Jibon activity conducted the TNA extensively and with rigor. The TNA exercise identified the needs of key FP service providers. It identified the training needs and bottlenecks and highlighted several challenges on: • FP myths in society and fear among clients regarding LARC and PM • Lack of career development opportunities • Providing Adolescent and Youth friendly services • Lack of context-specific refresher training on FP methods and services • Inadequate training on counseling • Inadequate practicum training for FP service providers • Inadequate supportive supervision • Improve the capacities of both in-house and outsourced trainers Inadequate on-the job training: An endline TNA has recently been conducted that will provide useful information for decision making. Areas of improvement • Specific action/ways could be explored to address the key bottlenecks and challenges identified during the TNA. NIPORT, DGFP and DGHS need to be actively engaged in revisiting the findings of TNA and developing a measurable, SMART action plan with definite timeline and responsibilities. Strong coordination among the DGFP and NIPORT leadership with individual training institutes for effective implementation of action plans • Shukhi Jibon needs advocate for inclusion of relevant activities in respective operation plans of next sector plan. • Ensure availability of essential training equipment (e.g., mannequin) in coordination with NIPORT and DGFP. Performance Evaluation of USAID’s Accelerating Universal Access to Family Planning (Shukhi Jibon) in Bangladesh | 27 d) Step 4- organizing divisional workshops and developing action plans Shukhi Jibon organized divisional workshops in Chattogram, Sylhet, Mymensingh and Dhaka (Figure-5) Policy makers, managers, and service providers from NIPORT, DGFP and DGHS participated in the workshops, identified gaps, and developed action plans through in-depth discussions. The action plans included adolescent health, postpartum family planning (PPFP), learning labs, mentorship and supportive supervision and capacity development. However, the implementation of the action plans was not systematically and regularly monitored with all parties. It is not followed up by DGFP and NIPORT leadership rather it was sporadic. Figure 5: Status of divisional workshops and developing action plans Achievements Division specific plans were developed through consultative process. Areas for Improvement • Plans need to be systematically and regularly monitored. A monitoring plan needs to be developed and agreed upon by all concerned. • Strong coordination among the DGFP and NIPORT leadership with individual training institutes for effective implementation of action plans. • Findings of the end line TNA need to be analyzed and shared with stakeholders for forward planning. Performance Evaluation of USAID’s Accelerating Universal Access to Family Planning (Shukhi Jibon) in Bangladesh | 28 e) Step 5-developing the training package The Shukhi Jibon activity developed a training package on CBT based on globally accepted standards. The training package included a comprehensive manual for trainers, which had 184 pages with several graphical illustrations (Figure 6). Technical experts from IntraHealth International HQ and Shukhi Jibon activity experts led the process with guidance from the technical committee headed by Director General, NIPORT along with the representatives from NIPORT, DGFP, and DGHS. The manual was pre-tested with trainers extensively before finalization. Figure 6: The Competency-based training manual for the trainers Developed a trainer’s manual. Partners 184 pages with graphical illustration Achievements • The training package was well-accepted by the DGFP and NIPORT policy makers and master trainers. A few of the trainers were asked to share their opinions in a five-point Likert scale. Most of the respondents strongly agreed on the acceptability, appropriateness, and feasibility of the training package. • NIPORT issued circulars for integrating CBT in the training system. • NIPORT organized Training for officials at own cost for integrating CBT in training institutes in Rajshahi, Rangpur, Barisal and Khulna (non Shukhi Jibon areas) • NIPORT included these interventions in their annual report to MOHFW. • NIPORT organized workshop for discussing sustainability of CBT and TMIS. • NIPORT integrated CBT in health care of disabled persons and hospital management curriculum. • DGFP integrated CBT in LARC& PM manual. • DGFP integrating CBT in PPFP manual. • Bangla version of the manual has been printed and is being distributed. Performance Evaluation of USAID’s Accelerating Universal Access to Family Planning (Shukhi Jibon) in Bangladesh | 29 f) Step 6-training of master trainers Shukhi Jibon organized a workshop in February 2019, where a total of 20 master trainers from NIPORT, DGFP and DGHS participated (Figure 7). The master trainers, representing 6 districts, were trained on developing curricula and training materials, facilitating sessions based on CBT principles. Figure 7: Training of master trainers and locations Areas of improvement • Manual added a list of participants in workshop to develop manual and circular on technical committee formation for the manual development, but does not have the list of key contributors and thus, future edition/ update may be challenging. • The Shukhi Jibon may take steps for having wide availability of soft copies of Bangla/English version of the training manual among the trainers as well as training institutes. Achievements The workshop was well-accepted by master trainers. A few of master trainers were asked to share their opinions in a five-point Likert scale. Most of the respondents strongly agreed on the importance, quality and acceptability of the training sessions, Trainer’s manual, and Training standards, as well as training, follow-up, and mentoring of trainers. Likert scale on training of master trainers showed that most of the master trainers were satisfied with the quality of the training. Master trainers agreed or strongly agreed on the confidence, importance, and acceptability. Performance Evaluation of USAID’s Accelerating Universal Access to Family Planning (Shukhi Jibon) in Bangladesh | 30 g) Step 7-training of trainers The master trainers trained 44 trainers of NIPORT, 268 of DGFP, 60 of DGHS and 2 of DGNM from November 2020 to June 2021 (Figure 8 and Figure 9). The participants were from NIPORT-HQ and sub-national institutes, national and sub-national managers of DGFP and DGHS, Consultants, Midwives, Medical Officers etc. List of CBT participants provided as Annex VIII. Figure 8: Timeline and participants of training of trainers Figure 9: Locations of training of trainers Achievements The training facilitated by the master trainers was well-accepted by trainers. Most of the trainers have reported that they are using the CBT principles in imparting FP-specific training. Performance Evaluation of USAID’s Accelerating Universal Access to Family Planning (Shukhi Jibon) in Bangladesh | 31 h) Step 8- Facilitation and Follow-up Support The Shukhi Jibon activity recruited 10 TCBOs to provide follow-up support after training. The TCBOs conducted periodic visits to the trainers (trained in CBT) and provided technical assistance to prepare their training materials based on the CBT principles. They also participated in the training sessions conducted by the trainers and assessed the integration and use CBT principles in those sessions. They provided personalized feedback to the trainers to discuss gaps and potential areas of improvements. Achievements • Although there was early resistance by the trainers regarding TCBOs, they eventually appreciated the technical assistance through follow-up support. • The trainers felt that the continuous support of TCBOs was helpful in adopting or following the CBT principles in training session facilitated by them. Areas of improvement • There are concerns regarding the sustainability of TCBOs as most of the respondents (trainers) felt the need of continuing this support. • Addition of extra manpower are usually reducing burden of some people, work that are usually neglected gets done but it is an expensive intervention. Considering the early encouraging results of TCBO intervention, the Shukhi Jibon may conduct an experimental study on TCBO actions and follow-up in the practical circumstances. • Shukhi Jibon may gradually reduce the dependence on TCBOs and hand-over the work-load to government staff. Performance Evaluation of USAID’s Accelerating Universal Access to Family Planning (Shukhi Jibon) in Bangladesh | 32 ii. TMIS intervention Shukhi Jibon conducted a landscape analysis in February 2020 on TMS/HRIS to assess the structure of NIPORT's TMS and map the stakeholders based on the Government’s understanding that enhancing the capabilities of NIPORT and its institutes along with the training missions of DGFP and DGHS is the pathway to improve workforce competencies. The landscape analysis identified the training needs and bottlenecks to improve workforce competencies. Shukhi Jibon took over the TMS where the MaMoni HSS activity left and to update and operationalize it for NIPORT. It took over 14 months to get the source code of TMS from MaMoni to initiate the updating and operationalization process. Shukhi Jibon updated the TMIS based on the need of NIPORT, and the needs were identified through consultative workshops involving the training institutes of NIPORT and DGFP. Shukhi Jibon also trained NIPORT and DGFP staff on using and maintaining the digitalized TMIS. Participant list of TMIS training provided as Annex IX. iii. TMIF intervention Shukhi Jibon started TMIF activities in activity year 3 and conducted 3 workshops on SWOT analysis with officials Achievements • The Shukhi Jibon expanded TMIS of MaMoni HSS in NIPORT’s training institutes, instead of duplicating the efforts by developing a new TMIS. • NIPORT issued circulars for operationalization of TMIS at 16 Training Institutes (currently has 48,000 training records) • NIPORT identified TMIS focal person • The digitalized TMIS improved the provision and capacity of NIPORT for real time monitoring and reporting of several training related indicators mentioned in the relevant operational plan related indicators mentioned in the relevant operation plan. • NIPORT hosted TMIS in the National Data Centre at Bangladesh Computer Council with NIPORT’s OP funds • NIPORT is operationalizing TMIS in Training Institutes in Rajshahi, Rangpur, Barisal and Khulna divisions using own funds Areas of improvement • Steps may be taken in providing access of TMIS to DGFP training institutes (MFSTC, MCHTI, FWTI) and build capacities of relevant officials. • There is no interoperability between the NIPORT’s TMIS and DGFP’s HR databases. Without the interoperability, DGFP or NIPORT will not be able to assess the true training needs and develop a comprehensive training plan. The Shukhi Jibon project has started taking some initiative to address the gaps. • Vacant positions and frequent transfer of staff at RTCs and RPTIs created obstacles in effective implementation and functionality of digitalized TMIS. Performance Evaluation of USAID’s Accelerating Universal Access to Family Planning (Shukhi Jibon) in Bangladesh | 33 of NIPORT (1st Workshop 10-11 Nov 2020, 2nd Workshop 27-28 March 2021 and 3rd Workshop 7-8 Sep 2021) and 2 workshops for DGFP (1st Workshop 15-16 Feb 2022 and 2nd Workshop 11-12 April 2022). A TMIF report was developed for each of the participating institutes (16 NIPORT institutes and 8 DGFP RCTCs) during the workshops. The individual TMIF report contained: a) Background information of the respective training Institute b) Catchment area c) Current human resource d) Training management practices e) Administration f) Technical Assistance of USAID’s Shukhi Jibon g) Process of TMIF development h) Findings of SWOT analysis i) Recommendations for strengthening the training institute following 9 dimensions j) Photos of consultative workshops of developing the TMIF 4.1.4 Conclusion of Evaluation Area 1 Achievements • The Shukhi Jibon activity involved multiple stakeholders in SWOT analysis, which helped in developing acceptance and some level of ownership of the findings. • Training institutes organized performance review meetings to assess the progress of the implementation of the recommendations. • All 16 training institutes of NIPORT implemented at least 3 recommendations including a) CBT integration in training programs, b) use of Trainers’ manual of CBT as reference manual, and c) opalization of TMIS Areas of improvement • Shukhi Jibon involved only one or two participants from each training institute in the SWOT analysis exercise. It was not enough to understand the bottlenecks and identify the root causes of each institute. Though Shukhi Jibon claimed that with the facilitation from the TCBOs of Shukhi Jibon, Principals and Training Officers of training institutes consulted with relevant other staff of respective training institutes before participating in the SWOT analysis exercise. However, during KII/GI, the other staff members were not aware of the process or the framework. • There were major gaps in the implementation of the recommended activities (outlined in the TMIF) in the NIPORT and DGFP institutions. Though all workshops were complete by early 2022, Shukhi Jibon claimed that they need more time for the implementation of the TMIF recommendations. Joint efforts with DGFP and NIPORT may be helpful to improve the situation. Moreover, recommendations could be revisited and developed as SMART action plans and staff members from different layers need to be involved in the process. Performance Evaluation of USAID’s Accelerating Universal Access to Family Planning (Shukhi Jibon) in Bangladesh | 34 The Shukhi Jibon activity was reasonably successful in achieving the specific objectives related to CBT, TMIS and TMIF. However, there are some areas of improvement, particularly in coordination among partner organizations and implementation of the institution-specific action plans. The Shukhi Jibon activity should also consider discussing with all concerned to review and amend some of the original plans based on context and real needs. This may significantly improve the value for money of investments and contribute towards sustainability. 4.1.5 Recommendations on Evaluation Area 1 a) There were gaps in coordination among DGFP, NIPORT and DGHS initiatives. A tri-party agreement involving all partners would have helped gaining better sensitization and ownership regarding the priority interventions. b) Selection of the training institutes of DGFP had issues with appropriateness and concerns regarding sustainability. The value for money could be improved by investing in approved and functional training institutes. c) The training institutes did not receive adequate support regarding critical training equipment, such as Mannequin. It compromised the provision of the training institutes in delivering high quality training. d) Representation of the training institutes in developing action plans was somewhat limited. Moreover, the execution of the action plans had several gaps and limitations. Shukhi Jibon could have addressed this with effective coordination and active facilitation with joint involvement of policy makers and senior officials of DGFP and NIPORT. e) There is no link between the TMIS and the personal MIS of DGFP. It compromises the utility and potential impact of the TMIS on planning and decision making. Shukhi Jibon could have taken more initiatives to address this specific gap. 4.2 Intervention Area 2: Mentorship and supportive supervision (M&SS) 4.2.1 Intervention Description Integrated Mentoring and Supportive Supervision (M&SS) is central to achieving Shukhi Jibon’s overarching activity objectives to increase the use of FP services by growing and strengthening the qualified FP workforce to improve the quality of FP service provision in the activity areas. Objectives of the approach were to introduce mentorship and supportive supervision in public sector, increased quality of FP service provision and integrate gender transformative supportive supervision approach. Mentoring is a positive developmental partnership in which an experienced, proficient, and empathetic mentor teacher and coaches a mentee or group of mentees, in person and/or virtually, to ensure competent workplace performance and provide ongoing professional development. Mentorship is driven primarily by the mentee with the primary aim of building mentee capability and self-reliance. Supportive Supervision is a process that promotes quality at all levels of the health system by strengthening relationships within the system; identifying and resolving problems; optimizing the allocation of resources; and promoting high standards, teamwork, and effective two-way communication. Supportive supervision focuses on the entire health facility and all health workers. Figure 10 identifies the differences between mentorship and supportive supervision. Performance Evaluation of USAID’s Accelerating Universal Access to Family Planning (Shukhi Jibon) in Bangladesh | 35 Figure 10: Objectives and focus of Mentorship and Supportive Supervision While mentorship and supportive supervision are distinct, they do have some overlapping functions. Shukhi Jibon developed integrated M&SS training and curricula to focus on common skills, such as leadership, problem solving etc. required for both mentorship and supportive supervision. Figure 11 shows the different steps of initiation and implementation of the intervention. M&SS was designed through two consultative workshops, held in 2019 with GOB and stakeholders and under guidance of the M&SS Working Group (WG) at the MOHFW, chaired by the Joint Secretary (Planning), Family Welfare Division of MOHFW. The WG consists of representatives from DGFP, NIPORT, DGHS, University of Dhaka, Ipas and Shukhi Jibon. Based on the recommendations, M&SS Trainer’s and Trainee’s manuals were developed by the Shukhi Jibon staff members and reviewed by M&SS working group. The trainer’s manual was developed for the Master Trainers, while the participants manual was developed for the Mentors. The manuals were published by the DGFP in November 2020. Job aids and tools [checklists, Root Cause Analysis (RCA) tool, Action Plan Handbook, Self￾assessment form, Consent form] for M&SS were developed at the same time to support the intervention. -demonstrate correct techniques -provide clinical instruction -coaching -conduct clinical case review -facilitate provision of support by others on provider support team Focus on the individual provider Introducing supportive supervision in FP program of Bangladesh Introducing mentorship for FP service providers in public health sector Increase quality of FP services provision in project implementation area Implement gender transformative supportive supervision approach - address facility infrastructure issues -monitors equipment, supplies and supply chain -review forms -reviews training, staffing and other HR issues -Appraise client satisfaction Focus on the health facility and all providers Mentorship Objective Supportive Supervision Performance Evaluation of USAID’s Accelerating Universal Access to Family Planning (Shukhi Jibon) in Bangladesh | 36 The Master Trainers and mentors were selected during training of all supervisors and through consultation with district managers. After training, mentors were paired with up to five mentees, who were selected by the mentors. After selection, individual mentee profiles were to be developed through SWOT analysis of the mentees in order to assess the baseline status and set targets for each mentee. Next, mentors and mentees met for periodic onsite mentorship sessions, which transitioned to virtual sessions during the COVID-19 pandemic. The topics and timing of the sessions were supposed to be determined based on mentee’s interest and identified areas for growth. On the other hand, supervisors conducted routine quarterly supervisory visits with a structured checklist. The mentorship intervention was scaled in 16 activity districts while supportive supervision is operational in all 32 districts. The evaluation team visited the districts where both Mentorship and Supportive supervision are operational. The findings are grouped in three sections: i. preparatory activities ii. field implementation iii. benefits, challenges and sustainability. Figure 11: Steps of Mentorship and Supportive Supervision Intervention 4.2.2 M&SS findings i. Preparatory activities a) Mentorship and Supportive Supervision Training Manual: The trainers and participant’s manuals were 200+ and 150 pages long and published in November 2020 by the DGFP . The manuals are unique resources for Preparatory Activities ▪ Literature review (study report) ▪ Landscape Analysis ▪ Consultation Workshops ▪ M&SS working group formation ▪ Development of training manuals and Job aids (checklists, RCA tool, Action Plan Handbook, Self￾assessment form, Consent form) ▪ Master Mentor training ▪ Training of Trainers ▪ M&SS Cascade training Field Implementation ▪ Mentor-mentee selection and pairing ▪ SWOT analysis (mentees) and Action Plan Development ▪ Mentorship sessions ▪ Mentorship mobile application: development and training ▪ Refresher training ▪ Mentor’s periodic meetings/ District FP progress review meeting ▪ Mentee graduation ▪ Supportive Supervisory visits Performance Evaluation of USAID’s Accelerating Universal Access to Family Planning (Shukhi Jibon) in Bangladesh | 37 any ideal mentorship program, however, our interviews reveal that because of language (English) and heavy content, many mentors found the manual difficult to understand. The recently translated draft Bangla version is also quite complex and may not be suitable for mid-level managers and supervisors. There is no structured training manual for the Mentees and they felt the need of a simple, user friendly, easy operational guideline in Bangla. Though the manual was published by DGFP, a lack of ownership was found during discussion with DGFP officials as they were not involved with the manual development process and only worked as reviewers. b) M&SS Training: There are 3 types of training: M&SS basic training, refresher training and training on the mentorship application. The respondents highly appreciated the quality of training, as the trainers followed a competency-based training approach and made the trainings interesting. However, the training was later condensed from 4 days to 2 days, which may not be sufficient for delivering a 25-hour training module. Till date, a total of 617 managers and supervisors in 32 districts were trained in M&SS. c) Mentor and Mentee Selection: Almost all the mentors who were interviewed were selected due to their position. When one mentor was transferred, her/his successor was expected to take over the role. On the other hand, mentees were selected by the mentors, in most of the cases they were the supervisors. When asked, most of the mentors and mentees responded that ‘good performers’ are selected as mentees, even though the protocol suggests to enroll the poor performers. Mentees also did not know why they were selected and neither mentors nor mentees were aware of any consent form. d) Job Aids: The job aids that were developed only for the M&SS intervention are self-assessment form, root cause analysis form, action plan handbook, observation checklists, FPCS-QIT checklist etc. Of these only the Action Plan handbook was used by a few mentors and the observation checklist was the main tool used during sessions. The checklists were highly appreciated by the respondents except by one or two mentors who suggested for customization and also who faced difficulties with the scoring. No filled-in form was available at DGHS facilities such as, UHC and CC. On the other hand, the supervisory checklist is used at a limited level only by the FPCS-QIT. We found only a few checklists that are duly filled-in. Some of the supervisors said the checklist is too long and not all components of the checklists can be filled in a single visit. ii. Field implementation a) Mentorship Sessions: Mentorship/coaching sessions were held as per convenience of the mentors and mentees, with no advance planning done. During the session, the mentor observed the service delivery skill of the mentee, scored and discussed the gaps with the mentee. As observation of LAPM was not possible due to lack of clients, simulation was used to assess Mentee’s skill. Shukhi Jibon staff remained present in all mentorship sessions and have supported the mentor throughout the process. Though initial plan was to hold 1 session per mentee per month, in reality, on an average, 2-5 sessions per mentee per year were held, although the initial plan was to hold at least one session per month. The reasons for a smaller number of sessions are: ▪ Busy schedules and conflicting priorities of mentors ▪ In-person session was not possible during the pandemic, so mentee’s performance assessment was not possible ▪ Mentors do not conduct structured mentorship session without Shukhi Jibon’s support, if they do, they don’t keep records so preparing for graduation is difficult ▪ Vacancies within project in Dhaka and Mymensingh resulted in low number of sessions Performance Evaluation of USAID’s Accelerating Universal Access to Family Planning (Shukhi Jibon) in Bangladesh | 38 b) Mentor’s Engagement: Among 617 trained supervisors, 190 participants of 16 districts were engaged as mentors (Table 2). The reason for universal coverage of training of all supervisors was to engage them in supportive supervision in both intervention and non-intervention areas. Of 190 selected mentors, 88 are currently active in M&SS, which means that only 14% of the trained providers are currently active as a mentor and over half of the mentors dropped out after enrolment as mentor. Highest attrition was at upazila health complex level and lowest at the specialized hospitals although the number of total mentors is low at this level. Average mentor and mentee ratio were 3.1 (highest 3.8 and lowest 1.6). Reasons for high drop-out were reported to be transfer (30%), retirement (12%), Exit without showing any reason (12%), pregnancy (1%), unwillingness and busy schedule of mentors (40%), study leave (3%) (Table 3). The reasons for reluctance/unwillingness as identified by Shukhi Jibon staff are: ▪ Perceives that Mentorship/mentorship sessions are time consuming and not willing to allocate time from service provision hours for mentorship ▪ Involvement with other project activities and job responsibilities such as some service providers (both mentor and mentees) are engaged in more than one facility/region and are over burdened ▪ Conflicting schedule with other project activities ▪ Sometimes denied structured mentorship and wanted to do unstructured mentorship Table 2: Trained mentors who are currently engaged Type of Facility # of Trained personnel on M&SS (32 districts) # of trained personnel engaged (Mentor) (16 districts) # of trained mentor currently active (16 districts) # of active mentees currently engaged against active mentors (16 dist) Mentor: Mentee Ratio District Hospital 31 20 12 28 2.3 Specialized Hospital 7 7 5 8 1.6 MCWC 15 12 5 22 4.4 UHC 175 43 11 38 3.5 Sadar FP Clinic/ MCH unit 268 67 36 119 3.3 UH&FWC 12 12 7 13 1.9 Non-Facility Admin. level 110 29 13 50 3.8 Total 617 190 88 278 3.1 Source: activity database Table 3: Reasons for drop-out of Mentors Reasons for drop-out of Mentors Number % Demise 1 1 Exit without showing any reason 15 12 Unwilling/Reluctant 49 40 Maternity leave 1 1 Performance Evaluation of USAID’s Accelerating Universal Access to Family Planning (Shukhi Jibon) in Bangladesh | 39 Transferred 36 30 Retired 15 12 Study leave 4 3 Total 121 100.00 Source: activity database c) Distribution of Mentees: A total of 345 mentees from DGFP and 169 mentees from DGHS were enrolled in the program (Table 4). Highest number was FWVs from DGFP. The percentage of providers who were enrolled as mentee against the total number of staffs in that position reveals that the coverage of the intervention is quite low. The highest proportion was MOMCH-FP (27%) and lowest was HA/HI/AHI (0.13%). Currently only 54% mentees are engaged in the process and the percentage of drop-out widely varies among different positions. Most of the doctors/Medical Officers dropped out from the program after enrollment. Table 4 shows reasons for drop-out of mentees include transfers (12%), Retired (3%) pregnancy (1%) and reluctance to continue (77%). The reasons for reluctance were almost same as the mentors. Table 4: Distribution and engagement of mentees by designation Designation Directorate # of Mentees enrolled % Mentees against total staff # Active Mentees % of Mentees active 1. FWA DGFP (345) 38 0.80% 27 71 2. FWV 248 20.00% 146 59 3. FPI 4 0.40% 4 100 4. FP Counsellor 1 6.20% 0 0 5. SACMO 27 5.00% 17 63 6. AFWO/AUFWO 4 8.00% 3 75 7. MO (MCH-FP) 22 27.00% 5 23 8. MO (Clinic) 1 6.25% 0 0 9. HA/AHI/HI DGHS (169) 8 0.13% 0 0 10. CHCP 15 0.50% 8 53 11. Midwife 68 43% 35 51 12. SSN/Asst. Nurse 68 24.00% 33 49 13. RMO 1 1.00% 0 0 14. MO 9 1.00% 0 0 Total 514 278 54 Source: activity database Table 5: Reasons for drop-out of Mentees Reason for drop-out of Mentees Number % Demise 1 0 Maternity leave 2 1 Retired 12 5 Study leave 3 1 Transferred 28 12 Performance Evaluation of USAID’s Accelerating Universal Access to Family Planning (Shukhi Jibon) in Bangladesh | 40 Inactive/Reluctance 179 77 Mentor inactive 6 3 Sickness 1 0 Total 232 100 Source: activity database d) Mentee’s progress by division: Each mentee was assessed on specific skills at the beginning as well as at each mentorship session. Baseline score and latest scores in key FP service delivery areas (FP counseling, OCP, Condom, Injectables, IUD and Implant) were compared to assess the progress. (Figure-12) shows that the last scores in all service areas were more than 80% while the baseline scores varied from 12% to 83% in different areas. Average baseline score of all components was 53.5% while the average current score is 90%. The team also examined the scores of individual mentees during field visits, which are consistent with the Shukhi Jibon database. However, there may be other contributing factors for increased knowledge and skills as the mentees are exposed to different trainings and interventions on maternal, child health and FP. Figure 12: Mentee's Progress in FP Service Delivery by Division Source: activity database e) Graduation of Mentees: Mentees who completed FP and MCH service-related sessions with direct mentoring by assigned mentor and consistently received at least =>80% competency score following checklists (FP checklist-FP Manual, MCH-FPCSQIT checklist) with three consecutive sessions for each service graduated from the program. The mentor also provided suggestions on whether the mentee graduated or not based on observations during services. Till date, 24 mentees have graduated which is 8.7% of 278 active mentees. Eight of them graduated from mentees to mentors. As per Shukhi Jibon staff members, more mentees are in the graduation process and a few will graduate soon. The reasons for low graduation are mainly due to its strict conditions of “consistently receiving at least =>80% score with three consecutive sessions for each service.” In many cases, no of total sessions held was not enough to assess the mentee on all topics. On an average, 2-5 sessions were held per mentee per year which was not adequate for fulfilling the graduation criteria. There are also data quality issues as mentors are reluctant to fill-up the checklist without Shukhi Jibon staff member’s active support. The other reason mentioned by the respondents are lack of awareness of graduation criteria and process, they couldn’t set the goal for achievement. Shukhi Jibon is closely tracking the progress and increasingly supporting more follow-up sessions of the Mentees to fulfill the graduation criteria. 0% 20% 40% 60% 80% 100% 120% FP Counseling OCP Condom Injectable IUD FP Counseling OCP Condom Injectable IUD FP Counseling OCP Condom Injectable IUD Implant FP Counseling OCP Condom Injectable IUD Implant Sylhet Chattogram Dhaka Mymensingh Scores obtained Topics of assessment Baseline Last Score Performance Evaluation of USAID’s Accelerating Universal Access to Family Planning (Shukhi Jibon) in Bangladesh | 41 During KII and GI, it was found that only 2 mentors heard about the graduation process but were not clear about the criteria and process. The mentees were not aware of the graduation criteria. f) Mentorship Mobile Application: The objective of developing this app was to shift from paper-based system to a digital system. It started in December 2021 and was implemented in phases. Shukhi Jibon database shows that about 20% of sessions are being conducted through mobile apps, and in 2022 a total of 160 sessions were held through mobile apps. However, during the field visits we found that out of 7 mentors and 14 mentees, only one mentor-mentee pair was successfully using the app. The remaining respondents received training and have the app installed on their phone but need more technical support from the activity to navigate it. Moreover, the app uses both online and offline version but many users faced trouble initially to use it online. Also, some users were reluctant to use their personal device as there is no tab supplied by the Shukhi Jibon activity. Use of the app was voluntary and tech savvy users adopted it quicker and rest of the users (majority) are adopting it slower. Shukhi Jibon is currently surveying to see patterns of use of the app and will be able to explain further later as well as take actions to increase the use accordingly. Shukhi Jibon is also advocating with Director MIS, DGFP to link the app with existing DGFP systems. g) Supportive Supervision: The intervention uses a 13-page long observation checklist that was endorsed by the DGFP. The checklist has three sections: Section-l is related to facility readiness and performance; Section-II contains FP method specific clinical skills observation checklists and Section-III is related to MCH services by component. Section 2 and 3 are also used by the mentors as part of mentorship visits. The components of Section-1 captures information on: ▪ Facility Infrastructure ▪ Human Resource ▪ Logistics/Supplies/commodities and Drugs ▪ Emergency Preparedness ▪ Infection Prevention ▪ Register, Manual, Checklists and Forms ▪ Training and services status of the providers of the facilities ▪ Service provisions and performances (last 3 months) Section-II: FP method specific observation checklists: ▪ Observation Checklist for Family Planning Counseling ▪ Observation Checklist for Clinical FP Methods (Tubectomy, NSV, lUD, Implant and lnjectable): checklists for both procedure and removal as appropriate ▪ Section-III is related to MCH services by component: ▪ Antenatal care ▪ Normal Labor, Childbirth & Immediate Newborn Care ▪ Postnatal Care “Nobody ever discussed with me the graduation criteria. My Mentor only discussed the gaps after assessment but I was not aware of the graduation. If I were clearly informed about the Mentorship process, I could do better.” – mentioned by a Mentee who recently graduated. Performance Evaluation of USAID’s Accelerating Universal Access to Family Planning (Shukhi Jibon) in Bangladesh | 42 As per protocol, the supervisors are expected to identify the problems, identify solutions and provide immediate feedback to the providers. The filled-in forms are supposed to be stored at the facility. However, evaluation team could only observe the observation checklists (Section 2 and 3) that were being used by the mentors. A government order for use of supervision checklists by FPCS-QITs was issued in July 2019. Since then a total of 1,070 supervisory visits took place in 32 districts (Table 6), which is equivalent to 11 visits per district per year. The majority of the visits were conducted by the DDFPs (4 visits per person per year) while FPCS-QITs, who are mainly responsible for supportive supervision, made only 0.4 visits per person per year. Maximum number of visits were at the UHFWC level (858 visits). All visits were accompanied by Shukhi Jibon staff who scanned the filled-in form and submitted to their head office. Use of these scanned forms in analytical decision-making process was not observed at the field level. Table 7 shows that out of 495 facilities that received at least one supervisory visit, 126 (25%) showed some kind of improvement. Out of the 126 facilities that were improved, 108 were UHFWCs. One of the major reasons for low progress include: Shukhi Jibon did not provide direct logistical support for improving facility readiness, rather mobilized GoB funds and advocated for ensuring supply of logistics, equipment, communication materials etc. Ensuring those through GoB channel is bureaucratic and time-consuming. For example, arranging a functional BP machine and weighing machine at a UH&FWC took six months. Also, there are data quality issues as stated by Shukhi Jibon staff member. Shukhi Jibon is continually advocating with FPCS-QIT and DGFP and such improvements are being documented through case studies and qualitative responses. Table 6: Distribution of Supportive Supervision visits by different level supervisors Performance Evaluation of USAID’s Accelerating Universal Access to Family Planning (Shukhi Jibon) in Bangladesh | 43 Table 7: Facility readiness improved through supervisory visits Type of facilities # of facility received supportive supervisory visit # of facilities improved 20 bed hospital 1 Community Clinic 2 District Hospital 8 4 MCWC 27 12 Medical College & Hospital 2 1 Satellite Clinic 11 Sub Center 3 UH&FWC 438 108 UHC 3 1 Grand Total 495 126 iii. Usefulness and sustainability: a) Benefits: All respondents recognized integrated mentorship and supervision as a unique approach for improving providers skills and thus overall improvement in quality of care. Participation in the process makes the mentees more confident and there is an improved rapport and trust between the mentor and mentee. Mentees also mentioned that they can seek support from the mentor whenever they face any problem or have a query regarding service delivery. This has also improved the referral system as mentors/supervisors are now more accessible to mentees. b) Challenges: The major challenge mentioned was scheduling sessions, as both mentors and mentees are busy with competing priorities. Drop-out of mentors and mentees is also disruptive. For external mentors and supervisors, long travel time was seen as a challenge for conducting regular sessions/visits. Both mentorship sessions and supportive supervisory visits were largely affected by the pandemic. c) Sustainability: The process is resource incentive requiring close support by Shukhi Jibon. The activity staff need to attend almost all mentorship sessions/visits to accompany the mentors/supervisors as well as to provide necessary support. Most of the respondents commented that although this is a unique approach, it would be difficult to sustain after the activity ends. For sustainability, it needs to be integrated within GOB’s existing supervision system with adequate resource allocation. Moreover, if the app is not aligned with DGFP systems, Performance Evaluation of USAID’s Accelerating Universal Access to Family Planning (Shukhi Jibon) in Bangladesh | 44 the efforts for developing the app will not sustain. If the app is only linked to DGFP, the intervention would be difficult to sustain in DGHS facilities. 4.2.3 Conclusion of Evaluation area 2 Over the past decade, the GOB has demonstrated its strong commitment to strengthening the capacity of health service providers. The 4th HPNSP Operation Plan of the GOB emphasizes the importance of strengthening the country’s supervision system. The 2015 Bangladesh Health Workforce Strategy also prioritizes strengthening supportive supervision to diversify and enhance skills and boost retention among members of the health workforce. Despite this progress, a critical gap remains. Efforts to strengthen the capacity of health service providers have traditionally emphasized training but provided little or no onsite post-training support. M&SS is a unique approach for the development and retention of FP provider skills as training alone is not enough to develop competencies to an expected level. Also, mentorship takes place in provider’s own service delivery environment where they get hands-on support and guidance from the mentor, which makes the provider more confident and competent. On the other hand, the supportive supervision approach replaces the traditional didactic supervision and helps problem solving for quality service delivery. Shukhi Jibon developed this international standard of intervention based on Pathfinder’s experience of implementing M&SS in different countries as well as in NGO service delivery program in Bangladesh. However, challenges of rolling out this intervention in public facilities were not anticipated at the time of intervention design and the intervention is suffering from high attrition (about 50%) of mentors and mentees, lack of compliance to the protocol (such as Root Cause Analysis and Smart Action Plan etc.), high dependence on Shukhi Jibon staff, poor documentation and low use of mentorship app. Coverage among different cadres of health and family planning providers is also low (0.13% to 27%). The training conducted by Shukhi Jibon was highly appreciated by the participants. However, the manuals were found to be complex in terms of language and content. Moreover, neither DGFP nor DGHS officials were involved during the manual development stage but only the M&SS working group at MOH&FW who reviewed and approved the manuals after development had representations from both the Directorates. Similarly, government officials were not engaged during development of the different job aids and there is a lack of ownership of the officials. The most challenging part is organizing mentorship sessions as both the mentors and mentees are busy with competing priorities. The intervention will only sustain with proper advocacy, government commitment and linking the intervention with existing GOB programs. Supportive supervision was also challenging as supervisors are reluctant to fill-in the checklists, document and use the findings for smart action plan and problem-solving approach. Organizing joint supervisory visit was also a major challenge for the activity. Despite all these challenges, the intervention was found to be highly effective in improving mentee’s performance which means performance of mentees who have gone through the process for a substantial period of time has increased significantly. The process makes the mentees competent and more confident, while building rapport with their mentors who are mostly their supervisors. The supportive supervision was another useful way to identify and resolve problems for delivering uninterrupted service, yet enough evidence was not available for a critical review of the approach. 4.2.4 Recommendations on Evaluation Area 2 a) Identify Operational Challenges: Integrated M&SS is a critical intervention that needs to be continued in the near future. However, the operational challenges need to be identified and resolved. The evaluation team noted a few challenges during the short review, although Shukhi Jibonmay organize consultation Performance Evaluation of USAID’s Accelerating Universal Access to Family Planning (Shukhi Jibon) in Bangladesh | 45 sessions/workshops involving national and sub-national GOB and NGO stakeholders. The workshop will identify the operational challenges as well as ways to overcome them. Engaging DGFP and DGHS in the process would help identify feasible solutions as well as develop ownership of GOB managers and providers. The following recommendations are to minimize or avoid a few operational challenges: ▪ In order to save time and resources, the training could be organized in 2 phases. A short training for all supervisors and then “training on mentorship” only for selected mentors who agreed to undertake the responsibility. ▪ Mentee selection criteria need to be set in consultation with the DGFP, “poor performers” should get preference and an informed consent process may be helpful. ▪ A brief, easy Bangla guideline needs to be developed for the mentees, followed by orientation on the process. ▪ Reasons for non-use of several job aids needs to be further explored and action needs to be taken accordingly. ▪ Instead of selecting 14 different cadres as mentees, intervention may focus on a few critical cadres who are responsible for clinical service delivery such as FWV, SACMO, Midwives, Nurses, Community Health Care Provider (CHCP) etc. This will also increase the coverage of specific cadres. ▪ Both the mentor and the mentee needs to be aware of the graduation criteria and learn to track progress. If the mentees are involved in their own success through a self-tracking process, they will be self-aware and responsible in their progress. ▪ The graduation criteria need to be revisited for developing a more realistic goal. Instead of graduating from the M&SS program in 3-4 years’ time, a mentee may graduate step by step such as counselling, OCP, IUD, ANC etc. A detailed plan for graduation to be developed at the beginning. Quarterly goals may be set. However, it should be communicated well with the mentors and mentees that the goal of this program is ‘improved performance’ and graduation is only an indicator for performance improvement. ▪ Ensure the use of dummies for clinical skills such as LAPM, normal labor etc. ▪ The mobile application needs to be hosted at the DGFP server and linked with other applications ▪ M&SS is seen as Shukhi Jibon’s intervention, but ownership of DGHS and DGFP is critical for smooth implementation of the intervention. Follow-up by departmental senior officials such as, Line Directors, Program Managers, DDFP may be helpful in reducing drop-outs and regular session conduction. ▪ USAID may undertake an operations research to identify feasible, acceptable and sustainable approach for mentorship and supportive supervision b) Document strong evidence: the current publications on M&SS are mostly based on qualitative information. The activity needs strong evidence (both qualitative and quantitative) on the effectiveness of the intervention and disseminate the findings among all relevant stakeholders including GOB and development partners. c) Coordination and collaboration: Shukhi Jibon needs to collaborate with the development partners (WHO, UNICEF, UNFPA) and jointly advocate for inclusion of M&SS in the upcoming sector plan. Also, the Shukhi Jibon activity should advocate and support GOB to develop a continuum of capacity development plan for health and family planning service providers in Bangladesh. Quality Improvement Secretariat, Health Economics Unit of MOHFW initiated development of “National Mentorship Strategy and Action Plan.” For sustainability, the activity needs to collaborate and link the M&SS approach with the national plan. 4.3 Evaluation Area 3: Interventions for Adolescents 4.3.1 Intervention description and findings The National Adolescent Health Strategy (NAHS) 2017-2030 of the DGFP) was developed to address the overall Performance Evaluation of USAID’s Accelerating Universal Access to Family Planning (Shukhi Jibon) in Bangladesh | 46 health needs of adolescents in Bangladesh by taking a broad and holistic understanding of the concept of health. The NAHS recognizes that adolescents lack the structural and social support to "develop their full potential" due to factors such as harmful social norms, poverty, and lack of education. It identified four priority thematic areas of intervention: adolescent sexual and reproductive health, violence against adolescents, adolescent nutrition and mental health of adolescents. The National Action Plan for Adolescent Health concedes the importance of using multiple platforms for reaching adolescents with information and services and as such has outlined 11 schools, 19 facilities and 20 community level activities. The Strategy document along with the Action Plan have identified a number of Ministries and the need for them to work closely for the effective implementation of the interventions on adolescents (Figure 13). Figure 13: Ministries mentioned in the NAHS and Action Plan One of the primary interventions identified in the strategy document and Action Plan was establishing a center for providing adolescent and youth friendly services (AYFS) at district and sub-district levels. This seemed pertinent as the Bangladesh Health Facility Survey 2017 showed that only 22% of health facilities meet the readiness criteria for family planning (FP) service provision. However, of those which are classified as FP ready, few can meet the special needs of adolescents. 4.3.2 Interventions at health facilities relating to Sub-IR 2.1 Prior to initiation of the Shukhi Jibon activity, the implementing partner had conducted a landscape analysis to help the activity align its activities with government priorities, as set out in the NAHS, address gaps and build on existing activities. The Shukhi Jibon activity implemented a constellation of activities in close collaboration with the MOHFW and other local partners to address adolescent SRHR, with the objective of creating an enabling environment at the facility and community levels and reaching married and unmarried adolescents and youth within public-sector health facilities. The interventions centered mostly around facilities and communities, and on schools in a limited scale. The activity has also implemented several strategies to engage adolescents and youth through digital platforms. It should be mentioned that an assessment of USAID’s A2H activity, that had a Ministry of Education inclusion of adolescent health issues in the formal school curriculum Ministry of Social Welfare: establishment of centers to meet the health needs of street children and juvenile delinquents Ministry of Women and Children Affairs: provision of services and skills training for young women and adolescent girls. Ministry of Local Government, Rural Development and Cooperatives: provision of adolescent health services in municipalities and all City Corporation areas; Management and Coordination Ministry of Youth and Sports Ministry of Law, Justice and Parliamentary Affairs Ministry of Disaster Management & Relief Ministry of Home Affairs Ministry of Religious Affairs Ministry of Labor and Employment Ministry of Information Ministry of Information and Communication Technology Performance Evaluation of USAID’s Accelerating Universal Access to Family Planning (Shukhi Jibon) in Bangladesh | 47 strong community linking mechanism, revealed that investing in making existing health facilities adolescent friendly is not necessarily associated with increased use of services. The report highlighted the need to better understand the health service needs of adolescents and to best serve those needs.4.3.2 Interventions at health facilities relating to Sub-IR 2.1 The activity Shukhi Jibon has aimed to enhance facility readiness of UH&FWCs, UHCs and MCWCs by providing training curricula, job aids, registers and other essential materials to selected facilities in activity supported sites. A total of 800 facilities had been identified of which 700 were completed. Additionally, the initiatives taken to ensure facility readiness for serving adolescent and youth included capacity building of service providers, in particular FWVs and SACMOs by providing a four-day training on AFHS plus another three-day training on counselling along with Mentoring and Supportive Supervision (M&SS). The Shukhi Jibon activity has promoted a ‘whole-site approach’ to create awareness on ASRHR among all facility staff including support and field staff of catchment areas. The activity used the following ten-point checklist (Figure-14) approved by DGFP for ensuring facility readiness for adolescents. Figure 14: Checklist to assess AYFS facility readiness based on WHO and DGFP guidelines: 4.3.3 Findings on the interventions at health facilities i. Client shortage: The adolescent corners at facilities are under-utilized, especially by boys as recorded in the registers kept at the health facilities visited by the evaluation team. Field observation coupled with KIIs and GIs Does the health facility have any service providers trained to provide AYFS? Does the health facility provide RH services, including FP, for A&Y clients free of cost? Does the health facility maintain both visual and auditory privacy for A&Y clients? Does the health facility provide gender-equitable services for A&Y clients? Does the health facility have any RH educational materials (pamphlets, booklets), posters, or job aids focusing specifically on A&Y? Does thehealthfacilityhave written guidelines or policies for providing services to A&Y? Does the health facility have provisions for sex- and age-disaggregated record keepingand reporting for services to A&Y? Does the health facility have an assigned service provider to manage the school healthprogram? Is a list of AYSRH services clearly displayed at this health facility? Is there a signboard or other indication to identify this health facility as A&Y-friendly? Performance Evaluation of USAID’s Accelerating Universal Access to Family Planning (Shukhi Jibon) in Bangladesh | 48 with service providers and adolescents point to a number of reasons explaining the low use rate. Firstly, the facilities are open from 9 am to 3:30 pm which is when school going adolescents are in school. Secondly, there seems to be a lack of promotion of these AYFS centers among adolescents – none of the 100+ adolescents interviewed by the evaluation team in Dhaka, Sylhet and Faridpur were aware of this service. Finally, an element of stigma is still attached with seeking SRH services; the respondents alluded to a preference for private providers when it comes to seeking SRH services by adolescents. Upgrading facilities with logistical support and trained service providers for making them adolescent friendly and functional takes time. COVID-19 hit in early 2020 just about a year after Shukhi Jibon activity was launched. Using HMIS data, we tried to gauge the change in the percentage of adolescents receiving counseling Figure 15: Rate of increase in counseling provided to adolescents between 2020 and 2022 Source: HMIS data, DGFP services at facilities in 2022 relative to 2020. For example, according to HMIS data, 5.4% of all eligible adolescents in the Barisal division were recorded to have received counseling in 2020. This figure had increased to 7.4% of adolescents in 2022 thus representing a 43% change in 2022 relative to 2020. Assuming that the rate of change in HMIS completeness and accuracy were uniform across all eight divisions over the study period 2020-2022, Figure 15 suggests that the increase in the use rate of facilities for counseling by adolescents was relatively higher in the non-intervention divisions of Barisal, Khulna, Rajshahi and Rangpur. This is possibly explained by a stronger linkage with the community in non-intervention areas or by cultural differences, among other factors. COVID related disruptions in implementing Shukhi Jibon interventions should be taken into consideration. Finally, the possibility of relatively better HMIS coverage and/or completeness in the non-intervention areas cannot be ruled out. Annex X shows the Annual number of counselling provided to adolescents ii. Product shortage and reasons for seeking healthcare: According to the registers where adolescent service statistics are recorded, the main reason for coming to these facilities is to seek treatment for general illnesses such as cold and cough. A limited number of female clients came in with menstruation and or UTI/STI issues or for collecting free sanitary napkins which are not always in stock. Service providers reported to the evaluation team that healthcare utilization will increase if a constant supply of products can be ensured. iii. Staff shortage: According to one FWV, “Adolescents comprise about 10% of my daily patients. I have to leave the waiting mothers in my office to walk to the adolescent corner to serve adolescents which is difficult. It will be useful to have more staff’. The SACMOs at the health facilities are predominantly male and as such not best placed to serve adolescent girls. iv. The 10-point checklist used by the Shukhi Jibon for assessing facility readiness was found to be inconsistent Performance Evaluation of USAID’s Accelerating Universal Access to Family Planning (Shukhi Jibon) in Bangladesh | 49 during field visits and in certain cases impractical. While some of the criteria for readiness using the checklist were universally followed, others varied from facility to facility. These include visual and auditory privacy, list of AYFS clearly displayed, provision of gender-equitable services and assigned manager to manage school health program. v. Adolescent service register: The registers provided by Shukhi Jibon at the facilities is the only means for recording adolescent service statistics while DGFP is still developing its electronic MIS to capture age and sex specific data. 4.3.4 Community Interventions for Adolescents and Youth relating to Sub-IR 2.4 At the community level, the ShukhiJibon activity has focused primarily on enhancing frontline health worker’s capacity of both DGFP and DGHS through orientation sessions on AYFHS. A total of 3,140 field staff have been oriented as of December 2022. Shukhi Jibon has supported community mobilization, engagement, and outreach by supporting DGFP to develop courtyard guidelines to assist Family Planning Inspectors (FPIs) and Family Welfare Assistants (FWAs) to reach the target audience. Moreover, the activity aimed to connect gatekeepers and community leaders through various committees. Community engagement sessions were arranged with Satellite Clinic committee members, UH&FWC committee members, Adolescent and Youth committee members, Union Parishad members, religious leaders, and DGFP and DGHS frontline service providers. In addition, Shukhi Jibon has promoted the newlywed intervention which is DGFP’s pilot intervention to delay first pregnancy and encourage healthy timing and spacing for newlyweds. The intervention engages religious and community leaders with FWAs and FPIs to provide an information box to newlywed couples to inform and encourage them on the use of family planning from the very beginning. While this intervention works as a Achievements of Shukhi Jibon interventions at health facilities • The Shukhi Jibon activity’s whole site approach is to be adopted in the Operational Plan • The Shukhi Jibon activity provided a register at each facility to record adolescent service statistics while DGFP is still developing its electronic MIS to capture age and sex specific data. That said, how completely and accurately these are filled out is at the FWV/SACMO’s discretion. The activity so far has ensured facility readiness of 700 out of the 800 facilities identified Challenges • There were staff shortages at the health facilities There were client shortages (low use rate) for which Shukhi Jibon strengthened its community activities What could be done differently? In addition to aligning its activities to government priorities, the Shukhi Jibon could have adopted innovative ways of supply and demand creation like other development partners do such as engaging adolescents through sports or at schools. Given the sensitive nature of the topic of ASRH, greater attention is needed to ensure confidentiality and privacy of adolescent clients. Performance Evaluation of USAID’s Accelerating Universal Access to Family Planning (Shukhi Jibon) in Bangladesh | 50 bridging point between community leaders and service providers, it also underscores the importance of preventing child marriage and the negative impact of early and unplanned pregnancy. Finally, the activity has had several activities in place for reaching underserved adolescent and youth groups such as transgender and those living in climate-prone areas. The activity has been supported by six local NGOs on the community level activities for adolescents in 15 outof the total 32 intervention districts since 2021. The initial Shukhi Jibon activities for adolescents were largely facility centered. In Year 3, Shukhi Jibon launched a grants program by engaging six local NGOs at the district level to reach adolescents and youth with sexual and reproductive health information and services that complement the public sector services. The grants aimed to contribute to a healthy social environment for rights￾based SRH information and services for young people. The activities were designed to be gender sensitive and to support ongoing Shukhi Jibon AYSRH programs or, embark on new and innovative initiatives, and look for integration with digital technology-based activities. The approaches were encouraged to be inclusive and to integrate diverse populations such as transgender communities, adolescents in urban slums, out of school and working adolescents, and those living in climate￾prone or hard to reach areas. The active participation period of the NGOs, namely, FPAB, SERAC, Lighthouse, ESDO, PHD and YPSA started in May/June of 2021 and is expected to continue till May 2023. The main activities of these sub-grantees were community based as seen in Table-8 The NGO SERAC has an additional activity of supporting adolescent friendly health corners with 40 youth peer volunteers. This is an activity that is to be adopted in the next sector program, as confirmed in interviews with DGFP. According to HMIS data, the number of services provided to adolescents on IFA distribution and counseling and on RTI/STI treatment and counseling are generally higher in the 15 NGO districts than in the rest of Shukhi Jibon intervention areas, possibly attributable to their innovative activities such as deploying peer educators at facilities. The trends observed in Figure 16 should be treated with caution as they may have been influenced by district-level differences in coverage and completeness of HMIS service statistics. Figure 16: Number of adolescent treatment and counselling services in NGO and non-NGO areas Performance Evaluation of USAID’s Accelerating Universal Access to Family Planning (Shukhi Jibon) in Bangladesh | 51 Source: HMIS Table 8: Primary Activities of the 6 NGO sub-grantees NGO name Community School Digital Facility PHD (Dhaka, Sylhet) -hard to reach adolescents in haor and tea-gardens -first time parents and newlyweds Works with 56 schools in 8 upazilas on CSE (after Covid) Tele-medicine in partnership with DGFP in hard to reach areas FPAB (Dhaka, Chattogram, Sylhet) -activities in Rangamati and with transgender -youth center ‘Tarar Mela’ CSE in selected schools ESDO (Mymensingh) -Orient gatekeepers on SRH -serve adolescents in disaster prone areas through satellite clinics YSPA (Chattogram) -Hard to reach areas in Chattogram Hill Tracts -Engage adolescents and gatekeepers at community level -Peer educator scheme on SRH Lighthouse (Dhaka) -training for journalists and imams - peer volunteers conduct session with school and out of school adolescents SERAC(Mymen singh, Dhaka) -advocacy meetings with DGFP & DGHS at community Support adolescent friendly health corners with 40 youth peer volunteers 2) Underserved A total of 45,921 members of marginalized communities in hard to reach areas have been reached by these NGOs so far. Among all the underserved groups, the activity, with the help of NGO partners, has been able to reach 20,725 adolescents in the Chattogram Hill Tracts so far followed by 11,605 adolescents in climate vulnerable areas. Figure 17 shows the number of underserved adolescents reached, broken down by population category. Performance Evaluation of USAID’s Accelerating Universal Access to Family Planning (Shukhi Jibon) in Bangladesh | 52 Figure 17: Breakdown of hard to reach groups reached by NGO sub-grantees 4.3.5 Findings on community level interventions 1. COVID-19 has been a big challenge when community activities were disrupted or even stopped. The six partner NGOs were brought on board in Year 3 of the activity (2021) to supplement Shukhi Jibon’s community activities. 2. Shukhi Jibon had relied on courtyard sessions in the 17 non-NGO intervention districts. Participation of the adolescents in these courtyard sessions is very low. This is partly due to the timing of these sessions, which coincide with school hours, and partly due to adolescents’ reluctance to attend these sessions that are attended by people of all ages, as reported by FPIs and FWAs. Moreover, monitoring mechanism or detailed information on courtyard session are not available for validation. 3. The courtyard guidelines developed by Shukhi Jibon are not very practical, for example, separate sessions cannot be held for sub-groups such as adolescents’ boys or unmarried girls where topics relevant to these groups can be discussed. 4. The courtyard sessions, however, are helpful for engaging gatekeepers of adolescents and community leaders; they also continue to be a first contact point for pregnant or newly married adolescents. Performance Evaluation of USAID’s Accelerating Universal Access to Family Planning (Shukhi Jibon) in Bangladesh | 53 4.3.6 Interventions at the school level and using digital platforms The school centere interventions were initiated in Year 2 with activities being undertaken with the DGFP, DGHS and Department of Secondary and Higher Education (DSHE). The main school-based activity has been training of SACMOs on adolescent SRHR who have been assigned for school health support. For DGHS, one of the partner NGOs FPAB has offered training to 24 teachers in 12 schools in Moulavibazar to enable them to teach the school curriculum on sex education. The DGHS is expected to expand the training program for school teachers in the next sector program, as outlined in the Maternal, Newborn, Child and Adolescent Health (MNC&AH) Operational Plan. The Shukhi Jibon activity has developed various adolescent responsive manuals for parents, gatekeepers and service providers. The Shukhi Jibon activity with DSHE has been establishing virtual SRH corners via Konnect platform which are resources for virtual classrooms based on textbooks. Shukhi Jibon provided technical assistance to IEM for continued education for call center officers and providers in delivering AYFS support through virtual platforms. Shukhi Jibon further promoted the development of a website and a connected Facebook platform developed by Aspire to Innovate (a2i) to create a connection between student and teachers 7 . 7 Shuki Jibon Year 4 Annual Report: September 2022 Challenges • Outreach and courtyard sessions were disrupted during the COVID-19 pandemic. Given the national and global COVID-related restrictions which came into place in 2020, just a year after the Shukhi Jibon activity started, implementation of interventions were severely delayed or even halted. Thus, the pandemic has been a huge impediment to the effective implementation of the activity during the initial years. • The courtyard guidelines for FPI/FWA, although developed by Shukhi Jibon in partnership with national Working Groups, were not very practical (e.g. the guideline to conduct separate meetings for boys and girls and for married and unmarried groups in courtyard sessions). Outreach and courtyard activities are not very effective for reaching school adolescents. This is attributable to timing that clashes with school timing and the fact that courtyard sessions include participants of all ages from the community, especially married females. Community workers reported to the evaluation team that adolescents are not comfortable being in the presence of adults when sensitive topics such as ASRH are being discussed. Achievements of Shukhi Jibon interventions at community level Courtyard sessions are very helpful for engaging gatekeepers of adolescents and community leaders, and for reaching married adolescents. What could be done differently? The separate sessions with boys and girls could have been done in schools Performance Evaluation of USAID’s Accelerating Universal Access to Family Planning (Shukhi Jibon) in Bangladesh | 54 4.3.7 Findings on Shukhi Jibon school-level interventions 1. Adolescent knowledge of reproductive health has increased as reported by adolescents and service providers interviewed by the evaluation team. This is predominantly due to increased access to digital platforms, as reported by the respondents. 2. There seems to have been minimal school-level interventions by Shukhi Jibon - monitoring of the SACMO’s school sessions only started this year, resulting in no data being available to assess the impact of providing close to 1,300 training sessions to SACMOs. The SACMOs are mostly male, and as such, involving SACMOs in school health programs in girls’ schools may not achieve maximum effectiveness. Special attention is also required for conducting health programs in religious institutes. 3. The students in our GIs who did receive school sessions from SACMOs or NGO partners claimed that they did not have sufficient information on RH, despite having received sessions. This underscores the need for repeated interaction with students. What evidence is there that these Shukhi Jibon interventions have improved adolescents’ health knowledge and/or care-seeking behavior? Ideally this question should be answered by doing pre and post-activity assessments or by comparing intervention and comparison data which were beyond the scope of this assessment. Based on our KIIs and GIs with service providers, activity implementers and adolescents, digital sources have significantly improved adolescents’ health knowledge. This is especially true for boys, the vast majority of whom have access to digital sources. However, this increase in adolescents’ health knowledge is not necessarily due to Shukhi Jibon work. None of the adolescents whom we interviewed had knowledge of any of the digital activities of the Shukhi Jibon activity. The registers with records of adolescent service statistics that are maintained at health facilities supported by the activity suggest that girls, albeit limited in numbers, are coming to facilities for sanitary napkins or to seek treatment for menstruation issues or RTI/STI. The main reason for visits by adolescents to these facilities however continues to be for seeking treatment for general illnesses such as cough and cold. Service providers often use this opportunity to provide counseling on AYFS or distribute iron and folic acid (IFA). This slow growing trend among girls to visit health centers could be at least partly due to counseling and community activities of Shukhi Jibon. On the other hand, adolescent boys visiting health facilities for seeking SRH is yet to become a practice. According to the first national representative adolescent survey in the country, 50% of unmarried female adolescents had sought any kind of healthcare during the six months preceding the survey and among them, less than ten percent had sought care for reproductive health issues (BAHWS 2019-20). Almost all the male respondents in the survey reported seeking care for general illnesses only. The vast majority of adolescents of both sexes in the survey reported to have sought care from a private provider. 4.3.8 Conclusions of Evaluation Area 3 USAID Bangladesh has been among the top development partners supporting the national adolescent health program in the country. Some of the successes of Shukhi Jibon activity have been working closely with the MOHFW to ensure readiness of health facilities in 32 low performing districts in 4 eastern Divisions - 700 out of 800 identified facilities have been upgraded to provide adolescent friendly health services. The activity’s community level activities include providing orientation sessions to DGFP and DGHS frontline works with 3,140 Performance Evaluation of USAID’s Accelerating Universal Access to Family Planning (Shukhi Jibon) in Bangladesh | 55 field staff having been oriented so far. The six local NGOs in 15 out of the 32 total intervention districts have provided support through innovative activities having reached many of the underserved communities such as transgender, adolescents living in CHT, tea-gardens or climate-vulnerable areas. The use of peer educators at facilities and the whole-site approach of sensitizing support staff are among the activities that are to be expanded or adopted in the next sector program. For the whole-site approach to be effective, it is necessary to develop a strong linkage with community and schools for reaching adolescents. COVID-19 was a big impediment to the effective implementation of activity activities, as were staff shortages/vacancies at the health centers. Behavioral change takes time and effort – the use rate of health facilities among adolescents continues to be low due to socio-cultural reasons. In addition to aligning its activities to government priorities, Shukhi Jibon could have adopted innovative ways of supply and demand creation like other DPs do, for example, engaging adolescents through sports or through activities in schools. Perhaps the missed opportunity was not having had more school level interventions when the secondary school enrolment rate in the country is over 75% (World Bank, 2021).8 At the initial design and planning phase, the Shukhi Jibon program would have benefited from including more school-based activities rather than relying solely on community interventions for demand generation. One of the findings from the assessment of the previous iteration of the USAID funded adolescent program in the country, Advancing Adolescent Health (2016- 2019), was that reaching adolescents through facilities is not very effective. The National Action Plan for Adolescent Health recognized the importance of using multiple platforms for reaching adolescents, having outlined 11 schools, 19 facilities and 20 community level activities. 4.3.9 Recommendations on Evaluation Area 3 ▪ Separate and targeted interventions are needed for specific adolescent groups - unmarried and married, girls and boys, and the hard to reach, using a combination of platforms. Boys may be best reached using digital media as they are unlikely to visit public-sector health facilities for information and services. Married adolescents and non-school going or hard to reach adolescents are best positioned to be reached through innovative community programs and referrals to facilities. ▪ In a country with a 75% secondary school enrolment rate, emphasis should be placed on school-based interventions such as a full-time counselor for reaching large numbers of unmarried adolescents. The training of teachers and students to act as peer educators in the school program of DGHS is to be expanded in the next sector program. ▪ Health facilities and schools should have a full-time dedicated peer- educator/counselor for providing information and referral services to adolescent clients. There is ample evidence from India and Africa that peer-led programs in schools and health facilities improve knowledge on ASHR.9 8 Retrieved from: https://data.worldbank.org/indicator/SE.SEC.ENRR?locations=BD 9 https://www.annalsofglobalhealth.org/articles/10.5334/aogh.2791/print/ https://www.childtrends.org/publications/school-based-strategies-support-adolescent-sexual-reproductive-health https://rutgers.international/news/peer-education-widely-used-but-wildly-undervalued/ https://reproductive-health-journal.biomedcentral.com/articles/10.1186/s12978-021-01251-3 Performance Evaluation of USAID’s Accelerating Universal Access to Family Planning (Shukhi Jibon) in Bangladesh | 56 ▪ The Shukhi Jibon activity had various components. The earlier iterations of the program had focused primarily on strengthening capacity of training institutes and of health service providers. As the adolescent component was new in the current phase, more time and innovative efforts are needed for the adolescent activities to have substantive impact. Adopting a multi-sectoral approach involving other Ministries such as MoE and MOWCA, as envisioned in the National Action Plan for Adolescent Health will help. Shukhi Jibon’s collaboration with the government sector has been limited to DGFP, DGHS and DSHE. 5. CONCLUSION AND RECOMMENDATIONS Conclusion In last 5 years Shukhi Jibon has accomplished a substantial amount of activities and tasks related to Competency-Based Training, Mentorship and Supportive Supervision and Adolescent Sexual and Reproductive Health Interventions. The Evaluation Team conducted rigorous review of documents, program data and conducted KIIs and GIs with government stakeholders, implementing NGOs, USAID and Shukhi Jibon staff members, service providers and adolescents. The findings were grouped into 3 evaluation areas and detailed recommendations are included in each section. This section summarizes overall feedback on Shukhi Jibon program operation and a few take away recommendations against each evaluation area: Evaluation Area 1: Strengthening Capacity of Training Institutes: The CBT intervention developed an international standard training manual, job aids and imparted trainings to a huge number of participants from NIPORT, DGFP and DGHS. Shukhi Jibon successfully on-boarded all selected training institutes through structured LoC and MoU. The Evaluation Team raised concerns regarding selection of FPCS-QIT’s offices as training centers and investing for their development which is a clear duplication of efforts while there are NIPORT training centers at the same district/location. The training need assessment was conducted thoroughly that identified the training needs and bottlenecks. Shukhi Jibon recently conducted an endline training need assessment which may be an important tool to demonstrate progress as well as for future decision-making process. CBT was found to be an acceptable, appropriate and feasible intervention that has already been integrated into NIPORT’s training system and to some extent DGFP’s training packages. Moreover, Shukhi Jibon provided mentoring and follow-up support to the trainers through TCBOs who are designated for CBT implementation at the field level. TMIS was found to be a useful tool for record-keeping and reporting and Shukhi Jibon trained a good number of staff on TMIS, it is only used by NIPORT as DGFP is yet to get access. However, as TMIS is located at National Data Centre at the Computer Council and managed by NIPORT, the Evaluation Team expects this intervention to sustain in future. The concept of TMIF is excellent as it aims quality improvement of training institutes through a structured mechanism of SWOT analysis and action plan development, only a few recommendations were implanted by the training institutes. Also, not all staff members of the institute were aware of the process and the action plans Performance Evaluation of USAID’s Accelerating Universal Access to Family Planning (Shukhi Jibon) in Bangladesh | 57 Evaluation Area 2: Mentorship and Supportive Supervision M&SS is an innovative and unique approach for improving FP providers skills as training alone is not enough to develop and retain competencies to an expected level. Also, in case of clinical service providers, mentorship gives opportunity for hands on training, demonstration and facilitates learning in the providers own service delivery environment. This intervention also developed international standard manuals and job aids to facilitate the implementation. The intervention, when implemented properly has demonstrated measurable outcomes in terms of providers knowledge and skill building. However, the evaluation team identified a few operational challenges that has been detailed out in section 4.2.2. Evaluation Area 3: Interventions for Adolescents Despite concerted efforts by the government and development partners to reach Bangladeshi adolescents with ASRH services, progress has been limited. This is due to both demand side (cultural norms and traditions that are hard to break) and supply side (limited resources for service provision) factors. The evaluation team assessed the interventions for adolescents at health facilities, school level and community level. The interventions at the health facilities identified client shortage, product shortage, staff shortage influenced the AFHS as well as adolescent’s health seeking behavior. The 10-point checklist used by Shukhi Jibon for assessing facility readiness was found to be inconsistent and to some extent impractical. However, Shukhi Jibon’s whole site approach was found useful and to be adopted in the operation plan. Shukhi Jibon provided a register at each facility to record adolescent service information, however, the registers were not duly filled-in. However, DGFP is developing electronic MIS to capture these information Community interventions are mostly implemented by the partner NGOs that cover limited areas of 17 intervention districts. In non-NGO areas, the activity had relied on courtyard sessions and developed a guideline to conduct these sessions. The guideline was identified as impractical by the community level service providers during our interviews. The courtyard sessions are helpful for engaging the gatekeepers of adolescents and community leaders as well as for some married adolescents. There seems to have been a minimal school-based intervention by Shukhi Jibon and few of them are imparted by the NGOs. In both NGO intervention areas and non-NGO areas, SACMOs have been trained to conduct school￾based sessions. However, due to lack of monitoring mechanism, the efficiency of the intervention was hard to assess. However, KIIs and GIs revealed that most of the SACMOs are males and involving SACMOs in school health programs in girl’s school may not be effective. The girls and boys in intervention area schools, who we interviewed did not have sufficient information on reproductive health, despite having received sessions. Performance Evaluation of USAID’s Accelerating Universal Access to Family Planning (Shukhi Jibon) in Bangladesh | 58 Recommendations Evaluation Area 1: Strengthening Capacity of Training Institutes: ▪ The evaluation team recommends to facilitate collaboration between NIPORT and DGFP and strengthen NIPORT facilities for both clinical and non-clinical training through active engagement of FPCS-QITs in the process ▪ The endline TNA should be carefully reviewed by Shukhi Jibon as well as by NIPORT and DGFP to identify progress, bottlenecks and future actions. Once identified, Shukhi Jibon need to support government to develop a long-term training plan as well as advocate for inclusion of relevant activities in next sector plan ▪ Shukhi Jibon developed divisional action plans through rigorous consultative process however, systematic and structured monitoring plan to be developed in collaboration with NIPORT and DGFP ▪ A huge number of different categories of participants were trained on CBT, however, selection of potential trainers who would be able to apply CBT approach could save resource and increase efficiency of the training. ▪ Though TCBOs were highly appreciated by the trainers, Shukhi Jibon may gradually reduce their dependence on TCBOs and hand-over the work to NIPORT and DGFP staff members ▪ Shukhi Jibon need to follow-up with DGFP to provide access of TMIS to DGFP staff and build capacities of relevant staff ▪ The evaluation team found the action plans need further refinement of TMIF action plans and active monitoring and follow-up mechanism with an accountability framework to be developed in consultation with the respective training institutes Evaluation Area 2: Mentorship and Supportive Supervision Identify the root cause of the operational challenges and their solutions in consultation with the mentors, mentees and DGHS, DGFP officials. The process will help to develop ownership of the government officials Shukhi Jibon need to document strong evidence and disseminate the findings among relevant stakeholders. It has been observed that the activity database has a lot of quantitative information, however, the Shukhi Jibon publications mostly captured qualitative data which may be less convincing to some audience. Shukhi Jibon may consider writing a scientific publication on M&SS implementation experience Evaluation team observed lack of collaboration and coordination with GOB and development partners in this regard. The activity needs to effectively collaborate with QIS, DGHS and DGFP. Shukhi Jibon also need to collaborate with WHO, UNICEF, UNFPA and other development partners and advocate with government for inclusion of mentorship and supportive supervision in next sector plan with adequate resource allocation. Evaluation Area 3: Interventions for Adolescents A better understanding of the barriers to accessing healthcare is much needed, especially in the low-performing districts. The take-away message from this assessment is that one size does not fit all. In order to reach Performance Evaluation of USAID’s Accelerating Universal Access to Family Planning (Shukhi Jibon) in Bangladesh | 59 adolescents effectively with reproductive health information and services, separate and targeted interventions are needed for specific adolescent groups such as unmarried and married adolescents, girls and boys, and the hard to reach. A combination of platforms needs to be utilized depending on the target group as opposed to relying on only health facilities or community outreach. There is ample global evidence that peer-led programs in schools and at health facilities help improve knowledge on ASRH. Thus far Shukhi Jibon’s interventions at the school level have been rather limited when the national secondary school enrolment rate is 75%. Moving forward, a stand-alone ASRH program as opposed to one integrated in a bigger activity, coupled with a multi￾sectoral approach involving other Ministries such as MoE and MOWCA may be more effective. Performance Evaluation of USAID’s Accelerating Universal Access to Family Planning (Shukhi Jibon) in Bangladesh | 60 ANNEXES Annex I: Shukhi Jibon Intervention Districts District Count of Upazila Bandarban 8 Brahmanbaria 10 Chandpur 9 Chattogram 17 Cox's Bazar 8 Cumilla 17 Dhaka 7 Faridpur 10 Feni 6 Gazipur 6 Gopalganj 5 Habiganj 8 Jamalpur 7 Khagrachhari 8 Kishoreganj 13 Lakshmipur 5 Madaripur 4 Manikganj 8 Maulvibazar 7 Munshiganj 6 Mymensingh 12 Narayanganj 5 Narshingdi 6 Netrokona 10 Noakhali 10 Rajbari 5 Rangamati 10 Shariatpur 6 Sherpur 5 Sunamganj 11 Sylhet 13 Tangail 11 Grand Total 273 Performance Evaluation of USAID’s Accelerating Universal Access to Family Planning (Shukhi Jibon) in Bangladesh | 61 Annex II: Result Framework Performance Evaluation of USAID’s Accelerating Universal Access to Family Planning (Shukhi Jibon) in Bangladesh | 62 Annex III: Guidelines and tools for evaluation 1. CBT, TMIS and TMIF (Please note: The guideline is to shape the conversations. The questions will not necessarily be asked directly, but the respondent will be invited to tell of their experiences, guided by these topics. The interviewer will use their discretion to decide the order and level of detail in which to cover each topic with each informant.) Name Workstation Designation Date 1. Key-informant interview guides: CBT interventions Can I confirm that you are aware of the use of AUAFP CBT interventions? Adoption Development of the intervention package 1) What do you do to make for CBT implementation process in Shukhi jibon for competency-based training and use of adult learning approaches? Please explain. (action) What types of key steps were taken to develop this ‘AUAFP CBT interventions’ in different stages (Probe: Situation analysis, Development, Implementation, Adoption, Recognition?) (adherence) 2) What does the CBT intervention package include? Input: What kind of intervention input has been included in ‘CBT intervention’ to integrate Competency-based Training (CBT) in Training System for improving quality of training? Please elaborate. (Probe: Conduction of Training Needs Assessments, Organized ToTs, Advocacy to integrate CBT in the training system for sustainability and next health sector program, advocated to use standard guidelines to identify and prepare practicum sites, Developing e-learning courses in on CBT for trainers, Training Capacity Building Officers to the trainers and training institutes, Trained trainers' observation using standard checklist to ensure they followed CBT approaches and provided feedback to improve) Whom do you provide intervention? Why? (action) (Probe: Trainers from DGFP, NIPORT and DGHS, Training Institutes (Tis) created trainers pool) When and how these steps implemented? Please elaborate. What was your role in this process? (adherence) Why do you think the steps were necessary? (advantageous) Materials 3. What type of materials are used for the intervention? Please elaborate. (Probe: Curriculums, manuals, guidelines, modules, handbooks, e-learning contents, handout, job aids, leaflets, posters, stories booklet, video etc.) Which one do you think is easy to use and which one is difficult? Training/orientation/workshop/meeting 1. Has the intervention provided any training/orientation/workshop/meeting on how to make or use competency-based training and adult learning approaches? Performance Evaluation of USAID’s Accelerating Universal Access to Family Planning (Shukhi Jibon) in Bangladesh | 63 Probe: what training/orientation/workshop/meeting was provided; who provided the training/orientation/workshop/meeting; when; was the training/orientation/workshop/meeting adequate? Was the training/orientation workshop/meeting useful? How can the training/orientation/workshop/meeting be improved? (Practical, advantageous) Changes/differences (now and before) 2. What are the changes/differences do you feel the way intervention are providing information now and before? (intention & action) - (Probe: used training need assessment report, Tis created trainers pool with trained trainers and engaged them in training conduction for service provider, DGFP integrated CBT in LARC&PM manual/in PPFP manual, integrate CBT in 16 TIs and use CBT Trainers’ manual as reference manual, developed and used posters to promote CBT, TI’s used standard guidelines to identify and prepare practicum sites, TIs changed training room seating arrangement to allow CBT based training conduction, eLearning courses are being developed for trainers as part of source of continuous learning for them for sustainability purpose, trained trainers observed) Acceptability, Feasibility and Sustainability 3. How easy do you think to inform beneficiaries of intervention about this? (Practical) 4. How do you think this intervention will have impact in raising competency-based training and adult learning approaches? (Probe: agreeable/suitable/sustainability) 5. Are the trainers/trainees interested about this intervention components? What kind of component do they want to know in this regard? (Practical) Probe: (practicability) 6. How this intervention can be more easily accessed by beneficiaries? (Feasibility) 7. What are the demonstrable active usages of the CBT across learning lab sites? 8. How was the setting to build provider competency approach at the national, divisional, district and upazila levels operationalized? 9. How was the cooperation across DGFP, DGHS, and NIPORT in operationalization, usability, and quality assurance? 10. Are there any enhance steps for governmental commitments (Extent of activity moving forward legal, policy and regulatory processes and standards) 11. How do you think it will sustain in future? Are there any logical connections with current HPNSP, PIP, Operational Plan, Training plan, donor support or commitment? 12. At this moment, you are implementing this CBT intervention, do you think this is appropriate or do you have any suggestion how to implement this better? 13. How do you think this intervention will have impact in competency-based training and use of adult learning approaches? (agreeable/suitable/sustainability) 14. Are the service providers or trainers interested to know this intervention? What kind of information do they want to know in this regard? (Practical) Probe: (practicability) 15. How well does the intervention align with government priorities? 16. Does the program represent a legitimate role for government? 17. To what extent is the program interventions producing worthwhile results (outputs, outcomes) and/or meeting each of its objectives? Performance Evaluation of USAID’s Accelerating Universal Access to Family Planning (Shukhi Jibon) in Bangladesh | 64 18. Overall, how effective was the intervention in assisting FP service providers to improve their skills? 19. How likely are you to recommend AUAFP CBT intervention to a friend? Barriers and Enablers Health care Providers/Trainers 20. How appropriate is the intervention for improving skills of health service providers and trainers? 21. Will the interventions be able to create any advantage to the health care providers and trainers in improving skills to provide FP services? 22. How easy do you think to inform trainers about this? (Practical) 23. What are the positive sides of using this intervention? Please elaborate. Probe: easy to delivery and understand 24. What challenges do you face in using this intervention? How did you overcome this challenge in different phases? Give an example. (Probe: Situation analysis, Development, Implementation, Adoption, Recognition phases etc.) 25. Which materials/job-aid is the easiest to explain and use? 26. Which materials/job aid do you think is difficult to explain and use? Why? 27. How this intervention is useful for beneficiaries, what do you think? 28. What kind of changes do you think will be helpful to provide this intervention to the beneficiaries? Please explain. Training Institutes: 29. How suitable are the ‘CBT intervention’ in educating training institute manager and faculty for competency-based training and use of adult learning approaches? ● Is there enough information, or could more information be included? ● Are the materials easy to understand? Could more materials be developed? Please elaborate. 30. What are the other steps should be taken to aware the faculty of training institutions? 31. Do you have any suggestions on how these interventions could be used to trainers in an easier way? 32. What are the challenges do you think the institute or trainers might face while using the intervention? How those challenges can be overcome? 33. Do you think it is necessary to take any steps for the training institutes about how to use the ‘CBT intervention’? If yes, is there any arrangement of it? Please explain. 34. What kind of steps could be taken to encourage the trainers about using ‘CBT intervention’? Please explain it. If they have already taken any steps, please try to know in detail. 2. Key-informant interview guides: TMIS interventions Can I confirm that you are aware of the use of AUAFP TMIS interventions? Adoption Development of the intervention package 1. What do you do to make Shukhi Jibon’s approach to developing a TMIS? Please explain. (action) What types of key steps were taken to develop this ‘AUAFP TMIS intervention’ for training institute systems strengthening, and certification? (adherence) Performance Evaluation of USAID’s Accelerating Universal Access to Family Planning (Shukhi Jibon) in Bangladesh | 65 2. What does the TMIS intervention package include? Input: What kind of intervention input has been included in ‘TMIS intervention’ for a systematic management information system to support the monitoring of its training system? Please elaborate. (Probe: Landscape analysis on TMIS/HRIS, needs based updating of the TMIS, development of dashboard, Training for capacity building of NIPORT staff for operationalizing TMIS, Technical Assistance to use, maintain and data driven decision making, Advocating for sustainability of TMIS and inclusion in the next health sector program) Whom do you provide different kind of interventions? Why? (action) When and how these interventions implemented? Please elaborate. What was AUAFP role in this process? (adherence) Why do you think the steps were necessary? (advantageous) 3. What are the achievements in operationalization on Training Management Information System (TMIS) for improving training data management? Materials 4. What type of materials are used for the intervention? Please elaborate. Which one do you think is easy to use and which one is difficult? Training/orientation/workshop/meeting 5. Has the intervention provided any training/orientation/workshop/meeting on how to make or use for training institute systems strengthening? Probe: what training/orientation/ workshop /meeting was provided; who provided the training/orientation/workshop/meeting; when; was the training/orientation/workshop/meeting adequate? Was the training/orientation workshop/ meeting useful? How can the training/orientation/workshop/meeting be improved? (Practical, advantageous) Changes/differences (now and before) 1. What are the changes/differences do you feel the way intervention are providing information now and before? (intention & action) Probe: TMIS was updated as per needs of NIPORT Tis, NIPORT trained staff use TMIS, NIPORT Tis used TMIS to improve training management efficiency (avoid overlapping or repeat of trainees), reduced overlapping/repeat training by service providers, TMIS rolled out in non-Shukhi Jibon areas, Access of TMIS to DDFP, NIPORT and DGFP utilize training data, NIPORT used own fund for hosting TMIS at National data center of Bangladesh Computer Council) Acceptability, Feasibility and Sustainability 6. How easy do you think to inform beneficiaries of intervention about the TMIS? (Practical) 7. How do you think this intervention will have impact in raising training institute systems strengthening? (Probe: agreeable/suitable/sustainability) 8. Are the training institutes/trainers interested about this intervention components? What kind of component do they want to know in this regard? (Practical) Probe: (please indicate practicability) 9. How this intervention can be more easily accessed by beneficiaries? (Feasibility) 10. What are the demonstrable active usages of the TMIS across FP system? Performance Evaluation of USAID’s Accelerating Universal Access to Family Planning (Shukhi Jibon) in Bangladesh | 66 11. How was the setting to build competent training institute at the national, divisional, district and upazila levels operationalized? 12. How was the cooperation across DGFP, DGHS, and NIPORT in TMIS operationalization, usability, and quality assurance? 13. Are there any enhance steps for governmental commitments (Extent of activity moving forward legal, policy and regulatory processes and standards) 14. How do you think it will sustain in future? Are there any logical connections with current HPNSP, PIP, Operational Plan, Training plan, donor support or commitment? 15. At this moment, you are implementing this TMIS intervention, do you think this is appropriate or do you have any suggestion how to implement this better? 16. How do you think this intervention will have impact in strengthening training system? (agreeable/suitable/sustainability) 17. Are the service providers or trainers interested to know this intervention? What kind of information do they want to know in this regard? (Practical) 18. How well does the intervention align with government priorities? 19. Does the program represent a legitimate role for government? 20. To what extent is the program interventions producing worthwhile results (outputs, outcomes) and/or meeting each of its objectives? 21. Overall, how effective was the intervention in assisting FP service providers to improve their skills? 22. How likely are you to recommend AUAFP TMIS intervention to a friend? Barriers and Enablers 23. How appropriate is the TMIS intervention for systematic management information system to support the monitoring of its training system? 24. Will the interventions be able to create any advantage to the training institutes in improving management information system for monitoring of its training system? 25. How easy do you think to inform training institutes about this? (Practical) 26. What are the positive sides of using this intervention? Please elaborate. Probe: easy to delivery and understand 27. What challenges do you face in using this intervention? How did you overcome this challenge in different phases? Give an example. 28. Which materials/job-aid related to TMIS is the easiest to explain and use? 29. Which materials/job aid related to TMIS do you think is difficult to explain and use? Why? 30. How this intervention is useful for beneficiary institutes, what do you think? 31. What kind of changes do you think will be helpful to provide this intervention to the beneficiaries? Please explain. 32. How suitable are the ‘TMIS intervention’ in educating training institute manager and faculty for strengthening training institutes? • Is there enough information, or could more information be included? • Are the materials easy to understand? Could more materials be developed? Please elaborate. 33. What are the other steps should be taken to aware the faculty of training institutions? Performance Evaluation of USAID’s Accelerating Universal Access to Family Planning (Shukhi Jibon) in Bangladesh | 67 3. Key-informant interview guides: TMIF interventions Can I confirm that you are aware of the use of AUAFP TMIF interventions? Adoption: Development of the intervention package 1. What do you do to make Shukhi Jibon’s approach to developing a TMIF? Please explain. (action) What types of key steps were taken to develop this ‘AUAFP TMIF intervention’ for strengthening training institutes for improving quality of training? What are the signs or indicators of training quality? (adherence) 2. What does the TMIF intervention package include to improve quality of training? Input: What kind of intervention input has been included in ‘TMIS intervention’ for strengthening training institutes for improving quality of training? Please elaborate. (Probe: Organize workshops for developing TMIFs in Tis of NIPORT and DGFP, organize workshops on Optimizing Performance and Quality (OPQ) and Improvement Collaborates (IC), Organize IC meetings, organize review meetings to monitor implementation the recommendations of TMIF, Technical assistance for implementing the recommendations, organize joint monitoring visit for implementing recommendations and operationalization of TMIF, Advocacy with the leadership for TMIF recommendations) What are the silent features of TMIF for improving quality of training? Whom do you provide different kind of interventions? Why? (action) When and how these interventions implemented? Please elaborate. What was AUAFP role in this process? (adherence) Why do you think the steps were necessary? (advantageous) What are the achievements in operationalization on Training Management Improvement Frameworks (TMIF) for Improving management, planning, and technical capacity of MOHFW FP Training Institutes? Materials 3. What type of materials are used for the TMIF? How are these used practically? Please elaborate Probe: Materials for optimizing performance and quality (OPQ) and improvement collaborates (IC). Which one do you think is easy to use and which one is difficult? Training/orientation/workshop/meeting 4. Has the intervention provided any training/orientation/workshop/meeting on how to make or use for training institute systems strengthening? Probe: what training/orientation/ workshop /meeting was provided; who provided the training/orientation/workshop/meeting; when; was the training/orientation/workshop/meeting adequate? Was the training/orientation workshop/ meeting useful? How can the training/orientation/workshop/meeting be improved? (Practical, advantageous) Changes/differences (now and before) 5. What are the changes/differences do you feel the way intervention are providing information now and before? (intention & action) (Probe: Training Institutes developed TMIF with specific recommendations to strengthen the quality of training (Please see MOV), NIPORT and DGFP participated for improving quality of training (Please see MOV and actions), monitor implementation the TMIF recommendations, Coordination improved among NIPORT, DGFP and DGHS, NIPORT initiated staff recruitment, Training room sitting arrangement changed to facilitate interactive and participatory training etc.) Performance Evaluation of USAID’s Accelerating Universal Access to Family Planning (Shukhi Jibon) in Bangladesh | 68 Acceptability & Feasibility 6. How easy do you think to inform beneficiaries of intervention about the TMIF? (Practical) 7. How do you think this intervention will have impact in strengthening training institutes for improving quality of training? What are the qualities the TMIF refers? (Probe: agreeable/suitable/sustainability) 8. Are the training institutes/trainers interested about this intervention components? What kind of component do they want to know in this regard? (Practical) Probe: (please indicate practicability) 9. How this intervention can be more easily accessed by beneficiaries? (Feasibility) 10. What are the demonstrable active usages of the TMIF across training system? 11. How was the setting to build competent training institute at the national, divisional, district and upazila levels operationalized? 12. How was the cooperation across DGFP, DGHS, and NIPORT in TMIF operationalization, usability, and quality assurance? 13. Are there any enhance steps for governmental commitments (Extent of activity moving forward legal, policy and regulatory processes and standards) 14. How do you think it will sustain in future? Are there any logical connections with current HPNSP, PIP, Operational Plan, Training plan, donor support or commitment? 15. At this moment, you are implementing this TMIF intervention, do you think this is appropriate or do you have any suggestion how to implement this better? 16. Are the NIPORT, Tis or DGFP interested to know this intervention? What kind of quality indicators do they want to know or change for improving training quality? (Practical) 17. How well does the intervention align with government priorities? 18. Does the program represent a legitimate role for government? 19. To what extent is the program interventions producing worthwhile results (outputs, outcomes) and/or meeting each of its objectives? 20. Overall, how effective was the TMIF intervention in strengthening training institutes for improving quality of training? Can you please cite examples of quality of training? 21. How likely are you to recommend AUAFP TMIS intervention to a friend? Barriers and Enablers 22. How appropriate is the TMIF intervention for strengthening training institutes for improving quality of training? 23. Will the interventions be able to create any advantage to the training institutes in strengthening training institutes for improving its training quality? 24. How easy do you think to inform training institutes about this? (Practical) 25. What are the positive sides of using this intervention? Please elaborate. Probe: easy to delivery and understand 26. What challenges do you face in using this intervention? How did you overcome this challenge in different phases? Give an example. 27. Which materials/job-aid related to TMIF is the easiest to explain and use? 28. Which materials/job aid related to TMIF do you think is difficult to explain and use? Why? 29. How this intervention is useful for beneficiary institutes, what do you think? 30. What kind of changes do you think will be helpful to provide this intervention to the beneficiaries? Please explain. Performance Evaluation of USAID’s Accelerating Universal Access to Family Planning (Shukhi Jibon) in Bangladesh | 69 31. How suitable are the ‘TMIF intervention’ in educating training institute manager and faculty for strengthening quality of training institutes? • Is there enough information, or could more information be included? • Are the materials easy to understand? Could more materials be developed? Please elaborate. 32. What are the other steps should be taken to aware the faculty of training institutions? Feedback on the intervention package: CBT Intervention Please circle the answer you feel is most appropriate for each of the following aspects of the AUAFP CBT intervention using following ratings: 1-Strongly disagree 2-Disagree 3-Neither agree nor disagree 4-Agree 5-Strongly agree Statements Rating Scale 1 The content of the CBT intervention package is technically correct (aligned with the national policy and guidelines) 1 2 3 4 5 2 The content of the CBT intervention package is technically adequate (contains enough information to strengthening training and skill of providers) 1 2 3 4 5 3 The CBT intervention has enough availability of training materials (multimedia, posters, flipchart, brochures, leaflets etc.) 1 2 3 4 5 4 The overall design of the CBT intervention package is simple and easy to follow 1 2 3 4 5 5 The design of the CBT intervention package will assist in translating required information in accelerating universal access to FP 1 2 3 4 5 6 The CBT intervention package covers all information required to improve skills of the health providers for strengthening FP 1 2 3 4 5 7 The CBT intervention package is easy to use 1 2 3 4 5 8 The training program of CBT intervention address an identified need 1 2 3 4 5 9 In your opinion, what are the limitations of the CBT intervention package (a) Not appropriate to local context (b) Tasks are complex and unclear (c) Will not motivate a satisfactory number of service providers to improve skills for providing FP services (d) May increase gaps among service providers (e) The users may feel burdened that they are being told what to do (f) Others 10 Your opinion on the Trainer’s Manual? (a) Acceptability: The perception among stakeholder that the Trainer’s Manual is agreeable and relatively advantageous to strengthen service provider skills in proving selected FP services 1 2 3 4 5 (b) Appropriateness: Suitability and applicability of the the Trainer’s Manual to strengthen service provider skills in proving selected FP services inability 1 2 3 4 5 Performance Evaluation of USAID’s Accelerating Universal Access to Family Planning (Shukhi Jibon) in Bangladesh | 70 Statements Rating Scale (c) Feasibility: The Trainer’s Manual is Practicable and implementable to strengthen service provider skills in proving selected FP services 1 2 3 4 5 11 Your opinion on the training of Master trainers (a) Confidence: The Adherence and capability of the Master Trainers in imparting training 1 2 3 4 5 (b) Importance: The intention and action to implement the intervention package to strengthen service provider skills in proving selected FP services 1 2 3 4 5 (c) Acceptability: It is agreeable and relatively advantageous to impart the training as a Master Trainer 1 2 3 4 5 11 What are your recommendations for improving the quality of the CBT intervention package? 12 Other comments Feedback on the intervention package: TMIS Intervention Please circle the answer you feel is most appropriate for each of the following aspects of the AUAFP TMIS intervention using following ratings: 1-Strongly disagree 2-Disagree 3-Neither agree nor disagree 4-Agree 5-Strongly agree Statements Rating Scale 1 The content of the TMIS intervention package is technically correct (aligned with the national policy and guidelines) 1 2 3 4 5 2 The content of the TMIS intervention package is technically adequate (contains enough information to strengthening training and skill of providers) 1 2 3 4 5 3 The TMIS intervention has enough availability of training materials (multimedia, posters, flipchart, brochures, leaflets etc.) 1 2 3 4 5 4 The overall design of the TMIS intervention package is simple and easy to follow 1 2 3 4 5 5 The design of the TMIS intervention package will assist in translating required information in accelerating universal access to FP 1 2 3 4 5 6 The TMIS intervention package covers all information required to improve skills of the health providers for strengthening FP 1 2 3 4 5 7 The TMIS intervention package is easy to use 1 2 3 4 5 8 The training program of TMIS intervention address an identified need 1 2 3 4 5 9 In your opinion, what are the limitations of the TMIS intervention package (g) Not appropriate to local context 1 2 3 4 5 (h) Tasks are complex and unclear 1 2 3 4 5 (i) Will not motivate a satisfactory number of service providers to improve skills for providing FP services 1 2 3 4 5 (j) May increase gaps among service providers 1 2 3 4 5 Performance Evaluation of USAID’s Accelerating Universal Access to Family Planning (Shukhi Jibon) in Bangladesh | 71 Statements Rating Scale (k) The users may feel burdened that they are being told what to do 1 2 3 4 5 (l) Others 1 2 3 4 5 10 In your opinion, what are the contributions of the intervention package? (a) Report on landscape analysis provides a roadmap for continuous learning to proceed with TMIS intervention 1 2 3 4 5 (b) Trained NIPORT staff use TMIS 1 2 3 4 5 (c) TIs using TMIS to improve training management efficiency (avoid overlapping or repeat of trainees) 1 2 3 4 5 (d) TMIS to detect Overlapping/repeat training by service providers 1 2 3 4 5 (e) NIPORT using own fund for procuring server for operationalizing TMIS 1 2 3 4 5 (f) NIPORT rolled out TMIS in non-Shukhi Jibon areas 1 2 3 4 5 (g) NIPORT provided access to DDFP of DGFP in TMIS to utilize training data 2 3 4 5 (a) DGFP using TMIS to utilize training data 1 2 3 4 5 (b) The TMIS application led training programs at NIPORT headquarters and training institutes 1 2 3 4 5 (c) The TMIS database of FP trainees and trainers, informs trainers performance feedback, tracks training information and trainee performance 1 2 3 4 5 11 What are your recommendations for improving the quality of the TMIS intervention package? 12 Other comments Feedback on the intervention package: TMIF Intervention Please circle the answer you feel is most appropriate for each of the following aspects of the AUAFP TMIF intervention using following ratings: 1-Strongly disagree 2-Disagree 3-Neither agree nor disagree 4-Agree 5-Strongly agree Statements Rating Scale 1 The content of the TMIF intervention package is technically correct (aligned with the national policy and guidelines) 1 2 3 4 5 2 The content of the TMIF intervention package is technically adequate (contains enough information to strengthening training and skill of providers) 1 2 3 4 5 3 The TMIF intervention has enough availability of training materials (multimedia, posters, flipchart, brochures, leaflets etc.) 1 2 3 4 5 4 The overall design of the TMIF intervention package is simple and easy to follow 1 2 3 4 5 5 The design of the TMIF intervention package will assist in translating required information in accelerating universal access to FP 1 2 3 4 5 6 The TMIF intervention package covers all information required to improve skills of the health providers for strengthening FP 1 2 3 4 5 Performance Evaluation of USAID’s Accelerating Universal Access to Family Planning (Shukhi Jibon) in Bangladesh | 72 7 The TMIF intervention package is easy to use 1 2 3 4 5 8 The training program of TMIF intervention address an identified need 1 2 3 4 5 9 In your opinion, what are the limitations of the TMIF intervention package (m) Not appropriate to local context 1 2 3 4 5 (n) Tasks are complex and unclear 1 2 3 4 5 (o) Will not motivate a satisfactory number of service providers to improve skills for providing FP services 1 2 3 4 5 (p) May increase gaps among service providers 1 2 3 4 5 (q) The users may feel burdened that they are being told what to do 1 2 3 4 5 (r) Others 1 2 3 4 5 10 In your opinion, what are the contributions of the intervention package? (a) TMIFs developed in all Tis with specific recommendations to strengthen the quality of training in all TIs 1 2 3 4 5 (h) Optimizing Performance and Quality (OPQ) and Improvement Collaborates (IC) workshops 1 2 3 4 5 (i) Trained NIPORT staff use TMIF 1 2 3 4 5 (j) TIs using TMIS to improve training management efficiency (avoid overlapping or repeat of trainees) assisted in improving quality of training of TIs 1 2 3 4 5 (k) Coordination improved among NIPORT, DGFP and DGHS for supporting each other 1 2 3 4 5 (a) Training room sitting arrangement changed to facilitate interactive and participatory training 1 2 3 4 5 (b) The TMIF strengthen training capacity of the TIs 1 2 3 4 5 (c) The TMIF helps each TI to improve its management practices 2 3 4 5 (d) The TMIF helps to increase the trainer’s skill and to strengthen the training institute and overcome training challenges 1 2 3 4 5 11 What are your recommendations for improving the quality of the TMIF intervention package? 12 Other comments Questionnaire to obtain information from Shukhi Jibon: SL Question Responses Remarks/ references Competency based Training (CBT) 1. What is the CBT and adult learning components? 2. Please list down of CBT input (what you have done? When did you do it? Who are the beneficiaries? How did you do it?) 3. Please quantify CBT input (e.g., training, workshop, logistics, equipment, follow-up) as against expectation that are supposed to offer? 4. Do you have AUAFP CBT training/workshop participants list (e.g., training of master trainers) by institutes? Is it possible to share the lists with us? Performance Evaluation of USAID’s Accelerating Universal Access to Family Planning (Shukhi Jibon) in Bangladesh | 73 SL Question Responses Remarks/ references 5. Please provide progress, achievements, and implementation of CBT interventions with means of verifications (e.g., Annual technical report etc.) 6. What are the demonstratives active usage of the CBT across learning lab sites? 7. How was the setting to build provider competency approach at the national, divisional, district and upazila levels operationalized? 8. How was the cooperation across DGFP, DGHS, and NIPORT in operationalization, usability, and quality assurance? 9. Enhancement of governmental commitments (Extent of activity moving forward legal, policy and regulatory processes and standards) 10. How do you think it will sustain in future? Logical connections with current HPNSP, PIP, Operational Plan, Training plan, donor support or commitment. 11. Challenges in operationalizing and implementing CBT interventions 12. What AUAFP data base information on CBT intervention are available: ● Adoption of CBT and adult learning methodology in the local context after the training (Checklist for session observation of Master Trainers (activity data) ● In-depth interviews with trainers (with low/high adoption) (Reflection report technical brief) ● Follow-up of CBT approach: In-depth interviews (AUAFP Managers, Training institutes heads) Reflection report technical brief ● Level of quality among trainees trained by CBT (Structured checklist for interview trainees of CBT (Reflection report technical brief) ● Views on support received by AUAFP? Quarterly reflection report ● What other information/report available in the AUAFP database? Training Management Information System (TMIS) 13. What are the TMIS components? 14. Please list down of TMIS input (what you have done? When did you do it? Who are the beneficiaries? How did you do it?) 15. Please quantify TMIS input (e.g., training, workshop, logistics, equipment, follow-up) as against expectation that are supposed to offer by TMIS interventions. 16. Do you have AUAFP TMIS training/workshop participants list by institutes? Is it possible to share the lists with us? Performance Evaluation of USAID’s Accelerating Universal Access to Family Planning (Shukhi Jibon) in Bangladesh | 74 SL Question Responses Remarks/ references 17. Please provide progress, achievements, and implementation of TMIS interventions with means of verifications (e.g., Annual technical report etc.) 18. What are the demonstratives active usage of the TMIS across NIPORT system? 19. How was the process of operationalizing the TMIS? 20. Do you have AUAFP TMIS data base? Is it possible to share with us? 21. What extent is the TMIS being used by training institutes to manage (i.e., plan, execute, track) training events? 22. Is it possible to share TMIS data base handled or used by NIPORT? 23. How was the cooperation across DGFP, DGHS, and NIPORT in operationalization, usability, and quality assurance? 24. Enhance governmental commitments (Extent of activity moving forward legal, policy and regulatory processes and standards) 25. How do you think it will sustain in future? Logical connections with current HPNSP, PIP, Operational Plan, Training plan, donor support or commitment. 26. Challenges in operationalizing and implementing TMIS interventions 27. What AUAFP data base information are available: 28. Reflection report-technical brief (Process of operationalizing the TMIS) 29. Reflection report-technical brief (Use of TMIS to manage (i.e., plan, execute, track) training events) 30. What other information/report available in the AUAFP database? - o 31. OTHER annual reports/technical reports - o Training Management Improvement Frameworks (TMIFs) 32. What are the TMIF components? 33. Please list down of TMIF input (what you have done? When did you do it? Who are the beneficiaries? How did you do it?) 34. Please quantify TMIF input (e.g., training, workshop, logistics, equipment, follow-up) as against expectation that are supposed to offer by TMIF interventions. 35. At what extent are the training institutes using the TMIF for decision-making. 36. What are uses of the TMIF database for FP trainees and trainers? 37. What are the demonstratives active usage of the TMIF across learning lab sites? 38. Do you have AUAFP TMIF data base? Is it possible to share with us? Performance Evaluation of USAID’s Accelerating Universal Access to Family Planning (Shukhi Jibon) in Bangladesh | 75 SL Question Responses Remarks/ references 39. How was the cooperation across DGFP, DGHS, and NIPORT in operationalization, usability, and quality assurance? 40. Enhance governmental commitments (Extent of activity moving forward legal, policy and regulatory processes and standards) 41. How do you think it will sustain in future? Logical connections with current HPNSP, PIP, Operational Plan, Training plan, donor support or commitment. Remarks or any comments or observations 2. Mentorship and supportive supervision (M&SS) 2.1. Key-informant interview guides (Managers-DGFP) (Please note: The guideline is to shape the conversations. The questions will not necessarily be asked directly, but the respondent will be invited to tell of their experiences, guided by these topics. The interviewer will use their discretion to decide the order and level of detail in which to cover each topic with each informant.) Name Workstation Designation Working in this position since Date Can I confirm that you are aware of the Mentorship intervention? And Supportive Supervision intervention of Shukhi Jibon? Adoption Development of the intervention package (Mentorship and Supportive Supervision) 1. Would you please explain the objective/purpose of Integrated Mentorship and Supportive Supervision (M&SS) approach? 2. What key steps were taken to develop M&SS interventions in different stages? When and how these steps were implemented? How were you and the other staff members of the Directorate was involved in the process? a. Situation analysis b. Manual Development c. Development of tools, guidelines, checklists d. TOT and training e. Mentee selection-pairing, SWOT analysis, action plan development f. Coaching sessions g. Planning and review meetings h. Record keeping, reporting i. M&SS mobile application development j. Supervisory visits using checklist Performance Evaluation of USAID’s Accelerating Universal Access to Family Planning (Shukhi Jibon) in Bangladesh | 76 3. Are you aware of the selection and pairing process of mentor and mentees? How this process could be improved? Materials/Job Aids 4. Have you seen the materials/job aids that are specifically developed and used for this intervention? Please elaborate. (Probe: Mentorship and Supportive Supervision Trainers and Participants manual and standard presentation, Observation checklist, Action plan handbook, self-assessment form, consent form). How helpful these were? Training/orientation/workshop/meeting 5. Has the intervention provided any training/orientation/workshop/meeting on how to make or use of mentorship approaches? What was the role of DGFP in these trainings? Effectiveness (now and before) 6. To what extent is the program interventions producing worthwhile results (outputs, outcomes) and/or meeting its objectives? 7. Are you aware of any differences between the skills of a provider who participated in mentorship program and who did not? Please share your experiences. Acceptability, Feasibility and Sustainability 8. How well does the intervention align with government priorities? 9. How was the cooperation across DGFP and DGHS in operationalization, usability, and quality assurance? Please give examples. 10. How do you think it will sustain in future? Are there any logical connections with current HPNSP, PIP, Operational Plan, donor support or commitment? Actions from MOHFW? 11. How does this intervention help the FP service providers? 12. What would be your suggestions/recommendations to MOHFW for integration of Mentorship strategy in their regular programs? 13. What was the role of Shukhi Jibon staff in the entire process? How helpful they were? (For GOB: How do you continue this intervention without Shukhi Jibon staff in place?) Barriers and Enablers 14. What are the positive sides of using this intervention? What helped? Please elaborate. 15. What are the challenges of implementation of this intervention? 16. What kind of changes do you think will be helpful to provide this intervention? Please explain. 2.2. In-depth interview guides (Mentors/Managers) (Please note: The guideline is to shape the conversations. The questions will not necessarily be asked directly, but the respondent will be invited to tell of their experiences, guided by these topics. The interviewer will use their discretion to decide the order and level of detail in which to cover each topic with each informant. The interviewer will raise the Mentorship and Supportive Supervision interventions one by one.) Performance Evaluation of USAID’s Accelerating Universal Access to Family Planning (Shukhi Jibon) in Bangladesh | 77 Name Designation/Workstation When and where did you receive master training/ training How long you are working as a Mentor? Total how many Mentees you have mentored? Their names, job titles and locations 1. 2. 3. 4. 5. Has any of them graduated? Who? MENTORSHIP Can I confirm that you are aware of the Mentorship intervention of Shukhi Jibon? Adoption 1. Please tell me what you know about the Mentorship intervention of Shukhi Jibon? 2. How was your selected as a mentor? Why did you agree to work as a mentor? 3. How your Mentees were selected and paired with you? What did you do after selection of your Mentees? 4. How did you assess his/her/their baseline skills and set the goals? (please show me any documentation available). Tell me about the use of initial SWOT analysis using RCA tool. May I see an example? Who conducted this analysis? What was your role? May I know your experience of this process? Training 5. Have you received any training/ orientation on AUAFP’s mentorship approach? When did you receive the 1st training? How many days was the training? Probe: what training was provided; who provided the training; when; was the training adequate? Was the training useful? How can the training be improved? 6. Did you receive any refresher’s training? When? Where? How many days? Probe: what training was provided; who provided the training; when; was the training adequate? Was the training useful? How can the training be improved? 7. Have you received training on Gender Integration approach? How many days? How do you apply the knowledge that you gained through this training? Is there any manual/handbook on gender integration approach? Information/use/effectiveness 8. What do you do to improve the skills of your Mentee/s? How do you track their progress? Please tell us about baseline and current skill set of your Mentee/s. Show us any documents available with you. 9. When did you have your last mentorship session (month, year)? Was the last session virtual or in-person? 10. Which areas did you assess in your last session? What were the gradings of different Mentees? May I see any document? 11. Have you shared the grades/marks with your Mentees? What are their gaps? What is plan to improve their skills? Performance Evaluation of USAID’s Accelerating Universal Access to Family Planning (Shukhi Jibon) in Bangladesh | 78 Materials/Job Aids 12. What type of materials/job aids you specifically used while participated in this intervention? Please show me the use of these materials? (Probe: Mentorship and Supportive Supervision Trainers and Participants manual and standard presentation, Observation checklist, Action plan handbook, self-assessment form, consent form, mentorship mobile app etc.) 13. When were you trained on the mobile app? How many times you have used it? What are the main features? How comfortable are you to use this app? Acceptability, Feasibility and Sustainability 14. How many coaching sessions did you initially plan? How many were achieved? May I see you plan? (in person/virtual) 15. Are you aware of the graduation criteria? Has any of your Mentees graduated? When do you think your mentees will graduate? 16. How do you think it will sustain in future? Are there any logical connections with current HPNSP, PIP, Operational Plan/ donor support or commitment? 17. What was the role of Shukhi Jibon staff in the entire process? How helpful they were? How do you continue this intervention without Shukhi Jibon staff in place? Barriers and Enablers 18. How does this intervention helps the FP service providers? 19. What are the positive sides of this intervention? Please elaborate. 20. What challenges do you face in using this intervention? How did you overcome this challenge in different phases? Give an example. 21. Are there any specific areas of the Mentorship intervention that can be improved? SUPPORTIVE SUPERVISION Can I confirm that you are aware of the Supportive Supervision intervention of Shukhi Jibon? Information, use, practice 1. Please tell me what you know about the Supportive Supervision intervention of Shukhi Jibon? 2. How many total visits you made/faced in last 1 year? When did you conducted the last supervisory visit (month, year)? Which areas did you assess/were assessed? 3. Have you ever conducted a supervisory visit where the facility was assessed with the structured checklist? After filling-in, what happened to the checklist? Where do you store it? May I see the copies of the filled￾in checklists. (May I see a copy of the filled-in checklist) 4. Please tell me about an incident when the findings of the checklist were used for facility readiness or to bring a change in the facility. 5. How comfortable are you to use the checklist? 6. Do you have a monthly plan for supervisory visits? If you are a Mentor, how do you manage both Mentorship visit and supervisory visit? Performance Evaluation of USAID’s Accelerating Universal Access to Family Planning (Shukhi Jibon) in Bangladesh | 79 Barriers and Enablers 7. How does this intervention help the FP service providers? 8. What are the positive sides of this intervention? Please elaborate. 9. What challenges do you face in using this intervention? How did you overcome this challenge in different phases? Give an example. 10. Are there any specific areas of the Supportive Supervision intervention that can be improved? Sustainability 11. How do you think the intervention will sustain in future? Are there any logical connections with current HPNSP, PIP, Operational Plan/ donor support or commitment? 12. What was the role of Shukhi Jibon staff in the entire process? How helpful they were? How do you continue this intervention without Shukhi Jibon staff in place? 2.3. In-depth interview guides (Mentees) (Please note: The guideline is to shape the conversations. The questions will not necessarily be asked directly, but the respondent will be invited to tell of their experiences, guided by these topics. The interviewer will use their discretion to decide the order and level of detail in which to cover each topic with each informant. The interviewer will raise the Mentorship and Supportive Supervision interventions one by one.) MENTORSHIP: Can I confirm that you are aware of the Mentorship intervention? Name Workstation Designation When was you first enrolled in this program? Who is your mentor? Name, designation & location Total how many Coaching sessions did you participate? #in-person # virtual Adoption/Training 1. Why and how you have been selected as a mentee? Why did you agree to join this program? Did anyone ask if you liked to join the program? Was there any formal consent process? 2. Have you received any training on M&SS approach? If yes, when? How many days? What was discussed? 3. How were you assessed at the beginning? How was your action plan developed? Who did what (you/mentor/supervisor/AUAFP) (please show me any documentation available) 4. When did you receive training on Gender Integration approach? How many days? How do you apply the knowledge that you gained through this training? Is there any manual/handbook on gender integration approach? Performance Evaluation of USAID’s Accelerating Universal Access to Family Planning (Shukhi Jibon) in Bangladesh | 80 Information/Use/Process 5. What do you know about this intervention? How did you know about the mentorship approach? 6. Which technical areas you are working to develop your skills through this mentorship approach? (probe: Skill in FP and MCH services, Oral Pill, Condom, Injection, Implant insertion and removal, IUD insertion and removal, Tubectomy, Vasectomy, ANC & PNC, AY SRH services, Good counselling skill) 7. How do you know if your skills are being improved? Please show me if there is any documents. 8. When did you have your last mentorship session (month, year)? Was the last session virtual or in-person? 9. Which area/s you were assessed in your last session? What was your grading in the last session? Please show me the checklist filled-in the last session. 10. Do you know about your grades/marks in each technical area? What are your gaps? What is your plan to improve your skills? Materials 11. What type of materials/job aids you used while participated in this intervention? Please show me the use of these materials? (Probe: Observation checklist, Action plan handbook, self-assessment form, consent form, mentorship mobile app etc.) 12. When were you trained on the mobile app? How many times you have used it? What are the main features? How comfortable are you to use this app? Acceptability, Feasibility and Sustainability 13. How many coaching sessions did you initially plan? How many were achieved? (in person/virtual) 14. Are you aware of the graduation criteria? Please explain? When do you think you can graduate? 15. What do you do differently after participating in Mentorship program? 16. What did you like about this approach? 17. Are there any specific areas of the Mentorship intervention that can be improved? 18. What is the role of Shukhi Jibon staff in the process? What do they do? How helpful are they? The Mentorship approach will continue same as before after Shukhi Jibon activity ends. How? Barriers and Enablers 19. How appropriate is the intervention for improving skills of health service providers? Why? 20. What are the positive sides of using this intervention? Please elaborate. (Probe: easy to deliver and understand) 21. What challenges do you face in using this intervention? How did you overcome this challenge in different phases? Give an example. (Probe: Self-assessment, RCA, Action planning, use of checklist, graduation etc.) SUPPORTIVE SUPERVISION: Can I confirm that you are aware of the Supportive Supervision intervention of Shukhi Jibon? Information, use, practice 22. Please tell me what you know about the Supportive Supervision intervention of Shukhi Jibon? Does your supervisor ever visit the facility and identify issues with a structured checklist? (Probe: Looks Performance Evaluation of USAID’s Accelerating Universal Access to Family Planning (Shukhi Jibon) in Bangladesh | 81 at training, logistics, infrastructure, registers etc.). When did s/he visited last time? 23. After filling-in, what happened to the checklist? Have you identified the problems and solutions and developed any action plan? (May I see the copy of a filled-in form) 24. Please tell me about an incident when the findings of the checklist were used for facility readiness or to bring a change in the facility. 25. How does this intervention help your service delivery? What are the positive sides of this intervention? Please elaborate. 26. What was the role of Shukhi Jibon staff in the entire process? How helpful they were? How do you continue this intervention without Shukhi Jibon staff in place? 3. Guidelines and Tools on Interventions for Adolescents A. Interview Guide for local NGO implementers/AUAFP Field Officers a) Please tell us about the adolescent interventions that you implement in your catchment area (facility/community) b) If you have activities in schools, how responsive are the adolescents? c) Do adolescents attend/participate in the community events? If no, why not? d) How do you monitor progress of the work you do on adolescents? e) Do you think adolescents are benefiting from these interventions? If so, how do you know? f) What do you think is the most effective platform (outreach, schools, facilities, digital) for reaching adolescents? g) What worked best? What were you not able to do so well? B. KII guide for service providers at facilities (FWV, SACMO). Name Workstation Designation Date 1. What training did you receive on providing services for adolescents (probe on details of when, where, duration and contents)? 2. How helpful has this been for your actual work with adolescents? 3. Do you think the adolescent health corners are being utilized sufficiently? If not, why? 4. Where do they currently go for services and why? 5. What are the services most commonly sought by adolescents? where? 6. Do you think adolescents attend the community outreach activities, if not, why? 7. What do you think is the most effective platform for reaching adolescents (probe – facility, outreach, digital, schools)? 8. Have you noticed any changes among adolescents before and after intervention? C. Checklist to assess AYFS facility readiness based on WHO and DGFP guidelines: • Does the health facility have any service providers trained to provide AYFS? • Does the health facility provide RH services, including FP, for A&Y clients free of cost? • Does the health facility maintain both visual and auditory privacy for A&Y clients? • Does the health facility provide gender-equitable services for A&Y clients? • Does the health facility have any RH educational materials (pamphlets, booklets), posters, or job Performance Evaluation of USAID’s Accelerating Universal Access to Family Planning (Shukhi Jibon) in Bangladesh | 82 aids focusing specifically on A&Y? • Does the health facility have written guidelines or policies for providing services to A&Y? • Does the health facility have provisions for sex- and age-disaggregated record keeping and reporting for services to A&Y? • Does the health facility have an assigned service provider to manage the school health program? • Is a list of AYSRH services clearly displayed at this health facility? • Is there a signboard or other indication to identify this health facility as A&Y-friendly? D. KII Guide: Adolescents at adolescent friendly health facilities I am going to ask you some questions regarding seeking information and/or service relating to health If you or a friend needs RH services, where would they go? (probe all the options they know of) 1.Did you visit any health facility in last 6 months for yourself? 2.How many visits did you make? 3.Where did you visit last? Probe Public medical college hospital NGO Private Pharmacy SMC On the last health facility, you visited, is there a separate corner for adolescents to provide information and service? Yes No DK In your last visit did you go for any information or services or both? Information Service Both What information and/ or services did you receive the last time you visited? Anything else? Menstrual problem management UTI/STI General illness Allergy/Rash/Itching FP services Now I would want to know if you like to know more about following topics? 1.Puberty 2.physical change 3.Menstruation 4.Marriage/Pregnancy 5.Family planning E. Group Interview (GI) Guide: Adolescents in the Community/School I will now tell you a story about a girl and ask you some questions related to what you think of her. This girl is made-up: that means she does not exist and is not somebody you know. Shahnaz is 17 years old and lives in [Upazila]. She recently got married and does not want to try having a baby until she is 21 years old. 1. What do you think of Shahnaz’s choice to not have a baby till she is 21? Why? 2. How much knowledge do you think Shahnaz has about not having a baby? Performance Evaluation of USAID’s Accelerating Universal Access to Family Planning (Shukhi Jibon) in Bangladesh | 83 3. What are the resources available to Shahnaz? How helpful are they? 4. How supportive are her husband/in-laws? Shahnaz decides to go to the local health clinic where she meets a lady who advises her to begin taking contraceptive pills. 1. What do you think Shahnaz should do? Why? 2. Do you feel that contraceptive pills would be preferred by Shahnaz, compared with a more long-term approach? Why? 3. Are there any other challenges Shahnaz might face in this situation? Have you heard of the Shukhi Jibon programme? If yes: 1. Please share 3 things you have learned from the programme. 2. Please share the main information/ services adolescent girls and boys in this community/ school need related to their health. Prompt – differences between girls/ boys, married/ unmarried 3. Any curriculum or sessions in school on this? By SACMO? Was it helpful? 4. Have you attended any session at community organized by NGO/ Shukhi Jibon /GOB/ Was it helpful? 5. Do you feel that the program has met your needs – why/ how? F. Interview guide for DGFP Name Workstation Designation Date 1. Adolescent sexual and reproductive health is one of the 4 priority thematic areas of intervention in the National Strategy– What is the role of USAID Shukhi Jibon? How does it support the government program? a) In your opinion, which interventions have been most effective (facility/school/community b) Which ones are likely to be integrated in the Operational Plans? c) Moving forward, what are the lessons learnt? d) Is there any evidence that these interventions have improved adolescents’ health knowledge and/or care-seeking behavior? Performance Evaluation of USAID’s Accelerating Universal Access to Family Planning (Shukhi Jibon) in Bangladesh | 84 Annex IV: List of documents reviewed Topic Document Category Activity documents • Monitoring, Evaluation and Learning Plan (MEL). July 23, 2018 – July 22, 2023 Service delivery protocol and tools • Shukhi Jibon Learning Lab: Adolescent and Youth Sexual and Reproductive Health-October 2021 • Shukhi Jibon Learning Lab: Competency-Based Training- October 2021 • Shukhi Jibon Learning Lab: Mentorship and Supportive Supervision- October 2021 • Learning Labs in Shukhi Jibon Program Approach- October 2021 • Shukhi Jibon Learning Lab: Training Management System- October 2021 • Concept Brief: Integrated Mentoring and Supportive Supervision (M&SS) Approach of USAID’s Shukhi Jibon activity 2019 • Overview: The Integrated Mentorship and Supportive Supervision (M&SS) approach • Mentorship Checklists • Action Plan Handbook for mentorship program • Self-Assessment form for mentorship program Service statistics • activity M&SS data repository • Distribution of mentors from receiving training to engaging as active mentors in M&SS intervention areas • Distribution of mentees from enrollment to mentorship graduation • Distribution of active mentees by facilities and designation • Distribution of supportive supervision visits to different category of facilities by different category of active supervisors • activity AUAFP DHIS2 Training documents and Manual • Competency Based Training- Trainer’s Manual • Number of Trainers Trained on Competency-based Training by Districts • Mentorship and Supportive Supervision Trainer’s and Participants Manual 2020 &2022 • Family Planning Manual, CCSDP, DGFP, June 2021 • National guideline for clinical mentorship program (draft_2023). qis.gov.bd Annual/periodic reports • Sukhi Jibon Work plan Y1, Y2, Y3, Y5 • Sukhi Jibon Annual Report Y1, Y2, Y3, Y4 Evaluation/assess ment • Final Facility Assessment Report_23_07_2019 • FP Training Need Assessment-Brief-final August 2019 • Assessment of FP service delivery at selected public health facility in Bangladesh: focusing on adolescent RH service, PPFP, PAC-FP and gender responsiveness- 2019 report Summary Performance Evaluation of USAID’s Accelerating Universal Access to Family Planning (Shukhi Jibon) in Bangladesh | 85 Topic Document Category Research Reports • The Power of Mentorship & Supportive Supervision in Bangladesh • DGFP & MOHFW. (2016). National Strategy for Adolescent Health 2017-2030. • DGFP & MOHFW. (2018). National Plan of Action for Adolescent Health Strategy 2017-2030. • NIPORT, MEFWD, MOHFW, ICDDR, B & D4I. (2021). Bangladesh Adolescent Health and Wellbeing Survey 2019–20. • Shankar, P., Sievers, D., & Sharma, R. (2020). Evaluating the Impact of a School- • Based Youth-Led Health Education Program for Adolescent Females in Mumbai, India. Annals of Global Health, 86(1), 108. https://doi.org/10.5334/aogh.2791 • The World Bank Data. School enrollment, secondary (% gross) – Bangladesh. https://data.worldbank.org/indicator/SE.SEC.ENRR?locations=BD • Akuiyibo, S., Anyanti, J., Idogho, O. et al. (2021). Reprod Health 18, 204. Impact of peer education on sexual health knowledge among adolescents and young persons in two North Western states of Nigeria. https://doi.org/10.1186/s12978- 021-01251-3 • Rutgers International. (2022). Peer education: widely used but wildly undervalued. https://rutgers.international/news/peer-education-widely-used￾but-wildly-undervalued/ • Sydney Briggs, Lisa Kim, Ellen Wilson, and Elizabeth Wildsmith. (2021). School￾based Strategies to Support Adolescent Sexual and Reproductive Health. Child Trends. https://www.childtrends.org/publications/school-based-strategies￾support-adolescent-sexual-reproductive-health Strategic exercise documents • HMIS data on selected indicators All divisions • Joint Supervisory visit report • M&SS review meetings report • Training Management Improvement Framework (10) • Report on Training Management Improvement Framework Workshop￾November 10-11, 2020 • Report on Improvement Collaboratives Meeting-1 March 2022 • Report on Regional Meeting on Improvement -Collaboratives for Improving Quality of Training-07 June 2022 • Report on Meetings of regional improvement collaborative Teams of Training Institutes-20 April 2022 • Report on CBT, TMIS and TMIF • Health, Population and Nutrition Sector Program 2017-2022 • Maternal, Newborn and Child and Adolescent Health (MNCAH) Operation Plan, DGHS • Maternal, Child, Reproductive and Adolescent Health (MCRAH) Operation Plan • Clinical Contraceptive Service Delivery Program (CCSDP) Operation Plan Circulars • NIPORT Circular on creating Resource Pool June 2020 • NIPORT Circular on Integration of CBT and use of CBT Manual at 11 TIs Nov 2020 • NIPORT Circular on Integration of CBT and use of CBT Manual at 5 TIs June 2021 • NIPORT Circular on CBT Integration in 3 Training Institutes_24 March 2022 • NIPORT Circular on Technical Committee on Monitoring Integration of CBT at NIPORT TIs June 2021 • NIPORT Circular on approval and use of checklist to observe trainers Nov 2021 • DGFP Circular on approval and use of checklist to observe trainers Dec 2021 Performance Evaluation of USAID’s Accelerating Universal Access to Family Planning (Shukhi Jibon) in Bangladesh | 86 Topic Document Category • NIPORT Circular for Data Entry in TMS Software June 2021 • NIPORT Circular for monitoring operalization of TMS Dec 2020 • NIPORT circular on identifying focal person for TMS for each TIs March 2022 • NIPORT Circular on approving access of TMIS for DGFP August 2022 • GOB approval for manual printing • DGFP circular on FPCSQIT TOR and checklist-2019 Annex V: List of meetings and stakeholders # Organization # of meetings Date # of participant Key participants 1. USAID 4 15 December 2021* 3 Umme Salma Jahan Meena, Taskeen Chowdhury, Fida Mehran 2. 12 January 2023* 5 Arick Proma, Taskeen Chowdhury, Umme Salma Jahan Meena, Marvin Crespin Gamez, Shahrear Farid 3. 31 January 2023# 2 Taskeen Chowdhury, Samina Chowdhury 4. 22 February 2023* 7 Umme Salma Jahan Meena, Taskeen Chowdhury, Samina Chowdhury, Marvin Crespin￾Gamez, Arick Proma, Shahrear Farid, Liza Talukder 5. AUAFP 8 19 December 2022* 10 Mahbub Alam, Sharmin Sultana, Ali Liaquat, Fatema Shabnam, Umme Salma Jahan Meena, Taskeen Chowdhury 6. 8 January 2023# 8 Caroline Crosbie, Liaquat Ali, Mahbub Alam, Mahbub Rashid, Murshida Rahman, Rayan Sharif, Sharmin Sultana, Saiful Hasan 7. 9 January 2023# 5 Shahina Sultana, Sharmin Sultana, Mahbub Alam, Liaquat Ali, Saiful Hasan 8. 16 January 2023# 7 Caroline Crosbie, Fatema Shabnam, Jeanett Shapla Chowdhury, Liaquat Ali, Mahbub Alam, Parveen Akter, Saiful Hasan 9. 22 January 2023# 5 Mahbub Alam, Sharmin Sultana, Liaquat Ali, Shahina Sultana, Saiful Hasan Performance Evaluation of USAID’s Accelerating Universal Access to Family Planning (Shukhi Jibon) in Bangladesh | 87 # Organization # of meetings Date # of participant Key participants 10. 29 January 2023# 12 Sharmin Sultana, Liaquat Ali, Fatema Shabnam, Mahbub Rashid, Akteruzzaman Bhuiyan, Parveen Akter, Sohrab Hossain, Murshida Rahman, Rayan Sharif, Jeanett Shapla Chowdhury, Saiful Hasan, Fahmina Khan 11. 9 February 2023# 3 Sharmin Sultana, Liaquat Ali, Shahina Sultana 12. 15 March 2023* 17 Caroline Crosbie, Mahbub Alam, Sharmin Sultana, Liaquat Ali, Fatema Shabnam, Mahbubur Rashid, Marufa Khan, Shiril Sarker, Saiful Hassan, Murshida Rahman, Jeanett Shapla Chowdhury, Shahina Sultana, Taskeen Chowdhury 13. RDM-Senior management 3 19,23,28 January 2023# 2 Shams El Arifeen, Quamrun Nahar 14. NIPORT 4 31 January 2023 4 Mohammad Shahjahan- DG, Md. Monirul Huda-Director Training, Abdul Hamid Moral￾DD Training, Hero Dhar-DD 15. 2 2 February 2023# Dhamrai 2 Abul Moin-Training Officer, Bijoy Chandra Sarker-AV Operator 16. 2 6 February 2023# Sylhet 2 Riajul Haque-Principal, Ebadot Khan Phatan-AV operator 17. 1 14 February 2023# Faridpur 1 Hori Chand Shil-Principal 18. 2 19 February 2023# Iswarganj 2 Nahidul Islam-Training Officer, Susmita Roy-Assistant Trainer 19. DGFP 3 1 February 2023# 3 Mahmudur Rahman-LD MCRAH, Abdul Latif Mollah-LD IEC, Nurun Nahar Begum-LD CCSDP 20. 5 2 February 2023# Dhamrai 5 Noor Rifat Ara-UH&FPO, Iqbal Kabir-MO MCH-FP, Mesbah Uddin-UFPO, Samsunnahar Shompa- FWV, Moly Akter-FWV 21. 3 12 February 2023# 3 Shahan Ara Banu-DG, Manjur Rahman-PM, Shahadat Hossain- MIS Performance Evaluation of USAID’s Accelerating Universal Access to Family Planning (Shukhi Jibon) in Bangladesh | 88 # Organization # of meetings Date # of participant Key participants 22. 3 5 February 2023# Sylhet 3 Sadik Miah MO-MCH, Abdul Mannan-Regional Consultant FPCS-QIT, Shilpi Akhter-FWV 23. 2 6 February 2023# Sylhet 2 Abul Mansur Ashzad- UFPO, Chaina Talukdar- FWV 24. 3 7 February 2023# Sylhet 3 Luthfunnahar Jasmin- DDFP, Kutub Uddin- Divisional Director-FP, Sumi Roy-FWV 25. 2 14 February 2023# Faridpur 2 Nurjahan Begum-FWV, Fatema Khatun-FWA 26. 3 15 February 2023# Faridpur 3 Mozammel Haque-DDFP, Kamrul Hasan-ADFP, Abdur Rahman- ADCC 27. 2 19 February 2023# Mymensingh 2 Mohammad Kamal Hossain￾UFPO, Nargis Mursheda- ADCC 28. 2 20 February 2023# Faridpur 2 Hanida Parveen Reshma- MO, Shariful Islam- MO 29. DGHS 2 6 February 2023# Sylhet 2 Najia Sultana-SACMO, Uma Rani Talukdar-CHCP 30. 4 7 February 2023# Sylhet 4 Ahmad Sirajum Munir￾UH&FPO, Dr Rizwana￾Consultant OBGYN, Rumana Akter-Midwife, Rasna Akter￾Midwife, 31. 1 14 February 2023# Faridpur 1 Amena Begum-SACMO 32. 1 1 March 2023# 1 Dr Amanullah-DPM, School Health 33. 1 4 March 2023# 1 Shamsul Hoque-PM, School Health 34. 1 19 February 2023# Ishwarganj 1 Lipi Begum-Midwife 35. MFSTC 2 11 February 2023# 2 Helena Jabeen-Senior Consultant, Sumana Das￾Medical Officer 36. Implementing Partners of AUAFP 1 13 February 2023# 7 Halima Begum-FPAB, Rezaul Karim Bhuiyan- PHD, Fatema Showkat Jahan-PHD, Songita Sarker- SERAC, Santosh Kumar Roy Tigga- ESDO, Abdul Bari Sarkar- ESDO, Salim Miah￾SERAC 9 67 144 *Senior management of RDM, icddr,b and the Evaluation Team were present # The Evaluation Team was present Performance Evaluation of USAID’s Accelerating Universal Access to Family Planning (Shukhi Jibon) in Bangladesh | 89 Annex VI: List of KIIs and GIs Table 1: Data collection for evaluation on Shukhi Jibon’s investments on FP Training Institutes Office/Facilities KII GI Persons identified Process Observation National Institute of Population Research and Training (NIPORT) 5 KII 4 KIIs: DG-NIPORT, Director Training Deputy Director Regional Population Training Center (RPTI)- Faridpur and Sylhet 2KII 2 KIIs: Principal in Faridpur and Sylhet Processes, tools, records and reports etc. Regional Training Center (RTC)- Dhamrai, Ishwarganj 2 GIs 2 GIs: with training officers and AV operator Processes, tools, records and reports etc. Regional Clinical Training Center (RCTC)- 2 KII 2 KIIs: Regional Consultant FPCS-QIT Processes, tools, records and reports etc. Table 2: Data collection for evaluation on Shukhi Jibon’s investments on mentorship, and supportive supervision Office/Facilities KII GI Persons identified/Methods Process Observation Community Clinic (CC), Union Health & Family Welfare Center (UH&FWC), Upazila Health Complex (UHC), Mother and Child Welfare Center (MCWC) ● Faridpur Sadar ● Ishwarganj, Mymensingh ● Sylhet Sadar 5 KIIs with mentors (1 from each category of facilities). 14 KIIs with mentees 1 from each category of facilities). MO MCH-FP, UFPO, FWV, Midwives, CHCP Mohammadpur Fertility Services and Training Center (MFSTC) 1GI 1 GI with senior consultant and MO Performance Evaluation of USAID’s Accelerating Universal Access to Family Planning (Shukhi Jibon) in Bangladesh | 90 Table 3: Data collection for evaluation on Shukhi Jibon’s investments on adolescent friendly health services Office/Facilities KII GI Persons identified Process Observation 6 Health Facilities (4 UH&FWC, 3 UHC, 1 MCWC, 1 FP model clinic, 1 CC) 4 Schools 3 Communities Faridpur, Sylhet, Ishwarganj 7 KIIs with PM, DPM, FWV, SACMO, district manager FPAB 5 GIs with adolescents in schools (schools implementing adolescent interventions 3 GIs with adolescents at the community level to determine their views on community adolescent sessions (Community sessions) Providers of AYFS services at facility and community levels, adolescents and teachers involved in the process 6 Observations (2 at each level) of adolescent interventions at adolescent corners Table 4: Data collection plan for evaluation on all Shukhi Jibon investment areas Office/Facilities KII GI Persons identified Directorate General of Family Planning (DGFP) 5 KIIs with relevant officials of Directorate General of Family Planning DG LD- MNRAH LD-CCSDP LD-FSD Director IEM Director MIS PM Directorate General of Family Planning (DGFP) at Sylhet, Faridpur, Dhamrai, Iswarganj 7 KII Divisional Director DDFP ADFP ADCC UFPO AUAFP activity HQ and Field staff 8 GIs with AUAFP management to discuss each evaluation question CoP (KII) Senior Advisor (KII) GI for CBT including TMIS, TMIF; M&SS; AYFS USAID/Bangladesh 1 GI with USAID AOR AOR (former and current) AUAFP activity regional/district staff 2 GIs with AYFS Implementing Partners regional/district managers (field level) Performance Evaluation of USAID’s Accelerating Universal Access to Family Planning (Shukhi Jibon) in Bangladesh | 91 Annex VII: Locations and catchment areas of 16 NIPORT institutes (NIPORT HQ, 5 RPTIs and 10 RTCs) and 8 DGFP institutes (MFSTC, MCHTI, FWTI and 5 RCTCs) SL# NAME OF TRAINING INSTITUTE CATCHMENT DISTRICTS 1 NIPORT HQ NATIONAL (ALL DISTRICTS) 2 RTC, DHAMRAI (NIPORT) DHAKA, MUNSHIGANJ, MANIKGANJ AND NARAYANGANJ 3 MFSTC (DGFP) ALL DISTRICTS 4 MCHTI (DGFP) DHAKA, MUNSHIGANJ, CUMILLA AND BARISAL 5 FWTI (DGFP) DHAKA, NARAYANGANJ, NARSHINGDI, GAZIPUR, MYMENSINGH, AND MUNSHIGANJ 6 RPTI, FARIDPUR (NIPORT) FARIDPUR, RAJBARI, GOPALGANJ, SHARIATPUR, MUNSHIGANJ AND MANIKGANJ 7 RCTC, FARIDPUR (DGFP) FARIDPUR, RAJBARI, GOPALGANJ, SHARIATPUR, MADARIPUR, BAGERHAT AND KUSHTIA 8 RPTI, TANGAIL (NIPORT) TANGAIL, MYMENSINGH, NETROKONA, KISHOREGANJ AND SHERPUR 9 RCTC, MYMENSINGH (DGFP) MYMENSINGH, KISHOREGANJ, NETROKONA, SHERPUR AND JAMALPUR 10 RPTI, SYLHET (NIPORT) SYLHET, MOULVIBAZAR, HABIGANJ AND SUNAMGANJ 11 RCTC, SYLHET (DGFP) SYLHET, MOULVIBAZAR, HABIGANJ AND SUNAMGANJ 12 RPTI, CUMILLA (NIPORT) CUMILLA, NOAKHALI, CHANDPUR, LAKSHMIPUR AND FENI 13 RCTC, CUMILLA. (DGFP) CUMILLA, NOAKHALI, CHANDPUR, LAKSHMIPUR, FENI, AND BRAHMANBARIA 14 RCTC, CHATTOGRAM (DGFP) ALL DISTRICTS OF CHATTOGRAM DIVISION 15 RPTI, RANGAMATI (NIPORT) FENI, CHATTOGRAM, KHAGRACHARI, BANDARBAN AND RANGAMATI 16 RTC, BHANGA (NIPORT) FARIDPUR, MADARIPUR, SHARIATPUR, MUNSHIGANJ 17 RTC, GHATAIL (NIPORT) TANGAIL, GAZIPUR AND NARSHINDI 18 RTC, ISHWARGANJ (NIPORT) MYMENSINGH, KISHOREGANJ AND NETROKONA 19 RTC, MELANDAHA (NIPORT) JAMALPUR, MYMENSINGH AND SHERPUR 20 RTC, JAMALGANJ (NIPORT) SUNAMGANJ, HABIGANJ, MOULVIBAZAR AND SYLHET 21 RTC, SITAKUNDA (NIPORT) CHATTOGRAM, CUMILLA, FENI AND KHAGRACHARI 22 RTC, KAPTAI (NIPORT) RANGAMATI, COX’S BAZAR, BANDARBAN AND 9 UPAZILAS OF CHATTOGRAM 23 RTC, SHAHRASTI (NIPORT) CHANDPUR, BRAHMANBARIA AND HOBIGONJ 24 RTC, NOAKHALI (NIPORT) NOAKHALI AND LAKSHMIPUR Performance Evaluation of USAID’s Accelerating Universal Access to Family Planning (Shukhi Jibon) in Bangladesh | 92 Annex VIII: List of Trainers Trained on CBT and Adult Learning Methodology 1. National Level Name Designation Place of Posting Trainers from DGFP 1 Dr. Nurun Nahar Begum Line Director, CCSDP DGFP 2 Dr. Md. Abdul Haque AD-CC/ FPCS-QIT Regional Consultant DGFP 3 Dr. Zebunnessa Hossain Principal, Faridpur FWVTI 4 Dr. Maksuda Begum Deputy Director MFSTC 5 Dr. Rehana Akter MO(MCH-FP) Mirpur, Dhaka cut 6 Dr. Md. Sanowar Hossain FPCS-QIT Faridpur 7 AKM Salim Bhuiyan ADFP Sylhet Trainers from NIPORT 1 Dr. Md Saiful Islam Assistant Director NIPORT 2 Md. Mahfuzur Rahman Assistant Director NIPORT 3 Biswajit Baishya Assistant Director NIPORT 4 Dr. Hari Chand Shill Principal, Faridpur FWVTI 5 Md.Obidur Rahman Sarder Principal, Rangamati RPTI 6 Anita Baroi Field Trainer RPTI, Faridpur 7 Shekhar Dev Training Officer 8 Md. Obeydur Rahman Sarder Principal RPTI, Rangamati 9 Amdadul Haque Khan Principal RPTI, Sylhet 10 Makbul Murshid Training Officer RTC, Ishwarganj 11 Shafiul Azmain Home Economist, Gazipur NIPORT Trainers from DGHS 1 Dr. Noor Rifat ara UH&FPO, Dhamrai DGHS Trainers from DGNM 1 Sayeda Mahfuja Aktar Midwife, DGNM Muktaghacha Dhaka Division: 2. Dhaka District 2 Name Designation Place of Posting Trainers from DGFP 1 Dr. Lutful Kabir Khan Deputy Director CCSDP 2 Dr. Nasreen Zaman Deputy Director CCSDP 3 Dr. Md. Rafiqul Islam Talukder Assistant Director CCSDP 4 Dr. Fahmida Shahnaz Assist. Training Coordinator MCHTI, Azimpur 5 Krishna Saha Trainer FWVTI, Dhaka 6 Sabina Nasrin Trainer FWVTI, Dhaka 7 Dr. Hosne Ara Akhter Jr. Consultant MFSTC, Dhaka 8 Dr. Suchitra Saha Medical Officer MFSTC, Dhaka 9 Dr. Gulnahar Begum Medical Officer MFSTC, Dhaka 10 Dr. Lutfunnaher Medical Officer MFSTC, Dhaka Trainers from NIPORT 1 Mokima Shirin Training Officer RTC, Dhamrai Trainers from DGHS 1 Dr. Md. Shahin MOCS CS Office, Dhaka 2 Dr. Zhumana Ashrafi Sweety Medical Officer CS office, Dhaka 3. Dhaka District 3 Performance Evaluation of USAID’s Accelerating Universal Access to Family Planning (Shukhi Jibon) in Bangladesh | 93 Name Designation Place of Posting Trainers from DGFP 1 Sayeda Umme Kaosar Ferdousi Assistant Director NIPORT 2 Dr. Helena Jabeen Sr. Consultant, MFSTC MFSTC 3 Dr. Anup Sarker Apu MO (MCH-FP), MFSTC MFSTC 4 Dr. Samsun Naher MO (Obs/Gyn), MFSTC MFSTC 5 Dr. Sumana Das MO (MCH-FP), MFSTC MFSTC 6 Dr. Nasima Begum Jr Consultant (Obs/Gyn), MCHTI MCHTI 7 Dr. Rokeya Khatun MO (Obs/Gyn), MCHTI MCHTI 8 Dr. Mahbuba Nargis Jr Consultant, MCHTI MCHTI 9 Dr. Rehana Akter MO (MCH-FP), Mirpur FPO Mirpur FPO 10 Dr. Shahnaz Khan MO (MCH-FP), Tejgaon FPO Tejgaon FPO 11 Dr. Nasrin Begum MO (Obs/Gyn), MCHTI MCHTI 12 Dr. Selina Begum MO (Obs/Gyn), MCHTI MCHTI 13 Dr. Nadira Afroz Jr. Consultant, MCHTI MCHTI 4. Dhaka District and Manikganj District 4 Name Designation Place of Posting Trainers from DGFP 1 Dr. Maksuda Khanom ADCC & District Consultant Manikganj 2 Md. Mezbah Uddin UFPO Savar 3 Dr. Qamrun Nahar MO (MCH-FP) Savar 4 Tania Parvin UFPO Dhamrai 5 Dr. Md. Iqbal Kabir MO-MCHFP Dhamrai 6 ABM Shahinuzzaman UFPO Manikganj 7 Md. Arifur Rahman UFPO Manikganj 8 Md. Abul Kasham UFPO Manikganj 9 Md. Qutub Uddin Chowdhury ADFP Manikganj 10 Momotaj Aktar UFPO Manikganj 11 Md. Golam Nobi DDFP Manikganj 12 Dr. Md. Nazrul Islam MOMCHFP Manikganj 5. Dhaka District 5 Name Designation Place of Posting Trainers from DGFP 1 Dr. Shamsun Nahar Jr. Consultant MFSTC 2 Dr. Selina Bulbul MO (Obs/Gyn) MFSTC 3 Dr. Tanzima Haque Nimmi MO (Obs/Gyn) MFSTC 4 Dr. Shahera Begum MO (Obs/Gyn) MFSTC 5 Dr. Sumaiya Fazrin MO(Paediatrics) MFSTC 6 Dr. Jasmin Ara Begum Senior Consultant (Obs/Gyn) MCHTI Performance Evaluation of USAID’s Accelerating Universal Access to Family Planning (Shukhi Jibon) in Bangladesh | 94 7 Dr. Nahid Sultana Assistant Coordinator(Training) MCHTI 8 Dr. Nilufer Yesmin Senior Consultant (Obs/Gyn) MCHTI 9 Dr. Nahida Naznin Senior Consultant (Obs/Gyn) MCHTI 10 Dr. Mahmuda Akhter Junior Consultant (Obs/Gyn) MCHTI 11 Most. Shamima Akter Nursing Instructor, MCHTI MCHTI 6. Dhaka District 6 Name Designation Place of Posting Trainers from DGFP 1 Dr. Syeda Shamianaz Lecturer (Medical) FWVTI, Dhaka 2 Dr. Kaisaree Farhat Arpita MOMCHFP FWVTI, Dhaka 3 Kamrun Nahar Nurse-Midwife FWVTI, Dhaka 4 Sabina Nasrin Field-Trainer FWVTI, Dhaka 5 Dr. Taraque Ahamed Shawon MO(Paediatrics) MCHTI, Azimpur 6 Dr. Md. Ahsan Habib MO(Anesthesia) MCHTI, Azimpur 7 Dr. Ishrat Afroz Antora MO(Gyn/Obs) MCHTI, Azimpur 8 Dr. Nur E Akhter Tahmina Quader MO MCHTI, Azimpur 9 Dr. Kohinoor Akhter Khanam MO(Paediatrics) MCHTI, Azimpur 10 Dr. Rahima Khatun Junior Consultant (Infertility) MFSTC, Dhaka 11 Dr. AKM Shahjanur Alam Shamim Junior Consultant (Anesthsiology) MFSTC, Dhaka 12 Dr. Swapna Rani Roy MO(Obs/Gyne) MFSTC, Dhaka 13 Dr. Tanvir Ahmed Bhuiyan MO MFSTC, Dhaka Trainers from NIPORT 14 Hero Dhar Instructor NIPORT 15 Dr. A F M Radwanoor Rahman Instructor NIPORT 16 Aklima Begum Instructor NIPORT 7. Faridpur District Name Designation Place of Posting Trainers from DGFP 1 Dr. Shahana Sultana MO-Clinic, MCWC Sadar 2 Md. Soyeb Ali Miah UFPO Modhukhali 3 A. H. M Mashiur Rahman UFPO Nagarkanda 4 Dr. Shireen Akter MO(MCH) Boalmari 5 Rabin Biswas UFPO Boalmari 6 Dr. Md. Sazzad Hossain MO (MCH-FP) Bangha 7 Sirajul Islam UFPO Sadarpur Trainers from NIPORT 1 Selina Akther Banu Field Trainer RPTI, Faridpur Trainers from DGHS 1 Dr. Md. Shafiq Ullah UH&FPO Sadarpur 2 Dr. Mahmudul Hasan UH&FPO Sadar Performance Evaluation of USAID’s Accelerating Universal Access to Family Planning (Shukhi Jibon) in Bangladesh | 95 8. Rajbari District Name Designation Place of Posting Trainers from DGFP 1 Dr. Md. Abdul Quddus MO (CC) Sadar 2 Dr. Gopal Chandra Sutradhar MO Clinic Sadar 3 Md. Mahmud Hasan Khan UFPO Baliakandi 4 Dr. Md. Neamot Ullah MOMCH-FP Sadar 5 Dr. Md. Abdur Rahman MOMCH-FP Goalanda 6 Mst. Latifa Pervin AFWO Goalanda 7 Khondaker Safiqul Islam UFPO Pangsha 8 Khan A Al Mamun AUFPO Pangsha 9 Sutapa Karmaker AUFPO Baliakandi 10 Dr. M. Zahiduzzaman MOMCH-FP Baliakandi 9. Gazipur District Name Designation Place of Posting Trainers from DGFP 1 Dr. Rownak Afroz Chowdhury MO(MCH-FP) Sadar 2 Sanjida Ahmed UFPO Kaliganj 3 Dr. Md. Golam Awal MO(MCH-FP) Tongi 4 Jinat Sharmin UFPO Sreepur 5 Dr. Md. Monzur Alom MO(MCH-FP) Sreepur 6 Dr. Abu Hasan Mostafa MO(MCH-FP) Kapasia 7 Dr. Jobayra Hasin MO(MCH-FP) Kaliakair 8 Mir Masud Rahman UFPO Kaliakair 9 Mohammad Abdur Rahim UFPO Kapasia 10 Dr. Md. Azizul Islam MO(MCH-FP) Sadar 11 Fowzia Asmat ADFP 12 Begum Noor-a-Jannat ADFP 10. Kishoreganj District Name Designation Place of Posting Trainers from DGFP 1 Dr. Husna Begum ADCC Sadar 2 Dr. Jashoda Dulal Saha FPCS QIT Sadar 3 Dr. Halima Akhter MO (CC) Sadar 4 Md. Safi Uddin UFPO Bhoirab 5 Md. Saidur Rahman UFPO Katiadi 6 Md. Selim Miah UFPO Karimganj 7 Md. Anjuman Islam UFPO Tarail 8 Dr. Md. Abdur Rahman Miah MO MCH FP Tarail 9 Masuma Khatun Safty UFPO Sadar 10 Dr. Md Mizanur Rahman MOMCHFP Pakundia 11 Dr. Mitali Das Gupta MOMCHFP Bhoirab Trainers from DGHS Name Designation Posting 1 Dr. Md. Mostafizur Rahman DCS Sadar 2 Dr.Jahir Ahmed Talukder MOCS Sadar 3 Dr.Tajrina Tayub UH&FPO Sador Performance Evaluation of USAID’s Accelerating Universal Access to Family Planning (Shukhi Jibon) in Bangladesh | 96 4 Dr. Saurovo Kishore Das MO Itna 5 Dr. Kishore kumar Dhar RMO Bhairab 11. Tangail District Name Designation Place of Posting Trainers from DGFP 1 Dr. Md. Farhad Ali Khan ADCC and FPCS-QIT Sadar 2 Eivy Yasmin AD (FP) Sadar 3 Dr. Akhtar Zahan MO Clinic Sadar 4 Dr. Md. Abdul Alim MO (MCH-FP) Sadar 5 Nusrat Rashid UFPO Sadar 6 Dr. Masud Murshed Talukder MO (MCH-FP) Kalihati 7 Dr. Md. Rafikul Islam MO (MCH-FP) Sakhipur 8 Dr. Khalid Bin Kashim MO (MCH-FP) Nagarpur 9 Md. Abdul Mannan UFPO Modhupur 10 Dr. Md. Majnu Miah MO (MCH-FP) Delduar 11 Dr. Priyanka Das MO (MCH-FP) Basail 12 Dr. Soheli Sharmin MO (MCH-FP) Mirzapur 13 Dr. Md. Arif Hossain MO (MCH-FP) Mirzapur Trainers from NIPORT 1 Md. Motiar Rahman Principal Sadar 2 Omar Faruk Training Officer Ghatail 3 Taslima Islam Lecturer Sadar 4 Sultana Razia Assistant Trainer Ghatail 5 Abdullah Al Masud Lecturer Ghatail Trainers from DGHS 1 Dr. Saifur Rahman Khan UH&FPO Ghatail 2 Dr. Nazma Khalil Consultant (Gyn) Dhanbari Mymensingh Division 12. Mymensingh District Name Designation Place of Posting Trainers from DGFP 1 Dr Hosne Ara Begum MO-CC Sadar 2 Dr. Shamima Anis MO-MCH FP Sadar 3 Kazi Mahfuzul Karim UFPO Sadar 4 Mohammad Kamal Hossain UFPO Ishwarganj 5 Dr. Muklesur Rahman Khan MO-MCH FP Ishwarganj 6 Md. Ali Amzad Doptori UFPO Muktagacha 7 Dr. Nargis Murshida MO-MCH FP Muktagacha 8 Mahbuba Aziz UFPO Goforgoan 9 Dr. Noor Mohammad MO-MCH FP Haluaghat 10 Dr. Ferdous Ara Akter MO-MCH FP Gouripur Trainers from NIPORT 1 Sandha Karmoker Assistant Trainer Ishwarganj RTC Trainers from DGHS 1 Dr. Rehana Naznin MO Ishwarganj 2 Dr. Rahat Hossain MO Mymensingh Sadar 13. Netrokona District Name Designation Place of Posting Trainers from DGFP Performance Evaluation of USAID’s Accelerating Universal Access to Family Planning (Shukhi Jibon) in Bangladesh | 97 Name Designation Place of Posting 1 Dr. Md. Abdul Qayyum Anwar ADCC Netrokona Sadar 2 Dr.Md.Fazlur Rahman District Consultant Netrokona Sadar 3 Md. Rafiqul Islam UFPO Netrokona 4 Shovon Kumer Ghosh UFPO Atpara 5 Sudeb Karmakar UFPO Mohonganj 6 Shahidul Islam Khan UFPO Purbodhola 7 Mashiur Rahman UFPO Durgapur 8 Dr. Abdul Karim Bhuiyan MOMCH-FP Madon 9 SM Toukir Alam UFPO Khaliajuri 10 Dr. Kripanath paul MOMCH-FP Netrokona 11 Dr. Papia Majumdar MO (Clinic) Sadar 12 Dr. Bijoy Prokash Biswas MOMCH-FP Kolmakanda 13 Dr. ABM Golam Faruk MOMCH-FP Mohonganj 14 Dr Rubina jahan Nusheen MOMCH-FP Purbodhola 15 Dr. Md. Firoj Khan Pathan MOMCH-FP Atpara 16 Mahmudul Hasan FP Facilitator Netrokona Sadar Trainers from DGHS 1 Dr. Uttam Kumar Paul MO CS Netrokona Sadar 2 Dr. Mahbuba Khan MO DC Netrokona Sadar 14. Jamalpur District Name Designation Place of Posting Trainers from DGFP 1 Humayun Kabir Talukadar Additional Director DGFP Head office 2 Dr. Md. Asadul Islam ADCC (FPCS-QIT Consultant) Sherpur District 3 Faruk Al Faisal UFPO Islampur 4 Md. Golam Rabbani UFPO Sharishabari 5 Kohinur Begum UFPO Melandah 6 Mazedur Rahman UFPO Sadar 7 Dr. Sonia Akhter MOMCH-FP Transfer to Tangail 8 Dr. Omme Habiba MO Clinic MCWC Sadar, Jamalpur Trainers from DGHS 1 Dr. K.M Shafiquzzaman DCS CS Office 2 Dr. Md. Fazlul Hoque UH&FPO Melandaha 3 Dr. A. A. M. Abu Taher UH&FPO Islampur 4 Dr. Saiful Islam UH&FPO Madarganj 5 Dr. Md. Mahfuzur Rahman AD 250 Bed General Hospital, Jamalpur 6 Dr. Syed Abu Ahammad Shafi UH&FPO Transfer to Sunamganj 7 Dr. Munayem Hussain Khan MO Melandaha Sylhet Division 15. Sylhet District Name Designation Place of Posting Trainers from DGFP 1 AKM Abdus Subhan Assistant Director (FP) (Divisional Office) Divisional Director-FP Office 2 Dr. Mst Omar GuI Azad MOCC/FPCS QIT Sadar, Sylhet 3 Dr. Nazrul Islam MOMCH FP Sadar, Sylhet 4 Dr. Hamida Begum MOMCH FP Balaganj, Sylhet Performance Evaluation of USAID’s Accelerating Universal Access to Family Planning (Shukhi Jibon) in Bangladesh | 98 5 Dr. Md. Sadik Miah MOMCH FP Companiganj, Sylhet 6 Abul Mansor Ashzad UFPO Bianibazar, Sylhet Trainers from NIPORT 1 Dr. Sumit Kumar Basak Lecturer (Medical) RPTI, Sylhet 2 Mrs. Rahima Akhtar Field Trainer RPTI, Sylhet 3 Mrs. Gul Nahar Begum Field Trainer RPTI, Sylhet 4 Mrs. Momtaz Begum Field Trainer RPTI, Sylhet Trainers from DGHS 1 Dr. Ahmed Sirajum Munir UHFWO Sadar 2 Dr. Amzad Hossain MO(CS), Sylhet CS Office, Sylhet 16. Sunamganj District Name Designation Place of Posting Trainers from DGFP 1 Biswajit Krisna Chakroborty UFPO Sadar, Sunamganj 2 Dr. Md. Jasim Uddin Khan MO-Clinic MCWC, Sadar, Sunamganj 3 Chowdhury Rajib Mustafa MO (MCH-FP) Upazila FP Office, Chattak 4 Ripon Chandra Das UFPO Upazila FP Office, Dowarabazar 5 Dr. Bipasha Mazumder MO (MCH-FP) Upazila FP Office, Dharmapasha 6 Dr. Md. Shahidul Islam MO (MCH-FP) Upazila FP Office, Chattak Trainers from NIPORT 1 Md. Abdullah Al Mamun Training Officer RTC, Jamalganj Trainers from DGHS 1 Dr. A. S. M. Abdul Momen Assistant Director District Hospital, Sunamganj 2 Dr. Mohammad Ashraful Haque DCS CS Office, Sunamganj 3 Mohammad Omar Faruq Ser. Health Education Officer CS Office, Sunamganj 4 Mohammad Abdur Rahman UFPO (In-Charge) Upazila FP Office, Biswamburpur 5 Dr. Manisar Chowdhury UH&FPO UHC, Jamalganj 6 Dr. Mahbubur Rahman UH&FPO UHC, Derai 7 Dr. Saumitra Chakrabarti UH&FPO Sadar, Sunamganj 8 Dr. Rajib Chakravarty UH&FPO UHC, Chattak 17. Moulvibazar District Name Designation Place of Posting Trainers from DGFP 1 Dr. Biswajit Vhowmik MO-Clinic Moulvibazar Sadar 2 Madhusudan Paul Chowdhury UFPO Kulaura 3 Md. Rashidul Hasan UFPO Moulvibazar Sadar 4 Dr. Md. Abdul Mukid MO-MCHFP Barlekha 5 Dr. Md. Sultan Ahmed MO-MCHFP Kulaura 6 Md Abdul Matin UFPO(In Charge) Juri 7 Dr. Ranjan Chandra Dash MO-MCHFP Sreemangal 8 Maksuda Yeasmin UFPO (In- Charge) Barlekha 9 Rantu Purkayestha UFPO (In- Charge) Sreemangal 10 Kane Laskar FP Facilitator Moulvibazar Sadar 11 Dr. Md. Abdul Mannan ADCC/ District Consultant Moulvibazar Sadar Trainers from DGHS Name Designation Posting 1 Dr. Roksana Wahid Rahi MO-CS Moulvibazar Sadar Performance Evaluation of USAID’s Accelerating Universal Access to Family Planning (Shukhi Jibon) in Bangladesh | 99 2 Dr. Priyanka Bhattacharjee Jr. Consultant-OBS-GYN Barlekha 3 Dr. Priyajoti Ghosh Anik RMO Juri 4 Dr. Binendu Bhowmik UH&FPO Moulvibazar Sadar 5 Dr. Tarafdar Tasneem Rahat MO Kamalganj 6 Md. Nasir Uddin Sr. Health Education Office Moulvibazar Sadar Chattogram Division 18. Chattogram District Name Designation Place of Posting Trainers from DGFP 1 Dr. Sheheli Nargis FPCS-QIT Chattogram 2 Dr. Shamoli Das MO MCHFP Bansh Khali 3 Dr. Shamima Hasnat MO MCHFP Boal Khali 4 Dr. Md.Hossain Al Mamun MO MCHFP Fatik Chari UFPO office 5 Dr. Protima Raani Tripura Assist. Surgeon Bakuliia MCWC 6 Dr. Shilpi Choudhury MO Clinic Pachurria MCWC 7 Subrata Kumar Chowdhury Assistant Director DDFP Office, Chattogram 8 Abdur Rohim Choudhury ADCC Chattogram Port 9 Md.Khorshed Alam UFPO UFPO Office, Chattogram 10 Nixon Choudhury UFPO DGFP HQ 11 Abu Saleh Md. Forkan UFPO UFPO Office, Chandainsh Trainers from NIPORT 1 Umma Kulsum Assistant Trainer RTC Sitakunda Trainers from DGHS 1 Dr.Md.Wazed Chowdhury MOCS CS Office, Chattogram 2 Dr.Nurul Haider MODC CS Office, Chattogram 19. Rangamati District Name Designation Place of Posting Trainers from DGFP 1 Dr.Lenin Talukder MO Clinic Rangamati 2 Dr. Susmita Sen MO (MCH-FP) Sadar 3 Dr.Robin Bhattacharjee MO (MCH-FP) Nanir Char 4 Dr. Kamrunessa Akter Dalia MO (MCH-FP) Kaw Khali 5 Dr.Shuvra Palit MO (MCH-FP) Belai Chari Trainers from NIPORT 1 Dr.Moshumi Dey LPHC RTC, Kaptai 2 Lipi Chakma Assist. Trainer RPTI, Rangamati 3 Bulbuli Rani Borua Assist. Trainer RPTI, Rangamati Name Designation Posting 4 Maching Ching Ching Assist. Trainer RPTI, Rangamati 5 Jharna Chakma Assist.Trainer RTC, Kaptai Trainers from DGHS 1 Dr.Utpal Borna Chakma Gynecologist CS Office 2 Dr.M A Hai Consultant Ped. CS Office 20. Cox’s Bazar District Name Designation Place of Posting Trainers from DGFP 1 Dr. Md. Rakib Ullah ADCC Sadar UFPO office Performance Evaluation of USAID’s Accelerating Universal Access to Family Planning (Shukhi Jibon) in Bangladesh | 100 2 Jashim Uddin Md. Usuf UFPO Ukhia FP OFFICE 3 Dr. Maziibul Hoque Sr. MO Ukhia FP OFFICE 4 Shruti Purna Chakma UFPO Teknaf UFPO office 5 Dr. AKM Hedayatul Islam MOMCH Teknaf UFPO office 6 Dr. Soma Chawdhury MO Clinic MCWC Sadar Cox Bazar 7 Dr Shirajum Monira MOMCH FP Chokoria 8 Md. Sadikur Rahman UFPO Ramu 9 Bidhan Kanti Rudraho UFPO Chokoria 21. Feni district Name Designation Place of Posting Trainers from DGFP 1 Md. Zakir Hossain UFPO Porshuram FP office 2 Ahammed Karim UFPO Fulgazi FP office 3 Dr. Taslima Akhter MO MCHFP Chagal Naya FP office 4 Dr. Nasrin Sultana MO MCHFP Sadar FP office 5 Saheda Hossain UFPO Sona gazi FP office 6 Dr. Nasrin Akhter MO MCHFP Sadar FP office 7 Md. Khalilur Rahman UFPO Dagonbhuyan FP office 8 Iffekher Ahmmed Chowdhury UFPO Sadar FP office 9 Siraj Ullah AUFPO Sadar FP office 10 Dr. Belal Uddin Ahmmed MO Clinic Sadar MCWC 11 Dr. Md. Rezaul Karim MO MCHFP Sona gazi FP office 12 Jahirul Huda AUFPO Chagal Naya FP office Trainers from DGHS 1. Dr. Tahira Khatun Consultant Gyn Sona gazi UHC 2. Dr. Shamima Akhter Consultant Gyn Chagal Naya UHC 3. Dr. Dilruba Yesmin Consultant Gyn Dagonbhuyan UHC 4. Dr. Kamrunnahar Consultant Gyn Fulgazi UHC 22. Bandarban District Name Designation Place of Posting Trainers from DGFP 1 Emran Hossain Chowdhury ADFP DDFP Office 2 Dr. NuruSafa Chowdhurys MOCC DDFP Office 3 Dr. Kamrul Monir Ribbon MO Clinic MCWC 4 Ashim Chakma UFPO Sadar FP Office 5 Sahana Begum AUFWO Sadar FP Office 6 Shanti Joy Tanchangya UFPO Rowangchari FP office 7 Md. Nazmul Hasan Sayed MO MCHFP Rowangchari FP office 8 Ratan Bikash Chakma UFPO Ruma FP Office 9 Anadi Ranjan Barua UFPO Thanchi FP Office 10 Zubara begum UFPO Lama FP Office 11 Shamsun Nahar Lipe AUFWO Lama FP Office 12 M.Didarul Alam UFPO Alikadama FP Office 13 Dr. Belal Uddin Ahmed MO MCHFP Alikadama FP Office 14 Ditiya Moy Chakma UFPO Naikhongchari FP Office Trainers from DGHS 15 Dr. Naznin Ahmed Junior Consultant (GYN) Sadar Hospital 23. Noakhali District Name Designation Place of Posting Performance Evaluation of USAID’s Accelerating Universal Access to Family Planning (Shukhi Jibon) in Bangladesh | 101 Trainers from DGFP 1 Dr. Md. Kamrul Hasan ADCC & FPCSQIT Sadar 2 Dr. Muhammad Zihadul Haque MOMCH-FP Sadar 3 Mahmudul Hasan UFPO Sadar 4 Mostafizur Rahman Chowdhury UFPO Begumganj 5 Mahedee Hasan AUFPO Hatiya 6 Md. Mahabubul Alam UFPO Subarna Char 7 Dr. CM Zahirul Haq MOMCH-FP Begumganj 8 Faruque Abdullah ADFP 9 Md. Lokman Hossain UFPO Kabirhut 10 Dr. Md. Ghulam Haider MOMCH-FP Kabirhut 11 Md. Mostafa Kamal UFPO Companyganj 12 MD.Nazmul Hasan UFPO Sadar, Lakshmipur 13 Maya Rani Datta Banik UFPO Ramgonj,Lakshmipur 14 Md. Mahmudul Hasan AUFPO Chatkhil 15 Md. Nizam Uddin AUFPO Companyganj Trainers from NIPORT 1 Debashis Dev Training Officer RTC, Begumganj 24. Chandpur District Name Designation Place of Posting Trainers from DGFP 1 Dr. Nasir Ahmed District Consultant Matlab(North 2 Dr. Tonmoy Barua MO-MCH-FP Matlab (South) 3 Dr. Aklima Akhter peya MO-MCH-FP Sharashti 4 Dr. Latifa Nasrin MO(Clinic) Sadar 5 Dr. Suborna Roy Tuli MO-MCH-FP Kochua 6 Dr. Sharin Afrin MO-MCH-FP Haim Char 7 A T M Burhan Uddin UFPO Matlab (South) 8 Md. Noor Nobi AUFPO Matlab(North) 9 Md. Mohfizur Rahman UFPO Kochua 10 Md. Tanvirul Islam AUFPO Chandpur Sadar Trainers from NIPORT 1 Mohamed Abdur Rahaman Training Officer Noakhali RTC 2 Dr. Md. Tahedul Islam Patwary Training Officer Sahrasthi RTC 3 Dr. Sharmin Akhter PHC Noakhali RTC Trainers from DGHS 1 Dr. Mohammad Golam Kawser UH&FPO Motlab(South) 2 Dr. Sajeda Begum UH&FPO Sadar 3 Dr. Md. Issa Ruhullah MO-CS CS Office 25. and 26. Cumilla District Name Designation Place of Posting Trainers from DGFP 1 Dr. Sumiya Khan MO (MCHFP) Nangolkot 2 Chowdhury Murshed Alam UFPO Addarsha Sadar 3 Md. Abu Kawsar UFPO Chandina 4 Fatema Begum Soma UFPO Addarsha Sadar 5 Dr. Gobinda Kumar Saha MO (MCH-FP) Addarsha Sadar 6 AKM Aminul Islam UFPO Daudkandi 7 Dr. Mohammad Zakir Hossain MO (MCH-FP) Addarsha Sadar 8 Dr. Md. Golam Sarwar MO (MCH-FP) Barura 9 Dr. Mahmuda Nasreen MO (MCH-FP) Chandina Performance Evaluation of USAID’s Accelerating Universal Access to Family Planning (Shukhi Jibon) in Bangladesh | 102 10 Dr. Ruby Das MO (MCH-FP) Monohorgonj 11 Dr. Kamrun Nahar Bhuiyan MO (MCH-FP) Devidwar 12 Dr. Faysal Mahmud Mazumder MO (MCH-FP) Barura 13 Dr. Abu Jafor Mohammad Saleh MO (MCH-FP) Titas 14 Dr. Shahnaz Arfin AD (CC) Adarsha Sadar Name Designation Posting 15 Md. Zakir Hossain UFPO Meghna 16 Muhammad Mizanur Rahman UFPO Muradnagar 17 Roksana Khanom Munni UFPO Burichong 18 Ranjit Sen UFPO Debidwar 19 Roksana Akhter UFPO Monohorgonj 20 Md. Faruk Ahmed UFPO Brahmmonpara 21 Dr. Rehena Ferdoues MO (MCH-FP) Muradnagar 22 Dr. AKM Hedayatul Islam MO (MCH-FP) Burichong 23 Md. Haroon-Al- Rashid AUFPO Homna 24 Rubal Chandra Das AUFPO Nangolkot Trainers from NIPORT 1 Gias Uddin Ahmed Principle RPTI RPTI, Cumilla 2 U.S Khaleda Field Trainer NIPORT 3 Mosammat Shamima Aktar Nurse Midwives Adarsha Sadar Trainers from DGHS 1 Dr. Md. Enamul Hoque MO CS, DGHS Adarsha Sadar 2 Dr. Ishrat Jahan MO(DRS) DGHS Adarsha Sadar Trainers from DGNM 1 Akbori Khanum Nursing Instructor in Charge Adarsha Sadar 27. Combined CBT (25-28 September 2022) Name Designation Place of Posting Trainers from DGFP 1 Dr. Anamika Das MO(Obs/Gyn) MFSTC, Dhaka 2 Dr. Md. Tariqul Islam MO(Paediatrics) MCHTI, Azimpur, Dhaka 3 Dr. Foysal Siddkee MOMCHFP Sadar, Mymensingh 4 Md. Kamal Hossain UFPO Gouripur, Mymensingh 5 Kanij Fatema UFPO Tongi, Gazipur 6 Dr. Farzana Haque MOMCHFP Pubail Gazipur 7 Md. Akib Uddin ADFP Cumilla 8 Dr. Abdul Quader MOMCHFP Sadar, Moulvibazar 9 Dr. Subimol Chanda MOMCHFP Juri Moulvibazar Trainers from DGHS 10 Dr. Ahmedul Haque Titas RMO UHC Dhamrai 11 Dr. Neher Banu Jr. Consultant (Gyne) UHC Dhamrai 12 Dr. Anisa Hossain MO UHC Dhamrai 13 Dr. Md Akib Hossain MODC UHC Dhamrai 14 Dr. Tajnaher Begum IMO UHC Dhamrai 15 Dr. Sowmen Roy MO Cumilla Medical College Hospital Performance Evaluation of USAID’s Accelerating Universal Access to Family Planning (Shukhi Jibon) in Bangladesh | 103 Annex IX: List of Participants of TMIS Training SL# Division District Upazila Name Designation Name of training 1 Dhaka Dhaka Tejgaon Didarul Alam Stenotypist TMS Data Entry and Orientation 2 Dhaka Dhaka Tejgaon Sheikh Mohammad Monaf Audio Visual Operator TMS Data Entry and Orientation 3 Chattogram Chandpur Sharasti Dr. Md. TaheduI Islam Patowary Training Officer TMS Data Entry and Orientation 4 Chattogram Chandpur Sharasti Asif Reza Home Economist TMS Data Entry and Orientation 5 Chattogram Chandpur Sharasti Afzal Mia Computer Operator TMS Data Entry and Orientation 6 Chattogram Chattogram Sitakunda Taslima Islam Lecturer TMS Data Entry and Orientation 7 Chattogram Noakhali Begumganj Debashish Deb Training Officer TMS Data Entry and Orientation 8 Chattogram Noakhali Begumganj Hasna Ara Akhter Office Assistant TMS Data Entry and Orientation 9 Chattogram Rangamati Kaptai Shakar Kanti Dev Training Officer TMS Data Entry and Orientation 10 Chattogram Rangamati Kaptai Mizanur Rahman Audio Visual Operator TMS Data Entry and Orientation 11 Chattogram Rangamati Rangamati Sadar Md. Lill Miah Head Assistant TMS Data Entry and Orientation 12 Dhaka Dhaka Dhamrai Bijoy Chandra Sarker Audio Visual Operator TMS Data Entry and Orientation 13 Dhaka Dhaka Dhamrai Harun Or Rashid Computer Operator TMS Data Entry and Orientation 14 Dhaka Dhaka Mirpur Md. Mahfuzur Rahman Assistant Director TMS Data Entry and Orientation 15 Dhaka Dhaka Mirpur Biswajit Baishya Assistant Director (Training) TMS Data Entry and Orientation 16 Dhaka Dhaka Mirpur Faridul Hasan AVO TMS Data Entry and Orientation 17 Dhaka Dhaka Mirpur Md. Abdur Rahman Instructor- NIPORT TMS Data Entry and Orientation 18 Dhaka Faridpur Bhanga Shafiul Azmain Home Economist TMS Data Entry and Orientation 19 Dhaka Faridpur Bhanga Md. Haydar Hossain Cashier TMS Data Entry and Orientation 20 Dhaka Faridpur Faridpur Sadar Md. Sohrab Hossain Stenotypist TMS Data Entry and Orientation 21 Dhaka Gazipur Gazipur Sadar Md. Riajul Haque Principal TMS Data Entry and Orientation 22 Dhaka Tangail Ghatail Md. Abdullah Al Mamun Training Officer TMS Data Entry and Orientation 23 Dhaka Tangail Ghatail Dr. Md. Aminul Islam Training Officer TMS Data Entry and Orientation 24 Dhaka Tangail Ghatail Md. Arshed Ali Audio Visual Operator TMS Data Entry and Orientation 25 Dhaka Tangail Tangail Sadar Md. Matiyar Rahman Principal TMS Data Entry and Orientation 26 Dhaka Tangail Tangail Sadar Parimal Chandra Singha Roy Stenotypist TMS Data Entry and Orientation 27 Mymensingh Jamalpur Melandaha Al Mamum Gyume Training Officer TMS Data Entry and Orientation 28 Mymensingh Jamalpur Melandaha S M Humayon Kabir Audio Visual Operator TMS Data Entry and Orientation 29 Mymensingh Mymensingh Ishwarganj Makbul Murshid Training Officer￾NIPORT TMS Data Entry and Orientation 30 Mymensingh Mymensingh Ishwarganj Saddam Hossain Audio Visual Operator TMS Data Entry and Orientation 31 Mymensingh Mymensingh Ishwarganj Shaurav Chandra Kar Computer Operator TMS Data Entry and Orientation 32 Sylhet Sylhet Sylhet Sadar Md. Abadat Khan Audio Visual Operator TMS Data Entry and Orientation 33 Chattogram Chattogram Sitakunda Dr. Mousumi Dey LPHC Training on TMIS 34 Chattogram Rangamati Kaptai Win Chakma Computer Operator Training on TMIS 35 Chattogram Rangamati Kaptai Ruksana-Fer￾Yesmin Assistant Trainer Training on TMIS Performance Evaluation of USAID’s Accelerating Universal Access to Family Planning (Shukhi Jibon) in Bangladesh | 104 36 Dhaka Faridpur Faridpur Sadar Md. Asiful Islam Raju Lecturer(Social Science) Training on TMIS 37 Dhaka Faridpur Faridpur Sadar Farzana Computer Operator Training on TMIS 38 Dhaka Gazipur Gazipur Sadar Masuda Khatun Nurse Midwife Training on TMIS 39 Dhaka Gazipur Gazipur Sadar Md. Ali Azam Dihider Cashier Training on TMIS 40 Dhaka Tangail Ghatail Md. Mokbul Hossain Office Assistant Cum Computer Operator Training on TMIS 41 Dhaka Tangail Tangail Sadar Abu Sayid Miah Head Assistant Training on TMIS 42 Mymensingh Jamalpur Melandaha Sabekunnahar Assistant Trainer Training on TMIS 43 Mymensingh Mymensingh Ishwarganj Susmita Roy Assistant Trainer Training on TMIS 44 Sylhet Sylhet Sylhet Sadar Md. Hridoy Mia Office Assistant cum Computer Typist Training on TMIS 45 Dhaka Dhaka Mirpur Mr. Abdul Hamid Moral Deputy Director(Training) Training on TMIS for DGFP 46 Dhaka Dhaka Mirpur A.F.M. Arafat Hossain ADFP Training on TMIS for DGFP 47 Dhaka Dhaka Mirpur Md. Nasir Uddin Howlader Stenotypist Training on TMIS for DGFP 48 Dhaka Dhaka Tejgaon Dr. Gopi Nath Basak AD & DPM (QA) Training on TMIS for DGFP 49 Dhaka Dhaka Tejgaon Abu Taher Md. Sanaullah Nuri DD & PM,MIS Training on TMIS for DGFP 50 Dhaka Dhaka Tejgaon Dr. Rafiqul Islam Talukder AD,DGFP Training on TMIS for DGFP 51 Dhaka Faridpur Faridpur Sadar Dr. Md. Sanowar Hossain Khan ADCC & District Consultant Training on TMIS for DGFP 52 Dhaka Faridpur Faridpur Sadar Md. Mozammel Haque DDFP Training on TMIS for DGFP 53 Dhaka Faridpur Faridpur Sadar Mariam Akter UFPA Training on TMIS for DGFP 54 Dhaka Gazipur Gazipur Sadar Lazu Shamsad Haque DD-FP Training on TMIS for DGFP 55 Dhaka Gazipur Gazipur Sadar Dr. Md. Majnu Miah ADCC Training on TMIS for DGFP 56 Dhaka Gazipur Gazipur Sadar Emdadul Islam Stenotypist Training on TMIS for DGFP 57 Dhaka Gopalganj Gopalganj Sadar Md. Moniruzzaman Khan DD-FP Training on TMIS for DGFP 58 Dhaka Gopalganj Gopalganj Sadar Dr. B.M Moniruzzaman ADCC Training on TMIS for DGFP 59 Dhaka Madaripur Madaripur Sadar Biplob Barua DDFP Training on TMIS for DGFP 60 Dhaka Madaripur Madaripur Sadar Md. Rezaul Karim Jr. Assistant Statistician Training on TMIS for DGFP 61 Dhaka Madaripur Madaripur Sadar Sadia Moriom Office Assistant Training on TMIS for DGFP 62 Dhaka Manikganj Manikganj Sadar Md. Golam Nobi DDFP Training on TMIS for DGFP 63 Dhaka Manikganj Manikganj Sadar MD. Abu Sayed Office Assistant Training on TMIS for DGFP 64 Dhaka Manikganj Manikganj Sadar Dr. Maksuda Khanom ADCC & District Consultant Training on TMIS for DGFP 65 Dhaka Munshiganj Munshiganj Sadar Md. Abdus Salam ADFP Training on TMIS for DGFP 66 Dhaka Munshiganj Munshiganj Sadar Ramjan Ali Khan Cashier Training on TMIS for DGFP 67 Dhaka Narayanganj Narayanganj Sadar Md. Shahjalal ADFP Training on TMIS for DGFP 68 Dhaka Narayanganj Narayanganj Sadar Md. Anwar Hossen DDFP Training on TMIS for DGFP Performance Evaluation of USAID’s Accelerating Universal Access to Family Planning (Shukhi Jibon) in Bangladesh | 105 69 Dhaka Narayanganj Narayanganj Sadar Md. Saiful Islam Khan Office Assistant Training on TMIS for DGFP 70 Dhaka Narshingdi Narshingdi Sadar Arabindo Datta DDFP Training on TMIS for DGFP 71 Dhaka Narshingdi Narshingdi Sadar Md. Maruf Rabbe Computer Operator Training on TMIS for DGFP 72 Dhaka Narshingdi Narshingdi Sadar Md. Fazlur Rahman Stenotypist Training on TMIS for DGFP 73 Dhaka Rajbari Rajbari Sadar Md. Golam Azam DDFP Training on TMIS for DGFP 74 Dhaka Rajbari Rajbari Sadar Md. Rafiqul Islam Office Assistant Training on TMIS for DGFP 75 Dhaka Rajbari Rajbari Sadar Shariful Islam Stenotypist Training on TMIS for DGFP 76 Dhaka Rajbari Rajbari Sadar Dr. Md. Neamot Ullah MO (MCH-FP) Training on TMIS for DGFP 77 Dhaka Shariatpur Palong (Sadar) Md. Sohel Parvez DDFP Training on TMIS for DGFP 78 Dhaka Shariatpur Palong (Sadar) Md. Borhan Howlader TFPA Training on TMIS for DGFP 79 Dhaka Tangail Tangail Sadar Dr. Md. Abdul Alim MO (MCH-FP) Training on TMIS for DGFP 80 Dhaka Tangail Tangail Sadar Md. Lutful Kibria DDFP Training on TMIS for DGFP 81 Dhaka Tangail Tangail Sadar Md. Sibli Sadik Stenotypist Training on TMIS for DGFP 82 Chattogram Chattogram Doublemooring Md. Dabir Uddin Office Super Training on Training Management System for DGFP Officials 83 Chattogram Chattogram Doublemooring Farhad Uddin Ahmed Bhuyan Statistical Assistant Training on Training Management System for DGFP Officials 84 Chattogram Chattogram Sitakunda Abdur Rahaman Training Officer Training on Training Management System for DGFP Officials 85 Chattogram Chattogram Sitakunda Chayan Barua Training Officer Training on Training Management System for DGFP Officials 86 Chattogram Rangamati Rangamati Sadar Dr. Baby Tripura MO-CC Training on Training Management System for DGFP Officials 87 Chattogram Rangamati Rangamati Sadar Md. Obaidur Rahman Principal Training on Training Management System for DGFP Officials 88 Dhaka Dhaka Mirpur Dr. Nahid Sultana Assistant Coordinator(Training) Training on Training Management System for DGFP Officials 89 Dhaka Dhaka Mirpur Most. Shamima Akter Nursing Instructor Training on Training Management System for DGFP Officials 90 Dhaka Dhaka Mirpur Dr. Syeda Shamianaz Principal Training on Training Management System for DGFP Officials 91 Dhaka Dhaka Mirpur Runa Parveen Office Assistant Cum Computer Operator Training on Training Management System for DGFP Officials 92 Dhaka Dhaka Mirpur Rowshan Ara Mokta Office Assistant Cum Computer Operator Training on Training Management System for DGFP Officials 93 Dhaka Dhaka Mirpur Md. Mahmudul Hasan Computer Operator Training on Training Management System for DGFP Officials 94 Dhaka Dhaka Mirpur Dr. Helena Jabeen Sr. Consultant Training on Training Management System for DGFP Officials 95 Sylhet Sunamganj Jamalganj Md. Abdullah Al Mamun Training Officer Training on Training Management System for DGFP Officials 96 Sylhet Sunamganj Sunamganj Sadar Niranjan Bondhu Dam DDFP Training on Training Management System for DGFP Officials 97 Sylhet Sunamganj Sunamganj Sadar Md. Didarul Alam Office Assistant Cum Computer Operator Training on Training Management System for DGFP Officials 98 Sylhet Sylhet Sylhet Sadar Md. Wasib Ul Haque Office Assistant Cum Computer Operator Training on Training Management System for DGFP Officials 99 Sylhet Sylhet Zakiganj Barun Chatry UFPA Training on Training Management System for DGFP Officials 100 Dhaka Dhaka Mirpur Debashis Dev Instructor Training on Training Management System: Advance Usages, Dashboard Performance Evaluation of USAID’s Accelerating Universal Access to Family Planning (Shukhi Jibon) in Bangladesh | 106 Management & Virtual Communication Linkage 101 Chattogram Chandpur Sharasti Dr. Md. Tahedul Islam Patwary Training Officer Training on Training Management System: Advance Usages, Dashboard Management & Virtual Communication Linkage 102 Chattogram Chandpur Sharasti Md. Atahar Ali Cleaner Training on Training Management System: Advance Usages, Dashboard Management & Virtual Communication Linkage 103 Chattogram Chattogram Sitakunda Sandip Mondol AVO Training on Training Management System: Advance Usages, Dashboard Management & Virtual Communication Linkage 104 Chattogram Cumilla Adarsho Sadar Md. Yasin Mia Computer Operator Training on Training Management System: Advance Usages, Dashboard Management & Virtual Communication Linkage 105 Chattogram Noakhali Begumganj Shamsul Alam Housekeeper Training on Training Management System: Advance Usages, Dashboard Management & Virtual Communication Linkage 106 Chattogram Rangamati Kaptai Shekhar Kanti Dev Training Officer Training on Training Management System: Advance Usages, Dashboard Management & Virtual Communication Linkage 107 Chattogram Rangamati Rangamati Sadar Mahmudur Rahman Saidy Lecturer (Social Science) Training on Training Management System: Advance Usages, Dashboard Management & Virtual Communication Linkage 108 Dhaka Dhaka Mirpur Dr. A F M Radwanoor Rahman Instructor Training on Training Management System: Advance Usages, Dashboard Management & Virtual Communication Linkage 109 Dhaka Dhaka Mirpur Aklima Begum Instructor Training on Training Management System: Advance Usages, Dashboard Management & Virtual Communication Linkage 110 Dhaka Dhaka Mirpur Shafiul Azmain Home Economist Training on Training Management System: Advance Usages, Dashboard Management & Virtual Communication Linkage 111 Dhaka Faridpur Bhanga Faiz Ahamed Tanim Training Officer Training on Training Management System: Advance Usages, Dashboard Management & Virtual Communication Linkage 112 Dhaka Faridpur Bhanga Dilip Kumar Haldar House Keeper Training on Training Management System: Advance Usages, Dashboard Management & Virtual Communication Linkage 113 Dhaka Gazipur Gazipur Sadar Shafiul Azmain Principal Training on Training Management System: Advance Usages, Dashboard Management & Virtual Communication Linkage 114 Dhaka Tangail Ghatail Abdullah Al Masud Lecturer Training on Training Management System: Advance Usages, Dashboard Management & Virtual Communication Linkage Performance Evaluation of USAID’s Accelerating Universal Access to Family Planning (Shukhi Jibon) in Bangladesh | 107 115 Dhaka Tangail Ghatail Omar Faruk Training Officer Training on Training Management System: Advance Usages, Dashboard Management & Virtual Communication Linkage 116 Dhaka Tangail Tangail Sadar Taslima Islam Lecturer Training on Training Management System: Advance Usages, Dashboard Management & Virtual Communication Linkage 117 Mymensingh Jamalpur Melandaha Dr. Masud Parvez Lecturer (Primary Health Care) Training on Training Management System: Advance Usages, Dashboard Management & Virtual Communication Linkage 118 Mymensingh Jamalpur Melandaha Al Mamun Gyume Training Officer Training on Training Management System: Advance Usages, Dashboard Management & Virtual Communication Linkage 119 Sylhet Sylhet Sylhet Sadar Md. Riajul Haque Principal Training on Training Management System: Advance Usages, Dashboard Management & Virtual Communication Linkage Performance Evaluation of USAID’s Accelerating Universal Access to Family Planning (Shukhi Jibon) in Bangladesh | 108 Annex X: Annual number of counsellings provided to adolescents Source: HMIS Data Note: The coverage and completeness of HMIS service statistics may vary by time and division. Quality of registration data is known to deteriorate during crisis periods. 0 50000 100000 150000 200000 250000 300000 350000 400000 Annual number of adolesscent counselling sessions, 2019- 2022, HMIS 2019 2020 2021 2022 Performance Evaluation of USAID’s Accelerating Universal Access to Family Planning (Shukhi Jibon) in Bangladesh | 109 Annex XI: Timeline Performance Evaluation of USAID’s Accelerating Universal Access to Family Planning (Shukhi Jibon) in Bangladesh | 110 Annex XII: Evaluation statement of work A. BACKGROUND National context Bangladesh has made remarkable strides in improving its citizens' health over the past 25 years-including reducing infant mortality (from 88 to 38 deaths per 1,000 live births) (DHS,2017-18) and maternal mortality (from 322 to 196 deaths per 100,000 live births) (Bangladesh Maternal Mortality Survey, 2016). The country has also seen reductions in the total fertility rate (TFR) from 3.4 to 2.3, and increases in the modern contraceptive prevalence rate (mCPR); from 36.6% to 52% during this period (DHS, 2017-18). These gains are due in great part to commitment from the Government of Bangladesh (GOB) in advancing to the Universal Health Coverage as outlined in the 2011 National Health Policy and the 2016 - 2021 Health, Nutrition, and Population Strategic Investment Plan (HNPSIP). Going forward, the GOB has made ambitious commitments as part of FP2020- including a further reduction in TFR to 2.0, increasing CPR from 62% to 75%, increasing the share of long-acting and permanent methods (LAPM) from 8.1 % to 20%, and reducing unmet need and method discontinuation. However, the country faces key challenges in achieving this further progress-including inequity across geographic areas and among age cohorts. For example, while mCPR nationally among married women is at 52%, it is much lower in the Accelerating Universal Access to Family Planning (AUAFP) focus divisions of Chattogram, and Sylhet. These inequities also exist between age cohorts-modem method use among young married women aged 15 to 19 and 20 to 24 is only 43.7% and 50.9%, respectively, versus 62.7% among women between 30 and 34 (DHS, 2017-18). Further, nearly half of users rely on short-acting methods-primarily oral contraceptive pills (OCPs). Social, cultural, and structural challenges to progress To overcome the current mCPR plateau, USAID would like to address the range of social, cultural, and structural factors that underpin these inequities through Pathfinder International. Early marriage, pressure to prove fertility, and subsequent early childbearing results in almost half of women giving birth by age 18. FP decisions are driven by male partners and gatekeepers (e.g., mothers-in-law), leading to discrepancies between women's desired and actual fertility levels. These factors also contribute to insufficient birth spacing, especially among young women. Socio-cultural characteristics of certain populations and geographies present further challenges. In Sylhet, financial security from remittances combined with traditional preference for larger family sizes promote a higher TFR. In Chittagong, the myriad cultural and linguistic differences of remote populations living in the Hill Tracts (Khagrachhari, Rangamati, and Bandarban) hinder interaction with and trust of the health system. The influx of Rohingya refugees into Cox's Bazar in Chittagong creates added challenges for the health system. More than half of the 480,000 refugees are women and girls-including an estimated 120,000 women of reproductive age and 24,000 who are either pregnant or breastfeeding (UNFPA, 2017). The public sector is the main provider of modem methods and faces many structural challenges related to its health workforce and facilities. Health facilities at all levels should be prepared to provide FP services-from community-based health workers and community clinics (CCs), which provide OCPs, male condoms, injectables, and emergency contraceptive pills, to facilities at the upazila level and above, which should be able to provide a full range of modern methods. Despite this, on average only 76.8% of union-level public facilities offer FP services every day-with significant disparity between urban and rural clinics (98 % versus 77%, respectively) and across Performance Evaluation of USAID’s Accelerating Universal Access to Family Planning (Shukhi Jibon) in Bangladesh | 111 divisions (e.g., 63.2% in Mymensingh, compared with Dhaka (77.5%), Chattogram (88.3% and Sylhet (89.5%) Bangladesh Health Facility Survey [BHFS], 2017). Specific Problems I. Weak training system A key factor limiting service availability is presence of trained staff. Nationally, the Bangladeshi public health sector is understaffed, "with twice as many doctors as nurses, clustered disproportionately in urban areas, while rural facilities are overburdened, understaffed, and insufficiently equipped" (Bangladesh Health System Review, 2015). Once vacancies are filled in rural or hard-to-reach areas, providers are professionally isolated, often spend significant time away from their posts, and seek transfers to urban areas- resulting in understaffing even as the GOB has increased the numbers of sanctioned staff. To meet the criteria of FP readiness, only one staff must have been ever trained on FP. (BHFS, 2017). The Bangladesh Medical and Dental Council (BMDC) was established with the goal of ensuring a minimum standard for provider basic and higher qualifications; however, these and other similar bodies (e.g., the State Medical Faculty [SMF], the Bangladesh Nursing Council [BNC]) have limited legal power and autonomy. Further, there are no formal continuing medical education (CME) requirements for medical personnel, including FP providers. Further complicating this, the FP training mandate is held by three entities- the National Institute of Population Research and Training (NIPORT), the Directorate General of Family Planning (DGFP) and the Directorate General of Health Services (DGHS). Hiring is also slowed by the fact that trainer recruitment is handled through the Public Service Recruitment Commission. This all occurs in a highly pluralistic political environment frequently affected by localized violence and instability. II. Populations with FP access barriers, especially adolescents and youth The Ministry of Health and Family Welfare’s progressive National Strategy for Adolescent Health 2017-2030 recognizes that adolescents lack the structural and social supports to "develop their full potential" due to harmful social norms, poverty, and lack of education. The strategy aims to ensure that "By 2030 all adolescents will lead a healthy and productive life in a socially secure and supportive environment where they will have easy access to quality and comprehensive information, education, and services." The BHFS 2017 reports that only 22% of health facilities (including community clinics) meet the readiness criteria for FP service provision. If community clinics are excluded, then 51% meet the criteria. However, of those who are classified as FP ready, few can meet the special needs of adolescents. While postpartum FP, post-abortion care FP, and post-menstrual regulation FP are offered in Bangladesh's public health system, much remains to be done to ensure that potential recipients of services are fully informed about their options, and that service provision is tailored to meet the needs of all types of clients, especially adolescents and youth. Development hypothesis A. The overall development hypothesis of the AUAFP Activity is as follows: “IF the public and private sector FP workforce in Bangladesh has appropriate and adequate training, coordination, mentorship, and supervision, and IF there is increased awareness and availability of public sector FP outreach and services, particularly those that meet adolescent- and youth-friendly health services criteria, THEN more Bangladeshis, particularly adolescents and youth, will use FP services, leading to improved health and human capital that will contribute directly to IR 2.3 of the USAID/Bangladesh Country Development Cooperation Strategy results framework.” Outreach to communities and to adolescents and youth, in particular, will be vital to increasing demand for FP services and ensuring increased use of FP. Performance Evaluation of USAID’s Accelerating Universal Access to Family Planning (Shukhi Jibon) in Bangladesh | 112 B. The activity The AUAFP activity , also known as Shukhi Jibon, contributes to the health and well-being of Bangladeshis and to accelerating family planning (FP) use through strengthening the capacity of the Ministry of Health and Family Welfare (MOHFW), particularly DGFP, DGHS, and (NIPORT. The activity supports the trainings and deployment of skilled and respectful FP providers; strengthens the delivery of quality FP services, especially for adolescents and youth, and postpartum women; and works with communities to transform norms around the use of FP information and services. Shukhi Jibon relies on strong collaboration and coordination among government divisions, USAID implementing partners, and other stakeholders, including community structures, non￾governmental organizations (NGOs), and the private sector. The activity, initiated in July 2018 will complete its five years tenure in July 2023. USAID aims to conduct a high￾level performance evaluation of different components of Shukhi Jibon and use the lessons learnt in future USAID’s programming. activity /Activity Title: USAID’s Accelerating Universal Access to Family Planning Award/Contract Number: Cooperative Agreement No. 72038818CA00004 Award/Contract Dates: July 23, 2018 to July 22, 2023 Activity/activity Funding: $36,956,930 Implementing Partner(s): Pathfinder International Activity/activity COR/AOR: Taskeen Chowdhury Activity/activity Start Date: July 23, 2018 Activity/activity End Date: July 22, 2023 C. Results framework Performance Evaluation of USAID’s Accelerating Universal Access to Family Planning (Shukhi Jibon) in Bangladesh | 113 D. purpose of the evaluation The purposes of this performance evaluation are to determine: 1) which investments in the FP Training Institutes are increasing health provider’s FP capacity; 2) which, if any, interventions to reach adolescent and youth populations for reproductive health/FP information and services are promising; and 3) which, if any, interventions on mentorship and supportive supervision increase service providers’ skills in providing selected FP services Findings will be used to inform future USAID/Bangladesh FP and adolescent reproductive health designs, and national policies and programming on adolescents. Also, to advocate for changes in government staff skill development policies and to promote further adoption and implementation of FPRH initiatives that originate from the learning labs. E. Audience The intended audience for this analysis is the USAID/Bangladesh Mission staff in OPHN-particularly those working in maternal and child health, family planning and adolescent and youth reproductive health programming, as well as USAID/Bangladesh offices working on youth programs and Mission leadership. F. Applications and use Findings will be used to inform future USAID/Bangladesh FP and adolescent reproductive health designs, and national policies and programming on adolescents. Findings may be used to advocate for changes in government training policies and to promote further adoption and implementation of FPRH initiatives that originate from the learning labs. G. Evaluation questions & matrix Evaluation Questions/Areas Suggested Data Sources Suggested Data Collection Methods Suggested Data Analysis Methods 1 Assess effectiveness/ progress of key interventions targeted to strengthen the capacity of training institutions (both at national and sub-national level)? What are the components that are likely to sustain beyond the AUAFP implementation period? - Training curriculum (competency-based training) - Training Management Information System (TMIS) AUAFP reports and records Primary qualitative and quantitative data Desk review of activity documents and results, Key Informant Interviews; group interviews; observations of trainers teaching sessions. Thematic/content analysis of qualitative data; Descriptive statistics of extracted data; and teaching observation checklist data Performance Evaluation of USAID’s Accelerating Universal Access to Family Planning (Shukhi Jibon) in Bangladesh | 114 Evaluation Questions/Areas Suggested Data Sources Suggested Data Collection Methods Suggested Data Analysis Methods - Training Management Improvement Framework (TMIF) 2 To what extent did competency-based training and use of adult learning approaches strengthen service provider (different cadres at different levels of facilities) skills in providing select FP services (PPFP, PAC￾FP)? Lessons learned? Sustainability? AUAFP reports and records; Qualitative and Quantitative data Desk review of activity documents and results, Key Informant Interviews; focus group discussions; observations of trainers teaching classes Thematic/content analysis of qualitative data; Descriptive statistics of extracted data; and Provider-client observation checklist data 3 To what extent did mentorship and supportive supervision strategies increase service providers’ skills in providing select FP services, such as provision of long-acting reversible contraceptives (LARC) and FP counseling? AUAFP reports and records; Qualitative and Quantitative data Desk review of activity documents and results, Key Informant Interviews; Group Interviews, observations of trainers teaching sessions Thematic/content analysis of qualitative data; Descriptive statistics of extracted data; and Provider-client observation checklist data 4 What are the interventions of Shukhi Jibon for adolescents and youth implemented at facilities and at the community? What are the key lessons learned from these interventions to reach them? What are the evidence that these interventions have improved adolescents’ health knowledge and/or care￾seeking behavior? AUAFP reports and records; qualitative and quantitative data Desk review of activity documents and results; Focus group discussions; Secondary data (facilities service statistics) Thematic/content analysis of qualitative data; Descriptive statistics of facilities data including reach of the interventions by age and sex of adolescents/z reached; types of intervention. Suggested lines of enquiry The suggested line of enquiry for the evaluation questions is given below. However, the evaluation team can propose any change as and when deemed. Performance Evaluation of USAID’s Accelerating Universal Access to Family Planning (Shukhi Jibon) in Bangladesh | 115 Guidelines for KII with stakeholders and implementers - Strengths and weaknesses of adult learning approaches, learning labs, mentorship and adolescent programmes, TMIS - Thoughts on how AUAFP is contributing to improving health situation of our country - Major successes and challenges - What should be the activity focus in the future - Thoughts on how these interventions can be sustained Guidelines for GIs - effectiveness of the AUAFP interventions at national and regional levels - ways to improve effectiveness - ways to sustain these activities - challenges faced in implementation and why Guidelines for observations - Interactive-ness of participants - Enthusiasm of trainers and quality of their training - Condition of equipment and resources used for training - Turnout rate of participants - Quality of service delivery at different tiers H. Data collection methodology a. Document and Data Review: The desk review will be used to provide background information on the activity /program and will also provide data for analysis for this evaluation. Documents and data to be reviewed include: AUAFP Program Description; AUAFP Annual Reports: Years 1, 2, and 3; AUAFP Quarterly Reports: Q 1 and Q2; AUAFP MELP. b. Secondary Analysis of Existing Data: This is a re-analysis of existing data, beyond a review of data reports. The data sources used will be activity MIS, Facilities service statistics and Training Institutions MIS. c. Key Informant Interviews: i. Relevant officials of National Institute of Population Research and Training (2-3 KIIs) ii. Relevant officials of Directorate General of FP (2-3 KIIs) iii. USAID AOR/COR/Activity manager (3 KIIs or 1 GI) iv. AUAFP management to discuss each evaluation question (2 KIIs with management) v. Training institute managers to assess utilization of management improvement frameworks (4 KIIs) vi. Mentors at different levels (4 KIIs) vii. Learning lab managers on effectiveness of learning lab approaches (2-3 KIIs) viii. Implementing Partner managers and field staff (4 KIIs) ix. Adolescents- school and community based (6 KIIs) d. Group Interviews: will be conducted among: i. AUAFP staff (3 GI) Performance Evaluation of USAID’s Accelerating Universal Access to Family Planning (Shukhi Jibon) in Bangladesh | 116 ii. Trainers to determine their views of effectiveness of competency-based training and adult learning approaches on FP provider skills; [3 GIs (8 participants each = 24 institutions)] iii. AUAFP Field Staff (Implementing Partners) (4 GI) iv. Discussion among supervisors (Trainers) on effectiveness of mentorship and supportive supervision on FP provider skills; [4 GIs] v. Mentees (4 GI) vi. Learning lab managers to discuss effectiveness of learning lab approach; [1-2 GIs] vii. Discussion with community- and facility-based providers on working with adolescents and youth; [3-4 GIs] viii. Discussion with adolescents using services to determine their views. [5 GIs] (2 with adolescents in schools and 3 with adolescents in community) (Adolescent friendly corners, school-based, community￾based interventions) Community sessions. KII and GIs will be conducted either in-person or virtually depending on type of interview and respondent groups. The numbers, location and mode of interviews will be finalized by the evaluation team. e. Observations i. Training sessions to determine whether trainers are performing up to international standards ii. FP provider interactions with clients to determine skills in PPFP, PAC-FP, LARC services and FP counseling iii. Observations of adolescent program interventions at the Adolescent corners I. Analysis plan • Analysis process will progress with desk review, qualitative and quantitative (from AUAFP, NIPORT, DGFP data base) information. • Quantitative information will be used to mapping the progress of AUAFP intervention • Completeness, accuracy, availability, quality, and usability of the will be assessed to examine access of skills • From the feedback of stakeholders and logical connections with government’s policies and program documents would be helpful to assess lesson learned and sustainability of the interventions • Identify areas of best practice to be replicated as well as seek solutions where challenges still exist • However, where data is found to be inadequate the team will seek alternate sources of information or verification. It is envisaged this will be more through interview of KII, GI where appropriate and observation through field visits. • Appropriate graphs will be used for data visualization. • Average of the five-points Likert scale will be used to assess the intervention package. Performance Evaluation of USAID’s Accelerating Universal Access to Family Planning (Shukhi Jibon) in Bangladesh | 117 J. Activities/timeline Evaluation of USAID's AUAFP Detailed Timeline Evaluation of the USAID's AUAFP activity Dec-22 Jan-23 Feb-23 Mar-23 Apr-23 Week 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 Evaluation launch/In￾brief with USAID (Online) X 15- Dec Finalize the SoW 24- Dec Desk review X X Team Planning Meeting/In-depth discussion with USAID X Draft data collection tools (Guidelines, checklists and dummy tables) 5- Ja n X Kick-off meeting with USAID 12 Jan Meeting with USAID Team Lead and POC X X X X Draft and submission of inception report (evaluation questions, methods, timeline, data analysis plan, and data collection tools) 15- Jan X USAID review and approval 18- Jan X X In-brief with the target activity /program X Online Interviews (KIIs and GIs) 14- Jan 20- Jan Performance Evaluation of USAID’s Accelerating Universal Access to Family Planning (Shukhi Jibon) in Bangladesh | 118 Fieldwork: Site visits and data collection (both qualitative and quantitative) 23- Jan X 18- Fe b X Data analysis, interpretation and triangulation (FCR matrix) for preliminary findings draft X X 23- Fe b Debrief with USAID with PowerPoint presentation on progress of the evaluation and preliminary findings 23- Fe b Finalize the draft report after incorporating USAID’s feedback during the presentation X X Submission of the draft final report to USAID 11 - Ma r X USAID review of the report 20 - Ma r X Final submission of the report after addressing USAID comments and submission to USAID 31- Ma r 1. Evaluation Launch/In-brief with USAID – A call/meeting with the USAID, RDM, and the Evaluation Team (Team) to initiate the evaluation and review expectations. USAID will review the purpose, expectations, and agenda of the evaluation. RDM will introduce the Team and review the initial schedule and other management issues. 2. Desk Review – Several documents are available for review for this evaluation. These include the AUAFP proposal, annual work plans, MEL plan, quarterly progress reports, and routine reports of activity performance indicator data, as well as survey data reports and national strategic documents. This desk review will provide background information for the Evaluation Team and will also be used as data input and evidence for the evaluation. 3. Team Planning Meeting – A three to four-day team planning meeting (TPM) will be held at the initiation of the evaluation and before the data collection begins. During the TPM, the Team will: ▪ Review and clarify any questions on the evaluation SOW Performance Evaluation of USAID’s Accelerating Universal Access to Family Planning (Shukhi Jibon) in Bangladesh | 119 ▪ Establish a team atmosphere, share individual working styles, and agree on procedures for resolving differences of opinion ▪ Review and finalize the evaluation questions ▪ Review and finalize the evaluation timeline ▪ Develop a draft of the data collection methods, instruments, and guidelines ▪ Review and clarify any logistical and administrative procedures for the evaluation ▪ Develop a preliminary data collection plan ▪ Draft the evaluation inception report (workplan and methodology) ▪ Develop a preliminary draft outline of the team’s report ▪ Assign drafting/writing responsibilities for the final report or final presentation 4. Inception Report (workplan and methodology) will be submitted to USAID followed by a review meeting. The inception report will include: ▪ Evaluation timeline ▪ Evaluation questions ▪ Proposed methodology ▪ Data collection strategy, sampling frame, and selection criteria ▪ Data analysis plan describing procedures that will be used to analyze qualitative and quantitative data ▪ Data and resource requirements ▪ Data collection instruments 5. Fieldwork: Site Visits and Data Collection – The Consultants Team will conduct online interviews and site visits for data collection. They will visit NIPORT, DGFP, MFSTC, Dhamrai, Faridpur, Sylhet Sadar and Ishwarganj. Detailed visit plan will be finalized in consultation with USAID. The Evaluation Team will outline and schedule key meetings and site visits prior to departing to the field. 6. USAID Briefings – The Team Lead (TL) will brief the USAID POC weekly to discuss progress. As preliminary findings arise, the TL will share these during the routine briefing, and in an email. A final debrief between the Evaluation Team and USAID will be held at the end of the evaluation and before the preparation of the final report, to present preliminary findings to USAID. During this meeting, a summary of the data will be presented, along with high level findings and draft recommendations. For the debrief, the Team will prepare a PowerPoint Presentation of the key findings, issues, and recommendations. The Evaluation Team will incorporate comments received from USAID during the debrief in the evaluation report. 7. Evaluation Report – The Evaluation Team will develop a report with findings and recommendations. Report writing and submission will include the following steps: ▪ Evaluation team will submit a draft final report to USAID ▪ USAID will review the draft report in a timely manner, and send their comments ▪ Team will incorporate and address the comments and submit the final report to USAID K. Deliverables, and timelines Expected Deliverables/Milestone dates Proposed Submission Dates 1. Kick off meeting with USAID 12 January 2023 Performance Evaluation of USAID’s Accelerating Universal Access to Family Planning (Shukhi Jibon) in Bangladesh | 120 2. Inception report with revised SOW detailing the methodology, field work plan, outline of the evaluation report and data collection tools 15 January 2023 3. Presentation of preliminary findings to USAID/Dhaka 23 February 2023 4. Draft Report Mid-March 2023 5. Final Report End March 2023 L. Final report The Final Report will follow USAID’s Criteria to Ensure the Quality of the Evaluation Report. ● The report will not exceed 25-30 pages (excluding executive summary, table of contents, acronym list, and annexes). ● The structure of the report will follow the Evaluation Report template, including branding found here. ● Draft reports will be provided electronically, in English to USAID. USAID Criteria to Ensure the Quality of the Evaluation Report (USAID ADS 201): ● Evaluation/Assessment reports will be written in clear, readily understandable language and will identify key points clearly, distinctly, and succinctly. ● The Executive Summary of the evaluation/assessment report will present a concise and accurate statement of the most critical elements of the report. ● Evaluation/Assessment reports will adequately address all assignment questions included in the SOW, or the assignment questions will be subsequently revised and documented in consultation and agreement with USAID. ● Assignment methodology will be explained in detail and sources of information properly identified. ● Limitations of the assignment will be adequately disclosed in the report, with particular attention to the limitations associated with the methodology (selection bias, recall bias, unobservable differences between comparator groups, etc.). ● Findings and conclusions will be specific, concise, and supported by strong quantitative or qualitative evidence. ● If assignment findings assess person-level outcomes or impact, they will also be separately assessed for both males and females. ● Recommendations will be supported by a specific set of findings and should be action-oriented, practical, and specific. Reporting Guidelines: The draft report will be a comprehensive, analytical, evidence-based evaluation/assessment report. It would detail and describe results, effects, constraints, and lessons learned, and provide recommendations and identify key questions for future consideration. The report shall follow USAID branding procedures. The report will be edited/formatted and made 508 compliant as required by USAID for public reports and will be posted to the USAID/DEC. The findings from the final report will be presented in a draft report at a full briefing with USAID and at a follow￾up meeting with key stakeholders. The report would use the following format: ● Abstract: briefly describing what was evaluated, questions, methods, and key findings or conclusions (not more than 250 words) ● Executive Summary: summarizes key points, including the purpose, background, Performance Evaluation of USAID’s Accelerating Universal Access to Family Planning (Shukhi Jibon) in Bangladesh | 121 questions, methods, limitations, findings, conclusions, and most salient recommendations (2-5 pages) ● Table of Contents (1 page) ● Table of Figures ● Acronyms ● Assignment Purpose and Questions: state purpose of, audience for, and anticipated use(s) of the assignment (1-2 pages) ● activity [or Program] Background: describe the activity /program and the background, including country and sector context, and how the activity /program addresses a problem or opportunity (1-3 pages) ● Methods and Limitations: data collection, sampling, data analysis, and limitations (1-3 pages) ● Findings (organized by Assignment questions): substantiate findings with evidence/data ● Conclusions ● Recommendations ● Annexes ○ Annex I: Assignment Statement of Work ○ Annex II: Methods and Limitations (if not described in full in the main body of the final report) ○ Annex III: Data Collection Instruments ○ Annex IV: Sources of Information ▪ List of Persons Interviewed ▪ Bibliography of Documents Reviewed ▪ Databases ▪ [etc.] ○ Annex V: Statement of Differences (if applicable) ○ Annex VI: Disclosure of Any Conflicts of Interest ○ Annex VII: Summary information about Team members, including ○ qualifications, experience, and role on the team. The assignment methodology and report will be compliant with the USAID Evaluation Policy and Evaluation Report Checklist. The final report will exclude any potentially procurement-sensitive information. As needed, any procurement sensitive information or other sensitive but unclassified (SBU) information will be submitted in a memo to USAID separate from the report. ----------------------------------------------------------- All data instruments, data sets (if appropriate), presentations, meeting notes, and report for this evaluation/analysis will be submitted electronically to USAID. All datasets developed as part of this assignment will be submitted to USAID in an unlocked machine-readable format (CSV or XML). The datasets will not include any identifying or confidential information. The datasets must also be accompanied by a data dictionary that includes a codebook and any other information needed for others to use these data. Qualitative data included in this submission will not contain identifying or confidential information. Names, addresses, and other confidential information that can easily lead to identifying the respondent will not be included in any quantitative or qualitative data submitted. M. Team composition and skills Performance Evaluation of USAID’s Accelerating Universal Access to Family Planning (Shukhi Jibon) in Bangladesh | 122 The Evaluation Team will consist of 3 Key Technical Staff. The Team will also have 1 junior researcher from RDM to support the key staff in the evaluation process, including data collection, transcription, translation etc. One Logistics Coordinator will be hired to undertake the coordination and management responsibilities. The technical team members should represent depth of knowledge related to health service delivery in Bangladesh; non-profit management, capacity building and financial sustainability; in-depth knowledge of quality of care, MCH and FP. The technical team members must have significant health program and evaluation experience. At least two members of the technical experts must have Bangladesh experience, speak Bangla, and be familiar with the MCH-FP service delivery structure in urban and rural areas. The junior members of the team should have basic knowledge about interview techniques and be able to provide translation services to other team members. All team members must have professional-level English speaking and writing skills. There will be one local evaluator to assist the key staff in the evaluation. S/he will help in desk review, data collection, analysis and data interpretation. S/he will collect relevant program materials and make clinic visits as instructed by the Team Leader. S/he will assist in translation of data collection tools and transcripts, as needed. S/he should have basic understanding about health service delivery system of Bangladesh. S/he need to have basic familiarity with essential health service packages, as well as experience in conducting structured interviews, in-depth interviews and focus group discussion, both facilitating and note taking. S/he should have a good command of English and Bangla. The position descriptions of key team members are provided below: 1. Lead Consultant Roles & Responsibilities: Management ▪ Lead a team of 2-3 consultants and will be responsible for overall management of the evaluation including coordinating and packaging the deliverables in consultation with other team members; ▪ Facilitate the Team Planning Meeting (TPM) or guide a facilitator to set the agenda and other elements of the TPM; ▪ Manage team coordination meetings in the field; ▪ Coordinate the workflow and tasks and ensure that team members are working as per the schedule; ▪ Ensure that team field logistics are organized. Preparations ▪ Draft the SOW for the evaluation with inputs from other team members ▪ Lead the preparatory activities to initiate the evaluation process including desk review; ▪ In consultation with other team members, develop technical evaluation proposal/concept note, tools for the assessment and evaluation work plan and share it with RDM, icddr,b; ▪ Lead the data collection activity in the field; ▪ Oversee the training of all engaged in data collection, ensuring the highest level of reliability and validity of data being collected; ▪ Provide quality assurance on all aspects of evaluation; ▪ Establish evaluation roles, responsibilities, and tasks for other team members; ▪ Lead introductory and debriefing sessions with USAID; Performance Evaluation of USAID’s Accelerating Universal Access to Family Planning (Shukhi Jibon) in Bangladesh | 123 ▪ In coordination with other team members, the Lead consultant will be responsible for submitting the agreed deliverables including a presentation to USAID to share preliminary findings and submission of the final report. The final report requires to be approved by USAID. Communications ▪ Handle conflict within the team, if any; ▪ Serve as the spokesperson for the team, as and when required; ▪ Debrief RDM, icddr,b as the evaluation progresses, and organize a final debriefing; ▪ Keep the team members apprised of progress challenges, work changes, team travel plans in the field, and report preparation via phone conversation or email at least once in a week; ▪ Make decisions about the safety and security of the team in consultation with the client. Qualifications and experiences: ▪ Must have MBBS from reputed Medical College with Masters in Public Health; ▪ At least 15 years of experience in public health activity management, activity implementation and training; ▪ Must have 5 years of experience in managing M&E procedures in large activity supported by USAID; ▪ Familiarity with USAID health programs/activity, particularly in the area of family planning and health systems will be preferred; ▪ Experience in conducting secondary analysis of existing data; ▪ Sound knowledge in designing and implementation of evaluations and/or assessments using mixed methods of data collection and analysis; ▪ Excellent presentation skills; ▪ Excellent skill in writing and spoken English. 2. Consultant-1 (Training and Curriculum Specialist) Roles & Responsibilities: ▪ The Consultant will serve as a key member of the evaluation team, providing expertise in ▪ Family Planning, Training and Curriculum related component of the evaluation ▪ Actively contribute in finalizing the SoW for the evaluation team ▪ Actively contribute in desk review, data collection tools development and drafting inception report development and submission ▪ S/He will participate in team meetings, stakeholders’ meetings and site visits ▪ Take the lead role while collecting and analyzing data related to Family Planning, Training and Curriculum related component of the evaluation ▪ Prepare and support in drafting the sections of the evaluation report relevant to his/her expertise and role in the team ▪ S/He will also participate in presenting the report to USAID or other stakeholders and be responsible for addressing pertinent comments provided by USAID Bangladesh or other stakeholders ▪ S/He will communicate with the Team Leader and other consultants to produce written notes to incorporate in the report as required for addressing comments and feedbacks from USAID ▪ S/He is required to make his/her contributions to the Team Leader within the timeline ▪ Support the Lead Consultant on relevant management aspect of the consultancy team. Performance Evaluation of USAID’s Accelerating Universal Access to Family Planning (Shukhi Jibon) in Bangladesh | 124 Qualifications and experiences: ▪ PhD in Statistics/Applied Statistics/Demography/Medical Demography/Economics ▪ At least 15 years of working experience in the field of public health and research along with technical knowledge and experience in results-oriented family planning and reproductive health program planning and implementation ▪ Familiarity with USAID health programs/activity s, particularly in the area of family planning and health systems will be preferred; ▪ Experience in conducting secondary analysis of existing data; ▪ Sound knowledge in designing and implementation of evaluations and/or assessments using mixed methods of data collection and analysis; ▪ Excellent presentation skills; ▪ Excellent skill in writing and spoken English. 3. Consultant-2 Roles & Responsibilities: ▪ The Consultant will serve as a key member of an evaluation team ▪ Actively contribute in finalizing the SoW for the evaluation team ▪ Actively contribute in desk review, data collection tools development, drafting and submission of inception report ▪ S/He will participate in team meetings, stakeholders’ meetings and site visits ▪ Work closely with the evaluation team to implement data collection, cleaning, analysis and ensure that data collected is of high quality ▪ S/he will administer questionnaires, assessment tools and conduct key informant interviews and in-depth interviews, both facilitating and note-taking, as required ▪ S/he will be responsible for ensuring that clean quantitative data and verbatim transcripts of qualitative data are provided in accordance with the evaluation design ▪ Assist the evaluation team in preparing presentation slides for sharing preliminary results with USAID and other relevant stakeholders ▪ S/He will communicate with the Team Leader and other consultants to produce written notes to incorporate in the evaluation report as required for addressing comments and feedbacks received from USAID during presentation ▪ Actively support the evaluation team in drafting and finalizing the evaluation report ▪ S/He is required to make his/her contributions to the Team Leader within the timeline ▪ Support the Lead Consultant on relevant management aspect of the consultancy team. Qualifications and experiences: ▪ An international consultant with PhD in Social Science/Demography/Population studies; ▪ At least 15 years of research and evaluation experience, particularly in the field of family planning, reproductive health, adolescent health or maternal health; ▪ Regional experience in conducting research and evaluation is essential; Performance Evaluation of USAID’s Accelerating Universal Access to Family Planning (Shukhi Jibon) in Bangladesh | 125 ▪ Familiarity with USAID health programs/activity s, particularly in the area of family planning and health systems will be preferred; ▪ Sound knowledge in designing and implementation of evaluations and/or assessments using mixed methods of data collection and analysis; ▪ Training on impact evaluation for the public health intervention is desirable; ▪ Knowledge of country health system would be an added advantage; ▪ Excellent and professional level writing skills in English. 4. Local Researcher Roles & Responsibilities: ▪ S/he will assist the key staff in the evaluation; ▪ S/he will help in desk review; ▪ S/he will accompany the data collection team to facilitate data collection (both qualitative and quantitative); ▪ S/he will support data analysis and interpretation; ▪ S/he will collect relevant program materials as and when suggested by the Team Leader; Qualifications and experiences: ▪ Must be a medical graduate with MBBS; ▪ MPH would be preferred; ▪ Must have the basic understanding about health service delivery system of Bangladesh; ▪ Familiarity with essential health service packages; ▪ Experience in conducting structured interviews, in-depth interviews and focus group discussion, both facilitating and note taking; ▪ S/he should have a good command of English and Bangla. 5. Local Logistics Coordinator Roles & Responsibilities: ▪ Under the guidance of the Evaluation Team Lead, s/he will support the Evaluation Team with all logistics and administration support to allow the team to carry out the evaluation successfully; ▪ With the support of the RDM administration team, s/he will liaise with hotel staff, arrange in-country transportation (ground and air), arrange meeting rooms and ensure business center support, e.g. copying, internet, and printing; ▪ For in country travel, s/he will ensure bills in compliance with the USAID rules with particular attention to VAT exemption issues; ▪ S/he will coordinate with the relevant implementing partner, GoB and development partners for setting up meetings and interviews throughout the evaluation period; ▪ S/he will take notes on the action points during any meeting and share among the evaluation team members; ▪ S/he will conduct programmatic administrative and support tasks as assigned and ensure the process moves forward smoothly; ▪ S/he will also be asked to assist in translation of data collection tools and transcripts, if required. Performance Evaluation of USAID’s Accelerating Universal Access to Family Planning (Shukhi Jibon) in Bangladesh | 126 Qualifications and experiences: ▪ Must be a Masters in any discipline ▪ At least 15 years of experience in activity administration and financial management ▪ Previous working experience under the USAID funded activity s or environment is preferred ▪ S/he will have knowledge of key actors in the health sector and their locations including MOH, donors, and other stakeholders ▪ S/he will have a good command in English and must be proficient in Bangla ▪ Requires excellent organizing and time management skills. 0. Staffing level of effort (LOE) matrix and anticipated travel LOE Chart: The following is an illustrative LOE Chart showing the level of effort in days for each Team member. Assignment Team Lead Consultant Consultant-2 Consultant-3 Local Researcher Logistics Coordinator Number of people 1 1 1 1 1 Task Planning and Evaluation Design 1 Evaluation launch/In￾brief with USAID (Online) 0.5 0.5 0.5 0 0 2 Finalize the SoW 2 1 1 0 0 3 Desk review/Team Planning Meeting and preparation of Inception Report 3 3 3 3 1 4 Team Planning Meeting/In￾depth discussion with USAID on inception report 0.5 0.5 0.5 0.5 1 5 Draft data collection tools (Guidelines, checklists and 3 2 2 2 0 Performance Evaluation of USAID’s Accelerating Universal Access to Family Planning (Shukhi Jibon) in Bangladesh | 127 Assignment Team Lead Consultant Consultant-2 Consultant-3 Local Researcher Logistics Coordinator Number of people 1 1 1 1 1 Task dummy tables) 6 Finalization of inception report (incorporating USAID's feedback) 2 2 1 1 0 7 In-brief with the target activity /program 0.5 0.5 0.5 0.5 1 8 International travels 4 4 2 0 0 Preparation for data collection (Logistics and travel) 9 Background work for field travel 0 0 0 0 8 10 Setting appointments for the interviews 0 0 0 0 10 11 Piloting of data collection tools 1 1 1 1 2 12 Travel days inter/intra￾district (road) 3 3 3 3 6 Data collection, documentation and analysis 13 Fieldwork: conduct data collection (KII/GI/FGD) including observation￾online/in￾person 10 11 9 11 0 14 Consultations with USAID regarding 0.5 0.5 0.5 0.5 0.5 Performance Evaluation of USAID’s Accelerating Universal Access to Family Planning (Shukhi Jibon) in Bangladesh | 128 Assignment Team Lead Consultant Consultant-2 Consultant-3 Local Researcher Logistics Coordinator Number of people 1 1 1 1 1 Task data collection 15 Conduct quantitative data mining and analysis 3 4 3 4 0 16 Organize, compile, clean, prepare, analyze, and triangulate data. 3 2.5 2.5 4 0 17 Prepare and Present preliminary findings to USAID (PowerPoint slides) 1 0.5 0.5 0.5 0 Total LOE before draft report 37 36 30 31 29.5 18 Draft evaluation report 9 7 7 7 2 19 Finalize evaluation report through incorporating USAID's Feedback 4 3 3 2 1 20 Adjustment of all bills and vouchers 0 0 0 0 2.5 Total LOE per person 50 46 40 40 35 N. Anticipated travel Performance Evaluation of USAID’s Accelerating Universal Access to Family Planning (Shukhi Jibon) in Bangladesh | 129 Three consultants will travel to Bangladesh. Local travel from Dhaka by all team members to selected activity areas in Sylhet, Mymensingh and Dhaka Divisions. Performance Evaluation of USAID’s Accelerating Universal Access to Family Planning (Shukhi Jibon) in Bangladesh | 130 Annex XIII: Disclosure of any conflicts of interest A. Name Shumona Shafinaz Title Consultant Organization icddr,b Evaluation Position ☒ Team Leader ☐ Team member Evaluation Award Number (contract or other instrument) Un ited States Agency for International Development (USAID) under the terms of USAID’s Research for Decision Makers (RDM) Activity, cooperative agreement no. AID-388-A-17-00006 USAID Activity(s) Evaluated (Include activity name(s), implementer name(s) and award number(s), if applicable) USAID’s Advancing Universal Access to Family Planning activity; Pathfinder International, Cooperative Agreement No. 72038818CA00004 I have real or potential conflicts of interest to disclose. ☐ Yes ☒ No If yes answered above, I disclose the following facts: Real or potential conflicts of interest may include, but are not limited to: 1. Close family member who is an employee of the USAID operating unit managing the activity(s) being evaluated or the implementing organization(s) whose activity(s) are being evaluated. 2. Financial interest that is direct, or is significant though indirect, in the implementing organization(s) whose activities are being evaluated or in the outcome of the evaluation. 3. Current or previous direct or significant though indirect experience with the activity(s) being evaluated, including involvement in the activity design or previous iterations of the activity. CONTINUED If yes answered above, I disclose the following facts: Real or potential conflicts of interest may include, but are not limited to: Performance Evaluation of USAID’s Accelerating Universal Access to Family Planning (Shukhi Jibon) in Bangladesh | 131 4. Current or previous work experience or seeking employment with the USAID operating unit managing the evaluation or the implementing organization(s) whose activity(s) are being evaluated. 5. Current or previous work experience with an organization that may be seen as an industry competitor with the implementing organization(s) whose activity(s) are being evaluated. 6. Preconceived ideas toward individuals, groups, organizations, or objectives of the particular activities and organizations being evaluated that could bias the evaluation. I certify (1) that I have completed this disclosure form fully and to the best of my ability and (2) that I will update this disclosure form promptly if relevant circumstances change. If I gain access to proprietary information of other companies, then I agree to protect their information from unauthorized use or disclosure for as long as it remains proprietary and refrain from using the information for any purpose other than that for which it was furnished. Date 20 March 2023 Signature B. Name Ahmed Al-Sabir Title Consultant Organization icddrb Evaluation Position ☐ Team Leader ☒ Team member Evaluation Award Number (contract or other instrument) United States Agency for International Development (USAID) under the terms of USAID’s Research for Decision Makers (RDM) Activity, cooperative agreement no. AID-388-A-17-00006. USAID Activity(s) Evaluated (Include activity name(s), implementer name(s) and award number(s), if applicable) USAID’s Advancing Universal Access to Family Planning activity, Pathfinder International, Cooperative Performance Evaluation of USAID’s Accelerating Universal Access to Family Planning (Shukhi Jibon) in Bangladesh | 132 Agreement No. 72038818CA00004 I have real or potential conflicts of interest to disclose. ☐ Yes ☒ No If yes answered above, I disclose the following facts: Real or potential conflicts of interest may include, but are not limited to: 7. Close family member who is an employee of the USAID operating unit managing the activity(s) being evaluated or the implementing organization(s) whose activity(s) are being evaluated. 8. Financial interest that is direct, or is significant though indirect, in the implementing organization(s) whose activities are being evaluated or in the outcome of the evaluation. 9. Current or previous direct or significant though indirect experience with the activity(s) being evaluated, including involvement in the activity design or previous iterations of the activity. CONTINUED If yes answered above, I disclose the following facts: Real or potential conflicts of interest may include, but are not limited to: 10. Current or previous work experience or seeking employment with the USAID operating unit managing the evaluation or the implementing organization(s) whose activity(s) are being evaluated. 11. Current or previous work experience with an organization that may be seen as an industry competitor with the implementing organization(s) whose activity(s) are being evaluated. 12. Preconceived ideas toward individuals, groups, organizations, or objectives of the particular activities and organizations being evaluated that could bias the evaluation. I certify (1) that I have completed this disclosure form fully and to the best of my ability and (2) that I will update this disclosure form promptly if relevant circumstances change. If I gain access to proprietary information of other companies, then I agree to protect their information from unauthorized use or disclosure for as long as it remains proprietary and refrain from using the information for any purpose other than that for which it was furnished. Date 20 March 2023 Performance Evaluation of USAID’s Accelerating Universal Access to Family Planning (Shukhi Jibon) in Bangladesh | 133 Signature C. Name Anadil Alam Title Assistant Scientist Organization icddr,b Evaluation Position ☐ Team Leader ☒ Team member Evaluation Award Number (contract or other instrument) United States Agency for International Development (USAID) under the terms of USAID’s Research for Decision Makers (RDM) Activity, cooperative agreement no. AID-388-A-17-00006. USAID Activity(s) Evaluated (Include activity name(s), implementer name(s) and award number(s), if applicable) USAID’s Advancing Universal Access to Family Planning activity, Pathfinder International, Cooperative Agreement No. 72038818CA00004 I have real or potential conflicts of interest to disclose. ☐ Yes ☒ No If yes answered above, I disclose the following facts: Real or potential conflicts of interest may include, but are not limited to: 13. Close family member who is an employee of the USAID operating unit managing the activity(s) being evaluated or the implementing organization(s) whose activity(s) are being evaluated. 14. Financial interest that is direct, or is significant though indirect, in the implementing organization(s) whose activities are being evaluated or in the outcome of the evaluation. Performance Evaluation of USAID’s Accelerating Universal Access to Family Planning (Shukhi Jibon) in Bangladesh | 134 15. Current or previous direct or significant though indirect experience with the activity(s) being evaluated, including involvement in the activity design or previous iterations of the activity. CONTINUED If yes answered above, I disclose the following facts: Real or potential conflicts of interest may include, but are not limited to: 16. Current or previous work experience or seeking employment with the USAID operating unit managing the evaluation or the implementing organization(s) whose activity(s) are being evaluated. 17. Current or previous work experience with an organization that may be seen as an industry competitor with the implementing organization(s) whose activity(s) are being evaluated. 18. Preconceived ideas toward individuals, groups, organizations, or objectives of the particular activities and organizations being evaluated that could bias the evaluation. I certify (1) that I have completed this disclosure form fully and to the best of my ability and (2) that I will update this disclosure form promptly if relevant circumstances change. If I gain access to proprietary information of other companies, then I agree to protect their information from unauthorized use or disclosure for as long as it remains proprietary and refrain from using the information for any purpose other than that for which it was furnished. Performance Evaluation of USAID’s Accelerating Universal Access to Family Planning (Shukhi Jibon) in Bangladesh | 135 Date 19 March 2023 Signature Anadil Alam D. Name Nahid Kamal Title Consultant Organization Icddr,b Evaluation Position ☒ Team Leader ☐ Team member Evaluation Award Number (contract or other instrument) United States Agency for International Development (USAID) under the terms of USAID’s Research for Decision Makers (RDM) Activity, cooperative agreement no. AID-388-A-17-00006. USAID Activity(s) Evaluated (Include activity name(s), implementer name(s) and award number(s), if applicable) USAID’s Advancing Universal Access to Family Planning activity, Pathfinder International, Cooperative Agreement No. 72038818CA00004 I have real or potential conflicts of interest to disclose. ☐ Yes ☒ No If yes answered above, I disclose the following facts: Real or potential conflicts of interest may include, but are not limited to: 19. Close family member who is an employee of the USAID operating unit managing the activity(s) being evaluated or the implementing organization(s) whose activity(s) are being evaluated. 20. Financial interest that is direct, or is significant though indirect, in the implementing organization(s) whose activities are being evaluated or in the outcome of the evaluation. 21. Current or previous direct or significant though indirect experience with the activity(s) being evaluated, including Performance Evaluation of USAID’s Accelerating Universal Access to Family Planning (Shukhi Jibon) in Bangladesh | 136 involvement in the activity design or previous iterations of the activity. CONTINUED If yes answered above, I disclose the following facts: Real or potential conflicts of interest may include, but are not limited to: 22. Current or previous work experience or seeking employment with the USAID operating unit managing the evaluation or the implementing organization(s) whose activity(s) are being evaluated. 23. Current or previous work experience with an organization that may be seen as an industry competitor with the implementing organization(s) whose activity(s) are being evaluated. 24. Preconceived ideas toward individuals, groups, organizations, or objectives of the particular activities and organizations being evaluated that could bias the evaluation. I certify (1) that I have completed this disclosure form fully and to the best of my ability and (2) that I will update this disclosure form promptly if relevant circumstances change. If I gain access to proprietary information of other companies, then I agree to protect their information from unauthorized use or disclosure for as long as it remains proprietary and refrain from using the information for any purpose other than that for which it was furnished. Date 20 March 2023 Signature Performance Evaluation of USAID’s Accelerating Universal Access to Family Planning (Shukhi Jibon) in Bangladesh | 137 Annex XIV: Evaluation Team Members Team composition and skills The Evaluation Team will consist of 3 Key Technical Staff. The Team will also have 1 junior researcher from RDM to support the key staff in the evaluation process, including data collection, transcription, translation etc. One Logistics Coordinator will be hired to undertake the coordination and management responsibilities. The technical team members should represent depth of knowledge related to health service delivery in Bangladesh; non-profit management, capacity building and financial sustainability; in-depth knowledge of quality of care, MCH and FP. The technical team members must have significant health program and evaluation experience. At least two members of the technical experts must have Bangladesh experience, speak Bangla, and be familiar with the MCH-FP service delivery structure in urban and rural areas. The junior members of the team should have basic knowledge about interview techniques and be able to provide translation services to other team members. All team members must have professional-level English speaking and writing skills. There will be one local evaluator to assist the key staff in the evaluation. S/he will help in desk review, data collection, analysis and data interpretation. S/he will collect relevant program materials and make clinic visits as instructed by the Team Leader. S/he will assist in translation of data collection tools and transcripts, as needed. S/he should have basic understanding about health service delivery system of Bangladesh. S/he need to have basic familiarity with essential health service packages, as well as experience in conducting structured interviews, in-depth interviews and focus group discussion, both facilitating and note taking. S/he should have a good command of English and Bangla. The position descriptions of key team members are provided below: A. Lead Consultant Roles & Responsibilities: Management ▪ Lead a team of 2-3 consultants and will be responsible for overall management of the evaluation including coordinating and packaging the deliverables in consultation with other team members; ▪ Facilitate the Team Planning Meeting (TPM) or guide a facilitator to set the agenda and other elements of the TPM; ▪ Manage team coordination meetings in the field; ▪ Coordinate the workflow and tasks and ensure that team members are working as per the schedule; ▪ Ensure that team field logistics are organized. Preparations ▪ Draft the SOW for the evaluation with inputs from other team members ▪ Lead the preparatory activities to initiate the evaluation process including desk review; ▪ In consultation with other team members, develop technical evaluation proposal/concept note, tools for the assessment and evaluation work plan and share it with RDM, icddr,b; ▪ Lead the data collection activity in the field; ▪ Oversee the training of all engaged in data collection, ensuring the highest level of reliability and validity of data being collected; ▪ Provide quality assurance on all aspects of evaluation; Performance Evaluation of USAID’s Accelerating Universal Access to Family Planning (Shukhi Jibon) in Bangladesh | 138 ▪ Establish evaluation roles, responsibilities, and tasks for other team members; ▪ Lead introductory and debriefing sessions with USAID; ▪ In coordination with other team members, the Lead consultant will be responsible for submitting the agreed deliverables including a presentation to USAID to share preliminary findings and submission of the final report. The final report requires to be approved by USAID. Communications ▪ Handle conflict within the team, if any; ▪ Serve as the spokesperson for the team, as and when required; ▪ Debrief RDM, icddr,b as the evaluation progresses, and organize a final debriefing; ▪ Keep the team members apprised of progress challenges, work changes, team travel plans in the field, and report preparation via phone conversation or email at least once in a week; ▪ Make decisions about the safety and security of the team in consultation with the client. Qualifications and experiences: ▪ Must have MBBS from reputed Medical College with Masters in Public Health; ▪ At least 15 years of experience in public health activity management, activity implementation and training; ▪ Must have 5 years of experience in managing M&E procedures in large activities supported by USAID; ▪ Familiarity with USAID health programs/activities, particularly in the area of family planning and health systems will be preferred; ▪ Experience in conducting secondary analysis of existing data; ▪ Sound knowledge in designing and implementation of evaluations and/or assessments using mixed methods of data collection and analysis; ▪ Excellent presentation skills; ▪ Excellent skill in writing and spoken English. B. Consultant-1 (Training and Curriculum Specialist) Roles & Responsibilities: ▪ The Consultant will serve as a key member of the evaluation team, providing expertise in Family Planning, Training and Curriculum related component of the evaluation ▪ Actively contribute in finalizing the SoW for the evaluation team ▪ Actively contribute in desk review, data collection tools development and drafting inception report development and submission ▪ S/He will participate in team meetings, stakeholders’ meetings and site visits ▪ Take the lead role while collecting and analyzing data related to Family Planning, Training and Curriculum related component of the evaluation ▪ Prepare and support in drafting the sections of the evaluation report relevant to his/her expertise and role in the team ▪ S/He will also participate in presenting the report to USAID or other stakeholders and be responsible for addressing pertinent comments provided by USAID Bangladesh or other stakeholders ▪ S/He will communicate with the Team Leader and other consultants to produce written notes to incorporate in the report as required for addressing comments and feedbacks from USAID ▪ S/He is required to make his/her contributions to the Team Leader within the timeline ▪ Support the Lead Consultant on relevant management aspect of the consultancy team. Performance Evaluation of USAID’s Accelerating Universal Access to Family Planning (Shukhi Jibon) in Bangladesh | 139 Qualifications and experiences: ▪ PhD in Statistics/Applied Statistics/Demography/Medical Demography/Economics ▪ At least 15 years of working experience in the field of public health and research along with technical knowledge and experience in results-oriented family planning and reproductive health program planning and implementation ▪ Familiarity with USAID health programs/activities, particularly in the area of family planning and health systems will be preferred; ▪ Experience in conducting secondary analysis of existing data; ▪ Sound knowledge in designing and implementation of evaluations and/or assessments using mixed methods of data collection and analysis; ▪ Excellent presentation skills; ▪ Excellent skill in writing and spoken English. C. Consultant-2 Roles & Responsibilities: ▪ The Consultant will serve as a key member of an evaluation team ▪ Actively contribute in finalizing the SoW for the evaluation team ▪ Actively contribute in desk review, data collection tools development, drafting and submission of inception report ▪ S/He will participate in team meetings, stakeholders’ meetings and site visits ▪ Work closely with the evaluation team to implement data collection, cleaning, analysis and ensure that data collected is of high quality ▪ S/he will administer questionnaires, assessment tools and conduct key informant interviews and in-depth interviews, both facilitating and note-taking, as required ▪ S/he will be responsible for ensuring that clean quantitative data and verbatim transcripts of qualitative data are provided in accordance with the evaluation design ▪ Assist the evaluation team in preparing presentation slides for sharing preliminary results with USAID and other relevant stakeholders ▪ S/He will communicate with the Team Leader and other consultants to produce written notes to incorporate in the evaluation report as required for addressing comments and feedbacks received from USAID during presentation ▪ Actively support the evaluation team in drafting and finalizing the evaluation report ▪ S/He is required to make his/her contributions to the Team Leader within the timeline ▪ Support the Lead Consultant on relevant management aspect of the consultancy team. Qualifications and experiences: ▪ An international consultant with PhD in Social Science/Demography/Population studies; ▪ At least 15 years of research and evaluation experience, particularly in the field of family planning, reproductive health, adolescent health or maternal health; ▪ Regional experience in conducting research and evaluation is essential; ▪ Familiarity with USAID health programs/activities, particularly in the area of family planning and health systems will be preferred; Performance Evaluation of USAID’s Accelerating Universal Access to Family Planning (Shukhi Jibon) in Bangladesh | 140 ▪ Sound knowledge in designing and implementation of evaluations and/or assessments using mixed methods of data collection and analysis; ▪ Training on impact evaluation for the public health intervention is desirable; ▪ Knowledge of country health system would be an added advantage; ▪ Excellent and professional level writing skills in English. D. Local Researcher Roles & Responsibilities: ▪ S/he will assist the key staff in the evaluation; ▪ S/he will help in desk review; ▪ S/he will accompany the data collection team to facilitate data collection (both qualitative and quantitative); ▪ S/he will support data analysis and interpretation; ▪ S/he will collect relevant program materials as and when suggested by the Team Leader; Qualifications and experiences: ▪ Must be a medical graduate with MBBS; ▪ MPH would be preferred; ▪ Must have the basic understanding about health service delivery system of Bangladesh; ▪ Familiarity with essential health service packages; ▪ Experience in conducting structured interviews, in-depth interviews and focus group discussion, both facilitating and note taking; ▪ S/he should have a good command of English and Bangla. E. Local Logistics Coordinator Roles & Responsibilities: ▪ Under the guidance of the Evaluation Team Lead, s/he will support the Evaluation Team with all logistics and administration support to allow the team to carry out the evaluation successfully; ▪ With the support of the RDM administration team, s/he will liaise with hotel staff, arrange in-country transportation (ground and air), arrange meeting rooms and ensure business center support, e.g. copying, internet, and printing; ▪ For in country travel, s/he will ensure bills in compliance with the USAID rules with particular attention to VAT exemption issues; ▪ S/he will coordinate with the relevant implementing partner, GoB and development partners for setting up meetings and interviews throughout the evaluation period; ▪ S/he will take notes on the action points during any meeting and share among the evaluation team members; ▪ S/he will conduct programmatic administrative and support tasks as assigned and ensure the process moves forward smoothly; ▪ S/he will also be asked to assist in translation of data collection tools and transcripts, if required. Performance Evaluation of USAID’s Accelerating Universal Access to Family Planning (Shukhi Jibon) in Bangladesh | 141 Qualifications and experiences: ▪ Must be a Masters in any discipline ▪ At least 15 years of experience in activity administration and financial management ▪ Previous working experience under the USAID funded activity or environment is preferred ▪ S/he will have knowledge of key actors in the health sector and their locations including MOH, donors, and other stakeholders ▪ S/he will have a good command in English and must be proficient in Bangla ▪ Requires excellent organizing and time management skills. Office of Population, Health and Nutrition U.S. Agency for International Development Madani Avenue, Baridhara Dhaka 1212, Bangladesh