EVALUATION MIDTERM PERFORMANCE EVALUATION OF STRENGTHENING EGYPT’S FAMILY PLANNING PROGRAM July 28, 2021 This publication was produced at the request of the United States Agency for International Development. It was prepared independently by Mildred Howard, Rafeek Ramzey, Nivine Ramses, Noha Hassan, Norhan Bader, Hanan Kwinana, Ehab Sakr, Wael Abdel Karim, Sawsan Refaat, Amal Refaat, Hany Mohamed, and Om Hashem Refaat, with support from Aliaa Abdella, Avinesh De Silva, Adriana Abreu, and Eric Benschoter of the QED Group. SEFPP FOR USAID EVALUATION MIDTERM PERFORMANCE EVALUATION OF STRENGTHENING EGYPT’S FAMILY PLANNING PROGRAM (SEFPP) July 28, 2021 Submitted by: The QED Group, LLC 2461 Eisenhower Ave, Suite 200 Alexandria, VA 22314, USA Tel.: 1.703.678.4700 www.qedgroupllc.com Egypt Office: The QED Group, LLC 1A Nadi El Etisalat off Ellaselky Street, New Maadi, 11435, Cairo, Egypt Office: +20 (0) 2 25226697 PHOTO CAPTION: Basic Task Sharing Training Course for Family Planning Nurses–Fayoum Governorate, December 2020 DISCLAIMER The authors’ views expressed in this publication do not necessarily reflect the views of the United States Agency for International Development or the United States Government. ACKNOWLEDGEMENTS The United States Agency for International Development in Egypt (USAID/Egypt) commissioned the Services to Improve Performance Management, Enhance Learning, and Evaluation (SIMPLE) Project to field an evaluation team to conduct a midterm performance evaluation of USAID/Egypt’s Strengthening Egypt’s Family Planning Program (SEFPP). SEFPP is implemented in Egypt by John Snow, Inc. (JSI), a public health management and research organization based in the United States. Because of the ongoing global COVID-19 pandemic and restrictions imposed throughout Egypt, this evaluation was conducted entirely through virtual meetings and phone contacts. The evaluation team thanks all informants who extended courtesy and cooperation in scheduling time and speaking with us under these unusual circumstances. They all contributed to our understanding of SEFPP. The virtual nature of this evaluation made it challenging for the evaluation team to reach a few of the informants we would have wished to interview. USAID/Egypt’s Office of Education and Health and Program Office staff were very responsive in facilitating contacts with some difficult-to-reach informants. In particular, we wish to express our appreciation to the headquarters and Egypt-based SEFPP staff who ensured that the evaluation team had access to organizations and individuals that had previously interacted with and/or benefitted from SEFPP support. i | MIDTERM EVALUATION OF STRENGTHENING EGYPT’S FAMILY PLANNING PROGRAM USAID.GOV TABLE OF CONTENTS Acronyms...................................................................................................................iii Executive Summary .................................................................................................. v Evaluation Purpose .............................................................................................................................................. v Activity Background............................................................................................................................................ v Evaluation Questions.......................................................................................................................................... v Evaluation Methodology, Limitations, and Mitigation................................................................................. vi Findings, Conclusions, and Recommendations........................................................................................... vii Introduction................................................................................................................1 Activity Information..................................................................................................1 Activity Background..................................................................................................2 Evaluation Purpose and Evaluation Questions.......................................................2 Evaluation Purpose .............................................................................................................................................. 2 Evaluation Questions.......................................................................................................................................... 2 Evaluation Methodology ...........................................................................................3 Findings, Conclusions, and Recommendations (FCRS).........................................5 Activity Achievements MATRIX — An Overview...................................................................................... 6 Findings, Conclusions, and Recommendations by Activity Objective...................................................10 DESIGN, THEORY OF CHANGE, AND SUSTAINABILITY 10 STRATEGIC INFORMATION 12 STRATEGIC PARTNERSHIPS 13 ACTIVITY MANAGEMENT 15 MONITORING, EVALUATION, AND LEARNING (MEL) 15 SERVICE DELIVERY: CONTRACEPTIVE SECURITY 17 SERVICE DELIVERY: SUPPLY-SIDE INTERVENTIONS 19 OPERATIONAL PARTNERSHIPS 21 SOCIAL AND BEHAVIOR CHANGE COMMUNICATION (SBCC) 22 Findings, Conclusions, and Recommendations—Training and Capacity Building...............................24 TRAINING AND CAPACITY BUILDING APPROACHES 24 TRAINING BENEFICIARIES: SERVICE PROVIDERS 25 TRAINING BENEFICIARIES: NON-SERVICE PROVIDERS 26 Findings, Conclusions, and Recommendations—Cross-cutting Issues.................................................29 COVID-19 29 GENDER 30 SUSTAINABILITY 30 Selected Recommendations Recapped.................................................................32 Annex 1: Evaluation Statement of Work..............................................................34 I. Purpose of the Evaluation.............................................................................................................................34 II. Summary Information...................................................................................................................................34 III. Background....................................................................................................................................................34 USAID.GOV MIDTERM EVALUATION OF STRENGTHENING EGYPT’S FAMILY PLANNING PROGRAM | ii IV. Evaluation Questions..................................................................................................................................39 V. Evaluation Design and Methodology........................................................................................................40 VI. Deliverables and Reporting Requirements............................................................................................41 Annex 2: SEFPP Midterm Evaluation Concept Note..........................................46 1. DESIGN 47 2. SAMPLING 48 3. DATA COLLECTION 48 4. DATA ANALYSIS 49 5. LIMITATIONS 49 Annex 3: List of Consulted Stakeholders..............................................................56 Annex 4: Evaluation Methodology .........................................................................58 Data Sources and Sampling .............................................................................................................................58 Data Collection..................................................................................................................................................59 Data Analysis & Quality Assurance ...............................................................................................................61 Limitations and Mitigation Measures.............................................................................................................62 LIMITATIONS 62 MITIGATION 62 Annex 5: Qualitative Data Analysis Code Book...................................................63 Annex 6: Quantitative Data Analysis Charts and Tables ....................................65 Part A: Service Providers Training Recipients............................................................................................65 Part B: Non-Service Providers Training Recipients...................................................................................76 Annex 7: SEFPP Data Collection Tools (Arabic and English) ............................82 Annex 8: List of Reviewed Documents................................................................134 Annex 9: Biographies.............................................................................................142 Mildred Howard, Team Leader....................................................................................................................142 Rafeek Ramzy, Subject Matter Expert ........................................................................................................142 Nivine Ramses, Evaluator...............................................................................................................................142 Noha Hassan, Evaluator.................................................................................................................................143 Norhan Bader, Evaluator...............................................................................................................................143 Hanan Kwinana, Evaluator.............................................................................................................................143 Ehab Sakr, Statistician .....................................................................................................................................144 TABLES Table 1. Summary of Activity Information............................................................................................................ 1 Table 2 -- ACTIVITY ACHIEVEMENTS – AN OVERVIEW.......................................................................... 7 Table 3. Selected Recommendations and Actions............................................................................................32 FIGURES Figure 1. Distribution of Service Providers by Occupation............................................................................25 Figure 2. Service Providers: “I am able to apply the learning I acquired in my work” (n=336).............26 Figure 3. Distribution of non-service providers by work governorates.....................................................27 Figure 4. Non-service Providers: “I am able to apply the learning I acquired in my work” (n=219)...27 iii | MIDTERM EVALUATION OF STRENGTHENING EGYPT’S FAMILY PLANNING PROGRAM USAID.GOV ACRONYMS AUC American University in Cairo CAPMAS Central Agency for Public Mobilization and Statistics COP Chief of Party COVID-19 Coronavirus Disease 2019 CS Contraceptive Security DO Development Objective DQA Data Quality Assessment EDHS Egypt Demographic and Health Survey EU European Union FCR Findings, Conclusions, and Recommendations FP Family Planning FP/RH Family Planning/Reproductive Health GOE Government of Egypt HECU Health Education and Communication Unit HIC Health Information Center HMIS Health Management Information System IP Implementing Partner IR Intermediate Result IUD Intrauterine Device JSI John Snow, Inc. KII Key Informant Interview LARC Long-acting Reversible Contraception LMIS Logistics Management Information System M&E Monitoring and Evaluation MEL Monitoring, Evaluation, and Learning USAID.GOV MIDTERM EVALUATION OF STRENGTHENING EGYPT’S FAMILY PLANNING PROGRAM | iv MELP Monitoring, Evaluation, and Learning Plan MOHP Ministry of Health and Population MOU Memorandum of Understanding NGO Non-governmental Organization NPC National Population Council OB/GYN Obstetrics and Gynecology PAD Project Appraisal Document PC Population Council (international) PHCU Primary Health Care Unit PIRS Performance Indicator Reference Sheet(s) PPC Postpartum Contraception PPE Personal Protective Equipment PP/PA/IUD Postpartum/Post-abortion Insertion of Intrauterine Devices PPR Performance Plan and Report RR Raedat Refiyat (Public Healthcare Worker) SBCC Social and Behavior Change Communication SEFPP Strengthening Egypt’s Family Planning Program SIMPLE Services to Improve Performance Management, Enhance Learning, and Evaluation SOW Scope of Work TEC Total Estimated Cost (of a funded activity) TFA Thematic Focal Area TOT Training of Trainers UNICEF United Nations Children’s Fund UNFPA United Nations Population Fund WB World Bank WHO World Health Organization v | MIDTERM EVALUATION OF STRENGTHENING EGYPT’S FAMILY PLANNING PROGRAM USAID.GOV EXECUTIVE SUMMARY EVALUATION PURPOSE The purpose of this mid-term evaluation is to identify areas for Strengthening Egypt’s Family Planning Program (SEFPP) to focus on for the remaining period and to identify evidence of good practices for a broad network of stakeholders.1 The evaluation aims to serve learning purposes by identifying any challenges incurred by the activity to date and formulating appropriate recommendations for corrective actions and effective implementation during the remaining years of the activity and future activities. ACTIVITY BACKGROUND Following a hiatus in USAID funding for the family planning (FP) sector in Egypt (from approximately 2010 to 2017), USAID re-entered the FP arena and worked with the Government of Egypt (GOE) to establish SEFPP. USAID and John Snow, Inc. (JSI) signed a five-year Cooperative Agreement in December 2017, and the SEFPP activity began full mobilization in early 2018. The Total Estimated Cost (TEC) of the activity is $18,788,799. Because of the seven-year gap in USAID’s involvement in the FP sector, in practical terms, the activity was a “new start.” Working relationships with the Ministry of Health and Population (MOHP), National Population Council (NPC), and other FP sector partners needed revitalization. SEFPP was designed to revitalize these relationships and address those factors that were hampering effective FP efforts in Egypt. The purpose of SEFPP is to address the country’s rapid population growth by reducing the unmet need2 of women and men for FP counseling services and methods. SEFPP prioritizes its support in the nine governorates of Upper Egypt and selected slum areas in Cairo and Alexandria, where the need for increased access to FP is greatest and on-going MOHP and NPC interventions can be scaled up. The activity design Theory of Change postulated that, “Use of evidence (that is informed by valid data) leads to progressive policy formulation, robust strategic program planning, and reliable decision￾making with good activity outcomes and results.” This Theory of Change in SEFPP’s design spelled out the basic, underlying approach to the activity’s strategy in three interrelated objectives: (1) improved evidence-based planning, (2) improved quality of services, and (3) improved social behaviors that work together to produce a strengthened national family planning program. It was assumed that when data is correctly collected, analyzed, and disseminated, decision-makers can use this data for planning, which will improve the quality of services. To strengthen the outcomes further, social behaviors need to be changed in order to simultaneously create a demand for FP services. EVALUATION QUESTIONS In response to the evaluation questions (EQs), the team examined the activity’s progress at the broad programmatic level, the operational level, and the service level, including factors underlying successes and challenges. In addition to assessing the cross-cutting effects of COVID-19 on the activity’s 1 See also Annex 1: Evaluation Statement of Work, and Annex 2: SEFPP Midterm Evaluation Concept Note. 2 Unmet need is the gap between women’s reproductive intentions and their contraceptive behavior. USAID.GOV MIDTERM EVALUATION OF STRENGTHENING EGYPT’S FAMILY PLANNING PROGRAM | vi operations, the team considered its performance in promoting gender equity and the sustainability of programming. This evaluation was charged with answering the following key questions: EQ1 – (Broad program level) To date, how successful has SEFPP been in achieving results under each component? What has been accomplished so far? What have been the challenges? What factors have led to specific accomplishments? Specific factors to examine include but are not limited to performance against indicators, relationship management with GOE stakeholders, internal SEFPP management, and implementation. EQ2 – (IP operational level) To date, given the current activity’s status and the results reported under each objective: (1) Strengthen evidence-based planning for decision making, (2) quality family planning/reproductive health (FP/RH) services supported, (3) selected health behaviors enhanced, in the quarterly progress reports, annual reports, and the Performance Plan & Report (PPR), how successful has the activity been, including the factors contributing to its success? What are the hindrances? EQ3 – (Service level) To date, in what ways did the SEFPP activity strengthen capacity and commitment in the local systems (health workforce, health information system, and health service delivery) to continue delivering consistent levels of access to and delivery of quality FP services? EVALUATION METHODOLOGY, LIMITATIONS, AND MITIGATION A detailed discussion of the evaluation methodology can be found in Annex 4 of this report. The following is a summary. This SEFPP midterm evaluation used a mixed-methods approach with qualitative research methods, complemented by a quantitative opinion survey. The evaluation focused on the activity’s structure, operations, and outputs. Possible outcomes and impact of the activity were beyond the scope of this evaluation. The qualitative research for this evaluation included ten key informant interview (KII) tools intended to systematically interview program policymakers, managers, and field service providers. The quantitative research involved a questionnaire administered to 599 service providers and non-service providers (administrative staff) who benefitted from SEFPP’s training and capacity building activities. Respondents were selected using a purposive sampling methodology. The evaluation team conducted descriptive statistical analyses on the quantitative data and employed a codable thematic analytical approach for the qualitative data. These methods identified explicit and implicit patterns in respondents’ answers relating to the EQs. The evaluation team conducted a data cleaning exercise and developed a coding tree for key informants. Limitations and Mitigation Measures. Several challenges and limitations experienced in this evaluation were mitigated as follows: ● With the exception of data provided to scale up postpartum intervention and concurrence to provide data for scaling up task sharing, the lack of data capture at the level of change (especially related to policy-level data) in most instances made it impossible to measure incremental changes and progress in the FP program or estimate SEFPP’s contribution to such changes. Notably, certain populations that the evaluation might have wanted to interview, such as recipients of FP services at clinic/community-level, were not accessible because of MOHP restrictions on direct survey data collection. Furthermore, inaccessibility of certain public sector statistics (such as “stock on hand”) made it impossible for SEFPP to help the MOHP with evidence-based planning to estimate national program performance supply requirements. Thus, the evaluation could not adequately assess SEFPP’s contributions to contraceptive forecasting and quantification. It was not possible to assess improvements in service delivery or the impact of behavior change components that might have vii | MIDTERM EVALUATION OF STRENGTHENING EGYPT’S FAMILY PLANNING PROGRAM USAID.GOV been attributable to SEFPP. To compensate for possible limitations in this evaluation that resulted from various restrictions on availability of data, the evaluation relied on output data and statistics that were generated from SEFPP reports and readily available in files or from discussions with informants. ● Limitations inherent in conducting phone surveys included time constraints (such as people being engaged in “at home” activities), difficulty in reaching respondents, and enumerators misstating a question or misinterpreting the answer. On the phone surveys, enumerators were trained and supervised throughout to avoid poor performance in conducting or notating interviews; digital data input (to SurveyMonkey) was controlled by a statistician/supervisor; respondents were contacted in advance, appointments were scheduled, and all contacts/interviews were sent advance copies of the questionnaire. Interviews were conducted in Arabic and were completed in about 10 minutes. Also, by calculating the possible non-responses in our phone survey target populations and then oversampling by that percentage, a statistically significant sample was achieved. ● The unwillingness of some target KIIs to participate in the evaluation introduced the possibility of sampling bias because, depending on the reasons for declining to participate, a respondent might have important but negative views about the subject being evaluated that it would have been helpful to know. To minimize the limitation of not being able to reach certain informants, the evaluators endeavored to pursue, to the extent possible, all targeted individuals. When challenges arose, such as missing contact information in a database, the team sought and received prompt assistance from both USAID and the IP. In addition, the data collection phase was extended to allow for more availability and participation of targeted respondents. ● Because COVID-19 restricted face-to-face interviews and group interviews, the quality of interactions with informants may have been limited. To compensate for the possible limitations of remote communication, phone and virtual contacts were copiously documented and digitized for recall during analysis. FINDINGS, CONCLUSIONS, AND RECOMMENDATIONS How FCR Discussion Is Organized This evaluation responds to three EQs and three cross-cutting issues (COVID-19, gender, and sustainability)3. Each EQ has a different focus, asking the evaluators to look at SEFPP from slightly different perspectives: • EQI asks for an overview of SEFPP’s performance in all activity components; • EQ2 asks for an analysis of SEFPP’s performance by activity objectives; and • EQ3 asks for an assessment of interventions most likely to strengthen capacity and commitment in local systems. For ease of investigation, analysis, discussion by topic, and reporting, the evaluators divided the activity into 12 components, or Thematic Focal Areas (TFAs), outlined below.4 1 Design, Theory of Change, and Sustainability 7. Service Delivery – Supply-side Interventions 2. Strategic Information 8. Operational Partnerships 3. Strategic Partnerships 9. Social and Behavior Change Communication (SBCC) 4. Activity Management 10. Training /Capacity Building Approaches 5. Monitoring, Evaluation, and Learning (MEL) 11. Training/Capacity Building – Service Providers 3 Sustainability is discussed as a cross-cutting issue and is also discussed in report sections concerned with Design, Theory of Change, and Sustainability. 4 In this evaluation, the terms “activity components” and “Thematic Focal Areas” are interchangeable. Both describe the same groupings or clusters of activities with similar processes and knowledge requirements. USAID.GOV MIDTERM EVALUATION OF STRENGTHENING EGYPT’S FAMILY PLANNING PROGRAM | viii 6. Service Delivery – Contraceptive Security 12. Training/Capacity Building – Non-service Providers Based on the different perspectives of the EQs, the 12 TFAs are discussed in this evaluation as follows: ● Activity Achievements – An Overview (analyzes all 12 TFAs); ● Findings, Conclusions, and Recommendations by Activity Objectives (discusses TFAs 1-9); and ● Findings, Conclusions, and Recommendations – Training and Capacity Building (discusses TFAs 10-12). This Executive Summary discusses FCRs as a Summary, followed by summarized discussion of each TFA and cross-cutting issue. FCR SUMMARY EQ1 – (Broad program level) To date, how successful has SEFPP been in achieving results under each component? What has been accomplished so far? What have been the challenges? What factors have led to specific accomplishments? Specific factors to examine include but are not limited to performance against indicators, relationship management with GOE stakeholders, internal SEFPP management, and implementation. In response to EQ1, see the Matrix, below. The Matrix provides an overview of SEFPP’s achievements for all activity components (that is, TFAs). The Matrix provides a “heat map” that allows the evaluators to develop a consensus (based on data, evidence, and intuitive professional judgement) about how well the activity has done in specific areas. On the color-coded heat map5, an example of a “Not successful” activity would be a scenario where SEFPP attempted an intervention that was curtailed at the MOHP’s direction. The reason why this cancellation occurred might appear in the Matrix as a “challenge,” but “Not successful” activities do not explicitly appear in the matrix. Activities that are assessed as having “Some Success” are a “mixed bag” where SEFPP has made some inroads in reaching its goals, but more effort or improved strategic direction is needed to facilitate the effort. “Good progress” suggests that activities have been productive and SEFPP should continue (perhaps with some refinement) to do what it has been doing. EQ2 – (IP operational level) To date, given the current activity’s status and the results reported under each objective: (1) Strengthen evidence-based planning for decision making, (2) quality family planning/reproductive health (FP/RH) services supported, (3) selected health behaviors enhanced, in the quarterly progress reports, annual reports, and the Performance Plan & Report (PPR), how successful has the activity been, including the factors contributing to its success? What are the hindrances? The response to EQ2 focuses on the activity’s three key objectives: (1) strengthen evidenced-based planning for decision-making, (2) support the quality of FP/RH services, and (3) enhance selected health behaviors. The activity’s successes and hindrances for each key objective can be summarized as follows: 5 Use of a color coded heat map is an intuitive assessment tool employed widely by USAID in Globl Health Security Agenda (GHSA) strategic planning and World Health Organization (WHO) Joint External Evaluations. ix | MIDTERM EVALUATION OF STRENGTHENING EGYPT’S FAMILY PLANNING PROGRAM USAID.GOV ACTIVITY ACHIEVEMENTS MATRIX – AN OVERVIEW MAIN RESULTS ACHIEVED CLASSIFIED AS: NOT SUCCESSFUL (NO EFFORT/ACTION TAKEN) SOME SUCCESS (SOME PROGRESS, BUT STRATEGY NEEDS TO CHANGE OR MORE EFFORT NEEDED) GOOD PROGRESS (EFFORTS ARE GOING IN RIGHT DIRECTION AND SHOULD BE CONTINUED OR ACCELERATED) TECHNICAL FOCAL AREAS SUCCESSES: MAIN RESULTS ACHIEVED FACTORS LEADING TO ACHIEVEMENTS CHALLENGES Design, Theory of Change and Sustainability USAID and SEFPP made program and agreement adjustments and proceeded to implement the activity despite the need to drop the research component of the activity because of a non- permissive environment for data access. SEFPP has made good progress in several areas likely to lead to positive outcomes and sustainability opportunities exist to do more. ((GOOD PROGRESS) USAID and SEFPP diligence and flexibility in being as responsive as possible to MOHP guidance; systematically eliminating some activities while retaining others. JSI’s flexibility in agreeing to work with a changed activity framework. . JSI applies principles of sustainability in development assistance in its internal strategic planning. Non-permissive environment for data access (including SEFPP’s non- involvement in service delivery and community-level measurement) has required SEFPP to make many technical adjustments and is still a concern, although there is a sense within SEFPP that the environment is improving. Strategic Information Highly productive technical assistance to NPC on MIS systems developments and upgrades Training of both MOHP and NPC on basic research protocols, and NPC staff demonstrating use of learned skills in follow-on research activity with the United Nations Children’s Fund (UNICEF) (SOME SUCCESS) NPC reception and cooperation. Effective SEFPP assistance. Limited coordination and collaboration between MOHP and NPC on systems developments and systems information outputs. Strategic Partnerships UNFPA and SEFPP agreed to a productive, ongoing working relationship. (SOME SUCCESS) MOHP’s role as mediator led to a better-defined working framework for both the United Nations Population Fund (UNFPA) and SEFPP. Willingness of the parties to resolve differences and work cooperatively. Limited number of partners in FP developments in Egypt. Limited opportunities to confer with other strategic partners. Activity Management JSI kept the activity going by gap-filling, and, despite staff shortages and internal pressures, achieved results and responded to MOHP priorities. (SOME SUCCESS) JSI was diligent and persistent in supporting SEFPP’s various programmatic and staffing challenges. SEFPP experienced high project staff turnover6, with the continuous need to shift technical and management staff around to fill gaps. 6 The severity of turnover took into consideration the rate of departures, but more importantly, its impact on the organization. Internal disruptions and uncertainties caused by senior management departures, use of interim substitutes, and on-going senior level vacancies were consistently reported and discussed by JSI headquarters management and SEFPP staff as being a consequential problem. The evaluators determined that the severity of the problem as described warranted characterizing the turnover as “high.” There was agreement with on this in the March 3, 2021 Validation Meeting. USAID.GOV MIDTERM EVALUATION OF STRENGTHENING EGYPT’S FAMILY PLANNING PROGRAM | x Recruitment of staff has been difficult because of uncertainties about the activity’s future. Monitoring, Evaluation, and Learning The MEL framework/indicators and Monitoring, Evaluation, and Learning Plan (MELP) were adjusted to reflect activity changes and modifications in a timely fashion and were approved by USAID. MEL introduced Click Up (a project management tool) and improved internal activity monitoring, communication, and learning. (SOME SUCCESS) JSI provided effective management guidance to its field office. SEFPP had technical staff who were effective in implementing MEL systems and plans. The MEL staff had to respond to required activity changes and modifications in a short timeframe. MEL and MELP productivity had to be maintained amid high staff turnover. Operational Partnerships Development of memoranda of understanding (MOUs) with private sector factories for FP awareness building and peer education was productive. Development of an MOU with the American University in Cairo (AUC) for youth engagement in FP awareness building and peer education was productive. (SOME SUCCESS) SEFPP found a receptive environment for awareness building activities in the private sector. SEFPP was attentive to the interest of participating companies and AUC and provided needed technical support. Participating partners need encouragement and capacity building in measuring and reporting on outcome results in addition to output results because outcome measures are more difficult to construct and carry out. Service Delivery- Contraceptive Security SEFPP has trained MOHP, NPC, and Ministry of Finance (MOF) personnel in forecasting and quantification methods. SEFPP has trained MOHP personnel in conducting warehouse assessments and will support the MOHP in carrying assessments. SEFPP has provided assistance to MOHP to define the criteria for selecting newly added FP methods in the public sector. (SOME SUCCESS) SEFPP was responsive to MOHP-expressed priorities. SEFPP is not involved with the MOHP’s upgrades and use of the LMIS, the critical information system that informs MOHP forecasting and quantification; thus, the substance of SEFPP’s training in this area will be limited. SEFPP’s lack of access to geographically defined behavioral data about FP users limits SEFPP and MOHP capacity to make evidence-based decisions about introducing new methods. By SEFPP’s assessment, the MOHP’s systems and procedures for testing new methods are weak; thus, the mere identification of candidate methods may not lead to adaptation and introduction. Service Delivery – Supply-side Interventions. SEFPP’s response to the MOHP by assisting the postpartum contraception programs and promotion of high priority national task-sharing has been productive and appreciated by the MOHP. (GOOD PROGRESS) SEFPP was responsive to MOHP-expressed priorities. SEFPP had the technical know-how to assist in the subject areas. SEFPP provided effective technical assistance. The overall environment for better use of evidence to inform planning is needed, allowing for these two programs (along with the MOHP FP sector) to improve measurement of outcome results. SBCC Based on established working relationships, SEFPP will assist NPC with updating the national SBCC strategy. (GOOD PROGRESS) SEFPP established an effective working relationship with NPC. SEFPP established technical competencies to assist NPC with evidence-based strategic planning. SEFPP attempted to undertake several interventions that the MOHP curtailed because of their close relationship to the use of intermediaries for messaging and implied access to facility and client/community-level data. xi | MIDTERM EVALUATION OF STRENGTHENING EGYPT’S FAMILY PLANNING PROGRAM USAID.GOV Training/Capacity Building – Approaches The training of trainers (TOT) approach is popular with individuals, programs, and agencies; e-learning is progressing; and qualification tracking is responsive to MOHP needs. (GOOD PROGRESS) On TOT, individuals, programs, and agencies recognize that TOT can pay dividends (such as upward career mobility and the sustainability of the training). SEFPP’s responses to the MOHP on e-learning and tracking have been strong. E-learning developments are reported by SEFPP staff and the MOHP to be proceeding well, but SEFPP needs to monitor progress closely. Training/Capacity Building – Providers and Non- providers Training beneficiaries give training high marks for relevance and applicability to their work. (GOOD PROGRESS) Training was based on needs assessments, and delivery was well-organized and supported by MOHP. SEFPP needed to switch to virtual platforms because of COVID- 19. . More training needs assessments are required to guide SEFPP and the MOHP with further prioritization, targeting of resources, and customizing training content. USAID.GOV MIDTERM EVALUATION OF STRENGTHENING EGYPT’S FAMILY PLANNING PROGRAM | xii Evidence-based planning Evidence-based planning for decision-making is a basic program development concept and is fundamental to SEFPP’s activity design. However, within one year of SEFPP’s inception, the activity faced a major challenge when SEFPP was advised by the MOHP of an emerging non-permissive environment for external partners such as JSI/SEFPP to have access to data or engage in activities that might be construed as data capture or data access at the operational systems or client or community level of the FP program. These restrictions presented a significant challenge to the activity and resulted in the activity having to drop the major research component of the original activity design and make many other programmatic and management changes. It meant that the activity could not conduct baseline surveys and could not conduct interventions that might involve service provision, client, or community-level data gathering. It limited the data available for monitoring and evaluation: measurement of activity outputs was within the activity’s domain, but measurement of related outcome results was and still is problematic. In addition to the problems of data access described above, SEFPP has also not been involved with the MOHP’s strategic information systems upgrades and access to the type of data/information the MOHP would need to use for strategic planning and decision making. While the activity has been productive in providing technical assistance to the NPC in upgrading and automating its Management Information System (MIS), the activity does not have a similar technical assistance partnership with the MOHP. It should be noted that while initially, the MOHP was open to SEFPP helping develop the HMIS a decision was made by the Ministry at some point not to utilize SEFPP’s expertise, but to instead use internal resources. Through the course of interviews various reasons have been given for this shift ranging from “political changes,” “leadership changes,” to availability of new GOE resources. However, while a clear chain of events was not definitively identified, the evaluators feel it vital to mention this as it had major implications for other parts of the activity’s portfolio. Specifically, because SEFPP is not involved with the MOHP’s upgrades and use of the LMIS, the information system that informs MOHP forecasting and quantification, SEFPP’s training in the area of quantification was not based on contemporary MOHP data, and thus, not as practical as it should be. Another consequence is that SEFPP’s lack of access to geographically defined behavioral data about FP users limits SEFPP and MOHP capacity to make evidence-based decisions about introducing new methods. Overall, the environment was also destabilizing internally for the activity in the early days, as evidenced by high staff turnover (the activity’s management outlook has improved after three-and-a-half years). Despite these challenges, USAID and SEFPP successfully made program and agreement adjustments and proceeded to implement the activity. Moving forward, a key to optimizing SEFPP’s outcome results and sustainability potential during its remaining years will be for SEFPP to undertake a strategic action plan to mitigate existing restrictions on data capture and data access and for USAID to additionally support SEFPP’s mitigation efforts through policy dialogue at appropriate levels with GOE counterparts. This evaluation recommends that SEFPP leadership re-engage the MOHP in dialogue to improve the overall working environment for data capture and access and in the generation and use of strategic data and information to inform evidence-based planning and decision making. The evaluation team also recommends that SEFPP identify strategic planning “champions” in the MOHP and NPC and, as opportunities arise, sponsor these champions to participate in regional or international forums that promote evidence-based planning and decision making. Examples might be relevant World Health Organization (WHO) conferences or a technical workshop on data for contraceptive forecasting. Furthermore, the team recommends that USAID explore (and implement, if indicated) a strategy to support and reinforce JSI/SEFPP efforts to undertake sustainable evidence￾based planning (such as being able to conduct service delivery and client-based data and information), and behavior change communications (such as community-based data gathering). It should be noted that SEFPP has recently made inroads in this area. SEFPP has been able to get client-based data for innovative interventions such as for postpartum client utilization data and has also received concurrence from MOHP to provide client utilization data to scale up the task sharing intervention. While these appear to be breakthroughs, to push the strategy further, USAID might include eliciting mission and embassy involvement at the dialogue level with the GOE, as appropriate. xiii | MIDTERM EVALUATION OF STRENGTHENING EGYPT’S FAMILY PLANNING PROGRAM USAID.GOV Quality of family planning services SEFPP has been relatively successful in contributing to MOHP priorities aimed at improving the availability of quality FP services. Notably, due to early successes, the activity has played an important role in training health workers in a program to promote and expand postpartum contraceptive services and to expand the range of FP services that nurses can provide to FP users (called “task￾sharing” training). Enhance selected health behaviors SEFPP has also made major contributions to demand creation through family planning, awareness building and peer education in organizing and supporting private sector workplaces such as factories, as well as supporting youth engagement activities. However, the activity has been less successful with undertaking several interventions that the MOHP curtailed because of the interpretation that the activities were too closely related to the use of intermediaries in gathering facility and community data. At the same time, SEFPP works well with NPC and, in the coming year, will assist NPC in updating the national Social and Behavior Change Communication (SBCC) strategy (concerned with FP demand creation). The evaluation’s recommendations, in terms of SEFPP’s various supply-side and demand￾side interventions aimed at improving quality and demand for services, are mixed: continue and expand those interventions that appear productive but be cautious about continued investments of staff time and attention in those areas that have been problematic up to now. This is particularly true about possible re-engagement with the MOHP on SBCC interventions that were previously curtailed. EQ3 – (Service level) To date, in what ways did the SEFPP activity strengthen capacity and commitment in the local systems (health workforce, health information system, and health service delivery) to continue delivering consistent levels of access to and delivery of quality FP services? EQ3 focuses on training and building skills and capacities among health workers who directly work with FP clients, in FP management systems, and with adjunct workers (such as media professionals). Training and capacity building The activity’s best news is in the area of training and capacity building. The approach of training trainers (called “TOT”) supported by SEFPP has proven to be popular with individuals, programs, and agencies because it is recognized that TOT can pay dividends (such as upward career mobility and the sustainability of the training). SEFPP is generally recognized for its training programs. Training beneficiaries give the trainings high marks for relevance and applicability to their work. The evaluators did note several challenges: SEFPP has had to switch to virtual training platforms because of COVID￾19 and is likely to continue and/expand these training modalities. There were several comments from informants to the effect that private sector pharmacists might need more customized training. However, in general, demand for training was high, noting that SEFPP should conduct more needs assessments to better prioritize, target resources, and customize training content. In addition, a phone survey suggested that SEFPP needs to give more priority to training private sector pharmacists. CROSS-CUTTING ISSUES COVID-19 SEFPP adapted quickly to the challenges of COVID-19 by supplementing MOHP needs for infection control supplies and moving to virtual and online formats for training and assisting the NPC in research and improving public awareness of COVID-19’s impact. This evaluation recommends that, in the interest of continuity of FP services, and based on MOHP requests and the availability of budgetary resources, SEFPP should continue or increase the provision of infection control supplies to more beneficiaries (service and non-service providers). In addition, SEFPP should continue and expand the use of virtual and online communications strategies so long as the COVID-19 threat and mitigation activities are required. USAID.GOV MIDTERM EVALUATION OF STRENGTHENING EGYPT’S FAMILY PLANNING PROGRAM | xiv Gender SEFPP originally put considerable effort into developing a comprehensive gender strategy and has implemented many aspects of the strategy but, by its own estimates, did not consistently ensure that all aspects of its strategy are culturally responsive and well-integrated and implemented in its activities as intended. This evaluation recommends that SEFPP complete its Year 4 plans to strengthen implementation of its gender strategy, including designating a gender focal point that will organize orientation meetings for SEFPP staff on basic gender equality concepts and applications. SEFPP should also develop and upgrade its gender mainstreaming guidelines to be used by staff in its tools, training materials, and manuals. Sustainability SEFPP’s understanding of sustainability appears to be consistent with USAID’s principles as defined in USAID policy and program design guidelines, and is articulated throughout its documentation and evident in many of the activities it has undertaken during the past three years. Sustainability is seen as a long-term goal of the activity that SEFPP believes will be achieved through the combined and interactive effects of interventions implemented throughout the program’s life. The combined effects of SEFPP’s interventions aim to ensure ongoing impact long after the program ends. As an outgrowth of its activities over the past several years, SEFPP has many opportunities to promote sustainable interventions, particularly in the areas of training and capacity building. 1 | MIDTERM EVALUATION OF STRENGTHENING EGYPT’S FAMILY PLANNING PROGRAM USAID.GOV INTRODUCTION The Strengthening Egypt’s Family Planning Program (SEFPP) activity began in December 2017, with anticipated completion in December 2022 with a Total Estimated Cost (TEC) established at $18,788,799. SEFPP’s Cooperative Agreement aims to assist the Government of Egypt (GOE) to re￾establish what was (in the early 2000s) a strong family planning (FP) system. SEFPP works to increase evidence-based planning, decision making, and policy formulation, including contraceptive security, in Egypt by supporting quality family planning/reproductive health (FP/RH) services and enhancing selected health behaviors. SEFPP, implemented by John Snow, Inc. (JSI), assists the GOE, through the Ministry of Health and Population (MOHP)/Family Planning Sector and National Population Council (NPC), and private sector partners and NGOs in addressing the country’s rapid population growth by reducing the unmet need7 of women and men for FP counseling services and methods. SEFPP prioritizes its support in the nine governorates of Upper Egypt and selected slum areas in Cairo and Alexandria, where the need for increased access to FP is greatest and on-going MOHP and NPC interventions can be scaled up. The primary audience of this evaluation report is USAID/Egypt, specifically USAID senior mission management, the Office of Education and Health (OEH), and the Program Office. Secondary audiences include the MOHP, NPC, and JSI. USAID/Egypt will share the executive summary, final report, and recommendations with the implementer, stakeholders, beneficiaries, and the general public via the Development Education Clearinghouse (DEC) within 90 days of report completion. ACTIVITY INFORMATION Table 1. Summary of Activity Information PROJECT NAME Strengthening Egypt’s Family Planning Program IMPLEMENTER John Snow, Inc. (JSI) COOPERATIVE AGREEMENT # AID-263-A-15-00022 TOTAL ESTIMATED COST OF THE EVALUATED PROJECT (TEC) $18,788,799 LIFE OF PROJECT December 2017–December 2022 ACTIVE GEOGRAPHIC REGIONS Assuit, Beni Suef, Fayoum, Giza, Luxor, Minya, Qena, Sohag, Aswan, and marginalized areas in Cairo and Alexandria DEVELOPMENT OBJECTIVES (DOS) 8 DO 3: Health outcomes for target populations improved and identified in the Health Project Appraisal Document (PAD) Intermediate Result (IR) 3.1: Quality family planning reproductive health services supported IR 3.2: Evidence-based decision making increased USAID/EGYPT OFFICE Office of Education and Health 7 Unmet need is the gap between women’s reproductive intentions and their contraceptive behavior. 8 As defined prior to development and approval of the current USAID/Egypt Country Development Cooperation Strategy 2020-2025. USAID.GOV MIDTERM EVALUATION OF STRENGTHENING EGYPT’S FAMILY PLANNING PROGRAM | 2 ACTIVITY BACKGROUND Following a hiatus in USAID funding for the FP sector in Egypt (from approximately 2010 to 2017), USAID re-entered the FP arena and worked with GOE to establish the SEFPP. SEFPP aims to assist the GOE in addressing the country’s rapid population growth by reducing the unmet needs of women and men for FP counseling services and methods. SEFPP’s long-term objective is to expand contraceptive use to reduce the number of high-risk and high-parity births. The activity works by helping the GOE to re-establish a strong FP system integrated with other related MOHP units such as Maternal and Child Health Care under the Primary Heal Care Sector, and the Obstetrics and Gynecology (OB/GYN) Departments within hospitals under the Curative Care Sector. Evidence￾based strategic planning is integral to this development objective. SEFPP provides technical assistance to the MOHP/FP Sector and the NPC to reinvigorate and sustain Egypt’s FP program. All SEFPP interventions are conducted through the activity’s implementation partners, namely, the MOHP/FP Sector, NPC, private sector partners and NGOs. EVALUATION PURPOSE AND EVALUATION QUESTIONS EVALUATION PURPOSE The purpose of this mid-term evaluation is to identify areas for SEFPP to focus on for the remaining period and to identify evidence of good practices for a broad network of stakeholders.9 The evaluation aims to serve learning purposes by identifying any challenges incurred by the activity to date and formulating appropriate recommendations for corrective actions and effective implementation during the remaining years of the activity and future activities. EVALUATION QUESTIONS EQ1: BROAD PROGRAM LEVEL To date, how successful has SEFPP been in achieving results under each component? What has been accomplished so far? What have been the challenges? What factors have led to specific accomplishments? Specific factors to examine include, but are not limited to, performance against indicators, relationship management with GOE stakeholders, internal SEFPP management, and implementation. EQ2: IMPLEMENTING PARTNER (IP) OPERATIONAL LEVEL To date, given the current activity’s status and the results reported under each objective in the quarterly progress reports, annual reports, and the performance plan and report (PPR), how successful has the activity been, including the factors contributing to its success? What are the hindrances? EQ3: SERVICE LEVELTo date, in what ways did SEFPP strengthen capacity and commitment in the local systems (health workforce, health information system, health service delivery) to continue delivering consistent levels of access to and delivery of quality FP services? Cross-cutting issues are: COVID-19 How is SEFPP responding to and adapting to the COVID-19 crisis in each results area, and how effective are these interventions in fulfilling MOHP needs? GENDER How has SEFPP responded to USAID’s general guidelines on improving gender-based violence (GBV) and social equity through FP interventions? 9 See also Annex 1: Evaluation Statement of Work and Annex 2: SEFPP Midterm Evaluation Concept Note. 3 | MIDTERM EVALUATION OF STRENGTHENING EGYPT’S FAMILY PLANNING PROGRAM USAID.GOV SUSTAINABILITY Has SEFPP planned/implemented strategies to ensure that SEFPP can end without disruption to FP program assistance in Egypt, and that activities initiated by SEFPP will continue beyond SEFPP’s lifetime? EVALUATION METHODOLOGY A detailed discussion of the evaluation methodology can be found in Annex 4: Evaluation Methodology. The following is a summary. This SEFPP midterm evaluation used a mixed-methods approach. The evaluation team applied qualitative research methods, complemented by a quantitative opinion survey. The evaluation focused on the activity’s structure, operations, and outputs. The possible outcomes and impact of the activity were beyond the scope of this evaluation. The qualitative research for this evaluation included ten key informant interview (KII) tools intended to systematically interview program policymakers, managers, and field service providers. The quantitative research involved a questionnaire administered to service providers and non-service providers (administrative staff) who benefitted from SEFPP’s training and capacity building activities. Data Sources and Sampling. Sources of qualitative data and information included information derived from activity documents and from interviews with FP/RH professionals who had directly participated in SEFPP technical assistance. The evaluation used purposive sampling (also known as judgment, selective, or subjective sampling) to define distinct categories of informants and beneficiaries, for which interview tools were developed and used. The source of quantitative data was a digital questionnaire administered to a random selection of physicians, nurses, midwives and pharmacists and other adjunct health/family planning professionals who responded via a phone survey. Data Collection. Data collection activities included: ● A desk review of activity-related documents ● Key informant interviews: The evaluation team conducted 55 KIIs. As the data collection phase took place during the second wave of COVID-19, all KIIs were conducted virtually, by Zoom, Google Meet, or telephone. ● Phone survey: The evaluation team conducted 599 phone surveys with training beneficiaries. The purpose of the survey was to gather overall impressions from those trained about the strategic relevance and usefulness of the training they received and the service providers’ perspective on the level of demand for and quality of FP. This evaluation did not intend to be an in-depth assessment of training content, curricula, or management of the training activities. ● Validation meeting: A validation meeting with SEFPP staff and USAID allowed the evaluation team to receive inputs on its findings. The team incorporated feedback from the validation meeting into its findings, conclusions, and recommendations (FCRs). Data Analysis and Quality Assurance. The evaluation team conducted descriptive statistical analyses on the quantitative data and employed a codable thematic analytical approach for the qualitative data. These methods identified explicit and implicit patterns in respondents’ answers relating to the evaluation questions. The evaluation team conducted a data cleaning exercise and developed a coding tree for key informants. For EQs 1 and 2, qualitative findings derived primarily from KIIs, since interview questions were directly keyed and coded to their specific subject matters. The team reviewed findings within the context of specific subjects and achieved a consensus about the validity of findings through triangulation and cross-referencing reports and other documents. For EQ3, on training beneficiaries, the evaluation USAID.GOV MIDTERM EVALUATION OF STRENGTHENING EGYPT’S FAMILY PLANNING PROGRAM | 4 team could cross-validate findings between the KIIs and phone surveys by comparing responses to questions focused on satisfaction and the usefulness of the respondents’ training experiences. A formal validation meeting with SEFPP took place to verify and receive the implementing partner’s feedback on the core findings. USAID staff attended to provide clarification when needed. The team considered and cross-checked the IP’s inputs before finalizing conclusions and recommendations. Limitations and Mitigation Measures. Several challenges and limitations experienced in this evaluation were mitigated as follows: ● With the exception of data provided to scale up post-partum intervention and concurrence to provide data for scaling up task sharing, the lack of data capture at the level of change (especially related to policy-level data) in most instances made it impossible to measure incremental changes and progress in the FP program or estimate SEFPP’s contribution to such changes. Notably, certain populations that the evaluation might have wanted to interview, such as recipients of FP services at clinic/community-level, were not accessible because of MOHP restrictions on direct survey data collection. Furthermore, inaccessibility of certain public sector statistics (such as “stock on hand”) made it impossible for SEFPP to help the MOHP with evidence-based planning to estimate national program performance supply requirements. Thus, the evaluation could not adequately assess SEFPP’s contributions to contraceptive forecasting and quantification. It was not possible to assess improvements in service delivery or the impact of behavior change components that might have been attributable to SEFPP. To compensate for possible limitations in this evaluation that resulted from various restrictions on availability of data, the evaluation relied on output data and statistics that were generated from SEFPP reports and readily available in files or from discussions with informants. ● Limitations inherent in conducting phone surveys included time constraints (such as people being engaged in “at home” activities), difficulty in reaching respondents, and enumerators misstating a question or misinterpreting the answer. On the phone surveys, enumerators were trained and supervised throughout to avoid poor performance in conducting or notating interviews; digital data input (to SurveyMonkey) was controlled by a statistician/supervisor; respondents were contacted in advance, appointments were scheduled, and all contacts/interviews were sent advance copies of the questionnaire. Interviews were conducted in Arabic and were completed in about 10 minutes. Also, by calculating the possible non-responses in our phone survey target populations and then oversampling by that percentage, a statistically significant sample was achieved. ● The unwillingness of some target KIIs to participate in the evaluation introduced the possibility of sampling bias because, depending on the reasons for declining to participate, a respondent might have important but negative views about the subject being evaluated that it would have been helpful to know. To minimize the limitation of not being able to reach certain informants, the evaluators endeavored to pursue, to the extent possible, contacting the targeted individual. When challenges arose, such as missing contact information in a database, the team sought and received prompt assistance from both USAID and the IP. In addition, the data collection phase was extended to allow for more availability and participation of targeted respondents. ● Because COVID-19 restricted face-to-face interviews and group interviews, the quality of interactions with informants may have been limited. To compensate for the possible limitations of remote communication, phone and virtual contacts were copiously documented and digitized for recall during analysis. 5 | MIDTERM EVALUATION OF STRENGTHENING EGYPT’S FAMILY PLANNING PROGRAM USAID.GOV FINDINGS, CONCLUSIONS, AND RECOMMENDATIONS (FCRS) How FCR Discussion Is Organized This evaluation responds to three EQs and three cross-cutting issues (COVID-19, gender, and sustainability)10. Each EQ focuses differently and asks the evaluators to look at SEFPP from slightly different perspectives: • EQI asks for an overview of SEFPP’s performance in all activity components, • EQ2 asks for an analysis of SEFPP’s performance by activity objectives, and • EQ3 asks for an assessment of interventions most likely to strengthen capacity and commitment in local systems. For ease of investigation, analysis, discussion by topics, and reporting, the evaluators divided the activity into 12 components, or Thematic Focal Areas (TFAs).11 The 12 TFAs are: 1. Design, Theory of Change, and Sustainability 7. Service Delivery – Supply-side Interventions 2. Strategic Information 8. Operational Partnerships 3. Strategic Partnerships 9. Social and Behavior Change Communication (SBCC) 4. Activity Management 10. Training /Capacity Building Approaches 5. Monitoring, Evaluation, and Learning (MEL) 11. Training/Capacity Building – Service Providers 6. Service Delivery – Contraceptive Security 12. Training/Capacity Building – Non-service Providers Based on the different perspectives of the EQs, the 12 TFAs are discussed in this evaluation as follows: • Activity Achievements – An Overview (analyzes all 12 TFAs); • Findings, Conclusions, and Recommendations by Activity Objectives (discusses FCRs for 9 of the 12 TFAs); and • Findings, Conclusions, and Recommendations – Training and Capacity Building (discusses FCRs for 3 of the 12 TFAs). The Background and Context discussion found at the beginning of each TFA provides information that supports the FCRs. A recommendation will be linked to one or more Key Findings or Conclusions. This evaluation identified many findings, but only pertinent key findings are presented in this report. 10 Sustainability of the program is examined as a cross-cutting issue and, because of its close links to evidenced-based design, planning, program development, sustainability is also covered extensively in the Design, Theory of Change and Sustainability sections of this report. 11 In this evaluation, the terms “activity components” and “Thematic Focal Areas” are interchangeable. Both describe groupings or clusters of activities with similar processes and knowledge requirements. USAID.GOV MIDTERM EVALUATION OF STRENGTHENING EGYPT’S FAMILY PLANNING PROGRAM | 6 FINDINGS, CONCLUSIONS, AND RECOMMENDATIONS BY ACHIEVEMENTS EQ1 – (Broad program level) To date, how successful has SEFPP been in achieving results under each component? What has been accomplished so far? What have been the challenges? What factors have led to specific accomplishments? Specific factors to examine include but are not limited to performance against indicators, relationship management with GOE stakeholders, internal SEFPP management, and implementation. ACTIVITY ACHIEVEMENTS MATRIX — AN OVERVIEW See Table 2, below. 7 | MIDTERM EVALUATION OF STRENGTHENING EGYPT’S FAMILY PLANNING PROGRAM USAID.GOV TECHNICAL FOCAL AREAS SUCCESSES; MAIN RESULTS ACHIEVED FACTORS LEADING TO ACHIEVEMENTS CHALLENGES Design, Theory of Change and Sustainability USAID and SEFPP made program and agreement adjustments and proceeded to implement the activity despite the need to drop the research component of the activity because of a non- permissive environment for data access. SEFPP has made good progress in several areas likely to lead to positive outcomes and sustainability opportunities exist to do more. ((GOOD PROGRESS) USAID and SEFPP diligence and flexibility in being as responsive as possible to MOHP guidance; systematically eliminating some activities while retaining others. JSI’s flexibility in agreeing to work with a changed activity framework. . JSI applies principles of sustainability in development assistance in its internal strategic planning. Non-permissive environment for data access (including SEFPP’s non-involvement in service delivery and community-level measurement) has required SEFPP to make many technical adjustments and is still a concern, although there is a sense within SEFPP that the environment is improving. Strategic Information Highly productive technical assistance to NPC on MIS systems developments and upgrades Training of both MOHP and NPC on basic research protocols, and NPC staff demonstrating use of learned skills in follow-on research activity with the United Nations Children’s Fund (UNICEF) (SOME SUCCESS) NPC reception and cooperation. Effective SEFPP assistance. Limited coordination and collaboration between MOHP and NPC on systems developments and systems information outputs. Strategic Partnerships UNFPA and SEFPP agreed to a productive, ongoing working relationship. (SOME SUCCESS) MOHP’s role as mediator led to a better-defined working framework for both the United Nations Population Fund (UNFPA) and SEFPP. Willingness of the parties to resolve differences and work cooperatively. Limited number of partners in FP developments in Egypt. Limited opportunities to confer with other strategic partners. Table 2 -- ACTIVITY ACHIEVEMENTS – AN OVERVIEW MAIN RESULTS ACHIEVED CLASSIFIED AS: NOT SUCCESSFUL (NO EFFORT/ACTION TAKEN) SOME SUCCESS (SOME PROGRESS, BUT STRATEGY NEEDS TO CHANGE OR MORE EFFORT NEEDED) GOOD PROGRESS (EFFORTS ARE GOING IN RIGHT DIRECTION AND SHOULD BE CONTINUED OR ACCELERATED) USAID.GOV MIDTERM EVALUATION OF STRENGTHENING EGYPT’S FAMILY PLANNING PROGRAM | 8 Activity Management JSI kept the activity going by gap-filling, and, despite staff shortages and internal pressures, achieved results and responded to MOHP priorities. (SOME SUCCESS) JSI was diligent and persistent in supporting SEFPP’s various programmatic and staffing challenges. SEFPP experienced high project staff turnover12, with the continuous need to shift technical and management staff around to fill gaps. Recruitment of staff has been difficult because of uncertainties about the activity’s future. Monitoring, Evaluation, and Learning The MEL framework/indicators and Monitoring, Evaluation, and Learning Plan (MELP) were adjusted to reflect activity changes and modifications in a timely fashion and were approved by USAID. MEL introduced Click Up (a project management tool) and improved internal activity monitoring, communication, and learning. (SOME SUCCESS) JSI provided effective management guidance to its field office. SEFPP had technical staff who were effective in implementing MEL systems and plans. The MEL staff had to respond to required activity changes and modifications in a short timeframe. MEL and MELP productivity had to be maintained amid high staff turnover. Operational Partnerships Development of memoranda of understanding (MOUs) with private sector factories for FP awareness building and peer education was productive. Development of an MOU with the American University in Cairo (AUC) for youth engagement in FP awareness building and peer education was productive. (SOME SUCCESS) SEFPP found a receptive environment for awareness building activities in the private sector. SEFPP was attentive to the interest of participating companies and AUC and provided needed technical support. Participating partners need encouragement and capacity building in measuring and reporting on outcome results in addition to output results because outcome measures are more difficult to construct and carry out. Service Delivery- Contraceptive Security SEFPP has trained MOHP, NPC, and Ministry of Finance (MOF) personnel in forecasting and quantification methods. SEFPP has trained MOHP personnel in conducting warehouse assessments and will support the MOHP in carrying assessments. SEFPP has provided assistance to MOHP to define the criteria for selecting newly added FP methods in the public sector. SEFPP was responsive to MOHP-expressed priorities. SEFPP is not involved with the MOHP’s upgrades and use of the LMIS, the critical information system that informs MOHP forecasting and quantification; thus, the substance of SEFPP’s training in this area will be limited. SEFPP’s lack of access to geographically defined behavioral data about FP users limits SEFPP and MOHP capacity to make evidence-based decisions about introducing new methods. By SEFPP’s assessment, the MOHP’s systems and procedures for testing new methods are weak; thus, the mere identification of candidate methods may not lead to adaptation and introduction. 12 The severity of turnover took into consideration the rate of departures, but more importantly, its impact on the organization. Internal disruptions and uncertainties caused by senior management departures, use of interim substitutes, and on-going senior level vacancies were consistently reported and discussed by JSI headquarters management and SEFPP staff as being a consequential problem. The evaluators determined that the severity of the problem as described warranted characterizing the turnover as “high.” There was agreement with on this in the March 3, 2021 Validation Meeting. 9 | MIDTERM EVALUATION OF STRENGTHENING EGYPT’S FAMILY PLANNING PROGRAM USAID.GOV (SOME SUCCESS) Service Delivery – Supply-side Interventions. SEFPP’s response to the MOHP by assisting the postpartum contraception programs and promotion of high priority national task-sharing has been productive and appreciated by the MOHP. (GOOD PROGRESS) SEFPP was responsive to MOHP-expressed priorities. SEFPP had the technical know-how to assist in the subject areas. SEFPP provided effective technical assistance. The overall environment for better use of evidence to inform planning is needed, allowing for these two programs (along with the MOHP FP sector) to improve measurement of outcome results. SBCC Based on established working relationships, SEFPP will assist NPC with updating the national SBCC strategy. (GOOD PROGRESS) SEFPP established an effective working relationship with NPC. SEFPP established technical competencies to assist NPC with evidence-based strategic planning. SEFPP attempted to undertake several interventions that the MOHP curtailed because of their close relationship to the use of intermediaries for messaging and implied access to facility and client/community-level data. Training/Capacity Building – Approaches The training of trainers (TOT) approach is popular with individuals, programs, and agencies; e-learning is progressing; and qualification tracking is responsive to MOHP needs. (GOOD PROGRESS) On TOT, individuals, programs, and agencies recognize that TOT can pay dividends (such as upward career mobility and the sustainability of the training). SEFPP’s responses to the MOHP on e-learning and tracking have been strong. The overall environment for better use of evidence to inform planning is needed, allowing for these two programs (along with the MOHP FP sector) to improve measurement of outcome results. E-learning developments are reported by SEFPP staff and the MOHP to be proceeding well, but SEFPP needs to monitor progress closely. Training/Capacity Building – Providers and Non-providers Training beneficiaries give training high marks for relevance and applicability to their work. (GOOD PROGRESS) Training was based on needs assessments, and delivery was well-organized and supported by MOHP. SEFPP needed to switch to virtual platforms because of COVID- 19. . More training needs assessments are required to guide SEFPP and the MOHP with further prioritization, targeting of resources, and customizing training content. USAID.GOV MIDTERM EVALUATION OF STRENGTHENING EGYPT’S FAMILY PLANNING PROGRAM | 10 FINDINGS, CONCLUSIONS, AND RECOMMENDATIONS BY ACTIVITY OBJECTIVE EQ2: IMPLEMENTING PARTNER (IP) OPERATIONAL LEVEL To date, given the current activity’s status and the results reported under each objective in the quarterly progress reports, annual reports, and the performance plan and report (PPR), how successful has the activity been, including the factors contributing to its success? What are the hindrances? Activity Objective 1: Strengthen Evidence-based Planning for Decision Making and Related Topics DESIGN, THEORY OF CHANGE, AND SUSTAINABILITY Background and Context At inception, JSI’s SEFPP Cooperative Agreement was anchored in a Theory of Change that postulated that, “Use of evidence (that is informed by valid data) leads to progressive policy formulation, robust strategic program planning, and reliable decision-making with sustainable outcome results.” This Theory of Change in SEFPP’s design was articulated in three interrelated objectives: (1) improved evidence-based planning, (2) improved quality of services, and (3) improved social behaviors—all of which are intended to work together to produce a strengthened, sustainable13 national FP program in Egypt. Challenges/Hindrances As stated in USAID’s ADS 201.3.4 guidelines14, several principles serve as the foundation for sustainable development assistance, including evidence-based activities and working through local actors to promote sustainability. Thus, within its first year of operation, SEFPP faced a major challenge when SEFPP was advised by the MOHP that external partners such as JSI/SEFPP could expect to experience restrictions in availability of direct access to data and engagement in activities that might be construed as data capture or access to local systems and client/community levels of the FP program. These restrictions severely constrained SEFPP’s ability to implement evidence-based activities and promote sustainability of interventions through evidence-based planning with key local actors, most notably the MOHP FP Sector. Generally speaking, the GOE and MOHP expect all such data collection activities to be managed by the GOE’s Central Agency for Public Mobilization and Statistics (CAPMAS). Limitations on strategic freedom such as those posed by the non-permissive environment in which SEFPP found itself in its first year of operation are nothing new in development assistance, and, over time, researchers have generated suggestions on how donors and implementers might handle such situations.15 Implementers are encouraged to use a three-pronged approach: (1) Acknowledge the restriction(s), (2) make financial and programmatic adjustments that make sense but leave room for ongoing bargaining, (3) develop longer-term action plans to mediate the restriction(s). This MOHP position had major implications for SEFPP’s structure and program scope and was extremely disruptive to the formative activity, but USAID, as the donor/financier, and JSI/SEFPP, as the implementer, made rapid accommodations and adjustments. The research component of SEFPP was dropped altogether, signaling the departure of one of JSI’s original consortium partners. Several key staff resigned. The challenges related to data capture and data access also necessitated substantial 13 Within the context of the SEFPP activity, the issue of sustainability asks the question, “Will the benefits of SEFPP last; to what extent will net benefits of SEFPP continue (or are likely to continue) after the activity ends?” 14 USAID Automated Directives System describing USAID organization and functions. 15 Doz and Prahalad, “How MNCs Cope with Host Government Interventions,” Harvard Business Review, March 1980. 11 | MIDTERM EVALUATION OF STRENGTHENING EGYPT’S FAMILY PLANNING PROGRAM USAID.GOV changes to the activity’s MEL plan. Over the past three years of operation, SEFPP has achieved progress in several areas likely to lead to positive outcomes and sustainability as listed in Key Findings, below but many challenges still exist. A key to optimizing SEFPP’s outcome results and sustainability potential during its remaining years will be: • for SEFPP to undertake a strategic action plan to mitigate existing restrictions on data capture and data access; and • for USAID to additionally support SEFPP’s mitigation efforts through policy dialogue at appropriate levels with GOE counterparts. Key Findings • SEFPP has an existing Sustainability Plan which it reviews periodically to determine if events or program changes that occurred during a review period will have an impact on the potential for sustainability of planned interventions, and to replan accordingly. ● USAID indicates that its bilateral communications and working relationships with the MOHP and NPC are cooperative, and it was observed that USAID endeavors to provide good backup support to SEFPP in facilitating the activity’s working relationships with local implementing partners. ● SEFPP’s efforts to align its interventions with MOHP FP Sector plans and priorities enhanced the potential sustainability of those activities. ● The TOT approach provided MOHP with cadres to address the shortage of trainers at the directorate level. It is expected that these trainers will continue to transfer knowledge beyond the project’s lifetime. ● Fostering the establishment of new functional relationships between the GOE (MOHP and NPC) and FP partners (such as AUC and private sector factories) is likely to result in cooperation that will endure beyond the life of the activity. ● The NPC has demonstrated a high degree of potential for sustainability by being able to utilize the cadre trained by SEFPP to implement new capacities (such as NPC’s recent partnership with UNICEF on population research). ● Similarly, nurses trained on task sharing are reported to be currently applying what they learned in SEFPP training and are expected to continue these practices after the project ends. ● Two pilot interventions (postpartum contraception and task sharing), and the NPC’s national observatory initiative are in various stages of national scale up. Conclusion ● SEFPP is able to promote and implement sustainable strategies and activities with NPC, but similar strategic freedom with the MOHP has been more challenging and requires further attention, particularly continued critical backup support from USAID. ● There may be opportunities for USAID to play a more active role at higher policy dialogue levels in helping to improve the enabling environment for SEFPP to work more closely with the MOHP on sustainable evidence-based planning and behavior change communications interventions. Recommendations ● SEFPP leadership should continue to engage the MOHP in dialogue aimed at improving the overall working environment for data capture and access, as well as the generation and use of strategic data/information to inform evidence-based planning and decision making. ● SEFPP should review and identify interventions that were productive and have the best potential to sustain themselves in the future. According to the findings of this evaluation, planning and implementation should be undertaken to develop consideration might be given, at a minimum, to interventions in the following areas: SEFPP investments in managerial and operational capacity building (such as systems developments that build institutional commitment); SEFPP investments in training FP influencers (i.e., peer educators); SEFPP investments in training workers to train others USAID.GOV MIDTERM EVALUATION OF STRENGTHENING EGYPT’S FAMILY PLANNING PROGRAM | 12 (e.g., TOT); and SEFPP investments in forging potentially sustainable partnerships (such as between the public and private sectors)16. ● SEFPP should continue or introduce measurements of results for interventions it has identified as potentially sustainable. Work Plans, targets, and quarterly monitoring of these sustainability-related interventions should be incorporated into SEFPP’s MELP. ● SEFPP should identify strategic planning “champions” in the MOHP and NPC and, as opportunities arise, sponsor these champions to participate in regional or international forums that promote evidence-based planning and decision making. Examples might be relevant World Health Organization (WHO) conferences or a technical workshop on data for contraceptive forecasting. It should be noted that SEFPP has attempted similar approaches in the past without success. It is worth another attempt with such a strategy as personnel in agencies change overtime, and a more conducive environment may emerge ● USAID/Egypt should explore (and implement, if indicated) a strategy to support and reinforce JSI/SEFPP efforts to undertake sustainable evidenced-based planning (such as being able to conduct service delivery and client-based data and information), and behavior change communications (such as community-based data gathering). This strategy might include eliciting Mission and even Embassy involvement at the dialogue level with GOE counterparts when appropriate or needed. STRATEGIC INFORMATION Background and Context Strategic information goes to the heart of GOE/MOHP’s concerns about external factors such as JSI/SEFPP having access to or engaging in activities that would provide access to data and information at the operational, client, and community levels. SEFPP’s Strategic Information Unit has had to engage in innovative and alternative ways of accommodating this challenge (such as attempting to work through intermediaries). Challenges/Hindrances SEFPP’s Strategic Information Unit did engage with the MOHP in offering assistance in strengthening the MOHP’s HMIS and Logistics Management Information System (LMIS). These are the two core information functions in the MOHP that generate and aggregate data about the system’s performance (service utilization statistics) and contraceptive consumption. After exploring the possibilities of SEFPP assisting with the HMIS and LMIS, early in SEFPP’s year 4, the MOHP expressed its preference to develop its HMIS and LMIS through its own resources, specifically, the MOHP’s Health Information Center (HIC). MOHP systems upgrades are still in the design phase, and actual work has not yet begun. Achievements Originally, SEFPP had proposed three components of assistance with information systems development that included both the MOHP and NPC, and SEFPP was to forge ahead with assistance to NPC in the area of MIS developments. Now that SEFPP has made concrete progress with NPC’s MIS module, SEFPP plans to re-engage with MOHP on this topic to discuss SEFPP’s support to MOHP in its HMIS. Despite the MOHP’s position on SEFPP not assisting with HMIS, SEFPP developed and shared data collection forms for quantification with the MOHP, and SEFPP conducted training and exercises on quantification. Plus, MOHP has requested SEFPP in Year 4 to review and update the Arabic quantification training materials and update the forecasting exercise based on updated data sources. 16 It should be noted that the categories of intervention as recommended may also be relevant and appropriate for consideration in future USAID project/activity designs and requests for assistance. 13 | MIDTERM EVALUATION OF STRENGTHENING EGYPT’S FAMILY PLANNING PROGRAM USAID.GOV In the meantime, SEFPP has succeeded in training MOHP and NPC staff in quantitative and qualitative research techniques and is working productively with NPC on developing an automated tracking system for MIS based on the GOE’s main health statistics data input forms from the field (called the “TA8 Application”). The GOE’s core “TA8” health information reporting form feeds both HMIS and LMIS. The MIS being developed by NPC is more comprehensive and gathers information related to FP and other population indicators, while MOHP is more concerned with beneficiary data and monthly reports. The NPC’s automated system will allow data collection from primary health care units (PHCUs) and outpatient clinics affiliated with non-governmental organizations (NGOs). SEFPP is supporting automated MIS developments in NPC with both hardware and software support. NPC software will also substantially enhance HMIS analytical functions such as contraceptive consumption data by target populations. SEFPP is also assisting NPC in establishing the NPC Observatory, which will serve as a resource hub for various decision-makers in need of FP and population information. In sum, two health information databases exist, an MIS with health sector components operated by NPC (automated), with technical support from SEFPP, with the other, to be managed by MOHP. Currently, it is not clear how the MOHP and NPC systems will coordinate, how each will complement the other, or how they may eventually converge. Key Findings ● Although SEFPP demonstrates care in aligning its objectives with those of its national partners (MOHP and NPC), this has not ensured a high level of buy-in from the MOHP, particularly in areas that touch on the MOHP’s concerns about SEFPP’s activities that would involve data capture and data access (such as HMIS and LMIS). ● SEFPP staff report that NPC has been a more receptive partner than the MOHP concerning HMIS/LMIS systems improvements and strategies for evidence-based planning. Conclusions Since SEFPP is not involved with the MOHP’s HMIS and LMIS upgrades and development, SEFPP also does not have access to HMIS/LMIS data and its uses in the MOHP. Particularly on LMIS, any assistance SEFPP provides the MOHP on training staff in LMIS areas such as quantification principles and methods will be theoretical, not practical, and of limited use in helping the MOHP to strengthen the technical capacity of staff in strategic information management. Recommendations ● In the near term, SEFPP should facilitate dialogue between MOHP and NPC on how MIS systems developments at NPC (assisted by SEFPP) can benefit MOHP efforts to upgrade its HMIS/LMIS and how both could work compatibly in the future. ● In the longer term, in addition to theoretical training on strategic information systems and its uses, SEFPP should re-engage with the MOHP on the possibilities of direct SEFPP assistance with strategic information systems upgrades (such as with LMIS) and related evidence-based planning. STRATEGIC PARTNERSHIPS Background and Context In its first years, SEFPP made early attempts to foster relationships with potential strategic family planning partners: the United Nations Population Fund (UNFPA), WHO, the World Bank, and the European Union (EU), the most relevant and enduring of which has been with UNFPA. SEFPP engaged with UNFPA in connection with an assessment on CS. SEFPP hired a well-known consultant to conduct a CS assessment to be presented in a high-level FP workshop. During the workshop however, MOHP USAID.GOV MIDTERM EVALUATION OF STRENGTHENING EGYPT’S FAMILY PLANNING PROGRAM | 14 observed that recommendations for a Contraceptive Security (CS) strategy should come from UNFPA (not SEFPP as a technical assistance activity). Following the workshop, the MOHP spearheaded discussions between the parties, an amicable agreement was reached, and the matter turned into a “learning event” for SEFPP, UNFPA, as well as the MOHP. SEFPP agreed to limit its contributions in the areas of CS to training on quantification, conducting warehouse assessments, and introducing new FP methods, while UNFPA would focus on broader strategy development. UNFPA and SEFPP now have a productive working relationship. Challenges/Hindrances It might also have been helpful for SEFPP to cultivate collegial relationships at appropriate technical levels with agencies such as the World Bank,17 WHO, and UNICEF, as these all have had historical interests in Egypt’s Population and FP program. However, those relationships did not evolve. Other operational distractions and high staff turnover within the activity probably limited SEFPP’s ability to undertake the time-consuming task of building other strategic partnerships. SEFPP believes this situation will improve as new leadership and staff become available to give attention to building these development partner relationships. Key Findings SEFPP made early attempts to foster partner relationships with potential strategic partners, including UNFPA and WHO. SEFPP had one meeting with the WHO, but there was no follow-up. There were no known initiatives with the World Bank. Conclusions ● Periods of high staff turnover and shortages of SEFPP staff to engage in the time-consuming task of cultivating working collaboration with other partners may have prevented SEFPP from developing working relationships with other entities. ● There may be opportunities for SEFPP to engage (or even initiate) NGO and development partner coordination forums around common agendas related to assistance to Egypt’s FP program. Recommendations ● SEFPP should establish a staff point of contact to undertake a current inventory of active NGO and development partner coordination forums and identify opportunities to forge collegial relationships with these strategic partners. SEFPP should decide which member of the team is most appropriate to engage in identifying and leading taking advantage of these opportunities. ● SEFPP should consider exploring the possibilities of sponsoring the MOHP to convene committees and forums to develop relevant implementation strategies (such as a national committee to promote long-acting reversible contraception in the public sector). This might include development partner level “brown bag” forums or virtual meetings, use of the MOHP’s e-learning platform, etc. ● In order to increase demand for services and further strengthen political will for FP programming, USAID might consider conducting a FP-specific political/ economic analysis or a similar mapping exercise to help identify important stakeholders to ensure they broaden their horizons and work with other (possibly Egyptian) stakeholders to improve the sustainability of the commitment to FP in the country. The evaluation team acknowledges that a similar exercise may have been done early on, however given SEFPP’s technical leadership changes as well as changes in local FP agencies and partner organizations, a second assessment will prove useful to help the activity continue to 17 Rabie, et al., “Transforming Family Planning Outlook and Practices in Egypt: A rights-based Approach,” World Bank, 2017. 15 | MIDTERM EVALUATION OF STRENGTHENING EGYPT’S FAMILY PLANNING PROGRAM USAID.GOV adapt. Given SEFPP’s remaining timeframe, this exercise might also be of use to future programs or inform their designs. ACTIVITY MANAGEMENT Background and Context During SEFPP’s first year of operation, restrictions on SEFPP data access, research and data collection emerged that caused programmatic and MEL system changes and upheavals that were, as claimed by SEFPP staff, major contributing factors in SEFPP’s difficulties in organizing, retaining staff, and directing its operations. The fact that SEFPP could not conduct client or community-level research meant that SEFPP could not provide relevant evidence to inform MOHP policy development, strategic planning, and decision making eliminating a core component of the activity’s design. Additionally, during the past three years, the original Chief of Party (COP) departed, and the leadership gap was filled by interim acting COPs. JSI recently hired a new permanent COP who began work in April 2021. . COP responsibilities appeared to be handled well by the currently designated Deputy Chief of Party. The Strategic Information Director’s position was vacant for some time and has recently been filled. Another key position, that of SBCC Director, was vacant at the time of the evaluation fieldwork having resigned in February 2021. At least one ex-staff member cited a general atmosphere of uncertainty about the activity and its future as a contributing factor to high staff turnover. Management guidance and direction to staff on implementation of SEFPP activities and interventions by the activity’s managers were also reported to be inadequate. There was a sense that, as one staff member put it: There wasn’t much cross-fertilization and information sharing among units of the project. MEL was not very effective in ensuring that the internal technical units were talking together and working for unified project purposes. I feel this is another reason why some staff resigned from the project. Key Findings ● Difficulties in SEFPP’s working understandings with the MOHP, general uncertainties about the activity’s future, and probable gaps in internal activity management led to high staff turnover during the activity’s first two years. ● JSI headquarters has been in constant and close communication about the local staff and activity management situation and some of its underlying causes and continues to make course corrections. Conclusion JSI appears to be in touch with the activity’s staffing and management difficulties and is taking corrective actions. The activity’s management environment is reported to be substantially improving, and SEFPP staff appear to be optimistic that the remaining two years of the activity will be productive. Recommendations Early in his/her administration, the new SEFPP COP should plan one or a series of virtual staff retreats to boost staff morale and cohesiveness covering such topics as SEFPP outlook through 2022, the evolving development environment for the SEFPP activity and the GOE, issues arising from MEL, review of FCRs from the midterm performance evaluation, etc. MONITORING, EVALUATION, AND LEARNING (MEL) Background and Context SEFPP’s MEL system is the backbone of its internal data, information, evidence, and activity management. Properly structured and implemented, the MEL should enable SEFPP to deliver effectively on program objectives by providing a dynamic monitoring and reporting system that documents, USAID.GOV MIDTERM EVALUATION OF STRENGTHENING EGYPT’S FAMILY PLANNING PROGRAM | 16 tracks, and assesses the output and outcome results of interventions throughout its lifecycle. The system’s indicators permit appraisal of SEFPP and help SEFPP to estimate (but not accurately measure) whether the activities (as planned or adjusted) are making progress toward improving the quality of FP/RH services. MEL should foster continuous collaboration, learning, and adapting within the internal activity management environment. A MEL plan (MELP) established shortly after SEFPP’s inception in December 2017 presents a logic model that explains the rationale and interlocking relationships between purposes/goals and stated objectives, sub-objectives, and activities. The MELP additionally outlines a methodology for data collection, data management, reporting, evaluation, learning, and dissemination. The MELP includes a set of performance standards and customized indicators to help track the progress of activities and assess SEFPP’s outputs and outcomes.18 Challenges/Hindrances The environment in which the MELP was implemented, however, differed from what was originally envisioned. Around SEFPP’s inception in December 2017, SEFPP management was advised by the MOHP FP Sector of a government-wide non-permissive environment for data collection, including any population or facility-based studies and/or qualitative or quantitative interviews. This non-permissive environment had profound implications for what activities SEFPP could implement and how it could measure outputs and outcomes. This resulted in some data quality issues (such as the reliability of alternative secondary data sources) as described in the FY 2019 Data Quality Assessment (DQA). These issues were addressed and improved significantly (as acknowledged in the FY 2020 DQA), although data access remains an issue that SEFPP is aware of and is working to resolve. In addition, there were many senior-level changes in the MOHP at that time, resulting in shifts and changes in SEFPP activities in an attempt to better align with MOHP needs and priorities. Additional adjustments to SEFPP’s work plan due to the global COVID-19 pandemic resulted in still other activity changes from the MOHP and FP implementers requiring changes in SEFPP’s activities/approaches and MELP revisions. Achievements To its credit, SEFPP and the MEL system responded robustly to these various changes and challenges. As described by one evaluation informant: “The MEL system actively19 revised the project’s M&E [monitoring and evaluation] framework while shifts were occurring (example, elimination of research activities) and used the issues arising from the many program changes to change official documents plus provide direction for internal project changes (and redirection of implementation). The MEL unit deserves credit for documenting the various changes (change of objectives, activities, and measurement indicators).” SEFPP also made a concerted effort to strengthen the MEL system’s internal communications, learning, and adapting processes. One evaluation informant summarized what appeared to be the consensus of many staff, saying that: “The project has been using the Click Up system for the past year. It links the work plan activities to indicators, and is accessible to all staff, even at home office. We have regular meetings with the senior management team in Egypt and the backstopping team at headquarters to stay on top of things. There are processes in place for the review of documents. There are multiple layers of communication, meetings, 18 SEFPP 5-year Revised MELP, December 2017, p. 8. 19 Interpreted as meaning, “immediately made changes and sought USAID approvals.” 17 | MIDTERM EVALUATION OF STRENGTHENING EGYPT’S FAMILY PLANNING PROGRAM USAID.GOV and processes and systems set up to make sure that there is good communication at a distance, and we’re able to provide support when needed.”20 There have also been notable improvements in internal staff communications and team collaboration since the introduction of output activity management tools and processes. The Click-up system is mentioned favorably by staff as having assisted in establishing a collaborative work environment. However, a downside to concentrating on output monitoring may be neglecting the ongoing need for primary sourced outcome results data for MEL purposes. The GOE and MOHP are reported to be in the planning stages of a “Family Health Survey,” a national household-level health/FP survey (similar in nature and content to the traditional Demographic and Health Surveys conducted in many countries) to be conducted by CAPMAS. It is expected that SEFPP and other development partners would have access to these survey data (including metadata, if necessary) to inform FP policy formulation and planning. Key Findings ● Currently, for national FP planning purposes, the best data and objective evidence of client and community-level FP/RH knowledge, attitudes, and practices by geographic area are based on 2014 national FP statistics (periodically updated from secondary FP data sources). ● The GOE and MOHP are in the planning stages of conducting a Family Health Survey through CAPMAS that will provide updated FP statistics. ● The current MELP reflects updates and activity modifications that were required after inception of the activity and complies with USAID functional and reporting requirements. ● The current MEL system functions well as an internal activity management and learning tool. Conclusion Despite secondary FP statistics now available, in the absence of relatively up-to-date primary national FP national statistics to inform policy and program direction, FP sector policymakers and strategic planners, as well as implementers such as SEFPP, remain restricted in being able to plan and implement a robust, evidence-based national FP program and could still benefit from improved access to qualitative and quantitative service delivery and community-level data. Recommendations In anticipation of the GOE’s planned Family Health Survey being completed within a workable timeframe for SEFPP, SEFPP should offer MOHP to sponsor and assist the MOHP in conducting a national dissemination workshop and a series of smaller forums to provide FP stakeholders with anticipated updated household-level health and FP data. Activity Objective 2: Support the quality of FP/RH services SERVICE DELIVERY: CONTRACEPTIVE SECURITY Background and Context Assisting the FP sector in achieving contraceptive security (CS) is a priority area for SEFPP because increased demand for FP services will not yield results unless contraceptive methods are available and accessible. SEFPP has agreed with the MOHP to focus its CS assistance around three interventions: quantification, supply chain strengthening (specifically warehouse assessments), and introducing new FP methods, with a view toward expanding the contraceptive options available. SEFPP’s documents articulate its additional understanding that the private sector must play a pivotal role and that the method mix must be enhanced with an increased focus on long-acting reversible contraception 20 Ibid. USAID.GOV MIDTERM EVALUATION OF STRENGTHENING EGYPT’S FAMILY PLANNING PROGRAM | 18 (LARC), the existing coordination mechanisms must be revitalized, data collection and quantification must be strengthened, and a supply and resource mobilization plan must be developed, with clear identification of the roles and responsibilities of the MOHP/FP Sector. Achievements In introducing new FP methods, SEFPP has provided assistance to MOHP to define the criteria for selecting newly added FP methods in the public sector. The MOHP selected three methods to be provided in the public sector, namely multi-load IUDs, hormonal IUDs, and Sayana Press, noting that all three of the methods have long been available in the private sector. In Year 4, SEFPP plans to provide technical assistance to the MOHP/FP Sector to finalize the selection of areas to pilot new methods based on community characteristics. Upon the MOHP/FP Sector’s request, SEFPP would then support disseminating the new method through a nationwide scale-up of the newly selected successful FP methods. It was noted, however, that, as in the past, SEFPP’s access to data describing community characteristics may be problematic. Further, a Year 2 SEFPP assessment found that the MOHP’s capacity to test new methods needed strengthening. On quantification, SEFPP has plans to train various cadres in quantification and forecasting principles and methods. To support such training, the MOHP shared consumption data and service data for 2017/2018. However, data on “stock on hand” was not shared with SEFPP, without which SEFPP could not help MOHP develop its supply plans or total commodity requirements and costs estimated during the quantification activity. Despite this, to date, SEFPP was able to assist MOHP to: • Develop forecasts for seven contraceptives included in MOHP’s quantification plan. • Provide refresher training for MOHP’s quantification team on “1- how to estimate total commodity requirements and costs, 2- develop supply plans, and 3- monitor the upstream pipeline and take appropriate action to ensure a continuous supply of contraceptives.” • SEFPP worked with the quantification team to build their capacity to use these Excel-based tools to create new forecasts for future use and update existing forecasts as more comprehensive data become available. It is noted, however, that aside from training GOE staff (MOHP, NPC, MOF, and other in-country stakeholders) in quantification principles, and refresher training for the MOHP’s quantification team, SEFPP does not have a role in development or strengthening of data collection and quantifications systems. Because SEFPP is not working with the MOHP on HMIS/LMIS systems upgrades, does not have access to contemporary LMIS data (such as stock-on-hand), and is excluded from the resource planning aspect of quantification (by agreement with the MOHP21), SEFPP may have a problem in that the relevance of SEFPP’s training and capacity building efforts may not be clear. SEFPP does plan to re￾engage with the MOHP to reassess a possible role for SEFPP to assist the MOHP’s HMIS/LMIS upgrades (now in the design phase) and to further discuss with MOHP possible collaboration with NPC on its SEFPP-related automated information systems upgrades. In the meantime, however, SEFPP is not clear about the structures, processes, and roles and accountability of the different players and stakeholders related to CS in general and quantification specifically. One key informant said that SEFPP’s strategic role in the development of quantification capacity in the FP sector was technically unclear: “In terms of capacity building on CS quantification, planning, and implementation in the MOHP, NPC, and MOF, the Project has not been able to move freely because: (1) the jurisdictional issue is still up in the air with UNFPA (about CS Strategy being its exclusive domain), (2) the NPC doesn’t want to be involved with areas such as quantification and financial sourcing because it doesn’t see this as its domain, and (3) the MOHP doesn’t see any need for the MOF to get involved with TA[technical assistance]/training in CS 21 See Service Delivery: Contraceptive Security section of this report, page 18. 19 | MIDTERM EVALUATION OF STRENGTHENING EGYPT’S FAMILY PLANNING PROGRAM USAID.GOV because it says it has already communicated its long-term and immediate needs to the MOF, and the MOF has responded satisfactorily on the budget allocations requested.” Key Findings ● The MOHP determined that SEFPP’s early CS assessment and assistance activities fell outside the intended mandate of SEFPP. The MOHP instead requested that SEFPP redirect its efforts to concentrate on quantification, warehouse assessments, and introducing new FP methods. ● SEFPP developed manuals and plans to train MOHP staff in conducting warehouse assessments. ● SEFPP reports that the MOHP’s capacity to conduct testing and authorize new contraceptive methods needs to be strengthened, but SEFPP is not involved in strengthening these product testing systems and has, up to now, been confined to identifying sourcing of one contraceptive (Silver Nova T) for testing as a potential public sector method, noting that Silver Nova T has been available and in use in the private sector for the past ten years. ● SEFPP’s Annual Work Plan describes an approach to supporting contraceptive quantification that mainly focuses on training various FP planners and managers on quantification principles and exercises. SEFPP assistance does not include any involvement in the development of an operational MOHP quantification system tied to a unified LMIS (which would be needed for a functional national contraceptive quantification/forecasting and supply chain system). Conclusions ● Since the MOHP requested that SEFPP disengage from LMIS development, it is not clear whether or how SEFPP can support critical components of a CS strategy, such as measurement, that engage the information systems that form the basis of forecasting and quantification. ● Future warehouse rehabilitation (and financial resources) might result from the warehouse assessments SEFPP is assisting the MOHP in conducting. Recommendations ● SEFPP should reassess the current environment and reopen dialogue with the MOHP to explore the possibility that its designated contributions (quantification, warehouse assessments, and introduction of new FP methods) might extend to make SEFPP a more vital and recognized partner in the CS strategy arena. An example might be for SEFPP to assist the MOHP in strengthening the MOHP system involved with testing new FP methods. ● SEFPP should have a less marginal and more clearly spelled out role in strengthening the MOHP’s quantification system and capacity. SEFPP should review its current assistance to determine whether it might offer to play a more technical and impactful developmental role (other than training) in assisting the MOHP with strengthening its contraceptive quantification system. An example might be for SEFPP to assist the MOHP in establishing a quantification unit and train technical staff in forecasting and quantification. SERVICE DELIVERY: SUPPLY-SIDE INTERVENTIONS Background and Context Over the past three years, SEFPP has supported two MOHP initiatives to expand services. First, task sharing is a strategy for expanding service delivery by broadening the scope of services that nurses can officially provide to potential family planning users. SEFPP has been assisting the MOHP in training task￾sharing nurses and the rollout of this program. Task-sharing is still considered a pilot phase because some MOHP officials believe that task-sharing will eventually require approval by the Nursing Council and vetting of job descriptions by Parliament. SEFPP cannot know with certainty that the training and application of new FP skills is making a difference in making FP services available, as it is not permitted by the MOHP to measure outcomes at the service delivery level. However, there is anecdotal evidence that the task-sharing training is paying dividends. A nurse from Qena who received the task-sharing training said after she passed the USAID.GOV MIDTERM EVALUATION OF STRENGTHENING EGYPT’S FAMILY PLANNING PROGRAM | 20 training post-test that she liked the content of the training and quality of knowledge provided by training and also commented positively on the skills of trainers. She also said that in January 2020, she started her new role providing injectables and pills to FP clients in PHCUs that lacked physicians. The nurse believes that providing these services improves access to FP services for FP clients, as this avoids losing them during referral to another PHCU. The nurse received follow-up visits from SEFPP, and MOHP physicians believe that nurses’ task-sharing could promise to improve access to FP services (according to eight service provider KIIs). According to one nurse in Qena, “We have a lot of supplies, and nothing can stop me from task sharing. I liked the task sharing very much. It is good practice, for me and the user as well as to benefit from my knowledge as now I know how to make sure the method won’t cause any complications for the women which is safe for the user. Task sharing is a good practice and easier for users as now they don’t need to go to doctors in far areas. Now support is closer to users, which increases the new users in the remote area.” The practice of postpartum and post-abortion insertion of Intrauterine Devices (PP/PA/IUD) is the second important strategy promoted by the MOHP to increase service availability. This was the first intervention that the MOHP moved to scale up nationally with SEFPP assistance. Here, SEFPP supports training implemented in an integrated fashion involving the MOHP’s FP, Curative, and Primary Health sectors. A physician (a Key Informant to the evaluation) in Alexandria who received the PP/PA/IUD training liked the training content and found it to be an excellent training program. She mentioned that providing task-sharing training to nurses and physicians in the same hospital was a great step to promote services by nurses to FP methods users and provide quality services by physicians, as they complement each other. The physician in Alexandria mentioned that the training resulted in introducing a new service in Dar Ismail Hospital and increased the number of IUDs inserted. The physician also stated that they insert around 80 IUDs monthly, but she believes there is room for improvement. Service providers who received the training found it to be a promising approach to increasing women’s access to FP services (eight service provider KIIs). Key Findings ● SEFPP developed the task-sharing curriculum, and 41 nurses received the task-sharing training in six governorates to start providing pills and injectables. SEFPP is following the MOHP’s lead on timing the full-scale rollout of the program. ● On task-sharing, SEFPP is constrained by MOHP restrictions from measuring progress at the service delivery supervisory level. SEFPP collects numbers at the output level, but SEFPP does not measure outcomes or qualitative studies with clients or beneficiaries. ● SEFPP indirectly monitors results using data reported to SEFPP in the PP/PA/IUD program at the facility level, with cooperation from the MOHP OB/GYN Department. Conclusions ● SEFPP has significantly assisted the MOHP in launching the task-sharing and PP/PA/IUD approaches, and national rollout of the PP/PA/IUD approach has begun. ● SEFPP has strategically important outcome data for the PP/PA/ID program that could be helpful in assisting the MOHP with planning a national rollout. ● On task-sharing training, SEFPP may have an opportunity (other than direct data collection) to engage the MOHP in gathering strategic information and evidence-based planning to scale up these activities. Recommendation In conjunction with scaling up task-sharing and the PP/PA/IUD program in the upper and lower governorates (as requested by the MOHP), SEFPP should leverage its substantial support for these interventions to encourage the MOHP to set up its own “qualitative research” protocols to measure outcomes and plan rollouts informed by evidence. 21 | MIDTERM EVALUATION OF STRENGTHENING EGYPT’S FAMILY PLANNING PROGRAM USAID.GOV Activity Objective 3: Enhance Selected Health Behaviors OPERATIONAL PARTNERSHIPS Background and Context SEFPP has actively pursued a strategy of working with private sector companies, organizations, and companies to promote FP acceptance through awareness building and peer education among the populations they work with and serve. JSI selected the international Population Council (PC), a U.S.- based population program development agency, as a consortium partner to work with SEFPP because of PC’s long involvement with Egypt’s FP program, particularly private sector developments. Of note, as far back as 2016, PC published a report on challenges and recommendations for engagement of private sector entities in Egypt in the national FP response. Achievements PC has helped SEFPP pursue an estimated 10–12 MOUs22 with private sector workplaces that have proven to be highly successful in engaging their workers in ancillary family planning messaging and communications. SEFPP developed the MOUs and PC then helped with conducting training for peer educators used in implementation of activities. As an outgrowth of its MOU with Marie Louis Factory to conduct awareness sessions targeting female factory workers, SEFPP conducted TOT courses for selected factory workers to create a cadre of trainers in the factories, who were then able to conduct FP orientation sessions with other workers and promote FP awareness activities with their families, friends, and neighbors. The activity has also established a working relationship with the American University in Cairo (AUC), focusing on engaging youth populations to build awareness of FP/RH issues and practices. Youth have proven (through AUC efforts) to be effective in peer-to-peer education and use of social media, and AUC is already thinking of engaging still younger participants (younger than 18 years of age) in the program in activities that would involve young school-age adults, by working with the Ministry of Education. Implementation of operational partnerships initiated by SEFPP is still formative but is part of a strategy for strengthening private sector engagement in FP programs. The partnerships forged to date demonstrate that these partnerships can enable SEFPP to extend the reach of FP advocacy and messaging indirectly. In turn, partnering with SEFPP enhances the agency of partners to serve as FP advocates and messengers. In general, these alliances also demonstrate the advantages of collaborating with the private sector to FP policymakers. Key Findings ● In partnership with PC, SEFPP focused on working with young, married girls and women who work in factories. These activities turned out to be popular. They introduced peer-to-peer approaches that allowed the factories to reach more girls and women in the community with FP messaging. ● AUC’s approach to engaging youth in family planning messaging has attracted many young people to the program, and the AUC youth engagement activity appears to be productive. ● More than one informant to this evaluation cautioned that the Ministry of Education has a reputation for being conservative on reproductive health matters and that it may be difficult for SEFPP to establish productive working relationships with this agency on a youth sex education program within the activity’s remaining timeframe. 22 Estimates vary because some MOUs are dormant because of COVID-19 restrictions, and others are at various stages of implementation. USAID.GOV MIDTERM EVALUATION OF STRENGTHENING EGYPT’S FAMILY PLANNING PROGRAM | 22 Conclusions ● Factory-based programs are an effective strategy for reaching individuals and communities with FP advocacy and messaging. ● AUC’s approach is less than one year old and should be further documented and assessed. ● Expansion of AUC’s activities to younger age groups at this time requires a feasibility assessment to determine the approach and timing needed. Recommendations ● SEFPP should conduct a joint workshop to draw lessons learned and adapt them to improve the current approach and tools for improving its current university-level activities. ● SEFPP should undertake a feasibility assessment to determine the approach and timing of introducing FP messaging to younger school-aged groups and working with ministries such as the Ministry of Education. ● SEFPP should continue and expand its highly productive and popular factory-based programs in continued partnership with the international Population Council . Consistent with suggestions from participating companies, awareness sessions should be continued, but more emphasis should be placed on peer education approaches. SEFPP should develop new outcome-oriented indicators and require implementing partners for both the youth engagement activities and the factory-based programs to measure and report on activity outcomes in addition to outputs, (noting that outcomes go beyond outputs to describe what was achieved as a consequence of what was done). SOCIAL AND BEHAVIOR CHANGE COMMUNICATION (SBCC) Background and Context In 2015, NPC led the development of a 15-year national population strategy, with active participation from all the relevant ministries. After strategy development, a 5-year strategic plan was developed in a participatory way with the involvement of an inter-ministerial standing committee. Achieving the strategic objectives of the national population strategy requires behavioral changes among specific population segments (regarding female genital mutilation, short inter-pregnancy spacing, gender-based violence, high fertility, etc.). The achievements of SEFPP’s SBCC Unit in implementing several SBCC interventions within the framework of the National Population Strategy were mixed due to the MOHP/FP’s concerns about the participation of external actors such as SEFPP in community-level activities. For instance, in Years 1 and 2, SEFPP provided FP training for media officers in targeted governorates, who were then expected to develop implementation plans for health education sessions, including health messages targeting women as part of their regular work (as reported in the Year 3 Annual Report). However, these plans were abandoned after about six months when the MOHP prevented any data collection at the community level. Also, in Year 3, SEFPP conducted roundtable discussions for community awareness teams comprised of media officers, community health worker supervisors, women’s health club supervisors, and nurse supervisors and introduced an approach for behavioral mapping of women and men of reproductive age to better understand behaviors and shape health education messages integrated into the Unified Health Education Reference Manual (also developed with SEFPP technical assistance). Once finalized, SEFPP tested the Manual with the Community Awareness Team (CAT) with plans for it to be distributed and used by CATs. However, similar to plans for implementation of mapping exercises, distribution of the Manual was initially held in abeyance over concerns that the Manual would be used to collect sensitive community-level data. It Fortunately the MOHP has recently cleared the Manual for distribution and use by CATs. 23 | MIDTERM EVALUATION OF STRENGTHENING EGYPT’S FAMILY PLANNING PROGRAM USAID.GOV Further, in Year 3, SEFPP implemented workshops with 126 media officers specifically keyed to postpartum contraception (PPC) in an effort to advocate for LARC. SEFPP’s assistance to the MOHP in conjunction with the OB/GYN Department is ongoing. Service providers’ perceptions regarding PPC and the related training they have received from SEFPP were generally positive, prompting the MOHP’s decision to undertake an evidence-based national scale up of the approach. SEFPP’s Year 4 Work Plan expresses the intention to again work with the MOHP/FP Sector to conduct mapping exercises on behavioral barriers to FP uptake and with service providers on barriers to effective FP delivery (including LARC) and to undertake “innovative” and diverse communication/outreach channels, engage media practitioners, and develop materials and training with “interpersonal influencers.” This suggests that the MOHP/FP Sector anticipates some easement of the non-permissive environment for involvement in sensitive community-level interactions that SEFP has experienced in the past. , Furthermore, in the coming year, SEFPP plans to collaborate with NPC to leverage SEFPP’s extensive technical background in SBCC to develop a national population strategy. NPC will take the lead in coordinating among partners to develop an evidence-based National Population SBCC Strategy. The goal is to develop and launch a 10-year National Population SBCC Strategy that supports achieving the 2015–2030 National Population strategic objectives and the new GOE/MOHP Egyptian Family Development Project, 2021-2023 by NPC conducting a scientifically sound behavioral segmentation analysis and coordinating between the relevant ministries to develop an evidence-based participatory SBCC strategy. Key Findings SBCC experienced a challenge in working with the MOHP’s Health Education and Communication Unit’s (HECU) district Media Officers. An advocacy tool to be used by Media Officers was developed, but the MOHP stopped the activity because it was seen as a tool for monitoring community behavior (which could also be interpreted as program “research”). SEFPP’s AY4 work plan again calls for assistance to the MOHP in such areas as mapping and working through media professionals, although these interventions have met with objections at MOPH policy levels. Among other assistance, SEFPP supported NPC in enhancing its online presence in social media. NPC’s visual identity in social media platforms was developed and approved by NPC, and SEFPP developed a training plan to build the capacity of NPC staff on social media management, writing of media releases, and how to upload them on the different platforms. Conclusions ● SEFPP has undertaken several demand-side interventions intended to align with demand creation strategies in the MOHP. Achievements have been mixed; most were problematic because they were seen by MOHP policymakers as being too close to community behavioral assessments. ● SEFPP’s assistance to NPC on social media management was productive. Recommendations SEFPP’s plan to assist NPC in developing and implementing a national SBCC strategy should be the main focus in the coming year to enable SEFPP to develop a more strategically effective SBCC vision and interventions for the future. USAID.GOV MIDTERM EVALUATION OF STRENGTHENING EGYPT’S FAMILY PLANNING PROGRAM | 24 • SEFPP should review and be very cautious about Year 4 plans for SBCC interventions through the MOHP/FP Sector until SEFPP can resolve previous objections expressed by MOHP/FP to SEFPP SBCC approaches and the pending development of NPC’s national SBCC strategy, which may provide directions for SEFPP SBCC programming. FINDINGS, CONCLUSIONS, AND RECOMMENDATIONS—TRAINING AND CAPACITY BUILDING EQ3: SERVICE LEVEL To date, in what ways did SEFPP strengthen capacity and commitment in the local systems (health workforce, health information system, health service delivery) to continue delivering consistent levels of access to and delivery of quality FP services? SEFPP’s basic approach to strengthening capacity and commitment in local systems has been to carry out a wide range of training events that enable capacity building in many SEFPP activity components, as discussed throughout this evaluation.23 SEFPP has engaged in the promotion of training methodologies, as well as conducting training events where knowledge and skills are imparted. TRAINING AND CAPACITY BUILDING APPROACHES Achievements Although it was a small sample, the physicians and nurses who received TOT agreed that it improved their skills in conducting training and helped them do on-the-job training more efficiently, especially for the participants working in supervisory roles. Respondents who received TOT said that it was their first time receiving such training, as they are used to only technical training (eight service provider KIIs). One key informant from Cairo said, “My communication skills improved. The way that I manage my team, my negotiation skills with clients improved, which help the client to take the decision of what method she wants to use.” SEFPP is engaged in two other interventions intended to improve opportunities for training. SEFPP is actively engaged with the MOHP in establishing an e-learning platform that will allow selected FP training modules developed by SEFPP to be made available electronically. When it is up and running, SEFPP also has plans to assist the MOHP in utilizing information from those registered in e-learning to identify high performers for TOT. In addition, SEFPP has developed a “Qualifications/Competency Tracking System” that allows the MOHP to identify low performers (as assessed through SEFPP FP training pre/post-tests) to target for additional training. Although the tracking system does not serve as a robust competency assessment tool, SEFPP staff report that it serves the limited purpose of identifying targeted training candidates. Key Findings ● Interviewed stakeholders (service providers, MOH, private sector) agreed on the effectiveness of the TOT component in terms of improving skills for conducting training, with increased sustainability potential. ● SEFPP has agreed with the MOHP to use a “cloud-based” internet platform for the MOHP e-learning materials developed by SEFPP to provide easy public access for all users. ● SEFPP has spent considerable time and effort developing a Qualifications/Competency Tracking System that addresses the needs of SEFPP and the MOHP to identify candidates for refresher training and training upgrades (such as TOT) and has plans to continue this effort in its Year 4 Work Plan. 23 Various SEFPP training and capacity building activities can be found in SEFPP’s Strategic Information component as well as Service Delivery – Contraceptive Security, Service Delivery – Supply-side Interventions, and Social and Behavior Change Communications. 25 | MIDTERM EVALUATION OF STRENGTHENING EGYPT’S FAMILY PLANNING PROGRAM USAID.GOV Conclusions ● TOT is an effective strategy for increasing the sustainability of capacity building in the FP program. MOHP will benefit from SEFPP’s support in this area. SEFPP has a clear plan for follow-up and monitoring, as well as for replication and scaling up. ● E-Learning has strong potential as a mechanism for increasing access to SEFPP’s informational and training resources. ● The qualification tracking system has been a helpful tool in identifying candidates for refresher training and identifying TOT candidates. Recommendations ● SEFPP should continue to advocate for and seek opportunities in the MOHP or private sector to introduce TOT approaches into training and capacity-building activities. ● SEFPP should continue with assistance to the MOHP in developing its e-learning programs but closely monitor these efforts to ensure that no further technical difficulties arise. ● SEFPP should explore with the MOHP how the tracking system could be made more robust in measuring competencies. TRAINING BENEFICIARIES: SERVICE PROVIDERS Background and Context As reported in annual reports and its MEL system, SEFPP trained, through its different capacity-building activities in the last three years, around 696 service providers, including different cadres of physicians and nurses. SEFPP also conducted four basic FP training workshops with 119 private pharmacists (Year 3 Annual Report). A total of 362 service providers responded to the evaluation phone survey. Over half of the sample (50.8 percent) were physicians, while 28.2 percent were nurses and 21 percent were pharmacists (see Figure 1 below). A quarter (25.4 percent) were from Assuit, 21.5 percent were from Gena, and the rest were from Cairo, Luxor, and Alexandria (service provider questionnaire, n=362). Figure 1. Distribution of Service Providers by Occupation All service providers agreed that the training was relevant to their job description, and most reported sharing their acquired knowledge and skills with their colleagues. Figure 2 below shows that a total of 336 service provider respondents (91 percent of the nurses, 77.6 percent of physicians, and 53.5 percent of pharmacists) confirmed (through agreeing or strongly agreeing) that the training received was relevant to their work responsibilities (service provider questionnaire, n=336). Pharmacist 21.0% Physician 50.8% Nurse 28.2% USAID.GOV MIDTERM EVALUATION OF STRENGTHENING EGYPT’S FAMILY PLANNING PROGRAM | 26 Figure 2. Service Providers: “I am able to apply the learning I acquired in my work” (n=336) Other selected findings from the phone survey are listed below. Key Findings ● All the service delivery training recipients liked the training content and rated it as very good or excellent. ● SEFPP is restricted from setting up the types of protocols that would allow it to measure or document training results. Still, there is a general perception among providers who received some form of SEFPP FP training that they are better prepared to respond to increasing demand for services. ● A total of 94.9 percent of service providers agreed or strongly agreed that the training content provided them with sufficient information and skills. ● Three-quarters (76.5 percent) of the total service provider respondents said that they can use the learning they gained in their work. ● Pharmacists, particularly those working in private pharmacies, believed that the training helped them provide better communication to women about FP methods but only 53.3 percent of pharmacists surveyed believed they could apply what they learned to their work. Recommendations ● SEFPP should continue its overall training and capacity-building activities while using its existing database to conduct training needs assessments to better inform its strategic choices in the remainder of the activity. ● SEFPP should conduct more training targeted to private sector pharmacists. Aside from needs identified by the trainees themselves (such as the need for additional supplies and media tools), a systematic training needs assessment should be conducted to discover ways of making the training curriculum more relevant to pharmacists’ work requirements. TRAINING BENEFICIARIES: NON-SERVICE PROVIDERS Background and Context Non-service providers are defined as professionals who work in companies or agencies that do not themselves provide family planning services or commodities but come in regular contact with potential 26.8% 12.1% 4.0% 19.7% 9.7% 5.0% 53.5% 77.6% 91.0% Pharmacists Physicians Nurses Disagree/Strongly disagree Neutral Agree/Strongly agree 27 | MIDTERM EVALUATION OF STRENGTHENING EGYPT’S FAMILY PLANNING PROGRAM USAID.GOV family planning clientele and could create public awareness and demand for services. This cadre includes, for example, media officers, media professionals, HMIS staff, peer educators, NGO volunteers, and NPC administrative staff. A total of 237 non-service providers responded to the midterm evaluation survey. Over half of the respondents (52.3 percent) were from Cairo (see Figure 3 below). Figure 3. Distribution of non-service providers by work governorates The rest of the respondents represent Qena, Alexandria, Luxor, and Assuit (non-service provider survey, n=237). As shown in Figure 4 below, a total of 91.8 percent of non-service provider respondents confirmed (through agreeing or strongly agreeing) that the received capacity building was relevant to their work responsibilities (non-service provider survey, n=219). Figure 4. Non-service Providers: “I am able to apply the learning I acquired in my work” (n=219) Challenges Although all training was generally well received by trainees, there were some notable challenges. For instance, activities of the MOHP Media Officers who received training and began use of interactive social behavior tools (such as the use of theatre to air public opinions and information) were curtailed Cairo 52.3% Alexandria 15.2% Assiut 11.0% Qena 17.3% Luxor 3.8% 18.2% 15.2% 16.0% 9.8% 0.0% 10.0% 0.0% 0.0% 2.4% 11.1% 71.8% 81.8% 84.0% 82.9% 88.9% Cairo Alexandria Assiut Qena Luxor Strongly Disagree/Disagree Neutral Strongly Agree/Agree USAID.GOV MIDTERM EVALUATION OF STRENGTHENING EGYPT’S FAMILY PLANNING PROGRAM | 28 after about six months because of MOHP concerns that the approach was too closely related to community-level research. Said one media officer from Assuit, ”We stopped working with this tool after six months of implementation, we go back to awareness sessions, but we try to do some work but not like before. Dec 2018–June 2019. I got instructions from MOHP to stop work using this tool. I wish this tool would return again. Women liked it a lot.” One journalist who worked for a publication observed that he did feel more equipped to speak about FP misconceptions and that the training he received was too short. He needed frequent updates from FP practitioners, and he was struggling with not having a sustainable mechanism for publishing FP messages. There were other cautionary notes about the non-service provider training: a SEFPP initiative with training of Community Health Workers was curtailed because it was ultimately determined that many other actors in the FP arena were similarly engaged and that SEFPP intervention in this area was not needed24. Despite these challenges, in the evaluation’s phone survey, over half of the total non-service provider respondents (55.7 percent) indicated that SEFPP the activity contributed to an increase in demand for FP services. Forty-one percent of Cairo non-service providers shared a belief that SEFPP contributed to an increase in FP service demand, compared to 66.7 percent from Alexandria, 76 percent from Assuit, 73.2 percent from Qena, and 66.7 percent from Luxor (non-service provider survey, n=219). Key Findings ● Although the training and capacity building provided by SEFPP proved to strengthen recipients’ capacities, all training provided to non-service providers was incomplete, either because of the short length of the training provided, a lack of follow-up, or instruction from the MOHP to stop interventions. This possibly reduced the overall effectiveness of the training. ● SEFPP-trained MOHP media officers were expected to develop implementation plans for health education sessions, including the health messages targeting women, as part of their regular work. ● SEFPP completed a series of workshops for media engagement and communication on FP issues that were attended by media professionals, MOHP/FP Sector team members, NPC representatives, and people from the MOHP Spokesperson’s Office. These capacity-building workshops were intended to improve attendees’ report writing skills and help them present their work to the media in a way that could influence the community. ● A total of 91.8 percent of non-service provider respondents confirmed (through agreeing or strongly agreeing) that the received capacity building was relevant to their work responsibilities. ● Further, 91.4 percent of non-service provider respondents agreed or strongly agreed that the training content provided them with sufficient information and skills. Conclusions ● SEFPP training for non-service providers has good potential to make contributions to strengthening the family planning sector by helping these non-service providers to improve public messaging and communication with potential family planning users. ● Synergies between service providers and non-service providers across governorates could enable knowledge exchange and promotion of technical messages in an attractive, simple format that allows broad outreach and faster adoption of users. ● The factory worker TOT approach proved beneficial and has adequate reach and positive results in promoting FP information among relevant targets. Follow-up, monitoring, replication, and scaling up of the intervention are not yet clear. 24 SEFPP will, however, train community health worker supervisors. 29 | MIDTERM EVALUATION OF STRENGTHENING EGYPT’S FAMILY PLANNING PROGRAM USAID.GOV Recommendations ● SEFPP should strengthen all of its private sector non-service provider training and capacity building by using a more structured format and longer-term follow-up to ensure the scalability and sustainability of training. ● SEFPP should consider forming a committee of doctors, media professionals, and journalists to meet regularly and create content, which may provide a model for social behavior promotion through online and offline channels that would sustain itself beyond the activity’s end. ● SEFPP should consider arranging interactions, discussions, and mini workshops among service providers/clinicians and media professionals to facilitate knowledge exchange and promotion of public health communications and FP education and messaging. FINDINGS, CONCLUSIONS, AND RECOMMENDATIONS—CROSS-CUTTING ISSUES COVID-19 Overview According to statistics reported by the U.S. Embassy in Cairo, as of mid-March 2021, there were 192,840 confirmed cases of COVID-19 and 11,431 deaths in Egypt, as acknowledged by the Ministry of Health and Population. Egypt implements 14-day quarantine periods and other preventative measures in all cases for people who test positive for COVID-19 while in Egypt. Intensive care unit occupancy rates remain near 90 percent. The general mobility of people is restricted, and everyone is required to exercise mitigation measures. Mass gatherings are canceled indefinitely. Government offices are closed to normal business, and communications and administrative activities are primarily conducted online or through virtual media. All of the above shape the restricted working environment in which SEFPP has operated since the COVID-19 pandemic began in early 2021. Key Findings ● Areas of SEFPP COVID-19 response included adding materials to online training modules, assisting NPC in developing a research paper on the impact of COVID-19 on FP services and population control (which was reviewed by USAID), and conducting virtual training and events. ● In addition, SEFPP supplemented MOHP needs by providing masks and infection control supplies in quarantine hospitals and distributed to Raedat Refiyats (RRs), which was appreciated by the MOHP, RRs, and media officers. ● Due to limited supplies and budget, the availability of COVID-19 supplies through SEFPP was not widely advertised and was directed to specific recipients by MOHP. It was also noted that only a few service and non-service providers were aware that SEFPP was a source for these supplies. Aside from the availability of limited COVID-19 supplies through SEFPP, none of the service or non-service providers were aware of any other SEFPP interventions related to COVID-19. Conclusion SEFPP adapted quickly to the challenges of COVID-19 by supplementing MOHP needs for infection control supplies and moving to virtual and online formats for training and assisting the NPC in research and improving public awareness of COVID-19’s impact. Recommendations ● In the interest of continuity of FP services, and based on MOHP requests and the availability of budgetary resources, SEFPP should continue or increase the provision of infection control supplies to more beneficiaries (service and non-service providers). USAID.GOV MIDTERM EVALUATION OF STRENGTHENING EGYPT’S FAMILY PLANNING PROGRAM | 30 ● SEFPP should continue and expand the use of virtual and online communications strategies so long as the COVID-19 threat and mitigation activities are required. GENDER Overview As stated in SEFPP’s Year 4 Work Plan, early in SEFPP’s implementation, the activity tailored its gender mainstreaming strategy around three strategic objectives: (1) equal opportunities for men, women, and youth in accessing FP/RH services, including counseling; (2) effective and sustainable gender equality within SEFPP and MOHP through engagement with the private sector and community institutions; and (3) evidence-based, gender-sensitive planning and gender-disaggregated data use for decision making and policy formulation, including increasing CS for women and men. In reality, and by its own assessment, SEFPP has not fully implemented many of these laudable objectives. The relatively robust MEL system now being implemented by SEFPP, including the “Click Up” activity management tool and the activities of SEFPP’s Implementation Unit, provides a platform and process around which SEFPP can make substantial progress on strengthening gender-related programming. Key Findings • SEFPP applied a systematic gender and development approach to the formulation of a SEFPP gender strategy that involves investigating gender situations, cultural norms, and power structures and identifies interventions geared to both women and men. • SEFPP routinely introduces the importance of gender, compliance, and women’s rights as a default presentation before all training programs. • However, SEFFP management does not feel that gender issues have received sufficient attention in SEFPP program management over the past three years and expressed the intention to give more concentrated attention to monitoring the implementation of gender programming in Year 4. Conclusion SEFPP originally put considerable effort into developing a comprehensive gender strategy and has implemented many aspects of the strategy but, by its own estimates, did not consistently ensure that all aspects of its strategy are culturally responsive and well-integrated and implemented in its activities as intended. Recommendations • SEFPP should complete its Year 4 plans to strengthen the implementation of its gender strategy, including designating a gender focal point that will organize orientation meetings for SEFPP staff on basic gender equality concepts and applications. • SEFPP should also develop and upgrade its gender mainstreaming guidelines to be used by staff in its tools, training materials, and manuals. SUSTAINABILITY Overview This evaluation defines sustainable activities as those that (1) address a specific need, (2) have internal momentum, and (3) have adequate resources (directly or indirectly) to continue into the future. SEFPP’s understanding of sustainability appears to be consistent with this definition and is articulated throughout its documentation and evident in many of the activities it has undertaken during the past three years. Sustainability is seen as a long-term goal of the activity that SEFPP believes will be achieved through the combined and interactive effects of interventions implemented throughout the program’s 31 | MIDTERM EVALUATION OF STRENGTHENING EGYPT’S FAMILY PLANNING PROGRAM USAID.GOV life. The combined effects of SEFPP’s interventions aim to ensure ongoing impact long after the program ends. Key Findings ● Alignment of SEFPP’s interventions with MOHP FP Sector plans and priorities enhanced the potential sustainability of those activities. ● The TOT approach provided MOHP with cadres to address the shortage of trainers at the directorate level. It is expected that these trainers will continue to transfer knowledge beyond the project’s lifetime. ● Fostering the establishment of new functional relationships between the GOE (MOHP and NPC) and FP partners (such as AUC) is likely to result in cooperation that will endure beyond the life of the activity. ● The NPC has demonstrated a high sustainability potential in utilizing the trained cadres in further research (such as through partnership with UNICEF). ● Similarly, nurses trained on task sharing are reported to be currently applying what they learned in training and are expected to continue these practices after the project ends. Conclusion SEFPP has proven its ability to promote and implement sustainable strategies and activities. Recommendations ● SEFPP should review and identify interventions that were productive and have the best potential to sustain themselves in the future. According to the findings of this evaluation, consideration might be given, at a minimum, to interventions in the following areas: ● SEFPP investments in managerial and operational capacity building (such as systems developments that build institutional commitment); ● SEFPP investments in training FP influencers (i.e., peer educators); ● SEFPP investments in training workers to train others (e.g., TOT); and ● SEFPP investments in forging potentially sustainable partnerships (such as between the public and private sectors). ● SEFPP should continue or introduce measurements of results for interventions it has identified as potentially sustainable. Work Plans, targets, and quarterly monitoring should be continued or incorporated into SEFPP’s MELP. USAID.GOV MIDTERM EVALUATION OF STRENGTHENING EGYPT’S FAMILY PLANNING PROGRAM | 32 SELECTED RECOMMENDATIONS RECAPPED This evaluation developed 31 Recommendations organized by 12 thematic focal areas. While all recommendations are important for SEFPP to note and act upon, some recommendations stand out as essential to the activity’s near-term achievements and long-term sustainability. These essential recommendations are recapitulated in Table 3 below. Table 3. Selected Recommendations and Actions SELECTED RECOMMENDATIONS SUGGESTED SEFPP ACTIONS Design, Theory of Change, and Sustainability 1. SEFPP leadership should re-engage the MOHP in dialogue to improve the overall working environment for data capture and access and in the generation and use of strategic data/information to inform evidence-based planning and decision making. ● Conduct internal strategy sessions ● Initiate meetings and discussions with MOHP officials Design, Theory of Change, and Sustainability 2. SEFPP should identify strategic planning “champions” in the MOHP and NPC. As opportunities arise, sponsor these champions to participate in regional or international forums that promote evidence-based planning and decision making. Examples might be relevant WHO conferences or a technical workshop on data for contraceptive forecasting. ● Identify candidates ● Cultivate dialogue and working relationships ● Identify events ● Plan and manage participation in events Design, Theory of Change, and Sustainability 3. USAID/Egypt should explore (and implement, if indicated) a strategy to support and reinforce JSI/SEFPP efforts to undertake sustainable evidenced-based planning (such as being able to conduct service delivery and client-based data and information), and behavior change communications (such as community-based data gathering). This strategy might include eliciting Mission and Embassy involvement at the dialogue level with GOE counterparts. ● [Specific to USAID internally] Strategic Information 4. In the near term, SEFPP should facilitate dialogue between MOHP and NPC on how HMIS/LMIS systems developments at NPC (assisted by SEFPP) can benefit MOHP efforts to upgrade its HMIS/LMIS and how both could work compatibly. ● Identify relevant offices and individuals in MOHP and NPC ● Facilitate initial meeting(s) Strategic Information 5. In the longer term, in addition to theoretical training on strategic information systems and its uses, SEFPP should re- engage with the MOHP on the possibilities of direct SEFPP assistance with strategic information systems upgrades (such as with LMIS) and related evidence-based planning. ● Conduct internal strategy sessions ● Initiate meetings and discussions with MOHP officials Operational Partnerships 6. SEFPP should develop new indicators and require implementing partners for both the youth engagement activities and the factory-based programs to measure and report on activity outcomes and results (in addition to outputs). ● Review/modify indicators as necessary ● Introduce any new indicators and measurement procedures to implementers Service Delivery – Contraceptive Security 7. SEFPP should have a less marginal and more clearly spelled out role in strengthening the MOHP’s quantification system and capacity. SEFPP should review its current assistance to determine whether it might offer to play a more technical and impactful developmental role (other than training) in assisting the MOHP with strengthening its contraceptive ● Engage a Contraceptive Logistics Expert to review SEFPP’s current SOW and recommend ways to strengthen it ● Redefine a proposed SOW 33 | MIDTERM EVALUATION OF STRENGTHENING EGYPT’S FAMILY PLANNING PROGRAM USAID.GOV quantification system. An example might be for SEFPP to assist the MOHP in establishing a quantification unit and training technical staff in forecasting and quantification (including information technology). ● Discuss proposed SOW with MOHP decision makers Service Delivery – Supply-side Interventions 8. In conjunction with scaling up the task-sharing and PP/PA/IUD programs in the upper and lower governorates (as requested by the MOHP), SEFPP should leverage its substantial support for these interventions to encourage the MOHP to set up its own “qualitative research” protocols to measure outcomes and plan rollouts informed by evidence. ● Internally develop a Concept Paper that demonstrates how PP/PA/IUD outcome data (available from OB/GYN) can be used for strategic planning of rollout. ● Hold discussions and get buy-in about the concept and research protocols from MOHP planners ● Discuss approach and get approval from MOHP decision makers Social and Behavior Change Communication 9. SEFPP’s plan to assist NPC in developing and implementing a national SBCC strategy should be the main focus in the coming year to enable SEFPP to develop a more strategically effective SBCC vision and interventions for the future. ● Hold discussions with NPC and develop a plan of action for SEFPP assistance during the timeline of preparations for strategy reviews and development Training and Capacity Building Approaches 10. SEFPP should continue to advocate for and seek out opportunities in the MOHP or private sector to introduce TOT approaches into training and capacity-building activities. ● Survey SEFPP TOT assistance to date to determine results and categories of potential partners ● Approach interested partners to offer assistance in establishing TOT activities Training – Service Providers 11. SEFPP should conduct more training targeted to private sector pharmacists. Aside from needs identified by the trainees themselves (such as the need for additional supplies and media tools), a systematic training needs assessment should be conducted to discover ways of making the training curriculum more relevant to pharmacists’ work requirements ● Re-assess pharmacists training (including curricula) ● Develop targeted training campaign to be implemented in Years 4 and 5 Sustainability 12. SEFPP should continue or introduce the measurement of results for interventions it has identified as potentially sustainable. Work Plans, targets, and quarterly monitoring should be continued or incorporated into SEFPP’s MELP. ● Identify priority activities with sustainability value ● Incorporate plans and monitoring into the MELP USAID.GOV MIDTERM EVALUATION OF STRENGTHENING EGYPT’S FAMILY PLANNING PROGRAM | 34 ANNEX 1: EVALUATION STATEMENT OF WORK I. PURPOSE OF THE EVALUATION A. Evaluation Purpose The USAID/Egypt is conducting a midterm activity performance evaluation of its Strengthening Egypt’s Family Planning Program (SEFPP). The purpose of this mid-term evaluation is to assess the activity’s performance and progress towards achieving its three key objectives; (1) Strengthen evidence-based planning for decision making, (2) Quality of Family Planning/Reproductive health (FP/RH) services supported and (3) Selected health behavior enhanced. The evaluation will also serve learning purposes by identifying any challenges incurred by the activity to date and formulate appropriate recommendations for corrective actions and effective implementation during the remaining years of the activity and for future activities. B. Audience and Intended users The primary audience of the evaluation report will be USAID/Egypt, specifically USAID Senior Mission management, the Office of Education and Health (OEH), and the Program Office. Secondary audiences include the Government of Egypt’s Ministry of Health and Population (MOHP), National Population Council (NPC), and the activity’s implementing partner John Snow, Incorporated. The mid-term evaluation will be used to identify areas for the program to focus on for the remaining period, and to identify evidence of good practices for a broad network of stakeholders. USAID/Egypt will share the executive summary, final report and recommendations with the implementer, stakeholders, beneficiaries, and the general public via the Development Education Clearinghouse (DEC) within 90 days of report completion. II. SUMMARY INFORMATION Activity Name Strengthening Egypt’s Family Planning Program SEFPP Implementer John Snow, Incorporated Cooperative Agreement # 72026318CA00001 Total Estimated Ceiling $18,788,799 Life of Activity December 2017- December 2022 Activity’s Geographic Regions Assiut, Beni Suef, Fayoum, Giza, Luxor, Minya, Qena, Sohag, and in marginalized areas in Cairo and Alexandria Development Objective(s) (DOs) DO 3: Health outcomes for target populations improved as identified in the Health Project Appraisal Document (PAD) IR 3.1: Quality family planning reproductive health services supported IR 3.2: Evidence-based decision making increased III. BACKGROUND A. DESCRIPTION OF THE PROBLEM/DEVELOPMENT CHALLENGE 35 | MIDTERM EVALUATION OF STRENGTHENING EGYPT’S FAMILY PLANNING PROGRAM: ANNEXES USAID.GOV According to the most recent census (2017), Egypt’s population rose from 72.6 million in 2006 to 94.8 million in 2017, and is projected to be 111.5 million by the end of 2025. This growth is primarily influenced by changes in fertility, from an average of 3.0 births per woman in 2008 to 3.5 in 2014. The Egypt Demographic and Health Survey 2014 (EDHS 2014) indicated a persistent 13% unmet need for family planning; a contraceptive discontinuation rate of 30% within the first 12 months of use; 16% of all pregnancies in the previous five years unwanted at the time of conception; and nearly half (48.2%) of all second or subsequent births occurring fewer than three years apart. The use of any Family Planning (FP) method declined slightly, to 59% from 60% in 2008, and while intrauterine device (IUDs) at 51% of all methods used remain the most popular, its share in the method mix declined from 36% to 30% between EDHS 2008 and EDHS 2014. Pills and injectables are the only two methods that have grown, from 12% to 16%, and from 7% to 8%, respectively. Subdermal implants (Implanon), though part of the choice of FP methods, are offered only in selected MOHP facilities in urban settings; emergency contraceptive pills are available only in private pharmacies and are frequently out of stock. However, these national averages belie geographic variations. In rural Upper Egypt, only 45% of women were using any modern method and 53% were not using any method at all (compared with 41% nationally). While data for slums in Cairo and Alexandria are limited, access to FP counseling and methods are known to be inadequate. Based on 2017 projections, the nine governorates comprising Upper Egypt show total fertility rates as high as 4.6, 4.3, and 4.2 in Fayoum, Souhag, and Assiut, respectively. In Souhag, the contraceptive prevalence rate is as low as 31%, and one in four people have an unmet FP need. Young men and women marry, on average, at the age of 23 and 21 years, respectively. However, women in rural Upper Egypt marry at a younger age (19 years) compared to their urban counterparts (23 years), though women in the slums of greater Cairo also marry young. While the fertility desires of young people increased between 2009 and 2014 from 2.6 to 2.9 children among unmarried youth, and from 2.8 to 3.1 among married youth, in rural Assiut and Souhag, married adolescent girls desire an average of 3.8 and 4.2 children, respectively. As a consequence of early marriage, not only are young couples likely to have more children, but young women are also more likely to have abortions and stillbirths than their older peers. The MOHP is still the largest provider of FP services—almost 57% of current users obtained their method at a government facility, while the remainder went to a private clinic (22%) or pharmacy (20%). Clinics in the nongovernmental organization (NGO) sector are insignificant (0.6%). Availability of printed materials on FP is adequate at MOHP and private facilities, as is information on method choice, and management of side effects, particularly when compared with pharmacies. However, shortfalls in quality at the MOHP facilities, including limited visual and auditory privacy; lack of female physicians; high physician turnover; and method stock outs, remain. The MOHP facilities—intended to serve people who cannot afford to pay for services— are also being used by 39% of women from the highest wealth quintile, in part because the private sector is hindered from playing a greater role in FP service provision (e.g., complicated registration procedures for new products, rigid pricing mechanisms, lengthy importation procedures, limited training opportunities). Given this contextual backdrop, there have been positive developments: the 2014 constitution refers to implementing a population program, President Sisi and his cabinet are acutely aware of the economic and social consequences of the rapidly increasing population, and the President repeatedly highlights these concerns in his speeches. Egypt’s Population Strategy 2015–2030 lays out innovative ideas for reaching different population segments. While the MOHP through the FP sector will continue to be responsible for Egypt’s population agenda, support of political, religious, and cross￾sectoral leadership at the highest levels is critical for achieving results. B. ACTIVITY SUMMARY Approach Given the above development challenge, the Government of Egypt (GOE), has identified population growth as one of the biggest challenges facing Egypt, and it has expressed a desire to strengthen its family planning and reproductive health services. As a result, the SEFPP activity focuses on providing support to the Family Planning/Reproductive Health (FP/RH) Sector of the Ministry of Health and Population (MOHP) and the National Population Council (NPC) to contribute to the higher impact of the family planning related to the maternal mortality. USAID.GOV MIDTERM EVALUATION OF STRENGTHENING EGYPT’S FAMILY PLANNING PROGRAM | 36 The SEFPP activity’s long-term objective is to expand contraceptive use to contribute to reducing the number of high￾risk and high-parity births, thereby reducing maternal mortality. Access to contraceptives also helps to prevent unwanted pregnancies, some of which result in unsafe abortions—one of the leading causes of global maternal deaths. The activity is doing so by helping the GOE to re-establish a strong FP system and integrate it into the GOE’s maternal and other health programs through the following strategic approach: Focus on Strengthening Capacity of the existing information systems of the MOHP at the community, governorate, and national levels through technical assistance, advocacy, and collaborative coordination meetings. Reinforce institutionalization of family planning in Egypt by putting the GOE in the lead, working closely with the MOHP and other government structures to take an evidence-based approach to identify program opportunities. Expanding the role of the NGOs in Upper Egypt for FP service provision (that decreased to 0.6% documented in the DHS 2014) is a cornerstone to improve access of the vulnerable groups (poor) to services. The activity will ensure availability of an operational clinic and commodities in each of the selected NGOs to help improve community awareness and scale up the services. Build upon public health achievements that have been made by government, donors, and private sector programs. The activity will quickly review and build upon these programs to close the circle of information from community, health facility, and national levels and consolidate best practices and strengthen bottom-up and top-down approaches that focus on clients and providers. Focus on youth, current FP clients are young couples who increasingly are technology savvy. The SEFPP activity will take advantage of social media and appropriate technologies to reach this group and apply programmatic innovation that will bring about real change to family planning. Engage the private sector: The private sector comprises three important groups, each of which can have greater participation in the country’s FP program. First are private-sector providers of FP services: doctors, nurses, and pharmacists as well as health care entities such as private hospitals and pharmaceutical companies. The second group is the private commercial sector that increasingly considers its corporate social responsibility. The third group includes NGOs and foundations working on FP service provision and community-based interventions related to health and awareness. These three groups and their respective subgroups are motivated by a variety of factors that will be explored, and where possible, leveraged for the benefit of Egypt’s FP program. The activity will develop and test several prototypes for private sector engagement and its linkages to the public sector—public-private partnerships (PPPs)— to ensure sustainability. Strengthen family planning system through community participation in selected slum areas: Based on the national population strategy, which makes RH a national priority, SEFPP developed an intervention methodology for slums that focuses on the collaboration of key social players and the government to ensure FP/RH sustainability and the integration of health services within the RH/FP program. SEFPP will work with the MOHP to strengthen this participatory approach to working in slums. Block Grants: SEFPP has allocated approximately $2.2 million for local, community-level activities through the disbursement of block grants to local organizations in targeted districts in the nine governorates and slum areas. The team will establish sub-granting guidelines and procedures and develop selection criteria in consultation with USAID. Grantees will be identified, based on agreed-upon criteria, through the Social Solidarity Directorate and will be thoroughly vetted for their technical, financial, and human resource capacities, as well as for FP and other compliance. Results will be monitored against pre-established expectations developed jointly by the SEFPP team and each grantee’s leadership Programmatic Modifications: The SEFPP activity recently finalized a modification of the cooperative agreement (CA), amendment #6 to reflect a revision to the cooperative agreement (CA) program description necessitated by the Ministry of Health and 37 | MIDTERM EVALUATION OF STRENGTHENING EGYPT’S FAMILY PLANNING PROGRAM: ANNEXES USAID.GOV Population’s (MOHP) restrictions on quantitative/qualitative research with removal of this component and also the expansion of the Health Management Information System upgrade. COVID-19 activity response During the COVID-19 pandemic, SEFPP redirected activities implemented within the FP sector to respond to this pandemic. Those activities included developing a COVID-19 information module as an e-learning module within the national platform for the FP sector, designing COVID-19 awareness package for FP sector Master trainers, updating Minimum Service Delivery standards for infection prevention measures resisting COVID-19 and developing IEC materials and short videos with information on preventive measures for COVID-19. In total US$180,000 was reprogrammed for this work. ACTIVITY’S THEORY OF CHANGE: The Theory of Change (TOC) illustrates the strategic approach employed and the change pathways that will improve FP/RH indices and increase contraceptive security. The “roadmap” of how and why desired changes are expected to occur starts by defining the long-term goal. The preconditions needed to reach that goal are then mapped backwards, with the activities designed to work in concert to achieve measurable and positive changes. The main purpose of SEFPP activity is the revitalization of the family planning and reproductive health program in Egypt. This support is done by ensuring that the national program is implementing all the fundamental requirements to meet the beneficiaries’ needs. These fundamental requirements fall under the umbrella of SEFPP’s three results which are: Result 1: Strengthening evidence-based planning for decision making; Result 2: Quality of FP/RH services supported; and Result 3: Selected health behavior enhanced. Key populations are required to help ensure these change pathways are implemented. These include a wide range of stakeholders including the MOHP, NPC, and other government entities; public and private sector providers; media and communication personnel; religious leaders; and men and women of reproductive age. C. ACTIVITY ACHIEVEMENTS TO DATE: With regards to the COVID-19 activity response, work is on-going for the relevant outreach materials, but the materials have not been finalized. JSI is also working on developing an online resource platform for providers. Result 1: Strengthening evidence-based planning for decision making Health Management Information System (HMIS) SEFPP completed activities pertaining to the assistance provided to the MOHP’s IT unit in identifying priority needs of the reporting system to support and strengthen data capture through the national HMIS. SEFPP conducted a HMIS/LMIS software development functionality workshop. Using the information gathered in the workshop, SEFPP developed an HMIS /LMIS request for proposal (RFP) and selected the company that is currently working on upgrading the Management Information System within the National Population Council. Contraceptive security (CS) increased ● A Contraceptive Security Assessment was completed and a comprehensive final report was issued. ● A new FP methods manual was developed, shared with MOHP/FP Sector with some methods used for pilot testing, for example:1) Hormonal IUD (Mirena), 2) Sayana Injectables, and 3) Silver IUD. ● An assessment mechanism was developed for regularly assessing and creating a database for the status of contraceptive warehouses. This has consisted of a warehouse assessment checklist tool developed for the MOHP. It has not been implemented systematically as yet, but could be a useful part of the HMIS implementation for MOHP. USAID.GOV MIDTERM EVALUATION OF STRENGTHENING EGYPT’S FAMILY PLANNING PROGRAM | 38 RESULT 2: Quality of FP/RH Services Supported FP/RH service delivery ● A Family Planning (FP) trainer guide was developed with the MOHP/FP central level. ● 91 Physicians received 7 TOT training on quality service delivery. ● 124 nurse supervisors attended 8 TOT courses for FP covering Cairo, Giza, Fayoum, Menya, Aswan and Sohag Governorates. ● 13 Basic FP training courses and 7 Decision Making Tool (DMT) training courses for MOHP/FP were conducted and attended by 132 FP newly recruited retired physicians to work in facilities lacking service providers in Fayoum, Menya, Assiut, Sohag, Qena, Luxor and Aswan governorates as recommended by the MOHP/FP Sector. Increase private sector access to quality FP/RH services ● A memorandum of understanding (MoU) was signed with Marie Louis Factory for Clothes, to conduct awareness training for their female factory workers and technical school students using a health and life skills education curriculum that built on the shared training document from the international Population Council. The intervention supported 16 awareness sessions addressing reproductive health topics, family planning advice, information, and decision making that benefit their health and their children’s health and well-being. Those training sessions provided up to date health information for hundreds of female factory workers and vocational school students. ● MOU was signed with the Medical and Pharmaceutical Syndicate to provide updated FP/RH information to hundreds of private sector doctors and pharmacists. ● MOU was signed with DKT to support the private sector practitioner service provision training on FP services, updates and best practice. Strengthen family planning system through community participation in selected slum areas ● SEFPP, in collaboration with MOHP/FP Sector, formed and trained task forces, comprising MOHP officials, local authorities, and community leaders, to conduct needs assessment in targeted slum areas using the developed assessment tools. RESULT 3: Selected Health Behaviors Enhanced Social Behavior change communication activities ● 6 rounds of FP outreach master trainer workshops were conducted and attended by 113 media officers. ● 7 Postpartum Contraceptive (PPC) workshops were completed and reached out to 126 media officers from both lower and Upper Egypt Governorates. (Cairo, Giza, Alexandria, Qaliubyia, Sharikia, Port Said, Assuit, and Sohag). ● The behavior change component has been reduced in scope over time and has now focused on capacity building for the media officers in the FP sector to conduct outreach as opposed to orchestrating an outreach campaign directly as with more traditional BCC based programs. D. RELEVANT DOCUMENTATION The Agreement Officer’s Representative, through USAID/Egypt’s Education and Health Office, will provide relevant activity documents that are available. These documents include: 1. SEFPP cooperative agreement and six amendments 2. SEFPP Annual Work Plan 3. SEFPP quarterly progress reports 4. SEFPP Fact Sheet 5. Recent updated MEL plan Other sources: 4. 2017 National population census 39 | MIDTERM EVALUATION OF STRENGTHENING EGYPT’S FAMILY PLANNING PROGRAM: ANNEXES USAID.GOV 5. 2014 EDHS 6. Egypt’s Population Strategy 2015–2030 In addition to the above list, the evaluator’s document review will consider other secondary literature determined relevant by the evaluation team. IV. EVALUATION QUESTIONS This section presents the fundamental questions that the evaluation will answer. The evaluation will produce a comprehensive analysis of the outcomes and the causal contributors to those outcomes. The specific EQs are: Questions Suggested Data Sources Suggested Data Collection Methods (other methods can be suggested by the Evaluation team) Data Analysis Methods (other methods can be suggested by the Evaluation team) 1) To date, how successful has SEFPP been in achieving the results under each component? What has been accomplished so far? What have been the challenges? What factors have led to specific accomplishments? (Specific factors to examine include but not limited to: performance against indicators, relationship management with GOE stakeholders, internal SEFPP management and implementation) 1.a To date, in what ways did SEFPP activity strengthen the capacity and commitment in the local system (health workforce, health information system & health service delivery) to continue delivering consistent levels of access to, and delivery of quality FP services? - MOHP central office staff - MOHP Governorates staff - Beneficiaries (such as Media officers, doctors, nurses) - Assessments conducted by the activity - MEL Plan of the activity showing progress against performance indicators . - IP staff - USAID AOR - PopCouncil director and staff - Center for development services (CDS) project staff -Desk review - Key informant interviews and focus groups -Site visits - Analyze the data of key informant interviews, focus groups and reports USAID.GOV MIDTERM EVALUATION OF STRENGTHENING EGYPT’S FAMILY PLANNING PROGRAM | 40 1. 2) How well are SEFPP’s COVID-19 Response activities fulfilling MOHP’s needs? - - MOHP central staff - MOHP governorate staff - IP staff - Activity field managers - Beneficiaries as doctors and nurses - Private sector partners as Marie Louis and medical and pharmaceutical syndicates -Desk review - Key informant interviews and focus groups -Site visits - Analyze data of key informant interviews, focus groups and reports V. EVALUATION DESIGN AND METHODOLOGY A mixed method approach to the evaluation that complies to the greatest extent possible with the USAID evaluation policy is recommended. https://www.usaid.gov/sites/default/files/documents/1870/USAIDEvaluationPolicy.pdf The above evaluation design and methodology matrix is illustrative. The evaluation team must include in the proposal an updated matrix (depending on the situation at the time of the implementation) and must propose specific qualitative and quantitative data collection and analyses methods that align with current circumstances to be utilized to appropriately answer the listed EQs . The team must aim to collect data using a sample size that is valid and representative. DATA COLLECTION METHODS: The Evaluation Team should provide Plan A “normal with precautions “ to include combined methodologies for both face to face and virtual meetings but with field data collection. The evaluation team should propose a Plan B to move to fully remote data collection. Field work would be carried out while strictly observing procedures and protection in response to COVID -19. The evaluation team must develop data collection tools that are consistent with the evaluation questions to ensure high quality analysis. The evaluation team is required to share data collection tools with the USAID Evaluation Program Manager for review, feedback and/or discussion with sufficient time for USAID’s review before they are applied in the field. The evaluation team must complete the review of all debriefing materials cited in the “Relevant Documentation” section prior to the team leader’s arrival in Egypt.” Data collection methods may include a combination of the following: 41 | MIDTERM EVALUATION OF STRENGTHENING EGYPT’S FAMILY PLANNING PROGRAM: ANNEXES USAID.GOV ● Desk review of relevant documentation: USAID/Egypt will provide the Evaluation Team with soft copies of the activity related documents as needed. All team members shall review these documents in preparation for the initial team planning meeting. ● Key informant interviews ● Group discussions with beneficiaries and other counterparts and stakeholders- sample size will be determined by the evaluation team. Interviews and Site visits: Fieldwork will be determined by evaluators within the geographical governorates of SEFPP. Key Informant and Group Interviews will include, but may not be limited to: ● USAID/Egypt Office of Health Activity Manager ● USAID/Egypt Office of Health & Education Director. ● JSI staff ● Field level staff from the targeted governorates ● MOHP staff. ● NPC staff. ● Beneficiaries: trained doctors, nurses and media officers, etc The evaluation team must provide a more detailed explanation of the proposed methodology for collecting data. A list of interviewees and key stakeholders will be provided by USAID prior to the assignment’s inception. The evaluation team must aim to collect data using a sample size that is valid and representative. DATA ANALYSIS PLAN: Prior to the start date of data collection, the evaluation team must develop and present, for USAID/Egypt review and approval, a data analysis plan that details how focus groups and key informant interviews will be transcribed and analyzed; what procedures will be used to analyze qualitative and quantitative data from key informant and other stakeholder interviews; and how the evaluation will weigh and integrate qualitative data from these sources with quantitative data from performance indicators and the activity performance monitoring records to reach conclusions about the effectiveness and efficiency of activities. Data will be disaggregated by sex, governorate, type of intervention, age group when applicable, to identify how the activity benefitted different types of project beneficiaries. DATA QUALITY STANDARDS: The evaluation team must ensure that the data they collect clearly and adequately represent answers to the evaluation questions, is sufficiently precise to present a fair picture of performance and is at appropriate level of details. The Data Quality Standards must adhere to ADS 201 standards of validity, integrity, reliability, precision, timeliness DATA LIMITATION: USAID expects that all issues affecting validity be discussed and documented during the evaluation planning. Measures to mitigate these issues will be addressed with all team members and USAID in the implementation phase and detailed in the final report. The activity could not collect baseline data so the evaluation team should provide ways/a methodology that can mitigate the effect of not having baseline data. VI. DELIVERABLES AND REPORTING REQUIREMENTS LOGISTICS: USAID will provide overall direction to the evaluation team, identify key documents, and assist in facilitating a work plan. USAID will assist in arranging meetings with key stakeholders identified by USAID prior to the initiation of field￾work. The evaluation team is responsible for arranging other meetings as identified during the course of this evaluation and advising USAID/ Egypt prior to each of those meetings. USAID.GOV MIDTERM EVALUATION OF STRENGTHENING EGYPT’S FAMILY PLANNING PROGRAM | 42 The evaluation team is also responsible for arranging transportation as needed for site visits in and around Cairo and selected governorates. The evaluation team will be responsible for arranging its own work/office space, computers, internet access, printing, and photocopying. The evaluation team is also responsible for procuring and paying for translation services for interviews, reports and any other evaluation related task. Evaluation team members will be required to make their own lodging and travel payments. USAID personnel will be made available to the team for consultations regarding sources and technical issues, before and during the evaluation process. DELIVERABLES ➔ Team Planning Meeting (TPM): A team planning meeting must be held in Egypt before the start of the field work. This meeting will allow USAID/Egypt to discuss the purpose, expectations, and agenda of the assignment with the Evaluation Team. In addition, the team will: • Finalize team members' roles and responsibilities; • Review and make recommendations for improving the precision of evaluation questions; • Review and finalize the assignment timeline; • Present and discuss data collection methods, instruments, tools and guidelines; and • Review and clarify any logistical and administrative procedures for the assignment. ➔ Work Plan: A well-written, detailed methodology and data analysis plan (evaluation design, data analysis steps and detail, operational work plan) must be prepared by the team and discussed with USAID during the planning meeting. The activity Manager will provide the evaluation team with an initial list of interviewees, from which the evaluation team can work to create a more comprehensive list. The evaluation team will construct an interview schedule that includes different stakeholders, and then share with USAID the updated lists of interviewees and schedules as meetings/interviews take place and informants are added to/deleted from the schedule. An updated list must be sent to USAID within three working days after the USAID Team Planning Meeting. The evaluation team should submit the work Plan and evaluation methodology (including data collection tools) to the evaluation manager for review and approval. ➔ Validation Workshop with IP: To validate/clarify preliminary findings and provide a venue for clarification of data collection and findings as a result of the field work. ➔ Final OEH/Program office Exit Briefing: The evaluation team is expected to hold a final exit briefing prior to leaving the country to discuss the preliminary findings and present the draft report. This presentation will be scheduled as agreed upon during the Team Planning Meeting. This briefing is mainly for the OEH Office and the Program office to present preliminary findings. ➔ Final presentation/Mission Debrief: prior to the team leader’s departure, the evaluation team must present their final findings to the USAID/Egypt Mission after submitting and approving the draft skeletal report. The Mission debriefing must include a discussion of achievements and issues as well as recommendations for the future activities designs and implementation. The team must consider any USAID/Egypt comments and revise the draft report as appropriate. ➔ Debriefing with Partners (Tentative): The team will present the major finding of the evaluation to USAID partners (as appropriate and as defined by USAID) through a PowerPoint presentation prior to the team’s departure from the country. The debriefing will include a discussion of achievements and activities and will incorporate partners’ comments accordingly, as appropriate. ➔ Draft final evaluation Report: The evaluation team must submit a draft evaluation report within 15 business days of the team leaders’ departure. Final Report Format. The report will address each of the questions identified in the SOW, including a finding, data source, conclusion, recommendation summary table for each question and any other issues the team considers to have a bearing on the objectives of the evaluation. Any such issues can be included in the report only after consultation with USAID. ➔ Mission & partners Peer Review: Once the initial draft evaluation report is submitted and the pre-final presentation is held, the draft report must undergo a peer review and the Mission will have 10 business days in which to review and comment on the initial draft, after which point the evaluation manager will submit the consolidated comments to the evaluation team. The team must consider all USAID/Egypt comments when submitting the final evaluation report. 43 | MIDTERM EVALUATION OF STRENGTHENING EGYPT’S FAMILY PLANNING PROGRAM: ANNEXES USAID.GOV ➔ Final Report and data sets: evaluation team will submit the final evaluation report to the USAID Evaluation Manager within 10 working days from receiving USAID’s comments. The written report must clearly describe findings, conclusions and recommendations for future programming. The submission date for the final evaluation report will be determined in the evaluation work plan. The final report must not exceed 30 pages in length (not including appendices, lists of contacts, etc.). The format must include an executive summary, table of contents, glossary, methodology, findings, and conclusions. The report must be submitted initially in English, electronically, and later, an Arabic translation of the Executive Summary will be submitted. All project evaluation data and records will be submitted in full and should be in electronic form in easily readable format, organized and documented for use by those not fully familiar with the intervention or evaluation, and owned by USAID. At the time of submission of the final English language report, the data collection instruments, interviews and data sets must be submitted on a flash drive to the evaluation program manager. USAID/Egypt intends to disseminate the report within USAID and to stakeholders. All quantitative data collected by the evaluation team must be provided in machine-readable, non-proprietary formats as required by USAID’s Open Data policy (see ADS 579). The anonymized data should be organized and fully documented for use by those not fully familiar with the activity or the evaluation. USAID will retain ownership of the survey and all datasets developed. ➔ Arabic Translation of the executive summary: After approval of the final report, the evaluation team must submit the Arabic translation of the report’s executive summary within 10 working days from approving the evaluation report. ➔ Infographics English and Arabic: After approval of the final report, the evaluation team must submit a maximum of two page infographic summarizing the main findings and other relevant information. USAID will have to review and approve. The English infographic must be finalized within 10 working days from approving the evaluation report and the Arabic infographic will be finalized within 10 business days from approval of the English version of the infographic. REPORTING REQUIREMENTS Final Report Format The evaluation final report should include executive summary; background of the local context and the strategies/projects/activities being evaluated; the evaluation purpose and main evaluation questions; the methodology or methodologies; the limitations to the evaluation; findings, conclusions, and recommendations. For more detail, see "How-To Note: Preparing Evaluation Reports" and ADS 20l, USAID evaluation Report Requirements. An optional evaluation report template is available in the Evaluation Toolkit. The executive summary should be 2- 5 pages in length and summarize the purpose, background of the activity being evaluated, main evaluation questions, methods, findings, conclusions, and recommendations. The executive summary will be translated into Arabic. The evaluation methodology will be explained in the report in detail. Limitations to the evaluation will be disclosed in the report, with particular attention to the limitations associated with the absence of the implementing partner and the evaluation methodology (e.g., selection bias, recall bias, unobservable differences between comparator groups, etc.) The annexes to the report will include: -The Evaluation SOW; -All data collection and analysis tools used in conducting the evaluation, such as questionnaires, checklists, and discussion guides; -All sources of information properly identified and listed; -Signed disclosure of conflict of interest forms for all evaluation team members, either attesting to a lack of conflicts of interest or describing existing conflicts of any "statements of difference" regarding significant unresolved differences of opinion by funders, implementers, and/or members of the evaluation team; and -Summary information about evaluation team members, including qualifications, experience, and role on the team. USAID.GOV MIDTERM EVALUATION OF STRENGTHENING EGYPT’S FAMILY PLANNING PROGRAM | 44 In accordance with ADS 201, the contractor will make the final evaluation reports publicly available through the Development Experience Clearinghouse within three months of the evaluation’s conclusion. CRITERIA TO ENSURE THE QUALITY OF THE EVALUATION REPORT Per ADS 201, Criteria to Ensure the Quality of the Evaluation Report, draft and final evaluation reports will be evaluated against the following criteria to ensure the quality of the evaluation report. - Evaluation reports should represent a thoughtful, well-researched, and well-organized effort to objectively evaluate the strategy, project, or activity. - Evaluation reports should be readily understood and should identify key points clearly, distinctly, and succinctly. - The Executive Summary of an evaluation report should present a concise and accurate statement of the most critical elements of the report. - Evaluation reports should adequately address all evaluation questions included in the SOW, or the evaluation questions subsequently revised and documented in consultation and agreement with USAID. - Evaluation methodology should be explained in detail and sources of information properly identified. - Limitations to the evaluation should be adequately disclosed in the report, with particular attention to the limitations associated with the evaluation methodology (selection bias, recall bias, unobservable differences between comparator groups, etc.). - Evaluation findings should be presented as analyzed facts, evidence, and data and not based on anecdotes, hearsay, or simply the compilation of people’s opinions. - Findings and conclusions should be specific, concise, and supported by strong quantitative or qualitative evidence and linked in a table to the sources of data. - If evaluation findings assess person-level outcomes or impact, they should also be separately assessed for both males and females. - If recommendations are included, they should be supported by a specific set of findings and should be action-oriented, practical, and specific. VII. EVALUATION COMPOSITION SCHEDULING Work is to be carried out over a period of approximately 15 weeks, o/a beginning of January 2021 Each week will be a 6 working day week. TEAM COMPOSITION: The team will include the following personnel and all efforts should be made for the team to be composed of a balanced number of male and female members. The evaluation team will sign COI forms that will be included in the report annexes. Evaluation Team Leader: A senior international consultant with at least 15 years of extensive experience in leading evaluation teams and the design, implementation, and/or evaluation of foreign assistance programs, including significant (12+ years) experience in health with focus on Family Planning issues (preferred). He should have experience in leading data collection teams, analyzing data and summarizing findings. As the evaluation Team Leader, the incumbent should be thoroughly familiar with the techniques of mid-term evaluations, and possess good organization and team-building skills, and excellent written and oral communication skills in English. Previous overseas experience in Egypt (preferred) or the Middle East region (required). The Team Leader will: - Finalize and negotiate with USAID/Egypt the evaluation work plan; - Design the evaluation plan. - Establish evaluation team roles, responsibilities, and tasks; 45 | MIDTERM EVALUATION OF STRENGTHENING EGYPT’S FAMILY PLANNING PROGRAM: ANNEXES USAID.GOV - Facilitate the Team Planning Meeting - Ensure that the logistics arrangements in the field are complete; - Manage team coordination meetings in-country and ensure that team members are working to schedule; - Coordinate the process of assembling individual input/findings for the evaluation report and finalizing the evaluation report; - Lead the preparation and conduct presentation of key evaluation findings and recommendations to USAID/Egypt team prior to departing Egypt. Senior Health Expert: the Evaluation Team shall include a local health expert. It is strongly recommended that the following characteristics be reflected in the health Expert: fluency in Arabic language; 10+ years of past experience in monitoring and evaluation of health development projects, preferably focused on family planning activities; extensive field experience in Egypt or the MENA region; strong written and verbal communication skills. Three Mid-level Monitoring and Evaluation Experts: the Evaluation Team shall include 3 local monitoring and evaluation experts to divide themselves on the selected governorates. The following characteristics must be reflected in the monitoring and evaluation expert in in order to maximize use of time and effectiveness of the survey: fluency in Arabic and English language; at least 7 years of past experience in monitoring and evaluation of health development projects focusing on family planning; extensive field experience in Egypt; and strong written and verbal communication skills. (to conduct interviews) The Local Logistics/Admin Coordinator: A local consultant will serve as local logistics coordinator. The person should be fluent in written and spoken Arabic. He/she will provide logistical, administrative, and clerical support to the team. He/she will have at least four years of experience in an administrative support role. USAID.GOV MIDTERM EVALUATION OF STRENGTHENING EGYPT’S FAMILY PLANNING PROGRAM | 46 ANNEX 2: SEFPP MIDTERM EVALUATION CONCEPT NOTE Concept Note Strengthening Egypt’s Family Planning Program (SEFPP) Activity Mid-term Performance Evaluation 1. INTRODUCTION The USAID/Egypt Program Office (PO) has tasked Services to Improve Performance Management, Enhance Learning and Evaluation (SIMPLE) to conduct a mid-term performance evaluation of the Strengthening Egypt’s Family Planning Program (SEFPP) activity, implemented by John Snow, Inc. (JSI). The evaluation will roughly cover the first three years of implementation, with the activity period of performance from December 2017, to the present. SIMPLE will engage a highly competent team comprised of a local subject matter expert and evaluators, led by an international specialized Team Leader. Answering the evaluation questions will require a mixed-methods approach, comprising a qualitative approach supplemented by quantitative methods (where appropriate and feasible) to collect primary data from SEFPP stakeholders and beneficiaries in Cairo and four other governorates including Luxor, Qena, Alexandria, and Assiut. Conditions permitting, data collection will be carried out both face-to-face and virtually to accommodate COVID-19 limitations, and in all cases proper precautions will be in place for the safety of the team and the evaluation respondents. In the event it is necessary, the team is ready to switch to fully virtual work. This Concept Note presents the evaluation purpose and questions from the Scope of Work (SOW) and provides an overview of SIMPLE’s evaluation approach. A detailed Work Plan, including a methodology and data analysis plan will be developed by the evaluation team as an outcome of the Team Planning Workshop/Meeting (TPW/M). The evaluation’s final Work Plan will be presented to USAID for approval. The evaluation will follow SIMPLE’s standard process, which is depicted in Figure 1 below. Figure 1: SIMPLE Evaluation Process T 2. PURPOSE As per the SOW, the evaluation purpose is to “assess the activity’s performance and progress towards achieving its three key objectives; (1) strengthen evidence-based planning for decision making, (2) quality of family planning/ reproductive health (FP/RH) services supported and (3) selected health behavior enhanced. The evaluation will also SOW Design/ TPW/M Data Data FCR/ Briefings Reporting 47 | MIDTERM EVALUATION OF STRENGTHENING EGYPT’S FAMILY PLANNING PROGRAM: ANNEXES USAID.GOV serve learning purposes by identifying any challenges incurred by the activity to date, and formulate appropriate recommendations for corrective actions and effective implementation during the remaining years of the activity and for future activities.” The evaluation will address the following three questions: 1. To date, how successful has SEFPP been in achieving the results under each component? What has been accomplished so far? What have been the challenges? What factors have led to specific accomplishments? (Specific factors to examine include but not be limited to: performance against indicators, relationship management with GOE stakeholders, internal SEFPP management and implementation) 2. To date, given the current activity’s status and results reported under each objective: (1) Strengthen evidence-based planning for decision making, (2) quality FP/RH services supported, (3) selected health behaviors enhanced, in the quarterly progress reports, annual reports and the performance plan & report (PPR), how successful have they been, including the factors contributing to their success? What are the hindrances? 3. To date, in what ways did SEFPP activity strengthen the capacity and commitment in the local system (health workforce, health information system & health service delivery) to continue delivering consistent levels of access to, and delivery of quality FP services? 4. How is the SEFPP activity responding and adapting to COVID-19 crisis on the level of each result? And How effective are these interventions in fulfilling the MOHP needs? 3. TECHNICAL SCOPE 1. DESIGN As part of the desk review, the team will review SEFPP monitoring reports as well as any relevant secondary data that may be available and that could provide contextual information for family planning and reproductive health in Egypt. There may also be quantitative data available from GoE or other donor sources, however we understand access to GoE data from CAPMAS may be restricted. Drawing on information gathered in the desk review, the evaluation will employ a multi-method design to allow for triangulation and verification of desk review material with primary data collected in the field by seeking multiple data sources and using a variety of data collection tools. Multiple methods will include: ● in-person or virtual (depending on COVID-19 restrictions and comfort levels) semi-structured, one-to-one key informant and group interviews with key stakeholders (i.e., USAID staff, JSI staff, Government of Egypt, private sector, and other institutional partners); ● group discussions with training recipients (doctors, nurses, and other medical professionals); ● group discussions or short questionnaire, as feasible, with family planning service users (only if feasible to get contacts from the IP and organize without special permissions or privacy concerns), and; ● site visits to locations receiving improvements. If direct data collection with family planning service users is not possible due to concerns for privacy and permission to gather primary data, the evaluation team will seek secondary data sources from either the project or health care facilities that may provide customer satisfaction information. At least one site visit will be conducted in each of the governorates of Cairo, Qena, and Alexandria where training facilities were refurbished. Evaluators will use a site visit tool to record their observations related to the evaluation questions. The tool will also contain a checklist to compare specific observations related to the refurbishment activities. This tool will be shared with the Program Office along with all of the other data collection instruments for review and approval. There may also be other opportunities identified for site visits during the desk review, consultations with JSI, and recommendations arising from the Team Planning Workshop/Meeting. Depending on a USAID.GOV MIDTERM EVALUATION OF STRENGTHENING EGYPT’S FAMILY PLANNING PROGRAM | 48 review of project monitoring data and availability of training recipients to convene for group discussions, the evaluation team may also consider a short SurveyMonkey questionnaire for medical professionals. More details regarding the design, data collection and sampling will be provided in the work plan for this evaluation. 2. SAMPLING Although SEFPP has conducted activities in eight governorates, SIMPLE will plan data collection activities in a purposive sample of governorates that have received the most assistance. USAID has identified the five governorates of Cairo, Luxor, Qena, Alexandria, and Assiut that meet this criteria. The evaluation team will focus its data collection in these five governorates. The evaluation team will identify the relevant staff and leadership at each stakeholder organization and will schedule visits and/or meetings for data collection with the key informants. As the Team Leader will work only from Cairo during the two week data collection period, she may also have the opportunity to conduct a few virtual key informant interviews related to the work accomplished in the other three governorates to augment the focus of the evaluation on the five governorates selected for field work. This may give the evaluation team the opportunity to better understand why certain governorates were more active than others. For this, we will select a purposive sample of leadership at key stakeholder organizations in these additional governorates. Regarding group discussions with training beneficiaries, SIMPLE understands that across all eight governorates there have been 91 doctors, 124 nurses and 132 other medical professionals trained as part of either a training-of-trainers activity or other basic family planning training. In addition, there were up to 239 media officers trained related to the social behavior change component of SEFPP. This is a relatively small beneficiary population for sampling and data collection. For the purposes of group discussions, participants will be identified and engaged according to availability. SIMPLE will need to determine the population pools, with consideration for gender participation, in each type of training activity for each of the five governorates targeted for data collection and select roughly 6 to 8 participants for each group discussion planned. If evaluators are able to conduct at least four group discussions in each governorate, the team will need to convene approximately 24 to 32 training beneficiaries per governorate, or roughly 120 to 160 in total, knowing those figures can changes once the evaluation team completes desk review. Data collection is planned to take place over two weeks, with teams spending roughly one week in each governorate. Depending on location access, group discussions may be convened at training facilities used for SEFPP or other benefiting facilities. Given the larger size of the general populations served by health facilities in each governorate, it will not be possible to obtain a statistically valid sample of family planning service users. However, we will plan to convene a convenience sample of service users at health facilities at the time when the evaluation team convenes group discussions with training recipients. Although this may not be feasible due to privacy concerns, if possible, the team will use the opportunity and location to pull together small groups for discussion, or to conduct short questionnaires related to access and quality of services, as well as any changes in health behaviors to help address EQ 2 and EQ 3. A detailed sampling plan, as well as all data collection instruments, will be shared and discussed with the PO during the Team Planning Meeting and will be included in the work plan the team will submit as part of this evaluation. If primary data collection with end users is not possible, the team will seek secondary data sources related to customer satisfaction that may be available from either the project or health care providers. 3. DATA COLLECTION Data collection will be undertaken by a team of six technical experts as follows: 1. Expatriate Team Leader, 2. One subject-matter expert (SME), and 3. Four Monitoring and Evaluation (M&E) Specialists/Evaluators 49 | MIDTERM EVALUATION OF STRENGTHENING EGYPT’S FAMILY PLANNING PROGRAM: ANNEXES USAID.GOV The team will be divided into three sub-teams of two people. Data collection will be conducted over a period of approximately two weeks. Due to COVID-19 travel concerns, the Team Leader will remain in Cairo for the duration of the data collection. In Sub-team # 1, the Team Leader will work with an M&E Specialist/Evaluator to collect data in Cairo, focusing on private sector, institutional and government partners, USAID and SEFPP staff. They will also conduct group discussions and site visits in El-Haggana, El-Darassa, and El-Asmarat areas. Sub-team # 2, including the SME and an M&E Specialist/Evaluator will travel to Luxor and Qena for data collection over roughly one week in each governorate. Sub-team #3 made up of two M&E Specialists/Evaluators will travel to Alexandria and Assuit. The evaluation team will be trained by the Team Leader on the data collection tools and approach in advance of data collection (including written observation and informant interview guidelines) to ensure consistency across teams. The Team Leader will coordinate all evaluation activities with the SIMPLE evaluation manager in Cairo. SIMPLE will request to establish communication protocols with the SEFPP implementing partner JSI and government counterparts. Before the evaluation start up, SIMPLE will ask the activity Agreement Officer Representative (AOR) at the Office of Education and Health (OEH) to inform the Ministry of Health and Population (MOHP) and National Population Council (NPC) contact points at the central level about this evaluation. After this is done, SIMPLE will ask the AOR to introduce the evaluation team to the JSI team. The IP will be asked to in: 1) introduce the evaluation team to officials at governorate level and benefiting entities, including local state hospitals, via a letter provided by SIMPLE; and 2) introduce the teams to other institutional partners, including the American University of Cairo, the NPC as well as private sector entities, such as Marie Louis and other medical and pharmaceutical syndicates and private care facilities, for the purposes of setting up key informant interviews and group discussions. 4. DATA ANALYSIS Following SIMPLE’s established evaluation process, a rigorous system of notetaking will be adopted to ensure data is documented and shared with all team members on a daily basis during data collection. Data analysis will take place over a 10-day team analysis workshop. Qualitative data will be categorized and analyzed to determine patterns and findings across governorates. The team will look for convergence and divergence in responses, as well as content patterns related to accomplishments and challenges encountered during the first three years of implementation. Responses will be analyzed according to the different categories of stakeholders, as well as across gender and geographic locations, to allow for better understanding of findings. Qualitative data will be examined during the desk review from and GoE or family planning donor sources available to the team. This may include national or localized statistical data and/or academic research. In addition, the team may also need to rely on secondary customer satisfaction data from health care providers, if available. SIMPLE utilizes the Findings, Conclusions and Recommendations (FCR) framework to analyze and present data, and this will be the basis for analysis, debriefing presentations, and reporting. 5. LIMITATIONS An obvious limitation of this evaluation is the COVID-19 situation. While data collection is planned to be a mix of face-to-face and virtual interactions with three small data collection teams visiting the five governorates, this could change if the situation requires. SIMPLE will be ready to switch to fully remote approaches if necessary, though this likely would come with limitations on reach and depth of data that can be collected in such a scenario. Other limitations may include our ability to convene group discussions with medical professionals in each governorate. This will depend on their availability, given the unique and demanding work schedule of doctors and nurses in particular. Should this pose a problem for data collection efforts, SIMPLE may consider a short online questionnaire. Biases arising from the sampling methods will be identified and addressed during the Team Planning Workshop to ensure credibility of results. USAID.GOV MIDTERM EVALUATION OF STRENGTHENING EGYPT’S FAMILY PLANNING PROGRAM | 50 4. TEAM COMPOSITION Given the scope of this evaluation involving visits to five governorates, this evaluation will configure a team of six specialists into three smaller teams of two each for field work. A statistician will be engaged to assist in sampling, data analysis and the preparation of data files for submission. A field coordinator will support the team in all the logistical work that needs to be put in place for the evaluation, and a few days of an interpreter may be needed for the Team Leader for any high-level meetings in which the second evaluator will be notetaking and thus may not be able to translate conversations on the spot. The full team in this evaluation and their estimated level of effort (LOE) is shown in Table 1 below. The CVs of the Team Leader and five other technical team members, including the Subject Matter Expert Table 1: Team Composition # Name, Role 1 Mildred Howard, Team Leader 2 Rafeek Ramzey, Subject Matter Expert 3 Nivine Ramsis, Evaluator 4 Noha Hassam, Evaluator 5 Hanan Kwinana, Evaluator 6 Norhan Bader, Evaluator 7 Ehab Sakr, Statistician 8 Translator 9 Wael AbdelKarim, Field Coordinator (SME) and Evaluators included in this evaluation, are provided in this Concept Note as a separate Appendix. For ease of reference, we also share below a blurb about the Team Leader, Mildred Howard and below it more information on the SME. Team Leader Ms. Mildred Howard has over 30 years of experience in over a dozen countries in international development assistance as a program manager, evaluator, and independent technical consultant. Her career has focused on the health sector where she has held several positions, including overseeing, directing and/or managing health programs for USAID/Missions in several countries, as well as leading teams conducting evaluations of health programs. These opportunities gave her solid understanding of the sector from both a programmatic and evaluative angle, including nine years of family planning specific experience. Ms. Howard is well known in Egypt . She has lived and worked in Egypt, including directly working on family planning activities. For example, in 2013 Ms. Howard served as team leader of the Evaluation of Health Systems Strengthening Technical Assistance Activity implemented by Social Impact, Inc. Earlier on, in 2001-2006 (five years), Ms. Howard lived and worked full-time in Cairo. As Health Sector Reform and Financing Advisor in the USAID Mission she was assigned to implement USAID’s support for the Government of Egypt’s Health Sector Reform Program which involved implementation of a national family health (inclusive of family planning) service delivery model. She has authored several publications, including a number of 51 | MIDTERM EVALUATION OF STRENGTHENING EGYPT’S FAMILY PLANNING PROGRAM: ANNEXES USAID.GOV evaluation reports and other research. She comes highly recommended by a QED Group colleague who has worked with her in the past. Ms. Howard holds a Masters’ Degree in Social Policy and Program Evaluation from the prestigious Harvard University in the US as well as several health-related certificates from other universities. Subject Matter Expert Dr. Rafeek Ramzey has 28 years of professional experience. He began his career as a physician and branched out to working on development activities, many of which were in the health and population sector. He has had direct work experience with population and/or reproductive health activities over the years. This has included the population and health activity of the Non-Governmental Organizations Service Center (NGOSC) funded by USAID; European Commission Support to Population Program in Upper Egypt; as well as close coordination with the Ministry of Health and Population (MOHP) under the primary health care through a community participation activity funded by the Catholic Coptic Diocese. He has worked extensively with NGOs and has been involved in several training efforts, including preparation of a training manual on peer education for Family Health International (FHI) more recently. In addition to direct work with reproductive health and population activities, he brings hands-on experience with training efforts which is very relevant to the SEFPP evaluation that includes looking at its training component. Dr. Rafeek has worked for prominent USAID and EU-funded activities. Through these opportunities, he has gained extensive experience in institutional capacity building, monitoring and evaluation, as well as management of a wide portfolio of development activities. He has also served as team leader for five evaluations. Dr. Rafeek began his career as a trained physician, and holds a Master’s degree in the field. USAID.GOV MIDTERM EVALUATION OF STRENGTHENING EGYPT’S FAMILY PLANNING PROGRAM | 52 5. DELIVERABLES & TIMELINE This evaluation will produce a number of deliverables, listed in the Table 2 below. All final deliverables will be in line with the current ADS 201, including the final evaluation report. We will prepare all metadata files in accordance with ADS 579, incorporating any subsequent revisions related to standards for submitting data to the DDL. Table 2: Deliverables & Timeline Deliverables Task # Task Name / Description Task Duration W - Days Start Due date 1 Team hiring and mobilization 5 December 2020 (Upon Concept Note approval) January 6, 2021 2 Desk review 5 January 10, 2021 January 14, 2021 3 Team Planning Workshop (Team will hold an initial meeting with USAID on the first week this workshop) 10 January 17, 2021 January 28, 2021 Deliverable 1 4 Team Planning Meeting: the team convenes with USAID/Egypt to 1) Clarify any inquires on evaluation questions; 3) finalize the timeline; 4) present and discuss methodology of evaluation, including data collection tools (English and Arabic if applicable) and guidelines; and 5) clarify any logistical and administrative procedures. 1 January 31, 2021 5 Work Plan: A detailed work plan, including final design, methodology and data analysis plan will be prepared by the team and discussed with USAID during the team planning meeting. Draft workplan and tools are submitted to USAID ahead of the TPM. After this meeting, the evaluation team will submit the final work plan which will also include the final data collection tools, interview schedule and updated list of informants. January 25 (draft tools submitted) January 27 (draft Work Plan narrative submitted) (assuming January 28 is a holiday in Egypt) 6 Team revises Work Plan and resubmits, completes pilot of tools 3 January 31, 2021 February 2, 2021 7 Data collection (2 weeks) 12 February 3, 2021 February 18, 2021 8 Data analysis workshop: the team analyzes and interpret data, prepares to conduct validation workshop with SEFPP and Team Leader starts developing the report and its annexes 10 February 20, 2021 March 6, 2021 53 | MIDTERM EVALUATION OF STRENGTHENING EGYPT’S FAMILY PLANNING PROGRAM: ANNEXES USAID.GOV 9 Validation Workshop with JSI: Team will validate and clarify preliminary findings and provide a venue for clarifications. 1 March 7, 2021 Team Leader departs Cairo, March 8, 2020 10 Draft Exit Briefing Presentation and Skeletal Report (consisting of word version of the presentation) prepared and submitted in advance of debriefings. March 11, 2021 Deliverable 2 11 Final OEH/Program Office Exit Briefing: the team delivers debriefing focusing on findings, conclusions, and recommendations (FCRs). 1 March 16, 2021 Deliverable 3 12 Final Presentation/Mission Debrief: Team delivers debriefing focusing on FCRs to a larger audience at the Mission. 1 March 18, 2021 13 Debriefing with Partners (Tentative): Present major findings to USAID partners. 1 TBC Deliverable 4 14 Draft evaluation report: submission of draft report to USAID (after internal review, copy editing and formatting). 15 March 15, 2021 April 5, 2021 15 Mission and Partner Peer Review: Evaluation team will track and respond to all draft report comments received and include changes in the final report. USAID reviews for 10 business days. 10 April 5, 2021 April 19, 2021 Deliverable 5 16 Final report: SIMPLE submits final report to USAID addressing comments received from the peer review process. Final report will comply with ADS 201. 10 April 20, 2021 May 6, 2021 USAID approval of final report (discounts Eid holiday) 10 May7, 2021 May 25, 2021 Deliverable 6 17 Metadata files and Data sets: SIMPLE prepares and submits the metadata files and data sets, as applicable and in line with ADS 579, 10 April 20, 2021 Deliverable 7 18 Draft Arabic translation of executive summary: SIMPLE translates executive summary into Arabic after approval of final report and submits it to USAID (discounts Eid holiday) 15 May 6, 2021 May 27, 2021 USAID reviews and sends comments in 10 business days (discounts American holiday) 10 May 31, 2021 June 13, 2021 Final Arabic translation of executive summary 10 June 13, 2021 June 24, 2021 Deliverable 8 19 Draft Infographic (English) of USAID Approved Executive Summary. Draft starts once the draft report is submitted. 10 April 6, 2021 April 20, 2021 USAID reviews and sends comments in 10 business days (discounts local holidays) 10 April 21, 2021 May 10, 2021 USAID.GOV MIDTERM EVALUATION OF STRENGTHENING EGYPT’S FAMILY PLANNING PROGRAM | 54 20 Final Infographic (English). SIMPLE reviews and resubmits. 10 May 10, 2021 May 24, 2021 Deliverable 9 21 Draft Infographic (Arabic). Translation begins once the final English is submitted to USAID. 10 May 24, 2021 June 7, 2021 USAID reviews and sends comments in 10 business days June 7, 2021 June 21, 2021 22 Final Infographic (Arabic) 10 June 21, 2021 July 5, 2021 USAID completes approval of all deliverables by July 18, 2021 23 Printing and posting on the DEC: SIMPLE submits printed and soft copies (via Google Drive) of all approved evaluation deliverables (report, Arabic executive summary, and infographic English and Arabic) and post those on the DEC. July 29, 2021 55 | MIDTERM EVALUATION OF STRENGTHENING EGYPT’S FAMILY PLANNING PROGRAM: ANNEXES USAID.GOV ANNEX A: ACRONYMS & TERMS USED COR Contracting Officer’s Representative DEC Development Experience Clearinghouse FCR Findings, Conclusions and Recommendations FP Family Planning FP/RH Family Planning/Reproductive Health GOE Government of Egypt JSI John Snow, Incorporated LOE Level of Effort M&E Monitoring and Evaluation MOHP Ministry of Health and Population NPC National Population Council OEH Office of Education and Health PO Program Office SEFPP Strengthening Egypt’s Family Planning Program SIMPLE Services to Improve Performance Management, Enhance Learning and Evaluation SME Subject Matter Expert SOW Scope of Work TPW/M Team Planning Workshop/Meeting USAID.GOV MIDTERM EVALUATION OF STRENGTHENING EGYPT’S FAMILY PLANNING PROGRAM | 56 ANNEX 3: LIST OF CONSULTED STAKEHOLDERS GOVERNORATE OR CENTRAL ORGANIZATION POSITION Central MOHP Senior Management, MOHP/FP Sector Central MOHP Central Administration of M&E Central MOHP Senior Management, Private sector and NGOs Department Central MOHP Clinical Supervisor –OB/GYN Department Central MOHP General Manager – Statistics Unit Central NPC Senior Management, National Population Council Central NPC Research General Manager Central NPC Planning Team Central NPC Information Center Team Central NPC General Manager -Statistics Unit Qena Governorate MOHP Senior Management, FP (Qena) Qena Governorate MOHP Senior Management- Nursing, Governorate Level, Qena Alexandria Governorate MOHP Senior Management FP – Alexandria – El Agamy Slum Luxor Governorate MOHP Senior Management FP, Governorate Level, Luxor Cairo Governorate USAID SEFPP AOR – Senior Project Management Specialist Cairo Governorate USAID Office of Education and Health (OEH) – Project Management (M&E) Specialist Cairo Governorate USAID Senior Program & Compliance Officer Cairo Governorate USAID Senior Project Management Specialist Cairo Governorate JSI Senior Management Cairo Governorate JSI M&E Team Cairo Governorate JSI Senior Management Cairo Governorate JSI Senior Technical SD Unit Cairo Governorate JSI MIS Management / Strategic Information Unit Cairo Governorate AUC Civic Engagement Unit: Senior Management Team Cairo Governorate AUC Civic Engagement Unit: Program Management Team Cairo Governorate UNFPA Senior Management Cairo Governorate JSI Program Management Cairo Governorate JSI Former SEFPP Senior Management Qena Governorate JSI SEFPP Project Management 57 | MIDTERM EVALUATION OF STRENGTHENING EGYPT’S FAMILY PLANNING PROGRAM: ANNEXES USAID.GOV GOVERNORATE OR CENTRAL ORGANIZATION POSITION Cairo Governorate JSI Former SI Unit Senior Management Cairo Governorate Pop Council International Egypt Senior Management Assuit Governorate JSI SEFPP Project Management Team Luxor Governorate JSI SEFPP Project Management Team Cairo Governorate JSI Implementation Management Team Cairo Governorate JSI Training Management Team: Clinical Cairo Governorate JSI SEFPP Field Officers at slum areas Cairo Governorate Marie Louis Senior Management Cairo Governorate Marie Louis Factory Management Cairo Governorate DKT Egypt Senior Management JSI HQ JSI Senior Research, Monitoring & Evaluation Team Cairo Governorate JSI Data Team Luxor Governorate MOHP Pharmacist Cairo Governorate MOHP Nurse Senior Management Cairo Governorate Al Ahram newspaper Journalist Qena Governorate MOHP Media Officer Qena Governorate MOHP Nurse Cairo Governorate Akhbar elyoum portal Journalist Assuit Governorate Smoking Factory Factory worker Assuit Governorate MOHP Physician Alexandria Governorate MOHP Physician Assuit Governorate MOHP Media Officer Qena Governorate MOHP Community Health Worker Cairo Governorate Teba Factory Factory worker Alexandria Governorate MOHP Physician Luxor Governorates MOHP Pharmacist Qena Governorate MOHP Pharmacist Cairo Governorate MOHP Web development Team USAID.GOV MIDTERM EVALUATION OF STRENGTHENING EGYPT’S FAMILY PLANNING PROGRAM | 58 ANNEX 4: EVALUATION METHODOLOGY This SEFPP midterm evaluation used a mixed-methods approach. The evaluation team applied qualitative research methods, including document reviews and informant interviews, complemented by an opinion survey.25 The evaluation focused on the activity’s structure, operations, and outputs and did not attempt to assess potential results and outcomes of the activity’s efforts. Reasons for the limited focus and limited use of typical qualitative and quantitative methods were two-fold: (1) SEFPP is not itself engaged in service delivery or community-level program measurement that would generate results and outcome data and information for evaluative purposes,26 and (2) constraints on visiting health facilities and service providers due to COVID-19 restrictions made it impossible to use in￾person data collection methods such as focus groups. The qualitative research for this evaluation included ten key informant interview (KII) tools designed for program policymakers, managers, and field service providers. 27 Six technical expert evaluators in sub-teams of two each coordinated with a statistician to develop a sample of individuals in specific categories to be interviewed. The sub-teams scheduled and conducted interviews, most of which averaged one hour in duration. Respondents received an advance copy of the questions to be discussed and were could choose English or Arabic for the interview. One evaluation team member moderated the discussion based on the interview questionnaire, while the other made copious notes, which were later typed and transcribed in a coded qualitative database. Most KIIs used virtual meetings, and several were conducted by phone. The quantitative research involved a questionnaire administered via phone individually and then entered digitally into SurveyMonkey.28 A random sample of respondents was drawn by a statistician/supervisor and allocated to each of four enumerators who scheduled interview appointments, conducted interviews, recorded results, and entered data into SurveyMonkey. Survey enumerators participated in a day-long pilot/practice and received daily supervision throughout the phone survey period. Survey respondents received an advance copy of the questions to ensure that they were well prepared to participate in the brief Arabic language interview, which lasted less than 10 minutes on average. There was no follow-up with survey participants. DATA SOURCES AND SAMPLING The evaluation’s sources of data and information included reviewable documents, interviewed key informants, and surveyed FP/RH professionals who had directly or indirectly participated in SEFPP activities. The population of individuals who had directly interacted with SEFPP did not lend itself to randomized sampling techniques because it was impractical to enumerate this population in advance. Instead, the evaluation used purposive sampling (also known as judgment, selective, or subjective sampling).29 Essentially, the evaluators needed to rely on technical judgments to select informants for the evaluation. The evaluators identified distinct categories of informants and beneficiaries, for which interview tools were developed. The total numbers of people chosen in each category were drawn from JSI, the implementing partner’s database, and others added by the evaluation team based on its 25 See Annex 3: List of Consulted Stakeholders. 26 SEFPP instead relies on FP program outcome data from secondary sources, where no attribution can be linked to SEFPP. 27 See Annex 5: Qualitative Data Analysis Code Book. 28 See Annex 6: Quantitative Data Analysis Charts and Tables. 29 Technically referred to as a “pre-identified non-probabilistic” sampling technique. 59 | MIDTERM EVALUATION OF STRENGTHENING EGYPT’S FAMILY PLANNING PROGRAM: ANNEXES USAID.GOV knowledge of SEFPP. The evaluators attempted to oversample all interview categories to mitigate against the limitations of purposive sampling (versus randomized sampling). The sampling situation was different for the training beneficiary population that received digital questionnaires. For this population, the evaluators did have a complete database in hand and could draw a randomized sample (building in such factors as the type of training and the geographical location of where the trainee works). Two types of training beneficiaries were invited to provide input to the digital questionnaire regarding their training experience: (1) service providers who work directly with FP clients, and (2) non-service providers such as adjunct health personnel and media professionals. A telephone survey with a sample of 764 beneficiaries allowed for an expected 30 percent non-response rate. DATA COLLECTION Data collection activities30 included: 1. A desk review31 of activity-related documents: The desk review included the SEFPP Scope of Work (SOW); Final SEFPP Concept Note; SEFPP Timeline Revised; the activity amendments; the activity annual and quarterly progress reports; Monitoring and Evaluation Plan, including performance indicator reference sheets (PIRS); the Contraceptive Security (CS) Assessment report; the gender strategy, the draft sustainability plan; and a Health Management Information System (HMIS) report, among others. In addition, JSI submitted a database of more than 4,997 trainees to the evaluation team for sampling purposes. 2. Key informant interviews: The evaluation team conducted 55 KIIs. As the data collection phase took place during the second wave of COVID-19, all KIIs were conducted virtually, by Zoom, Google Meet, or telephone. 3. Phone survey: The evaluation team conducted 599 phone surveys with training beneficiaries. The purpose of the survey was to gather overall impressions from those trained about the strategic relevance and usefulness of the training they received and the service providers’ perspective on the level of demand for and quality of FP. This evaluation did not intend to be an in-depth assessment of training content, curricula, or management of the training activities. 4. Validation meeting: A validation meeting with SEFPP staff and USAID allowed the evaluation team to receive inputs on its findings. The team incorporated feedback from the validation meeting into its findings, conclusions, and recommendations (FCRs). Table 1 below summarizes the data collection tools developed and used for the evaluation. Tools were available in both English and Arabic and administered according to the language preference of the interviewee. Table 2 below shows sampling results and data collected for both the quantitative phone survey and the qualitative KIIs. Table 3 below shows additional details of the quantitative survey by geographical coverage. Note that the Governorates of Assiut, Qena, and Luxor (plus “Others”) and two slum areas in Cairo and Alexandria were designated for the SEFPP evaluation as the geographical areas that have received the most SEFPP assistance. 30 See Annex 7: SEFPP Data Collection Tools. 31 See Annex 8: List of Reviewed Documents. USAID.GOV MIDTERM EVALUATION OF STRENGTHENING EGYPT’S FAMILY PLANNING PROGRAM | 60 TABLE 1. TARGET STAKEHOLDERS AND DATA COLLECTION TOOLS INFORMANT TOOLS USAID T0 – USAID KII MOHP – Central Level, National Population Council (NPC) T1 – Central Government KII MOHP – Governorate Level T2 – Local government KII Strategic and Operational Stakeholders T3 – Partners/stakeholders KII JSI – Management Level T4 – IP Management level KII JSI – Operational Level T5 – IP Operational/Governorate-level KII Training Recipients – Service Providers T6 – Service Providers KII T8 – Service Providers phone Survey Training Recipients – Non-service Providers T7 – Non-service Providers KII T9 – Non-service Providers Phone Survey TABLE 2. DATA COLLECTION (TARGETED VERSUS ACTUAL) INFORMANT TARGETED ACTUAL QUANTITATIVE DATA COLLECTION Training Recipients – Service Providers 410 362 (88.3%) Training Recipients – Non-Service Providers 354 237 (66.9%) Training Recipients – Total 764 599 (78.4%) QUALITATIVE DATA COLLECTION MOHP – Central Level 10 4 MOHP – Governorate Level 10 4 NPC 5 5 JSI – Management Level 14 12 JSI – Operational Level 6 6 Strategic and Operational Stakeholders 7 4 Training Recipients – Service Providers 9 8 Training Recipients – Non-Service Providers 15 8 USAID 4 4 Qualitative Informants Total 80 55 Note: the planned quantitative numbers were the sample size calculated based on a 95 percent confidence level and 5 percent margin of error, as well as an additional 30 percent to cover potential non-responses; the team met or exceeded the response rate required. Similarly, the number of KIIs conducted under each category is adequate for qualitative data collection purposes. 61 | MIDTERM EVALUATION OF STRENGTHENING EGYPT’S FAMILY PLANNING PROGRAM: ANNEXES USAID.GOV TABLE 3. QUANTITATIVE SURVEY DETAILS SERVICE PROVIDERS NON-SERVICE PROVIDERS GOVERNORATE* N % GOVERNORATE* N % Cairo 72 19.9% Cairo 124 52.3% Alexandria 50 13.8% Alexandria 36 15.2% Assiut 92 25.4% Assiut 26 11.0% Qena 78 21.5% Qena 41 17.3% Luxor 68 18.8% Luxor 9 3.8% Others** (Sohag, New Valley) 2 0.6% Others*** (Gharbia) 1 0.4% Total 362 100.0% Total 237 100.0% Non-Response Rate 30.7% Non-Response Rate 34.5% Margin of Error 4.3% Margin of Error 4.6% * The Governorates of Assiut, Qena, and Luxor (plus “Others”) and two slum areas in Cairo and Alexandria were designated for the SEFPP evaluation as the geographical areas that have received the most SEFPP assistance. ** One survey from Sohag was counted as from Assuit, as per information from the database, and one from New Valley was counted as from Luxor. *** One survey from Gharbia was counted as from Cairo, as per information from the database. DATA ANALYSIS & QUALITY ASSURANCE The evaluation team conducted descriptive statistical analyses on the quantitative data and employed a codable thematic analytical approach for the qualitative data. These methods identified explicit and implicit patterns in respondents’ answers relating to the evaluation questions. Data analysis began during the data collection phase, and the data collection phase was extended for an extra week to allow the MOHP KIIs to finalize urgent fieldwork and be available for the interviews. The evaluation team conducted a data cleaning exercise and developed a coding tree for key informants. For Evaluation Questions 1 and 2, qualitative findings derived primarily from KIIs, since interview questions were directly keyed and coded to the Thematic Focal Areas. The team reviewed findings within the context of the Thematic Focal Areas and achieved a consensus about the validity of findings through triangulation and cross-referencing reports and other documents. For Evaluation Question 3, on training beneficiaries, the evaluation team could cross-validate findings between the KIIs and phone surveys by comparing responses to questions focused on satisfaction and the usefulness of the respondents’ training experiences. A formal validation meeting with SEFPP and USAID staff took place to verify and receive the implementing partner’s feedback on the core findings. The team considered and cross-checked the IP’s inputs before finalizing conclusions and recommendations. USAID.GOV MIDTERM EVALUATION OF STRENGTHENING EGYPT’S FAMILY PLANNING PROGRAM | 62 LIMITATIONS AND MITIGATION MEASURES LIMITATIONS • The lack of data capture at the level of change (especially related to policy-level data) made it impossible to measure incremental changes and progress in the FP program or estimate SEFPP’s contribution to such changes. • Certain populations that the evaluation might have wanted to interview, such as recipients of FP services at clinic/community-level, were not accessible because of GOE restrictions on direct survey data collection. Thus, it was not possible to assess improvements in service delivery or the impact of behavior change components that might have been attributable to SEFPP. • Limitations inherent in conducting phone surveys included time constraints (such as people being engaged in “at home” activities, difficulty in reaching respondents, language difficulties, and enumerators misstating a question or misinterpreting the answer. • Inaccessibility of certain public sector statistics (such as “stock on hand”) made it impossible for SEFPP to help the MOHP estimate national program performance supply requirements. Thus, the evaluation could not adequately assess SEFPP’s contributions to contraceptive forecasting and quantification, recognizing that SEFPP does not have all of the information needed to conduct these analyses.32 • The unwillingness of some target KIIs to participate in the evaluation introduced the possibility of sampling bias because, depending on the reasons for declining to participate, a respondent might have important but negative views about the subject being evaluated that it would have been helpful to know. • Because COVID-19 restricted face-to-face interviews and group interviews, the quality of interactions with informants may have been limited. MITIGATION The midterm evaluation team anticipated these limitations and considered several measures to mitigate their influence and ensure sufficient data for evaluation. Among these measures were: • To compensate for possible limitations in the level of data available, the evaluation relied on output data and statistics that were readily available in files, plus data created through its own research. • By calculating the possible non-responses in our target populations and then oversampling by that percentage in both the qualitative KII and quantitative phone survey, a statistically significant sample was achieved in each category. • On the phone surveys, enumerators were trained and supervised throughout to avoid poor performance in conducting or notating interviews; digital data input (to SurveyMonkey) was controlled by a statistician/supervisor; respondents were contacted in advance, appointments were scheduled, and all contacts/interviews were sent advance copies of the questionnaire. Interviews were conducted in Arabic and were completed in about 10 minutes. • When challenges arose, such as missing contact information in a database, the team sought and received prompt assistance from both USAID and the IP. • The data collection phase was extended to allow more participation and ensure the availability of targeted respondents. • To compensate for the possible limitations of remote communication, phone and virtual contacts were copiously documented and digitized for recall during analysis. 32 SEFPP and MOHP need data that MOHP owns and uses to inform policy, not just secondary data. 63 | MIDTERM EVALUATION OF STRENGTHENING EGYPT’S FAMILY PLANNING PROGRAM: ANNEXES USAID.GOV ANNEX 5: QUALITATIVE DATA ANALYSIS CODE BOOK SEFPP EVALUATION QUALITATIVE DATA ANALYSIS CODE BOOK CORE CODES (Column A in Data Analysis Worksheets) SUBCODES (Column B in Data Analysis Worksheets) EQ 1 (01) Design, Relevance, and Change Theory (T -Zero) 1.1 Needs of Beneficiaries 1.2 Adapted to government capacities 1.3 Is Coop Agree. Appropriate 1.4 Gov/project objectives aligned 1.5 Synergy - other USAID projects 1.6 Past experiences/Lessons learned 1.7 Vertical LFM logical 1.8 Horizontal LFM logical 1.10 Good practices/lessons learned 1.11 Delays/challenges 1.12 Implementation mechanisms achieving results? 1.13 Add something? (02) Strategic Information (T- 1 and T- 2) 2.1 Data capture (effect on HMIS/LMIS) 2.2 Research, M&E, data usage . Areas of success (such as, HMIS/LMIS) 1.2 Success factors 1.3 MOH/NPC support/cooperation 1.4 MOHP/NPC Sustainability of capacity 1.6 Policy formulation 1.7 research/information for decision making 8 advocacy for policy change (03) Strategic Partnerships (T-3) (UN family & strategic private sector) 1.2 Benefits of the partnership 1.4 Project performance (coordination, info. Sharing) 4.1 Challenges in the partnership 1.5 (T-3) More productive partnership? 1.5 (T-1 & T-2) private sector strategic partnerships 2.9 Private sector commodities planning (04) Project Management (T-4) (Selected management issues TBD) 1.12 (T-0) Implementation mechanisms, achieving results 2.19 Sustainability TBD EQ 2 (05) Monitoring, Evaluation & Learning (MEL) (Multiple Tools) (operational/implementation level) 1.9 (T-0) MEL system adequate (learning/leadership)? 2.20 (T-4) MEL effectiveness 2.21 (T-5) Data gathering challenges 1.8, 1.9, 1.10 (T-1), (strategic level may include MEL references) (06) Operational Partnerships (T3) (AUC, Marie Louis) 1.2 Contributed to results 1.5 More productive partnership? 2.10 Peer-to-peer training (factories) 4.1 Main challenges (07) 2.4 Contraceptive Security (assessment/strategy) USAID.GOV MIDTERM EVALUATION OF STRENGTHENING EGYPT’S FAMILY PLANNING PROGRAM | 64 Service Delivery (T 4) (CS Assessment, PP IUDs, task sharing) 2.4 Conceptive Security (capacity MOHP, NPC, MOF) 2.4 Conceptive Security (introducing new methods) 2.9 Private sector commodities 2.8 Task sharing 2.12 Post-partum FP counseling/methods 2.13 Community participation strengthened (slums) (08) Social and Behavior Change Communication (SBCC) (T 4) 2.14 Capacity of HECU 2.15 Behavior mapping 2.16 Innovations in behavior change 2.17 WHO, MPHP, NPC collaboration – media messaging￾2.18 Capacity building - media (09) Training/Capacity Building Approaches (T-4) 2.5 Qualifications tracking 2.6 TOT effectiveness 2.7 E-learning approaches EQ 3 (10) Service Providers/Training Beneficiaries (T-6) 6.0 SEFPP engagement and Training details 6.1 Training quality and relevance to trainee needs 6.2 Alternative complementary resources 6.3 Training application 6.4 Sustainability 6. 2.1 Demand increase drivers / demand assessment and messages 6.6 Innovation and good practices (11) Non-Service Providers/Training Beneficiaries (T-7) 7.0 SEFPP engagement and Training details 7.1 Training quality and relevance to trainee needs 7.3 Training application 7. 2.1 Demand increase drivers / demand assessment and messages 7. 2.2 Knowledge raise community awareness and create demand 7. 2.3 Challenges 7.4 Sustainability 7.6 Innovation and good practices GENDER (Cross-cutting) (T-4 and T-5) GN 1.1 GN 1.2 GN 1.3 COVID-19 (Cross-cutting) (Multiple Tools) 4.1 COVID-19 4.2 COVID-19 6.4.1 COVID-19 7.4.1 COVID-19 65 | MIDTERM EVALUATION OF STRENGTHENING EGYPT’S FAMILY PLANNING PROGRAM: ANNEXES USAID.GOV ANNEX 6: QUANTITATIVE DATA ANALYSIS CHARTS AND TABLES PART A: SERVICE PROVIDERS TRAINING RECIPIENTS TABLE A-1: DISTRIBUTION OF SERVICE PROVIDERS BY OCCUPATION OCCUPATION N % Pharmacist 76 21.0% Physician 184 50.8% Nurse 102 28.2% Total 362 100.0% FIGURE A-1: DISTRIBUTION OF SERVICE PROVIDERS BY OCCUPATION TABLE A-2: DISTRIBUTION OF SERVICE PROVIDERS BY OCCUPATION AND WORK GOVERNORATE GOVERNORATES PHARMACIST PHYSICIAN NURSES TOTAL N % N % N % N % Cairo 18 23.7% 35 19.0% 19 18.6% 72 19.9% Alexandria 2 2.6% 32 17.4% 16 15.7% 50 13.8% Assiut 7 9.2% 54 29.3% 31 30.4% 92 25.4% Qena 16 21.1% 31 16.8% 31 30.4% 78 21.5% Luxor 33 43.4% 30 16.3% 5 4.9% 68 18.8% Others (Suhag, New Valley) 0 0.0% 2 1.1% 0 0.0% 2 0.6% Total 76 100.0% 184 100.0% 102 100.0% 362 100.0% Pharmacist 21.0% Physician 50.8% Nurse 28.2% USAID.GOV MIDTERM EVALUATION OF STRENGTHENING EGYPT’S FAMILY PLANNING PROGRAM | 66 FIGURE A-2: DISTRIBUTION OF SERVICE PROVIDERS BY WORK GOVERNORATE TABLE A-3: DISTRIBUTION OF SERVICE PROVIDERS BY PARTICIPATION IN ANY TRAINING ACTIVITY/EVENTS WITH SEFPP, DISAGGREGATED BY OCCUPATION EVER PARTICIPATED IN ANY TRAINING ACTIVITY/EVENTS WITH SEFPP PHARMACIST PHYSICIAN NURSES TOTAL N % N % N % N % Yes 71 93.4% 165 89.7% 100 98.0% 336 92.8% No 5 6.6% 19 10.3% 2 2.0% 26 7.2% Total 76 100.0% 184 100.0% 102 100.0% 362 100.0% FIGURE A-3:DISTRIBUTION OF SERVICE PROVIDERS BY PARTICIPATION IN ANY TRAINING ACTIVITY/EVENTS WITH SEFPP Cairo 19.9% Alexandria 13.8% Assiut 25.4% Qena 21.5% Luxor 18.8% Others 0.6% Yes 92.8% No 7.2% 67 | MIDTERM EVALUATION OF STRENGTHENING EGYPT’S FAMILY PLANNING PROGRAM: ANNEXES USAID.GOV TABLE A-4: DISTRIBUTION OF SERVICE PROVIDERS BY PARTICIPATION IN ANY TRAINING ACTIVITY/EVENTS WITH SEFPP, DISAGGREGATED BY WORK GOVERNORATE EVER PARTICIPATED IN ANY TRAINING ACTIVITY/EVENTS WITH SEFPP CAIRO ALEXANDRIA ASSIUT QENA LUXOR OTHERS TOTAL N % N % N % N % N % N % N % Yes 6 5 90.3% 47 94.0% 89 96.7% 7 0 89.7% 6 3 92.6% 2 100.0 % 3 3 6 92.8% No 7 9.7% 3 6.0% 3 3.3% 8 10.3% 5 7.4% 0 0.0% 2 6 7.2% Total 7 2 100.0% 50 100.0% 92 100.0% 7 8 100.0 % 6 8 100.0 % 2 100.0 % 3 6 2 100.0 % TABLE A-5: DISTRIBUTION OF SERVICE PROVIDERS BY THEIR PERSPECTIVE ON THE RELEVANCE OF TRAINING TOPICS TO THEIR WORK RESPONSIBILITIES, DISAGGREGATED BY OCCUPATION THE TRAINING TOPICS ARE RELEVANT TO THEIR WORK RESPONSIBILITIES PHARMACIST PHYSICIAN NURSES TOTAL N % N % N % N % Strongly disagree (1) 2 2.8% 1 0.6% 0 0.0% 3 0.9% Disagree (2) 4 5.6% 4 2.4% 0 0.0% 8 2.4% Neutral (3) 6 8.5% 3 1.8% 0 0.0% 9 2.7% Agree (4) 25 35.2% 30 18.2% 10 10.0% 65 19.3% Strongly agree (5) 34 47.9% 127 77.0% 90 90.0% 251 74.7% Total 71 100.0% 165 100.0% 100 100.0% 336 100.0% Mean Rank 4.2 4.7 4.9 4.6 FIGURE A-5: DISTRIBUTION OF SERVICE PROVIDERS BY THEIR PERSPECTIVE OF THE RELEVANCE OF TRAINING TOPICS TO THEIR WORK RESPONSIBILITIES 0.9% 2.4% 2.7% 19.3% 74.7% 0.00% 10.00% 20.00% 30.00% 40.00% 50.00% 60.00% 70.00% 80.00% Strongly disagree (1) Disagree (2) Neutral (3) Agree (4) Strongly agree (5) USAID.GOV MIDTERM EVALUATION OF STRENGTHENING EGYPT’S FAMILY PLANNING PROGRAM | 68 TABLE A-6: DISTRIBUTION OF SERVICE PROVIDERS BY THEIR PERSPECTIVE ON THE RELEVANCE OF TRAINING TOPICS TO THEIR WORK RESPONSIBILITIES, DISAGGREGATED BY WORK GOVERNORATE THE TRAINING TOPICS ARE RELEVANT TO THEIR WORK RESPONSIBILITIES CAIRO ALEXANDR IA ASSIUT QENA LUXOR OTHER S TOTAL N % N % N % N % N % N % N % Strongly disagree (1) 0 0.0% 0 0.0% 1 1.1% 0 0.0% 2 3.2% 0 0.0% 3 0.9% Disagree (2) 2 3.1% 1 2.1% 0 0.0% 2 2.9% 2 3.2% 1 50.0 % 8 2.4% Neutral (3) 3 4.6% 1 2.1% 1 1.1% 0.0% 4 6.3% 0 0.0% 9 2.7% Agree (4) 1 2 18.5 % 1 0 21.3% 1 5 16.9 % 1 3 18.6 % 1 5 23.8 % 0 0.0% 6 5 19.3 % Strongly agree (5) 4 8 73.8 % 3 5 74.5% 7 2 80.9 % 5 5 78.6 % 4 0 63.5 % 1 50.0 % 2 5 1 74.7 % Total 6 5 100. 0% 4 7 100.0% 8 9 100. 0% 7 0 100. 0% 6 3 100. 0% 2 100. 0% 3 3 6 100. 0% Mean Rank 4.6 4.7 4.8 4.7 4.4 3.5 4.6 FIGURE A-6: DISTRIBUTION OF SERVICE PROVIDERS BY THEIR PERSPECTIVE ON THEIR ABILITY TO APPLY THE LEARNING THEY ACQUIRED IN THEIR WORK 6.3% 6.5% 10.4% 25.0% 51.5% 0.00% 10.00% 20.00% 30.00% 40.00% 50.00% 60.00% Strongly disagree (1) Disagree (2) Neutral (3) Agree (4) Strongly agree (5) 69 | MIDTERM EVALUATION OF STRENGTHENING EGYPT’S FAMILY PLANNING PROGRAM: ANNEXES USAID.GOV TABLE A-7: DISTRIBUTION OF SERVICE PROVIDERS BY THEIR PERSPECTIVE ON WHETHER THE TRAINING THEY RECEIVED PROVIDING SUFFICIENT INFORMATION AND SKILLS, DISAGGREGATED BY OCCUPATION THE TRAINING PROVIDED THEM WITH SUFFICIENT INFORMATION AND SKILLS PHARMACIST PHYSICIAN NURSES TOTAL N % N % N % N % Strongly disagree (1) 0 0.0% 1 0.6% 0 0.0% 1 0.3% Disagree (2) 2 2.8% 3 1.8% 1 1.0% 6 1.8% Neutral (3) 1 1.4% 7 4.2% 0 0.0% 8 2.4% Agree (4) 22 31.0% 34 20.6% 10 10.0% 66 19.6% Strongly agree (5) 45 63.4% 119 72.1% 89 89.0% 253 75.3% Don’t Know 0 0.0% 1 0.6% 0 0.0% 1 0.3% Missing 1 1.4% 0.0% 0 0.0% 1 0.3% Total 71 100.0% 165 100.0% 100 100.0% 336 100.0% Mean Rank 4.6 4.6 4.9 4.7 FIGURE A-7: DISTRIBUTION OF SERVICE PROVIDERS BY THEIR PERSPECTIVE ON WHETHER THE TRAINING THEY RECEIVED PROVIDED SUFFICIENT INFORMATION AND SKILLS 0.3% 1.8% 2.4% 19.6% 75.3% 0.00% 10.00% 20.00% 30.00% 40.00% 50.00% 60.00% 70.00% 80.00% Strongly disagree (1) Disagree (2) Neutral (3) Agree (4) Strongly agree (5) USAID.GOV MIDTERM EVALUATION OF STRENGTHENING EGYPT’S FAMILY PLANNING PROGRAM | 70 TABLE A-8: DISTRIBUTION OF SERVICE PROVIDERS BY THEIR PERSPECTIVE ON WHETHER THE TRAINING THEY RECEIVED PROVIDING SUFFICIENT INFORMATION AND SKILLS, DISAGGREGATED BY WORK GOVERNORATE THE TRAINING PROVIDED THEM WITH SUFFICIENT INFORMATION AND SKILLS CAIRO ALEXANDR IA ASSIUT QENA LUXOR OTHER S TOTAL N % N % N % N % N % N % N % Strongly disagree (1) 0 0.0% 0 0.0% 1 1.1% 0 0.0% 0 0.0% 0 0.0% 1 0.3% Disagree (2) 2 3.1% 0 0.0% 1 1.1% 2 2.9% 1 1.6% 0 0.0% 6 1.8% Neutral (3) 4 6.2% 1 2.1% 2 2.2% 0 0.0% 1 1.6% 0 0.0% 8 2.4% Agree (4) 1 3 20.0 % 9 19.1% 1 3 14.6 % 1 3 18.6 % 1 7 27.0 % 1 50.0 % 6 6 19.6 % Strongly agree (5) 4 6 70.8 % 3 6 76.6% 7 1 79.8 % 5 5 78.6 % 4 4 69.8 % 1 50.0 % 2 5 3 75.3 % Don’t Know 0 0.0% 1 2.1% 0 0.0% 0 0.0% 0 0.0% 0 0.0% 1 0.3% Missing 0 0.0% 0 0.0% 1 1.1% 0 0.0% 0 0.0% 0 0.0% 1 0.3% Total 6 5 100. 0% 4 7 100.0% 8 9 100. 0% 7 0 100. 0% 6 3 100. 0% 2 100. 0% 3 3 6 100. 0% Mean Rank 4.6 4.8 4.7 4.7 4.7 4.5 4.7 FIGURE A-8: PERCENT DISTRIBUTION OF SERVICE PROVIDERS BY THEIR PERSPECTIVE ON THE CONTRIBUTION OF THE PROJECT TO INCREASING ACCESS TO QUALITY FAMILY PLANNING (FP) METHODS. 0.0% 1.5% 6.0% 28.9% 56.0% 0.00% 10.00% 20.00% 30.00% 40.00% 50.00% 60.00% Strongly disagree (1) Disagree (2) Neutral (3) Agree (4) Strongly agree (5) 71 | MIDTERM EVALUATION OF STRENGTHENING EGYPT’S FAMILY PLANNING PROGRAM: ANNEXES USAID.GOV TABLE A-9: DISTRIBUTION OF SERVICE PROVIDERS BY THEIR PERSPECTIVE ON THEIR ABILITY TO APPLY THE LEARNING THEY ACQUIRED IN THEIR WORK, DISAGGREGATED BY OCCUPATION ABILITY TO APPLY THE LEARNING THEY ACQUIRED IN THEIR WORK PHARMACIST PHYSICIAN NURSES TOTAL N % N % N % N % Strongly disagree (1) 8 11.3% 12 7.3% 1 1.0% 21 6.3% Disagree (2) 11 15.5% 8 4.8% 3 3.0% 22 6.5% Neutral (3) 14 19.7% 16 9.7% 5 5.0% 35 10.4% Agree (4) 19 26.8% 49 29.7% 16 16.0% 84 25.0% Strongly agree (5) 19 26.8% 79 47.9% 75 75.0% 173 51.5% Don’t Know 0 0.0% 1 0.6% 0 0.0% 1 0.3% Total 71 100.0% 165 100.0% 100 100.0% 336 100.0% Mean Rank 3.4 4.1 4.6 4.1 TABLE A-10: DISTRIBUTION OF SERVICE PROVIDERS BY THEIR PERSPECTIVE ON THEIR ABILITY TO APPLY THE LEARNING THEY ACQUIRED IN THEIR WORK, DISAGGREGATED BY WORK GOVERNORATE ABILITY TO APPLY THE LEARNING THEY ACQUIRED IN THEIR WORK CAIRO ALEXANDR IA ASSIUT QENA LUXOR OTHER S TOTAL N % N % N % N % N % N % N % Strongly disagree (1) 6 9.2% 1 2.1% 6 6.7% 2 2.9% 5 7.9% 1 50.0 % 2 1 6.3% Disagree (2) 7 10.8 % 3 6.4% 2 2.2% 5 7.1% 5 7.9% 0 0.0% 2 2 6.5% Neutral (3) 6 9.2% 5 10.6% 6 6.7% 9 12.9 % 9 14.3 % 0 0.0% 3 5 10.4 % Agree (4) 1 6 24.6 % 8 17.0% 2 2 24.7 % 2 2 31.4 % 1 6 25.4 % 0 0.0% 8 4 25.0 % Strongly agree (5) 3 0 46.2 % 2 9 61.7% 5 3 59.6 % 3 2 45.7 % 2 8 44.4 % 1 50.0 % 1 7 3 51.5 % Don’t Know 0 0.0% 1 2.1% 0 0.0% 0 0.0% 0 0.0% 0 0.0% 1 0.3% Total 6 5 100. 0% 4 7 100.0% 8 9 100. 0% 7 0 100. 0% 6 3 100. 0% 2 100. 0% 3 3 6 100. 0% Mean Rank 3.9 4.3 4.3 4.1 3.9 3.0 4.1 USAID.GOV MIDTERM EVALUATION OF STRENGTHENING EGYPT’S FAMILY PLANNING PROGRAM | 72 TABLE A-11: DISTRIBUTION OF SERVICE PROVIDERS BY THEIR PERSPECTIVE ON THE IMPROVEMENT OF THEIR FP DELIVERY PRACTICES FROM TRAINING, DISAGGREGATED BY OCCUPATION THE TRAINING IMPROVED THEIR FP DELIVERY PRACTICES PHARMACIST PHYSICIAN NURSES TOTAL N % N % N % N % Strongly disagree (1) 0 0.0% 3 1.8% 2 2.0% 5 1.5% Disagree (2) 6 8.5% 7 4.2% 4 4.0% 17 5.1% Neutral (3) 10 14.1% 10 6.1% 2 2.0% 22 6.5% Agree (4) 19 26.8% 63 38.2% 29 29.0% 111 33.0% Strongly agree (5) 14 19.7% 58 35.2% 57 57.0% 129 38.4% Don’t Know 22 31.0% 24 14.5% 6 6.0% 52 15.5% Total 71 100.0% 165 100.0% 100 100.0% 336 100.0% Mean Rank 3.8 4.2 4.4 4.2 FIGURE A-11: DISTRIBUTION OF SERVICE PROVIDERS BY THEIR PERSPECTIVE ON THE IMPROVEMENT OF THEIR FP DELIVERY PRACTICES BY THE TRAINING 1.5% 5.1% 6.5% 33.0% 38.4% 0.00% 5.00% 10.00% 15.00% 20.00% 25.00% 30.00% 35.00% 40.00% 45.00% Strongly disagree (1) Disagree (2) Neutral (3) Agree (4) Strongly agree (5) 73 | MIDTERM EVALUATION OF STRENGTHENING EGYPT’S FAMILY PLANNING PROGRAM: ANNEXES USAID.GOV TABLE A-12: DISTRIBUTION OF SERVICE PROVIDERS BY THEIR PERSPECTIVE ON THE IMPROVEMENT OF THEIR FP DELIVERY PRACTICES BY THE TRAINING, DISAGGREGATED BY WORK GOVERNORATE THE TRAINING IMPROVED THEIR FP DELIVERY PRACTICES CAIRO ALEXANDRI A ASSIUT QENA LUXOR OTHERS TOTAL N % N % N % N % N % N % N % Strongly disagree (1) 1 1.5% 1 2.1% 2 2.2% 1 1.4% 0 0.0% 0 0.0% 5 1.5% Disagree (2) 6 9.2% 0 0.0% 3 3.4% 3 4.3% 5 7.9% 0 0.0% 1 7 5.1% Neutral (3) 5 7.7% 3 6.4% 4 4.5% 1 1.4% 9 14.3% 0 0.0% 2 2 6.5% Agree (4) 1 5 23.1% 1 9 40.4% 3 4 38.2% 2 1 30.0% 2 1 33.3% 1 50.0% 1 1 1 33.0 % Strongly agree (5) 2 1 32.3% 2 0 42.6% 4 0 44.9% 3 3 47.1% 1 5 23.8% 0 0.0% 1 2 9 38.4 % Don’t Know 1 7 26.2% 4 8.5% 6 6.7% 1 1 15.7% 1 3 20.6% 1 50.0% 5 2 15.5 % Total 6 5 100.0 % 4 7 100.0% 8 9 100.0 % 7 0 100.0 % 6 3 100.0 % 2 100.0 % 3 3 6 100.0 % Mean Rank 4.0 4.3 4.3 4.4 3.9 4.0 4.2 TABLE A-13: DISTRIBUTION OF SERVICE PROVIDERS BY THEIR PERSPECTIVE ON THE CONTRIBUTION OF THE PROJECT TO INCREASING ACCESS TO QUALITY FP METHODS, DISAGGREGATED BY OCCUPATION THE PROJECT HAS CONTRIBUTED TO THE INCREASED ACCESS TO QUALITY FP METHODS PHARMACIST PHYSICIAN NURSES TOTAL N % N % N % N % Strongly disagree (1) 0 0.0% 0 0.0% 0 0.0% 0 0.0% Disagree (2) 3 4.2% 2 1.2% 0 0.0% 5 1.5% Neutral (3) 7 9.9% 11 6.7% 2 2.0% 20 6.0% Agree (4) 32 45.1% 46 27.9% 19 19.0% 97 28.9% Strongly agree (5) 18 25.4% 92 55.8% 78 78.0% 188 56.0% Don’t Know 11 15.5% 14 8.5% 1 1.0% 26 7.7% Total 71 100.0% 165 100.0% 100 100.0% 336 100.0 % Mean Rank 4.1 4.5 4.8 4.5 USAID.GOV MIDTERM EVALUATION OF STRENGTHENING EGYPT’S FAMILY PLANNING PROGRAM | 74 TABLE A-14: DISTRIBUTION OF SERVICE PROVIDERS BY THEIR PERSPECTIVE ON THE CONTRIBUTION OF THE PROJECT TO INCREASING ACCESS TO QUALITY FP METHODS, DISAGGREGATED BY WORK GOVERNORATE THE PROJECT HAS CONTRIBUTED TO THE INCREASED ACCESS TO QUALITY FP METHODS CAIRO ALEXANDR IA ASSIUT QENA LUXOR OTHER S TOTAL N % N % N % N % N % N % N % Strongly disagree (1) 0 0.0% 0 0.0% 0 0.0% 0 0.0% 0 0.0% 0 0.0% 0 0.0% Disagree (2) 1 1.5% 0 0.0% 0 0.0% 1 1.4% 3 4.8% 0 0.0% 5 1.5% Neutral (3) 6 9.2% 3 6.4% 5 5.6% 1 1.4% 5 7.9% 0 0.0% 2 0 6.0% Agree (4) 1 8 27.7 % 1 6 34.0% 2 1 23.6 % 1 7 24.3 % 2 5 39.7 % 0 0.0% 9 7 28.9 % Strongly agree (5) 3 0 46.2 % 2 6 55.3% 6 0 67.4 % 4 5 64.3 % 2 6 41.3 % 1 50.0 % 1 8 8 56.0 % Don’t Know 1 0 15.4 % 2 4.3% 3 3.4% 6 8.6% 4 6.3% 1 50.0 % 2 6 7.7% Total 6 5 100. 0% 4 7 100.0% 8 9 100. 0% 7 0 100. 0% 6 3 100. 0% 2 100. 0% 3 3 6 100. 0% Mean Rank 4.4 4.5 4.6 4.7 4.3 5.0 4.5 TABLE A-15: DISTRIBUTION OF SERVICE PROVIDERS BY THEIR PERSPECTIVE ON THE CONTRIBUTION OF THE PROJECT TO INCREASING DEMAND FOR FP, DISAGGREGATED BY OCCUPATION THE PROJECT HAS CONTRIBUTED TO THE INCREASE OF THE DEMAND FOR FP SERVICES PHARMACIST PHYSICIAN NURSES TOTAL N % N % N % N % Strongly disagree (1) 0 0.0% 1 0.6% 1 1.0% 2 0.6% Disagree (2) 5 7.0% 3 1.8% 1 1.0% 9 2.7% Neutral (3) 7 9.9% 10 6.1% 3 3.0% 20 6.0% Agree (4) 31 43.7% 56 33.9% 31 31.0% 118 35.1% Strongly agree (5) 8 11.3% 60 36.4% 59 59.0% 127 37.8% Don’t Know 20 28.2% 34 20.6% 5 5.0% 59 17.6% Missing 0 0.0% 1 0.6% 0 0.0% 1 0.3% Total 71 100.0% 165 100.0% 100 100.0% 336 100.0% Mean Rank 3.8 4.3 4.5 4.3 75 | MIDTERM EVALUATION OF STRENGTHENING EGYPT’S FAMILY PLANNING PROGRAM: ANNEXES USAID.GOV FIGURE A-15: PERCENT DISTRIBUTION OF SERVICE PROVIDERS BY THEIR PERSPECTIVE ON THE CONTRIBUTION OF THE PROJECT TO INCREASING DEMAND FOR FP TABLE A-16: DISTRIBUTION OF SERVICE PROVIDERS BY THEIR PERSPECTIVE ON THE CONTRIBUTION OF THE PROJECT TO INCREASING DEMAND FOR FP, DISAGGREGATED BY WORK GOVERNORATE THE PROJECT HAS CONTRIBUTED TO THE INCREASE OF THE DEMAND FOR FP SERVICES CAIRO ALEXANDRI A ASSIUT QENA LUXOR OTHERS TOTAL N % N % N % N % N % N % N % Strongly disagree (1) 0 0.0% 1 2.1% 1 1.1% 0 0.0% 0 0.0% 0 0.0% 2 0.6% Disagree (2) 3 4.6% 0 0.0% 1 1.1% 3 4.3% 2 3.2% 0 0.0% 9 2.7% Neutral (3) 4 6.2% 3 6.4% 6 6.7% 3 4.3% 4 6.3% 0 0.0% 2 0 6.0% Agree (4) 2 1 32.3% 1 8 38.3% 2 7 30.3% 2 4 34.3% 2 8 44.4% 0 0.0% 1 1 8 35.1 % Strongly agree (5) 1 7 26.2% 1 7 36.2% 4 4 49.4% 3 1 44.3% 1 7 27.0% 1 50.0% 1 2 7 37.8 % Don’t Know 2 0 30.8% 8 17.0% 9 10.1% 9 12.9% 1 2 19.0% 1 50.0% 5 9 17.6 % Missing 0 0.0% 0 0.0% 1 1.1% 0 0.0% 0 0.0% 0 0.0% 1 0.3% Total 6 5 100. 0% 4 7 100.0% 8 9 100. 0% 7 0 100. 0% 6 3 100. 0% 2 100. 0% 3 3 6 100. 0% Mean Rank 4.2 4.3 4.4 4.4 4.2 5.0 4.3 0.6% 2.7% 6.0% 35.1% 37.8% 0.00% 5.00% 10.00% 15.00% 20.00% 25.00% 30.00% 35.00% 40.00% Strongly disagree (1) Disagree (2) Neutral (3) Agree (4) Strongly agree (5) USAID.GOV MIDTERM EVALUATION OF STRENGTHENING EGYPT’S FAMILY PLANNING PROGRAM | 76 PART B: NON-SERVICE PROVIDERS TRAINING RECIPIENTS TABLE B-1. DISTRIBUTION OF NON-SERVICE PROVIDERS BY WORK GOVERNORATE GOVERNORATES TOTAL N % Cairo 124 52.3% Alexandria 36 15.2% Assiut 26 11.0% Qena 41 17.3% Luxor 9 3.8% Others (Gharbia) 1 0.4% Total 237 100.0% FIGURE B-1. DISTRIBUTION OF NON-SERVICE PROVIDERS BY WORK GOVERNORATES TABLE B-2: DISTRIBUTION OF NON-SERVICE PROVIDERS BY THEIR PARTICIPATION IN ANY TRAINING ACTIVITY/EVENTS WITH SEFPP, DISAGGREGATED BY WORK GOVERNORATE EVER PARTICIPATED IN ANY TRAINING ACTIVITY/ EVENTS WITH SEFPP CAIRO ALEXAND RIA ASSIUT QENA LUXOR OTHER S TOTAL N % N % N % N % N % N % N % Yes 1 1 0 88.7 % 3 3 91.7% 2 5 96.2 % 4 1 100.0 % 9 100.0 % 1 100.0 % 2 1 9 92.4 % No 1 4 11.3 % 3 8.3% 1 3.8% 0 0.0% 0 0.0% 0 0.0% 1 8 7.6% Total 1 2 4 100. 0% 3 6 100.0% 2 6 100. 0% 4 1 100. 0% 9 100. 0% 1 100. 0% 2 3 7 100. 0% Cairo 52.3% Alexandria 15.2% Assiut 11.0% Qena 17.3% Luxor 3.8% Others 0.4% 77 | MIDTERM EVALUATION OF STRENGTHENING EGYPT’S FAMILY PLANNING PROGRAM: ANNEXES USAID.GOV TABLE B-2:DISTRIBUTION OF NON-SERVICE PROVIDERS BY THEIR PARTICIPATION IN ANY TRAINING ACTIVITY/EVENTS WITH SEFPP TABLE B-3: DISTRIBUTION OF NON-SERVICE PROVIDERS BY THEIR PERSPECTIVE ON TRAINING TOPICS’ RELEVANCE TO THEIR WORK RESPONSIBILITIES, DISAGGREGATED BY WORK GOVERNORATE THE TRAINING TOPICS ARE RELEVANT TO THEIR WORK RESPONSIBILITIES CAIRO ALEXANDR IA ASSIUT QENA LUXOR OTHER S TOTAL N % N % N % N % N % N % N % Strongly disagree (1) 2 1.8% 0 0.0% 3 12.0% 0 0.0% 0 0.0% 0 0.0% 5 2.3% Disagree (2) 2 1.8% 2 6.1% 2 8.0% 1 2.4% 0 0.0% 0 0.0% 7 3.2% Neutral (3) 4 3.6% 1 3.0% 0.0% 0 0.0% 0 0.0% 0 0.0% 5 2.3% Agree (4) 2 4 21.8% 4 12.1% 2 8.0% 8 19.5% 3 33.3% 0 0.0% 41 18.7 % Strongly agree (5) 7 7 70.0% 2 6 78.8% 1 8 72.0% 3 2 78.0% 6 66.7% 1 100.0 % 16 0 73.1 % Don’t Know 1 0.9% 0 0.0% 0 0.0% 0 0.0% 0 0.0% 0 0.0% 1 0.5% Total 1 1 0 100. 0% 3 3 100.0% 2 5 100. 0% 4 1 100. 0% 9 100. 0% 1 100. 0% 21 9 100. 0% Mean Rank 4.6 4.6 4.2 4.7 4.7 5.0 4.6 Yes 92.4% No 7.6% USAID.GOV MIDTERM EVALUATION OF STRENGTHENING EGYPT’S FAMILY PLANNING PROGRAM | 78 FIGURE B-3: DISTRIBUTION OF NON-SERVICE PROVIDERS BY THEIR PERSPECTIVE ON TRAINING TOPICS’ RELEVANCE TO THEIR WORK RESPONSIBILITIES TABLE B-4: DISTRIBUTION OF NON-SERVICE PROVIDERS BY THEIR PERSPECTIVE ON PROVIDING SUFFICIENT INFORMATION AND SKILLS BY THE TRAINING THEY RECEIVED, DISAGGREGATED BY WORK GOVERNORATE THE TRAINING PROVIDED THEM WITH SUFFICIENT INFORMATION AND SKILLS CAIRO ALEXANDR IA ASSIUT QENA LUXOR OTHERS TOTAL N % N % N % N % N % N % N % Strongly disagree (1) 3 2.7% 0 0.0% 0 0.0% 0 0.0% 0 0.0% 0 0.0% 3 1.4% Disagree (2) 3 2.7% 2 6.1% 0 0.0% 1 2.4% 0 0.0% 0 0.0% 6 2.7% Neutral (3) 3 2.7% 2 6.1% 0 0.0% 2 4.9% 1 11.1% 0 0.0% 8 3.7% Agree (4) 2 5 22.7% 5 15.2% 1 4.0% 8 19.5% 1 11.1% 0 0.0% 4 0 18.3% Strongly agree (5) 7 4 67.3% 2 4 72.7% 2 4 96.0 % 3 0 73.2% 7 77.8% 1 100.0 % 1 6 0 73.1% Don’t Know 1 0.9% 0 0.0% 0 0.0% 0 0.0% 0 0.0% 0 0.0% 1 0.5% Missing 1 0.9% 0 0.0% 0 0.0% 0 0.0% 0 0.0% 0 0.0% 1 0.5% Total 1 1 0 100.0 % 3 3 100.0% 2 5 100. 0% 4 1 100.0 % 9 100.0 % 1 100.0 % 2 1 9 100.0 % Mean Rank 4.5 4.5 5.0 4.6 4.7 5.0 4.6 2.3% 3.2% 2.3% 18.7% 73.1% 0.00% 10.00% 20.00% 30.00% 40.00% 50.00% 60.00% 70.00% 80.00% Strongly disagree (1) Disagree (2) Neutral (3) Agree (4) Strongly agree (5) 79 | MIDTERM EVALUATION OF STRENGTHENING EGYPT’S FAMILY PLANNING PROGRAM: ANNEXES USAID.GOV FIGURE B-4: DISTRIBUTION OF NON-SERVICE PROVIDERS BY THEIR PERSPECTIVE ON PROVIDING SUFFICIENT INFORMATION AND SKILLS BY THE TRAINING THEY RECEIVED TABLE B-5: DISTRIBUTION OF NON-SERVICE PROVIDERS BY THEIR PERSPECTIVE ON THEIR ABILITY TO APPLY THE LEARNING THEY ACQUIRED IN THEIR WORK, DISAGGREGATED BY WORK GOVERNORATE ABILITY TO APPLY THE LEARNING THEY ACQUIRED IN THEIR WORK CAIRO ALEXAN DRIA ASSIUT QENA LUXOR OTHER S TOTAL N % N % N % N % N % N % N % Strongly disagree (1) 10 9.1% 2 6.1% 2 8.0% 3 7.3% 0 0.0% 0 0.0% 1 7 7.8% Disagree (2) 10 9.1% 3 9.1% 2 8.0% 1 2.4% 0 0.0% 0 0.0% 1 6 7.3% Neutral (3) 11 10.0% 0 0.0% 0 0.0% 3 7.3% 1 11.1 % 0 0.0% 1 5 6.8% Agree (4) 31 28.2% 8 24.2% 3 12.0 % 1 4 34.1 % 2 22.2 % 1 100.0 % 5 9 26.9 % Strongly agree (5) 48 43.6% 1 9 57.6% 1 8 72.0 % 2 0 48.8 % 6 66.7 % 0 0.0% 1 1 1 50.7 % Don’t Know 0 0.0% 1 3.0% 0 0.0% 0 0.0% 0 0.0% 0 0.0% 1 0.5% Total 11 0 100. 0% 3 3 100.0 % 2 5 100. 0% 4 1 100. 0% 9 100. 0% 1 100. 0% 2 1 9 100. 0% Mean Rank 3.9 4.2 4.3 4.1 4.6 4.0 4.1 1.4% 2.7% 3.7% 18.3% 73.1% 0.00% 10.00% 20.00% 30.00% 40.00% 50.00% 60.00% 70.00% 80.00% Strongly disagree (1) Disagree (2) Neutral (3) Agree (4) Strongly agree (5) USAID.GOV MIDTERM EVALUATION OF STRENGTHENING EGYPT’S FAMILY PLANNING PROGRAM | 80 FIGURE B-5: DISTRIBUTION OF NON-SERVICE PROVIDERS BY THEIR PERSPECTIVE ON THEIR ABILITY TO APPLY THE LEARNING THEY ACQUIRED IN THEIR WORK TABLE B-6: DISTRIBUTION OF NON-SERVICE PROVIDERS BY THEIR PERSPECTIVE ON THE CONTRIBUTION OF THE PROJECT TO INCREASING THE DEMAND FOR FP, DISAGGREGATED BY WORK GOVERNORATES THE PROJECT HAS CONTRIBUTED TO THE INCREASE IN DEMAND FOR FP SERVICES CAIRO ALEXAN DRIA ASSIUT QENA LUXOR OTHER S TOTAL N % N % N % N % N % N % N % Strongly disagree (1) 1 0.9% 0 0.0% 0 0.0% 0 0.0% 0 0.0% 0 0.0% 1 0.5% Disagree (2) 1 0.9% 2 6.1% 0 0.0% 1 2.4% 0 0.0% 0 0.0% 4 1.8% Neutral (3) 5 4.5% 1 3.0% 1 4.0% 1 2.4% 3 33.3 % 0 0.0% 1 1 5.0% Agree (4) 30 27.3 % 8 24.2% 5 20.0 % 5 12.2 % 2 22.2 % 0 0.0% 5 0 22.8 % Strongly agree (5) 15 13.6 % 1 4 42.4% 1 4 56.0 % 2 5 61.0 % 4 44.4 % 0 0.0% 7 2 32.9 % Don’t Know 58 52.7 % 8 24.2% 5 20.0 % 9 22.0 % 0 0.0% 1 100.0 % 8 1 37.0 % Total 11 0 100. 0% 3 3 100.0 % 2 5 100. 0% 4 1 100. 0% 9 100. 0% 1 100. 0% 2 1 9 100. 0% Mean Rank 4.1 4.4 4.7 4.7 4.1 4.4 7.8% 7.3% 6.8% 26.9% 50.7% 0.00% 10.00% 20.00% 30.00% 40.00% 50.00% 60.00% Strongly disagree (1) Disagree (2) Neutral (3) Agree (4) Strongly agree (5) 81 | MIDTERM EVALUATION OF STRENGTHENING EGYPT’S FAMILY PLANNING PROGRAM: ANNEXES USAID.GOV FIGURE B-6: DISTRIBUTION OF NON-SERVICE PROVIDERS BY THEIR PERSPECTIVE ON THE CONTRIBUTION OF THE PROJECT TO INCREASING DEMAND FOR FP 0.5% 1.8% 5.0% 22.8% 32.9% 0.00% 5.00% 10.00% 15.00% 20.00% 25.00% 30.00% 35.00% Strongly disagree (1) Disagree (2) Neutral (3) Agree (4) Strongly agree (5) USAID.GOV MIDTERM EVALUATION OF STRENGTHENING EGYPT’S FAMILY PLANNING PROGRAM | 82 ANNEX 7: SEFPP DATA COLLECTION TOOLS (ARABIC AND ENGLISH) Quantitative and Qualitative Data Collection ToolS Activity Name Strengthening Egypt’s Family Planning Program SEFPP Implementer John Snow, Incorporated Life of Activity December 2017- December 2022 Activity’s Geographic Regions Assiut, Beni Suef, Fayoum, Giza, Luxor, Minya, Qena, Sohag, and in marginalized areas in Cairo and Alexandria USAID Office Office of Health and Education LIST OF DATA COLLECTION TOOLS 5 TOOL TYPE TARGET GROUP T0 Key Informant Interview Protocol (KII) USAID T1 Key Informant Interview Protocol (KII) Ministry of Health and Population (MOHP) Central Level National Population Council (NPC) T2 Key Informant Interview Protocol (KII) Ministry of Health and Population (MOHP) Governorate Level T3 Key Informant Interview Protocol (KII) Stakeholders – Strategic and Operational Levels T4 Key Informant Interview Protocol (KII) Implementing Partner (IP) – Management T5 Key Informant Interview Protocol (KII) Implementing Partner (IP) – Field Staff Members T6 Key Informant Interview Protocol (KII) Service Providers Training Recipients T7 Key Informant Interview Protocol (KII) Training Recipients (non-service providers) T8 Phone Survey Service Providers Training Recipients (Clinicians) T9 Phone Survey Non-Service Providers Training Recipients 83 | MIDTERM EVALUATION OF STRENGTHENING EGYPT’S FAMILY PLANNING PROGRAM: ANNEXES USAID.GOV SEFPP MID-TERM EVALUATION QUALITATIVE DATA COLLECTION TOOLS T0 – KII – USAID Tool Description: Data Source (Informant Category): USAID Type of Tool KII Specific Target Informants for the Tool: SEFPP AOR, Health and Education Office M&E Officer Time per tool implementation: 90 – 120 minutes per tool administration with a respondent - net time 150 minutes per tool – including pre-and post-arrangements: Logistical Needs: Preparation: - Inviting participants - Fill in the key informant data - Observing COVID safety precautions - Ensuring electric plugs availability for laptops - Online connectivity (if needed) Materials Needed: - Hard copies of fill-in tool (for field notes) – in case of outage/technical problems - Stationary (notebooks, pens, staples, folders and envelopes, clips, etc.) Basic Data – to be filled by evaluators (per tool administration/per KII): Team Members Administering the Tool: Governorate: Date: (mm/dd/yy) District (if applicable): # of Interviewees/Participants: Total # ( ) Male ( ) Female ( ) Entity Name: Type of Entity: (USAID – IP - National Partner – Private Sector - Training Provider – Service Provider – Training Recipient - Other) Interviewee Name and Phone Number: Position: Interviewee Name and Phone Number: Position: Interviewee Name and Phone Number: Position: How long have you been in this position/area? USAID.GOV MIDTERM EVALUATION OF STRENGTHENING EGYPT’S FAMILY PLANNING PROGRAM | 84 TOOL Questions 1. Relevance 1.1 Does the project constitute an adequate response to the current needs and rights of the target groups / end beneficiaries? (needs of beneficiaries) 1.2 Is the project adapted to the present institutional, human and financial capacities of the partner government and/or other key stakeholder(s) with a role in implementation? 1.3 Is the choice of implementing modality (Cooperative Agreement) proving to be appropriate? (project modality) 1.4 Do all government stakeholders demonstrate effective commitment to the objectives of the project (i.e., ownership)? 2. Coordination and Complementarity 2.1 Is the project likely to benefit from any complementarity / synergies with other projects funded by the USAID and other entities (donors, public and private) such as information systems, contraceptive quantification, FP clinical training programs? 3. Intervention Logic, Monitoring and Learning 3.1 To what extent does the project, as currently designed and implemented, take into account past experiences in the sector, good practices and lessons learnt from previous interventions? (learning culture) 3.2 What is the current quality of the intervention logic? Are planned outputs and outcomes coherent and feasible, and have key assumptions and risks been clearly identified? 3.3 Is the horizontal logic of the Logical Framework Matrix (LFM) adequate? I.e., choice of indicators, data availability, baseline data, target values and relevant disaggregation. 3.4 Does the project have an adequate internal monitoring system? (M&E system) )(supplemental information will also be in the DQA reports) 3.5 Are there any lessons learned and good practices that would be useful to share beyond the project context? (good practices & lessons learned) 4. Efficiency 4.1 Has the project encountered any delays/challenges (e.g., data collection at community level) and was the planning revised accordingly? (activities) 4.2 Are the implementation mechanisms proving to be appropriate to achieve planned outputs and contribute to outcomes? (implementation mechanisms) Other Questions/ Ending: Is there anything you would like to add? 85 | MIDTERM EVALUATION OF STRENGTHENING EGYPT’S FAMILY PLANNING PROGRAM: ANNEXES USAID.GOV SEFPP MID-TERM EVALUATION QUALITATIVE DATA COLLECTION TOOLS T1 – KII – CENTRAL LEVEL – MOHP AND NPC Tool Description: Data Source (Informant Category): MOHP NPC Type of Tool KII Specific Target Informants for the Tool: - Head of the MOHP/FP Sector (or his /her delegate) - The Head of Central Administration of M&E - Head of Central Administration of RR - Head of Media Department - Head of CS Department - Head of Private sector and NGOs Department - Head of Training Department - Head of Information Center (Focal Person for Slum Areas) - Head of the PHC sector - Head of HECU Unit - Head of HIC at MOHP - Director of NPC (or his/her delegate) - Head of the Policy Unit at NPC Time per tool implementation: 45 - 60 minutes per tool administration with a respondent - net time 90 minutes per tool – including pre-and post-arrangements: Logistical Needs: Preparation: - Inviting participants - Fill in the key informant data - Observing COVID safety precautions - Ensuring electric plugs availability for laptops - Online connectivity (if needed) Materials Needed: - Hard copies of fill-in tool (for field notes) – in case of outage/technical problems - Stationary (notebooks, pens, staples, folders and envelopes, clips, etc.) USAID.GOV MIDTERM EVALUATION OF STRENGTHENING EGYPT’S FAMILY PLANNING PROGRAM | 86 Basic Data – to be filled by evaluators (per tool administration/per KII): Team Members Administering the Tool: Governorate: Date: (mm/dd/yy) District (if applicable): # of Interviewees/Participants: Total # ( ) Male ( ) Female ( ) Entity Name: Type of Entity: (USAID – IP - National Partner – Private Sector - Training Provider – Service Provider – Training Recipient - Other) Interviewee Name and Phone Number: Position: Interviewee Name and Phone Number: Position: Interviewee Name and Phone Number: Position: How long have you been in this position/area? Standard SEFPP Evaluation Introductory Statement: We are members of an independent evaluation team contracted by the USAID Mission in Egypt to conduct a mid-term performance evaluation of the Strengthening Egypt’s Family Planning Program (SEFPP). The findings of the evaluation will assist USAID in informing decisions regarding: 1) the effectiveness of SEFPP technical assistance approaches, 2) the most effective/sustainable approaches regarding the promotion of family planning in Egypt, And; 3) improvements that can be made by SEFPP implementers in the remaining years of the activity, including potential areas of future technical assistance based on the lessons learned from this project. Your participation is entirely voluntary, but your participation is important to the results of this evaluation. Results will be anonymized (no personally identifiable information) and will be shared with project stakeholders. TOOL Questions Introductory Questions: What is the level of engagement with SEFPP? EQ Questions: EQ1.1. Based on your work with SEFPP, where do you see the areas of success of the project? (Probing: As needed mention examples of areas of success - as related to their position and role): - Quality of FP/RH Programs - Private sector access and support to quality FP/RH service - Increased post-partum counselling - Community Participation - and its effect on strengthening the FP system - Enhanced FP behaviours among target populations - Improved data capture, data use, and data interpretation - and use for policy reformulation - Increased contraceptive security - NPC's role in shaping population and development policies 87 | MIDTERM EVALUATION OF STRENGTHENING EGYPT’S FAMILY PLANNING PROGRAM: ANNEXES USAID.GOV - The general FP climate in Egypt EQ 1.2. What factors affected the achievement of results and supported/hindered their success? (Probing: Provide examples of such factors as needed: - Flexible planning - Participatory partnership - Coordination - Structural elements - Organizational factors - Operational factors - Cultural norms EQ.1-3. How did MOHP / NCP address the challenges that SEFPP encountered ? EQ. 1.4.A What is your viewpoint on SEFPP’s approach to building the capacities of local health systems (health workforce, HMIS, health service delivery, supply chain/logistics management, etc.)? EQ. 1.4.B How effective have SEFPP CB activities been in this regard? How does it complement what MOHP/NPC is doing in this area? EQ. 1.4.C How will MOHP/NPC ensure the continued utilization of the improved capacities and systems (to ensure consistent level of access and delivery of quality FP services)? EQ 1.5 How do you envision the role of the private sector/NGOs (private clinics/ hospitals/pharmaceutical companies) in promoting the national FP program? How has SEFPP supported the role of the private sector in this regard? EQ.1.6. In your view, how has SEFPP contributed to the decision-making and policy shaping (e.g. national population strategy)? EQ. 1.7. How effective was the project in assessing your informational and research needs? Please elaborate on how SEFPP enhanced the ability of MOHP/NPC in decision-making and evidence based planning? EQ.1.8. How effective have the SEFPP advocacy actions (probe by type of advocacy actions: task-sharing, quantification, budget allocation, and public information/education, others) to date? Are there any additional areas of advocacy that the project might consider? EQ. 4.1. How has SEFPP supported MOHP in responding to COVID-19 crisis? How effective has this support been? EQ. 4.2. What other types of support that SEFPP can provide in this area? Other Questions/ Ending: Is there anything you would like to add? USAID.GOV MIDTERM EVALUATION OF STRENGTHENING EGYPT’S FAMILY PLANNING PROGRAM | 88 تقییم منتصف المدة لمشروع تعزیز برنامج مصر لتنظیم الأسرة في مصر الأدوات الكیفیة الأداة (T1 – (مقابلة مع مصدر معلومات رئیسي – وزارة الصحة والسكان – المجلس القومي للسكان - المستوى المركزي وصف الأداة: مصدر البیانات (الفئة المستھدفة): وزارة الصحة والسكان المجلس القومي للسكان نوع الأداة: مقابلة مع مصدر معلومات رئیسي مصادر المعلومات الرئیسیین للأداة: - رئیس قطاع تنظیم الأسرة بوزارة الصحة والسكان (أو من ینوب عنھ) - رئیس الإدارة المركزیة للمتابعة والتقییم - رئیس الإدارة المركزیة للرائدات الریفیات - مدیر إدارة الإعلام - مدیر إدارة تأمین وسائل تنظیم الأسرة - مدیر إدارة القطاع الخاص والجمعیات الأھلیة - مدیر إدارة التدریب - مدیر مركز المعلومات (المسئول عن المناطق العشوائیة) - رئیس قطاع الرعایة الصحیة الأولیة - مدیر وحدة الصحة والتعلیم والتواصل - مدیر المركز القومي المعلومات الصحیة - رئیس المجلس القومي للسكان (أو من ینوب عنھ) - مدیر وحدة السیاسات بالمجلس القومي للسكان 45 – 60 دقیقة لكل استخدام للأداة مع مصدر معلومات رئیسي الوقت المتوقع لتنفیذ الأداة: 90 دقیقة شاملة الترتیبات السابقة واللاحقة للمقابلة الاحتیاجات اللوجستیة: الإعداد: - دعوة المشاركین في المقابلات - ملء قائمة بیانات المشاركین/ات - التأكد من احتیاطات وشروط السلامة الخاصة بفیروس كورونا - توفیر مصدر للكھرباء (فیشات) - التوصیل بشبكة الإنترنت (إذا لزم الأمر) المستلزمات المطلوبة: - نسخ من أدوات جمع البیانات (لتسجیل محضر المقابلة) في حالة انقطاع التیار الكھربائي. - أدوات كتابیة (دفاتر، أقلام، دوسیھات حفظ، مظاریف، دبابیس، الخ) البیانات الأساسیة: (یتم ملء الخانات مع كل استخدام للأداة لكل مصدر معلومات رئیسي) أسماء فریق العمل المنفذین للأداة: ًا / أو تحدید المحافظة: التاریخ: (یوم/ شھر / سنة) المركز (لو انطبق): مركزی عدد المشاركین في المقابلة: الإجمالي ( ) رجال ( ) نساء ( ) اسم الجھة: نوع الجھة: 89 | MIDTERM EVALUATION OF STRENGTHENING EGYPT’S FAMILY PLANNING PROGRAM: ANNEXES USAID.GOV (الوكالة الأمریكیة للتنمیة – الشریك المنفذ – الشریك القومي – القطاع الخاص – مقدمي التدریب – مقدمي الخدمة - متلقو التدریب – آخرون: تحدید ...) اسم الشخص ورقم الھاتف: الوظیفة داخل المنظمة / مدة الوجود بالمنصب :اسم الشخص ورقم الھاتف الوظیفة داخل المنظمة / مدة الوجود بالمنصب :اسم الشخص ورقم الھاتف الوظیفة داخل المنظمة / مدة الوجود بالمنصب الأداة (T1 – (مقابلة شخصیة مع مصدر معلومات رئیسي – المستوى المركزي تقدیم وافتتاحیة: صباح الخیر، أنا اسمي--------------------: و معي زمیلي/زمیلتي في العمل ---------------------------- . نحن أعضاء فریق تقییم مستقل تعاقدت معھ الوكالة الأمریكیة للتنمیة الدولیة في مصر لإجراء تقییم منتصف المدة لأداء برنامج تعزیز تنظیم الأسرة في مصر ستساعد نتائج التقییم الوكالة الأمریكیة للتنمیة الدولیة بمصر على اتخاذ قرارات فـیما یخص: 1) فعالیة مناھج المساعدة الفنیة المستخدمة من المشروع 2) النھج الأكثر فعالیة / استدامة فیما یتعلق بتعزیز تنظیم الأسرة في مصر بشكل عام 3) التحسینات التي یمكن لمنفذي المشروع القیام بھا في سنواتھ المتبقیة بما یتضمن مجالات المساعدة الفنیة المحتملة المبنیة على الدروس المستفادة من المشروع. ً مشاركتك معنا الیوم طوعیة تماما ولكنھا ھامة للغایة لنتائج ھذا التقییم. لن یتم الافصاح عن أي بیانات شخصیة للمشاركین/ات فى ھذا التقییم و لكن سیتم مشاركة نتائج التقییم مع أصحاب المصلحة في المشروع. أسئلة الأداة: أسئلة افتتاحیة: ما مدى مشاركتكم في مشروع تعزیز برنامج مصر لتنظیم الأسرة؟ الأسئلة المتعلقة بأسئلة التقییم: س 1-1 بناء على خبرة عملكم مع مشروع تعزیز برنامج مصر لتنظیم الأسرة (المشروع)، ما ھي النجاحات التي حققھا البرنامج؟ (للمناقشة یمكن ذكر أمثلة لبعض المجالات من القائمة التالیة – حسب وظیفة ودور المشارك): - جودة برامج تنظیم الأسرة والصحة الإنجابیة - وصول القطاع الخاص - ازدیاد المشورة في فترة النفاس - مشاركة مجتمعیة - تعزیز السلوكیات الداعمة لتنظیم الأسرة بین قطاع المستھدفین - جمع واستخدام وتفسیر البیانات واستخدامھا في صیاغة السیاسات - زیادة تأمین وسائل تنظیم الأسرة - دور المجلس القومي للسكان في وضع السیاسات السكانیة والتنمویة - المناخ العام لتنظیم الأسرة في مصر س 2-1 ما ھي العوامل التي ساھمت في أو أعاقت الوصول للنتائج؟ USAID.GOV MIDTERM EVALUATION OF STRENGTHENING EGYPT’S FAMILY PLANNING PROGRAM | 90 (من أمثلة ھذه العوامل للمناقشة: - التخطیط المرن - الشراكة - التنسیق - عوامل ھیكلیة - عوامل تشغیلیة - الأعراف س 3-1 كیف واجھت وزارة الصحة والسكان والمجلس القومي للسكان التحدیات التي واجھت المشروع? س 4-1 –أ. ما ھي وجھة نظرك في النھج الذي استخدمھ المشروع في بناء قدرات النظم الصحیة المحلیة (القوى العاملة في المجال الصحي، نظام إدارة المعلومات الصحیة، تقدیم الخدمات الصحیة، سلسلة الإمداد/إدارة الخدمات اللوجستیة، وغیرھا)؟ س 4-1 –ب. ما مدى فعالیة أنشطة بناء القدرات بالمشروع في ھذا الصدد؟ كیف تكمل تلك الأنشطة عمل وزارة الصحة والمجلس القومي للسكان في ھذا المجال؟ س 4-1 – ج. كیف ستضمن وزارة الصحة والسكان والمجلس القومي للسكان استمراریة استخدام النظم والقدرات المطورة (لضمان مستوى ثابت من إتاحة وتقدیم خدمات تنظیم أسرة ذات جودة)؟ س 5-1 كیف ترى دور القطاع الخاص / (العیادات الخاصة / المستشفیات / شركات الأدویة) /المنظمات غیر الحكومیة في تعزیز البرنامج القومي لتنظیم الأسرة؟ كیف دعم المشروع دور القطاع الخاص في ھذا الصدد؟ س 6-1 من وجھة نظرك، كیف ساھم المشروع في صنع القرار وتشكیل السیاسات (مثل الاستراتیجیة القومیة للسكان)؟ س 7-1 ما مدى فعالیة المشروع في تقییم احتیاجاتك المعلوماتیة والبحثیة؟ یرجى توضیح كیفیة تعزیز المشروع لقدرة وزارة الصحة والمجلس القومي للسكان في صنع القرار والتخطیط القائم على المعلومات؟ س 8-1 ما مدى فعالیة إجراءات الدعوة الخاصة بالمشروع (التعمق في المناقشة حول أنواع أنشطة الدعوة مثل مشاركة المھام، والتحدید الكمي، وتخصیص الموازنة، ونشر المعلومات والتوعیة، وغیرھا) حتى الآن؟ ھل یمكن للمشروع الأخذ في الاعتبار أي مجالات إضافیة في ھذا المجال؟ س 1-4 كیف دعم المشروع وزارة الصحة في الاستجابة لجائحة فیروس كورونا المستجد (COVID19-(؟ ما مدى فعالیة ھذا الدعم؟ س 4 2- ما ھي أنواع الدعم الأخرى التي یمكن أن یقدمھا المشروع في ھذا المجال؟ أسئلة أخرى / إنھاء المقابلة: ھل ترغب في إضافة شيء آخر؟ 91 | MIDTERM EVALUATION OF STRENGTHENING EGYPT’S FAMILY PLANNING PROGRAM: ANNEXES USAID.GOV SEFPP MID-TERM EVALUATION QUALITATIVE DATA COLLECTION TOOLS T2 – KII – GOVERNORATE LEVEL – MOHP Tool Description: Data Source (Informant Category): MOHP Type of Tool KII Specific Target Informants for the Tool: - Undersecretary of MOHP - Head of Private sector and NGOs Department at the Governorate Level - Director of FP at Governorate Level - PHC Director at Governorate Level - Curative Medicine Director at Governorate Level - Head of Nursing Department at the Governorate Level - Head of RR Department Time per tool implementation: 45 - 60 minutes per tool administration with a respondent - net time 90 minutes per tool – including pre-and post-arrangements: Logistical Needs: Preparation: - Inviting participants - Fill in the key informant data - Observing COVID safety precautions - Ensuring electric plugs availability for laptops - Online connectivity (if needed) Materials Needed: - Hard copies of fill-in tool (for field notes) – in case of outage/technical problems - Stationary (notebooks, pens, staples, folders and envelopes, clips, etc.) USAID.GOV MIDTERM EVALUATION OF STRENGTHENING EGYPT’S FAMILY PLANNING PROGRAM | 92 Basic Data – to be filled by evaluators (per tool administration/per KII): Team Members Administering the Tool: Governorate: Date: (mm/dd/yy) District (if applicable): # of Interviewees/Participants: Total # ( ) Male ( ) Female ( ) Entity Name: Type of Entity: (USAID – IP - National Partner – Private Sector - Training Provider – Service Provider – Training Recipient - Other) Interviewee Name and Phone Number: Position: Interviewee Name and Phone Number: Position: Interviewee Name and Phone Number: Position: How long have you been in this position/area? Standard SEFPP Evaluation Introductory Statement: We are members of an independent evaluation team contracted by the USAID Mission in Egypt to conduct a mid-term performance evaluation of the Strengthening Egypt’s Family Planning Program (SEFPP). The findings of the evaluation will assist USAID in informing decisions regarding: 1) the effectiveness of SEFPP technical assistance approaches, 2) the most effective/sustainable approaches regarding the promotion of family planning in Egypt, and; 3) improvements that can be made by SEFPP implementers in the remaining years of the activity, including potential areas of future technical assistance based on the lessons learned from this project. Your participation is entirely voluntary, but your participation is important to the results of this evaluation. Results will be anonymized (no personally identifiable information) and will be shared with project stakeholders. TOOL Questions Introductory Questions: What is the level of engagement with SEFPP? EQ Questions: EQ1.1. Based on your work with SEFPP, where do you see the areas of success of the project? (Probing: As needed mention examples of areas of success - as related to their position and role): - Quality of FP/RH Programs - Private sector access and support to quality FP/RH service - Increased post-partum counselling - Community Participation - and its effect on strengthening the FP system - Enhanced FP behaviours among target populations - Improved data capture, data use, and data interpretation - and use for policy reformulation - Increased contraceptive security - NPC's role in shaping population and development policies - The general FP climate in Egypt EQ 1.2. What factors affected the achievement of results and supported/hindered their success? 93 | MIDTERM EVALUATION OF STRENGTHENING EGYPT’S FAMILY PLANNING PROGRAM: ANNEXES USAID.GOV (Probing: Provide examples of such factors as needed: - Flexible planning - Participatory partnership - Coordination - Structural elements - Organizational factors - Operational factors - Cultural norms EQ.1-3. How did MOHP / NCP address the challenges that SEFPP encountered and hindered the achievement of results? EQ. 1.4.A What is your viewpoint on SEFPP approach to building the capacities of local health systems (health workforce, HMIS, health service delivery, supply chain/logistics management, etc.)? EQ. 1.4.B How effective have SEFPP Capacity Building activities been in this regard? How does it complement what MOHP/NPC is doing in this area? EQ. 1.4.C How will MOHP/NPC ensure the continued utilization of the improved capacities and systems (to ensure consistent level of access and delivery of quality FP services)? EQ 1.5 How do you envision the role of the private sector/NGOs (private clinics/ hospitals/pharmaceutical companies) in promoting the national FP program? How has SEFPP supported the role of the private sector in this regard? EQ.1.6. In your view, how has SEFPP contributed to the decision-making and policy shaping (e.g. national population strategy)? EQ. 1.7. How effective was the project in assessing your informational and research needs? Please elaborate on how SEFPP enhanced the ability of MOHP/NPC in decision-making and evidence based planning? EQ.1.8. How effective have the SEFPP advocacy actions (probe by type of advocacy actions: task-sharing, quantification, budget allocation, and public information/education, others) to date? Are there any additional areas of advocacy that the project might consider? EQ. 4.1. How has SEFPP supported MOHP in responding to COVID-19 crisis? How effective has this support been? EQ. 4.2. What other types of support that SEFPP can provide in this area? Other Questions/ Ending: Is there anything you would like to add? USAID.GOV MIDTERM EVALUATION OF STRENGTHENING EGYPT’S FAMILY PLANNING PROGRAM | 94 تقییم منتصف المدة لمشروع تعزیز برنامج مصر لتنظیم الأسرة في مصر الأدوات الكیفیة الأداة (T2 – (مقابلة مع مصدر معلومات رئیسي – المستوى المجافظة وصف الأداة: مصدر البیانات (الفئة المستھدفة): وزارة الصحة والسكان نوع الأداة: مقابلة مع مصدر معلومات رئیسي مصادر المعلومات الرئیسیین للأداة: - ةكلاء وزارة الصحة بالحافظات - مدیر إدارة القطاع الخاص والجمعیات الأھلیة - رئیس قطاع تنظیم الأسرة بالمحافظات - رئیس قطاع الرعایة الصحیة الأولیة بالمحافظات - رئیس الطب العلاجي بالمحافظات - مدیر إدارات التمریض بالمحافظات -مدیر إدارة الرائدات الریفیات بالمحافظات 45 – 60 دقیقة لكل استخدام للأداة مع مصدر معلومات رئیسي الوقت المتوقع لتنفیذ الأداة: 90 دقیقة شاملة الترتیبات السابقة واللاحقة للمقابلة الاحتیاجات اللوجستیة: الإعداد: - دعوة المشاركین في المقابلات - ملء قائمة بیانات المشاركین/ات - التأكد من احتیاطات وشروط السلامة الخاصة بفیروس كورونا - توفیر مصدر للكھرباء (فیشات) - التوصیل بشبكة الإنترنت (إذا لزم الأمر) المستلزمات المطلوبة: - نسخ من أدوات جمع البیانات (لتسجیل محضر المقابلة) في حالة انقطاع التیار الكھربائي. - أدوات كتابیة (دفاتر، أقلام، دوسیھات حفظ، مظاریف، دبابیس، الخ) البیانات الأساسیة: (یتم ملء الخانات مع كل استخدام للأداة لكل مصدر معلومات رئیسي) أسماء فریق العمل المنفذین للأداة: ًا / أو تحدید المحافظة: التاریخ: (یوم/ شھر / سنة) المركز (لو انطبق): مركزی عدد المشاركین في المقابلة: الإجمالي ( ) رجال ( ) نساء ( ) نوع الجھة: (الوكالة الأمریكیة للتنمیة – الشریك المنفذ – الشریك القومي – القطاع الخاص – مقدمي التدریب – مقدمي الخدمة - متلقو التدریب – آخرون: تحدید ...) اسم الجھة: اسم الشخص ورقم الھاتف: الوظیفة داخل المنظمة / مدة الوجود بالمنصب :اسم الشخص ورقم الھاتف الوظیفة داخل المنظمة / مدة الوجود بالمنصب :اسم الشخص ورقم الھاتف الوظیفة داخل المنظمة / مدة الوجود بالمنصب 95 | MIDTERM EVALUATION OF STRENGTHENING EGYPT’S FAMILY PLANNING PROGRAM: ANNEXES USAID.GOV الأداة (T2 – (مقابلة مع مصدر معلومات رئیسي – وزارة الصحة على مستوى المحافظة تقدیم وافتتاحیة: صباح الخیر، أنا اسمي--------------------: و معي زمیلي/زمیلتي في العمل ---------------------------- . نحن أعضاء فریق تقییم مستقل تعاقدت معھ الوكالة الأمریكیة للتنمیة الدولیة في مصر لإجراء تقییم منتصف المدة لأداء برنامج تعزیز تنظیم الأسرة في مصر ستساعد نتائج التقییم الوكالة الأمریكیة للتنمیة الدولیة بمصر على اتخاذ قرارات فـیما یخص: 1) فعالیة مناھج المساعدة الفنیة المستخدمة من المشروع 2) النھج الأكثر فعالیة / استدامة فیما یتعلق بتعزیز تنظیم الأسرة في مصر بشكل عام 3) التحسینات التي یمكن لمنفذي المشروع القیام بھا في سنواتھ المتبقیة بما یتضمن مجالات المساعدة الفنیة المحتملة المبنیة على الدروس المستفادة من المشروع. ً مشاركتك معنا الیوم طوعیة تماما ولكنھا ھامة للغایة لنتائج ھذا التقییم. لن یتم الافصاح عن أي بیانات شخصیة للمشاركین/ات فى ھذا التقییم و لكن سیتم مشاركة نتائج التقییم مع أصحاب المصلحة في المشروع. أسئلة الأداة: أسئلة افتتاحیة: ما مدى مشاركتكم في مشروع تعزیز برنامج مصر لتنظیم الأسرة؟ الأسئلة المتعلقة بأسئلة التقییم: س 1-1 بناء على خبرة عملكم مع مشروع تعزیز برنامج مصر لتنظیم الأسرة (المشروع)، ما ھي النجاحات التي حققھا البرنامج؟ (للمناقشة یمكن ذكر أمثلة لبعض المجالات من القائمة التالیة – حسب وظیفة ودور المشارك): - جودة برامج تنظیم الأسرة والصحة الإنجابیة - وصول القطاع الخاص - ازدیاد المشورة في فترة النفاس - مشاركة مجتمعیة - تعزیز السلوكیات الداعمة لتنظیم الأسرة بین قطاع المستھدفین - جمع واستخدام وتفسیر البیانات واستخدامھا في صیاغة السیاسات - زیادة تأمین وسائل تنظیم الأسرة - دور المجلس القومي للسكان في وضع السیاسات السكانیة والتنمویة - المناخ العام لتنظیم الأسرة في مصر س 2-1 ما ھي العوامل التي ساھمت في أو أعاقت الوصول للنتائج؟ (من أمثلة ھذه العوامل للمناقشة: - التخطیط المرن - الشراكة - التنسیق - عوامل ھیكلیة - عوامل تشغیلیة - الأعراف المجتمعیھ س 3-1 كیف واجھت وزارة الصحة والسكان والمجلس القومي للسكان التحدیات التي واجھت المشروع وأعاقت الوصول لنتائجھ؟ س 4-1 –أ. ما ھي وجھة نظرك في النھج الذي استخدمھ المشروع في بناء قدرات النظم الصحیة المحلیة (القوى العاملة في المجال الصحي، نظام إدارة المعلومات الصحیة، تقدیم الخدمات الصحیة، سلسلة الإمداد/إدارة الخدمات اللوجستیة، وغیرھا)؟ USAID.GOV MIDTERM EVALUATION OF STRENGTHENING EGYPT’S FAMILY PLANNING PROGRAM | 96 س 4-1 –ب. ما مدى فعالیة أنشطة بناء القدرات بالمشروع في ھذا الصدد؟ كیف تكمل تلك الأنشطة عمل وزارة الصحة والمجلس القومي للسكان في ھذا المجال؟ س 4-1 – ج. كیف ستضمن وزارة الصحة والسكان والمجلس القومي للسكان استمراریة استخدام النظم والقدرات المطورة (لضمان مستوى ثابت من إتاحة وتقدیم خدمات تنظیم أسرة ذات جودة)؟ س 5-1 كیف ترى دور القطاع الخاص / المنظمات غیر الحكومیة (العیادات الخاصة / المستشفیات / شركات الأدویة) في تعزیز البرنامج القومي لتنظیم الأسرة؟ كیف دعم المشروع دور القطاع الخاص في ھذا الصدد؟ س 6-1 من وجھة نظرك، كیف ساھم المشروع في صنع القرار وتشكیل السیاسات (مثل الاستراتیجیة القومیة للسكان)؟ س 7-1 ما مدى فعالیة المشروع في تقییم احتیاجاتك المعلوماتیة والبحثیة؟ یرجى توضیح كیفیة تعزیز المشروع لقدرة وزارة الصحة والمجلس القومي للسكان في صنع القرار والتخطیط القائم على المعلومات؟ س 8-1 ما مدى فعالیة إجراءات الدعوة الخاصة بالمشروع (التعمق في المناقشة حول أنواع أنشطة الدعوة مثل مشاركة المھام، والتحدید الكمي، وتخصیص الموازنة، ونشر المعلومات والتوعیة، وغیرھا) حتى الآن؟ ھل یمكن للمشروع الأخذ في الاعتبار أي مجالات إضافیة في ھذا المجال؟ س 1-4 كیف دعم المشروع وزارة الصحة في الاستجابة لجائحة فیروس كورونا المستجد (COVID19-(؟ ما مدى فعالیة ھذا الدعم؟ س 4 2- ما ھي أنواع الدعم الأخرى التي یمكن أن یقدمھا المشروع في ھذا المجال؟ أسئلة أخرى / إنھاء المقابلة: ھل ترغب في إضافة شيء آخر؟ 97 | MIDTERM EVALUATION OF STRENGTHENING EGYPT’S FAMILY PLANNING PROGRAM: ANNEXES USAID.GOV SEFPP MID-TERM EVALUATION QUALITATIVE DATA COLLECTION TOOLS T3 – KII – STAKEHOLDERS – STRATEGIC AND OPERATIONAL LEVELS Tool Description: Data Source (Informant Category): 1. Strategic Level 2. Operational Level Type of Tool KII Specific Target Informants for the Tool: Strategic Level Stakeholders: - WHO - UNFPA Operational Level Stakeholders: - Marie Louis - Microsoft - DKT - Egyptian Family Planning Association (EFPA) Time per tool implementation: 45 - 60 minutes per tool administration with a respondent - net time 90 minutes per tool – including pre-and post-arrangements: Logistical Needs: Preparation: - Inviting participants - Fill in the key informant data - Observing COVID safety precautions - Ensuring electric plugs availability for laptops - Online connectivity (if needed) Materials Needed: - Hard copies of fill-in tool (for field notes) – in case of outage/technical problems - Stationary (notebooks, pens, staples, folders and envelopes, clips, etc.) USAID.GOV MIDTERM EVALUATION OF STRENGTHENING EGYPT’S FAMILY PLANNING PROGRAM | 98 Basic Data – to be filled by evaluators (per tool administration/per KII): Team Members Administering the Tool: Governorate: Date: (mm/dd/yy) District (if applicable): # of Interviewees/Participants: Total # ( ) Male ( ) Female ( ) Entity Name: Type of Entity: (USAID – IP - National Partner – Private Sector - Training Provider – Service Provider – Training Recipient - Other) Interviewee Name and Phone Number: Position: Interviewee Name and Phone Number: Position: Interviewee Name and Phone Number: Position: How long have you been in this position/area? Standard SEFPP Evaluation Introductory Statement: We are members of an independent evaluation team contracted by the USAID Mission in Egypt to conduct a mid-term performance evaluation of the Strengthening Egypt’s Family Planning Program (SEFPP). The findings of the evaluation will assist USAID in informing decisions regarding: 1) the effectiveness of SEFPP technical assistance approaches, 2) the most effective/sustainable approaches regarding the promotion of family planning in Egypt, and 3) improvements that can be made by SEFPP implementers in the remaining years of the activity, including potential areas of future technical assistance based on the lessons learned from this project. Your participation is entirely voluntary, but your participation is important to the results of this evaluation. Results will be anonymized (no personally identifiable information) and will be shared with project stakeholders. TOOL Questions Introductory Questions: What is the level of engagement with SEFPP? EQ Questions: EQ1.2. How has this partnership contributed to the achievements of SEFPP intended results? How has this partnership benefited your organization? EQ1.4. How would you assess the project performance in terms of: responsiveness, coordination, integrated actions and sharing information? Examples are requested EQ4.1. – What have been the main challenges that affected the partnership (Including COVID-19) and how were these challenges mitigated? EQ.1-5. What would you recommend to enhance more productive partnership? What are other areas of engagement with SEFPP may be possible in the future? 99 | MIDTERM EVALUATION OF STRENGTHENING EGYPT’S FAMILY PLANNING PROGRAM: ANNEXES USAID.GOV EQ. 4.1. How has SEFPP supported MOHP in responding to COVID-19 crisis? How effective has this support been? EQ. 4.2. What other types of support that SEFPP can provide in this area? Other Questions/ Ending: Is there anything you would like to add? USAID.GOV MIDTERM EVALUATION OF STRENGTHENING EGYPT’S FAMILY PLANNING PROGRAM | 100 تقییم منتصف المدة لمشروع تعزیز برنامج مصر لتنظیم الأسرة في مصر الأدوات الكیفیة الأداة (3T – (مقابلة مع مصدر معلومات رئیسي – الأطراف الفاعلة وصف الأداة: مصدر البیانات (الفئة المستھدفة): .1 المستوى الاستراتیجي 2ز المستوى التنفیذي نوع الأداة: مقابلة مع مصدر معلومات رئیسي مصادر المعلومات الرئیسیین للأداة: الفاعلین على المستوى الاستراتیجي: - منظمة الصحة العالمیة - صندوق الامم المتحدة للسكان الفاعلین على المستوى التنفیذي: - مصنع ماري لویس للأزیاء - شركة مایكروسوفت - منظمة DKT - الجمعیة المصریة لتنظیم الأسرة 45 – 60 دقیقة لكل استخدام للأداة مع مصدر معلومات رئیسي الوقت المتوقع لتنفیذ الأداة: 90 دقیقة شاملة الترتیبات السابقة واللاحقة للمقابلة الاحتیاجات اللوجستیة: الإعداد: - دعوة المشاركین في المقابلات - ملء قائمة بیانات المشاركین/ات - التأكد من احتیاطات وشروط السلامة الخاصة بفیروس كورونا - توفیر مصدر للكھرباء (فیشات) - التوصیل بشبكة الإنترنت (إذا لزم الأمر) المستلزمات المطلوبة: - نسخ من أدوات جمع البیانات (لتسجیل محضر المقابلة) في حالة انقطاع التیار الكھربائي. - أدوات كتابیة (دفاتر، أقلام، دوسیھات حفظ، مظاریف، دبابیس، الخ) البیانات الأساسیة: (یتم ملء الخانات مع كل استخدام للأداة لكل مصدر معلومات رئیسي) أسماء فریق العمل المنفذین للأداة: ًا / أو تحدید المحافظة: التاریخ: (یوم/ شھر / سنة) المركز (لو انطبق): مركزی عدد المشاركین في المقابلة: الإجمالي ( ) رجال ( ) نساء ( ) نوع الجھة: (الوكالة الأمریكیة للتنمیة – الشریك المنفذ – الشریك القومي – القطاع الخاص – مقدمي التدریب – مقدمي الخدمة - متلقو التدریب – آخرون: تحدید ...) اسم الجھة: اسم الشخص ورقم الھاتف: الوظیفة داخل المنظمة / مدة الوجود بالمنصب :اسم الشخص ورقم الھاتف الوظیفة داخل المنظمة / مدة الوجود بالمنصب :اسم الشخص ورقم الھاتف الوظیفة داخل المنظمة / مدة الوجود بالمنصب الأداة (3T – (مقابلة مع مصدر معلومات رئیسي – الأطراف الفاعلة على المستوى الاستراتیجي والتنفیذي 101 | MIDTERM EVALUATION OF STRENGTHENING EGYPT’S FAMILY PLANNING PROGRAM: ANNEXES USAID.GOV تقدیم وافتتاحیة: صباح الخیر، أنا اسمي--------------------: و معي زمیلي/زمیلتي في العمل ---------------------------- . نحن أعضاء فریق تقییم مستقل تعاقدت معھ الوكالة الأمریكیة للتنمیة الدولیة في مصر لإجراء تقییم منتصف المدة لأداء برنامج تعزیز تنظیم الأسرة في مصر ستساعد نتائج التقییم الوكالة الأمریكیة للتنمیة الدولیة بمصر على اتخاذ قرارات فـیما یخص: 1) فعالیة مناھج المساعدة الفنیة المستخدمة من المشروع 2) النھج الأكثر فعالیة / استدامة فیما یتعلق بتعزیز تنظیم الأسرة في مصر بشكل عام 3) التحسینات التي یمكن لمنفذي المشروع القیام بھا في سنواتھ المتبقیة بما یتضمن مجالات المساعدة الفنیة المحتملة المبنیة على الدروس المستفادة من المشروع. ً مشاركتك معنا الیوم طوعیة تماما ولكنھا ھامة للغایة لنتائج ھذا التقییم. لن یتم الافصاح عن أي بیانات شخصیة للمشاركین/ات فى ھذا التقییم و لكن سیتم مشاركة نتائج التقییم مع أصحاب المصلحة في المشروع. أسئلة الأداة: أسئلة افتتاحیة: ما مدى مشاركتكم في مشروع تعزیز برنامج مصر لتنظیم الأسرة؟ الأسئلة المتعلقة بأسئلة التقییم: س 2-1 كیف ساھمت تلك الشراكة في تحقیق نتائج المشروع؟ وكیف أفادت تلك الشراكة منظمتك / شركتك؟ س 4-1 كیف تقیم اداء المشروع فیما یخص الاستجابة - التنسیق – الانشطة التكاملیة – مشاركة المعلومات ؟ برجاء سرد امثلھ. س -4 1 ما ھي التحدیات الرئیسیة التي اثرت على تلك الشراكة (متضمنھ جائحة كورونا) وكیف تم التعامل مع تلك التحدیات؟ س 5-1 ما ھي توصیاتك لتعزیز تلك الشراكة؟ وما ھي مجالات الشراكة الاخرى المحتملة التي یمكن العمل علیھا مع المشروع فى المستقبل؟ أسئلة أخرى / إنھاء المقابلة: ھل ترغب في إضافة شيء آخر؟ USAID.GOV MIDTERM EVALUATION OF STRENGTHENING EGYPT’S FAMILY PLANNING PROGRAM | 102 SEFPP MID-TERM EVALUATION QUALITATIVE DATA COLLECTION TOOLS T4 – KII – IMPLEMENTING PARTNER (IP) – MANAGEMENT Tool Description: Data Source (Informant Category): JSI – Management Type of Tool KII Specific Target Informants for the Tool: - SEFPP CoP - Former CoP - SEFPP DCoP - SEFPP Units Directors - Former JSI Units Directors - Policy and advocacy Manager - HMIS Manager / Strategic Information unit - M&E and Research & Learning unit/PPP - Senior Technical SD Unit Director - Contraceptive Security Unit Director/Manager - Communication unit / SBCC unit (former Director) - Training Manager: Clinical / Implementation Time per tool implementation: 90 – 120 minutes per tool administration with a respondent - net time 150 minutes per tool – including pre-and post-arrangements: Logistical Needs: Preparation: - Inviting participants - Fill in the key informant data - Observing COVID safety precautions - Ensuring electric plugs availability for laptops - Online connectivity (if needed) Materials Needed: - Hard copies of fill-in tool (for field notes) – in case of outage/technical problems - Stationary (notebooks, pens, staples, folders and envelopes, clips, etc.) 103 | MIDTERM EVALUATION OF STRENGTHENING EGYPT’S FAMILY PLANNING PROGRAM: ANNEXES USAID.GOV Basic Data – to be filled by evaluators (per tool administration/per KII): Team Members Administering the Tool: Governorate: Date: (mm/dd/yy) District (if applicable): # of Interviewees/Participants: Total # ( ) Male ( ) Female ( ) Entity Name: Type of Entity: (USAID – IP - National Partner – Private Sector - Training Provider – Service Provider – Training Recipient - Other) Interviewee Name and Phone Number: Position: Interviewee Name and Phone Number: Position: Interviewee Name and Phone Number: Position: How long have you been in this position/area? Standard SEFPP Evaluation Introductory Statement: We are members of an independent evaluation team contracted by the USAID Mission in Egypt to conduct a mid-term performance evaluation of the Strengthening Egypt’s Family Planning Program (SEFPP). The findings of the evaluation will assist USAID in informing decisions regarding: 1) the effectiveness of SEFPP technical assistance approaches, 2) the most effective/sustainable approaches regarding the promotion of family planning in Egypt, and 3) improvements that can be made by SEFPP implementers in the remaining years of the activity, including potential areas of future technical assistance based on the lessons learned from this project. Your participation is entirely voluntary, but your participation is important to the results of this evaluation. Results will be anonymized (no personally identifiable information) and will be shared with project stakeholders. TOOL Questions Introductory Questions: Briefly describe your role in SEFPP. EQ Questions: O1- Strengthen evidence-based planning EQ 2.1To what extent was the project effective in strengthening data capture for NPC & MOHP's Central Administration and MOHP/FP Sector HMIS/ logistics management information system (LMIS) units? Any challenges and how did the project address them? EQ 2.2 How successful was the project support to (1) MOHP and NPC research, M&E, and IT staff to make use of available data and (2) build capacity to interpret and visualize results for policy formulation? What have been the challenges? Share examples for both. EQ 2.4 CS Strategic Level. What are the project contributions in terms of: (1) development of RH Contraceptive Security Strategy, (2) CB of MOHP, NPC, MOF, and other stakeholders in national quantification, and development of a supply plan that includes financing and resource mobilization, & (3) feasibility of introducing new FP methods Q2 - Quality FP/RH services supported USAID.GOV MIDTERM EVALUATION OF STRENGTHENING EGYPT’S FAMILY PLANNING PROGRAM | 104 EQ 2.5 Training. How has the project supported the MoHP/FP sector in terms of the FP Trainer qualification tracking and updating (central office)? Examples EQ 2.6 Training. How effective is the project FP TOT interventions targeting FP nurse supervisors and newly hired physician? EQ 2.7 Training. How did the project strengthen e-learning systems and modules on MoHP/FP sector website for in￾service training of all FP providers (physicians, pharmacists, nurses, and nurse/midwives)? EQ 2.8 Training. What is the current status of FP CB and promoting (advocating for) task-sharing of nurses/nurse midwives? Any challenges and how did the project address them? EQ 2.9 PPP. How successful was the project in increasing private sector access to high-quality RH/FP services (e.g., 1- planning and available for FP commodities in the private sector -2- collaboration with DKT Egypt)? EQ 2.10 PPP. How effective were the private sector partnerships; in terms of (1) integration of peer-to-peer FP training program into the factory’s vocational school curriculum, (2) TOT to create a cadre of trainers to promote FP awareness activities with factory peers and in their community with families, friends, and neighbours. (sustainability) EQ 2.12 SD (Service Delivery). How effective was the project in enhancing postpartum and post-abortion (miscarriage) FP counselling and methods? EQ 2.13 SD. To what extent has the project task force strengthened FP systems through community participation in selected slum areas? (all targeted governorates) (improving community standards of living and health status and encouraging developmental projects) (sustainability)- Success Stories O3 - Selected health behaviors enhanced EQ 2.14 SBCC (Social and Behaviour Change Communication) . How successful was the project in building capacities of Media Officers/Media Professionals? To what extent the Media Officers/Media Professionals were able to influence health behaviours? (sustainability) Success Stories shared by Media Officers/News articles shared by Media Professionals EQ 2.15 SBCC. Project faced challenges in conducting the behavioural mapping exercise of women and men of reproductive age on behavioural barriers to FP uptake for all methods, including LARC, to inform segmentation strategy. How did the project address this hurdle? Any project design adjustments (restrictions imposed by the GOE on access to data and conducting research)? EQ 2.16 SBCC. What are the most effective, innovative and diverse communication and outreach channels used by the project to engage and influence targeted population behaviour? Provide examples EQ 2.17 SBCC. How effective has the collaboration been with WHO, MoHP & NPC in engaging & developing the capacities of “influential media professionals”? EQ 2.18 SBCC. How effective was the Capacity Building on FP/RH topics of the media practitioners in providing the information and skills needed to write about family planning now and long after the program ends (sustainability)? Share Media Articles, if possible. EQ 2.19 CoP/DCoP. How did the project incorporate sustainability measures in its interventions? EQ 2.20 CoP/DCoP. How effective was the project MEL system in informing project management and USAID decision making? What can be improved? GN 1.1 CoP/DCoP SI. What is the contribution of this project to achieving gender equality outcomes? GN 1.2 SEFPP Functional Units. What is the contribution of your unit to achieving gender equality outcomes? EQ. 4.1. COVID-19. How did the project address C-19 major changes & challenges in activities design and implementation pertaining to: (1) the work environment and accommodating the GOE regulation changes (e.g., C-19), (2) continuing SEFPP support of the MOHP/FP Sector at the governorate level? Please give examples. (Examples changing FP training to online mode, postponed refurbishment of government training centers) Other Questions/ Ending: What are your recommendations for the remaining project period (start date Dec 2017 – end date Dec 2022) Is there anything you would like to add? 105 | MIDTERM EVALUATION OF STRENGTHENING EGYPT’S FAMILY PLANNING PROGRAM: ANNEXES USAID.GOV SEFPP MID-TERM EVALUATION QUALITATIVE DATA COLLECTION TOOLS T5 – KII – IMPLEMENTING PARTNER (IP) – FIELD STAFF MEMBERS Tool Description: Data Source (Informant Category): JSI – Field Staff Type of Tool KII Specific Target Informants for the Tool: - SEFPP Project Managers at governorate level (per governorate) - SEFPP Field Officers at slum areas (in Cairo & Alex) - SEFPP Data Officer (at SI Unit) Time per tool implementation: 90 – 120 minutes per tool administration with a respondent - net time 150 minutes per tool – including pre-and post-arrangements: Logistical Needs: Preparation: - Inviting participants - Fill in the key informant data - Observing COVID safety precautions - Ensuring electric plugs availability for laptops - Online connectivity (if needed) Materials Needed: - Hard copies of fill-in tool (for field notes) – in case of outage/technical problems - Stationary (notebooks, pens, staples, folders and envelopes, clips, etc.) USAID.GOV MIDTERM EVALUATION OF STRENGTHENING EGYPT’S FAMILY PLANNING PROGRAM | 106 Basic Data – to be filled by evaluators (per tool administration/per KII): Team Members Administering the Tool: Governorate: Date: (mm/dd/yy) District (if applicable): # of Interviewees/Participants: Total # ( ) Male ( ) Female ( ) Entity Name: Type of Entity: (USAID – IP - National Partner – Private Sector - Training Provider – Service Provider – Training Recipient - Other) Interviewee Name and Phone Number: Position: Interviewee Name and Phone Number: Position: Interviewee Name and Phone Number: Position: How long have you been in this position/area? Standard SEFPP Evaluation Introductory Statement: We are members of an independent evaluation team contracted by the USAID Mission in Egypt to conduct a mid-term performance evaluation of the Strengthening Egypt’s Family Planning Program (SEFPP). The findings of the evaluation will assist USAID in informing decisions regarding: 1) the effectiveness of SEFPP technical assistance approaches, 2) the most effective/sustainable approaches regarding the promotion of family planning in Egypt, and 3) improvements that can be made by SEFPP implementers in the remaining years of the activity, including potential areas of future technical assistance based on the lessons learned from this project. Your participation is entirely voluntary, but your participation is important to the results of this evaluation. Results will be anonymized (no personally identifiable information) and will be shared with project stakeholders. TOOL Questions Introductory Questions: Briefly describe your role in SEFPP. EQ Questions: Q2 - Quality FP/RH services supported EQ 2.5 Training. How has the project supported the MoHP/FP sector in terms of the FP Trainer qualification tracking and updating (governorate level)? Examples EQ 2.6 Training. How effective is the project FP TOT interventions targeting FP nurse supervisors and newly hired physician? (governorate level) EQ 2.7 Training. How effective were the e-learning courses conducted to FP providers (physicians, pharmacists, nurses, and nurse/midwives)? EQ 2.8 Training. What is the current status of FP Capacity Building and promoting (advocating for) task-sharing of nurses/nurse midwives? Any challenges and how did the project address them? (Assiut governorate pilot) EQ 2.10 PPP. How effective were the private sector partnerships; in terms of (1) integration of peer-to-peer FP training program into the factory’s vocational school curriculum, (2) TOT to create a cadre of trainers to promote FP awareness activities with factory peers and in their community with families, friends, and neighbors. (sustainability) 107 | MIDTERM EVALUATION OF STRENGTHENING EGYPT’S FAMILY PLANNING PROGRAM: ANNEXES USAID.GOV EQ 2.12 SD. How effective was the project in enhancing postpartum and post-abortion (miscarriage) FP counselling and methods? (Cairo, Alex, & Assiut Governorates) EQ 2.13 SD. To what extent has the project task force strengthened FP systems through community participation in selected slum areas? (all targeted governorates) (improving community standards of living and health status and encouraging developmental projects) (sustainability)- Success Stories O3 - Selected health behaviors enhanced EQ 2.14 SBCC. How successful was the project in building capacities of Media Officers/Media Professionals? To what extent the Media Officers/Media Professionals were able to influence health behaviours? (sustainability) Success Stories shared by Media Officers/News articles shared by Media Professionals EQ 2.16 SBCC. What are the most effective, innovative and diverse communication and outreach channels used by the project to engage and influence targeted population behaviour? Provide examples EQ 2.21 MEL. What challenges do you face in gathering data for monitoring purposes? What are the mitigation measures to address those challenges? Your suggestions for further improvements. GN 1.3 What measures do you follow to ensure gender equality considered in all implemented activities? EQ. 4.1. COVID-19. How did the project address C-19 major changes & challenges in activities design and implementation pertaining to: (1) the work environment and accommodating the GOE regulation changes (e.g., C-19), (2) continuing SEFPP support of the MOHP/FP Sector at the governorate level? (Examples changing FP training to online mode, postponed refurbishment of government training centers) Other Questions/ Ending: What are your recommendations for the remaining project period (start date Dec 2017 – end date Dec 2022) Is there anything you would like to add? USAID.GOV MIDTERM EVALUATION OF STRENGTHENING EGYPT’S FAMILY PLANNING PROGRAM | 108 تقییم منتصف المدة لمشروع تعزیز برنامج مصر لتنظیم الأسرة في مصر الأدوات الكیفیة الأداة (T5 – (مقابلة مع مصدر معلومات رئیسي – إدارة المشروع – الفریق التنفیذي على مستوى المحافظة وصف الأداة: مصدر البیانات (الفئة المستھدفة): إدارة المشروع – الفریق التنفیذي على مستوى المحافظة نوع الأداة: مقابلة مع مصدر معلومات رئیسي مصادر المعلومات الرئیسیین للأداة: - مدراء المشاریع على مستوى المحافظة - المسؤول بالمناطق العشوائیة (في القاھرة والإسكندریة) - مسؤول البیانات في وحدة المعلومات الاستراتیجیة (Unit Information Strategic ( 90 - 120 دقیقة لكل استخدام للأداة مع مصدر معلومات رئیسي الوقت المتوقع لتنفیذ الأداة: 150 دقیقة شاملة الترتیبات السابقة واللاحقة للمقابلة الاحتیاجات اللوجستیة: الإعداد: - دعوة المشاركین في المقابلات - ملء قائمة بیانات المشاركین/ات - التأكد من احتیاطات وشروط السلامة الخاصة بفیروس كورونا - توفیر مصدر للكھرباء (فیشات) - التوصیل بشبكة الإنترنت (إذا لزم الأمر) المستلزمات المطلوبة: - نسخ من أدوات جمع البیانات (لتسجیل محضر المقابلة) في حالة انقطاع التیار الكھربائي. - أدوات كتابیة (دفاتر، أقلام، دوسیھات حفظ، مظاریف، دبابیس، الخ) البیانات الأساسیة: (یتم ملء الخانات مع كل استخدام للأداة لكل مصدر معلومات رئیسي) أسماء فریق العمل المنفذین للأداة: ًا / أو تحدید المحافظة: التاریخ: (یوم/ شھر / سنة) المركز (لو انطبق): مركزی عدد المشاركین في المقابلة: الإجمالي ( ) رجال ( ) نساء ( ) نوع الجھة: (الوكالة الأمریكیة للتنمیة – الشریك المنفذ – الشریك القومي – القطاع الخاص – مقدمي التدریب – مقدمي الخدمة - متلقو التدریب – آخرون: تحدید ...) اسم الجھة: اسم الشخص ورقم الھاتف: الوظیفة داخل المنظمة / مدة الوجود بالمنصب :اسم الشخص ورقم الھاتف الوظیفة داخل المنظمة / مدة الوجود بالمنصب :اسم الشخص ورقم الھاتف الوظیفة داخل المنظمة / مدة الوجود بالمنصب الأداة (T5 – (مقابلة مع مصدر معلومات رئیسي – إدارة المشروع – الفریق التنفیذي تقدیم وافتتاحیة: 109 | MIDTERM EVALUATION OF STRENGTHENING EGYPT’S FAMILY PLANNING PROGRAM: ANNEXES USAID.GOV صباح الخیر، أنا اسمي--------------------: و معي زمیلي/زمیلتي في العمل ---------------------------- . نحن أعضاء فریق تقییم مستقل تعاقدت معھ الوكالة الأمریكیة للتنمیة الدولیة في مصر لإجراء تقییم منتصف المدة لأداء برنامج تعزیز تنظیم الأسرة في مصر ستساعد نتائج التقییم الوكالة الأمریكیة للتنمیة الدولیة بمصر على اتخاذ قرارات فـیما یخص: 1) فعالیة مناھج المساعدة الفنیة المستخدمة من المشروع 2) النھج الأكثر فعالیة / استدامة فیما یتعلق بتعزیز تنظیم الأسرة في مصر بشكل عام 3) التحسینات التي یمكن لمنفذي المشروع القیام بھا في سنواتھ المتبقیة بما یتضمن مجالات المساعدة الفنیة المحتملة المبنیة على الدروس المستفادة من المشروع. ً مشاركتك معنا الیوم طوعیة تماما ولكنھا ھامة للغایة لنتائج ھذا التقییم. لن یتم الافصاح عن أي بیانات شخصیة للمشاركین/ات فى ھذا التقییم و لكن سیتم مشاركة نتائج التقییم مع أصحاب المصلحة في المشروع. أسئلة الأداة: أسئلة افتتاحیة: صف دورك في مشروع تعزیز برنامج مصر لتنظیم الأسرة (SEFPP(بإیجاز. الأسئلة المتعلقة بأسئلة التقییم: نتیجة 2 - جودة خدمات الصحة الإنجابیة وتنظیم الأسرة س 5-2 - التدریب. كیف دعم المشروع وزارة الصحة والسكان / قطاع تنظیم الأسرة من حیث متابعة وتطویر كفاءة مدربي تنظیم الأسرة (على مستوى المحافظة)؟ أمثلة ًا؟ س 6-2 – التدریب. ما مدى فاعلیة برنامج تدریب المدربین لتنظیم الأسرة لمشرفات الممرضات والأطباء المعینون حدیث (على مستوى المحافظة) س 7-2 - التدریب. ما مدى فاعلیة دورات التعلم عن بعد (التعلم الإلكتروني) التي تم تقدیمھا لمقدمي برامج تنظیم الأسرة (الأطباء والصیادلة والممرضات / القابلات)؟ س 8-2 - التدریب. ما ھو الوضع الحالي لبناء قدرات تنظیم الأسرة وتعزیز (الدعوة إلى) تقاسم المھام مع الممرضات / القابلات الممرضات؟ ھل كانت ھناك تحدیات وكیف واجھھا المشروع؟ (تجربة محافظة أسیوط) س 10-2 .PPP- ما مدى فاعلیة شراكات القطاع الخاص ؛ من حیث (1) دمج برنامج تدریب تنظیم الأسرة مثقفي الأقران في منھج المدرسة المھنیة للمصنع ، (2) تدریب المدربین لتكوین كادر من المدربین لتعزیز أنشطة التوعیة ببرنامج تنظیم الأسرة مع أقرانھم في المصنع وفي مجتمعھم مع العائلات والأصدقاء ، والجیران. (الاستدامة) س 12-2 – .SDما مدى فاعلیة المشروع في ازدیاد المشورة و وسائل تنظیم الأسرة ف فترة النفاس وبعد الإجھاض. (محافظات القاھرة والإسكندریة وأسیوط) س 13-2 – .SD إلى أي مدى دعم فریق عمل المشروع أنظمة تنظیم الأسرة من خلال مشاركة المجتمع في المناطق العشوائیة المختارة؟ (جمیع المحافظات المستھدفة) (تحسین مستوى المعیشة للمجتمع والحالة الصحیة وتشجیع المشاریع التنمویة) (الاستدامة) - قصص نجاح نتیجة 3 - تعزیز السلوكیات الصحیة المختارة س 14-2 - .SBCC ما مدى نجاح المشروع في بناء قدرات المسؤولین الإعلامیین / الإعلامیین المھنیون؟ إلى أي مدى استطاع المسؤولون الإعلامیون / الإعلامیون المھنیون التأثیر على السلوكیات الصحیة؟ (الاستدامة) قصص النجاح التي شاركھا المسؤولون الإعلامیون / المقالات الإخباریة التي شاركھا الإعلامیون المھنیون. س 16-2 .SBCC ما ً ھي قنوات الاتصال والتواصل الأكثر فاعلیة وابتكار ً ا وتنوعا التي یستخدمھا المشروع لإشراك المستھدفین من السكان والتأثیر فd سلوكھم؟ قدم أمثلة س 21-2 – .MEL ما ھي التحدیات التي تواجھھا في جمع البیانات لأغراض المتابعة؟ ما ھي تدابیر التخفیف لمواجھة تلك التحدیات؟ اقتراحاتك لمزید من التحسینات. أسئلة أخرى / إنھاء المقابلة: USAID.GOV MIDTERM EVALUATION OF STRENGTHENING EGYPT’S FAMILY PLANNING PROGRAM | 110 س 1-4 – -19COVID - كیف عالج المشروع التغییرات والتحدیات الرئیسیة لجائحة كورونا في تصمیم الأنشطة وتنفیذھا المتعلقة بـ: (1) بیئة العمل واستیعاب التعلیمات التنظیمیة للحكومة المصریة، (2) استمرار دعم المشروع لوزارة الصحة والسكان / قطاع تنظیم الأسرة على مستوى المحافظة؟ (أمثلة على تغییر تدریب تنظیم الأسرة إلى الوضع عبر الإنترنت، وتأجیل تجدید مراكز التدریب الحكومیة) أسئلة أخرى / إنھاء المقابلة: توصیات لفترة المشروع المتبقیة (تاریخ البدء دیسمبر 2017 - تاریخ الانتھاء دیسمبر 2022) ھل ترغب في إضافة شيء آخر؟ 111 | MIDTERM EVALUATION OF STRENGTHENING EGYPT’S FAMILY PLANNING PROGRAM: ANNEXES USAID.GOV SEFPP MID-TERM EVALUATION QUALITATIVE DATA COLLECTION TOOLS T6 – KII – SERVICE PROVIDERS TRAINING RECIPIENTS Tool Description: Data Source (Informant Category): Service Providers Training Recipients Type of Tool KII Specific Target Informants for the Tool: - Public service providers (e.g., physicians, nurses, nursing supervisors, midwives, RR, GYN/OP, Pharmacists, administrative staff). - Private service providers (e.g., private clinics, private hospitals). Time per tool implementation: 45 – 60 minutes per tool administration with a respondent - net time 90 minutes per tool – including pre-and post-arrangements: Logistical Needs: Preparation: - Inviting participants - Fill in the key informant data - Observing COVID safety precautions - Ensuring electric plugs availability for laptops - Online connectivity (if needed) Materials Needed: - Hard copies of fill-in tool (for field notes) – in case of outage/technical problems - Stationary (notebooks, pens, staples, folders and envelopes, clips, etc.) USAID.GOV MIDTERM EVALUATION OF STRENGTHENING EGYPT’S FAMILY PLANNING PROGRAM | 112 Basic Data – to be filled by evaluators (per tool administration/per KII): Team Members Administering the Tool: Governorate: Date: (mm/dd/yy) District (if applicable): # of Interviewees/Participants: Total # ( ) Male ( ) Female ( ) Entity Name: Type of Entity: (USAID – IP - National Partner – Private Sector - Training Provider – Service Provider – Training Recipient - Other) Interviewee Name and Phone Number: Position: Interviewee Name and Phone Number: Position: Interviewee Name and Phone Number: Position: How long have you been in this position/area? Standard SEFPP Evaluation Introductory Statement: We are members of an independent evaluation team contracted by the USAID Mission in Egypt to conduct a mid-term performance evaluation of the Strengthening Egypt’s Family Planning Program (SEFPP). The findings of the evaluation will assist USAID in informing decisions regarding: 1) the effectiveness of SEFPP technical assistance approaches, 2) the most effective/sustainable approaches regarding the promotion of family planning in Egypt, and 3) improvements that can be made by SEFPP implementers in the remaining years of the activity, including potential areas of future technical assistance based on the lessons learned from this project. Your participation is entirely voluntary, but your participation is important to the results of this evaluation. Results will be anonymized (no personally identifiable information) and will be shared with project stakeholders. TOOL Questions Introductory Questions: How are you engaged in SEFPP project? Can you specify the training activities you participated in under SEFPP? EQ Questions: EQ3.1. How would you assess the relevance of received capacity building (e.g., training, coaching, incentive programs, policies, IT resources) to your needs and practice jobs? EQ3.2. What alternative resources (e.g., institutional supports and programs, other agencies support) have you identified and utilized as a complement to what you already have from the project? EQ.3.3. How did the knowledge and skills received translate into practice in relationship to enhancing access/quality of FP services? In case of receiving multiple training, which training program is the most beneficial? and why? Provide examples. E.Q.3.4. In your opinion, what are the chances of sustaining the FP services accessibility (such as affordability) and quality? Do you have the required resources to apply and sustain the learnings derived from the capacity building? What gaps are still there? 113 | MIDTERM EVALUATION OF STRENGTHENING EGYPT’S FAMILY PLANNING PROGRAM: ANNEXES USAID.GOV E.Q.2.1. What can drive and increase demand in FP methods in your community? What tools do you have to assess the demand? How do you shape your messages to create demand? E.Q.3.6. What are the innovations and best practices that have proven to be effective and worth replicating in the accessibility and delivery of quality FP services? E.Q.4.1. How has COVID-19 affected the training programs and your ability to deliver services? How the project contributed to addressing those challenges? Other Questions/ Ending: Is there anything you would like to add? USAID.GOV MIDTERM EVALUATION OF STRENGTHENING EGYPT’S FAMILY PLANNING PROGRAM | 114 تقییم منتصف المدة لمشروع تعزیز برنامج مصر لتنظیم الأسرة في مصر الأدوات الكیفیة الأداة (T6 – (مقابلة شخصیة – مقدمي الخدمات المتلقین للتدریب وصف الأداة: مصدر البیانات (الفئة المستھدفة): مقدمي الخدمات المتلقین للتدریب نوع الأداة: مقابلة مع مصدر معلومات رئیسي مصادر المعلومات الرئیسیین للأداة: - مقدمي الخدمات من القطاع العام (الاطباء و الممرضات و مشرفي الممرضات و الرائدات الریفیات و قابلات و أخصائي أمراض النساء والتولید والصیادلة والموظفین الاداریین) - مقدمي الخدمات من القطاع الخاص (العیادات الخاصة والمستشفیات الخاصة) 45 – 60 دقیقة لكل استخدام للأداة مع مصدر معلومات رئیسي الوقت المتوقع لتنفیذ الأداة: 90 دقیقة شاملة الترتیبات السابقة واللاحقة للمقابلة الاحتیاجات اللوجستیة: الإعداد: - دعوة المشاركین في المقابلات - ملء قائمة بیانات المشاركین/ات - التأكد من احتیاطات وشروط السلامة الخاصة بفیروس كورونا - توفیر مصدر للكھرباء (فیشات) - التوصیل بشبكة الإنترنت (إذا لزم الأمر) المستلزمات المطلوبة: - نسخ من أدوات جمع البیانات (لتسجیل محضر المقابلة) في حالة انقطاع التیار الكھربائي. - أدوات كتابیة (دفاتر، أقلام، دوسیھات حفظ، مظاریف، دبابیس، الخ) البیانات الأساسیة: (یتم ملء الخانات مع كل استخدام للأداة لكل مصدر معلومات رئیسي) أسماء فریق العمل المنفذین للأداة: ًا / أو تحدید المحافظة: التاریخ: (یوم/ شھر / سنة) المركز (لو انطبق): مركزی عدد المشاركین في المقابلة: الإجمالي ( ) رجال ( ) نساء ( ) نوع الجھة: (الوكالة الأمریكیة للتنمیة – الشریك المنفذ – الشریك القومي – القطاع الخاص – مقدمي التدریب – مقدمي الخدمة - متلقو التدریب – آخرون: تحدید ...) اسم الجھة: اسم الشخص ورقم الھاتف: الوظیفة داخل المنظمة / مدة الوجود بالمنصب :اسم الشخص ورقم الھاتف الوظیفة داخل المنظمة / مدة الوجود بالمنصب :اسم الشخص ورقم الھاتف الوظیفة داخل المنظمة / مدة الوجود بالمنصب الأداة (T6 – (مقابلة مع مصدر معلومات رئیسي – مقدمي الخدمة المتلقین للتدریب تقدیم وافتتاحیة: صباح الخیر، 115 | MIDTERM EVALUATION OF STRENGTHENING EGYPT’S FAMILY PLANNING PROGRAM: ANNEXES USAID.GOV أنا اسمي--------------------: و معي زمیلي/زمیلتي في العمل ---------------------------- . نحن أعضاء فریق تقییم مستقل تعاقدت معھ الوكالة الأمریكیة للتنمیة الدولیة في مصر لإجراء تقییم منتصف المدة لأداء برنامج تعزیز تنظیم الأسرة في مصر ستساعد نتائج التقییم الوكالة الأمریكیة للتنمیة الدولیة بمصر على اتخاذ قرارات فـیما یخص: 1) فعالیة مناھج المساعدة الفنیة المستخدمة من المشروع 2) النھج الأكثر فعالیة / استدامة فیما یتعلق بتعزیز تنظیم الأسرة في مصر بشكل عام 3) التحسینات التي یمكن لمنفذي المشروع القیام بھا في سنواتھ المتبقیة بما یتضمن مجالات المساعدة الفنیة المحتملة المبنیة على الدروس المستفادة من المشروع. ً مشاركتك معنا الیوم طوعیة تماما ولكنھا ھامة للغایة لنتائج ھذا التقییم. لن یتم الافصاح عن أي بیانات شخصیة للمشاركین/ات في ھذا التقییم ولكن سیتم مشاركة نتائج التقییم مع أصحاب المصلحة في المشروع. أسئلة الأداة: أسئلة افتتاحیة: سؤال افتتاحي: ما ھو شكل مشاركتكم في مشروع تعزیز برنامج مصر لتنظیم الأسرة؟ ھل یمكن تحدید الأنشطة التدریبیة التي شاركت فیھا في إطار مشروع تعزیز برنامج مصر لتنظیم الأسرة؟ الأسئلة المتعلقة بأسئلة التقییم: س 1-3 ما ھو تقییمك لملائمة برامج بناء القدرات التي حصلتم علیھا (على سبیل المثال، التدریب، والتوجیھ، وبرامج التحفیز، والسیاسات، وموارد تكنولوجیا المعلومات) لاحتیاجاتك وممارسة مھام وظیفتك؟ س 2-3 ما ھي الموارد الأخرى (على سبیل المثال، الدعم المؤسسي والبرامج، ودعم الوكالات الأخرى) التي حددتھا واستخدمتھا كمكمل لما لدیك بالفعل من المشروع؟ س 3-3 كیف قمت بتحویل المعارف والمھارات التي حصلت علیھا إلى ممارسات فیما یتعلق بتحسین إمكانیة الوصول وتعزیز جودة خدمات تنظیم الأسرة؟ في حالة تلقي تدریبات متعددة، ما ھو البرنامج التدریبي الأكثر فائدة؟ ولماذا؟ یرجى تقدیم أمثلة. س 4-3 في رأیك، ما ھي فرص استدامة تحسین إمكانیة الوصول إلى خدمات تنظیم الأسرة (مثل القدرة على تحمل التكالیف) وتحسین جودة تلك الخدمات؟ ھل لدیك الموارد المطلوبة لتطبیق الدروس المستفادة من بناء القدرات والحفاظ علیھا؟ ما ھي الفجوات التي لا تزال موجودة؟ س 1-2 ما الذي یمكنھ أن یحفز ویزید الطلب على وسائل تنظیم الأسرة في مجتمعك؟ ما ھي الأدوات التي لدیك لتقییم الطلب؟ كیف تصیغ رسائلك لخلق الطلب على تلك الخدمات؟ س 6-3 ما ھي الابتكارات وأفضل الممارسات التي أثبتت فعالیتھا وتستحق تكرارھا في تعزیز إمكانیة الوصول وتقدیم خدمات تنظیم الأسرة عالیة الجودة؟ س 1-4 كیف أثرت جائحة كورونا على برامج التدریب وقدرتك على تقدیم الخدمات؟ كیف ساھم المشروع في مواجھة تلك التحدیات؟ أسئلة أخرى / إنھاء المقابلة: ھل ترغب في إضافة شيء آخر؟ USAID.GOV MIDTERM EVALUATION OF STRENGTHENING EGYPT’S FAMILY PLANNING PROGRAM | 116 SEFPP MID-TERM EVALUATION QUALITATIVE DATA COLLECTION TOOLS T7 – KII – TRAINING RECIPIENTS (NON-SERVICE PROVIDERS) Tool Description: Data Source (Informant Category): Training Recipients (non-service providers) Type of Tool KII Specific Target Informants for the Tool: Non-service delivery personnel: - Media officers - Media professionals - HMIS information staff - Peer educator - NGOs volunteers - NPC administrative staff Time per tool implementation: 45 – 60 minutes per tool administration with a respondent - net time 90 minutes per tool – including pre-and post-arrangements: Logistical Needs: Preparation: - Inviting participants - Fill in the key informant data - Observing COVID safety precautions - Ensuring electric plugs availability for laptops - Online connectivity (if needed) Materials Needed: - Hard copies of fill-in tool (for field notes) – in case of outage/technical problems - Stationary (notebooks, pens, staples, folders and envelopes, clips, etc.) 117 | MIDTERM EVALUATION OF STRENGTHENING EGYPT’S FAMILY PLANNING PROGRAM: ANNEXES USAID.GOV Basic Data – to be filled by evaluators (per tool administration/per KII): Team Members Administering the Tool: Governorate: Date: (mm/dd/yy) District (if applicable): # of Interviewees/Participants: Total # ( ) Male ( ) Female ( ) Entity Name: Type of Entity: (USAID – IP - National Partner – Private Sector - Training Provider – Service Provider – Training Recipient - Other) Interviewee Name and Phone Number: Position: Interviewee Name and Phone Number: Position: Interviewee Name and Phone Number: Position: How long have you been in this position/area? Standard SEFPP Evaluation Introductory Statement: We are members of an independent evaluation team contracted by the USAID Mission in Egypt to conduct a mid-term performance evaluation of the Strengthening Egypt’s Family Planning Program (SEFPP). The findings of the evaluation will assist USAID in informing decisions regarding: 1) the effectiveness of SEFPP technical assistance approaches, 2) the most effective/sustainable approaches regarding the promotion of family planning in Egypt, and 3) improvements that can be made by SEFPP implementers in the remaining years of the activity, including potential areas of future technical assistance based on the lessons learned from this project. Your participation is entirely voluntary, but your participation is important to the results of this evaluation. Results will be anonymized (no personally identifiable information) and will be shared with project stakeholders. TOOL Questions Introductory Questions: How are you engaged in SEFPP project? Can you specify the training activities you participated in under SEFPP? EQ Questions: EQ3.1. How would you assess the relevance of received capacity building (e.g., training, coaching, incentive programs, policies, IT resources) to your needs and practice jobs? EQ.3.3.33 How did the knowledge and skills received translate into practice in relationship to your professional functions? In case of receiving multiple training, which training program has been the most beneficial? and why? Provide examples. E.Q.3.4. In your opinion, what are the chances of sustaining the FP services accessibility (such as affordability) and quality? Do you have the required resources to apply and sustain the learnings derived from the capacity building? What gaps are still there? 33 Other training beneficiaries who are not media staff USAID.GOV MIDTERM EVALUATION OF STRENGTHENING EGYPT’S FAMILY PLANNING PROGRAM | 118 E.Q.2.1. What can drive and increase demand in FP methods in your community? What tools do you have to assess the demand? How do you shape your messages to create demand? EQ.2.2.34 How did the knowledge and skills received translate into practice in relation to raising community awareness (interpreting and presenting data) and creating demand for FP services? In case of receiving multiple training, which training program has been the most beneficial? Why? Provide examples. EQ.2.3 What are the key challenges you face conducting your work? E.Q.3.4. Do you have the required enabling environment to apply and sustain the learnings/action plans derived from the training? What gaps are still there? E.Q.3.6. What are the innovations and best practices that have proven to be effective and worth replicating in the accessibility and delivery of quality FP services? E.Q.4.1. How has COVID-19 affected the training programs and your ability to deliver services? How the project contributed to addressing those challenges? Other Questions/ Ending: Is there anything you would like to add? 34 (media officers and media professionals and NGOs volunteers) 119 | MIDTERM EVALUATION OF STRENGTHENING EGYPT’S FAMILY PLANNING PROGRAM: ANNEXES USAID.GOV تقییم منتصف المدة لمشروع تعزیز برنامج مصر لتنظیم الأسرة في مصر الأدوات الكیفیة الأداة (7T – (مقابلة شخصیة – متلقو التدریب (من غیر مقدمي الخدمات) وصف الأداة: مصدر البیانات (الفئة المستھدفة): متلقو التدریب (من غیر مقدمي الخدمات) نوع الأداة: مقابلة مع مصدر معلومات رئیسي مصادر المعلومات الرئیسیین للأداة: متلقو التدریب من غیر مقدمي الخدمة: - مسئولین الإعلام - الإعلامیون - موظفو المعلومات في نظام إدارة المعلومات الصحیة - مثقفي الأقران - المتطوعون من المنظمات غیر الحكومیة - الموظفون الإداریون في المجلس القومي للسكان 45 – 60 دقیقة لكل استخدام للأداة مع مصدر معلومات رئیسي الوقت المتوقع لتنفیذ الأداة: 90 دقیقة شاملة الترتیبات السابقة واللاحقة للمقابلة الاحتیاجات اللوجستیة: الإعداد: - دعوة المشاركین في المقابلات - ملء قائمة بیانات المشاركین/ات - التأكد من احتیاطات وشروط السلامة الخاصة بفیروس كورونا - توفیر مصدر للكھرباء (فیشات) - التوصیل بشبكة الإنترنت (إذا لزم الأمر) المستلزمات المطلوبة: - نسخ من أدوات جمع البیانات (لتسجیل محضر المقابلة) في حالة انقطاع التیار الكھربائي. - أدوات كتابیة (دفاتر، أقلام، دوسیھات حفظ، مظاریف، دبابیس، الخ) البیانات الأساسیة: (یتم ملء الخانات مع كل استخدام للأداة لكل مصدر معلومات رئیسي) أسماء فریق العمل المنفذین للأداة: ًا / أو تحدید المحافظة: التاریخ: (یوم/ شھر / سنة) المركز (لو انطبق): مركزی عدد المشاركین في المقابلة: الإجمالي ( ) رجال ( ) نساء ( ) نوع الجھة: (الوكالة الأمریكیة للتنمیة – الشریك المنفذ – الشریك القومي – القطاع الخاص – مقدمي التدریب – مقدمي الخدمة - متلقو التدریب – آخرون: تحدید ...) اسم الجھة: اسم الشخص ورقم الھاتف: الوظیفة داخل المنظمة / مدة الوجود بالمنصب :اسم الشخص ورقم الھاتف الوظیفة داخل المنظمة / مدة الوجود بالمنصب :اسم الشخص ورقم الھاتف الوظیفة داخل المنظمة / مدة الوجود بالمنصب USAID.GOV MIDTERM EVALUATION OF STRENGTHENING EGYPT’S FAMILY PLANNING PROGRAM | 120 الأداة (T7 – (مقابلة مع مصدر معلومات رئیسي – متلقو التدریب من غیر مقدمي الخدمة تقدیم وافتتاحیة: صباح الخیر، أنا اسمي--------------------: و معي زمیلي/زمیلتي في العمل ---------------------------- . نحن أعضاء فریق تقییم مستقل تعاقدت معھ الوكالة الأمریكیة للتنمیة الدولیة في مصر لإجراء تقییم منتصف المدة لأداء برنامج تعزیز تنظیم الأسرة في مصر ستساعد نتائج التقییم الوكالة الأمریكیة للتنمیة الدولیة بمصر على اتخاذ قرارات فـیما یخص: 1) فعالیة مناھج المساعدة الفنیة المستخدمة من المشروع 2) النھج الأكثر فعالیة / استدامة فیما یتعلق بتعزیز تنظیم الأسرة في مصر بشكل عام 3) التحسینات التي یمكن لمنفذي المشروع القیام بھا في سنواتھ المتبقیة بما یتضمن مجالات المساعدة الفنیة المحتملة المبنیة على الدروس المستفادة من المشروع. ً مشاركتك معنا الیوم طوعیة تماما ولكنھا ھامة للغایة لنتائج ھذا التقییم. لن یتم الافصاح عن أي بیانات شخصیة للمشاركین/ات في ھذا التقییم ولكن سیتم مشاركة نتائج التقییم مع أصحاب المصلحة في المشروع. أسئلة الأداة: أسئلة افتتاحیة: سؤال افتتاحي: ما ھو شكل مشاركتكم في مشروع تعزیز برنامج مصر لتنظیم الأسرة؟ ھل یمكن تحدید الأنشطة التدریبیة التي شاركت فیھا في إطار مشروع تعزیز برنامج مصر لتنظیم الأسرة؟ الأسئلة المتعلقة بأسئلة التقییم: س 1-3 ما ھو تقییمك لملائمة برامج بناء القدرات التي حصلتم علیھا (على سبیل المثال، التدریب، والتوجیھ، وبرامج التحفیز، والسیاسات، وموارد تكنولوجیا المعلومات) لاحتیاجاتك وممارسة مھام وظیفتك؟ س 353-3 كیف قمت بتحویل المعارف والمھارات التي حصلت علیھا إلى ممارسات فیما یتعلق بتحسین إمكانیة الوصول وتعزیز جودة خدمات تنظیم الأسرة؟ في حالة تلقي تدریبات متعددة، ما ھو البرنامج التدریبي الأكثر فائدة؟ ولماذا؟ یرجى تقدیم أمثلة. س 2-2 36كیف قمت بتحویل المعارف والمھارات التي حصلت علیھا في التدریبات إلى ممارسات بخصوص رفع وعي المجتمع (تفسیر وعرض البیانات) وخلق/زیادة الطلب على خدمات تنظیم الأسرة؟ في حالة تلقي تدریبات متعددة، ما ھو البرنامج التدریبي الأكثر فائدة؟ ولماذا؟ برجاء تقدیم امثلھ س 3-2 ما ھي التحدیات الرئیسیة التي تواجھھا أثناء تنفیذ عملك؟ س 4-3 في رأیك، ما ھي فرص استدامة تحسین إمكانیة الوصول إلى خدمات تنظیم الأسرة (مثل القدرة على تحمل التكالیف) وتحسین جودة تلك الخدمات؟ ھل لدیك الموارد المطلوبة لتطبیق الدروس المستفادة من بناء القدرات والحفاظ علیھا؟ ما ھي الفجوات التي لا تزال موجودة؟ س 6-3 ما ھي الابتكارات وأفضل الممارسات التي أثبتت فعالیتھا وتستحق تكرارھا في تعزیز إمكانیة الوصول وتقدیم خدمات تنظیم الأسرة عالیة الجودة؟ س 1-4 كیف أثرت جائحة كورونا على برامج التدریب وقدرتك على تقدیم الخدمات؟ كیف ساھم المشروع في مواجھة تلك التحدیات؟ المستفیدون الآخرون من التدریبات لیسوا إعلامیین 35 المسؤولین الإعلامیون والمتطوعون من المنظمات غیر الحكومیة 36 121 | MIDTERM EVALUATION OF STRENGTHENING EGYPT’S FAMILY PLANNING PROGRAM: ANNEXES USAID.GOV أسئلة أخرى / إنھاء المقابلة: ھل ترغب في إضافة شيء آخر؟ USAID.GOV MIDTERM EVALUATION OF STRENGTHENING EGYPT’S FAMILY PLANNING PROGRAM | 122 SEFPP MID-TERM EVALUATION QUANTITATIVE DATA COLLECTION TOOLS T8 – SERVICE PROVIDERS TRAINING RECIPIENTS (CLINICIANS) QUESTIONNAIRE This section to be filled by the enumerator: Implementation Date: Enumerator Name: Participant name: Participant phone number: Occupation (Pharmacist-Physician-Nurse) Beneficiary ID Introduction: Good morning. I am ………. I am calling you on behalf of an independent evaluation team for the Strengthening Egypt’s Family Planning Program that is implemented by the Ministry of Health in coordination with USAID and JSI. We are contacting you because – at the mid-term of the project, we were asked an as independent evaluation team to evaluate the previous period of the project to assess the effectiveness of the implemented activities and to identify the most successful approaches. This will help the project to better plan for the coming years and the design of new similar projects. As you were one of the training recipients or participated in workshops and similar activities, we would like to ask your opinion about those trainings and their impact n the project’s implementation. If you agree, I will ask you a few questions and won’t take more than a few minutes of your time. Participants respond to the questions from this point onwards: Work Governorate District/Area: Job Title: 1- Most Beneficial Trainings or Workshops: 1-1 Have you ever participated in any training activity/events with SEFPP? The enumerator might need to remind the respondent of types of events like: Training – workshops –round table discussions – workshops for reviewing a training or procedural manual – panning workshop, … etc. a) Yes (Continue to next Question) b) No (End the Survey) 123 | MIDTERM EVALUATION OF STRENGTHENING EGYPT’S FAMILY PLANNING PROGRAM: ANNEXES USAID.GOV 1-2 If yes, what was the most beneficial training(s) that you received with SEFPP? Training/Workshop/Activity/Event Titles 2- Respondent Opinion about training and effect on his work: We will say some statement related to the project’s performance and the training component. Please rate the statements with a score from 1 to 5, where 1 is strongly disagree and 5 is strongly agree as follows: 1- Strongly disagree 2- Agree 3- Neutral 4- Agree 5- Strongly Agree 2-1. The training topics were relevant to your work responsibilities. Strongly disagree (1) Disagree (2) Neutral (3) Agree (4) Strongly agree (5) Don’t Know 2-2. The training provided you with sufficient information and skills Strongly disagree (1) Disagree (2) Neutral (3) Agree (4) Strongly agree (5) Don’t Know 2-3. You were able to apply the learning you received in the training in your workplace (through which you attended the training). Strongly disagree (1) Disagree (2) Neutral (3) Agree (4) Strongly agree (5) Don’t Know 2-4. Comments/Clarifications/Reasons 3- Availability of the FP methods at the health Facilities: USAID.GOV MIDTERM EVALUATION OF STRENGTHENING EGYPT’S FAMILY PLANNING PROGRAM | 124 3-1. The project has contributed to the increased access to quality FP methods Strongly disagree (1) Disagree (2) Neutral (3) Agree (4) Strongly agree (5) Don’t Know 3-2. The project has contributed to the improved quality of FP services Strongly disagree (1) Disagree (2) Neutral (3) Agree (4) Strongly agree (5) Don’t Know 4- Enhancing Selected Community behaviours: 4-1. The project has contributed to the increase of the demand for FP services Strongly disagree (1) Disagree (2) Neutral (3) Agree (4) Strongly agree (5) Don’t Know 125 | MIDTERM EVALUATION OF STRENGTHENING EGYPT’S FAMILY PLANNING PROGRAM: ANNEXES USAID.GOV مشروع تعزیز برنامج مصر لتنظیم الأسرة تقییم منتصف المدة أدوات جمع البیانات الكمیة أداة (T8 (- استمارة متلقو التدریب من مقدمي الخدمات یقوم جامع البیانات بملء ھذا الجزء تاریخ تنفیذ الاستمارة منفذ الاستمارة اسم المبحوث رقم التلیفون: الفئة (طبیب – ممرض – صیدلي): الرقم التعریفي: المقدمة صباح الخیر انا .......... بكلم حضرتك نیابة عن فریق تقییم مشروع تنظیم الأسرة اللي بتنفذه وزارة الصحة بالتعاون مع المعونة الأمریكیة وھیئة جون سنو بكلمك عشان عدا نص مدة المشروع واتطلب مننا كفریق تقییم مستقل نعمل تقییم للفترة اللي فاتت من المشروع عشان نحدد فاعلیة الأنشطة اللي اتنفذت ونحدد ایھ اكتر الطرق كانت مناسبة – وكمان ده ھیساعد المشروع یخطط بشكل افضل للسنوات الباقیة من المشروع وكمان یخطط افضل لمشروعات مماثلة بعد كدة. احنا بكلم حضرتك عشان حضرتك من الناس اللي حضرت بعض التدریبات أو شاركت في ورش عمل او أنشطة شبیھة تبع المشروع واحنا كفریق تقییم حابین نأخد رأیكم في التدریبات دي وتأثیرھا على تنفیذ المشروع. لو حضرتك موافق ھاسألك بعض الأسئلة ومش ھاخد من وقتك اكتر من دقایق. یقوم المبحوث بالرد على الأسئلة بدءا من ھذا الجزء: محافظة العمل الأساسیة: المركز / المنطقة: المسمى الوظیفي: USAID.GOV MIDTERM EVALUATION OF STRENGTHENING EGYPT’S FAMILY PLANNING PROGRAM | 126 ً – أكثر التدریبات وورش العمل فائدة: أولا 1-1 ھل سبق انك شاركت في تدریبات أو ورش عمل مشروع تنظیم الأسرة الذي تنفذه الوزارة مع المعونة الأمریكیة وھیئة جون سنو؟ على الباحث تذكرة المشارك بأنواع الأنشطة التي یمكن أن یكون قد شارك فیھا مثل: التدریب – ورش العمل – مناقشات مائدة مستدیرة – ورش عمل لمراجعة دلیل تدریبي أو دلیل إجرائي – ورش عمل تخطیطیة – وغیرھا .1 نعم (الاستمرار الى السؤال التالي) ب. لا (انھي الاستمارة) ً بالنسبة لك بشكل عام؟ .2-1 اذا كانت الإجابة بنعم ، ما ھي أكثر التدریبات أو ورش العمل فائدة اسم التدریب/ورشة العمل/الاجتماع/النشاط ًا – رأي المشارك في التدریبات وتأثیرھا على عملھ: ثانی .2 ما ھو رأیك في التدریبات التي حصلت علیھا بشكل عام؟ سنقوم بسرد بعض العبارات المتعلقة بأداء المشروع بشكل عام وبمكون التدریبات. رجاء تقییم العبارة بإعطائھا درجة من 1 إلى 5 بحیث تعني 5 "أوافق جدًا" و1 ً "لا أوافق تماما" وھي كالتالي: -1 ً لا أوافق تماما -2 لا أوافق -3 محاید -4 أوافق -5 أوافق جدًا 1-2 موضوعات التدریبات ملائمة لمسئولیات عملك لا أوافق تماما (1) لا أوافق (2) محاید (3) اوافق (4) اوافق جدا ( 5) لا اعرف 2-2 التدریب وفر لك معلومات ومھارات كافیة لا أوافق تماما (1) لا أوافق (2) محاید (3) اوافق (4) اوافق جدا ( 5) لا اعرف 127 | MIDTERM EVALUATION OF STRENGTHENING EGYPT’S FAMILY PLANNING PROGRAM: ANNEXES USAID.GOV 3-2 استطعت تطبیق ما تم مناقشتھ في التدریب في مكان عملك (الذي حضرت التدریب من خلالھ) لا أوافق تماما (1) لا أوافق (2) محاید (3) اوافق (4) اوافق جدا ( 5) لا اعرف 4-2 ملحوظات / شرح / الأسباب: ثالث - توافر وجودة وسائل تنظیم الأسرة فى المرافق الصحیة: ًا 1-3 ساھم المشروع في زیادة إتاحة وسائل تنظیم الأسرة ذات الجودة لا أوافق تماما (1) لا أوافق (2) محاید (3) اوافق (4) اوافق جدا ( 5) لا اعرف 2-3 المشروع ساھم في تطویر جودة خدمات تنظیم الأسرة لا أوافق تماما (1) لا أوافق (2) محاید (3) اوافق (4) اوافق جدا ( 5) لا اعرف ًا – تعزیز السلوكیات الصحیة المختارة: رابع 1-4 ساھم المشروع في زیادة الطلب على خدمات تنظیم الأسرة من المنتفعین لا أوافق تماما (1) لا أوافق (2) محاید (3) اوافق (4) اوافق جدا ( 5) لا اعرف USAID.GOV MIDTERM EVALUATION OF STRENGTHENING EGYPT’S FAMILY PLANNING PROGRAM | 128 SEFPP MID-TERM EVALUATION QUANTITATIVE DATA COLLECTION TOOLS T9 – TRAINING RECIPIENTS (NON-SERVICE PROVIDERS) (MEDIA OFFICERS, MEDIA PROFESSIONALS, HMIS INFORMATION STAFF, PEER EDUCATOR, NGOS VOLUNTEERS, NPC ADMINISTRATIVE STAFF) QUESTIONNAIRE This section to be filled by the enumerator: Implementation Date: Enumerator Name: Participant name: Participant phone number: Occupation (Media officers, Media professionals, HMIS information staff, Peer educator, NGOs volunteers, Factory Workers, NPC administrative staff, Other) Beneficiary ID Introduction: Good morning. I am ………. I am calling you on behalf of an independent evaluation team for the Strengthening Egypt’s Family Planning Program that is implemented by the Ministry of Health in coordination with USAID and JSI. We are contacting you because – at the mid-term of the project, we were asked an as independent evaluation team to evaluate the previous period of the project to assess the effectiveness of the implemented activities and to identify the most successful approaches. This will help the project to better plan for the coming years and the design of new similar projects. As you were one of the training recipients or participated in workshops and similar activities, we would like to ask your opinion about those trainings and their impact n the project’s implementation. If you agree, I will ask you a few questions and won’t take more than a few minutes of your time. Participants respond to the questions from this point onwards: Work Governorate District/Area: Job Title: 1- Most Beneficial Trainings or Workshops: 1-1 Have you ever participated in any training activity/events with SEFPP? The enumerator might need to remind the respondent of types of events like: Training – workshops –round table discussions – workshops for reviewing a training or procedural manual – panning workshop, … etc. 129 | MIDTERM EVALUATION OF STRENGTHENING EGYPT’S FAMILY PLANNING PROGRAM: ANNEXES USAID.GOV b) Yes (Continue to next Question) b) No (End the Survey) 1-2 If yes, what was the most beneficial training(s) that you received with SEFPP? Training/Workshop/Activity/Event Titles 2- Respondent Opinion about training and effect on his work: We will say some statement related to the project’s performance and the training component. Please rate the statements with a score from 1 to 5, where 1 is strongly disagree and 5 is strongly agree as follows: 6- Strongly disagree 7- Agree 8- Neutral 9- Agree 10- Strongly Agree 2-1. The training topics were relevant to your work responsibilities. Strongly disagree (1) Disagree (2) Neutral (3) Agree (4) Strongly agree (5) Don’t Know 2-2. The training provided you with sufficient information and skills Strongly disagree (1) Disagree (2) Neutral (3) Agree (4) Strongly agree (5) Don’t Know 2-3. You were able to apply the learning you received in the training in your workplace (through which you attended the training). Strongly disagree (1) Disagree (2) Neutral (3) Agree (4) Strongly agree (5) Don’t Know 2-4. Comments/Clarifications/Reasons USAID.GOV MIDTERM EVALUATION OF STRENGTHENING EGYPT’S FAMILY PLANNING PROGRAM | 130 3- Enhancing Selected Community behaviours: 4-1. The project has contributed to the increase of the demand for FP services Strongly disagree (1) Disagree (2) Neutral (3) Agree (4) Strongly agree (5) Don’t Know 131 | MIDTERM EVALUATION OF STRENGTHENING EGYPT’S FAMILY PLANNING PROGRAM: ANNEXES USAID.GOV مشروع تعزیز برنامج مصر لتنظیم الأسرة تقییم منتصف المدة أدوات جمع البیانات الكمیة أداة (T9 (- استمارة متلقو التدریب من غیر مقدمي الخدمات )المسؤولون الإعلامیون و المنسقون الاعلامیون و موظفي نظم المعلومات ومثقفي الاقران و متطوعي المنظمات الغیر حكومیة و الموظفون الإداریون لدى المجلس القومي للسكان) یقوم جامع البیانات بملء ھذا الجزء تاریخ تنفیذ الاستمارة منفذ الاستمارة اسم المبحوث رقم التلیفون: الفئة (مسئول اعلامي – منسق اعلامي – نظم معلومات – مثقف – متطوع بمنظمة غبر حكومیة – موظف إداري- عامل مصنع – أخرى): الرقم التعریفي: المقدمة صباح الخیر انا .......... بكلم حضرتك نیابة عن فریق تقییم مشروع تنظیم الأسرة اللي بتنفذه وزارة الصحة بالتعاون مع المعونة الأمریكیة وھیئة جون سنو بكلمك عشان عدا نص مدة المشروع واتطلب مننا كفریق تقییم مستقل نعمل تقییم للفترة اللي فاتت من المشروع عشان نحدد فاعلیة الأنشطة اللي اتنفذت ونحدد ایھ اكتر الطرق كانت مناسبة – وكمان ده ھیساعد المشروع یخطط بشكل افضل السنوات الباقیة من المشروع وكمان یخطط افضل لمشروعات مماثلة بعد كدة. احنا بكلم حضرتك عشان حضرتك من الناس اللي حضرت بعض التدریبات أو شاركت في ورش عمل او أنشطة شبیھة تبع المشروع واحنا كفریق تقییم حابین نأخد رأیكم في التدریبات دي وتأثیرھا على تنفیذ المشروع. لو حضرتك موافق ھاسألك بعض الأسئلة ومش ھاخد من وقتك اكتر من دقایق. یقوم المبحوث بالرد على الأسئلة بدءا من ھذا الجزء: محافظة العمل الأساسیة: منطقة العمل الأساسیة: المسمى الوظیفي: ً – أكثر التدریبات فائدة: أولا USAID.GOV MIDTERM EVALUATION OF STRENGTHENING EGYPT’S FAMILY PLANNING PROGRAM | 132 ً – أكثر التدریبات وورش العمل فائدة: أولا 1.-1 ھل سبق انك شاركت في تدریبات أو ورش عمل مشروع تنظیم الأسرة الذي تنفذه الوزارة مع المعونة الأمریكیة وھیئة جون سنو ؟ على الباحث تذكرة المشارك بأنواع الأنشطة التي یمكن أن یكون قد شارك فیھا مثل: التدریب – ورش العمل – مناقشات مائدة مستدیرة – ورش عمل لمراجعة دلیل تدریبي أو دلیل إجرائي – ورش عمل تخطیطیة – وغیرھا .1 نعم (الاستمرار الى السؤال التالي) ب. لا (انھي الاستمارة) ً بالنسبة لك بشكل عام؟ .2-1 ما ھي أكثر التدریبات أو ورش العمل فائدة اسم التدریب/ورشة العمل/الاجتماع/النشاط ًا – رأي المشارك في التدریبات وتأثیرھا على عملھ: ثانی .2 ما ھو رأیك في التدریبات التي حصلت علیھا بشكل عام؟ سنقوم بسرد بعض العبارات المتعلقة بأداء المشروع بشكل عام وبمكون التدریبات. رجاء تقییم العبارة بإعطائھا درجة من 1 إلى 5 بحیث تعني 5 "أوافق جدًا" و1 ً "لا أوافق تماما" وھي كالتالي: -1 ً لا أوافق تماما -2 لا أوافق -3 محاید -4 أوافق -5 أوافق جدًا 1-2 موضوعات التدریبات ملائمة لمسئولیات عملك لا أوافق تماما (1) لا أوافق (2) محاید (3) اوافق (4) اوافق جدا ( 5) لا اعرف 2-2 التدریب وفر لك معلومات ومھارات كافیة لا أوافق تماما (1) لا أوافق (2) محاید (3) اوافق (4) اوافق جدا ( 5) لا اعرف 133 | MIDTERM EVALUATION OF STRENGTHENING EGYPT’S FAMILY PLANNING PROGRAM: ANNEXES USAID.GOV 3-2 استطعت تطبیق ما تم مناقشتھ في التدریب في مكان عملك (الذي حضرت التدریب من خلالھ) لا أوافق تماما (1) لا أوافق (2) محاید (3) اوافق (4) اوافق جدا ( 5) لا اعرف 4-2 ملحوظات / شرح / الأسباب: ًا – تعزیز السلوكیات الصحیة المختارة: رابع 1-4 ساھم المشروع في زیادة الطلب على خدمات تنظیم الأسرة من المنتفعین لا أوافق تماما (1) لا أوافق (2) محاید (3) اوافق (4) اوافق جدا ( 5) لا اعرف USAID.GOV MIDTERM EVALUATION OF STRENGTHENING EGYPT’S FAMILY PLANNING PROGRAM | 134 ANNEX 8: LIST OF REVIEWED DOCUMENTS # DOCUMENT NAME 1 Annex 1 – EQ 2 Sub-questions 2 Annex 1 – EQ 3 Sub-questions 3 Approved 2020.05.13-SEFPP_Planned_Activities_6Month-WithCOVID-19-SEFPP_24 june 2020 4 JSI CA 72026318CA00001 (1) 5 JSI CA 72026318CA00001 6 JSI CA 72026318CA00001 7 JSI MOD 03 signed 8 List of Desk Review Documents - 21 Jan 2021 9 MOD P005 (1) 10 Mod P008 - SEFPP- JSI 12072020 - Fully Executed (1) 11 P002 - Countersigned Modification 12 SEFPP MELP updated codes10th-DecV2 13 SEFPP Mod 1_fully executed 14 SEFPP Mod 1_fully executed 15 SEFPP Mod 4_fully executed 16 SEFPP MOD P006 - fully executed (3) 17 SEFPP Annual Report Y1_Final 18 SEFPP Annual Report Y1_Final 19 SEFPP Annual Report Y2_Final 20 SEFPP Annual Report Y3_Final (1) 21 SEFPP Annual Report Y3_Final (2) 22 SEFPP Annual Report Y3_Final (3) 23 SEFPP Annual Report Y3_Final 24 SEFPP Annual Report Y3_Final 25 SEFPP Annual Report Y3_Final 26 SEFPP_Y1 Annual Workplan_FINAL 27 SEFPP_Y1 Annual Workplan_FINAL 28 SEFPP_Y2 Annual Workplan_FINAL 29 SEFPP_Y3 Annual Workplan_FINAL 30 SEFPP_Y4 Annual Work Plan_ 29 Dec.2020 (1) 135 | MIDTERM EVALUATION OF STRENGTHENING EGYPT’S FAMILY PLANNING PROGRAM: ANNEXES USAID.GOV # DOCUMENT NAME 31 AY3_Q2_Quarterly Report_Jan-Mar2020_SEFPP_Response 2 32 PY1-Q3_ Quarterly Report_final 33 PY1_Q2_Quarterly Report _final (1) 34 PY1_Q2_Quarterly Report _final 35 PY1_Q2_Quarterly Report _final 36 PY2-Q1 Progress Report to MOHP_ Oct.-Dec. 18_ Final 37 PY2-Q2_SEFPP QuarterlyProgressReport_Final 38 PY2-Q3_SEFPP QuarterlyProgressReport_July 29 39 PY3-Q1 Quarterly Report_FINAL-Oct-Dec2019 40 PY3-Q3 Quarterly Report_FINAL-April-June2020 41 El Galaa teaching hospital visit report 42 MSDS for FP training venue-Galaa 43 MSDS for Public FP training clinics-Galaa 44 El Matareya Teaching hospital visit report 45 MSDS for FP training venue-Matareya 46 MSDS for Public FP training clinics-Matareya 47 El Monera hospital visit report 48 MSDS for FP training venue - المنیرة 49 MSDS for Public FP training clinics - المنیرة 50 20-Jan2020-MOHP needs assessment 51 SEFPP_Cs Assessment report Arabic version_word_NY_2018 52 SEFPP_Cs Assessment report English version_word_NY_2018 53 SEFPP_Egypt Quantification Report_word_BL.NY Feb 2019 54 Criteria of selection for FP physician - Service provider 55 SOW for FP physician - Service provider 56 Criteria of selection for FP Trainer - Nurse supervisor 57 SOW for FP Trainer - Nurse supervisor 58 1-SOW for FP Trainer - Physician 59 2-Criteria of selection for FP Trainer - Physician 60 3-SKILLS CHECKLIST FOR FP TRAINER - Physician 61 Gap Analysis Session presentation Arabic-Final-S Edit USAID.GOV MIDTERM EVALUATION OF STRENGTHENING EGYPT’S FAMILY PLANNING PROGRAM | 136 # DOCUMENT NAME 62 Gap Analysis and Needs Assessment Workshop Report July 19 2018 63 Gap Analysis and Needs Assessment Workshop Report July 19 2018 64 20-OctPY2 Reporting GAP 65 Consultant Assignment Workplan V2 66 GAP_USAID_Final- resp USAID comments_28Mar2018_v2 67 Gender Implementation Plan AY4-5 68 Worksheet in C Users User Desktop SEFPP Final Deliverables Annual Reports SEFPP AY3 Annual Report Final 69 2020.09.28-Gender Mainstreaming Strategy v2- USAID comments addressed_Jan.19th 70 HMIS Assessment Final Report-USAID comments+SEFPP Response - USAID Approved-clean 71 USAID Staff List 72 WHO Workshop for Media Practitioners_24-25 Feb 2019 73 SEFPP- ME Manual-Y4.workplan 74 Training register summary 75 Final-English CS Assessment Report 76 Capacity Building Workshop on Behaviour Segmentation Report August 5 – 9 2018 - Final Version - V2 77 Capacity Building Workshop on Behaviour Segmentation Report August 5 – 9 2018 - Final Version - V2 78 Core trainers workshop_ Report 26 _ 27 Dec 2019 - Final 79 Core trainers workshop_ Report 26 _ 27 Dec 2019 - Final 80 SBCC Capacity Building Workshop Report_Feb 2019_ Revised Final 81 SBCC Capacity Building Workshop Report_Feb 2019_ Revised Final 82 Email to UNFPA 83 SBCC CB Agenda 84 Advocacy Assessment in Arabic with Answers 85 BE Assessment in Arabic with Answers 86 IPC Assessment in Arabic with Answers 87 SBCC Assessment in Arabic with Answers 88 Advocacy Assessment in Arabic 89 BE Assessment in Arabic 90 IPC Assessment in Arabic 91 SBCC Assessment in Arabic 92 Micro Teaching Evaluation 137 | MIDTERM EVALUATION OF STRENGTHENING EGYPT’S FAMILY PLANNING PROGRAM: ANNEXES USAID.GOV # DOCUMENT NAME 93 Overall Evaluation 94 Participant_s Checklist Evaluation 95 SBCC Facilitator Guide - Final 96 Handouts slides 97 Advocacy Unit 98 BE Unit 99 Embrace Life - always wear your seat belt 100 IPC Unit 101 SBCC Unit ما ھو ختان البنات؟ 102 103 SBBCC - Participant Guide 104 Arabic PLGHA_Media 105 Introduction 106 Unit 1 _ 2 (SBCC _ IPC) 107 Unit 3 (BE) 108 Unit 4 (Advocacy) 109 1- Facilitator Guide 110 2- Participant Guide 111 3- PowerPoint 112 Agenda 113 Advocacy Assessment in Arabic with Answers 114 BE Assessment in Arabic with Answers 115 IPC Assessment in Arabic with Answers 116 SBCC Assessment in Arabic with Answers 117 Advocacy Assessment in Arabic 118 BE Assessment in Arabic 119 IPC Assessment in Arabic 120 SBCC Assessment in Arabic 121 Micro Teaching Evaluation 122 Overall Evaluation 123 Participant_s Checklist Evaluation USAID.GOV MIDTERM EVALUATION OF STRENGTHENING EGYPT’S FAMILY PLANNING PROGRAM | 138 # DOCUMENT NAME 124 SEFPP_New method action plan for pilot phase_word_NY_2020 125 SEFPP_New method data collection Guide1,2,3_word_NY_2020 126 SEFPP_New method Geograph Scope MOHP letter_NY_2020 127 SEFPP_New method Geograph ScopeWorkshopReport_word_NY_2020 128 SEFPP_New method Instrument to collect data-form 3_word_NY_2020 129 SEFPP_New method Instrument to collect data-form1_word_NY_2020 130 SEFPP_New method introduction Guideline arabic_word_NY_2020 131 SEFPP_New method manual validationWorkshopReport_word_NY_2019 132 SEFPP_New method meeting report_word_NY_1-2Feb2019 133 SEFPP_New method workshop I Reort_word_NY_2018 134 SEFPP_New method Workshop II Report_ word_JF.NY_Jan2019 135 SEFPP_New method Instrument to collect data- form2_word_NY_2020 136 Monitoring Plan Arabic 137 Monitoring Plan English 138 0-TOT Nurses and Physicians - مجمع- 0- 139 00-Cadre_s Manual-word 140 1-Pre- Post Test Ar-nu 141 10.Overall Evaluation-ph 142 2-Key Answer-Ar-nu 143 3-Facilitation Evaluation Ar-nu 144 4-Overall Evaluation Ar-nu 145 5-pre- post test final, Feb 2019-ph 146 6-pre- post test final, Feb 2019-answers-ph 147 7.Checklist for Implanon insertion _ removal-ph 148 8.Checklist for IUD insertion _ removal-ph 149 9.facilitaion Evaluation-ph 150 TOT Nurses and Physicians - مجمع 151 1-Course Objectives لماذا تنظیم الاسرة2- 152 تعلیمات 153 154 1-Anatomy and physiology 139 | MIDTERM EVALUATION OF STRENGTHENING EGYPT’S FAMILY PLANNING PROGRAM: ANNEXES USAID.GOV # DOCUMENT NAME 155 2-contraceptive methods overview 156 3-Clinical Assessment _ IUD insertion 157 5-Overview of Counseling 158 Ovulation the menstrual cycle - Narrated 3D animation - YouTube.avi 159 Presentation1 160 MEC 2015 161 MEC full-2015 162 Wheel contraception 163 1-IUD 164 2-Problems associated with IUD 165 3-RTIs 166 How Intrauterine Device Prevent Pregnancy - Manipal Hospital 167 1-Combined hormonal contraception 168 2-Problems associated with combined hormonal methods مكافحة العدوي3- 169 170 6 step hand cleaning technique 171 1-Progestin only contraception 172 2-Problems associated with progestin only hormonal methods (2) 173 3-Emergency contraception 174 barrier , local _ natural 175 CleanVersion_SEFPP Private Sector Strategy- 210120 176 2020.06.29-SustainabilityReport-Draft- USAID comments-SEFPP Responses - Based on the old version_Not Approved yet 177 SEFPP__Sustainability Plan_ May 22 2019_FINAL_Approved_Old Version 178 SEFPP__Sustainability Plan_14.12.2020-Final_Not Approved yet 179 Communication Skills for Media Engagement_AR-Final 180 Communication Skills for Media Engagement_EN_Final 181 Data Visualization Manual_AR 182 Data Visualization Manual_EN 183 Reporting _ Writing Skills_AR_Final 184 Reporting _ Writing Skills_EN_Final 185 2.1.5.2. Data Visualization Workshop Final Report AR USAID.GOV MIDTERM EVALUATION OF STRENGTHENING EGYPT’S FAMILY PLANNING PROGRAM | 140 # DOCUMENT NAME 186 2.1.5.2. Data Visualization Workshop Final Report EN 187 2.1.5.2. Reporting and Writing Skills Workshop Final Report AR 188 2.1.5.2. Reporting and Writing Skills Workshop Final Report EN 189 Basic CB Workshop_Dealing with the Media_6-9 February 2020_FInal Report AR 190 Basic CB Workshop_Dealing with the Media_6-9 February 2020_FInal Report EN 191 1-TOT Universities and Factories - Agenda and Objectives 192 2-SEFPP Compliance Standard PPT - Final Arabic 12 Jan 2019 193 3-SEFPP Program Introduction PPT الإقتصاد السلوكي - Economics Behavior4- 194 عربي - Gender5- 195 196 6-COVID-19 Protection 197 7-Training of Trainer Final - Ar 198 Facilitation Evaluation Ar H -التقییم الختامي للدورة 199 200 Private Pharmacist Pre-PostTest_ 201 Private Pharmacist key answers Pre-PostTest_ 202 Private Pharmacists Training agenda - Master En. لماذا تنظیم الاسرة 1- 203 204 2-SEFPP Program Introduction PPT 205 3-SEFPP Compliance Standard PPT - Final Arabic 12 Jan 2019 عربي - Gender4- 206 الإقتصاد السلوكي - Economics Behavior5- 207 وسائل تنظیم الاسرة6- 208 الشائعات 7- 209 رسائل اعلامیة للصجة الانجابیة8- 210 الاختبار القبلي-لا یستخدم 211 المحتوي التدریبي 212 H - للتمریض - MEC- قائمة إختیار الوسائل الھرمونیة 213 214 Private Physician Training agenda - Master En. 215 Private Physicians pre-post Test_ Answers 216 1-SEFPP Program Introduction PPT 141 | MIDTERM EVALUATION OF STRENGTHENING EGYPT’S FAMILY PLANNING PROGRAM: ANNEXES USAID.GOV # DOCUMENT NAME 217 2-SEFPP Compliance Standard PPT - Final English 12 Jan 2019 عربي - Gender3- 218 الإقتصاد السلوكي - Economics Behavior4- 219 220 5-Contraceptive methods overview الشائعات 6- 221 222 7- MEC - 2015 223 global-handbook-2018-full-web 224 MEC أسئلة استبعاد الحمل - موعد بدء الوسائل 225 226 PY1-2 indicators 227 PY3 indicators USAID.GOV MIDTERM EVALUATION OF STRENGTHENING EGYPT’S FAMILY PLANNING PROGRAM | 142 ANNEX 9: BIOGRAPHIES MILDRED HOWARD, TEAM LEADER Mildred Howard has over 30 years of experience in over a dozen countries in international development assistance as a program manager, evaluator, and independent technical consultant. Her career has focused on the health sector, where she has held several positions, including overseeing, directing, and/or managing health programs for USAID missions in several countries, as well as leading teams conducting evaluations of health programs. These opportunities gave her a solid understanding of the sector from both a programmatic and evaluative angle, including nine years of family planning-specific experience. Ms. Howard has lived and worked in Egypt, including directly working on family planning activities. For example, in 2013, Ms. Howard served as team leader of the Evaluation of the Health Systems Strengthening Technical Assistance Activity, implemented by Social Impact, Inc. Earlier, from 2001 to 2006, Ms. Howard lived and worked full-time in Cairo. As Health Sector Reform and Financing Advisor for USAID/Egypt, she was assigned to implement USAID’s support for the Government of Egypt’s Health Sector Reform Program, which involved implementation of a national family health (including family planning) service delivery model. She has authored several publications, including many evaluation reports and other research. She comes highly recommended by a QED Group colleague who has worked with her in the past. Ms. Howard holds a master’s in social policy and program evaluation from the prestigious Harvard University in the United States, as well as several health-related certificates from other universities. RAFEEK RAMZY, SUBJECT MATTER EXPERT Dr. Rafeek Ramzy has 28 years of professional experience. He began his career as a physician and branched out to working on development activities, many of which were in the health and population sector. He has had extensive direct work experience with population and/or reproductive health activities over many years. This has included the population and health activity of the Non-Governmental Organizations Service Center (NGOSC) funded by USAID, the European Commission Support to Population Program in Upper Egypt, and close coordination with the Ministry of Health and Population (MOHP) under primary health care through a community participation activity funded by the Catholic Coptic Diocese. He has worked extensively with NGOs and has been involved in several training efforts, including recently preparing a training manual on peer education for Family Health International (FHI). In addition to direct work with reproductive health and population activities, he brings hands-on experience with training efforts, which is very relevant to examining the training component for the SEFPP evaluation. Dr. Rafeek has worked for prominent USAID and EU-funded activities. Through these opportunities, he has gained extensive experience in institutional capacity building, monitoring, and evaluation, as well as management of a wide portfolio of development activities. He has also served as team leader for five evaluations. Dr. Rafeek holds a master’s degree in the field. NIVINE RAMSES, EVALUATOR With more than 25 years of experience in the field of development and humanitarian work, Ms. Ramses is an expert evaluator in various sectors, with a special focus on gender. Her expertise is Monitoring Evaluation, Accountability, and Learning (MEAL)-focused, carrying out program and project evaluations; conducting situation analyses, pre-project research, evaluability assessments, baselines, endlines, and organizational assessments; developing monitoring and evaluation systems; building evaluation capacities in organizations; designing and implementing multi-themed surveys; and implementing participatory planning and participatory monitoring and evaluation. Some organizations 143 | MIDTERM EVALUATION OF STRENGTHENING EGYPT’S FAMILY PLANNING PROGRAM: ANNEXES USAID.GOV that Ms. Ramses has worked with include U.N. Women, the World Food Programme (WFP), the U.N. High Commissioner for Refugees (UNHCR), Food and Agriculture Organization (FAO), USAID/Egypt’s SIMPLE Project, CARE International, Save the Children, Plan International, UNICEF, Face, Oxfam, and GreenWorld. Her most recent consultancy assignments varied broadly, covering topics such as national food loss programs, education programs for refugees, economic empowerment for women through value chains and asset transfer, youth and smallholder farmers, climate change resilience, gender equality, gender transformative strategies, gender-based violence, agribusiness development, and cultural heritage. NOHA HASSAN, EVALUATOR Noha Hassan is a monitoring and evaluation (M&E) specialist with a master’s in development management from the London School of Economics. She has extensive experience working with national, multinational, and government entities in Egypt, the United States. and the United Kingdom. Ms. Hassan’s fields of expertise are monitoring and evaluation, entrepreneurship, and economics. Ms. Hassan worked with clients such as USAID, the European Union, GIZ, Plan International, the Drosos Foundation, Oxfam Novib, the International Organization for Migration (IOM), and Christian Aid. NORHAN BADER, EVALUATOR Norhan Bader is a physician by training, having graduated from Cairo University, with extensive experience in the field of Sexual and Reproductive Health and Rights (SRHR) for more than ten years. She is currently working as a Planning, Monitoring, Evaluation, and Learning (PMEL) Specialist in RNW Media at the Love Matters Arabic project. She worked as an M&E Officer at the Population Council for two years, in the Egypt office of the “Expanding the role of the private sector in addressing family planning needs of Egyptian young people” project. Additionally, Norhan acted as a Director of Projects for two years at International Youth Alliance for Family Planning (IYAFP), where she supervised the implementation of advocacy projects performed by 81 country coordinators. As a researcher, she is interested in studying such topics as Female Genital Cutting (FGC), Intimate Partner Violence (IPV), puberty and sexuality, and unmet needs for family planning. She completed her Master’s in Public Health (MPH), with a concentration in health promotion and community health, at the American University of Beirut (AUB) after being awarded a scholarship by the World Health Organization’s (WHO) Special Program for Research and Training on Tropical Diseases. She was selected as a Young Leader in Women Deliver Young Leaders program in 2015 and, in 2017, as a winner of 120 under 40, organized by The Bill & Melinda Gates Institute for Population and Reproductive Health at Johns Hopkins Bloomberg School of Public Health for her efforts in the reproductive health field. HANAN KWINANA, EVALUATOR Ms. Kwinana is a results-oriented quality management expert with hands-on experience in project cycle management and design, monitoring & evaluation (DME) experience in qualitative & quantitative participatory tools and theory-based models of complex projects, documentation of lessons learned and sustainability measures, baseline/endline research studies, and human and institutional capacity assessment and development. She is experienced in various social and economic development sectors: health (primary, secondary, and tertiary healthcare systems), education (basic & higher education, TVET, employment, employability, youth development, and transition from school to work, labor market studies, active labor market strategies, and career guidance), entrepreneurship, micro/small and medium enterprises development, and others. She has managed USAID.GOV MIDTERM EVALUATION OF STRENGTHENING EGYPT’S FAMILY PLANNING PROGRAM | 144 development cooperation and technical assistance interventions engaging the private sector, government institutions, donor communities, and development organizations: IFAD, KfW, UN Women, Oxfam GB, GIZ, EU/EC, DFID, USAID, World Bank Group, SIDA, AFD, Italian SWAP, UNDP, ILO, Qatar Foundation, and others. Ms. Kwinana is fluent in English and Arabic. EHAB SAKR, STATISTICIAN Ehab Sakr is a Ph.D. candidate and assistant lecturer in the Department of Demography and Biostatistics at the Institute of Statistical Studies and Research of Cairo University in Egypt. Mr. Sakr holds a master’s degree in statistics from the university’s Faculty of Economics and Political Science. He has more than 18 years’ experience in research, working at both the national (e.g., National Population Council and the Industrial Modernization Center) and international (e.g., United Nations Population Fund, Demographic and Health Surveys Program, and ILO) levels. He contributes to questionnaire design and provides guidance on the quality of data collection, data entry, and data analysis using statistical packages, including STATA and SPSS. He also has experience in software and web development, as well as digital marketing. U.S. Agency for International Development 1300 Pennsylvania Avenue, NW Washington, DC 20523