USAID/JORDAN: EVALUATION OF JORDAN HEALTH COMMUNICATION PROGRAM OCTOBER 2011 This publication was produced for review by the United States Agency for International Development. It was prepared by Gary Lewis, Carolyn Barnes, and Inaam Khalaf through the Global Health Technical Assistance Project. USAID/JORDAN: EVALUATION OF THE JORDAN HEALTH COMMUNICATION PROGRAM DISCLAIMER The views of the authors expressed in this publication do not necessarily reflect the views of the United States Agency for International Development or the United States Government. This document (Report No. 11-01-500) not available online. Online documents can be located in the GH Tech website library at http://resources.ghtechproject.net. Documents are also made available through the Development Experience Clearinghouse (http://dec.usaid.gov). Additional information can be obtained from: The Global Health Technical Assistance Project 1250 Eye St., NW, Suite 1100 Washington, DC 20005 Tel: (202) 521-1900 Fax: (202) 521-1901 info@ghtechproject.com This document was submitted by The QED Group, LLC, with CAMRIS International and Social & Scientific Systems, Inc., to the United States Agency for International Development under USAID Contract No. GHS-I-00-05-00005-0. ACKNOWLEDGMENTS The evaluation team would like to acknowledge the generous support and assistance it received. Special thanks go to Laura Slobey, Team Leader, Population and Family Health Section, and Basma Khraisat, Project Management Specialist, Population and Family Health Section, who kindly gave their time to brief the team and meet with team members to provide valuable feedback. The team greatly appreciates the support and assistance provided throughout the evaluation process by Leslie MacKeen, Project Manager in USAID's Population & Family Health Section. She ensured that the evaluators had all essential background documents for the preparatory phase of the evaluation and was available throughout the process to address team questions. In addition, special thanks go to USAID/Jordan Acting Director Kevin Rushing, Deputy Mission Director Douglas Ball, Program Officer Mohammed Yassien, Project Management Specialist Ali Arbaji, and Project Management Specialist Ziad Muasher for their interest in and support of the evaluation team’s efforts. The team acknowledges the extraordinary access and forthright nature of the dialogue with the Government of Jordan officials working with USAID and JHCP. Their inputs helped the team better understand complex issues like partnerships and institutionalization. While the number of officials the team met with precludes acknowledging individual contributions, the team would like to single out the following individuals for providing their time, wisdom, and insights: His Excellence Dr. Daif Allah Al Lozi, His Excellence Raed Al Adwan, His Excellence Abd Al Rahman Ibdah, His Excellence Dr. Mohammad Al Khalayleh, His Excellence Marwan Qteshat, His Excellence Khalid Abu Zaid, Her Excellence Professor Raeda Al Qutob, Dr. Bassam Hajawi, Dr. Rowaida Rasheed, Dr. Randa Baqa’een, His Excellence Dr. Taher Abu Al Samen, Dr. Salah Khalyleh, Dr. Abd Al Rahman Tubaishat, Dr. Marwan Rayahnieh, Mr. Mohammad Al Amri, Shaik Abdalla Rababa’ah, Dr. Ali Zitawi, Dr. Munther Shboul, Mr. Mohammad Aqeel, Shaik Mohammad Al Jiboul, Dr. Bashar Abu Saleem, and Mr. Abdel Salam Al Omoush. The Jordan Health Communication Partnership project director, deputy director, and technical staff provided an informative briefing, answered myriad questions, and provided a comprehensive set of project documents. In addition, JHCP planned the team schedule. The JHCP staff worked together with the evaluation team to accompany the team to various meetings and gave their valuable time to the team through the evaluation process. The team would like to thank the entire JHCP staff for their time, resources and assistance throughout the evaluation process. The team would also like to thank the numerous partners of JHCP who gave of their limited time to discuss their experiences, opinions on the project, and their individual partnerships. In addition, several people from USAID cooperating agencies met with team members to discuss their own projects and coordination with JHCP. The team gained much from these meetings. The team is grateful to the religious leaders, university students, and the staff, trainers, and trainees of Arab Women Speak Out, who gave team members insight on the perspective of beneficiaries through the focus group discussions. The University Ambassadors made a special effort to attend the focus group discussion and the team thanks them for their contributions. USAID/JORDAN: EVALUATION OF THE JORDAN HEALTH COMMUNICATION PROGRAM i ii USAID/JORDAN: EVALUATION OF THE JORDAN HEALTH COMMUNICATION PROGRAM CONTENTS ACRONYMS AND COMMONLY USED ARABIC WORDS............................. v Acronyms................................................................................................................................... v Commonly Used Arabic Words.......................................................................................... vi EXECUTIVE SUMMARY .....................................................................................vii Introduction ............................................................................................................................. vii Selected Evaluation Findings................................................................................................. vii Evaluation Findings on Other Project Elements................................................................ x Recommended USAID Future Role in FP Behavior Change Communication............ x I. INTRODUCTION ......................................................................................... 1 Evaluation Purpose and Objectives ...................................................................................... 1 Summary Statement of Work................................................................................................ 1 II. BACKGROUND............................................................................................. 3 Factors Influencing Population Growth Rate in Jordan.................................................... 3 III. EVALUATION METHODOLOGY.............................................................. 7 IV. EVALUATION FINDINGS ON BCC ACTIVITIES................................... 9 Overview of the Project ......................................................................................................... 9 Findings......................................................................................................................................11 V. EVALUATION FINDINGS ON OTHER PROJECT ELEMENTS........... 33 Monitoring and Evaluation....................................................................................................33 PARTNERSHIPS ......................................................................................................................36 JHCP Management and Structure .......................................................................................40 VI. RECOMMENDED USAID FUTURE ROLE IN FP BEHAVIOR CHANGE COMMUNICATION ................................................................................... 43 Rationale for Further Investment........................................................................................43 Recommended Strategy for FP BCC Project...................................................................43 USAID/JORDAN: EVALUATION OF THE JORDAN HEALTH COMMUNICATION PROGRAM iii APPENDIXES APPENDIX A. SCOPE OF WORK.................................................................... 51 APPENDIX B. PERSONS CONTACTED......................................................... 59 APPENDIX C. REFERENCES ............................................................................ 65 APPENDIX D. FOCUS GROUP INSTRUMENTS AND SUMMARY OF FOCUS GROUP DISCUSSIONS ...................................... 69 APPENDIX E. RECOMMENDATIONS ON THE HEALTH COMPETENT SCHOOLS INITIATIVE............................................................ 95 APPENDIX F: RECOMMENDATIONS ON CONSULT AND CHOOSE..... 97 iv USAID/JORDAN: EVALUATION OF THE JORDAN HEALTH COMMUNICATION PROGRAM ACRONYMS AND COMMONLY USED ARABIC WORDS ACRONYMS AWSO Arab Women Speak Out BCC Behavior change communication CA Cooperating agency CBO Community-based organization CC Consult and Choose CSPD Civil Status and Passports Department CPR Contraceptive prevalence rate DHS Demographic and Health Survey EC Emergency contraception FGD Focus group discussion FP Family planning FP/RH Family planning/reproductive health GDA Global Development Alliance GIS Geographic information system GoJ Government of Jordan HCSI Health Competent Schools Initiative HHC Higher Health Council HPC Higher Population Council HPI Health Policy Initiative HPP Health Policy Project HSS-II Health System Strengthening Program IEC Information, education, and communication Iftaa Mufti Department JHCP Jordan Health Communication Partnership JHU/CCP Johns Hopkins University Center for Communication Programs JNFW Jordanian National Forum for Women JPFHS Jordan Population and Family Health Survey JRTV Jordanian Radio & Television Corporation JTV Jordan TV JUST Jordan University of Science and Technology KS Knowledge station USAID/JORDAN: EVALUATION OF THE JORDAN HEALTH COMMUNICATION PROGRAM v LDD Local Development Directorate MCH Maternal and child health M&E Monitoring and evaluation MOE Ministry of Education MOH Ministry of Health MOI Ministry of Interior MAIAHP Ministry of Religious Affairs, Trusts and Holy Places NGO Non-governmental organization NITC National Information Technology Center PMP Program monitoring plan PSP Private Sector Project RH Reproductive health RHAS Royal Health Awareness Society RL Religious leader SOW Scope of work TFR Total fertility rate TOT Training of trainers COMMONLY USED ARABIC WORDS Awqaf Trust or endowment for properties or other assets donated to Islam and managed by the Ministry of Religious Affairs and Holy Places Iftaa The Mufti Department Imam Religious leader and preacher at a mosque Fatwa An interpretation of Islamic law Sharia Islamic system of religion-based law Waizat Woman religious leader Hayati Alha My life is more beautiful Mabrouk Congratulations vi USAID/JORDAN: EVALUATION OF THE JORDAN HEALTH COMMUNICATION PROGRAM EXECUTIVE SUMMARY INTRODUCTION The Jordan Health Communication Program (JHCP) Evaluation was commissioned by USAID/Jordan to analyze progress and achievements of the project, approximately 15 months prior to the project’s conclusion. The main objective was two-fold: first, to review progress and achievements for each of the objectives listed in the scope of work (SOW), and second, to inform future directions and priority setting for USAID-supported family planning (FP) behavior change communication in Jordan. The evaluation was designed to assess consistency of interventions with both Jordanian national and USAID health strategies, and to assess activities for effectiveness and impact on large family norms and birth spacing. The evaluation also assessed the role of partnerships between JHCP and related USAID cooperating agencies (CAs), relevant government ministries, non￾governmental organizations (NGOs), and the private sector. Effective partnerships are essential for institutionalization, capacity building, and leveraging of impacts. The recommendations in this report should inform USAID/Jordan’s decision making on extending, discontinuing, or transferring current FP communication programming, as well as decisions regarding the scope for new approaches and activities. JHCP is an eight-year cooperative agreement implemented by the Johns Hopkins University, Center for Communication Programs, with a funding level of $26 million. Designed in 2004, the program’s purpose was to promote healthy lifestyles by: (1) implementing a national health communication strategy using the life-stages approach; (2) assisting local communities in adopting healthy behaviors; and (3) supporting child spacing and the small family norm. Starting in May of 2008, the project has focused more on FP rather than broader healthy lifestyle issues. The evaluation team was directed to focus primarily on JHCP’s performance against the program’s revised objectives to promote FP behavior change through promotion of a three-year birth spacing interval and promotion of a cultural shift in favor of a smaller desired family size. Annex A provides the evaluation SOW. SELECTED EVALUATION FINDINGS The evaluation team found that JHCP’s activities have been effective in training selected audiences and communicating behavior change communication (BCC) messages through media and Internet campaigns. Involvement of religious leaders has shown to increase discussion of FP messages, but not all trained leaders integrate FP messages into their sermons. The media campaign was well-designed, but the fragmented nature of media in Jordan prevented it from reaching the broadest possible audience. JHCP has used innovative Internet activities to provide internal-standard health information to a wide audience. Despite the training provided by JHCP to health service recipients, health care providers, and religious leaders, many trainees still report high ideal fertility rates, and rates of contraceptive discontinuation remain high. Evaluation findings on BCC activities and other project elements are detailed below. Religious Leaders • JHCP facilitated the issuance of a fatwa (an interpretation of Islamic law) that found FP, contraception, and birth spacing to be acceptable practices. The fatwa addressed the widespread misconception that large families are an Islamic value, when they are in fact a traditional or cultural value. USAID/JORDAN: EVALUATION OF THE JORDAN HEALTH COMMUNICATION PROGRAM vii • The involvement of religious leaders in addressing FP issues was a powerful intervention that increased the impact of all other JHCP activities and other USAID FP projects. • Monitoring data suggest that the training and follow-up need to be improved to increase religious leader activity and address their remaining biases and discomfort discussing FP. The unique participation of Jordanian women religious leaders in the training facilitated more direct, less structured communication with women in the community. • The involvement of religious leaders in discussions about birth spacing and contraception has opened up public dialogue on these previously taboo topics. • The strong partnership between the two religious agencies and JHCP is reflected in the stakeholder’s stated desire to institutionalize training and expand activities to the national level. • Participating religious leaders have a high opinion of USAID, despite earlier misgivings. Hayati Ahla Media Campaign • The media campaigns have been well designed, according to many Jordanian officials and media figures, an observation confirmed by the reported levels of recall and actions taken by audience members. The media also used the JHCP spots to fill unsold space at no cost to the project. • The evaluation results suggest that media coverage was not as far-reaching as expected, probably due to the fragmented nature of Jordanian media and the dominance of international and regional media. JHCP Internet Activities—Sehetna • JHCP has been innovative and creative in its use of the Internet, particularly as a tool to reach young people. • The Web site Sehetna1 successfully provides international-standard health information, which helps counteract misconceptions held by youth, health services recipients, and health providers. It also effectively disseminates JHCP and Ministry of Health (MOH) materials. • JHCP’s Internet technical capacities have strengthened JHCP’s partnership with MOH and the Government of Jordan (GoJ) and set standards for quality. • JHCP has institutionalized Internet resources by ensuring local ownership of Sehetna and Healthcom. However, the sustainability of these resources is unclear, though many aspects of the sites are owned and controlled locally. Mabrouk Packets for First Birth Couples and Newly Married Couples • Mabrouk is a cost-effective, innovative way to reach a primary audience for FP messages at a national scale. There are no dissemination costs beyond materials production. • JHCP’s work with Civil Status and Passports Department (CSPD) is an innovative partnership. • The CSPD has taken ownership of Mabrouk to the degree that it has allocated an operating budget for the activity, but the budget is inadequate for reproduction of Mabrouk packets. TANMIA Software for Development Planning • A planning tool such as the TANMIA software falls outside the JHCP focus on FP and general health BCC. It also appears that TANMIA may duplicate activities in other parts of the GoJ. 1 Sehetna, Healthcom.gov.jo. Accessed via the Knowledge Station, 5-10-15. viii USAID/JORDAN: EVALUATION OF THE JORDAN HEALTH COMMUNICATION PROGRAM • TANMIA was designed to support local planning under decentralization. Since Jordan’s timeline for decentralizing planning is unclear, this activity could be premature. • The resources needed to fully implement TANMIA are significant. There seem to be no guarantees that these resources exist or will be committed by GoJ or USAID. Arab Women Speak Out Community Mobilization Activity • Arab Women Speak Out (AWSO) is a powerful tool for empowering women with positive models and information. Presenting sensitive FP messages in the context of larger health and social issues helped women accept them. • AWSO reached a secondary audience, with many women reporting that they talked to friends and neighbors about FP and other topics. In addition, AWSO materials and techniques have been adopted by health projects that work at the community level. Health Competent Schools Initiative • JHCP was successful in institutionalizing Health Competent Schools Initiative (HCSI) in the Ministry of Education. The activity will continue to get USAID Office of Education support. • HCSI is cost effective and low maintenance. Once materials were distributed to schools and the teachers trained, there was limited continuing involvement needed with individual schools. • While community outreach was not a major component in the design, during implementation schools used it extensively with parents, then with the larger community, introducing health competency to many more beneficiaries than a school-based program could reach. • Behavior change and expanded awareness of health competency issues among students was substantial, suggesting a long-term impact on student health. Ambassadors/University Training for Selected Students as Ambassadors • The training for ambassadors was insufficient to address student shyness about FP. It also failed to give them tools for addressing FP issues (e.g., gender-specific groups, instructional DVDs or protected Web sites, external expert speakers). • Irregular follow up from JHCP and a lack of a longer-term plan of activities reduced longer￾term impact and institutionalization. • Student peer approaches to communicating are effective, but are labor intensive, have a relatively small reach, require audience-specific messages and material, and must include continuing motivation. Consult and Choose Counseling Materials • While Consult and Choose (CC) training and materials and presence of a designated midwife counselor have increased the quantity and quality of information provided to clients, there are still missed opportunities for disseminating information. • The counseling training did not fully address provider biases. • Staffing should be an issue of concern for MOH and the Health Systems Strengthening Program. Several people with counseling assignments shifted responsibilities depending on workloads. • Discussions with women at health centers point to gaps in private sector support for contraceptive methods, with counseling rarely available at pharmacies. The centers should offer counseling to all clients, especially for user-dependent methods like the pill and condoms. USAID/JORDAN: EVALUATION OF THE JORDAN HEALTH COMMUNICATION PROGRAM ix • CC exemplifies a collaborative working relationship with other USAID projects focused on FP. EVALUATION FINDINGS ON OTHER PROJECT ELEMENTS Monitoring and Evaluation • The lack of a clearly articulated higher-level FP results statement or objective has contributed to JHCP’s being pulled away from its FP focus. It also makes it difficult to judge progress toward an objective that is not clearly defined. • Although USAID believes it has actively promoted the development of project outcome measures, JHCP has not yet incorporated them in its project and activity descriptions. • JHCP has done good formative research and activity evaluation, but has failed to follow up with secondary research and analysis. The program appears to favor performance monitoring at the expense of programmatic analysis. Partnerships • The network of local partners that JHCP has built is strong and partners regard their relationship with JHCP as positive. The partnerships deserve credit for the success of some activities and for the institutionalization and going-to-scale that has taken place. • The large number of JHCP partners and activities has probably limited the potential contributions of some activities due to human and financial resource constraints. • JHCP has established a good model for public-private partnerships. With the precedent now established, the positive experience with Fine Paper Products has opened the door for MOH to undertake future partnerships. • The four USAID projects have cooperated, with JHCP’s sharing benefitting the other projects. Many other opportunities for collaboration exist. JHCP Management • Although there have been issues related to turnover in project directors and staff, the current management system and structure seem to be working well. • JHCP’s change in focus from healthy lifestyles to FP has led to a continuation of health messaging. In some instances, this has provided an entry into discussion of FP and partnership-building opportunities, but the balance between topics has not always favored FP. • JHCP has generally been successful in balancing project and political agendas and the program agendas of partners. Achieving this balance has come at a cost to the project, but also explains its effective partnering and credibility with the GoJ. RECOMMENDED USAID FUTURE ROLE IN FP BEHAVIOR CHANGE COMMUNICATION The evaluation team recommends that USAID/Jordan continue to support BCC and that BCC be a major thrust for USAID for the next three to four years. This recommendation arises from a review of the demographic data (Jordan Population and Family Health Survey – 2007 and 2009) and is supported by the following evaluation findings: • The stated goal of the GoJ and USAID are a reduction in the growth rate through a reduction in fertility. This can only be achieved by a change in the cultural norms favoring large families (mean desired number of children in Jordan: 4.2). x USAID/JORDAN: EVALUATION OF THE JORDAN HEALTH COMMUNICATION PROGRAM • Although quality of care can be improved, FP service provision is not a constraint to the adoption and use of contraception. Access to public and private health services, contraceptive knowledge, and contraceptive use levels are already high. Cost is not a constraint to use. Improved quality of care can affect fertility timing, but will not change fertility levels without a change in desired fertility. • The structure and partnerships already exist for communication initiatives, an area in which USAID already has credibility. • Success in changing fertility norms will increase the effectiveness and impact of other USAID programs. Rationalization of roles and better coordination between USAID FP projects will increase synergy and impact. A review of the population sector portfolio is the first step in this rationalization. Based on the evaluation’s findings, BCC priorities should be the following: the involvement of religious leaders, as key influencers of social norms, to champion FP at a national level; and a mass media strategy using a range of available channels to support religious leaders’ messages, educate couples and providers, and create demand for modern FP methods. The evaluation team’s recommendations on current JHCP activities that should be continued are based on the independent effectiveness of each activity. The team also recommends that a future project should be more strategic as well as horizontally integrated; of these two recommendations, the project’s strategy should take priority. Each JHCP activity should be reexamined in light of the project-wide strategy; if an activity does not fit, it should be discontinued. USAID/JORDAN: EVALUATION OF THE JORDAN HEALTH COMMUNICATION PROGRAM xi I. INTRODUCTION EVALUATION PURPOSE AND OBJECTIVES The Jordan Health Communication Program (JHCP) is an eight-year cooperative agreement implemented by the Johns Hopkins University’s Center for Communication Programs, with a funding level of approximately $26 million. The program has the following objectives: • Coordinate and integrate all health behavior change communication (BCC) activities and programs in Jordan • Provide strategic, integrated, cross-cutting BCC to achieve health competence and sustainable health behavior change across life stages • Enhance Jordanian capacity in BCC and institutionalize sustainable BCC systems • Invoke and sustain support among leaders, policy makers, stakeholders, and decision makers for an environment conductive to behavior change and adoption of healthy lifestyles As designed in 2004, the program’s purpose was to promote healthy lifestyles by: (1) implementing a national health communication strategy using the life-stages approach; (2) assisting local communities in adopting healthy behaviors; and (3) supporting child spacing and small family norm. Since May 2008, the project has shifted its focus from broad-based healthy lifestyles to family planning (FP). USAID/Jordan commissioned the JHCP evaluation approximately 15 months prior to the program’s conclusion to analyze JHCP’s progress and achievements. The main objectives of the evaluation are two-fold: first, review progress and achievements for each of the objectives in the scope of work (SOW), and second, inform priority setting and future directions for USAID￾supported BCC in Jordan. The evaluation team was directed to focus primarily on JHCP’s performance against the revised program objectives from May 2008: Promote behavior change regarding FP through (1) promoting spacing of at least three years between pregnancies, and (2) promote a cultural shift toward small family size (See Annex A for the SOW). The evaluation was designed to assess consistency of interventions with Jordanian national and USAID health strategies, and assess activities for effectiveness and impact on large family norms and birth spacing. The evaluation also assessed the effective use of partnerships between JHCP, the related USAID cooperating agencies (CAs), and relevant government ministries, non￾governmental organizations (NGOs), and the private sector. Effective partnerships are essential for institutionalization, capacity building, and impact leveraging. The recommendations in this report should inform USAID/Jordan’s future decisions on extending, discontinuing, or transferring current FP communication programming, and the scope for new approaches and activities. SUMMARY STATEMENT OF WORK The evaluation was conducted during a five-week period in September and October 2011. The evaluation methodology is summarized in Section III of this report. A team of three technical experts with backgrounds in reproductive health, family planning, maternal and child health, behavior change communication, gender issues, and program management and evaluation conducted the evaluation. The deliverables included the evaluation work plan, including interview instruments, a presentation of findings and recommendations, the draft evaluation report, and a final report, reflecting suggestions by USAID/Jordan, as appropriate. USAID/JORDAN: EVALUATION OF THE JORDAN HEALTH COMMUNICATION PROGRAM 1 2 USAID/JORDAN: EVALUATION OF THE JORDAN HEALTH COMMUNICATION PROGRAM II. BACKGROUND FACTORS INFLUENCING POPULATION GROWTH RATE IN JORDAN Jordan has a population of approximately 6.1 million and a relatively advanced health care system compared to other countries in the region. The population growth rate of 2.2% is one of the highest in the region, and continues to be a major development constraint, especially in light of the quantity and quality of services that need to be provided to accommodate this rapidly increasing population and the fact that Jordan has a limited natural resource and economic base. The 2009 Jordan Population and Family Health Survey (JPFHS) found a total fertility rate (TFR) of 3.8, with virtually no change from 3.7 in 2007. The stagnation of fertility decline is a cause for concern, as declines in fertility rates are typically associated with increases in a population’s rate of education, increased social and economic opportunities for women, and awareness of the individual and social cost of having many children. Also of concern is that, while fertility rates are stagnant, the absolute number being added to the population increases as the growing population base of young men and women enter the reproductive stage of life. The lack of change is due in large part to a broadly held social value in favor of large families. The 2009 JPFHS reported an ideal family size of 4.2 children. The widespread desire for large families is reflected in stagnant national contraceptive prevalence rates (CPR). Contraceptive prevalence has leveled off for all methods at 57% (JPFHS, 2009), as compared to 56% in 2002 (JPFHS, 2002). Many other factors contribute to Jordan’s high fertility and plateauing levels of contraceptive use, including high discontinuation rates, use of traditional FP methods, government incentives for large families, shortages of female health care providers, health provider biases against some modern methods and for large families, poor quality and missed opportunities for counseling, a gender bias in favor of sons, traditional values and lifestyles, and cultural and religious beliefs that favor large families. The Jordanian Environment for Family Planning It is essential to understand the environment that influences the way FP is integrated into Jordanian society. Demographic goals. The current National Population Strategy is directed at reducing population growth by reducing fertility to replacement levels –TFR 2.1 – by 2030. This policy emphasizes achieving fertility reduction by changing family size norms.2 Proven FP interventions that do not have a direct impact on fertility, like improved access, quality of care, introduction of new methods, and so on are considered secondary. Political will. Interviews with stakeholders in the Ministry of Health (MOH), the Higher Population Council, the Higher Health Council, and USAID indicate that there are concerns at the highest levels of government about rapid population growth. These concerns are generally not translated into strong programmatic support for FP, because (1) the large family norm is culturally strong and gives the appearance of having some basis in religious values, making direct opposition politically risky; and (2) FP has a lower priority than other health agendas (e.g., chronic health issues and maternal health) among those involved in health policy decision making. 2 The Higher Population Council, The Demographic Opportunity in Jordan “A Policy Document, Amman, October 2009, page 27. The document states: “Policies that accelerate the demographic shift and reach the demographic opportunity period – Achieve targeted fertility rate of 2.1 births per woman in 2030, passing through the rate of 2.5 in year 2017 according to the recommendations of the National Agenda.” USAID/JORDAN: EVALUATION OF THE JORDAN HEALTH COMMUNICATION PROGRAM 3 Large family norms. The 4+ ideal family size transcends education, income, urban/rural residence, and other traditional fertility preference determinants. Also important is an almost universal belief in the importance of having sons (JPFHS, 2009). Based on interviews with beneficiaries, the commonly stated ideal family is “two boys and two girls.” In effect, current fertility levels in Jordan can be characterized as desired. Given this reality, there will not be significant declines in fertility and population growth rates, or increases in use of effective contraception, until the desired family size changes. A strong health service delivery system. Health services are widely available and used in Jordan. Nearly all women deliver in hospitals (99%) and receive antenatal care (99%); most have their children fully immunized (88%). The country’s public sector health infrastructure has virtually complete coverage in urban and rural areas. There is an active private sector health delivery system ranging from private practitioners to specialized private hospitals, with the private sector providing 54% of FP services. FP services are widely available – only 0.1% of women not intending to use FP gave cost as a reason, and just 0.2% gave access as the reason for not using it in the future (JPFHS, 2007). On average, FP supplies (pills and condoms) are available in drug stores throughout the country (one out of three pill and condom users buy supplies in private pharmacies). The poor have access to free services and contraceptives in public clinics, although services can be limited by a lack of contraceptives or trained providers, as well as by provider biases. Jordan’s health care system has resulted in low maternal and neonatal mortality and high immunization rates. This raises two issues: first, the health delivery system obviates traditional maternal and child health rationales for FP. With the exception of preventing high-parity births, FP will have only marginal impacts on maternal health. Second, a glaring “missed opportunity” in the system is a lack of postpartum FP services and FP counseling for women during their regular visits to health facilities. The Health Systems Strengthening Program (HSS-II) is attempting to address “missed opportunities” in public sector health facilities. A homogenous population. A common approach to development interventions – and, specifically, health communication – is to segment the audience into target groups for specific messages and behavior change. This approach is not particularly effective in Jordan. The traditional determinants of health behavior and FP are age, education, urban/rural residence, and economic status. In Jordan, these variables correlate in the expected way – fertility is higher among the poor than among the wealthy – but the difference between classes is small. For example, if interventions targeted the poorest quintile of the population to raise their FP use to the level of the next highest group, and the interventions succeeded in this goal, the CPR would increase by less than one percentage point. Various studies3 using multivariate analysis usually find that these variables explain very little of the differences found. This suggests that segmenting audiences or targeting subpopulations will produce limited impact at the national level; it also suggests that national strategies are likely to generate the biggest impact, given that Jordan is a small country. That said, segmenting can be an effective strategy to reach youth, who use different channels of communication, need a different set of messages, and require greater sensitivity to norms related to sex. With youth more open to new ideas, any attitude or behavior change will produce long-term impacts. 3 JHCP, Evaluation of the Mabrouk II: You've Become a Mother and Father Initiative, November 2010, and Religious Leaders' Knowledge, Attitudes, Practices and Public Speaking about Family Planning and Reproductive Health: Results from Baseline and Endline Surveys in Zarqa Governorate, Jordan, 2010. June 2011. 4 USAID/JORDAN: EVALUATION OF THE JORDAN HEALTH COMMUNICATION PROGRAM The youth bulge. The rapid rate of population growth guarantees a large and growing number of young people. This bulge in the age profile of Jordan is a concern of the Government of Jordan (GoJ), as it is in many Middle East countries. The cost of providing social services and the potential for political and social unrest are common concerns. Further sensitization on this issue has two potential impacts: (1) it can increase the policy and programmatic support for FP; and (2) it contributes to an increased acceptance and awareness of messages focused on the importance of “quality over quantity” in making family size decisions. USAID/JORDAN: EVALUATION OF THE JORDAN HEALTH COMMUNICATION PROGRAM 5 6 USAID/JORDAN: EVALUATION OF THE JORDAN HEALTH COMMUNICATION PROGRAM III. EVALUATION METHODOLOGY The evaluation team assessed project performance through the following steps: • The team reviewed a wide range of documents and data, such as JHCP project-specific documents, reports on USAID health activities, JPFHS data, country-specific documents, and GoJ documents. (Appendix C contains a list of references consulted.) • Interviews were held with a variety of stakeholders in both individual and group discussions, including project beneficiaries, project officials in the various partner ministries, NGOs, and USAID CAs. Some sources were interviewed more than once to clarify issues arising from fieldwork or to achieve better understanding. Translators accompanied the team to provide impartial translation of the discussions. Dr. Khalaf also provided translation or validation of translations. Appendix B provides a list of stakeholders contacted; Appendix D contains interview and focus group discussion guides used by the evaluation team. • Site visits were carried out to selected clinics, schools, NGO offices, and project sites to view activities and speak with implementers. • The team solicited beneficiary and participant views using seven focus group discussions (FGDs) and group interviews. FGDs were held with university student participants in the Ambassador initiative, religious leaders participating in training of trainers (TOT) in Irbid and Zarqa, TOT participants in Arab Women Speak Out (AWSO), and community members receiving AWSO training. Discussion groups were held with religious leaders who received training from TOT participants in Irbid and Zarqa. Appendix D summarizes FGD findings. JHCP staff developed an agenda for the site visits, selecting the sites and people to be interviewed outside Amman. For the FGDs, JHCP staff provided the names of 20 individuals and Dr. Khalaf randomly selected and invited 10 from each list. One goal in developing the evaluation methodology was to ensure forthright discussions by maintaining the anonymity of the respondents. To provide access and introduce the team, JHCP staff attended many of the meetings, but did not participate unless asked a direct question. When possible, the staff provided introductions and then left the room. The discussions were positive on the subject of JHCP support, regardless of JHCP staff presence. When respondents discussed activities, they raised both positive and negative issues, backing up their points with illustrations and explanations. A typical balanced criticism was “it was very difficult in starting up, but now we understand each other and everything is very easy.” The team had no reason to believe that the presence or absence of JHCP staff had a substantive impact on the content of the discussions. Of more concern was the role of JHCP in planning the field exercises. In Zarqa and Irbid, site visits were made to schools and clinics where the team could see a full range of project activities. By definition, these facilities were successful in their implementation, and were able to inform the team of what had worked and what had not. Facilities that had failed to implement or were still in the implementation process were not seen, and therefore did not inform the evaluation. Participants in the focus groups and discussions were those available to JHCP and knowledgeable about activities; the team did not speak with dropouts and those who failed to maintain contact with the implementing group, whether JHCP or NGOs. The seven FGDs were conducted with one evaluation team member working as a facilitator and a note taker with no relation to JHCP. With the dialogue ranging from laudatory to critical, the team believed it received a fairly complete picture of what did and did not work in implementation, with only a slight slant toward the positive. USAID/JORDAN: EVALUATION OF THE JORDAN HEALTH COMMUNICATION PROGRAM 7 8 USAID/JORDAN: EVALUATION OF THE JORDAN HEALTH COMMUNICATION PROGRAM IV. EVALUATION FINDINGS ON BCC ACTIVITIES OVERVIEW OF THE PROJECT Alignment with GOJ Policies and Strategies JHCP’s focus on FP addresses the goals of the Updated National Population Strategy 2000–2020. This strategy seeks to lower the fertility rate to 2.5 by 2015, and 2.1 (replacement levels) by 2030. This decline in fertility will be achieved by increasing contraceptive use with attention to modern methods, along with increasing the participation of men in FP-related programs and improving information, education, and communication (IEC). Unfortunately, the policy statements do not mention reducing desired family size, which is key if there is to be an increase in contraceptive use and a concomitant decrease in fertility. In addition, the JHCP focus on FP supports specific aspects of the National Health Communication Strategy of 2004–2008 and the National Health Strategy 2011, which recommend that the high total fertility rate should be a priority area to address among peri￾marital young adults (see JHCP activities: Mabrouk, Ambassadors, Health Competent Schools Initiative [HCSI]). Also, the Ministry of Health Strategic Plan 2008–2012 centers on preventive and curative health services, with a greater focus on primary health care (see Hayati Alha, Consult and Choose, Sehetna). Contribution to USAID Country Strategy The USAID/Jordan Country Strategy 2010–2014 supports the GOJ’s strategy to lower the fertility rate and increase the CPR with attention to modern methods, male participation, and improved IEC. The JHCP contribution falls under the USAID/Jordan sub-intermediate result: “use of family planning services is increased.” JHCP contributes to the following USAID/Health performance results indicators: “percent of women who have heard a specific FP/RH (reproductive health) message, percent of women who waited at least three years since previous birth, and percent of women who report preference for a small family size.” JHCP FP Approach and Activities JHCP’s stated objective for its BCC interventions is: “to provide strategic integrated BCC to achieve health competence and sustainable health behavior change across life stages.” The project uses a range of communication channels to increase awareness and knowledge to generate changes in attitudes and behaviors. The messages include couple dialogue, child spacing, religious and cultural beliefs regarding FP, gender equity in valuing children, use of modern reversible contraceptive methods, and encouraging couples to visit a health provider and to use FP services. The table below provides an overview of the BCC activities that are covered in this section of the report. USAID/JORDAN: EVALUATION OF THE JORDAN HEALTH COMMUNICATION PROGRAM 9 Table 1. Overview of JHCP BCC Activities Activity Communication Channels Coverage Target Group(s) Main Partners Mass Media Hayati Ahla Radio TV Newspapers Print Billboards National General population Memac Oglivy advertizing firm (creative and media plan) MOH and MAIAHP (review) Internet - Sehetna - Healthcom Internet Web sites National and beyond Adults, health care providers, youth MOH, (content), and NITC Knowledge stations (Internet), MOICT Mabrouk Print materials National Registering couples - Newly married - First births CSPD TANMIA Planning tool for district government Interactive planning tool, mapping, and database Pilot in Irbid, for use in local government directorates MOI local development directorates MOICTMOI/LDD Irbid Religious Leaders (RLs) Training - RL sermons, teaching, and counseling -Print materials Zarqa and Irbid governorates Religious leaders and their communities MAIAHP and Iftaa AWSO-trained facilitators to mobilize and empower communities Six 2-hour community sessions using interactive educational tools Select areas in Zarqa and Irbid governorates Women (also men in Zarqa) 22 CBOs in Zarqa and JNFW in Irbid Health Competent Schools Initiative (HCSI) - Print materials -Teachers -Events/dramas -Contests -Parent mtg. 115 secondary schools Secondary school students (2) Parents and community Ministry of Education, Fine Paper Products Co. (private sector partner) University Interpersonal training Print materials Trained selected students as ambassadors Three universities in Irbid University students and secondarily local community JUST, Yarmouk, and Irbid Alahlia universities Consult and Choose Print materials DVD National through HSS-II Health facility FP counselors and clients MOH HSS-II 10 USAID/JORDAN: EVALUATION OF THE JORDAN HEALTH COMMUNICATION PROGRAM FINDINGS Religious Leaders Objectives and Implementation JHCP involved the Ministry of Religious Affairs, Trusts and Holy Places (MAIAHP) and the Mufti Department (Iftaa) as key partners in building on previous JHCP work with Muslim religious leaders (RLs). MAIAHP oversees the work of more than 3,000 religious leaders (imams and waizat [women RLs]), who are employees of the ministry. The MAIAHP is also responsible for the content of the preaching and guidance programs. Iftaa is a government-sponsored department involved in the interpretation and explanation of Sharia (Islamic law). Its role in interpreting religious texts is both spiritual and secular. These interpretations are presented as a fatwa, which has the standing of a law. Iftaa is independent and an equal partner with the MAIAHP. It is headed by the Grand Mufti and has a staff of 30 muftis located throughout the kingdom. Since cultural norms and practices are often incorrectly attributed to Islam, the Iftaa partnership has been critically important in establishing Sharia positions related to FP. Since March 2007 JHCP has supported the MAIAHP Family Health Program as part of an overall strategy to link spiritual, social, and physical well-being to create an inclusive definition of family health. It is under the rubric of family health that issues related to FP are addressed. Starting in July 2008, JHCP supported three study tours for 25 men and 5 women religious leaders to the International Islamic Center for Population Studies and Research in Egypt. The purpose of the visits was to gain first-hand knowledge of Egyptian Islamic support for FP, reproductive health, youth, and gender issues. In early 2010 JHCP and MAIAHP launched the initiative to train religious leaders to address the MAIAHP’s family health agenda. The training used a step-down approach: select RLs who participated in the TOT course, then work in pairs to train other RLs. The two-day step-down training covered numerous issues, including spousal communication, the positive position of Sharia on the acceptance of modern, reversible contraceptive methods,4 contraceptive technology, the health/economic/social benefits of FP, communication skills, gender equity, and related health topics. The training prepared participants to incorporate these issues into their Friday sermons, religious lessons, and counseling. The training in Zarqa Governorate served as a pilot and led to finalization of the training process and materials. The changes made in Irbid based on the Zarqa experience were participation of two Mufti in the TOT as trainees, who then led sessions in each step-down training to respond to RL questions, and the addition of two days to the TOT to improve group facilitation skills. Iftaa issued a fatwa titled “Family Health” that supports FP within a family health context. The fatwa states Sharia acceptance of reversible, nonpermanent modern contraceptive methods, “quality not quantity” of children for the health of the child and the parents, birth spacing, and gender equity (to address the cultural preference for sons). In total, 47 RLs, including 11 women, participated in the TOTs. They trained 972 RLs, (including 201 women): 372 in Zarqa and 600 in Irbid. As a result, 93% of all RLs in Zarqa and 75% in Irbid participated in the training. All received materials including sermon guides with verses from the Quran that support the training messages; flash cards about population growth; and information on nonpermanent modern FP methods. The step-down training ended with development by the participants of a six-month plan for applying the knowledge and materials in their work. 4 The Zarqa training used a fatwa from Egypt. USAID/JORDAN: EVALUATION OF THE JORDAN HEALTH COMMUNICATION PROGRAM 11 Data from JHCP's monitoring of the application of participants' six-month plan are presented below. Governorate Time Frame Religious Lessons Conducted Friday Sermons Conducted Community Members Attending Religious Lessons and Friday Sermons Men Women Total Zarqa Mar.2010– Sept. 2010 203 98 51,824 4,410 56,234 Irbid Mar. 2011 – July 2011 460 226 90,260 13,360 103,620 Total 663 324 142,084 17,770 159,854 The numbers of community members reached is large, but closer examination suggests that religious leader training is not delivering its potential or expected impact. Of the 771imams trained, fewer than half gave sermons incorporating messages on family health (i.e., related to FP), even though the training incorporated models and verses to address the messages in Friday sermons. The data shown above from Zarqa, covering six months, indicate that less than a third of the 323 trained imams integrated family health into one of their sermons during this period. In the four-month reporting period for Irbid, it appears that among the 448 trained imams, no more than half addressed family health in one of their sermons.5 It should be noted that performance may have been under-reported if the reporting instructions used a very narrow definition of sermons with messages. The findings from the JHCP evaluation of the Zarqa RL trainees and the evaluation team's findings (presented below) suggest likely reasons for the relatively low number of sermons on family health. Results JHCP evaluated a sample of the Zarqa trainees with a self-administered questionnaire prior to training and six months after the training. The results show increases in imam trainee knowledge and approval of the FP and population issues covered in the training. Generally, the indicators rose between 15 and 25% but never reached100% for any of the knowledge or approval indicators. Only 51% in the post-training study agreed with the statement that couples should space pregnancies at least three years and the reported ideal number of children a family should have was unchanged at six, in the pre- and post-study. Possible reasons for these findings include shortcomings in the trainings and external cultural factors. Data from the three evaluation team FGDs with men and women RLs in the Irbid and Zarqa governorates provide additional information on the effect of the training (see Appendix D). The trainers reported that the TOT increased their communication abilities, decreased shyness, enabled them to discuss issues in front of large groups, and empowered them to be stronger individuals. The most frequent questions they received from trainees were on the position of Islam on FP and spacing of children. The fatwa was used as a point of reference in addressing questions. Both RL trainers and trainees discussed FP issues from the context of the quality of children, the health and welfare of the family and women and children, child care and rights, women's inheritance rights, spacing, couple’s relationships, planning for the future, and communication between children and parents. Imams did not discuss specific FP methods, which they considered inappropriate, in their Friday sermons. However, the women RLs indicated that they discuss such issues in their religious lessons. The FGDs found more approval and support 5 As the data do not reveal whether certain imams included messages more than once, this very basic analysis indicates the maximum proportion of trained imams who included family health in their sermons during the reporting period. 12 USAID/JORDAN: EVALUATION OF THE JORDAN HEALTH COMMUNICATION PROGRAM for the messages than anticipated from the Zarqa pre-post test and the monitoring data on the six-month work plans. Most of the religious leaders in the FGDs reported that the sessions changed their perspectives on FP and other health issues presented. The changes included considerations of the relationship between males and females; the importance of quality rather than quantity of children; the need to avoid discrimination between boys and girls; and importance of consideration of the wife's feelings during pregnancy. Many of the Zarqa participants said that they are convinced of the importance of birth spacing and have advised their own children to plan their families and ensure appropriate spacing between children. One interesting issue raised in the training was the aim of the project and its funding agency. The presence of the muftis and their clarification on the messages, project, funding agency, and partnership changed attitudes positively. An RL noted: “Before the training we did not trust the project and those who direct it from outside Jordan. We used to think that this is a step toward the limitation of the number of the children, but after we participated in the training we discovered that the project is a trustful [trustworthy] one as well as the funding organization.” Views of Participants and Key Stakeholders • The participants in the evaluation team's FGDs were satisfied with their training, but both trainers and trainees recommended spreading the training out over more days to include more overall training time, but to shorten training days to under eight hours. • Participants suggested a physician attend the training to present a more technical discussion of modern FP methods and answer questions. • In the Zarqa FGD with AWSO trainees, participants wanted women religious leaders to participate in one of their AWSO sessions, indicating the important influence of these RLs. • In the team's interviews with religious leaders and the mufti in Irbid and Zarqa, it was widely acknowledged that the training encouraged discussion on key topics related to FP. The introduction of FP has been difficult since having children is often perceived as a form of worship. The problem was made more difficult by the misinterpretations by some RLs of the position of Islam and participation by a number of pronatalist RLs. The training has been successful in getting even conservative RLs to listen and consider the issues. • There is widespread acknowledgment by JHCP key partners and stakeholders that the involvement of religious leaders has been a critical step toward desensitizing the topics around FP, increasing the acceptance of the role of FP in family health, and fostering the use of modern contraceptive methods. Capacity Building and Institutionalization • The MAIAHP and the Mufti Department have been actively engaged in the technical and managerial aspects of the training of religious leaders. They were successful in introducing social and health issues into the normally spiritual roles of the mosque. During the coming months MAIAHP plans to enact steps to systemize guidance on talking about family health topics in Friday prayer in Irbid and Zarqa. At MAIAHP's suggestion RLs are being trained in the Jerash Governorate, a high-fertility area. • JHCP has been successful in building relationships and transferring ownership to MAIAHP and Iftaa. Both organizations have adopted FP as part of their mandate and JHCP, as a technical advisor, provides resources for fulfilling this mandate. • Many of the trainers in the FGDs were keen to serve as trainers in other governorates to spread the benefit of the training. "This program is owned by us. If JHCP leaves we will carry USAID/JORDAN: EVALUATION OF THE JORDAN HEALTH COMMUNICATION PROGRAM 13 on," said a ministry official. Both religious agencies are committed to expanding the family health and FP communication activities nationwide. In the near future, though, GoJ budgetary support for the activity is unlikely without continued partnership with expansion of external donor support for the training throughout Jordan. • The messages about health and FP are being institutionalized by their addition to the preservice curriculum of religious leaders. This will be done under JHCP in response to a request from key RLs. • The issuance of the fatwa will continue to define the dialogue on FP and smaller families in Jordan, with or without JHCP. A key committed mufti has recently suggested that muftis provide advisory services on the content and wording of media messages. Conclusions • The involvement of religious leaders in addressing FP issues was a powerful intervention that increased the impact of all other JHCP activities and the other USAID FP projects. Religious leaders were able to achieve this because they were able to identify attitudes toward FP and large families as a cultural or traditional value and not a value with a religious basis. They were then able to provide a religious basis for using FP. • JHCP, MAIAHP, and Iftaa have established a strong partnership, as evidenced by the two religious agencies actively identifying opportunities for advancing key FP messages. This effective partnership offers expanded opportunities for embedding FP messages with an emphasis on health benefits into the work of RLs. • As MAIAHP and Iftaa are well organized and managed, there is capacity for effective geographic and technical expansion of their role. They also have a grassroots infrastructure that lets them operate at any level. In Irbid and Zarqa they have champions at all levels able to credibly address the community on issues related to FP. • The two day step-down training was a success and made advocates of some of the participants. Nevertheless, the Zarqa pre-post evaluation of the training indicates uneven knowledge and acceptance of key FP messages. Even more unsettling was the lack of support for addressing the critical issue – high fertility. The Zarqa training did not sufficiently address the fact that religious leaders often believe in a high ideal family size (six). Some modifications were made for the Irbid training. The forthcoming pre- and post-test results should indicate whether the Irbid training led to any changes in the social ideal of high fertility and whether there were greater positive changes related to knowledge and acceptance related to FP. If not, the training strategy should be reviewed, and the key factors constraining change should be identified and addressed. • The evaluation team's findings suggest that the relatively low number of sermons given by the trained RLs may be related to the length of the training, the absence of a mufti and physician at trainings to answer questions, use of a fatwa from Egypt related to modern contraceptives (none existed in Jordan at the time), and the monthly guidance on sermons given by an imam for his governorate. The team was pleased that the MAIAHP plans to address the latter. • The unique participation of Jordanian women religious leaders, or waizat, in the training expanded the dialogue and facilitated more direct, less structured communication with women in the community. While the waizat do not deliver sermons in mosques, they give lectures, religious lessons, and counseling in mosques or in Islamic women centers. In these centers, they have the opportunity to provide more detailed FP information directly to women (i.e., method-specific information). • A broader and more strategic approach to the RL training might have generated more impact. Illustrative activities could include refresher training, specialized training for women 14 USAID/JORDAN: EVALUATION OF THE JORDAN HEALTH COMMUNICATION PROGRAM RLs, follow-up on trainees by the MAIAHP, and greater recognition of imams and mosques with successful activities, • The religious leader training and implementation, even with its limited trial, was a powerful tool in addressing cultural biases against FP. The infrastructure and willingness are in place to take the initiative to a national level. The single most important channel of communication to address the large family norm, fear of modern methods, health provider biases, and inertia of some key policy makers is to have the religious community speak with one voice on the importance of family health and FP. Without continued support from USAID, it is highly unlikely that the training of RLs on FP within the context of family health will be scaled up nationwide. Hayati Ahla—National Family Planning Campaign Objectives and Implementation The JHCP national Family Planning Campaign is carried out under the logo of Hayati Ahla, meaning “My life is more beautiful.” Under the BCC objective, the campaign centers on reaching people with FP messages through mass media (TV, radio, Internet, press, and outdoor signs/banners). Besides placing ads on the TV and radio and in the press, the campaign has also provided content to newspapers, TV, and radio programs. The campaign is intended to increase knowledge and influence behaviors. An interim campaign was launched through media placements between July and September 2009 (by another USAID project PSP), which included spots on two specific contraceptive methods, the IUD and the pill, drawing on materials from a 2002 JHCP campaign. It also included religious spots on gender and spacing of children, which were timed to coincide with the holy month of Ramadan. This interim campaign was a joint effort of JHCP, the Private Sector Project (PSP), the MOH. The first FP campaign under the Hayati Ahla logo was carried out between December 2009 and March 2010. It included four TV commercials and dramatic spots. The action tag for the campaign focused on choosing the modern contraceptive method that is right for you. An advertising agency carried out creative design and pretesting, and managed the reviews with key stakeholders (JHCP, MOH, MAIAHP). One interesting result of pretesting was that emotional messages were not as effective as fact-based messages. This suggests that there is already an interest in FP, but credible information is not readily available. Three TV spots were broadcast on Jordan Television for free after the conclusion of the paid commercials.6 All of the messages were developed through consensus of the key stakeholders. Building consensus was initially slow, but has improved with experience and a better understanding of various partner viewpoints. The campaign has included a variety of ways to present the messages, which have been tailored to specific audiences and disseminated through different media channels as appropriate. For example, the national campaign from May to August 2011 included four cartoon TV spots reflecting concepts of FP, spacing, spousal communication, and gender equity. Messages during the campaign continue to be disseminated through TV, radio spots, newspaper ads and articles, advertising handouts, and outdoor billboards, banners, and posters. In addition to the campaign, which had a limited timeframe, a local radio station began a weekly program titled Hayati Ahla; press releases were added in local newspapers; and TV carried out interviews with JHCP staff and partners. The media attention continues to keep a range of health topics, including FP, in the media and in public view. The media campaign stimulated the interest of some media journalists in Amman. Thereafter, JHCP supported media coverage by providing workshops in mid 2011 to increase journalists’ 6 This is generally done with spots that have the highest production values and the most public interest. USAID/JORDAN: EVALUATION OF THE JORDAN HEALTH COMMUNICATION PROGRAM 15 understanding of FP issues and skills needed to professionally address these issues. The media trainings in Amman and Aqaba, held in cooperation with the Higher Population Council, enabled journalists to better understand attitudes toward FP. The trainings highlighted the need for correct information and discussed how journalists could meet those needs with properly developed messages, introducing approaches to using media as an educational tool. As a result of the trainings, media journalists have to varying degrees incorporated FP issues into their work. This coverage complements the JHCP-paid Hayati Ahla media messages: for instance, a TV talk show host invites key people to discuss FP issues on her program, and a health economist writes newspaper articles that include the economic implications of high population growth. Hayati Ahla Internal Evaluation Results JHCP bought questions on the Nielsen Omnibus Survey (a recurring market research survey) to evaluate the Hayati Ahla 2010 campaigns. Recall of the Hayati Ahla campaign was lower than might be expected for a multimedia campaign – 23% unprompted and 30% when prompted by the Hayati Ahla logo. This is probably a function of the fragmented coverage of Jordanian media. Higher socio-economic households and women had more exposure to the messages. Messages on TV (Jordan TV) had the highest recall. Among those who heard the messages, the level of follow-up actions was high, nearly 40%. Actions included discussing the issues covered with others and convincing spouses to space future pregnancies and to use modern contraceptives. Views of Key Stakeholders • According to the three journalists with whom the evaluation team met, the involvement of religious leaders and the media campaign have broken down the barriers to free discussion of population and FP issues in the mass media. The journalists felt that the JHCP training was extremely useful and gave them skills to cover social issues, including FP and population issues. The journalists reported that contests associated with radio coverage of FP issues has generated word-of-mouth promotion of the specific program and increased listenership in the Amman area. Some listeners, intent on winning the contest, take notes while listening to the program. • A number of stakeholders commented on the high quality of the media campaign. The evaluation team learned that the royal family saw and liked the spots. • Memac Ogilvy Advize was responsible for the creative development of the messages and for media placement. The agency believes that it has had a powerful impact, in that it opened the media up to formerly taboo topics. “Birth spacing was once taboo, but is now a tired old message.” Memac Ogilvy also got considerable recognition from the parent company for doing socially responsible advertising, doing it with high production values, and partnering with the government. • The consensus among stakeholders is that the campaign would never have happened without USAID support and JHCP leadership. Conclusions • The involvement of religious leaders in discussions around birth spacing and contraception has opened up media and public dialogue on these previously taboo topics. • The media campaigns have been well-considered and designed. This opinion is held by many highly placed Jordanian officials and media people, and confirmed by the reported levels of recall and action taken by audience members. The evaluation results suggest that the media coverage was not as far-reaching as would be expected in a mass media campaign with multiple channels and lots of bought and free time and space. This is probably due to the fragmented nature of media in Jordan, where radio and TV are mostly local; weekly or daily newspapers serve more local audiences; and satellite TV, which has much higher viewership 16 USAID/JORDAN: EVALUATION OF THE JORDAN HEALTH COMMUNICATION PROGRAM than local TV, was unavailable to the campaign. Jordan TV has high viewership, but only during news programs (when Hayati Alha spots were bought). The fees for media space are based on the size of the audience reached; coverage would increase if the media budget were higher and more local-level media sources involved. • The media training and health information provided by JHCP established a network of radio, TV, and print journalists who are able to integrate FP issues into their work. • There was clearly a synergy between the messages of the religious leaders and the media campaign. • The survey instrument used in the JHCP evaluation of Hayati Ahla includes questions on Islam’s position on a variety of related issues, and the results showed very high levels of agreement between Islam and the messages in the campaign. However, the design of the questionnaire suggests the level of agreement may reflect when the questions were asked (after the questions on exposure to religious messages) and the use of all positive statements, which can encourage an automatic response. The phrasing of the questions should be reexamined before the next Omnibus survey is launched. JHCP Internet Activities Objectives and Implementation To increase awareness and knowledge, JHCP under its BCC objective has used the Internet to reach youth and provide health professionals access to a source of empirically based health information, including but not limited to reproductive health. This is important because both health care providers and the public have high levels of misinformation. Use of the Internet has also provided access to JHCP products. Materials can be downloaded from the Sehetna or the MOH’s Health.com Web site. These Web sites disseminate JHCP materials for use in non￾project areas and by NGOs not involved in the project. Jordanians have relatively high levels of access to Internet services (38%),7 so the Internet is a good channel for reaching these people. Currently none of the other USAID health projects have used the JHCP Internet channels for dissemination. Sehetna (Sehetna.com, Sehetna.com.jo, and Sehetna.com.eg) Sehetna.com began in first half of 2007 as a single Web site built by JHCP in partnership with the MOH and the Egyptian Communication for Healthy Living Project. As a site of health information in Arabic, it has attracted visitors from all over the world. Moreover, in late 2010, Sehetna became three parallel Arabic health Web sites designed to increase the health awareness of the public, especially youth. The Jordanian Web site (sehetna.com.jo) is tailored to address national health priorities, as defined by the MOH. The sites are also used by other JHCP activities: the Health Competent Schools Initiative and the University Ambassadors activity. As part of the JHCP University youth-focused activities in the Irbid Governorate, three awareness sessions centered on Sehetna were conducted at the Jordan University for Science and Technology (JUST), attended by 1,320 students. Sehetna is heavily visited, with almost 27,000 visits in the last quarter of 2010. It is also connected through Facebook™ and has around 500 followers on Twitter.™8 Sehetna contains about 1,200 health “articles,” which are one- to two-page summaries of the most current empirically based information, written for a nontechnical audience and reviewed by a panel of physicians. Sehetna is currently expanding the reproductive health content. Other features 7 Information and Communication Technology Association, Ministry of Information and Communication Technology, “ICT & ITES Industry Statistics and Yearbook – 2010,” Amman Jordan, p. 15. 8 JHCP Quarterly Report for April-June 2011[[There are JHCP 2011 reports in the reference section but can’t tell if this is one of them. USAID/JORDAN: EVALUATION OF THE JORDAN HEALTH COMMUNICATION PROGRAM 17 include all JHCP products for public consumption, weekly Jordanian health news, a health calculator, daily age-specific health tips, useful Internet links, and “ask-the-expert.” This feature has generated 3,000 questions since its initiation. In most instances the “expert” refers the person to a document on the site, but otherwise a knowledgeable health professional responds. Sehetna is hosted in the United States because it is less expense, and the systems are more stable. In the phase out of JHCP, however, MOH is expected to take full responsibility for the Web site. 5-10-15 5-10-15 is an interactive life planning program for use by young people. The tool, currently in testing, is meant to enable Jordanian youth to plan their future by connecting four areas of their life: career, family, lifestyle, and finance. 5-10-15 helps users see how decisions in one area affect all other areas. For example, users can see how the number of children they have will affect finances and lifestyle. When it is finished, 5-10-15 will be supported by the National Information Technology Center (NITC), which is an independent operational extension of the Ministry of Information Communication Technology. The Knowledge Station The Knowledge Station is a program sponsored by the GoJ to support social development by providing Internet access to poor communities in underserved areas. There are 183 stations, each with computers and a manager/advisor. The program is run by the NITC, which also maintains all of the GoJ Internet servers (gov.jo). JHCP collaborated with NITC to train a staff member and to get the stations in the Irbid and Zarqa governorates to promote Sehetna. The NITC will also be the home and promoter of 5-10-15 when testing is finished. Healthcom.gov.jo JHCP helped develop and institutionalize the MOH Web site, which is hosted by NITC and maintained by the Health Information and Communication Directorate in MOH. Healthcom is primarily intended for use by MOH staff in both Amman and in the governorates. It has a large archive of materials, including those developed with JHCP, policy statements, reports, health Internet links, and materials from all the directorates in the Ministry. NITC provides hosting for the GoJ and hosts Healthcom, but they are not as stable and do not have the development support available outside the country. Views of Key Stakeholders Sehetna is widely recognized as professional, credible, accurate, and current. These attributes are highly respected by MOH officials, who feel the Web site is a major accomplishment and the standard against which other indigenous Internet products will be judged. JHCP and their partners are given full credit for their contribution. The MOH and other partners value the credibility of an international organization ensuring the accuracy of the information on Sehetna. NITC thinks that Sehetna makes “excellent” use of the Knowledge Station network in Zarqa and Irbid to connect with hard-to-reach segments of the population. They feel that the activity with the Knowledge Station should be national in scale, with more staff trained and engaged in promoting the Web site through posters and handouts. While JHCP has institutionalized Internet resources by ensuring local ownership of Sehetna and Healthcom, the sustainability of these resources is unclear. GoJ partners are not confident in local capability and skills to keep the sites up to date and responsive to audience needs, though many aspects of the sites are owned and controlled locally. The partners also feel that the local groups lack the network and the skills to find and incorporate the latest international health information. Another concern is that while many Jordanian Web sites are hosted locally, the 18 USAID/JORDAN: EVALUATION OF THE JORDAN HEALTH COMMUNICATION PROGRAM servers are normally located outside the country, making it easier for an international organization like JHCP to quickly address server-based problems. . Conclusions • JHCP has been innovative and creative in its use of the Internet, particularly as a tool to reach young people. • Sehetna successfully provides international-standard health information, which often resolves misconceptions by youth, health services recipients, and health providers. It also effectively disseminates JHCP materials, and strengthens JHCP’s partnership with MOH and GoJ. • JHCP has institutionalized Internet resources by ensuring local ownership of Sehetna and Healthcom. However, the sustainability of these resources is unclear. • The Web sites provide an archive that ensures that JHCP materials in Arabic will continue to be available to the widest possible audience in Jordan and the Arab world. Mabrouk and the Civil Status and Passports Department Objectives and Implementation Under its BCC objective, JHCP works in partnership with the Civil Status and Passports Department (CSPD), which is responsible for registering all births of Jordanian citizens and issuing the mandatory “Family Book” to register every newly married Jordanian couple.9 These functions offer a unique and innovative opportunity to distribute FP and related materials to these two groups of clients who are making decisions about family formation. The Mabrouk (meaning “congratulations”) Initiative centers on 76 national CSPD offices that distribute tailored packets of materials to couples who recently married (Mabrouk I) and parents of all first-born babies (Mabrouk II). Implementation began in 2008, with CSPD staff from all governorates attending an orientation session. Subsequently, 75,000 copies of both Mabrouk I and Mabrouk II packets were distributed to all of the CSPD offices in Jordan. In 2011, another orientation session was held for 200 CSPD staff to introduce the new versions of Mabrouk10 and to discuss the findings from the 2010 evaluation of the initiative. An additional 100,000 copies of each packet were provided to CSPD for distribution. The cost of reproduction of each packet was US $0.87. In addition, select, larger CSPD offices have video screens playing Mabrouk messages together with other information, and all offices have Mabrouk posters. JHCP support for the CSPD computer system was part of the partnership on Mabrouk. The system is functioning and has earned CSPD awards for its quality customer service. On the Web site, individuals can register for birth certificates and Family Books, and SMS messages are sent to alert them when their documents are ready for pickup together with their Mabrouk packet. Internal Evaluation Results An evaluation consisted of telephone interviews with newly married couples who were issued their Family Book between July and November 2008, and had provided a telephone number to CSPD. The sample is not representative so caution should be used in generalizing the findings. The evaluation found that those who had received the package liked it, most had read the materials, and almost all who read the materials found them beneficial, particularly the FP information. Similarly, Mabrouk II was evaluated through telephone interviews among a sample 9 A couple receives a “Family Book” from the CSPD upon marriage registration Similar to a small passport, the “Family Book” serves as a family’s government ID. 10 This updated version was also authorized by USAID to be distributed without the logo, because it was perceived to politicalize the materials and some people did not accept them. USAID/JORDAN: EVALUATION OF THE JORDAN HEALTH COMMUNICATION PROGRAM 19 of couples who registered their first child between 2008 and December 2009. Most of the sample of 639 persons found it useful, and men especially found the FP information useful. Views CSPD considers promotion and distribution of the Mabrouk materials as its social responsibility. CSPD reports that the parents often request the Mabrouk II package when they are registering a child who is not their first-born. Capacity Building The system is institutionalized and is projected to be sustained by GoJ budget allocations, although the budget is insufficient to cover all reproduction costs of Mabrouk materials. CSPD reports that Mabrouk is integrated into the organization – “originally it was just another task, but now it is totally ours.” One interesting aspect of the Mabrouk activity was the change in attitude of the CSPD staff. Mabrouk started as another job, and was then transformed into a social responsibility and a source of pride. The senior managers at CSPD used the JHCP staff as models to be emulated for their dedication to the well-being of the people. JHCP has increased the capacity and efficiency of the CSPD document process by supporting the development of the department's Web site to accept electronic applications. CSPD's expansion to SMS messages to notify applicants that their documents are ready for pick-up has facilitated the acquisition of a higher number of telephone numbers, allowing a wider base for sampling in JHCP evaluations. CSPD is also institutionalizing a system to track distribution of Mabrouk materials. Conclusions • Mabrouk is a cost-effective way to reach a primary audience. There are no dissemination costs beyond materials production. • Mabrouk packets have national coverage because they are being distributed throughout the kingdom to Jordanian citizens. • JHCP’s work with CSPD represents an innovative partnership. • CSPD has an allocated budget for the initiative (2013-2017), but it is inadequate for reproduction of Mabrouk packets and continuation without financial support is unlikely. Given that the distribution of health education materials is not the mandate of CSPD, funding for material production will likely remain a problem. • Based on the JHCP evaluation, both Mabrouk I and II were well received by the intended audience. Newly married couples and new fathers found the FP information most beneficial. • The JHCP evaluation yielded information on issues that should be addressed in the future, including lowering the ideal or intended number of children and reducing the fear of side effects of modern FP methods. TANMIA Objectives and Implementation Note: TANMIA is not a FP activity, so it falls only loosely under the scope of the evaluation. Also, the evaluation team does not have the background to assess economic and social planning models. Because of these limitations, the comments of the team should be considered in the context of a limited examination of the activity. Implemented under the BCC objective, TANMIA is an interactive planning tool for use by the Ministry of Interior (MOI), whose officers head the government structures at the governorate and district levels. The tool is based on geographic information system (GIS) mapping that allows 20 USAID/JORDAN: EVALUATION OF THE JORDAN HEALTH COMMUNICATION PROGRAM for geographic and other kinds of data to be overlaid on a map, drawing on information from two MOI databases and data from other sources. The initial plan in early 2011 was for JHCP to introduce a “demonstrable prototype” focused on health planning in the Irbid Governorate. However, the director of the Local Development Directorate (LDD)/MOI governor in Irbid requested that the Irbid model incorporate other development data to ensure the sustainability of the tool within LDD/MOI and across the kingdom. JHCP agreed to two phases: 1. JHCP would develop the tool to cover health and four other sectors so that comprehensive data could be used in development planning in Irbid. 2. JHCP would build the capacity of a core group of LDD staff in Irbid to carry out the tool’s implementation across the remaining 11 governorates. In the latter phase MOI expects JHCP to train some 200 persons and to provide computers. These two phases of TANMIA require a significant portion of the JHCP budget. In the course of the evaluation team's meetings, it was learned that the HPC and MOH recently completed a GIS mapping of health facilities. Also GoJ has a unit that is responsible for GIS mapping. TANMIA appeared to be redundant with activities in other parts of the GoJ. JHCP reports that other agencies were inventoried to see if there had related mapping or database development activities. The one undertaken by the MOH (and funded by the GoJ) has been dropped in favor of the MOI effort (funded by USAID). Conclusions • A planning tool such as TANMIA falls outside the JHCP focus on FP and general health communication for behavior change. • TANMIA is redundant with activities in other parts of the GoJ. JHCP reports that its inventory of other agencies found related mapping and database development activities that have now been integrated into the MOI effort. • TANMIA will be used to support local planning when Jordan decentralizes development planning. Since there is no definite timeline for decentralization, this activity could be premature. • TANMIA started out as a health planning tool, but has metamorphosed into a broad￾spectrum economic development planning tool. In any case, its justification for incorporation into a FP project is extremely slim. • The resources to fully implement TANMIA are significant (i.e., training 200 staff, computers, 154 data entry staff, and the technical, managerial, and financial capacity to constantly update the tool). There are no guarantees of these resources on either the GoJ or the USAID side. • It is not clear if the development planning field was assessed to determine if the Johns Hopkins software is the most appropriate product for Jordan. • TANMIA may be the cost of developing a strategic partnership with MOI for future cooperation at local levels. However, many other projects seem to work at the local level, with the MOH taking the lead and MOI officials being informed. Consult and Choose Objectives and Implementation Consult and Choose (CC) is an activity to improve the quality of family planning/reproductive health (FP/RH) counseling in GoJ health facilities. The primary role of JHCP was to develop client materials, provider job aids, and training materials. The utilization of the materials is the responsibility of the MOH with support from the Health System Strengthening Program II (HSS￾USAID/JORDAN: EVALUATION OF THE JORDAN HEALTH COMMUNICATION PROGRAM 21 II).11 The materials were developed in collaboration with all the partners, including MOH. The activity started in the last part of 2008. JHCP produced a number of materials for the CC initiative: 2,000 posters for health centers on modern contraceptive methods; 3,500 FP medical eligibility wheels, 3,885 FP handbooks for health providers, and 4,225 service provider cue cards, as well as 250 client counseling checklists and 752,845 client cue cards. The latter are given to clients on the method they have chosen and include information on when to return for the second visit, how to deal with side effects, dialogue with their husbands, and religious messages. The materials are being distributed on a national level together with Hayati Ahla slogan pins and 2,235 service provider lab coats with the slogan. JHCP assisted HSS-II in integrating a section on proper incorporation of the materials into the curriculum and guidelines for the five-day comprehensive FP/RH training. JHCP provided special IEC materials to clinics, called “customized centers”: posters using pictures and testimonials from clinic staff and clients, and a flyer to inform clients of what to expect during the clinic’s counseling sessions. JHCP also developed a video/DVD called "Wait and Learn," using materials developed by all partners. Twelve health facilities in Irbid and Zarqa were equipped with screens and DVD players for use in the waiting areas of the clinics. The aim of the DVD was to increase demand for modern contraceptive methods and child spacing. In October 2010, JHCP started a four-hour follow-up training in Irbid. This orientation covered the proper use of CC using the FP tutorial counseling DVD. The orientation reached 100% of the midwives and 60% of the physicians at maternal and child health (MCH) centers in Irbid. Evaluation Team Observations: • A variety of FP IEC materials were evident at the two clinics visited.12 Most materials focused on the use of FP but not sufficiently on the critical behavior change messages related to smaller families and use of modern FP methods. • The policy is that all women clients are screened for FP needs by the MCH/general health provider. Referrals are sent to the midwife for counseling. However, it is not clear how effective this screening is in ensuring that all women are screened and counseled. • In the clinics observed, the clients getting counseling were those already seeking FP services, so there are still missed opportunities for counseling. • The clinics’ staffing structure for counseling was confusing. Midwives, doctors, and counselors all do counseling, but at one clinic, the counselor seemed to be the least busy person. • Clinic staff rated the training as “good but limited.” Suggestions for improvements include more practical/applied training; follow-up training; and a better training module on dealing with problem clients. Staff also recommended improvements to the training implementation, including added incentives and moving off-site. • There is a high discontinuation rate of oral contraceptive use. Improved counseling, especially on side effects, could reduce the discontinuation rate. • Trained doctors still hold biases against some modern methods. 11 The CC initiative is targeted at improved communication in FP counseling sessions between health care providers and clients to increase demand for modern contraceptive methods. The approach centers on counseling as a partnership between the client and service provider for client-informed decision making. CC aims at providing proper information and elimination of service provider biases by fostering professionalism and positive attitudes regarding modern contraceptives, pregnancy spacing, and small family size. 12 The team visited one health center in Zarqa and one in Irbid. 22 USAID/JORDAN: EVALUATION OF THE JORDAN HEALTH COMMUNICATION PROGRAM • The medical ruling on midwifes' inserting IUDs only under the supervision of a doctor did not seem to bother the two visited clinics. They had women doctors to insert,13 midwives trained to insert, and a doctor to supervise. One clinic had two clinics and one maternity hospital within five kilometers to refer clients to if no one was available to insert the IUD. • Counselors had the job aids. Since no counseling sessions were observed (privacy issues), it is not clear to what degree they were being used. During the visit to these two health centers, most of the women waiting for services came for their children's health issues. When asked about where they go for FP services, only one of the seven women, from Zarqa, said she uses the FP services at the health center. Two women used a resupply method (pills and condoms), purchased at a pharmacy. A third woman went to a private physician. The other women, each with three to four children, said they were beyond reproductive age. Internal Evaluation Results JHCP conducted an evaluation of the results of the CC training and material utilization in Zarqa. Exit interviews were held with women who were visiting the health center for an FP consultation or service. Exit interviews have a number of methodological constraints, so caution should be used when interpreting the results. The evaluation found that the FP providers generally followed most, but not all, of the recommended counseling practices; most used the counseling job aids and materials during counseling; clinic posters were effective; and clients valued the method cards. Views of Stakeholders People in MOH and other key stakeholders believe the focus on improved FP counseling standards by health care providers is essential to increasing contraceptive use. Specifically mentioned in MOH was the need to overcome provider biases toward short-term and less effective methods, and the need for delaying pregnancies after marriage. Some MOH respondents felt that male doctors tended to lack respect for the right of women clients to decide on an FP method among methods not contraindicated. Among the top recommendations made by participants at the September 2011 Higher Population Council's Reproductive Health/Family Planning Symposium is one that addresses FP method choice: The Symposium recommended: communication campaigns including use of health facility based interventions to promote FP concepts and market choices of modern FP methods, including long-acting methods among targeted groups. Institutionalization The MOH is in the process of institutionalizing CC. Its 2012 budget is projected to include 50% of the cost of materials. CC materials have been distributed throughout the system. HSS-II is doing training in selected districts and intends to expand. MOH is also proposing to do counseling refresher training for the clinical staff of all clinics. Conclusions • The results of the exit interviews suggest that CC materials supported by the training have made a difference in the quantity and the quality of information provided to clients. This conclusion is supported by the observations of the evaluation team, which found materials and staff with counseling strategies based on the training. The presence of a designated midwife counselor has also helped inform clients. 13 Women doctors are relatively scarce within the GoJ health system compared to the private sector health system. USAID/JORDAN: EVALUATION OF THE JORDAN HEALTH COMMUNICATION PROGRAM 23 • Staffing is an issue of concern for MOH and HSS-II. Several people had counseling assignments. They shifted the responsibility around depending on the situation in the facility. “When everyone has a job, nobody has the job.” • Opportunities were still being missed. • Discussions with some women at the health centers point to the role of the private sector in supplying short-term contraceptive methods. • CC exemplifies a collaborative working relationship with other USAID projects focused on FP. Irbid Hayati Ahla University Intervention Objectives and Implementation The Irbid Hayati Ahla /University Ambassadors activity is one part of JHCP’ youth outreach component. This activity was intended to inform and empower university students with the knowledge, skills, and practices to improve and sustain a healthy lifestyle. The implementation of the Ambassador activity started with the recruitment of ambassador/students at JUST (15 students); Yarmouk (5 students); and Irbid Alahlia (5 students). The ambassadors were trained by a JHCP team for two days to introduce the concepts of planning, including FP, career planning, financial planning, and healthy lifestyles to other students. They also used sections from the AWSO training guide and other material from JHCP and other USAID projects concerned with youth and FP. JHCP also integrated the AWSO tool in the community health curriculum for the nursing students at JUST University. All ambassadors were introduced to the Sehetna.com Web site as a reference for the health education sessions. Besides talking individually with students, the ambassadors at the three universities set up six Hayati Ahla booths to distribute 2,600 pamphlets and 900 posters, and give health talks. One university carried out community outreach with information booths in public places; two universities held two Hayati Ahla days. One university developed and performed a drama covering the issues for students and the community. All three universities attended and participated in the Hayati Ahla campaign at Al Hassan City in Irbid. A JHCP report on the ambassador activity noted several challenges: a short period for conducting activities; a drop-off in student participation after the first two semesters; improper selection of the ambassadors at one university; weak cooperation for integrating the Hayati Ahla concept within the curriculum at one university; difficulty in arranging meetings for the ambassadors and low attendance at these meetings (two universities); and an unclear incentive scheme for the ambassadors and supervisors, which reduced motivation (one university); weak media campaigns at the universities; and weak documentation of the activities by JHCP. Views of Ambassadors The evaluation team conducted an FGD with 13 ambassadors from the three universities (see Appendix D). They reported that the training covered healthy lifestyles, FP, financial and occupational planning, and water and population resources. Eight of the13 FGD participants felt that they needed more training on the different issues. Most of the participants felt that they need more training, in general. Family planning was the most awkward topic to discuss, especially in mixed gender groups. Capacity Building and Institutionalization The university ambassadors were interested and motivated to be part of this activity, although from August 2011 there was no follow up from JHCP to the ambassadors. Approximately half of the students trained as ambassadors are still active. 24 USAID/JORDAN: EVALUATION OF THE JORDAN HEALTH COMMUNICATION PROGRAM The role of the universities in the ambassador activity was not clear to the evaluation team. The universities approved the activity, but provided neither leadership nor resources to the effort. It also appears that they took no ownership of the activity. Conclusions • The training for the ambassadors did not adequately address the students’ shyness about FP. It also failed to give them mechanisms to address these issues (e.g., gender-specific groups, instructional DVDs or protected Web sites, external expert speakers). • Students perceived their primary role as raising awareness on healthy lifestyles. Generally they did not feel comfortable and informed enough about FP methods, and religious positions and cultural beliefs concerning FP, to make it the focal point of discussions. • In working with students, the FP messages are best presented as part of a broader healthy lifestyle package. Even in this context, the FP messages require extra effort such as additional counseling and training to transmit. • Irregular follow up from the JHCP and the lack of a longer-term activity plan reduced institutionalization and longer-term impact. • The number of activities and the continuing activity of some ambassadors suggest that peer￾to-peer activities can play a role in changing reproductive health knowledge and attitudes. • Creating a sense of ownership by the ambassadors and recipients of the issues would augment simply putting the information in the curriculum. • Student peer approaches to communicating health messages work, but they are labor intensive, require audience specific messages and material, and must include continuing motivation. They also have a relatively small reach. For these reasons the University Ambassador Initiative is not recommended for continuation. Arab Women Speak Out Objectives and Implementation AWSO aims to empower women through an interactive training package. Introduced into Jordan in 2002 by JHCP, the AWSO training manual was adapted and implemented under JHCP's BCC strategy. The JHCP AWSO training guide includes a mix of exercises and presentations of case studies and other talks, which facilitate participatory discussions. The guide is based on six two-hour modules. It covers a range of topics such as differences between generations, demands of modern life, planning for the future, negotiating skills, facing social pressure, couples sharing the decision to have a small family, Islam's position on FP, and contraceptive methods. It also covers health topics, such as women's health and child care. The AWSO model is designed to empower women to make informed decisions on a healthy lifestyle and a good domestic environment. This broader approach allows sensitive topics like contraception to be covered as part of a larger package of health information. The guide and the topics have been tested and applied in multiple countries in the region. It has been adapted to the situation and culture in Jordan and is an effective tool for training. PSP and HSS-II also use modules from AWSO. The content is extensive; any additions would probably exceed the time allotted for the training. It would, however, be possible to emphasize selected topics, like the benefits of small families and the side effects of modern FP methods. The guide’s coverage of social and broader health issues could be diminished with some caution. The interactive training approach facilitates participant discussions on issues and sharing experiences. At first glance only one section of the training guide (Men and Women Planning Their Family) stands out as focused on FP. However, sections leading to this FP section USAID/JORDAN: EVALUATION OF THE JORDAN HEALTH COMMUNICATION PROGRAM 25 incorporate important associated factors. For example, one section centers on comparing gender roles and economic life with those of your mother and grandmothers. This topic leads to a discussion of changing social norms – the small family – in response to the demands of modern life. Other examples of topics that lead up to the session on FP are life planning and negotiating skills. The title of the training and its packaging of FP and associated information appear to attract participants. AWSO was implemented in Zarqa Governorate in 2009–2010 as a pilot activity, working through government directorates and 22 community-based organizations (CBOs) that participated in the implementation. Although the initiative was aimed at women, when men expressed an interest, a few CBOs initiated separate training sessions for men. Some CBOs provided trainees with photocopies of materials from the guide so they would have a printed source to help them reflect on the training they received Other CBOs invited guest speakers such as health care providers and religious leaders. Approximately 5,000 women and 320 men attended the AWSO sessions given by approximately 115 trained facilitators. Two sessions a week were held over a three-week period to complete the training package. As a result of lessons learned in the Zarqa pilot, when JHCP launched AWSO in Irbid in February 2011, and under the guidance of the Irbid Steering Committee, they selected one NGO - Jordanian National Forum for Women (JNFW) – to implement and manage the activity to increase efficiency and the potential for institutionalization. In addition, to reduce the potential for drop-outs, the 12 hours of sessions were all held within a one-week period. JHCP hired a training group to train the facilitators and periodically monitor the quality of facilitators' sessions. A total of 51 women have been trained as facilitators, but due to lack of funds, to date they have trained 806 women. JHCP also supported JNFW in training five facilitators in Zarqa and their training of community members in mid 2011.14 Those attending the training often discuss issues with their spouses, friends, and other family members. To enable trainees to be informal peer educators, the plan for 2012 is to ask women who complete the AWSO training to disseminate information to at least 10 people. To enable them to correctly present information, JHCP will support the production of cue cards for distribution to these trainees. JNFW is also requesting a transport allowance to enable trainees to reach women outside their immediate communities. Findings In early 2011, JHCP conducted an evaluation of AWSO in Zarqa using telephone interviews with a sample of participants and a control group. The evaluation found that AWSO participants were more strongly committed to joint decision making in the family than nonparticipants. The number of AWSO training sessions attended positively influenced approval of FP, birth spacing, and spousal communication, but had no impact on attitudes on ideal family size.15 The reported impact of participation also differed between men and women: a higher proportion of men than women had discussed FP with their spouses. The evaluation team did one FGD with AWSO-trained facilitators, one with trainees in Irbid, and one FGD with both facilitators and trainees in Zarqa. • The AWSO facilitators in Irbid reported that they benefited from the TOT training on different contraceptive methods and their side effects, and how to address misconceptions and misinformation about serious side effects (i.e., methods cause infertility). They also 14 In Irbid, JHCP also trained five Peace Corps volunteers who have been active in promoting AWSO training in their sites. 15 This was based on comparing those who attended all six sessions with those who attended four to five sessions. 26 USAID/JORDAN: EVALUATION OF THE JORDAN HEALTH COMMUNICATION PROGRAM learned that modern, reversible contraceptive methods are allowed by Islam, and that using contraception is not shameful. • The trainees in Irbid said that they expanded their knowledge about contraceptive methods beyond pills and IUDs. • All participants had discussed the FP topics with their friends and neighbors and all accepted the idea of a small family. • The participants were convinced about the importance of FP and its acceptability in Islam. • All the trainees in the Irbid FGD said that they were satisfied with the information they received. They also believed that the trainers were skillful and that their approach to delivering information was excellent and convincing. • The referral cards given to AWSO participants got positive feedback in the Irbid and Zarqa FGDs for getting women to go to health centers for FP services. • Zarqa participants related small family size to economic well-being and better outcomes for children. • The Irbid FGD trainees mentioned the range of issues raised, including FP, motivating others, raising children, breast self-exam, and domestic violence. In comparison, the Irbid FGD facilitators listed, in descending order of importance: healthy lifestyle, MCH, planning for the future, and the need for couples to participate in organizing and ensuring their family health. • The training changed the perception of female participants, especially in Zarqa, about the need to make decisions to improve one's quality of life. Many of the participants commented on the ignorance of community members who have too many children and how they are not able to raise or care for them.16 FGD participants gave a range of responses about changes they made after training: opposing early marriage, seeking antenatal and postnatal care, taking care of their own health, and being more self-confident and responsible. • The title of the training – Arab Women Speak Out – has drawn some negative comments. It has also drawn attention because it is labeled as a women's empowerment tool. Particularly given the Arab Spring and its aftermath, several people noted that the title could be misinterpreted. • The team heard comments from people in Zarqa, Irbid, and Amman, who felt that the AWSO sessions on FP topics helped dispel misinformation on Islam's position, specific modern FP methods, and, combined with Hayati Ahla, has made discussions about FP no longer “shameful." • The FGDs covered the topic of men's participation in AWSO, since some men attended in Zarqa. In Irbid the facilitators thought it important that men be trained on the different contraceptive methods and their side effects. Some participants thought the sessions for men should include domestic violence, FP, resources, women’s health, and planning for the future. Some also suggested that a man should do the training and that the scheduling be more flexible. • Participants in all the FGDs indicated that community members need the AWSO training and it is important to continue. • In Zarqa some facilitators continue to train others but they could do more if they had financial support. Most of the facilitators in the Irbid FGD thought JNFW could take over implementation of AWSO, but the NGO needs financial support. 16 Comments included: “a woman delivers a baby and then immediately she is pregnant”;“a boy is without the slippers or trousers and the mother is pregnant and breast feeding another child”; “we have many problems with early marriage”; and ”the woman does not go for postnatal care for herself or her child.” USAID/JORDAN: EVALUATION OF THE JORDAN HEALTH COMMUNICATION PROGRAM 27 Capacity Building and Institutionalization. In the FGD conducted with four trainers and three trainee participants from Dalial in Zarqa, three of the four trainers reported that they continue to train and counsel other women even after the withdrawal of financial support from JHCP. They have the training manual, and use different settings at CBO s to provide the training. Two of the participants reported that they are involved in the women’s association and are using their training to address women’s needs. The findings indicate that the AWSO initiative in Zarqa built capacities and has been informally institutionalized in some areas. In Irbid, JHCP is providing guidance and technical support to JNFW to seek funding from other sources to enable them to continue to offer AWSO after completion of the JHCP project. JHCP may also support training of facilitators and provision of materials for another large NGO to adopt the AWSO model. Conclusions • The interactive training approach and training tools are of high quality. Parts of the training guide have been used to train university ambassadors and by the HSS-II in its community mobilization work, as mentioned in other sections of this report. • The topics covered topics that were in line with its goal to empower trainees to make educated FP choices. However, it appears that the links between the topics and FP are too subtle for some participants and do not support reproductive behavior change. • AWSO does not formally integrate the Family Health Fatwa into the training guide because it was produced after the training tool. • The finite time available in sessions constrains a more thorough presentation on FP, including more detailed information on method effectiveness and long-term methods. • The training in Zarqa led some facilitators to continue with the training sessions and incorporate the messages in their work after completion of JHCP support, which highlights the power and applicability of the training messages. • AWSO is a powerful tool that increases women's knowledge about a range of topics and increases their self-confidence. While the Zarqa JHCP evaluation found that AWSO was less effective in terms of FP, the FGDs suggest that some participants did increase their knowledge and thereafter actively disseminated FP messages. The FGDs in Zarqa, where fertility is high, appear to have increased knowledge and led to changing attitudes about FP and spacing. • Men are an important target group because the training related to FP tends to have a greater effect on them. This may be because their initial knowledge level is lower than that of women. Women in the FGDs felt that men should have the opportunity for training, suggesting that men should be a target group for interpersonal, interactive community outreach communication. • Limiting the number of CBOs or NGOs participating in the activity eases management and quality control, but it can reduce coverage and the diversity of populations served. Health Competent Schools Initiative HCSI is an activity to bring healthy lifestyles to secondary schools. JHCP has provided material for students, teachers, and school administrators, and has also supplied dynamic teaching methodologies. The participating schools also receive a stipend to get activities started. The goal is to get students to adopt and maintain healthy lifestyles while acting as champions in their families and in their communities. The initiative covers a range of topics. Passport to My Future covers future planning (including small happy families) through a booklet and exercises and is available through the school’s internet. Puberty and personal hygiene were addressed in the 28 USAID/JORDAN: EVALUATION OF THE JORDAN HEALTH COMMUNICATION PROGRAM Talking Frankly Initiative (with private sector sponsorship). Issues of bullying, diversity, coping skills, self esteem, and friendships were covered in Living Together. All of these issues were also supported with teaching materials and activity guides. HCSI is in 115 schools public schools, 15 military schools, and 10 private schools. Jordan has a total of 5,000 schools (3,800 government, 800 private, and 400 military). The Ministry of Education (MOE) is the primary partner in HCSI. One of the unique aspects of this activity was the public-private partnership with the Fine Hygienic Paper Company. Fine Co. sponsored materials and workshops, and equipped health rooms (tables, cots, decorations, scales). The company got involved because it has an active corporate social responsibility policy; its staff has enjoyed the activities. Its participation was low key in terms of branding, which was helpful in dealing with the MOH. Fine Co. saw it as an opportunity to do good and to grow the market for paper products (toilet paper, facial tissues, and feminine pads). Because of its large market share, anything that grows the market grows its sales. As a result, it felt no need to brand it support. HCSI also partnered with the Royal Health Awareness Society (RHAS). The society's role was to integrate HCSI materials within its own already developed materials to be implemented in its Madrasati Initiative schools with a capacity of 30-50 schools a year. The society is also an implementing partner to the MOE and USAID/Education in the scaling up of HCSI. The evaluation team’s site visit to Koufar Yoaba Secondary Girls School, Irbid, an award-winning Health Competency School, yielded the following information: • The MOE, in consultation with JHCP, selected the school to participate in HCSI. The faculty liked the idea and actively implemented it. • Activities include: a drama competition (tied for first in the national completion); a chorus singing and dancing to health songs written by students; posters and exhibits; a popular health room; community and parent involvement; a Facebook page and YouTube for activities; health day clinics for the community; and banning junk food (“chips”) in the school. • As part of the cleanliness messages, the chemistry teacher set up a “factory” where the students make cleaning disinfectant, liquid hand soap, and glycerin-alcohol rub. They buy the raw materials and empty bottles. They estimate the cost at .35JD compared with a 2JD cost for the equivalent commercial product. They are planning to expand production to include tooth paste. This activity raises the question of selling products within the community to generate funds to support other health activities. However, this is not possible because of strict MOE policies that prohibit external funding for public schools. • The program actively sought out parental involvement, with a positive reaction. Parents were invited to events, special lectures (e.g., a presentation by a gynecologist for the mothers of students), and exhibits at the school. The faculty members were also available to address questions or concerns. • Project support was relatively small: 100JD, materials, and a two-day training. HCSI Internal Evaluation Results This activity addressed a large range of health behaviors that are beyond the current scope of JHCP. However, the findings provided some interesting support for school-based strategies. • Exposure to the materials was high. • There was a strong gender difference in the interest in specific topics. • Students discuss the messages with parents, siblings, and friends. • Self-reported behavior change was common for a number of messages. USAID/JORDAN: EVALUATION OF THE JORDAN HEALTH COMMUNICATION PROGRAM 29 • There was a strong positive association between the number of materials a student was exposed to and behavior change – what is called dose effect. This is an important finding because it reaffirms the quality and impact of the materials, highlights the importance of multiple channels (e.g., booklets Web sites, classroom) for messages, and strongly suggests that messages should be embedded in the environment to allow multiple exposures and constant reinforcement. Views of Key Stakeholders An MOE official described the project as a “beautiful” and “excellent project,” providing “good quality support.” The JHCP project officer is highly regarded. An official in the RHAS noted they were impressed with the shared ownership, the ongoing efforts to institutionalize the activity, and the way multiple partners increased the available technical capacities. Fine Co. was positive about the partnership with JHCP: “They were very professional”; “There was a very strong sense of team work”; “All planning was done jointly and transparently”; “They were an excellent broker for the partnership with the MOH and MOE”; “Without JHCP there would have been no partnership.” Both MOH and MOE were suspicious of private sector participation. For this public-private partnership to work, JHCP had to play the role of “trustworthy advocate” to sell an unconventional relationship; act as a “buffer” between the two partners to resolve issues arising from different perspectives and approaches; and provide “technical expertise” for design and implementation. Fine Co. learned the following lessons: • Health competency is more difficult to achieve in boy’s schools. They need clearer messages and more reinforcement. For boys, there is a sense that health behaviors are optional. Girls are easier to work with because they get immediate benefits (feminine products). • The current materials and activities are the product of a considerable amount of accumulated wisdom. • The public sector does not know how to work with the private sector. One observer noted that he biggest problems are that the “public sector has a very short-term focus, and they are not strategic.” Capacity Building and Institutionalization While the program has been supported in 140schools, the HCSI materials are available to all schools in Jordan from the MOE Intranet site Edu-wave. The healthy lifestyles content of HCSI is being incorporated into curricula and text books. New teaching techniques were introduced to build capacity. A large number of teachers and school administrators can now train and use their schools and students as models to scale up the health competency program. All of the partners and participants want to see the HCSI expanded to every school in the country. Originally the idea of certifying health competent schools was proposed. The MOE is in the process of certifying schools using a broad range of criteria, which caused some confusion. However, JHCP was able to get health competency added as one of the criteria for certification, further institutionalizing the content of HCSI in the education system. HCSI has been transferred to the MOE and receives funds through a project in USAID/Jordan's Education Office. Conclusions • JHCP was proactive and successful in institutionalizing HCSI. • HCSI is cost effective. 30 USAID/JORDAN: EVALUATION OF THE JORDAN HEALTH COMMUNICATION PROGRAM • HCSI is a low maintenance activity. Once the materials were distributed to the schools and the teachers were trained, there was limited continuing involvement with individual schools. • Getting youth actively involved in health competency is a logical part of a long-term strategy for health in Jordan. • Because of the community outreach, the activity introduced health competency to many more beneficiaries than a school-based program would be projected to reach. • The behavior change and expanded awareness of health competency issues was substantial, suggesting a long-term impact on student health. USAID/JORDAN: EVALUATION OF THE JORDAN HEALTH COMMUNICATION PROGRAM 31 32 USAID/JORDAN: EVALUATION OF THE JORDAN HEALTH COMMUNICATION PROGRAM V. EVALUATION FINDINGS ON OTHER PROJECT ELEMENTS MONITORING AND EVALUATION Formative Research Communication activities need a strong empirical base to ensure the clarity and appropriateness of messages, media, and audience. The formative stage of design benefits from a strong theoretical basis, which the Johns Hopkins University Center for Communication Programs (JHU/CCP) has because of its academic roots. The JHCP media materials are screened by panels from the partner organizations. Then options are pretested with focus groups made up of members of the intended audiences. As many as six rounds of audience pretesting and review by the partners were required in the early stages of material development. The formative research practices of JHCP are standard in the field of communication. Monitoring The JHCP activity technical manager monitors each JHCP activity. For the religious leaders' training, JHCP has hired a person to monitor implementation in Irbid and persons to follow up the trainees in Irbid and Zarqa. For the most part communication between the activity and the JHCP manager has been regular. A few activities such as the University Ambassadors and Knowledge Stations reported a lack of follow-up. The monitoring tends to focus on inputs and outputs, and does not sufficiently consider the implementation process or provide the best tool to support the potential scaling up of an activity. Since FY 7 (Oct. 2010–Sept. 2011) the JHCP monitoring and evaluation (M&E) plan has included project performance outcome measures, which are carried over in FY 8. However, the plan does not meet USAID guidance for such plans. Efforts by USAID/Jordan to improve the performance monitoring system and the quality of chosen indicators have not yet resulted in improvements. The M&E plan includes output, outcome, monitoring, and outcome evaluation indicators for each activity, and in some instances there are similarities between the different types of indicators for an activity. As of October 2011, the FY 7 data were not available. Evaluation JHCP should have a results statement or objective with outcome indicators that reflect what it expects to accomplish by the end of the project. These indicators should reflect FP-related indicators in the USAID/Jordan's Social Sector's Results Framework. Indeed, the current research and evaluation officer maintains a core of four to five indicators related to the USAID indicators in all of the activity-level evaluations. However, JHCP’s lack of a clearly stated objective has contributed to the multiplicity of activities, and, in some cases, a lack of synergies. The evaluation team observed that the 2011 JHCP annual partners meeting did not begin with a brief review of what JHCP is expected to achieve or a high level statement about its FP focus. JHCP used the standard quantitative impact evaluation technique of baseline and follow-up surveys, but they have had problems that limited the utility of the data. The partners involved in implementing the surveys were not as involved as they needed to be; they did not respond to the need for additional analysis and did not provide the raw data set for the project to use. Another problem was that the baseline survey focused on healthy lifestyles, and FP was just one piece of the data set. The follow-up survey took place after the change of focus, so it had limited baseline data on FP. USAID/JORDAN: EVALUATION OF THE JORDAN HEALTH COMMUNICATION PROGRAM 33 The research and evaluation officer, together with a research specialist from JHU/CCP, has been responsible for the evaluations of the BCC activities. Once the reports are available they are translated into Arabic and shared with the responsible partners. The evaluation reports on religious leader training and AWSO were not available before these activities were launched in Irbid. Nevertheless, the expansion to Irbid did benefit from the monitoring in Zarqa and experiences gained in implementation there. Also, the evaluation of Mabrouk I led to changes in some of the materials and a workshop to motivate CSPD staff to be more diligent in distributing packages to the intended recipients. The evaluation team found no instances of JHCP using evaluation findings to conduct secondary research, conduct secondary analysis of the findings, or rethink and refine their communication activities to address critical issues, such as fears associated with use of modern FP methods. An analysis across activity evaluations would have indicated that there JHCP faces inter-related challenges in increasing the percentage of women who wait at least three years between births, the percent who prefer a small family size, and the percent who report the ideal family size to be less than four children. Also, at the individual activity level it appears that the evaluation findings were inconsistently used to refine or redirect the activity. The activity evaluation reports lack sufficient information on the sampling design for data users to understand the representativeness, the quality and coverage of the sample frame, etc., to allow for findings to be extrapolated with confidence. In addition, the evaluations normally include a question on ideal number of children in a family, which tests a social norm. The evaluations have not sought to find out if the attitudes, intent, or behaviors of the respondents differ from the norm. As a result, evaluations only vaguely measure the USAID indicator on small family size. Conclusion • The lack of a clearly articulated, higher-level FP results statement or objective for activities and the project has contributed to JHCP’s being pulled away from its FP focus. It also makes it difficult to judge progress toward an objective that is never clearly defined. • USAID believes that it has actively promoted the development of project outcome measures, but to date JHCP has not incorporated them in its project and activity descriptions. • JHCP has done good formative research, but has failed to follow up with secondary research and analysis. – Secondary research is used to answer questions that arise from the primary research or program implementation. Two obvious examples of JHCP issues for secondary research are ideal family size and husband’s roles in FP. Family size norms are the critical issue for behavior change communication in Jordan. JPFHS provides a measure of ideal family size, but “ideal” is an abstract concept. Secondary research should clarify the determinants, definitions, and process used to refine “ideal” into “I want/wanted X children.” Another issue is the role of husbands. Both AWSO and Mabrouk h found that men were interested in the FP information provided. High levels of condom use also indicate men are active in the household’s FP decisions. Secondary research could have examined men’s attitudes on family size and spacing, sources of information, method knowledge attitudes, roles of in-laws, etc. These studies could have led to a whole new line of activities. – Secondary analysis is done to maximize the utility of existing data. The evaluation team found a number of common issues as it went through the various activity evaluations and research documents (related to, media exposure, use of health services, method knowledge, various attitudes). The staff of JHCP may have absorbed and used some of 34 USAID/JORDAN: EVALUATION OF THE JORDAN HEALTH COMMUNICATION PROGRAM the information, but this is at best a limited use of existing information. If the qualitative and quantitative data had been synthesized and documented, they could be used consistently across JHCP activities, by new JHCP staff, by the partners, and for future project design. • JHCP appears to favor monitoring for performance at the expense of programmatic analysis. This is not surprising given that performance as measured by the Results Framework is USAID’s primary management and learning tool. JHCP should be using its M&E tools to learn what works. They do use findings to fine tune activities, but a higher-level analysis of the meaning at a program level is not in evidence. For this higher0order analysis to happen, skills in measurement, FP, and communication must be merged in one specialist or in a team doing a joint analytical exercise. • Without a clear description of the sample design in the evaluation reports, it is difficult to know whether the results are representative of the program or related only to the sample studied. Recommendations • JHCP should produce lessons learned and best practices reports for use in planning any follow-on projects. These reports would also be useful for the partners and other donors. Activities that are transferring (HCSI, CC) or ending (University Ambassadors) should start this analysis before the information is lost to staff changes and time. • The Results Framework should be reconsidered in the context of Jordan’s unique demographic and programmatic situation. • There is an over-reliance on the Demographic and Health Survey (DHS). The monitoring plan should use more process indicators and tailored data collation techniques, like sentinel sites, telephone and mail-in surveys, and observations of clinics to measure change. Omnibus surveys and sentinel sites are good examples of methodologies that measure change well. • A shared FP research officer could support all four FP projects. This could provide: higher level program analysis (including lessons learned and best practice reporting), integrated M&E plans, shared data and analysis, coordinating project M&E officers, and the cross-use of findings in program planning. • In the evaluations planned for the remainder of JHCP, all evaluation reports need to clearly state the criteria for selection of the sample and the sampling process. • Cost effectiveness should be a standard part of all evaluations of JHCP activities. • The evaluation team has recommended better monitoring of the quality of clinical services using existing tools like the “Quick Inventory of Quality.” Though the DHS is often over￾used, it is applicable to this context. The standard DHS has a number of questions that can be used as indicators of changes in the quality of clinical services – for example, more detailed knowledge of methods; informed choice; reasons for not using in the future – specifically “health concerns, fear of side effects”; reasons for source choice (public vs. private); side effects discussed; breast feeding; and FP. The JPFHS-Interim can also be used to monitor quality and to provide a benchmark for other efforts to measure quality.17 17 See Measure Evaluation Project, “Quick Inventory of Quality” or DHS Project, “Quality of Care Assessment Tool.” USAID/JORDAN: EVALUATION OF THE JORDAN HEALTH COMMUNICATION PROGRAM 35 PARTNERSHIPS JHCP Activities in Partnering As the name implies, the Jordan Health Communication Partnership was mandated from its inception to collaborate with other organizations that shared its health agenda or provided technical or logistical resources that could be leveraged to achieve the project objectives. The size of partner network is impressive: five ministries, two higher counsels, four universities, two private firms, three independent government agencies, one Royal Society, several media outlets, and numerous national and local NGOs. Throughout this report partners are quoted on the positive quality of their relationship with JHCP. The often forgotten definition of a partnership is an agreement between equals for mutual benefit. Clearly the partners have been treated as equals, have had meaningful involvement in project activities, and have open channels of communication that facilitate cooperation. Building some of the partnerships was difficult and took time and considerable effort. Perhaps the best example is the MAIAHP, which had no interest in partnering with a U.S. government agency on a sensitive issue like FP. Now they are fully committed to the partnership, merging FP concepts into their spiritual commitment to make “good people.” It is likely that MAIAHP will be the most critical partner in changing social norms around large families. The positive sense of partnership has benefits for both JHCP and USAID. For JHCP, the sense of ownership fostered by the sense of equal partnership has facilitated institutionalization. CSPD owns Mabrouk, and the MOH is implementing CC nationally. The MAIAHP is changing the imam curriculum to include FP and the concept of “quality family.” JHCP has also earned respect for their technical expertise. This has given JHCP the role of technical arbitrator when partners disagree over technical content or approach. In playing this role JHCP has cut the design time and increased a strong sense of consensus that has increased local ownership of the process and the products. Because the partners are closely involved in the development and review of activities and materials, they have gotten a better sense of the importance of quality and teamwork. An official described the development of the religious leaders training manual as “EVERY word in the Manual was argued over, and reviewed, and then reviewed again. There is also a very strong connection between the JHCP staff and their partner counterparts. Partners were very quick to tell the evaluation team that (JHCP staff) was “critical to the success” of the activity and that “they were always available if there was a problem.” The effectiveness of the JHCP network of partners has also been helpful to USAID. Some current partners originally distrusted USAID’s motives, especially the motive behind support for FP. USAID is now seen as a donor-of-choice with a strong commitment to partnerships and a willingness to respect local issues. Public-Private Partnerships The Global Development Alliance (GDA) is USAID’s strategy for increasing public-private partnerships in development. JHCP has a GDA-defined partnership with the Fine Hygiene Paper Products Co. The context of this partnership is described in more detail in the assessment of the HCSI activity. Fine Paper has an active social responsibility policy, so they had experience with partnering. As a result, their briefing raised several issues that highlight difficulties with public private partnerships, and why the JHCP/MOH/MOE/Fine partnership was so successful: • JHCP kept the discussion professional, and emphasized team work, joint planning, and transparency. • JHCP had to play the role of “trustworthy advocate.” 36 USAID/JORDAN: EVALUATION OF THE JORDAN HEALTH COMMUNICATION PROGRAM • Both MOH and MOE were very suspicious of private sector participation. • The public sector partners had to be sold an unconventional relationship, with JHCP acting as the “broker.” • JHCP provided a “buffer” to resolve the conflicts arising from different perspectives and approaches. • JHCP provided the design and implementation skills none of the other partners had. • It took a lot of accumulated wisdom to produce high-quality training material. Other Partnerships The relationship between the advertising agency Memac Ogilvy Advize and JHCP evolved into a contractual partnership. Ogilvy was contracted to do the mass media creative development and media placement for the Health Life Style Campaign. Originally the project was considered uninteresting because the messages had to be simple and straightforward. Ogilvy increased its support when management and staff became engaged in the objectives and the complexity of working with the MOH, HPC and the MAIAHP on message design. They also enjoyed “learning￾as-you-go” in developing socially responsible advertising and working with the Government. Like the Fine partnership, Ogilvy felt that without JHCP as a broker and a buffer, the partnership with the Government would have been impossible. Cost of Partnering JHCP has had some costs in building and maintaining its partnerships. Considerable management and staff time goes into building and even more into maintaining the partnerships, in addition to the direct cost of meetings, travel, and materials. Building consensus among partners can be difficult and time consuming; moreover, the products of consensus building often tend to be weak, as they often reflect the position of the weakest partner (e.g., spacing as a lead message rather than family size norms). Staff turnover requires extra effort to bring new staff up to speed. While the considered opinion is that the costs of partnerships are high, the benefits outweigh the costs. Partner participation in the development process, and in some cases the implementation, has resulted in a strong sense of ownership and ultimately the institutionalization of activities and processes. Participation in design and implementation has increased the partners’ technical capacity. Multiple partners in an activity have ensured the presence of technical capacity somewhere within the partnership. On the negative side, expectations and demands of some partners have led to resources being directed away from achieving the FP focus of JHCP. Some partners appear to be unaware of basic USAID requirements, such as those on reporting and allowable expenses within a project framework. Implementation of AWSO in Zarqa through 22 CBOs contributed to diluting management and quality control, suggesting that adding partners may not lead to quality performance. Furthermore, the large number of partners has contributed to a relatively low level of understanding and commitment to JHCP's higher purpose. Hence, at times there has been more of a competition between key partners than a sense of a partnership among partners, with the exception of the approval of media materials. In general, the implementing partners and JHCP tend to regard each activity as a separate project. Partnering with USAID Cooperating Agencies The Behavior Change Communication Task Force, established by USAID, facilitates partnering across the four FP projects. Meetings every two months provide a structured sharing of opportunities for collaboration, leveraging resources, and generating synergies. USAID/JORDAN: EVALUATION OF THE JORDAN HEALTH COMMUNICATION PROGRAM 37 Health Systems Strengthening II Project (HSS-II)—HSS-II works to expand and improve health systems, and increase access and quality of MCH and primary health care. It works at all geographic levels, with all public sector health providers, with all six divisions of the MOH, and uses a wide range of approaches to its objectives (accreditation, training, community mobilization, management information system, capacity building, etc.). It defines itself as the supply side to JHCP’s demand creation role. The two projects have overlapping interests in the areas of community mobilization, health awareness, counseling, changing provider attitudes on FP, and youth life planning. They also overlap in the Irbid Governorate, where both do field implementation. The partnership can best be described this way: JHCP develops and provides materials (clinic IEC posters and pamphlets, counseling job aids), content (training materials), and models (CC, AWSO, 5-10-15), which HSS-II introduces into the health delivery system. HSS-II has no communication design, testing, or production capacity and relies on JHCP or the MOH for these services. HSS-II is taking CC counseling job aids and training to a national level with MOH funding support. AWSO training materials provide content for HSS-II’s community mobilization activities. HSS-II is using 5-10-15 and the University Ambassadors content in its youth life planning activities. This division of labor is somewhat confusing in Irbid, but the two projects collaborate and both use Irbid as a testing ground. Also confusing is the distinction made between the role of HSS-II in addressing the supply side and JHCP’s role of demand creation. USAID made a programmatic decision to have JHCP operate at a national level. So at the community or service delivery levels, HSS-II is carrying out both supply and demand activities, often using materials developed by JHCP. Both projects meet in regularly scheduled BCC Taskforce meetings, and both feel there is a positive collaboration. Private Sector Project (PSP)—PSP works to strengthen private sector provision of services. Strategies include certification, improved pricing structures, community-based outreach to do referrals, public education events, pharmacist training, strengthening the role of professional associations, and revitalizing service delivery of the Jordan Association of Family Planning and Protection's network of services. PSP participates in the “demand side” meetings. A major collaboration was the use of JHCP resources to buy additional media time for PSP spots and then coordinating the timing to ensure messages did not overlap. The partnership with JHCP includes using the Hayati Ahla logo on materials when appropriate. PSP does its own materials design; it does not provide content or promote Sehetna.com. PSP has a different set of partners than JHCP (and HSS-II), so there is little exchange or synergy at the partners level; any synergy is at a national level. The Health Policy Project (HPP)—HPP works to create a positive policy environment for FP. It also advocates for effective implementation of new policies. Its primary partner is the Higher Population Council, but it has recently been working with MOH on improving operational policies. HPP attends the BCC Taskforce meeting. The one collaboration with JHCP is to produce a film to present findings in the Rapid software to all levels of government in collaboration with the HPC and HPP. Conclusions • The network of local partners built by JHCP is strong, and partners regard their relationship with JHCP as positive. The partnerships deserve major credit for the success of some activities, and for the institutionalization and going-to-scale that has taken place to date. • Functional partners are involved in implementation (MOH MCH Div., CSPD). Structural partners are strategic, long term, and involved in policy, going to scale, and overall direction (JHPC, MOI, HPC). JHCP partner problems stem from the lack of clarity over roles and 38 USAID/JORDAN: EVALUATION OF THE JORDAN HEALTH COMMUNICATION PROGRAM functions among the partners, the functional partners' lack of understanding and commitment to a common higher purpose to which they are expected to contribute, and the lack of understanding of some basic USAID project requirements. Furthermore there is a lack of partnership among the different functional partners, except for vetting and approval of media materials. • The large number of partners and activities is likely to have constrained the potential contributions of some activities due to JHCP’s human and financial resource limitations. • JHCP has established a good model for public-private partnerships. More important, the precedent has now been established and the positive experience with Fine Paper Products has opened the door for MOH to undertake future partnerships. The four USAID projects have cooperated. Most of the sharing is done by JHCP to the benefit of other projects (CC, AWSO, clinic materials). Despite the good intentions of BCC taskforce members, collaboration and efficiencies are not comprehensive or particularly creative. Even the evaluation team’s superficial look at the other projects identified opportunities for collaboration: promotion of Sehetna.com by the other CAs; advocacy by public education (demand creation for policy change); joint research activities; joint media activities (production and messaging, media buys, evaluation); broader use of JHCP’s Internet capabilities; and networking and sharing of partners. Among the projects, HSS-II has the strongest partnership with JHCP based on its collaboration in Irbid. The partnerships with PSP and HPP are more limited. Recommendations • Local partners, both government and nongovernment, seem to have little understanding of how USAID works, particularly related to issues such as reporting requirements; inflexibility on funding levels and on use of funds; contract language; allowable expenses; the USAID/CA relationship; and the role of Washington. This misunderstanding has lead to complaints and inappropriate recommendations. A presentation for partners on how USAID works could improve future collaboration. • A formalized handover strategy was useful for the HCSI activities. A handover strategy facilitates institutionalization and partner confidence in taking over responsibility and should be a standard part of any movement of implementation responsibility from the project to the local partners. • The public-private partnership in HCSI is a good model. It should be documented and promoted. The promotion would reward Fine Paper Products, encourage other businesses to develop a social responsibility plan, and give credit to the MOE and MOH for their innovative efforts to leverage resources. • USAID can facilitate public/ private partnerships by: – Promoting partnerships to build understanding, trust, and a positive attitude to future collaboration by the HCSI partners. – Helping public sector partners develop a written policy statement on partnerships, and providing participatory training to key personnel. – Asking the health CAs to seek and incorporate private sector partnerships into their activities with the GoJ. – Looking for opportunities for the GoJ to serve as a resource to the private sector. For example, GoJ could provide health education materials for distribution to private sector employees; conduct health screenings in the workplace; provide contraceptives; and provide training opportunities for the staff of employer-provided health services. USAID/JORDAN: EVALUATION OF THE JORDAN HEALTH COMMUNICATION PROGRAM 39 • The partnerships between USAID projects are not proactive or creative. USAID should take a more active role in clarifying the roles and responsibilities of each project, and require greater integration and cooperation on the USAID strategy and the achievement. JHCP MANAGEMENT AND STRUCTURE JHCP Management JHCP has experienced some management issues over the life of the project. Turnover in project directors has resulted in periods of lack of leadership either because interim managers have been in charge or the periodic transitions have caused problems. In addition, there has been a high turnover of technical staff. Only one person remains from the beginning of the project in 2004. Nearly all of the current technical staff have joined JHCP in the last three years. The USAID/Jordan policy decision to focus its limited health resources on FP was consistent with the GoJ’s growing concern with rapid population growth. This change in focus meant that in May 2008 JHCP moved from a healthy-lifestyles approach (including FP) to concentrate on FP. Many of the activities and materials were already in progress and could not be turned around easily, in part because of the partners' commitment to the broad healthy lifestyles approach. For sensitive audiences (religious leaders, students, AWSO), some of the health messages provided, and still provide, a safe context for discussing FP. Also, the technical partners, especially the MOH, still have a broader health mandate and still want and need technical support available only from JHCP. USAID has periodically allowed changes in the responsibilities and relationship of its four FP projects. For example, JHCP was mandated to work with media at the national level, but PSP has its own media production and campaigns, and at least once the JHCP budget was used to buy media supporting PSP messages. JHCP is supposed to be national, except that it works at the local level in Irbid and Zarqa governorates. HSS-II is mandated to work on health systems, but is doing community mobilization around the clinics with which it works, using sections from some tools developed by JHCP. The HSS-II community mobilization differs significantly from the community approach used by JHCP, so the work is not redundant. It is, however, clear that there is some confusion over scope at the operational levels, and that project managers are sensitized to boundaries. JHCP is less affected by this because it is in its final year, and it is focused on finishing and handing over its activities. JHCP activities are not horizontally integrated. They are stand-alone activities with different audiences, channels of communication, and partners. Managing several independent activities is a more difficult task. JHCP has addressed this by having good technical staff and having staff involved in more than one project. Furthermore, JHCP has a BCC working group composed of the technical managers of the BCC activities and headed by a senior technical officer. On the basis of the evaluation team's observations, review of documents, and some interviews, the following were noted: • Technical officers have concerns with the JHCP management decision to permit TANMIA to absorb a large portion of the budget, which means that other activities are financially constrained. • Technical officers from JHU/CPP have played a key role in providing technical guidance, such as for the activity evaluations and start-up activities such as AWSO and TANMIA. • Staff are technically knowledgeable about their activities. • Technical staff are usually assigned to one or two activities, which in some cases leads to heavy workloads 40 USAID/JORDAN: EVALUATION OF THE JORDAN HEALTH COMMUNICATION PROGRAM • Technical staff are generally knowledgeable about the entire portfolio of activities. • The work environment is comfortable and collegial, without visible internal conflicts. • The organizational structure is fairly flat, so there is direct communication between management and implementing staff. • The administration and logistics of the project appear good and efficient. Based on observations, the JHCP office runs smoothly, with staff expecting the support that allows them to stay focused on technical responsibilities. • Based on experience, the partners take for granted a high level of logistical and administrative support from JHCP. Conclusions • Although there have been issues related to turnover in project directors and heavy turnover of staff, the current management system and structure seem to be working well. • The change in the focus of JHCP from healthy lifestyles to FP has led to a residual carryover of the former. In some instances, this has provided an entry into discussion of FP, but the balance between topics has not always favored FP. Recommendations • The evaluation team has made recommendations on current JHCP activities that should be continued. These recommendations are based on the independent effectiveness of each the activities. The team has also recommended that a future project should be more strategic (and horizontally integrated). Of the two recommendations, the project strategy should take priority. Each JHCP activity should be reexamined in light of the project-wide strategy, and if it does not fit it should be discontinued. • FP communication will continue to benefit from being linked with broader health messages. USAID/JORDAN: EVALUATION OF THE JORDAN HEALTH COMMUNICATION PROGRAM 41 42 USAID/JORDAN: EVALUATION OF THE JORDAN HEALTH COMMUNICATION PROGRAM VI. RECOMMENDED USAID FUTURE ROLE IN FP BEHAVIOR CHANGE COMMUNICATION RATIONALE FOR FURTHER INVESTMENT Jordan is experiencing stagnation in lowering TFR and raising contraceptive use rates due to high desired fertility. As noted previously, the 2009 the JPFHS found a total fertility rate of 3.8, virtually unchanged from 3.7 in 2007. The CPR has leveled off for all methods at 59% (42% modern methods, and 17% traditional methods) as compared with 56% in 2002. A number of factors have contributed to the plateauing levels of TFR and CPR, as covered in the Background section of this report. There is high use of less effective contraceptive methods. Discontinuance rates are high among women who began using the three most popular modern contraceptive methods in Jordan: the IUD (11.8% discontinuance in one year), the male condom (43.6%), and the pill (46.5%). The major reasons for discontinuance, wanted to become pregnant and became pregnant, account for 60% of the reasons for discontinuance (Health Policy Initiative [HPI]) BCC to change a high fertility preference and adopt the appropriate behaviors to achieve a decline is essential for high fertility to decline. USAID investment in such a FP BCC project would support the National Population Policy and key champions with a macro-level concern about the impact of high TFR on Jordan resources, and key champions with a micro-level concern about the impact of high fertility on the quality of family health and welfare. RECOMMENDED STRATEGY FOR FP BCC PROJECT On the basis of both demographic data and the evaluation’s findings, it is recommended that the strategy and the new implementing project support activities centered on reducing high fertility. In addressing this objective, the project should focus on activities that are likely to generate a national impact. In addition, USAID/Jordan could consider more focused interventions for areas with high fertility and for reaching youth. These two subgroups require more tailored approaches. The BCC activities should be conceptually linked to the national strategy and should not be implemented at the expense of the national strategy. The message content should focus on reducing fertility by changing the values that promote large families, high fertility, and rapid population growth. Supporting messages could address quality over quantity in family size; environmental and social consequences of rapid population growth; why and how to plan a family; misinformation about FP; promoting providers as a source of information and service; improving family health and well-being (using the broad definition applied in Islam); reducing gender preferences; and reducing high-risk pregnancies, especially high-parity births. Other messages have a more marginal impact on completed fertility,18 such as preventing higher risk pregnancies of women under age 20 and over 35, increasing demand for quality services, and increased spacing of births. Because religious leaders (including mufti) are gatekeepers to social values, the new project should make their active involvement a top priority. The USAID FP portfolio should be considered as having a short-term and a long-term strategy. The short-term strategy should focus on BCC to change cultural norms on family size, and on policy change to create an enabling environment for an improved FP program. Currently, JHCP, 18 The family size of a woman who cannot have, or doesn’t want, more children. USAID/JORDAN: EVALUATION OF THE JORDAN HEALTH COMMUNICATION PROGRAM 43 HPP, and monitoring by the JPFHS represent the implementation of the short-term strategy, which focuses on changes in cultural norms and policy. The long-term strategy, implemented by HSS-II and PSP, focuses on improving quality, access, provider and client knowledge, and effective use of contraception. . The GoJ’s demographic goal of lowering fertility rates in order to slow population growth cannot be achieved without changing the cultural norms that favor large families. This is the role of a communication initiative that uses mass media, champions (like RLs), community mobilization, and health providers as a channel for messages. The objectives of the long-term strategy are to change the contraceptive behavior of individual clients, and to respond to the demand for effective FP when family size norms change. The strategy is long term because it is unlikely to have significant impacts on fertility and concomitant declines in the growth rate until the short-term goal of changing re has been some success in reducing desired fertility. The long-term strategy will also take longer to affect fertility because it addresses issues that are not a constraint to contraceptive use. Access to health services, contraceptive knowledge, and contraceptive use levels are already high. Cost does not seem to be a constraint to access. Quality of care has some room for improvement, especially in client￾provider interaction, but on the whole is quite good by the usual standards of a USAID program. The private sector already plays a major role in the provision of FP services. Jordan’s unique method mix suggests some problems: widespread and increasing use of traditional methods, high discontinuation rates, the lack of more modern methods in the mix (i.e. pills, emergency contraception), and a heavy reliance on only one effective modern method – the IUD. 19The long-term strategy responds to a different set of objectives than the short term strategy. The evaluation team recommends that future programming focus on key behavior change messages of reduction of ideal family size and increased use of modern contraceptive methods. Health messages can continue to be used because they generate policy support and strengthen the case for FP and smaller families as a component of a healthy lifestyle. USAID and the implementing partners should consider the balance of health and FP content in considering each behavior change activity As part of its strategy, USAID/Jordan should require that the design of future projects include an assessment of cost-effectiveness. This will allow the partners to better plan for institutionalization of activities, and allow USAID to make more informed decisions on continuation /and or modification of specific activities. USAID should also consider how much health messaging is necessary in new projects, and clearly define the parameters for using health content in FP messages. New project design might also include the development of a performance results framework that the USAID/Jordan FP office could use to engage strategic and potential functional partners on discussion of key activities to achieve the intermediate results and possibly discussion of the higher level indicators. The framework would be a tool for clarification of the role of FP content of the new project, compared to general health messages such as healthy lifestyles. Clarity among partners is essential for ensuring synergy among activities and a common focus. As part of the efforts to clarify roles, functions, and impact, USAID should consider a population sector review to examine the projects as parts of the larger FP program; prioritize approaches and activities for potential impact on the fertility and health sub-results; examine the breadth 19These issues may not be perceived as problems by a Jordanian woman with three children, but who wants four or five children. Her decision to use withdrawal is rational if she does not perceive any real cost if the method fails and she gets pregnant. 44 USAID/JORDAN: EVALUATION OF THE JORDAN HEALTH COMMUNICATION PROGRAM and complexity of each project to identify gaps between expected results and resources; review the relevance of technical inputs to the stated USAID and GoJ objectives; and identify the constraints that might prevent each activity, each project, and the program from achieving its objectives. This information could be synthesized to produce recommendations for future activities. Incorporation of this information would help increase synergy, optimize resource use, achieve more focused activities, maximize management efficiencies, facilitate going to scale, and increase the project’s impact in achieving USAID’s goals. USAID/Jordan should continue to involve all CAs in a BCC taskforce, but change the structure and mandate to increase cooperation and impact. USAID should clearly state this in project agreements and require that work planning be integrated. This could be done by having a single population sector work plan, individual work plans that are prepared collectively, or draft annual work plans that are reviewed by taskforce members to highlight areas for coordination, avoid duplication of effort, and facilitate going to scale on key components. The CAs should be charged with actively seeking areas of cooperation and coordination to increase the overall impact of the USAID/Jordan FP program. The BCC taskforce, cooperation of the four FP projects, and periodic meetings have not created the best possible synergy among activities. In addition to integrated work plans, USAID could also require review and comment of quarterly reports and encourage shared office space. At the FP program level, the proposed project should specify its scope relative to the efforts of the other partners, as well as the overall FP strategy. Clarity among partners implementing the FP related projects is essential for ensuring synergy among activities. A well-designed interactive communication element should be considered in high-fertility areas where traditional norms are strong. One strategy for USAID would be to do BCC operations research that could provide proven models and tools that other projects and the MOH could use to reach the most conservative elements of Jordanian society. Summary Recommendations on the Future of JHCP Current Activities The evaluation team has determined the following key recommendations, based on its assessment of the current JHCP BCC activities: • Religious Leaders: Maintain this activity, scale up the training of religious leaders, and expand the scope of activities with MAIAHP and Iftaa. • Hayati Ahla Campaign: Maintain the Hayati Ahla (or related) media campaign and focus future campaigns on reducing the desired number of children and on correcting common misunderstandings about FP. • Internet Activities: Maintain current Internet activities and expanded them with an increased focus on FP. • Mabrouk: Maintain this activity, as it has a broad reach among key audiences at a low cost and level of effort. • Ambassadors: Discontinue this activity and have JHCP expand the use of digital media to reach younger audiences and consider integration of FP and small family norms into university curriculums. AWSO: This activity will be transferred to JWNF and The Queen Zein Al Sharaf Institute for Development • A future project should consider using parts of the AWSO training manual to reach men and women in high-fertility areas. USAID/JORDAN: EVALUATION OF THE JORDAN HEALTH COMMUNICATION PROGRAM 45 • Health Competent Schools Initiative: This activity has been transferred within USAID from the Health to the Education Office. Recommendations on this Initiative are in Appendix E. • Consult and Choose: This activity will be transferred to MOH and HSS-II. Recommendations are in Appendix F. • TANMIA: Review this activity for relevance to FP, develop a close-out and handover strategy, and, if appropriate, continue technical and financial support from the USAID civil society portfolio. Specific Recommendations on JHCP Activities These recommendations are based on an assessment of the independent effectiveness of each of the activities. The team also recommends that a future project should be more strategic (and horizontally integrated). For future communication initiatives, the project strategy should take priority. Each JHCP activity should be reexamined in light of the new project-wide strategy, and if it does not fit it should be discontinued. Religious Leaders • The RL activity should be the first and broadest initiative of any expanded FP behavior￾change strategy. The RLs have the authority to change social values around FP and fertility reduction so that other channels of communication face less resistance. The effective management, deep national reach, and existing partnership with USAID further support their continued and expanded participation. • In Irbid, the participation of muftis in TOT and their participation in the step-down training added legitimacy to the training and facilitated effective responses to concerns on the Islamic position on FP, spacing, and small families. They should continue to participate in TOTs and step-down training. Other expert guest trainers, such as physicians speaking about contraception, might also increase the effectiveness of training. The partnership with RLs should recognize the contribution the leaders are making and build in rewards. These rewards could include domestic travel as a trainer or in an imam exchange program, media recognition, opportunities to “work” for the project as trainers, participation in national events, citations, and annual conferences. • A broader and more strategic approach with RLs should be discussed with MAIAHP and Iftaa to identify activities that would generate more impact. Activities could include: – Supervision by MAIAHP representatives to encourage active participation and correctness of messages, and to identify needed changes in training and outreach approaches. – (Development of a) community diagnostic tool to help RLs identify community concerns, champions, and groups that need extra attention or resources (mufti visit, special meetings) – Special aides to assist RLs in counseling couples who plan to marry. – Mosque-organized community follow-up – health fairs, community transport to health centers, midwife clinics for counseling in the community – Greater recognition of imams and mosques with strong family health activities (certificates, prizes, media recognition, haj trips) • A policy expert might also contribute to the training to help religious leaders relate to the policies and national priorities. RLs tend to relate better to messages of personal and community interest and less to nationally or globally related messages, according to MAIAHP. A policy presentation might be able to increase awareness and interest in national issues that affect local or personal situations. 46 USAID/JORDAN: EVALUATION OF THE JORDAN HEALTH COMMUNICATION PROGRAM • The expansion of the training course for trainers and trainees should be informed by issues raised in JHCP internal evaluations in Zarqa and Irbid, and by the end-of-project status related to RL’s use of training, including information on factors contributing to RL’s low use of training. Formative research based on observations, one-on-one interviews, and group discussions would contribute to increased understanding of the issues. If the findings identify a training problem, an operations research model could be used for training. The training should be varied as well as tailored to address questions raised by the Zarqa evaluation: why leaders were only partially convinced of the FP messages; why there was variation in the levels of change in attitudes on specific issues; why some issues are resistant to change (e.g., ideal family size); and why some imams are not sharing the messages. Actions based on the results must ensure that RLs willingly participate in the sharing of messages. • Periodic refresher training should be used to allow the sharing of experiences, incorporation of new, more focused messages based on attitude changes, and participant motivation. • Consideration should be given to adding sessions tailored for women RLs to take advantage of their unique access to women and their ability to talk about sensitive issues, including the use of specific contraceptive methods. The small numbers of Waizat suggest that funding transport could enable them to deliver family health lessons in a larger number of mosques. • Men’s support and involvement in family health is an objective for USAID and the GoJ, and imams are ideal candidates to be change agents for men. Special materials and training should be available to help reach men in the community. • Iftaa should be a partner on the advisory group to develop messages for Hayati Ahla, as well as materials for training RLs. • Though the capacity to perform sterilization is available in Jordan, MAIAHP and Iftaa guidance is not clear on the risk and appropriateness of the method when used to protect a woman’s health or life. Hayati Ahla and Mass Media • A mass media campaign should be part of an ongoing behavior change project. It is the umbrella that brings together efforts at the clinic and community level. It also reaches the most people and stimulates the dialogue at all social levels. It is such dialogue that ultimately defines a changing set of values. • Jordan’s fragmented media coverage limits the audience of any specific channel. In the future, more channels should be used to send and reinforce messages. Channels should be chosen based on their likely effectiveness and evaluations should be done to determine the cost￾effectiveness of each. Train and work with local level media journalists as well as those at the national level to expand coverage if an evaluation of those trained under JHCP justifies this approach. • There might be an emphasis on digital messages, which may be especially accessible to younger audiences. These messages can be used in a variety of ways (Internet, training, digital billboards, clinics with monitors, etc.). An evaluation of such an approach after no more than two years should signal whether it is cost-effective and worth continuing. • There should be a strong, strategic collaboration between the new communication project and HSS-II. The communication efforts should be used to create demand for high-quality and complete service. Demand creation can force health providers to consider their role and improve service. • There should be a stronger collaboration with HPP to use the media to promote policy change. • All communication activities using mass media share a common set of activities: formative research message development, creative design, media placement, and evaluation of reach USAID/JORDAN: EVALUATION OF THE JORDAN HEALTH COMMUNICATION PROGRAM 47 and impact. One group should coordinate the media work of all USAID health projects and possibly even projects in other development sectors. Better coordination of media scheduling would prevent overlapping messages and competition for time and space. Better coordination would also allow projects to take advantage of existing media networks (like the journalist trained by JHCP). Coordination might reduce costs with shared messaging like health and civil society. Coordinated evaluations could share data collection costs. • The new project should include a time line for messaging. The time line should use the steps of a behavior change model (awareness, approval, intention, use, advocacy). It should also consider different audiences and their informational needs, media access, constraints to behavior change, and pace of change. The time line is more important now because the taboos on FP messages have been broken, so the pace of change will increase. Messages will wear out sooner. Demand for information will increase. New media channels will develop. Incorporating a time line will ensure a more dynamic approach to behavior change communication. • JHCP monitoring of Hayati Ahla indicates that women have higher recall of messages than men. This may be a result of men not focusing on messages more relevant to women, or perhaps a result of men not having as much exposure to media. Once the cause is determined the media plan can consider messages more tailored to men and/or message placement to target men. The future media campaign might also consider using a “Swami Siaga” or “caring husband” campaign, first used very successfully in Indonesia to increase men’s involvement in preventing maternal morality. An illustrative message might be “the caring husband supports his wife using family planning to space out births three years.” Internet Activities • JHCP has been effective in using the Internet as a channel of communication. In the future, the Internet might be promoted from a tool to a strategy. It could be used to empower clients with health-care-seeking tools, get feedback from clients or providers, do advocacy at local and national levels, more effectively reach young and school-based populations, and augment other media by providing more detailed or sensitive information (e.g., by informing them that more information is available on a specified Web site.). The Internet may also be used to promote champions, recognize community or clinic achievements, promote community mobilization among local leaders and officials, and so forth. The approach could also serve specific groups with their own health Web sites. The two most obvious groups would be public and private health care providers and religious leaders. These sites could facilitate communication, offer continuing education, provide forums for discussion of issues, update information, and provide job aids. • The Healthcom Web site is intended for MOH use but is also available to the public. It would be more publicly visible and user-friendly to have two sites – one focused on educating the general public and the other for health professionals with relevant technical content. A professional version accessible by both public and private sector health providers, would also allow for dissemination of policies and policy changes, training opportunities, innovations, success stories, continuing education, technical updates, and feedback. • Sehetna should add a “Common Family Planning Myths and Misconceptions” section to address the constraints of misinformed providers and clients. • USAID support for a health Internet strategy should be housed with the communication initiatives, but all USAID health projects should be involved. Involvement should include provision of content, promotion of Web sites in project materials, and provision of funding and staff. • There should be active promotion of the sites as part of the media strategy. 48 USAID/JORDAN: EVALUATION OF THE JORDAN HEALTH COMMUNICATION PROGRAM • The project should use the Internet and Web sites in a strategic manner to reach youth with appropriate information about FP, health, and life planning. • Web sites of use to young people should be promoted by the project in the media, Knowledge Stations, mandatory university health classes, youth centers, and secondary schools. • Capacity for Web site development, maintenance of the site, and the servers should be considered. • To determine the cost-effectiveness of an Internet strategy, monitoring and evaluation of use of the information, and effects on attitudes and behaviors of selected Web sites should be done. Mabrouk • Mabrouk is a low-cost channel of communication to two primary audiences. It has a good partner and is low maintenance for JHCP and USAID. USAID should continue to subsidize the cost for reproduction of Mabrouk I and II packets, because they represent a good deal for all. • During the first year of the new project, an evaluation should be conducted to determine which items in each package are valued and useful and which might be eliminated or replaced. Then CSPD might test distribute the package without the low-valued items in select sites. • The Mabrouk packets should be reviewed periodically to ensure they continue to meet the changing information needs of the target audience. • JHCP should consider a Mabrouk III. The current packet goes to only first birth registrations. However, there is a demand for a packet from CSPD clients when they are registering additional births. It is unlikely that families will retain the packets for when their next baby arrives. • If acceptable to CSPD, the texting ability of the CSPD computer system should be explored, with the goal of expanding the phone data collected and using text messages to communicate with clients. For example, send birth spacing messages two months after a registered birth; contraceptive use messages to mothers of six months (who are likely to be ending full breastfeeding); vaccination reminders; and reminders to talk to providers about FP when going in for antenatal care. • Mabrouk would be an ideal partner for a private sector sponsorship of materials production. • The birth data CSPD has could be used to track fertility indicators. For example plotting the proportions of higher birth orders – 5+ – would give a sensitive measure of fertility change. TANMIA The Conclusion Section for TANMIA raises some critical issues. Addressing those issues should inform the USAID decision-making process on whether TANMIA is better suited for USAID support from the Civil Society Office or whether any assistance beyond current commitments should be considered. University Ambassadors • Discontinue the ambassador initiative in favor of curriculum change and use of alternate media. • If the decision is made that university students are a priority, and the necessary resources are available, and the activity can go to national scale within the life of the project, include the Ambassador approach in any FP communication project. USAID/JORDAN: EVALUATION OF THE JORDAN HEALTH COMMUNICATION PROGRAM 49 • Work with universities to develop curriculum content around life planning, FP, reproductive health, and relationship building that fits the focus of the school – general, religious, or medical. Some schools already have courses that could be augmented with family health and life planning materials. Most of the content already exists in the Ambassador Initiative, HCSI, AWSO, and RL training. Partnership with the MOH, HHC, and the HPC would provide the credibility and access to the universities. • Develop a media strategy that would inform youth on various family heath issues, while protecting privacy and respect for the sensitivity around the issues. Channels could include online courses, Sehetna and related Web sites, Ask-the-Expert, Web sites and cell phone messaging, print materials, DVDs, health fairs, and school media. • Actively promote the integration of Sehetna as a resource for existing related courses, using teacher workshops and printed instructor guides. AWSO Jordan has a highly literate population with good access to various media channels. It also has a health system that facilitates frequent contacts with health providers. It is recommended that USAID communication initiatives focus on these two channels, but also have tools like AWSO to do interpersonal and interactive communication at the community level. The strength of community level initiatives is that they can focus on the more traditional communities, which are most difficult to change. The characteristics of communities that would be candidates for this kind of communication activity would include high fertility, early age at marriage, limited access to other information channels, and low use of IUDs even though provider services are available. • Depending on the situations the activities could be addressed to couples or to men and women in separate groups. Within such situations, the interpersonal, interactive communication might be targeted to groups with specific characteristics: antenatal and postnatal women, high-parity couples, or young couples. • The approach should be informed by the AWSO training method and ought to involve multiple contacts with the same individuals over a set period of time. Parts of the AWSO tool kit specifically related to FP, including spousal communication, the difference between a wish and a plan, the fatwa on family health, and effectiveness of different methods, should be used for training facilitators, who would receive a stipend, and community members. The trainees should be encouraged to discuss the topics of each session with others. After completion of the training, willing participants could become a channel of communication and community-level change agents provided with cue cards for dissemination of messages and low-cost brochures on modern FP methods for distribution to community members. During their first year, the “participant graduates” should receive periodic supervision and monitoring to help ensure that they are correctly and effectively providing messages to their peers. • The interpersonal, interactive outreach to community members could be undertaken through health promoters based in health facilities with community outreach responsibilities, or under the auspices of CBOs or NGOs. It is important that the person has credibility within the community and the necessary attitude and communication skills. An institutional base will be important for regular monitoring and support to the “trainers” or peer educators. It is suggested that the new project pilot various approaches based on an initial community and institutional assessment. 50 USAID/JORDAN: EVALUATION OF THE JORDAN HEALTH COMMUNICATION PROGRAM APPENDIX A. SCOPE OF WORK I. TITLE Activity: USAID/Jordan: Evaluation of Jordan Health Communication Program Contract: Global Health Technical Assistance Project (GH Tech), Task Order #1 II. PERFORMANCE PERIOD o/a September 13, 2011 – October 13, 2011 III. FUNDING SOURCE Field Support- Jordan IV. OBJECTIVES AND PURPOSE OF THE ASSIGNMENT USAID/Jordan plans to conduct an evaluation of the Jordan Health Communication Program, managed by the Johns Hopkins University, Center for Communication Programs. The communications program comprises the behavior change communication component of the USAID Strategic Objective, “Improved Health Status of all Jordanians” under the 2010-2014 USAID Strategy. V. BACKGROUND Jordan has one of the fastest growing populations in the world. Between 1979 and 2009, the population grew from 2.1 to about 6.1 million people. At the current growth rate of 2.3% the population of Jordan will double in about 30 years. The Government of Jordan (GOJ) recognizes that this is a prospect it can ill afford, given Jordan’s limited natural resource and economic base. While overall health conditions in Jordan are good, the population growth rate continues to be a major development constraint. The 2009 Demographic and Health Survey (PFHS) found total fertility rate at 3.8 with virtually no change from 3.7 in 2007. Contraceptive prevalence has leveled off for all methods at 57% as compared to 56% in 2002. The following factors might have contributed to the plateauing levels of TFR and CPR: • Desire for Large Family Size: More than two-thirds (66%) of women in the 2009 PFHS consider the ideal family size to be at least four children, compared to 70 percent in 2002. • High risk closely spaced births: Birth intervals in Jordan are still among the shortest in the world. In fact approximately 60% of births in the five years preceding the 2009 PFHS were less than three years apart. • High discontinuation rates: Forty four percent of women discontinue use of a modern contraceptive method within 12 months of beginning a method according to PFHS 2009, a figure that is 5% more than the preceding survey of 2002. High discontinuation is combined with a high contraceptive failure rate due to the fact that 17% of current users utilize traditional methods. • Missed Opportunities: Several lost opportunities exist in the health care system: One is the premarital exam which could be a good opportunity to educate young couples about birth spacing and family planning. The second is associated with postnatal care. Although antenatal care is universal in Jordan, only 30% of antenatal care attendees receive post-natal medical checkup within 8 weeks from delivery. This period is a crucial period for family planning as many women are again at risk for unwanted pregnancy. Women who have spontaneous abortions (miscarriage) also represent another missed opportunity as they do not receive USAID/JORDAN: EVALUATION OF THE JORDAN HEALTH COMMUNICATION PROGRAM 51 contraception as part of post-abortion care (PAC). Furthermore, current packaging of service delivery at primary health care settings allows for easy to tackle missed opportunities for family planning. • Inadequate access to female providers: Jordanian women generally prefer to visit female physicians for family planning services. However, there are not enough female physicians engaged in family planning to meet this demand. Midwives, who are allowed to deliver pregnant women, are more available and acceptable, and have been trained under USAID supported projects to insert IUDs. However, expanding their medical license faces legal problems. • Providers’ biases: The absence of clear operational polices at public service sites has opened the door for providers’ own policies and biases. Physicians’ use of personally determined criteria to decide whether or not a woman is eligible for receiving certain services limits access, especially to permanent and long-acting methods. Also, many physicians lack proper training and accurate information about methods. • Quality of counseling and short encounter time: Negative attitudes toward counseling and inadequate quality and short encounters limit the amount of information provided to women and, therefore, their ability to choose the type of services best suited to their needs and contributes to the high discontinuation rates. Jordan has an extensive public health service delivery system reaching the majority of citizens. The current rates of infant mortality at 23 per 1000 live births and under- five-mortality at 28 per 1,000 live births are still considered high and can affect Jordan’s ability to achieve the Millennium Development Goals. Services for maternal health have expanded to national coverage with 98% of pregnant women in contact with MOH antenatal and birth services and 94% of children age 12-23 months are fully immunized. Over 98% of deliveries take place in hospitals in Jordan with an estimated maternal mortality ratio of 19 per 100,000 (1995). While reasonably good quality care is available from MOH services, the effectiveness of the services and quality of the contacts requires more attention. Jordan has been experiencing an epidemiological transition as part of the demographic transition to an older population. Infectious diseases are still a cause, albeit less prominent cause, of morbidity and mortality, while at the same time chronic non-communicable diseases are reaching alarming levels. Population and Family Health Strategy USAID seeks to help Jordan achieve sustained declines in fertility, stable population growth, and improved family health for all Jordanians. The program is carried out in close coordination with the Government of Jordan with emphasis on the Ministry of Health and the Higher Population Council. The Mission’s health sector goal, as revised in 2009, is to “Improve the Health Status of all Jordanians” by focusing on the following three results: (1) “Maternal and Neonatal Services Improved”; (2) “Use of Family Planning Services Increased”; and (3) Capacity of Health Systems Strengthened.” JHCP is responsible for generating the demand for these services and effecting positive behavior changes related to family planning, reproductive health and healthier lifestyles. Program Description The Jordan Health Communication Program (JHCP) is an 8-year Cooperative Agreement implemented by the Johns Hopkins University with a funding level of approximately $26 million. The purpose of this program designed in 2004 was to promote healthy lifestyles by: (1) implementing a national health communication strategy using the life-stages approach; (2) assisting local communities to adopt healthy behaviors; and (3) supporting child spacing and the small family norm. Over the last several years the program has focused more on promotion of 52 USAID/JORDAN: EVALUATION OF THE JORDAN HEALTH COMMUNICATION PROGRAM the small family norm, birth spacing and gender equity rather than on broad based healthy lifestyles through multi-media channels. Data from the 2007 mid-term evaluation show that exposure to messages is high – with 73% of women having recalled seeing or hearing a family planning message in the preceding six months. JHCP works through a number of local partners, including partners not traditionally involved in health promotion efforts, such as religious leaders, civil society groups, the private sector. In 2008 JHCP began working with the Civil Status and Passports Department (CSPD) of the Ministry of the Interior (MOI) on the Mabrouk I and II Initiatives. The CSPD is able to reach almost all Jordanians and through this program targets engaged couples, newlyweds, and parents registering their first baby. JHCP also launched a targeted campaign in Zarqa in 2010 and currently in Irbid through the local Governorates and with the local governors, the MOI and religious leaders. JHCP also has ongoing efforts working with the Ministry of AWQAF and Islamic Affairs educating religious leaders on birth spacing, family planning and reproductive health. In addition, Web-based health information is disseminated through Shetna.com. VII. STATEMENT OF WORK The team will review programmatic strengths and weakness and results of JHCP since its inception in 2004. The original goal of the JHCP as laid out in the original Grant Agreement was to “Improve the Practice of Healthy Lifestyles”. Indicators were related to exercise, smoking and quality of life as well as desired family size and birth spacing. As USAID began to focus its program increasingly on family planning, these healthy lifestyle indicators were eliminated from the USAID Performance Monitoring Plan (PMP). Therefore, the evaluators will not be asked to evaluate the extent to which the project has been successful in achieving the original indicators. However, they will be asked to comment on progress (and/or relevance) toward the original program objectives outlined below: • To coordinate and integrate all Health Behavior Change Communication activities/programs in Jordan. • To provide strategic integrated cross cutting Behavior Change Communication to achieve health competence and sustainable health behavior change across life stages. • To enhance the Jordanian capacity in Behavior Change Communication and institutionalize sustainable Behavior Change Communication systems. • To invoke and sustain support among leaders, policy makers, stakeholders and decision￾makers for an environment conductive to behavior change and adoption of healthy lifestyles JHCP’s performance should primarily be evaluated against the revised program objectives from May 2008: To promote behavior change regarding family planning through (1) promoting spacing of at least 3 years between pregnancies and (2) promotion of a cultural shift toward small family size. The evaluators will review progress toward the indicators identified in the Mission’s PMP including but not limited to: • Percentage of mothers who waited at least three years since the previous birth • Percentage of women who report preference for a small family size (3 children or less) • Percentage of women who have seen or heard a specific FP/RH message, by media channel To achieve these objectives JHCP has structured their initiatives through the following components: 1. Entrenching FP as a main component of a modern and healthy lifestyle USAID/JORDAN: EVALUATION OF THE JORDAN HEALTH COMMUNICATION PROGRAM 53 2. Community-level interventions that focus on vulnerable groups 3. Bolstering existing social networks to achieve national adoption of healthy FP practices 4. Linking FP with women’s empowerment and gender equity 5. Promoting quality FP service delivery through modeling service providers behavior, skills, quality services, provision of communication aids and training in IPC/ Counseling (in cooperation with CAs working on service delivery) 6. Enhancing men’s involvement in FP communication, including a review of JHCP’s work with religious leaders 7. Reaching youth through electronic media 8. Integrating ‘life planning’ initiatives into existing health education curriculum within the Jordanian school system and at the University level (as outlined under JHCP’s 2011 work plan) The evaluators will review progress and achievements for each of the 8 objectives listed above. Item 9 was subsequently removed from as a JHCP objective and replaced with strengthening capacity for behavior change communication at the MOH. The evaluators should also comment on program linkages and collaboration among implementing partners. They need to assess coordination between JHCP and the other USAID Implementing organizations and the degree to which the JHCP communication program has assisted with materials and other support necessary for USAID supported projects to expand quality family planning and reproductive health services. Illustrative questions for the evaluation team include the following: • To what extent are JHCP achievements contributing to the USAID Strategic Objective for Health? • Assess how the JHCP program advances GOJ priorities as expressed in GOJ strategy documents such as the National Agenda, the 5-year Health Strategy, the National Population Strategy, the Reproductive Health Action Plan and the National Communication Strategy. • Are the target groups selected appropriate? Are any segments of the population left out? • Comment on JHCP selection of local partners. How effective are each of these partners in helping to achieve USAID family planning/reproductive health objectives? • To what extent has the program been successful in helping to create an environment that is supportive of the small family norm, of child spacing and adoption of modern family planning methods? • Comment on the effectiveness of the communication channels approaches and media channels used. • Comment on the effectiveness of community based approaches. How effective has JHCP been in linking mass media messages with community level interventions? • Assess the effectiveness of JHCP in building capacity and institutionalizing in-country capacity. • How has the private sector contributed to the success of this program? • Comment on in-country capacity, especially in the private sector to develop communication materials and implement activities • How effective is the Health Communication Strategy? Comment on the process for developing this Plan. • Has the JHCP successfully implemented the recommendations contained in the 2009 Program Assessment? 54 USAID/JORDAN: EVALUATION OF THE JORDAN HEALTH COMMUNICATION PROGRAM The evaluation report should also contain recommendations for the future in terms of: • Assess the need for and scope of continued USAID assistance in health communication • Identify appropriate target groups such as populations with higher fertility rates and critical content areas for future behavior change communication program and suggest innovative and focused strategies to promote family planning, the small family norm and healthy lifestyles, especially among disadvantaged groups and taking into consideration gender roles in Jordan as appropriate; • The relative emphases of family planning and health in the program; • Identify any new approaches to message dissemination; • Recommend reasonable and implementable options for ensuring the long-term sustainability of activities; • Assess extent to which the private sector may provide financial or other support for health communication activities in the future • Recommend ways to increase private sector involvement, e.g. corporate social responsibility and social marketing opportunities, in the Jordan health communication program; • Suggest new approaches to capacity building or interventions necessary to institutionalize project interventions, i.e. Mabrouk, work with Religious Leaders and partnerships with the MOI and MOE; • Recommend adjustments necessary to improve current monitoring and evaluation efforts; • Identify the major components and specific activities which should be continued and identify new areas of emphasis which should be included in any new design; • Comment on ways to strengthen community based approaches. VI. PROPOSED METHODOLOGY The methods to be used in completing this evaluation will include, but not be limited to: reviewing documentation, interviews, site visits, stakeholder meetings, focus group discussions etc. The Mission will be responsible for arranging all in country logistics, site visits and meetings. The Evaluation Team is expected to consider all stakeholders in completing this evaluation. Therefore, it is expected that the contractor will interview, meet with, and review the documents of: • USAID/Jordan • Social Sector/PFH Team • USAID’s Implementing Partners • JHCP sub-contractors • Other organizations working in communication, i.e., RHAS, private sector • Ministry of Health Officials • Ministry of Education, Ministry of Interior and Ministry of AWQAF and Key private/commercial sector counterparts • Others to be determined during the evaluation Below is a list of specific Background Reading Material that the Team should review (either prior to arriving or in-country): • USAID Strategic Documents, including the Strategy Status Check of 2008 and the draft Country Strategy: 2009–2014, March 2009 USAID/JORDAN: EVALUATION OF THE JORDAN HEALTH COMMUNICATION PROGRAM 55 • Jordan’s National Population Strategy • National Health Communication Strategy • Health Sector strategy • Ministry of Health Strategy • USAID-funded Reproductive Health Action Plan II • PFHS 2009 Survey Results • 2009 Program Assessment • JHCP Omnibus and other activity-level evaluations • JHCP Cooperative Agreement and Amendments • Annual work plans, quarterly reports • JHCP baseline and mid-term survey • Omnibus and other data collection, monitoring and evaluation plans and studies All above documents will be made available on a CD after selecting the consultants. VII. SKILLS AND LEVEL OF EFFORT Team Composition The team will consist of 3 individuals with the following skills mix: health communication, family planning/reproductive health, population policy, training, community based approaches, corporate social responsibility, USAID project assessment and evaluation. Familiarity with the health service delivery system in Jordan would be an advantage as would experience working in the Middle East. All individuals need to be experts with international experience. Proposed composition of the three consultants: 1. The Team Leader should be a health communications/behavior change specialist with at least 10 years of experience supporting similar program requirements, related to project design and evaluation. The team leader will take the lead on preparations, coordinating team member input, and submitting, revising and finalizing the assignment report. 2. A second health communications professional with at least 5 years experience managing or evaluating USAID programs. The second health communications professional will, as agreed with and assigned by the Team Leader during team planning meetings, contribute to the writing of the final report. 3. One Arabic speaking public health communication expert. The public health communication expert will, as agreed with and assigned by the Team Leader during team planning meetings, contribute to the writing of the final report. This team member will also arrange focus group discussion as necessary. Deliverables 1. Work Plan: The consultant team will prepare a work plan which will include the methodologies and questionnaires to be used during the evaluation. The work plan will be submitted for discussion and approval during the team’s in-briefing with USAID/Jordan. 2. Debriefing with USAID/Jordan: The team will debrief USAID prior to the development of the report and then will follow up with an official oral debrief to the Mission while still in country. They will present the major findings of the fieldwork through a PowerPoint presentation. The debriefing will include a discussion of the findings, conclusions, recommendations for next steps and outline of the evaluation report. The consultant team will consider USAID comments and incorporate those comments and changes into the draft report, as appropriate, prior to submission to USAID. 56 USAID/JORDAN: EVALUATION OF THE JORDAN HEALTH COMMUNICATION PROGRAM 3. Draft Evaluation Report: A draft report of the findings and recommendations will be submitted to USAID/Jordan prior to the consultant team’s departure. The report will be submitted in English, electronically. The written report should clearly describe findings, conclusions and recommendations including next steps. USAID will provide comments within 10 business days of receiving the draft report. The team leader will coordinate the report writing efforts and lead this process. The report should not exceed 40 pages, not including annexes. A suggested outline for this report includes the following: • Summary • Evaluation Methodology • Background • Rationale for Demand Creation Program in Jordan • Description of Program (Objectives and Activities) • Changes in Design over Time • Achievements • Evaluation Findings – External Factors – Comments related to original (2004) Program Objectives – Findings related to USAID Strategic Objectives and specified indicators (since 2008) – Findings related to program objectives and indicators (since 2008) – Findings related to Project Activities (planned and unplanned) – Findings related to recommendations for future BCC activities and initiatives in Jordan • Key Constraints and Obstacles hindering achievement of program objectives • Management Issues (JHCP and USAID) • Conclusions and Recommendations • Recommendations for Future USAID Assistance in Health and Family Planning • Lessons Learned 1. Final Report: The consultant team will submit a final report that incorporates responses to Mission comments and suggestions one week after USAID/Jordan provides written comments on the draft evaluation report. GH Tech will provide the edited and formatted final document approximately 30 business days after USAID provides final approval of the report. GH Tech will provide 10 hard copies along with an electronic final copy. The final draft of the report may be used by USAID/Jordan for planning purposes during the editing/formatting process. The final report will be a public document. Timing The evaluation team will be given three working days in the U.S. to review background material. The consultants will be expected to arrive in Jordan o/a 11 September 2011 and to spend approximately 4 weeks in Jordan to complete the evaluation. A six-day workweek is authorized while in country. Prior to departing Jordan, a draft version of the evaluation and its findings should be submitted to the Mission and the team will present the findings in a presentation and oral debrief to the Mission. USAID/Jordan will review the draft and return comments to the team within two weeks. The Team is then expected to submit the final report to GH Tech USAID/JORDAN: EVALUATION OF THE JORDAN HEALTH COMMUNICATION PROGRAM 57 within 10 calendar days. The Level of Effort for this evaluation is 37 days, including LOE pre- and post-in country reading and report drafting. Time in country is approximately 26 days. An illustrative table of Level of Effort (LOE) follows: LOE Local Consultant International Consultants Review of documents / preparation 3 days 3 days Scheduling FGDs (JHCP will schedule interviews) 3 days - Travel to country - 2 days In-country Team Planning Meeting and Meeting In-briefing at USAID/Jordan 2 days 2 days Attend JHCP Workplan 1 day 1 day In brief with JHCP, site visits and meetings with counterparts/stakeholders and FGDs 14 days 14 days Analysis of findings 2 days 2 days Preliminary debrief with USAID/Jordan of findings for comments 1 day 1 day Draft report writing based on USAID/Jordan comments 5 days 5 days Oral debrief with USAID/Jordan and submit draft report 1 day 1 day Consultants depart country - 1 day Finalize report in Washington/ remotely 5 days 5 days TOTAL LOE (estimated) 37 days 37 days A six-day work week is authorized while working in country. Delivery Schedule: USAID/Jordan requires a preliminary presentation of findings by 6 October with a comprehensive draft report on 13 October prior to departure. VIII. LOGISTICS This review will be carried out by the GH Tech Project. The contractor will provide all logistical arrangements such as flight reservations, country cable clearance, in-country travel, airport pick￾up, lodging and interpreters, as necessary. USAID/Jordan will provide overall direction to the consultant, provide key documents and background materials for reading and help arrange the in-briefing and debriefing. As space is very limited at USAID, it would be helpful if the team arranged working and meeting space at the hotel. USAID will participate in key meetings with the Ministry of Health and other stakeholders as appropriate. The Mission will be responsible for arranging all in country logistics, site visits and meetings. IX. MISSION CONTACTS Laura Slobey, Leslie MacKeen, or Basma Khraisat USAID/Jordan Social Sectors Office Population and Family Health Program 58 USAID/JORDAN: EVALUATION OF THE JORDAN HEALTH COMMUNICATION PROGRAM APPENDIX B. PERSONS CONTACTED U. S. AGENCY FOR INTERNATIONAL DEVELOPMENT/JORDAN Kevin Rushing USAID/Jordan, Acting Director Douglas Ball Deputy Mission Director Mohammed Yassien USAID/Jordan, Program Officer Leslie MacKeen Project Manager, Population and Family Health Section Laura Slobey Team Leader, Population and Family Health Section Dr. Basma Khraisat Project Management Specialist, Population and Family Health Section Ziad Muasher Project Management Specialist Ali Arbaji, MD, MPA Project Management Specialist/ Population and Family Health Section JOHNS HOPKINS UNIVERSITY/ USA Alice Payne Merritt Director, Global Program Amrita Gill Bailey Team Leader\Near East Program Heather Sanders Program Specialist, Asia\Near East Douglas Storey Research Director Carol Underwood Research Office for Jordan USAID COOPERATING AGENCIES Jordan Health Communication Partnership (JHCP) Edson Whitney Chief of Party Rula Dajani Deputy Chief of Party Lina Qardan Senior Technical and Advocacy Advisor Huda Murad Senior Specialist: Service Delivery and Community Interventions Wisam Qarash Senior Health Education and Communication Specialist Shaima Alieh Training and Business Development Specialist Ahmad Nofal Community Mobilization and Outreach – Project Officer Amjad Hiary ICT & Web Management Specialist Bashar Husban Mass Media and Communication Officer Sarah Kamhawi Monitoring and Evaluation Program Officer Hanan Qobrosi Documentation and Online Management Officer Rawan Qurashi Web Management and Content Provision Officer Mohammad Samhouri Program Assistant/ IT Bashar Kafafi Irbid Project Coordinator USAID/JORDAN: EVALUATION OF THE JORDAN HEALTH COMMUNICATION PROGRAM 59 Strengthening Family Planning Project (SHOPS) Private Sector Project (PSP) Reed Ramlow Project Director Houda Khayame Private Sector and Social Marketing Manager Maha Shadid Deputy Chief of Party Minki Chatterji Research Director (USA) Health Systems Strengthening II (HSS-II) Dr. Sabry Hamza Chief of Party Hala Al-Sharif Community Health Team Leader Health Policy Project Basma Ishaqat Country Director GOVERNMENT OF THE KINGDOM OF JORDAN Higher Health Council Dr. Taher Abu Al Samen Secretary General Dr. Jamal Abu Saif Research and Evaluation Director Higher Population Council Professor Raeda Al Qutob General Secretary Rania Alabadee Director of the Technical and Information Unit Ministry of Education Dr. Salah Khalyleh General Director Dr. Munther Shboul Director of General Education Mr. Mohammad Ghazi Head of School Health Division Mr. Assad Ameera Member in the School Health and Nutrition Program Ms. Sahar Mahasan Member in the School Health and Nutrition Program Mr. Muhammad Alksawani Member in the School Health and Nutrition Program Ms. Ketaam Hatar Member in the School Health and Nutrition Program Ministry of Information and Communication Technology - National Information Technology Center Dr. Nabeel Fayoumi General Manager/ Chief Information Officer Ms. Auhood Majali Knowledge Station Director 60 USAID/JORDAN: EVALUATION OF THE JORDAN HEALTH COMMUNICATION PROGRAM Civil Status and Passport Department (CSPD): HE Mr. Marwan Qteshat CSPD General Manager Mr. Abdullah Al Qudah CSPD Assistant Director General Mr. Khalid Taha Management Development and Training Assistant Director Mr. Hussain Al Dabbas Administrative and Financial Manager Mr. Taiel Al Sarheed Management Development and Training Officer AWQAF Ministry Abd Al Rahman Ibdah Secretary General Assistant for Preaching and Guidance Youssef Salabie Ministry of Interior (MOI) Dr. Raed Al Adwan Local Development Director Ministry of Health (MOH)/Amman Dr. Daif Allah Al Lozi Secretary General Dr. Bassam Al-Hijawi Director of Primary Health Care Administration Dr. Rowaida Rasheed Head of Woman and Child Health Directorate Dr. Khawla Kawaa Woman and Child Health Directorate Dr. Randa Baqa’een IFTAA Directorate Dr. Mohammad AL Khalayleh Secretary General Dr. Hasan Abu Arqooab GOVERNMENT OF JORDAN—IRBID His Excellence Mr. Khalid Abu Zaid Irbid Governor Ministry of Health (MOH)/ Irbid Dr. Abd AL Rahman Tubaishat Irbid Health Director Dr. Ali Zitawi Howara Health Clinic Head Ministry of Religious Affairs (MAIAHP) - Director /Irbid Marwan Rayahnieh Ramtha AWQAF Director Mohammad Al Amri AWQAF Irbid Director Shaik Abdalla Rababa’ah Irbid Mufti Ahmad Nosier Task Force Liaison officer/ LDU in Irbid JNFW Irbid Ms. Ghada El Telhab JNFW Irbid Director USAID/JORDAN: EVALUATION OF THE JORDAN HEALTH COMMUNICATION PROGRAM 61 Khadijah Um Al Mo’mienien School Siham Al Shourman School Director GOVERNMENT OF JORDAN—ZARQA Health Directorate of Zarqa Governorate Bashar Abu Saleem Health Director of Zarqa Governorate Nabeel Al Hejawi Deputy Health Director Sukaina School / Zarqa Ms Kholoud Jarrad School Director Ms Duh Al-Safwan Physical Fitness Teacher Ms. Hayat Al- Rowad Computer Teacher Dulail District Governorate Mr. Abdel Salam Al Omoush Dulail District Governor Mr. Khalaf Al Khaldi Dulail District Governorate Mr. Isoud Al Masa’eed Local Community Mr. Abdalla Abu Hudie Local Community Mrs. Fatmeh Al Dahamsheh Local Community Mrs. Amal Bra’meh A nurse in Dulail Medical Health Center/ AWSO trainer Zarqa Local Development Unit Mr. Mohammad Aqeel Head of LDU/ Zarqa Mr. Ayman Al Romi LDU Team Ola Jaradat LDU Team AWQAF/Zarqa Shaik Mohammad Al Jiboul AWQAF Irsaifeh Director RLs and waizat trainees: Yehia Shahin Afaf Al Khatieb San’a Al Khaldi Rahmeh Al Dimour Zidan Humaidan Former AWQAF Director & RL Trainer Hatem Ishaimat RL Trainer 62 USAID/JORDAN: EVALUATION OF THE JORDAN HEALTH COMMUNICATION PROGRAM Memec Olgilvy Advize Fouad Abdel Latif Partner Luma Aloul Account manager for JHCP Mohammad Kamal Managing Partner Fine Hygiene Paper Products, Nuqal Group Bassam Salah FINE Country Manager Waseem Nuqul Trade Marketing Manager Royal Health Awareness Society Enaam Barrishi General Manager The Queen Zein Al Sharaf Institute for Development Dr Sawasan Majali Director, The Queen Zein Al Sharaf Institute for Development and Deputy Executive Director of Strategic Planning JOHUD USAID/JORDAN: EVALUATION OF THE JORDAN HEALTH COMMUNICATION PROGRAM 63 64 USAID/JORDAN: EVALUATION OF THE JORDAN HEALTH COMMUNICATION PROGRAM APPENDIX C. REFERENCES PUBLICATIONS Callister, L.C., and I. A. Khalaf, Spirituality in Childbearing Women. The Journal of Perinatal Education. Spring 2010, Volume 19, Number 2. Clark, C.J., J. Silverman, I. A. Khalaf, B. A. Ra'ad, Z. A. Al Sha'ar, A. Al Ata, and A. Batieha. (2008). Intimate partner violence and interference with women's efforts to avoid pregnancy in Jordan. Studies Family Planning. Vol 39(2), 2008. Department of Statistics [Jordan] and ICF Macro. 2010. Jordan Population and Family Health Survey 2009. Calverton,Maryland, USA: Department of Statistics and ICF Macro. El-Maaddawi, Yl., M. N. ABoul-Ella, M. N. ABdael-Moneim, and A. B. Tantawy, Study of the effect of birth spacing on maternal health. The Journal of the Egyptian Public Health Association, 5, 725– 731, 1992. Farsoun, M., N. Khoury, and C. Underwood. In their own words: A qualitative study of family planning in Jordan. IEC Field Report Number 6. Baltimore, MD: Johns Hopkins University Center for Communications Programs, 1996. Giger, J. M., and R. Davidhizar, Culturally Competent Care: Emphasis on understanding the people of Afghanistan, Afghanistan Americans, and Islamic culture and religion. International Nursing Review, 49, 79–86, 2002. Huber, Douglas. A, Guide to Family Planning for Community Health Workers and Their Clients (Adapted from the WHO's Decision-Making Tool for Family Planning Clients and Providers) PowerPoint presentation, CORE Group Community Child Health Work Group webinar/online meeting, Aug. 24, 2011 JHCP, Evaluation of the Mabrouk II: You've Become a Mother and Father Initiative, November 2010, and Religious Leaders' Knowledge, Attitudes, Practices and Public Speaking about Family Planning and Reproductive Health: Results from Baseline and Endline Surveys in Zarqa Governorate, Jordan, 2010. June 2011. Jordan Health Communication Partnership and John Hopkins School of Public Health Center for Communication Programs. Assessment of the Jordan Health Competent School Initiative 2010. Amman Jordan, n.d. Jordanian Iftaa Department, Family Health Fatwa, Amman: Ministry of AWQAF, Islamic Affairs and Holy Places. 2010. Khalaf, Inaam A., F. Abu-Moghli, L. Callister, A. Mahadeen, K. Kaawa, and A. Zomot, Jordanian Health Care Providers’ Perceptions of Postpartum Health Care. International Nursing Review. Volume 56 Issue 4, 2009. Khalaf, Inaam A., and Lynn Callister, Cultural Meanings of Childbirth: Jordanian Muslim Women Living in Jordan. Journal of Holistic Nursing Vol. 15: No 4: 373 – 388, 1997. Khalaf, I. Exploring the Use of Modern Contraceptive Methods among Jordanian Women: A Qualitative Study. 2004. Dirasat, Medical and Biological Sciences. 31 (1): 46 - 66. USAID/JORDAN: EVALUATION OF THE JORDAN HEALTH COMMUNICATION PROGRAM 65 Khalaf, Inaam A., F. Abu-Moghli, L. Callister, L. Clark and R. Rasheed, Jordanian Women's Experiences With the Use of Traditional Family Planning, Health Care for Women International, 29:5, 2008. Khalaf, I., and Fathieh Abu-Moghli. Reproductive Health Needs Survey: Integrating Health and Empowerment of Women in the South Region. Government of Jordan / Ministry of Health, Japan International Cooperation Agency (JICA) and Higher Population Council General Secretariat (HPC/GS), Amman, 2007. Khalaf, I.A., F. A. Abu-Moghli, A. I. Mahadeen, L. C. Callister, and M. Al-Hadidi. Jordanian women's perceptions of post-partum health care. International Nursing Review. 54(3), 2007. Kridli, S. A., and K. Libbus. Contraception in Jordan: A cultural and religious perspective. International Nursing Review, 48, 2001. Kridli, S. A., and K. Libbus. Establishing reliability and validity of an Instrument measuring Jordanian Muslim women’s contraceptive beliefs. Health Care for Women International, 23, 2002. Libbus, K., and S. Kridli. Contraceptive decision making in a sample of Jordanian Muslim women: Delineating salient beliefs. Health Care for Women International, 18, 1997. M&E under GHI: The Causal Pathway, Results Framework and Strategy Matrix. SHI Interagency M&E Working Group. PowerPoint presentation at the Asia Regional Workshop. April 2011. Sharma, S., T. Pandit-Rajani, R. Sanders, A. Krishna, and I. Almasarweh. A Trend Analysis of the Family Planning Market in Jordan: Informing Policy and Program Planning. Amman: Futures Group International, Health Policy Initiative, Task Order 1. 2009. The Hashemite Kingdom of Jordan. The National Health Communication Strategy 2004-2008. April 2005 The Hashemite Kingdom of Jordan. The Ministry of Health Strategic Plan 2008-2012. (n.d.) The Hashemite Kingdom of Jordan. The Higher Health Council. Jordan National Health Accounts 2008, Technical Report No. 2, March 2011. The Higher Population Council. RH/FP Symposium Recommendations. September 19-10, Amman, Jordan. Oct. 2011. (3 page document of recommendations). The Higher Population Council. The Demographic Opportunity in Jordan: A Policy Document Amman, Jordan. Oct. 2009. Underwood, Carol. Islamic Precepts and Family Planning: The Perceptions of Jordanian Religious Leaders and Their Constituents. International Family Planning Perspectives. Vol 26, Nu. 3, Sept. 2000. USAID/Jordan Country Strategy: 2010-2014. Amman, Jordan. March 2010. USAID/Jordan. Performance Management Plan. Assistance Objective 3: Social Services are Improved. Intermediate Result 3.2: Improved Health Status for All Jordanians. December 2010. USAID. Country Development Strategy Guidance, version 3. USAID. USAID Country Health Statistical Report: Jordan. 2010. Warren, C. Fertility and Family Planning in Jordan. Studies in Family Planning, 21, 33–39. 1990 66 USAID/JORDAN: EVALUATION OF THE JORDAN HEALTH COMMUNICATION PROGRAM JHCP PROJECT DOCUMENTS Communication Partnership for Family Health Baseline Survey: Key Results. 2005 Communication Partnership for Family Health Midterm Survey {Results} 2008. Jordan Health Communication Partnership, YR 7 Programmatic Strategy and Monitoring Plan Key findings from JHCP's recent studies, Dec. 2, 2010: Omnibus Results. Jordan Health Communication Partnership "Arab Women Speak Out" Guide. Draft translation in English. Sept. 2011. Jordan Health Communication Partnership Evaluation of the Arab Women Speak Out (AWSO) Initiative in Zarqa Governorate, Jordan, 2009-2010. June 2011. Jordan Health Communication Partnership Civil Status and Passports Department (CSPD) Employees Orientation Meetings on the new versions of Mabrouk Packages, Jan. 4-8th, 2011. Mini-report. N.d. Jordan Health Communication Partnership Communication Programs. Evaluation of the Consult and Choose Initiative in Zarqa Governorate, Jordan-2011. July 2011. Jordan Health Communication Partnership. Hayati Ahla Universities Component Phase Report, 2011. Jordan Health Communication Partnership. Hayati Ahla ICT &Youth Intervention Project Document, 2011 Jordan Health Communication Partnership. Youth Project Summer Report. A Power Point Presentation, 2011. . Jordan Health Communication Partnership, Evaluation of the "Mabrouk II: You've Become a Mother and Father" Initiative. Nov. 2010. Jordan Health Communication Partnership, "Hayati Ahla" Campaign Media Survey Results, Nov. 2010. Jordan Health Communication Partnership and John Hopkins School of Public Health Center for Communication Programs. Religious leaders' knowledge, attitudes, practices and public speaking about family planning and reproductive health: Results from baseline and endline surveys in Zarqa Governorate, Jordan, 2010. June 2011. Jordan Health Communication Partnership and John Hopkins School of Public Health Center for Communication Programs. Summary Report of Jordan Health Communication Partnership Activities: Focus on the Zarqa Initiative, October 2009 - September 2010 (FY6) Aug. 2011. Religious Leaders' Training Manual on Family Health. Prepared by the Ministry of AWQAF, Islamic Affairs and Holy Places (MAIAHP), in cooperation with the Higher Population Council and Jordan Health Communication Partnership. Amman, Jordan (first draft of Arabic version in English, Sept. 2011). Jordan Health Communication partnership (JHCP). 2011. Hayati Ahla Universities Component Phase Report. Jordan Health Communication partnership (JHCP). 2011. Youth Project Summer Report. A PowerPoint Presentation. USAID/JORDAN: EVALUATION OF THE JORDAN HEALTH COMMUNICATION PROGRAM 67 Jordan Health Communication Partnership (JHCP). 2011. Hayati Ahla ICT &Youth Intervention Project. OTHER PROJECT INFORMATION SOURCES Details on the RLs Family Health program implementation in Zarqa and Irbid 2010-2011. Family Planning Mass Media Campaign: Baseline Indicators and Major Conclusions. June 2006. PowerPoint. Mabrouk I Initiative Evaluation Results: Mini Report. (n.d.) Mabrouk Initiative Evaluation Survey, Sept. 2009 PowerPoint. Quarterly Reports submitted to USAID/Jordan, between March 2005 to June 2011. Annual Report submitted to USAID/Jordan for 2008. Semi-annual reports submitted to USAID/Jordan for October 2006-Mar. 2007, April - Sept. 2007, and October 2007-March 2008. 68 USAID/JORDAN: EVALUATION OF THE JORDAN HEALTH COMMUNICATION PROGRAM APPENDIX D. FOCUS GROUP INSTRUMENTS AND SUMMARY OF FOCUS GROUP DISCUSSIONS MODERATOR’S GUIDE Ambassadors/ University Students Semi-structured FOCUS GROUP/ Discussion Introduction: Moderator: Good morning. My name is.................................................. I am a researcher in reproductive health issues, and I have been asked to serve as the moderator for this focus group discussion. Welcome, we would like to thank you for sharing your time with us in this discussion. We are working with GH Tech. We are going to ask you about your experience in the Ambassadors activity. The objective of these focus group discussions is to evaluate the Ambassadors activity. We want you to know that your opinions are very important. My colleague ......................... will write down our discussions. I am sure that she will not be able to write the entire dialogue, so we decided to record this meeting if you have no objection. The content of the discussions is confidential and limited to the use or the purpose of analysis and report writing. Notes writer Please check participants' names and compare it with the list you have that was written by the regional coordinators. Please fill out participants' characteristics forms and write down their names on cards to be fixed on their cloths. Preliminary rules Moderator: (Try to encourage participants to share in the discussion.) Remember that in this discussion there is no such thing as a right or wrong opinion or correct or incorrect on shares. We want and need you to express your opinions freely. USAID/JORDAN: EVALUATION OF THE JORDAN HEALTH COMMUNICATION PROGRAM 69 Please feel free to talk frankly and share your points of view. I am interested to hear from all of you. I am not going to take part in this dialogue because my role is to facilitate the group discussion. You should know that today’s discussions and other meetings will be treated confidentially and will be used only for research purposes. Please do not talk among yourselves so that we can listen to the woman who is talking and respect her point of view. Please do not interrupt each other. Our meeting today is nearly two hours. We will take a break at the middle of the discussion and some refreshments will be served. I will be turning off my cell phone so that I can listen to you effectively, so please turn off your cell phone if you have one so that you can concentrate on the discussion. Before we get started, I'd like everyone to introduce themselves to the group. Please speak up. Discussion about Ambassadors Activity: 1. How were you selected to be an ambassadors/university student? (Probe: by whom, e.g., university or MOI) Is it the same for every one of you? 2. What is the main purpose/objective of the Ambassadors/University Students project? (Ask a number of people about their perception of the purpose of ambassadors.) 3. What are the main messages/topics you talk about as ambassadors? Among all the messages/topics, which four do you focus on the most? (Have each person write their four topics on a piece of paper and then later we can calculate.) 4. Are there any topics that you do not feel comfortable discussing? Which? (List the topics then read and have each hold up their hand if they also do not feel comfortable discussing; count the number of hands raised.) 5. Are there some messages/topics for which you do not feel you have the correct information to answer questions raised? Which? (Use same procedure as above.) 6. What are the main activities that you have carried out? Were they done with a group or individual? (If for a group, ask what is the main approach, e.g., lecturing, discussions, role play.) 70 USAID/JORDAN: EVALUATION OF THE JORDAN HEALTH COMMUNICATION PROGRAM 7. Now I would like to know if you have reached more male students than female students. (Raise hand if more males and then count the number of hands.) 8. In the training you received as an ambassador, were you provided any materials or references? What? Do you find these materials adequate in helping you to carry out your activities? Explain. 9. To conclude please rate the degree to which you find them adequate: very adequate, adequate, less than adequate, not adequate. (Read category and have them raise hand if this is their rating.) 10. Do you receive any payment as an ambassador? (What for; monthly amount.) 11. How many people were trained as ambassadors from your university? How many of these from your university are still active as ambassadors? (Ask one person from each university to respond.) 12. I'd like each of you to share if being an ambassador has had an effect on you as an individual? Please cite the main effect and explain. USAID/JORDAN: EVALUATION OF THE JORDAN HEALTH COMMUNICATION PROGRAM 71 MODERATOR’S GUIDE Focus Group with Religious Leaders (TOT PARTICIPANTS) Introduction: Moderator: Good morning. My name is.................................................. I am a researcher in reproductive health issues, and I have been asked to serve as the moderator for this focus group discussion. Welcome, we would like to thank you for your time to share us in this discussion. We are working with GH Tech. We are going to ask you about your experience in The Religious Leaders activity. The objective of these focus group discussions is to evaluate the Religious Leaders activity. We want you to know that your opinions are very important. My colleague ......................... will write down our discussions. I am sure that she will not be able to write the entire dialogue so we decided to record this meeting if you have no objection. The content of the discussions is confidential and limited to the use or the purpose of analysis and report writing. Notes writer Please check participants' names and compare it with the list you have that was written by the regional coordinators. Please fill out participants' characteristics forms and write down their names on cards to be fixed on their cloths. Preliminary rules Moderator: (Try to encourage participants to share in the discussion.) Remember that in this discussion there is no such thing as a right or wrong opinion or correct or incorrect on shares. We want and need you to express your opinions freely. Please feel free to talk frankly and share your points of view. I am interested to hear from all of you. I am not going to take part in this dialogue because my role is to facilitate the group discussion. 72 USAID/JORDAN: EVALUATION OF THE JORDAN HEALTH COMMUNICATION PROGRAM You should know that today’s discussions and other meetings will be treated confidentially and will be used only for research purposes. Please do not talk among yourselves so that we can listen to the person who is talking and respect his or her point of view. Please do not interrupt each other. Our meeting today is nearly two hours. We will take a break at the middle of the discussion and some refreshments will be served. I will be turning off my cell phone so that I can listen to you effectively, so please turn off your cell phone if you have one so that you can concentrate on the discussion. Before we get started, I'd like everyone to introduce themselves to the group. Please speak up. Discussion about Religious Leaders Activity Trainers (TOT PARTICIPANTS): 1. What are your opinions about the training you received to enable you to train other imams and waizat (women religious leaders)? IF THEY DO NOT MENTION, ASK, Did the training on facilitation skills assist you in training others? In your working with lay people? 2. When you trained the other imams and waizat, what were their most frequent questions or comments? 3. In your personal opinion, what portion of those you trained were resistant about modern family planning methods? Focus on quality of child's life rather than quantity of children? 4. What do you think needs to happen to reduce resistance? 5. Do you think that the training adequately empowered them to (LEAVE AT LEAST 15 minutes for question 7—SKIP 5 & 6 if time running out.) 6. Use the guide for Friday prayers? 7. Better understand the fatwa issued on modern family planning methods, equality of males and females, and quality of child's life in the family? 8. What do you think needs to be done to empower them? 9. As individuals with families, has the training you received influenced your family life? Explain. USAID/JORDAN: EVALUATION OF THE JORDAN HEALTH COMMUNICATION PROGRAM 73 Discussion about Religious Leaders Activity Second Level: From the training you received, what do you consider the single most important thing that you learned? The manual has many verses for sermons under a number of categories. • From which category have you chosen the most verses on which to preach? In your religious lessons and counseling have you talked about Islam acceptance of use of modern family planning methods? If yes, what have been the reaction and comments? In your religious lessons and counseling have you talked about being able to care for the children you have: quality rather than quantity? If yes, what have been the reaction and comments? Is someone asks you, how do you explain providing quality life for children? From your personal view, if I ask a layperson the ideal family size does the answer reflect the size of family the person intends to have? 74 USAID/JORDAN: EVALUATION OF THE JORDAN HEALTH COMMUNICATION PROGRAM MODERATOR’S GUIDE AWSO Semi-structured FOCUS GROUP/ Discussion (Trainers): Introduction: Moderator: Good morning. My name is.................................................. I am a researcher in reproductive health issues, and I have been asked to serve as the moderator for this focus group discussion. Welcome, we would like to thank you for sharing your time with us in this discussion. We are working with GH Tech. We are going to ask you about your experience in the AWSO activity. The objective of these focus group discussions is to evaluate the AWSO activity. We want you to know that your opinions are very important. My colleague ......................... will write down our discussions. I’ am sure that she will not be able to write the entire dialogue so we decided to record this meeting if you have no objection. The content of the discussions is confidential and limited to the use or the purpose of analysis and report writing. Note’s writer Please check participants' names and compare it with the list you have that was written by the regional coordinators. Please fill out participants' characteristics forms and write down their names on cards to be fixed on their cloths. Preliminary rules Moderator: Try to encourage participants to share in speech. Remember that in this discussion there is no such thing as a right or wrong opinion or correct or incorrect on shares. We want and need you to express your opinions freely. Please feel free to talk frankly and share your points of view. I am interested to hear from all of you. I am not going to take part in this dialogue because my role is to facilitate the group discussion. USAID/JORDAN: EVALUATION OF THE JORDAN HEALTH COMMUNICATION PROGRAM 75 You should know that today’s discussions and other meetings will be treated confidentially and will be used only for research purposes. Please do not talk among yourselves so that we can listen to the woman who is talking and respect her point of view. Please do not interrupt each other. Our meeting today is nearly two hours. We will take a break at the middle of the discussion and some refreshments will be served. I will be turning off my cell phone so that I can listen to you effectively, so please turn off your cell phone if you have one so that you can concentrate on the discussion. Before we get started, I'd like everyone to introduce themselves to the group. Please speak up. Discussion About AWSO: Discussion starters: How long have you been involved? Do any of you work outside the house? What is your husband’s occupation? Etc. Try to see if there are background characteristics that affect selection or implementation (for example, husband approves of AWSO, etc.) 1. How did you come to be involved in AWSO? 2. What motivated you to be an AWSO facilitator? 3. Now that you have experience as facilitators, what are your views/opinions about the strength and weaknesses of the facilitator training? 4. Do you have any suggestions for improving the training? 5. How do you recruit (encourage people to come for the training)? 6. Is it difficult to get the people to come? NOTE: 5 and 6 may generate redundant responses, especially if they are easy to recruit. 76 USAID/JORDAN: EVALUATION OF THE JORDAN HEALTH COMMUNICATION PROGRAM 7. In some communities, men attend the AWSO training sessions. Has this occurred in your training sessions? If so, are they training sessions or discussion sessions? – IF Yes Probe – 7A In your opinion, why do they come? – 7B. Do men come for specific sessions or do they just come to see what is happening? – 7C. If men come for specific sessions,. which sessions are most interesting to them? – 7D. Does having men participate have an impact on the sessions, and is the impact positive or negative? – 7 E. If the impact is negative or mixed (Probe) Do you think a separate session for men to help them understand AWSO would be helpful? 8. Facilitator does group ranking: I'd like you to think about the six sessions (READ Session names). As a group can you rank the six sessions by the most to the least interesting? Ties are acceptable. (Give time for discussion and negotiations) 9. Now can you pick the session that was the most helpful to you in your role as an AWSO facilitator? – 9A. Which session was the least useful? 10. What are the common questions and comments do you get when discussing Planning the Family and Securing Its Health? 11. As a facilitator, do you have any recommendations for improving the Facilitation Guide? 12. Has your AWSO discussion and work had any impact on the women and couples in your community? 13. Has being a facilitator had any impact/ effect/change on you? USAID/JORDAN: EVALUATION OF THE JORDAN HEALTH COMMUNICATION PROGRAM 77 AWSO Semi-structured FOCUS GROUP/ Discussion Participants (Lay people) 1. What motivated you to begin attending the AWSO training sessions? 2. What are the two most important things that you learned from the sessions you attended, and why you consider these the most important? 3. Did you acquire any new skills? What is the single most important skill you learned? 4. Are there any topics that you felt should have been discussed or explained more thoroughly? 5. There is a session on Participation of Men and Women in Planning Their Family and Securing the Family’s Health. In this session there is training and discussion on modern family planning contraceptive methods and the benefits of smaller families. – 5A.Did you learn anything new from the discussion on all the modern family planning contraceptive methods? – 5B. Did you talk to your husband about what you heard in the session? – 5C. Did you talk to your friends about what you heard in the session (Probe:) What was their response? – 5D. If you wanted to use family planning today, what modern method would you be most interested in? 6. How long should you wait to have your first child? 7. How long should you wait between having children? 8. Is choosing to have fewer children against the teachings of Allah? or What do the teachings of Islam say about the number of children a couple should have? 9. The training covered many important topics. Has attending the training sessions led you to do anything differently? Please explain. (Probe:) Why didn't you do this before the training? (LEAVE AT LEAST 15 MINUTES FOR DISCUSSION OF THIS QUESTION) 78 USAID/JORDAN: EVALUATION OF THE JORDAN HEALTH COMMUNICATION PROGRAM FOCUS GROUP FINDINGS The AWOSO Trainers Irbid Introduction The participants indicated that they were recruited as trainers through an ad from the Jordanian National Forum for Women (JNFW). They had a meeting with the responsible administrator from JHCP; 27 women attended. They were offered full information about the training; accordingly they were motivated to participate. A member of the assessment team selected the participants for the focus group from a list sent by JHCP. The participants were five female trainers, with an age range from 26 to 55 years old; three participants were married, one engaged, and one single. One participant had four children, one participant had six children, and one participant had 13 children. One participant had a BSc. education, one was a two-year college graduate, and three were high school graduates. The family monthly income ranged from $500 to $800 with an average of $650. All participants indicated that what motivated them is that the idea was good and needed, and they had money for the transportation and for the hospitality provided to the women. They want to train so it is a good opportunity and they were able to do it. Many of the participants indicated that there is a need for them to educate other women who lack the information related to women’s health, and family planning. Major Findings • All participants were motivated. One participant indicated that “the training is needed for them as they needed the information and many of them they lack the information and the training manual has complete information related to the family health which is needed for all of us, in addition of the motivation to be active in the community.” One participant said “the best reason for participating that you could do the activity at your home and you could select the women you want to train.” • All the participants indicated that all topics in the training were needed and they learned from them; however, they agreed that the most important were the women’s health topics and the information related to hypertension and diabetes and the family planning part. The participants agreed that all the information in the training was important. • Most participants indicated that they are interested in the training manual, and after the training they were able to communicate with people. They learned the most about the importance of the women’s empowerment, and they were interested about the first topic, which is the Arab woman speaks out. One participant said “we learned (from) most of the topics” and the other participants agreed. • The participants indicated that the manual was comprehensive but they needed more time to discuss it and they felt that more time is needed to cover most of the topics, especially the one related to the different diseases and the health issues. Many participants indicated that one of the weakness points is that the training time of six sessions is not enough and a participant said “in Bani Kenana where she lives the population 10,000 and to spread the training to all we need more sessions to be conducted.” Another participant said “the training was very condensed to have 2 sessions in one day is exhausting.” One participant said “it is difficult to bring the women to the sessions, but mostly they agreed as they receive the money for the transportation and the hospitality so they were able to do the session and no problems.” • To improve the training one participant suggested giving the women advance training about health and illnesses topics. Some participants indicated that they need the training to be longer and they think that some of the topics were covered within a very short period and many issues were covered superficially as the time was not enough. They recommended USAID/JORDAN: EVALUATION OF THE JORDAN HEALTH COMMUNICATION PROGRAM 79 having good trainers who are able to use the communication actively. They think that they want the training to be continued for at least two years and they need the follow up and the financial support. They also thought that the follow up without the financial support would make it difficult to continue. • Many participants indicated that there was follow up from the liaison officer from the governorate, and another participant indicated that “Ms. Huda in the second training she assigned a woman from the JNFW to do the follow up and writes the reports about the supervision and the follow up.” • The participants recruited other women for their training sessions through different approaches. Many recruited their friends, relatives, and neighbors, while others used schools and NGOs. Many participants said that “the women association through the governate help us to put ads about the training in the schools and the different agencies,” one participant said “the liaison officer use to send an official letter for the schools and the NGOs to recruit the women to attend the sessions.” All the participants said it was not difficult to recruit other women to attend the training sessions; many said that they recruited different age groups, and one participant said “it all depend on the trainer, many indicated that offering the women with food help in the recruitment.” • All the women except one indicated that women only attended their training and they thought it is hard for men to attend as they are busy with their jobs. They think it is important to train the men and preferably men to train men, and the men need to be trained about the different family planning methods and its side effects. Most of the participants thought that men can participate in the training or attend the different session. There is nothing awkward for men except the topics (condom and the inflammation of the reproductive system). A participant indicated that “the embracement culture should be resolve from the community.” The only participant who had men in her session said “it was okay to have men in the sessions, and the participation was usual and it was accepted, but they attended only one session as they were busy to attend the other sessions.” • The participants indicated that men need to attend sessions related to domestic violence, family planning, resources, women’s health, and planning for the future. • The participants indicated that all the topics are important and it is hard to rank them. But if they need to rank them they think the 1st is the healthy lifestyle, 2nd maternal child health, 3rd planning for the future, and 4th the participation of women and husbands in organizing and assuring their families health. But after this they all agreed that all topics were important. • Most of the participants (4 out of 5) thought that the AWSO activity could be institutionalized in the JNFW, but they need the financial support; one woman said the “Ministry of Social Affairs.” • The participants agreed that all the topics helped them to implement the training and they thought that women’s health and planning for the future are very important. They said all the topics were important at the same level. • The participants indicated that in the family planning session the most frequent questions were related to rumors about family planning methods. For example, do the pills make the woman infertile and cause cancer; will IUDs cause the woman to have cancer and to be infertile and anemic; and will the injectable make the women infertile. • The participants indicated that although they know about all the methods, they prefer to use the IUDs more than the other methods because they are safer and easy to use. • The participants indicated that they discussed these issues with their neighbors and friends, and they were interested to know about it. In addition, many of them discussed it with their husbands, and they think that men need the information. 80 USAID/JORDAN: EVALUATION OF THE JORDAN HEALTH COMMUNICATION PROGRAM • The participants indicated that they influenced the community and they talked about the training to their husbands and families. One participant said “my brother in law told me that he is interested to attend all the sessions.” • Many participants indicated that they changed in many ways Some said they started to take care of their health, other said they had a better social situation, they started to refuse the early marriage, more pregnant women started to go for antenatal care, and more cared for themselves and the child post-natally. • All the participants asked for continuing the training, to reach more people. USAID/JORDAN: EVALUATION OF THE JORDAN HEALTH COMMUNICATION PROGRAM 81 FOCUS GROUP FINDINGS The AWSO Trainee Irbid Introduction • The participants indicated that they were recruited as trainees through their friends, families, neighbors, and the Jordanian National Forum for Women (JNFW). A member of the assessment team selected the participants for the focus group from a list sent by JHCP. The participants were five female trainers, with an age range from 18 to 50 years old. Three participants were married and two single; one participant had one child, and two participants had five children. One participant had a master’s degree, one participant was a two-year college graduate, and three were high school graduates. The family monthly income ranged from $500 to $700, with an average of $530. • Many participants were motivated to attend the sessions as they felt that they needed the information to raise their families’ awareness. Major Findings • Many participants were motivated to attend the sessions as they felt that they needed the information to raise their families’ awareness. One participant said “to get information about the family planning issues and the maternal child health, I gained information as well as my neighbors who attended the sessions.” One participant said “I benefited from the information at my work as a teacher, I use to tell my students about the information I gained and they will transfer the information to their families.” Many participants indicated that the older women who attended said they wished that this type of information were given to them so they had less children and better life and health. Another participant said “I learned how to deal with my traditional father and the other male in my family.” • Three participants indicated that most important thing they learned from the sessions is the family planning and how the spacing is useful for the mother and the fetus and this way each child will have enough care. They said that this is important for them because they learned how to convince the other about such issues. The other two participants said that they learned about raising children and they needed this information to help them to deal with their adolescents. One participant said “I needed the information related to the breast self exam,” and another participant said “domestic violence.” • Most of the study participants indicated that they acquired the effective communication skills and they are more skillful in communicating with their kids and in laws. Two participants said that they learned more about the family planning issues and breast self exam. One participant said “I was empowered as a woman and I gained more self confidence.” • All the participants said that they had enough information, and they are satisfied with the information they had. They said that the trainers were skillful and discussed everything and demonstrated in front of them. Their approach in delivering the information was excellent and they were convincing, and they gave them brochures to read. One participant said that “the training was positive and effective,” and another participant said “the referral card is important in which they can go to the health center and they will help them to select the appropriate family planning which suite them.” Another participant said “in the sessions there were many young and old mothers and all of them they were interested to know about the family planning.” • All the participants indicated that they learned from the family planning session, and they knew about the injectables. They used to know only about the IUDs and pills, and they knew after the sessions that IUDs will not cause infertility. 82 USAID/JORDAN: EVALUATION OF THE JORDAN HEALTH COMMUNICATION PROGRAM • Most of the participants said that they did not discuss the family planning topic with their husbands. One participant said “I did not discuss with my husband as we have our agreement to plan between our children”; another participant said “I did not as I am the one who is responsible about the family planning.” Another two are not married but one of them discussed the topic with her brother to convince her brother to let his wife do the tubal legation as she has 5 children and her health is not good and the best for her is tubal ligation, and she said her brother was convinced. The other one said she believes that men need to know and they should as well use methods for men. • All the participants discussed the family planning topic with their friends and neighbors and they all accepted the idea of the small family. One participant said her mother was happy to participate, another participant reported that after discussing with a young neighbor, the neighbor went for an IUD insertion and intends to space three years between her children. All the participants were convinced about the importance of family planning. • The participants indicated that although they know about all the methods, two prefer to use the IUDs and three prefer the pills. • All participants said they will not wait after marriage to have children and they thought that from a social perspective they will not be able to wait. But they will space three years between their children. • All the participants agreed with the idea of having a small family as they will suffer economically and they will have less time with the husband which might create problems. But most of them consider four children as a small family. And they know that spacing between children is accepted by Islam but not limiting the number of children. • All the participants were empowered after the training. They have more self confidence and are more responsible. One participant encouraged her daughters to attend the training, and her daughter is convinced about the small family idea, the idea of not getting married at a young age, the importance to wait after abortion, and not getting pregnant before 20 years. • The participants recommended training of the men through the mosques and at their work. USAID/JORDAN: EVALUATION OF THE JORDAN HEALTH COMMUNICATION PROGRAM 83 FOCUS GROUP FINDINGS The AWOSO Zarqa Trainers & Trainee Introduction Participants indicated that they were recruited as trainers and trainees through the NGO associations; trainees were recruited through visits to their houses. The focus group discussion included seven women (four trainers, three trainee participants) from the Dulail District governorate ranging in age from 37 to 53 years. Six participants were married and one was single. One participant had eight children, one had seven children, one had five children, one had four children, and one had no children. Two participants had a BSc. education, three were two￾year college graduates, and two were high school graduates. The family monthly income ranged from $500 to $800, with an average of $650. Major Findings: The trainers participants indicated they were recruited to be trainers through the NGOs associations, the trainee were recruited through visiting them in their houses, One trainee participant indicated that “they benefited from the training as the trainers were able to transfer the information to them very well, especially in Dulail area the community members need the awareness and they have a lot of children.” • All the participants indicated that they were interested and motivated to train others and one participant said “many trainers were interested in the training and financially we benefited as they use to give us 270JD for each workshop we use for hospitality.” Another participant said “in the field we discovered how much people lack the awareness, and through the AWSO project we had the opportunity to know the community.” Another participant said “I am a teacher and through my work I trained many teachers, parents and students.” A nurse participant said “I conducted 10 workshops, one of the workshops I conducted all the participants were male and the Mouktar was one of the participants. I discussed with them issues related to adolescents health, spacing between children, and chronic illnesses and cancer.” Another participant said “I talked about the early marriage problems”; another participant said “I conducted 12 workshops through the women association (JUMIEH), we reached the places and people who are not able to reach us and I talked and discussed with them the spaces between children and the economical situation of the area.” Another participant said “we were distributed in different area in Dulail and Alhalabat, and I conducted 2 workshops in Alhalabat, and we use to do the workshops in the health centers, in the houses and the Association.” Another participant said “what helped us in the AWSO project was that we know the people; for example I am the President for the JUMAIAH.” • The participants indicated that it is important to raise the men’s awareness in the different topics. A nurse participant said “I know all the area I talked to the leader of the area (ALMouktar) and all the families. I conducted 10 workshops, and one of the workshops I conducted all the participants were male and the Mouktar was one of the participants. I discussed with them all the topics including issues related to adolescents’ health, spacing between children, and chronic illnesses and cancer.” She added “the men participants were positive. I had no problems in anything. Before the training the women use to come for the family planning services at the health centers without informing their husbands. After I discussed with them the importance of discussing family planning with their husbands they started to come back to the health center with their husbands.” However, one participant said “sometimes it is hard to reach the men.” • Many participants in the FGD indicated that they continued to train and counsel other women related to what they were trained in. Three out of the four trainers reported that 84 USAID/JORDAN: EVALUATION OF THE JORDAN HEALTH COMMUNICATION PROGRAM they continue to train and counsel other women even after the withdrawal of financial support. They indicated that what they learned should be transferred to other women as other women need the information, and since they have the training manual, they can use the different CBO settings to provide the training. Two of the participants reported that as they are involved in the women’s association, empowering and raising women awareness is part of their work and their training through the JHCP project helped them to be more responsive to the women’s needs. One participant said “we started to ask and look for centers for women.” Many women reported that they conduct the training sessions at the health center, NGOs, and in homes. Most of the participants said that they are still in need for conducting more training session as many areas are still not covered with the training. • The participants indicated different media to educate the community about the family planning issues; however, all of them said pamphlets and posters, and other participants added television, cellular messages, comedian spots, and Friday Prayers. One participant said “once I heard in the Friday speech the imam was talking about family planning.” • Many of the participants complained of the ignorance of the community members of having too many children and how they are not able to raise or care for them. One participant said “ the woman deliver a baby and then immediately she is pregnant.,”; another participant said “the boy without the slipper or trousers and the mother pregnant and breast feeding another child”; another participant said “we have many problems the early marriage, the woman does not go for postnatal care for herself or her child.” • All the trainees were interested in the sessions they attended. One participant indicated that “the trainers invited me to attend to benefit from the information, especially I am a school teacher…,” Another participant said “all the topics were interested and important we use to attend always.” One participant said “the trainer conducted a session at my home about the family planning and early marriage. She added at my area the girls get engaged at 12 or 13 years old and they get married at 15 years and then they will have a child at 16 ….” One participant added “they taught us about breast self exam and how to do it. I liked the idea and as well I learned through the training about the home economics.” Another participant said “I learned about family relationships and home economics.” • The participants’ trainers indicated that “we learned how to plan our time, we learned about the family health, and after this training we believe that we should not have more than three to four children.” • The participants indicated that the topics that were most interesting to them were the family planning, the health topic, the roles of the Arabic women, planning for the future, and breast self exam. All the participants agreed that all the topics are important but the most important was the family planning topic. The nurse participant said “the pregnant women and the women in the postnatal period use not to visit the health center as they used after the training.” • The participants indicated that the positive things about the project are: increased our information, empowered us, helped us to conduct the training sessions, and covering the expenses of the training. However they indicated that they needed specialized people for the specialized medical topics, and they asked for religious women leaders to be available in the sessions. • The participants indicated that the JHCP used to follow up the project, and one participant said they still call us. • Focus Group Findings USAID/JORDAN: EVALUATION OF THE JORDAN HEALTH COMMUNICATION PROGRAM 85 RELIGIOUS LEADERS TRAINERS/IRBID Introduction • The participants were four female trainers and seven male trainers. All the trainers were recruited for the training purposes through the Ministry of MAIAHP. A member of the assessment team selected the participants for participation in the focus group from a list sent by JHCP. The participants’ ages ranged from 31 to 55 years old. Ten participants were married and one was divorced. One participant had no children, one participant had one child, one participant had two children, three participants had four children, and five participants had more than four children. Most of the participants had a master’s degree or a BSc. education (eight participants), and one participant is a high school graduate. The family monthly income ranged from $500 to $900 with an average of $750. • Most of the participants were satisfied with their training and recommended more training They discussed and talked about family planning issues. Most of them used the information in the religious sessions and less frequently in the Friday prayers. Major Findings • All the participants were selected through the Ministry of MAIAHP who asked the directorate of Irbid MAIAHP to select the participants. However, some of the focus group participants said the selection was from the director who might chose according to what he feels or whom he knows, and some thought this not fair, but others said that the director need to select the individuals who will accept the idea of the JHCP project. Many participants participated in many activities on family planning or other training with JHCP like visiting Al Zaher. • Most of the participants indicated that the training increased their communication abilities, decreased shyness and enabled them to discuss issues in front of a large number of trainees, and empowered them to be stronger individuals. Many thought that four days training is not enough but they were able to cover the topics. One participant said “but with more time we might have more time for discussion,” and they thought that they need more time for the IFTA presentation. One participant said “the good management of time and the training method helped us to overcome the time limitation.” One participant said “some of the trainers said that they had 10 children but they said if we knew the information that we had from the training and how the too many children will affect our health we will not have this number of children.” The participants were very motivated to learn everything in the training. All the participants were very satisfied with the training materials and with the trainers who trained them. • All participants indicated that the most frequent question asked by the other religious leaders whom they trained, was if it is religiously allowed to family plan and space between children, and they use to answer them from the Holy Quran and Hadith. One participant said “we use to tell them it is planning and not limiting the number of the children and when they understand it, they all accepted the idea of family planning.” Many participants said that the trainees use to ask about who is funding the project, but the presence of the mufti helped a lot in changing their way of thinking and they were very satisfied. • Most of the participants indicated that they talk about and discuss family planning issues and how it is important for the quality of children and the health and welfare of the family, the woman and the child. They mostly covered these aspects more than discussing any scientific issues related to the family planning methods. They thought that if a physician is present in the training that would be helpful. • Most of the participants felt that it is not wise to connect directly the need for family planning with the idea of having enough money to raise them, as most people will say that 86 USAID/JORDAN: EVALUATION OF THE JORDAN HEALTH COMMUNICATION PROGRAM God is the one who will help them with the money…, but mostly they connect the benefit of family planning with the health, education, and care of the child, the woman, and the family. Many as well discussed the need to be equal and fair to both the male and female children, and many families want children because they need to have boys in the family. One participant indicated that “it is okay to use different approaches to have a male baby, but limiting in Islam is what is forbidden.” • Most of the participants were not in support of using the modern family planning methods because of the many side effects. Many women said we try it and we suffered from many side effects, but other participants said that they can go to the physician who could decide. • Most of the participants said to overcome the difficulties in convincing others about the modern family planning methods they can use real examples from life about how important it is to plan the family and they used statements from the Holy Quran, and Hadith. • All participants said that they used the general idea of family planning and its importance to the health of families including children and women, they thought that talking about the modern family planning methods in Friday speech is not convenient, but the women religious leaders indicated that they discuss these issues in their religious lessons. All participants said that they used the religious fatwa related to family planning in their religious lessons and sessions. • All participants recommended empowering other religious leaders through training them like they were trained. • Most of the participants indicated that the training affected them and change their life from different perspectives. One participant said “it affected my idea about the relationship between male and female,” many said we now believe more on the importance of the quality of children and not the quantity, to be fair to girls, planning our life, not to discriminate between boys and girls, one female participant said “I knew through the training that the male who decide the sex of the baby,” one male participant said “I am more considerate to my wife feelings during pregnancy.” • The participants recommended that they need the training time to be longer than two days for the trainee, to have enough time to discuss the material, and to add one hour of training for a physician to discuss specialized issues related to family planning and one hour for a mufti to answer questions related to the religious aspects. One participant commented that “to have the workshop from 8-2 is too long and it is better to have more days, and less hours,” another participant said “we need to have training in how to be leaders.” USAID/JORDAN: EVALUATION OF THE JORDAN HEALTH COMMUNICATION PROGRAM 87 FOCUS GROUP FINDINGS Religious Leaders Trainee/ Irbid Introduction: The participants were nine religious leaders, four females and five males who were trained by the other religious leaders in Irbid. All the trainees were recruited for the training purposes through the Ministry of MAIAHP. A member of the assessment team selected the participant for participation in the focus group from a list sent by JHCP. The participants’ ages ranged from 21 to 60 years old. Seven participants were married and two were single. Three participants had two children, three participants had four children, and one participant had more than four children. Most of the participants had two years college or a BSc. education (eight participants), and one participant was a high school graduate. The family monthly income ranged from $300 to $700, with an average of $600. Major Findings: • Most of the participants indicated that they learned and talked in their Friday speeches about raising the children, the family concept, equity, how to communicate with others, and planning for life. One participant said “in Friday speech we can achieve objectives and talk about the equal treatment between the kids and how to care for the women.” Another participant said “we discussed the difference between spacing and limiting the number of children, we concentrated more in the quality of the children.” • All the participants indicated that after the training they were convinced that spacing is acceptable and they indicated that they learned about the family planning methods. One participant said “before the project we did not trust the project or who directed it from outside Jordan, but after we participated in the training we discovered that the project is a trustful one as well as the funding organization.” One participant said that “the topic about the relation between the husband and wife was a good topic and the idea of family planning to protect the woman’s health, and the importance of having a quality children more than quantity, and to give the woman her right to protect her health and to give time to raise the children is very convincing.” Another participant emphasizes the role of religious leaders in preserving life and improving the health of the community and to protect the resources. Another participant said “I liked the good relation between the man and the woman.” • The participants in the last four weeks in their religious speeches and sessions covered the followings topics: the child care and his rights for care, the women rights in heritage, three participants discussed the family planning and the spacing between children and its importance, the relationship and the communication between the children and their parents, the relationship between the husband and wife, the quality of children, and planning for the future. • All the male participants said it was not appropriate to discuss family planning methods in their religious speeches, but it was accepted to talk about family planning in general. The women religious leaders indicated that they discussed family planning issues and methods in their religious lesson, but most of the discussion and questions from the attendee were about the side effects of modern family planning methods, and they recommended to have a physician to explain this specialized part. 88 USAID/JORDAN: EVALUATION OF THE JORDAN HEALTH COMMUNICATION PROGRAM • All the participants discussed the idea of the importance of having quality children and the importance of giving the children their rights in a good quality life and not to be raised on the streets. The participants said that the attendee said that the topics were new to them and they thanked them for the information. • All the participants did not give the number of children they want to have and they said to space three years between children. USAID/JORDAN: EVALUATION OF THE JORDAN HEALTH COMMUNICATION PROGRAM 89 FOCUS GROUP FINDINGS Religious Leaders Zarqa trainers & Trainee Introduction The participants from Zarqa were six trainers (three women and three men) and one trainee (man). All were recruited for the training purposes through the Ministry of MAIAHP. The participants’ ages ranged from 21 to 50 years old. Six participants were married and one was single, one participant had three children, two participants had four children, and three had more than four children. Most of the participants had a two-year college education or a BSc. education (six participants), and one participant is a high school graduate. The family monthly income ranged from $300 to $800, with an average of $650. Most of the participants were satisfied with their training, more training was recommended, they discussed and talked about family planning issues, most of them used the information that they were given to them in the training. Major Findings • Most of the participants indicated that the training was adequate and comprehensive and enable them to train others, the participants said we were trained comprehensively for 3 days and we did the training in 4 days and during the training we were trained how to convince the others with our ideas. The training included pictures and presenting a film. One participant said “they explained for us very well everything related to family planning.” Another participant said “they discuss with us for one hour the religious fatwa related to family planning.” • Most of the participants said that the JHCP personals were very organized everything was ready and they provided us with ever thing we needed and they use to have an attendance sheet. • The participants indicated that all the religious leaders in Zarqa Governorate were trained and each one of them trained around 70 persons. One participant said “we train everywhere,” another participant said “we train people in associations, mosques, homes, and schools.” • The participants said they gave us information about breast feeding, family planning from its religious perspectives, many participants indicated how important to plan their families and to space between their children and how it’s difficult to raise children nowadays economically and to cover their expenses. Another participant said “that the withdrawal as a family planning was used from the beginning of Islam, but the modern family planning methods sometimes have side effects,” another woman said “not that much, I used the pills and only the first month I had headache and drowsiness and then I was okay,” another participant said “the religion ask us to be organized we need civilization and to be advance, the nutrition now not like before the woman will not be able to have a lot of children.” Another participant said “my husband was a teacher and we were not able to cover the clothing expenses of our children we use to buy them the clothes only twice a year during Eid, and we were not able to teach them at universities out of five children we were to teach only one.” • Many participants said that after the training they were convinced that they needed to space at least three years (36months) between children and that is accepted from a religious perspective, and that they advice their children to plan their families and to space between them. • All the participants indicated that they discuss family planning issues from the perspective of having quality children through providing quality life to the children, and they believe on it and said that Islam wants quality and healthy children and families and not quantity. Many 90 USAID/JORDAN: EVALUATION OF THE JORDAN HEALTH COMMUNICATION PROGRAM were discussing the advantage of small families and how large families might lead to violence in the family as a result of inabilities to provide the family members with their needs. • Many participants (4) said that they are still active and they talk about family planning, one participant said she is not talking about family planning, four participants said that they discuss different issues related to health affairs, one participant said we discussed how to raise children and the importance of providing quality life for them, another one indicated that they discussed the women health and women rights, another participant said not all the time in special occasional days we are not able to talk about family planning as we have other things to discuss. One participant said “I am not active as before and many person calls and they want training.” • One participant said “one disadvantage of the project it stops and without support it is hard to continue.” • Most of the participants did not give a number for the children they want to have but they said that the ideal number way to have a family is to space three years between children. • Many participants indicated that the training and the trainer helped them to understand the importance of family planning and that nothing bad hidden from providing such training (hidden Agenda). One participant said “before the training the religious leaders used not to like talking about such issue, but after the training we were more convince with the idea of family planning, before we were confuse and not able to differentiate between spacing and limiting.” • Many participants indicated that the participants in their sessions asked them about the side effects of the different family planning methods, and others that they asked about domestic violence and violence in schools. Other participants said that they were asked about the religious aspects related to using the family planning methods. One participant said “that sometimes we needed an expert to answer the very specialized questions related to family planning methods.” One participant said “what helped us that we were competent in the information and we supported our discussion with the religious saying.” • All the participants indicated that the training influenced their life within their families and they are spacing between their children and they advise their children to space between their children. One participant said “after training I told my daughters in law to space between their children.” • Most of the participants recommended that JHCP to continue the communication with them and to benefit from them in training other religious leaders, and to continue the follow up even through phone calls. One participant said “to have us train in a different place in addition to the local community.” One participant said “to continue the TV ads about family planning,” another participant said “the TV is useful but not interesting another participant said to communicate and train the community directly is more useful,” another participant indicated that” using the religious leaders is convincing and people trust them.” USAID/JORDAN: EVALUATION OF THE JORDAN HEALTH COMMUNICATION PROGRAM 91 FOCUS GROUP FINDINGS Irbid Hayati Ahla Campaign/University Students Ambassadors Introduction The ambassadors/university students were selected through King Abdallah II fund for development (seven students), The Students Deanship at the University (four students), and other friends (two students). A member of the assessment team selected the participant for participation in the focus group from a list sent by JHCP. The participants in the focus group were seven males and six females recruited from three universities: Yaromouk University, Jordan University of Science and Technology (JUST), and Irbid Alahlia University. The participants’ ages ranged from 19 to 24 years old and all were single. One male participant who had four brothers and five sisters indicated that he want to have one child. One female participant and one male participant who had two brothers and two sisters indicated that they wanted to have two children; five participants who had between one brother and eight siblings indicated that they wanted to have three children. Four participants who had between zero and 10 siblings indicated that they wanted to have 4 children. Three students are in their second year, one participant in his third year, six are in their fourth year, and three are in their fifth year. One participant was studying nursing, one statistics, one economics, one graphic design, one art, one anthropology, one media, one medicine, three engineering, and two religion. The training of the students included healthy lifestyles and family planning, and the four topics they focused on most during their activities were the following; first topic is healthy lifestyle, the second is family planning, the third is financial and occupational planning, and the fourth is water and population resources. Most of the participants felt that they needed more training. Main Findings: • The FGD participants indicated that the main purpose/objective of the Ambassadors/University Students activity were; raising health awareness of students (three participants) one participant said “the idea of the activity is related to Hayatee Ahla, health, planning,” another participant indicated “health education,” two participants said “family planning,” one participant said “health care in general,” another participant said “ the main purpose of our project is to understand the basic of the awareness I need to conduct among other university student, so to be prepared to conduct other training for the new students and it includes time management and correcting the different social concept.” Another participant indicated “as an ambassadors our job is to communicate the message to other students from a family and health perspective,” one participant said “as an ambassadors to conduct youth activities and health activities such as diabetes, hypertension, family planning and planning for the future,” another participant said “planning for the future,” one participant said “creating an aware youth and a new health educated Jordan,” a participant said “to disseminate and mobilize an awareness culture through the youth and their friends.” • The participants indicated that the main messages/topics they discussed as ambassadors were; family planning, planning for the future, the ideal weight, healthy lifestyle, financial and health planning, population issues, smoking, another participant said health and physical activities, when the participants were asked to rank the four most topics they focused upon, they reported after a long discussion that the four topics they focused upon the most in their activities, from higher to lower order, were healthy lifestyle, family planning, financial and occupational planning, and water and population resources. 92 USAID/JORDAN: EVALUATION OF THE JORDAN HEALTH COMMUNICATION PROGRAM • When the participants were asked which topics that they don’t feel comfortable discussing, most of the participants found it difficult to discuss family planning issues especially the methods. A participant said “it is hard to expand a lot on religious issues for example the difference between family planning and limiting the number of children.” Another participant said “it was difficult to explain the ideas of family planning and its relation to limiting the number of children.” One male participant added “it is difficult for youth to explain and discuss the family planning methods,” another female participant added “and also to females was difficult to discuss it,” in general. Also, most of the them said it is not easy and awkward to discuss family planning methods among male and female altogether they were okay to talk about it with a person of the same gender. One participant said “all the topics are logical and needed.” • Eight out of 13 FG participants felt that they need more training on the different issue in the training they did receive, one participant said “in Amman workshop the information given to us was not sufficient and the training related the financial and occupational planning was not enough” ne participant said “ they told us that they will give us more training and we are waiting, it is good if they train us as trainers and then to let us train other youth this will be good and cost effective…”. • All the participants felt that they have the information to the general issues related to the healthy lifestyle, smoking, and family planning; however, one participant said “that they did not discussed the details of the family planning methods.” Three participants said we were not able to answer the specialized questions related to the family planning like questions related to how to use a family planning or the medical problems related to it such as side effects. One participant said “it is important to have a specialized person in IFTA with us to answer questions related to religion and family planning.” A participant from the school of Islamic religion said “I did not need that as I know the information,” another medical student said “I was able to answer the questions.” One participant said “I had no problem to answer the questions I have the information.” One participant said “I am able to answer all the questions related to smoking and passive smoking.” • The participants indicated that they implemented many activities, some of which they organized with the local community. Most of the participants distributed pamphlets and posters related to healthy lifestyle and family planning at the universities, the international commercial center, Irbid shopping mall, in the local community, and in King Abdulla hospital. One participant said that “they might take the posters for the kids to play with.” Another said “some will tear it without reading it,” and one participant said “we distributed the posters and we told the people that we have a competition in which you need to read the posters in order to be able to answer and win the competition.” • The participants indicated that they attended and participated in Hayati Ahla campaign at Al Hassan city, in addition to participating in the comic drama called Zaal wa Khadra done by JHCP related to JHCP initiatives. One participant said “we participated in the activities of the information stations.” Another participant said “I talked with the youth about healthy diets.” One more discussed the ideal weight, many were discussing smoking and its influence on health, and some gave the youth an apple for two cigarettes. Six participants discussed family planning issues but males with males and females with females as it is awkward to do it with the opposite sex. One male participant said “I was discussing family planning. One female participant from the nursing school was with the group and when she heard the topic she left.” One participant said “to discuss and to talk about it in relation to population growth and resources is accepted with different sex. One of participants said “I discussed the family planning with people visiting the shopping mall,” 3 discussed it at the university, and another participant said “we conducted a play about family planning in the Hayati Ahla campaign.” One participant commented that “the family planning topic is not embarrassing,” but another participant said “it is hard to talk about it and as students we will not able to discuss it.” USAID/JORDAN: EVALUATION OF THE JORDAN HEALTH COMMUNICATION PROGRAM 93 • In relation to the number of male and female students who were reached by the participants, one participant commented that “males prefer to reach females, and females prefer to reach males,” while another female said “when I discussed the smoking topic more male youth attended.” Another female said “in Yarumok University more females attended the activities.” Seven participants said mostly equal number of males and females attended the activities, three participants said more females attended the activities, and two participants indicated that when they discussed smoking more male attended the activities. • The participants indicated that in the training they received as ambassadors, they were provided by the AWSO training manual, and they attended a workshop and they were given a bag with material produce by the JHCP in addition to a T shirt. Twelve participants find the resources useful and helped them to educate others, one find it not useful, 11 of the participants found it adequate and helpful in the training, 1 participant found it very adequate, and another 1 inadequate. • The participants indicated that they received money for transportation (40 JD for 2 days for transportation purposes to attend the training sessions for each participant), and sometimes they got lunch and mostly they bring juice and water for them. • Although there was no follow-up from JHCP to the ambassadors after August 2011. Five students of 14 from Yarmok University are still active and 9 out of 18 from JUST University are still active. At Irbid University they trained six participants who then trained three more and all nine are still working. Most of the participants indicated that the JHCP followed them up according to the activities for example when they had Hayati Ahla campaign, others said that a doctor called Hana came and met with the and gave them posters and she always followed up; three participants said they had a supervisor but didn’t do any activities. • The participants noted that they need more communication with the JHCP and to have a face book special for this, another participant noted the need for regular meetings, workshops and training, as well as a plan to do more activities with Dr. Hana’s follow up. The students wanted to have the meetings on Saturday as they have no school on that day. One participant said “we need the workshops to be more of practice than theory we need to be more active learners.” Two participants said they needed more information. One participant said “we need to have an advance training for the ambassadors and more coordination between JHCP and the universities and the students.” 94 USAID/JORDAN: EVALUATION OF THE JORDAN HEALTH COMMUNICATION PROGRAM APPENDIX E. RECOMMENDATIONS ON THE HEALTH COMPETENT SCHOOLS INITIATIVE The USAID/Jordan Office of Education has recently taken responsibility for support to this initiative in partnership with the Ministry of Health. The evaluation team's assessment of HCSI has led to the following recommendations. • HCSI should be implemented on a national scale. This will require a strategy to decide the following: if there are geographic priorities, if there are school-specific priorities (girls schools, schools in poor communities), the role of local governments and MOE staff, and other key factors. The MOE is enthusiastic about being a partner in this effort. • Look at expanding the use public-private partnerships. This could be done at the school level or multiple schools. • For USAID this activity requires only funding and some updating, and possibly some new material. The technical competence has been transferred. • Use the existing schools with dynamic health competency activities to model and be motivators other schools. • Consider increasing the 100JD allowances to schools and introducing additional incremental allowances if activities are effective. Currently, teachers are subsidizing some of the costs. • Incorporate community involvement into the activity. Community involvement was not part of the original HCSI content. However, the visited schools enrolled parents and then the larger community in the activities and events. This innovation expanded the reach of health messages, reinforced behavior change, increased attitude change, and empowered students to adopt healthier lifestyles. The HCSI should support and develop a strategy, materials, tools, and models to engage first parents and then the larger community in building health competence. This has the potential to be a large and effective community mobilization activity with a low cost and limited effort. It may also provide a new opportunity to address spacing and small, quality families. USAID/JORDAN: EVALUATION OF THE JORDAN HEALTH COMMUNICATION PROGRAM 95 96 USAID/JORDAN: EVALUATION OF THE JORDAN HEALTH COMMUNICATION PROGRAM APPENDIX F: RECOMMENDATIONS ON CONSULT AND CHOOSE This activity is being transferred to MOH and HSS-II. Consult and Choose (CC) is the first step in addressing missed opportunities. When the health system is capable of educating clients and their partners on family planning; changing cultural values around family size, child bearing practices, and gender preference; and then providing appropriate services when desired by the clients, there will be no more missed opportunities, and no need for community outreach, mass media, or any other communication activities. Recommendations • Client education and addressing traditional norms should continue to be supported until better health facility systems are in place. • Advocacy for operational and budgetary changes should be made to stabilize the staffing capacity to provide counseling in all health facilities. • More training and more information should be provided to health providers to address continuing provider biases. This should be matched with client education to create a demand for modern methods. • Providers are still hesitant to discuss the benefits of smaller families. Additional training and development of a protocol for referrals to facilities that provide the client desired method should be implemented. • CC should better address the use of traditional methods and less-effective methods. USAID/JORDAN: EVALUATION OF THE JORDAN HEALTH COMMUNICATION PROGRAM 97 For more information, please visit http://resources.ghtechproject.net Global Health Technical Assistance Project 1250 Eye St., NW, Suite 1100 Washington, DC 20005 Tel: (202) 521-1900 Fax: (202) 521-1901 www.ghtechproject.com