CRTU RESEARCH UTILIZATION ASSESSMENT REPORT RESEARCH WITH A PURPOSE DECEMBER 2008 This publication was produced for review by the United States Agency for International Development. It was prepared by Julie Solo, Sarah Harbison, Emma Ottolenghi, and Nandita Thatte through the Global Health Technical Assistance Project. CRTU RESEARCH UTILIZATION ASSESSMENT REPORT RESEARCH WITH A PURPOSE DISCLAIMER The authors’ views expressed in this publication do not necessarily reflect the views of the United States Agency for International Development or the United States Government. This document (Report No. 07-001-114) is available online. Online documents can be located in the GH Tech web site library at www.ghtechproject.com/resources/. Documents are also made available through the Development Experience Clearinghouse (www.dec.org). Additional information can be obtained from The Global Health Technical Assistance Project 1250 Eye St., NW, Suite 1100 Washington, DC 20005 Tel: (202) 521-1900 Fax: (202) 521-1901 info@ghtechproject.com This document was submitted by The QED Group, LLC, with CAMRIS International and Social & Scientific Systems, Inc., to the United States Agency for International Development under USAID Contract No. GHS-I-00-05-00005-00. CRTU RESEARCH UTILIZATION ASSESSMENT REPORT: RESEARCH WITH A PURPOSE i CONTENTS ACRONYMS ..................................................................................................................................................iii EXECUTIVE SUMMARY ...............................................................................................................................v INTRODUCTION ....................................................................................................................................v PROCESS ...............................................................................................................................................v ACHIEVEMENTS...................................................................................................................................vi FUTURE ................................................................................................................................................vii I. INTRODUCTION ................................................................................................................................... 1 II. METHODOLOGY .................................................................................................................................. 3 III. FINDINGS ............................................................................................................................................. 5 A. OVERALL: HOW AN ORGANIZATION CHANGES ITS MENTALITY.......................................... 5 B. PROCESS OF UTILIZATION ........................................................................................................... 6 C. ACHIEVEMENTS OF UTILIZATION ............................................................................................. 17 D. FUTURE OF UTILIZATION ............................................................................................................ 22 IV. CONCLUSIONS ................................................................................................................................. 25 APPENDIX 1: LIST OF GLOBAL INTERVIEWEES ................................................................................. 27 APPENDIX 2: COUNTRY REPORTS ...................................................................................................... 29 CRTU RESEARCH UTILIZATION COUNTRY REPORT: KENYA .................................................. 29 CRTU RESEARCH UTILIZATION COUNTRY REPORT: MADAGASCAR.................................... 39 CRTU RESEARCH UTILIZATION COUNTRY REPORT: RWANDA .............................................. 47 CRTU RESEARCH UTILIZATION COUNTRY REPORT: UGANDA .............................................. 53 APPENDIX 3: SCOPE OF WORK ............................................................................................................. 59 ii CRTU RESEARCH UTILIZATION ASSESSMENT REPORT: RESEARCH WITH A PURPOSE CRTU RESEARCH UTILIZATION ASSESSMENT REPORT: RESEARCH WITH A PURPOSE iii ACRONYMS ADRA Adventist Development and Relief Agency APHIA Population and Health Integrated Assistance Project ART Antiretroviral treatment ARV Antiretroviral medications CA (USAID) Cooperating agency CBD Community-based distribution CCC HIV/AIDS Comprehensive Care Centers CME Continuing Medical Education CNLS Commission Nationale de Lutte Contre le SIDA COC Combined oral contraceptive CONRAD Contraceptive Research and Development Program CPR Contraceptive prevalence rate CRS Catholic Relief Services CRTU Contraceptive and Reproductive Health Technologies and Research Utilization CSTS+ Child Survival and Technical Support Plus CTPH Conservation Through Public Health CTR Contraceptive Technology and Research DHS Demographic and Health Survey DMPA Depo Medroxyprogesterone Acetate DRH Division of Reproductive Health (Kenya Ministry of Health) ECP Enhanced Country Program FHI Family Health International FHI/NC Family Health International/North Carolina office FP Family planning FPRWG Family Planning Research Working Group FPWG Family Planning Working Group FY Fiscal year GTZ German Technical Cooperation HBC Home-based care (for HIV/AIDS) HIV Human Immunodeficiency Virus ICL I Choose Life Project/Kenya IPPF/WHR International Planned Parenthood Federation/Western Hemisphere IRH Institute for Reproductive Health (Georgetown University) IRs Intermediate results IUD Intrauterine device JHU/CCP Johns Hopkins University Center for Communication Programs KAIS Kenya AIDS Indicators Survey LAPM Long-acting and permanent methods M&E Monitoring and evaluation MAP Madagascar Action Plan iv CRTU RESEARCH UTILIZATION ASSESSMENT REPORT: RESEARCH WITH A PURPOSE MCH Maternal and child health MEC Medical eligibility criteria (WHO) MOH Ministry of Health MOU Memorandum of understanding MSH Management Sciences for Health NASCOP Kenya MOH HIV/AIDS Division NGO Nongovernmental organization PATH Program for Appropriate Technologies in Health PEPFAR President’s Emergency Plan for AIDS Relief PMTCT Prevention of mother-to-child transmission PQC Product quality and compliance PRH Office of Population and Reproductive Health (USAID) PSI Population Services International RFA Request for application RH Reproductive health RtoP Research to Practice RTU USAID’s Research Technology and Utilization Division SDI Service Delivery Improvement SDM Standard Days Method SubQ Depo Subcutaneous injection for DMPA TA Technical assistance TWG Technical working group UNFPA United Nations Population Fund USAID United States Agency for International Development URC University Research Corporation VCT Voluntary counseling and testing WHO World Health Organization CRTU RESEARCH UTILIZATION ASSESSMENT REPORT: RESEARCH WITH A PURPOSE v EXECUTIVE SUMMARY INTRODUCTION Family Health International (FHI) is a well-known and highly respected research institution, but how well does it promote research findings to contribute to evidence-based reproductive health policies and programs around the world? The USAID-funded Contraceptive and Reproductive Health Technologies and Research Utilization (CRTU) Program includes a significant focus on this important aspect of the research process. USAID organized a four-person team to conduct an assessment to explore how well this program is working and to develop recommendations to improve research utilization in the future. Sources of data included review of key documents including a detailed self-assessment prepared by FHI, interviews with 35 global partners and stakeholders, and visits to four countries—Kenya, Madagascar, Rwanda, and Uganda—where the team met with 144 individuals. The assessment focused on three main areas: (a) the process of promoting research results, (b) the achievements of research utilization, and (c) conclusions and suggestions for future improvement. PROCESS FHI began to focus explicitly on research utilization in 2001 with its Research to Practice (RtoP) Initiative after encouragement from USAID. This focus has grown significantly under CRTU, to the point where over 25 percent of project resources are dedicated to research utilization. Initial efforts before RtoP focused primarily on information dissemination, but research utilization has become more proactive in recent years, with a broad range of strategies, including the following key components: (a) more formal partnerships, (b) focus country programs, and (c) targeted advocacy to key users of research results through more tailored dissemination and study tours. While much has been achieved, there is still much to be done to fully change the mindset of the organization, fully modify the research process, and fully realize the potential of being ―one FHI‖ where the research and programs divisions are working together more closely. This reorganization has tremendous potential given the extensive field presence of FHI’s HIV-focused programs, but requires ―consciousness￾raising‖ of FHI staff regarding the organization’s family planning tools and research results. Partnership has been essential to research utilization efforts. Formal memoranda of understanding (MOU) were deliverables under CRTU, and the project began with eight signed MOUs, later adding three more. While these often led to positive outcomes, they were also very time- and labor-intensive, with FHI headquarters staff acting as relationship managers for each of the MOU partners. Such formal arrangements are most likely not necessary for future research utilization efforts. Instead, there should be more efforts to improve partnerships by ensuring that all partners get credit, regularly sharing information, and strategically identifying the most appropriate partners. CRTU has done a good job of working closely with ministries of health (MOH) in a number of countries and ensuring ownership. One of the most complicated issues in partnership in research utilization is the question of handoff and clearly defining the roles of research organizations and service delivery organizations. These lines get more blurred as FHI takes on a greater role in research utilization and service delivery organizations engage in more research activities. Many saw the need for better coordination, from the level of USAID down to field-level implementation. A staff member from FHI called the Enhanced Country Program the ―backbone of research utilization efforts.‖ The idea of having focus countries comes from experience that showed the importance of having a country presence in order to effectively identify local needs and facilitate local change. The four main focus countries have been Kenya, Madagascar, South Africa, and Uganda. It was clear from country visits that the presence of FHI staff in-country greatly facilitates research utilization and allows for opportunities that could not be realized from a distance. However, it is clearly too expensive to have a large number of focus countries, making it important to learn how best to expand to non-focus countries. Experience in Rwanda shows the substantial potential of having research utilization impact in a non-focus country, through consciousness-raising of FHI staff and study tours for partners to see community-based distribution (CBD) of Depo Medroxyprogesterone Acetate (DMPA) in Uganda and family planning (FP)/HIV integration in Kenya. vi CRTU RESEARCH UTILIZATION ASSESSMENT REPORT: RESEARCH WITH A PURPOSE Practice should feed into research just as research feeds into programs and policies. CRTU has tried to include ideas generated at the country level into the overall research agenda, but this has been challenging for a number of reasons, including: timing issues, in part due to different funding cycles of cooperating agencies; difficultly in balancing global and local research priorities; and the lack of successful ways to solicit ideas from the field and turn them into projects. Information programs have gone beyond simple dissemination to create more synthesis materials and tailored dissemination. CRTU has also used more action-oriented dissemination meetings and study tours to see programs in action. The latter have been particularly effective for CBD of DMPA. While tools and job aids have been widely disseminated and adopted in many countries, it is difficult to know how much they are actually used. ACHIEVEMENTS CRTU is tasked with addressing three intermediate results (IRs), with IR3 being the most relevant for this assessment of research utilization: Use of contraceptives, microbicides, and reproductive health technologies optimized and expanded. Research utilization efforts have led to many policy and programmatic changes that help achieve this result. The main research results that have been successfully promoted by CRTU include: (1) CBD of DMPA; (2) the pregnancy checklist and screening checklists (combined oral contraceptives [COC], DMPA, IUDs); (3) FP/HIV integration; and (4) long-acting and permanent methods (LAPM) of FP. CRTU has developed a set of 14 research utilization indicators to measure reach, capacity building, collaboration, and use of research. However, reporting of results is not comprehensive, and it is challenging to pull together the information to tell a clear story. It is also difficult to link utilization to impact, but this is important for prioritizing efforts—to focus on the end goal and choose efforts that best help to meet this goal. For example, while there has been small-scale success in intrauterine device (IUD) revitalization in Kenya, it appears that there is more latent demand for implants, perhaps indicating that investing greater efforts in implants is a more effective strategy for increasing use of LAPM and meeting women’s needs. CRTU has recognized this need and is engaging in a large project to facilitate registration and availability of a generic two-rod implant. At the policy level, ―WHO looks to FHI for guidance,‖ USAID staff explain. FHI has provided extensive input to WHO’s Four Cornerstones for Evidence-Based Guidance for Family Planning: (1) Medical Eligibility Criteria (MEC) for Contraceptive Use; (2) Selected Practice Recommendations for Contraceptive Use; (3) Decision-Making Tool for Family Planning Clients and Providers; and (4) Family Planning: A Global Handbook for Providers. This global-level policy guidance has important country-level impact; for example, at least 50 national program guidelines have drawn on recommendations from the MEC and Selected Practice Recommendations. Important policy changes have also been made at the country level. For example, CBD of DMPA has been included in the Government of Madagascar’s Norms and Procedures in Reproductive Health, and the checklist to rule out pregnancy has been incorporated into the Madagascar Ministry of Health and Family Planning training manual for providers, the Ugandan MOH Basic Family Planning Clinical Skills Curriculum, the Senegalese Protocoles de Services de la Santé de la Reproduction, and the Ghana MOH Reproductive Health Service Protocols. While programmatic changes are more difficult to measure for a process like research utilization, there have been identifiable changes at the country level. For example, in Moramanga District in Madagascar, FHI research utilization efforts and strong partnerships with the MOH and local nongovernmental organizations (NGOs) contributed to a contraceptive prevalence rate (CPR) increase from 20.9 percent in 2006 to 24.7 percent in 2007 and 35 percent in the second quarter of 2008. During this time, the number of CBD agents trained in CBD of DMPA increased from 23 to 85, and 16 percent of DMPA users obtained their method from a CBD agent. CRTU RESEARCH UTILIZATION ASSESSMENT REPORT: RESEARCH WITH A PURPOSE vii FUTURE The team heard generally favorable opinions about the research utilization efforts by CRTU, supporting the idea of this model of including research utilization in a research project. The team heard and saw many positive examples of programmatic changes due to specific efforts—particularly the focus country programs, advocacy efforts such as study tours, and influence on WHO guidelines. As a USAID respondent stated, ―the research process is a continuum from idea generation through getting it utilized.‖ Such a definition highlights how utilization is an essential component of research, not an optional add-on. However, this does not mean the model is complete, as many felt that service delivery groups also need to receive funding to put research findings into practice: ―Where do you put the money? First you put it in research organizations, get them to put information in a form that others utilize, then you create a forum to share that and put responsibility to utilize information into service delivery contracts‖ (USAID). Specific suggestions are given below, for CRTU and for USAID. It should be noted that there is limited time left in the CRTU, so these recommendations apply both to the project and to future efforts by FHI and other research organizations. For CRTU 1. Build on the “one FHI” potential: Additional consciousness-raising among FHI staff—as was done successfully in Rwanda—will enable even more country staff to be aware of organizational resources. 2. Reorient the partnership approach from formal MOU arrangements to more strategic and complementary partnerships. There should also be efforts to expand beyond the focus on the public sector to work more with the private sector. 3. Improve efforts to acknowledge the contribution of partners and share attribution, addressing the perceptions of this need among many partners. 4. Explore additional ways in which critical research findings from other organizations can be effectively incorporated into CRTU’s utilization efforts. 5. Build on the successes of the focus country approach, including more efforts in non-focus countries: Learn from the lessons in Uganda and Madagascar—much can be achieved with fairly minimal inputs. For non-focus countries, there should be a strategic approach involving the identification of facilitating factors (for example, strong government support for FP, emphasis on community health, etc.), using the on-the-ground presence of the 40 HIV-focused FHI field offices. 6. Continue to focus on systematic advocacy, for example, incorporating findings into USAID bilaterals in collaboration with USAID staff, continuing input into WHO guidance documents, assisting in moving guidelines into practice, and in general acting as a catalyst more than a service implementer. 7. Develop more effective ways of incorporating needs identified in the field into global research agenda. This means listening better to country needs and more consideration of research utilization when choosing research studies to conduct. 8. Allow for more autonomy and data analysis at the country level, which will enhance local partnerships and could also reduce the central-level costs associated with focus countries. This has to be done with close attention to maintaining high-quality research standards. 9. Include more attention to building local research capacity to generate research results, transform them into actionable findings, and use them in programs. 10. Incorporate more consideration of behavior change theory/motivation. This is particularly important, for example, in promoting practices that require behavior change from health providers. 11. Regularly collect and analyze cost information on proposed interventions to facilitate efforts in scaling-up and a greater focus on sustainability. viii CRTU RESEARCH UTILIZATION ASSESSMENT REPORT: RESEARCH WITH A PURPOSE For USAID: 1. Create better linkages between the Research Technology and Utilization (RTU) Division and other Office of Population and Reproductive Health (PRH) Divisions, especially Service Delivery Improvement (SDI): ―[T]ear down the walls!‖ as one USAID staff member stated, possibly through a joint RTU/SDI request for applications (RFA), pairing up staff from each unit, or finding ways beyond open staff meetings to interact. 2. Develop processes and indicators for findings to be systematically considered and used in global projects and bilaterals: This will require advocacy and better linkages between USAID/Washington and field Missions to enhance Mission involvement in research and commitment to scaling-up successful pilot projects. 3. Allow for some flexibility in timing and funding: Provide flexible funds for service delivery innovations and cutting-edge research issues. 4. Conduct a parallel assessment to better understand the extent and process of use of best practices in service delivery programs, both globally and in bilaterals. 5. Fund study tours for USAID staff from USAID/Washington and Missions. 6. Assess the feasibility of and appropriate process for promotion of a package of evidence-based best practices to service delivery organizations and field Missions. In addition, respondents suggested several priority research topics for the future, including: How best to improve motivation and retention of CBD agents and sustainability of CBD programs How best to effectively provide integrated FP/HIV services in a range of settings Understanding and addressing the issue of discontinuation of use of various contraceptive methods Improving understanding of how to change provider attitudes, address motivation, and remove other barriers to improving quality of services Improving understanding about impediments to wider use of LAPM. Overall, the team recommends that research utilization continue to be a key part of CRTU and other research projects, and that this be complemented by efforts on the service delivery side as well. Three key words sum up the recommendations for research utilization by FHI and others for the future: balanced, systematic, and strategic. 1. Balanced: For partnerships to work, there must be a balance in terms of resources, respect, responsibility, and research priorities: a. Resources must be made available for both research and service delivery organizations to work together. b. Credit must be given to all partners. c. Using evidence to inform policies and programs must be everyone’s mandate. d. Research priorities must reflect needs identified by a range of partners, at both the global and national levels. 2. Systematic: Research utilization should be an integral part of research efforts and service delivery planning and implementation. For this to happen there is a need to systematically: a. Assess the potential usefulness of every research study. b. Work with host-country colleagues to increase demand for using evidence. c. Incorporate evidence-based best practices into USAID bilaterals. d. Bring together service delivery and research organizations. CRTU RESEARCH UTILIZATION ASSESSMENT REPORT: RESEARCH WITH A PURPOSE ix 3. Strategic: Research utilization efforts come with associated costs, of course, not only in terms of the actual costs of the efforts but also in terms of diverting resources from research. It is essential therefore to make strategic choices about where to focus research utilization efforts, in terms of: a. Choice of focus countries b. Choice of key findings c. Choice of key partners d. Choice of most effective research utilization efforts. These principles are part of the CRTU’s research utilization approach, but they can be further strengthened to build on the momentum achieved over the past several years and continue to move knowledge into practice. x CRTU RESEARCH UTILIZATION ASSESSMENT REPORT: RESEARCH WITH A PURPOSE CRTU RESEARCH UTILIZATION ASSESSMENT REPORT: RESEARCH WITH A PURPOSE 1 I. INTRODUCTION Family planning (FP) is typically considered to be a successful global health intervention and a key component of development efforts. However, unmet need around the world remains high. Family Health International (FHI) has played a key role over the years in helping to develop new contraceptive methods and improve access to and use of existing methods, a role that continues to be extremely important. It has become known as a well￾respected institution that conducts high-quality research. From 1995 to 2005, FHI implemented the Contraceptive Technology and Research (CTR) Project, funded by the United States Agency for International Development (USAID). In April 2005, the five-year Contraceptive and Reproductive Health Technologies and Research Utilization (CRTU) Program was awarded to FHI as a follow-on to CTR. CRTU differed from its predecessor in a number of ways, with the most striking being a greatly increased focus on research utilization. FHI had begun a Research to Practice (RtoP) initiative under CTR, but this was a small component. Under CRTU, more than a quarter of the CRTU project’s resources and deliverables are directly related to research utilization outcomes. ―One of our biggest problems in our field is getting out what we know. We could make huge inroads if we could just get out the interventions that we know [are effective],‖ explains one USAID staff member. One way to try to do this is through putting funds into a research organization to facilitate the use of evidence in policies and programs, as has been done with the CRTU Project. This report discusses the process and the potential of this model, and provides suggestions for improvement and modification to have even greater impact. 2 CRTU RESEARCH UTILIZATION ASSESSMENT REPORT: RESEARCH WITH A PURPOSE CRTU RESEARCH UTILIZATION ASSESSMENT REPORT: RESEARCH WITH A PURPOSE 3 II. METHODOLOGY USAID decided to evaluate the research utilization component of CRTU in order to both assess the progress of the project’s utilization efforts and to determine to the extent possible whether or not the incorporation of such a utilization component into a research project is the most efficient and effective way to promote research utilization. Conducting this assessment midway through the cooperative agreement, in 2008, made sense, given that it was far enough into the agreement to assess progress but with enough time remaining to still allow for adjustments. A four-person team conducted the assessment from August to October 2008. Team members included two external consultants and two staff members from USAID. Data collection included review of background materials, discussions with FHI staff in North Carolina, interviews with partners and key stakeholders at the global level, and field visits to four countries. The team interviewed 35 representatives of global-level organizations, including USAID, other collaborating agencies, and multinational groups such as WHO (Appendix 1). The four countries visited included three focus countries (Kenya, Madagascar, and Uganda) and one non-focus country (Rwanda). Two members of the team traveled to Kenya and Uganda, while the other two visited Rwanda and Madagascar. The team met with 144 respondents in the four countries (Table 1). A non￾focus country was chosen so that the team could see a country where FHI has limited utilization activities to help explore the effectiveness of the CRTU’s targeted technical assistance for research utilization taking place in focus countries. In all four countries, the team assessed program implementation, research progress, and evidence of collaboration and research utilization (Appendix 2). TABLE 1: FIELD VISITS BY ASSESSMENT TEAM Country Dates of Visit Number of Respondents Kenya Aug 31–Sept 10, 2008 50 Uganda Sept 11–17, 2008 39 Rwanda Sept 3–6, 2008 24 Madagascar Sept 10–18, 2008 31 Following a discussion of overall issues regarding integrating research utilization into FHI, findings are presented under the three main areas that were addressed in the assessment: (a) the process of promoting research results, (b) the achievements of research utilization, and (c) conclusions and suggestions for improvement for the future. There are limitations to this assessment. Given the large number of projects and partners under CRTU and the relatively short amount of time for the assessment, the team could not explore all issues in depth. In particular, the team’s time in each of the four countries visited was fairly brief, limiting the chance to meet with more than a small number of health providers or clients. In addition, a number of respondents did not clearly distinguish work conducted under the CRTU project from other FHI activities. Finally, much of the data for this report are qualitative in nature. However, clear themes emerged in the interviews, and the large number of respondents strengthens the validity of the conclusions. 4 CRTU RESEARCH UTILIZATION ASSESSMENT REPORT: RESEARCH WITH A PURPOSE CRTU RESEARCH UTILIZATION ASSESSMENT REPORT: RESEARCH WITH A PURPOSE 5 III. FINDINGS A. OVERALL: HOW AN ORGANIZATION CHANGES ITS MENTALITY FHI sees its main identity as being an organization that ―conducts world class research…our number one value is excellence.‖ While this has not necessarily changed, incorporating a focus on research utilization has meant a lot of change to the organization. The goal is ―to embed research utilization into the DNA of every staff member,‖ but just how much has this happened? FHI recognizes that change can be a slow process, ―an evolution not a revolution,‖ and it is challenging to change the mindset of an organization. There is certainly more of the feeling that the end point is changing policy and programs, not just publication. ―This changes how you do things,‖ explain FHI staff, and the details of this will be described below. While FHI’s self-assessment states that research utilization is now ―how we do business,‖ the document later points out that ―institutionalizing research utilization throughout FHI and across all programs remains incomplete.‖ Some staff are concerned that the increased focus on utilization takes away time and resources from research. For example, FHI staff mentioned potential studies around bleeding issues associated with hormonal methods that had to be shelved and studies looking at the interaction of hormonal contraceptives and antiretroviral treatment (ART) that had to be scaled back. FHI staff point out that in both cases this happened during the prioritization necessary as part of the workplan review and budgeting process with USAID, while USAID highlighted the general resource constraints for research. FHI’s identity is also undergoing a change due to structural reorganization, creating ―one FHI.‖ Rather than the research and programs being separate, at the country level these are now merged under one country director who takes responsibility for all activities. This process has important implications for research utilization, discussed below, but remains a challenge. As FHI points out in their self-assessment, ―collaborative efforts to incorporate RH into primarily HIV-oriented offices require all the relationship-building typical of a new partnership.‖ This reorganization can enhance an existing strength of FHI—having a unique mix of skills and expertise in one organization; ―there is a value in having a critical mass of expertise in one place that is difficult to quantify.‖ This leads to many interesting opportunities that might otherwise be missed. For example, when the question arose regarding the grace period for women coming for a follow-up injection of DMPA, rather than conducting an expensive study, an FHI staff member came up with the idea of using an existing data set from another study to answer this question. Findings that the grace period could safely be extended from two to four weeks have now been adopted into WHO guidelines, a change with significant programmatic implications that was brought about at very low cost. FHI’s identity is a key factor in its ability to promote utilization of research and it comes from both its long￾standing work in contraceptive technologies and in HIV; ―FHI is so well known and there is a good reputation that comes with them. This is one of their biggest advantages. Country partners know of them, often through their HIV work, and they can leverage that.‖ This points to how FHI can take advantage of its reorganization both in terms of building of infrastructure and reputation. FHI staff acknowledge that there is still a need to better address the ―piece at the beginning—thinking about the potential usefulness of a particular research finding or technology.‖ There have been increased efforts and systems to review key documents such as concept proposals and protocols to ensure that research utilization concepts are included. But this process needs to start even a step earlier, by choosing more strategically to conduct projects that are most potentially useful. It is interesting to note that in their self-assessment, FHI describes how research utilization ―begins with the research results,‖ but in fact it begins even before that. 6 CRTU RESEARCH UTILIZATION ASSESSMENT REPORT: RESEARCH WITH A PURPOSE The explicit focus on research utilization started with RtoP under CTR after encouragement from USAID, but has grown and changed under CRTU. Figure 1 estimates the growth in expenditures on research utilization from 2001 to 2008. Overall, in FY 2007, 69 percent of CRTU funds were spent on research, 12 percent on RtoP, 10 percent on the Enhanced Country Program (ECP), and 9 percent on Information Programs. The model of research utilization has also changed. The RtoP conceptual framework was a push model, seeing utilization as generally a linear process, and was vague about the detail of processes to encourage utilization. FHI staff now note that ―there is not a lot in this research utilization process that turns out to be linear.‖ There is now a greater emphasis on specific processes, including memoranda of understanding with partners, a focus country program, and targeted advocacy efforts through information programs and efforts such as study tours. Figure 1: CTR/CRTU Growth in Total Research Utilization Expenditures by Fiscal Year CTR/CRTU: Growth in Total RU Expenditures by Fiscal Year $1,600,000 $3,000,000 $2,500,000 $2,300,000 $3,500,000 $3,500,000 $5,400,000 $5,600,000 $0 $1,000,000 $2,000,000 $3,000,000 $4,000,000 $5,000,000 $6,000,000 2001 2002 2003 2004 2005 2006 2007 2008 Year USD (approx.) B. PROCESS OF UTILIZATION Question: Has the CRTU project made progress in promoting the applicability and utilization of research results related to products, tools, and approaches to improve family planning service delivery, both FHI’s own results and those of other organizations? Key Research Results That Have Been Promoted The following are some of the key research results that the CRTU Project has promoted under its research utilization efforts, some of which came from research conducted under the CTR Project and some from CRTU. CBD of DMPA. While community-based agents have provided DMPA in countries in Latin America and Asia for many years, this practice is relatively new to Africa. In 2004, FHI collaborated with the MOH and Save the Children in Uganda to conduct a successful pilot showing the safety and effectiveness of this approach. Since then, the practice has been replicated in additional sites in Uganda and other countries. ―We’ve been able to work consistently on getting the word out—that kind of focus has made an impact. Also timing is important—it comes at a time where there is a revitalization of family planning and growing popularity of DMPA—this is a success of our practice to research efforts—this was a need,‖ states one staff member. Currently, this practice is being implemented in Ethiopia, Madagascar, Malawi, Kenya, and Uganda, and Rwanda is conducting a preliminary feasibility study. There have been policy changes in Madagascar and Malawi, and in these two countries and in Uganda it was included in the USAID bilaterals. Several other countries have expressed interest, including Mali, Nigeria, Tanzania, and Zambia. CRTU RESEARCH UTILIZATION ASSESSMENT REPORT: RESEARCH WITH A PURPOSE 7 Checklists. A very important role of FHI has been creating job aids, disseminating them widely, and assisting with service delivery training on using them; ―you can’t just throw something at the MOH and expect to get it used in facilities.‖ Of all the checklists, it is the pregnancy checklist (a checklist to be reasonably sure that a client is not pregnant) that is most well known and widely used. The COC and DMPA checklists were also mentioned as very useful. Having the MOHs in Uganda and Kenya adopt the checklists as their official documents, with only their MOH logo, has been a successful approach. FP/HIV Integration. FHI is known as a leader in the area of FP/HIV integration. Unfortunately, ―efforts are still hampered by the fact that there is a gaping hole in the evidence on this topic‖ in terms of identifying effective models of service integration. The evidence FHI is using includes modeling evidence; unmet reproductive health needs of clients of HIV services, feasibility and acceptability of integration, safety and effectiveness of FP methods for HIV-positive women—this is all evidence for why, but there is still a need for evidence on how. CRTU’s research agenda includes a number of evaluations of FP/HIV integration models, based on this need identified in the field. An important avenue for pursuing FP/HIV integration is within FHI itself—increasing the number of FHI country programs that are working systematically to integrate FP and HIV services. Youth. With the end of the YouthNet project, the topic of youth was taken on as its own strategy under CRTU. At that point, FHI was funded to do knowledge management around youth issues. When asked what research was being used, an FHI staff member responded that they were a ―catalyst and springboard to focus on the area of youth more than promoting specific results.‖ A staff member from USAID speculated that ―we are more comfortable focusing on methods than on segments such as young people.‖ Under the CRTU, FHI is continuing the Youth InfoNet monthly e-newsletters; coordinating the Interagency Youth Working Group; and promoting targeted dissemination of tools, both new tools and those developed under YouthNet (for example, the Muslim and Christian Family Life Education manuals and the HIV Counseling and Testing Manual for Youth). IUD. FHI did a terrific job on the IUD working group at the global level, according to several partners, and there has been some small-scale success in Kenya in work with the MOH, the ACQUIRE Project, and other partners. However, this came after extensive efforts and substantial resources, calling into question the feasibility of scaling up the efforts around IUD revitalization, which involved a range of advocacy, training, and service strengthening inputs. ―At what point do you say it’s not worth it?‖ asked one respondent. Another agreed that ―I’m prepared to say this horse is not winning,‖ but did not see FHI being at fault in this; rather, there are a lot of problems intrinsic with the IUD. In addition, there are still unanswered questions about how to change provider attitudes. The challenges in bringing about a dramatic impact with the work with the IUD points to a larger issue: It is not just a question of using a finding, but using a useful finding. While the IUD has met with limited success, there is increasing interest in implants, which some see as the ―great LAPM hope.‖ FHI has become involved in a new initiative to expand access to implants through providing assistance for registration and use of a generic version of the two-rod implant. The Sino-implant will be available at less than one-third of the cost of currently available implants such as Jadelle, and so has the potential to make implants much more widely available. FHI was able to leverage modest CRTU funding to obtain a large grant from the Bill & Melinda Gates Foundation to work in 10 countries with 14 implementing partners. FHI was uniquely suited to move this forward, especially given the presence of the Product Quality and Compliance (PQC) Division in the organization—PQC staff were able to visit the Chinese facility, help put together the dossier, and also provide connections to key personnel at WHO to expedite prequalification. One respondent expressed some concern about FHI becoming too involved with this product, as it might compromise their neutrality in working more broadly on implants. This highlights the importance of a research organization being seen as an unbiased source of the best available evidence. During the team’s visits to Kenya and Uganda, health facility staff stated there is a pent-up demand for implants that will make this method very popular once it is available; this may lead to a decrease in IUD demand, even 8 CRTU RESEARCH UTILIZATION ASSESSMENT REPORT: RESEARCH WITH A PURPOSE where established: many women who came intending to use implants had received the IUD as a second option because implants are not available in most facilities. The team also heard that DMPA stockouts had increased IUD uptake. Similarly, work in Rwanda to expand access and use of LAPM has found that implants are significantly more popular than IUDs. While these are just anecdotal pieces of information, they might be indicative of important trends that should influence prioritization of efforts. Of course, there are many factors in research utilization that are beyond the control of the research organization, some of which facilitate adoption and some of which hinder use of research. For the former, often it is a question of good timing. For example, the MOH and other partners in Rwanda have been very receptive to findings around FP/HIV integration because many partners had acknowledged the importance of meeting the needs of couples and helping HIV-positive women avoid pregnancy if they chose to; one respondent explained how the Minister of Health would often say to him, ―are your HIV-positive women still having babies? What are you doing? Please do something for them.‖ Likewise, CBD of DMPA was picked up in Rwanda and Madagascar, both countries where there is a strong focus on FP from the government, and an emphasis on community health. While FHI cannot make these facilitating conditions happen, they can identify such conditions and tailor research utilization efforts to take advantage of such possibilities. This draws attention to the importance of an on-the-ground presence to be able to identify and understand whether and where such conditions exist, an issue that is discussed more in the section on focus countries. There is a need to test interventions that can be scaled up; ―you shouldn’t test it unless you have scale-up in mind,‖ stated FHI staff. This draws attention again to the need to be thinking about utilization at the initial planning stages of a project, and of including better cost information about interventions to help in planning and implementing scale-up of successful interventions. In fact, given the expertise in economic evaluation at FHI, the assessment team was surprised that there were not more cost studies being undertaken by CRTU. There is some tension between FHI and other partners regarding different models of scale-up, with FHI generally being more cautious and some partners pushing for more rapid scale-up. For example, this is seen in different approaches in Madagascar in scaling up community-based distribution of DMPA compared to the Standard Days Method (SDM), with the latter moving much more rapidly. However, these are very different technologies, and there are many partners, including the MOH, who want to proceed more cautiously with expanding DMPA to the community level. Work on the Sino-implant highlights this same issue about speed; ―we’re playing a conservative role and some partners are criticizing us,‖ explained FHI staff. It is interesting to note that the main findings that people mention and that have been taken up are from studies that have been conducted by two or three staff members, and that several respondents mention and praise these staff specifically; ―they’ve been out there trying to make things work in a creative kind of way…they understand the need to get methods more available and look at what stands in the way.‖ They are also praised for being proactive. It would be worthwhile to look at what these staff members are doing and try to instill some of these ideas into the organization more broadly, namely listening to the problems in the field and developing research based on these problems and then proactively getting the word out. Best practices need to be adapted to the context in which they are going to be implemented. There will always be adjustments based on the realities at the country level. For example, in Kenya, the MOH thought that giving 13 packets of pills at one time would not be feasible due to commodity concerns—this is not a failure of research to practice, but is simply a realistic adjustment. However, some respondents felt that there should be more standardization, for example with training of CBD agents to provide DMPA, which the team was told ranges from three weeks in some settings to three days in others. Often, a pilot study will still be needed when a new country adopts a practice. CBD of DMPA is a good example of this. Such task-shifting to lower-level cadres remains controversial in most settings, and often the MOH in a country will want to proceed cautiously and test this intervention on a small scale before expanding it. CRTU RESEARCH UTILIZATION ASSESSMENT REPORT: RESEARCH WITH A PURPOSE 9 It is also important to recognize that different types of research results—for example, a clinical finding that the IUD is safe for HIV-positive women, a checklist, or the fact that CBD agents can safely provide DMPA—will require different types of research utilization efforts. In addition, some research studies will not lead to identification of a successful strategy or intervention. Not everything will be used, and this is not necessarily a failure of research utilization. Some research has negative findings; ―the cost of being at the cutting edge is that certain things won’t work,‖ notes FHI staff. That is not to say that there are no lessons from ―unsuccessful‖ projects; for example, much has been learned from the microbicide trials even though these trials did not achieve their desired results. One USAID staff member posed an important question: ―[D]oes FHI see itself as a global leader in research or as promoting FHI and its own findings?‖ FHI has mostly focused on findings generated by their own research, under both the CTR and the CRTU projects. However, there are a few examples of promoting findings from other organizations, in particular use of the SDM in partnership with the Institute for Reproductive Health (IRH) and the systematic screening tool, developed by the Population Council. When asked about the role of other organizations when FHI promotes their findings, they used the example of Population Council and explained how Population Council was involved in reviewing the report and methodology, but they admitted that it was ―tricky because Population Council felt an ownership role and they worried about us screwing it up. They would have liked to have more participation…but for logistics reasons, we didn’t take them up on their offer of technical assistance.‖ From their side, Population Council staff felt that ―we were purposefully being kept at arm’s length.‖ Interestingly, at the field level, FHI staff in Madagascar did not see a difference in promoting findings from FHI research (for example, CBD of DMPA) compared to findings from others (for example, systematic screening), though they had more success with the former. Based on the team’s assessment, it is clear that promoting other organization’s findings is not happening very much under CRTU; what is less clear is whether it would be effective for them to do more of this. This also raises the issue of bundling results, or promoting best practice packages compared with only one specific finding or tool. CRTU experience thus far does not provide a clear model for how best to promote a package of findings, although lessons can be learned from experiences in Madagascar. Partnership Partnership has been essential to research utilization efforts. As CRTU staff in Madagascar explain, ―our job is to connect people…and make sure everything is synergistic. And in-country strategic partnership building‖ is the key to FHI’s success in research utilization. The team heard many positive comments about partnership with FHI. ―FHI has a culture of sharing results, being inclusive, and good collaborators,‖ according to a DC-based partner. In Rwanda, partners praised the fact that ―FHI never does research alone, they are always with national institutions.‖ However, many issues were raised that point to the need for FHI to create better balance in their partnership and to be more strategic in their choice of partners, looking for logical fitting together of pieces and complementarity. A key factor in utilization is the hand-off—which means that partnership is key. Many talked about the confusion in roles—what does a service delivery group do and what does a research group do? The assessment team heard mixed responses to this question, but overall, it is clear that the lines have blurred and this has led to tensions. One USAID staff member felt that ―there is never really a handoff because the research guys are always learning new things—it’s not like we’re producing cans of peas.‖ Some see a need for clear division of labor; ―if we need to get the MOH to change national guidelines or norms, yes, I’d bring in researchers, but if I’m training providers, do I need FHI? No. At the policy level, yes, you need someone there with the evidence. Service delivery, that we can do quite well,‖ states a staff member from a service delivery organization. FHI’s role should be ―technical support but not the implementers,‖ according to a partner in Rwanda. Others talk about the need to pair research and service delivery groups, and in particular to improve the linkages between the Service Delivery Improvement (SDI) and Research Technology and Utilization (RTU) Divisions at 10 CRTU RESEARCH UTILIZATION ASSESSMENT REPORT: RESEARCH WITH A PURPOSE USAID. ―We haven’t had an RFA [Request for Application] that combines SDI and RTU, but we have done it in bilaterals—then they can see themselves as one team rather than competitors.‖ Respondents highlighted the importance of ownership among partners so that research results are taken on in a sustainable manner, particularly with government partners. At the country level, the team heard many positive comments about CRTU working in close partnership with the MOH. USAID Mission staff in Rwanda praised FHI for this: ―they are good at pushing the program, but stepping back at the same time—this is an MOH activity.‖ With the Kenyan materials on FP/voluntary counseling and testing (VCT) integration, it is ―all about the Ministry of Health—FHI is very invisible,‖ according to FHI staff. However, a partner in Kenya disagreed and stated that all the names in documents published are FHI staff and that their staff were not listed. In many countries, tools developed by FHI are now branded as MOH tools. MOH branding is very important for partners to adapt and use FHI materials. It is recognized that this is a developmental process, and that early in the process of developing a tool or conducting research, it is very likely and even appropriate that a product or curriculum or approach will be identified with FHI. Of course, at this stage there should be close collaboration with partners and government, and as soon as possible, official branding should be changed. One thing that can limit a partner’s sense of ownership is when they do not feel that they are given adequate credit or acknowledgement, an issue raised by many respondents. People saw this lack of credit to others as evidenced in the pregnancy checklist and the CBD of DMPA work, both often touted as being FHI work, while in fact multiple partners were involved in the development and implementation of these practices. FHI staff acknowledged the importance of the issue of credit when briefing the assessment team, citing a quote by Harry Truman (see box). There were mixed feelings about how well FHI communicated—the majority felt that FHI was good at keeping them informed, but some felt that FHI was not always good at reporting back after meetings or completion of research projects. Likewise, some FHI staff felt that sometimes it was difficult to get data from partners. Such communication difficulties are often due to time constraints, but are also sometimes a function of competition. Whatever the cause, systematic and open communication is clearly essential to effective partnerships. Another important role played by CRTU is taking a lead in working groups and coordinating entities, at both the global and country level. In addition to bringing partners together and sharing technical updates, in Uganda this group has played a key advocacy role; if that were not happening, family planning would be off the screen, according to respondents. Respondents repeatedly talked of imbalances in partnerships, primarily due to funding issues. Such partnerships are not typically a deliverable for service delivery groups in the way they are for FHI under CRTU, and linked to this, they do not have funds for it—when you have a model, it becomes a service delivery issue and needs funding. ―Without committed resources up front it’s hard to hold partnerships together.‖ This issue of a real or perceived imbalance in funding arose in most interviews with partners. In discussing the challenges of collaboration, one partner explained that, ―we have one agenda and they have another. They are funding the study so we aren’t in a strong position to negotiate…it was fine until we started talking about the budget.‖ Another service delivery partner expressed this same idea: ―we were not able to be an equal partner because of resource disparities.‖ One way to collaborate better is to have direct sources of funding to work on it, explained another partner. FHI staff also noted that ―the imbalance in resources was the key issue‖ in the challenges faced in their partnership with Adventist Development and Relief Agency (ADRA), which was eventually terminated. In retrospect, they feel that they should have created a fund to subsidize the imbalance. Some of this imbalance may also stem from how USAID central core funds are distributed. Research is a core function of USAIDs Office of Population and Reproductive Health and as a result, research cooperating agencies (CAs) receive a large amount of central core funding. This works well since research CAs are less likely to leverage field support funds from Missions, where the focus is on improving delivery of existing ―It is amazing what you can accomplish if you do not care who gets the credit.‖ — Harry Truman CRTU RESEARCH UTILIZATION ASSESSMENT REPORT: RESEARCH WITH A PURPOSE 11 services rather than on new research. Service delivery projects are supported heavily through Mission field support but receive less central core funds. The issue becomes complicated when core funds are used for operations research and research utilization, which begin to blur with the work of service delivery organizations. Without strong, balanced partnerships, turf issues become battles, with partners saying things like, ―organizations that have specialties [for example, research] should stay with their specialty,‖ and ―I do get a little frustrated with FHI—they step into service delivery and move into implementation. I question whether that is appropriate.‖ Sometimes USAID creates barriers to effective utilization. For example, FHI has significant expertise and experience to offer in the area of FP/HIV integration, but in Uganda the USAID Mission has ―assigned‖ this area to EngenderHealth. It is important to identify ways in which FHI’s resources can be made available and promoted to partners in Uganda. In South Africa, CRTU is also limited somewhat by USAID; it has been mandated to work only on FP/HIV integration, limiting its ability to work on a broader range of FP and reproductive health (RH) issues. Another factor that complicates partnerships is the different perspectives of research and service delivery groups. For example, ―we work on different timelines, we work at a different pace…what you fundamentally have is different organizations with different cultures and different sources of funding—what drives research is doing research and getting results used, what drives services is increasing and improving the services—those differences and the differences in funding are big problems,‖ a staff member from a service delivery organization stated. Some of this points to misperceptions and a lack of appreciation that all partners are working toward the same goal. Another respondent talked about the differing orientation as it played out in identifying research priorities: ―when talking about counseling [as a way to address discontinuation], FHI said we can study the cost-effectiveness of counseling and we said it doesn’t matter, we’re going to do counseling no matter what.‖ Memoranda of Understanding CRTU included more formal partnership through signing memoranda of understanding (MOUs) with a number of organizations at the headquarters level. MOUs are a deliverable under CRTU and an attempt to quantify and be held accountable for partnership. In addition to a signed memorandum, CRTU has a staff member serve as the relationship manager for each MOU partner. CRTU began with signed MOUs with eight partners: 1. ADRA 2. Contraceptive Research and Development Program (CONRAD) 3. EngenderHealth/ACQUIRE Project 4. The Johns Hopkins Center for Communication Programs (JHU/CCP)/INFO Project 5. Management Sciences for Health (MSH)/Center for Leadership and Management 6. Program for Appropriate Technologies in Health (PATH) 7. Population Council 8. Save the Children. MOUs were later signed with Pathfinder’s Extending Service Delivery Project, Macro’s Child Survival and Technical Support Plus (CSTS+) Project, and an activity-specific MOU was signed with Georgetown University’s Institute for Reproductive Health (IRH). While the MOUs were seen as useful on several occasions in terms of clearly identifying roles and responsibilities, they were also seen as difficult and time-consuming. It is not clear that there was substantial benefit to having a formal MOU compared to a more informal partnership. Even when they worked, the field offices often were not aware that formal MOUs had been signed at the headquarters level. Almost every partner gave examples of research projects that they tried to collaborate on, but which never came to fruition, often due to timing or funding issues. 12 CRTU RESEARCH UTILIZATION ASSESSMENT REPORT: RESEARCH WITH A PURPOSE The MOU with the MSH Leadership, Management, and Sustainability (LMS) Project draws attention to the challenges and limits of this arrangement. The partnership was certainly useful in that it led to successful collaboration on two virtual learning platforms through the Global Exchange Network, one on CBD of DMPA and one on LAPM, in collaboration with the ACQUIRE Project. However, both sides talked about how they ―really struggled to find a way to work together‖ and that it was ―a little limiting‖ since the MOU was between the CRTU and the LMS Project, rather than MSH as a whole. FHI staff point out that ―our challenge was lack of complementarity, but the MOU helped overcome the challenge. We really had to search for ways to collaborate.‖ Overall, it is not worth pursuing this approach, and this has clearly been noted by FHI as the new project, PROGRESS, does not include this. As FHI staff point out, ―work with those who are best placed to get things done vs. pre-set partners,‖ and select partners at the country level more than global MOU partners. Enhanced Country Program/Focus Countries A staff member from FHI called the Enhanced Country Program (ECP) the ―backbone of research utilization efforts.‖ The idea of having focus countries comes from experience that showed the importance of having a country presence in order to effectively identify local needs and facilitate local change. As staff from a partner organization explained, ―with in-country presence you get better research and better utilization.‖ The lack of in￾country presence beyond Kenya had been noted as a gap in the 2003 CTR evaluation.1 There were four main objectives of the ECP, with a goal of having impact at the country level: 1. Establish in-country partnerships 2. Identify and prioritize local RH research and program needs that correspond to the global CRTU research agenda 3. Promote the use of research results at the country level 4. Support scale-up of improved programs. Focus countries include four tier 1 countries (Kenya, Madagascar, South Africa, and Uganda) and three tier 2 countries (India, Nigeria, and Tanzania), with more intensive efforts and resources going into first-tier countries. In terms of choosing the countries, FHI staff pointed out that so much boils down to political will and getting buy-in from the Mission, so these two factors were key in selection of focus countries. In the three focus countries that the team visited, the FHI office had a strong relationship with the MOH. Kenya was chosen as a focus country due to the preexisting staff and relationships in the country. As Table 2 shows, Kenya is an outlier compared to other focus countries, in terms of using significantly more resources, and so is not likely to be a model to replicate in other countries. Kenya is also a regional office, providing support to other countries in Africa, including Nigeria, Rwanda, Tanzania, and Uganda. It is also important to note that a significant amount of ECP resources goes to ―ECP implementation,‖ which covers North Carolina staff time and travel. FHI should move toward further decentralization for greater autonomy of their field offices and more resources at the field level compared to headquarters, with the understanding that headquarters staff will continue to play a key role in technical assistance. 1 Claudia Morrissey Conlon, Gordon Duncan, Noel McIntosh, and Julie Solo, Assessment of the Contraceptive Technology Research Project at Family Health International 1995–2005, POPTECH, 2003. CRTU RESEARCH UTILIZATION ASSESSMENT REPORT: RESEARCH WITH A PURPOSE 13 TABLE 2: EXPENDITURES FOR ENHANCED COUNTRY PROGRAMS* Country 2005 2006 2007 2008 Total Kenya — 853,607.71 979,861.68 211,203** 1,767,011.46 Uganda — 49,106.79 147,299.24 198,222.44 394,628.47 Madagascar — — 59,614.73 186,652.94 246,267.67 South Africa — 72,446.39 — — 72,446.39 ECP Implementation 230,122.58 541,107.03 457,228.83 468,483.13 1,696,941.57 * Figures provided to assessment team by FHI staff. ** This figure is for the reporting year rather than the fiscal year. The dramatic decrease in funds under the Kenya focus country designation came from a USAID request to not charge Kenya infrastructure costs to the focus country program, since these costs were general management costs that should be spread across projects. It is useful to compare the experiences in the different focus countries to inform future efforts. Kenya. FHI opened its office in Kenya in 1992, but their work in the country began in the 1980s. CRTU built on existing CTR staff, structures, reputation, and relationships. Currently, CRTU has 12 staff in the Kenya office, by far its largest field presence. CRTU research utilization priorities were based on knowledge of the country realities, input and consensus from partners, and existing research results and experience. The priorities are to: (1) develop and scale up FP integration into HIV programs; (2) support programs with improved FP service access and quality, institutionalize FP checklists and other job aids and training materials, revitalize LAPM, and lower the provider level requirement to allow midlevel providers to insert IUDs and implants and community health workers to provide DMPA; and (3) address youth RH and HIV needs. Overarching these priorities are efforts to build capacity within the MOH and other partner programs. Since its inception, CRTU has leveraged $6.3 million in field support. Uganda. Technical assistance from the Kenya office was essential in launching activities for CRTU in 2005. Currently, FHI serves as the secretariat for the Family Planning Research Working Group (FPRWG), which was formed in 2006 and is led by the MOH. CRTU worked with MOH, the United Nations Population Fund (UNFPA), and others to update the MOH FP curriculum, which now includes WHO simplified instructions on missed pills, guidance on advance provision of pills, FP provider checklists, provision of multiple pill packs, and quick-start of pills. The CBD of DMPA study conducted in 2004 has had significant impact throughout Africa. While it has been expanded in Uganda, the policy has not yet changed. Overall, CRTU has played an important advocacy role in promoting collaboration among partners through the FPRWG and through provision of technical assistance. The country director has also made vigorous efforts to share and promote use of FHI’s results and tools in the areas of youth and FP/HIV. This is greatly appreciated by partners. As of March 2008, CRTU had leveraged $500,000 from the Mission. Experience from Uganda highlights the importance of having an adequate number of staff to respond to the requests for technical assistance that result from the excellent reputation of FHI and the in-country presence. Currently, it is challenging for the three-full time staff to meet all of the requests, so FHI is planning to engage additional staff. Madagascar. CRTU’s work in Madagascar began through two primary efforts: introducing CBD of DMPA and promoting the Best Practices Package. Its office opened in February 2007, though work began in 2006 with one staff member based at the Population Services International (PSI) office. CRTU has worked closely with the MOH, ―linking everything with the country’s vision‖ on efforts that ―matched the needs.‖ The Minister of Health called the CBD of DMPA work a ―groundbreaking experience for us…it has been a grand success and has really helped increase contraceptive prevalence.‖ The Best Practices 14 CRTU RESEARCH UTILIZATION ASSESSMENT REPORT: RESEARCH WITH A PURPOSE Package has been less successful; in part this could be due to the different processes of introduction—CBD of DMPA was introduced through training and follow-up while the tools of the package (pregnancy checklist and systematic screening) were shared in a workshop. Madagascar is not a PEPFAR country, and not a country where FHI had a presence and reputation through HIV/AIDS work (HIV prevalence is currently estimated at 0.1 percent, so low that the upcoming Demographic and Health Survey (DHS) will not include HIV testing). CRTU has received over $1 million in funding from the USAID Mission, an important sign of how highly their work is valued. South Africa. In contrast to Madagascar, where HIV/FP integration is not a high priority, this is the sole focus of CRTU’s efforts in South Africa, as mandated by USAID. FHI opened its office in South Africa in October 2003 and currently has six technical staff working on the CRTU program. The South Africa office has leveraged $4 million in PEPFAR funds. Research utilization efforts have led to important changes in policies and training curricula to ensure that FP is included in prevention of mother-to-child transmission (PMTCT) programs. One example of this is that the National PMTCT policy was changed in December 2007 to include FP needs of HIV-positive individuals. Rwanda. While not a focus country, the assessment team’s trip to Rwanda provided important insights into the research utilization process. Rwanda is a country with great potential for research utilization; as a leader in HIV/AIDS explained, ―we are a country that wants to make evidence-based decisions. We are results￾based, so we need to be evidence-based.‖ FHI had developed a strong presence and reputation in Rwanda through its HIV/AIDS work. CRTU was able to build on this, through involving partners in Rwanda in an educational tour to Uganda to see the CBD of DMPA project and through a study tour to Kenya to see the FP/VCT integration work. An important aspect of the work in Rwanda was the consciousness-raising of FHI staff, showing a potential model for other non-focus countries where FHI has a presence (for example, Nigeria, where FHI has roughly 350 staff). One benefit to having a local presence through the focus country approach is that it allows for serendipity. For example, in Kenya, Madagascar, and Uganda, CRTU staff found resources to be responsive to requests that fell into their lap because of the reputation of FHI. In some cases, the USAID Mission has facilitated this by directing local partners to FHI as a resource for technical assistance. This reputation and presence of FHI through its HIV/AIDS work remains a largely untapped potential route for research utilization. But work in Rwanda shows exciting possibilities for how to spread research utilization effectively beyond a small number of focus countries. Given FHI’s presence in 40 countries through HIV/AIDS programs, this could be an important route of expanding impact. As noted, one way to do this is to look for facilitating factors to take advantage of possibilities in non-focus countries. For example, Rwanda is similar to Madagascar in that there is strong ownership and focus on FP as being key to the development of the country. Practice to Research ―We need to start from the reality because the goal of research is to improve the situation…the key is to identify the right topic for research,‖ explained MOH staff in Madagascar, an idea repeated by many respondents in many different settings. As FHI headquarters staff explain, research utilization is not a simple, linear process. Staff in Madagascar echo this idea, and expand upon it, highlighting the importance of obtaining input from partners, and how practice feeds into research just as research feeds into programs and policies, seeing the process as a circle rather than one or the other being the starting point of a linear process; ―it’s like asking what comes first, the chicken or the egg.‖ FHI recognizes that there is a need to improve their efforts to include ideas generated at the country level into the overall research agenda; ―when viewed in comparison to concepts developed by researchers in North Carolina, country-level concepts are often at a disadvantage,‖ according to their self-assessment. FHI staff suggested that they could improve the stakeholder process, provide more technical assistance (TA) in writing CRTU RESEARCH UTILIZATION ASSESSMENT REPORT: RESEARCH WITH A PURPOSE 15 proposals, and set aside an allocation so field-generated proposals are not always competing with the other projects. CRTU experience highlights the ongoing challenges in balancing global and national needs. There is a need to improve communication and collaborative efforts to identify research priorities in constructive and balanced ways. A staff member from USAID explained that USAID invites CAs and CRTU to informal quarterly meetings where they all look at ideas and CRTU does listen. However, it would be better if such processes were formalized so they could look at ideas more systematically and choose priorities together. The new PROGRESS project has this written into it. In their self-assessment, FHI writes, ―those closest to service delivery often have inadequate time, resources, or skills to fully contribute to identifying and prioritizing research priorities. They may know what their problems are but may be unable to translate those problems into testable hypotheses.‖ That should not be a problem, as it should be the responsibility of the research group to provide assistance to translate service delivery problems into research hypotheses. CRTU tried to solicit ideas and proposals through stakeholder meetings in Kenya and Uganda, but this process met with problems. Participants generated ideas and concept papers but they did not get funded; this can be damaging to perceptions and partnership. In Uganda, for example, a meeting was held and partners developed concept proposals—however, very few proposals were approved and approvals took at least one year, leading to frustration on the part of the CAs and organizations involved. It is important to clarify and be transparent about the process by which assessments on a country or global level will be used to identify country or global research priorities. It is also essential to clearly communicate the CRTU’s mandate, as many of the concept papers involved projects that were outside of this focus of the CRTU. Information Programs Information Programs is a formal part of the research team. Key activities by Information Programs that support research utilization include publishing research results, synthesizing and contextualizing information, and conducting tailored dissemination. They track the way materials are used by others and track media coverage. A strong partnership with INFO Project helps get results out. Some illustrative examples include: (1) the FP Effectiveness Chart is included on the back of the FP Handbook, (2) FHI took the lead to coordinate efforts with different CAs to develop the IUD toolkit, and (3) FHI suggested many of the 29 indicators adopted by the HIPNET group to measure success of information products and services. A meeting with a somewhat different format from most dissemination meetings was held in Madagascar, and seems to be an effective approach to possibly replicate in other settings. Held on June 12–13, 2008, the ―Best Practices in Reproductive Health‖ meeting brought together the country’s 22 Regional Directors of Health, along with a range of technical partners, and aimed to disseminate information on best practices and obtain commitments to use these research results. A number of best practices were presented, including SDM, presented by IRH, and fistula care presented by UNFPA. Participants felt that this meeting was unique in that ―we made commitments, not recommendations.‖ A follow-up meeting was planned for October 2008, and FHI should assess the effectiveness of this approach and whether it should be replicated in other countries. A number of partners expressed the need for FHI to continue to improve on the language they use to communicate; ―they need to present the data so we can understand it,‖ as one respondent explained. Staff from a service delivery partner mention the need to put research facts into context and present the results in a way that they can be utilized. They explain that there has already been a shift in FHI with more attention to how the data are packaged, but that there is still room for improvement. FHI has done extensive dissemination of tools, and it is clear that many people know about them. But it is more difficult to know how they are used. For example, CSTS+ praised FHI tools—―everything FHI comes out with we send to our partners. They do a very good job of testing these tools, so you know what you’re sending is of good quality.‖ However, they admit that ―it’s hard to know how much it actually gets used in a program.‖ It is generally not sufficient to just distribute the tools—one thing that helps is ―putting a name to these tools‖ so that partners feel more supported and have someone they can actually talk to. Another issue with the tools is their 16 CRTU RESEARCH UTILIZATION ASSESSMENT REPORT: RESEARCH WITH A PURPOSE cost—the laminated versions are fairly expensive and it would be useful to explore ways to reduce the costs of reproducing these tools so that costs can be taken on by local partners, including Ministries of Health. By distributing laminated versions, FHI can sometimes unwittingly create dependence. FHI does currently make these tools available as PDF files, but there might be a need to provide some technical assistance as well to help partners develop more sustainable means of maintaining supplies of the tools. CRTU has also been able to use the combination of its research and communication expertise to respond rapidly to controversies. For example, in March 2008, a controversy and potential crisis arose in Zambia when there were claims that a shipment of DMPA had been contaminated with HIV after a lab technician got a positive test result. This was reported in the newspaper in Zambia, and the government withdrew DMPA from health facilities. FHI was informed about this issue, and the Product Quality and Compliance (PQC) Division prepared an investigation plan, conducting tests to explain this positive result (which meant the DMPA was NOT contaminated), and compiled a report that led to the Zambian MOH rescinding their ban on DMPA. This highlights the importance of being viewed as a high-quality and unbiased source of information, an essential aspect for research utilization, as well as showing an example of quickly putting findings into use. Unfortunately, there has been little success to date in developing a safe and effective microbicide. However, a good deal of useful information has come from the microbicide trials conducted by FHI. In particular, their management of communication issues and their communications materials have been used by many partners in microbicide efforts. In addition to these materials, many organizations have learned from and adopted the behavioral work and community engagement that has accompanied the clinical trials managed by CRTU. Other Specific Utilization Efforts Study tours have played an important role in research utilization under CRTU, particularly for CBD of DMPA and FP/HIV integration. The study tours for CBD of DMPA included assistance with developing a country plan, and FHI staff stayed in touch—―the extra effort to make sure it didn’t waft away.‖ Also, the tours included an FHI staff member from the country office to help it move along. It could be useful to explore the possibility of study tours that are focused on changing policy, tours that could take place both between countries and within countries. For example, in Kenya, FHI and the MOH are considering taking key decisionmakers to see the CBD of DMPA project to help facilitate policy change. It is interesting to see the continuing impact of South-to-South exchanges; for example, while initially Uganda was the site of study tours, now the Madagascar program has also become a destination for learning, with a team from Malawi visiting in June 2008. It seems that study tours can be particularly effective for projects involving allowing lower-level cadres to provide certain health services. For example, this was done effectively by Ipas and partners in 1997, with a study tour to Ghana to see midwives offering postabortion care services, leading to several countries adopting this practice. 2 These are the types of projects where there is some resistance and controversy, so it is helpful to actually see directly that it works and is safe. These tours, of course, have mixed success. For example, an educational tour to Uganda in February 2008 for participants from Rwanda, Nigeria, and Tanzania led to the Rwanda team pushing for a pilot project in their country. In Nigeria, however, the team was told that there were mixed reactions to the idea of CBD of DMPA, although there has been encouraging movement forward in recent months. Another CRTU strategy has involved working with, assisting, and strengthening local champions. One such effort was developing a Network of Champions where FHI identified individuals in select countries to act as ―change agents,‖ helping to introduce or facilitate evidence-based change in programs and policies. These champions have mostly focused on FP/HIV integration. Costing roughly $100,000 per year in FY 2007 and FY 2008, these champions have had several achievements in terms of training and awareness-raising, but it is unclear whether they have had significant impact on policies or programs. CRTU is also trying a new method that uses the champion philosophy—seconding staff. In August 2008, CRTU seconded a staff member to the 2 D. L. Billings, Training Midwives to Improve Postabortion Care. Summary Report of a Study Tour to Ghana, October 12–19, 1997, 1998. CRTU RESEARCH UTILIZATION ASSESSMENT REPORT: RESEARCH WITH A PURPOSE 17 MOH in Rwanda to coordinate activities in FP/HIV integration. There is also interest in Uganda of adopting this approach. C. ACHIEVEMENTS OF UTILIZATION Question: In terms of meeting the goals and objectives of the cooperative agreement and of the Office of PRH, what is the evidence of impact of FHI’s research utilization efforts under the CRTU project? The CRTU is tasked with addressing three Intermediate Results (IRs), with IR3 being the most relevant part of the agreement in terms of this discussion of research utilization: IR1—Improved and new contraceptive and reproductive health technologies developed, evaluated, and approved IR2—Microbicides and microbicides/spermicides developed, evaluated, and approved IR3—Use of contraceptives, microbicides, and reproductive health technologies optimized and expanded. CRTU has developed a set of 14 research utilization indicators that are used to measure reach, capacity building, collaboration, and use of research. In addition, FHI developed an research utilization database to track and sort these indicators. However, there is still a need to better capture achievements; ―the information is all over the place…and there is an overload of information. If someone asks what they are doing, it is hard to say because it is so big,‖ explains a respondent from USAID. There is no attempt to link use with impact, which makes it more difficult to consider the relative importance of promoting different findings. While CRTU is not responsible for meeting targets of increasing CPR or similar impact measures, keeping an eye on these ultimate outcomes can help guide choices for the research agenda and utilization activities. 18 CRTU RESEARCH UTILIZATION ASSESSMENT REPORT: RESEARCH WITH A PURPOSE TABLE 3: CRTU INDICATORS AND EXAMPLES FOR RESEARCH UTILIZATION Indicator Example REACH 1. Examples of CTR/CRTU-related products being adapted or translated by recipient The Tanzanian Food and Drug Administration translated the pregnancy and COC checklists into Swahili and incorporated them into their 2006 training manual for drug shop managers. 2. Examples of reprinting, significant redistribution, or reuse of CTR/CRTU￾related product The Population Council used the pregnancy checklist in its Balanced Counseling Toolkit. 3. Number of people who attended FHI trainings or research dissemination meetings, type and duration of training or meeting Approximately 25 people attended a two-hour satellite session on HIV and contraception, which FHI hosted at the 2006 International AIDS Conference in Toronto. CAPACITY BUILDING 4. Examples of capacity building FHI assisted the Division of Reproductive Health (DRH) of the Kenyan MOH to develop a website, and the DRH has since assisted another MOH-associated agency to develop a web site. 5. Examples of secondary trainings or research dissemination meetings as a result of the original The Uganda Private Midwives Association has been using the provider checklists during training. 6. Examples and amounts of funds leveraged as a result of CTR/CRTU￾related efforts The American Urological Association has invited an FHI staff member to be part of the seven-person team developing clinical guidelines for vasectomy. 7. Examples and scope of South-to-South collaboration Jhpiego learned from the Kenya IUD revitalization efforts for a similar project in Malawi. COLLABORATION 8. Examples of partners having a substantial role in the identification, design, implementation, and/or follow￾on of a study or activity FHI is working with PSI and International Planned Parenthood Federation/Western Hemisphere (IPPF/WHR) on a training guide to the ―HIV Counseling and Testing for Youth: A Manual for Providers.‖ USE 9. Examples of technical assistance, research results, or information products being used in policy in guidelines, and scale and impact of that effort where known The Senegalese RH guidelines include adaptations of FHI’s checklists for the initiation of COCs, DMPA, and the IUD, as well as the pregnancy checklist. CRTU RESEARCH UTILIZATION ASSESSMENT REPORT: RESEARCH WITH A PURPOSE 19 10. Examples of technical assistance, research results, or information products being used in program planning and management, and scale and impact of that effort where known FHI contributed to global priority setting and program planning surrounding the female condom. 11. Examples of technical assistance, research results, or information products being used in service delivery, and scale and impact of that effort where known A monitoring activity in the Dominican Republic found that 56 percent of providers at 34 health facilities had used FHI job aids that had been distributed at a series of workshops. 12. Examples of technical assistance, research results, or information products being used in training, and scale and impact of that effort where known The ACQUIRE Project’s ―No-Scalpel Vasectomy Curriculum‖ incorporated results of CRTU research. 13. Examples of technical assistance, research results, or information products being used in research, and scale and impact of that effort where known Researchers at University of California at Berkeley (UC/Berkeley) requested questionnaires used in the CBD of DMPA study in Uganda to use for a similar study in Ethiopia. 14. Continued use of technical assistance, research results, information products or program model post-intervention, and scale and impact of that effort where known FHI assisted the Ugandan MOH in establishing a national Repositioning FP Working Group, which has met quarterly since 2005, and is being conducted with minimal FHI support. FHI acknowledges in its self-assessment that reporting of results is sometimes opportunistic rather than a methodical, systematic process. While there is a need to make such reporting more institutionalized, there is much that could be done with the existing data. It seems that there is great potential to tap into this database to come up with more condensed syntheses that better communicate how research has had an impact on policies and programs. This database could also be used more to explore questions about research utilization. While it would also be useful and important to conduct research on utilization efforts, this existing database provides potential data that are ready to be analyzed. FHI’s self-assessment recognizes that ―Research utilization outcomes related to the scale-up and replication of research results have proven the most challenging. Reasons for this include (1) the need for results to provide a new model or approach that is clearly better than what exists, so that it is deemed worthy of replication or scale￾up; (2) the necessity of handing off the implementation of scale-up to other agencies; and (3) issues related to timing, funding, and logistics.‖ While these outcomes are challenging to achieve, they are also perhaps the most important. Another way that results get used is through being incorporated into other proposals developed by FHI, either to USAID or other donors. When asked about this, FHI provided a list of 57 proposals submitted from January 2007 through August 2008 that had some link to CTR or CRTU results, or which built on the capacities built under the CTR and CRTU. This includes proposals to the National Institutes of Health, the Wellcome Trust, the Bill and Melinda Gates Foundation, the Centers for Disease Control and Prevention, the Tides Foundation, the World Health Organization, and others. It is important to think about what the world would look like without research utilization. For example, with CBD of DMPA, ―in the absence of research utilization, you wouldn’t have the same scenario,‖ assert FHI staff, and this seems a valid point. Without the efforts described above, the study tours, the follow-up, and the sharing 20 CRTU RESEARCH UTILIZATION ASSESSMENT REPORT: RESEARCH WITH A PURPOSE of information and lessons, it is likely that this practice would not have been picked up by as many countries and programs as it has. Impact of research results on policies ―WHO looks to FHI for guidance,‖ USAID staff explained. FHI has had extensive involvement with providing input into WHO’s Four Cornerstones for Evidence-Based Guidance for Family Planning: (1) The Medical Eligibility Criteria (MEC) of the WHO; (2) Selected Practice Recommendations for Contraceptive Use, WHO; (3) Decision-Making Tool for Family Planning Clients and Providers; and (4) Family Planning: A Global Handbook for Providers. This has been done through bringing questions to WHO, conducting systematic reviews, conducting original research, and participating as a member of the WHO secretariat and guidelines working group. Feeding evidence into these international guidelines is an extremely important step and should not be undervalued, because it is still a step away from direct programmatic impact. FHI is able to have this influence due to its reputation for high-quality research, pointing to the importance of maintaining this primary strength and focus. Table 4 outlines some key results from FHI’s own research and/or FHI’s research synthesis and technical input that have been incorporated into WHO guidelines. According to a 2006 article, at least 50 national program guidelines have drawn on recommendations from the MEC and Selected Practice Recommendations. 3 Taking the extended DMPA grace period as an example highlights the potential impact of this work. As the April–June 2008 FHI News Briefs explains, ―The extended grace period—if widely adopted—could prevent more unwanted pregnancies and help boost continuation rates for DMPA. Current one-year continuation rates are typically reported to be no higher than 60 to 70 percent.‖ TABLE 4: EXAMPLES OF RESEARCH RESULTS INCORPORATED INTO WHO GUIDELINES, BY METHOD Method Research Results Oral contraceptives Quick-start of OCs anytime during menstrual cycle Advance provision of multiple pill packs New instructions on missed pills IUD Safe for HIV-positive women Safe for young/nulliparous women DMPA Reinjection window can be extended from two to four weeks Implants Prequalification of Sino-implant Vasectomy Improving counseling on effectiveness (wait three months rather than 20 ejaculations) Use no-scalpel technique and occlusion of the vas using fascial interposition All Methods Contraceptive effectiveness chart 3 Herbert Peterson, and Kathryn Curtis, editorial, “The World Health Organization’s Global Guidance for Family Planning: An Achievement to Celebrate,” Contraception, 2006. CRTU RESEARCH UTILIZATION ASSESSMENT REPORT: RESEARCH WITH A PURPOSE 21 FHI staff recognize that this alone does not change services—that there is a need to ―transfer knowledge and skills from books to actual practice.‖ This follows a number of steps, beginning with revising country guidelines, implementing guidelines through developing reference and training materials for providers, and improving provider practice through job aids and identifying best practices. In Madagascar, for example, FHI’s active role in ensuring that WHO guidelines inform the national norms and procedures is critical, and then following up with development of job aids to put these norms into practice further enhances the chances for global-level research to inform and influence services at the national level. Impact of research results on programs As you get further away from the actual research, it becomes more difficult to attribute programmatic changes to specific research results. However, it is possible to assert ―plausible attribution‖ for the impact of CRTU research on programs. Below are two examples to show CRTU research contributions to quantifiable programmatic achievements. Increased Access and Use of DMPA in Madagascar: In Madagascar, coverage has increased from 61 trained agents to 208 working in seven of the 22 regions of the country, covering 14 of 111 districts. In Moramanga, one of the two pilot districts, CPR has increased from 20.9 percent in 2006 to 24.7 percent in 2007 and had jumped to 35 percent in the second quarter of 2008. During this time, the number of CBD agents trained to provide DMPA has increased from 23 to 85, and currently 18 or 21 communes have trained agents. It is notable that of 9,567 users of DMPA, 16 percent obtained their method from a CBD agent. Without study tours, targeted advocacy, and strong partnerships it is unlikely that CBD of DMPA would have progressed as far as it has in Madagascar. Increased Access and Use of the IUD in Kenya: Data collected by the ACQUIRE Project in Kenya show the impact of IUD revitalization efforts, in which CRTU was a key partner (Figure 2). The project started in January 2005 and ended in January 2007. The increase is encouraging, but it is also important to note that there were stockouts of DMPA at the time of the dramatic increase in IUD uptake in the first quarter of 2007. More important, it is unclear how successfully this is being scaled up in the country. Figure 2: Quarterly Provision of IUD in 13 ACQUIRE-Supported Facilities in Kisii District, Kenya 0 20 40 60 80 100 120 140 160 Jan- Mar '04 Apr￾Jun '04 Jul￾Sep '04 Oct￾Dec '04 Jan- Mar '05 Apr￾Jun '05 Jul￾Sep '05 Oct￾Dec '05 Jan- Mar '06 Apr￾Jun '06 Jul￾Sep '06 Oct￾Dec '06 Jan- Mar '07 Apr￾Jun '07 Jul￾Sep '07 Oct￾Dec '07 Jan- Mar '08 IUCD It would be useful for CRTU to collect more data like the above to show concrete results and quantifiable impact. This would not necessarily require substantial effort, as many partners will already have these data. Clearly, this impact is not due to FHI’s efforts alone, but FHI can reasonably argue that their efforts contributed to these results, acknowledging that this impact happened through partnership with many other organizations, including MOH and NGO partners. In addition, such data showing potential impact can help prioritize findings that should receive greater focus in research utilization efforts. 22 CRTU RESEARCH UTILIZATION ASSESSMENT REPORT: RESEARCH WITH A PURPOSE D. FUTURE OF UTILIZATION Question: What is the added value, if any, of including a research utilization component in a primarily contraceptive and reproductive health research project? Is this strategy effective and worth the effort? General conclusions on this model The team heard generally favorable opinions about the research utilization efforts by CRTU, supporting the idea of this model of including research utilization in a research project. The team heard and saw many positive examples of programmatic changes due to specific efforts in research utilization—particularly the enhanced country programs, advocacy efforts such as study tours, and targeting WHO guidelines. As a respondent at USAID stated, ―the research process is a continuum from idea generation through getting it utilized.‖ Such a definition highlights how utilization is an essential component of research, not an optional add-on. However, this does not mean the model is complete, as many felt that service delivery groups need to be full participants in research utilization and also need to receive funding to put research findings into practice: ―It’s all right to leave it where it is if they can draw in enough partners to assist them in the utilization‖ (USAID). ―Utilization should go to those who are doing service delivery‖ (service delivery partner). ―Where do you put the money? First, you put it in research organizations, get them to put information in a form that others utilize, then you create a forum to share that and put responsibility to utilize information into service delivery contracts‖ (USAID). ―It should be both—we should give more support to the service delivery groups too and have more flexible funds for innovation‖ (USAID). Suggestions for the future For CRTU 1. Build on the “one FHI” potential. There should be more consciousness-raising of FHI staff so that country staff are aware of organizational resources. These staff, often with expertise in and focus on HIV/AIDS, should be particularly well placed to move forward the issue of FP/HIV integration. 2. Reorient the partnership approach from formal MOU arrangements to more strategic and complementary partnerships. There should also be efforts to expand beyond the focus on the public sector to work more with the private sector. 3. Improve efforts to acknowledge the contribution of partners and share attribution. This is a key factor in making partnerships more balanced and equal. 4. Explore additional ways in which key research findings from other organizations can be effectively incorporated into CRTU’s utilization efforts. 5. Build on the successes of the focus country approach, including more efforts in non-focus countries. For future projects (whether PROGRESS or any follow-on to CRTU), learn from the lessons in Uganda and Madagascar—much can be achieved with fairly minimal inputs. For non-focus countries, there should be a strategic approach involving the identification of facilitating factors (for example, strong government support for family planning, emphasis on community health, etc.) and using the on-the-ground presence of the 40 HIV-focused FHI field offices. 6. Focus on systematic advocacy, for example, getting findings into USAID bilaterals in collaboration with USAID staff, continued input into WHO guidance documents, and assisting in moving guidelines into practice, in general acting as a catalyst more than a service implementer. CRTU RESEARCH UTILIZATION ASSESSMENT REPORT: RESEARCH WITH A PURPOSE 23 7. Develop more effective ways of incorporating needs identified in the field into the global research agenda. This means listening better to country needs and more consideration of research utilization when choosing research studies to conduct. 8. Allow for more autonomy and data analysis at the country level, which will enhance local partnerships and could also reduce the central-level costs associated with focus countries. This has to be done with close attention to maintaining high-quality research standards. 9. Include more attention to building local research capacity to generate research results, to transform them into actionable findings, and to use them in programs. Capacity building was not an explicit component of CRTU, but it is for the new PROGRESS project. 10. Incorporate more consideration of behavior change theory/motivation. This is particularly important, for example, in promoting practices that require behavior change from health providers. 11. Regularly collect and analyze cost information to facilitate efforts in scaling up and a greater focus on sustainability. For USAID 1. Create better linkages between the Research Technology and Utilization (RTU) Division and other PRH Divisions, especially Service Delivery Improvement (SDI). ―[T]ear down the walls!‖ as one USAID staff member stated, possibly through a joint RTU/SDI RFA, or pairing up staff from each unit, or finding ways beyond open staff meetings to interact. 2. Develop processes and indicators for findings to be systematically considered and used in global projects and bilaterals. This starts with advocacy to develop guidelines that need to be followed for design of activities, both global and bilaterals, procedures that mandate utilization of the best information available. This will require better linkages between USAID/Washington and field Missions to enhance Mission involvement in research and commitment to scaling up successful pilot projects. 3. Allow for some flexibility in timing and funding. Have flexible funds for innovations for service delivery and for cutting-edge issues for research (for example, when SubQ Depo becomes available, to be able to move quickly to implement operations research studies). 4. Conduct a parallel assessment to better understand the extent and process of use of best practices in service delivery programs, both globally and in bilaterals. 5. Fund study tours for USAID staff, at both global and Mission levels. 6. Assess the feasibility of and appropriate process for promotion of a package of evidence-based best practices to services delivery organizations and field Missions. Current priority research needs How best to improve motivation and retention of CBD agents and sustainability of CBD programs How best to effectively provide integrated FP/HIV services in a range of settings Understanding and addressing the issue of discontinuation of use of various contraceptive methods Improve understanding of how to change provider attitudes, address motivation, and remove other barriers to improving quality of services Further research to understand impediments to wider use of LAPM How best to safely and effectively provide SubQ Depo Operations research around provision of Sino-implants, for example, training lower-level providers 24 CRTU RESEARCH UTILIZATION ASSESSMENT REPORT: RESEARCH WITH A PURPOSE CRTU RESEARCH UTILIZATION ASSESSMENT REPORT: RESEARCH WITH A PURPOSE 25 IV. CONCLUSIONS FHI has made substantial progress in advancing efforts in research utilization. Through these efforts, a number of areas have been identified where FHI and USAID can continue to strengthen the use of evidence to inform programs, thereby enhancing the ability of the Office of Population to meet its goals. Overall, the team recommends that research utilization continue to be a key part of CRTU and other research projects, but this needs to be complemented by efforts on the service delivery side as well, with a greater focus on the fact that all are working toward the same goals. Three key words sum up the recommendations for research utilization by FHI and others for the future: balanced, systematic, and strategic. 1. Balanced: For partnerships to work, there must be a balance in terms of resources, respect, responsibility, and research priorities: a. Resources must be made available for both research and service delivery organizations to work together b. Credit must be given to all partners c. Using evidence to inform policies and programs must be everyone’s mandate d. Research priorities must reflect needs identified by a range of partners, at both global and national levels. 2. Systematic: Research utilization should be an integral part of research efforts and service delivery planning and implementation. For this to happen there is a need to systematically: a. Assess the potential usefulness of every research study b. Work with host-country colleagues to increase demand for using evidence c. Incorporate evidence-based best practices into USAID bilaterals d. Bring together service delivery and research organizations. 3. Strategic: Research utilization efforts come with associated costs, of course, not only in terms of the actual costs of the efforts but also in terms of diverting resources from research. It is essential, therefore, to make strategic choices about where to focus research utilization efforts, in terms of: a. Choice of focus countries—focus on countries where FP has support and momentum (for example, Madagascar, Rwanda), or countries where advocacy is needed to move FP (for example, Francophone West Africa), or a combination? b. Choice of key findings—which will have the most potential impact? c. Choice of key partners—who is best placed in each setting to help scale up important findings? d. Choice of most effective research utilization efforts—for example, when should study tours be used, etc.? These principles are part of the CRTU’s research utilization approach, but they can be further strengthened to build on the momentum achieved over the past several years and continue to move knowledge into practice. 26 CRTU RESEARCH UTILIZATION ASSESSMENT REPORT: RESEARCH WITH A PURPOSE CRTU RESEARCH UTILIZATION ASSESSMENT REPORT: RESEARCH WITH A PURPOSE 27 APPENDIX 1: LIST OF GLOBAL INTERVIEWEES Name Organization 1. Mihira Karra USAID 2. John Townsend Population Council 3. Jim Foreit Population Council 4. Victoria Jennings IRH 5. Rebecca Lundgren IRH 6. Jeff Spieler USAID 7. Maureen Norton USAID 8. Elizabeth Schoenecker USAID 9. Mary Ann Abeyta-Behnke USAID 10. Karen Sherk MSH 11. Cary Perry MSH 12. Milka Dinev Extending Service Delivery (ESD) 13. Jeanette Kessleman ESD 14. Salwa Bitar ESD 15. Carla White ESD 16. Uchechi Obichere ESD 17. Sonja Pilusa FHI/South Africa 18. Scott Radloff USAID 19. Rushna Ravji USAID 20. Roy Jacobstein EngenderHealth 21. John Pile EngenderHealth 22. Mia Foreman CSTS+ Project 23. Peggy D’Adamo INFO Project 24. Ruwaida Salem INFO Project 25. Winnie Mwebesa Save the Children 26. Janet Schafer CONRAD 28 CRTU RESEARCH UTILIZATION ASSESSMENT REPORT: RESEARCH WITH A PURPOSE 27. Marianne Callahan CONRAD 28. Rebecca Callahan Formerly with USAID 29. Patricia Coffey PATH 30. Victoria Graham USAID 31. Jim Shelton USAID 32. Dana Vogel USAID 33. Mark Rilling USAID 34. Maggie Usher-Patel WHO 35. Parmindar (Darsi) Lotay Pharm Access Africa CRTU RESEARCH UTILIZATION ASSESSMENT REPORT: RESEARCH WITH A PURPOSE 29 APPENDIX 2: COUNTRY REPORTS CRTU RESEARCH UTILIZATION COUNTRY REPORT: KENYA USAID has conducted an assessment of research utilization activities under FHI’s CRTU Project. As part of this assessment, two members of the four-person assessment team visited Kenya from August 31 to September 10, 2008. This visit, along with additional country visits to Uganda and Madagascar, provided the team with an overview of CRTU’s Focus Country strategy achievements and challenges in different settings. Kenya activities included interviews/conversations with more than 50 individuals representing central-level and provincial MOH and cooperating agency (CA) representatives and other local and global stakeholders including technical and program officers and AIDS, Population, and Health Integrated Assistance Projects (APHIAs) managers. In facilities the team met with facility management and a variety of providers. The team visited two facilities in each of two USAID-funded APHIAs designed to support the MOH’s HIV/AIDS, malaria, tuberculosis (TB), reproductive health, and maternal and child health (MCH) services: The Rift Valley Province APHIA where FHI is lead partner with Jhpiego, World Vision, Catholic Relief Services (CRS), Social Impact, and a Kenyan NGO (NOPE) The Western Province APHIA led by PATH with Jhpiego, World Vision, and two Kenyan NGOs. CRTU in Kenya and the APHIA Program CRTU in Kenya initiated with a seamless transition from CTR in 2005. By fortunate coincidence, the APHIA program initiated a few months later. FHI was awarded bilaterals in two of the eight APHIAs, providing a built￾in linkage between service delivery and research utilization. Its partners include a local NGO, Jhpiego, CRS, Social Impact, and others. In addition, FHI serves as the secretariat for the Kenya MOH/FP Working Group and is a member of the USAID Partners’ Working Group, which provide fora for materials distribution, dissemination, and exchange of experiences among the lead CA partners and other key players. FHI’s office opened in Kenya in 1992, but their activities began in the 1980s. CRTU’s staff consists of 15 full-time or part￾time persons, including two regional staff, and benefits from the availability of staff based in North Carolina as well as staff from other projects. FHI has a very broad and increasing volume of activities in Kenya, of which the team witnessed only a small number. Kenya FP and HIV situation: 1998 and 2003 DHS and 2007 Kenya AIDS Indicator Survey (KAIS) Until the 1980s, experts viewed Kenya as a model country, with impressive changes in FP uptake, closing the gap toward a transition stage. However, between 1998 and 2003, the total fertility rate (TFR) and unmet need rose slightly (4.7 to 4.9, and 24 percent to 25 percent, respectively) and CPR stagnated at 32 percent. IUD use decreased from 33 percent to 5 percent, COC use went from 27 percent to 15 percent female sterilization remained almost unchanged at below 10 percent, and DMPA use increased from five percent to 45 percent. During the same period, HIV prevalence increased in 15-to-49-year-old women from 8.7 percent (2003 DHS) to 9.2 percent (2007 KAIS) and in men, from 4.6 percent to 5.8 percent; yet, four out of five HIV-positive persons do not know their status. Besides women, the most affected are young people of both sexes, and 25 percent of all new cases are vertically infected infants. In 2003, pregnant women’s prevalence was 7.3 percent, increasing to 9.6 percent in 2007. With antiretroviral (ARV) treatment, a growing number of perinatally infected girls and boys are surviving and reaching puberty with associated significant risks. CRTU’s priorities The above data justify CRTU’s choice of priorities for research results utilization: To develop and scale-up FP integration into HIV programs; 30 CRTU RESEARCH UTILIZATION ASSESSMENT REPORT: RESEARCH WITH A PURPOSE To support FP services by promoting best practices through the use of FP checklists, job aids, and training materials; revitalizing LAPM; lowering level of service providers allowed to insert IUDs and implants; and changing policy to allow the CBD of DMPA; and To address youth’s RH and HIV needs. Overarching these priorities are efforts to build capacity within the MOH and other partner programs. Findings Informants were almost unanimous in their appreciation of FHI’s research quality and availability for TA, and praised it for support of and participation in the USAID Partners’ Working Group and the MOH RH and HIV Working Groups. FHI acts as the secretariat for the RH and HIV groups. In addition, job aids and other materials developed by FHI and branded by the MOH are considered very useful by managers and providers. Most frequently mentioned were the pregnancy and method checklists and the chart summarizing WHO’s MEC. In addition, single toolkits have been developed through collaboration with service delivery CAs. FHI assists the MOH in distributing and training users of those materials and the WHO/USAID Family Planning Handbook for Providers and MOH Family Planning Guidelines for Providers. However, there were also critical comments about the initial steps with MOU partners operating in Kenya; expectations had been raised by requesting research utilization concept papers, which were very labor intensive but in the end received no financing. A few informants also complained of the time spent at multiple meetings, some of them on the same topic. Observations during visits to APHIA facilities In all facilities, the team was received very cordially by facility management and staff, was given a tour, and had the opportunity to discuss services. They were clearly supportive and aware of the value of FP-HIV integration. For confidentiality reasons the team did not observe services being rendered. Great differences between facilities were noted: two, one in each visited APHIA, were well maintained and busy and the providers were enthusiastic and motivated. One of these reported 10 or more IUD insertions per day and many more births attended—there were two new mothers and their infants per bed in the postpartum ward. One of these facilities had considerable community participation and volunteers, and modest financial support. In contrast, in the other two facilities, waiting rooms and wards were gloomy in appearance and practically empty, and staff appeared unmotivated and less than energetic. There were few births and five to seven IUD insertions per month. MOH/FHI checklists and toolkits, the MEC summary chart, and the WHO and MOH Family Planning books for providers were present in all FP and HIV service areas, and most providers were knowledgeable about the tools and how they should be used; they stated that the tools improved their efficiency and service quality as well as access to methods. The team did not notice any FHI method effectiveness chart or Population Council’s Balanced Counseling flipchart and checklist for HIV integration into FP services. Additional observation: Comprehensive postabortion care (PAC) services are not in place in any visited facility; only in one was there a dedicated space with literally nothing but an exam table in it. The team was informed that it is difficult to obtain MVA kits; they are on the equipment list but the MOH cannot afford to purchase them. Findings related to CRTU’s priorities: FHI priority 1: FP integration into HIV/AIDS programs. ―For a woman, the risks for HIV and unplanned pregnancy are the same‖ (VCT counselor in APHIA Western Province). CRTU RESEARCH UTILIZATION ASSESSMENT REPORT: RESEARCH WITH A PURPOSE 31 In Kenya as in most countries, MOH RH and HIV divisions are vertical and separate, have different funding streams ($138 million in PEPFAR funds for HIV compared to $13.2 million in population funds for RH) and use different service providers. HIV programs depend on nonclinical counselors and ARV nurse/clinicians with little FP/RH knowledge, and RH service providers had outdated knowledge and biases on contraception for HIV-positive people. Feasibility, acceptability of FP and VCT integration were demonstrated by CTR in 2002 and implementation has started. A rapid assessment study (2005) in VCT centers in three provinces showed major gaps still present: Provider inertia: they do not counsel men on FP; condoms are not promoted or provided. Providers lack skills to discuss sexuality and behavior change communication with men and youth. Providers do not have tools to help clients assess personal risky behaviors and act to reduce these. No increase in FP uptake was demonstrated. Process ―FHI led integration process; they were way ahead of us. You can see what they are doing on the ground‖ (USAID informant). FHI coordinates FP-VCT stakeholders’ subcommittee. VCT free-standing services are widely implemented and are particularly important because their clients are predominantly men and youth who do not seek services in clinical settings. FHI is assisting the MOH to include FP counseling and selected method provision in them. This intervention has the potential to be very effective but, due to the fact that sites do not identify clients (except by number), it is impossible to track them and evaluate behavior change. FHI is lobbying the MOH to allow trained VCT nonmedical counselors to provide DMPA. CRTU and stakeholders developed a toolkit on contraception for HIV-positive women and couples that is informing the development of a national orientation package for training ART services providers. CRTU and partners are undertaking assessments to inform the strengthening of FP into PMTCT, and addition of FP into ART/CCC (HIV/AIDS Comprehensive Care Centers) and HIV home-based care (HBC) services, through the APHIAs and other service delivery organizations. FHI lobbied to have FP added to basic HBC provider training but only six hours were added to a two-week curriculum. Most visited VCT, CCC/ARV, and PMTCT sites were provided with MOH/CRTU checklists, the MEC chart and MOH FP Guidelines, and the WHO FP book for providers. COCs and condoms (and DMPA in fewer sites) were observed in most HIV service delivery sites. Providers stated the pregnancy and method checklists and the MEC chart were particularly useful for their FP work. Penis models (four of them) were only seen in one site. Though ideal, integrated counseling in CCC/ARV sites may not be feasible due to heavy client load and provider shortages. The CRTU is testing different models of integrating FP into CCC to identify the most effective model. Additional observation: The University Research Corporation (URC) 2007 flipchart observed in PMTCT room does not mention FP as component of pre-pregnancy, antenatal care, or postpartum counseling. Only Nevirapine is discussed as PMTCT. 32 CRTU RESEARCH UTILIZATION ASSESSMENT REPORT: RESEARCH WITH A PURPOSE Challenges: Both MOH divisions were very interested from day one, but vertical programs hinder and retard integration. Though informed of new WHO MEC, providers do not trust that HIV-positive women and couples can safely use any FP method, and not all providers are convinced of integration benefits. At lower-level facilities, one provider is responsible for both RH and HIV services but she or he may be too busy to provide necessary counseling. Competitive attitude between CA partners sometimes prevents ideal collaboration. Provider turnover, resistance, and negative attitudes (for example, that HIV-positive people have no right to be sexually active) require much special training and retraining. Monitoring and evaluation (M&E) for FP uptake is difficult in VCT centers where clients are not identified by name. Funding for job aids is a challenge: existing materials are too expensive, and user agencies depend on CRTU for constant resupply. APHIA supports training, and as it identifies materials it would like to use, it covers reproduction costs. However, the districts not covered by APHIA have no available funds for this purpose. According to more than one informant, the process of research utilization is too long and difficult to coordinate with what service delivery agencies do, and ―materials, in some cases, are ready before research results are available‖ (NASCOP informant). According to FHI, the time-consuming process is a product of the length of time before research results are available and the time it takes to get government buy-in and ownership, which can facilitate sustainability. Achievements: FHI is perceived by key stakeholders as ―the go-to agency for TA on integration.‖ Acceptance at Central MOH; strong support by HIV (NASCOP) and RH Divisions. ―Before starting to work with FHI, with FP, I was on square one‖ (high-level MOH/NASCOP informant). NASCOP/FHI integration research done collaboratively. ―Ministry decides how to utilize results in programs. There is reason to capture best practices‖ (NASCOP). Materials including toolkits adapted, translated into Swahili and branded with MOH. VCT counselors inform clients on FP, and provide condoms and COCs; on-site nurses provide VCT; counselors inform clients on FP, provide condoms and COCs; and on-site nurses provide DMPA and clients referred for other services. Checklists are present in counseling and testing sites and providers are knowledgeable of their use. Future needs identified by interviewees and assessment team: Despite CRTU’s achievements, integrated services are a relatively new modality; time, persistence, and resources are needed to make them available countrywide and sustainable. To address high provider turnover, new training strategies need to be developed and training capacity transferred to service delivery organizations. For true integration, six-hour training on FP embedded in a two-week curriculum for HBC HIV service providers is inadequate. On-the-job training needs to be developed to address shortfall. Provider training on sexuality and addressing men and youth are needed, stressing proper condom use demonstration and promotion for dual protection. Complete MOH materials ownership. ―All the names are FHI but NASCOP not among them.‖ Advocate to leverage PEPFAR and other donor resources for integration efforts. Reduce cost of materials and job aids and transfer duplication to user organizations. CRTU RESEARCH UTILIZATION ASSESSMENT REPORT: RESEARCH WITH A PURPOSE 33 HBC clients with no side effects should be able to receive method resupply from CBDs. Simplify integration process and develop simpler tools for community providers. With partners, assist MOH to advocate policy changes on who can do what in service delivery, including CBD of DMPA. Pre-service training on integrated services is needed in all professional schools. The team recommends closer collaboration and data collection with Population Council and other research organizations conducting alternate forms of integration (for example, integrating HIV risk assessment and counseling with FP clients). Integration indicators should be developed and data collection strengthened to allow progress monitoring and to address shortcomings. Operations research needed: Men Test strategies, including illustrated talks with condom demonstrations and videos for group education in busy waiting rooms (for example, CCC/ARV services) Test strategies to track persons who attend VCT sites for behavior change, with informed consent from clients. FHI priority 2: Improve FP services: Revitalize LAPM and lower provider level for IUD and implant services and for DMPA to community health workers. To further this goal: develop, distribute, and institutionalize use of FP checklists, other job aids, and training materials. ―The wheel doesn’t have to be reinvented. We wish to see that people can choose the method they use. USAID is developing a strategy to help partners to share‖ (USAID). CTR work on IUDs gave CRTU a head start in terms of development of materials, establishment of strong relationships with the MOH, and positive relationships with the USAID mission and service delivery CAs. Most informants stated that FHI deserves much credit for driving the change process to address high unmet need, increase access and improve quality, and revitalize LAPM with the MOH and USAID partners. The APHIA structure provides an excellent built-in linkage of service delivery and research utilization; CRTU has focused primarily on tool development, training in tool use, and tool distribution as a primary strategy for this priority. Process/FHI contribution The MOH/DRH Family Planning Working Group established by CTR is very active today under the MOH. ―Best Practices‖ has entered the discourse within the MOH/DRH. A LAPM strategy was developed; Best Practices and IUD Task Forces merged to form the FP Working Group. Job aids and toolkits in English and Kiswahili were presented to the Family Planning Working Group (FPWG) and branded by the MOH. FHI distributed and trained service delivery partners in job aid use. Following a study tour to Uganda, the MOH has approved a CBD of Depo pilot starting in late 2008 in the Jhpiego-led APHIA. Challenges Changing WHO FP MEC and guidelines in service delivery is difficult and time consuming. 34 CRTU RESEARCH UTILIZATION ASSESSMENT REPORT: RESEARCH WITH A PURPOSE Continuous personnel changes at central level retard progress. Provider shortages and reassignments demand continuous, expensive, and unsustainable trainings. Achievements The MOH now ―owns‖ Family Planning Working Group, with continued support from FHI secretariat; FPWG prioritizes and promotes service delivery changes. Job aids were branded by MOH (but many still refer to them as ―FHI‖ materials); the MOH has assumed responsibility for bringing job aids stepwise from central level to service delivery facilities. Checklists and other materials are available in all APHIAs. Interviewed APHIA managers report receiving tools/materials. Materials have been distributed and providers are being trained to use them. Visited providers appear knowledgeable of their contents and use. With FHI strong input, LAPM toolkits produced independently by CAs were ―harmonized‖ into single toolkit, which now has replaced older versions in programs. Data provided by ACQUIRE and FHI collaboration on IUD revitalization initiated in 2005 shows sustained demand. RH curriculum introduced in pre-service training institutions. Future FHI efforts There are still a large number of efforts needed to improve FP service quality and access at all levels; FHI has very extensive but not limitless skills, capacity, or funding. It will have to make strategic choices of what to prioritize for the next 18 months. Needs expressed by many informants include: Assist the MOH in revision of the 2005 FP guideline. Efforts are still needed toward strengthening promotion of best practices. CRTU and Population Council should support the Research Working Group to be proactive about linking partners, and the MOH to select research priorities based on identified country needs. Service delivery agencies need capacity building on research. FHI and Population Council can lead this by focusing on management information system (MIS) improvements, data analysis, and ―translating‖ results to service delivery improvement. Data for decision-making culture is still not in place; this needs support from CRTU and partners. If the CBD of DEPO pilot is effective, FHI, the MOH, and partners must advocate for policy change. The team was informed that many FP clients choose the IUD because implants are not available. Once Sino-implant becomes available, CRTU should assess the extent of pent-up demand in all APHIAs. If IUD demand is significantly replaced by implant demand, FHI needs to make strategic decisions on where to concentrate LAPM efforts. There is interest in SDM, especially for Muslim and Catholic populations; for example, in the North Eastern Province APHIA, which has a large Muslim population, SDM has been introduced with strong support from CRTU. Still, in most cases, CycleBead procurement and cost are barriers to its utilization. There is a need for alternative commodity production and/or communication strategies (for example, paper-based calendar ―beads‖). Another alternative would be to encourage the mosques and churches to purchase the commodity for their parishioners. Client materials mirroring information provided in checklists as an opportunity to utilize client’s time in waiting rooms is currently being explored; a range of approaches should be considered. Research is still needed to address the following questions: What are essential conditions causing widely different provider attitudes, service quality: leadership, training, supervision, community involvement, or all of these? CRTU RESEARCH UTILIZATION ASSESSMENT REPORT: RESEARCH WITH A PURPOSE 35 What are alternative sustainable models of supportive supervision? What is the best way to work on stigma with providers, communities, and clients? What are the best approaches for educating clients in busy waiting rooms in antenatal, CCC/ART, and other service sites? Recurring issues needing efforts, by CRTU with appropriate bilateral partners: Demand creation once supply is in place. Training the community on DMPA and midlevel providers on implants and IUDs. Appropriate materials for each level will be increasingly important as task-shifting occurs. Logistics are seriously deficient with frequent stockouts. MSH is the partner to lead this effort because ―with no products there is no program.‖ FHI priority 3: Youth-centered programs. The team visited two new CRTU activities with youth agencies; they illustrate the value of having flexibility to take advantage of emerging opportunities: PCEA St. Andrews (Episcopalian) Church project educating and supporting housegirls on RH, behavior change, and income generation by more professional training. The University of Nairobi ―I choose Life‖ (ICL) or the Abstinence, Be faithful, Condoms (ABC) project directed to students. Both work with important populations, are expanding, and will benefit from assistance in costing, scaling-up and developing sustainability plans. The team also visited two youth centers in APHIA facilities. Process FHI is building consensus among stakeholders that a special youth focus needs implementation in all services to include HIV and FP information and education, sexuality, and behavior change counseling and all methods of FP without neglecting the condom. The two above-mentioned NGOs came to FHI’s attention by serendipitous coincidences. FHI has taken full advantage of this opportunity to provide TA on FP and HIV integrated services. ICL and FHI conducted a baseline survey to inform ―edutainment‖ activities; the survey highlighted high unmet need (64 percent of second-year students are sexually active and take risks). ICL, with University of Nairobi and PEPFAR funding and FHI TA has trained peers to offer comprehensive education, VCT, and care services for infected youth. Challenges Negative provider attitudes and stigma associated with serving youth. Youth Centers in Facilities lack educational materials. YouthNet materials are not well known. Provider training does not include sexuality and behavior change communication. Youth projects are difficult to evaluate because target populations are mobile/difficult to track. Achievements ICL and housegirls’ projects, with FHI support, are now considered models for other youth programs. They still need TA to inform decisions on scaling-up and to leverage funds for sustainability and replication with programs working with young men and women. 36 CRTU RESEARCH UTILIZATION ASSESSMENT REPORT: RESEARCH WITH A PURPOSE Future needs Evidence is needed on best ways to deliver youth services. CRTU and partners need to support implementation of specialized youth services and to develop training toolkits for providers, with modules on reducing stigma. YouthNet materials need to be disseminated for Youth Centers and other programs for in- and out-of-school youth. Develop proxy indicators for impact of youth services. SUMMARY Much has been achieved by CRTU to date; it has taken full advantage of its unblemished research reputation; its close collaborative work with the MOH, the APHIA system and its CAs; and its constant attention to the working groups it coordinates. It also has been able to act when new innovative opportunities appear. Much remains to be achieved in the next 18 months. CRTU should use this assessment to strategically identify where to invest its considerable capacity to further utilization of research results that fit into Kenya’s priority needs. LIST OF KEY INFORMANTS FHI Kenya Title/Responsibility 1. Maureen Kuyoh CRTU Director 2. Jennifer Liku FP-HIV integration 3. Dr. Marsden Solomon Regional Medical Advisor 4. Alice Olawo CBD of Depo 5. Constance Ambasa-S. and Willis Odek Youth 6. Monica Wangiru Research Utilization in Kenya 7. Violet Bukusi FP Checklists and other tools 8. Erika Martin Deputy Director, Africa Regional Office USAID Kenya-OPH 9. Lynn Adrian Director OPH 10. Dr. Sheila Macharia Program Manager 11. Jerusha Karuthiru Program Management Specialist MOH 12. Dr. Bartilol Kigen FP Program Manager, MOH/DRH 13. Anne Njeru Head Trainer, MOH/DRH 14. Dr. Robert Ayisi Deputy Director, NASCOP CRTU RESEARCH UTILIZATION ASSESSMENT REPORT: RESEARCH WITH A PURPOSE 37 LIST OF KEY INFORMANTS FHI Kenya Title/Responsibility 15. Margaret Gitau Program Manager, NASCOP 16. Rift Valley Provincial Medical Office (PMO) team 4 persons 17. Njoro and Gilgil MOH Facilities Facility management and staff 18. Western PMO team 5 persons 19. Kakamega Prov. Hosp and Mbale Health Center Facility management and clinical staff MOH Central 20. Dr. Juma Mwangi Director 21. Alice Wambugu Trainer/MCH-FP Service provider APHIA II Rift Valley (FHI) 22. Charity Maturi Representing Deputy Director 23. Dr. Japheth Kituu Senior Technical Advisor 24. Violet Ambundo Technical Officer 25. George Avosa Senior Technical Advisor 26. Janet Namwebya Regional Senior Technical Support 27. Dr. Frank Mwangemi Senior Technical Advisor: HIV prevention, care, and treatment APHIA II Western (PATH) 28. Dr. Ambrose Misore Project Director 29. June Omollo Senior Technical Advisor, Civil Society 30. Cornelius Kondo RH/FP and Malaria Technical Advisor Population Council 31. Ian Askew Director 32. Harriet Burungi Associate 33. Charlotte Warren Associate EngenderHealth 34. Feddis Mumba Country Director 35. Dr. Frederick Ndede Medial Associate 38 CRTU RESEARCH UTILIZATION ASSESSMENT REPORT: RESEARCH WITH A PURPOSE LIST OF KEY INFORMANTS FHI Kenya Title/Responsibility 36. Dr. Job Obwaka APHIA II/Nyanza Director Jhpiego 37. Dr. Kenneth Chebet APHIA II Eastern PATH/Nairobi 38. Rikka Trangsrud Country Director 39. Margaret Waithaka M&E Advisor MSH 40. Dr. Mary Wangai Deputy Regional Tech Advisor/Director PCEA St. Andrews Church 41. Rev Kanga 42. Michael Wainaina P.I. Housegirls Project I Chose Life Foundation/U Nairobi 43. Mike Mutumbi CEO 44. Pascal Wambua Program Coordinator FHOK (IPPF Affiliate) 45. Dr. Oteba Program Director WHO 46. Dr. Joyce Lavussa National Professional Officer RH KOGS 47. Njoroge Waithaka Secretary 48. Dr. F.X.O.Odawa Journal Editor Telephone Interviews 49. Dr. Jennifer Othigo MOH Coast-Prov RH Coordinator 50. APHIA Coast (FHI) North Eastern 2 persons, including David Adriance, APHIA NEP Director CRTU RESEARCH UTILIZATION ASSESSMENT REPORT: RESEARCH WITH A PURPOSE 39 CRTU RESEARCH UTILIZATION COUNTRY REPORT: MADAGASCAR BACKGROUND The fourth-largest island in the world, Madagascar is diverse in its land, culture, and people. With a population of 19.5 million and a growth rate of about 3.0 percent, the population of Madagascar is estimated to increase to close to 25 million by 2015 and 30 million by 2025 (UN Data Record). The total fertility rate is estimated at 5.4, and despite a relatively high unmet need for contraceptive methods (24 percent), the modern contraceptive prevalence rate is still only 18 percent. Madagascar’s efforts to address family planning are impressive. In 2003, the Ministry of Health officially changed its name to the Ministry of Health and Family Planning, and the government recently launched the Madagascar Action Plan aimed to improve family planning, health, and other development indicators by 2012. This USAID/Washington-initiated assessment focused on evaluating research utilization activities under Family Health International’s Contraceptive and Reproductive Health Technologies Research and Utilization (CRTU) Program. Two members of the four-person assessment team traveled to Madagascar during September 10–19, 2008, and conducted interviews with 31 individuals, including government officials, donor representatives, and community-based health workers. The findings address the three main objectives of the assessment: (1) the process of research utilization, (2) the impact of research utilization, and (3) the future of research utilization, including conclusions and suggestions for improvements. 1. THE PROCESS OF RESEARCH UTILIZATION Successful research utilization is a function of many factors. In addition to financial and human resources, strong partnerships, ownership, and active efforts to promote utilization are imperative to success. Financial and Human Resources The FHI office was opened in Antananarivo, Madagascar in February 2007. The office operates with four technical staff and is funded through CRTU global funding and field support from the USAID Mission. In FY 2007 and FY 2008, a total of $246,267.67 was spent for the Madagascar Enhanced Country Program through CRTU, and additional funds ($50,000) were used to support a dissemination meeting. FHI leveraged $800,000 from the USAID Mission in field support, highlighting the value of FHI’s work by USAID/Madagascar. Partnership: ―Here partners work together to achieve results‖ Given the small staff and limited funding, FHI Madagascar has relied heavily on partners to help promote successful research utilization. Though relatively small in numbers, Madagascar has a highly functioning group of partners that allow for strong in-country strategic partnership building. FHI has been instrumental in creating a coordination committee to facilitate regular communication among the partners, the MOH, and other donors. Respondents stressed the need for partners to support the objectives of the MOH in order to ensure success. This has been recognized by the MOH, and one official commented, “Here the partners work together to achieve results.” In addition, FHI’s inclusion of partners at the inception of projects has received positive feedback and has helped partners ―internalize findings.‖ For example, PSI was involved from the beginning of the CBD of DMPA pilot and has maintained a strong role in the development of the CBD of DMPA program since then. According to PSI staff, during the pilot study for CBD of DMPA, “they [FHI] really involved people during the study, during presentation of results, etc. PSI input was really considered.” Regional and district health officers were also involved in improving the training manuals and materials for the pilot in addition to supporting the implementation. As one official explained, “one thing that really pleased us— they involved us in improving the training manual, they didn‟t just inform us.” 40 CRTU RESEARCH UTILIZATION ASSESSMENT REPORT: RESEARCH WITH A PURPOSE FHI’s work has focused on the public sector. Efforts to partner with the private sector may increase utilization of tools and research findings. Finally, partner collaboration in Madagascar has been informal, without MOU structures. This has worked well given the small health community and open communication among partners. Ownership: ―You need to really work hand in hand with the country government… the government has to share the vision—this is key.‖ Government ownership of research results is imperative to successful utilization. In Madagascar, the government recently developed a Madagascar Action Plan (MAP 2007–2013) outlining eight commitments to rapid development. MAP Commitment #5 is for Health, Family Planning and the Fight against HIV/AIDS. Family Planning, which is seen as a key to poverty reduction, is one of the top priorities, and the MAP goal is to achieve 30 percent CPR by 2012. In addition to the MAP, the Government has taken ownership of other successful programs such as the Champion Communes, which make contracts with the country to meet certain development goals. FHI staff stressed the importance of “linking everything with the country‟s vision,” enabling support from the MOH and programs such as CBD of DMPA and others to move forward. Efforts to promote utilization: ―We made commitments, not recommendations‖ Action-oriented dissemination: Successful dissemination of research findings is instrumental to getting research results used. FHI has encouraged ―Action Oriented‖ dissemination to ensure results are not only disseminated, but actively taken up by partners, donors, and the MOH. For example, in June 2008, the ―Best Practices in Reproductive Health‖ meeting was organized by the MOH and National Leadership Institute of Madagascar (NLIM) in collaboration with USAID and FHI. This meeting was unique in that it used a ―Mini-University‖ approach, which enabled participants to present findings from the field and engage in real-time discussions with each other regarding results. Participants included the 22 Regional Directors of Health and a range of other technical partners and donors. The purpose of the meeting was to disseminate information on best practices and obtain commitments to use these research results in the field. “We made commitments, not recommendations,” as one respondent said. A range of practices were highlighted including CBD of DMPA by FHI, the Standard Days Method (SDM) by IRH, and fistula care by UNFPA. A follow-up meeting was planned for October 2008 to assess progress. Study tours: Study tours to foster south-to-south exchanges have worked well in promoting CBD of DMPA. After a successful pilot program largely inspired from the lessons learned in Uganda, the CBD of DMPA program has expanded in Madagascar, and today, Madagascar has become a destination for learning. A team from Malawi visited in June 2008 to learn from the country’s efforts at introducing and scaling-up CBD of DMPA. Models of scale-up: Rapid versus gradual scale-up can also impact research utilization. IRH is using the ExpandNet approach for scale-up of SDM. This nine-step model has enabled over 1,500 agents to be trained in SDM with plans for countrywide training by 2012. The model for expanding CBD of DMPA has been more gradual. Sixty-one CBD agents were trained in DMPA injection for the pilot study in 2006, and to date just over 200 agents have been trained in DMPA injection. Determining the appropriate method for scale-up depends on the intervention. For example, the repercussions for poor training in CBD of DMPA are great, and one adverse event can result in complete termination of the program. SDM, while less ―clinical,‖ requires significant training and retraining, and issues of quality and effectiveness of the method are still being researched. It is worthwhile noting the different process of introducing CBD of DMPA compared with the Best Practices Package. CBD of DMPA was introduced in a three-day training course with hands-on sessions outlining the 14- CRTU RESEARCH UTILIZATION ASSESSMENT REPORT: RESEARCH WITH A PURPOSE 41 step process for an agent to provide DMPA, and included close monitoring and supervision. The Best Practices Package was introduced through a workshop where tools were shared, a workplan developed, and partners encouraged to follow up and monitor use. Lack of clear direction for the Best Practices may explain why uptake has not been as successful as CBD of DMPA in Madagascar. Finally, decisions for scale-up depend on higher government authorities. Scale-up of SDM has been encouraged by the MOH despite some additional research needs raised by IRH. As one staff member put it, “when the MOH says „Scale-up!‟, you scale-up.” This has some implications in that support from other donors might be compromised. Though feasibility and acceptability of SDM has been reported, and has been recognized by WHO, UNFPA still does not consider the method as ―modern,‖ and thus has yet to supply commodities. In addition, there are continued reservations from the local Mission about supporting activities until clear evidence of CPR reduction has been demonstrated. IRH is continuing research efforts to demonstrate efficacy during the scale-up process. Advocacy: Perhaps as important as the information being transmitted is the source of the information and advocating for research utilization. Comments from respondents highlight the importance of research and evidence to overcome resistance, and research organizations are often the most credible source of information. For example, resistance at the MOH level to CBD of DMPA was overcome by presentations from FHI about the experience in Uganda. After further discussions and presentations of data, the MOH agreed to do the pilot study. This has important implications for continuing a model where research organizations are given support for utilization efforts. Practice to research: ―We are trying to collect concerns from the field‖ Just as research results need to be used in practice, it is important to use program experience to inform research questions. ―We need to start from the reality because the goal of research is to improve the situation…the key is to identify the right topic for research,‖ explained MOH staff, an idea repeated by many respondents. FHI has been instrumental in holding regular partner meetings where issues from the field are raised and new research ideas proposed. Most recently, participants were interested in learning about various issues around cost￾effectiveness, including not only the cost for each community health worker, but also understanding the cost per client. FHI/Madagascar has now approached FHI/NC for technical assistance in designing such a study. As one FHI staff person stressed, ―We can miss this kind of question if we don’t ask people.‖ 2. IMPACT OF RESEARCH UTILIZATION Research utilization is a process. As a result, measuring impact is more challenging than measuring impact of a particular intervention. Nevertheless, it is possible to assess relative impact of FHI research utilization efforts by identifying impacts on policies and programs. Impact of research results on policies In Madagascar, FHI has been instrumental in helping to develop the Government of Madagascar-mandated Norms and Procedures in Reproductive Health. For the first time, guidelines for reproductive health were collected and published into a single book. FHI also played a role in helping put these policies into usable practice for providers via job aids, flip books, posters, etc. Distribution of DMPA by community health workers has been included in the guidelines as a result of FHI’s work demonstrating the efficacy of community-based distribution of DMPA. Though use of the pregnancy checklist does not appear explicitly in the norms and guidelines, it is a common practice and has been included in the MOH FP training manual for providers. 42 CRTU RESEARCH UTILIZATION ASSESSMENT REPORT: RESEARCH WITH A PURPOSE Impact of research results on programs The MOH staff credit the expansion of CBD of DMPA with contributing to an increase in CPR. Though it is difficult to attribute this difference exclusively to FHI’s work, the change is notable, and FHI’s efforts have certainly played a role in fostering partnerships, training, and advocating for CBD of DMPA. Overall coverage has increased from 61 trained agents to 208, working in seven of the 22 regions of the country, in 14 of 111 districts. The number of trained workers is still a relatively small proportion of the total of roughly 6,000 community health workers in Madagascar; however, the potential benefits go beyond just FP, as CBD agents could potentially be involved in providing other injections and services, as well. In Moramanga District, CPR increased from 20.9 percent in 2006, to 24.7 percent in 2007, to 35 percent in the second quarter of 2008. During this time, the number of CBD agents trained to provide Depo has increased from 23 to 85, and currently 18 or 21 communes have trained agents. Of 9,567 Depo users, 16 percent (1,531) obtained their method from a CBD agent. FHI’s success in research utilization has focused around CBD of DMPA, whereas the use of other practices, such as the Best Practices Package, has been less successful. In 2006, a multidisciplinary team including representatives from WHO, the MOH, SanteNet, and FHI conducted field visits, considered different tools and approaches, and finally selected systematic screening and the pregnancy checklist as being the most appropriate practices to respond to the identified needs. There seem to be a number of factors in why this process was less successful. First, the findings on efficacy were not clear. FHI will be holding a data interpretation workshop to further analyze the findings and identify how to move forward. The pregnancy checklist was quite successful, which raises the possibility that FHI is more effective in promoting its own research and tools with which they are more comfortable compared with those developed by others. For example, systematic screening was developed by the Population Council, and according to Population Council staff in Washington, they offered to send an expert in this tool to provide assistance to FHI, but FHI declined this offer. Another factor is that the government wanted to address a wide range of services in the tool (eight), despite the tool typically incorporating fewer services. Perhaps if someone with more expertise had been involved, some of these issues could have been addressed. From the field perspective, however, there seemed to be little difference in the promotion of tools, as the research and development of tools is done at the headquarters level. Another area where success has been limited is youth. Though FHI has supported materials for youth, uptake of materials particularly outside the urban capital has been limited. It is unclear whether additional research is needed to identify ways to promote materials to youth in these areas or whether current methods of dissemination have not been successful. 3. FUTURE OF UTILIZATION Strengths and weaknesses of this model The Madagascar experience highlights how specific efforts to promote utilization are fundamental and essential components of research. Without technical support and targeted advocacy, it is unlikely that CBD of DMPA would have progressed as far as it has in Madagascar. This includes advocacy from FHI and from USAID, where USAID/Washington staff were also advocates for CBD-DMPA and heavily involved in getting the process started. Similarly, FHI’s active role in ensuring that WHO guidelines (many of which are based on FHI research) inform the national norms and procedures is critical. The ability to follow up and create job aids and tools to put these norms into practice further enhances the chances for global-level research to inform and influence services at the national level. The issue of ―handoff,‖ that is when research organizations transfer responsibility for implementation and scale￾up to those in service delivery, is still up for debate, and there may not be a clearly defined answer. Even within Madagascar there were differing opinions. Some felt that researchers should maintain their competitive advantage by focusing on research and disseminating results and allowing local NGOs and bilaterals to manage CRTU RESEARCH UTILIZATION ASSESSMENT REPORT: RESEARCH WITH A PURPOSE 43 implementation and scale-up. Others felt that the role of researchers should continue through scale-up and implementation as research questions will continue to arise as programs go to scale. Probably the most commonly preferred approach was that of a gradual handoff. For example, USAID and MOH staff felt that “Researchers have to be involved in at least the next step in scaling-up, and then when it reaches a certain level, they give up little by little.” The role of a research organization such as FHI in advocacy was much clearer. All respondents agreed that advocacy by a research organization is critical to influence the MOH, donors, and other partners. In Madagascar, FHI credibility stands so high with the MOH that they are approached to continue work in areas outside of their expertise. For example, the MOH has suggested FHI take on the issue of contraceptive security despite FHI’s lack of experience in this area. FHI can, however, serve as an advocate to the MOH for DELIVER, a project that has recently started focusing on contraceptive security in Madagascar. Recommendations For FHI: Conduct more systematic communication and feedback with partners, including presentation of research results in more ―user-friendly‖ ways. Continue to remain involved in scaling up research results in Madagascar, but with an eye to appropriate time for handoff. Maintain a strong advocacy role with other partners, donors, and the MOH. Increase awareness of attribution and continue to stress ownership of MOH. Work to increase capacity of research and research utilization at the local and country levels. For USAID: Provide support to allow countries to share experiences (via online seminars, study tours, etc.). Continue advocacy and technical support from USAID/Washington. Future Research Needs Madagascar has had great success in community health, particularly with the community-based distribution of DMPA and the use of tools such as pregnancy checklists. However, additional research is needed and includes: How to improve the sustainability and retention of community health workers Understanding the large unmet need (24 percent) for family planning How to better supply distant health facilities and to better understand the causes of stockouts What is the cost-effectiveness of CBD-DMPA? This includes not only the cost for each worker, but also understanding the cost per client. 44 CRTU RESEARCH UTILIZATION ASSESSMENT REPORT: RESEARCH WITH A PURPOSE LIST OF RESPONDENTS Name Title Organization 1. Serge Raharison Project Director FHI 2. Ny Lova Rabenja Senior Research Associate FHI 3. Nadia Razakamandimby Finance/Administration Officer FHI 4. Ralaiaritinasoa Esperance Operations/Logistics Officer FHI 5. Olivier Rahoelison Driver FHI 6. Benjamin Andriamitantsoa FP/Child Survival Program Manager USAID/Madagascar 7. Lalah Rambeloson Director of Operations PSI 8. Ietje Reerink Reproductive Health Program Director PSI 9. Iarimalanto Rabary Director of Research, M&E PSI 10. Bodo Rakotomalala NGO Program Coordinator PSI 11. Paul Richard Ralainirina Minister of Health Ministry of Health and Family Planning (MOHFP) 12. Marie Perline Rahantanirina Vice Minister of Health MOHFP 13. Bako Nirina Rakotoelina Director of Family Planning MOHFP 14. Berthin Andrianaina Director Adolescent Reproductive Health MOHFP 15. Christine Ravaonoro Technical Officer Family Planning MOHFP 16. Simone Rasoarimalala Reproductive Health officer Moramanga District Office 17. Agnes Ravaoarimanana Reproductive Health Chief officer Alaotra Mangoro Regional Office 18. Herly Daniel Ramiandrisoa Medical Officer Moramanga District Office 19. Celestin Razafinjato Regional Health Director Alaotra Mangoro Regional Office 20. Veromaritra Andrianjafitrimo Technical Health Coordinator ADRA 21. Harisoa Rasoanandrasana Regional Health Coordinator ADRA 22. Jean Claude Rakotomalala Executive Secretary ASOS 23. Randrianarivony Techincal Health Officer ASOS 24. Marie Francine Tsiapetraka Community Based Distributer Ambolibolakely Village, Madagascar CRTU RESEARCH UTILIZATION ASSESSMENT REPORT: RESEARCH WITH A PURPOSE 45 25. Hubert L. Rakotoarivony Deputy General Director National Leadership Institute of Madagascar (NLIM) 26. Volkan Cakir Director RTI International (SantéNet2) 27. Jeanine Johanesa Rahelimahefa Family Planning Director SantéNet 2 28. Justin Ranjalahy Rasolofomananana Director Institut National de Santé Publique et Communautaire (INSPC) 29. Josea Ratsirarson Country Representative Medical Care Development International (MCDI) 30. Jerry Rakotozafy MCDI 31. Rija Fanomeza MCDI REFERENCES 1. United Nations Data Record—Madagascar www.data.un.org. 2. Community Based Distribution of DMPA: Madagascar Study Tour Report. June 16–20, 2008. 3. Normes et Procedures en Santé de la Reproduction; Repoblikan’I Madagasikara; Ministere de la Sante et du Planning Familial. 4. Madagascar Action Plan 2007–2012: A Bold and Exciting Plan for Rapid Development. www.madagascar.gov.mg/MAP. 5. MOH and NLIM. 2008. Colloque International: Les Bonnes Pratiques en “Santé de la Reproduction”— Livret du participant. En collaboration avec USAID et FHI. Iavoloha—Antananarivo, 12 et 13 Juin. 6. MOH and NLIM. 2008. Colloque International: Les Bonnes Pratiques en “Santé de la Reproduction”— Resume des Engagements. En collaboration avec USAID et FHI. Iavoloha—Antananarivo, 12 et 13 Juin. 46 CRTU RESEARCH UTILIZATION ASSESSMENT REPORT: RESEARCH WITH A PURPOSE CRTU RESEARCH UTILIZATION ASSESSMENT REPORT: RESEARCH WITH A PURPOSE 47 CRTU RESEARCH UTILIZATION COUNTRY REPORT: RWANDA “WE ARE RESULTS-BASED, SO WE NEED TO BE EVIDENCE-BASED” ―We are a country that wants to make evidence-based decisions,‖ explains a leader in HIV/AIDS in Rwanda, adding ―we are results-based, so we need to be evidence-based.‖ Perhaps in part it is this mentality that has contributed to successful research utilization efforts by Family Health International’s (FHI) Contraceptive and Reproductive Health Technologies Research and Utilization (CRTU) Project in Rwanda. As FHI staff explain, there is ―something about Rwanda—the efforts being done are coordinated at the national level.‖ USAID organized an assessment of research utilization activities under the CRTU Project, which included visits to four countries: Kenya, Madagascar, Uganda, and Rwanda. Two members of the four-person assessment team traveled to Rwanda during September 3–6, 2008, conducted interviews with 24 individuals, and visited two health facilities (one urban, one rural). Rwanda is different from the other three countries in that it is not a focus country under the CRTU Project, and therefore it provides important lessons in how to effectively promote research utilization in a non-focus country. The report presents the findings under three main areas: the process of promoting research results, the impact of research utilization, and conclusions and suggestions for improvement for the future. I. PROCESS OF UTILIZATION. Has the CRTU project made progress in promoting the applicability and utilization of research results related to products, tools, and approaches to improve family planning service delivery, both FHI’s own results and those of other organizations? FHI-Rwanda employs 70 staff, but there are currently no full-time CRTU staff. As of June 2008, $47,000 of CRTU core funds had been spent in Rwanda, along with additional resources in terms of staff time from headquarters and regional staff. Even with just that minimal investment, there has been important research utilization in Rwanda. There are two primary outcomes or ―spillovers,‖ in the words of one FHI staff member, of CRTU work on programs in Rwanda: 1. HIV/FP integration 2. Community-Based Distribution of DMPA. FHI has ―spearheaded‖ efforts in integration; according to several respondents, ―they are at the frontline of integration,‖ explains USAID staff. Many people mentioned lessons from Kenya when discussing integration, in particular highlighting a study tour by staff from the Capacity Project in Rwanda who visited Kenya to learn firsthand about experiences in integrating FP and HIV services. Others agree that FHI has played a key role, but highlight how there were many efforts at integrating services before FHI began to proactively push the issue forward in 2006–07. Other forces also helped to make it an opportune moment for moving this issue forward— in particular, many partners saw the importance of meeting the needs of couples and helping HIV-positive women avoid pregnancy if they chose to. As staff at the MOH explained, ―in these days we are seeing increasing numbers of pregnancies in HIV-positive women.‖ ―The discussion started not just at FHI, but at TRAC, MOH, etc.‖ explained FHI staff. One respondent explained how the Minister of Health, a strong advocate for family planning, would often say to him, ―are your HIV-positive women still having babies? What are you doing? Please do something for them.‖ What is missing now in integration is national policy and direction. To help address this, CRTU has seconded a staff person to the MOH to focus on integration, to help to move this issue forward and create a ―road-map to scale-up.‖ He is currently collecting documents to develop a strategy document, and will possibly travel to other countries to see other models of integration. This staff member has important technical support from FHI/Rwanda and FHI/NC. This model is a ―nice transition for the handoff‖ from research to service delivery, according to FHI staff. The origin of the idea for this staff member is interesting; the USAID/Rwanda CTO for FHI was at a State-of-the-Art (SOTA) course, became intrigued at how to use CRTU better in Rwanda, and 48 CRTU RESEARCH UTILIZATION ASSESSMENT REPORT: RESEARCH WITH A PURPOSE realized that one thing missing was a person at the MOH with technical expertise in integration, so USAID approached the MOH and explained that FHI had resources to provide assistance. In discussing FHI’s influence on integration in Rwanda, a staff member from FHI explained, ―That wasn’t planned out, but that is probably the greatest accomplishment.‖ On the other hand, utilization of CBD of DMPA was ―more set up.‖ Efforts began with FHI regional staff visiting FHI/Rwanda to discuss their work in having community-based health workers provide DMPA (see timeline below). This was followed by an educational visit to Uganda of a six-person team including three staff from the MOH, two staff members from IntraHealth, and one FHI staff member. A report on this trip was then presented to the FP Technical Working Group (TWG) and to a senior management meeting at the MOH. The CBD of DMPA findings were timely in Rwanda; there is a strong focus on family planning from the government, and an emphasis on community health, including a recently developed Community Health Policy: ―all these efforts are quite favorable for a CBD program for contraceptive use.‖ Timeline for Promotion of CBD of DMPA in Rwanda June 2007 Feb 2008 March 2008 April 2008 May 2008 June 2008 July 2008 August 2008 Sept 2008+ FHI regional staff visit FHI/Rwanda and discuss CBD of DMPA Study tour to Uganda to see CBD of DMPA Delegates’ report submitted to MOH and FPTWG Delegates and MCH Task Force discuss rapid assessment FHI sends job aids and advocacy kits in French MOH approves proposed CBD of DMPA CBD of DMPA included in draft of FP/RH norms Planning for rapid assessment presented to FP TWG August 13 Rapid assessment implemented with report due November 15 Currently, there are plans to conduct a rapid assessment in 10 districts to determine the feasibility of using community health workers to distribute DMPA. Results will be available in November. Many partners are on board—most importantly, the MOH, but also IntraHealth, PSI (through German Technical Cooperation [GTZ] support), and UNFPA. However, there is remaining skepticism, which highlights some of the difficulties in utilization. Rwanda will not allow community health workers (CHWs) to provide DMPA to new users, only for resupply or follow-up. The team heard this concern from multiple respondents, including FHI staff, highlighting the need to adapt to local contexts, and also the difficulty of changing attitudes, as is often seen in such task￾shifting recommendations. Interestingly, the Executive Director of Commission Nationale de Lutte Contre le SIDA (CNLS) mentioned non-CRTU FHI research conducted in Rwanda looking at having nurses provide ARV treatment, and based on these results, this has now been adopted as national policy, indicating that there is an openness to allowing lower-level cadres of health staff to provide a broader range of services. In addition, Rwanda is one of the countries where FHI is trying to facilitate registration of Sino-implants. MOH staff that the team met with had not heard about this (the product is not yet registered in Rwanda), but were excited at the possibility of a cheaper implant, given the high cost of Jadelle and its popularity and frequent stockouts. The dossier was recently submitted to authorities in Rwanda, and FHI has had discussions with IntraHealth to assist with training under a grant from the Gates Foundation to expand use of Sino-implants in 10 countries. Given that Rwanda is not a focus country, it is not surprising that respondents did not talk much about other FHI research findings or tools. Interestingly, in the two health facilities visited, the team saw one where the pregnancy checklist was prominently displayed, and another where it was absent and not known. At the latter facility, the team also noted the need to disseminate the findings regarding the expansion of the DMPA grace period from two weeks to four weeks. The team asked the FP nurse what they did if a woman came back later than three months for her DMPA shot, and she said if it was within two weeks, they gave the injection, but if it was longer than two weeks, they needed to check if the woman was pregnant. This was done by pregnancy test, CRTU RESEARCH UTILIZATION ASSESSMENT REPORT: RESEARCH WITH A PURPOSE 49 but ―if the woman didn’t want to wait, then she comes back when she is menstruating.‖ Such comments highlight that it would be useful to CRTU to work more with the FHI/Rwanda office to expand promotion of other FP and RH tools. Partnership Respondents said many positive things about FHI: ―They are good at making things happen.‖ ―FHI never does research alone, they are always with national institutions.‖ ―FHI is at the forefront of many innovations.‖ ―They carry weight.‖ While there is a bit of competition among the clinical partners, according to USAID, there are also very good exchanges of information through the technical working groups. Some say that FHI does a good job of sharing information, while others feel that this could be improved and they would like to be informed more. ―Information sharing is not always optimal,‖ explains one partner, continuing to say that sharing often happens at the initiative of a couple of people rather than being systematic. As one example, some partners had not yet seen any of the preliminary results of FHI’s five-country integration study, even though they were consulted during the process; ―it’s FHI’s research, and you want to toot your own horn which might preclude sharing preliminary results.‖ Ownership It is widely acknowledged that a sense of ownership is critical to utilization and sustainability. Respondents talked about the importance of this, that the ―authority owns the process and owns the results.‖ As USAID staff states this idea, ―the MOH should drive the program, FHI shouldn’t drive it.‖ This highlights again the importance of favorable conditions, such as the MOH moving toward community health, which makes this an opportune moment for something like CBD of DMPA, so the findings are an ―easy sell‖ and it is a question of providing appropriate technical assistance to facilitate adoption of the practice in a manner where the practice becomes institutionalized. USAID praises FHI for this; ―they are good at pushing the program, but stepping back at the same time—this is an MOH activity.‖ Specific efforts at promoting utilization Study tours have clearly been effective for Rwanda, particularly in promoting CBD of DMPA, though according to a member of the team that went from Rwanda, it was much less effective for Nigeria…showing that there is no simple all-purpose method that will work with every situation or every partner. CRTU regional staff also helped spread information through presenting findings at key meetings, including conferences and technical working-group meetings. One FHI staff member talked of the importance of ―consciousness-raising‖ both among partners at the national level and FHI staff. Rwanda provides an interesting model, not only of research utilization, but of creating stronger linkages between the two worlds of FHI: FP/RH research and HIV/AIDS programs. While organizational restructuring has made the organization theoretically ―one FHI,‖ this unity is not yet a widespread reality. But making it happen can be a key mode of utilization, particularly in the area of FP/HIV integration. The above were useful processes, but experiences in Rwanda also highlight efforts that are less successful. When asked about other tools or materials, FHI staff explained that they passed these off to the MOH, but ―to be honest, I don’t think they did much with them.‖ Not surprisingly, just making materials available is not enough. 50 CRTU RESEARCH UTILIZATION ASSESSMENT REPORT: RESEARCH WITH A PURPOSE One of the most important efforts at promoting utilization happens before a study even starts—this is choosing a study that matters, that is needed. This is perhaps easier to do in Rwanda, where research studies have to be approved by a research committee, based on whether the research is needed, the cost is fair, and the people doing it are qualified. ―Is it useful? If it isn’t, we don’t approve it,‖ explained a respondent in describing how this committee functions, further explaining that to get approved, there has to be a plan of utilization. This committee included civil society, implementing partners, donors, etc., that is, ―people who should benefit, people with expertise, and people who implement.‖ This committee works because there are penalties for bypassing it—―if people don’t go through this committee, this is communicated to Washington and the organization can lose PEPFAR funding—if it’s not serious, people will bypass.‖ What is the researchers’ role after the research study is completed? The following quotes highlight different perspectives. According to one local partner, ―after dissemination, it is the responsibility of the national institution.‖ Another partner talked about a continuing role for FHI, in terms of providing ―technical support, but not being the implementers.‖ Practice to Research Some FP-implementing partners talk about the lack of a mechanism to feed their needs into FHI’s research, which is not surprising given the lack of FP presence in the FHI/Rwanda office. For example, when one partner was asked whether FHI responds to their needs, there was silence, then the comment that ―there isn’t a mechanism to collect these needs.‖ Asked if they would like such a mechanism, they said yes, if there was money and expertise, but the challenge would be making the timing work. But some comments highlight a seeming lack of respect, a dismissive attitude toward research and lessons from non-research partners, for example, as seen in the lack of recognition of early integration work done by IntraHealth, or minimal attention to research conducted under Twubakane of gender-based violence. IntraHealth admits that they could do a better job of documentation, that some of their early PMTCT work was not particularly well documented, a common problem among service-delivery groups. II. IMPACT OF UTILIZATION. In terms of meeting the goals and objectives of the cooperative agreement and of the Office of PRH, what is the evidence of impact of FHI’s research utilization efforts under the CRTU project? IR3: Use of contraceptives, microbicides, and reproductive health technologies optimized and expanded Impact of research results on policies and programs It is too early to see the impact of CBD of DMPA and integration activities, though it is likely that both will have important effects on policies and programs in Rwanda. It will be particularly important to evaluate the impact of the seconded staff in the MOH on both policies and programs to determine whether this is a model that should be tried in other countries. It is interesting to note that some of FHI’s FP materials are cited in the Family Planning protocols (which are still only in draft form), including the following: Vérifier les critères d’éligibilité en utilisant les listes de vérification selon la méthode choisie en distribution en base communautaire (voir annexes à la fin du documents—Liste de vérification pour l’initiation de la pilule de FHI à adapter) (translation : Verify the eligibility criteria by using the checklist according to the method chosen in distribution at the community [see the annexes at the end of the document—Checklist for the initiation of the pill adapted from FHI]). III. FUTURE OF UTILIZATION. What is the added value, if any, of including a research utilization component in a primarily contraceptive and reproductive health research project? Is this strategy effective and worth the effort? The CRTU project has had some important success in transferring lessons into programs and policies in Rwanda. It is important to acknowledge the favorable conditions that existed for utilization of CBD of DMPA, CRTU RESEARCH UTILIZATION ASSESSMENT REPORT: RESEARCH WITH A PURPOSE 51 FP/HIV integration, and Sino-Implants. Such conditions are not under the control of any organization, but identifying such conditions is potentially possible; ―at a certain moment, specific issues come up and you need to deal with them,‖ as one person said. This could inform future efforts at promoting utilization in non-focus countries, but does require an on-the-ground presence to be able to identify and understand whether such conditions exist. Given FHI’s extensive network of offices involved in HIV/AIDS work, the organization should continue to find ways to tap into this potential to expand the impact of its work under CRTU. Suggestions for the Future: For FHI: Conduct more consciousness raising among FHI’s HIV/AIDS staff about CRTU research findings, using lessons from the process in Rwanda. Make greater efforts at sharing information with partners. Given the impressive structures and processes in Rwanda, hold a meeting with partners to disseminate this report and discuss implications for research utilization. Evaluate the model of having a seconded staff in the MOH to assist with HIV/FP integration to determine whether this is a model that should be tried in other countries. For USAID: Ensure that evidence is used for project design; ―we are not very good at using evidence-based information for project design,‖ admit USAID staff. ―Here it is just reporting, reporting, and we are not analyzing what we’re doing and not being strategic…because of the shortage of staff.‖ Set aside funds for USAID mission staff to go on study tours. Staff from GTZ proposed an interesting model: when creating a new program, those programs have access to a research fund—essentially, you have a team of researchers available to service/program groups to do essential accompanying research, perhaps organized with three to five key topics, but with flexibility within that. This idea is based on a successful model they use for providing assistance in preparing proposals for the Global Fund. According to GTZ, ―the model is so easy, it can be adapted.‖ Current priority Research Needs There is a need in Rwanda for more research around FP/HIV integration. Interestingly, the CNLS has provided support to FHI to supplement the five-country assessment of integration. This study looked at integration of FP with VCT and ART services, and CNLS wanted to supplement this by looking at PMTCT, as well. ―It is an encouraging outcome that the government is using its own money to complete the study,‖ one respondent commented. Many would like to see a clearer answer regarding what works best for integration of FP and HIV. As a health worker said, ―I would like to know the most successful experience in Africa. This will help us.‖ MOH staff were also interested to better understand why women do not come for the recommended four antenatal care visits and do not deliver at health facilities. 52 CRTU RESEARCH UTILIZATION ASSESSMENT REPORT: RESEARCH WITH A PURPOSE LIST OF RESPONDENTS Name Title Organization 1. Jessica Price Country Director Family Health International (FHI) 2. Fabienne Shumbusho Deputy Director FHI 3. Livinus Bangendanye Senior Technical Officer FHI 4. Innocent Tourate ART Service Coordinator FHI 5. Pulcherie Mukangwije Health Manager, DNP FHI 6. Emmanuel Muyambanza Study Coordinator FHI 7. Fidele Ngabo MCH/FP/Immunization Coordinator MOH 8. Anicet Nzabonimpa Coordinator for HIV/FP Integration MOH (seconded from FHI) 9. Blaise Uhagaze Director Muhima Hospital 10. Raisa Management of HIV/AIDS clinic Muhima Hospital 11. Agnes Binagwaho Executive Secretary CNLS 12. Mary Kabanyana Senior Clinical Services Advisor USAID/Rwanda 13. Soukey Traore Head of MCH USAID/Rwanda 14. Kristina Lantis Acting HPN USAID/Rwanda 15. Noni Gachuhi HIV Prevention Advisor USAID/Rwanda 16. Jessica Forrest GH Fellow USAID/Washington 17. Laura Hoemeke Director Twubakane, IntraHealth 18. Defa Wane Quality and Community Health Team Leader Twubakane, IntraHealth 19. Suzanne Twubakane, IntraHealth 20. Etienne Karita Director Project San Francisco 21. Elisabeth Girrbach Coordinator, Health Sector GTZ 22. Cornelia Vanzyl Country Director Elizabeth Glaser Pediatric AIDS Foundation 23. Beatrice Mukandori Director Kigoma Health Center 24. Jean D’Arc Kigoma Health Center CRTU RESEARCH UTILIZATION ASSESSMENT REPORT: RESEARCH WITH A PURPOSE 53 CRTU RESEARCH UTILIZATION COUNTRY REPORT: UGANDA USAID has conducted an assessment of research utilization activities under Family Health International’s (FHI) Contraceptive and Reproductive Health Technologies and Research Utilization (CRTU) Project. As part of this assessment, two members of the four-person assessment team visited Uganda during September 11–17, 2008. This visit, along with additional country visits to Kenya and Madagascar, provided the team with an overview of CRTU’s Focus Country strategy achievements and challenges in different settings. Uganda faces many challenges in the areas of FP/RH and HIV/AIDS. Modern family planning use is extremely low (18 percent), unmet need is approximately 40 percent, and total fertility is very high (6.7 in 2006); HIV/AIDS prevalence was estimated to be 6.5 in 2006, and efforts to integrate these two areas is relatively recent, really not started until 2006. In the context of these challenges, the FHI/CRTU Uganda focus country office seems relatively small. It clearly benefited during the startup phase from technical assistance provided by the Nairobi CRTU office, and the Uganda office continues to be well served by this arrangement. As work expands, however, it will be essential to assure adequate staffing to respond quickly and comprehensively to country needs. FHI has really been active in Uganda only since the beginning of CRTU, and work continues to expand. Nevertheless, project staff have been able to achieve a great deal in the relatively short duration of the project. FHI is a valued partner among CAs, and the MOH, respected for its quality research, its tools, and materials, and appreciated for its collaborative and flexible approach. Because of CRTU’s good reputation, expectations are high, and people think of going to FHI when a TA need arises. The current staff size (two professional staff, plus one administrative assistant) seems inadequate to respond quickly and effectively to all these requests. There was a plan to hire a research associate in the fall of 2008. However, the current program officer was expected to go on maternity leave in November 2008. Early Steps The 2004–05 World Health Organization/Implementing Best Practices (WHO/IBP) Entebbe Regional meeting developed in partnership with Jhpiego, CTR/Kenya, and other CAs had significant impact on future activities in Uganda and on the CRTU/Uganda agenda. Uganda defined the need to revitalize FP (not only to revitalize the IUD). FHI participated in an FP policy update, as did other CAs. An FHI stakeholders meeting resulted in defining priorities for research and program strengthening, and led to the formation of Family Planning and Reproductive Heath Working Group (FPRWG) in 2006 led by MOH, with FHI serving as the secretariat. A separate Partners’ Council (which included FHI’s MOU partners, as well as other CAs and the MOH) led by USAID with EngenderHealth, Save the Children, Pathfinder, JHU (PATH, Population Council not present) was established. The last meeting was held January 2008: New members: Conservation Through Public Health (CTPH) and MIHV—these were the new partners for the CBD/Depo initiative. Pre-CRTU TA from the Kenya office for IUD revitalization, establishment of the FPRWG, and Continuing Medical Education (CME) activities were essential in launching activities. 54 CRTU RESEARCH UTILIZATION ASSESSMENT REPORT: RESEARCH WITH A PURPOSE Initiation of CRTU With the establishment of the Focus Country Program/Uganda, the office expanded in resident staff to four (the three previously mentioned staff, and one person who focused exclusively on the RHODES project), and CTRU took the lead in activities of the FPRWG, the Partners’ Council, and coordination with MOU and non-MOU health facility staff and partners. The CRTU workplan for the first year was developed with significant input from North Carolina. Difficulties included the concept proposal process: very few proposals were approved and approvals took at least one year to be processed, leading to frustration on the part of the CAs and organizations involved. It is important to note that the percent of concepts approved for implementation is low systemwide, and that the approval process, which does take almost a year, is related to internal CRTU processes, and the USAID/Washington workplan approval process. Slowly, as local staff have come on board in focus countries, North Carolina has been moving toward local country office control/ownership of subsequent year annual workplans. There is still a need for local staff/budgeting to allow for flexibility to respond to important requests for TA and research utilization activities as they arise. FHI has developed a very strong relationship with the Uganda MOH as their prime partner. FHI/Uganda leveraged $400,000 from the Mission and PEPFAR in the first year. Accomplishments CBD of DMPA: John Stanback (2004) approached Save the Children, Uganda. Save the Children CBDs started providing pills and condoms in two sites in one district. The activity went smoothly and was well received by the communities. MOH permission was received to develop a pilot test of CBD provision of DMPA. Materials were developed, trainings held in collaboration with the MOH and Save the Children, and activities were launched. Initial pilots were successful, Save the Children sites were expanded, and MOH, only (without Save the Children) sites were launched. At present, DMPA policy allows only Medical Officers, Clinical Officers, and nurses to provide the injection. Assistant nurses, CBDs, pharmacies, and drug shops are not allowed to inject, but there is actually some lack of clarity about this in official guidance (see official MOH FP Guidelines, p. 16). CRTU is leading the effort with multiple partners to modify the existing policy by providing evidence from scaled-up pilot activities to provide local and global evidence. With MOH and the FPRWG fully on board, a larger scale-up strategy is in place in 20 sites in seven MOH districts, and also with Minnesota International Health Volunteers and CTPH, a conservation NGO. With TA from CRTU, training programs have been developed at participating MOH districts. MOH is providing funds to continue training providers. Uganda has become a leader in CBD of DMPA strategy and in on-the-ground experience. Kenya, Madagascar, and Rwanda study tours to Uganda have had great effect on these countries, including policy change. CRTU is developing a range of materials, including an advocacy kit to change policy. There is considerable caution on this issue in Uganda and the process is expected to be long. The MOH likes CBD of DMPA strategy and materials and will brand these even if a policy is not in place. MOH FP Support CRTU collaborated with UNFPA and other partners to update the MOH FP curriculum. Tools included in this curriculum include: the WHO simplified instructions on missed pills, guidance on advance provision of pills, family planning provider checklists, provision of multiple pill packs, and providing quick start of pills. CRTU RESEARCH UTILIZATION ASSESSMENT REPORT: RESEARCH WITH A PURPOSE 55 MOH has poor training capacity and appreciated TA from CRTU, although no training of trainers was conducted with them on the updated curriculum (with the exception of CBD of DMPA). The strategy is to increase partners’ FP training capacity by Contraceptive Technology Update (CTU) and CME events on the revised curriculum, which is now almost ready to be utilized. Checklists and other job aids (for example, MEC comprehensive chart, FP guidelines, tools for oral contraceptive provision) have been widely distributed and cascaded down levels by MOH and FHI partners; these are well known and appreciated. Among these, the pregnancy checklist and the MEC are mentioned most. Now many materials are branded by MOH. It is important to encourage MOH branding, and when that branding has been achieved, to refer to the tools/products as Ministry products. It is time to stop calling them FHI materials/job aids and start calling them MOH materials. CRTU has provided capacity building to the MOH to date at the central level and at the district level in CBD of DMPA training and continuing medical education in family planning. Additional activities Working to add RH/FP to expand youth strategy with established AIDS organizations, and to raise the issue with MOH Supporting an FP/HIV integration champion within the MOH Working with EngenderHealth and Centers for Disease Control (CDC) to integrate FP for couples living with HIV Male circumcision: needs assessment conducted, including the facility situation for providing expanded services. MOH intends to conduct a mass-media campaign to promote use of medical circumcision services. The survey was funded by the Mission and conducted by FHI with Makerere University School of Public Health; it includes assessment of community attitudes and practices regarding medical and traditional circumcision, and sexual concurrency research. Lessons learned CRTU staff have played an important FP/RH advocacy role in promoting collaboration among partners through the FPRWG, through outreach to other organizations, and through provision of technical assistance to a wide range of formal and informal partners. They have effectively promoted the utilization of checklists, manuals and other materials through presentations and provision of the materials at the FPRWG and directly to partners. These materials have been incorporated into MOH documents, including training curricula and guidelines. The materials have also been used in the CBD of Depo sites for the training of facility staff, training of the CBDs, and provision to the CBDs for use in their activities. CRTU staff have played an important role in advocating for policy change to officially permit the CBD of Depo. This remains a challenge, but FHI is taking the lead to influence policy. The activities of FHI, Save the Children, and the MOH to promote CBD of DMPA have been collaborative and highly effective. Together, they have managed challenges and implemented a program of community delivery of DMPA, which has demonstrated acceptability, quality, and impact in terms of contraceptive users in a very short period of time (2.5 years in the Save the Children sites, approximately eight months in the non-Save the Children, MOH-only sites. At the outset of CRTU, MOUs signed at HQ were not known by country partners. These MOUs did not allocate funds for locally important proposals, which may not rank highly in North Carolina-based global efforts, causing frustration of partners due to raised expectations and significant loss of time. Partner organizations do not necessarily provide documentation of utilization of CRTU assistance; for example, on IUD revitalization, FHI has difficulty obtaining data on uptake trends since using the toolkit, checklists, training manuals, etc. probably originates in competitive attitudes. 56 CRTU RESEARCH UTILIZATION ASSESSMENT REPORT: RESEARCH WITH A PURPOSE MOH branding is very important for partners to adapt/use FHI materials. One-on-one meetings are more effective for discussion of specific issues. Research and utilization proposals do not need Working Group approval but do need Mission approval. Key Issues for the Future There is an urgent need to advocate for and collaborate in the drafting of an explicit policy supporting the provision of DMPA by community-based agents. Although there is an RH policy on adolescents in place, there is a need to get the issue back ―on the screen‖ of the MOH and of CAs. Both FHI and Pathfinder have extensive expertise in this area, and could collaborate on such an effort. A component of these activities should be a policy allowing the provision of FP/RH services in the schools. Similar to the package of advocacy, operations research, and development of training and counseling materials that was developed for the IUD in Kenya, a package of activities should be developed to permit the provision of permanent contraception by lower-level medical providers (clinical officers and nurse midwives). There is documented and growing demand, and CRHWs report interest in permanent contraception on the part of their clients. There is a need for a technical review of materials and provider knowledge on the issues of reinjection window, treatment of side effects, and the initiation of contraception. There is a need for a systematic review of training and counseling materials in FP/RH. The flipchart developed by The Johns Hopkins University is the main counseling tool currently being used by CRHWs. However, this flipchart, at least the version for the CRHWs, does not have the ―ruling out pregnancy checklist.‖ In addition, the production of the checklists, in the context of other materials (including the flipbook) currently available in Uganda, needs to be reviewed for their role in training, counseling, and strategy development. While it may be that each of these products does serve a unique role, duplication of effort is a concern. While integration of FP and HIV is a sensitive area of work for CRTU given the Mission’s ―assignment‖ of this area to EngenderHealth, it is essential to identify ways in which the extensive resources of FHI in this area can be made available and promoted to partners in Uganda. Should CRTU be working deeper or wider? Should CRTU focus on perhaps three main issues in a highly intensive way, or should it try to be responsive to a broad range of requests for TA, materials, etc.? The need for more work on men’s attitudes toward FP, and the role they play in the contraceptive decision making of women, and the development of some formative research and operations research was raised repeatedly. CRTU is well placed to take on such a task. CRTU RESEARCH UTILIZATION ASSESSMENT REPORT: RESEARCH WITH A PURPOSE 57 LIST OF RESPONDENTS Organization People Family Health International Dr. Angela Akol Erika Martin Ms. Patricia Wamala EngenderHealth Grace Nagendi Henry Kakande Nakaseke District MOH Dr. Badru Sessimba Dr. Violet Nabatte Justine Bulyaba Busia District MOH Dr. George Oundo Agnes Lojjo Pathfinder Dr. Abeja Apunyo Rhobinah Babirye CTPH Conservation Through Public Health Stephen Rubanga Sylvia Nandago UHMG AFFORD (JHU CCP) Godfrey Magumba Uganda Health Marketing Group (Beads) William Nyombi Busia District - Bulumbi Hlth. Centre III Acen Rose Lam Mayende Reproductive Health Uganda (RHU) (IPPF Affiliate) Dr. Peter Ibembe USAID Sereen Thaddeus MIHV International Volunteer Dr. Kaur Marjolein Moreaux Norah Nakate MOH AIDS Control Program (ACP) Dr. Elizabeth Madraa Dr. Godfrey Esiru Save the Children Martha Bekiita Richard Talagwa John Kisaka 58 CRTU RESEARCH UTILIZATION ASSESSMENT REPORT: RESEARCH WITH A PURPOSE Luwero District Kamira Health Centre III Dr. Joseph Okware Ruth Acam Sula Buyinza Bugiri District Buluguyi Health Centre III Dr. Stephen Kirya Misaki Baliddawa Rose Ofwono MOH, RHD Dr. Anthony Mbonye Dr. Miriam Sentongo Dr. Miriam Mutabazi JHU/HCP Cheryl Lettenmaier Barbara Katende Donna Sherard List of Background Documents 1. FHI Briefs. 2. CRTU Uganda Program Summary of Key Accomplishments. 3. FHI Uganda: CRTU Portfolio Highlights – CBD of DMPA, Hormonals and FP/HIV. 4. USAID Quarterly Review, April 28, 2008. 5. Memorandum of Understanding between Family Health International and EngenderHealth. 6. Notes from Partners’ Council Meeting, Kampala, Uganda, 16 January. 7. FHI – CRTU Management Review Family Health Internationals’ Response to Questions. 8. Solo, Julie, Susan McIntyre, and Elizabeth Warnick. 2005. ―To Be of Use: An Assessment of FHI’s Research to Practice Initiative.‖ June. 9. Baine, Dr. Sebastian, and others. 2008. ―Situation Analysis to Determine the Acceptability and Feasibility of Male Circumcision Promotion in Uganda, P.I.‖ CRTU RESEARCH UTILIZATION ASSESSMENT REPORT: RESEARCH WITH A PURPOSE 59 APPENDIX 3: SCOPE OF WORK I. BACKGROUND The Contraceptive and Reproductive Health Technologies and Research Utilization (CRTU) Program was awarded to Family Health International in April 2005. Its title, as well as the accompanying Program Description, addressed USAID’s interest in both developing and evaluating contraceptive and reproductive health technologies, as well as ensuring that research findings are quickly disseminated and used to influence service delivery programs. The CRTU project is unique in that it represents the first time that a primarily research-focused cooperating agency (CA) and research project have been tasked with optimizing and expanding utilization of contraceptive and reproductive health research results. More than a quarter of the CRTU project’s resources and deliverables are directly related to research utilization outcomes. In 2008, far enough into the cooperative agreement to assess progress but still allow for adjustments, USAID would like to evaluate the significant research utilization component of the CRTU in order to both assess the progress of the CRTU project’s utilization efforts and to determine to the extent possible whether or not the incorporation of the such a utilization component into a research project is the most efficient and effective way to promote research utilization. This external evaluation will complement a second thematic evaluation currently underway and being carried out by FHI itself focused on the topic of HIV and family planning integration. Research utilization, as a key component of the CRTU, involves all parts of the program. It includes taking practical, program-related questions into consideration when setting the research agenda, planning for utilization in the course of developing research projects, disseminating results, and working with others to ensure that the best evidence is applied. Research results can include tested service delivery methodologies, tools and specific clinic and community-based practices among other things. The CRTU’s focus country program (FCP) is linked to these utilization efforts by informing, disseminating and concentrating FHI’s activities within an individual focus country to maximize impact. Therefore, by examining the CRTU’s research utilization efforts, one can obtain a good understanding of the CRTU Project overall. The CRTU has developed a detailed project monitoring plan (PMP) with specific utilization related indicators. The project’s success will ultimately be measured by how it performs against these indicators. II. PURPOSE This thematic evaluation will assess FHI’s activities to increase the utilization of research findings, both FHI’s own results and others, under the CRTU during its first three years as defined by the project’s PMP. To the extent possible, the impact of these efforts will also be assessed. The utilization component of the CRTU is unique and it has not been determined whether having such a breadth of scope incorporated into one program effectively allows for streamlining and the strengthening of synergies between research and service delivery programs. The proposed seeks to address this issue by evaluating the utilization efforts across the CRTU agreement. Expenditures associated with accomplishing the tasks will also be assessed. The assessment should devote roughly equal time to reviewing past performance and future opportunities. Specifically, the evaluation team is expected to answer the following questions: Has the CRTU project made progress in promoting the applicability and utilization of research results related to products, tools, and approaches to improve family planning service delivery, both FHI’s own results as well as those of other organizations? In terms of meeting the goals and objectives of the cooperative agreement and the Office of PRH, what is the evidence of impact of FHI’s research utilization efforts under the CRTU project? What is the added value, if any, of including a research utilization component in a primarily contraceptive and reproductive health research project? Is this strategy effective and worth the effort? III. METHODOLOGY 1. Self-Assessment: FHI will first prepare a self-assessment of the CRTU’s research utilization program, based largely on the general questions included at the end of this scope of work. This report will be provided to the evaluation team as part of the background materials. 60 CRTU RESEARCH UTILIZATION ASSESSMENT REPORT: RESEARCH WITH A PURPOSE 2. Preparation of the Evaluation Workplan: The evaluation team will consist of three to four members including two outside consultants and one senior USAID staff member. A junior USAID staff member may also be included if needed. The team will initially meet with the USAID staff (RTU Division) to be briefed on the CRTU Project and USAID’s interest with respect to research utilization. The team will then meet to further refine and prioritize the key questions to be addressed in the evaluation and review and discuss the general methodology to be used The team will then be responsible for developing the overall final evaluation workplan, defining the responsibilities of individual team members, developing interview questionnaires including a standardized data collection instrument to be used by all team members during their individual country visits, agreeing on a schedule for specific activities (in consultation with FHI for field travel), and addressing other operational and logistical issues as needed. The workplan and budget will be reviewed and approved by USAID before beginning field work. 3. Review of Background Documents/Materials: The following documents will be provided to the evaluation team. Other documents may be added or requested as needed or deemed appropriate. Cooperative Agreement GPO-A-00-05-00022-00 Notes from the last two CRTU management reviews The 2007-2008 CRTU Annual Workplan, and the 2006-2007 Annual Report Self-assessment report from FHI The CRTU M&E Framework, Gap Analysis, and Country Results and Logic Matrices Significant CRTU research publications, as determined by relevance to research utilization theme FHI’s Research Utilization briefs and other disseminated materials of relevance to research utilization List of country programs and a proposed list of contact persons A brief synopsis of the CRTU MOU partnerships 4. Interviews: In consultation with USAID’s RTU Division, the evaluation team will extensively interview selected USAID staff, including the Microbicides team, CSL, SDI and PEC division staff. The team will interview FHI headquarters and field level staff as well as staff from other cooperating agencies (including MOU partners), multinational groups such as WHO and IPPF, donors, selected ministries of health, researchers, advocates, or other parties chosen by the evaluation team. In most cases, it is expected that interviews with US-based USAID or FHI staff will be conducted in person. Some interviews will need to be conducted via conference call (e.g. calls with WHO, IPPF, and EngenderHealth), during which the entire evaluation team may be present. Though country visits will likely involve only one or two team member(s) per country, if deemed necessary, interviews may be done though international conference or video calls with all team members. A list of suggested interviewees (key informants) at FHI, USAID and other stakeholders will be provided by USAID/W with FHI input. 5. Field Visits: The evaluation team will tentatively travel (one or two member(s) to each country, or perhaps one member to two countries including a non-focus country) to visit ongoing CRTU subprojects. Likely countries to be visited include three of the CRTU focus countries: Kenya, Madagascar, Uganda- and Rwanda (a non-focus country). By visiting countries where FHI has limited or no specific utilization activities the team will be able to determine the effectiveness of the CRTU’s targeted technical assistance for research utilization taking place in focus countries. The team will assess program implementation, research progress, and evidence of collaboration and research utilization. Final selection of countries to be visited will be determined by degree of local Mission support, level of resources invested in the CRTU program, and the size and diversity of in-country subprojects. IV. DELIVERABLES 1. Trip Reports: Each team member will complete a trip report that summarizes their key findings (including direct quotes to illustrate significant points) from the country visits. These will be shared with all team members and, in conjunction with information obtained from the interviews the team has conducted together in the U.S., will be used to develop a common final evaluation report. The trip reports are intended to be source documents and a means of expediting the overall report. Any recommendations they may contain should be country-specific; overall recommendations will be discussed and decided upon by the team as a whole. CRTU RESEARCH UTILIZATION ASSESSMENT REPORT: RESEARCH WITH A PURPOSE 61 2. Evaluation Report: The evaluation report should describe the methodology, provide conclusions on the key evaluation questions and offer recommendations for the future. A complete draft report, not to exceed 30 pages (not including annexes) and which will include a clear 3-5 page executive summary with attachments as needed to clearly illustrate or highlight key points will be submitted to USAID/W at a time to be determined in collaboration with USAID when the assignment timeline is finalized. USAID/W will have two weeks to provide comments and suggestions to the evaluation team which will be addressed in the final report. A near-final draft should be shared with USAID and FHI for corrections of facts and feedback. Recommendations should be those of the evaluation team as a whole. This report is primarily intended for internal USAID use in assessing the performance of the CRTU Program and defining future program needs. All or parts of the report will be shared with FHI. However, any recommendations to USAID regarding future procurement issues may be kept internal to USAID. After comments have been provided to the team, the final executive summary and full report will be prepared incorporating the comments received from the review of the draft. The team will submit the final but unedited report for USAID approval no later than one week after USAID has provided comments on the draft. USAID will share the final but unedited report with the partners/grantees to give them an opportunity to review and make comments. After the final but unedited draft report has been reviewed by USAID and the partners/grantees, GH Tech will have the document edited and formatted, and will provide the final report to USAID; each mission will receive a copy of their country study and a copy of the final report. 3. Debriefings: The evaluation team will provide separate debriefings in Washington D.C. to both USAID and FHI’s CRTU key staff. V. TEAM COMPOSITION The evaluation team must be qualified and be sufficiently respected and influential so that their recommendations will be considered to be authoritative. It is expected that three team members, and possibly a fourth junior USAID staff member who may be involved as needed, with complementary knowledge in this technical area will be sufficient for the evaluation team. The team should have expertise in the following areas: Knowledge of contraceptive and reproductive health research, including basic knowledge of the various research methods (e.g. clinical, operations and behavioral research) Familiarity with the ideas of research utilization and the research-to-practice, practice-to-research continuum. Experience in the management of USAID-sponsored family planning and other reproductive health programs in developing countries Knowledge of issues related to information dissemination and utilization of research for program improvement Developing country experience Good writing and communication skills with experience in producing team-based reports For any team member going to Madagascar, fluency in French is desirable. Potential candidates for this team may include: senior and possibly retired persons with careers related to contraceptive research and development, and/or reproductive health care in developing countries. The candidates must be able to work as team members, evaluate and synthesize information quickly, make clear and well-founded recommendations, and contribute to the written report and debriefings. Careful judgment should be used to recruit consultants who are knowledgeable and highly respected in this field, but are unbiased about this technical area and its future directions. Approximately four weeks, though not necessarily consecutive, of effort will be required for each of the team members, with an additional possible week for the team leader. A suggested breakdown of time is included in the following table. Timelines should be adjusted based on the individual requirements of team members as long as the team as a whole can function coherently and complete the task. The following timeline is based upon the calendar approved in the meeting between USAID and GH Tech Project staff on April 30, 2008 (timeline is attached for reference). 62 CRTU RESEARCH UTILIZATION ASSESSMENT REPORT: RESEARCH WITH A PURPOSE Activity Days Team Leader + USAID Staff (Team 1*) Days Consultant + USAID Staff (Team 2*) Preparatory work Review of background documents/materials 2 2 Team Planning Meeting with USAID (including travel) 3 3 Continue review of documents/materials, develop interview instruments, schedule DC-based interviews and make Missions/travel arrangements 2 2 Data Gathering Conduct DC-based interviews and other US and/or international conference calls 3 3 Visit to FHI North Carolina office (includes travel day) 3 3 Field Visit Country 1 (Kenya) and writing country report (includes travel) 0 8 Field Visit Country 2 (Uganda) and writing country report (includes travel) 0 7 Field Visit Country 3 (Madagascar) and writing country report (includes travel) 6 0 Field Visit Country 4 (Rwanda) and writing country report (includes travel) 6 Full-Team Meeting in Kenya 3 3 Data analysis/drafting report (includes required expense reporting, and required time for team to review all trip reports, agree on conclusions and recommendations, and develop common themes and overall conclusions and recommendations) 18 13 Submit draft report Receive feedback Revise report based on feedback 2 2 Submit revised report - Debrief at USAID (includes travel) 3 3 Finalize report based on feedback 2 2 Submit final report - Total Days 53 51 *Teams include 1 consultant + 1 USAID staff member each CRTU RESEARCH UTILIZATION ASSESSMENT REPORT: RESEARCH WITH A PURPOSE 63 VI. SCHEDULING AND LOGISTICS Once the evaluation team members are identified and recruited, the process for document review and interviews with key informants can begin immediately. Field visits will be completed in the summer of 2008. The above timeline will be revised as necessary to ensure sufficient opportunity for report writing, including edits and revisions, as well as final debriefings. Additionally, prior to drafting the full report, an outline should be provided to USAID for review and comment. The evaluation team should adhere to the agreed upon timeline. Prior to the debriefings, the team will prepare its major findings and recommendations. It is anticipated that the full team will make two trips to Washington, and one trip to North Carolina. There will be two teams comprised of two team members each. Each team will visit two countries. FHI staff will assist with in-country arrangements for the interviews. Questions to be Addressed by the Research Utilization Assessment I. Has the CRTU project made progress in promoting the applicability and utilization of research results related to products, tools, and approaches to improve family planning service delivery, both FHI’s own results and those of other organizations? 1. Are the goals and outcomes of the CRTU being met, particularly those which have a research utilization component? (Reference the CRTU M&E Framework) 2. To what extent has FHI been effective in promoting the uptake and use of important research results, both FHI’s own results as well as other findings? 3. What is the unique role of the focus country program in aiding or enhancing the CRTU's efforts to promote research utilization efforts at a country level? Country assessment visits should seek to document examples. (Reference also the country matrices). 4. Considering the MOU partnerships, and work with multinational organizations such as WHO, describe the roles of FHI and its other partners in research utilization efforts. Identify specific examples of collaboration. What factors facilitated or hindered these collaborations? 5. What are the roles of national governments, service delivery programs, and donors in enhancing research utilization? 6. In the opinion of partners and stakeholders, are there significant contraceptive or reproductive health research findings that are not being incorporated into existing programs? What are the potential reasons why these findings are not being used or incorporated? II. In terms of meeting the goals and objectives of the cooperative agreement and of the Office of PRH, what is the evidence of impact of FHI’s research utilization efforts under the CRTU project? 7. What impact has been achieved as a result of the CRTU research utilization efforts? 8. Under the CRTU, what evidence is there of the balance FHI has struck in managing its responsibility to provide global technical leadership with its interest in being responsive to USAID Missions and country￾specific interests? 9. Describe the characteristics of the CRTU’s successful research utilization efforts as well as specific obstacles to getting research results utilized faced by the CRTU. 10. How do the CRTU’s dissemination efforts support and advance the research utilization effort? Describe the scope of these efforts and, to the extent it has not already been addressed, the impact. 64 CRTU RESEARCH UTILIZATION ASSESSMENT REPORT: RESEARCH WITH A PURPOSE III. What is the added value, if any, of including a research utilization component in a primarily contraceptive and reproductive health research project? Is this strategy effective and worth the effort? 11. The CRTU Program was intentionally distinguished from its predecessor, the CTR, by its emphasis on research utilization. Describe the processes by which the CRTU has placed emphasis on research utilization under this Cooperative Agreement. What are the strengths and weaknesses of the current process? 12. What might USAID do to further encourage the identification of and support for contraceptive and reproductive health research priorities, as well as the use of research results and evidence-based practices? 13. What could FHI do to further improve its effectiveness in getting research utilized? 14. Are there strategic areas and/or research findings that FHI should be concentrating on more or less heavily? For more information, please visit http://www.ghtechproject.com/resources/ Global Health Technical Assistance Project 1250 Eye St., NW, Suite 1100 Washington, DC 20005 Tel: (202) 521-1900 Fax: (202) 521-1901 www.ghtechproject.com