USAID/DRC: Mid-Term Performance Evaluation for the Advancing Social Marketing for Health in the Democratic Republic of the Congo, Award GHH-I-05-07-00062-00 DISCLAIMER The views of the authors expressed in this publication do not necessarily reflect the views of the United States Agency for International Development or the United States Government. Submitted November 2012. ii TABLE OF CONTENTS ACRONYM LIST .............................................................................................................................................................. iv EXECUTIVE SUMMARY .................................................................................................................................................. 1 I. INTRODUCTION ................................................................................................................................................... 4 A. Statement of Work ................................................................................................................................................. 5 B. Evaluation design and methodology ..................................................................................................................... 5 II. FINDINGS .................................................................................................................................................................. 7 A. Theory of Change .................................................................................................................................................... 7 B. Supply and Diversity of Health Products and Services Distributed and Delivered .................................. 9 i. Confiance Contraceptives ................................................................................................................................... 9 iii. Aqua Tabs and PUR .......................................................................................................................................... 14 iv. Clean Delivery Kits ........................................................................................................................................... 15 v. Other products ................................................................................................................................................... 16 C. Awareness and Demand for Health Products and Services ........................................................................ 16 i. TRaC Study Findings ........................................................................................................................................... 17 ii. Consumer Commentary from Focus Groups ............................................................................................. 17 D. Appropriate Approaches for Behavior Change Communication .............................................................. 20 i. Mass Media ............................................................................................................................................................ 18 ii. Peer Educators and Community Organizers ............................................................................................... 19 E. Enhanced Capacities of Commercial Sector, Private Sector Entities and local Non- Governmental Organizations ............................................................................................................................................................... 20 i. Commercial Sector Wholesalers .................................................................................................................... 20 ii. Private Sector Clinics ........................................................................................................................................ 20 iii. NGO Partners .................................................................................................................................................. 20 F. Integrated Services at National, Provincial, District and Facility level ....................................................... 21 G. Joint Planning and Coordination with the Government of the Democratic Republic of the Congo, United States Government partners and other partners .................................................................................. 22 i. Government Partners ........................................................................................................................................ 22 ii. Other USG partners.......................................................................................................................................... 23 iii. Non-USG partners............................................................................................................................................ 23 H. The Effectiveness of the Rural Strategy ............................................................................................................ 24 I. The Gender and Youth Perspective within Social Marketing for Health ............................................. 24 i. Gender ................................................................................................................................................................... 24 ii. Youth ..................................................................................................................................................................... 26 J. Institutional Capacity of the Associations de Santé Familiale ....................................................................... 27 iii IV. CONCLUSIONS ................................................................................................................................................ 31 V. RECOMMENDATIONS ................................................................................................................................... 32 VI. ANNEXES ............................................................................................................................................................ 37 A. List of Materials Consulted .................................................................................................................................. 37 B. List of People Interviewed ................................................................................................................................... 37 C. Tools Used in Focus Groups and In-depth Interviews ................................................................................. 37 D. Original and Revised Scopes of Work ............................................................................................................. 37 E. Full Evaluation Methodology ................................................................................................................................ 37 iv ACRONYM LIST AIDS Acquired Immune Deficiency Syndrome AIDSTAR AIDS Support and Technical Assistance Resources   ASF Association de Santé Familiale- Family Health Association CDK Clean Delivery Kit COC Combined Oral Contraceptive COR Contracting Officer’s Representative DFF Department Femme et Famille de L’Eglise du Christ au Congo- Family and Women Department of the Christ Church, Congo DHS Demographic and Health Surveys DMPA Depot medroxyprogesterone acetate-Depo Provera DOD Department of Defense DRC Democratic Republic of the Congo DTK Diarrhea Treatment Kit FC Franc Congolese GDRC Government of the Democratic Republic of the Congo HIV Human Immunodeficiency Virus ITNs Insecticide Treated (bed-)Nets MARP Most At Risk Population MCZ Médecin Chef de Zone MDG Millennium Development Goals MICS Multiple Indicator Cluster Survey MIP Médecin Inspecteur de Province MSM Men who have sex with men NGO Non-Governmental Organization OAM Opportunity, Ability and Motivation OIG Office of the Inspector General PEPFAR President’s Emergency Plan for AIDS Relief PNLS Programme National pour la Lutte contre le Sida- National Program for the Fight against AIDS PNLMD Programme National pour la Lutte contre les Maladies Diarrheque-National Program for the Fight against Diarrheal Disease PNMLS Programme National Multi sectorial pour la lutte contre le Sida-National Multisectoral Program for the fight against AIDS POP Progesterone Only Pill PR Prime Recipient v PPIUD Postpartum Inter-uterine Device PROSANI Project de Santé Intégré – Integrated Health Project ProVIC Projet de lute contre le VIH Intégré au Congo – Integrated HIV/AIDS Project PSI Population Services International SI Social Impact SR Sub Recipient STIs Sexually Transmitted Infections TRaC Tracking Results Continuously UNICEF United Nations Children’s Fund USAID United Stated Agency for International Development VCT Voluntary Counseling and Testing WASH Water and Sanitation for Health EXECUTIVE SUMMARY At the time of this evaluation in September 2012, The Advance Social Marketing for Health in the Democratic Republic of the Congo had just ended its third year of implementation. Using qualitative evaluation methodologies, including conducting over 30 focus groups in four provinces, in-depth interviews and data review, the evaluation team found that PSI/ASF has run a successful project. It will achieve all but three of the goals set in 2009. In some cases, such as the distribution of Aqua tab and PUR, they have more than doubled the amount of product they have sold. In cases where they are behind target during one quarter, they usually are able to make it up in subsequent quarters. They have suffered from stock-outs and product disruption, mostly due to larger external environmental factors, such as the government of the DRC changing its exoneration and product registration policies. Nonetheless, they have largely mitigated the negative impact of these stock-outs and are on track for product distribution. PSI/ASF has been the leader in condom social marketing in the DRC for over 25 years and this reflected in the fact that its brand name, Prudence, has now become synonymous for condom, much like “Bic” is for pen. This has put it at a disadvantage during the recent campaign by OK, a new entrant in the market place. However, PSI/ASF believes that the unethical marketing practices of OK (telling vendors that Prudence was dead during a period of stock out of male condoms, etc), its expensive costs and old￾fashioned marketing practices do not make it a long-term threat, once the novelty factor wears off. Even if it stays on the market, PSI’s analysis indicates that the market is far from saturated and that there is plenty of room for two brands because the goal is to get as many people as possible buying and using condoms. Nonetheless, they are planning to invigorate their marketing campaign when they introduce the new product Prudence Sensuel in January 2013. They implement activities based on known elements of a clear Theory of Change that guides their strategy. They use appropriate channels for behavior change communication but have not kept pace with the technological changes present in the DRC, particularly the rapid increase in cell phone penetration and the improving internet network. Any future project must be sure to embrace these technologies. They have correctly identified two future opportunities, for Emergency Contraceptives and PostPartum IUD (PPIUD) and are beginning the foundational work necessary for these products to be promoted. They have also made the correct business decision to drop the sales of Clean Delivery Kits (CDKs) because there was no way to optimize profit and the kits did not address some of the determinants of maternal mortality present in the DRC, such as limited access to caesarean section births. They have expanded the distribution networks they use and have gathered more private clinics into their networks. In particular, the institutional development work with local NGOs is positive. The nine NGOs which have already received assistance from PSI/ASF have been able to improve their capacity in strategic design and project management. They have also been able to create cadres of peer educators. Overall, PSI/ASF is appreciated by all the partners it works with, including the Government of the DRC (GDRC), other USAID-funded projects and other multi-lateral groups, such as UNICEF. PSI/ASF has been responsive to USAID and are implementing a rural strategy which supports the overall Health Office objectives. However, the evaluation team was unable to definitively answer whether this 2 investment has been worth it as it has only just begun. One issue raised by partners and which has not been addressed is the low level of disposable income in the rural economy and how this with affect PSI’s ability to increase sales. PSI/ASF was not contractually obligated to work with either Youth or on Gender but they have done both during the course of the project. For gender, their research echoes many of the findings that are in the recent USAID gender report, including the fact that it is very important to include men in the decision making on using contraceptives. For Youth, they have segmented the market to include a young cohort of 15-17, which they are specifically targeting. This makes eminent programmatic sense, given that according to the 2010 Report on Progress toward the Millennium Development Goals one in five girls under the age of 15 has a child and the fact that one of the GDRC’s overarching objectives is to have a generation free of HIV/AIDS. PSI/ASF has also contributed to the sustainability of USAID’s investment by leveraging funds from other donors and investing in the capacity development of their local counterpart NGO, L’Association pour Santé Familiale (ASF). At this juncture, ASF believes it would pass any administrative and finance assessment conducted by the USAID Office of Financial Management and that they are ready to submit grant proposals on their own. If they are successful, the next step would be to renegotiate the working relationship ASF enjoys with PSI. Among the conclusions that the team came to are the following:  PSI/ASF successfully integrates its development Theory of Change into the implementation of its project and this allows it to better track impact from their interventions.  PSI/ASF is doing a good job and will meet all but three of the project objectives but they could be doing a more dynamic marketing job given their comparative advantage of market knowledge and experience.  Because they have been a market leader for so long, PSI/ASF has stagnated a little in their creativity and approach to marketing and has not exploited trends in technology for information sharing.  PSI/ASF is distributing a good gamut of products that respond to identified health needs, if they add PPIUD and emergency contraceptive, they will have a very comprehensive product range. Given that the evaluation slipped by a year, that PSI/ASF will have only nine months of program implementation post-evaluation and that they have had their funding cut by USAID, it will be difficult for PSI/ASF to achieve an extensive “mid-course correction” but there are still improvement that can be made during this remaining contract time. To this end, the evaluation team has proposed a series of recommendations for both USAID and PSI. Recommendations that can be implemented in the remaining nine months of program activity include:  PSI/ASF should continue its work with PNSR to change the national reproductive health policies on PPIUD; having gotten the permission of the PNSR to try this method delivery in five clinics in Kinshasa, it will be important to build momentum. Given that there is almost a 70% facility based birth rate in the DRC, this represents a significant opportunity to promote PPIUD. The PPIUD allows clients to leave the facility with an effective method that does not affect breastfeeding and can be long- or short-acting based on clients’ needs. In addition, PPIUDs are cost-effective for both the client and the health system: for clients, there is no need to return to the clinic 3 multiple times for FP; for providers, the process requires only minutes to perform post-delivery and needs fewer instruments than an interval insertion. When breastfeeding women use PPIUDs, they also do not need to “transition” to another method when they begin complementary feedings of their babies. Policy changes will be focused on training methodologies and accepting insertion in the postpartum period.  USAID and PSI/ASF should pilot social marketing approach for Emergency Contraceptives, targeting young women not in union. This will support behavior change away from using illegal and dangerous products to prevent contraception.  Based on the past success PSI/ASF enjoyed from working closely with ProVIC, they need to continue to link demand creation for VTC with immediate testing so as not to lose clients when they are referred to other testing sites and should reach out to all partners, including the Global Fund to make this happen.  PSI/ASF should focus its Prudence Sensuel campaign to respond to market competition and recapture those consumers who have gone to using the Ok brand of condom.  PSI/ASF should rethink its placement of billboards as well as its post-campaign management of billboards to avoid having dirty, torn and defaced billboards associated with its brands.  PSI/ASF should address provider bias in prescribing to young girls and vendor behavior around selling condoms to young men.  Recommendations for issues to consider during the follow-on design include:  Placing an emphasis on the marketing of long term methods needed to address short birth intervals and higher fecundity. It is vital to include long term methods in all marketing efforts to avoid undue reliance on pills and injectables, which have an appropriate place in the gamut of methods but which do not necessarily best serve older women who have completed their family.  PSI/ASF and USAID need to engage in policy dialogue with the government on removing the limitations of advertising branded family planning methods, once done the Confiance campaigns could be revitalized.  Future programming but support communication and behavior change efforts that allow project staff to capitalize on the growing use of smart phones and mobile phones within the DRC.  USAID should ensure that all future reproductive health and maternal health collaborate with government on a renewed program for FP education/awareness in girls under age 15.  Given that one in five girls under 15 is a mother, USAID should review the messaging around youth and abstinence and increase efforts for improving campaign relevance to the DRC socio￾cultural context.  USAID should drop its insistence that ASF’s institutional viability must be divorced from ongoing liaison with PSI and not be overly focused on the inter-institution arrangements.  USAID should pilot providing grant funds directly to ASF as the prime recipient, providing they submit a proposal that is responsive to USAID criteria. 4  USAID should determine if it wants to continue funding the rural strategy given the changing urbanization of the DRC and the challenges that need to be addressed within the existing large cities.  If the rural strategy is maintained, and PSI wins the follow contract, PSI/ASF should evaluate whether collaboration with road-building projects would allow it to increase market penetration in the rural sector and mitigate against some of the enormous transportation issues they have currently identified. I. INTRODUCTION The Advance Social Marketing for Health in Democratic Republic of the Congo was awarded to PSI, with its partners ASF, QED, Social Impact, and Hope Consulting in September, 2009. The award was for four years, from October 1, 2009 to September 30, 2013 and at an expected level of 23 million dollars but in the last year of the project, they have received an approximate 75% reduction in the PEPFAR funds they had anticipated. The evaluation was conducted from September 1, 2012 to September 29, 2012 and looked at the first three years of project implementation. The program has four main objectives, which are as follows: 1. Increase the supply and diversity of health products and services that are to be distributed and delivered through the private sector, in conjunction with the public sector, for disease prevention and control as well as integrated health service delivery. 2. Increase the awareness and demand for health products and services to emphasize prevention of childhood illnesses, unintended and unsafe pregnancies, HIV infections and STIs, and to build an informed, sustainable consumer base. 3. Develop and or enhance the ability of the commercial/private sector entities to socially market health products and services including behavior change communication activities. 4. Integrate service delivery and other activities, emphasizing prevention, at national, provincial, district, facility, and community levels through joint planning with the GRDC, other United States Government (USG), and non-USG partners. To achieve these objectives, PSI uses a sophisticated theory of change model that was revised in 2004 and has adapted elements from existing frameworks. The overarching theory looks at determinants for health status, risk reducing behavior and quality of life, under which PSI uses an algorithm on Opportunity, Ability, and Motivation (OAM). “These three elements were theorized as being the summary determinants of behavior change, which education, marketing and law were, to varying degrees, capable of influencing.” Within these three elements, PSI has adapted sixteen “bubbles”, all of which have an effect. Opportunity encapsulates availability, brand appeal, brand attributes, quality of care and social norms. Ability includes knowledge, social support and self￾efficacy. Motivation includes attitudes, intention, subjective norms, locus of control, threats, belief, outcome expectations and willingness to pay. The foundation piece in their Theory of Change is social marketing and its four “P”s, product, place, price and promotion. 5 A. STATEMENT OF WORK The USAID Mission in the DRC developed a statement of work for the evaluation, in consultation with regional USAID offices and with input from Washington. However, because the evaluation slipped a year,1 the Statement of Work was revised and agreed upon during the first week of the evaluation September 2, 2012. The original SOW and the full revision are contained in Annex D. The evaluation questions which were revised and then approved are as follows: 1. To what extent has PSI/ASF increased the supply and diversity of health products and services distributed and delivered through the private sector, in conjunction with the public sector, for disease prevention and control as well as integrated health service delivery? 2. To what extent has PSI/ASF increased awareness of and demand for health products and services to emphasize prevention of childhood illness, unintended and unsafe pregnancies, HIV infection and STIs, and to build an informed, sustainable consumer base? 3. To what extent has PSI/ASF enhanced the capacities of commercial and private sector entities and local NGOs to socially market health products and services? 4. To what extent has PSI/ASF used appropriate approaches for behavior change communication activities that further reinforce the increased demand for socially market health products and services? 5. To what extent has PSI/ASF integrated service delivery and other activities, emphasizing prevention, at national, provincial, district, facility, and community levels? 6. To what extent has PSI/ASF used joint planning and coordination with the GDRC, other United States Government (USG), and non-USG partners to achieve the desired integration of activities? 7. How effective has been the rural expansion strategy to date? Is it worth continuing to fund? What are the lessons learned? 8. To what extent has the project been able to integrate the gender perspective within its activities and which of these approaches and activities will be useful to continue in the future as support to the overall USAID gender objectives? An additional task was to assess the institutional capacity of ASF to determine whether it could take grant funds in the near future as proposed in the USAID Forward strategy. B. EVALUATION DESIGN AND METHODOLOGY According to the original statement of work, the Mission’s point of view on the evaluation was: “this is a formative performance evaluation whose purpose will be to clarify the project’s strengths and weaknesses and identify solutions and recommendations to better equip the project to achieve its targets. When possible, comparison will be made with any data from the final evaluation of the previous social marketing project in a bid to establish areas of improvement. Due to the lack of baseline data, the evaluation will use a “simple one-shot design” – looking at a group or intervention at one point in time during or after an intervention.” However, because the timing slipped for the review, the evaluation 1 The review was planned to be done by a consortium partner, QED. They had to drop out because of a conflict of interest and then the review was further delayed as the Mission sought a team leader who met their criteria. Thus an activity that should have been completed by September 2011, was only commenced in September 2012. 6 focused more on future directions then mid-course corrections. The work of the evaluation was done by a four person team, including an internationally recruited team leader, two national experts in social surveys and evaluation and a USAID Washington colleague with knowledge of the distribution issues and the DRC. During the first week of the evaluation, the methodology and questions were reviewed and revised. The evaluation used a mixed methodological approach (quantitative and qualitative). The following data collection tools and data resources were used:  In-depth interviews with key informants, including partners, government counterparts at the national and provincial level, other USG program staff and multi-lateral donors.  Focus group interviews with: men aged 30 to 50, women aged 20-40 with at least one under five child at home, female commercial sex workers2, young women between the ages of 15-19, young men aged 15-19 and product distributors from both the urban and rural setting and at different points in the commercial distribution changed. There was one focus group with the clients of commercial sex workers and one focus group of youth that were also peer educators before the team was able to refine the criteria for the groups. All told, there were approximately 300 informants.  Existing quantitative surveys, including the 2007 DHS and the 2010 MICS.  Project documents as well as other strategy documents and research findings funded by bilateral or multilateral donors.  Consultation with ASF staff to further understand some of the data points and to clarify items that had been reported in the three years worth of quarterly and annual reports. The team developed the focus group guides; they were designed to prompt discussion that would yield data to help answer the key questions. The tools were piloted during three focus group discussions in Kinshasa and then revised, the final tools are in Annex C. Because of the security conditions in the East, Ebola virus outbreaks in the North, and internal USAID security regulations limiting air travel to very few flights, the team was unable to get to all the project provinces. The team was able to go four of the provinces of the project: Bas Congo, Kasai Oriental, Kinshasa and Katanga. In the provinces, capital cities, secondary urban sites and limited rural sites (in Katanga only) were included. The team started working on September 2nd and field work began on the 12th of September. Field work was from the 12th to 18th, and again from the 23 to 25th. A debriefing was done with the USAID team in on the 26th of September and with the ASF team in the afternoon. The draft final report was submitted October 10th, 2012. There were limitations and challenges to getting the field work done. Because of an Ebola virus outbreak in the Northeast, on-going security challenges in the East (such as prohibition by the local government to have small group meetings because of security and PSI’s concerns about challenges from rebels on having outsiders, particularly foreigners, present and grenades being launched in Goma) and because of USAID internal travel regulations, the number of field sites available was limited. Time constraints also limited the number of visits to the rural sector so the rural strategy was not adequately assessed. In addition, the team leader developed pneumonia and was unable to travel as extensively as she wished, all of which limited the sites the team was able to visit. Other minor challenges included 2 The focus groups of commercial sex workers were mostly adult women. However in Mbuji-Mai, two of the participants were under the age of 15 and in Matadi, one group had relatively older women. 7 getting all the commercial sex workers together, translating from one of the four national languages into French and the administrative logistics of getting the national consultants hired quickly. Overall though these challenges were mitigated and the team does not think the quality of the evaluation was compromised. II. FINDINGS A. THEORY OF CHANGE To one extent or another, all of the elements outlined in the “Theory of Change” by PSI are used in their work. They drive how PSI and ASF work to increase sales points, invest in behavior change communication and integrate preventive services into the country framework. In their Tracking Results Continuously (TRaC) research, they use the sixteen bubbles to analyze research results. For example, in the study on the knowledge and utilization of water treatment products in South Kivu among women with children less than five years old, they stated findings such as:  Knowledge: Women who have a good awareness on causes of diarrhea are seven times more likely to use PUR.  Social Norms: Utilization of the product to treat water was increased among women who had previously used PUR.  Social Support: Women who had the encouragement of their husbands and friends are more inclined to use the product PUR. Other examples include looking at the foundational theory aspects of price and promotion. PSI/ASF believes that consumers would be willing to pay an increased amount for male and female condoms and is going to do pricing studies October to December 2012 to get the evidence to support this. They are also evaluating subjective norms and social support around the messaging of abstinence for very young (12-15 year old) sexually active adolescents. PSI/ASF has mostly focused on education and marketing to obtain desired outcomes, leaving aside policy and law, except they are engaging on Post-Partum Inter-Uterine Device (PPIUD) policy and revising the antiquated family planning law. There are elements within the GDRC legislative context that hamper how PSI/ASF is able to do its marketing and optimize financial benefit. For example, PSI/ASF is limited in how it can market contraceptives because of the GDRC policy that only allows family planning products to be promoted generically. In addition, because of extreme national poverty, PSI/ASF has found that some of its commercial products, such as the Clean Delivery Kits, are undercut by free donations from multi-lateral organizations, making it difficult to sell product. Nonetheless, they have succeeded in almost all their objectives. Table I. captures progress to September 30, 2012 for all the project objectives. 8 Table I. Summary of Progress to Date Compared to Life of Project Expected Results Result Planned LOP Achieved To 9/30/12 Percent Achieved Comments Objective One Male Condoms 107,000,000 76,768,950 74% Likely to be achieved with introduction of new Prudence Female Condoms 3,400,000 2,593,116 77% Likely to be achieved COC 3,740,000 2,092,474 61% Possibly to be achieved POP 660,000 10,507 2% Two year stock out DMPA 750,000 558,533 76% Likely to be achieved IUD 10,250 8,454 81% Likely to be achieved Cycle Beads 22,200 26,001 118% Exceeded Implants 6,300 5,139 82% Likely to be achieved Deliverans 50,000 44,308 96% Will achieve target but because not commercially feasible, product will be cancelled Aquatabs 7,250,000 15,438,698 213% Exceeded PUR 7,000,000 8,767,567 125% Exceeded Diarrhea Treatment Kits 700,000 0 Product expected Jan 2013 Objective Two Number of people reached during HIV/AIDS activities who are oriented to VCT site 26,933 31,854 118% Exceeded Number of individuals reached with individual and/or small group preventive interventions primarily focused on abstinence and/or being faithful that are based on evidence 59,096 31,616 53% Unlikely to be achieved, abstinence and fidelity messages very socio-cultural dependent and not yet resonating with sexually active youth Number of MARP reached with individual and/or small group level interventions that are based on evidence and/or meet the minimum standards 48,137 58,847 118% Exceeded Number of targeted condom service outlets 7,152 7,152 100% Achieved Number of individuals who participated in communitywide event focused on HIV/AIDS 900,000 405,093 45% Last quarter quadrupled numbers reached, project anticipates achieving this Number of media outlets including HIV/AIDS messages in their program 48 39 81% On Track Number of media broadcasts that promote responsible behavior 24,390 26,912 100% Target increased and achieved No. of peer educators who successfully completed an in￾service training program 665 665 100 Target was increased 10% from original contract, achieved FP service delivery points (pharmacies and clinics) added to the Confiance FP network 204 199 98 Will be achieved or exceeded USG-assisted service delivery points experiencing stock￾outs of specific tracer drugs 45 Number of people reached during outreach activities promoting the use of water purifier products 800,000 926,153 116 Exceeded Number of people reached during outreach activities promoting the use of oral-rehydration sachets to treat diarrhea 125,000 0 0 Demand creation delayed until product on market Number of service delivery points social marketing delivery kits 400 481 120 Exceeded Percentage of service delivery points reporting stockouts of ORS/zinc tablets at any time 60% 0 0 Product not yet on the market, expected January 2013 Objective Three Number of socially marketed health products or services transitioned to the private sector 1 0 0 Product was not commercially viable Number of trained/refreshed private sector distributors, NGOs, associations and community health workers trained in social marketing and/or BCC techniques 20 19 95 On Track, will be achieved Objective Four Number of external technical/coordination meetings attended at national/provincial/district levels with stakeholders 373 297 80 On Track, will be achieved 9 B. SUPPLY AND DIVERSITY OF HEALTH PRODUCTS AND SERVICES DISTRIBUTED AND DELIVERED Overall, the evaluation found that PSI/ASF has very successfully increased the supply, distribution and diversity of products during the project. They have brought four new products, Jadelle, CDKs, Aquatab and PUR to the market and are expected to bring a fifth product ORS/Zinc, and a sixth product Prudence Sensuel to the market in January 2013. They have also almost doubled the number of clinics where Confiance products are available. PSI/ASF used innovative distribution techniques to enlarge the sales network to include hair salons for female condoms. However, during the evaluation focus groups, most women did not like the idea of being able to buy products at the hair dresser as they thought it demeaned the value of the product. Many of the focus group members indicated that outside of pharmacies, the only reasonable commercial venues to purchase condoms were bars and hotels, as that is where there was immediate need of product. During field work, the evaluation team found that bars and hotels were able to command higher prices for the product, rising to as much as 500 Congolese Francs for Prudence homme, compared to the usual 50 to 100 FC. The distribution system used by PSI/ASF capitalizes on existing commercial systems within the DRC. All products are first received and stored centrally in the Kinshasa PSI/ASF warehouse. Each of the eight provinces has a PSI/ASF warehouse managed by the provincial PSI/ASF staff, who monitor stock levels and report to the central warehouse. At the next level of the supply chain are the pre-approved network of pharmaceutical wholesalers who place orders with the provincial level warehouses. From the start of the task order, PSI/ASF has increased the number of pharmaceutical wholesalers from 10 in Kinshasa and Katanga, to a total of 70 wholesalers in the eight project provinces. Although there are systems in place with standard operating procedures to ensure the quality of product, the evaluation team found that PSI’s control over the stock environment is not able to be implemented throughout the distribution network. Provincial warehouses keep stock in air￾conditioned rooms and according to international norms. The wholesalers try to do this but frequently have electricity disruptions and have open enough storage systems that dust and humidity are present. At the individual pharmacy level, there is very little consistent stock management. During field visits, the team found condoms and other Confiance products stored on the floor in plastic bags, on shelves near open windows and in wooden crates. If product stays long under the conditions, it is at risk for deterioration and these conditions at the sales points might be a contributing factor to consumer’s complaints on the quality of condoms. i. Confiance Contraceptives The product coverage is extensive and there is now continuous availability of products at the central and provincial warehouses. The Confiance product line in particular exists in a strong network of partner clinics that consistently provide counseling on and access to the diversity of methods. 12 of 15 women interviewed in Mbuji-Mai said they preferred Confiance products over any other. The project has experienced challenges with both oral contraceptives. The first challenge concerns the Combination Oral Contraceptive (COC) that replaced the previous COC Duofem. Duofem stocked out at the provincial levels in March 2011 resulting in a period of paused distribution of stock for four months until Combination 3 was registered in July 2011. PSI/ASF rushed the product to market, without using the overbranding of Confiance and consumers bought the product in much lower quantities, assuming it was 10 an inferior product. Once overbranding was available, COC sales shot up immediately and were almost three times higher for the third quarter than the previous quarter, increasing from 183,694 cycles to 512, 213 cycles. The second challenge has been the registration of Microlut, the new progesterone only pill. Delays by the manufacturer in registering the product coupled with the long review process of the product’s technical dossier by the 3e Direction of the Ministry of Health (MOH) prevented the product from being registered and distributed almost throughout the life of the project, making this one objective that they will be unable to achieve. The 3e Direction restructured in June of 2011, shifting the responsibility of reviewing product registration applications from one individual to a committee that meets only four times a year, so there is no guarantee that this sort of challenge will be quickly mitigated. DMPA is the second most popular method of all Confiance Contraceptives. The PSI/ASF TRaC Studies showed use rates of 4% in Equateur, 6.9% in North Kivu, 2.3% in Katanga and 1.6% in Kinshasa, of which almost all was Confiance Depo-Provera. This use has increased, in the first three months of the project, PSI distributed 21,140 vials and in the last three months (Q11) PSI distributed 66,033 vials, an increase of a 212.4%. This significant rise in use happened despite a global recall of Pfizer’s Depo-Provera, in part because women cite the product as affordable, ranging from 1000 to 1200 Franc for the three month dose. Despite these challenges, Graph I on the next page shows that compared to the previous project, the sales of COCs and DMPA are steadily increasing and contributing significantly to the increase in Couple Years of Protection. In the three years of this current project, PSI/ASF has managed to exceed total sales done in the five years of the previous project. The evaluation team finds this is due to the better network management and expansion of points of sale. Graph I. Distribution of Oral Contraceptives and DMPA Data source: PSI/ASF records Cyclebeads, another method in the Confiance brand, have been sold at rates that exceed the target for the project. Many of the service providers interviewed thought of the Cyclebeads as the perfect method for girls under the age of 15 who were sexually active. They stated it was the only method they felt comfortable prescribing but given that many young women have irregular menstrual patterns during their first years of menarche, it is not clinically evident that this is the best method for them. The law 0 500,000 1,000,000 1,500,000 2,000,000 2,500,000 3,000,000 3,500,000 4,000,000 COC POP DMPA Total 2004‐ 2009 Total to Date (3 years) Target Distribution of Oral Contraceptives and DMPA 11 in the DRC does not allow long term methods to be used in girls under the age of 15 but it does allow for COCs to be used so clinicians are limiting legal options because of bias. The Confiance network also includes implants and IUDS. Jadelle is the newly licensed implant for the DRC and implant insertion training to all USAID funded service delivery providers was completed by the Dutch SALIN project in the first quarter of 2011, slightly later than planned. PSI/ASF supplied Jadelle as soon as the product was registered during the following quarter 2011. In the first nine months of distribution, they sold 1,299 and in the second nine months they sold 3,532. While this is positive, from focus groups it is evident that knowledge of this new method has not reached many women in comparison to the other well-known modern contraceptive methods. Use of the implant has also been hampered by a stock out of the insertion trocar necessary to deliver the implant below the skin. For IUDs, overall national sales levels are increasing slightly every year. However, in an intra-province analysis over time, there is no consistent trend, as shown in graph two. Graph II. Province Sales of IUDs, by quarter for Life of Project to Date Data sources: ASF distribution data PSI/ASF is working to increase the formal support for PPIUDs in the DRC. They arranged a study tour to Zambia for key Ministerial counterparts and PSI/ASF staff are engaged in helping the MOH change national policy on PPIUDs. Five clinics in Kinshasa have had training in how to insert an IUD postpartum and the project anticipates an uptick in utilization. ii. Prudence Male and Female Condoms After nearly 25 tears of marketing, the Prudence brand is so well known that it now synonymous for “capote” and “preservative” similar to referring to all disposable ink pens as a “Bic”. Because condoms 0 200 400 600 800 1,000 1,200 Q1 Q2 Q3 Q4 Q5 Q6 Q7 Q8 Q9 Q10 Q11 Q12 Province Sales of IUDs, by quarter for Life of Project to Date 12 are the preferred modern contraceptive method of choice in the DRC3, there is a large market potential. The Prudence male condom and Prudence Femme female condom are the most widely recognized brand of condoms in the PSI/ASF targeted provinces, although in the Bas Congo, where rival brand Ok has invested significant resources, Prudence is under threat. Repeatedly in all of the target focus groups (men, women, youth and sex workers), similar messages were heard. They included:  Ok is ribbed and provides more pleasure  Ok is not over lubricated and so feels better  Ok has more per packet  Ok does not tear so it provides better protection PSI/ASF staff believes that there is a novelty factor at play and that while clients are buying Ok to try them out, they will return to Prudence. Ok has also engaged in underhanded marketing techniques, for example taking advantage of a Prudence male condom stock-out to declare “Prudence is dead and there are no more Prudence.” In Kasai, they are providing commercial sex workers with large cartons of product and asking that they use them with all their clients. USAID also believes that the marketing strategy for Ok is not sustainable, because their stock is not duty free and therefore more expensive. When asked by a client for a condom but not a specific brand, many pharmacists stated that they generally give a Prudence condom without questioning the client’s preference. Despite this practice, vendors also said that clients sometimes respond by saying “Not that Prudence, Ok prudence.” This extraordinary success of the Prudence is to be recognized but so is the need for continually renewed marketing strategies. As an ASF province coordinator said “Just because Coca-Cola is number one in the DRC, they don’t ever stop marketing.” Wholesalers in the Bas Congo reported that their stock of Prudence was taking longer to move than before and they needed help it getting it out, they attribute this to the uptake in Ok condom sales. There will be new scented Prudence condoms on the market soon (Prudence Sensuel) and PSI/ASF is anticipating renewed levels of sales because of the new product. 3 DHS 2007 Two minor girls (ages 14 and 15) who were commercial sex works in Mbuji‐May spoke of getting OK from clients. They said “Usually, it is us that bring Prudence to the clients when we have sex. Put Indo‐ Pakistani clients in the mining zones say that Prudence is easily torn and they pressure us to use OK. After a visit, they give us several cartons of OK, they have big boxes. For us, it is a good brand because it doesn’t have too much oil and it has a nice odor. Clients don’t know we have already had sex because the banana smell hides the fact there have been previous clients. 13 Graph III: Distribution of Prudence Male Condoms by Year Despite its wide distribution network in-country, barriers to access, particularly for youth and in rural areas, continue to be evident. Focus groups participants and interviews with pharmacies in the Katanga Province cited the reluctance to sell condoms to youth. Some youth report they receive a lecture on abstinence first before they are able to procure condoms. Youth added that they prefer not to buy condoms from older generations because of the implied disapproval of their sexual activity. In Likasi only one of 14 young women had ever heard of the female condom, although all 14 had heard of the male condom. 100% of the women, both the housewives and the young women, in all focus group said they would not want to purchase female condoms in the salons. Two primarily explanations were given for their preferences. The first is that they do not want the owners and the other women in the shops to believe they are having sexual relations or engaging in prostitution and tell others in the community. The second reason is that the women believe the quality of standards for storing the condoms are lower than in the pharmacy. One participant said “For me, condoms should primarily be sold in pharmacies and not everywhere in salons and boutiques where small children can see them, it is bad to find them everywhere. Pharmacies sell medications that can protect and since the condom can also protect, it should also be sold there.” (Matadi women’s focus group) Compared to other African countries, the DRC has a high procurement rate for female condoms. In the past 6 years, DRC has been the third largest recipient of donor funded female condoms (almost 11 0 2,000,000 4,000,000 6,000,000 8,000,000 10,000,000 12,000,000 14,000,000 Year 1 Year 2 Year 3 Distribution of Prudence Male Condoms by 14 million). Zimbabwe is the largest recipient, receiving 30.5 million and South Africa received 11.7 million. Of the 11 million that have come to the DRC, USAID delivered 8.5 million, the remainder were donated by UNFPA and UNDP. These three donors only cover about 20% of the country and we do not have any additional insight on procurement and use of female condoms in the remaining 80% of the DRC. 50% of the nine Commercial Sex Workers interviewed in Kinshasa used female condoms. PSI/ASF has not yet conducted market research to determine why this is so but some informed opinions include:  Some professional sex workers prefer to use the female condom as it can be put in place well before sexual relations and sometimes it can be used without the knowledge to the client, thus protecting them while at the same time avoiding negotiation with a client.  Some of the sales are now driven by use of the condom by men having sex with men (MSM), as the ring makes it more practical. In Bas Congo and Katanga, other creative alternative uses of female condoms emerged during the focus group discussions. Young women and professional sex workers buy the condoms to use the ring of the condom as a bracelet. Men have also been known to use the condom, specifically the lubricant, to polish shoes. PSI/ASF does not track the sales data by client so what percentage of the 2.7 million female condoms distributed to date are for the alternative uses is not clear. iii. Aqua Tabs and PUR Aqua Tabs and PUR essentially sell themselves. Access to potable water remains very limited in the DRC; the 2007 DHS found only 46% of households had access to an improved water source for potable water. In addition, in the rural areas, 59% of the households had to go more than 30 minutes to get clean water. Table Two: Distribution of Aqua Tabs and PUR over time by Province Distribution of Aqua Tabs and PUR over time by Province Year 1 Aquatabs Year 1 PUR Year 2 Aquatabs Year 2 PUR Year 3 Aquatabs Year 3 PUR Kinshasa 343,624 606,952 2,384,304 1,174,200 4,100,608 1,105,312 Katanga 635,692 313,400 571,526 702,720 279,520 268,092 Bas-Congo 69,632 76,217 95,136 50,724 238,144 159,360 South Kivu 647,820 532,640 765,552 784,196 1,031,557 347,040 North Kivu Province Oriental 0 0 239,040 149,388 144,800 80,640 Equateur 0 0 56,511 63,384 859,720 603,600 Kasai Occidental 169,933 69,170 257,920 154,210 236,448 231,360 Kasai Oriental 34,240 143,520 171,920 250,696 142,792 227,040 Maniema 0 0 124,792 90,965 0 0 Total 1,900,941 1,741,899 4,666,701 3,420,483 7,033,589 3,022,444 Another reason for high sales is that the visual demonstration of how the product works is very compelling, particularly for a low literacy rural society. In addition, the DRC continues to be wracked by cholera epidemics across the country so many people understand the link between good health and potable water and are willing to pay for it. Focus groups also confirmed the popularity of the product for these reasons. More Aquatabs than PUR have been distributed thus far; one explanation for this may 15 be due to the difference in clean water output. Aquatabs are able to purify twice as many liters of water than PUR and at a lower price per liter, which is of interest to consumers with limited money. iv. Clean Delivery Kits-Deliverans PSI/ASF used funds from the PSI Enterprise fund to start the procurement of Clean Delivery Kits (CDKs). The kits contain gauze, gloves, a sterile razor for cutting the umbilical cord and soap, among other small items. While providers in the PSI/ASF clinic network stated they had insisted that women purchase the kits prior to delivering, as it helped the providers manage scarce clinic supplies, overall there was not a consistent uptake of the product. Its distribution is shown in Graph Four, which includes all province data except data from Kinshasa which were anomalous, as the first year there were almost 16,000 kits distributed in Kinshasa and the second year there were just 79. Graph IV. CDKs Distributed by Year and Province, save Kinshasa The kits were to be transitioned to a private sector partner. However, Hope Consulting, one of the partners in the project consortium, did a spin-off analysis and found that the CDK did not meet the criteria of value, social impact or synergy necessary for spin-off and recommended that the kits not be considered as a viable spin-off. This decision was further supported when UNICEF distributed a significant number of kits for free in South Kivu, under-cutting what had been an important market for PSI. While this was a worthwhile experiment, the evaluation team agrees with the business decisions made and the current decision to no longer promote these kits once existing stock is finished. While this could be construed as a setback to USAID’s desire to promote safe delivery, it is actually consistent with the technical discussions written in the USAID BEST action plan of July, 2011. Given that most of maternal deaths that occur in the DRC are due to postpartum hemorrhage, which is compounded by poor clinical practices at the facility level, low rate of c-section and limited obstetric emergency drugs, the CDK was really only a band-aid. 0 500 1000 1500 2000 2500 3000 3500 4000 4500 5000 Year 1 Year 2 Year 3 CDKs Distributed by Year: All Provinces save Kinshasa 16 v. Other products The ORS and Zinc kits, known as Diarrhea Treatment Kits (DTKs) have not yet been imported. The procurement of the DTKs was delayed at the Washington level due to approval of the source/origin and pharmaceutical waiver in Quarter 5. This resulted in the original bid for the production to expire. Thus the bidding had to be reinitiated and product is estimated to arrive in country in January 2013. It can be expected to be distributed to provincial levels in early 2013. PSI/ASF proactively submitted the request for registration on behalf of the manufacturer and has been able to secure registration well in advance of the product arriving in country. They have conducted consumer preference tests and have preferred packaging, thus there is possibility that they will come within striking distance of meeting the overall project objective with this product. C. AWARENESS AND DEMAND FOR HEALTH PRODUCTS AND SERVICES The evaluation question for this section is: To what extent has PSI/ASF increased awareness of and demand for health products and services to emphasize prevention of childhood illness, unintended and unsafe pregnancies, HIV infection and STIs, and to build an informed, sustainable consumer base? PSI/ASF has certainly increased awareness, particularly for condoms. The focus groups found that almost everyone knew that condom usage helped prevent HIV/AIDS, however systematic usage is problematic. 8 out of 15 young girls in Kasumbalasa said you should use condoms for every sexual union but agreed that they did not do so. PSI/ASF has done a good job of creating demand for Voluntary Counseling and Testing (VCT) and distributes cards that show sites where the testing is available. However this referral process contributes to a loss of clients following through on the testing, as they perceive it as difficult to go back to another site. When PSI/ASF worked closed with ProVIC, they were able to get as many as 100 clients tested in a night, as the demand creation activities and the testing were done at the same time in the same venue. 8 out of 15 young girls in Kasumbalesa had already seen or heard messages on HIV/AIDS. They said they would prefer to get more information by television, youth conferences in schools, in pamphlets they could read at their leisure and also in church-organized activities. Demand for water purification products is very high for two reasons. One, the marketing techniques use visual aids that show how water turns clean and this is very striking for a low literate population as they don’t have to read anything to understand the message. Two, the DRC has suffered from frequent cholera epidemics so the messaging on the importance of safe water has been repeated from multiple donors and in multiple venues, saturating the market with campaigns and increasing awareness. Some insight into the demand for female condoms can also be gleaned from responses from the focus groups. One commercial sex worker had a concern: “I am scared to use them because some men can enter beside the vagina opening and if that is the case, then I can get a sexually transmitted disease” while another said “Me, when I go looking for men on the Angolan border, it’s the female condom that I use and I am at ease.” In terms of an educated consumer base that is sustainable, the team found that subjective rumors negatively impact the sustained use of certain methods. Participants in the focus groups who were already users of modern contraceptives still had concerns regarding their method choice and did not 17 have adequate access to venues that could dispel their concerns. Rumors that were repeated in the groups include that lubricants on the condoms were cancer-causing, that injectables made you sterile and that pills were “noxious”. The rumors were enough of a concern that some participants said they had stopped using the methods. i. TRaC Study Findings The 2007 DHS indicates that modern contraceptive usage is approximately 6.7% for all women and when stratified geographically, is higher in the urban areas. However, PSI/ASF did studies in four provinces in 2009 and found the demand much higher. The average usage of modern contraceptives was 20.5% among a total sample size of 6,049 women. The regional differences were marked though, in Katanga only 11% were using modern methods versus 24% in North Kivu. Women are more likely to use modern family planning if they have discussed it with their partners and women who believe their partners have a positive attitude are more likely to use modern family planning methods. These studies were also consistent in demonstrating that utilization of family planning and demand for family planning was greatly influenced by male attitudes to use and whether or not they approved of family planning. Five out of eight housewives in Kolwezi used contraceptives that they bought at the hospital after a consultation. Youth and truck drivers are more likely to use condoms if they believe their peers use condoms and if they could ensure that they use a condom correctly, then they were more likely to use condoms. Awareness of transmission modes of HIV/AIDS also increased the likelihood of use of condoms among youth, truck drivers and military. The studies also indicate the need for systematic usage of condoms, which is much harder to sustain than situational use. The demand for products that influence child health and mitigate diarrhea were affected by awareness. Women who had a positive perception of the effectiveness of PUR to make drinking water clean are more likely to have used the product than those who do not perceive its efficacy. Women who had used the product PUR had heightened perceptions on the severity of diarrheal disease among children under five years, as well as the frequency. They are therefore more consistently aware that many children have died of diarrhea in their environment and thus understand how the use of PUR could be beneficial. The results of all the TRaC studies were appropriately used by PSI/ASF to shape interventions and increase demand for products or healthier behavior. ii. Consumer Commentary from Focus Groups Use and knowledge of condoms by professional is remarkably high. 100% of the professional sex workers in Boma and Matadi confirmed the availability of condoms in pharmacies or directly from ASF peer educators. 100% of the professional sex workers keep their own personal stock in Matadi, and 12 out of 15 kept a stock in Boma. In Kowalezi, the majority of respondents either kept or bought their own condoms. In the focus groups, commercial sex workers were adamant that they have moved to using condoms with every client. Sex workers in Kasumbalesa said “For us, Prudence is like our merchandise, without Prudence, we don’t work. Even if clients propose giving us much more money than usual, no sex without a condom.” Male clients of commercial sex workers in Kasumbalesa said they used condoms with them but just 4 out of 8 said they used condoms with their wives and then only to space births. One of the participant responses from Matadi validates the appropriate channels for 18 access that PSI/ASF has created, “Me, I wish they were sold in more bars because if you are sitting with your girlfriend and drinking beer and you have the urge to have sex, you can buy them directly and then go.” Among young people interviewed, there was a limited understanding of other family planning methods besides condoms and a dangerous use of other drugs, such as Tetracycline and Cytotex, to prevent pregnancy. The young girls in Kazenze said “We have never heard of the Confiance brand. We put sugar in our vagina or drink salted water, aspirin and valium to avoid pregnancy. We know we should use condoms but we have sexual relations without them.” This same group knew where to get Prudence condoms but did not know how much they cost. There were also many basic questions, such as “Can I use a condom twice?” and “Is it even safer when I use two condoms at the same time?” The responses from men in Kinshasa supported the findings from the gender study and from PSI/ASF’s own work that men are vital as partners in family planning; “Wives should not use modern family planning methods without me knowing about it because the day I find out, there will be problems in the household and I would even consider getting a divorce;” and “Me, if my wife used those methods, I would think first of all that she is being unfaithful and that she is using them to prevent bringing a pregnancy into the household that is not mine.” D. APPROPRIATE APPROACHES FOR BEHAVIOR CHANGE COMMUNICATION The evaluation question for this section is “To what extent has PSI/ASF used appropriate approaches for behavior change communication activities that further reinforce the increased demand for socially market health products and services?” PSI/ASF behavior changes efforts are driven by their Theory of Change framework and assessed for their contributions to improving health and mitigating risky behavior, as well as reflecting social norms. They currently use all forms of mass media as well as a more personal approach of peer education and home visits. The following section discusses the findings. i. Mass Media PSI/ASF maintains that the use of billboards is appropriate and that after each new campaign, they get a bounce in sales. Recent visits to the field by the project director indicate that the tag line they are using “Vrai Djo” is actually what customers are asking for when they go into pharmacies in Kisangani. However, government partners question whether the placement of billboards is appropriate and wonder if there should be billboards placed within the more dense urban slum areas of Kinshasa and not just on the big boulevards. A second issue with billboards is that the current contract does not call for maintenance. Thus, if an owner of a billboard does not have a new advertisement to place, he lets the existing one stay. The result of this is that in the Bas Congo, and other project sites, there are billboards for Prudence products which are torn, dirty and defaced, as well as faded. This sends a subliminal message that the product associated with the ad might also not be of top quality. PSI/ASF also uses print media and has recently produced three copies of a magazine called 100% Jeune, even though their own market research indicated that in urban areas in the DRC, the young do not read. While the content of the magazine is appropriate and well done, and draws on PSI’s pan-Africa experience with a similar campaign in other countries, the impact is still not known. During focus 19 groups, very few of the young people had heard of the magazine and they also repeated the finding that they do not particularly use reading as a way to seek out information. Current distribution is 4,000 copies because of budget constraints and this does not reach a very large audience. TV and radio are also used although during the field visits, the evaluation team learned from women in the rural areas that they do not have much time to listen to the radio and they do not find it an effective medium for getting information. Mobile video has been successful for raising awareness for HIV/AIDS and clinics have also benefited from receiving DVDs they can screen. In the Bas Congo, two of the clinic sites complained that their DVDs were no longer functioning and that they had not received replacement materials. PSI/ASF has invested in two toll free numbers for the Ligne Verte hotline and they average approximately 3000 calls a month, the preponderance of which are male callers. The limitation of the medium is that it is only staffed during office hours and during the week day. PSI/ASF has not adequately exploited using the explosion of mobile phones as a channel of communication for family planning, prevention of diarrheal disease and HIV/AIDS. One telecommunications source (Budde Intelligence Reports) states that as of 2012, approximately 27% of the population has access to phones, which is more than triple when the project began. The World Bank Institute believes that by 2013, 47% of the DRC population will have access to a phone and 50% of the rural population will be in areas covered by cell phones. PSI did use short messaging services when they were marketing Insecticide Treated Nets (ITNs) so they have experience in the field but they have not used that experience during this project. They have also not examined whether or not cyber cafes and the expanded access to internet might be a way to reach the young in urban areas. ii. Peer Educators and Community Organizers During field visits, PSI/ASF received commendation from clients on the skills and outreach capabilities of their community organizers and their peer educators. Women in the Bas Congo groups felt that the community organizers were very supportive and they were pleased that the relationship included receiving the personal telephone numbers of the educators so they could call them with questions at any time. However, during the evaluation, the team also learned that PSI/ASF was reducing the number of community organizers because they are expensive and the project has been subjected to budget reductions. The Peer Educators stated they would like to have more visible branding to identify them with the project and wished that PSI/ASF would re-issue hats, vests and umbrellas. They felt this also helped them when they went into neighborhoods and stimulated interest and conversation. 20 E. ENHANCED CAPACITIES OF COMMERCIAL SECTOR, PRIVATE SECTOR ENTITIES AND LOCAL NON- GOVERNMENTAL ORGANIZATIONS The evaluation question for this section is: “To what extent has PSI/ASF enhanced the capacities of commercial and private sector entities and local NGOs to socially market health products and services?” i. Commercial Sector Wholesalers While PSI/ASF has a system in place in which they can respond to any questions or issues with wholesalers, basically wholesalers are indifferent to any intervention except providing a good product at a good price. Of the wholesalers interviewed, most were very pleased with the product line offered by PSI/ASF and thus felt they really did not need much other support. They have received guidance on the stock management of product and by and large, keep product in good condition and move it fairly fast. Other commercial sector entities include sales points that are pharmacies, hotels and bars and other venues, such as hair salons. It is at this point in the sales chain that there are continuing capacity development needs. Very few of the vendors in the small pharmacies could answer any technical questions about the products, putting them at a disadvantage when trying to get consumers to buy PSI/ASF products. They also did not respect the norms and standards for keeping stock and the evaluation team noted product on the floor, product in dusty bags and product kept in very hot conditions. ii. Private Sector Clinics PSI/ASF has almost doubled the number of private clinics that are in its networks. Of those visited during the evaluation, there was clear evidence of sound family planning counseling practices. The Tiarht Amendment language was posted on walls, as were posters showing all the available family planning methods. PSI/ASF has enhanced the capacity of these clinics as distribution points for the Confiance products by investing in clinical training (including leveraging funds from other donors) and providing regular supervision visits to support the staff. Some of the clinicians interviewed do not adhere to counseling on the full range of allowable methods when they counsel younger women. Instead of offering them the range of methods allowed by the government, they over-promote Cyclebeads as the only appropriate method. PSI/ASF is also reaching out to other private sector venues and has a letter of agreement, signed by the Ministry of Mines, that they can engage with mining companies to try and serve the reproductive health needs of their workforce. iii. NGO Partners Under the teaming agreement of the award contract, institutional development of local NGO partners was to be the bailiwick of Social Impact (SI). SI suffered a grievous personnel loss when their country representative was killed in an airplane accident. Nonetheless, they have completed the initial assessment of NGOs and they have conducted initial training in strategic planning and have provided some management support to the identified group of recipients. 21 One of the local NGOs in Katanga stated that its collaboration with ASF has “reinforced our expertise when we are in the field, we are comfortable with answering questions on HIV/AIDS, sexually transmitted diseases and also how to convince the population to have a more responsible behavior. “ While there is an appreciation of the relationship with ASF, it is not without some contention. For example, another one of the NGOs in Katanga cited the following difficulties within their relationship with ASF:  Hard to keep with up the constant demand to evaluate the impact of activities  Hard to meet ASF goals of having meetings with 100 commercial sex workers monthly because of government prohibitions on gatherings and also because they had no access to incentives to provide to the workers  Limited ability to renegotiate the contract demands within their relationship with ASF. A more global issue is that the NGO partners are hungry for more than just training. For example, Department Femme et Famille de L’Eglise du Christ au Congo (DFF) in the Bas Congo is one of the partner organizations being supported by SI. They were identified as a partner 18 months ago and they have had four meetings with SI. The last was an eight-day training on strategic planning, which was very participative and which showed them not only what they did well but what they could do to improve. Now, however, as they work on their action plan, they are alone and they have need for some additional support, yet they have not had a meeting with ASF Matadi office in five months. APROVEMAC, which just began to get support from ASF within the past six months, also feels equally bereft of follow up. They too very much appreciated the quality of training that ASF supplied for the training of peer educators but also wish there was more consistent and regular follow-up. They understand the principle is for them to develop sufficiently so they might be able to get independent donor funding but they don’t feel that with the current level of support, they will be able to achieve significant growth. SI has taken this into account and developed a proposal whereby there will be two mentors per province that will do the follow up and further support for strategic planning. However, this has not yet been implemented although they plan to roll out the mentoring in January, 2013. F. INTEGRATED SERVICES AT NATIONAL, PROVINCIAL, DISTRICT AND FACILITY LEVEL The evaluation team was asked to determine “To what extent has PSI/ASF integrated service delivery and other activities, emphasizing prevention, at national, provincial, district, facility, and community levels?” The evaluation team found this question problematic because of the emphasis on service delivery, which is only a small part of the overall PSI/ASF project and approach. However, the team noted the following:  PSI/ASF integrated its demand creation for HIV testing services with ProVIC’s counseling and testing activities. This enabled them to reach significant numbers of people. However in Kasumbalesa, men in the focus groups said they get their counseling and testing from Medecines Sans Frontier. 22  Counseling on family planning has been integrated in the prenatal care delivered at private clinics, with follow-up plans in place for women to select a method once they deliver.  In doing community outreach, PSI/ASF integrates family planning and HIV/AIDS messages when they discuss condom use.  PSI/ASF has integrated all of its activities within the larger national and provincial strategies and action plans and in many cases, has a discrete set of actions that they are expected to implement to contribute to achieving the overall national health and development goals.  ProVIC and PSI have a joint work plan which integrates the individual approaches of each project.  PSI is integrated into many of the national working groups and is able to coordinate its program efforts for maximum benefit. G. JOINT PLANNING AND COORDINATION WITH THE GOVERNMENT OF THE DEMOCRATIC REPUBLIC OF THE CONGO, UNITED STATES GOVERNMENT PARTNERS AND OTHER PARTNERS The evaluation question was: “To what extent has PSI/ASF used joint planning and coordination with the GDRC, other USG, and non-USG partners to achieve the desired integration of activities?” PSI/ASF receives high marks from all its partner entities for transparent and willing collaboration and coordination. PSI/ASF works closely with the government, at both the national and regional level, other USAID funded projects, other US government entities, notably the Department of Defense, and with a multitude of non-USG partners, that are both bilateral donors to the DRC and multilateral agencies, such as UNICEF. i. Government Partners The national level partners who were interviewed feel that PSI/ASF supports the national action plan for health and that they are very mindful of working within state norms. In Katanga, the MIP said “Once there were points of contention between ASF and the government because of the social marketing strategy. But now PSI/ASF is much closer to the standards and guidelines of the Ministry. From a vertical intervention strategy in the health zones, they have moved to a more holistic strategy where the commercialization of ASF products supports our goals.” They did state that they differentiate between what PSI/ASF is able to contribute in terms of technical support and development of materials or program strategies versus providing actual fiscal resources. The Programme National pour la Lutte contre les Maladies Diarrhetique(PNLMD) was clear in stating that they needed to know how much money was being contributed to their national action plan but that they held USAID itself responsible for this. At the provincial level, all of the Médecin Chef de Zone (MCZ) interviewed knew PSI/ASF regional staff well and felt that they were an integral part of their team. ASF is viewed to be a significant contribution to reaching national goals, including creating a generation free of HIV/AIDS and improving the management of diarrheal disease. 23 PNLS made one comment that since PSI/ASF is a private organization that sells product- even if the sales of the product goes back into buying additional products, they are still a private, for-profit4 organization and that is one aspect that sets them apart from other NGOs and the ability to work with the government. An example of this is that the government made the decision to do free distribution of ITNs in order to reach the MDGs and so PSI/ASF had to stop the marketing of the Serena brand ITNs, even though they were very successful. Government counterparts in Katanga indicated that they should play a role in helping ASF/PSI get their products to the market place. The Médecin Inspecteur de Province (MIP) said “Government should exempt products ASF / PSI. This is a major support which will enable ASF / PSI effectively meet its mission to improve the health of the population as the level of coverage for contraception and purifying water is still very low.” ii. Other USG partners PROSANI spoke highly of their collaboration with PSI, citing one example of PSI’s response to a cholera epidemic that had been very useful. They said that they enjoy excellent institutional collaboration but there is a need to make sure meetings happen on a regular basis and are not ad hoc. PROSANI mentioned that travel regulations, including items such as per diem, need to be harmonized between all USG partners, including PSI. They also questioned PSI/ASF provincial staffing levels if they were going to continue to expand their rural strategy and indicated they thought PSI/ASF should have more people in the field. This was of interest since the evaluation team found that PSI/ASF has actually closed some provincial offices and consolidated staff because of budget cuts. ProVIC has had a good working relationship with PSI/ASF and in the case of a combined approach to VCT, where PSI/ASF did the “animation” and ProVIC did the immediate counseling and testing, they were able to reach significant numbers of Most At Risk Populations (MARPS). In Kisangani, PSI/ASF and ProVIC actually have a joint action plan that has been guided by USAID, however they don’t do this any of the other project sites. PSI/ASF supports the Department of Defense program through supplying condoms and is currently working on communication materials and all the efforts to date have been collaborative and fruitful. iii. Non-USG partners UNICEF’s interaction with PSI/ASF has been focused within the Water and Sanitation for Health (WASH) sector and the distribution of Aqua Tabs and PUR. UNICEF staff found that PSI/ASF was very responsive and able to move product quickly when the need arose because of cholera epidemics. They were able to use their logistics and distribution network to get adequate stocks were UNICEF needed them. UNICEF remarked that PSI/ASF always adhered to their social marketing principles and requested payment for product but that this was not a deterrent to collaboration. 4 This is a quote from the government about their perception of PSI as a private organization, legally PSI is a registered non-profit under US laws and regulations. 24 H. THE EFFECTIVENESS OF THE RURAL STRATEGY Because access to health products in the rural area remains critically low and erratic, USAID asked that PSI/ASF become involved in getting product out the priority health zones. At this junction, USAID wishes to know how effective has been the rural expansion strategy to date and is it worth continuing to fund? The evaluation team found that it is too soon to tell if the strategy is working as it has only been piloted in a few areas. Among the issues raised during this evaluation were the following:  One USG partner was concerned whether PSI’s strategy adequately took into consideration the low level of monetization in the rural economy. For example, they suggested that if PSI/ASF could accept three cups of rice or payments in installments for Cyclebeads, then the rural strategy might work better.  PSI/ASF correctly identified bicycle merchants “pedaleurs” as the appropriate venue for distribution of products. But in Kanzenze, one of the sites where the rural strategy has been rolled out, the team found that condoms were not being sold door to door, like the Aquatabs and PUR, but were being sold out of the house of the middle man and he was refusing to sell condoms to young people.  The pedaleurs have an irregular schedule because of the enormous challenges of the road system and sometimes it can take two months to complete a circuit. This is problematic for those clients who only have the funds to purchase small quantities of supplies that may not last until the pedaleur returns.  The pedaleurs wanted more corporate identification because of their involvement in the program. They wanted items like hats and t-shirts or at least stickers to identify them.  Current data on the growth trends for the DRC shows an increasing urbanization and an increasing need for interventions in cities such as Kinshasa,, thus limiting progress that can be made in national health indicators from the rural zones. One of the community agents in Kanzenze was frustrated and said “Here in Kanzenze, all we do is demonstrate how to use PUR and Aquatabs. We have been waiting for product since the 14th of August but until now, we still have not gotten anything. This despite the fact that the local population, miners and travelers are begging for the product.” The team found that keeping adequate supplies in the pipeline for the rural zones is very challenging and is not helped by the overall transportation problems of bad roads. I. THE GENDER AND YOUTH PERSPECTIVE WITHIN SOCIAL MARKETING FOR HEALTH i. Gender Because of changes in USAID global programming, the evaluation team was asked to look at “To what extent has the project been able to integrate the gender perspective within its activities and which of these approaches and activities will be useful to continue in the future as support to the overall USAID gender objectives?” The pages below provide information not only on gender but also on youth, an equally important constituent group. 25 While gender work is very important within the DRC context, under PSI’s current contracting mechanism for the Social Marketing for Health, they are not specifically asked to have a gender focus. However from a corporate perspective, PSI’s president stated in 2010: “By treating women around the world as customers, by incentivizing the private sector that already interacts with these women to carry life-saving products as well as soap or cooking oil, by using marketing to encourage behavior change the same way we were encouraged to wear a seat belt or are now encouraged to Twitter, we reach more women and we change more lives” in response to Secretary of State Hilary Clinton’s exhortation to include more women. Thus, despite not having a contractual obligation to demonstrate their responsiveness to gender needs, PSI/ASF has an institutional philosophy and ethic which recognizes the importance of gender. Recently, USAID commissioned an extensive review of the gender situation in the DRC and the assessment revealed: Deeply-embedded traditional customs of Congolese culture – the pre-conscious, “meta-level” patterns that condition (but do not determine) thoughts, perceptions, speech and actions – which define men as dominant and women as submissive. Fundamental to masculine and feminine personal identity, they are supported by sanctions on behavior in extended family and community life, bolstered by beliefs in religion and witchcraft, and reinforced by the Family Code. These attitudes are very evident in the work that PSI/ASF does within family planning and HIV/AIDs prevention, although less so in child health. They have disaggregated data by gender to estimate their impact; after two years of project implementation PSI/ASF has reached almost one million women with interpersonal counseling, but only about 250,000 men, a four to one ratio. Additionally, they had reached 23,000 men with the use of their call-in hotline, the Ligne Verte, but only about 7,700 women, a ratio of approximately three men for every one woman. They had not anticipated that the hotline would be used primarily by men and they have not done sufficient exploration of why men are drawn to this medium. One possibility is that according to the 2007 DHS, only 18% of female headed households have a cell phone, compared to 24% of male headed households. We can presume then that men might have greater access to phones and thus are better able to use a call-in service. Other findings, drawn from both PSI’s research and the focus groups conducted for the evaluation indicate:  In order for women to access family planning, there is need for joint counseling to engage men in FP; this is actually one of the same findings that the USAID gender report highlighted.  Women identify men as a barrier to use but are becoming more emboldened to make independent decisions about the use of family planning;  Older women are turn to commercial sex as income generation and engaging in risky practices;  Commercial sex workers are using female condoms so they do not have to negotiate with their clients and are able to protect themselves at the same time.  Existing family planning legislation, “code de la famille” which dates to the Belgian colonial era, is unduly discriminating against women and not in line with current constitutional hopes but it is still the law. PSI/ASF tried to be responsive to improving access to contraceptive methods for women by expanding distribution networks for the female condom to include hair salons. The success of this effort is not yet known because the focus group participants did not find that these commercial sites very worthwhile, as 26 they felt that it cheapened the value of the product to have it available in sites that were not managed by a pharmacist. Of course, a probable unspoken concern was that the purchase of product in a hair salon is not very private and thus would leave them vulnerable to speculation by other women that they were CSWs. PSI/ASF also reached out to women’s NGOs by targeting six of the nine NGOs for institutional development support, these six groups are: Uwaki, Dynamique des Femmes pour le Développement du Congo (DFDC), Centre Féminin de Formation et d’Information pour le Développement (CEFIDE), ONG Women’s Muakaji, Bureau Femmes et Familles de l’Eglise du Christ au Bas Congo (BFF/ECC) and Réseau National des ONG pour le Développement de la Femme en RDC (RENADEF). SI, the partner working with these groups, has helped them sharpen their vision and develop strategic plans. For example, Uwaki, a women’s agricultural collective that deemed closely spaced pregnancies were detrimental to income generation, began working with PSI/ASF in 2004 on family planning. Currently, through the PSI/ASF/SI intervention, they have decided that their forward focus will be on female entrepreneurship and increased access to capital. Future USAID programming is going to require that programs deal explicitly with identified gender issues. The principle development aim is to redress gender imbalances related to health, to promote the empowerment of women and girls, and to improve health outcomes for individuals, families and communities. Given the work to date, PSI/ASF will be well positioned to take on this challenge. ii. Youth PSI/ASF also did not have a specific mandate to deal with youth but as they implemented their program, they revised project efforts to better target youth, including segmenting the overall youth target to include a specific group ages 15-17 . They did this in order to support the government’s goal of creating a generation free of HIV/AIDs and also in response to identified behaviors. For example, the studies they did that looked at condom usage indicated that approximately 34% of youth used a condom with their regular partner but only 15%5 used a condom with an occasional partner and only 17% used a condom with a commercial sex worker (which is a discrepancy from what the CSWs self-reported during focus groups.)6 Despite almost universal awareness that condom usage prevents the transmission of HIV/AIDs, systematic use of condoms is significantly lower than occasional use. In fact, during one focus group with young men many of them asked if there was not something else they could do as they were bored with using condoms. PSI/ASF is trying to address this by introducing a new scented condom, designed to appeal to the youth market. In the DRC, birth rates to girls under the age of 15 confirm early onset of sexual activity and during the focus groups, many of the youth indirectly confirmed they were sexually active. Some of the youth 5 These data are the averages from studies done with youth ages 15-24 in Bukavu and Kinshasa in 2010 and refer to the use of a condom during the last sexual contact and not to systematic use of the condom, which is significantly lower. 6 These data were gathered in 2009 and we were interviewing Commercial Sex Workers in late 2012. So the CSW’s claim that they “always” use condom can be interpreted in a number of ways. One, they know they should always use a condom so they say it, even though in practice they don’t. Two, behavior has changed and if those same youth were interviewed now, we might find that they consistently use condoms with CSW or three, “always” is open to interpretation and it means they use a condom with a man they don’t know, or a man they think looks unclean. 27 indicated that the messages of abstinence and fidelity, which PSI/ASF are required to promote, were appropriate but many more were dismissive of these messages. Other highlights that emerged during the focus groups with youth include:  When youth go to certain pharmacies, they are receiving lectures from distributors on how they should abstain from sex, before they are able to buy condoms. This makes them uncomfortable;  Youth have a great many unanswered questions on the basics of family planning and reproduction so that even though they are sexually active, they are not always making informed decisions;  Youth are using dangerous methods to avoid pregnancy, such as ingesting large amounts of tetracycline or Cytotex Clinicians within the PSI/ASF network of clinics confirmed they are often approached by younger people asking for family planning but do not always adhere to the GDRC policy that allows for the use of condoms and COCs in girls under the age of 15. Counseling for young women remains an area of importance for PSI/ASF because current legislative frameworks offer insufficient protection to young women, allowing them to legally get married at the age of 15 and thus vulnerable to unplanned pregnancies. PSI, again in partnership with SI, has done institutional strengthening to RACOJ, which is a national youth network against AIDS. Because youth unemployment is so high, reaching almost 80%, RACOJ have turned their focus on environmental sanitation as domain where youth could develop jobs. Given that 44% of the current population is under the age of 15, which is almost 32 million people, the need for focusing on youth remains pertinent to national development strategy. AJIS, another local NGO that has received support from PSI, feels that its institutional support for the youth peer educators has been particularly significant. They said “The peer educators who have already improved their competence will also be available tomorrow because they have the facility now to meet members of the community and discuss their daily problems in face of HIV/AIDS. Both individually and in a collective, it is already like a snowball in terms of the activities.” J. INSTITUTIONAL CAPACITY OF THE ASSOCIATIONS DE SANTÉ FAMILIALE Institutional capacity development has taken on an increasing importance over the last five years, in part because of USAID’s new global strategy, USAID Forward. Among its seven approaches, USAID Forward intends to do the following: USAID is changing its business processes—contracting with and providing grants to more and varied local partners, and creating true partnerships to create the conditions where aid is no longer necessary in the countries where the Agency works. To achieve this, USAID is streamlining its processes, increasing the use of small businesses, building metrics into its implementation agreements to achieve capacity building objectives and using host country systems where it makes sense. The USAID Mission in the DRC is keenly interested in what potential local partners might be viable candidates for grants and contributors to the global network of “true partnerships.” From SI’s analysis 28 referenced below, the sobering answer is that for the DRC, there are very few that will be able to take grants and at the same time comply with the fiscal and reporting requirements, including the more stringent evaluation metrics, that USAID requires. USAID/DRC is particularly interested in knowing the institutional evolution of ASF, which has worked in partnership with PSI since 1987. Before going into details on the evolution of ASF and whether or not PSI accomplished what they detailed in their proposal, it is important to understand the context of NGOs in the DRC in general. SI did an initial assessment of different DRC NGOs in late 2010. Two keys points from that assessment are: 1. Several pre-selected civil society organizations pre-identified as “NGOs” are in fact small, community-based organizations with very limited geographical and topical coverage and with very limited institutional viability. Several pre-selected “NGO networks” are in fact NGOs that have been legally established, have a leader and limited staff (mostly part time) and performs a variety of activities in the geographical area they cover. 2. The risk areas most NGOs/networks of NGOs that completed the pre-assessment questionnaires obtained low scores in are: internal policies, management processes and procedures; availability of sufficient human and financial resources used for activities they undertake; number of source of financial resources; technical capacity to formulate programs and projects; monitoring, evaluation and learning; sustainability of NGOs/networks of NGOs. Some of these same risk areas are highlighted in a report from the Global Fund on the management of malaria funding. They found that one of their principal recipients had not managed sub-grantee fiscal responsibility:  Weak Sub-Recipient (SR) financial management systems- identification of unsupported expenditures: An extended verification of SR expenditures underlined significant unsupported (partially or fully) and suspicious expenditures, among the expenditures tested and incurred by two SRs of this Round 8 malaria grant (total tested: USD 554,000, 67% presenting irregularities to be further investigated). The Office of the Inspector General has launched an investigation across the entire portfolio of the DRC.  Lack of Prime Recipient (PR) oversight over SR financial management and expenditures: The PR did not perform adequate monitoring and controls over SR expenditures during the first 18 months of Phase 1, as evidenced by the verification findings. Given this was an organization with significant donor funding history and sound management systems in place, similar to the existing relationship between PSI and ASF, it underscores the vulnerability of NGOs working in DRC. Any local NGO which receives USAID funding under the USAID Forward initiative must ensure that its partners are viable institutions that comply with DRC laws pertaining to the development of non-governmental organizations and with US requirements for strong financial management systems. This brings us to the discussion of ASF. 29 In Section C of the proposal PSI submitted to USAID for the AIDS Support and Technical Assistance Resources (AIDSTAR) project they indicated three possible ways in which they would support institutional development. These were: 1. PSI and ASF will engage in the institutional reform of the local entity, ASF, to improve its legal framework and its capacity for self-governance and in so doing, position ASF to play a true leadership role among local institutions. (page 9 of 46) 2. PSI/ASF’s management approach features Congolese staff in leadership roles with increasing responsibility over the life of the project….PSI/ASF will build ASF local staff capacity so that staff are better able to work with local NGOs to implement social marketing activities (page 19 of 46) 3. PSI/ASF has brought partner Social Impact onto the team to build local institutional capacity to address key health changes in DRC…..SI will work with approximately 20 local organizations over the life of the project to build their operational capabilities such as strategy and planning, finance and administration, and key skills. (page 20 of 46) They have accomplished significant institutional reform and done what they said they would do in points one and two above. In December, 2010, with the hiring of a senior Congolese Managing Director for ASF (who is also a Country Director for PSI), who has many years of international management experience in the private sector, they have done the following:  Redone the organogram and put senior, skilled Congolese in as the heads of most departments;  Developed an institutional pay scale intrinsic to ASF which has created equity among employees and which has resolved the issue of multiple systems that were in place just because of multiple donor funding;  As a follow up to the equalizing of salaries, they are doing a market survey of other NGOs in the country and seeing what the basket of salary and benefits is like in similar institutions. They are doing this because they have had staff hired away from them for economic reasons and they want to make sure they are competitive within the NGO sector and thus able to retain their trained and senior staff;  Hired a staff member who is a “Capacity Building Champion” who works alongside the Director and who develops personalized development plans for each staff employee in response to assessed strengths and weaknesses; as a follow-up, ASF/PSI invested in training opportunities for most of their senior staff in the last two years.  They have reviewed internal management processes and tried to streamline them while at the same time keeping adequate controls, which is necessary in the larger environment of the DRC where financial irregularity is allowed to flourish. However, some staff are reporting that the process is so heavy now that program objectives are hindered because funds cannot be released in time.  Begun to develop ideas for an alternative resource of funding other than donor funding so they have money that can be used for bridging, can cover program activities that are not funded or allowed by different donors or which can be used as an endowment to support institutional sustainability; 30  Developed a five year strategic plan which is in its final stages and which is expected to be approved by the Board of Directors in late December 2012 (and is in institutional embargo until approved so it was not able to be shared with the evaluation team);  Provided training and skill development so that the local senior managers are now charged with writing proposals and developing budgets and this is no longer outsourced to PSI global staff;  Made an institutional commitment to using English as the global business language and thus have invested in developing the English language skills of their Francophone staff with the goal that every staff person will be able to communicate both orally and in writing in English. What ASF has not done is severed their relationship with PSI and nor do they intend to. PSI will move into a supporting role and will be accessed as technical resources. The President of PSI has stated the following “PSI is headed toward an operating model in which more and more of our platforms are locally governed yet still affiliated with us. In ten years, for the sake of planning, we should assume that most of our platforms will have some degree of local governance, and in some cases complete local control. From this assumption, our vision for PSI in ten years is of a network of strong locally governed organizations that maximize their health impact specifically because they maintain a close, organic and recognized association with PSI.” How they will cost these services when they submit proposals is still not clear, in some cases PSI might serve as sub-grantee, in other cases they might be billed as consulting services. The reasons for the continued relationship include:  The use of a known brand, PSI/ASF. For 25 years, the two organizations have worked in tandem, so much so that a government partner who was recently interviewed asked “Where does PSI end and ASF begin?” Likewise, an ASF staff person said that it was ASF who had been the prime recipient of Global Fund resources during Round Eight, while Global Fund documents indicated that PSI was the recipient, with a sub-grant to ASF. This is a symbiotic7 relationship that is working so why create separation?  By continuing to partner with PSI, they will be able to draw on PSI’s Pan-Africa experience and bring in innovation and best practices that have been trialed elsewhere and which can work in the DRC.  The continued relationship will give ASF financial leveraging because they will still be able to rely on the audit tracking systems and software that are proprietary to PSI and which are not available on the local market or at a cost that is affordable. This will also ensure that they have the capacity to meet the stringent fiscal oversight requirements that most donors insist on in the DRC.  For PSI, the relationship will allow them to continue their work in the DRC but also be responsive to changes in how donor funding is being delivered. It will also ensure that the DRC remains part of their global network and experiences gained here can be shared elsewhere. For USAID, one possible drawback of this partnership model is the quality of written English in the project documentation if PSI no longer has an oversight role. The USAID reporting requirements are onerous and new guidance coming out establishes more and more expectations for quality, including 7 A relationship of mutual benefit or dependence. 31 writing in professional English.8 Writing in English is much harder than speaking English and it will be important for USAID to address how non-native speakers of English who are partners are expected to meet the standards of documentation. Will the Contracting Officer’s Representative (COR) be expected to put a final polish on project documents? Will the hiring of a bilingual editor be an allowable cost to ensure that documentation adequately captures in English all the success expressed in French? On the third point of institutional development, from ASF to other local NGOs, efforts are underway but have been delayed and do not yet represent the full changes that were desired. These have been elaborated earlier in section G of this report. Even as ASF grows in institutional capacity, they will never be able to handle all the complexities of managing a program in the diverse provinces of the DRC without a network of sound local partners. While ASF itself is able to take USAID funding and has adequate controls in place, future programming should put more emphasis on extending local institutional development of smaller, regional NGOs and view it as an investment cost. IV. CONCLUSIONS From our findings and review we have arrived at a series of conclusions. They are as follows:  PSI/ASF successfully integrates its development Theory of Change into the implementation of its project and this allows it to better track impact from their interventions.  PSI/ASF is doing a good job and will meet all but three of the project objectives but they could be doing a more dynamic marketing job given their comparative advantage of market knowledge and experience.  Despite the variances in distribution from quarter to quarter, which was caused by politics, customs clearance and a short-term low level of stock, distribution rates for male condoms and female condoms have been high throughout the project, continually meeting quarterly targets and will most likely reach the project target.  Despite these high sales of Prudence Homme, Ok condoms is a threat to sales even though PSI/ASF believes that because there is nowhere near market saturation, there is plenty of room for competing brands.  It is good that sales of Depo Provera and Confiance pills are increasing but the overall method mix needs to have higher number of sales for long-term methods to be responsive to women’s reproductive health needs.  Because they have been a market leader for so long, PSI/ASF has stagnated a little in their creativity and approach to marketing and has not exploited trends in technology for information sharing.  PSI/ASF is distributing a good gamut of products that respond to identified health needs.  Given that there is almost a 70% facility based birth rate in the DRC, this represents a significant opportunity to promote PPIUD and PSI has rightly identified this as an opportunity to add to the contraceptive method mix. 8 At this writing, most of the proposals are required in English. But as USAID goes forward in its changes, there might be recognition that better proposals will be submitted if they are written in native languages and translation costs will just be a part of doing business. 32  The team concludes it is short-sighted to reduce the number of community organizers because of budget reasons, given that PSI’s research has consistently demonstrated that it is necessary to reach out to couples and engage men in the family planning decision process and that this is best done within the household, using community organizers.  Reaching out to industry as an expansion of private sector partners is a sound tactic. This approach could reach increasing numbers of consumers, particularly as mining companies have large all male workforces who use commercial sex workers and need access to condoms and information on VCT.  PSI/ASF is thoroughly integrated into the GDRC health strategies and implementation plans and is viewed as a key player in obtaining national objectives.  PSI/ASF has worked very well with other USG partners, particularly in creating demand for VCT when they do joint activities.  ASF, the local NGO long affiliated with PSI, is mature enough to be able to take USAID grant funds independently.  The evidence is not yet compelling that efforts in the rural sector will help reach national goals. PSI/ASF has identified key gender issues that need to be addressed to move forward in family planning and they have the knowledge and strategies to respond appropriately.  PSI/ASF has appropriately segmented the youth market and is concentrating its focus on the younger cohort, which is appropriate given the government goals of a generation without HIV/AIDS and the fact that 20% of girls under 15 have had babies.  PSI/ASF is not as strong as they need to be in following up with consumers who are using products but who have outstanding questions on efficacy and side effects and this increases the risk that clients will be family planning drop-outs.  While not specifically tasked to focus on gender, because of their current research and field experience, PSI is poised to do more work on gender issues and reproductive health. V. RECOMMENDATIONS The recommendations below are both for immediate implementation by PSI/ASF and for long-term consideration for USAID as they move forward with their Country Development Cooperation Strategy. The recommendations are loosely grouped by category, based on the evaluation questions. PSI will have only nine months to make the changes, as their contract goes only until September, 2013. USAID will have the current planning cycle to evaluate whether they wish to incorporate these recommendations into future strategies or design. Products and Distribution  PSI/ASF should align pre-product placement activities (approval, exoneration, market studies, branding materials) with product arrival to reduce lag time between demand creation and market availability (Microlut/DTKs).This will maximize its ability to meet program objectives.  USAID should facilitate its different partners in the coordination of a united approach to product registration, increasing efficiency and mitigating stock out periods. 33  USAID should not revisit the question of CDKs, there are other contributions to safe motherhood such as the promotion of long term methods and increased access to contraception.  Despite the already broad gamut of products they distribute, PSI should add PPIUD and emergency contraceptive, so they will have a very comprehensive product range that meets the needs of women in the DRC.  PSI/ASF should continue its work with PNSR to ease barriers to access for PPIUD and try to increase the number of IUDs provided to women immediately after delivering. USAID and PSI/ASF should pilot social marketing approach for Emergency Contraceptives, targeting young women not in union. This will support behavior change away from using illegal and dangerous products to prevent contraception.  PSI/ASF should complete its pricing study scheduled for the first quarter 2013 to support increase of product prices, as prices are currently lagging behind economic growth and not maximizing opportunity for revenue.  USAID should look at the root cause of stock-outs and apply its comparative advantage and long experience in procurement to mitigate the ongoing problems of stock out which have affected trocars for implants, progesterone only pills and even sometimes condoms.  PSI/ASF should move from “good to great” through a renewed marketing campaign that is future focused and creative and not resting on past accomplishments and its role as the long￾term market leader. Awareness and Demand Creation  PSI/ASF needs to target vendors in small pharmacies with appropriate messages for clients and how to manage stock, consider the creation of a laminated checklist that would provide stock answers to frequently raised questions, such as whether or not the lubricant on Prudence condoms is cancer inducing.  PSI/ASF needs to increase the sales of long term methods, needed to address short birth intervals and higher fecundity. It is vital to include long term methods in all marketing efforts to avoid undue reliance on pills and injectables, which have an appropriate place in the gamut of methods but which do not necessarily best serve older women who have completed their family.  PSI/ASF and USAID need to engage in policy dialogue with the GDRC on removing the limitations of advertising branded family planning methods, once done the Confiance campaigns could be revitalized.  Because of consumer resistance indicated in the evaluation focus groups, PSI/ASF should re￾evaluate hair salons as venue and determine if product is really moving there.  Based on the past success PSI/ASF enjoyed from working closely with ProVIC, they need to continue to link demand creation for VTC with immediate testing so as not to lose clients when they are referred to other testing sites. This linkage can be between PSI and any partner working in the VTC sector, such as the Global Fund and is not limited to just USAID partners. Behavior Change Communication Methodologies  PSI/ASF should consider other community points for demand creation and increased use of community outreach workers- such as village water pumps and cyber cafes; the work with cyber 34 cafes would entail an assessment to understand the user profile and to determine what kind of messaging could be promoted at what price.  PSI/ASF should evaluate whether the print media associated with 100% Jeune is appropriate in a low literate society or whether there are other venues to get out the messages focused on youth.  PSI/ASF should evaluate how to retain its community mobilizers in the face of increasing budget constraints, given their proven effectiveness and appropriateness to the socio-cultural context of both urban and rural settings within the DRC.  PSI/ASF should focus its Prudence Sensuel campaign to respond to market competition and recapture those consumers who have gone to using the Ok brand of condom.  PSI/ASF should rethink its placement of billboards as well as its post-campaign management of billboards to avoid having dirty, torn and defaced billboards associated with its brands.  PSI/ASF should add strategies that allow it to capitalize on the growing use of smart phones and mobile phones within the Congo.  ASF should send some of their senior staff to the Mobile Technology Conference in Dar Es Salaam, November 7th to 9th, 2012 which covers Leveraging Mobile Technology to Meet Healthcare Needs in the Developing World. The mHealth Alliance actively promotes the use of mobile platforms in underserved communities to increase support for recognized health interventions, promote well being and help save lives Some of the topics included are: o Adopt mHealth solutions to advance the use, value and power of mobile technologies. o Assess how mobile technologies provide cost‐effective solutions in remote communities. o Enable remote health workers, improve remote monitoring and diagnostics, and impact patient outcomes. o Empower the public in underprovided communities with health information via mobile platforms. o How mobile technology is being used and how it is impacting patient safety. o Existing roadblocks to widespread mobile technology adoption and IT support. Gender and Youth  PSI/ASF should act on the recommendations included in the USAID Gender Report which promotes addressing teenage girls particularly on issues such as delayed first pregnancy, child spacing, antenatal care, and maternal mortality.  PSI/ASF should collaborate with government on renewed program for FP education/awareness in girls under the age of 15.  PSI/ASF should address provider bias in prescribing to young girls and vendor behavior around selling condoms to young men.  Given that 20% of girls aged15 are already mothers, 9 USAID and PSI/ASF should review the messaging around youth and abstinence and increase efforts for improving campaign relevance to the DRC socio-cultural context. 9 This statistic is from the earliest MICS. While it is a shocking number, the latest DHS does not offer any evidence to suggest that it is grossly over inflated. According to the DHS, 25% of girls 15-19 are already in union, and of those 18% had been sexually active by the time they were 15. More than half (51%) of young women ages 15-19 35  Given that young women are using dangerous products to avoid pregnancy, PSI/ASF should increase campaign messages that address the prevention of unwanted pregnancy.  Conduct in-depth analysis of why men are drawn to the use of the Ligne Verte hotline, and try to establish whether men are drawn to the hotline because of anonymity, or because there are no other venues appropriate for men to get information or because men alone have access to phones, among possible reasons. Depending on the results of this analysis, PSI/ASF should identify other possible venues to provide family planning information to men.  Implement the recommendations from the USAID Gender Report which emphasize the need to address men’s health, responding first to their perceived health needs, then increasing their knowledge of and involvement with maternal and child health issues and family planning (keys to addressing safe motherhood)  Give women negotiating skills for discussing family planning, consider role-playing during the interpersonal counseling sessions done with both men and women.  Given that there is not yet firm evidence as to why female condoms are selling so well, PSI/ASF should conduct targeted research on female condoms including a determination if “off-license” use as jewelry, shoe polish and product for men who have sex with men (MSM) account for sales. Institutional Development/Partnership with NGOs, Private Sector and Commercial Sector  USAID should drop its insistence that ASF’s institutional viability must be divorced from ongoing liaison with PSI and not be overly focused on the inter-institution arrangements.  USAID should pilot providing grant funds directly to ASF as the prime recipient, providing they submit a proposal that is responsive to USAID criteria.ASF should consider getting a Communication Director (preferably a woman) and split Marketing and Communication responsibilities.  ASF should regularize the two classes of drivers to promote better institutional unity.  ASF should review internal management controls to analyze whether process is impeding program ability.  ASF should negotiate any future contract funding to allow disbursement of grant funds to small partner NGOs, which will support the implementation of institutional development promoted through training and coaching.  PSI/ASF should continue to build local NGO networks but increase “face time” and mentoring, rather than only conducting training.  PSI/ASF should continue the dialogue with mining companies as to how they can optimize linkages and create new markets for their reproductive health product line. were sexually active. 33% of women interviewed had had a baby before the age of 18. A study by IRC suggested that this high number of births to very young girls is part because of the “spoils of war” and the terrible epidemic of rape that women in the DRC have endured, irrespective of age. 36 Rural Strategy • USAID should determine if it wants to continue funding the rural strategy given the changing urbanization of the DRC and the current challenges that need to be addressed within the existing large cities. • If the rural strategy is maintained, PSI/ASF should evaluate whether collaboration with road￾building projects would allow it to increase market penetration in the rural sector and mitigate against some of the enormous transportation issues they have currently identified.  PSI/ASF should formalize the public identity of a “pedaleur” associated with its range of products, increasing visibility through branding with decals and t-shirts. • PSI/ASF should evaluate whether women can have adequate access to contraceptives and condoms given the protracted time between village visits and the low level of monetization in villages.    37 IV. ANNEXES A. LIST OF MATERIALS CONSULTED B. LIST OF PEOPLE INTERVIEWED C. TOOLS USED IN FOCUS GROUPS AND IN-DEPTH INTERVIEWS D. ORIGINAL AND REVISED SCOPES OF WORK E. FULL EVALUATION METHODOLOGY F. EXAMPLES OF HOW MOBILE TECHNOLOGY SUPPORTS FAMILY PLANNING PROGRAM OBJECTIVES 38