SMBCI PROJECT FINAL PERFORMANCE EVALUATION SMBCI PROJECT FINAL PERFORMANCE EVALUATION FINAL REPORT OCTOBER 2, 2012, REVISED NOVEMBER 2012 This publication was produced for review by the United States Agency for International Development. It was prepared by Betty Ravenholt, Jen Bergeson-Lockwood, Jennifer Cashin, Shaymi DeSilva, and Karen Heckert Diggs Utts through Management Systems International. SMBCI PROJECT FINAL PERFORMANCE EVALUATION FINAL REPORT Contracted under GPO-A-00-06-00005-00 DISCLAIMER The author’s views expressed in this publication do not necessarily reflect the views of the United States Agency for International Development or the United States Government. SMBCI PROJECT FINAL PERFORMANCE EVALUATION i CONTENTS CONTENTS ............................................................................................................................................ i Acknowledgements ................................................................................................................................ ii LIST OF ACRONYMS ........................................................................................................................ iii EXECUTIVE SUMMARY ................................................................................................................... 1 Key Findings (12) .......................................................................................................................................... 1 Conclusions (12) ............................................................................................................................................ 2 Recommendations (9)................................................................................................................................... 3 BODY OF REPORT ............................................................................................................................. 5 Description of The Project.......................................................................................................................... 6 Evaluation Methods....................................................................................................................................14 FINDINGS, CONCLUSIONS, AND GAPS OR MISSED OPPORTUNITIES ............................. 16 Hiv and Aids .................................................................................................................................................16 Reproductive Health and Family Planning .............................................................................................33 Child Survival ...............................................................................................................................................49 SMBCI CROSS-CUTTING ISSUES ................................................................................................. 56 Cost-Effectiveness.......................................................................................................................................56 Localization...................................................................................................................................................57 Gender...........................................................................................................................................................58 APPENDIX A ...................................................................................................................................... 60 Scope of Work .............................................................................................................................................60 STATEMENT OF WORK ................................................................................................................. 60 III. Statement of Work: ..............................................................................................................................62 IV. Methodology:.........................................................................................................................................63 APPENDIX B ...................................................................................................................................... 70 Persons Contacted.......................................................................................................................................70 APPENDIX C ...................................................................................................................................... 74 Team Interview Questionnaires................................................................................................................74 APPENDIX D ...................................................................................................................................... 85 Documents Consulted................................................................................................................................85 APPENDIX E ...................................................................................................................................... 89 PSI Research under USAID 2008-2012 ..................................................................................................89 APPENDIX F ....................................................................................................................................... 96 Guiding Principles of the Evaluation.......................................................................................................96 APPENDIX G ...................................................................................................................................... 98 Observed Commercial Sector Condom Prices.......................................................................................98 SMBCI PROJECT FINAL PERFORMANCE EVALUATION ii ACKNOWLEDGEMENTS The team would like to thank Robin Mardeusz, MCH Team Leader; Patricia Orlowitz, Project Development Officer; and the staff of USAID's Office of Population Health and Education (OPHE) for the excellent guidance and support they provided to the evaluation team. Special thanks are also given to the leadership and staff of PSI/Cambodia for their many briefings and presentations, informative field visits, and responsiveness in providing us with additional information and patience in answering our many questions. This evaluation would not have been possible without their assistance. SMBCI PROJECT FINAL PERFORMANCE EVALUATION iii LIST OF ACRONYMS ARI Acute Respiratory Illness BCC Behavior Change Communication BROS Behavioral Risk on site Serosurvey BSS Behavioral Surveillance Survey CCR Continuity of Care Record CD Country Director CDD Control of Diarrheal Disease CDHS Cambodia Demographic and Health Survey CMDG Cambodia Millennium Development Goals COC Continuity of Care CPR Contraceptive Prevalence Rate DFID United Kingdom Department for International Development DTK Diarrhea Treatment Kit EW Entertainment Worker FHI Family Health International FP Family Planning FWT Full Well Trading GFATM Global Fund to Fight AIDS, Tuberculosis and Malaria HIV/AIDS Human Immunodeficiency Virus/Acquired Immune Deficiency Syndrome HMIS Health Management Information System HRUM High-Risk Urban Men HSS HIV Sentinel Surveillance iBCC Integrated Behavior Change Communication IMS Information Management System IEC Information, Education, Communication IPC Interpersonal Communication IUD Intrauterine Device KfW Kreditanstalt fur Wiederaufbau KHANA Khmer HIV/AIDS NGO Alliance LOP Life of Project MAP Measuring Access and Performance MARPs Most-at-Risk Populations MCH Maternal and Child Health MDGs Millennium Development Goals MIS Management Information System MOH Royal Government of Cambodia Ministry of Health MORD Royal Government of Cambodia Ministry of Rural Development MoU Memorandum of Understanding MSM Men who have Sex with Men NAA National AIDS Authority NCHADS National Center for HIV/AIDS, Dermatology and STD NGO Non-Governmental Organization OPHE Office of Population Health and Education OHA Office of HIV/AIDS ORS Oral Rehydration Salts ORT Oral rehydration therapy PHC Primary Health Care PMP Performance Management Plan SMBCI PROJECT FINAL PERFORMANCE EVALUATION iv PRH Office of Population and Reproductive Health PSI Population Services International PSI/C Population Services International/Cambodia RACHA Reproductive and Child Health Alliance RGC Royal Government of Cambodia RH Reproductive Health RHAC Reproductive Health Alliance of Cambodia SMBCI Social Marketing and Behavior Change Interventions for HIV/AIDS, Reproductive & Sexual Health & Child Survival in Cambodia SMO Social Marketing Organization SQHN Sun Quality Health Network STI Sexually Transmitted Infection(s) TMA Total Market Approach TRaC Tracking Results Continuously TS Toeuk Sovatepheap TWG Technical Working Group UHN United Health Network USAID United States Agency for International Development WRA Women of Reproductive Age VCCT Voluntary Confidential Counseling and Testing SMBCI PROJECT FINAL PERFORMANCE EVALUATION 1 EXECUTIVE SUMMARY USAID/Cambodia’s Office of Public Health & Education (OPHE) convened a team of experts with a diverse skill set and background—Social Marketing and relevant health sector experience in HIV/AIDS, Reproductive Health and Family Planning, and Child Survival—to conduct an evaluation of the Mission’s Social Marketing and Behavior Change Interventions for HIV/AIDs, Reproductive and Sexual Health and Child Survival in Cambodia (SMBCI) Project currently implemented through Population Services International (PSI). The purpose of this evaluation was to conduct a summative evaluation of the existing program which aims to improve the health status of Cambodians through increasing sustainable access to health products and services and long-term improvements in health-seeking behavior related to HIV/AIDS, RH, and child survival. The Evaluation was carried out during the months of July and Aug 2012 and included document review, stakeholder interviews (80 individuals), and site visits to area commercial and public sector distributers in Kampong Chhang, Pursat, and Battambang provinces as well as in Phnom Penh. The team included three consultants from Management Systems International (MSI), as well as participation from two USAID/W based specialists. The project-wide findings, conclusions and recommendations include the following: KEY FINDINGS (12) The SMBCI Project has made significant contributions to the availability of very low-priced condoms, short￾term contraceptive methods, ORS+zinc, and safe water tablets; and the introduction of long-term and reversible contraceptive methods such as IUDs and implants. PSI/Cambodia (PSI/C) has taken on multiple roles under this project as a distributor of socially marketed condoms, a behavior change communication (BCC) and HIV/AIDS prevention partner, and a source of strategic and policy advice to the Royal Government of Cambodia (RGC). In some cases, these roles were conflicting. For example, PSI’s role as a social marketing organization (SMO) in the condom market, providing 80 percent of the products may conflict with its role as an advisor to the RGC or USAID on total market approaches (TMAs). TMAs often require trade-offs between long-term market impacts and short-to-medium term health impacts. There is a continuing need for the social marketing of selected health products in Cambodia––both to generate increased demand for and correct use of products and to ensure that necessary health products are accessible to the poor and vulnerable. The market for the project's health products and services is not currently segmented. There is evidence that a considerable number of consumers who are able to pay for products are obtaining them nearly free from the public sector or at a subsidized price from the social marketing project. Prices in the marketplace are clustered at the two extremes––very high-priced commercial products and very low-priced social marketing products. The limited range of condom and contraceptive method/brand choices in both the social marketing project and the commercial market constrains the ability to respond effectively to the special needs/desires of targeted segments of the population with products throughout each consumer's reproductive life cycle. Broader participation of the commercial sector in the market for condoms and contraceptives has not been strategically and persistently pursued as part of the social marketing project's overall strategy. This is SMBCI PROJECT FINAL PERFORMANCE EVALUATION 2 evidenced in project reports, interviews, and by the fact that subsidized brands account for such a large percentage of the existing market. Market research data that describes (socioeconomic status as well as other characteristics) the actual consumers of project products, the actual consumers of similar non-project products (either commercial or public sector), and the non-consumers of those products has not been collected. The current unavailability of this information greatly constrains the ability of the social marketing project to evaluate its success in reaching targeted populations and in designing more effective and strategic behavior change interventions. HIV/AIDS BCC programs have a greater understanding of the social and behavioral determinants of limited use of oral rehydration salts (ORS), appropriate treatment for acute respiratory illness (ARI)/pneumonia, and preference for traditional contraceptive methods (especially among urban and wealthier women). However, Tracking Results Continuously (TRaC) findings indicate that trust-based determinants for consistent condom use also need to be reexamined; and most at-risk populations (MARP) interpersonal communication (IPC) intervention data falls short of reporting a key HIV/AIDS prevention behavior, namely the numbers (percent) of those receiving voluntary confidential counseling and testing (VCCT). Community-based (NGO outreach partners) and facility-based (Sun Quality Health network [SQHN]) services are not well integrated across and within health areas. Systematic monitoring and supervision is not evident at the provincial and operational district levels to ensure that the dual practices of SQHN midwives do not impair the availability of reproductive health (RH)/family planning (FP) services in the public sector or facilitate leakage of public sector resources (products, supplies, equipment) into private practices. TRaC and Measuring Access and Performance (MAP) trend data confirm the decreased condom demand in hot spots and other high-risk venues, and a leveling off or decline in self-reported consistent condom use for certain high-risk populations such as high-risk urban men (HRUM) with sweethearts. While the SMBCI Project has created a "culture of research," some important variables that have been monitored and tracked over time have not remained consistent. It is difficult to demonstrate with confidence trends in some important areas of project performance because the variables used and/or questions asked changed constantly. Additionally, all relevant stake holders have not been fully engaged in the research process or use the valuable data for joint planning and evaluation. CONCLUSIONS (12) • The PSI SMBCI Project has succeeded in making significant contributions to the widespread availability of very low-priced commodities, including condoms, short-term contraceptive methods, ORS+zinc, and safe water tablets, as well as the introduction of long-term and reversible contraceptive methods like IUDs and implants. • PSI/C has taken on multiple roles under this project as a distributor of socially marketed condoms, a BCC and HIV/AIDS prevention partner, and a source of strategic and policy advice to the RGC. In some cases, these roles have conflicted. The role of articulating trade-offs and advising donors and governments on SMBCI PROJECT FINAL PERFORMANCE EVALUATION 3 policy options to maximize sustainability should be played by a neutral entity with no organizational stake in the commodity market. • There is a continuing need for the social marketing of selected health products in Cambodia––both to generate increased demands for and correct use of the products and to ensure that necessary health products are accessible to the poor and vulnerable. • The market for the project's health products and services needs to be better segmented. There should be widely available mid-priced offerings, especially for condoms and contraceptives, in the commercial marketplace. • The range of condom and contraceptive method/brand choices in both the social marketing project and the commercial market should be expanded. Some examples include products such as a progestin-only pill, another type of IUD, several mid-priced oral contraceptive brands, and several mid-priced condoms with special attributes (like ribs, dots, colors, scents). • The broader participation of the commercial sector in the market for condoms and contraceptives needs to be strategically and persistently pursued as part of the overall strategy of the social marketing project. For example, Bayer's efforts to introduce a mid-priced oral contraceptive should be aggressively supported by the SMBCI. • Market research data that describes (socioeconomic status as well as other characteristics) the actual consumers of project products, the actual consumers of similar non-project products (either commercial or public sector), and the non-consumers of those products needs to be collected. This, presumably, would result in the design and implementation of more effective and strategic marketing and behavior change interventions. Use of this data would also allow for a more accurate evaluation of the achievement of project purpose and for effective market segmentation. • To develop effective BCC strategies, a better understanding of the social and behavioral determinants of limited use of ORS (despite near universal knowledge of the product), appropriate treatment for ARI/pneumonia, and preference for traditional contraceptive methods (especially among urban and wealthier women) is needed. • Community-based (NGO outreach partners) and facility-based (SQHN) services need to be better integrated across and within health areas. • Systematic monitoring and supervision is needed at the provincial and operational district levels to ensure that the dual practices of SQHN midwives do not impair the availability of RH/FP services in the public sector or facilitate leakage of public sector resources (products, supplies, equipment) into private practices. • Intensified condom distribution and availability in high-risk venues may be warranted based on TRaC and MAP trend data that confirm decreased condom demand in hot spots and other high- risk venues, and a leveling off or decline in self-reported consistent condom use for certain high-risk populations such as HRUM with sweethearts. • Project research tools that use variables that remain consistent over time need to be designed. RECOMMENDATIONS (9) • PSI/C has taken on multiple roles under this project as a distributor of socially marketed condoms, a BCC and HIV/AIDS prevention partner, and a source of strategic and policy advice to the RGC. In some cases, these roles have conflicted. The role of articulating trade-offs and advising donors and governments on policy options to maximize sustainability may be best played by a neutral entity with no organizational stake in the commodity market. SMBCI PROJECT FINAL PERFORMANCE EVALUATION 4 • The SMBCI Project should revisit ways to achieve “improved behavior change/ marketing interventions.” This could be accomplished with more specific knowledge about target consumers and their behavioral determinants; and the better targeting of BCC, advertising, IPC messages, and health product characteristics/benefits to better-defined target market segments. This effort should include: (1) consumer and market research that identifies the socioeconomic status and other characteristics of users and non￾users, social marketing brand consumers, and non-SM brand consumers; (2) an adequate range of product/brand and service offerings; (3) integrated IPC; (4) mass and community media that targets and promotes appropriate prevention and treatment beliefs, perceptions, and practices; (5) a national condom strategy developed and owned by key relevant partners; and (6) community-level support/promotion of behavior change. • Community-based (NGO outreach partners) and facility-based (SQHN) services are not well integrated across and within health areas. For instance, contraceptive counseling and service provision should be a regular part of antenatal, post-partum, and post-abortion care protocols. Also, ORS and ARI counseling should be a regular part of health care interactions with mothers of young children. Additionally, routine risk assessments and referrals for HIV testing should be part of all sexually transmitted infection (STI) screening and treatment. • Program improvements can be made in the targeted distribution of products and services. This would be based on improved penetration of outlets that are accessible to targeted populations, and improved targeting of products to outlet types (facilitating the segmentation of the market among public, NGO/social marketing, and commercial sectors). • SMBCI Project staff should develop and begin implementation of a medium and long-term sustainability plan for SMBCI products and services delivery. • Priority should be placed on the training of private sector providers that are delivering integrated services. Major activities would include training for the providers, supportive supervision, and community-level promotion of service delivery sites. • Improved segmentation of the market for health products should be based on consumer and market research and implemented through distribution, pricing, and promotional strategies. In particular, USAID/Cambodia should focus on: (1) strategies to better target social marketing products to the poor and vulnerable, (2) strategies to attract able-paying consumers away from subsidized products toward private/commercial sector products, and (3) strategies to bring commercial mid-priced health products into market. • Improved interaction/collaboration among public, NGO, and commercial sectors should be integral to future SM activities. This would enable: (1) the greater acceptance and understanding of TMA; (2) research/data generation for health market analysis and rational market segmentation; (3) opportunities for interaction among sectors; and (4) inclusion of commercial sector representatives in relevant working or technical groups to discuss data or issues of mutual interest/importance. At the policy level, attention should be paid to the introduction of quality of care standards and practices in the private sector. • Improved strategic information should be developed, collected, analyzed, utilized, and disseminated by multiple collaborating U.S. government, Ministry of Health (MOH), and NGO partners for joint decision￾making and accountability. SMBCI PROJECT FINAL PERFORMANCE EVALUATION 5 I. BODY OF REPORT USAID/Cambodia’s Office of Public Health and Education (OPHE) convened a team of experts to conduct a final performance evaluation of the Mission’s Social Marketing and Behavior Change Interventions for HIV/AIDS, Reproductive and Sexual Health, and Child Survival in Cambodia (SMBCI) Project, currently implemented by PSI. Key Evaluation Questions 1. What are the major outcomes, achievements, constraints, and lessons learned of the USAID funded social marketing program in Cambodia? 2. Did the project successfully address gender issues? 3. How cost effective has this program been? Which interventions have been most/least cost effective? 4. How effective are the outreach and demand creation activities of PSI? Which would be most valuable to continue? 5. How effective are the various commodities socially marketed by PSI? Which would be most valuable to continue? 6. How optimal has the balance between supply-focused activities (distribution) and demand focused (promotion) activities been? 7. Is the total market approach appropriately structured (public, social market, and commercial sector) to optimize coverage while supporting long-term sustainability? 8. What are the lessons learned from the localization process? 9. How does the current project contribute toward policy reforms? Is their participation in technical working groups (TWGs) paying off? 10. How to improve engagement with the RGC/MOH and other relevant players? 11. Is the current technical approach, scale, and geographic coverage sufficient to achieve the overall goal of USAID? Are any changes recommended? 13. How to improve engagement with the RGC/MOH and other relevant players? 14. Is the current monitoring system effective and are there any changes recommended? SMBCI PROJECT FINAL PERFORMANCE EVALUATION 6 The evaluation team has organized the report of its findings, conclusions, and recommendations into three primary areas: (1) HIV/AIDS, (2) RH and FP, and (3) child survival /maternal and child health (MCH). Within each health area, the team discussed the project's performance in four primary output areas. As stated in project documents,1 these expected outputs are: • Increased access to selected products such as condoms, water-based lubricants, modern birth spacing products, ORS+zinc, safe water treatment products, and ARI treatments; • Improved knowledge, awareness, and the supportive attitudes necessary to change behaviors among target populations2 ; • Increased Cambodian national capacity to manage and sustain results over the long-term with reduced dependence on donors; and • Increased knowledge and evidence base for effective and efficient social marketing and behavior change interventions. Additionally, the evaluation team presented, within each health area, its findings related to: • Gaps and missed opportunities in project implementation • Lessons learned/conclusions • Cost-effectiveness • Localization • Gender issues In the final section, the team summarized by area the major recommendations it drew from its evaluation of the SMBCI Project. DESCRIPTION OF THE PROJECT Through support from USAID and DfID, PSI/C in close collaboration with the RGC, implemented the Social Marketing and Behavior Change Intervention for HIV/AIDS, Reproductive and Sexual Health, and Child Survival in Cambodia (SMBCI) program. The overall goal of the program is to improve 1 Implementing Partner: Population Services International (PSI) Cambodia, Semi-Annual Progress Report to USAID/Cambodia, submitted by Yasmin Madan, May 1, 2012, page 1. 2 The team notes that this output falls short of defining actual behavior change as a project output; however, the project purpose is "to increase consistent and correct use of and access to high-quality and affordable health products and services among poor and vulnerable populations." The evaluation team chose to follow the statement of project purpose and regards this output as behavior change––that is, increased demand for and use of selected products and services among targeted populations SMBCI PROJECT FINAL PERFORMANCE EVALUATION 7 the health status of Cambodians through increasing sustainable access to health products and services and long-term improvements in health-seeking behavior related to HIV/AIDS, RH, and child survival. The project purpose is to increase the consistent and correct use of and access to high-quality and affordable health products and services among poor and vulnerable populations. Within this framework, PSI/C outputs are: 1.) Increased access to condoms, water-based lubricants, modern birth spacing products, ORS+zinc, safe water treatment products, and ARI treatment 2.) Improved knowledge, awareness, and supportive attitudes to change behaviors among target populations 3.) Increased Cambodian national capacity to manage and sustain results over the long-term with reduced dependence on donors 4.) Increased knowledge and evidence base for effective and efficient social marketing and behavior change interventions USAID’s assistance to national, provincial and district-level efforts has greatly contributed to the positive results reported in the 2010 Cambodian Demographic and Health Survey (CDHS). MCH has been a core strategic health objective of USAID/Cambodia since 1993 and constitutes a large portion of the total $372 million in health funding obligated to Cambodia thus far. USAID remains the largest and most active donor in maternal and newborn health in Cambodia with an annual budget of $30-35 million in the areas of HIV/AIDS, MCH, FP, tuberculosis, and nutrition. USAID/Cambodia’s health strategy is reflected in the recent Country Development Cooperation Strategy Results Framework. Developed in coordination with the RGC and other development partners, the framework outlines the key necessary results of USAID to achieve the overarching health development objective, “Improved health status with emphasis on vulnerable populations.” The intermediate results of this objective are: 1) Improved maternal and child health practices in communities and facilities 2) Strengthened health systems and governance 3) Improved effectiveness and efficiency of infectious disease control programs The USAID/Cambodia Health portfolio includes multiple activities implemented with a mix of local and international partners. To reach rural, marginalized, and poor Cambodians and strengthen community-level public sector health services, the U.S. government directly funds three Cambodian NGOs and four international organizations. They in turn fund more than 40 local NGOs that deliver services throughout the country. USAID/Cambodia’s portfolio is diverse––each mechanism is designed and implementing partner chosen to address specific goals in achieving USAID/Cambodia’s health objectives. SMBCI PROJECT FINAL PERFORMANCE EVALUATION 8 A. Background As of 2008, the total population of Cambodia was estimated at 13.4 million people, according to the General Population Census.3 The country is predominantly rural, with 80 percent of the population living in rural areas and 72.3 percent working in agriculture. Cambodia’s age structure is typical of a young population with recently declining fertility rates. At present, 45 percent of the total population is under the age of 20. The annual population growth rate in Cambodia is 1.54 percent, and the total fertility rate is 3.0 (2.2 in urban areas, 3.3 in rural areas). The average household size is 4.7, and the male to female sex ratio is 94.7. Although Cambodia is considered a low income country, it has made substantial improvements in poverty reduction over the past two decades. In terms of economic growth, Cambodia’s GDP per capita has increased from $248 in 1994 to $739 in 2008 and $830 in 2010.4 Despite the crisis of food prices in 2008 and the global financial crisis in 2009, Cambodia has returned to its pre-crisis path of economic development and boasts a 5.9 percent annual growth 3 Royal Government of Cambodia. General Population Census 2008. Phnom Penh: National Institute of Statistics, 2009. 4National Institute of Statistics, 2010. SMBCI PROJECT FINAL PERFORMANCE EVALUATION 9 rate.5 The poverty rate in 2010 was estimated at 25.8 percent, representing marked and consistent improvement from 47.0 percent in 1993, 34.7 percent in 2005, and 30.1 percent in 2007.6 In general, economic growth has been accompanied by distinct improvements in health indicators. For instance, the maternal mortality ratio decreased significantly from 472 per 100,000 live births in 2005 to 206 per 100,000 live births in 2010. Furthermore, the proportion of births attended by a skilled health provider increased dramatically from 2000 (31.8 percent) to 2010 (71.0 percent). Table 1 illustrates trends in relevant reproductive, maternal, and newborn health indicators in Cambodia from 2000 to 2010 as well as Cambodia Millennium Development Goal (CMDG) 2015 targets where applicable. Table 1: Selected Reproductive, Maternal, and Newborn Health Indicators and CMDG Targets Indicator CDHS 2000 CDHS 2005 CDHS 2010 CMDG target 2015 1 Maternal mortality ratio (per 100,000 births) 437 472 206 250 Neonatal mortality rate (per 1,000 births) 37 28 27 22 Percentage of births preceded by antenatal care 37.7 69.3 89.1 90.0 Percentage of births preceded by four antenatal care visits 8.9 27.0 59.4 90.0 Percentage of births delivered by trained attendant 31.8 43.8 71.0 87.0 Percentage of births occurring in health facility 9.9 21.5 53.8 80.0 Married women using modern birth spacing 18.5 27.2 34.9 60.0 Total fertility rate 4.0 3.4 3.0 3.0 Percentage of women with at least one abortion in the past five years 1.9 3.5 5.2 - Percentage of women with BMI < 18.5 20.7 20.3 19.1 8.02 Percentage of women with anemia 57.8 46.6 44.4 19.02 Percentage of pregnant women with anemia 66.4 57.1 52.7 33.02 1 Royal Government of Cambodia. Achieving Cambodia’s Millennium Development Goals. Phnom Penh: Ministry of Planning, 2010. 2 Royal Government of Cambodia. Fast Track Initiative: Roadmap for Reducing Maternal and Newborn Mortality 2010-2015. Phnom Penh: Ministry of Health, 2010. 5 Council for Agriculture and Rural Development (CARD). Review Report: Strategic Framework for Food Security and Nutrition in Cambodia 2008-2012 (SFFSN). Phnom Penh, 2011. 6 Ibid SMBCI PROJECT FINAL PERFORMANCE EVALUATION 10 While Cambodia’s modern contraceptive prevalence rate (CPR) has increased over the past 10 years, currently only 34.9 percent of married women report using a modern contraceptive method. Between 2005 and 2010, there was an increase in the prevalence of traditional methods for family planning with 15.7 percent of married women reporting the use of a traditional method in 2010 as compared with 12.8 percent in 2005. Interestingly, women in urban areas are more likely to use a traditional method (24.1 percent) than women in rural areas (13.8 percent). Similarly, married women in rural areas are more likely to use modern contraception (35.8 percent) than women in urban areas (30.7 percent). Approximately five percent of women of reproductive age have had an abortion in the past five years (representing an estimated three percentage point increase from the 2000 CDHS). According to the 2010 CDHS, about one￾quarter of women who have ever undergone an abortion have had more than one. This represents a significant improvement from 2005 when 44 percent of women who have had an abortion reported multiple abortions.7 The estimated unmet need for family planning in Cambodia is 16.6 percent, as compared with approximately 25 percent in 2005. In terms of child health and mortality, the period from 2000 to 2010 was characterized by a marked reduction in the infant mortality rate, which fell from 95 per 1,000 live births in 2000 to 45 per 1,000 live births in 2010. Furthermore, the under-5 mortality rate was more than halved between 2000 and 2010. Table 2 presents trends in child health indicators from 2000 to 2010 as well as the CMDG 2015 targets where applicable. Newborn deaths (within the first 28 days of life) now account for more than half of all under-5 deaths in Cambodia. During the post-neonatal period, the leading cause of death among children under-5 years is pneumonia, followed by diarrhea. Approximately 15 percent of CDHS 2010 respondents stated that their children under-5 suffered from diarrhea in the last two weeks, while 6.4 percent reported ARI symptoms in the previous two weeks. Globally, under-nutrition is the underlying cause of 35 percent of deaths in children under-5 years. Despite reductions in poverty and improvements in MCH, reductions in the prevalence of under-nutrition among women and children have stagnated. As seen in Table 2, much of the progress made from 2000 to 2005 was stalled or reversed between 2005 and 2010. The prevalence of stunting or chronic under-nutrition decreased only slightly and continues to affect nearly 40 percent of children under 5-years of age. The prevalence of wasting or acute malnutrition actually increased from 8.4 percent in 2005 to 10.9 percent in 2010. The prevalence of children that are underweight, which is a composite measure of chronic and acute under-nutrition, was about the same in 2010 (28.3 percent) as it was in 2005 (28.1 percent). Anemia continues to affect more than half (55.1 percent) of all children under the age of five in Cambodia. Table 2: Selected Child Health Indicators and Cambodia MDG Targets Indicator CDHS 2000 CDHS 2005 CDHS 2010 CMDG target 2015 Infant mortality rate (per 1,000 births) 95 66 45 50 7 National Institute of Statistics, Directorate General for Health, & ICF Macro. Cambodia Demographic and Health Survey 2010. Phnom Penh, Cambodia and Calverton, Maryland, USA: National Institute of Statistics, Directorate General for Health, and ICF Macro, 2011. SMBCI PROJECT FINAL PERFORMANCE EVALUATION 11 Under 5 mortality rate (per 1,000 live births) 124 83 54 65 Percentage of children < 5 underweight 38.5 28.1 28.3 19.0 Percentage of children < 5 stunted 49.8 42.7 39.9 - Percentage of children < 5 wasted 16.8 8.4 10.9 - Percentage of children < 5 with anemia 63.4 61.9 55.1 - Percentage of children < 5 with symptoms of ARI in the last two weeks 19.8 8.5 6.4 - Percentage of children < 5 with diarrhea in the last two weeks 18.9 19.5 14.9 - Percentage of infants (0-5 months) exclusively breastfed 11.4 60.0 73.5 70.0 Prevalence of early initiation of breastfeeding (within one hour) 11.0 35.1 65.2 - Percentage of children (12-24 months) using bottle with nipple - 11.8 24.7 - Percentage of children (6-59 months) supplemented with vitamin A 28.5 34.5 70.9 90.0 Cambodia’s response to the HIV/AIDS epidemic has been remarkably successful, as evidenced by the reduction in HIV prevalence from a high of 2.4 percent among the general population in 1998 to 0.8 percent in 2010. The epidemic is concentrated among high-risk groups, specifically EWs, men who have sex with men (MSM), high-risk urban men (HRUM), and clients of commercial sex workers. The prevalence of HIV among these high-risk groups ranges from 1.6 percent (among men who have sex with women) to 13.9 percent (among EWs with 15 or more clients per week). The original 2015 target for CMDG 6, “Combat HIV/AIDS, malaria, and other diseases to reduce HIV prevalence to 2.3 percent” was modified to a target of less than 0.9 percent. This MDG has already been met. According to the Ministry of Planning’s 2010 Update for Achieving Cambodia’s Millennium Development Goals, prevention efforts should continue to focus on the most at-risk populations as well as spousal and mother-to-child transmission which represent approximately 43 percent and 30 percent of new infections, respectively. The CDHS 2010 includes a number of HIV/AIDS knowledge and sexual behavior questions to inform national prevention priorities. For example, 77 percent of women and 82 percent of men believe that a woman is justified in refusing to have sex with her husband if she knows he has sex with other women. Ninety-four percent of women and 96 percent of men believe that a woman is justified in asking her husband to use a condom if he has an STI. Among men with two or more partners in the past 12 months, 40 percent report having used a condom during their last encounter. The CDHS also reports that 11 percent of men have paid for sexual intercourse. Eighty-two percent of men who have paid for sex in the past year report using a condom during their most recent, paid sexual encounter. The RGC is committed to improving the health of its citizens, and efforts to improve services and health-related behaviors are country-led. The Cambodia Health Strategic Plan 2008-2015 sets forth an ambitious set of priorities and activities that address current deficiencies in the health care system. The draft National Strategy for Reproductive and Sexual Health for Cambodia 2012-2016 aims to build on and consolidate considerable achievements made over the past decade in reproductive and sexual health through improvements in RH coverage and the provision of a supportive environment in which this can occur. Additionally, the MOH recently launched the Fast Track Initiative: Road Map for Reducing Maternal and Newborn Mortality as a means of scaling up evidence￾based interventions aimed at sustaining improvements in the maternal mortality ratio and reducing the neonatal mortality rate. Additionally, the Cambodia Child Survival Strategy outlines approaches to reducing child mortality SMBCI PROJECT FINAL PERFORMANCE EVALUATION 12 in Cambodia through the universal coverage of a limited set of essential evidence-based, cost-effective interventions. B. Health Financing Nature of Market for Health Products and Services in Cambodia The predominant use of private sources of health products and services in Cambodia is consistent with trends in the region more broadly. Approximately 78 percent of Cambodians seek care in the private sector; however, compared with its neighbors in the region, it is hypothesized that more Cambodians seek care in lower level private sector outlets (pharmacies, drug shops, individual provider practices) than their neighbors in wealthier countries where private sector care is likely to be sought in higher level private clinics or hospitals. Table 3: Source of Care in Public/Private Sectors: Cambodia and Regionally8 (DHS Data: Cambodia 2005, Indonesia 2007, Philippines 2003, Vietnam 2002; NHA Data from WHO 2006) The CDHS also documents a strong preference in the use of the private sector for initial treatment of illnesses or injuries. In urban areas, almost 66 percent of respondents reported first seeking treatment in the private sector for illness or injury, compared with almost 26 percent who chose the public sector for their first treatment. In rural areas, a slightly lower percentage (56 percent) sought treatment in the private sector while 29 percent of rural residents first accessed treatment in the public sector. The private sector is also a significant source of family planning commodities. According to the CDHS 2010, across all methods, slightly over half of all users of modern contraceptive methods obtain their contraceptive commodities from the public sector, but a significant 30 percent report the private medical sector (hospital, clinic, or pharmacy) as their source. 9 Health Financing Context Cambodia has extremely high rates of out-of-pocket spending for health care, reflecting high usage rates in private sector facilities as the first point of care-seeking for illness, user fees charged in public sector facilities, as well as transportation and other related costs. In 2005, 60 percent of total expenditures on health in Cambodia came from out-of-pocket spending, similar to rates in Vietnam (64 percent) but significantly higher than Thailand (28 percent) and Indonesia (35 percent). Per capita out-of-pocket spending on health has been estimated at between $15 and $18 per year in Cambodia (World Bank Poverty Assessment 2006 and WHO NHA). The institution of user fees in public sector hospitals reportedly reduced the prevalence of informal payments in these facilities, but concerns remain regarding the high financial barriers to accessing health care faced by low income people who make up a large portion of Cambodia’s population. As a result of these concerns, donors and the RGC has launched several health financing mechanisms including health equity funds, contracting, vouchers, 8 UNICEF, Non-State Providers and Health Service Delivery in the EAP Region, April 2010 9 Cambodia Demographic and Health Survey, p. 81 SMBCI PROJECT FINAL PERFORMANCE EVALUATION 13 community-based health insurance, and conditional cash transfers. Health Equity Funds provide a mechanism through which NGOs reimburse user fees to eligible clients who receive services in the public sector. Health equity funds reportedly will not reimburse private facilities, even if the services they provide are unavailable in public sector facilities. 10 Dual Practices and Regulatory Issues It is well documented that public sector health care workers are poorly paid in Cambodia and as a result, many public health civil servants conduct private practices concurrently to augment their official government salaries. One study found that in Phnom Penh, 90 percent of public sector physicians’ incomes were derived from their private practices rather than their public sector positions.11 There are serious concerns regarding the poor quality of services in the private sector, which remains largely unregulated. As a result, the RGC has recently instituted and enforced regulations in an attempt to address quality issues in the private sector. For example, during a field visit, the evaluation team was told that private sector midwives had recently been prohibited from assisting with deliveries in their private clinics, as part of an effort to encourage institutional delivery in government health centers. During an interview with the Primary Health Care (PHC) Director in Kampong Chhnang, the issue of unlicensed or unregistered private practices was raised, and PSI was praised for only working with “official” or registered private providers as part of its SQHN. This broadly stated desire on the part of the public sector to reduce unlicensed practices and narrow the scope of practice of licensed, registered practitioners may create an atmosphere of uncertainty for private health practices. Quality issues extend beyond the service delivery context to that of pharmaceutical products as well. The MOH reportedly lacks the capacity to enforce existing quality standards for pharmaceuticals, and there is general agreement that substandard and counterfeit drugs are routinely sold in drug shops and other unregistered outlets. 10 Per the Riggs-Perla et al mid-term review 11 Cited in Ferrinho, 2004 SMBCI PROJECT FINAL PERFORMANCE EVALUATION 14 EVALUATION METHODS To gather the information needed to evaluate the PSI SMBCI Project, the team utilized a number of resources. Personal interviews were conducted by the team using a set of questionnaires developed as discussion guides for each category of stakeholder or potential stakeholder. Use of the questionnaires helped to ensure the consistency and completeness of information gathered during the interview process. (These questionnaires are found in Appendix C.) Relevant stakeholders were identified for the team by USAID/Cambodia’s health office technical staff and PSI project staff. The team also sought representatives from commercial pharmaceutical and distribution companies for their input. Those interviewed by the team for this evaluation included representatives of national and provincial level public sector health offices; technical staff of international donors working in HIV/AIDS prevention, RH/FP, and CS; USAID technical staff working in health; PSI/C staff; representatives of NGO stakeholders; private sector service providers participating in the SQHN; representatives of international pharmaceutical companies selling contraceptive products in Cambodia; representatives of commercial pharmaceutical and consumer product distribution companies; and private sector shop and pharmacy owners. (See Appendix B for a list of persons contacted.) Interviews were conducted during project site visits to Kampong Chhang, Pursat, and Battambang provinces as well as in Phnom Penh and surrounding areas. Documents consulted by the team during its evaluation of the PSI SMBCI Project include among others: USAID health strategy documents; program authorization documents, and recent health and HIV/AIDS program evaluations; PSI SMBCI Project Cooperative Agreement with USAID/C; PSI SMBCI Project annual and semi￾annual reports to USAID/Cambodia; PSI SMBCI Project annual work plans; and national MOH strategy documents. Research results relevant to the evaluation were accessed through a review of: PSI SMBCI Project research reports; CDHS reports for 2000, 2005, and 2010; and other project reports. (A list of all documents consulted is contained in Appendix D.) PSI technical staff also made a series of Power Point presentations to the evaluation team in which the project's social marketing goals, activities, outputs and achievements, and issues of particular importance were described in relation to HIV/AIDS prevention, RH and FP, and CS. The team acknowledges the following limitations inherent in its evaluation of the PSI SMBCI Project: • The evaluation was conducted during an in-country work period of 20 days • Due to time constraints, project site visits were limited to three provinces (Kampong Chhnang, Pursat, and Battambang) and the Phnom Penh area • There are no retail audit data available for the pharmaceutical sector in Cambodia • No market research has been done that establishes consumer profiles for PSI SMBCI Project products • There are no data available that allow the team to reliably estimate a denominator for the population at high risk of HIV/AIDS SMBCI PROJECT FINAL PERFORMANCE EVALUATION 15 STUDY LIMITATIONS The task of evaluating the USAID funded, PSI implemented SMBCI Project is complex due to the nature of the project. The project uses multiple interventions implemented across multiple health areas and supports these interventions through multiple sources of funding that are intertwined throughout project activities. For example, the network of private sector SQHN clinics provides services in RH/FP, often in STI treatments and referrals, and sometimes in the treatment of diarrheal disease among children. Virtually all SQHN service providers work in a public sector health center as well as in their private sector network clinics. The project training of clinic providers, therefore, likely strengthens both public and private sector service delivery infrastructure. USAID funded ten percent of the work of the project with SQHN in two targeted provinces, while complementary funding from another donor source supports the clinic’s network in other provinces. Available funding from a large anonymous donor may be used for the promotion and provision of long-term contraceptive methods within private sector SQHN clinics, but not for the promotion and provision of short-term methods nor public sector service delivery. The evaluative task is further complicated by ambiguities in the statements of purpose and expected outcomes of the SMBCI Project. The Revised Technical Application appended to the Cooperative Agreement, signed in 2008, between USAID/Cambodia and PSI for the implementation of the SMBCI Project, states that the purpose of the project is "to increase consistent and correct use of high-quality essential health products and services among priority at-risk populations, especially the poor and vulnerable." Further, the work of the project "will focus on rural areas." The Application document continues by stating that "the overarching strategy to achieve project objectives lies in developing long-term sustainable markets." Development of a sustainable market on the basis of providing products and services to the rural poor seems inherently contradictory to the evaluation team. Additionally, allocation requirements associated with multiple sources of donor funding for the SMBCI Project limits the project's ability to make strategic investments of project resources both among and within health areas. For example, the amount of PEPFAR funding for Cambodia ensures that approximately one-half of project activities are devoted to HIV/AIDS prevention, while earmarked funds for RH/FP and CS each account for approximately one￾quarter of project activities. These proportional allocations occur within the context of a country where HIV/AIDS prevalence is 0.8 percent of the general population12 (but higher among MARPs: HIV/AIDS prevalence ranges from 1.6 percent for men who have sex with women to 13.9 percent for female EWs with more than 15 clients per week),13 and where the prevalence of modern method contraceptive use among women of reproductive age is only approximately 35 percent. 14 Additionally, people aged 0 to 20 years represent approximately 45 percent of the total population,15 whereby stunting affects 40 percent of children under age five16 and an estimated 55 percent are anemic.17 There are several guiding principles that have influenced the team's evaluation of the performance of the PSI SMBCI Project. These principles include the team's definition of good marketing practices, effective market segmentation, TMAs, sustainability, and social franchises/branded networks. The team's statement of these principles can be found in Appendix F. 12 National Center for HIV/AIDS Dermatology and STD Report Estimations and Projections of HIV/AIDS in Cambodia 2010-2015, October 2011. 13 National Center for HIV/AIDS Dermatology and STD Report Estimations and Projections of HIV/AIDS in Cambodia 2010-2015, October 2011 and FHI & NCHADS, 2010, Report on BROS, Khmer Study. 14 CDHS, 2010, page 75 15 CDHS, 2010, page 10. Under five years of age = 11.3 percent of total population, 5-9 years = 11.1 percent, 10-14 years = 12.1 percent, and 15-19 years = 10.6 percent. 16 CDHS, 2010, page 149 17 CDHS, 2010, page 162 SMBCI PROJECT FINAL PERFORMANCE EVALUATION 16 FINDINGS, CONCLUSIONS, AND GAPS OR MISSED OPPORTUNITIES HIV and AIDS 1. Increased Access to Condoms and Lubricant The SMBCI Project sought to increase access to condoms and lubricants for HIV/STI prevention, as stated in the statement of work. In this document, three key objectives specifically related to increasing access to condoms were identified to facilitate greater sustainability in the Cambodian condom market: (1) creating space in the private sector by graduating Number One brand condoms to full cost-recovery; (2) continuing to support the OK brand for low income Cambodians at risk for HIV; and (3) working with the public sector to increase the efficiency of free condom distribution for the poorest segments of the population.18 This section will examine whether PSI/C effectively pursued these objectives over the life of the project, in addition to assessing its performance toward the outcome of increasing access to condoms and lubricants in the Cambodian market. Access: Geographic and Financial The CDHS confirms that condom use for FP purposes remains extremely low in Cambodia, at 2.7 percent of the CPR of women of reproductive age (WRA). Therefore, the vast majority of condoms sold in Cambodia can be assumed to be used for disease prevention. Cambodia has a high acceptability of condoms as a disease prevention method. During the 100 percent Condom Use Program, extremely high levels of condom use with commercial sex workers (over 90 percent) contributed to the reduction of HIV prevalence rates among adults 15-49 years from a peak of approximately 2.4 percent to less than 1 percent currently.19 PSI distributes condoms in Cambodia through a variety of different outlets as part of a strategy to ensure coverage of high-risk populations for the purposes of HIV/AIDS prevention. In 2011, the vast majority of OK condoms20 were distributed through pharmacies (31.4 percent) and United Health Network (UHN) NGOs (31.9 percent), followed by high-risk venues (12.4 percent). Table 4: PSI OK Condom Sales by Type of Outlet, 2011 Outlet Type Establishment Type OK Condom Sales FY2011 Percent of Total High-Risk Venue Barbeque restaurants, beer gardens, brothels, club/restaurants, guesthouses, hotels, massage/karaoke establishments 12.4% Health Care Provider Consultation Room, health care room, maternity/midwife practice 8.7% Clinic/Hospital Clinic, hospital, clinical pharmacy 7.1% 18 PSI application for the Social Marketing and Behavior Change intervention Project – Evidence to Action Partnership, p. 16 19 NCHADS, Estimation of the HIV Prevalence among General Population in Cambodia, 2010. July, 2011. 20 According to PSI, OK condoms are positioned for disease prevention to MSM and high frequency commercial sex workers. SMBCI PROJECT FINAL PERFORMANCE EVALUATION 17 Commercial Outlet Drink shop, mini-mart, convenience store, village shop, market stall, drug store, grocery store 2.4% Pharmacy Pharmacy 31.4% Street Vendor Street vendor 0.3% United Health Network NGO distribution 31.9% Other Consumer, distributor, SQHN provider, other 5.9% PSI has focused particular efforts on expanding the physical access of socially marketed condoms through non￾traditional venues. The rationale for this strategy is that high-risk urban men, EWs, and MSM are more likely to use a condom if it is easily available nearby the establishments where high-risk sexual activity takes place. PSI’s annual MAP Study attempts to assess condom availability in high-risk locations. The most recent study concluded that “condom coverage remains high around high-risk venues, with PSI condoms being available in most areas.”21 PSI’s approach to reaching high-risk groups such as MSM has been through UHN partners, in addition to ensuring availability at non-traditional venues. UHN partners receive socially marketed condoms at wholesale prices and are expected to mark-up and distribute them to users or other retailers. The team observed that at least one UHN partner, Men’s Health Social Services in Battambang, was distributing condoms received from PSI at wholesale prices. Due to the absence of Information Management System (IMS) or other commercial retail audit data for Cambodia, it was difficult to obtain reliable information on the availability of commercially distributed condoms within the time frame available to the evaluation team. During site visits in Phnom Penh and Battambang, the team visited a small number of pharmacies and found that both commercial and socially marketed (OK and Number One) condoms were readily available in those locations. However, per other surveys the team found that less than 30% of pharmacies selling any commercial brand condom, and less than 15% selling more than one commercial brand. PSI noted that the availability is much lower in non-traditional outlets like hotels, guesthouses etc. In reviewing PSI research and planning documents, it is not evident that PSI employed a geographic segmentation strategy, either between Number One and OK or between the two socially marketed brands and commercial sector brands. Review of project documents and discussions with PSI staff indicated that PSI’s strategy for physical access emphasized the presence and availability of socially marketed condoms in traditional and non-traditional outlets, rather than segmentation or focus in particular geographical areas or types of outlets. Financial Access Condoms have both high acceptability and physical accessibility in Cambodia, but a further component of overall availability includes the financial ability to access condoms by high-risk groups. Financial access is considered both in terms of the purchasing power of high-risk individuals, including EWs, HRUM, and MSM as well as the relative pricing of socially marketed condoms, both in relation to commercial condoms and to other comparable consumer goods. The systematic collection of data on pricing, through the PSI commissioned market survey conducted by A.C. Nielsen as well as anecdotal reports of market signals around pricing, are discussed in this section. 21 Cambodia 2011 MAP study evaluating the availability of PSI condoms and lubricants, 7th Round, p.21. SMBCI PROJECT FINAL PERFORMANCE EVALUATION 18 Per capita income in Cambodia is low, at $39 per month, 22 and extreme income disparities exist between lower and higher quintiles of the population. While poverty rates have seen improvements in the past decade, over one-third of Cambodia’s population remains under the poverty line. The imperative to ensure the affordability of socially marketed products was repeatedly emphasized by stakeholders during the team’s field visits and interviews. Accordingly, PSI’s pricing strategy with OK and Number One condoms has prioritized keeping prices low to ensure financial access by target groups. PSI TRaC surveys of MSM, EWs, and HRUM show variability in the monthly incomes of these target groups based on residence, and in the case of EWs, their workplace settings. MSM appear to have the lowest income, averaging $94 per month with a low of $59 per month in Battambang and a high of $144 per month in Sihanoukville in 2010. EWs in the Phnom Penh survey reported incomes of $226 per month in karaoke establishments, $192 per month in beer gardens, and $120 per month in massage parlors. Of the three target groups for condom promotion, HRUM have the highest incomes averaging between more than $300 per month in Phnom Penh to $204 in Sihanoukville in 2010. 23 Another factor considered by the team in terms of the affordability of condoms, particularly in high-risk sexual encounters between HRUM and their sweethearts, was the overall cost of the encounter, which likely includes drinks, dinner, transportation, and payment for a guesthouse room in addition to the condom. When these factors are considered, it would appear that the cost of the condom is a small contributor to the total cost of the high￾risk sexual encounter for HRUM. This perspective was validated by an official at the National Center for HIV/AIDS, Dermatology and STD (NCHADS), who opinioned that “if a man can pay for sex, he can pay for a commercial condom.” The positive association between wealth, risk behavior, and HIV prevalence is supported by a recent study on HIV/AIDS prevalence in Cambodia, 24 which found that men with the highest scores in wealth also had the highest rates of HIV prevalence. In addition, among urban men, older men aged 30 – 49 had a higher prevalence than those aged 20-29; and employed men were more likely to be HIV positive, with manual labor/service sector workers having higher prevalence rates than professional/technical/sales workers. Extremely low rates of HIV prevalence were found among male agricultural workers and the unemployed. Another common measure of financial access is the comparable cost of consumer products that users purchase and use on a regular basis. The team was only able to do limited, opportunistic sampling of consumer prices in outlets selling PSI-branded condoms, but found in a village shop in Pursat that items such as beer ($0.375/can of the cheapest brand and $0.50/can, Black Panther or Angkor brands), cigarettes ($0.70/pack Fine brand), shampoo ($0.05/individual use sachet or $0.12/pack of three sachets), and chips ($0.12/package) etc. were priced higher than the unit price of either OK or Number One condoms. Sales of Condoms Over Life of Project Total sales of PSI/C managed condom brands rose over the life of the project, from 19.7 million units in FY 2009 to 25.4 million units in FY 2011.25 This outcome is in contrast with PSI’s own prediction in its application that 22 Cambodia Socio-economic Survey, 2009. This figure underestimates income and overestimates both the degree and extent of poverty in Cambodia for a few reasons. Firstly, income fluctuates substantially by season, especially among the majority of the population engaged in agricultural production. Furthermore, determining household and per capita income is difficult in a country like Cambodia, where most households have some income from self-employment and there are no depreciation of investments such as tools and animals, resulting in a relatively large number of households with negative incomes. Additionally, respondents may be unwilling to report actual income from self-employment due to fear of attention from tax authorities. Mean disposable income is significantly lower than mean consumption (or expenditure). According to the 2009 CSES, the average monthly consumption is $62 per capita. 23 PSI TRaC surveys for MSM, EW, and HRUM, 2010, and Data on income collected in the TRaC survey are likely to be subject to the many of the same limitations identified in the CSES 2009, including under-reporting due to multiple streams of income from informal employment and agriculture. 24 Sobheap, JAIDS 2012 25 PSI Indicator Data, FY 2009 - 2011 SMBCI PROJECT FINAL PERFORMANCE EVALUATION 19 “…sales of socially marketed condoms will decrease over the life of the project [while] the total condom market will experience modest grown with better targeted subsidies.”26 The market share of commercial condoms also increased during this period from 10 percent of the total market in 2009 to 16 percent in 2010. The share of public sector condoms distributed during this period is also believed to have increased slightly. PSI’s FY 2012 Annual Work Plan describes metrics for monitoring the project’s progress toward the TMA goals. These include: • Is the market reaching an increasing percentage of the at-risk population? (Impact) • Is the market becoming less dependent on subsidies? (Sustainability) • Is the market evolution benefiting the poor and vulnerable? (Equity) Table 5: Total Condom Sales in Cambodia, By Sector 2008 – 2012 (Mills) Source: PSI Indicator Data Sector 2008 2009 2010 2011 2012 Social Marketing 26.5 19.7 18.2 25.4 8.75 (Q1&Q2) Commercial 2.5 2.5 3.8 Not avail Not avail Public 2.0 1.7 1.1 Not avail Not avail TOTAL 31.0 23.9 23.1 Overall, increases in the total condom market and modest increases in the commercial sector show progress toward improved sustainability in the overall condom market in Cambodia over the past five years. However, subsidized27 PSI promoted brands (Number One and OK) continued to dominate overall sales in private outlets. Market Segmentation and Pricing The three PSI-marketed condoms––OK, OK Plus, and Number One––dominate the Cambodian condom market. As a result of efforts to reposition Number One as a brand “to impress your sweetheart,” PSI increased its unit (one condom) price from a baseline of $0.019 ($0.075 per 4-pack28) to $0.083 ($0.25 per 3-pack29) in April 2010, 30 where it currently remains. The pricing of OK condoms has remained fixed since 2008 at $0.12 per 3-pack.31 As a result of what PSI now sees as an ill-conceived decision to raise Number One’s price over 300 percent without significantly changing the value proposition of the product, sales of Number One fell and the market shares of PSI’s two socially marketed condoms flipped between 2008 and 2011. Sales of Number One dropped from 10 million in 26 Ibid 27 PSI reports that it has achieved cost recovery for both of these brands as of FY 2012. This means that their sales revenue from these brands covers product, packaging, and shipping costs, but not promotion or other overhead expenses. For the purposes of this report, both Number One and OK products are considered to be subsidized by donor resources as the current PSI definition covers only cost of goods sold (COGS) and not total costs associated with each product. 28 300 Riel ($1.00 = 4,000 Riel) 29 1000 Riel ($1.00 = 4,000 Riel) 30 Interim price increases in 2008 were also made. 31 500 Riel ($1.00 = 4,000 Riel) SMBCI PROJECT FINAL PERFORMANCE EVALUATION 20 FY 2009 to 7.8 million32 in FY 2011, while sales of OK rose from 9.6 million in FY 2009 to 17.5 million in FY 2011. The brand positioning of OK as a condom that offers “value for money” proved stronger than the newly repositioned and costlier Number One, and caused consumers to shift between the two subsidized social marketing brands. It is not clear that the efforts to reposition Number One had any significant effect on the total market, or commercial brands in particular, as the new high-priced Number One still remains significantly below the lowest price commercial brand, Okamoto. PSI contends that a significant price gap exists in the Cambodian condom market between Number One (at $0.25 per 3-pack) and commercially available products (starting at $1 per 3-pack). The HIV MAP 2011 indicates the following prices: OKAMOTO $1.55, Mood $1.52, Strast $1.02, Romantic $1.40, and Kamasutra $1. With a less representative sample, the team observed a range of condom prices at private pharmacies in Phnom Penh and Battambang, and are presented in Appendix G. Overall, the team’s brief visits to pharmacies indicated that at least in urban areas, and Phnom Penh specifically, a range of both socially marketed and commercial condom brands were available. Pricing varies significantly, including for OK and Number One, which are sometimes sold at a price different from the suggested retail price, and commercial condoms are present in the urban market at between $0.40 - $1.00 per 3-pack compared with OK and Number One at $0.12 and $0.25 respectively. Commercial brands appear to emphasize specialized product attributes including thinness, ribs, dots, flavors, and scents. Table 6: Pricing Analysis and Recommendations for Socially Marketed Condoms (A.C. Nielsen) Number One Classic 450 – 2400 1350 1947 1000 Number One Flavors 500-1625 1480 2310 1000 OK 300-1625 900 n/a 500 PSI commissioned a market segmentation analysis by A.C. Nielsen, which offered several interesting insights related to pricing among urban male, high-income and high-volume users. According to the Nielsen analysis, all PSI brands are underpriced according to consumer’s willingness to pay. Furthermore, Nielsen found that the new flavored variants of Number One (strawberry, banana), OK, and the commercial Okamoto condom were perceived by consumers to be of similar quality. Okamoto condoms were associated with greater emotional benefits, reflecting the effective positioning of that brand, and OK and Number One were both associated with a low price and ease of use. 33 The report offered the following conclusion regarding pricing: “Although the price of new [Number One] and its flavors can be sold at $0.375, PSI decided to keep the current price at $0.25 for all and will observe the market for 6 months before making higher adjustment. The focus is on shifting value proposition rather than price; retailers will push the price up if they see the value. 34” The evaluation team confirmed that both Number One Classic and flavored variants are currently being sold in the market for $0.25 per 3-pack of condoms. MSM and Condom Market Segmentation 32 The 7.8 million in sales in FY 2011 includes the donation of six million Number One condoms to the RGC (see PSI Annual Report for FY 11, p.12). 33 A.C. Nielsen Vietnam, A Market Research Study on Number One & OK Condoms in Cambodia Market 34 A.C. Nielsen Vietnam, A Market Research Study on Number One & OK Condoms in Cambodia Market SMBCI PROJECT FINAL PERFORMANCE EVALUATION 21 Since its launch in FY 2009, OK Plus (a package of two condoms and two sachets of lubricant priced at $0.12) has been marketed to MSM and high-frequency commercial sex workers as a disease prevention product. The condom and lubricant (provided by the Global Fund to Fight AIDS, Tuberculosis and Malaria (GFATM)) are packaged together with the objective of increasing its ease of use and safety in anal sex. Sales of OK Plus rose in FY 2009 from 223,206 (data point may include some Number One Plus condoms which were discontinued) to 1.8 million in FY 2010, and fell to 1.2 million in FY 2011. No Findings: PSI/C’s UHN partners distribute OK Plus directly to MSM high-risk venues and through outreach activities. Due to the lack of client profile data, it is difficult to determine whether OK Plus is meeting the demand of this target group or to assess the extent of brand switching between OK Plus and other PSI supported social marketing brands and commercial brands. Okamoto also commercially markets two varieties of condoms with extra lubrication in the Cambodian market, as well as two types of lubricant in pump-top containers (though these products are not specifically marketed to MSM). 2. Improved Knowledge, Awareness, and Supportive Attitudes to Change Behaviors (Behavior Change, Increased Demand, See Detailed Findings in Appendix H.) The SMBCI Project aims to increase the correct and consistent use of condom among high-risk populations using a TMA toward the increased supply of, accessibility to, and demand for HIV prevention products and services that includes an integrated behavior change communication (iBCC) strategy featuring multiple mass media and IPC interventions. While there are a number of instances where indicators have changed or been dropped since the start of the project, either at the request of USAID or program management, in general the SMBCI Project has demonstrated progress in meeting and exceeding their objectives in changing awareness, knowledge, supportive attitudes, and behaviors. While the SMBCI Project’s social marketing of condoms is likely to have contributed to the overall decline in HIV prevalence in Cambodia, it is more difficult to quantify PSI/C’s contribution to changes in HIV prevalence (and incidence) within MARPs. Ultimately, the goal of PSI’s Behavior Change Framework is to reduce new HIV infections in Cambodia by increasing consistent condom use among EWs, male clients, and MSM as measured by the four outputs and 10 indicators in the performance management plan (PMP) listed below. Table 7: HIV/AIDS Outputs and Indicators35 Output 1: Increase consistent condom use and services Indicator Baseline Cumulative LOP Target Source Percent of EWs report consistent condom use with sweethearts 41% 46% 55% EWs TRaC Percent of HRUM report consistent condom use with EWs 85% 66.3% 72% HRUM TRaC Percent of HRUM report consistent condom use with sweethearts 40.7% 2005 58% 2008 66.3% 72% HRUM TRaC 35 Extracted from PSI/C SM for BCI Annual Report Oct 2010-Sept 2011 SMBCI PROJECT FINAL PERFORMANCE EVALUATION 22 Percent of MSM who report consistent use (sh - short hair) (lg – long hair) 44% sh 79% 85% 2006 BSS MSM TRaC Output 3: Reduce high-risk behaviors for STI/HIV prevention Indicator Baseline Cumulative LOP Target Source Percent of EWs who believe condoms are appropriate & necessary with sweethearts 87% 85% 95% EW TRaC Mean score of scaled beliefs (trust) constructed among EWs and sweethearts 2.81 3.0 3.4 EW TRaC Percent of HRUM who believe condoms are appropriate & necessary with sweethearts 93% 91% N/A HRUM TRaC Mean score of scaled beliefs (trust) constructed among HRUM & sweethearts 3.12 3.02 3.4 HRUM TRaC Output 4: Increased national capacity to manage and sustain social marketing programs Indicator Baseline Cumulative LOP Target Source Number of local NGOs contracted & supported to conduct effective BCC with MARPs & assist with social marketing & services 0 21 (all UHN) 130 (43 unique NGOs) Project UHN Reports Output 6: Conduct targeted outreach activities using evidence to action approach in entertainment establishments Indicator Baseline Cumulative LOP Target Source Number of MARPs reached with IPC or small group evidence-based interventions 55,921 829,773 825,755 Project MIS data Measured improvements were seen in increased condom penetration and sales (OK, OK Plus, and Number One), protective behaviors, and appropriate health-seeking behaviors through PSI/C’s annual TRaC surveys and routine monitoring of their NGO partners’ performance. These data, when combined with results from the national Behavioral Surveillance Survey (BSS), suggest that PSI/C’s integrated behavior change (iBCC) strategy and behavior change interventions are making a difference. The relatively recent implementation of the Law on Suppression of Human Trafficking and Sexual Exploitation led to the strict enforcement of brothel closures, resulting in a significant increase in and dispersion of entertainment establishments such as massage parlors, karaoke venues, and beer gardens and contributing to an increase in estimated EWs at risk for HIV. Despite these challenges and the reduced penetration of PSI condoms inside beer gardens in Phnom Penh from 35 percent in 2008 to 15 percent in 2011, the availability of PSI condoms at convenience shops within 50 meters of beer gardens and karaoke and massage parlors has increased from 76 percent in 2008 to 80 percent in 2011 and from 51 percent in 2008 to 84 percent in 2011, respectively. 36 The number of condoms sold or distributed through PSI social marketing was above 23 million in both FY 2009 and FY 2010.37 Condoms and lubricant (for MSM) are widely available 36 HIV MAP 2011 37 This represented a decline from approximately 31 million condoms sold or distributed in FY 2008. PSI increased the price of their Number One condom in FY 2009 as part of a long-term sustainability strategy that segmented consumers by their ability to pay and this had an immediate impact on sales volume. High stock levels in 2008 also contributed to the reduction in condoms sold in 2009. Tracking of behavioral trends through PSI research indicates there was no decrease in consistent condom use due to the price increase. USAID/Cambodia HIV/AIDS Portfolio Evaluation USAID/Cambodia Public Version Final Report, Public Version July 2011 SMBCI PROJECT FINAL PERFORMANCE EVALUATION 23 through both condom social marketing in more than 5,400 traditional and non-traditional outlets (FY 2010) and free distribution to MARPs. The results of the HIV Sentinel Surveillance (HSS) 2010 show HIV/AIDS prevalence to be 4.1 percent for female EWs with fewer than 14 clients per week and 13.9 percent for female EWs with more than 15 clients per week.38 The HSS data for MSM showed HIV prevalence rates ranging from 3.1 percent for “short haired” MSM to 3.6 percent for “long haired” (transgender) MSM.39 In 2010, additional data on HIV prevalence among MSM and HRUM became available from the Behavioral Risk on site Serosurvey (BROS) among at Risk Khmer’ Men study conducted by FHI and NCHADS as part of the national HIV BSS. This study found the prevalence of HIV/AIDS to be 1.6 percent in men who have sex with women, 2.1 percent for men who have sex with men only, and 2.2 percent for men who have sex with men and women.40 FHI reported that between 0.5-1.9 percent of the male population in six Cambodian cities were sexually active MSM, with half the surveyed men reporting also having female sex partners within the past 30 days. Estimating the size of the most at-risk populations is important for determining denominators for arriving at meaningful measures for coverage, reach, and magnitude to demonstrate impact in HIV transmission and prevalence reductions in these populations. Results from the NCHADS Estimations and Projections of HIV/AIDS in Cambodia 2010-2015 Report (October 2011) showed that consistent condom use among female EWs and beer promoters (indirect sex workers) steadily increased during the period between 1997 and 2007.41 Analysis revealed that the level of consistent condom use was 81.5 percent among female EWs who reported up to 14 partners per week and 89.2 percent among female EWs reporting more than 15 partners per week.42 PSI/C’s 2011 TRaC survey among EWs also documents persistently high consistent condom use in the last three months, 93 percent in 2009 and 92 percent in 2011. Moreover, according to the 2011 TRaC, 72 percent of EWs reported having had an HIV test in the past 12 months, representing a significant increase in positive behavior change from 64 percent in 2009.43 According to PSI/C, 78 percent of men have had sex with a commercial partner, with 6.2 commercial partners and on average and 3.4 sex acts during each encounter in the last 12 months.44 While 94 percent reported consistent condom use with commercial partners, only 48 percent reported consistent use with sweethearts.45 Consistency in prevalence and behavior change data for HIV at-risk populations between NCHADS’ HSS, BSS, and PSI/C’s TRaC survey results affirms the reliability of the data (despite accepted limitations in self-reported behaviors shared by all data sources) and the SMBCI Project’s contribution to mitigating the concentrated epidemic in Cambodia. Effective targeted behavior change interventions with selected at-risk sub-populations such as “sweethearts” and EWs with more than 14 partners per week continue to be a high project priority. Like PSI/C’s TRaC survey data, NCHADS HSS trends in the rate of consistent condom use with sweethearts show little change since reaching a peak in 2003 of 49-53 percent. In 2010, self-reported consistent condom was still no higher than 48 percent. 46 TRaC research conducted by PSI/C with sexually active men who visit entertainment establishments and women working in karaoke bars in Phnom Penh also found declines in self-reported consistent condom use with 38 National Center for HIV/AIDS, Report of the HIV Sentinel Surveillance 2010 39 National Center for HIV/AIDS Dermatology and STD Report Estimations and Projections of HIV/AIDS in Cambodia 2010-2015, October 2011 * Reporting HIV Incidence is reliant on modeling. Since HSS data are routinely collected, it may be more practical to adhere to the HIV prevalence indicator. 40 BROS Khmer 2010 Behavioral Risks On-Site Sero-survey among At-Risk Urban Men in Cambodia, FHI & NCHADS, 2010 41 National Center for HIV/AIDS Dermatology and STD Report Estimations and Projections of HIV/AIDS in Cambodia 2010-2015, October 2011 42 National Center for HIV/AIDS Dermatology and STD Report Estimations and Projections of HIV/AIDS in Cambodia 2010-2015, October 2011 43 PSI/C TRaC survey among EWs 2011 44 PSI Cambodia HIV/AIDS Prevention Program Review July 2012 (ppt) 45 PSI TRaC report 2011 46 National Center for HIV/AIDS Dermatology and STD Report Estimations and Projections of HIV/AIDS in Cambodia 2010-2015, October 2011 SMBCI PROJECT FINAL PERFORMANCE EVALUATION 24 “sweethearts in the last three months,” from 51 percent in 2009 to 46 percent in 2011.47 The study further revealed that the major barrier to consistent condom use in these relationships concern “beliefs about background and trust in partner.” The same study also found that condom use is significantly lower among women who do not believe that condoms are “appropriate to use with sweethearts.”48 A new campaign reframes the connection between trust and condom use in sweetheart relationships by associating condom use with mutual trust. Messages aim to dispel the notions that condoms are not needed if a man trusts his sweetheart and that his sweetheart will not trust him if he suggests using condoms with her. The intervention focuses on a few short and simple messages delivered one at a time, repeatedly, to increase recall of campaign and interpersonal communication (IPC) messages. Exposure to IPC has increased steadily since 2008. In 2011, more than half of the TRaC respondents reported that they had been exposed to IPC. According to the 2011 TRaC, exposure to PSI/C IPC interventions among HRUM is positively associated with increased and consistent condom use with sweethearts, namely 70 percent of consistent condom use for one IPC exposure and 82 percent with three IPC exposures.49 Enhanced efforts by PSI to utilize targeted IPC to address HIV among HRUM include brief interventions conducted by outreach teams with male patrons at urban entertainment venues. This is an important role for the HIV/AIDS NGO members of the UHN in the urban areas of Phnom Penh and other provinces, like Battambang. Similar IPC techniques were employed to reach HRUM and EWs, and were observed by the evaluation team in Battambang. Using an approach adapted from local private sector strategies of marketing beer and cigarettes through promoters that move from table to table, mixed-sex pairs of outreach workers deliver education, condoms, and recommend VCCT and STI service referrals to men in beer gardens, karaoke bars, and other hotspots. PSI/C’s prevention programs targeting female EWs in these same venues also provide education, commodities, and VCCT, STI, and RH service referrals. While there are a number of instances where indicators have changed or been dropped since the start of the project either at the request of USAID or program management, in general PSI/C demonstrates progress meeting and exceeding their objectives to change awareness, knowledge, supportive attitudes, and behaviors.50 Increased Cambodian National Capacity (Organizational and Financial Sustainability) A. Organizational Capacity (See Detailed Findings in Appendix I) According to the Cooperative Agreement, PSI/C and other U.S. government partners are expected to support the “Three Ones” principles by harmonizing efforts among U.S. government and RGC partners to ensure strong linkages between programs and research. MOH partners at NCHADS expressed their appreciation of the long￾standing productive relationship with PSI/C. NCHADS further affirmed the continued importance of the social marketing of condoms led by PSI/C to support declines in HIV prevalence and prevent its resurgence among MARPs. PSI/C reports active participation in multiple thematic working groups, including the Government-Donor Joint Technical Working Group on HIV/AIDS and the MOH Contraceptive Security Working Group. Government counterparts interviewed at the central and provincial levels during the evaluation demonstrated detailed knowledge 47 PSI/C TRaC survey among EWs 2011 48 PSI TRaC for HRUM 2011 49 PSI TRaC for HRUM 2011 50 Extracted from PSI/Cambodia Semi-Annual Progress Report to USAID/Cambodia for Oct 1, 2011 to March 31, 2012 SMBCI PROJECT FINAL PERFORMANCE EVALUATION 25 of, direct familiarity with, and appreciation for the SMBCI Project and HIV/AIDS prevention activities targeted at MARPs. In addition, national partners expressed the importance of sustaining PSI/C’s social marketing contribution into the future, acknowledging that PSI/C makes important contributions by ‘filling the gap’ and ‘helping [to] link the community to the public health services’ that the MOH would not be able to fulfill. PSI/C supports NCHADS’ BCC and IEC goals and seeks NCHADS partnership in and review of strategies and guidelines for promoting and monitoring the use of information, education, and communication (IEC) materials, coverage of BCC interventions, outreach, peer education, and the 100 percent Condom Use Program, as well as coordination through specific technical working groups (TWGs). Based on results reported by PSI/C in their Oct 2010-Sept 2011 Annual Report, capacity building via in-service training did not ramp up until 2011, and technical assistance to NGO partners in monitoring and evaluation has only recently been strengthened and appears to be accelerating with the introduction of a structured and more rigorous performance-based reporting system and a renewed commitment to carrying out regular site visits.51 PSI has a history of training providers in birth spacing knowledge and long term methods since 2008. It is somewhat less clear how PSI/C provides technical assistance in institutional capacity building among its NGO partners. However, according to the Oct 2011-March 2012 semi-annual report, PSI/C is collaborating with the National AIDS Authority (NAA) to host quarterly MARP Community Partnership Initiative meetings in Phnom Penh, Siem Reap, Battambang, and Sihanoukville (eight meetings held in FY2011) to bring together owners of entertainment establishments with provincial and local authorities to ensure that condoms and/or promotional materials are widely available at high-risk venues such as guesthouses, karaoke bars, and beer gardens. The meetings also aim to ensure that Interpersonal Communicators are welcome to conduct activities in entertainment establishments to promote consistent condom use among MARPs. 52 PSI has described to the team that their approach is to provide monitoring visits where the team will do on-site training, quarterly IPC tool training, and bi￾annual program review meetings with the ultimate goal of strengthening institutional capacity of IHN partners. Table 8: HIV/AIDS Outputs and Indicators53 Output 7: Increased sustainability through increased capacity of local institutions and individuals Indicator Baseline Cumulative LOP Target Source Number of health care workers successfully completing in-service training* N/A 611 916 Project MIS Number of local organizations provided with technical assistance for strategic information* 6 67 98 Project reports Number of local organizations provided with technical assistance for HIV-related institutional capacity building* 42 121 118 (43 unique) Project report Output 4: Increased national capacity to manage and sustain social marketing programs Indicator Baseline Cumulative LOP Target Source Number of local NGOs contracted & supported 0 95 (all 130 Project 51 PSI/C SM for BCI Annual Report Oct 2010-Sept 2011 52 PSI/C Semi-annual Report Oct 2011-March 2012 53 Extracted from PSI/C SM for BCI Annual Report Oct 2010-Sept 2011 SMBCI PROJECT FINAL PERFORMANCE EVALUATION 26 to conduct effective BCC with MARPs & assist with social marketing & services UHN) (43 unique) UHN Reports *New PEPFAR indicators for which PSI/C requested USAID approval in 2012 B. Organizational Sustainability PSI/C currently receives multiple streams of funding from donors to support an integrated BCC, social marketing, and policy reform agenda in Cambodia. The funding from the USAID/DfID social marketing project represents one such stream. PSI/C has creatively managed the respective opportunities and constraints related to each stream of funding in order to address what it feels are the most pressing public health challenges in Cambodia. Therefore, while PSI/C has been able to leverage other sources of donor funding (primarily via GFATM for HIV/AIDS) and program income from the sale of donated commodities, these streams of funding are ultimately tied to donor resources rather than commercially sustainable ones. Financial Sustainability: TMA Goals and Metrics PSI/C states that its approach to condom social marketing under the USAID funded SMBCI Project has “emphasized appropriate targeting to ensure that scarce donor subsidies are reaching the poorest and most at-risk populations while engaging the commercial sector for longer term sustainability.”54 PSI/C’s conceptualization of the TMA introduces concepts that are beyond the standard definition of this perspective, 55 such as a calculation of “universe of need” or the estimation of the total number of condoms needed for protected sex acts. In general, the TMA focuses on market segmentation by socioeconomic status and maximizing market efficiency by matching the level of subsidy to the economic needs of consumers. PSI/C’s TMA approach for the condom market in Cambodia included objectives of increasing market coverage for at-risk populations, increasing space for new commercial entrants to the market, and ensuring that the poor and vulnerable are not left behind. PSI/C presented “updated” TMA objectives to the evaluation team which included: meeting the universe of need in the condom category, decreasing the level of total subsidy in the market, and increasing the role of the commercial sector. PSI/C used an evolving set of metrics to track progress toward these goals, as illustrated in work plans and annual reports from FY 2009- 2011. From data shared with the evaluation team, it is not clear how PSI/C monitored progress toward its TMA objectives, or if it used data to make adjustments to implementation strategy or whether USAID provided input or direction on these issues. TMA Results As described above, the specific goals of the project related to implementing the TMA for condoms appear to have evolved over the life of project. Irrespective of routine indicators that were being used for monitoring progress, two important markers of TMA progress are: (1) growth in the size of the overall market and (2) growth in the size of the commercial sector share. A third important factor is the greater alignment of subsidy in the market with need, or socioeconomic segmentation, but data are lacking in terms of the socioeconomic profiles of the users of various condom brands, so this element is difficult to assess. 54 PSI Cambodia FY 2012 Work Plan Submitted to USAID, p. 7 55 The TMA is a widely accepted approach to social marketing of health products that seeks to support the development of sustainable markets through demand creation, market segmentation and targeting of subsidies to those who need them most. The long-term objective of the TMA is to transition highly subsidized markets to those requiring fewer subsidies in order to maintain or increase the use of health products and services by target groups. SMBCI PROJECT FINAL PERFORMANCE EVALUATION 27 • Market Size: According to PSI’s data, the condom market in Cambodia decreased in size from 30.4 million in 2007 to 25 million in 2011 due to a supply glut, wastage and pipelines at outlet levels. The figures did not represent use. • Market Share: Overall, the changes in market share for both commercial and public sector condoms over the life of the project have been small but moving in a positive direction. Commercial sector share has increased modestly from 2 percent to 6 percent of the market and free condoms distributed by the public sector increased their share slightly from 14 percent to 15 percent. From a sustainability perspective, donor subsidized condoms have decreased from 98 percent of the total condom market in Cambodia to 94 percent between 2007 and 2011. PSI/C’s routine indicator reporting between 2008 and 2011 included condom sales/distribution from public, social marketing, and commercial sectors, but in 2011 a recommendation was made by the project to cease reporting public and private sector distribution data due to data reliability constraints. It is unclear how PSI plans to assess progress toward TMA goals for condoms in the absence of reliable data on the size of the total market and relative shares of public and commercially provided products. Table 9: “Health of the Cambodian Condom Market”: Data Presented by PSI/C to Evaluation Team Condom Brand/Sector 2007 2011 Number One 77% 14% Commercial Sector 2% 6% Public Sector (free) 14% 15% Appropriateness of TMA Goals vis a vis Income Distribution of Target Groups Any assessment of the effectiveness of a TMA to condom promotion and distribution for HIV prevention purposes should take into account the need for and role of subsidies in facilitating preventive behaviors among each target group. All three target groups have average incomes over the national per capita average, which is extremely low at $39 per month. MSM appear to have the lowest average monthly incomes ($94 per month) which may warrant greater subsidies to the OK Plus condom, which is targeted at that category of users. EWs with lower incomes, such as massage workers who have monthly incomes of $120 on average, may also require subsidies to facilitate regular use, particularly if they have a high number of clients. Higher income EWs and HRUM who earn between $200 and $300 per month may be appropriate segments of the high-risk market to shift to commercial brands. Additional information is needed regarding the purchasing power and patterns of target groups for HIV prevention interventions, as well as ability to pay studies (in addition to willingness to pay), before developing segmentation strategies and setting goals for shifting segments of target population(s) to sources of product with lower subsidies. Results Related to Approach One: Creating Space for the Private Sector in the Condom Market PSI/C’s application proposed two primary approaches to creating space for the private sector in the condom market: (1) the transfer of the Number One brand to full cost-recovery and management by the commercial sector, and (2) a partnership with J.K. Ansell to introduce lower-priced commercial condoms to the Cambodia market. PSI/C reported in FY 2010 that the Number One condom had reached 160 percent of cost-recovery in product and packaging, but transfer of the brand to commercial sector management and distribution has not occurred. PSI/C reports difficulty finding a commercial sector partner that was willing to take over the Number One brand under terms it found to be acceptable, i.e., without increasing the unit price of the product to $1 or more. The “manufacturer’s model” partnership with Ansell did not occur. SMBCI PROJECT FINAL PERFORMANCE EVALUATION 28 Beyond specific strategies proposed in the application, the TMA also emphasizes the reduction and targeting of overall subsidies within a market. In Cambodia, the overall impact of product and promotion subsidies provided by donors, and managed through PSI/C on the condom market, has been to set an artificially low floor price for condoms. The combination of the small population size in Cambodia and the dominance of social marketing brands may prove to be a disincentive for commercial sector players to compete in the market at lower prices. An exception to this is the Okamoto brand, which is headquartered in Japan and distributed in Cambodia by the Full Well Trading Company (FWT). FWT has seen slow but steady growth of the Okamoto line of condoms, which they estimate to account for 50 percent of commercial condom sales in Cambodia. Despite the dominance of social marketing brands in the Cambodian condom market, FWT has demonstrated interest in expanding its distribution and sales, including among high-risk groups. For example, FWT distributes directly to guesthouses in order to meet the demand among HRUM and their sweethearts and/or commercial sex workers. Internal market research data provided by FWT illustrated that a proportion of their sales occurring in Phnom Penh and the top five provinces in terms of population size, are comparable to PSI’s sales of Number One and OK condoms. Data on the availability of other commercial brands by province was not available. Table 10: Proportion of Sales by Geographic Location, Okamoto and PSI products, 2011 Phnom Penh Top Five Provinces Other Provinces Okamoto-Made in Japan Series 70% sales 20% sales 10% sales PSI Number One and OK 63.3% sales 20.9% sales 15.8% sales Percent of Total Population 10% 35% 55% The executive director of FWT expressed the view that a constraint to expanding distribution of Okamoto condoms at high-risk venues was the presence of subsidized social marketing condoms in the same locations. The commercial sector’s interest in this market segment and non-traditional distribution points may indicate an opportunity for partnership to achieve both FWT’s sales goals and USAID’s public health objectives. Results Related to Approach Two: Social Marketing of OK Condoms PSI/C proposed “aggressive segmenting of the market through better management of price and distribution” of the OK condom in order to target the product at lower income groups at-risk for HIV infection. OK pricing went from $.02 per piece at the time of the proposal to $.04 per piece in 2008, and has remained there. OK Plus, introduced in 2009, at the same package price as OK, is slightly more expensive as is contains two condoms and two sachets of lubricant instead of just three condoms. The team could not find data to suggest that OK was aggressively segmented, as it was available in many urban pharmacies alongside Number One. In fact, the wholesale switch of consumers from Number One to OK following Number One’s price increase demonstrated that consumers saw both condoms as comparable in quality and brand position. Despite efforts to aggressively segment the market, a brand equity study conducted by PSI/C in 2011 “showed little differentiation between consumer’s perceived value between Number One and OK condoms.”56 The second component of approach two stated in the application was to transition OK distribution to traditional retail outlets in the commercial sector. PSI/C is currently piloting 56 PSI/Cambodia Annual Workplan 2012, p. 11. SMBCI PROJECT FINAL PERFORMANCE EVALUATION 29 commercial distribution of FP products, but it has not to date attempted to transition condom distribution to the commercial sector. Results Related to Approach Three: Facilitating Public Sector Availability of Condoms PSI/C proposed to advocate to the RGC to increase the procurement and distribution of condoms through the public sector, and to provide technical assistance and coordination to ensure that public sector condoms are targeted to the poorest and most vulnerable. The evaluation team saw and heard evidence of PSI/C’s collaboration with NCHADS, the NAA, and other public sector entities related to a range of issues including prevention programming for people living with HIV/AIDS, extension of the 100 percent Condom Use Program to entertainment establishments, and involvement of the Private Sector Working Group of the NAA around workplace HIV/AIDS initiatives. PSI/C also demonstrated willingness to donate condom samples to public sector sponsored HIV prevention events, and has donated six million condoms to fill a supply shortage in the public sector in FY 2011. None of these activities appear to specifically respond to the proposed approach of assisting the RGC in better targeting its condoms; however, given the small proportion of public sector condoms in the market, the lack of targeting of free products is unlikely to have as significant a total market impact as the lack of targeting of OK and Number One. 3. Increased Knowledge and Evidence Base (Use and Dissemination of Formative and Evaluative Research for Decision-Making, See Detailed Findings in Appendix J.) The strategic collection and dissemination of formative and evaluative data are critical for priority-setting and decision-making in effective program planning, implementation, and evaluation where key stakeholders are fully engaged throughout the entire research process. NCHADS leadership confirmed the value of PSI/C’s participation and contribution to the National HIV/AIDS Research Committee, but also acknowledged and took responsibility for the fact that this committee does not meet regularly. It is therefore somewhat difficult to determine the extent to which PSI/C’s significant time and human resource allocation for research is utilized by NCHADS to influence national HIV/AIDS program planning. On the other hand, it appears PSI/C works closely with the NAA to support the government’s efforts to reach the MARPs. For example, it was reported that a working group chaired by the NAA was established during the design phase of the 2011 MSM quantitative TRaC study. Members included technical representatives from FHI, Khmer HIV/AIDS NGO Alliance (KHANA), and NCHADS, all of whom contributed at every stage of planning including the development of the methodology, questionnaire, and protocols, and the conducting of field work. PSI/C has successfully created a culture of research and program evaluation by developing, testing, and implementing reliable, formative, and on-going research as well as program monitoring and evaluation tools. A complete list of the research studies and M&E data collection carried out since the beginning of the current PSI/C Cooperative Agreement is appended to this report in Appendix E, PSI Research under USAID 2008-2012. PSI/C uses formative methods to develop and test campaign concepts, messages, and images; to pre-test behavior change and marketing intervention materials; and to explore questions related to client perceptions and behavior change barriers or facilitators identified through TRaC studies. MAP is conducted quarterly to map and enumerate entertainment venues, entertainment workers, and prospective male clients in their catchment areas. This information is used to address potential gaps in coverage. For example, while the results of the 2011 MAP show little change since previous years, the penetration of condoms in hotspots and high-risk venues has declined whereas condom coverage near and around these venues has increased, e.g., more SMBCI PROJECT FINAL PERFORMANCE EVALUATION 30 than half of all potential condom outlets sell condoms. These MAP data prompted PSI/C to set up a network of retail outlets where sex workers and their clients can access condoms in close proximity to the venues where they meet or work. TRaC is a repetitive survey typically administered annually to identify the main barriers to behavior change experienced by target groups. PSI/C also uses TRaC surveys to track changes in behavior over time and to measure the effectiveness of communication and behavior change interventions. Recently, TRaC data have shown positive impacts from the IPC activities of PSI/C partner UHN on the perceptions, attitudes, and communication of HRUM and MSM about condom use with sweethearts and male partners. PSI/C and partners use project MIS systems to monitor key project indicators, such as sales by geographic location and type of outlet, over time. Results from targeted interventions such as numbers of people reached during specific IPC activities are also tracked, as well as the number of trainings, workshops, and activities conducted to monitor progress against project indicators and benchmarks. PSI/C aims to disseminate all research findings to key stakeholders, including NGO and community based organizations, the private sector, donors, and government agencies via workshops in-country and printed production of materials and the internet to help inform the program decisions and interventions of the health community. PSI/C and its partners strive to participate in all government-led TWGs to develop indicators, share program data, coordinate research proposals, and share findings. Table 11: HIV/AIDS Outputs and Indicators57 Output 5: Increased knowledge and evidence base for effective and efficient social marketing, including increased access for the poor and vulnerable Indicator Baseline Cumulativ e LOP Target Source Number of research and evaluation findings from project dissemination shared with partners 1 56 10 Project Data Number of internationally presented posters, presentations, publications 0 11 6 Reviews PSI/C appears to place more emphasis on breaking down key data for relevant and appropriate use by local NGOs, including specially designed dissemination sessions that guide partner NGO teams. Some UHN partners were also involved in the design of the questionnaire to ensure its appropriateness and the phasing of questions, and members of UHN also assisted in data collection. Summary Conclusions in HIV/AIDS • PSI/C’s social marketing program contributes significantly to the core package of evidenced based services tailored for MARPs. The quality of prevention interventions has improved through 57 Extracted from PSI/C social marketing for BCI Annual Report Oct 2010-Sept 2011 SMBCI PROJECT FINAL PERFORMANCE EVALUATION 31 development of standard packages of activities, innovative models, and messaging based on operational research. • Tailored IPC interventions have been developed, tested, and utilized to convey evidence-based behavior change messages to high-risk HIV/AIDS populations including high-risk urban men and men having sex with men not reached by other projects/providers. For example, TRaC surveys show high awareness of safe behaviors among targeted populations. TRaC surveys show a consistent rate of condom use during the last sexual encounter at around 75 percent among high-risk urban men. TRaC survey results also show that exposure to IPC has increased steadily since 2008 and increased IPC activity exposure correlates positively with four measures of positive beliefs about condom use.58 However, the OEM construct and scales for measuring the role of trust for increasing consistent condom use in HRUM relations with EWs is not entirely validated and may necessitate re-testing and modifications. • Results from MAP studies demonstrate that the recall of key messages for behavior change campaigns and the visibility of promotional materials, display boxes, and posters for condom promotion is less than 50 percent, indicating the need for modified approaches. • The most recent logic frame reports that most targets have been met or exceeded for the penetration of condom distribution within 50 meters of high risk outlets in Phnom Penh.59 However, improvements are needed to penetrate entertainment venues themselves. • Engaging the partnership of local, indigenous NGOs comprised of members of affected populations most at risk for HIV/AIDS is an effective and efficient approach to access hard-to-reach populations (exemplified by Men’s Health Social Services (MHSS) work with MSM). • Though PSI is not currently partnering with RACHA. RACHA has capacity to reach village shop keepers and keep village shops supplied with condoms and other health products is an important approach to “harmonizing” partner advantages and prioritizing the most vulnerable. Gaps and Missed Opportunities in HIV/AIDS The evaluation team identified gaps and missed opportunities related to PSI/C’s implementation of the SMBCI Project, based on review of documents, interviews with stakeholders, and field visits to sites in Kampong Chhang, Battambang, and Phnom Penh. They are outlined here based on the findings presented above. • There are many opportunities to collaborate more strategically, productively, and in a cost-efficient manner with other U.S. government partners (such as FHI) who are working with similar target populations (such as HRUM, MSM, MSMO, MSMW) to collect, analyze, report, and disseminate data (e.g., the Bros Khmer Study by FHI, the TRaC studies and Nielsen study by PSI/C, etc.). Reducing duplication and overlap with other USAID implementing partners by improving the coordination of human and financial resources and integrating program components and performance reporting according to a shared results framework would benefit all. • Joint ownership of operational research across national partners is imperative for a stronger and more integrated program, as well as the benefits of cost savings. • There may be a missed opportunity to collaborate with FHI to expand and create a more robust bi￾directional IPC intervention design that includes managers of guest houses, beer gardens, and hot￾spot establishments to help create an enabling or supportive environment for one-on-one IPC with 58 TRaCs HRUM, MSM & EW, 2008-2012 59 MAP Study Evaluating the availability of PSI condoms and lubricants, 7th Round, PSI/C 2011 SMBCI PROJECT FINAL PERFORMANCE EVALUATION 32 male clients, and to potentially include male-EWs and/or male-SH communication messages about mutual trust and decision-making to protect one another. • There may be potential for UHN sales of short-term FP methods for dual protection to EWs, in addition to on-going UHN referral of EWs to local providers for RH services. • The TRaC survey methodology could be more responsive to all HIV/AIDS partners to improve the accountability of the U.S. government and evidence for its investments in concentrated epidemics by measuring associations between exposure to the programs and promoted behaviors, such as consistent condom use with sex partners, HIV testing behavior, and accessing of STI screening services. • It appears that HIV knowledge levels of men who have sex with women are much lower than among MSMs, and they are much less likely to have received an HIV test. There may be an opportunity for PSI/C to reach these men who have sex with EWs to convey information about HIV, its transmission, and the benefits of counseling and testing. • There may be a missed opportunity for SQHN providers to screen women at elevated risk for HIV, or to refer women for STI diagnosis and treatment and VCCT in health centers. SMBCI PROJECT FINAL PERFORMANCE EVALUATION 33 REPRODUCTIVE HEALTH AND FAMILY PLANNING 1. Improved Supply of Selected Products and Services/Increased Access There are two factors involved in accessing RH/FP services: • A ready and consistent supply of contraceptive products, and • The presence of services––such as client counseling, insertion and removal of IUDs and implants, injections, and surgical services for vasectomy and tubal ligation––necessary for their safe and correct use. In this section of the evaluation report, we discuss the contributions of the SMBCI Project to the improved supply of contraceptive products in the marketplace and the increased availability of quality services necessary for their safe and correct use. a. Contraceptive Products Consumer access to a ready and consistent supply of contraceptive products requires three conditions: • Presence of contraceptive products in the marketplace • Availability of these contraceptive products in consumer accessible/appropriate outlets • Product prices that make them accessible to a range of current and potential users Table 12: Sales of SMBCI FP Products Over the Life of Project Contraceptive Product Units Sold FY60 2008 Units Sold FY 2009 Units Sold FY 2010 Units Sold FY 2011 Units Sold Q1 & Q2 only FY 2012 OK pill 2,811,274 3,166,476 (+12.6percent ) 3,359,652 (+6.1percent) 3,968,664 (+18.1percent ) 2,535,612 OK injectable 301,395 367,430 (+21.9percent ) 565,645 (+53.9percent ) 489,880 (- 13.4percent) 201,840 IUD Not Applicable Not Applicable61 Not Applicable62 14,700 12,742 Jadelle implant63 Not Not Not Not 60 FY = October 1 to September 1 61 Available only on special "event days" at franchised clinics and public sector outlets, according to FY 2009 annual report to USAID 62 5,139 IUDs inserted in 192 public health facilities through training efforts of PSI SM Project SMBCI PROJECT FINAL PERFORMANCE EVALUATION 34 Applicable Applicable64 Applicable 3,990 Available65 Implanon implant Not Applicable Not Applicable66 Not Applicable 910 Not Available67 Sales data taken from PSI SMBCI Project Annual Reports, FY 2008, 2009, 2010, and 2011 In FY 2011, according to the figures above, the SMBCI Project sold the equivalent of a year's supply of oral contraceptives for 305,282 women68 and the equivalent of a year's supply of injectable contraceptive for 112,470 women.69 Project product sales also provided 14,700 women with an IUD and 4,900 women with an implant contraceptive. From these numbers, one can infer that approximately 437,000 Cambodian women had access to contraceptive supply in FY 2011 as a result of the availability of SMBCI Project products. According to the 2012 Population Reference Bureau Population Data Sheet, in mid-2012 Cambodia had a population of 15 million. The 2010 CDHS estimates that 50.5 percent of the total population falls in the range of 15- 49 years of age, or 7,575,000 people. Since half of this group is female, we may estimate that there are approximately 3,787,500 WRA in Cambodia currently. If the SMBCI Project supplies approximately 437,000 WRA with contraceptives, then the project in FY 2011 supplied the contraceptive needs of roughly 11.5 percent of WRA, which is approximately one-third of the number of WRA currently using a modern contraceptive. The estimated overall contribution of the SMBCI Project (made above) to modern method CPR in FY 2011 is generally supported by the source of contraceptive supply70 data reported in the 2010 CDHS: • Private clinic 8.2 percent • Pharmacy 14.2 percent • Other private medical 6.1 percent • Shop 5.1 percent Total 33.6 percent While the percentage contribution by SMBCI Project products to contraceptive availability in FY 2011 appears significant, it must be noted that until FY 2011 the social marketing contraceptive product line was limited to only two methods: one brand of oral contraceptive and one brand of injectable contraceptive. Additionally, the project's contribution to contraceptive availability is still heavily dependent on the sales of these two products. (The availability of the injectable contraceptive has been limited in the past year because PSI is controlling its release into 63 The management team of Bayer in Cambodia reported in an interview that while PSI had originally projected annual sales of 7,000-8,000 sets of Jadelle (a product manufactured by Bayer), only 6,400 sets in total have been sold since 2008. 64 Available only on special "event days" at franchised clinics and public sector outlets 65 Total sales of implants for Q1 and Q2 = 3,326 66 Available only on special "event days" at franchised clinics and public sector outlets 67 Total sales of implants for Q1 and Q2 = 3,326 68 Assumes a year's supply of OCs is 13 cycles 69 Assumes a year's supply of injectable contraceptives is four vials 70 Very few commercial brands of modern contraceptive methods are sold now in Cambodia. Social marketing project contraceptive products are estimated by project staff to account for as much as 99 percent of oral contraceptive sales in the commercial sector. SMBCI PROJECT FINAL PERFORMANCE EVALUATION 35 the distribution system since it has not yet secured funding for additional injectable product purchases.71) The SMBCI Project has met or exceeded targets for its sales of oral contraceptives and injectable contraceptives according to the logical framework included in its FY 2012 annual report to USAID. The project’s contribution to the availability of a wider range of modern methods – especially of long-term methods – is recent and relatively small. According to the 2010 CDHS, the public sector remains the main supply source for 56.5 percent of IUD users and 46.7 percent of implant users. The project has plans for the introduction of a second oral contraceptive (a mid-priced Continuity of Care (COC)) and an emergency contraceptive pill in the next three months. Access to contraceptives depends not only on the presence of a range of methods in the family planning marketplace but also on the number, type, and location of outlets where contraceptives can be obtained. The Cooperative Agreement for the SMBCI Project emphasizes the importance of making project supported products available in rural areas down to the village level.72 According to the SMBCI Project Annual Report for FY 2011, the social marketing project has met its targets and increased the availability of its oral contraceptive products from 50 percent of target area rural villages73 in FY 2008 to 65 percent in FY 2011, whereby village availability is defined as having at least one official outlet (traditional/non-traditional) carrying its oral contraceptive product. During the same period, the availability of project oral contraceptives in urban74 areas increased from 85 percent in FY 2008 to 95 percent in FY 2011. In the 2011 RH TRaC Survey, it was reported that 62.5 percent of oral contraceptive users interviewed said they bought oral contraceptives in their village, while 27.5 percent said they had bought oral contraceptives in a neighboring village. It is not possible, however, for the team to comment on the trend in the availability of oral contraceptives in villages because similar data were not included in the 2011 RH TRaC Survey report. (The evaluation team has found that inconsistencies from year to year in the questions asked or type of information collected/reported across a number of the project's routine research interventions diminished the usefulness of the information collected.) The table below is illustrative of the distribution of social marketing brand contraceptive products available across Cambodia in FY 2011. Table 13: Percent SMBCI Project FP Product Sales by Geographic Area in FY 2011 Contraceptive Product Phnom Penh Next five Most Populous Provinces75 All Other Provinces OK pill 57.5% 18.6% 23.9% OK injectable Data not available Data not available Data not available 71 Semi-Annual Progress Report to USAID/Cambodia, May 1, 2012, page 13. 72 A MOH source stated that there are approximately 10,000 villages in all of Cambodia. 73 For the project, all areas that are not urban, as defined by the Ministry of Planning, are considered rural. 74 The project has adopted the Ministry of Planning definition of urban as an area where population density exceeds 200 per square kilometer, male employment in agriculture is below 50percent, and total population of the commune exceeds 2,000. 75 Includes Siem Reap, Battambang, Kampong Cham, Kandal, and Prey Veng SMBCI PROJECT FINAL PERFORMANCE EVALUATION 36 IUD 31.4% 29.6% 39% Jadelle implant 17.8% 41.3% 40.9% Implanon implant 5.2% 7.6% 87.2% Based on PSI SMBCI Project product sales/province data for FY 2011 For comparison, PSI project staff reported to the evaluation team that 85 percent of commercial sector pharmaceutical sales occur in Phnom Penh. Bayer, an international pharmaceutical company with an office in Cambodia estimates, however, that only 60 percent of total pharmaceutical sales occur in Phnom Penh. The urban/rural distribution of project IUD and implant sales may be explained by the geographic distribution of SQHN providers, the primary private providers of IUD and implant insertions, who are for the most part located outside of Phnom Penh. The following table shows the availability of oral contraceptives by outlet type. To the extent that pharmacies serve as "wholesalers" for the owners of village shops in surrounding areas, a portion of pharmacy sales may actually represent ultimate product sales in more rural areas or villages. Among the pharmacies visited by the evaluation team in Phnom Penh, several reported sales of larger quantities of PSI product, especially condoms, to customers for resale in their shops. Table 14: Sales of SMBCI Project Oral Contraceptives by Outlet Type in FY 2011 Outlet Type Number of OC Cycles Sold Percentage of Total OC Sales Pharmacy 2,866,920 72.2percent Clinic/hospital 275,628 6.9percent Continuity of Care Record/health care room 516,516 13.0percent Mini-Mart/shop 223,944 5.6percent UHN 33,024 0.8percent Other 52,632 1.39percent Total 3,968,664 Taken from PSI SMBCI Project sales distribution/outlet type data for FY 2011 The third factor affecting the availability of or access to modern contraceptive methods is price. If contraceptives are priced beyond the ability of the WRA who want and need them to pay, then those contraceptives are not in fact accessible. The 2008 Cooperative Agreement document states that the purpose of the SMBCI Project is "to increase consistent and correct use of high-quality essential health products and services among priority at-risk populations, especially SMBCI PROJECT FINAL PERFORMANCE EVALUATION 37 the poor and vulnerable." The SMBCI Project retail price for its OK oral contraceptive is low and set to be accessible to the very poor: 1,000 riel or about $0.25 per cycle.76 The retail price of the OK injectable is 2,000 riel ($0.50) per vial; however, this product is purchased by providers who take it to their clinics or cabinets where they charge the user $0.60 to $0.70 for the product, plus its injection. Similarly, providers purchase social marketing IUD ($0.50 per device) and social marketing implant ($10 per set) products and provide them to the end-user for a price that includes both the product and its insertion. In the SQHN, providers charge clients on average $8.60 for an IUD insertion. Other private providers may charge as much as $60 for an IUD insertion in Phnom Penh and $15 in rural areas.77 No stakeholder interviewed by the evaluation team expressed a concern that the current price of the OK pill is too high for low income consumers in rural areas. Many did note, however, that there is a percentage of the overall population, often estimated at 25 percent or more, who live below the poverty line and qualify for completely free services in the public sector. Staff at the central office of Reproductive Health Alliance of Cambodia (RHAC) told the team that they estimated that 30-40 percent of households in rural areas quality for completely free services in the public sector. The 2010 CDHS reports, however, that 43.2 percent of the poorest quintile and 53 percent of the second poorest quintile of women currently using an oral contraceptive purchase the OK social marketing brand oral contraceptive. Where consumers choose to obtain their contraceptive supply––whether for free or nearly free in the public sector or for a higher price in the private sector––is viewed by some FP program managers around the world as a statement of the degree to which products and services in the private sector are affordable to consumers. Increasingly, however, program managers in many countries are observing a difference between the health care consumer's ability to pay and her/his willingness to pay for the products and services s/he needs. In particular, there is a frequently observed willingness of consumers to pay very high prices for curative care, but less willingness to pay much lower prices for preventive care. The evaluation team was told several times by stakeholders that Cambodians "expect" to receive free FP services. In Cambodia, according to the 2010 CDHS, 51.6 percent of modern contraceptive method users use the public sector as their source of supply, while 43.6 percent use the private sector. In the public sector, a service fee of approximately 50,000 riel includes consultative/counseling services as well as an IUD insertion. Implant contraceptives are just now becoming available in the public sector. Public sector clients pay a service fee of 1,000 to 1,500 riel for a consultative/counseling visit to a health center and three cycles of oral contraceptives; whether the 52 percent of modern contraceptive method users are able to pay more than the public sector fee for these services is unclear. The table below illustrates the wealth quintiles for women who currently use an oral contraceptive and choose the public sector as the source of their oral contraceptives. Table 15: Percent of Current OC Users Who Use Public Sector OC and SMBCI Project OC by Wealth Wealth Quintile Public Sector Pill OK Pill 76 4,000 riel = US$1 77 Reproductive Health MAP survey 2011 and SQHN longitudinal study 2011 SMBCI PROJECT FINAL PERFORMANCE EVALUATION 38 Lowest 55.5% 43.2% Second 45.0% 53.0% Middle 52.4% 46.3% Fourth 50.9% 46.5% Highest 29.8% 59.0% Total 47.8% 49.1% 2010 CDHS Most striking in this table are the data that indicate almost 30 percent of oral contraceptive users from the wealthiest quintile choose to obtain their products from the public sector where it is nearly free (500 riel or about $0.12 per single cycle), and 43 percent of oral contraceptive users from the poorest quintile choose to use the OK pill which can be purchased for 1,000 riel or $0.25 per single cycle. Curiously, a larger percentage of oral contraceptive users in the second poorest quintile purchase OK pills than oral contraceptive users in the middle and next wealthiest quintiles. These data appear to indicate some misplacement of the benefits of both public sector and social marketing project subsidies, as well as the existence of a sizable market segment that appear to be able to pay more for oral contraceptive than they currently do. The management team of Bayer, manufacturer of oral contraceptive brands Yasmin and Diane, which are sold in Cambodia for $10 and $6.50 per cycle respectively, believes that more than 20 percent of current oral contraceptive users in Phnom Penh and perhaps 5-10 percent of users outside of Phnom Penh can afford to purchase Yasmin. Willingness to pay studies have differed in their approximation of this number, suggestion it overestimates ability to pay. There are no performance indicators in the FY 2012 logical framework that relate directly to contraceptive price; they only indirectly do so as price affects cost-recovery. Further aspects of the pricing of SMBCI Project contraceptive products are discussed in the following section on cost-recovery. b. Access to Services – The Sun Quality Health Network of Clinics Over the project period, the SMBCI Project increased the number of SQHN clinics from 29 in 2008 to 229 in 2012, operating in 16 provinces. The majority of SQHN sites are rural (75 percent) and small – cabinets or health care rooms. Through the SQHN, PSI provides support and training to improve the private sector providers’ ability to offer quality FP services. By training private providers in long-term methods including IUDs and implants, PSI is increasing availability and access to a wider mix of methods, especially in districts where public sector health clinics do not yet offer these services.78 A majority of SQHN providers report that they have received IUD training within the past two years; nearly 80 percent of providers received initial IUD training directly from PSI (other major sources of training were the MOH and Reproductive and Child Health Alliance [RACHA]).79 Similarly, 98 percent of SQHN providers received training on implants from PSI. Increasing the method mix available to women has the potential to reach previously un-served populations. For example, PSI found that among IUD users at SQHN, 55 78 OPHE Mid-term MCH Evaluation 79 PSI Provider Longitudinal Study Exploring Opportunity, Ability and Motivation (IUD) Among SQHN providers and Provider Satisfaction Survey, October 2011 SMBCI PROJECT FINAL PERFORMANCE EVALUATION 39 percent were 'new to modern FP.'80 Through the SQHN, PSI has strengthened provider behavior by training SQHN providers in client counseling techniques and counseling tools. Most SQHN providers (96 percent) found the network to be of value. Specifically, 90 percent believe that participation in the SQHN helps to attract more clients. Frequent and supportive supervision visits help to reinforce positive practices; nearly all providers felt that supervision visits improved their abilities in birth spacing counseling, as well as their skills in providing IUDs and implants.81 A survey of all SQHN clinics in October 2011 (N=151) found that nearly all clinics provide oral contraceptives, injectables, IUDs, implants, and condoms, with very low stock-outs at the time of the survey. As PSI shifts its distribution of FP products to a local distributor, Mega, it will be important to ensure that the availability of commodities continues. Because the SQHN clinics offer a variety of services in addition to FP counseling and provision, these settings offer an opportunity for the integration of services and the potential to reduce missed opportunities to reach WRA with FP messages. The PSI longitudinal survey of SQHN providers conducted in 2011 found that clinics offered a wide range of services including antenatal care, postnatal care, abortion services, and general services. More than half of the clinics also provide labor and delivery care, STI screening and treatment, and child health services. Though it is unclear the extent to which PSI has emphasized the importance of taking advantage of these many opportunities to reach women with FP messages, the SQHN clinic setting offers new ways to reach women with unmet need for FP, including post-partum and post-abortion women, and women with children under 2. 2. Behavior Change – Increased Demand for and Use of Selected Products and Services Among Targeted Populations The SMBCI Project recognized that in order to increase prevalence of the consistent and correct use of modern contraceptive methods in Cambodia, the behaviors of both WRA and their health care providers must change. Project staff has utilized a number of communications channels and behavior change interventions in reaching FP service providers to improve the supportive environment for women's adoption and continuation of modern contraceptive use: • Training service providers in good FP counseling and referral techniques • Providing service providers with correct information on modern contraceptive methods • Providing midwives and other providers with clinical training in IUD and implant insertion and removal • Providing follow-up supervisory visits to trained providers • Detailing pharmacists, physicians, and midwives with current information on modern contraceptive methods and their use 80 Email from Alysha, Monday, July 23, 2012, based on MIS data. 81 PSI Provider Longitudinal Study exploring Opportunity, Ability and Motivation (IUD) Among SQHN providers and Provider Satisfaction Survey, October 2011 (Slides 68, 69) SMBCI PROJECT FINAL PERFORMANCE EVALUATION 40 These provider-related behavior change interventions have occurred through SQHN operations. The SQHN offers a valuable mechanism to improve knowledge, awareness, and improve FP behaviors. Providers are trained to counsel on FP, as well as strategies to manage or reduce side effects. Additionally, SQHN providers are also trained to refer clients to the Women’s Health Hotline, which presents women with an opportunity to learn more about the RH options available to them. IPC has been the primary tool used by the project to reach WRA with FP behavior change messages. Channels for IPC have included both community outreach workers accessible through partnerships with local NGOs and RH service providers, especially midwives. Mass media such as television, radio, billboards, and mobile video units have also been used as channels for delivering behavior change messages to women; however, mass media in the current project has served mainly as a "back drop" to IPC interventions to provide coherency in messaging. The product marketing process has also been used by the project as a tool to foster behavior change among WRA. Effective marketing and behavior change interventions are targeted to specifically identified segments of the population. In contraceptive social marketing, the more specific behavior benefits; product benefits, characteristics, price, outlets; and communications messages and channels are designed to appeal to and reach a particular group of potential users, the more likely it is that members of that market segment will be positively influenced to adopt product use and/or targeted behaviors. In 2010, the earliest year for which the team has information on the project's marketing strategies, the segment of the population targeted by project FP behavior change interventions was identified as poor and vulnerable WRA. Project communication efforts aimed at influencing the FP behavior of this large and relatively undefined target market promoted the positive benefits of FP and "functional benefits" of each contraceptive method. Functional benefits of contraceptive methods included: ease of use, safe, infection prevention (condoms); worry free, convenient, safe (injectables); easily reversible, safe (OCs); worry free, effective for 3-5 years, no intrusive exams (implant); and worry free, effective for 10 years, quickly reversible, and non-hormonal (IUD). SMBCI project efforts to segment the family planning market in Cambodia have been tied, until recently, to one brand each of two short-term contraceptive products: the OK pill and the OK injectable, both priced to be affordable to the very poor. Such a narrow product line can limit the effectiveness of behavior change interventions by limiting the ability to appeal to the particular desires and needs of targeted consumers at various points in their reproductive life cycles. Within the last year and a half, two long-term contraceptive products have been added to the social marketing product line: an IUD and an implant contraceptive, both priced to be affordable to lower income consumers. The focus of behavior change/marketing efforts shifted in 2011 to a two-pronged approach: (1) promotion of long￾term (IUD and implant) contraceptive method use among WRA who were current non-users of a contraceptive method or were users of a traditional method, and (2) promotion of short-term (OC and injectable) contraceptive method use among WRA who were either non-users of a contraceptive method or were inconsistent users of contraceptives. Analysis of available data led project staff to conclude that 55 percent of new users of long-term methods came from the targeted segments of current non-users and traditional method users. The evaluation team, however, has not found data that supports this conclusion. SMBCI PROJECT FINAL PERFORMANCE EVALUATION 41 BCC efforts during this later period addressed the fears and misconceptions surrounding the use of short-term hormonal methods as these were determined to be the critical barriers to the increased adoption of short-term modern contraceptive methods.82 Segmentation of the short-term contraceptive market, however, has remained dependent on the same two contraceptive brands, the OK pill and the OK three-month injectable, that have been sold by the social marketing project even prior to the beginning of the current project in 2008. In the later months of 2012, the marketing/behavior change focus will be on reaching urban women in the middle to highest wealth quintiles who are inconsistent users of modern contraceptive methods. This market segment can be targeted for behavior change/marketing interventions largely because of the planned introduction of a newer generation of internationally manufactured oral contraceptives that will be sold at the higher price of $2.50 per cycle. This market segment is targeted for behavior change interventions because data reported in the 2010 CDHS indicate that the prevalence of traditional method use among women in the wealthiest quintile has increased to 24.6 percent. It is not possible to attribute contraceptive behavior change directly to project behavior change interventions. The SMBCI Project research staff have, however, implemented a series of annual RH TRaC surveys that allow them to measure contraceptive behavior change in the 13 provinces in which SQHN clinics operate. The CPRs demonstrated by these surveys are used as project performance indicators by USAID. The PSI/Cambodia social marketing for behavior change interventions Annual Report for FY 2011 states the following changes in the "percentage of currently married women of reproductive age using modern birth spacing methods": • FY 2009 44 percent • FY 2010 46 percent • FY 2011 43.5 percent There is a considerable difference between the modern method CPR of 34.9 percent reported in the 2010 CDHS and the 46 percent modern method CPR reported by the TRaC survey in the same year. The discrepancy between the two may be caused in part by the population samples utilized in each survey. The CDHS used a nationally representative sample that included respondents in all provinces. The TRaC survey used a sample taken only from project provinces. Additionally, the TRaC survey uses only WRA who do not intend to have another child for at least two years in its sample selection. For the evaluation team, this selection criterion appears likely to create a sample skewed toward present contraceptive use. The project’s research staff has also created a "scaled construct" to calculate the extent to which women interviewed in the RH TRaC surveys believe or do not believe selected statements about the safety and health effects of oral contraceptives. While this number is reported to demonstrate project performance in reducing health concerns among WRA as a barrier to contraceptive use,83 a trend cannot be established for recent years because the FY 2011 survey was implemented in 13 provinces while the FY 2010 survey was only implemented in two. Beliefs about birth spacing remain a barrier according to PSI. 82 The 2010 CDHS report does not include a table showing women's reasons for non-use or discontinued use of contraceptives. The 2011 RH TRaC Survey, however, reports that of those women interviewed who were not currently using a birth spacing method, 63.3 percent had never used a modern method. Approximately 53 percent reported that the last method they discontinued using was a modern method. Side effects were the most commonly reported reasons for the discontinued use of a modern method in the 2010 RH TRaC Survey (53.5 percent) and in the 2011 RH TRaC Survey (43.6 percent). 83 "Amended from initial log frame indicator of 'Decrease from 35.8 percent to 15 percent by the end of the project in the percentage of married women of reproductive age who cite health concerns as the reason not to use modern methods of contraception.' " SMBCI PROJECT FINAL PERFORMANCE EVALUATION 42 Currently, the SMBCI Project does not collect data that allows it to describe the socioeconomic, or any other characteristics, of the users of its contraceptive products or the users of contraceptive products from other sources. The absence of these data makes it impossible to evaluate whether or not project products are being purchased by its targeted consumers. PSI stated that they believe low quintile WRA are using oral contraceptives based on monthly income and assets owned by household and data related to a concentration index. 3. Improved Local Capacity For and Sustainability in Delivering Selected Products and Services to Targeted Populations The evaluation team considered project performance in the following areas related to improved local capacity for and sustainability of RH/FP products and services delivery: • Financial sustainability of the project's social marketing of contraceptives • Creation or expansion of a sustainable commercial market for contraceptives • Improvement of the technical capacity of local health care providers and community organizations to deliver quality FP services, including counseling and long-term methods in an environment of informed choice • The expansion of a branded network of health care providers able and willing to deliver high-quality services in targeted areas of reproductive and MCH. 3.1 Financial Sustainability of Social Marketing Activities The financial sustainability of PSI/C depends in large part on its ability to continue to attract significant donor funding from a variety of sources. Cost-recovery through sales of its branded products, however, may play an important role in the sustainability of its social marketing of contraceptives. Currently, the SMBCI Project measures cost-recovery as a percentage of the cost of the product, packaging, shipping, insurance, and import fees that are covered by the income received from sales of the product to distributors or directly to retail outlets. "Full cost-recovery" in SMBCI Project reports to USAID means that the equivalent of 100 percent of the money spent for product procurement, including packaging and shipping, has been generated through sales of the product. The significant costs of marketing and promotion, in-country distribution, and marketing management related to the sale of each project product are not included in this measurement. Currently, project management does not appear to have ready access to analysis that informs them of total project costs or products sold. Beyond a stated, but somewhat tentative, intention to raise the retail price of project condom brands and introduce a higher-priced oral contraceptive brand into the market in the next six months, the project does not seem to have an explicit or well developed strategy for the medium or long-term sustainability of its social marketing activities in FP. In fact, the project is currently experiencing a shortage of injectable contraceptives because at the time of this Evaluation, it had not yet found donor funding for new procurements. Indicators of project performance shown in the FY 2011 Logical Framework related to cost-recovery refer to only one project contraceptive – the OK pill. In each year of the project, the SMBCI Project has met or exceeded the cost￾recovery targets set for the OK pill. Cost-recovery, as defined above, has moved from 78 percent indicated in the 2007 baseline to 107-109 percent in FY 2011. Project reports indicate that this favorable cost-recovery position for SMBCI PROJECT FINAL PERFORMANCE EVALUATION 43 the OK pill has been largely achieved through cost-savings afforded by contracting with a commercial firm for the distribution of project oral contraceptive products. The project had previously distributed its own oral contraceptive products and continues to distribute condoms. There are a number of indications in the marketplace and recent research that suggests that there may be considerably more potential for cost-recovery through increased prices in the social marketing of contraceptives than is currently being realized. Data from the 2010 CDHS, for example, provides an interesting perspective on the market potential for higher-priced oral contraceptives and for the better targeting of subsidies in the public and social marketing/NGO sectors. Project staff members report that they currently have no national income distribution data and no research-generated data on the socio-economic characteristics of contraceptive consumers who do and do not purchase SMBCI Project products. It is difficult to make effective pricing and cost-recovery decisions without the benefits of such guidance. The SMBCI Project does not currently have a strategy that includes sustainable support through––cross￾subsidization by higher-priced products––for product subsidies that benefit lower income consumers. Creation or Expansion of a Sustainable Commercial Market for Contraceptives The 2008 Cooperative Agreement describes in considerable detail project strategies for the increased involvement of the commercial sector in condom and contraceptive sales and distribution as part of a TMA to contraceptive availability in the Cambodian marketplace. Further, one of three stated approaches in the SMBCI Project Cooperative Agreement for increasing access to modern FP/RH products is "creating space in the FP private sector." This approach is further described as the facilitation of the introduction of a number of new contraceptive brands targeting middle and upper income women in urban areas. "With support," the document continues, "private sector brands will achieve 20 percent of the total oral contraceptive market share within five years of the project." The FY 2009 project work plan stated that discussions of the possible introduction of a mid-price commercial oral contraceptive brand by Bayer was on-going, and mentioned the possible licensing of the project's Number One condom brand to a commercial firm in the future. By the time of the FY 2010 annual report, however, there is no further mention of possible collaboration with Bayer in the introduction of a mid-priced commercial oral contraceptive product into the market. FY 2010 did mark the initiation of the transfer of some project distribution tasks to the commercial sector. A contract was signed that year with Mega Lifesciences for the distribution of OK injectable contraceptives. Since FY 2010, project activities related to a possible TMA have seemed to focus on participation in the Contraceptive Security Working Group under the aegis of the MOH. There is no commercial sector representation in the working group, which is responsible for providing assistance to the MOH in projecting contraceptive commodity needs and ensuring that there is effective collaboration among stakeholders who either provide or distribute contraceptive commodities in Cambodia. There are no longer any indicators reported in the logical framework that relate to increased involvement of the commercial sector in the contraceptive market or to contraceptive market share by sector. Public and commercial sector sales/distribution figures are no longer collected or reported because those data are said by the project to be unreliable. SMBCI PROJECT FINAL PERFORMANCE EVALUATION 44 Improvement of the Technical Capacity of Local Health Care Providers and Community Organizations to Deliver Quality Family Planning Services, Including Counseling and Long-Term Methods, in an Environment of Informed Choice Over the course of the project, PSI has been the main partner of the MOH responsible for supporting the private sector health system, especially in FP and RH. Stakeholders widely acknowledged that Cambodia needs a strong commercial private sector in order to improve health outcomes and the overall health status of Cambodians. MOH officials recognize that the public sector is not able to meet the additional health needs of Cambodians, nor are national systems able yet to fully monitor and regulate private sector service delivery points though improvement has been made in recent years, especially with the closure of unlicensed pharmacies and drug shops. PSI provides a valuable service through the SQHN by monitoring private sector service delivery points, ensuring providers follow national guideline and treatment protocols, and facilitating the licensing and registration of clinics. Practices are supported by frequent supervision visits–– each clinic is visited at least quarterly, and more frequently for new members, or those with identified problems. According to the USAID Health Program Mid-Term Evaluation, supervision visits in 2010 showed 83 percent of franchisees meeting minimum quality assurance standards. Most private providers, including those in SQHN clinics, work in public sector clinics in addition to their private practices. Though it is not among the stated objectives of the SQHN, PSI training and efforts to improve the capacity of SQHN providers also strengthens their ability to provide care in public sector settings. As an example, half of SQHN providers surveyed in October 2011 also provide IUDs in public clinics.84 The SMBCI Project mid-term report for the period Oct 2011-Mar 2012 states that “Sun Quality Health Network are supported by on-going training, supervision and in-kind support to increase availability of both short-term and long￾term methods and provide access to trained providers of high-quality clinical services and counseling as well as continuity of services and follow-up. PSI/C conducts Supportive Site Supervision visits to SQHN franchise clinics in 16 provinces.” Expansion Of a Branded Network of Health Care Providers Able and Willing to Deliver High-Quality Services in Targeted Areas of Reproductive and Maternal and Child Health In addition to improving access to FP though providing technical assistance to SQHN providers, PSI also improves local organizational capacity for SQHN private providers. Network members receive assistance and support to become legally registered private clinics, improve client records management, and data incorporation into the national Health Management Information System (HMIS). Though PSI’s SQHN activities, such as technical assistance, trainings, and supportive supervision, are supported exclusively through donor funds, the services provided by SQHN clinics are not subsidized. Each provider determines their prices and clients pay the full cost of services and commodities, though providers report that they 84 Provider Longitudinal Study exploring Opportunity, Ability and Motivation (IUD) among SQHN Providers and Provider Satisfaction Survey. (Slide 106) SMBCI PROJECT FINAL PERFORMANCE EVALUATION 45 have discussed pricing with PSI staff during supportive supervision visits.85 Though this does not contribute to the long-term sustainability of the SQHN, it does improve the stability of the overall market for health services, as customers become accustomed to the real price of services and commodities. Table 16: Services Offered at Sun Quality Health Network Clinics Source: PSI Provider Longitudinal Study Exploring Opportunity, Ability and Motivation (IUD) among SQHN Providers and Provider Satisfaction Survey, PSI Research Division, October 2011 During the life of the SMBCI Project, the SQHN was expanded from three to 13 provinces and from 40 to approximately 240 clinic outlets. 4. Implementation and Dissemination of Formative and Evaluative Research Necessary for Effective Programmatic Decision-Making The SMBCI Project has developed, tested, and implemented formative and routine research methods such as TRaC to measure barriers to contraceptive behavior change, changes in contraceptive behavior over time, and the effectiveness of communications and behavior change interventions. The project has utilized qualitative research methods to test concepts, messages, and images for FP/RH communications interventions, including mass media, small media, and IPCs. Through analysis of research-generated data, project staff has identified health side effects, misconceptions, and fears as the primary barriers to the consistent use of hormonal contraceptives and adoption of method use by current non-users. These data have formed the basis for development of mass media advertising and IPCs designed to foster contraceptive behavior change among targeted consumer segments. 85 Provider Longitudinal Study exploring Opportunity, Ability and Motivation (IUD) among SQHN Providers and Provider Satisfaction Survey. SMBCI PROJECT FINAL PERFORMANCE EVALUATION 46 Data have not been collected and used, however, to develop descriptions of specific segments of the overall population of WRA that can be used to target behavior change interventions most effectively. Additionally, variables have not always been consistently maintained in evaluative research mechanisms, such as periodic TRaC surveys, from one round to the next. Consequently, trends in behavior or attitude change cannot always be established. SMBCI Project research results are largely used as the basis for monitoring and evaluating project performance against indicators that appear in the project's logical framework document accompanying each annual report to USAID. The TRaC surveys are also used to improve SQHN provider quality. Rh/FP research results have been disseminated by project staff through specially organized presentations and project publications and through project participation in various TWGs. NGO partners and MOH counterparts both reported to the evaluation team that the project had shared with them the results of its research efforts on a number of occasions. It is unclear to the evaluation team to what extent the project has shared relevant research results with representatives of the commercial sector––whether its distribution partner, pharmaceutical company representatives, or private providers like those in the SQHN network. The country director of one international pharmaceutical company indicated to the team that she had seen the 2010 CDHS survey report, but she may have gained access to those results through the Demographic Health Survey web site. Lessons Learned/Conclusions for Reproductive Health/Family Planning The following are the key lessons learned during the life of the SMBCI Project in the application of the social marketing process to the delivery of RH and FP services and products in Cambodia: • The SMBCI Project plays a significant role in the availability of low-priced modern methods of contraception in Cambodia. • An expanded social marketing project or commercial sector contraceptive product line is necessary to facilitate maximum behavior change among WRA. There is little choice now within product/method categories on which to base behavior change/marketing interventions targeted to specifically defined segments of the overall population of WRA. • The project has not collected sufficient relevant data on which to base maximally effective market segmentation decisions. Targeted consumer segments are not always described with sufficient specificity for development of maximally effective marketing strategies. There are no available data that describe contraceptive users who currently purchase social marketing products versus contraceptive users who currently obtain their method from other sources. Therefore, project managers cannot demonstrate that targeted consumers are the actual consumers of social marketing products, nor can they demonstrate that intended beneficiaries are receiving the price and other subsidies invested in project products. • Not enough is known about the factors influencing women's, especially wealthier urban women's, increasing reliance on traditional methods of FP to develop a fully effective behavior change intervention. • There is an insufficient distribution of contraceptive product prices in the Cambodian marketplace. Prices are clustered at the very top and the very bottom of the range. PSI can and should be more aggressive in testing increased prices for selected products. • PSI needs to review and renew its efforts to engage the commercial sector more actively in the contraceptive market in Cambodia. It is not necessary to achieve the "perfect" low commercial sector price SMBCI PROJECT FINAL PERFORMANCE EVALUATION 47 in early collaborations. It is more important to begin to engage the commercial sector in reaching a broader range of contraceptive consumers than they are presently and to demonstrate the existence of a market for "mid-range" contraceptive prices that the commercial sector can feasibly sustain. • The current shortage of injectable contraceptive products, due to the absence of donor funds for procurement, demonstrates the immediate need for the development of an SMBCI Project sustainability strategy for the short, medium, and long terms. • Local NGO partners can provide effective grass-roots access to community residents for delivery of IPC messages for behavior change and for promotion of products, methods, and service providers. Summary Conclusions in Reproductive Health (RH) • Number One and OK condoms are underpriced. PSI has pursued an extremely conservative course with regard to price increases for Number One and OK condoms. As a result, the price of both products appear to be significantly lower than what the market could bear without resulting in financial barriers for important target groups. Both formal analyses such as the Nielsen study as well as anecdotal reports of both brands being sold at higher than recommended prices at retail outlets support this finding. • Further segmentation of the condom market is needed. The team found promising signs of commercial sector interest in the Cambodian condom market, both in terms of commercial sector actors willing to invest promotion funds and distribute to non-traditional outlets,86 and in terms of the range and variety of commercial brands and products available in urban areas. This level of interest in a market where subsidized brands account for 80 percent of the market indicates the missed opportunities for further partnership and collaboration with the commercial sector, and the need to better target subsidized products in order to create space for commercial sector expansion. • Distribution of socially marketed products through NGOs and other subsidized channels should be rationalized. PSI/C should consider strategically the channels that socially marketed condoms are distributed through and ensure that UHN partners are marking up products appropriately before selling them to retailers or clients, both to protect the brand value of OK and to support the sustainability of the NGOs. • TMA metrics need to be refined and consistently collected and reported to monitor progress. In order to track progress toward an expanded total market and monitor shifting market shares between public, social marketing, and commercial sectors, reliable data sources or estimation methodologies are needed. The decision to drop the tracking of public and private sector condom distribution87 (given the lack of reliable information) presents a serious constraint to measuring project progress and the impact of the TMA as implemented. • The type of data (specific variables) collected and reported in the annual TRaC reports should be consistent. In order to identify trends across the same or similar data points/variables, data must be consistent. For example, the 2010 TRaC reports show that EWs know where to go for RH services, (i.e., have high knowledge) but small numbers actually seek these services. This variable is reported differently in the 2011 TRaC report; namely, we see self-reports of condom use for FP and reports of STI services accessed by type of service sites. 86 The Full Well Trading Company reported the intention to invest $100,000 in their own funds to support promotion of Okamoto products. These funds in part have supported development of a TV advertisement with a strong HIV prevention message. They are already distributing to guesthouses as part of targeting the high risk market in urban areas. 87 2011 Indicator Reporting note SMBCI PROJECT FINAL PERFORMANCE EVALUATION 48 Gaps and Missed Opportunities in Project Implementation for Reproductive Health/Family Planning While the SMBCI Project makes a very important contribution to the availability of contraceptives and RH services in Cambodia and has met or exceeded virtually all of the indicators that remain for it in the FY2011 logical framework, there are still some areas where the evaluation team feels project performance may be usefully improved. • The definition of market segments for project products does not seem fully developed. Targeted market segments are not clearly and specifically described in many instances. Illustrative targeted segments could include, for example, breastfeeding women, women who have completed their families, young "modern" women, women with short-term spacing needs, women with long-term spacing needs, women just starting to use contraceptives, long-time contraceptive users, women with special health needs, women concerned about side effects, women for whom social status is important even in health care-seeking behaviors, women for whom what is "natural" is important, middle-aged women who want to feel young and confidently attractive, women who want or need to conceal contraceptive use (privacy), and those who have achieved life-cycle contraceptive use or ready to move to the "next" method. The current differentiation of project products from other brands seems limited primarily to price. • No data are currently available that describe the characteristics of the consumers of the project's contraceptive products or the consumers of non-project contraceptive products; who is receiving the benefits of subsidized products cannot be discerned. What percentage of product purchasers are new users/new adopters also cannot be stated. • There is a lack of research-based information on current and potential contraceptive users' ability and willingness to pay and their income distribution. The absence of such data makes it difficult, if not impossible, to project the size of the potential market or market share growth for each contraceptive product or method. • The absence of retail audits/shelf surveys over time limits the project's ability to perceive and/or monitor changes in market share, product movement, geographic penetration, stock shortages, and the like. • Attention was given to FP method discontinuation late in the project's life. • Post-partum and post-abortion FP services need further attention. According to the 2011 RH TRaC survey, 42% of women who have ever had an abortion, reported that their provider recommended using a contraceptive method after the abortion. However, among women who have received an abortion in the last 12 months, 50% reported that their provider recommended a contraceptive method (2011 TRaC). In 2012, this percent increased from 50% to 62%. While there is much work to be done in this area, particularly at pharmacy level, this data demonstrate a significant positive trend. • Immediate post-partum IUD insertion, LAM, and POPs are key modern methods, and their provision could have been strengthened during the life of the project. • There has been insufficient attention given to the development of a strategy for promoting and marketing contraceptive methods to women at different stages of the reproductive life cycle. • There is a gap in knowledge of the attitudes of traditional method users and the drivers of their method choices. • There is a lack of IEC, IPC, and BCC messages related to the short-comings of key traditional methods such as withdrawal and rhythm. • The SQHN is supported exclusively by donor funds. Individual clinic operators do not pay a service or franchise fee for benefits received through the network. • Oversight of SQHN providers should include some supervision to ensure that service delivery is not detracting from the provider's public sector responsibilities nor facilitating the leakage of public sector commodities into the private sector. • SQHN provider knowledge about IUD medical eligibility and side effects should be improved. SMBCI PROJECT FINAL PERFORMANCE EVALUATION 49 CHILD SURVIVAL This section will evaluate the progress of the SMBCI Project toward goals and targets in the area of child survival as defined in the Cooperative Agreement, and its adaption and expansion throughout the course of the project. The decision to incorporate child survival products and activities into the existing PSI/C portfolio was well informed by the current health situation in Cambodia, where childhood illnesses are a major driver of morbidity and mortality. Despite significant improvement in recent years, Cambodia’s infant and under-5 mortality rates continue to rank among the highest in Southeast Asia. The under-5 mortality rate in Cambodia, at 54 per 1,000 live births, is significantly higher than in neighboring Vietnam (23 per 1,000 live births) and Thailand (13 per 1,000 live births). Pneumonia and diarrhea are the leading causes of death of children under-5 years after the neonatal period, and major barriers to appropriate treatment of these diseases exist. Globally, under-nutrition is an underlying cause in an estimated 35 percent of all child deaths, and Cambodia’s high rates of under-nutrition put children under-5 years at increased risk of mortality. Approximately 40 percent of children under-5 are stunted and an estimated 11 percent are acutely malnourished (wasted). The goal of the SMBCI Project in the area of child survival is to contribute to the RGC goal of reducing child mortality in Cambodia from a 2005 baseline of 83 per 1,000 live births88 to 65 per 1,000 live births by 2013. 89 The purpose of this project component is to increase consistent and correct use of high-quality products and services (including socially marketed products and services) for child survival among priority populations, especially the poor. At the time the Cooperative Agreement was signed, the only child survival product being socially marketed by PSI/C was the Orasel Diarrhea Treatment Kit (DTK). As such, the PMP contains only one indicator to measure progress toward this goal: the percentage of children under-5 with diarrhea treated with ORS or Oral Rehydration Therapy (ORT) (Indicator 1.6). In the most recent PMP, the data source for the baseline is from the 2010 CDHS (34.1 percent of children with diarrhea in the last two weeks were treated with ORS), despite the fact that Orasel distribution began at the start of the project in FY 2008. Progress toward this goal was not measured until FY 2011, when questions related to diarrhea treatment were added to a TRaC survey of WRA. The target for the life of the project was set at 40 percent of caregivers reporting that they treated their child’s diarrhea with ORS, and the cumulative result to date is 49 percent. However, it is worth noting that the target for the life of the project is set lower (40 percent) than the actual achievement in FY 2011 (49 percent). The project has been successful in increasing the number of children with diarrhea who are appropriately treated from the 2010 baseline and national average (34.1 percent) to 49 percent in target areas as of FY 2011. However, it is worth noting that the later addition of this indicator in the project (2010), a lack of data, and changing targets make it difficult to track trends in this specific behavior from year to year.90 Toeuk Sovatepheap (TS), or safe water tablets, were added to the child survival portfolio in FY 2010. Indicators to measure usage of TS tablets were added to the FY 2011 Logical Framework, 91 but do not appear in the 2012 88 National Institute of Statistics, Directorate General for Health, & ICF Macro. Cambodia Demographic and Health Survey 2005. Phnom Penh, Cambodia and Calverton, Maryland, USA: National Institute of Statistics, Directorate General for Health, and ICF Macro, 2006. 89 Royal Government of Cambodia. Achieving Cambodia’s Millennium Development Goals. Phnom Penh: Ministry of Planning, 2010. 90 From FY 2010 Logical Framework: “Increase from 58 percent in 2005 to 75 percent by 2010 the percentage of children under-5 with diarrhea who are treated with ORS or ORT therapy” (CDHS as data source). From 2011 Logical Framework: “Increase from 58 percent in 2005 to 65 percent in 2011 of children under-5 who had diarrhea in the last two weeks who were treated with ORS or ORT therapy.” 91 TS Indicators from 2011 Logical Framework: (1) increase the percentage of caregivers who have treated household drinking water with a promoted HWT method in the last three days, (2) increase percentage of caregivers who state diarrhea prevention methods/practices are done consistently, not only when a child/household member is ill. SMBCI PROJECT FINAL PERFORMANCE EVALUATION 50 version. However, data from the 2010 and 2011 TRaC surveys of WRA in four target provinces show an improvement in water treatment behavior. While usage of TS tablets is low (0.2 percent in 2010 and 0.3 percent in 2011), 88.2 percent of respondents reported treating their drinking water in 2011 as compared with 77.8 percent in 2010. 1. Increased Access to Affordable Key Health Commodities for Child Survival (ORS+zinc and Safe Water Treatment Products and ARI Treatment) PSI/C rapidly scaled up coverage of Orasel DTK and TS tablets during the course of the SMBCI Project. In FY 2008, Orasel was available in only two provinces (Pursat and Siem Reap), and as of FY 2012, both Orasel and TS are available in 15 provinces.92 The project has leveraged, through partnerships with RACHA, RHAC, and local UHN sub-grantees, the distribution of Orasel and TS to end-of-line village outlets in underserved rural areas. Improved access to ORS is measured by the number of Orasel DTK and/or ORS sold and distributed through PSI social marketing (Indicator 2.9). The life of the project target for DTKs sold is 523,000 units and as of June 2012, only 310,456 units have been sold (approximately 59 percent of target). The inability to reach the project target has largely been due to stock-outs and stock shortages. In FY 2009, when the target was set at 25,000 units, a complete stock-out led to only 23 units being sold. A stock shortage between April and September 2011 led to the achievement of only 62.5 percent of the FY 2011 target. As of June 2012, almost 94 percent of the FY 2012 target number of units sold had been achieved, with the help of 11,870 units being sold to humanitarian NGOs for flood victims. The universe of need for ORS is estimated at 2,613,005 episodes of diarrhea per year among children under-5 in PSI/C target areas. 93 Based on sales throughout the life of the project, the Orasel DTK has contributed approximately two percent of the total need for diarrhea treatment from both the public and private sectors in the past five years. Contribution to the total universe of need each year ranged from about 1 percent in FY 2008 to more than 3 percent in FY 2010. These rough estimations illustrate the fact that while the project has increased the availability of diarrhea treatment, the need for ORS is still substantial. From FY 2008 to FY 2010, the project seemed to put a priority on increasing the amount of ORS available in the total market, from the public and private sectors as well as through social marketing. However, indicators to measure achievement in this area were dropped after FY 2011.94 Progress toward increasing access to TS tablets is measured by the number of TS (Aquatab) sachets distributed (Indicator 2.11). One sachet contains 10 tablets, each of which can be used to treat 10 to 20 liters of water. The TS product was introduced in FY 2010 in seven provinces and scaled up to 15 provinces by FY 2012. The life of project target is 471,210 TS sachets distributed and 507,075 sachets have been distributed to date (June FY 2012), meaning that the project exceeded the target by almost eight percent. However, it is worth noting that the number of sachets distributed in FY 2012 is inflated due to the fact that PSI/C donated 60,300 sachets to MOH and sold 70,140 to 92 Pursat, Siem Reap, Prey Veng, Banteay Meanchey, Koh Kong, Kampong Chhnang, Svay Rieng, Pailin, Battambang, Preah Sihanouk, Kampong Cham, Kampong Speu, Kampong Thom, Kratie, and Kandal 93 Universe of need= Estimated number of children under-5 in target areas (1,045,202) x Average number of diarrhea episodes per year (2.5) 94 Number of DTK and/or ORS social marketing sold and distributed in total market (MIS); ORS and or Zinc supplied (social marketing, public and private sectors); ORS distributed by the private sector (private companies); ORS distributed by the public sector (RACHA) SMBCI PROJECT FINAL PERFORMANCE EVALUATION 51 humanitarian NGOs for distribution to flood victims in late 2011. Without the donation to MOH, the project would have met almost 95 percent of the target number of TS sachets distributed. It is also worth noting that the amount of sachets distributed does not necessarily measure the number of sachets sold or used for water treatment. The final indicator used to measure increased access to key health commodities for child survival is number of liters of drinking water disinfected by target populations using a U.S. government supported household water treatment product (Indicator 2.10). Each 10-tablet TS sachet can disinfect 200 liters of water. As the TS product is the only U.S. government supported household water treatment product included in this figure, achievement in this indicator mirrors the number of TS sachets distributed. The project exceeded the overall target of 94,242,000 by almost eight percent. However, this includes the 1,206,000 liters contributed by the donation to MOH during FY 2012. The project has been successful in leveraging partnerships with local NGOs, including UHN sub-grantees, to increase the availability of Orasel and TS tablets at the village level. As of FY 2011, Orasel DTKs and TS sachets were available at a total of 4,051 outlets in 10 provinces, of which 3,125 were village shops supported by RACHA and UHN partners. RACHA works with 2,836 village shopkeepers to provide training and product distribution support, and estimates 20 percent coverage of village shops in target communities.95 The PSI sales team reached 926 outlets (pharmacies, drug stores, and clinics) in 11 provinces (including Phnom Penh) during FY 2011. The division of distribution, in which the PSI sales team reaches larger outlets with medical detailing and local NGOs reach village shops in rural areas, appears to be an effective model. 2. Improved Behavioral Determinants in Order to Encourage Positive Family Health Care of Children The SMBCI Project works to improve diarrhea treatment and prevention behavior through a variety of channels. From FY 2010 to FY 2012, safe water roving teams that disseminate messages on diarrhea prevention, hygiene, and the correct and consistent use of TS conducted 636 sessions, reaching an estimated 372,638 people in seven provinces.96 Safe water mobile video unit teams conducted 182 sessions reaching 83,172 people in 12 provinces.97 Diarrhea treatment mobile video unit teams that disseminate messaging on diarrhea prevention and treatment conducted 145 sessions reaching 70,803 people in 11 provinces.98 School education program roving teams disseminating IEC messages on diarrhea prevention and hygiene practices conducted 37 interactive sessions in 2011, reaching 19,255 students in four provinces.99 In FY 2012, diarrhea prevention (TS tablets and other forms of water treatment plus hygiene) and treatment (Orasel DTK or ORS) were integrated, and roving teams conducted 158 outreach sessions reaching an estimated 75,989 people in 10 provinces. The only indicator used to measure improved behavioral determinants to promote appropriate treatment and prevention of childhood diseases is the percentage of caregivers of children under-5 in target areas who know that the best treatment for mild diarrhea is ORS plus zinc (Indicator 3.6). The life of project target for this indicator is 45 percent, and the cumulative result to date is only 9 percent (according to a 2011 RH TRaC survey that includes child survival questions). The SMBCI Project did not achieve the target, which was only set in 2011, due to an unrealistic estimate of the baseline, confusion regarding the appropriate data source, and a lack of data about diarrhea treatment 95 Meeting with Ms. Chan Theary, Executive Director of RACHA (07/24/12) 96 Kompong Chhanang, Pursat, Siem Reap, Prey Veng, Pailin, Koh Kong, and Battambang 97 Siem Reap, Battambang, Kompong Chhanang, Banteay Meanchey, Preah Vihear, Prevy Veng, Kompong Speu, Pursat, Koh Kong, Pailin, Kandal, and Svay Rieng 98 Siem Reap, Kompong Chhanang, Battambang, Banteay Meanchey, Svay Rieng, Kompong Speu, Kandal, Pursat, Prey Veng, Koh Kong, and Pailin. 99 Battambang, Kompong Chhanang, and Siem Reap SMBCI PROJECT FINAL PERFORMANCE EVALUATION 52 behavior in general. The data source for this indicator was listed in the CDHS (where data on this indicator is not collected) until FY 2011, when the source was changed to TRaC survey data. The child survival TRaC survey currently being conducted is intended to increase the amount of available information on the behavioral determinants of diarrhea treatment. The SMBCI Project strategy for improving water treatment behaviors acknowledges that improved access to safe water and decreased incidences of diarrhea is dependent on a number of related behaviors. Communication materials focus not only on water treatment options, but on the role that unsafe water plays in causing diarrhea; how to identify unsafe water sources, proper water storage, hygiene practices such as hand washing; and dispelling the myth that clear water is safe water. However, there are no indicators to measure improvements in drinking water treatment behavior included in the PMP and as such gauging progress in this area is not possible. Two relevant indicators100 were suggested in the FY 2011 work plan, but they did not appear in the FY 2011 or FY 2012 logical frameworks. In terms of provider behavior, UHN partners have been training village shopkeepers on diarrhea treatment and prevention since FY 2010. PSI/C began addressing provider behavior through medical detailing for child survival products and training SQHN providers in FY 2012. 3. Increased National Capacity to Manage and Sustain Social Marketing Programs The SMBCI Project has leveraged partnerships with UHN partners to extend coverage of child survival products and services. Indeed, the project’s ability to provide access to diarrhea treatment and prevention products in underserved rural areas is dependent on the training and support that local partners provide to village shopkeepers. In FY 2010, following a competitive process and evaluation, two UHN NGO partners were awarded one-year sub-grants to implement BCCs and social marketing of diarrhea treatment and prevention products. In FY 2012, PSI/C signed a memorandum of understanding (MoU) with RACHA to continue expansion of child survival activities in target areas. PSI/C participates in two national-level TWGs related to child survival: the TWG on Diarrhea and ARI Prevention and Control (MOH) and the Water and Sanitation TWG Ministry of Rural Development (MoRD). In FY 2012, PSI/C was engaged in the review and discussion on findings, future directions, and recommendations for National Acute Respiratory Infection and Control of Diarrheal Disease (ARI-CDD) program activities, and on the update of the diarrhea treatment protocol. Also in FY 2012, PSI/C participated in provincial level TWG meetings to present the child survival project in Battambang, Kampong Speu, and Pailin. According to representatives from relevant government agencies, PSI/C regularly shares research findings and cooperates well with governmental and non-governmental partners. In terms of the ARI case management project, PSI/C worked extensively to gain approval and input from relevant departments in MOH. In FY 2012, the project was presented to and accepted by the RN/MCH Task Force, and the National ARI-CDD program provided input for the packaging and insert for the ARI treatment product (Pneumox). 100 Two relevant indicators are: (1) increase from x percent to y percent of caregivers who know that water can cause diarrhea even if it looks clean, and (2) increase in percent of caregivers of children under-5 years who state that diarrhea can kill children under five. SMBCI PROJECT FINAL PERFORMANCE EVALUATION 53 Advocacy work with RGC and MOH to ensure that zinc and ORS are included in procurement plans, actually procured, and available in the public system was identified as a child survival priority in FY 2008. However, there are no indicators included in the PMP to measure this type of engagement with the government. Early project work plans included measures of the total number of ORS units available in the private, public, and social marketing sectors, but these were dropped by FY 2012. 4. Increased Knowledge and Evidence Base for Effective and Efficient Social Marketing, Including Increased Access for the Poor and Vulnerable Progress toward an increased knowledge and evidence base for social marketing is measured by the number of research and evaluation findings from the project that were disseminated and shared with partners working in Cambodia (Indicator 5.1), and the number of internationally presented papers/posters at conference and/or published studies (Indicator 5.2). While PSI/C has exceeded the life of project targets for both indicators, there was no child survival related research that was formally shared with partners in Cambodia or internationally. This is not surprising given that child survival is a recently added health area for PSI/C, and Orasel DTK and TS tablets were only widely introduced in 2008 and 2010 respectively. Safe water questions were included in the 2010 and 2011 TRaC surveys of WRA in target areas, though the inclusion criteria for these studies did not appear to require the respondent to have a child under 5-years of age. The 2010 TRaC identified the slight chlorine smell and taste as a barrier to using TS tablets, and these findings informed messages about how the chlorine smell alerts the user that the water is safe to drink. Questions about diarrhea treatment were included along with water treatment behaviors in the 2011 TRaC study of WRA. While these studies likely informed project activities, it is not apparent whether they were shared with any NGO or government partners. Similarly, messages and packaging for child survival products were pre-tested with caregivers in target areas, but these findings were used only to inform project interventions. In FY 2011 and 2012, research was conducted to inform the ARI case management report, including a qualitative study (FY 2011), ARI market survey, ARI pricing and concept testing, and a child survival TRaC survey that included ARI questions (FY 2012). Conclusions – Child Survival • The SMBCI Project has contributed to improvements in diarrhea treatment behavior in target areas and successfully increased the number of children with diarrhea who are treated appropriately. Because the Orasel Diarrhea Treatment Kit (DTK) was the only child survival product to be socially marketed at the time of the Cooperative Agreement, diarrhea treatment is the only indicator for measuring improvements in child survival behaviors. • The SMBCI Project has contributed to improvements in water treatment behavior in target areas despite the fact that TS tablets do not seem to be the preferred method of treatment. • While the SMBCI Project failed to reach its total target for DTK sales, evidence suggests that the project was increasingly successful in reaching targets from year to year. SMBCI PROJECT FINAL PERFORMANCE EVALUATION 54 • Rough estimations of the SMBCI Project’s contribution to the diarrhea treatment universe of need illustrate the fact that while social marketing of Orasel DTK has increased the availability of diarrhea treatment, the need for ORS is still substantial. • The SMBCI Project was successful in reaching TS tablet distribution targets, though TRaC survey data suggests that the tablets are not a preferred method of water treatment. • The division of distribution, in which the PSI sales team reaches larger outlets with medical detailing and local NGOs reach village shops in rural areas, appears to be an effective model. • The SMBCI Project did not achieve the target, which was only set in 2011, due to an unrealistic estimate of the baseline, confusion regarding the appropriate data source, and a lack of data about diarrhea treatment behavior in general. There is a need for additional research on the determinants of diarrhea treatment and prevention (specifically water treatment) behaviors. • The SMBCI Project was successful in increasing national capacity to effectively social market child survival products. However, it is unclear whether advocacy related to public sector procurement of ORS and zinc, considered a priority in FY 2008 but dropped as an indicator by FY 2012, continues to be an important step toward sustainability. • The lack of child survival-related research shared with partners over the course of the project is not surprising given that the health area was recently added to the PSI/C portfolio (Orasel DTK in 2010 and TS tablets in 2008). There is need for additional research related to care-seeking behavior for children under-5, both to inform project activities as well as add to the general knowledge base. • Orasel TDK, TS tablets, and Pneumox are essential health commodities that, if accessible and widely used, have significant potential to improve the health of children under-5 in Cambodia. However, there are significant barriers to appropriate treatment of childhood illness, specifically diarrhea and ARI/pneumonia. While 94.5 percent of respondents to the 2010 CDHS know about ORS, only 34.1 percent report treating their child’s diarrhea with ORS and only 58.9 percent sought advice or treatment from a healthcare provider in the public or private sector. In terms of ARI, only 39.1 percent of respondents to the 2010 CDHS reported treating their child’s ARI with antibiotics and only 64.2 percent reported seeking the assistance of a trained provider.101 • Given that diarrhea and pneumonia are the two main killers of children under-5 years after the neonatal period, these suboptimal behaviors are extremely dangerous. Interestingly, in terms of both children under￾5 with diarrhea and ARI, a mother’s education level affected the likelihood of seeking treatment, but socioeconomic status (wealth quintile) did not. This suggests that barriers to appropriate treatment cannot be attributed primarily to poverty. There is an urgent need to better understand the barriers and motivators to appropriate treatment of diarrhea and ARI through the addition of relevant indicators and regular data collection. 101 National Institute of Statistics, Directorate General for Health, & ICF Macro. Cambodia Demographic and Health Survey 2010. Phnom Penh, Cambodia and Calverton, Maryland, USA: National Institute of Statistics, Directorate General for Health, and ICF Macro, 2011. SMBCI PROJECT FINAL PERFORMANCE EVALUATION 55 • The SMBCI Project increased access to diarrhea treatment and prevention throughout the life of the project and built local capacity to socially market child survival products. Based on the limited data available, the project also seems to have improved diarrhea treatment and water treatment behaviors in target areas. More research is needed in order to add to the evidence base and better understand the behavioral determinants of childhood illness treatment. Gaps and Missed Opportunities for Child Survival While the project succeeded in increasing the availability of and access to essential child survival products, there were missed opportunities that if seized may have increased progress toward project outputs and targets. Given that the child survival products introduced by PSI/C during the SMBCI Project were new technologies for Cambodia, there have been a number of lessons learned throughout the course of the project that will undoubtedly prove valuable in making further headways in improving child health status in the country. The following gaps in the area of child survival were identified by the evaluation team: • There is a need to better understand and address the behavioral determinants of diarrhea treatment from multiple channels. Given the high levels of knowledge about ORS reported in the 2010 CDHS, it is surprising that a major reason for the improper treatment of diarrhea among children under-5 years is the lack of knowledge about ORS. According to a 2011 TRaC survey of WRA, 64.3 percent of respondents who did not use ORS to treat their child’s diarrhea gave the reason that they had never heard of ORS. Disturbingly, 42.8 percent of respondents felt that the best treatment for diarrhea is a “cocktail” of pills. • Safe Water tablets are not being widely used, and people that do use them do so infrequently. According to a 2011 TRaC survey with WRA that included questions about Safe Water tablets and Orasel, only 0.2 percent of respondents reported that their current method of treating water was with the Safe Water product. Usage did not increase from 2010, when only 0.3 percent reported Safe Water tablets as their current method of water treatment. Furthermore, exposure to Safe Water messaging increased by only about one percentage point from 2010 to 2011 (29.1 percent to 30.5 percent). Of the 7.6 percent of respondents who have ever used Safe Water tablets, 74.9 percent reported using the product once per month, only during certain seasons, or only once. • Child survival was only integrated with RH activities starting in FY 2012. As of mid-FY 2012, 273 RH community mobilizers have participated in a child survival orientation and basic training on the Orasel kit. Integrating child survival into community mobilizer activities and the SQHN earlier in the project may have improved the diarrhea treatment behaviors of mothers accessing FP services from private and public providers. Mothers often bring their children with them to the clinic, and providers reportedly treat childhood illnesses such as diarrhea, fever, and cold. Given that provider behavior is a major barrier to the appropriate treatment of child hood illnesses, it would have been beneficial to train SQHN providers in diarrhea treatment and prevention from the start of the project. Inappropriate treatments such as drips, antibiotics, and anti-motility could have been initially framed as contrary to SQHN policies and perhaps as grounds for expulsion from the network. • Medical detailing for child survival products only started in FY 2012. Access to Orasel and TS tablets may have increased if detailing had begun earlier in the project. SMBCI PROJECT FINAL PERFORMANCE EVALUATION 56 SMBCI CROSS-CUTTING ISSUES COST-EFFECTIVENESS Cost-effectiveness is the relationship between monetary inputs and the desired outcome, for example, the relationship between expenditure on an advertising campaign and increases in sales revenue. Cost-effectiveness analysis, in a health services environment, compares the costs and health effects of an intervention to assess the extent to which it can be regarded as having provided value for money. This analysis informs decision makers who have to determine where to allocate limited healthcare resources. Donor resources for investment in healthcare in Cambodia are shrinking. Decisions have to be made in the near-term regarding which healthcare interventions to continue to support and to what extent they should be supported with available donor funds. To help inform those difficult decisions, the evaluation team has been asked in its scope of work to answer the following questions: • How cost-effective has this program been? Which interventions have been most/least cost-effective? • How effective are the outreach and demand creation activities of PSI? Which would be most valuable to continue? • How effective are the various commodities socially marketed by PSI? Which would be most valuable to continue? This evaluation team cannot provide data-informed answers to these questions for several reasons: • Data are not currently available on project costs per service or product provided. • Data are not currently available that allow health impact or behavior change to be linked directly to given project interventions. • Project funding from multiple donor sources, often co-mingled, makes it particularly difficult to trace the value and impact of USAID's singular investment. • The technical expertise of the evaluation team does not include health economics. Given these considerable fundamental constraints, the evaluation team offers the following comments on the "cost￾effectiveness" of project interventions: • Decades of USAID experience in development assistance have demonstrated that one of the most cost￾effective interventions in the process of economic and social development is ensuring that FP products and services are available and accessible to every couple that needs and wants them. The 2010 CDHS estimates that there is an unmet need for contraceptive among almost 20 percent of the WRA in Cambodia. The modern contraceptive method prevalence rate as reported in the 2010 CDHS is only 34.9 percent. • Cost-efficiency is not always an appropriate factor in programmatic investment decisions. For example, long-term methods such as IUDs, implants, tubal ligation, and vasectomies, are considered from a supply point of view to be more cost-effective to provide than short-term methods such as oral contraceptives, condoms, and injectables. Focusing on or promoting one or more modern contraceptive methods over SMBCI PROJECT FINAL PERFORMANCE EVALUATION 57 others for reasons of cost-effectiveness of supply, however, presents significant infringements on the individual's right to choose and contrary to U.S. government directives. • The widespread availability of condoms is essential to maintain HIV/AIDS infection rate in Cambodia at its present low levels. However, the better targeting of product subsidies to those for whom they are intended could increase the cost-effectiveness of project interventions––especially in the case of very low-priced social marketing condoms (and oral contraceptives for FP). Data do not currently exist that demonstrate the percentage of project products purchased by those within targeted segments of the overall population–– particularly when enabled by a supportive environment where stigma is not acceptable, appropriate preventive and treatment health-seeking behaviors are valued, and where knowledge of one’s HIV status ensures care and treatment. • Diarrheal disease and ARI/pneumonia are the primary killers of children under five in Cambodia. Timely use of ORS-zinc and appropriate antibiotics can greatly reduce, if not eliminate, deaths from these diseases. While safe water tablets are a good product and safe water supply is an important development issue, safe water tablets do not at this time (for a variety of reasons) have the immediate impact on the reduction of childhood deaths due to diarrheal disease that ORS+zinc treatment kits have. • The availability of products and services without correct and consistent use has little value. Behavior change/adoption of use is an essential partner of ready supply. More specifically targeted behavior change interventions and communications efforts, based on research-informed consumer profiles, have the potential to make expenditures on BCC and IPC more "cost-effective" in achieving desired results. • Elimination of duplication and/or increased integration in IPC and outreach activities among USAID cooperating agencies and among local NGO partners would not damage project impact and could reduce project costs. • Increased participation of the commercial sector in making condoms, contraceptives, and other health products available in the marketplace can reduce the need for donor investments in supply-related interventions (like procurement of product for resale and subsidized distribution systems), making it "cost￾effective" in the long-term. However, some initial public sector/donor investment in demand generation/creation of a commercially sustainable market must be made––especially in a health product market that is as weakly segmented as Cambodia's. LOCALIZATION The statement of work for the SMBCI Project detailed an approach to organizational sustainability and localization that would be carried out by PSI/C over the life of the project. This approach included the following steps: • Establishment of a local SMO in Cambodia, by fully localizing PSI/W’s operations in year three of the project. • Building local human resources (“Cambodian leaders”) through staff training, study tours, and exchanges. • Strengthening systems to prepare for localization, with an emphasis on human resources and financial systems. Overall, it appears that PSI/C has pursued these steps toward the localization of the organization, but at a significantly slower pace than initially proposed. For example, the recruitment of a Cambodian Deputy Country Representative, initially planned for year one of the project, actually occurred in year four (Mary 2011). If the two year timeframe for mentorship by an expatriate Country Director (CD) mentioned in the application still holds, this will mean that Cambodian leadership of the localized PSI/C will not be in place by the end of the project period. SMBCI PROJECT FINAL PERFORMANCE EVALUATION 58 Annual work plans and reports detail the painstaking process that PSI has been engaged in for over four years102 related to localization. These include organizational assessments, trainings, constitution of an advisory board that is intended to transition to a government board, analysis of the international/local NGO law and its potential impact on the localization process, and perhaps most importantly, working with staff to better understand, accept, and support the concept of a localized PSI/C. PSI/C is currently staffed by a mix of expatriate and locally hired Cambodian nationals. Currently, expatriates serve key roles in management and technical areas but express the intent to transition those roles to their Cambodian counterparts over the next two years. According to project documents, PSI/C’s expatriate staff members play an advisory rather than management role within the organization. While it is difficult to assess the true roles of expatriate and national staff as an outside team, it was notable that Cambodian senior staff attended all briefings of the evaluation team and that the majority of presentations were made and questions answered by expatriate advisors and the country director. PSI/C is seriously and deliberately pursuing the localization process. This process has taken longer than planned, and is likely to continue according to the organization's own pace and timeframe. GENDER The PSI SMBCI Project Cooperative Agreement states that project implementation will take gender differences into account by addressing risky social norms and practices. • Apply vigilance with regard to male roles and masculinity stereotypes that harm women. • Increase resources to reduce the risk of HIV transmission as a result of paid sex by intensively promoting consistent condom use with men who visit sex workers, and utilizing these opportunities to foster respect for the rights and bodily integrity of sex workers. • Address rape and violence in communication materials for women and men. • Integrate HIV prevention messages into FP programming whenever possible. • Include both men and women in the development of campaigns and materials. • Enforce gender mainstreaming so that planning, administration, and implementation of activities consider gender issues and gender equality at all levels. • Actively involve women in managing the project at all levels of the organization. HIV/AIDS Prevention PSI/C and other U.S. government partners have explored ways to influence and change male norms and behaviors in Cambodian society to ensure that HIV/AIDS prevention programming is not only reaching commercial sex workers, but their clients as well. Transactional sex venues have expanded to include a larger number of non-brothel based settings. In addition to on-going and proven successful interventions in high-volume brothel settings, PSI/C targets male clients of entertainment workers in the integrated behavior change campaign at other high-risk venues to alter 102 The statement of work in the original application details a number of steps that had been taken prior to the start of the project in 2008. SMBCI PROJECT FINAL PERFORMANCE EVALUATION 59 existing gender norms and promote correct and consistent condom use and other safe sex behaviors among males. The SMBCI Project's NGO partners routinely perform outreach and IPC work through male-female paired teams. Reproductive Health/Family Planning In the area of RH/FP, the PSI SMBCI Project addresses its marketing activities to both men and women. While it is true that condoms are used for pregnancy prevention by only a small percentage of the Cambodian population of reproductive age (approximately 2 percent), the project has targeted males/husbands with communications interventions designed to involve them as active supporters of FP behavior through an improved understanding of the benefits of birth spacing and size limitations for families. The SQHN empowers female health care providers through project training and technical assistance that improves their business and clinical service delivery capacities. Child Survival The SMBCI Project's child survival products (ORS+zinc and AquaTabs) are distributed primarily through village shops, which are often operated by women. By linking shop operators with nearby product distributors, providing shop operators with training in correct product use, and promoting participating shops within communities as a source of important health products, the project has strengthened the business and influential standing of these businesswomen. Organizational PSI/C currently employs a number of women in senior management roles in areas such as research, communications, and RH/FP. The current country director is female. SMBCI PROJECT FINAL PERFORMANCE EVALUATION 60 APPENDIX A SCOPE OF WORK Social Marketing Project Final Performance Evaluation USAID/Cambodia STATEMENT OF WORK I. Purpose: USAID/Cambodia’s OPHE is convening a team of experts with a diverse skill set and background (social marketing and relevant health sector experience in HIV/AIDS, reproductive health and family planning, and child survival) to conduct an evaluation of the Mission’s Social Marketing program, currently implemented through Population Services International (PSI). The purpose of this evaluation is to conduct a final performance evaluation of the existing program. The overall aim of the exercise is to evaluate current USAID supported social marketing activities in Cambodia. The Mission envisions having this evaluation consultation carried out in July-Aug 2012. The Mission seeks the service of a five-person team including three consultants (a Team Leader, a Design Expert, and a local consultant – all procured by GHFP), as well as participation from two USAID/Washington based specialists and strong participation from the Mission’s OPHE office. II. Background: PSI is a leading global health organization with programs in HIV/AIDS, reproductive health and family planning, child survival and malaria. PSI was established in Cambodia in 1993 when it began the Number One condom Social Marketing program to support the Royal Government of Cambodia’s (RGC) 100 percent Condom Use Program. Since then, PSI/C has expanded its health programs to including HIV prevention, reproductive health and family planning, diarrhea prevention and treatment, as well as malaria prevention and treatment. In addition to health programming expansion, PSI/C has also focused on enhancing its local roots, ensuring local leadership with a Cambodian Executive Director and local governance structure in Cambodia. PSI/C is on track towards realizing its institutional development vision and registering as a local NGO with the aim of ensuring sustainable health impacts that are addressing the needs of the poor and vulnerable in Cambodia long into the future. PSI’s involvement in family planning and reproductive health began with the launch of the OK oral contraceptive in 1997, followed by the OK injectable contraceptive in 2002. In 2007, PSI expanded this portfolio with intensive SMBCI PROJECT FINAL PERFORMANCE EVALUATION 61 support for the uptake of long-term methods, including IUDs and implants. Since 2002, PSI/C has supported the Sun Quality Health Network, a social franchise of private sector health clinics providing high-quality, affordable family planning and health service delivery. Medical detailing teams reach over a thousand providers with products, information, and training to improve quality of care and support for services, products, and behaviors. Behavior change communications utilize mass media and targeted outreach through mobile video units and community mobilization to increase the use of modern contraceptives. PSI/C also contributes and supports the public sector through collaboration with other USAID supported partners in improving clinical and practical skills such as IUD and implant insertion and removal, and referral of women of reproductive age to public facilities for family planning and reproductive health services. In 2009, PSI/C revitalized support for the Orasel DTK (the world’s first diarrhea treatment kit combining ORS and Zinc), and supports safe water and hygiene programs focusing on a point-of-use household water product, treatment tablets. In 2002, PSI/Cambodia created the UHN, a partnership that aims to improve healthcare in Cambodia by supporting local NGOs to create widespread access to products and behavior change through innovative behavior change interventions. As a key partner of the RGC, PSI/C helps support health systems strengthening by harnessing the private sector to better meet the health needs of the poor and vulnerable in Cambodia. PSI/C does so through a portfolio of interventions that includes a franchise network of private providers for the provision of quality services; and medical detailing and product distribution through private sector outlets to provide high-quality, affordable, and easy-to￾access products. PSI/C’s programs such as medical detailing, training programs, supportive supervision visits, and targeted outlet support aim to improve provider behaviors–– from pharmacies to clinics, entertainment establishments to small shops––to meet health needs. Using innovative techniques and evidence-based decision￾making as a foundation of programs, PSI/C encourages healthier behaviors by ensuring poor and vulnerable populations are equipped with the opportunity, ability, and motivation to make healthier choices. PSI/C also takes a TMA to understand the role of all sectors (commercial, social marketing, and public sector) in providing solutions for long-term health impact. TMA have helped PSI/C better understand the role of social marketing in growing the category of health products and services while emphasizing sustainability to effectively use limited donor resources. TMA helps ensure all income groups are appropriately targeted so that low income and vulnerable populations have access to free or subsidized products while leaving open space and encouraging an increased role for the commercial sector to improve the market. TMA for the HIV health area has included introduction of a cost recoverable, repositioned Number One condom brand that is targeted toward higher income men, and support to build the “value for money” OK condom brand that is targeted toward low income populations in high-risk relationships. Strengthened behavior change interventions reach those most at-risk of HIV, focusing on entertainment workers, MSM, and male clients, while supporting prevention for positive efforts. TMA for reproductive health: PSI/C was entrusted by the MOH to lead the first ever population based contraceptive projection estimates up to 2020. PSI/C provided technical support in calculating contraceptive needs for the country based on historical data trends of the entire market including public, commercial, and social marketing sectors. Through this effort, the MOH has renewed its commitment to increasing the allocation and use of the national budget for contraceptive supply from 2015 onwards. PSI/C is also engaged with the commercial sector to bring into the market mid-range oral contraceptives to provide more choices for women and to create a healthy oral contraceptive market in the country. PSI/C’s diversified donor base includes USAID, DfID, KfW, MOH, private foundation support, UNFPA, and the GFATM. In support of the activities to be evaluated under this scope of work, DfID is contributing a total of 4.4 million British Pounds to be programmed through USAID/Cambodia. SMBCI PROJECT FINAL PERFORMANCE EVALUATION 62 PSI/C’s programs annually deliver over 550,000 CYPs, socially market over 20 million condoms, treat over 110,000 cases of diarrhea, provide over 20 million liters of household drinking water, and treat over 100,000 cases of malaria to sustainably improve the lives of millions of Cambodians every year. The availability of socially marketed contraceptives is especially critical in Cambodia given the uncertainty of contraceptive commodity security in the public sector. Donor support for public sector contraceptive procurement will end in 2015 at which point the Cambodian government must take up financing. While a similar transition for essential drug procurement was successful in the past, there appears to be less political will to take on family planning due to entrenched pro-natalist sentiment, stemming largely among politicians outside of the MOH. While it is assumed the RGC will be able to avoid stock-outs of contraceptives in the future, the expanded availability of products through social marketing and the private sector will help mitigate, to some degree, potential stock-outs of public sector contraceptive supplies. III. Statement of work: USAID/Cambodia requests that a team be assembled to conduct a performance evaluation of the Mission’s current social marketing activity implemented by PSI. The evaluation should explore relevant issues and areas of focus (HIV/AIDS, family planning/reproductive health, and child survival) as determined by OPHE. The team will need to assess the country’s current health sector issues, including existing gaps, resources, and challenges. In addition to meeting with relevant USAID/Cambodia health partners, key government counterparts, and development partners, the team will conduct field visits to project sites to interview health providers and beneficiaries. Key Evaluation Questions The following issues will need to be addressed in close consultation with the Mission: I. Performance Evaluation of Current USAID Funded Social Marketing Efforts: Per the USAID 2011 Evaluation Policy: “Performance evaluations focus on descriptive and normative questions: what a particular project or program has achieved (either at an intermediate point in execution or at the conclusion of an implementation period); how it is being implemented; how it is perceived and valued; whether expected results are occurring; and other questions that are pertinent to program design, management and operational decision-making. Performance evaluations often incorporate before-after comparisons, but generally lack a rigorously defined counterfactual.” [ADS203.3.2.1]  What are the major outcomes, achievements, constraints, and lessons learned of the USAID funded Social Marketing program in Cambodia?  Did the project successfully address gender issues?  How cost-effective has this program been? Which interventions have been most/least cost-effective?  How effective are the outreach and demand creation activities of PSI? Which would be most valuable to continue? SMBCI PROJECT FINAL PERFORMANCE EVALUATION 63  How effective are the various commodities socially marketed by PSI? Which would be most valuable to continue?  How optimal has the balance between supply-focused activities (distribution) and demand focused (promotion) activities been?  Is the total market approach appropriately structured (public, social market, and commercial sectors) to optimize coverage while supporting long-term sustainability?  What are the lessons learned from the localization process?  How does the current project contribute towards policy reforms––is their participation in TWGs paying off?  How to improve engagement with the RGC/MOH and other relevant players?  Are the current technical approach, scale, and geographic coverage sufficient to achieve USAID overall goal?  Is the current monitoring system effective and are any changes recommended? IV. Methodology: The evaluation team should consider a range of possible methods and approaches to collecting and analyzing the information, which will be required to assess evaluation objectives and address questions outlined in section III of this SOW including but not limited to the review of background documents, key informant interviews, site visits, and a series of team planning meetings with Mission staff as deemed necessary. Document Review • USAID/Cambodia will provide the evaluation team with key documents prior to the start of in-country work. • Prior to conducting field work, the evaluation team will review existing national documents including RGC/MOH policies; guidelines; strategic plans; relevant program evaluations; national surveys; annual operational plans of national programs, provincial, and operational districts; reports, etc. • In addition, the team will review relevant USAID funded project documents, including proposals, implementation plans, monitoring and evaluation plans, progress reports, review/evaluation reports, training curricula, etc. Team Planning Meeting The evaluation team will have three days before the evaluation begins to finalize the plan and meet with Mission staff. The initial briefing with the Mission staff will allow the team to discuss the purpose, expectations, and agenda of the assignment. The team will: • Clarify team members’ roles and responsibilities • Establish a team atmosphere, share individual working styles, and agree on procedures for resolving differences of opinion • Review and develop final evaluation questions • Review and finalize the assignment timeline and schedule SMBCI PROJECT FINAL PERFORMANCE EVALUATION 64 • Develop data collection methods, instruments, tools and guidelines • Review and clarify any logistical and administrative procedures for the assignment • Develop a preliminary draft outline of the evaluation team’s report and activity designs • Assign in drafting responsibilities for final deliverable Internal Mission/Evaluation Team meetings will include, at a minimum: • Initial organizational/introductory meeting in which the evaluation team will present an outline and explanation of the design of the evaluation • Mid-evaluation review for the Mission outlining progress and implementation problems • Final evaluation debrief/summary of the data and presentation of the draft report Field Visits/Interviews: • A general list of relevant stakeholders and key partners will be provided to the evaluation team. In consultation with the Mission staff, the evaluation team will develop a list of visits and interviews with key MOH partners (national programs, provincial offices, and district health teams), relevant USAID implementing partners, as well as key donors and stakeholders. The MCH Team at the Mission can assist with arranging these meetings. • The site visits will involve interviews with government staff, health workers, implementing partners, and both individuals and communities who were targeted to benefit from the project. • The evaluation team will conduct interviews with donor organizations, selected NGOs, and other key respondents identified during the planning meeting. V. Team Composition: The evaluation team will consist of five members including two international consultants, one local consultant, and two specialists from USAID/Washington Bureau of Global Health. The team should have a combination of significant social marketing (and other private sector approaches) expertise, extensive knowledge of USAID programming, and considerable hands-on applied program evaluation and design expertise. In depth knowledge and experience in Cambodia and the region is highly desirable. In addition to this five-member external team, members of the Mission’s MCH team will contribute extensively to the entire process, working closely as part of the larger evaluation team. Consultant 1/Team leader (Social Marketing Expert) - Liaise with client/AID and represent team - Ensure logistics arrangements in the field are complete - Develop Evaluation Framework, present MSI client/USAID with the framework for the evaluation on day three of the assignment in-country SMBCI PROJECT FINAL PERFORMANCE EVALUATION 65 - Responsible for overall design and structure of evaluation, including the finalization and negotiation of the team work plan for the assignment - Facilitate team planning meetings or work with a facilitator and/or the Mission’s Health Office Program Assistant to set the agenda and other elements of evaluation - In consultation with team members, responsible for instruments used in evaluation - Supervises, supports, and performs data collection activities in Cambodia with team members per work plan - Responsible for mid-term and debrief presentations to client in the form of PowerPoint presentations. Deliverables from team leader include a Final Evaluation Report (both a copy for internal use and a sanitized public version). - Take the lead in preparing, coordinating team member input, submitting, revising, and finalizing the Final Evaluation Report per AID Evaluation guidelines: (http://www.usaid.gov/policy/evalweb/documents/TIPS-ConstructinganEvaluationReport.pdf, http://kambing.ui.ac.id/bebas/v01/DEC-USAID/Other/publications-style-guide.pdf) - Ensure timely submission of drafts and deliverables to MSI. This will involve: • Managing and contributing significantly to the process of drafting, finalizing, and submitting of deliverables • Leading/facilitating team coordination meetings in the field • Coordinating the workflow and tasks and ensuring that team members are working to schedule Consultant 2/International Expert - Support evaluation planning, design, and execution - Responsible for developing instruments for data collection - Perform data collection in-country - Contribute to mid-term and final PowerPoint presentations - Analyze data per design framework and develop findings, conclusions, and recommendations - In addition to producing sections of Draft Evaluation Report - Write sections of the draft report and submit per work plan - Edit and finalize final report with team leader Consultant 3/Local expert - Support evaluation planning, design, and execution - Support logistics arrangements for team planning meeting and interviews as directed by the evaluation team leader - Responsible for contributing to the development of instruments for data collection - Perform data collection in-country - Contribute to mid-term and final PowerPoint presentations SMBCI PROJECT FINAL PERFORMANCE EVALUATION 66 - Analyze data per design framework and develop findings, conclusions, and recommendations - Write sections of the draft report and submit per work plan USAID/Washington contribution: It is expected that two members from USAID/Washington will also join the team: two senior technical experts with expertise relevant to the various aspects of the social marketing project (the Office of HIV/AIDS (OHA) and the Office of Population and Reproductive Health (PRH)). USAID/Cambodia contribution: In addition, the Mission’s MCH team members will all play a role in the planning, implementation, and finalization of this evaluation activity. USAID/Cambodia can assist with limited support including logistical arrangements such as lodging, in-country travel, scheduling meetings, etc., but the team should be prepared to work and travel as independently as possible. The Mission’s MCH team will be available to the team for consultations regarding resources and technical issues, before and during the assessment process. A six day work week is authorized while working in-country. The team should bring to post laptops to prepare reports. VI. Timeline and LOE: USAID/Cambodia anticipates that the entire evaluation would be completed within a 5-week period including preparatory work before arrival to the country and the finalization of deliverables after the trip. This would include preparation days, in-country work, and the completion of written deliverables. SMBCI PROJECT FINAL PERFORMANCE EVALUATION 67 Proposed timeline: VII. BACKGROUND MATERIALS:  Relevant background materials will be shared electronically, and will include project documents as well as documents outlining the context and background (e.g., Cambodian government policies and strategies) of the development environment. Task/Deliverable Duration Team Leader Program Expert Local Expert Other Party 1. Scheduling of interviews with local stakeholders 3 3 2. Travel to Cambodia 2 days 2 2 0 July 14-15 3. Reading of Materials 1 1 0 4. Team Planning Meeting and meetings with Mission staff 1 day 1 1 1 1 16-Jul 5. Preparation of data collection tools, protocols and workplan 3 days 3 3 1 3 6. Information and data collection: meeting with key partners and government officials in Phnom Penh, as well as field visits. 10 days 10 10 10 Jul 30 Aug 3 7. Discussion, analysis and preparation of draft Evaluation report. 5 days 5 5 5 8. Debrief meetings with Mission (preliminary draft report due). 1 day 6-Aug 9. depart Cambodia 1 day 1 1 1 6-Aug 10. feedback received from Missions on Preliminary Draft 1 1 0 14-Aug 10 11. Edit finalization of DRFT RPT by Team 3 3 1 12Edit at MSI formatting/technical 5 TOTAL LOE 27 27 22 NA SMBCI PROJECT FINAL PERFORMANCE EVALUATION 68 VIII. Deliverables: Phase One (In-Country) Deliverables: 1. Evaluation Framework: Present USAID with the framework for the evaluation on day three of the assignment in-country. This will include materials produced during the team planning meeting described in Section IV above. OPHE staff will be available for consultation during the initial planning time. 2. Debriefings: The team will conduct debriefings at the end of weeks two, three, and four. The first debrief will take place near the end of the second week to discuss the preliminary findings of the evaluation. The second debrief will take place part way through the concept design phase to ensure that the team and USAID/OPHE are in agreement on concept trajectory and priorities. The final debrief will present key findings and recommendations. The format for debriefs will be agreed upon between OPHE and the evaluation team in the first meeting. 3. Draft Evaluation Report: The first draft report of the evaluation will be due before the team departs the country. The draft report will include key findings, conclusions, and recommendations. USAID will provide comments on the draft report within five days of submission. Phase Two (Finalized Remotely) Deliverables: 1. Final Evaluation Report: The team incorporates USAID comments and submits the final report within five day after receiving it. USAID/Cambodia requests an electronic version of the final report in MS Word or PDF format. The final report shall be direct and brief, but thorough. Annexes shall be included with the final report. The report should have the following structure: • Executive summary • Table of contents • Introduction – purpose, audience, summary of statement of work • Background – brief overview of health situation in Cambodia, including the main trends and challenges relevant to social marketing • USAID assistance to date – a description of the USAID program strategy and activities implemented in response to the problem (coverage, implementing partners, funding levels) as well as strategic directions • Methodology • Qualitative/quantitative findings • Conclusions based on findings SMBCI PROJECT FINAL PERFORMANCE EVALUATION 69  Have findings and issues identified with respect to design and technical aspects, management and implementation, and policy  Project achievements and outcomes  Project gaps and missed opportunities • References • Annexes o SOW o List of persons contacted o Activity timeline/schedule o Bibliography SMBCI PROJECT FINAL PERFORMANCE EVALUATION 70 APPENDIX B PERSONS CONTACTED No. Name Position Organization/Agency 1 Ms. Tara Milani Deputy Director of OPHE USAID/Cambodia 2 Mr. Robin Mardeusz, MCH Team Leader 3 Ms. Patricia Orlowitz Program Officer 4 Ms. Robin Martz HIV/AIDS Team Leader 5 Dr. Sotheara Nap Development Assistance Specialist for Infectious Diseases 6 Dr. Sopheanarith Sek Development Assistance Specialist for MCH 7 Ms. Janet Hayman PEPFAR Interim Coordinator 8 Dr. Sok Bunna HIV/AIDS Technical Team Lead, AOR for FHI and KHANA 9 Mr. Billy Pick HIV Technical Advisor USAID/Washington 10 Mr. Craig Riegler Regional Agreement Officer USAID/Bangkok 11 Ms. Yasmin Madan Country Representative PSI/Cambodia 12 Dr. Sok Sokun Deputy Country Representative 13 Dr. Heng Kheng Health Services Director 14 Mr. Monte Achenbach Technical Advisor 15 Ms. Alysha Beyer Senior Technical Advisor 16 Mr. Khun Veasna Medical Representative manager 17 Ms. Long Dianna Strategic Information Director 18 Mr. Buth Lorn Sales and Distribution Director 19 Ms. Gnoun Sophy Medical representative 20 Ms. Yan Rady PSI sales representative 21 Ms. Yim Sineth IPC coordinator for Battambang province 22 Ms. Ang Asima IPC officer Battambang 23 Mr. Eat Panha IPC officer in Battambang 24 Mr. Doung Vechny PSI support supervisor, Battambang 25 Dr. Eng Veng Eang Supportive Supervision Manager SMBCI PROJECT FINAL PERFORMANCE EVALUATION 71 26 Ms. Chea Sophan Healthcare provider, Russei Keo, Phnom Penh Sun Quality Health 27 Ms. Um Samedy Nurse and owner of a private health care room, Network Kompong Chhanang 28 Ms. Ung Savoeun Healthcare provider, Battambang 29 Ms. Kong Srey Mom Owner of a private health room, Kompong Chhanang - 30 Ms. Chan Sophy Village shop keeper, Pursat - 31 Ms. Nov Sarath Pharmacy Depot B owner, Battambang - 32 Ms. Meng Solada Pharmacy Depot B owner, Battambang - 33 Mr. Dork Pagna Program Manager Men’s Health Social Services, Battambang 34 Ms. Chan Theary Executive Director RACHA 35 Dr. Sun Nasy Program Implementation Advisor 36 Dr. Chan Ketsana Team Leader- Child Health and Nutrition 37 Dr. Eam Mony Safe Motherhood Team Leader 38 Ms. Pal La Ine Assistant Provincial Coordinator in Pursat 39 Mr. Sou Penh Provincial child health officer, Pursat 40 Dr. Var Chhivorn Associate Executive Director 41 Dr. Iv Ek Navapol Planning & Development Coordinator RHAC 42 Mr. Ngudup Paljor MCH Advisor 43 Dr. Ly Penh Sun Deputy Director, NCHADS MOH 44 Dr. Ouk Vichea Technical Bureau Chief, NCHADS 45 Dr. Tung Rathavy Director of NMCHC, Director of National Reproductive Health Program 46 Dr. Chhroeung Sokhan Chairman of the Contraceptive Security Working Group, Former Director of Drug and Food Department 47 Dr. Chhorn Veasna Director of National Programe for ARI/CDD 48 Dr. Prak Vun Provincial Health Department Director, Kampong Chhnang 49 Mr. Klem Sakun Pursat provincial health department director 50 Mr. Khuy Dy Deputy provincial health department director 51 Mr. Lim Hour Peov (John) Executive Director Full Well Trading (Okamoto distributor) SMBCI PROJECT FINAL PERFORMANCE EVALUATION 72 52 Mr. Andy Chang General Manager 53 Mr. Lin Hwa Fu Managing Director 54 Ms. Stefanie Wallach Program Director Marie Stopes International 55 Mr. Emerson Mar Deputy Director 56 Mr. Narendra Kumar Country Director Mega Life Sciences 57 Dr. Heak Bunroeun National Sales Manager 58 Ms. Maline Srun Branch Manager 59 Mr. Soratha Chan Personal Assistant to Branch Manager Bayer Cambodia 60 Dr. Nhauch Te National Sales Manager Primary Care 61 Dr. Sin Somuny Executive Director MEDiCAM 62 Dr. Chris Vickery Health Advisor AusAID 63 Mr. Christophe Grundmann Chief of Party URC 64 Mr. May Tum Assistant Representative UNFPA 65 Dr. Sam Sochea National Programme Officer, Reproductive and Maternal Health 66 Ms. Amy Weismann Associate Director, Prevention and Mitigation FHI 360 67 Mr. Heng Saly Technical/Program Officer, MSM 68 Dr. Tith Khimuy Deputy Director, IP’s Program Management KHANA 69 Ms. Malalay Ahmadzai MCH Specialist 70 Ms. Ung Vanny Health Education Officer UNICEF 80 Mr. Chum Aun Child Health Officer SMBCI PROJECT FINAL PERFORMANCE EVALUATION 73 SMBCI PROJECT FINAL PERFORMANCE EVALUATION 74 APPENDIX C TEAM INTERVIEW QUESTIONNAIRES Public Sector (Bureaucrats not Providers) Central Level: 1. Name and position 2. Areas of responsibility 3. Areas of contact/collaboration with PSI project 4. Experience with PSI project: - What is/was the nature of PSI project's contribution to your area/field? - How was it decided what the PSI project would do/contribute? - How would you describe the value of PSI project's contribution? (Did the PSI project contribution solve problems, address priority areas, extend your reach, bring new skills/training to your staff, facilitate new policy development, facilitate involvement of private sector resources in your field/area, open new opportunities, add funding?) - How would you describe the PSI project's willingness to work collaboratively with you/your office? - How well has the PSI project and other organizations worked together to assist you in your work? Have there been overlaps, duplications, difficulties in working together? - How would you describe the quality/qualifications of PSI project staff working with your office/area? - What has been the efficiency/effectiveness of PSI project reporting and management systems viz a viz your work/office? - How well has PSI project shared information, lessons learned, research results, and other resources important to your work with you and your staff? - What are the priority current challenges/opportunities in your area of work? - How could the PSI project best help you in the future to meet those challenges/opportunities? 5. What, in your opinion, is an appropriate role for the private/commercial sector in health care delivery in Cambodia? Are there any obstacles/issues/problems that you believe currently stand in the way of the private/commercial sector's ability to fulfill the role that you envision? Are there any special opportunities that exist now for the expanded participation of the private/commercial sector in health care services delivery? What about in reproductive health and preventive health care in particular? 6. Do you currently have any opportunities for collaboration or exchange of information with representatives of the private sector in health care? How willing do you perceive the private/commercial sector to be in working with you/your office or with the MOH, in general? SMBCI PROJECT FINAL PERFORMANCE EVALUATION 75 7. How do you feel that the public sector and the private/commercial sector might best work together to ensure that everyone has access to the health care services and products they need? Is this possible? Would any policy, regulatory, or attitudinal changes be required for this to occur? 8. Do you feel that your office now has the funds or other resources it requires to fulfill all its responsibilities/health care service delivery roles? 9. What is the current funding status within the government for the procurement of commodities like contraceptives, diarrhea treatment kits, and the like? How, if at all, do you think that status will change during the next five years? 10. Does the MOH have a plan for maintaining levels of contraceptive products in the overall environment that is necessary to meet demand as funding status changes in the future? 9. How, if at all, will decentralization affect the ability or likelihood of the public sector and private/commercial sector working together effectively in health care delivery? 10. What are the channels/resources now at your disposal for reaching the population with behavior change/healthy behavior messages? Are these channels well-funded, in your opinion? By whom or through which channels? 11. How are village health volunteers recruited, trained, managed, and otherwise supported? What are the current tasks assigned to village health volunteers? In how many health areas do they now work? For how many people/families is each volunteer responsible? How do village health volunteers relate to the community motivators of the PSI project, for example? Provincial/Local Level 1. Name and position 2. Areas of responsibility 3. Areas of contact/collaboration with PSI project 4. Experience with PSI project: - What is/was the nature of PSI project's contribution to your area/field? - How was it decided what PSI project would do/contribute? - How would you describe the value of PSI project's contribution? (Did PSI project contribution solve problems, address priority areas, extend your reach, bring new skills/training to your staff, facilitate new policy development, facilitate involvement of private sector resources in your field/area, open new opportunities, add funding?) - How would you describe PSI project's willingness to work collaboratively with you/your office? - How well have PSI project and other organizations worked together to assist you in your work? Have there been overlaps, duplications, difficulties in working together? - How would you describe the quality/qualifications of PSI project staff working with your office/area? - What has been the efficiency/effectiveness of PSI project reporting and management systems viz a viz your work/office? - How well has the PSI project shared information, lessons learned, research results, and other resources important to your work with you and your staff? - What are the priority current challenges/opportunities in your area of work? SMBCI PROJECT FINAL PERFORMANCE EVALUATION 76 - How could the PSI project best help you in the future to meet those challenges/opportunities? 5. What are the channels/resources now at your disposal for reaching the population in your area with behavior change/healthy behavior messages? Are these channels well-funded, in your opinion? By whom or through which channels? 6. How are village health volunteers recruited, trained, managed, and otherwise supported? What are the current tasks assigned to village health volunteers? In how many health areas do they now work? For how many people/families is each volunteer responsible? How do village health volunteers relate to the community motivators of the PSI project, for example? 7. To what extent do you rely on social marketing products or products distributed through NGOs to ensure that sufficient commodities like contraceptives, diarrhea treatment kits, and the like are available for all the people in your geographic area who want them? 8. Under decentralization, do you expect to have more control over the quantity of health commodities available in your health centers or other health outlets? Which types of health-related commodities do you expect to have highest priority in a decentralized system? 9. What is the priority status of health care needs/funding, in your opinion, for the local government? Which health care needs/health areas are most important, in your opinion, in your geographic area? 10. What role, if any, do private/commercial sector providers play in serving the health care needs of the people in your geographic area? In your opinion, what are the primary strengths and/or weaknesses of private/commercial sector providers in your area? Are you interested in actively engaging the private sector in working to meet the health needs of people in your area? Why or why not? How do you think you can most effectively engage the private sector in working with you to meet common public health goals? Other Donors 1. Name and position 2. Areas of responsibility 3. What are your agency's current priorities for health assistance in Cambodia? 4. What are your agency's current health programs here? 5. How, if at all, will your agency's funding for health programs in Cambodia change in the next five years? 6. Do international donors working in Cambodia have a mechanism for regular collaboration and cooperation with each other? Has it been effective? Do you have any special problems, issues, or success stories? 7. In your experience, does it work well when two or more donors are funding the same implementing organization? What are the challenges, opportunities, or benefits? 8. Experience with PSI project: - What is the nature of PSI project's contribution to/involvement in your overall health portfolio? - How would you estimate the role of the PSI project as a percentage of your total health assistance here? What is your total support of PSI project, and how can it be broken down? - Has the PSI project's work with USAID in any way diminished, in your opinion, its ability to work effectively with your agency? Has it added value to its work for your agency? 9. What do you see are the highest priority areas for assistance/highest priority needs in health care in Cambodia over the next five years? SMBCI PROJECT FINAL PERFORMANCE EVALUATION 77 10. What do you see are the probable roles of the private/commercial sector in health care delivery and financing during the next five years? In what areas of health care is the private/commercial sector most likely to operate willingly and/or sustainably? 11. What do you envision are the likely and effective means of involving the private/commercial sector more broadly in health care delivery, especially in areas of greatest health need (e.g., vouchers, incentives, insurance, etc.)? Has your agency had any experience to date in working with any of these mechanisms? What have been the results? 12. Which are the most important policy issues facing health care delivery and funding in Cambodia, now and in the next five years? How best do you think these policy issues can be addressed? Which agencies/organizations in Cambodia do you feel are best equipped/positioned to advocate for these policy changes? Are there existing advocates for the role of the private/commercial sector in health care? USAID 1. Name and position 2. Areas of responsibility 3. How has the PSI project been involved in the work of your technical area/office? What percentage of the funding for the PSI project does your area/office provide? 4. What are the primary contributions that you feel the PSI project has made to the achievement of the goals of your technical area/office? Do you feel that those contributions commensurate with the amount of funding that your area/office has provided to the project? 5. How would you describe the technical strengths of the PSI project, its leading competencies, and weaknesses? How have these strengths and weaknesses shown themselves in the course of project implementation? 5. How nimble do you feel that the PSI project has been in responding to changes in the environment or to recommendations for change? How quickly, in your opinion, has the PSI project staff recognized the need for change in strategy or implementation on its own and initiated with you the process of "mid-course" corrections? Can you give us an example? 6. Have you ever felt that the PSI project could provide better assistance/more impact in your technical area if the breadth of its activities/responsibilities were diminished in any way? If so, under what circumstances did you feel this? 7. What do you think are/can be the most important contributions of the social marketing process to the achievement of improved public health in your area of technical responsibility? 8. Where do you feel that the private sector is/can be of the most use in achieving the health goals of your technical area/office? Are there policies or other obstacles that you see that could prevent the private sector from participating fully and effectively? 9. Are there currently any effective advocates for the private/commercial sector's role in health care in Cambodia? 10. What do you see as the highest priority areas for assistance/highest priority needs in health care in Cambodia over the next five years? 11. What do you see are the probable roles of the private/commercial sector in health care delivery and financing during the next five years? In what areas of health care is the private/commercial sector most likely to operate willingly and/or sustainably? SMBCI PROJECT FINAL PERFORMANCE EVALUATION 78 12. What do you envision are the likely and effective means of involving the private/commercial sector more broadly in health care delivery especially in areas of greatest health need (e.g., vouchers, incentives, insurance, etc.)? Has your office had any experience to date in working with any of these mechanisms? What have been the results? 13. What are the most important policy issues facing health care delivery and funding in Cambodia now and in the next five years? How best do you think these policy issues can be addressed? Which agencies/organizations in Cambodia do you feel are best equipped/positioned to advocate for these policy changes? Commercial Sector Pharmaceutical Company Managers 1. Name and position, company name 2. Areas of responsibility 3. Which, if any, products related to contraceptives and reproductive health, treatment of childhood respiratory infections and pneumonia, treatment of diarrheal disease, or childhood nutritional supplements does your company currently market in Cambodia? Can you please give us a list of these brands and the retail price of each? Is there local production of any of these product types in Cambodia? 4. What percentage of your total revenues in Cambodia would you estimate is represented by the sale of contraceptive products, excluding condoms? Is this a line of business that you would like to expand here? How important to your parent company's overall business is the sale of contraceptive products, excluding condoms? 5. Do you have plans for the introduction of any new contraceptive brands/products into this market within the next few years? Does your parent company have a progestin-only oral contraceptive, injectable contraceptive, or IUD? 6. How would you describe the target market for each of your contraceptive brands? Do you see possibilities for growth in the size of your target market in Cambodia for each of these contraceptive brands? 7. Do you see any potential in Cambodia for a mass market, lower-priced oral contraceptive brand? Why, or why not? What level of unit sales would be required, in your opinion, to make a lower-priced oral contraceptive brand profitable for/interesting to your company? At what price? At that price and level of unit sales, what would be the investment in product launch, product promotion, product detailing, etc. that your company could likely make? 8. What has been the rate of growth in unit sales in the overall commercial market in Cambodia for oral contraceptives in the past five years? What do you anticipate over the next five years? For injectable contraceptives, IUDs, or condoms? Has the rate of growth of your company's oral contraceptive brands been about the same as that of the overall market? What about injectables, IUDs, or condoms? 9. What is the level of market penetration that your current distributions of oral contraceptives achieve? How many pharmacy outlets do your oral contraceptives reach? What percentage of total pharmacy outlets do you reach? What percentage of your oral contraceptive sales are accounted for by the largest three (or four) cities? 10. Which is your highest selling (unit sales) oral contraceptive brand in Cambodia? Which is your most profitable oral contraceptive brand in Cambodia? 11. How many medical representatives does your company employ/use for the promotion of its products? Which types of physicians do you target with your medical representatives? How many physicians do they reach? How often do they see each physician? How often within an annual cycle do your medical SMBCI PROJECT FINAL PERFORMANCE EVALUATION 79 representatives present information to physicians about your contraceptive products? Which products? Does your company sponsor any seminars or other training events for physicians or for other types of providers? Are contraceptive products ever a part of these programs? How often? When was the last such educational event that you sponsored that focused on a contraceptive product/brand? 12. Would you say that physicians in Cambodia are resistant to prescribing contraceptives in general? Resistant to hormonal contraceptives or IUDs? If so, why do you think they are resistant to them? How best can this resistance be overcome in your opinion? 13. Do you collaborate with the MOH either in providing information/training to its physicians or in responding to MOH procurement tenders? Do you collaborate in any other ways? Does the MOH in general appear to be open to collaboration with the private/commercial sector? 14. Does the public sector provision of free contraceptives to some consumers affect your business? If so, to what extent and how? 14. Do you market any brands of condoms? At what prices and through which distribution and retail outlets? 15. How would you describe your target market for condoms? Do you foresee growth in the size of this target market? Why or why not? 16. Are there any particular obstacles that you can identify for us that inhibit your sales of condoms or contraceptive products in the Cambodian market? 17. Do you reach rural areas with any of your products/brands? How? Which products/brands? Which product types can you sell profitably in rural areas? 18. Is it particularly difficult to register new pharmaceutical products in Cambodia? Why? 19. Are there particular policy or regulatory issues that make it difficult for you or pharmaceutical companies, in general, to do business in Cambodia? Are there governmental price controls or other pricing regulations? If so, how might these issues best be resolved or overcome? 20. Have you had any direct contact with the PSI project in Cambodia? If so, under what circumstances? Have you collaborated with the PSI project in any activities that were helpful to you and your business? Can you give us an example? Have the activities of the PSI project in any way hindered, in your opinion, your ability to do business profitably in Cambodia? Can you give us an example? 21. The PSI project has undertaken a number of market research activities in Cambodia. Has the project shared the results of any of this research with you? How? Was the research helpful to you or your business? How? 22. Have you ever heard of the Cambodian Demographic and Health Survey supported by USAID? Have your ever been presented with any of the data or data analysis gained from this survey? Condom Distribution Companies 1. Name and position, company name 2. Areas of responsibility 3. What are the brands of condoms that you sell in Cambodia? Can you please give us a list of each brand that you sell and its retail price? Is there any local production of condoms in Cambodia? 4. How would describe your target market for each of your condom brands? Do you foresee any growth in this target market during the next five years? Why or why not? SMBCI PROJECT FINAL PERFORMANCE EVALUATION 80 5. What has been the rate of growth in unit sales in the overall commercial market in Cambodia for condoms in the past five years? What do you anticipate over the next five years? Has the rate of growth of your company's condom brand sales been about the same as that of the overall market? Better? 6. What is the level of penetration into the market that your current distribution of condoms achieves? How many pharmacy outlets do your condoms reach? What percentage of total pharmacy outlets do you reach? What other types of outlets stock your condom brands? How do your condom sales in other outlets compare with your condom sales in pharmacies? What percentage of your condom sales are accounted for by the largest three (or four) cities? 7. Do you actively advertise or promote your condom brands? How? About what percentage of sales revenues do you invest in the advertising and promotion of condom brands? Has advertising to consumers or promotion of the trade more effective in increasing your condom sales? 8. Do you reach rural areas with any of your condom brands? How? Which brands? Which product types can you sell most profitably in rural areas? What, in your opinion, is the most effective means of promotion and/or advertising to use in rural areas? Which is your highest selling (unit sales) condom brand in Cambodia? Which is your most profitable condom brand in Cambodia? 10. What percentage of your company's total revenue in Cambodia would you estimate is represented by the sale of condoms? Is this a line of business that you would like to expand here? 11. Do you have plans for the introduction of any new condom products into this market within the next few years? 12. Do new condom brands have to be registered through the MOH before they can be sold or imported into Cambodia? Is this process particularly difficult? Why? Are there governmental price controls or other price regulations? Are there particular policies or regulatory issues that make it difficult for you or other condom distribution companies, in general, to do business in Cambodia? If so, how might these issues best be resolved or overcome? 14. Does the public sector provision of free condoms have an impact on your business? If so, to what extent or how? 15. Have you had any direct contact with the PSI project in Cambodia? If so, under what circumstances? Have you collaborated with the PSI project in any activities that were helpful to you and your business? Can you give us an example? Have the activities of the PSI project in any way hindered, in your opinion, your ability to do business profitably in Cambodia? Can you give us an example? 16. The PSI project has undertaken a number of market research activities in Cambodia. Has the project shared the results of any of this research with you? How? Was the research helpful to you or your business? How? Pharmacies and Pharmacists 1. Name and position, name and location of pharmacy 2. Areas of responsibility 3. Which contraceptive products do your pharmacy regularly stock and what is the price of each (oral contraceptives, IUDs, injectables)? 4. About how many cycles of oral contraceptives do you sell in a month here? IUDs? SMBCI PROJECT FINAL PERFORMANCE EVALUATION 81 5. Which is the best-selling brand of oral contraceptives in your pharmacy? About how many cycles of that brand do you sell in a month? 6. What do you think is the most important factor in a woman's choice of oral contraceptive brands (e.g., doctor's recommendation/prescription, friend/family recommendation, advertisements, price)? How many women come with a doctor's prescription for their oral contraceptives (most, some, only a few)? 7. About what percent of your total sales revenue is represented by the sale of oral contraceptives, IUDs, injectables, and other contraceptive products? (Does this include or not include condoms? Break out condoms.) 7. Which condom brands do your pharmacy regularly stock and what is the price of each? 8. Who most frequently purchases condoms at your pharmacy – men or women? 9. About how many packs of condoms do you sell here per month? 10. Which is the best-selling brand of condoms in your pharmacy? Why do you think your customers prefer it? 11. What do you think is the most important factor in the customer's choice of condom brand? 12. About what percent of your total sales revenue is represented by the sale of condoms? 13. Do medical representatives call on you in your pharmacy? What types of products do they discuss with you? 14. Do pharmaceutical salesmen call on you in your pharmacy? Do they offer you special promotions or discounts for certain products? Do these ever include contraceptive products and/or condoms? What kind of offers do they make? 15. Do you ever run out of stock or have trouble obtaining contraceptive brands or condom brands for your pharmacy? If so, which brands? 16. Do representatives of OK and Number One condoms call on you? How often? Do they give you information about their products? Do they offer you special discounts or other promotions? Do representatives of OK oral contraceptives ever call on you? Do they give you information about the product or offer you special discounts or other promotions? 17. Have you ever had OK condom/oral pill or Number One condom display materials for your pharmacy? When did you last receive any? What types of materials? 18. Do your customers ever ask you questions about condoms or contraceptive products? What kinds of questions do they have? Do your customers ever ask you what to do for family planning? What recommendations do you give them? Do you customers ever ask you about "safe sex"? What recommendations do you give them? NGOs NGO managers 1. Name and position, name of organization 2. Areas of responsibility 3. Focus/mission of NGO SMBCI PROJECT FINAL PERFORMANCE EVALUATION 82 4. How long has your organization been established? 5. What are your principal sources of support? 6. Where do you operate? How many clients do you currently serve? 7. How long have you been working with the PSI social marketing project? 8. How were you selected to participate? 9. What do you do under the PSI project? 10. What support (financial, training, information sharing, advertising, etc.) do you receive from project participation? 11. What do you do as part of the PSI project? Are these activities new to your organization or activities that you had already been involved in? 12. Would your organization be able to continue project activities if PSI project funding or other types of PSI project support changed? To what extent? How would this be possible? 13. Has PSI project participation helped you to fulfill your organizational mission? How? 14. What would you say have been the most import benefits to your organization of participation in the PSI project? How has your organization changed, if at all, because of participation in the PSI project? 15. Have there been any particular problems for your organization in working with PSI as part of the social marketing project? NGO Providers (Sun Quality Health Network Providers) 1. Name 2. Name and location of clinic/outlet 3. Clinic hours 4. Type of provider 5. Length of time in practice 6. Do you also work somewhere else? Where? How often/when? 7. Principal services provided to clients 8. Average number of clients seen per month (per service and client type)? 9. Types of services now provided that are related to SQHN/PSI project. How much demand do you have in your clinic for these services (number of clients per month)? 10. Percentage of total business each month attributable to PSI project related services 11. How were you selected to participate in the Sun franchise? 12. What are the requirements for your participation in the Sun franchise? How easy/difficult has it been for you to meet those requirements? Why? 13. What have been the main benefits to you of participating in the Sun franchise? What have you received from PSI? Has what you received been helpful to you in your practice? How? 14. How do your clients know that you are a part of the Sun clinic franchise? What do you think it means to your clients that you are a Sun clinic provider? SMBCI PROJECT FINAL PERFORMANCE EVALUATION 83 15. How often do you have contact with someone from PSI/Sun franchise? What type of contact? 16. Has your business changed since you became a Sun clinic provider? How? 17. How far away/close is the next Sun clinic that you know about? The next MOH health center/clinic? 18. How is your clinic/practice different from that of a provider who is not participating in the Sun clinic franchise? Do your colleagues wish they could be a part of the Sun franchise too? Why? PSI Products 1. Does PSI/C depend on donor-procured products? 2. Can PSI/C source quality products locally from the private or public sector to ensure product availability? 3. Can PSI/C procure its own products through open international tenders? 4. Has PSI/C developed distribution agreements with commercial partners or negotiated other partnerships with commercial partners to ensure product supply? Price 1. How often does the PSI/C review prices for possible increases or decreases? 2. Does PSI/C adjust prices for inflation? 3. Does PSI/C track differences between actual and suggested retail prices and adjust prices when the actual price exceeds the suggested one? 4. Has the PSI/C conducted willingness to pay surveys with its target groups? 5. If so, are prices increased when people are willing to pay significantly higher prices than what is being charged? 6. Does the pricing policy reflect market segmentation strategies? 7. Does the recommended retail price for each target group fall within the range that each target group is willing to pay? 8. What percentage is the unit price to the trade of the unit percent cost of goods sold? Promotion and Communication 1. What is the ratio of brand-specific advertising and promotion expenses to sales revenues from the brand? 2. What is the ratio of brand-specific advertising and promotion to generic behavior change communication? 3. Are commercial entities investing in branded communication for health products related to PSI/C’s objectives? 4. Does PSI/C have monitoring systems to ensure that target groups are exposed to branded and generic communication? 5. Does PSI/C use communication strategies that work through local institutions that are able to assume message dissemination beyond the life of a campaign? SMBCI PROJECT FINAL PERFORMANCE EVALUATION 84 6. Does PSI promote generic products or are promotion activities always tied to one of their brands? Sustainability 1. Please describe the nature/extent of involvement of PSI sales/distribution staff vs. commercial distribution/sales within the project. 2. Does the project account system allow for segregation of costs related to good sold versus promotion/overhead activities? Can PSI calculate the total costs related to each product? 3. What is PSI’s current legal status as an organization? Are there any barriers to conducting commercial activities/earning profit as an NGO? 4. Of all product categories that PSI is engaged in, in how many is PSI/C the market leader? In those cases, what share of the market does the PSI brand have? 5. What metrics are being used to track progress toward achieving an expanded total market for key products under the USAID project? SMBCI PROJECT FINAL PERFORMANCE EVALUATION 85 APPENDIX D DOCUMENTS CONSULTED A Market Research Study on Number One & OK Condoms in Cambodia Market, A.C. Nielsen Vietnam, Final Report 2012 Achieving Cambodia's Millennium Development Goals, Royal Government of Cambodia, Ministry of Planning, 2010 BROS Khmer 2010 Behavioral Risks On-Site Sero-survey among At-Risk Urban Men in Cambodia, FHI & NCHADS, 2010 Cambodia Demographic and Health Survey, 2010, National Institute of Statistics, Directorate General for Health, and Measure DHS, September 2011 Cambodia Global Health Initiative Strategy, September 9, 2011. Available at: http://www.ghi.gov/ country/190250.htm Cambodia Socio-Economic Survey, 2009 Cambodia USAID Health Program Mid-Term Evaluation, Joy Riggs-Perla et al., March 2011 Child Survival (ppt), PSI/C Overview, PSI/Cambodia, July 2012 Cooperative Agreement 442-A-00-08-00001-00, Social Marketing and Behavior Change Interventions for HIV/AIDS, Reproductive and Sexual Health and Child Survival in Cambodia, Award Document dated February 5, 2008. Estimations and Projections of HIV/AIDS in Cambodia 2010-2015, National Center for HIV/AIDS Dermatology and STD Report, October 2011 Evidence to Action: Partnership for Health: Social Marketing & Behavior Change Interventions for HIV/AIDS, Reproductive and Sexual Health and Child Survival in Cambodia, Revised Technical Application, submitted to USAID, by PSI in collaboration with The United Health Network of NGOS, KHANA, RHAC, RACHA, and VBNK, July 31, 2007 SMBCI PROJECT FINAL PERFORMANCE EVALUATION 86 Fast Track Initiative: Roadmap for Reducing Maternal and Newborn Mortality 2010-2015, Royal Government of Cambodia, Ministry of Health, 2010 FY2011 Annual Progress Report, Reporting Period October 1, 2010 – September 30, 2011, Social Marketing & Behavior Change Intervention for HIV/AIDS, Reproductive and Sexual Health and Child Survival in Cambodia, PSI/Cambodia, November 1, 2011 General Population Census 2008, Royal Government of Cambodia, Phnom Penh: National Institute of Statistics, 2009 HIV/AIDS Portfolio Evaluation USAID/Cambodia Public Version Final Report, USAID Cambodia, July 2011 HIV/AIDS Prevention (ppt), PSI/C Program Overview, PSI/Cambodia, July 2012 MAP Study Evaluating the Availability of HIV Prevention Products, 6th Round, 2010, PSI Research Division, 2010 MAP Study Evaluating the Availability of PSI Birth-Spacing and Abortion Products and Services, 7th Round, PSI Research Division, 2011 MAP Study Evaluating the Availability of PSI Condoms and Lubricants, 7th Round, PSI Research Division, 2011 National Strategic Framework and Operational Plan on HIV, AIDS and STI for Men who Have Sex with Men in Cambodia, National AIDS Authority, June 2008 – June 2011 Non-State Providers and Health Service Delivery in the EAP Region, UNICEF, April 2010 PSI Overview (ppt), PSI/Cambodia, July 2012 PSI Provider Longitudinal Study Exploring Opportunity, Ability and Motivation (IUD) among SQHN Providers and Provider Satisfaction Survey, PSI Research Division, October 2011 PSI/Cambodia FY 2008 Annual Report, submitted to USAID November 3, 2008 PSI/Cambodia FY 2009 Annual Report, submitted to USAID November 6, 2009 SMBCI PROJECT FINAL PERFORMANCE EVALUATION 87 PSI/Cambodia FY 2010 Annual Progress Report, submitted to USAID November 1, 2010 PSI/Cambodia FY 2011 Annual Progress Report, submitted to USAID November 1, 2011 PSI/Cambodia HIV/AIDS Prevention Program Review, July 2012 (ppt) PSI/Cambodia Performance Indicator Tables, FY 2009-2011 PSI/Cambodia Sub-Grant Partners 2008-Present (spreadsheet) PSI/Cambodia Work Plan, submitted to USAID, FY 2009 PSI/Cambodia Work Plan, submitted to USAID, FY 2010 PSI/Cambodia Work Plan, submitted to USAID, FY 2011 PSI/Cambodia Work Plan, submitted to USAID, FY 2012 Report of the HIV Behavioral Surveillance (BSS), National Center for HIV/AIDS, 2007 Report on BROS Khmer Study, FHI and NCHADS, 2010 Reproductive Health/Family Planning (ppt), PSI/C Program Overview, PSI/Cambodia, July 2012 Reproductive Health Among Women at Reproductive Age in Cambodia, TRaC Population Based Survey, 2nd Round, PSI Research Division, 2009 Reproductive Health Among Women at Reproductive Age in Cambodia, Semi-TRaC Survey, 3rd Round, PSI Research Division, 2010 RH Product Assessment Survey among SQHN Providers, Top Line Findings Presentation, PSI/Cambodia, 2012 Reproductive Health Survey Among Women of Reproductive Age in Cambodia, Summary Report, PSI, 2011 SMBCI PROJECT FINAL PERFORMANCE EVALUATION 88 Review Report: Strategic Framework for Food Security and Nutrition in Cambodia 2008-2012 (SFFSN), Council for Agriculture and Rural Development (CARD), Phnom Penh, 2011 Semi-Annual Progress Report to USAID/Cambodia, Population Services International (PSI) Cambodia, submitted by Yasmin Madan, May 1, 2012 Strategic Plan for HIV/AIDS and STI Prevention and Care, MOH, Cambodia 2011-2015, NCHADS Total Market Approach to Family Planning in Cambodia, Case Study, PSI, 2011 TRaC Study Evaluating Consistent Condom Use with Male Partners among Men who have Sex with Men from Phnom Penh, Siem Reap, Battambang, Sihanouk and Banteay Meanchey, Round One, 2010, Summary Report TRaC Survey among EWs, PSI Research Division, 2011 TRaC Survey among HRUM, PSI Research Division, 2011 TRaC Survey among MSM, PSI Research Division, 2011 TRaC Surveys for MSM, EW and HRUM, PSI Research Division, 2010 TRaC Survey for Reproductive Health, PSI Research Division, 2010 TRaC Survey for Reproductive Health, PSI Research Division, 2011 USAID/Cambodia Health Program Design FY2009 – 2015, Activity Approval Document, Final USAID/Cambodia Health Program Design FY2014 – 2018, Activity Approval Document, Draft April 2012 USAID/Cambodia HIV/AIDS Portfolio Evaluation USAID/Cambodia Public Version Final Report, Public Version, July 2011 USAID/Cambodia Results Framework Paper, 2013-2017 Cambodia Development Cooperation Strategy, Draft March 29, 2012 SMBCI PROJECT FINAL PERFORMANCE EVALUATION 89 APPENDIX E PSI Research under USAID 2008-2012 SMBCI PROJECT FINAL PERFORMANCE EVALUATION 90 Nº Year Name of Research Project HIV/AIDS Research - 2008 1 2008 TRaC Study Evaluating Condom Use with Sweethearts among High Risk Urban Men from four cities in Cambodia (1st round) 2 2008 MAP Study Evaluating the Availability of HIV Prevention Product (4th round) 3 2008 Pre-testing Survey on OK condom TVC 4 2008 Pre-testing Survey defining OK condom lubricant packaging design and the instruction design 5 2008 Pre-testing Survey on concept executions to brand and packaging preference testing and memorable of OK condom 6 2008 Pre-testing Survey on ‘Trust” iBCC TVC storyboards and Re-positioned OK condom TVC storyboard (Baseline) 7 2008 Pre-testing Survey on ‘Trust” iBCC TVC storyboards and Re-positioned OK condom TVC storyboard (Follow up) Research - 2009 8 2009 TRaC Study Evaluating Condom Use with Sweethearts among High Risk Urban Men from four cities in Cambodia (2nd round) 9 2009 TRaC Study Evaluating Condom Use with Sweethearts among Female Entertainment Workers in Phnom Penh (1st round) 10 2009 MAP Study Evaluating the Availability of HIV Prevention Product (5th round) 11 2009 Qualitative Study on Sexual Health Among Entertainment Workers in Phnom Penh 12 2009 Qualitative Study on Sexual Health Among Men who have Sex with Men in three cities in Cambodia 13 2009 Quick Survey among Retail Traders to inform Trade Marketing Plan for Number One and OK Condom 14 2009 Pre-testing Survey on Re-positioned Number One condoms with new foil packaging design 15 2009 Pre-testing Survey on Subjective Norms Poster Research - 2010 16 2010 TRaC Study Evaluating Condom Use with Sweethearts among High Risk Urban Men from four cities in Cambodia (3rd round) SMBCI PROJECT FINAL PERFORMANCE EVALUATION 91 17 2010 TRaC Study Evaluating Condom Use with Sweethearts among Female Entertainment Workers in Phnom Penh (2nd round) 18 2010 TRaC Study Evaluating Consistent Condom Use with Male Partners among Men who have Sex with Men in Phnom Penh, Siem Reap, Battambang, Sihanouk and Banteay Meanchey (1st round) 19 2010 MAP Study Evaluating the Availability of HIV Prevention Product (6th round) 20 2010 Quick Survey among Trader and Consumer of New Number One condom 21 2010 Satisfaction Survey among United Health Networks 22 2010 Pre-testing Survey on New Number One Tips of Love Poster 23 2010 Pre-testing Survey on Man-Up Poster on Background and Trust 24 2010 Flash FoQus Study on Number One and OK condom Brand Equity Research - 2011 25 2011 TRaC Study Evaluating Condom Use with Sweethearts among High Risk Urban Men from four cities in Cambodia (4th round) 26 2011 TRaC Study Evaluating Condom Use with Sweethearts among Female Entertainment Workers in Phnom Penh (3rd round) 27 2011 TRaC Study Evaluating Consistent Condom Use with Male Partners among Men who have Sex with Men in Phnom Penh, Siem Reap, Battambang, Sihanouk and Banteay Meanchey (2nd round) 28 2011 MAP Study Evaluating the Availability of HIV Prevention Product (7th round) 29 2011 Market Segmentation Survey (Baseline) 30 2011 Qualitative Study on Quality of Inter-Personal Communication 31 2011 Qualitative Study among Entertainment Workers in Kampong Cham, Svay Rieng and Takeo province 32 2011 Qualitative Study among High Risk Urban Men in Phnom Penh 33 2011 Quick Survey with PSI Commercial Partner On distribution system Research - 2012 34 2012 TRaC Study Evaluating Condom Use with Sweethearts among High Risk Urban Men from four cities in Cambodia (5th round) 35 2012 TRaC Study Evaluating Condom Use with Sweethearts among Female Entertainment Workers in Phnom Penh (4th round) SMBCI PROJECT FINAL PERFORMANCE EVALUATION 92 36 2012 TRaC Study Evaluating Consistent Condom Use with Male Partners among Men who have Sex with Men in Phnom Penh, Siem Reap, Battambang, Sihanouk and Banteay Meanchey (3rd round) 37 2012 MAP Study Evaluating the Availability of HIV Prevention Product (8th round) 38 2012 Market Segmentation Survey (Follow up) 39 2012 Market Feedback Survey among New Number One Variances User (Baseline and Follow up) 40 2012 Mapping Survey among Men who have Sex with Men Outlets 41 2012 Qualitative Study among Entertainment Workers in Phnom Penh REPRODUCTIVE HEALTH Research - 2008 1 2008 MAP Study on Coverage and Quality of Coverage of PSI Birth Spacing Products in Cambodia (4th round) 2 2008 Mystery Client Survey for MDP Research - 2009 3 2009 MAP Study on Coverage and Quality of Coverage of PSI Birth Spacing Products in Cambodia (5th round) 4 2009 TRaC Study on Modern Contraceptive Methods Among Women at Reproductive Age in Kampot and Kampong Thom, Cambodia (2nd round) 5 2009 Provider Satisfaction Survey Sun Quality Health Network Program Research - 2010 6 2010 MAP Study on Coverage and Quality of Coverage of PSI Birth Spacing Products in Cambodia (6th round) 7 2010 TRaC Study on Modern Contraceptive Methods Among Women at Reproductive Age - nationwide (1st round) 8 2010 Mystery Client Survey on Women Hotline, (1st round) 9 2010 UHN Satisfaction Survey Research - 2011 10 2011 MAP Study on Coverage and Quality of Coverage of PSI Birth Spacing Products in Cambodia (7th round) SMBCI PROJECT FINAL PERFORMANCE EVALUATION 93 11 2011 TRaC Study on Modern Contraceptive Methods Among Women at Reproductive Age - nationwide (2nd round) 12 2011 Mystery Client Survey on Women Hotline, (2nd round) 13 2011 Provider Longitudinal Study among Sun Quality Health Network (1st round) (funded by WHP) 14 2011 Mystery Client Survey among MDP provider (1st round) 15 2011 Market Research on EC Research - 2012 16 2012 MAP Study on Coverage and Quality of Coverage of PSI Birth Spacing Products in Cambodia (8th round) 17 2012 TRaC Study on Modern Contraceptive Methods Among Women at Reproductive Age in Kampot and Kampong Thom, Cambodia (5th round) 18 2012 Mystery Client Survey among MDP provider (2nd round) 19 2012 Price Sensitivity Measure Study for Mid-range Oral Contraceptive Pill 20 2012 Voucher Program Evaluation Study 21 2012 Pre-testing on EC packaging design and branding CHILD SURVIVAL Research - 2009 1 2009 Pre-testing on 10-table strip dispensers (Toeuk Sovatepheap) Research - 2010 2 2010 Pre-testing on Safe Water Consumer Leaflet Research - 2011 3 2011 Current Practices and Health Seeking Behavior for Diarrhea and Pneumonia Treatment 4 2011 Story Board / Animation Pre-testing of TVC for Diarrhea Treatment Kit 5 2011 Pre-Testing on Toeuk Sovatepheap TVC Storyboard 6 2011 Pre-testing on New Packaging of Orasel KIT SMBCI PROJECT FINAL PERFORMANCE EVALUATION 94 EC : Emergency Contraception FoQus : Framework for Qualitative research in Social Marketing iBCC : Integrated Behavioral Communication Campaign MAP : Measuring Access and Performance TRaC : Tracking Result Continuously TVC : Television Commercial Research - 2012 7 2012 Pre-testing on ARI Antibiotic packaging design and branding 8 2012 TRaC Study on Oral Rehydration Salt Usage to treat Diarrhea among Caregiver of Children under 5 in Cambodia 9 2012 Market survey on Availability and Price of Supplementary product for Children Under Five and Women at Reproductive age 10 2012 Market survey on Availability and Price of Antibiotic for Children Under Five’s ARI Treatment 11 2012 Qualitative Study among Caregiver and Women at Reproductive Age SMBCI PROJECT FINAL PERFORMANCE EVALUATION 95 SMBCI PROJECT FINAL PERFORMANCE EVALUATION 96 APPENDIX F GUIDING PRINCIPLES OF THE EVALUATION • An effective social marketing program targets its products, prices, promotion, and distribution (the "four Ps" of the marketing process) to well-defined segments of the overall population. • Definition of market segments are not usefully limited to willingness and ability to pay but also include consumer characteristics such as disease status, level of risk, age, education, parity, geographic location, current source of supply, product preferences, and position on the behavior change continuum (such as previous/discontinued use, never before used, intention to use, opposition to use) among others. Knowledge of multiple characteristics of potential and actual consumers is necessary for successful social marketing. • Effective implementation of the total market approach (TMA) requires not only that the public, private/non-profit, and private/for-profit (commercial) sectors be present in the overall marketplace, but also that the work of each sector is well targeted to a specifically defined segment of the population of potential and actual consumers. • The purpose of a TMA is two-fold: (1) to ensure that all potential consumers are reached by the service delivery sector that can best meet their needs, is most acceptable to them, and most supportive of their desired healthy behaviors; and (2) to ensure that there is efficient use––that is, without unnecessary overlap in consumer segments served––of health service delivery resources and without unnecessary subsidization of service delivery––that is, to consumers that do not need it. • Sustainability considerations in social marketing programming relate to at least two levels: (1) the overall sustainability of the total amount of subsidy being delivered through the social marketing program, and (2) the sustainability of the social marketing organization (SMO) itself. In a context where donor resources are shrinking, a long-term plan should be developed to gradually withdrawing subsidies being delivered through increased cost-recovery and better targeting of market segments in need, while increasing the commercial sector share of the market (i.e., through the TMA as described above). SMOs should also prepare for the eventual reduction or withdrawal of donor funding by diversifying their sources of income, expanding their product lines to include more cost-recovery and commercially priced products, and cross￾subsidizing heavily subsidized products with those revenues. Sustainability at both of these levels may be a short, medium, or long-term consideration depending on the country context, but in all cases it should be explicitly planned for and incorporated into the SMO's strategic thinking, planning, and implementation. • In recent years, the social marketing approach has extended beyond products to health care services. These models are designed to expand the supply of and increase demand for a menu of health products and services (often family planning and reproductive health). They achieve economies of scale in training, procurement, and marketing; allow for rapid expansion to increase coverage; increase financial access by lowering prices for targeted consumers; and ensure quality and brand recognition. They comprise a group of affiliated health service providers organized together in an umbrella organization that is lead and managed by a central management structure. The structure and operation of such models can vary, and generally fall into three major categories: social franchises, not-for-profit, and commercial provider networks, as well as hybrids of these forms. The models vary greatly and span a continuum from the more commercially sustainable to the less so, depending on the level and extent of subsidy provided both to the umbrella organization and by the umbrella organization to its member providers. More sustainable models include both preventive and curative services as part of the menu covered by the branded network, charge franchise fees to providers who benefit from training and other inputs, provide financing at commercial rates to providers who require loans to make facility or other improvements, etc. Less sustainable models focus exclusively on preventive care services and provide training, renovations to facilities, and drugs/supplies for free or at cost. SMBCI PROJECT FINAL PERFORMANCE EVALUATION 97 SMBCI PROJECT FINAL PERFORMANCE EVALUATION 98 APPENDIX G Observed Commercial Sector Condom Prices Unit price (in US$) of commercial condom brands* Condom Brand Pharmacy #1 (Phnom Penh) Pharmacy #2 (Phnom Penh) Pharmacy #3 (Phnom Penh) Pharmacy #4 (Battambang) OK 0.04 0.08 0.04 0.04 Number One 0.08 0.08 0.07 Okamoto (various) 0.25-0.68 Okamoto (lubricant) 0.47 Okamoto Dot 0.83 Okamoto 003 1.17 Okamoto Alo 1.50 Okamoto Skinless 0.16 Okamoto Orange box 0.20 Okamoto Blue box 0.20 KamaSutra 0.43 0.08 Romantic 0.43 Trojan 2.10 Strast 0.41 * Observed by the evaluation team in a limited number of pharmacy and shop visits SMBCI PROJECT FINAL PERFORMANCE EVALUATION 99