Final Evaluation Social Marketing Strategies for Maternal and Child Health in the States of Uttar Pradesh, Uttaranchal & Jharkhand, India October, 2002 – May, 2005 Implemented by Population Services International HFP-A-00-02-00042-00 May 30, 2005 Final Evaluation Report, Social Marketing Strategies for MCH Page i ____________________________________________________________________________________________________________________________________________________________________________________________________ Authors of this Report External John Davies, International Health Consultant, Team Leader Internal Carmen Chan, Programme Director, Family Health Program, PSI/India and Vibha, Project Director, Social Marketing Strategies for MCH Project, PSI/India. Final Evaluation Report, Social Marketing Strategies for MCH Page ii ____________________________________________________________________________________________________________________________________________________________________________________________________ Contents Authors of this Report ........................................................................................................i Contents ..............................................................................................................................ii Abbreviations and Acronyms ..........................................................................................iv A. Executive Summary..................................................................................................5 B. Assessment of Results and Impact of the Program..................................................7 1. Results: Summary Charts.....................................................................................7 2. Results: Technical Approach.............................................................................10 3. Results: Cross-cutting approaches.....................................................................18 C. Program Management ...........................................................................................23 1. Planning .............................................................................................................23 2. Staff Training .....................................................................................................24 3. Supervision of Program Staff.............................................................................24 4. Human Resources and Staff Management.........................................................25 5. Financial Management .......................................................................................25 6. Logistics.............................................................................................................26 7. Information Management...................................................................................26 8. Technical and Administrative Support ..............................................................27 9. Management Lessons Learned...........................................................................28 D. Other Issues Identified by the Team.......................................................................28 1. Developing and expanding the Saadhan Network.............................................28 2. Live infotainment...............................................................................................29 E. Conclusions and Recommendations ......................................................................30 1. Achievement of Project objectives. ...................................................................30 2. Major achievements, best practices and lessons learned. ..................................30 3. A major constraint..............................................................................................30 4. Managerial sustainability. ..................................................................................31 5. Recommendations..............................................................................................31 6. Potential for Scale up .........................................................................................32 F. Results Highlight....................................................................................................33 Annex A. Evaluation Team Members and Titles.........................................................34 Annex B. Evaluation Assessment Methodology ...........................................................35 Annex C: Persons Interviewed and Contacted............................................................36 Annex D: CD....................................................................................................................38 Annex E-1: Training & Capacity Building Approach, PSI/India ..............................39 Annex E-2: PSI’s Approach to Behavior Change........................................................40 Final Evaluation Report, Social Marketing Strategies for MCH Page iii ____________________________________________________________________________________________________________________________________________________________________________________________________ Annex E-3: PSI’s Social Marketing Research Process................................................42 Annex E-4: Saadhan MCH Network and the Marketing Cycle .................................43 Annex E-5: ASHA and the National Rural Health Mission........................................45 Annex E-6: Rapid CATCH Comparison Table ...........................................................48 Annex F-1: Project Data Sheet Form ...........................................................................51 Annex F-2: Project Data Sheet Form – updated version............................................55 Final Evaluation Report, Social Marketing Strategies for MCH Page iv ____________________________________________________________________________________________________________________________________________________________________________________________________ Abbreviations and Acronyms ANC Antenatal Care ANM Auxiliary Nurse Midwife ASHA Accredited Social Health Activists BCC Behavior Change Communication BCG Bacillus Calamatte Gurine CDK Clean Delivery Kit DPT Diphtheria Pertussis Tetanus GOI Government of India Hb Hemoglobin HIHT Himalayan Institute Hospital Trust IFA Iron-Folic Acid IMCI Integrated Management of Childhood Illness ISM&HPsIndian Systems of Medicine and Homeopathic Practitioners MCH Maternal and Child Health NRHM National Rural Health Mission OCP Oral Contraceptive pills OPV Oral Polio vaccine ORS Oral Re-hydration Salts PSI Population Services International RCH Reproductive and Child Health RDI Rural Development Institute RTI Reproductive Tract Infections SCOVA Standing Committee of Voluntary Agencies STI Sexually Transmitted Infection SMS Social Marketing Strategies for MCH Project SWS Safe Water System TBA Traditional Birth Attendant TNA Training Needs Assessment TOT Training of Trainers UP Uttar Pradesh UR Uttaranchal USAID United States Agency for International Development WRA Women of Reproductive Age Final Evaluation Report, Social Marketing Strategies for MCH Page 5 ____________________________________________________________________________________________________________________________________________________________________________________________________ A. Executive Summary The goal of the Social Marketing Strategies for Maternal and Child Health Project is to reduce infant and child mortality and morbidity in three Indian States. The general strategy for the Project is to test the applicability of social marketing for Maternal and Child Health with a view to using lessons learned as a basis for scaling up the activities, if warranted. The project has two distinct components. Component One is social marketing of Maternal & Child Health products; its broad strategy is to test methods of increasing consumer uptake of condoms, oral contraceptive pills, iron-folic acid tablets, oral rehydration packets, and clean delivery kits. Component Two is development of the Saadhan Network of health providers in selected low-income (slum) areas of Uttaranchal; its broad strategy is to improve MCH behaviors through a community network of trained private health providers and female Community workers. Results In Component One, all sales targets at the purpose level of the Component One Logframe were met or exceeded by March, 2005. Sales of Neotral ORS sachets were especially noteworthy. Sales of Vitalet￾Preg iron-folic acid tables were much higher than targeted. Sales of Safewat water disinfectant were particularly heartening because Safewat is a new product in a new category: sales were almost double expectations; this popularity was supported by the Team’s observations during street theatre and discussions in Uttaranchal, as well as by the India Clen survey. All targets at the Logframe Objective level for Component One were also met or exceeded. On the supply side, access to Saadhan brand products was greatly increased by the opening of more than 45,000 outlets. On the demand side, advertising of products was increased through a variety of advertising and support media including billboards, shop signs, wall paintings, radio campaigns, health practitioners meetings, orientations for traditional birth attendants, NGOs, public sector agencies and the well-attended infotainment activities such as sidewalk stalls, and special events such as street theatre. Financial sustainability of this social marketing activity was high because the sales revenues were more than the cost of procuring the products. In Component Two, all Logframe purpose level behaviors improved, except for results showing that parents gave less fluid during their child’s previous diarrheal episode (this problematic issue is now under review). Two achievements were particularly noteworthy: use of iron-folic acid tablets and treatment of drinking water. The achievements under Component Two represent progress in harnessing the energies of the Saadhan Network of private health providers including ISMPs, female Community Workers, pharmacists and other retailers. Many of these achievements are impressive, showing the motivation of community health providers to support MCH activities. Some outstanding achievements include counseling by 97% of the Saadhan members about the need for iron-folic acid supplements during pregnancy, 70% of members counseling about diarrhea management, more than 35,000 counseling events by members, 3000 outlets selling MCH products, and the involvement of state government officials. These achievements are particularly impressive given the short period of fieldwork – 16 months. Managerial sustainability was strengthened as PSI managers and field workers gained skills from training and field experience in social marketing techniques. Institutional sustainability is progressing as some social marketing activities will continue after donor support is completed on May 31, 2005 – although without the paid community-level support such as Community Workers and infotainment. Final Evaluation Report, Social Marketing Strategies for MCH Page 6 ____________________________________________________________________________________________________________________________________________________________________________________________________ PSI gained experience in working in integrated MCH activities, including dissemination of best practices including reports on the PSI website Priority Conclusions and Lessons Learned Component One: Social Marketing of branded products 1. PSI’s social marketing system, which coordinates supply and demand, is an effective method for increasing uptake of new MCH products in a short period. 2. Safewat, a new product in a new category, was accepted quickly, probably because it used a variety of community media that appealed to low-income families. 3. Well planned, executed and evaluated Behavior Change Communication techniques exemplified by the Promotion methods used to create demand for PSI products are a major factor in the effectiveness of field activities. 4. PSI’s rigorous approach to integrating three high-quality technical systems – Training, Behavior Change Communication and Research – is a major key to development of effective fieldwork. Component Two: Social Marketing of the Saadhan MCH Network 1. The Saadhan Network represents a useful system for coordinating four factors – Behavior Change Communication, referrals, products and franchising – that can quickly lead to improved MCH in urban slums. 2. Social Marketing is an effective approach to community-based improvement of Mother & Child Health among low-income urban families 3. PSI/India’ innovative community-based Behavior Change Communication methods, such as street theatre, stalls, games and product VAT have combined to produce a major influence on the effectiveness of field activities. 4. Well-trained, paid female Community Workers provide effective counseling and demonstration of MCH products in their communities. 5. Female Community Workers in urban slums develop strong feelings of self-worth, while increasing family income. 6. Well-trained ISM&HPs will provide useful counseling and sell or promote branded products to their patients. 7. Birth spacing concepts and products are popular when included in a “basket” of other Mother & Child Health interventions. 8. PSI’s VAT (Visibility, Accessibility & Taste) behavior change system is an effective method of promoting MCH products. 9. Reducing the risk of diarrhea appears to be accomplished by providing accurate knowledge, handwashing and ORS as needed for diarrhea. 10. Franchising and training ISM&HPs is an effective way of providing quality MCH information and services in low-income setting 11. Marketing techniques can be effectively applied to MCH products and to community-based counseling services. * * * * * Final Evaluation Report, Social Marketing Strategies for MCH Page 7 ____________________________________________________________________________________________________________________________________________________________________________________________________ B. Assessment of Results and Impact of the Program 1. Results: Summary Charts Component One Table 1. Component One: Sales Achievements at Logframe Purpose Level, March, 2005 Social Marketing Strategies for MCH Products, India Products End-of Project Target Total Sales to March, 2005 Achieved (%) Masti Condoms 77,000,000 pieces 80,435,364 104 Pearl Oral Contraceptive Pills 2,332,000 cycles 2,330,653 100 Neotral ORS 738,000 sachets 1,363,246 184 Vitalet-Preg Iron-Folic Acid 2,700,000 tablets 6,158,654 228 Clean Delivery Kits 20,000 kits 29,650 148 Safewat water disinfectant (Uttaranchal only) 10,200 bottles 19,200 188 Source: PSI Field Force Management System Table 2. Component One: Sales Achievements at Logframe Objective Level, March, 2005 Social Marketing Strategies for MCH Products, India. Results Hierarchy Indicators Achievement Objective 1: Increased access to MCH products 1. Complete basket of MCH products launched in UP, UR and JH 2. Saadhan products available in 80% of Districts 3. 6,000 new outlets opened in program States · Yes · Yes · 45,522 new outlets opened Objective 2: Increased awareness of Saadhan products among low￾income families 1. Number and type of communications activities for Saadhan MCH products Jharkhand: · 83 Billboards, · 470 shop signs, · 236 wall paintings. UP & UR: · 1 radio campaign, · 166 health practitioners’ meetings, · 6 TBA orientations Objective 3: Improved awareness and collaboration between public, private and NGOs regarding integrated MCH 1. Number of NGOs working with PSI to promote Newborn CDKs 2. Number of health providers promoting Vitalet-Preg 3. Collaboration for promotion of Safewat SWS in urban slums 4. Lessons shared with White Ribbon Alliance for Safe Motherhood · 6 · 885 · 15 Community depots established. · Yes. Participation in 3 meetings. Final Evaluation Report, Social Marketing Strategies for MCH Page 8 ____________________________________________________________________________________________________________________________________________________________________________________________________ Table 2. Component One: Sales Achievements at Logframe Objective Level, March, 2005 Social Marketing Strategies for MCH Products, India. Results Hierarchy Indicators Achievement Objective 4: PSI/India’s capacity to implement a more sustainable integrated family health program strengthened 1. State Management Unit set up in UP. 2. SMS Project Director participates as member of the Executive Committee 3. Training capacity strengthened 4. Capacity to market new MCH products strengthened 5. Revenues from new MCH products cover 100% of cost of goods sold. · Yes · Yes. Participation in 6 out of 7 held. · Yes. · At PSI/Delhi office & project offices · Yes Objective 5: PSI’s capacity in integrated MCH expanded 1. Increased dissemination of best practices: distribution of IFAs, CDKs, and SWS. 2. Increased related documentation available on PSI Intranet. · Yes. In all 3 states 1. Yes. Training and other materials. Sources: PSI Field Force Management System, PSI Communication Activity Monitoring Reports, PSI Product Cost Analysis and Pricing Structure. Component Two Table 3. Component Two: Log Frame Purpose Level Achievements, Pilot Saadhan Network Of Private Health Providers in Low-Income Urban Areas of Dehradun and Hardwar, Uttaranchal State Results Hierarchy Targets Achievement, March 31, 2005 Purpose: Increase in healthy practices related to maternal and child health 1. Increase in spacing methods from 38% to 44% among low-income parents 2. Increased use of CDK from 18% to 30% during home deliveries 3. Increase TT coverage from 84% to 90% 4. Increase IFA coverage from 61% to 70% 5. Increased use of ORT from 15% to 25% during child’s previous diarrhea episode 6. Decrease from 30% to 20% of parents who give less fluid during their child’s previous diarrheal episode 7. Increased use of treatment to improve drinking water quality from 5% to 20% 8. Increased proper storage of drinking water from 26% to 35% · 41.3% · 28.3% · 88.5% · 72.2% · 56% · 55.9% · 38% · 31.1% Final Evaluation Report, Social Marketing Strategies for MCH Page 9 ____________________________________________________________________________________________________________________________________________________________________________________________________ Table 4. Component Two: Log Frame Objective Level Achievements, Pilot Saadhan Network Of Private Health Providers in Low-Income Urban Areas Of Dehradun and Hardwar, Uttaranchal State Results Hierarchy Indicators Achievement Objective 1: Improve access to basic MCH￾related information 1. % of network members who provided child spacing information during a post￾partum check-up 2. % of network members who provided birth preparation and complications readiness information during ANC 3. % of network members who provided counseling on anemia and IFA during antenatal check-ups 4. % network members who counseled on IFA, vitamin A and breastfeeding during post-partum check up. 5. % of network members who provide diarrhea prevention and management information during check-up for children suffering from diarrhea 6. % network providers who asked about and counseled on immunization status of children 7. % of clients satisfied with the quality of information received 8. Number and type of communication activities promoting MCH under the Saadhan logo 9. Number and type of communications activities promoting the Saadhan network of providers, · Not applicable, as reported to USAID · 85% · 55% · 97% Not applicable, as reported to USAID · 70% · 60% · 92% Four types : 780 women’s meetings, six rounds of IPC; 5626 local media; print material Eight types, as described later. · Billboards, tin plates, signage newspapers inserts, vouchers. Objective 2: Improved access to quality MCH services 1. Number of active network members who have completed the full MCH training course and are operating as per the terms of reference 2. Number of families registered with their practitioners. 3. Number of clients/consultations for MCH-related causes 4. % of population using Saadhan network services in catchment areas · 98 completed full training courses; 70 now in place. · Not applicable: registration not common. · 35,754 · 4.2%* Final Evaluation Report, Social Marketing Strategies for MCH Page 10 ____________________________________________________________________________________________________________________________________________________________________________________________________ Table 4. Component Two: Log Frame Objective Level Achievements, Pilot Saadhan Network Of Private Health Providers in Low-Income Urban Areas Of Dehradun and Hardwar, Uttaranchal State Results Hierarchy Indicators Achievement 5. % of clients satisfied with the quality of services rendered 6. % of effective referrals · 86% · 6% Objective 3: Improved access to essential MCH products 1. Sales of social marketing CDKs, ORS, IFA, OCPs, condoms and Safewat 2. Number of outlets with social marketing MCH products in network areas 2,780,476 condoms ; 104,978 OCP ; 23,004 ORS; 112,000 IFA; 3300 CDK & 19,200 safewat · 3,323 Objective 4: Improved enabling environment for MCH promotion in project areas 1. UR State government representatives participate in network supporting activities 2. Lessons learned shared with UR State government and WRAI 3. Project Advisory Group (PAG) meetings held 4. SMS Project Director participation in the UR Urban RCH Society · Yes · Yes · PAG not formed but quarterly meetings held individually with key officials for updates · Participated in NGO related meeting of the Society convened by Chief Secretary Source: Mode Services (research agency), TNS India (research agency), PSI Saadhan network monitoring reports *Public sector gets only about 22% of the entire clientele. As this was spontaneous response, the demarcation between Saadhan doctor, any ISMP, MBBS, child spec and RMP is quite blurred as very few respondents generally know about the degrees that the doctors hold. 2. Results: Technical Approach a) Brief Overview of the Project The goal of the Social Marketing Strategies for Maternal and Child Health Project is to reduce infant and child (under 5) mortality and morbidity in the Uttar Pradesh, Uttaranchal and Jharkhand. These three states are USAID focus states because the people are among the poorest in northern India. The duration of the Project is relatively short: 30 months, including 16 months of Saadhan network operation ending May 31, 2005. The general strategy for the Project is to test the applicability of social marketing for Maternal and Child Health, with a view to using lessons learned as a basis for scaling up the activities, if warranted. This involved Social Marketing of PSI products throughout Uttar Pradesh and in urban areas of the other two states, plus a pilot test of the Saadhan community-based concept in urban slums of Uttaranchal state. Final Evaluation Report, Social Marketing Strategies for MCH Page 11 ____________________________________________________________________________________________________________________________________________________________________________________________________ Table 5 summarizes some dimensions of the Project including the interventions, products, geographic areas, total populations, target population profiles and target population sizes. Table 5. Summary of Interventions, Products, Areas, and Populations, SMS Project Intervention and Level of Effort Product Geographic Area Total Pop (Est.) Target Pop. Profile Target Pop. (Est.) CONDOMS 3 States: urban mainly 42,667,300 Men of reproductive age (23%) CHILD SPACING 9,823,479 (25% LOE) OCPs 3 States: urban mainly 42,667,300 Non sterilized WRA (15%) 6,400,095 MATERNAL & NEW BORN CARE (20% LOE) CDKs Selected Districts of UR and Jharkhand 900,000 Pregnant women (2.7%) 24,300 MATERNAL NUTRITION (20% LOE) IFA 3 States: urban mainly 42,667,300 Pregnant women (2.7%) 1,152,017 ORS 3 States: urban mainly 42,667,300 Children under five (13%) CONTROL 5,546,749 OF DIARRHEEAL DISEASES (20% LOE) SWS disinfectant Selected slums in Dehradun and Hardwar 215,000 Slum dwellers (all household members) 180,000 BREASTFEEDING(5% LOE) IMMUNIZATION (5% LOE) NUTRITION (5% LOE) Selected slums in Dehradun and Hardwar 215,000 Pregnant women and children < 5 46,260 Source: Detailed Implementation Plan, SMS Project The project has two distinct components. Component One is social marketing of Maternal & Child Health products. The broad strategy for Component One is to test methods of increasing consumer uptake of condoms, oral contraceptive pills, iron-folic acid tablets, oral rehydration packets, and clean delivery kits. Component Two is development of the Saadhan Network of health providers in selected low-income (slum) areas of Uttaranchal. Component Two seeks to use the Saadhan Network to improve knowledge and use of four maternal and child health issues. The broad strategy for Component Two is to test the effectiveness of community-based social marketing techniques for increasing the knowledge and use of affordable MCH products in low-income communities. The Saadhan Network is a mechanism for training private health providers, retailers and female Community Workers to provide detailed information about the MCH issues, and to promote PSI’s brands of MCH products. The Saadhan health providers are ISMPs (Indian Systems of Medicine and Homeopathy Practitioners). PSI selected ISMPs because they are registered with the government and because of their proximity, affordability and credibility among slum dwellers. Saadhan includes 130 female Community Workers, most of them selected from existing local committees formed by District Urban Development Agency (DUDA). They received an initial five days’ training by PSI on inter-personal communication for maternal and child health issues. Each Community Worker covers approximately 260 households every month and receives an Final Evaluation Report, Social Marketing Strategies for MCH Page 12 ____________________________________________________________________________________________________________________________________________________________________________________________________ honorarium of Rs 1000 per month. The Saadhan network is pilot testing the following four interventions: 1. Birth spacing · Increased use of spacing as a means to mother and child health 2. Maternal and newborn care · Increased use of IFA, CDK in case of home deliveries & ante-natal check-ups · Improved health seeking behavior · Counseling by franchise health practitioners, 3. Control of diarrheal diseases · Increased use of ORS, home fluids and water treatment practice · Intensive community based approach & training of franchise providers. · Breast feeding 4. Child Nutrition and immunization · Improved complete immunization · Retaining and using immunization cards. b) Progress by intervention area i. Comparison of baseline and final evaluation surveys Component One As shown in Table 1, all sales targets at the purpose level of the Component One Logframe were met or exceeded by March, 2005 (Pearl Oral Contraceptives actual achievement was 99.9%). Sales of Neotral ORS sachets were noteworthy, reflecting the high levels of diarrhea among children and adults reported during interviews and meetings throughout the project area. Sales of Vitalet-Preg iron-folic acid tables were much higher than targeted, reflecting the results of interviews and a mini-focus group with project Community Workers in Uttaranchal, who remarked on the high popularity of Vita-Preg among mothers. Sales of Safewat water disinfectant were particularly heartening. Safewat is a new product in a new category: sales were almost double expectations; this popularity was supported by the Team’s observations during street theatre and discussions in Uttaranchal, as well as by the India Clen survey that showed that 22% of the respondents had used Safewat. Recommendation # 1. The PSI model for Social Marketing of MCH products, including Safewat, is successful and should be continued and expanded. Justification: These results indicate that PSI’s model can produce impressive sales results in a relatively short time and may be cost-effective because users pay for some of the costs of the activities. As shown in Table 2, all targets at the Logframe Objective level for Component One were also met or exceeded. On the supply side, access to Saadhan brand products was greatly increased by the opening of more than 45,000 new outlets (Objective 1). On the demand side, advertising of Saadhan products was increased through a variety of advertising and support media including billboards, shop signs, wall paintings, radio campaigns, health practitioners meetings, orientations for traditional birth attendants, NGOs, public sector agencies and the very well-attended infotainment activities such as sidewalk stalls, and special events such as street theatre (Objectives 2,3,4). Financial sustainability of this social marketing activity exceeded 100% as measured by comparing retail prices with cost of goods sold (Objective 4); Managerial sustainability was strengthened as PSI managers and field workers gained Final Evaluation Report, Social Marketing Strategies for MCH Page 13 ____________________________________________________________________________________________________________________________________________________________________________________________________ from training and field experience in social marketing techniques (Objective 5). Progress toward institutional sustainability was also evident as measured by the continuation ability of some social marketing activities to continue after donor support is completed on May 31, 2005 – although without the powerful, paid community-level support such as Community Workers and infotainment – while PSI gained experience in working in integrated MCH activities, including dissemination of best practices including reports on the PSI website (Objective 5). Component Two This component is a pilot activity operating in urban slums of Uttaranchal state. Unlike Component One, which is aimed at increasing use of branded products, Component Two uses Behavior Change Communication to increase healthy practices without necessarily referring to PSI products. As shown in Table 3, all Logframe purpose level behaviors improved, except for one: there was an increase in the percentage of parents who gave less fluid during their child’s previous diarrheal episode (this issue is discussed below). Although most of these achievements did not reach targeted levels by March, 2005, some targets may be met by end-of-project date, May 31. Two achievements exceeded their targets – iron-folic acid tablets and drinking water quality. Government officials in Uttaranchal stated that the program’s communication activities were responsible for additional demand for MCH services at government facilities. This suggests that the observed changes in the objectives probably would not have occurred in the absence of the SMS program. PSI’s analysis of the program’s pre- and post- survey results provides some useful information concerning attribution of effects. For example, when Endline survey respondents were divided into two groups – those who reported having been exposed to the program’s communication activities, and those who reported not having been exposed – a multivariate analysis that controlled for factors such as religion, caste, income, education and occupation, was used to show any statistically significant differences between the two groups that could be attributed to program interventions. The analysis showed that the significant increases in current use of any birth spacing method, use of tetanus toxoid, use of IFA tablets, and treatment of drinking water were probably due to the program’s interventions because the levels of significance were quite high: p. value less than .05 or less than .01. The difference between the two groups concerning increase in ORS had a low significance: p. value of .10. And the increase in the proportion of parents giving less fluid to children during the previous diarrheal episode showed no significant difference between the two groups: therefore the behavior change should not be attributed to the program intervention. Although there is no evidence to suggest that the increase in the proportion of parents who gave fewer fluids to their children who had diarrhea could be explained by program activities, the problem is troubling because many children are being deprived of necessary fluids. PSI/India is examining the problem, including evidence from other studies in India that parents believe diarrhea is a casual problem that needs no treatment; that ORS cures diarrhea, that there is little awareness of the dangers of dehydration or that water is a source of disease, while diarrhea is a sign that the child has too much fluid and therefore should be given less breastmilk and other fluids during diarrhea. Recommendation # 2. As part of its examination of the problem PSI should commission a brief, in￾depth observational study of family behaviors to learn about common beliefs and practices surrounding the etiology, prevention and treatment of diarrhea and dehydration among young children. Justification: Such a study – perhaps using a medical anthropological perspective – should help to uncover the cultural roots that lead to the problematic behavior. Analysis of results could then be used to form action concepts that can guide Behavior Change Communication aimed at explaining to parents why the child should have more fluids – not less – during diarrhea in order to prevent dehydration and Final Evaluation Report, Social Marketing Strategies for MCH Page 14 ____________________________________________________________________________________________________________________________________________________________________________________________________ its dangers. This in-depth observational study, rather than a survey, should be particularly useful for identifying the real behaviors rather than the reported behaviors, because in South Asia, survey respondents are known to have a courtesy bias and a cultural bias aimed at satisfying the interviewer rather than describing the true situation. Recommendation # 3. To the extent possible, PSI’s surveys should enquire into the use of specific brands when studying changes in knowledge and behavior. Justification: brand information can be very helpful in attributing effects to correct sources, particularly when several brands are in the market, such as multiple brands of condoms, iron-folate tablets and ORS. Brand information is less important for unique products such as Safewat water disinfectant. Recommendation # 4. PSI should continue to monitor the work of its research contractors, including making visits during interviewing to ensure that subcontractors consistently supervise their interviewers and back-check respondents. Justification: It is generally accepted that the quality of Indian survey research is above the regional average. Furthermore, a careful review of the methodological approach of PSI’s subcontractor for baseline measurements (SRI-IRMB) suggests a sophisticated technical approach to the task, including clear descriptions of each of the links in the survey research chain such as instrument design, instrument pretesting, sample design, selection of interviewers who have the necessary language skills, training of interviewers, supervision of interviewers, collection of data, coding of responses to open-ended questions, back-checking a sample of respondents, computer data entry, data cleaning and data analysis according to a tabulation plan approved by the client. Furthermore, the research agency provided useful descriptions of each indicator and denominator population, and provided a well-written report of the findings. In spite of these policies and professional methods, it is known that quality of fieldwork can decline if interviewers concentration or motivation wanes, if interviewer teams are not rotated often, if supervision is less than adequate or if research executives forget to adequately back-check a randomly selected sample of respondents. Table 4 shows the achievements at the Logframe Objectives level for Component Two. The achievements represent progress in using the energies of the Saadhan Network of private health providers including ISMPs, other medical practitioners, female Community Workers, pharmacists and other retailers. Many of these achievements are impressive, showing the motivation of community health providers to support MCH activities. Some outstanding achievements include counseling by 97% of the Saadhan members about the need for iron-folic acid supplements during pregnancy, 70% of members counseling about diarrhea management, more than 35,000 counseling events by Saadhan members, more than 3000 outlets selling MCH products, and the involvement of state government officials. These achievements are impressive given the short period of field work – 16 months. Other sources of information. PSI also commissioned Rapid CATCH studies aimed at measuring changes in indicators for Maternal Care, Newborn Care, Childhood Diarrhea and Safe Water Systems. Baseline and endline information for the Rapid CATCH study are summarized in Annex E-6. ii. Factors Affecting Achievement · PSI’s Social Marketing System. PSI/India’s Social Marketing system was a major factor in the achievements described above. The system has at least three strengths: (1) adherence to proven commercial marketing principles such as development of supply of appropriately priced products and services that is coordinated with development of demand through behavior change communication and promotion of the PSI brands, (2) in-house technical management for a triad of key support activities – Behavior Change Communication, training and research, and (3) more than thirty years of Social Marketing experience in South Asia. Final Evaluation Report, Social Marketing Strategies for MCH Page 15 ____________________________________________________________________________________________________________________________________________________________________________________________________ · A Concept-driven System. It is generally accepted that concept-driven activities are more successful than those that are not driven by a concept. PSI/India’s concept of the Saadhan (“The Way”) Network helped to ensure that the focus of all energies was on the Mother and Child pictured in the center of the graphic in Annex E-4. The various elements shown in the Saadhan graphic fit nicely with the second graphic in Annex E-4 – the Marketing Cycle. The central position of research in the Marketing Cycle is important because it indicates that evidence generated by research is of great importance to development of a successful social marketing program. iii. Contributing factors to objectives not fully achieved · Setting behavior change targets. In Component Two, five out of eight behavior change targets were not met, although the changes were very substantial. A contributing factor was setting objectives that were overly optimistic given the short period of Saadhan network operation – only 16 months. Recommendation # 5. PSI should consider avoiding setting numerical targets for first efforts. Instead, PSI could set targets only after measuring outputs in test markets for a few months before going to scale. Justification: Creating targets for new types of behavior change in culturally different areas is not very useful because there is no evidential basis for the targets. · Tests of referrals. On the one hand, the two tests – both of which used coupons aimed at increasing demand for counseling by ISMPs about MCH issues – did stimulate demand, and the Saadhan ISMPs reported that discounted prices helped to build client relationships. The tests may have been more successful if fielded for longer periods. Recommendation # 6. PSI should continue to test coupon offers. Justification: PSI coupon offers have proved very successful in other countries, for example in Sri Lanka for creating direct mail links between the project and couples interested in using oral contraceptives, and for creating demand for a free brochure that described various contraceptive methods. Furthermore, there is much evidence among multinational marketing firms that discount coupons are an effective marketing tool. · Post-partum check-ups. As indicated in Table 4, Objective 1, the Project could not obtain data on birth spacing information provided by Saadhan clinics during post-partum check-up, as clients tend to go to the health center where the delivery was conducted for post-partum check-ups. This situation was reported to USAID in the program’s 2003 and 2004 annual reports. · Record-keeping among ISMPs. In Objective 2, the project could not obtain family registration information. The information was unavailable because ISMPs do not commonly undertake such record-keeping. iv. Main Successes and Lessons Learned Component One. Social Marketing of branded products PSI’s social marketing system, which coordinates supply and demand, is an effective method for increasing uptake of new products in a short period. Safewat, a new product in a new category, was accepted quickly, probably because it used a variety of media that appealed to low-income families. Well planned, executed and evaluated Behavior Change Communication techniques exemplified by the Promotion methods used to create demand for PSI products are a major factor in the effectiveness of field activities. Final Evaluation Report, Social Marketing Strategies for MCH Page 16 ____________________________________________________________________________________________________________________________________________________________________________________________________ PSI’s rigorous approach to integrating three high-quality technical systems – Training, Behavior Change Communication and Research – is a major key to development of effective fieldwork. Component Two. Social Marketing of the Saadhan MCH Network The Saadhan Network represents a useful system for coordinating four factors – Behavior Change Communication, referrals, products and franchising – that can quickly lead to improved MCH in urban slums. Social Marketing is an effective approach to community-based improvement of Mother & Child Health among low-income urban families PSI/India’ innovative community-based Behavior Change Communication methods, such as street theatre, stalls, games and product VAT have combined to produce a major influence on the effectiveness of field activities. Well-trained, paid female Community Workers provide effective counseling and demonstration of MCH products in their communities. Female Community Workers in urban slums develop strong feelings of self-worth, while increasing family income Well-trained ISM&HPs will provide useful counseling and sell or promote branded products to their patients. Birth spacing concepts and products are popular when included in a “basket” of other Mother & Child Health interventions PSI’s VAT (Visibility, Accessibility & Taste) behavior change system is an effective method of promoting MCH products. Reducing the risk of diarrhea, appears to be accomplished by providing accurate knowledge, handwashing and ORS as needed for diarrhea. Franchising and training ISM&HPs is an effective way of providing quality MCH information and services in low-income setting Marketing techniques can be effectively applied to MCH products and to community￾based counseling services. v. Special outcomes and unexpected successes or constraints · Overall achievements: Although the duration of the Saadhan network operation was unusually brief, PSI accomplished surprisingly large amounts of testing, training, a wide variety of Behavior Change Communication, monitoring, evaluation, staff development and creation of useful relationships with state government officials. The fact that many knowledge and behavior indicators showed significant gains in such a short period is testimony to the quality and motivation of PSI/India’s staff and to the PSI model. One example is the Safewat success. Being a new category of product, it required much more than mere distribution to retailers and promotion: PSI developed a community-based campaign, including interpersonal communication about the nature and value of the product, including demonstrations and instructions for correct use. This successful introduction was accomplished in only eight months; PSI has passed lessons learned about Safewat to its other programs. · Quality of research. A review of PSI/India’s research policies and of its subcontractors methodologies indicate a rigorous and professional approach to market research, for example, a policy on selection of subcontractors, and cross-checking results of sales recorded by the PSI distribution system and results of independent retail audits by AC Nielsen, a highly-regarded international research organization. Final Evaluation Report, Social Marketing Strategies for MCH Page 17 ____________________________________________________________________________________________________________________________________________________________________________________________________ · Continuity rates for PSI products. Compliance rates for pharmaceutical products requiring daily use over several months are generally disappointing, specially among low-income or poorly educated people. However, before-and-after studies for this Project show that the proportion of women who consumed PSI’s brand of iron-folic acid tablets for up to three months increased significantly whereas the proportion of women who consumed for up to two months has decreased significantly. This is an indication of success for the IFA campaign that urged a minimum of three months of consumption. A factor in this beneficial outcome is probably the support given to users by trained community-level providers, such as the female Community Workers, ISM&HPs and government doctors. vi. Applying lessons learned to future activities The SMS project is not scheduled to continue beyond May 31, 2005. This is regrettable because both Components showed great promise and therefore deserve to be continued and expanded so that additional low-income families can benefit. Selected activities under Component One will continue, supported by revenues from product sales. The lessons learned in this program are being compiled for sharing within PSI and with interested external parties. Training manuals and promotional materials developed under the program are posted on the PSI intranet. vii. Potential for scale-up There is solid potential for scaling up for the reasons described above. Recommendation #7. Components One and Two should be taken to scale throughout urban and rural areas of the three states. PSI should add other appropriate products and services to the Saadhan network, for the ISM&HPs and referral doctors, such as IUDs, injectable contraceptives, RTI/STI interventions, Emergency Contraceptives, DOTS for TB medication, zinc preparations and HIV counseling. A special effort is needed to develop community-based referral systems for emergencies such as obstetric complications and infant distress; this important task will require a public-private partnership involving the use of the government’s clinical resources; a detailed description and recommendation is offered in Section D, below. Justification: Scaling up, including new products and services will benefit more low-income families who need affordable MCH products and services. c) New tools or approaches developed or used Component Two is a pilot activity aimed at measuring the success of new approaches to improving MCH among low-income communities with a view to scaling them up if warranted. Examples of PSI/India’s new approaches and tools are: · Birth spacing. PSI/India changed its approach from promotion of a two-year birth space to three years. This new campaign was placed in radio programs in Uttar Pradesh and Uttaranchal. It is too early to know about the impact of the new message. · Clinic client counts and follow-up. PSI/India instituted two new tools in support of its franchising of ISM&HPs. Researchers undertook client counts at ISM&HPs’ clinics before training which was later monitored monthly by record slips filled by ISM&HPs to ascertain changes in client volumes. Trainers undertook structured follow-up observations and interviews of the trained ISM&HPs aimed at providing support and identifying additional needs of the ISM&HPs. Final Evaluation Report, Social Marketing Strategies for MCH Page 18 ____________________________________________________________________________________________________________________________________________________________________________________________________ · Recognition system for ISM&HPs. PSI/India distributes newsletter to its franchise ISM&HPs with photos and names of top performers and PSI/India placed names and photos in newspapers of all those who were initially trained and branded under the Saadhan network. · New technical system in support of Social Marketing. PSI has developed a coordinated technical system that links three important technical support activities: research, training and Behavior Change Communication. Each of the three following activities provides important technical assistance to PSI’s implementing teams: o Training of health providers. PSI worked with IntraHealth to develop a state-of-the-art system for training medical practitioners and female Community Workers as part of the Saadhan initiative. The system is based on a Performance Improvement Approach that begins with a Performance Needs Assessment. Results are used to design the training curriculum along with teaching aids, handouts, and activities such as role-playing sessions. Then monitoring tools are designed. The system also includes a feedback loop to ensure that evidence-based improvements can be made to the system. A summary of PSI’s training approach is provided in Annex E-1. o Behavior change communication. PSI now uses a rigorous system called Performance Framework for Social Marketing (PERForM) that sets out elements and correlations of interest in measuring social marketing performance. PERForM is a tool for guiding PSI’s segmentation of potential target groups, concept development, pretesting, output monitoring, and evaluation of behavior changes. Additional information about PSI’s Behavior Change Communication model is in Annex E-2. o Research management. Research has played an important underpinning role in development of all MSS project strategies and activities. This pervasive use of social research is a reminder of the important role that commercial marketers give to market research: commercial marketers know that high-quality research is an important key to successful marketing. The central place of research in the marketing process is shown graphically in Annex E-3. The Technical Support Team at PSI/INDIA is an example of the expansion of PSI’s vision from a narrow approach in the 1970s and 80s that emphasized distribution and sales of contraceptives to the current approach that maintains an emphasis on products but also on services. 3. Results: Cross-cutting approaches a) Community Mobilization i. How effective was the approach for community mobilization? Although the DIP did not emphasize community mobilization as a cross-cutting activity, Component Two contained mobilization, for example, enthusiastic community participation in BCC events such as courtyard discussions led by the Project’s female Community Workers and street theater events. ii. Were the objectives met for community mobilization? No formal objectives were listed in the DIP. iii. What lessons were learned for future community mobilization? An important lesson is the need for community mobilization to deal with referrals based on Project referral tests that showed little interest among private specialists (Obs-Gyne and pediatricians) to be Final Evaluation Report, Social Marketing Strategies for MCH Page 19 ____________________________________________________________________________________________________________________________________________________________________________________________________ involved in low-income community activities. Recommendation # 8. If additional resources become available under Component Two, PSI/India should consider mobilizing community members to recognize signs of obstetric and pediatric emergencies, then provide adequate funds and transport to obtain emergency assistance at public sector clinics without delay. Community men, as well as women will be needed. Justification: Such referral systems are urgently needed in low-income communities. SMS Project experience to-date indicates that PSI/India can probably succeed in this effort, using proven social marketing techniques. While trained females will be needed to recognize symptoms and signs and raise the alarm, men will be needed to organize transport. iv. Is there demand in the community for program activities to continue? How was this measured? Yes, there is demand for more. A media assessment study clearly showed that people learn from such activities and want to have more. v. What are the plans for sustaining these activities after program closes? Activities will be downscaled by reducing frequency of individual contact, both with community through IPC and with network doctors. Local media will be stopped completely. However product marketing will continue, supported by revenues from product sales. vi. Are the sustainability plans realistic? Yes, the plan outlined above is realistic because some revenues will be available to support some activities under Component One, but it must be emphasized that Component Two cannot be continued without availability of additional financial resources. b) Communication for Behavior Change PSI’s Behavior Change Communication system, including promotion of specific products and advocacy plays an exceptionally important cross-cutting role that accounts for a major proportion of resources invested in field activities. i. How effective was the approach for behavior change communication? Use of multiple media tools like games, street theatre, women’s meetings, demonstration stalls, school activities and house-to-house inter-personal communication by female Community Workers were very effective, based upon evidence from the KAP endline results. For example, these community-based Behavior Change Communication activities were almost solely responsible for the successful uptake of Safewat, given that other media, such as TV, were not used. ii. Were the behavior change objectives met? Yes, as shown by KAP results discussed earlier, most Behavior Change target were met and exceeded and most succeeded well. iii. Lessons learned · Innovative product visibility, accessibility and taste (VAT). PSI/India’s innovative community￾based VAT for promoting its products through street theatre, stalls and games, when combined with counseling by well-trained ISM&HPs in their offices plus door-to-door visits by female Community Workers, can lead to a very effective project. Final Evaluation Report, Social Marketing Strategies for MCH Page 20 ____________________________________________________________________________________________________________________________________________________________________________________________________ · Special events. In Component One, PSI used street stalls and promotional games to promote specific products such as Safewat in Uttaranchal. Street theatre, which requires scripting and directing, is now operating in rural areas of Jharkhand; it is described in a later section. · Phasing. PSI/India has learned that phasing of BCC topics is important, given the very large numbers of sub-topics included under the four major issues in the project’s scope of work. Although it may be tempting to load each message with many topics, phasing in of individual MCH issues and products over time for promotion by Community Workers and PSI/India’s VAT system is probably more powerful and therefore more cost-effective iv. Sustaining behaviors once the program closes? In general, useful public health behaviors such as the MCH issues addressed in this project, such as hand-washing, feeding fluids during diarrhea, contraception to lengthen birth intervals and breastfeeding behaviors, will probably be sustained from generation to generation, and even taught in schools to some extent. Studies in western countries in the 20th century showed that such preventive health knowledge was even more important in extending life expectancy than more dramatic advances in curative medicine such as new pharmaceuticals and surgery. The useful knowledge will probably remain a strong factor in those urban Uttaranchal areas covered by Component Two, where the trained Community Workers and ISM&HPs will continue to pass on the beneficial information to people who will benefit most, namely poorly educated family members, neighbors, friends and clients. However, when the program closes, it would probably be unrealistic to expect proliferation of such information beyond the current areas. To ensure that this successful Behavior Change Communication system will thrive and proliferate, PSI must find additional financial resources v. Are the sustainability plans realistic? The DIP does not include plans for sustainability of BCC activities. vi. How was the impact of BCC interventions measured/evaluated? Two studies were conducted for this purpose: baseline and end line KAP studies and a mid-term media assessment study. Results of the baseline-end line studies, as they applied to Logframe indicators, are reported in the tables on pages 4-7. c) Capacity Building Approach Discuss the capacity strengthening results of this program. This may include how the program improved the capacity of the PVO, the public sector partners, NGOs and/or community-based partners. Use the questions below to guide the assessment. i. Strengthening the PVO Organization · This grant has improved the capacity of the PVO to operate child survival programs by providing for the development of social marketing systems, undertaking operations research under Component Two and developing management and leadership qualities among staff. · Effects of this grant have influenced other programs operated by PSI in India particularly by passing on lessons learned about research systems, training systems and BCC systems, specially the value of infotainment and community-based activities such as training community practitioners and female Community Workers who can undertake valuable woman-to-woman communication in the local community Final Evaluation Report, Social Marketing Strategies for MCH Page 21 ____________________________________________________________________________________________________________________________________________________________________________________________________ ii. Strengthening Local Partner Organizations · An assessment of the local partner, PSI/India, during this final evaluation, included a brief workshop at the head office in New Delhi. Results of the workshop and discussions with senior and middle-level staff indicated that most officers had the skills required to undertake their jobs. Senior management had good leadership skills and a useful range of technical skills. · Visits to the PVO offices in the three states also included mini-workshops and discussions with a variety of officers. Most officers appeared skilled in planning and software use as a result of in￾house training by project team members. It was noted, however, that considerable staff turnover had occurred in recent months as employees departed when they learned that the project would soon end. A senior officer reported that new staff had quickly learned to undertake follow-up studies of trained ISMPs. · Discussion with the key officer of the Rural Development Institute (RDI) in Uttaranchal indicated that IntraHealth, the US Technical Assistance contractor, had built the capacity of RDI to produce high-quality training of local health providers. iii. Health Facilities Strengthening · The only health facilities involved in this project were the clinics of ISM&HPs. Tools developed by INTRAH health were used for training and ‘follow-up’ activity. These activities cannot be sustained in the absence of funding of staff required for follow-up activities. iv. Strengthening Health Worker Performance · The project worked with three types of health workers: private ISM&HPs, traditional birth attendants and community workers and their performance improved. Tools were developed for ISM&HPs which showed improvement in performance. In case of community workers, behavior change tracking tool was used to assess their performance. v. Training · PSI’s training strategy for the ISM&HPs and the female Community Workers under Component Two was noticeably effective. Observations of both categories in Uttaranchal showed, for example, that they included PSI’s Visibility, Accessibility and Taste (VAT) method of personal communication. Observation of ISM&HPs in the field, combines with a perusal of the ISM&HPs Training Needs Assessment and the ISM&HPs Training Manual produced by IntraHealth under subcontract to PSI, confirmed that these health providers were abiding by the training maxim “I teach as I have been taught”. IntraHealth techniques included short, before and after paper and pencil tests for ISMP trainees so they could perceive their own progress after each training session. The useful role of IntraHealth was confirmed by the Rural Development Institute (RDI), whose members had been trained by IntraHealth in preparation for training the ISM&HPs. Initial training of the ISM&HPs was spread over four days, followed by one day refresher and follow-up clinic visits. Trainers of the Community Workers also used IntraHealth’s participatory techniques including role-playing and product VAT techniques. The Community Workers became the cornerstones of woman-to-woman Behavior Change Communication meetings and house-to-house visits in their own communities. · Testimonials by Community Workers in Dehradun showed that they had personally benefited in many ways from their training and fieldwork. Some testimonials were truly touching, such as: “I am now recognized as a helpful person”, “I have the confidence to talk about personal things like family planning”, “I am now a wage earner and I save some of my earnings for myself”, and “My daughter insists on adding a drop of Safewat to her bottle of water that she takes to school.” Final Evaluation Report, Social Marketing Strategies for MCH Page 22 ____________________________________________________________________________________________________________________________________________________________________________________________________ · PSI’s quantitative training targets were met. Importantly, also, the quality of training was uniformly high. For example, male street performers who carried out various types of public presentations and plays were well trained in infotainment techniques, providing interesting, humorous yet informative communications in Uttaranchal and Jharkhand. Furthermore, product salesmen – some employed by PSI, others by contract stockists – used powerful show and tell methods reminiscent of techniques used by “detail men” employed by large pharmaceutical firms in North America when discussing their products with doctors and pharmacists. · There is strong evidence that PSI’s training strategy and techniques represent new ways of doing things. For example, observations throughout India and neighboring countries over the past thirty years has shown that training and education institutes depended almost entirely on group lectures where a pedagogue read materials to the class, sometimes writing on a blackboard while students took notes that were to be regurgitated verbally or – more commonly – on exam papers. The IntraHealth techniques represented a quantum leap to a new dimension, incorporating teaching games and role playing combined with shaping sessions that led to correct training habits. · An important lesson learned was that “I teach as I have been taught” can lead to truly interesting and effective counseling and group teaching methods for a variety of teaching situations including home counseling and street performances. Another lesson is that training should not be a stand alone, one-off activity; it should be followed by on clinic Follow-up meetings with trainees and supportive supervision, as needed. Use of participatory methodologies is essential for making it interactive. The frequency of refresher training should be more, at least thrice, instead of just one time. · PSI/India has also benefited from the training component. Some of its employees are not master trainers, while the Training section has developed a training and capacity building approach that permeates its training activities. It begins with a performance needs analysis that is then matched with LogFrame deliverables and converted into training modules that are pretested, adjusted and used in the field. Monitoring and evaluation results are then fed back into the loop with a view to improving future training programs. · There are no plans to sustain SMS Project training after this project closes on May 31, because there will be no funds available. d) Sustainability Strategy The DIP did not address a specific sustainability strategy. However, three types of sustainability can be addressed here: financial, managerial and institutional. i. Financial Sustainability Financial sustainability refers to ability to maintain project activities after financial support is completed. As with public health programs for low-income people in every country, the beneficiaries of the Social Marketing Strategies Project cannot afford to pay all the costs of the program. However, they do pay part of the costs of Component 1, through purchases of products. This sales revenue is not inconsequential; it provides many millions of rupees that are used for additional programming. However, PSI has not yet applied the same policy to its services, such as training doctors and community workers. Final Evaluation Report, Social Marketing Strategies for MCH Page 23 ____________________________________________________________________________________________________________________________________________________________________________________________________ Recommendation # 9. PSI/India should set up field trials to test user fees for training. Private doctors may be willing to pay for each training, according to their income. It will be useful to begin the tests with relatively low fees, then raise them by 15-20% per year. Justification: Fees for training are based on the philosophy that “I value what I pay for”. Furthermore, the fees will help to offset training costs while improving cost-efficiency. Recommendation # 10. PSI/India should estimate unit costs for various aspects of the SMS Project, for example, annual cost per capita for the region served, cost per doctor trainee. Such estimates will not give conclusive results because of the many assumptions required, such as estimating the amounts of time that various employees provide to a specific activity, but the results will nevertheless be indicative and therefore useful. Results of this research will supplement the cost per CYP information routinely generated by PSI projects. Justification: Estimates of cost-efficiency, ie, unit costs at the output level, will be helpful for budgeting and also as support for proposals. ii. Managerial sustainability Managerial sustainability refers to the ability of staff to work effectively after the project is completed. Managerial sustainability reflects formal training provided during employment, on-the-job training during project planning and implementation and the type and amount of supervision and monitoring provided. Senior and middle managers may find that their experience and training during this project will lead to placement with other PSI/India projects or with other employers. Technical people, such as trainers, researchers and communication professionals may find that their experience with this project will be of considerable help to their career. iii. Institutional sustainability Institutional sustainability refers to the maintenance of fieldwork after project support has been completed. While observations during this evaluation showed that most employees knew their job and worked efficiently, it is difficult to see how management of project activities could be maintained after project closure without salary support. However, the training maxim that “training is forever” will probably hold, especially for the Community Workers and ISMPs who will continue to teach as they have been taught – counseling many people in new and powerful ways that will lead to better lives for many people. C. Program Management 1. Planning a) How inclusive was the program planning process and what effect did this have on the implementation process? Project planning was inclusive. For example, Managers of Component Two frequently used participatory methods, such as brainstorming to develop the Component. Managers also used formal needs assessment methods when planning training of ISM&HPs, as explained above. Monthly review meetings with Community Workers led to adjustments in community communication activities among the Community Workers and street performers. Managers also used informal interaction with community members attending various BCC activities to adjust plans for future activities such as street theatre. Final Evaluation Report, Social Marketing Strategies for MCH Page 24 ____________________________________________________________________________________________________________________________________________________________________________________________________ b) To what extent was the DIP work plan practical? Based on the PVO and its partner’s experience with this program, what could be added to the DIP preparation and review process that would have strengthened implementation? PSI used participatory processes when developing the DIP, involving local staff. This process was very helpful in producing an effective DIP and gave a sense of ownership among participants. However, the DIP has been criticized for its many details and fear of rigidity which would not allow local managers to make tactical changes on the spot, when needed. Recommendation # 11. DIPs, although very useful, should require fewer details and allow for iterative planning as the project develops. Justification: iterative planning allows – in fact, invites – tactical changes based on field experience. Organizations that use iterative planning report that the majority of managers can be trusted to use filed evidence and participation with local staff, subcontractors and beneficiaries before changing tactics. The end result is quicker, more cost-efficient planning and implementation. Commercial marketers produce relatively brief, evidence-based Marketing Plans that are adjusted from time-to-time as indicated by successes and failures in the field. Commercial marketers also use brief test markets in circumscribed areas such as one small city, then scale-up based on results of the test market. c) What were the gaps in the DIP and how were they addressed by the program staff? The DIP had very few gaps. Two gaps were inadequate attention to Community Mobilization and sustainability issues. 2. Staff Training a) What change is there in the knowledge, skills and competencies of the program and partner's staff? Is there evidence that the staff has applied these skills both within the program and in another context? PSI staff training included Behavior Change Communication training, Sales training, and MCH counseling b) Were adequate resources dedicated to staff training? A mini-workshop during this final evaluation showed a strong need for training in Project Management. Recommendation # 12: PSI should invest in formal Project Management training for its senior and mid-level managers. Justification: Project Management is a pervasive activity. Such training will help to ensure managerial and institutional sustainability during absences of senior managers and for sustainability of projects after donor support is completed. c) Overall lessons learned about building the capacity of program staff? The main lesson is that knowledge and skills gained in staff trainings have been useful in project management, training of community workers and training product salesmen. 3. Supervision of Program Staff a) Was the supervisory system adequate? The supervisory system appeared adequate at the country office and at the state level, although it is not known how well deputized managers led state teams during temporary absences of senior managers. Final Evaluation Report, Social Marketing Strategies for MCH Page 25 ____________________________________________________________________________________________________________________________________________________________________________________________________ b) Is the supervisory system fully institutionalized and can it be maintained? This remains an open question to be answered after May 31. It is doubtful, however, that adequate staff and supervision will ensue after donor support is withdrawn on May 31. c) Is there evidence that the program’s approach to strengthening supervisory systems has been adopted beyond the program? No evidence has been observed. 4. Human Resources and Staff Management a) Are essential personnel policies and procedures of the grantee and partner organizations in place to continue operations that are intended to be sustainable? Yes, the PSI country office has an “Employees Handbook that covers many personnel issues such as Expectations, Work Guidelines, Compensation, Benefits, Separation Policy and Code of Conduct. b) Describe the morale, cohesion and working relationships of program personnel and how this affected program implementation. The project has three teams located in three project states plus the head office Team in New Delhi. The teams appear to have very good working relationships and high morale. This is remarkable given the upcoming closure on May 31. c) Describe the level of staff turnover throughout the life of the program, and the impact it has had on program implementation. Almost 60% staff left in last seven months because of project closure. This has put considerable pressure on the remaining staff and affected testing of referral activities in Uttaranchal. d) Plans to facilitate staff transition to other paying jobs at the end of the program Yes, a large proportion of the remaining staff in the states will be placed in other PSI/India positions. 5. Financial Management a) Adequacy of financial management and accountability The budget was adjusted once to increase the field personnel budget line to accommodate the intensive IPC efforts, while the furniture and equipment budget line was increased to accommodate higher than anticipated equipment costs. The promotion, advertising and research budget lines were decreased based on more refined cost estimates. Project managers have adequate budgeting skills to estimate costs and elaborate on budgets for future programming. PSI financial systems are adequate: monthly project financial reports are monitored by the PSI India office then forwarded to PSI headquarters. PSI also operates an internal audit. b) Adequacy of resources to sustain activities beyond the cooperative agreement Component one will continue, whereas component two will proceed with minimal inputs. Final Evaluation Report, Social Marketing Strategies for MCH Page 26 ____________________________________________________________________________________________________________________________________________________________________________________________________ c) Technical assistance available to assist development of financial plans for sustainability No financial plans for sustainability were developed, although PSI/India expected follow-on funds for continuation and scaling up of the Saadhan pilot activities. Unfortunately, these have not materialized. 6. Logistics a) Impact of logistics on implementation Logistics activities have proceeded well, helped by PSI’s thirty-year history of knowledge gained to develop procurement, packaging and distribution systems for many products in many countries. b) Adequacy of the logistics system to support operations and activities intended to be sustained The logistics system is adequate for support and sustain future activities. 7. Information Management a) Effectiveness of the system to measure progress towards program objectives Component One: PSI used its global Field Force Management System (FFMS) and ORG retail audit data to measure progress. Component Two: Internal systems were set up to measure progress of LogFrame indicators via several research agencies. The program being barely for 16 months on the ground, there was no scope of any mid-term assessment of objectives. However, information from the Safe Water System study conducted by INDIAClen on behalf of USAID provided useful information. b) Was there a systematic way of collecting, reporting and using data at all program levels? Cite examples of how program data was used to make management or technical decisions. Yes. Behavior change tracking system and MCH record slips at clinics provided useful information. Community Workers collected household use of products through door-to-door visits. c) Is the program staff sufficiently skilled to continue collecting program data/information and to use it for program revisions or strengthening? Yes, remaining staff will continue to collect data for all Component One activities and for some Component Two activities, including Safewat sales. d) Did the program conduct or use special assessments, mini survey focus groups, etc. to solve problems or test new approaches? Give examples of the research, use of data, and outcomes. Yes, as described above, PSI used several in-house research, training and BCC assessments, mini-focus groups and monitoring surveys, plus assessments by external agencies. For example, INDIA Clen was contracted directly by USAID to monitor prevention and management of diarrhea. e) To what extent did the program strengthen other existing data collection systems (i.e. government)? Not applicable Final Evaluation Report, Social Marketing Strategies for MCH Page 27 ____________________________________________________________________________________________________________________________________________________________________________________________________ f) Do the program staff, headquarters staff, local level partners, and the community have a clear understanding of what the program has achieved? Yes, virtually all stakeholders, including state government officials in Uttaranchal and USAID/India have a solid understanding of achievements, including results of the Endline Survey. g) How have the program’s monitoring and impact data been used beyond this child survival program? PSI/India is using data collected during this project in proposals to USAID and other donors for projects in diarrhea prevention and control. USAID/AIDSMARK water project is using SMS Project data in its forthcoming RFP responses for Zinc and Polio. 8. Technical and Administrative Support a) Discuss types and sources, timeliness, and utility of external technical assistance the program has received to date. As described above, INTRAHealth provided useful technical assistance for training ISM&HPs. PSI provided all other types of technical assistance required. b) What assistance did the program need that was not available? How could PVO headquarters and/or USAID better plan for the technical assistance needs of PVO programs? PVO headquarters provided excellent assistance on many matters. Only one topic could have been more completely addressed, namely Sustainability issues. c) PVO headquarters and regional technical and managerial support of the field program. Approximately how much time has been devoted to supporting this program? PVO headquarters and regional technical and managerial support consisted of the following: 1. The HQ-based Child Survival Coordinator provided assistance in the development of the DIP in Year 1. 2. The HQ-based Program Manager and Contracts Department provided assistance in negotiating and drafting the consulting agreement with IntraH for the program’s training component, and the consulting agreement with John Davies for the program’s final evaluation. 3. The Program Manager provided general backstopping support to the program, including regular budget tracking, review of annual plans, and the preparation and submission to USAID of our request for a two-month no-cost extension. 4. The Program Manager coordinated Vibha’s visit to PSI HQ, USAID and CDC to share our integrated BCC approach for MCH. 5. The HQ-based Research Department rolled out training in PERForM, a newly developed performance framework for social marketing, and the ‘Dashboard’ tool to guide evidence-based decision-making. For items 1 to 4 above, the estimated total HQ level of effort is approximately 20 days per year. For item 5, it would be difficult to determine India’s share of the cost of the development and roll-out of PERForM and the Dashboard. Final Evaluation Report, Social Marketing Strategies for MCH Page 28 ____________________________________________________________________________________________________________________________________________________________________________________________________ 9. Management Lessons Learned The management style and system used by PSI has been developed through thirty years of experience in many developing countries. Its hallmarks include integration of technical systems under the general rubric of Social Marketing, and decentralization of project management. Benefits of the PSI system include technical assistance as needed, fast tactical changes of direction in the field, and strong feelings of ownership among staff. D. Other Issues Identified by the Team 1. Developing and expanding the Saadhan Network. As shown graphically in Annex E-4, the Saadhan Network includes four factors that work together to improve mother and child health: behavior change communication, socially marketed MCH products, health provider franchising and referrals. This section outlines each of the four factors then addresses three other factors that PSI should add in order to produce a complete, synergistic public-private partnership capable of reducing the unacceptably large numbers of preventable maternal and infant deaths in low-income communities. a) Four factors in the current Saadhan Network · Behavior Change Communication. The Project has shown that evidence-based, carefully coordinated BCC can improve knowledge and behavior through a combination of woman-to￾woman counseling by well-trained Community Workers, counseling by trusted health providers, such as ISM&HPs, infotainment and other media. · Socially marketed MCH products. The SMS project has shown how the marketing model succeeds by having the right products in the right place at the right price, with the right promotion. · Health Provider franchising. The SMS project has shown how PSI created successful formal partnerships with ISM&HPs, then trained them in modern MCH techniques and supplied them with PSI’s products. · Referrals. The SMS project experimented with referrals at the community level to show how the use of a coupon could bring different tiers of the community health system to work together. The experiment did not include emergency referrals. The project also tried to build a referral link between ISM&HPs and medical doctors/specialists. Unfortunately, there were insufficient resources to develop this link to its full potential. b) Additional factors required for a powerful referral system · Male participation. The SMS project has emphasized woman-to-woman communications in the communities. Although this emphasis is absolutely required, has worked well, and was a good first step, there is a need to increase the role of men in the Saadhan Network, because males are the major decision-makers in the households, including decisions about who will receive medical care and when. · Community mobilization. The SMS project did not conceptualize its community health activities in terms of community mobilization although it did sow the seeds by training community ISM&HPs and female Community Workers about MCH issues. Community mobilization will be a key to Final Evaluation Report, Social Marketing Strategies for MCH Page 29 ____________________________________________________________________________________________________________________________________________________________________________________________________ organizing emergency referrals, including quick provision of adequate transport. · Public-Private Partnerships. Expansion of the fledgling Saadhan Network to include referral of emergencies will require involvement of the public sector in at least three ways. First, when Saadhan activities are expanded to rural communities, Project managers will need to create public￾private partnerships with the local Panchayats. Second, in rural areas Saadhan managers will benefit from working with female public sector workers such as Anganwadi Workers, and the new ASHA described in Annex E-5. Third, community residents will need to refer emergencies to government hospitals and clinics having surgical facilities that include general anesthesia and blood transfusions. Recommendation # 13: Donor partners should consider expansion of the Saadhan Network along the lines described above, including mobilization of a community-based referral system for obstetric and pediatric emergencies linked to public sector surgical facilities. Justification: Many of the key components for expansion are in place under the existing Saadhan Network. Managers have the necessary knowledge and experience to develop the necessary additional components including community mobilization, male participation and operational involvement of the public sector. SMS Project officers are in a good position to create such partnerships with government because they have regularly exchanged information with Uttaranchal’s government officers about the relationships between, on the one hand, the government’s Health and Populations Policy, the new National Rural Health Mission (NRHM) including its Accredited Social Health Activists (ASHA) workers, the potential for working through SCOVA/Uttaranchal, and on the other hand the possibility for working within the framework of USAID’s Innovations in Family Planning Services Project.(IFPS) Phase Two, which supports Reproductive and Child Health activities that advance public-private partnerships in northern states. Benefits of an expanded version of the Saadhan Network would include improved health for low-income mothers and children and reductions in preventable deaths resulting from obstetric and pediatric emergencies in rural slums and in village. During feasibility and planning of the expansion PSI can find useful lessons from experience in rural Maharashtra where a USAID-funded project is implementing a somewhat similar set of activities.1 2. Live infotainment PSI has developed a powerful behavior change process that emphasizes live entertainment for promoting products and behavior change. This information-with-entertainment is called infotainment. It was developed in the context of the Saadhan Network where small-scale community-based communications were needed to reach slum dwellers close to their homes. Formats include small stalls staffed by trained communicators who used microphones to entertain the crowd and introduce them to educational games, as well as to PSI products, such as Safewat. These street events always incorporated PSI’ system called VAT (Visibility, Accessibility and Taste) to give potential users an intimate moment with the product. The infotainment system has been elevated to a new level in rural Jharkhand where PSI has combined its Social Marketing Strategy resources with Packard Foundation resources to take live performances to villages – both large and small. Comedy-drama is popular and informative when played out in a courtyard or under a huge shady tree. A recent event featured four characters: a young bride, her young husband, her mother in law and later, a doctor. The mother berates the young woman for not having more babies. They argue for a while, both trying to influence the confused young man. Finally, after much laughter and self-realization among the audience, the doctor arrives, takes over the microphone 1 “Community-led Infant and Child (CLIC) Project, implemented by the Aga Khan Foundation/USA and the Mahatma Gandhi Institute of Medical Science, Sewagram, Maharashtra. Final Evaluation Report, Social Marketing Strategies for MCH Page 30 ____________________________________________________________________________________________________________________________________________________________________________________________________ and explains about the values of birth spacing with oral contraceptives, including its benefits and side effects. As the drama ends, the crowd breaks up and a surprisingly large number of young wives rush to the tiny office of the real village ISM&HPs, clamoring for more information about Pearl oral contraceptives. Many of the women are counseled by the ISM&HPs, who had been trained about the pill and the other PSI products in his office, including Masti condoms and Neotral Oral Rehydration Salts. The PSI Manager in Jharkand expects to produce about 8,000 infotainment performances throughout Jharkhand this year. Recommendation # 14. PSI/India’s well scripted, well directed and well acted theatre being watched by an animated, mixed crowd in a small village should be expanded and replicated in the three states. Justification: Live sociodramas coupled with immediate, nearby availability of useful MCH information and products is probably a cost effective method of increasing use of the products E. Conclusions and Recommendations 1. Achievement of Project objectives Based on the data from the baseline and endline presented in the summary charts, the Project has met its objectives at the Purpose level and most of its objectives at the Objectives level. These achievements are especially noteworthy because they were made in a relatively short time – just 30 months of social marketing and 16 months of Saadhan network operation. 2. Major achievements, best practices and lessons learned. Each of the following achievements represents a best practice and a lesson learned: 1. Solid increases in distribution and sales of MCH products under Component One: social marketing. 2. Creation, development and success of the Saadhan Network in Uttaranchal, which illustrated that well-trained ISM&HPs and female Community Workers, could develop major improvements in Mother and Child Health knowledge and practices in slum areas in just a few months. 3. Infotainment, including live theatre in Jharkhand villages (with help from the Packard Foundation) where PSI develops demand for the products through lively outdoor theatre that combines entertainment, such as a soap opera format that includes information about one or more PSI products. 4. The very fast development of Safewat, a new product category that is used to disinfect piped and water stored from community taps before drinking; Safewat is in high demand in the pilot area of Uttaranchal thanks to the well-planned and implemented Social Marketing system including distribution to necessary outlets along with a mix of excellent behavior change communication through various media including one-on-one counseling, demonstrations at street stalls, promotion by doctors, ISM&HPs and pharmacies. 5. Use of an integrated triad of high-quality technical assistance based at PSI/Delhi – training, behavior change communication and research. 6. Coordination of all of the above activities by competent, hard-working PSI employees and subcontractors at various levels. 3. A major constraint One unfortunate constraint is the difficulty now facing the Project, namely the inability to continue and to develop Saadhan because of the apparent end of financial support that will inhibit opportunities to Final Evaluation Report, Social Marketing Strategies for MCH Page 31 ____________________________________________________________________________________________________________________________________________________________________________________________________ scale up and expand the Project. The Saadhan Network is an excellent model for India’s urban slums; it should be nurtured, developed to include an emergency referral system, and expanded to other slum areas and to villages. 4. Managerial sustainability Managerial sustainability was strengthened as PSI managers and field workers gained from training and field experience in social marketing techniques. Progress toward institutional sustainability was also evident as measured by the continuation ability of some social marketing activities to continue after donor support is completed on May 31, 2005 – although without the powerful, paid community-level support such as Community Workers and infotainment – while PSI gained experience in working in integrated MCH activities, including dissemination of best practices including reports on the PSI website 5. Summary of Recommendations Recommendation # 1. The PSI model for Social Marketing of MCH products, including Safewat, is successful and should be continued and expanded. Recommendation # 2: As part of its examination of the problem PSI should commission a brief, in￾depth observational study of family behaviors to learn about common behaviors surrounding the etiology, prevention and the treatment of diarrhea and dehydration among young children Recommendation # 3. To the extent possible, PSI should enquire into the use of specific brands when studying changes in knowledge and behavior. Recommendation # 4. PSI should continue to monitor the work of its research contractors, including making visits during interviewing to ensure that subcontractors consistently supervise their interviewers and back-check respondents. Recommendation # 5. PSI should consider avoiding setting numerical targets for first efforts. Instead, PSI could set targets only after measuring outputs in test markets for a few months before going to scale. Recommendation # 6. PSI should continue to test coupon offers. Recommendation #7. Components One and Two should be taken to scale throughout urban and rural areas of the three states. PSI should add other appropriate products and services to the Saadhan network, for the ISM&HPs and referral doctors, such as IUDs, injectable contraceptives, RTI/STI interventions, Emergency Contraceptives, DOTS for TB medication, zinc preparations and HIV counseling. A special effort is needed to develop community-based referral systems for emergencies such as obstetric complications and infant distress; this important task will require a public-private partnership involving the use of the government’s clinical resources Recommendation # 8. If additional resources become available under Component Two, PSI/India should consider mobilizing community members to recognize signs of obstetric and pediatric emergencies, then provide adequate funds and transport to obtain emergency assistance at public sector clinics without delay. Community men, as well as women will be needed. Final Evaluation Report, Social Marketing Strategies for MCH Page 32 ____________________________________________________________________________________________________________________________________________________________________________________________________ Recommendation # 9. PSI/India should set up field trials to test user fees for training. Private doctors may be willing to pay for each training, according to their income. It will be useful to begin the tests with relatively low fees, then raise them by 15-20% per year. Recommendation # 10. PSI/India should estimate unit costs for various aspects of the SMS Project, for example, annual cost per capita for the region served, cost per doctor trainee. Such estimates will not give conclusive results because of the many assumptions required, such as estimating the amounts of time that various employees provide to a specific activity, but the results will nevertheless be indicative and therefore useful. Results of this research will supplement the cost per CYP information routinely generated by PSI projects. Recommendation # 11. The Detailed Implementation Plan, although very useful, should require fewer details and allow for iterative planning as the project develops. Recommendation # 12. PSI should invest in formal Project Management training for its senior and mid-level managers Recommendation # 13. Donor partners should consider expansion of the Saadhan Network along the lines described above, including mobilization of a community-based referral system for obstetric and pediatric emergencies linked to public sector surgical facilities Recommendation # 14. PSI/India’s well scripted, well directed and well acted theatre being watched by an animated, mixed crowd in a small village should be expanded and replicated in the three states 6. Potential for Scale up The potential for scaling up both components is strong, given the successes of both and the great need for improving mother and child health in low-income areas of India. Component One might be scaled up at relatively small cost because its cost-efficiency is high due to high levels of cost recovery from sales revenues. Component Two might be specially useful in rural areas where the community-led efforts in this urban slum projects could be piloted, adjusted and scaled up. Final Evaluation Report, Social Marketing Strategies for MCH Page 33 ____________________________________________________________________________________________________________________________________________________________________________________________________ F. Results Highlight Community Networks Improve Health of Mothers & Children in India’s Slums India’s northern states have some of the poorest slums and lowest levels of mothers’ health and children’s health. Maternal death rates of 400 or more can be compared with about 50 in Sri Lanka and 10 in western countries. Infant mortality in the slums is also much higher than in middle-class areas. But much of this tragic burden of disease can be prevented when life-saving products are made readily available at affordable prices and necessary health information is given at the doorstep or through street entertainment. The USAID-supported Social Marketing Strategies for Maternal and Child Health Project has shown remarkable success in USAID’s focus states: Uttaranchal, Uttar Pradesh and Jharkhand. Under a Child Survival and Health Grant Program from USAID, Population Services International (PSI), a US-based Private Voluntary Organization that specializes in social marketing, is now selling its affordably-priced brands of iron-folic acid tablets for anemic mothers, oral rehydration salts for children’s diarrhea, oral contraceptive pills for birth spacing, clean delivery kits for home births, and condoms for birth spacing and prevention of sexually transmitted disease in more than 45,000 retail stores. And in Uttaranchal state, PSI has gone further in urban slums by creating a Mother and Child Health Network called Saadhan (“The Way”). PSI trains local medical practitioners and community women. The medical practitioners become Saadhan franchise holders while the women become Community Workers. Both teach parents about reliable means of contraception, proper use of iron-folic acid tablets to prevent anemia, oral rehydration salts to treat diarrhea, and prevention of diarrhea through proper hand washing and promotion of an important new product – bottles of Safewat for purifying drinking water that is often polluted. The Community Workers spread the word through woman-to-woman counseling and courtyard meetings, Saadhan clinics provide quality products, information and counseling on MCH issues while PSI “infotainment” specialists provide information through demonstrations and games on street corners. After 30 months of program implementation, distribution of PSI’s products in the three states were beyond expectations, including sales of more than 80 million condoms, 2 million cycles of oral contraceptives, 6 million packets of iron tablets and 1 million packets of oral rehydration salts. After only 16 months of operation, the Saadhan Network in the urban slums of Uttaranchal has led to substantial, positive behavior changes. For example, use of birth spacing methods among low-income parents increased from 38% to 41%. Use of a clean delivery kit during home deliveries increased from 18% to 28%. Tetanus toxoid coverage among pregnant women increased from 84% to 89%. IFA coverage increased from 61% to 72%. Use of oral rehydration therapy by parents during their child’s previous diarrheal episode jumped from 15% to 56%. Treatment of drinking water to improve its quality increased from 5% of households to 38%, while the practice of proper storage of drinking water increased from 26% to 31%. And there is good news for American taxpayers. PSI’s socially marketed products are paying for themselves. Even at prices that slum dwellers can afford, sales revenues cover the procurement cost of the products. * * * * * Final Evaluation Report, Social Marketing Strategies for MCH Page 34 ____________________________________________________________________________________________________________________________________________________________________________________________________ Annex A. Evaluation Team Members and Titles External: John Davies, International Health Consultant, Team Leader Internal: Carmen Chan, Director, Family Health Program, PSI/India and Vibha, Director, Social Marketing Strategies for MCH Project, PSI/India. The following officials from the government of Uttaranchal state, who were aware of the pilot Saadhan Network, participated during the Team’s visit to Uttaranchal: · Principal Secretary, Health & Family Welfare - Mr S K Das · Executive Director, SCOVA – Dr Umakant Panwar · Advisor to the Secretary – Dr I S Pal · Additional Director for Health – Dr B C Pathak Dr. D. Roy, Training Coordinator, HIHT-Rural Development Institute, who had worked intimately with IntraHealth on the development and implementation of the training component for Saadhan health practitioners also joined the Evaluation Team in Uttaranchal Many PSI managers from the Delhi office and offices in the three states assisted for short periods, often through mini-focus group meetings. The team leader and Vibha also interviewed many health providers such as ISM&HPs, Community Workers and pharmacists trained by the Project. PSI/India invited officials from USAID/Washington, USAID/India and PSI/ Washington, but they could not join because of previous commitments. Final Evaluation Report, Social Marketing Strategies for MCH Page 35 ____________________________________________________________________________________________________________________________________________________________________________________________________ Annex B. Evaluation Assessment Methodology Officers from PSI Headquarters and PSI India organized this evaluation. The three key team members worked together throughout the evaluation. The Team Leader was the lead author of this report; the internal members also contributed to the writing. Comment by the Team Leader This joint evaluation system succeeded well, partly because the beneficial mix of responsibilities, backgrounds and perspectives of the three key members provided more insights than would have otherwise been possible, while temporary members provided specialized information and participated in general discussions, from time to time. While it is sometimes argued that internal members of an evaluation team may tend to show external people only the successes, while hiding or playing down the problems, my experience as evaluator and as evaluatee in India and in other countries, has shown that most people want to provide the entire story, including difficulties and failures, as well as successes. For example, Carmen Chan and Vibha readily discussed factors in the non-achievement of some of the program targets, and areas of the program that did not reach full potential. In my experience, the external-internal system is superior to the purely external option, which can sometimes be perceived as a “we versus them” exercise, reminiscent of teams of financial auditors who search first and foremost for shortcomings and sometimes ask the wrong questions while sometimes being unaware of important technical, personal, social or cultural considerations. * * * * * Final Evaluation Report, Social Marketing Strategies for MCH Page 36 ____________________________________________________________________________________________________________________________________________________________________________________________________ Annex C: Persons Interviewed and Contacted A. New Delhi PSI India office Carmen Chan – Programme Director, Family Health Sushant Banerjee- Technical Director Aditi Verma- Research Manager Jaishree Nair- Training Manager B. Uttaranchal PSI Project office: Vibha-Project Director Amit Rawat-Project Coordinator Neeraj Dixit-Project Officer, Neeraj Jham and Ranjeet Samariyar-Inter-personal Communication Coordinators, Manoj Verma – Area Sales Manager Rural Development Institute Deb Brath Roy, Faculty, RDI-HIHT State government S K Das - Secretary Health and Family Welfare Umakant Panwar – Executive Director, SCOVA B C Pathak – Additional Director, Health and Family Welfare I S Pal – Advisor to the Secretary Bharti Dangwal – NGO Coordinator, Health and Family Welfare Project partners – community level Dr A K Pundir-ISMP Dr Akhilesh Bhatnagar-ISMP Mini-Focus Group Discussion with Community Workers Observed Community Workers courtyard meeting and product demonstration Observed local street media game Observed Safewat stall activity C. Uttar Pradesh PSI office, Lucknow Dheeraj Chawla – State Sales Manager A P Prajapati, Ravi Prajapati, A K Goel, Mahesh Mathur, Ajay Singh - Area Sales Managers Vivek Dwivedi – Field Officer Final Evaluation Report, Social Marketing Strategies for MCH Page 37 ____________________________________________________________________________________________________________________________________________________________________________________________________ Project Partners, community level Dr Ruksana and Dr Ravi Saxena - Private practitioners Mr Agarwal – Super Stockist for PSI products D. Jharkhand PSI office, Ranchi Mathew Joseph – Project Director Pritam Sunder - State Sales Manager Additional officers Final Evaluation Report, Social Marketing Strategies for MCH Page 38 ____________________________________________________________________________________________________________________________________________________________________________________________________ Annex D: CD Please see enclosure Final Evaluation Report, Social Marketing Strategies for MCH Page 39 ____________________________________________________________________________________________________________________________________________________________________________________________________ Annex E-1: Training & Capacity Building Approach, PSI/India Jayashree Nair, Training Manager, FHP, PSI/India May 19, 2005 Trainings are planned on the basis of the Performance Improvement Approach (PIA). This approach focuses on the need to do a thorough need assessment to identify the performance gap areas in the following: ÿ Knowledge and skills required for a job ÿ Clear job expectations ÿ Clear and immediate performance feedback ÿ Adequate physical environment, including proper tools, supplies and workspace ÿ Motivation and incentives to perform as expected. During need assessment, baseline research findings such as Knowledge Attitude and Practices (KAP) survey results, segmentation results are referred. Structured tools that help us capture the above performance factors are developed to identify training needs. Analysis of the above helps us to focus on input areas that will bring about an intended/required change in the attitude, knowledge, skills and behavior of the recipients of training. The results of the Performance Needs Assessment (PNA) helps the trainer to give focus to the training issues that needs to be addressed. The results are then focused towards the program deliverables and logframe indicators. Post PNA results; the training strategy, identification of trainers, budget, learning objectives of the training, curriculum designing is done. Instruments to evaluate the training, job aids, handouts etc are developed. Learning guide cum checklists is developed to ensure transfer of learning and follow-up of training inputs. Simultaneously the training materials, logistics and roles and responsibilities of each person involved in it are clarified. The training venue and materials required for the implementation of training are listed. Once the entire package is ready, the training is delivered. The principles of adult learning are followed during training and hence the training is planned to be interactive in nature giving participants enough opportunities to learn from their own experiences. Each session has a carefully planned learning objective that is Specific, Measurable, Achievable, Realistic & Time bound (SMART). During training the knowledge and skills are evaluated with a pre and post assessment. The assessment results are confidentially shared with all the participants and areas that need to be improved are highlighted. The learning guide cum checklist is also used post training to monitor & assess transfer of learning. Each training has follow-up plan to ensure transfer of learning from the classroom to the field. The follow-up is done through supportive supervision. The trainers follow up the participants and provide on the ground hands – on support. These follow-ups are done by using the knowledge and skills assessment checklist. Supportive feedback is provided to the participants and later an analysis of the results of follow-up inputs are done and areas that need further strengthening or new areas that requires inputs are identified and refresher training is organized. Inputs to ensure the other performance factors such as physical environment, motivation, tools, clarity in job expectations etc are also followed up and feasible inputs are provided. Refresher training is organized to cater to areas that require further strengthening or any additional input area that may enhance the performance of the participants. Final Evaluation Report, Social Marketing Strategies for MCH Page 40 ____________________________________________________________________________________________________________________________________________________________________________________________________ Annex E-2: PSI’s Approach to Behavior Change. Effective social marketing encourages behavior change through a combination of commercial sector marketing techniques (often mass media) and health sector approaches to interpersonal communication IEC. PSI believes that an individual's behavior tends to be influenced by a number of factors, and that a behavior change project needs to consider these factors to create a real impact. Generally, these factors belong to the following categories: · Material conditions: disposable income, commodities prices, access to products · Social support: traditions, religious influences, gender differences, interaction between couples · Individual predisposition: personal risk assessment, self efficacy, product knowledge and perception, knowledge of disease PSI has developed a behavior change approach that incorporates theoretical constructs into a framework combining key concepts from the fields of behavioral sciences and marketing. In some cases, certain factors must pre-exist to arrive at the next level (for example, one must be aware of the disease before understanding its transmission or appreciating its severity), however as this diagram depicts, the complex process of behavior change is largely non-linear. The framework is based upon increasing awareness of the problem, awareness of the transmission of the problem, and the perceived severity of the problem, as without fully understanding a health risk and its detrimental impact, one is less likely to initiate initial contemplation of behavior change. Once personal risk is assessed, a desirable solution must be available and one must have confidence in his/her ability to adjust behaviors as well as community and social support to reinforce the positive behavior change. Factors used in this model can be defined in simple sentences describing an individual's perceptions: Awareness of problem: "I have heard of anemia. It is a condition that is caused by insufficient iron intake." [Capitalization required: use this respondent in peer group education sessions. Explore the depth of her and her peers’ knowledge. Add one knowledge layer to that knowledge store. Deflate one myth associated with that knowledge (if evident). Show one practical way to improve birth outcome through an intervention involving a birth planning exercise. Associate that way with a key product. Discuss the financial implication of the proposed intervention.] Final Evaluation Report, Social Marketing Strategies for MCH Page 41 ____________________________________________________________________________________________________________________________________________________________________________________________________ Understanding causation: "I know that drinking unpurified water is one cause of diarrhea.” [Capitalization required: Explore the concept of impure water and when it is okay and not okay to use it. Add one more knowledge layer to the store of knowledge. Deflate one myth associated with that knowledge (if evident). Show one more way to improve health through an intervention involving water Associate that way with a key product. Discuss the financial implication of the proposed intervention.] Appreciating severity: "Diarrhea is especially dangerous for young children. It can kill." Solution efficacy: "I believe that iron-fortified micronutrient supplements protect against anemia.” Personal risk assessment: "My child is at risk of poor fetal development if I do not consume the proper amount of iron during pregnancy." Social support: "People whose opinion I value will support my using safe water systems. It's normal." Affordability: "ORS is worth the money. I know I can afford to buy and use the solution during my child’s episodes of diarrhea." Availability: "I can find CDKs. CDKs are sold at places I find accessible." Brand appeal: "The attributes of the branded Saadhan products meet my requirements. They are products for someone like me.” Self-efficacy: "I believe that I have the ability to take action in the prevention of diarrhea. It is important that I convince the economic power-holder that the use of Safewat necessary. It is imperative that I find the means to purchase Safewat for me and my family." Not all factors influence all population groups in the same way, nor do they have the same importance from one group to another, and not all factors evolve at the same time or in the same direction. Tradition, and its prescribed means of prevention, may be strong enough to deter many women from seeking more effective means as defined by modern medicine, such as Safewat. Because each group responds in a unique way to a set of influencing factors it is essential to analyze those groups separately. Epidemiological, demographic, attitude-specific, behavioral, and even a combination of several characteristics can segment the potential market. In addition, different communications channels may be better suited for influencing certain factors. For example, mass media may be best for influencing social support and creating positive brand associations, while interpersonal communication of generic prevention messages are more effective in increasing personal risk perception. Final Evaluation Report, Social Marketing Strategies for MCH Page 42 ____________________________________________________________________________________________________________________________________________________________________________________________________ Annex E-3: PSI’s Social Marketing Research Process Aditi Varma, Research Department, FHP, PSI, New Delhi 18 May, 2005 The objective of social marketing is behavior change. Social marketers need valid, timely, comprehensible, and actionable recommendations from researchers at each stage of the social marketing process. Target audiences are influenced to exchange their current behavior for a healthier (or safer or environmentally protective) behavior through the creation of opportunity, ability, and motivation (Rothschild, 1999). Creating these three elements of the intervention depends upon the four Ps of the traditional marketing mix: product, price, place and promotion. Research is a key element of Social Marketing, enabling Social Marketers to learn from the customer to develop relevant, high impact programs. Research also allows Social Marketers to evaluate the impact of the program on health status, and modify it to increase its success. PERForM (A Performance Framework for Social marketing) defines the scope of social marketing research within PSI. The PERForM concept is shown on the next page. PSI Social Marketing Research Process within PERForM Segmentation is the division of a population into homogenous groups or markets on the basis of need, behavior, opportunity, ability, and motivation, and population and intervention characteristics. Concept development is the systematic description of perceptions, behavior processes, and preferences of priority population segments for purposes of marketing mix decisions. Pre-testing is determining the extent to which the concepts, products, messages and materials that make up the social marketing intervention are understood and persuasive among target segments. Monitoring is measurement of levels and trends in the awareness and recall of key aspects of the marketing plan and of opportunity, ability, and motivation, given a set of population characteristics. Evaluation extends monitoring to include an examination of whether the marketing and or project plans changed opportunity, ability, and motivation; behavior; health status; or quality of life. At the center of the research process is the “bubbles” framework that shows how behavior comprises a set of psychosocial and marketing constructs thought necessary to create a behavior change. Each bubble (except for behavior) can be grouped or summarized as relating to opportunity, ability or motivation. Segmentation, Monitoring and Evaluation tables together are called “dashboards” because of their utility over the program tenure in determining: What should we do in the program? (Segmentation) How far have we come and how fast are we going? (Monitoring) Is this behavior change due to PSI’s work? (Evaluation) * * * * * Final Evaluation Report, Social Marketing Strategies for MCH Page 43 ____________________________________________________________________________________________________________________________________________________________________________________________________ Annex E-4: Saadhan MCH Network and the Marketing Cycle Source: SMS Project: adapted by Final Evaluation Team Saadhan MCH Network: PSI’s Integrated Approach to Child Survival Cross cutting all activities: Research & Training Emergency Referrals Doctor Franchising Behavior Change Communication ÿ Counseling ÿ Infotainment ÿ Couponing ÿ Radio Products > Doctors’ clinics > Retail shops Saadhan MCH Network: PSI’s Integrated Approach to Child Survival Cross cutting all activities: Research & Training Emergency Referrals Practitioner Franchising Behavior Change Communication ÿ Counseling ÿ Infotainment ÿ Couponing ÿ Radio Products > In clinics > In retail shops > Training > Branding Final Evaluation Report, Social Marketing Strategies for MCH Page 44 ____________________________________________________________________________________________________________________________________________________________________________________________________ The Social Marketing Cycle 5. AnalyzeSatisfied Target Groups JJJJJ 1. AnalyzeUnsatisfied Target Groups 2. DevelopSatisfying Products / Services 4. DevelopDemand • Train counselors • Advertise counselors • Infotainment 3. Develop Supply • Train salesmen • Increase outlets Research • Train outlet staff Final Evaluation Report, Social Marketing Strategies for MCH Page 45 ____________________________________________________________________________________________________________________________________________________________________________________________________ Annex E-5: ASHA and the National Rural Health Mission ASHA - Accredited Social Health Activists The Government of India has decided to address the health needs of rural populations, especially the vulnerable sections of society through the National Rural Health Mission (NRHM), which is described below. The NRHM will include a new band of female community based functionaries, named Accredited Social Health Activist (ASHA). A system similar to that of ASHA was started in Pakistan in the 1990s with the help of a World Bank loan. An evaluation showed that these paid Lady Health Workers were trained to undertake useful public health tasks but that more supervision was required to maintain their productivity. Since ASHA will not be paid employees, it will be important to develop a mechanism to sustain their motivation to continue working. · Key Roles of ASHA o ASHA will take steps to create awareness and provide information to the community on determinants of health such as nutrition, basic sanitation & hygienic practices, information on existing health services and the need for timely utilization of health & family welfare services. o She will counsel women on birth preparedness, importance of safe delivery, breastfeeding and complementary feeding, immunization, contraception and prevention of common infections including Reproductive Tract Infection-Sexually Transmitted Infection (RTIs/STIs) and care of the young child. o ASHA will mobilize the community and facilitate them in accessing health and health related services available at the village/sub-center/primary health centers, such as immunization, antenatal check-ups, and postnatal check-ups sanitation and other services being provided by the government. o She will work with the Village Health & Sanitation Committee of the Gram Panchayat to develop a comprehensive village health plan. o She will arrange escort/accompany pregnant women & children requiring treatment or admission to the nearest pre- identified health facility i.e. Primary Health Centre/ Community Health Centre/ First Referral Unit. o ASHA will provide primary medical care for minor ailments such as diarrhea, fevers, and first aid for minor injuries. She will be a provider of Directly Observed Treatment Short￾course (DOTS) under Revised National Tuberculosis Control Program. o She will also act as a depot holder for essential provisions being made available to every habitation such as Oral Rehydration Therapy (ORS), Iron Folic Acid Tablet (IFA), chloroquine, Disposable Delivery Kits (DDK), oral contraceptive pills and condoms. * * * * * Final Evaluation Report, Social Marketing Strategies for MCH Page 46 ____________________________________________________________________________________________________________________________________________________________________________________________________ National Rural Health Mission approved by cabinet in Jan, 2005 Source: “India Together” Website, April, 2005 7 February 2005 – The UPA Government’s list of provisions for improving the health of the population as put forth in its Common Minimum Program (CMP) was short. It promised to raise the public expenditure on health, increase investment in the control of communicable diseases, ensure healthcare for the poor through a national health insurance scheme, improve the availability of life￾saving drugs at a reasonable cost and introduce a targeted population control program in 150 districts in the country. But following criticism about its plans for a ‘sharply targeted population control program, the central government (including the Prime Minister) made repeated assertions that coercion cannot be part of any population decentralized policy. As a step towards fulfilling its promises, on January 4, 2005 the UPA cabinet approved the formation of a National Rural Health Mission (NRHM). The mission will aim at integrating different vertical programs, decentralized health care service delivery at the village, and improving intersectoral action. (‘Vertical’ implies that machinery responsible for planning; implementing and monitoring are unique for every centrally funded program and operate without any coordination with the others. Thus when the pulse polio machinery or the HIV/AIDS machinery start rolling they do not consider the implications of such a campaign on the delivery of other health care services, say maternal health or regular decentralized services.) The mission is expected to make a substantial reduction in maternal and infant mortalities from communicable diseases in the next four years. The government has provided the mission with a budget of Rs 6510 crores (1 crore = 10 million) and the body is expected to become functional from April this year. The mission will have a steering group chaired by the Prime Minister and will be located in the Ministry of Health and Family Welfare. Ministers of related Ministries, the Deputy Chairman of the Planning Commission and public health activists from civil society will be members of the mission. Who the specific invitees from civil society are has not yet been decided. The bottlenecks One of the main bottlenecks to the effective delivery of comprehensive healthcare services at the community level has been the multiplicity of vertical national health programs. While all these programs have depended upon the lowly multipurpose health worker (officially called the Auxiliary Nurse Midwife – ANM) for their implementation, the programs’ different planning, monitoring and supervisory systems bring about very uneven pattern of service delivery. The interminable rounds of the Pulse Polio campaign have a serious impact on routine, decentralized programs as well as delivery of other essential services like maternal health. Also the family planning program often takes precedence over all other interventions in the absence of any integrated decentralized (bottom-up) planning. The NRHM proposes to address these problems by a number of innovations at the community level. Access at the local level Firstly the NRHM proposes to appoint a community level health worker who has been named ASHA or Accredited Social Health Activist. It is proposed that over two hundred and fifty thousand such health workers will be appointed and they will provide first contact care to people in villages. These health workers will be women and will work closely with the Anganwadi Worker (village level worker of the Integrated Child Development Services) and the ANM. They will be chosen by and be accountable to the women in the community. This means that women get to choose who would become the ASHA Final Evaluation Report, Social Marketing Strategies for MCH Page 47 ____________________________________________________________________________________________________________________________________________________________________________________________________ from their villages, and the ASHA would be answerable to the women as well. Health education services (covering a wide range from drinking water, sanitation, maternal and child health) would be available on a more regular basis. Health education has been a very weak point in the system currently. The ANM is either a non-resident or has too many other responsibilities (including filling up of a large number of registers, a separate one for each vertical government program). The ASHA is also expected to provide curative assistance for fever, diarrhea, etc. In today’s scenario these services are supposed to be available at the sub-centre/village, however the ANM is seldom present there. Now the proposal is that there will be two ANMs so that one person can be doing outreach services while the other is present at the sub-centre. Due to health education and referrals from the ASHA, deaths in high risk pregnancy and childbirth cases may reduce. This is a major problem in under-served areas where the maternal death figures are up to 700 maternal deaths/100,000 live births. (Compare with 30 in Sri Lanka, and less than 10 for European countries.) * * * * * Final Evaluation Report, Social Marketing Strategies for MCH Page 48 ____________________________________________________________________________________________________________________________________________________________________________________________________ Annex E-6: Rapid CATCH Comparison Table Comparison of the Rapid Catch Indicators for Child Survival Project in Dehradun and Haridwar District of Uttaranchal. S.No Indicator Numerator Denominator Base Description Proportion CONFIDENCE LIMITS with design effect of 1.5# BASE LINE END LINE BASE LINE END LINE BASE LINE END LINE + - Sig. Leve l 1 Percentage of children age 0-23 months who are underweight (-2 SD from the median weight-for￾age, according to the WHO/NCHS reference population) 389 NA 982 NA All children who were weighed 39.6 NA NA NA NA 2 Percentage of children age 0-23 months who were born at least 24 months after the previous surviving child 175 551 480 819 All children age 0-23 who have an older sibling less than 5 yrs of age 36.5 66.6** 70.6 62.6 99% 3 Percentage of children age 0-23 months whose births were attended by skilled health personnel 241 341 603 719 All Mothers 40.0 47.4** 51.9 42.9 99% 4 Percentage of mothers with children age 0-23 months who received at least two tetanus toxoid injections before the birth of their youngest child 433 591 603 719 All Mothers 71.8 82.2** 85.6 78.8 99% 5 Percentage of children age 0-5 months who were exclusively breastfed during the last 24 hours 43 57 146 157 All mothers with children in the age range 0-5 months 30.1 36.3 45.5 27.1 75% # For calculating the confidence interval - n’ = n/ 1.5 as per the KPC 2000+ Guidelines, where n refers to either the sample size or the sub-sample. The confidence interval has been calculated using P = p +- Z *sqrt (pq/n’), where P = population proportion, p = sample proportion, q = 1-p, Z = 1.96 at 95% level of significance. Final Evaluation Report, Social Marketing Strategies for MCH Page 49 ____________________________________________________________________________________________________________________________________________________________________________________________________ S.No Indicator Numerator Denominator Base Description Proportion CONFIDENCE LIMITS with design effect of 1.5# BASE LINE END LINE BASE LINE END LINE BASE LINE END LINE + - Sig. Leve l 6 Percentage of children age 6-9 months who received Breast milk and complementary foods during the last 24 hours2 35 108 115 141 All mothers with children in the age range 6-9 months 30.4 76.6** 85.2 68.0 99% 7 Percentage of children age 12-23 months who are fully vaccinated (against the five vaccine￾preventable diseases) before the first birthday NC3 195 NC 613 All children age 12-23 months NC 31.8 36.3 27.3 NC 8 Percentage of children age 12-23 months who received a measles vaccine (as reported in verified vaccination card) 53 217 308 360 All mothers with children in the age range 12-23 months 17.3 60.3** 66.5 54.1 99% 9 Percentage of children age 0-23 months who slept under an insecticide-treated net (in malaria risk areas) the previous night 0 20 603 719 All Mothers 0.0 2.8** 4.3 1.3 99% 10 Percentage of mothers with children age 0-23 months who cite at least two known ways of reducing the risk of HIV infection4 148 197 603 719 All Mothers 24.6 27.4 31.4 23.4 75% 2 Complementary Breastfeeding Rate is defined as the proportion of infants age 6-9 months receiving breast milk and solid / mashed foods 3 Not calculable as information was only collected on the basis of vaccination card and not maternal report/recall. 4 These are only the recommended methods as given in KPC 2000+ (all those who coded any two of the following – Abstain from sex, Use condoms, Limit sex to one partner, Limit number of sexual partners, Avoid sex with Final Evaluation Report, Social Marketing Strategies for MCH Page 50 ____________________________________________________________________________________________________________________________________________________________________________________________________ S.No Indicator Numerator Denominator Base Description Proportion CONFIDENCE LIMITS with design effect of 1.5# BASE LINE END LINE BASE LINE END LINE BASE LINE END LINE + - Sig. Leve l 11 Percentage of mothers with children age 0-23 months who report that they wash their hands with soap/ash before food preparation, before feeding children, after defection, and after attending to a child who has defecated 10 510 603 719 All Mothers 1.7 70.9** 75.0 66.8 99% 12 Percentage of mothers of children age 0-23 months who know at least two signs of childhood illness that indicate the need for treatment 305 512 603 719 All Mothers 50.6 71.2** 75.3 67.1 99% 13 Percentage of sick children age 0-23 months who received increased fluids and continued feeding during an illness in the past two weeks 6 18 603 719 All Mothers 1.0 2.5* 3.9 1.1 96% 14 Percentage of children age 0-23 months who received ORS during an episode of diarrhoea in the last two weeks 14 7 128 65 All Mothers who reported their children had their diarrhoea in the last two weeks 10.9 10.8 20.0 1.6 2% prostitutes, Avoid sex with persons of the same sex, Avoid sex with persons who inject drugs intravenously, Avoid blood transfusions, Avoid injections, Avoid sharing razors / blades) Final Evaluation Report, Social Marketing Strategies for MCH Page 51 ____________________________________________________________________________________________________________________________________________________________________________________________________ Annex F: Project Data Sheet Form - DIP Child Survival Grants Program Project Summary On Line Data Form PSI India Field Contact Information First Name: Carmen Last Name: Chan Address: C-445 Chittaranjan Park City: New Delhi Zip/Postal Code: 110019 Country India Telephone: 91 11 26278375 Fax: 91 11 26275919 E-mail: carmen@psi.org.in Project Web Site: http://www.psi.org Project Information: Project Description: The goal of PSI India's CSHGP award, Social Marketing Strategies for MCH in Uttar Pradesh, Jharkand and Uttaranchal, is to reduce infant and child (under 5) mortality and morbidity in the States of UR, UP, and JH. The purpose is to increase positive MCH behaviors among, and increase the use of essential MCH products by low￾income users. Essential MCH products include CDKs, IFA, SWS, ORS, OCPs for birth spacing and condoms for the dual purpose of disease prevention and birth spacing. The project has two components: I) Social marketing of a basket of 6 essential MCH products in the three States, and II) Pilot of the Saadhan referral network of private medical providers in low-income urban centers of Dehradun and Hardwar of Uttaranchal. Partners: PSI India, Prime-INTRAH, State DOH of Uttaranchal, Uttar Pradesh and Jharkand, local NGOs such as RDI/HIHT, private medical practitioners. Project Location: Uttaranchal, Uttar Pradesh and Jharkand States of India (Pilot in low-income areas of Dehradun and Hardwar, Uttaranchal) Final Evaluation Report, Social Marketing Strategies for MCH Page 52 ____________________________________________________________________________________________________________________________________________________________________________________________________ Grant Funding Information: USAID Funding:(US $) $1,299,952 PVO match:(US $) $1,563,475 Target Beneficiaries: Type Number 0-59 month old children: 5,546,749 Non sterilized women 15-49: 6,400,095 Pregnant women (annual): 1,152,017 Beneficiary Residence: Urban/Peri-Urban % Rural % 85% 15% General Strategies Planned: Social Marketing Private Sector Involvement M&E Assessment Strategies: KPC Survey Community-based Monitoring Techniques Participatory Evaluation Techniques (for mid-term or final evaluation) Behavior Change & Communication (BCC) Strategies: Social Marketing Mass Media Interpersonal Communication Support Groups Capacity Building Targets Planned: PVO Non-Govt Partners Other Private Sector Government Community US HQ (CS unit) Field Office HQ CS PVOs (Int'l./US) e.g: INTRAH, EHP Pharmacists Non-traditional medical product distributors (stockists and retailers) Private Allopathic and ISM State and District level health authorities Local health staff TBAs CHWs Final Evaluation Report, Social Marketing Strategies for MCH Page 53 ____________________________________________________________________________________________________________________________________________________________________________________________________ Project Team Local NGOs health providers Interventions: Immunizations 5 % ** HF Training *** Classic 6 Vaccines Nutrition 5 % ** HF Training *** Comp. Feed. from 6 mos. *** Cont. BF up to 24 mos. *** Growth Monitoring Micronutrients 20 % ** HF Training *** Iron Folate in Pregnancy Control of Diarrheal Diseases 20 % ** CHW Training ** HF Training *** Water/Sanitation *** Hand Washing *** ORS/Home Fluids *** Feeding/Breastfeeding Maternal & Newborn Care 15 % ** HF Training *** Emerg. Obstet. Care *** Neonatal Tetanus *** Recog. Of Danger signs *** Newborn Care *** Post partum Care *** Delay 1st preg Child Spacing *** Integr. with Iron & Folate *** Normal Delivery Care *** Birth Plans Final Evaluation Report, Social Marketing Strategies for MCH Page 54 ____________________________________________________________________________________________________________________________________________________________________________________________________ Child Spacing 30 % ** HF Training *** Child Spacing Promotion *** Pre/Post Natal Serv. Integration Breastfeeding 5 % ** HF Training *** Promote Excl. BF to 6 Months Indicator Numerator Denominator Estimated Percentage Confidence line Percentage of children age 0-23 months who are underweight (-2 SD from the median weight-for-age, according to the WHO/NCHS reference population) 389 982 39.6 3.8 Percentage of children age 0-23 months who were born at least 24 months after the previous surviving child 175 480 36.5 5.3 Percentage of children age 0-23 months whose births were attended by skilled health personnel 241 603 40.0 4.8 Percentage of mothers of children age 0-23 months who received at least two tetanus toxoid injections before the birth of their youngest child 433 603 71.8 4.4 Percentage of infants age 0-5 months who were exclusively breastfed in the last 24 hours 43 143 30.1 9.2 Percentage of infants age 6-9 months receiving breastmilk and complementary foods 35 115 30.4 10.3 Percentage of children age 12-23 months who are fully vaccinated (against the five vaccine-preventable diseases) before the first birthday 64 151 42.4 9.6 Percentage of children age 12-23 months who received a measles vaccine 53 308 17.2 5.1 Percentage of children age 0-23 months who slept under an insecticide-treated bednet the previous night (in malaria-risk areas only) 0 603 0.0 0.0 Percentage of mothers who know at least two signs of childhood illness that indicate the need for treatment 305 603 50.6 4.9 Percentage of sick children age 0-23 months who received increased fluids and continued feeding during an illness in the past two weeks 6 603 1.0 1.0 Percentage of mothers of children age 0-23 months who cite at least two known ways of reducing the risk of HIV infection 148 603 24.5 4.2 Percentage of mothers of children age 0-23 months who wash their hands with soap/ash before food preparation, before feeding children, after defecation, and after attending to a child who has defecated 10 603 1.7 1.3 Final Evaluation Report, Social Marketing Strategies for MCH Page 55 ____________________________________________________________________________________________________________________________________________________________________________________________________ Final Evaluation Report, Social Marketing Strategies for MCH Page 56 ____________________________________________________________________________________________________________________________________________________________________________________________________ Final Evaluation Report, Social Marketing Strategies for MCH Page 57 ____________________________________________________________________________________________________________________________________________________________________________________________________ Final Evaluation Report, Social Marketing Strategies for MCH Page 58 ____________________________________________________________________________________________________________________________________________________________________________________________________ Final Evaluation Report, Social Marketing Strategies for MCH Page 59 ____________________________________________________________________________________________________________________________________________________________________________________________________ Final Evaluation Report, Social Marketing Strategies for MCH. page 1 _________________________________________________________________________________________________________________________________________________________________________________________________________