Senegal Maternal Health/Family Planning Project: Mid-Term Evaluation Report, November 2003 John Pollock Malcolm Bryant John McKenney Amelie Sow November 2003 Senegal Maternal Health and Family Planning Project This report was made possible through support provided by the US Agency for International Development, under the terms of Cooperative Agreement Number 685-A-00-00-00113-00. The opinions expressed herein are those of the author(s) and do not necessarily reflect the views of the US Agency for International Development. Management Sciences for Health 165 Allandale Rd Boston, MA 02130 Telephone: (617) 524-7799 www.msh.org 2 SM/PF Project Mid-Term Evaluation Report Table of Contents Acronym List 3 Acknowledgements 5 Introduction & Background 6 Methodology & Scope of Work 7 Project Strategy & Principles (Applicability and Effectiveness) 9 Results & Progress Towards Results 17 Technical Components (Volets) Logistics 20 Clinical 24 Behavior Change & Communication 36 Advocacy 48 Project Systems, Resources, and Motivations 53 Context that influences Results and the Potential for Achievement 58 Annexes Annex I Persons Consulted 61 Annex II Documents Reviewed 65 Annex III Scope of Work 66 Annex IV Proposed Expansion of the Pont d’Or 69 Annex V Project Monitoring Tool 73 Annex VI Demand Generation 84 Annex VII Quelques Résultats d’étape / Certains Indicateurs 89 3 ACRONYMS ADEMAS Agence de Développement du Marketing Social AIDS Acquired Immunity defiency Syndrome ANC Ante-Natal care ARPV Association des Relais polyvalents BCC Behavior Change & Communication CA Cooperating Agency CBD Community Based Distribution CEFOREP Centre de Formation et de Recherche en Santé de la Reproduction COPE Client-Oriented Provider-Efficient CPN Consultations prénatales CPR Contraceptive Prevalence Rate DEE Direction de l'Enseignement Elementaire DISC Développement des Initiatives Sanitaires et Communautaires DLCS Devision de la Lutte contra la SIDA DSR Division de la Santé de la Reproduction ECR Equipe Cadre de Région ECS Equipe Communautaires de Sante FFSDP Fully Functional Service Delivery Point FP Family Planning GIS Geographic Information System HIV Human Immuno-deifiency Virus ICP Infirmier Chef de Poste IEC Information, Education, Communication IPC/C Interpersonal Communication/Counseling IPPF International Planned Parenthood federation IPT Intermittant Preventive Treatment IUD Intra-Uterine device JHU/CCP Johns Hopkins University/Center for Communications & Policy M&E Monitoring & Evaluation MCD Médecin Chef de District MCR Médecin Chef de Région : MH Maternal Health MSH Management Sciences for Health MSP Ministère de la Santé et de la Prévention PC Persuadeur Communautaire PICG Performance Improvement Consultative Group PMTCT Prevention of Mother to Child Transmission PNA Pharmacie Nationale d’Approvisionnement PNC Post-Natal Care PNLP Programme Nationale Lutte contra Paludisme PP&R Performance Planning & Review REPS Reseau des partenaires du Senegal RH Reproductive Health RPM+ Rational Pharmaceuticals Management Plus Project SCS/PF Senegal Child Survival/Family Planning Project SDP Service Delivery Point SERDHA Service d’Etudes et de Recherche pour le Développement Humain de l’Afrique 4 SM/PF maternal health/Family Planning Project SO3 Strategic Objective 3 SOAG Strategic Objective Agreement SOUB Soins Obstétricaux d’Urgence de Base SP Sulfadoxine-Pyrimethamine STI Sexually Transmitted Infection TFGI The Futures Group International USAID United States Agency for International Development 5 ACKNOWLEDGEMENTS The assessment team is grateful for the efforts made by the MH/PF Project Chief of Party, Deputy, and all of the members of the Team to provide careful thought and input into the design of this assessment and for the magnificent job that was done in planning and carrying out the complex logistical arrangements required to allow us to meet the many people who have a role in the implementation of this project and a stake in its outcome. Its also important to acknowledge the leadership role of Colonel Adama Ndoye of the Division de la Santé de la Reproduction of the MSP in providing reliable strategic guidance to the project and the positive impact of his careful attention and steady availability. The USAID SO team also extended its open and supportive relationship with the project team to the process of this assessment, offering key inputs into the process as well as significant contributions to the content. The utility of this exercise has depended heavily on the availability and candid thoughts of many collaborating groups and individuals. Leaders in the health system at regional, district, and health post levels all gave time willingly and provided important information. Service delivery staff made time for us and provided essential perspective on their work and the context in which they operate. Private practitioners, Relais (members of the community linked to the health information system), religious leaders, and, representatives of the media, and the staff of other USAID cooperating agencies provided important reflections on strategic considerations that will impact on results. The core project team participated fully and at each step in this process. Each has made a willing investment of time (the scarcest project resource) that has assured an outcome where we have been able to link our observations accurately to project elements and processes. Thanks to all. John Pollock Malcolm Bryant John McKenney Amelie Sow 6 INTRODUCTION & BACKGROUND The Senegal SM/PF Project was awarded in May 2000 to MSH as a prime contractor, with JHU/CCP, and TFGI as sub-contractors. A five year cooperative agreement was signed for $5,826,458. This has subsequently been increased through several contract amendments and changes in scope of work to $7,423,506. The Project responds to USAID Key Intermediate Indicators 3.1 and 3.2 under Strategic Objective 3 (Increased and sustainable use of reproductive health services in the context of decentralization) with linkages to KIR 3.3. KIR 3.1 Improved access to quality services IR 3.1.1 Functionality of existing public health SDPs improved IR 3.1.2 Network of private sector SDPs offering RH services expanded IR 3.1.3 Coordination between public and private sector services improved IR 3.1.4 Program management and technical monitoring of public and private sector services improved KIR 3.2 Increased Demand for quality services IR 3.2.1 Increased knowledge of the benefits of reproductive health services IR 3.2.2 Increased participation of opinion leaders (religious, political, civil) in social mobilization IR 3.2.3 Private sector information, education, and communications activities expanded KIR 3.3 Increased financing of health services from internal sources IR 3.3.1 A monitoring system for the legal and regulatory framework for health made functional. (With reference to integration of contraceptives into the essential medicine system) The Project is now beyond the mid point and appears to be moving towards achieving key objectives. MSH’s policy is to conduct a mid-project assessment of all its field projects as part of its internal quality assurance process, and to provide guidance to project staff, donor and local counterparts on future directions building on the successes and lessons learned, and to make recommendations for changes should they be indicated. The assessment also supplements the routine monitoring, evaluation, and reporting process in place in the project. In addition to the preceding, the Senegal SM/PF Project has undergone substantial changes since its original design, and it is important to confirm that the project remains on-track technically. 7 METHODOLOGY AND SCOPE OF WORK Underlying principles of the review : • The process is designed to assist the Project team and MSH in identifying any unexpected or previously unanticipated problems with the execution of the Project. • The review team’s primary goal is to be able to validate the approaches taken by the Project and to give guidance on areas where successes can be built upon. Where areas of weakness or need for corrective action may be identified the team will offer possible actions to be taken. • An important additional output of the review will be the ability to describe (in a systematic fashion) those things that have worked and why they have worked, so that we can elaborate a clear framework for how each critical area and/or system in the project works. This is especially important, because documenting why something is working is not a contractual requirement of the project, but is an essential reference for future planning. • The evaluation will be comprehensive, but not exhaustive. In addition to the above, the team will keep three questions in mind: • Are targets being met? • Is this program laying the ground work for the next phase? • Is the project’s approach the most efficient way of achieving the strategic objective.? Specific Objectives of the review: 1. Verify if the original strategy for the Project remains valid and has been implemented effectively. 2. Assess the progress towards achieving the Project Objectives and results 3. Evaluate the quality and implementation of the four Project technical volets at central, regional, district, and community levels 4. Assess the overall technical and administrative management of the Project 5. Evaluate the effectiveness of the collaborative relationships between key partners (Reseau des partenaires du Senegal (REPS) The detailed Scope of Work is attached as Annex *** 8 Methodology: A ‘Performance Improvement’ approach is the basis for this exercise, which is designed to include all of the key participants and stakeholders in the process of strategic review and discussion of progress and impact. This model illustrates the guiding role of monitoring and evaluation for a project and provides a ‘map’ for observations and recommendations in that it can be clear to the team members, for whom the assessment needs to become a working tool, whether a recommended action is a routine extension of current activities, a call to review an intervention or to consider alternative approaches, or a newly identified (or emphasized) factor that requires a reexamination of root causes. This approach has been used by the Project team over the past two years to analyze and steadily improve it own communication and operational function and it is a positive fit for this exercise. The model was adapted by the USAID-supported Performance Improvement Consultative Group (PICG) from the work of the International Society for Performance Improvement. GET and MAINTAIN STAKEHOLDER AGREEMENT AND PARTICIPATION IN THE PI PROCESS CONSIDER INSTITUTIONAL CONTEXT MISSION GOALS STRATEGIES CULTURE CLIENT and COMMUNITY PERSPECTIVES DESCRIBE ACTUAL PERFORMANCE DEFINE DESIRED PERFORMANCE FIND ROOT CAUSES Why does the performance gap exist? SELECT INTERVENTIONS What can be done to close the performance gap? IMPLEMENT INTERVENTIONS MONITOR AND EVALUATE PERFORMANCE The Performance Improvement The Performance Improvement Process Process Strengths to build on and gaps CONSULTANT FACTORS – ROLE ASSUMED, COMMUNICATION SKILLS 9 The composition of the review team: The team is lead by John Pollock, Senior Fellow in Health Reform & Financing at MSH, and member of MSH’s Center for Health Systems and Services. Other team members are: Malcolm Bryant, Project Support Leader to the Senegal SM/PF Project and Senior Fellow for Development at MSH; John McKenney, Senior Financial Services Officer from MSH’s field office support unit; and Amelie Sow, Africa Division Program officer and Senegal Project backstop from JHU/CCP. Joe Deering, of the Futures Group, was unable to participate in this group exercise, but was able to visit the project in September to make his assessments, which are available in a separate report. PROJECT STRATEGY AND PRINCIPLES The original Project design was based around the conceptual framework of the Fully Functional Service Delivery Point. This Framework consists of three major elements: • Progressive stages of Management development • Establishing Fully Functional Service Delivery Points • Ensuring an appropriate mix of management systems at all levels The evaluation team is satisfied that the original Project design was excellent and that the main strategies have been effectively followed and elaborated on. In addition, the FFSDP strategy has evolved to the point where it can now be extended to include a system of certification that builds on the Pont d’Or strategy. Specific comments are outlined below: 1. Progressive stages of management development. The goal of any program is to attain optimum utilization of services and program sustainability. There are four well defined stages of program development which can be described as Start-up, Growth, Maturity, and Sustainability. These stages are characterized by different levels of development in Access, Quality, Demand, and Management and Leadership (Institutional Strengthening). It is clear that there has been real progress in increased access, improved quality, increased demand and evolution of management systems and leadership capacity, in the last three years. 10 Stage I Low supply, low demand, poor utilization Stage II Good supply, growing demand Improving utilization Stage III Good supply, good demand, good utilization Stage IV Optimum Utilization of Services Access - Limited # service delivery channels - Huge barriers to access - Multiple service delivery channels - Still many barriers to access Multiple service delivery channels, effective public-private collaboration - Few remaining barriers to access - Widespread network of service delivery channels - Virtually no barriers to access Quality - Major focus is provider training - Little monitoring of quality - Increasing attention to quality of care, specifically norms and standards, supervision, monitoring of quality - Quality concerns expand to include choice, convenience, “client focus” - More attention to monitoring of quality - Services perceived to be of high quality and “client focused” - Ongoing quality assurance program in place Demand - Awareness limited to more urban, more educated populations - IEC messages/ materials broadly focused - Expanded awareness - Nascent community involvement - BCC/IEC messages/materials more targeted, exploring alternative media - Widespread awareness; harder-to- reach populations beginning to express demand - Growing community involvement- BCC/IEC messages well targeted, using multimedia approaches - Widespread awareness - Widespread expression of demand- Active community involvement￾Demonstrated “behavior change” Management and Leadership - Basic systems for planning, training, logistics, management info., financial management - Supervision rarely takes place - Increased attention to strategic planning, coordination - New focus on data for decision-making, efficiency, cost recovery - Management systems strengthened - Supervision takes place but is a “control” and is intermittent - Expanded focus on strategic planning, data for decision-making, control over resources - Management systems more flexible, efficient, sophisticated-Supervision remains intermittent but is supportive and facilitative - Management systems robust, able to adapt to changes in the environment - Supervision integrated into all activities and is interactive and problem solving Access: At the outset of the project, there was estimated to be good access, which would lead to a classification of Stage II, with some elements of Stage III. At the time however, the access was predominantly through public health services and mostly facility based. Work had begun on removing barriers to access. At this point in time, there have been significant advances in increasing channels of access for both family planning and maternal health services. A major impact has come through the piloting of the CBD approach for contraceptives and PCs for Maternal Health, and when the ARPV program is implemented these approaches will be throughout the project areas. Similarly, piloting work with the private sector has greatly expanded geographical access to maternal health and family planning services. 11 There have also been innovative approaches to increasing access to specific services to overcome resource constraints – such as through training of nurses to conduct emergency obstetric care, and the use of religious leaders, the Ministry of Education, and the Parliamentary Network to reach more people. The Project is well established in both increasing access points and in reducing barriers to access, and firmly bridges Stages II and III. There is little doubt that the Program should achieve Stage III – Maturity by the end of the Project, and that many elements of sustainability should also be present. Quality: The Project began very much in the Start-up phase for quality of care in both family planning and maternal health. This is not because work had not been done in the past, but because the focus had been on training almost exclusively. During the first three years of the Project, the focus has been radically shifted from training to formative supervision. Norms and standards have been developed and implemented. Responsibilities have been decentralized, and a process of supervision designed and implementation begun. The Project currently bridges Stages II and III, with a strong tendency towards maturity in both family planning and maternal health. However the project gains are fragile, and as outlined in other parts of this report, the maintenance of this state of affairs is far from sustainable at present, and significant care an attention needs to be placed on ensuring that the outcome of the next three years is moving to Stage IV so that project gains are not lost immediately inputs end. Demand: The Project began at Stage II – Growth and has remained at this level. This has however been a conscious strategy, and will be addressed in the coming three years when increased focus will be paid to converting the high level of knowledge about family planning and maternal health into increased use of services. Management & Leadership: The project began at level I. Despite past efforts in this area, the discontinuity in support between the SCS/PF Project and the current USAID portfolio meant that management systems suffered and much of the gains were lost. However, the project has rapidly been able to boost the management and institutional strengths to Level II, and is rapidly progressing to Level III. If some of the key recommendations of this evaluation are carried out – Leadership training for supervisors, improvements in the data for decision-making process; and if the current trajectory of community involvement and leadership continues, we can expect to achieve a mature level within the next two years. As with quality however, these gains will be tenuous without strong attention to issues of sustainability and advocacy at the Policy level. 12 2. Fully Functional Service Delivery Points: The focus on the Service Delivery Point has remained vital to the implementation of the Project, and is one of its greatest strengths. By having clear criteria for intervention at the Service Delivery Point, and by dictating that a balance be found between both the provision of services and the community involvement, this Project has developed the unique character described in the technical sections. Most importantly the focus on the SDP has also become the framework used by other CAs in Senegal in their implementation. What is even more encouraging is the way in which the Pilot program Pont d’Or, has evolved from the FFSDP concept and how this lends itself to a certification and self improvement program for SDPs. This is described below. 3. Ensuring an appropriate mix of management systems at all levels MANAGEMENT SUPPORT drug logistics information system human resources financing organizational structure communication, coordination & planning quality assurance supervision & training marketing community mobilization FULLY FUNCTIONAL SERVICE trained & motivated staff sufficient equipm ent, drugs, & supplies adequate infrastructure functioning referral system PROVIDER QUALITY DELIVERY OF INTEGRATED RH/CS MENUS SERVICE DELIVERY POINT Has rights to: Good supplies Good working environment Good management and supervision Clear information and training CLIENT H as rights to: Clear information Access to services C hoice of FP m ethod Safe services Privacy Confidentiality Dignity, com fort Free expression of opinion Continuous supplies FULLY SUPPORTIVE COMMUNITY Social acceptance Attention to under-served Affordable services Informed choice SOCIAL M OBILIZATION Social M arketing C om m unity Participation Community Ownership BCC Advocacy Cost Recovery W ITH: Local NGOs C om m unity groups Village Committees Local governments INVOLVED in: Planning Implementation M anagement M onitoring Policy environment Enables providers to perform Enables clients to act on demand Social environment dialogue & advocacy SUPPLY DEMAND Local ownership Sustainable use of quality services Figure 2: Fully Functional SDP Model 13 The systems top model was proposed to focus on the efficacy of the health system for meeting client demand by matching the appropriate mix of management interventions at each level of the system. Originally proposed as a means by which to determine the appropriate mix of management interventions at each level of the system, this element has been the least used in the Project. COMMUNITY HUMAN RESOURCES FORMATIVE SUPERVISION LOGISTICS SYSTEMS ORGANIZATIONAL STRUCTURE QUALITY ASSURANCE COORDINATION & PLANNING COMMUNITY MOBILIZATION INFORMATION SYSTEM NATIONAL REGION DISTRICT S.D.P. MUNICIPALITY FACILITY THE SYSTEMS TOP A representation of the levels and functions of the Senegalese Health System to the Service Delivery Point Figure 3 The efficacy of the system is dependent on the appropriate mix of management systems at and within each level of the health system. 14 COMMUNITY NATIONAL DISTRICT S.D.P. FORMATIVE SUPERVISION AT DISTRICT LEVEL FORMATIVE SUPERVISION AT NATIONAL LEVEL FORMATIVE SUPERVISION AT FACILITY LEVEL ANALYZING THE SENEGAL SYSTEM A representation of the allocation of Formative Supervision at each level of the Senegalese Health System MUNICIPALITY FACILITY Figure 4 REGIONAL The element has been useful, and is descriptive of the level of effort required at the different levels of the system. The tool was used in the early stages of the Project in assessing the relative level of effort and priority in each of the different management elements. It would also be valuable for the team to take the time on an annual basis to reflect on the actual level of effort being dedicated to each management element to ensure that the pressures of the field do not detract from creating a balanced and sustainable program. Implementing the FFSDP Framework As noted earlier, the FFSDP Framework is at the heart of the Project’s success to date, and will be crucial for ultimately achieving the Project goals. Most comments on the Framework are dealt with in the relevant technical sections, but the following do not fit well into those and are included here. Trained staff in SOUB is a problem that the project has been able to provide an innovative and effective solution to. The lack of midwives is acute, and to wait for new midwives to be trained and posted would be unacceptable. The Project has therefore been able to develop a new curriculum for state nurses that provides them with the specialized skills to carry out emergency obstetric care to the same level that a midwife does, with the exception of the use of the Ventouse for operative delivery. 15 This is both innovative and unusual in that it has not run into problems from the nursing council or other statuary bodies that regulate provider behaviors Referral mechanisms are a huge problem for obstetrical care. While the channels are well defined – each health post and health center knows where to refer to; the actual system itself is flawed. The most important site to be able to conduct operative and emergency obstetric care is the district, and yet the majority of districts are not equipped to provide this care. They have neither an operating theatre, nor staff competent to perform the procedures. The result is that referral times are greatly extended while patients pass by health centers to go to regional centers, and mortality and morbidity is greatly increased because of both the delay and the poor condition of the roads. While it is not within the Project’s scope to make recommendations for policy change, it is important that the Project take a strong advocacy role to bring about the policy changes necessary to designate every health center as an obstetric reference center. These centers should be equipped with an operating theatre capable of conducing caesarian section and other operative deliveries. Staff at the district (both MCD and other doctors), should receive specific training in order to be able to conduct operative delivery. In addition to the above, the Project staff should continue to emphasize the importance of the community contributing to maintaining emergency transportation at the health post and health hut level so that complicated delivery can be referred promptly. Information management has been made a high priority by the Project at the SDP. Presentation of statistics has been a fundamental part of the COPE exercise, as has the interpretation of the statistics. There is need for better communication between USAID Projects on the nature of how information use is being taught and implemented at the SDP so that efforts are not duplicated. In addition, it is worth exploring how information can be presented more simply using graphical means rather than verbal reports. At district and regional levels this may be most effective using a simple G.I.S. Evolution of the FFSDP framework The Project adopted the Pont d’Or strategy (described later), early on in the project. This built on the concept of the FFSDP and has been very effective in improving both quality of services and utilization of services in the 6 pilot sites. The challenge that faces the Project is how to scale this up to all project sites. The evaluation team proposes that the two strategies be merged into a single concept, which will focus on the achievement of Functional status for all health facilities (i.e. all facilities meet the minimum requirements for service delivery and community engagement). Beyond that, we propose that a system of incentives and certification be introduced that builds on the Pont d’Or concept and results in the creation of a multi-tiered excellence program. This is represented graphically below. The concept has been discussed in some depth with the team, and if it is accepted, then will need careful work to define the criteria for the community involvement elements. Service delivery is already covered effectively 16 with the Project’s supervisory system. The development of community measures will be a challenge as they will have to be both quantifiable and replicable. Assistance can be provided by home office teams as required 17 RESULTS AND PROGRESS TOWARDS RESULTS Attaining Project Goals The strategic objective of increased use of maternal health and family planning services in the context of decentralization remains valid. Indeed, it is fair to say that increasing use remains one of the most important goals in the Senegalese health care system at this time. Similarly, the intermediate results of increasing access and increasing demand remain the main channels to achieving increased use, especially when linked with increasing availability of financial resources. From discussions with counterparts at all levels, with USAID, with community leaders, and with other collaborating agencies, the prevailing opinion is that the Project has the right focus, has achieved a good balance in its emphasis on the different intermediate results, and has a high likelihood of producing a significant improvement in both access to services, improvement in the quality of services, and increasing the demand for services. The four technical interventions remain the most important way to achieve the Project goals, although there have been very interesting developments in these volets. • The Clinical volet is somewhat misnamed, in that its real focus is increasing access to quality services, and for its part includes a wide range of management, human capacity development and organizational development activities in addition to clinical skill development. • The Advocacy volet has turned out to be exceptionally effective and much more prominent in the Project’s scope than originally expected. Originally conceived as a supplement to IEC/BCC as a way to generate demand, the volet has played a significant role in increasing access to services through its work with Persuadeurs Comaunitaires and Infirmiers Privées, and has also introduced some of the most far-seeing and sustainable interventions such as those in collaboration with the Ministry of Education. • The Logistics volet has proven to be more important than anticipated, especially in having a national role rather than simply for the project focus areas. The Project’s response to extend the technical assistance in this area has ensured continued inputs. • The IEC/BCC volet has not reached its full potential at this point. Partly by design at the DSR level (it was stated explicitly that access and quality of services needed to be assured before wide scale demand generation took place), IEC/BCC has remained narrowly focused, and largely limited to pilot areas. The Pont d’Or has been an important initiative in this area. There has been interesting work done in radio and local media, but the time has arrived where the IEC/BCC program needs to be expanded. The challenge that faces the IEC/BCC volet is not to increase knowledge about family planning – the ANNUAL SUPPLEMENTAL SURVEY has already shown that to be high, - but to increase utilization. New IEC/BCC strategies need to be focused on this. 18 One of the most important characteristics of the SM/PF project is its decentralized nature. The strategic decision to have resident regional coordinators has been very important in the ability to maintain contact with regions and districts. In addition to creating a local presence for the Project, the districts seem to claim some form of “ownership” of the regional coordinators. It was commented on in several different forums (from a local politician to the head of the PNLP), that the Project is unlike any previous Project in its decentralized nature. The routine presence at health centers and health posts; the consistent contacts with local politicians; the focus on drawing on community power; and, the approach to extending project activities into the community through the ‘Reduce’ model have all had significant effects. However, this decentralized approach has its toll. The project is having difficulty in achieving the geographical scope originally envisaged. At present the project is fully engaged in 15 districts, and by the end of the year will be in 21. This is already stretching the project resources to the maximum. Vehicles are used to the maximum (often breaking down) and staff are pushing themselves towards burn-out. It is necessary to slow the pace somewhat. For this reason, the discussion with USAID that the Project will continue to the anticipated end of the SOAG between the US and the government of Senegal is welcome, as it enables a change of pace and will ensure that the full 29 districts can be reached by the end of September 2006. There are however significant financial constraints to achieving the anticipated results. The most important is the rapid devaluation of the dollar (approximately 15% during the life of the Project). The current pipeline predicts a shortfall of approximately $800,000 (or 11% of the total). If this is not dealt with in the next workplan period, the Project is either going to have to significantly curtail technical activities, or elect to maintain the level of activity, but bring the end-date forward to December 2004. The other important budgetary factor is the need to ensure that as the SM/PF project instrument ends, that adequate funding is provided to whatever instrument and implementation mechanism USAID chooses, to ensure that there is not a forced reduction of activities. One approach would be to provide an overlap of instruments, but other means would be available, and the evaluation team simply wants to point the need to address this problem before it becomes a crisis and affects the ability to achieve the project goals. Project Monitoring Plan USAID Program monitoring USAID has established an excellent process to measure the impact of its programming in Senegal. Because of the fact that there are multiple projects that bring different components to achieving the USAID strategic objectives, the conducting of an Annual Supplemental Survey which measures key indicators enables USAID to have a good overview of the success of its program. While there are indicators that relate to the objectives of each Project, the work of each Project is also interrelated, and so it is not easy to tease out the specific contribution of each project to many of the indicators. For example, how to differentiate the contribution of clinical training in ante-natal care to 19 lowering maternal mortality in comparison to funding a mutuelle that enables women to have better financial access to that care? The most recent ANNUAL SUPPLEMENTAL SURVEY was released in October 2003, and looked at the 15 focus districts. Conducted by an independent organization (SERHDA), the study provides interim data between Demographic and Health Surveys, and it shows that USAID’s program is “on-track”. The full report is not summarized here, but with specific relevance to the Project it shows that CPR is currently at 9.8%, while knowledge about family planning amongst eligible women is very high – 94%. This reinforces the earlier statement that IEC/BCC focus needs to be on utilization rather than knowledge. There is not a wide mix of methods used – the majority of women using pills or injectables, but Norplant, IUD, condoms/spemicides and natural methods are also used. The survey supports the need to expand access points, especially into the private sector, but also to community sources such as CBD. It also strongly reinforces the need to provide a focus on men and the use of religious channels to reach families. In reference to maternal health, it is interesting to note that 81% of pregnant women have attended three prenatal visits (although this is much lower in rural areas). The quality of antenatal care seems to be generally satisfactory, although there are some concerns that relate to the supply of medications and counseling about both danger signs and care of the child after birth. Developing emergency obstetric care capacity, increasing the number of women who deliver under the care of a trained midwife, and providing effective means of operative intervention at a local level are all critical to improving maternal mortality. The low use of bednets shown in the survey is of concern and requires a concerted effort by the Project and partners. The ANNUAL SUPPLEMENTAL SURVEY therefore provides encouraging data on the progression towards achievement of overall project goals, and has provided good guidance to the Project about where to place emphasis in the coming year. Routine Project Reporting The USAID requirement of the Project for reporting relates to the state of implementation of the Project workplan. The work plans are developed in conjunction with USAID and the DSR, and as such represent the best assessment of what is required to achieve the overall project results. By measuring the progress in achieving the activities contained in the work plan, it is assumed that one is measuring progress towards achieving the overall goals. However, this is an assumption that needs to be questioned. The Project has performed well according to this reporting requirement, achieving 70% of activities in year 1, 80% in year 2, and is on track to achieve 90% in the current year. The measure of workplan activities completed however, measures the quality and 20 realistic nature of the workplan, and the ability of the Project to complete that workplan. It does not report on either the quality of the work done, nor on the impact of the work completed. For this reason, the Project has recently introduced an internal instrument to measure project performance. Project monitoring tool: This tool is attached as Annex V The project has selected 48 indicators from the 4 technical volets that are used to measure both progress towards achieving specific goals, and which measure the quality of the interventions being undertaken. This tool serves to guide the team in the next year’s workplanning, and also enables better distinction of the contributing factors to achieving overall project goals. The evaluation team worked with the project team in examining the indicators in the tool, and several were modified using the approach: What is it that is being changed? How will you know it has changed? What is the result of the change being made? PROGRESS IN THE FOUR TECHNICAL VOLETS The Project focus on the FFSDP results in a division of the technical volets into two main streams. The service delivery (supply), side, and the community (demand) side. The two technical volets of Logistics and Clinical care constitute the main activities on the supply side. This section presents an overview of the strategic approach being used by each volet; observations on the implementation; an assessment of the effectiveness of the approach based on both qualitative and quantitative information; and finally measures what is being done against the FFSDP framework. LOGISTICS Strategic approach The logistics volet is unique in the Project in that it is not restricted to the districts in the Project focus regions, but is expected to focus on every district in the country. The Project is building on the work that was done during the previous bilateral project. In that project a central depot for contraceptives was established within the National Pharmacy (PNA). This was run in-parallel to the system used by the PNA and at the time there was no interaction between the PNA and the Project’s logistics system. In the interim period between the SCS/PF Project and the start of the SM/PF project, contraceptive logistics was a lower priority and the level of supply fell dramatically. With the start-up of the SM/PF project and a renewed focus on contraceptive logistics there has been a dramatic improvement in the system. Statistics from the most recent quarterly report demonstrate how effective the intervention has been at the district level: Indicator 2001 2002 1st half of 2003 % of district stores that have no stock-outs of contraceptives 42% 60% 84% 21 % of district stores that no stock-outs attributable to local factors 86% 94% 98% This demonstrates both an absolute improvement in the functioning of the system, but also demonstrates a dramatic improvement in the way in which the central system is functioning. The Project’s strategy has been simple. Introduce a set of basic management tools at each level of the system which have been suitably adapted to reflect the specific logistics needs at that level, train the individuals in their use, and reinforce the efficient use of the tools through supervision, with periodic audit every six months. The project has continued to maintain a separate system for contraceptive logistics, running in conjunction with the PNA system for the delivery of essential medicines. The goal is to integrate the contraceptive logistics system into the PNA system before the end of the project to ensure the long-term sustainability of the program. Observations 1. Stock-outs at peripheral levels The tools that the project has developed are the standard tools of drug and inventory management adapted to the local situation in Senegal: • Stock/bin cards • Standardized ordering system • Stock Management Manual • Introduction of a “Sales Register” • Introduction of a “Prescription Register” Staff at all districts and posts have been trained in the basic use of these tools and the tools are all available and used at both district and health posts. However, the Project monitoring system indicates that while the level of performance is extremely high at the district level (in excess of 90% compliance with use of tools), it remains low and erratic at the health post level. This is an area of major concern for the project staff, and has become a major focus in the last 6 months. Analysis of the situation suggests that the problem comes down to the ineffectiveness of the supervisory system. As with so many other technical skills being dealt with by the Project the new skills learned in training, if not reinforced by in-situ learning, tend to be quickly forgotten or not applied. In this situation, the dispensing pharmacy assistant at the health post should be supervised by the nurse in-charge. However for reasons that vary from poor prioritization, to misunderstanding, to inadequate time, that supervision is not taking place, with the result that stock-outs in contraceptives are common at the health post level. The Project’s response to this is two-fold. First, the process of formative supervision that has been introduced in the last few months transforms the learning process from the traditional mode of “train and forget”, to a new mode of “sequential building of skills and 22 reinforcing capacity”. Secondly, the introduction of the Inventory Management and Assessment Tool provides senior managers with a new and effective tool for understanding their system and being more engaged at their own level and in their supervisory role. It is too early to say whether this approach will provide the results looked for, but based on the success at the regional and district level there is every reason to expect that within the next 3 years we will see stock-out levels at the health posts that mirror the progress seen at the district level. 2. Integration of contraceptives into the essential drug management system The Project has the goal of integrating contraceptive management into the overall essential drug management system before the end date. Five years ago this would have been an impossible goal, but the major reforms that have taken place at the PNA make this a more reasonable goal. The establishment of PNA as an autonomous unit, and the appointment of a senior and extremely competent management team lead by Lt. Colonel Issa Diop, has resulted in dramatic improvements in the management of PNA. The improvement is such that supplies are so much better that the PNA has insufficient room for its stocks and is asking the Project for additional space, or to move the contraceptives. (This was later the topic of heated discussion with the DSR). PNA has also recently purchased a number of new trucks for transportation of drugs to peripheral depots. There are many issues that arise when considering integration into the PNA/essential drugs system. Some of the reasons it is desirable include: • It will overcome the current dysfunctional arrangement whereby essential drugs and contraceptives are managed through parallel systems; • It will facilitate the reduction of prejudice against contraceptives that is often seen at the peripheral level; • It will greatly enhance the ability to ensure that other essential medications necessary for the use of contraceptives (e.g. local anesthesia for Norplant) is available; The director of the PNA and his staff seem to work very closely with the Project staff, and there is clearly a strong working relationship. The Project has provided all the technical training for the PNA in the last three years, and has also been responsible for the redesign of the tools used for management. This is very significant because it means that the tools used for management of the logistics and essential drug systems have already been drawn together as a launching point for a more integrated system. There remain a number of issues that need to be directly addressed: - Procurement of contraceptives: it seems that there is a strong desire on the part of the PNA to take over the responsibility for the procurement of contraceptives. However this does not translate into PNA financing the purchase of contraceptives. While it is important for PNA to improve its international 23 procurement capacity, it is not reasonable for USAID to hand this responsibility to the PNA when USAID can make more efficient and effective purchases on the international market; - The pricing of contraceptives needs to be regulated and regularized to ensure continued widespread access; - 3. Integration into the Bamako Initiative Contraceptives have been integrated into the Bamako Initiative at the health center and health post level for several years. However there remain some serious constraints to their effective use. Firstly, contraceptives are a not a high source of revenue for the local community, with the result that if there is prioritization to be done, contraceptives are often left out of orders made from the SDP to the depot. Secondly, there are no firm price guidelines, with the result that individual communities are able to set prices at will, and there is considerable variation around the country. This is clearly unacceptable because: 1. Higher prices are likely to reduce access; 2. The contraceptives are a donation by USAID, and should be sold at the minimum cost necessary to recover distribution costs, not as a source of profit. Solutions are being considered for both of these constraints. The process of community engagement through the use of community COPE will ultimately enable community leaders and managers to focus better on the need to maintain contraceptive supplies. This does however need to be a specific focus of the community COPE exercises. In regard to the price: there have been discussions at senior levels concerning the fixing of prices across the country. Earlier this year a guideline was published to the regions and districts on this subject (but seems to have been widely ignored). The PNA is strongly in agreement that contraceptive prices should be fixed, and at a sufficiently low price that it prevents the “price gouging” that takes place through private practitioners and private pharmacies. It is anticipated that once agreement can be reached on this subject that we would see the recommended prices printed on the packaging of contraceptives. As has already been demonstrated however, the problem is less the willingness of the central level to agree to a suitable pricing model, and more how to monitor the actual charges being made at the periphery. Once again the solution lies in the education of, and use of the community’s power to manage this, and the Project must address this directly in community COPE exercises. 4. Social Marketing and Logistics The Project’s logistics volet has little to do with the social marketing of contraceptives. There are some very real issues that arise within Senegal that relate to social marketing and which greatly impair the ability to expand to new markets. Most significantly is the prohibition on marketing any brand-name product. This means that only generic versions 24 can be marketed, or marketing can only advertise “oral contraceptives” or “injectables”. While this is not insurmountable – ADEMAS has been able to market over 40,000 cycles of its product Securil, it is a serious set-back. Where the closest overlap between the Project’s logistics system and social marketing will arise is amongst private practitioners who have the opportunity to purchase drugs directly from Regional Depots as well as socially marketed products. At this point in time contraceptives are not part of the 30 drugs that private practitioners can purchase, but it is only a matter of time before generic versions will be included. 5. Other logistic issues In addition to contraceptive logistics, the project is likely to be involved in two other areas of drug logistics. Firstly the distribution of SP for Intermittent Preventive Treatment of malaria will need to be dealt with. The policy is new, and the mechanism whereby distribution will be carried out has not yet been decided by the PNLP. There is currently discussion going on as to whether this should be initially established as a parallel system to ensure supply, and integrated into the essential drug system at a later date, or whether it should be integrated immediately. The project’s input will be valuable in the debate with the PNLP and it would be advisable to suggest an intermediate approach. If SP were integrated into the contraceptive logistics system – with its rapid improvement in eliminating stock-outs and its focus on women’s health, then the highest possible immediate impact will be achieved. As the plan is for later integration into the essential drug system, then sustainability of the SP distribution system can be assured in the next three years. The other, and far more complex issue is that of the distribution of antiretroviral agents and drugs to combat opportunistic infections. As the Project expands to include the prevention of mother to child transmission (PMTCT), this will potentially become an issue. At this point it is mostly theoretical, and the project needs to actively engage with USAID, the DSR and the DLCS in strategizing this approach. Given the limited experience in Senegal, it will be very important to draw on external, more experienced resources such as the RPM Plus project. CLINICAL CARE Strategic approach The project is building on the approach established by the predecessor SCS/PF Project. One of the observations of the SCS/PF project is that while it was able to develop excellent curricula and training tools, there was insufficient time to implement training and establish new skills. Analysis of this problem makes it clear that while part of the problem was simply insufficient time, there was a fundamental flaw in the overall training approach being followed. In response to the analysis above, the Project has taken an extremely innovative and bold approach to improving the quality of care. Under the direction of the DSR, the Project 25 has broken with the traditionally established training approach that has been prevalent in Senegal for many years. In the past, it was normal that whenever a new intervention or technology was introduced, or a change in policy was made, it was introduced in the form of a wide scale training program. Frequently a Training of Trainers approach, with cascade training was used, but in each case, the training took up to 45 days. It inevitably took place at a site remote to the place of work, and rarely received regular, sustained follow-up. The result was significant disruption to the health care delivery system, weak results in changing health care provider behaviors, and significant waste of human and financial resources. At the outset of the SM/PF Project, it was determined to follow an approach that focused on the long-term development of skills that would result in sustained behavior change by health care providers to improve the quality and scope of care. The main elements of this approach consist of a short introduction to the basic skills required (in 5 – 6 days), followed up by a program of “Formative Supervision” which takes place over an extended period of time in the workplace. The ultimate goal being that the basic skills are practiced in the period between training and first supervision. At the first supervision, problems encountered can be discussed, and application of the skills observed. From this point onwards an individualized competence improvement program can be developed and implemented at the regular supervisory meetings. This has also been supplemented by a pilot distance learning exercise which will provide radio broadcasts directed to care providers who have received basic training to update their skills and knowledge and link with the supervisory circuit. The other innovation in developing quality is related to the FFSDP strategy, but involves the aggressive adaptation of COPE to include the community at health center and health post level. By laying out minimum quality standards that refer to the 10 basic rights of COPE, the community is actively engaged in first understanding what constitutes quality care, and then is drawn upon to rectify structural and socio-political elements that impair the maintenance or achievement of quality. As the process develops, the decentralized power of community will be increasingly drawn upon as a resource to achieve and monitor the quality of services being developed. It has been a very important strategy for the Project leadership that facilities must be capable of offering basic quality of services before widespread demand generation takes place. This strategy is based on the assertion that demand that is unmet leads rapidly to disillusionment and non-use of services, so that later improvement in responsiveness may be futile. For this reason, the focus on service provision and quality has been throughout the whole project area, while the demand generation has been more highly focused, especially around the pilots of the PONT D’OR approach. As the Project moves into the second half, we can reasonably expect to see much more broadly focused demand generation activities integrated with the service delivery activities. Observations 1. Skill development in specific clinical areas: 26 The project has developed new, short curricula in a wide range of topics in reproductive health: Prenatal care; Contraceptive technology; Policy norms and protocols; Emergency Obstetric Care; Norplant insertion; and, infection prevention have all been developed and introduced at regional, district and post level. Curriculum development is in process for Intermittent Preventive Treatment of Malaria in pregnant women (Based on the new strategy for the country), and for Post Abortion Care (based on the addition of this as a new element in the Project. In addition, the Project has developed a training program for district and regional leaders in formative supervision, the use of the supervision grid, the introduction of the training manual for community based care, and the use of blood and blood derivatives. As outlined in the strategy discussion in the previous section, the Project’s curricula have been specifically adapted to the short-course with follow-up format. Training has taken place throughout the majority of facilities in the project and a clear plan of action is in place for assuring that the remaining facilities are reached in the coming months. The quality of the curricula is excellent. It has been a collaborative process between the DSR and the Project, with other expertise being drawn on as required. Training has been a huge challenge for the project simply because of the number of individuals being trained and the logistics of conducing so many training sessions. As far as we can tell, the training has been of good quality, but there are several steps that the Project could take to either improve on the quality or better measure what it is doing. - Identifying a set of competent “associates” who are trained in MSH’s overall approach to skill development, so that we can assure a uniform quality and standard of training when using non-staff for training; - Establishing a slightly slower pace of training to allow for reflection, modification and better preparation between sessions; - Introduction of a “training assessment” at the end of every training session to enable participants to provide direct feedback on the quality of training as they perceive it. The project has established a database of all facilities at which staff have been trained and the individuals trained at those facilities (classified by their professional standing), and this is directing further training efforts. The data base developed for this volet is now being adapted by the other volets of the Project. The Project has to resolve how to conduct further training during the life of the Project. As the emphasis of Project moves from training to formative supervision, the limited staff cannot conduct all the basic training and lead the development of the formative supervisory system. One suggestion is the basic training be contracted out to an organization such as CEFOREP. The concern of staff is that by doing so, they will lose the close contact that is established during basic training which can then be continued into supervision. It is agreed that while this concern is valid, our recommendation is 27 that he Project should move during the course of the next year to increasingly contract out training. This has significant advantages in that it contributes significantly to the sustainability of the intervention, it enables more efficient use of limited funds, and it reduces the burden on Project staff. 2. Supply of equipment The Project conducted a thorough inventory of the equipment available in all health centers and health posts, and in under the direction of the DSR a complete list of essential equipment was developed for Reproductive Health. On the basis of this list and the inventory, an equipment plan has been developed for all the Project facilities, equipment procured, and is in the process of being provided during the on-going formative supervision process. There has been some confusion and delay in this process however. Because of the overlap between the three USAID Projects (DISC, SM/FP, and HIV/AIDS), there have been cases where duplication of equipment has occurred. For this reason, the Project delayed its final procurement until this year to ensure that it was able to ensure that there would not be duplication of equipment and that gaps could be filled. It is reasonable to expect that at this point, all facilities will have the physical capacity to deliver services. However, the question of sustainability is high on the minds of staff members. A single injection of equipment, without plans for amortization will simply result in failure as the equipment breaks down. An external agency purchasing recurrent cost items – registers, stock cards, etc., is only effective for as long as the external agency is present. For this reason, a specific focus of the Project’s interventions with the community related to quality has to include specific plans to maintain and reequip facilities, and also to cover the recurrent costs. 3. Development of tools The project has developed or modified a number of tools that relate to service provision. Drug Logistics tools are described elsewhere, the Reduce Model has been translated into Arabic and Wolof, and training curricula have been specifically adapted, manuals have been developed for community-based distribution. However the most significant piece of work in the last year has been the elaboration of a supervision grid. The supervision grid is somewhat misnamed. It is not simply a “checklist” that enables the supervisory team to confirm the presence of absence of a required element. It is in￾fact a tool that guides the supervisor and the facility staff through a process that covers all the elements of quality care provision. The tool has taken a considerable time to develop. It derives from the supervisory experience of the project in the first two years, and builds on the norms and protocols established in reproductive health. It also incorporates both the service provision and community involvement sides of the FFSDP concept. It is the core of the formative 28 supervision process that was introduced this year, and is likely to become the official Ministry protocol for supervision (after review and modification by the DSR). At first glance it appears to be an unwieldy tool – running to over a hundred pages and necessitating jumping from one section to another. However it has now been field tested in 164 sites with remarkable ease of use and effectiveness. The project has developed a mechanism for analysis of the data using a spreadsheet-based system, and is in the process of modifying the tool on the basis of the experience. At this stage the tool has allowed the establishment of a highly detailed baseline of the performance of centers and posts in the Project areas, and will be a key tool in evaluating the impact of project interventions. It will also serve as a planning tool and a guide to supervisory teams in prioritizing their efforts. There is scope for experimenting with the use of handheld computers in the application of the tool. This could be a very cost effective means of gathering data, would save considerable time in data entry into electronic format, and would greatly facilitate analysis and possibly action during the supervisory visit. This could be increasingly important with future supervisory visits when comparison with previous visits is important. 4. Process of formative supervision The process of formative supervision was first introduced into the Project in 2002 as part of the new training strategy. Based on that experience, the Project and the DSR has developed a much more formalized system of formative supervision using the supervisory tool mentioned above, and linked to training. Starting in June 2003, the Project began a process that will result in every facility receiving four supervisory visits per year. Two will be supported by the Project and conducted in collaboration with the DSR staff, while two will be conducted by the Regional and District teams themselves. The process consists of seven steps 1. Observe the facility 2. Collect information on functioning of the facility 3. Look at the management capacity 4. Look at Service delivery and performance 5. introduce the Inventory Management Assessment Tool 6. Conduct a combined facility/community COPE 7. Conduct training on the Prevention of infection The preliminary results of the implementation of the process at the first 164 centers are encouraging. The use of the methodology has signaled begun a fundamentally new process to all levels of the health system and to the communities involved. For the first time in Senegal, a long-term perspective has been taken to skill development and 29 maintenance, while at the same time the community has been systematically and formally involved in accepting responsibility for the quality of service provision. At this point there are observations by those participating, and there is qualitative data suggesting that the impact of the exercise has been profound. However it will have to wait until the follow-up detailed exercise early next year to begin to document changes. It was related by DSR staff that “Once you start the change in focus it is very hard to put reverse it. When a care provider get help and not control, they will demand more and more of it” The process has certainly had the following positive effects: • There is a new engagement between health staff and community. • As a result of the supervision problems were actually solved, not simply identified. • COPE was a very important relationship building exercise and produced real and manageable products • The community has become more financially involved with the health center • Each SDP now has a realistic and manageable workplan The process is not without its difficulties: • Political leaders such as Mayors were often unavailable and without them the process cannot be effectively implemented. This can cause enormous scheduling problems. • The process must be kept “light” if it is to be sustainable. At this early stage there are huge project inputs and support to get the process moving, but the DSR and MSP must be able to assume the oversight role within the next 18 months and th project’s role switch to a supportive role. • Supervision is still not regarded as a technical priority in the MSP – it is still in essence a Project intervention. It must be institutionalized as soon as possible – the Project must determine who at the MSP or DSR would be the right champion of this approach. • Related to the above points, the competence either exists or can be developed to run a formative supervisory program in the MSP, but resources must be allocated to ensure that activities continue after the end of the Project. 5. COPE at facility level and extending to lower levels in the system The Project’s decision to use COPE extensively has been a very good one. There has been a considerable variety of experience in the Project in applying COPE. This has involved health provider only COPE, community only COPE, and integrated community and facility COPE. The lessons learned from the six Pont d’Or sites where community and facility COPE were conducted separately and action plans reconciled, has been important in understanding community perspectives and in developing a framework for community involvement. However, the combined approach introduced in the formative supervision is more practical and should be the model for movement towards achieving FFSDPs and leading into the Pont Bronze, Pond D’Argent, and Pont d’Or program. 30 There was some considerable discussion about the level to which COPE can be introduced in the system during the evaluation, and it is our recommendation that the Project encourage application of COPE down to the lowest facility level in the system – the Case Santé. The Project has already created the tools to be able to do this with the development of a large Boîte à Images. This tool can be used by the health post nurse in facilitating COPE at the lower level. This approach has the advantage that the skills to manage cope will have already been developed at the Post level, and will not draw on additional resources from the Project. The evaluation team took part in one of the first follow-up visits to a health center following the original formative supervision at Joal. We were met by a large contingent of both health staff and community members, including both the health management committee and the political leaders from the Commune. It was explained to us by the Mayor that the COPE had been a “ground breaking” exercise. That he and the community had: - Been helped to find out that there were problems that the community did not even know existed; - Been given a powerful source of motivation to improve services; - Been encouraged to change the behavior of patients as well as changing the behavior of center personnel; - Been given the tools to improve patient satisfaction; - Been given an entry point into participating with the health staff; The commune feels that they have seen visible changes in the last three months in the way in which the center is run, while the health staff members have seen dramatic changes in the resources available to them and the physical infrastructure – renovations, new construction and vehicle repairs. 6. Developing a new culture of supervision There are several important constraints that also apply to the supervisory capacities and these mostly relate to the change in culture from one of controlling to facilitating and training. There is no doubt that the staff members who were recipients of formative supervision found it desirable and powerful and want it to continue. However they remain very uncomfortable about being supervisors themselves, and until we can assure that district level staff will be competent and motivated supervisors, the intervention will not be sustainable. In addition, unless we can address this at the facility level, internal supervision will not become a priority and problems such as those observed with the essential drug program will continue. Analysis of the problem reveals several specific problems: - Unfamiliarity with the Process - Lack of appropriate role definition - Lack of technical competence and/or experience - Lack of leadership skills . 31 The problem of unfamiliarity and discomfort is easily resolved by facilitating and accompanying supervisors until they have developed confidence. However this is greatly complicated by the fact that many MCD are not trained for the supervisory role that they must play, nor does their role allow for this. The current role of the MCD is predominantly administrative, and yet the supervisory needs at the facility are predominantly clinical. If the MCD or supervisor is unable to provide input based on clinical knowledge or experience, they become of diminished value, and are in many cases are looked on with some disrespect or mistrust by health post staff. The Project is not able to actively redefine the role of district staff, nor can the Project engage in debate with professional societies about what competence is required to conduct particular clinical procedures. However, the project should remain actively engaged with the DSR and professional associations to attempt to formally change the role of the MCD to that of clinical supervisor of staff at the peripheral level and not simply an administrator. The project could then participate in developing the skills necessary to play this role, but ultimately it would require a different focus at training schools to develop this capacity. The issues are not however simply technical and clinical. A strong, inspirational leader is capable of providing guidance, input and help, even if they lack specific expertise. A good leader is also able to plan with, inspire, and engage a team. Leadership skills are uniformly lacking amongst District staff. This is because staff have not been taught, not because they lack overall competence or motivation. In discussion with Project and DSR staff it was agreed that the most pivotal intervention to improve the capacity in supervision and to initiate a sustainable change in culture would be to initiate a program of leadership training for the MCD. As with all other training in the Project, this should not be a single event, but part of a process, that is started with a formal training event, but includes some form of distance learning (perhaps through electronic means – all districts have computers and so can use CD ROM or e￾mail), or though radio – which is being used at lower levels in the system. It is to be hoped that the Management and Leadership Program could be recruited to provide assistance to the Project. 7. CBD (Reproductive health Community-Based Services) The project has been very involved in the pilot program to test the value of CBD as a means to expand access to contraceptives. The pilot in Kebemer has shown that there is a demand for CBD, and while there are some questions about the findings of the pilot based on the demographics of the area studied, and some concern that contraceptives are not as popular as other products, it seems entirely appropriate that the project should consider adopting a CBD approach on a wider scale. There are no barriers to CBD and with the advent of the ARPV in the new year the opportunity will present itself. The only caution is that the Project should engage in a relatively small effort to begin with until the cost-benefit can be determined. 32 8. ARPV, PC and outreach There is general recognition that the concept of the ARPV is a powerful one that can make a significant difference in the ability of multiple projects to have community level impact and to increase access points. There is also frustration about the fact that it has taken two years to get to the point where the ARPV may be functional. This frustration is shared by all parties, USAID, Projects and MSP. There has been excellent collaboration in developing the concept, and it will be exciting when the program does come into existence in the coming months. The ARPV in reproductive health will have an important role in community education, outreach, CBD, and advocacy. They will be important arms of the facilities in the community, and with the program as it is anticipated, issues of motivation will be largely addressed. In the interim, while the ARPV program has been under development, the Project has paid special attention to implementing a model ARPV at the 6 Pont d’Or sites. This has provided valuable experience and has enabled the project to plan for the ARPV program effectively. Perhaps more importantly, the Project has developed a new class of community workers that will need to be incorporated into the ARPV program. The Persuadeurs Comaunitaires represent a novel and powerful tool in providing maternal health outreach. By identifying pregnant women in their community and taking personal responsibility for their presenting to the health services and safe delivery, the PCs provide a mechanism for significantly improving the quality of ante natal, intra-partum, and post-natal care. Initial results from the work of the 200+ PCs that have been recruited demonstrate a success rate in excess of 90% in identifying pregnant women in the community and ensuring their full care at the facility. If this can be replicated throughout the project areas as the ARPV program comes into being, then we can anticipate dramatically altered service provision statistics, and can also anticipate an impact on maternal mortality. 9. Private Practitioners There is reason to believe that by engaging with private practitioners the Project would be able to significantly increase the number of access points at which both family planning and maternal health services are available. There are two types of private medical/nursing practitioners in Senegal (outside of traditional practitioners). Private doctors are mostly restricted to large urban and serve a relatively limited population. Retired nurses (Infirmiers Privées) have set up community practices in both urban and rural areas and appear to have access to large populations who have limited geographic access to public facilities. This is especially true in the smaller urban centers. The Project has undertaken a year-long pilot with Privées in Kaolack that has enormous potential, and demands significant attention towards replication in other centers. These nurses are retired from public service, but are encouraged to continue practice in the 33 community. Working with the District health office, the Project has successfully recruited the Collectif des Infirmiers Privées in Kaolack. This group represents 48 private practices throughout the city and outlying areas, and serve a significant population (each sees up to 500 patients a week). After conducting a census of Privées, skill development was carried out through a series of workshops and training sessions in maternal health and family planning. During this time, reporting and referral mechanisms were developed between the Collectif and the District. At the end of the year, an assessment of all the Privées was made and official licenses to practice issued by the medical district. This served two purposes: 1. It assured that all those who were licensed had an appropriate level of quality in service delivery; and, 2. It enabled the District to stop unqualified and incompetent practitioners from practicing. At present 16 of the original 48 are licensed and working closely with the district. They submit regular reports, make referrals to the district and receive back-referrals. They have also begun to participate in the family planning program, and would be an ideal site for basing both PCs and ARPVs in the community. The Project needs to consider replication of this initiative in all other smaller urban centers in the project area, and also needs to explore with the DSR and MSP as to how the licensing process could be adjusted to enable those Privées who don’t currently meet the criteria can be brought to a standard where they can practice. 10. Challenges: a. IUD vs Norplant USAID has expressed a concern that if Senegal is to achieve the desired CPR, then continued use of Norplant is not realistic because it costs too much. An alternative long-term method would be the IUD. The question of whether the IUD can be re-introduced in Senegal as the preferred long-term method of contraception is very hard to answer. It was very popular, but also very controversial. The complications of the use of the IUD, (e.g. uterine perforation), while rare, tended to be highly visible (as opposed to mal-insertion of Norplant, which rarely makes national news). Thus there was always a PR struggle. The advent of Norplant, and the complete dropping of the IUD, not only has driven awareness of the IUD down, but has had a negative reinforcing effect – because the health services no-longer promote it, it has confirmed in the minds of many women that it was a “bad” contraceptive. Thus the IUD could not easily be introduced without a very carefully crafted IEC campaign which starts with careful formative research into perceptions about the IUD, and moves onto promotion. 34 However there is an equally large problem with the supply side. Because the IUD is so little used in Senegal at this time, there are very few clinicians who have current practical experience in their insertion - even those who have been trained in the past lack current experience. Thus in order to meet any form of demand that would be generated, extensive training and practical experience would need to be generated. In essence, the program would have to be built from “scratch”. It would involve training, study tours to develop skills in a country where there would be sufficient numbers of IUDs being inserted, and then an extensive IEC campaign. There is also the consideration that because the program was allowed to languish, with some negative feelings, that it may be harder to re-introduce the IUD than if the IUD was introduced from “scratch”. b. Natural family planning There was some speculation as to whether the demand for family planning services can be met promptly just using modern methods, and whether there might be advantage in promoting natural methods more actively. There is some merit in this point of view and there is definitely need for increased emphasis on natural family planning methods. The recently completed Annual Supplemental Survey indicates that the CPR for natural methods is only 1.5%. This is despite the fact that natural methods are actively endorsed by both Islam and the Catholic Church. Between them, these two religious groups cover the majority of the Senegalese population, and it would certainly behoove the Project to more actively promote natural methods. The favored method is the Rhythm method. This recommendation however is not to suggest that there be any change in the emphasis on modern methods. Modern methods remain eminently superior to natural methods in preventing pregnancy, and natural methods play no role in preventing HIV/AIDS or the transmission of STIs. We suggest that education of health workers on natural methods receive additional emphasis in the Family Planning curriculum, and that it be among the choice of methods offered during family planning consultations. However, in the family planning consultation it should be stressed that natural methods are unreliable, and should only be a choice if the woman is unable to use an other method (because of pressure by husband or religion), or is not concerned if she becomes pregnant. c. Malaria The project became actively involved in the malaria program in Senegal as concerns about the high rate of infection in pregnant women grew, and as a 35 response to the high maternal mortality rate in the country. The Project’s interventions have been mostly at the central level, but the Project’s value to the PNLP has been its decentralized nature and the close contact with the community and service delivery points. The Chief of Party has been an active participant in the PNLP treatment policy group which recently made the decision to switch from Chloroquine to SP for treatment of pregnant women. This was not an easy decision, and the project played a noticeable advocacy role through providing sound MCH data and international evidence (especially in collaboration with RPM plus staff). The Project is currently involved in revising the training curriculum in IPT to accommodate these changes, and has already included key indicators of IPT into the supervision tool that has been tested at 164 centers. There will be a considerable burden on the Project as the new policy is put into action. At present decisions have not been made about the specific logistics system that will apply to SP, but even though there has been a communication with doctors and nurses about the changed policy, there seems to be poor compliance because of confusion about previous contra-indications for the use of SP in pregnancy. There will need to be careful education of peripheral staff to overcome this confusion. The PNLP has also asked the Project to facilitate the establishment of a sentinel surveillance system using the network of community centers that the Project has been able to establish. d. PMTCT The Project is in the process of engaging actively with USAID and the MSP in the expansion of the PMTCT program in Senegal. The evaluation team held several discussions on this topic with Project staff, FHI, and USAID. The only thing that the team can state clearly is that there is enormous confusion about how to proceed with the program! The pilot conducted in Dakar has led to extension to Regional centers, which include three of the Project Regions. For this reason, the Project will become actively engaged with the Program, however, the how of this is not clear – mostly because the government’s policy is not yet clear. In follow-up to our discussions with USAID, the results of a strategy brainstorming was shared with the Project staff. The Project has been asked to provide input that can be the basis for further discussions with USAID, other CAs, the DSR, and the DLCS. MSH’s HIV/AIDS team will be requested to provide specific and targeted input and support to the Project staff. 36 BEHAVIOR CHANGE AND COMMUNICATION Strategic approach The Project’s strategic approach in BCC is based on the following findings: • 29% of men (-proposal) give religion as a reason for not using contraception; Among women who do not use contraception and who have no intention of using it, 3% and 3,4% give religion and opposition to FP (religion?) respectively as a reason. • 11% find no access to family planning • 2% and 8% of those who have never used contraception and do not intend to in the near future complain about possible the side effects and infertility. The original strategy was centered around the formation of IEC committees at district and regional levels to help project implementation. This strategy however was delayed by the discussion around the ECS and ARPVs. The Project responded therefore by developing an alternative strategy focused on community dialogue to support the development and sustainability of the FFSDP. This has been articulated as the Pont d’Or approach. The approach aims to increase use of quality MH/FP services through a focus on both providers and community’s perceptions of quality. The strategy has been supported by a focus on broadcast IEC/BCC through capacity building and coordination efforts; journalist training in MH/FP, BCC material production; curriculum and strategy development. All demand efforts are preceded by advocacy activities aimed at getting buy-in by stakeholders (district/rural community administrative authorities, district health authorities, community members, and service providers). Observations 1. The Pont d’Or approach This approach targets the community and the service providers: AT THE COMMUNITY LEVEL: The community is empowered to materialize their rights to quality services using: 1. Participatory action planning and management with services providers to improve quality of services 2. Education of the community on the benefits of MH and FP services through: • Rural and regional radio programming to raise awareness of the importance of MH and FP services and encourage the community to use these services • Partnership with women’s groups to sensitize women about the benefits of MH and FP services and educate them to recognize signs of danger in pregnancy and take action. 37 • Knowledge and skills improvement for community Health workers to conduct interpersonal communication activities in the community on MH and FP (traditional training + distance learning refresher) AT THE SERVICE DELIVERY LEVEL: Service providers are empowered to assert their rights and value those of the community through: 1. Joint action planning and management with community members to improve quality of services 2. Skills improvement: clinical, client-provider interaction, and logistic management skills 3. Formative supervision: to support clinic skills improvement 4. Outreach (journees SR): to increase access to MH/FP services Pont d’Or was initiated in 2001 as a pilot in 3 service delivery points at first and then expanded to 6 service delivery points throughout the regions of Louga and Thies and combines the COPE and the JHU bridging approach. Implementation is done through the following steps: The bridging exercise: Using the community COPE, service providers and community members (youth, women, then men) separately, identify blocks to quality in their health structure. These problems are analyzed based on each group’s perception of quality and the clients’ rights chart developed by IPPF. Preparation of separate action plans based on the problem identified in COPE. Each group prepares a quality improvement action plan. Merging of action plans in a second stage of facilitated work, service providers and community members are brought together to discuss their action plans and merge them into a consensual action plan with defined responsibilities and timeline. A quality commission of about 15 members is formed with service providers and community members to follow-up on the quality action plan implementation Relays are given orientation in MH/FP to conduct Behavior Change Communication (BCC) activities in the community around the Pont d’Or sites, in collaboration with service providers. Relays are supervised by ICP or MCD. Presently, 25 relays are trained around each Pont d’Or site. Pont d’Or accreditation/certification based on: (i) level of implementation of consensus action plan (80%),(ii) performance results in formative supervision, (iii) monitoring results. Six new sites are expected to be added in the Ziguinchor region to supplement the 3 in the Thiadiaye district and 3 in the Linguere district (see Annex IV). 38 Promotion of Pont d’Or sites The Pont d’Or sites will be promoted in local languages on radio, and during community mobilization events as FFSDP which are partners of the community and where they can get quality MH/FP services. Print materials featuring the Pont d’Or could also be produced for the health center and Relais working around the Pont d’Or sites. Strengths • The Pont d’Or strategy makes the FFSDP model concrete by making community and service providers actively involved in the improvement and use of quality MH/FP health services. • The approach allows true dialogue with the community and brings to the fore, for the first time, the community’s right to give their opinion of the health facility. The strategy restores confidence between community and services by creating a forum to express clients’ views about the quality of health services. Through this approach, the community is involved in decision making for health with the same level of importance as local authorities and health providers. • Most activities on the consensus action plan are funded by the comité de santé which is usually made up of community members. Follow-up on implementation of the action plan is supervised by the commission de qualité, which is made up of service providers and community members. The Commission therefore enables a high degree of community control and allows for coordination through their relation with the Relais. • The Strategy creates healthy competition between neighboring communities to improve the quality of their health services. • Although improvement of the community’s knowledge of the benefits of using MH/FP services, give them status and sense of worth in their community remains key to the motivation of the relays. Challenges In view of sustainability and scaling-up of the approach, three major weaknesses prevail: 1. Initial selection of sites to enroll in the Pont d’Or strategy is somewhat subjective, as it relies on the MCD’s judgment/ advice on which “sites are most likely to succeed ” if enrolled in the strategy. 2. The accreditation criteria of a Pont d’Or site are not clearly defined. How can we rationalize/quantify them for standardization? 3. The status given to relays through community and the sense of worth and responsibility that derives from their community work is not enough to maintain their motivation in the long-term. Recommendations In planning for scaling-up of the Pont d’Or approach and in view of the MH/FP project exit in 2006, the Pont d’Or approach needs to be standardized and packaged 39 to be property of the MSP/DSR for expansion to other non-USAID districts in Senegal. Present collaboration of the SM/PF project with the MSP/DSR and district health teams will build capacity at central and regional levels to replicate the approach if resources are provided by other donors. Most importantly we recommend that the Project combine the current strategy designed to achieve FFSDPs through a process of formative supervision and COPE, to a system whereby incentives are added to encourage facilities to move beyond the barest minimum standards, and a form of certification is introduced. This expanded Pont d’Or program is laid out in more detail in Annex IV. 1. Increasing demand for quality MH/FP services: Community interpersonal communication and outreach activities The MH/FP project contracted with various community based structures 1) Association des Relais de Kaolack (2001- 2003) MH/FP group discussions and home visits were organized with 3,800 women from 19 Centres de nutrition communautaire and 1,650 women from 33 Groupements de Promotion Feminine and Centres d’alphabétisation in Kaolack. Activities included expert talks, video viewing, home visits, and interviews. In total, 24,500 persons (the majority being women) were reached in the districts of Kaolack and Guinguineo. Similar activities were conducted in military bases with the association of military wives in Kaolack. Anecdotally, the number of women who come to the base health center for ante natal care has doubled. The Association des Relais also conducts interpersonal communication activities in Koranic communities such as the dahra of Bouchra in Kaolack which has more than 1,000 members: men, women, youth, children. 2) With Women’s groups In addition to the work done in the Kaolack and Fatick regions with women’s groups, and through the Association des Relais de Kaolack, the Project works directly with Federations of women’s groups in Thies, Louga, Mbour, and with Pont d’Or relays in Thiadiaye, to organize interpersonal communication activities in the community to disseminate safe motherhood messages predominantly among women. In total, about 17,500 persons were reached. The women’s groups IPC/C activities are centered around the Wure, Wer, Werle (W3) which deals with pregnancy risk factors and action to take if any occurs. 3) With traditional artists Traditional MH/FP BCC materials contests and award ceremonies were organized in Kaolack (2001) and in Louga (2003) to create traditional communication materials such as poems (taalifs) and songs (Taasu) and sketches on Maternal Health and Family Planning topics such as: the importance of FP for the family’s health; the importance of complete prenatal visits coverage for the health of the mother and child; the importance of avoiding the 3 delays relating to pregnancy and birth: recognizing the signs of danger/ delay in getting to health facility/delay in receiving proper care once at health facility; the importance of post natal visits for the health of the mother and child. These traditional materials are recorded on audio tapes and available for further community activities and 40 radio broadcast. The traditional BCC materials development events usually gather about 1,000 people. 4) Reproductive health days: (Journees SR) These events have been held in Thies, Linguere, and Louga. They take place over two days to increase access to RH information and services. Clinical services such as gynecology, prenatal, sonogram for pregnant women, post natal visits, family planning, are available. BCC activities include: counseling, group discussions, conference debates, and material dissemination. Evaluation of these 2 days events is done with “registres consultations” and ‘cahier des causeries”. In the first quarter of 2003, 2,016 persons were reached. Strengths • Gather a lot of people and gives an opportunity to disseminate MH/FP messages and discuss them to help understanding and action such as use of service or further advocacy for the services • Advocacy opportunity as religious leaders, and other community leaders are invited. Their participation may result in support for MH/FP services and their use. • Service providers are invited to the events to ensure that the information disseminated is accurate and complete • Opportunity to involve women and empower them to take part in activities other than health: in Thies, women were encouraged to participate in local comités de santé management in order to better impact their own health. As a result, several women had been elected president Comité de Santé, many others had positions as important as Tresoriere de Comité de Santé. • Women’s groups are strong forums where women can reflect together, tap in their networks (e.g. Femmes Islamologues) to do something about the messages they receive. • Collaboration between health providers, outreach personnel, and district/regional medical team • Invitation of community leaders gives opportunity for advocacy for MH/FP services. • Provides access to MH/FP services to a multitude in a limited time. Challenges In relation to recording the impact of the work conducted with the military: The MCD at the military base acknowledged increase in the number of ANC visits in relation to the IPC/C conducted with women in the base, but these data are not fed to the health district general data base. The promotion of the Wure, Wer, Werle (W3) use was stopped by a conflict with the resource person that created the play related to his non-conformity to our consultant hiring policy. 41 Recommendations To continue community mobilization activities as they allow the Project to reach an important number of people and anecdotal evidence shows that they have a positive impact. Efforts can be made to develop more effective quality BCC materials to support Relais, women groups in their community activities Military Relais and Persuadeurs Communautaires can be trained in the military, gendarmes, and firemen bases, to increase the impact of FP and MH messages in these communities A similar IPC/C approach can also be developed with public transportation drivers (Syndicats des Transporteurs in the Dakar region for example) Associations of wrestlers, of fishermen, and in religious communities In remote areas where they are present, the project could collaborate with peace corps volunteers for community mobilization and outreach activities Research other resource persons to continue women’ group training in the Wure, Wer, Werle (W3) use. Design a similar play using our IEC materials (danger sign cards). 3. Mass-media Radio broadcasts The SM/PF project contracts with various public and private radio stations in Kaolack, Louga, Thiès. These include SudFM, Dunya FM, RTS and communty radio stations:, Dieri FM Keur Momar Sarr Louga, La cotiere Joal, Penc Mi Fissel Thiadiaye. The programming includes: - Radio spots on RH, - 16-episode radio soap opera on Men’s responsibility for their family’s health, - Radio debates on Islam and FP, - Traditional poems and songs on the benefits of RH services, - Radio programs on FP/ANC/attended birth, post natal care/prevention of malaria in pregnancy. Radio program formats are varied based on the radio station: For example in RTS Thies, Fissel, Sud FM, and Kaolack they are interactive with invited experts and call￾ins); In Fissel they are non-interactive – recorded and rebroadcast. (Impact evaluation of Keur Momar Sarr and Mbour M/FP radio programs) Decentralised reproductive health programs: Antennes SR decentralisees In Thies, Thiadiaye, Mbour, Fissel, Kaolack, Louga: radios cover community activities organized in collaboration between women’s groups and service providers. Local radio stations record the events, and rebroadcast them. During these events, MH/ FP messages are disseminated. Local community, administrative, and religious leaders are invited and often make statements supporting MH/FP. The community event is recorded by a radio journalist who will later make arrangements to broadcast it several times. It also 42 happens that the show is broadcast live and later rebroadcast. Approximately 100 such programs have been developed to-date. Distance learning radio program for Relay workers The program has been designed in collaboration with DSR, other ministries, and other USAID cooperating agencies: ADEMAS, BASICS, FHI, regional offices of Education pour la santé and community health workers. The program consists of 26 episodes on various reproductive health topics each. The main themes of the program include: maternal health and prenatal visits, malaria prevention in pregnancy, child survival, family planning, adolescent reproductive health, STI/HIV/AIDS, behavior change communication, rural taxation, and decentralization. Each theme is treated in one or several episodes. A participant’s handbook will be produced, and for each episode, the handbook will present to the Relais the key points to retain, and a practice exercise. The following episode will include a review of key points in the last episode and answers to the question and exercise. A follow-up system and supervision will be put in place to support the relays. The program will serve as refresher training for relays who will be trained with the traditional Relais polyvalants integrated IPC/C curriculum. The program is presently being produced after pre-testing of the first two episodes with relay workers and health service providers around the health posts of Ndiaganiao and Fissel. Analysis of the results have shown that the program is acceptable in their socio-cultural context and is understood in terms of content, language. Relays found the program attractive: they liked the “enter-educate” approach Strengths • Community ownership, involvement, and local initiative: Women’s groups and or relay workers (around Pont d’Or sites) work with the radio stations and service providers for programming of the radio shows (Fissel radio was created by a villagers’ association) • Radio has large coverage: a station can cover several kilometers (Fissel rural radio covers an area of 50 km in which the estimated population is about 2 million inhabitants. Although the station is based in the rural community of Fissel, District of Thiadiaye, Region of Thies, it can be listened to in the regions of Fatick, Diourbel, Dakar) • Communities react to programming through phone, letters, and in person • There is a perceived Increase in prenatal visits rate due to both the community work done by relays and the radio • The radio programming shows positive impact in changing perception about FP and increasing safe motherhood practices1 • Radio distance learning program will allow both community health workers and community at large to get health messages and will contribute to providing an improved status to the relays. Challenges Some of the radio spots broadcast are outdated (for example: that on the locations where you can get FP services) 43 Recommendations Radio is a critical communication channel in Senegal where illiteracy rates are high especially among women. The Project should capitalize on Rural and community radio which is indispensable in the dissemination of accurate and complete MH/FP messages. Moreover, Radio is culturally accepted in Senegal where exists an extended culture of oral tradition. Also radio is effective in disseminating messages in local languages in rural areas to make sure that they reach those who need them most. Most community health personnel involved in supporting services in health posts (ASC, conseilleres, TBAs) are not literate and could benefit from information on radio: this can be achieved through the radio distance learning program in production. Also a radio show specifically addressing performance in technical areas for providers “provider to provider” show on clinical or counseling advice dealing with FP or MH in collaboration with local community could benefit health personnel. 4. Client-Provider Communication(CPC) Interpersonal communication and counseling training for service providers and health centers/health posts counselors has been taking place using the integrated reproductive health interpersonal communication and counseling training curriculum developed in collaboration with DSR, Plan, ASBEF, SANFAM and reproductive health regional coordinators. The training was conducted in the disctircts of Kaolack, Thiadiaye, Bignona, Oussouye, Linguere, and Louga. 223 service providers, including military, and private service providers in the district of Kaolack. 280 frontline counselors were also trained in Interpersonal communication and counseling. A training was also conducted in the Sakal to improve PLAN’s community health staff counseling skills. Under Pont d’Or, 25 relays were trained in interpersonal communication and counseling in each district where the pilot is being implemented. Strengths: Interpersonal communication and counseling training allows access to a large number of community members with MH/FP messages. Recommendations There seems to be a lack of IPC/C support materials/aids for service providers and community health workers to use in the health centers and during community outreach mobilization activities. The project could reproduce FP flipcharts in appropriate sizes and make them available to the community health workers for interpersonal communication activities in the community. Also, the brochure on risk factors related to pregnancy and birth could be adapted as a standing flipchart more suitable for IPC/C activities in groups. Given that knowledge of FP is very high (90%) but CPR remains low, the GATHER (BERCER) model of FP counseling could be adapted to include profiling questions in the Ask stage to identify what method will most interest the client according to her profile: birth spacer, breastfeeder, limiter, multi-partner. Posters reflecting this could be placed in counseling rooms of health centers to assist providers and counselors in counseling clients. Also Cue cards on the various FP methods with description of the method, how it works, the advantages and benefits, side effects and disadvantages and a list of frequently asked questions and answers 44 to assist the provider respond accurately to queries. Simple pictorial leaflets on the various FP methods could be produced to give to clients who would choose the method. The generic pamphlet on the various methods can be simplified to give away to clients who have been counseled but need more time to choose a method. Prenatal and post natal Counseling cards to assist providers can be developed to make sure providers cover all aspects: such as the reason why ANC and PNC are important, what to expect during ANC and PNC visits, what to do…. Finally, given that the duration of counseling training for service providers was shortened from 21 to 6 days, regular supervision of counseling services should take place to ensure the quality of the counseling services 5. IEC/BCC materials The project has produced a number of important materials: Print materials A calendar with 26 risks factors associated with pregnancy and delivery was produced for community members, community health workers, and service providers. The calendar has been adapted as a booklet for community health workers and service providers. A similar calendar for FP was then adapted in booklet was developed for service providers and relay workers. A pamphlet describing the various FP methods was also develop and adapted as a brochure. A brochure on malaria prevention and treatment for the community includes a description of chloroquine posology and encourages the use of bednets. Posters: • on malaria prevention using bednets and one on use of chloroquine were developed. • Posters developed under the SCS/FP project have been reprinted for health centers: Ismael Lo poster encourages men to speak with their wives about FP; • Poster encouraging men and women to seek FP services; poster encouraging men and women to use a FP method; One last poster encouraging women to go for ANC. • An IPC/C kit with samples of the various FP methods and their description was developed for the service providers and relay workers. Audio and video materials • The songs and poems (Taasu, taalifs) created by traditional artists in Kaolack were recorded for distribution to relays and radio stations for broadcast. • Spots on the benefits of FP, where to get services, an “argumentaire religieux” on Islam and FP, and a 16-episode radio soap opera produced under the CS/FP project are being broadcast. 45 • Interactive radio programming done with the various radio stations and the recordings from the “Antennes SR decentralisees” (which are sometimes broadcast live) are re￾broadcast. • Video materials developed under the SCS/PF project have been made available which deal with FP, Prenatal care, and safe delivery. Games The Wure, Wer, Werle (W3) was developed with the 26 risk factors related to pregnancy and delivery: players need to know the risk factors, know how to search for them, and what to do to reduce the risk. The game is effective in raising awareness among women about the risks factors, knowing how to check for them and what to do when they are exposed. The game is distributed to women’s groups who organize tournaments. Promotional materials such as t-shirts and cups have been produced Recommendations Although most of the materials produced under the CS/FP project, are still current in terms of encouraging couple communication on FP, use of FP and antenatal care services, messages could be more targeted in terms of concrete actions to be taken by men who usually hold the decision-making power. For example, men and religion are often mentioned as a reason for the disconnect between high FP knowledge and low use of FP. Radio/ TV spots/infomercials and print materials (billboards, giant poster stickers for public transportation) targeting men and religious leaders on the direct benefit of FP and MH for their family could be developed. Messages on the 3 delays related to pregnancy and birth, antenatal and post natal care as well as birth preparedness can be featured by the materials produced. The religious leaders’ FP kit developed under the CS/FP project will be reviewed, updated and complemented with MH messages. A 30-mn entertainment-education video targeting women and men can be used as a center-piece for TV broadcast, viewing, and debate in community mobilization activities. This video can feature a story on safe motherhood in the Senegalese context (family-in-law, religion…) and can include messages on prenatal care, the 3 delays, post natal care, and family planning. The video will be in Wolof, possibly sub-titled in French. The same story will be recorded on audio tape for distribution to relays, health centers, public transportation, and radio stations for broadcast. BBC activity coordination Development of a Maternal Health Behavior Change Communication strategy in collaboration with the Direction de la Sante Reproductive and other directions of the ministry of health, regional representations of Direction de l’Education a la Sante, various NGOs and other development partners. The strategy centered around the 3 46 delays model for safe motherhood complements the FP strategy developed under the CS/FP project. In collaboration with DSR and other USAID cooperating agencies, the MH/FP project developed an integrated reproductive health IPC/C curriculum to train service providers and an IPC/C curriculum for Relais polyvalents to train members of the ARPVs. The radio distance learning mentioned above will serve as refresher training to the traditional approach. Finally, despite many efforts being done by the MH/FP project to coordinate field activities with ECD and ECR, there appears to have challenges of workplan integration and implementation at the community level. Recommendation The draft MH BCC strategy will be combined with the FP/STI/HIV/AIDS strategy developed under the CS/FP project and sent to the DSR and MH/FP project for review and validation before finalization as a single document. To facilitate coordination of demand generation activities in the field, local IEC/BCC working groups could be put in place under the supervision of the ECS ECS –Equipe Communautaires de Sante. The regional, district and rural IEC groups will be made up of representatives from private and public sectors and community based organisations such as traditional communicators (griots, theatre troupes, groupements de promotion feminine, ARPVs…) The local IEC/BCC working groups will coordinate local planning, and implementation of all local actors’ IEC interventions. By being involved in approving, designing, and implementing IEC activities, the group will develop capacity in IEC (participatory approaches, message materials design…community mobilization), and be able to continue their role locally and initiate IEC activities when needed for the community, even when projects end. The increased coordination and communication between local partners will allow to avoid activity overlap in such instances as training, mobilization of resources, and cost sharing for efficiency. This coordination body will facilitate and coach field work in their various communities and will , Provide local leadership and advocacy and support for community involvement and buy in for BCC and advocacy activities. General recommendations on BCC The time spent in the field during this evaluation made it clear that men and religion remain determinant in the decision-making process and access to FP and MH services. These findings were partially confirmed by the Rapport d’enquete supplementaire dans 15 disctricts sanitaires de l’USAID- Octobre 2003 47 Fir FP, it is reported a rate of 93% general knowledge among rural and uneducated women 1 while, use of FP services remains low: total Contraceptive Prevalence Rate remains at 9.8% for modern FP. On the other hand, it is worthy to mention that 54% of married women in the survey area expressed desire to use one FP method in the coming year. Seven percent of women surveyed, their husband/partner or someone else refuses that they use FP. Similarly, 14% of women who discontinued a method did so on their husband/partner’s decision. This is understandable in a country where it is socio￾culturally accepted that Men have decision-making power almost exclusively. Twenty￾nine percent of men cite religion (Islam) as a reason for not using FP2 In the communaute rurale of Gagnik (district of Guinguineo, Fatick region) the village religious leaders told the evaluation team “if we tell people to go for services they will. If we tell them not to, they won’t” This statement was confirmed by testimonies from many service providers, relay workers, and women’s groups who mentioned “men” as the reason why women do not use FP. Additionally, couple communication on FP is rare and when that occurs, it is usually initiated by women. -On the access front, only 32% of women surveyed reported being satisfied with the FP services they received. Seventeen percent of women who discontinued a FP method gave side effects or other problems related to the method as a reason. Other less important reasons for discontinuing FP include “sterility”, difficult access and poor client-provider communication (welcoming, availability) Given this context, it is critical that we target Men in as many associations/forums as we can reach them, including religious leaders. Messages that will be disseminated to them will refer to the pillar role of women in the society and the fact that the community cannot afford not to help them take care of their health particularly their reproductive health. Taking care of women’s health include using FP as appropriate for and desired by clients, and going for prenatal, attended delivery and post natal care. The fact that Islam is not against family planning and encourages family heads to take responsibility for the health of their families. Such messages will be disseminated through mass media and community mobilization activities. To that end, the MH/FP project will expand its collaboration with the Association Nationale des Imams et Oulemas du Senegal, the Reseau Islam et Population, and get in contact with Imam Cheikh Assane Cisse in Kaolack and other Muslim brotherhoods representatives, to design and implement activities with religious leaders and community. Illustrative activities include: updating of the religious leaders kit developed under the CS/FP project, training of trainers and peer educateur for religious leaders and lay Muslim persons in IPC/C for MH/FP and its importance for the women/family health and success/future of children. They will then organize interpersonal communication and community mobilization activities in men and women’s and youth darah and dahiras. They could use for example the MH/FP video described above (which could also include a famous Imam’s statement in support of MH/FP) 1 Rapport d’enquete supplementaire dans 15 disctricts sanitaires de l’USAID- Octobre 2003 2 Enquete Demographique et de Sante-Senegal 1997 48 Men will be targeted elsewhere for example with the Syndicats National des Transporteurs for orientation to speak for 5-10 minutes to their minibus or taxi clients about FP and MH. The audio tape adapted from the central campaign video will be given to “chauffeurs” to play for clients. Wrestlers, fishermen, military and “other hommes de tenu” will also be targeted with similar community mobilization and IPC/C activities. To increase men’s access to anonymous MH/FP information and counseling, a hotline project could be piloted in Kaolack and held in turn by relay workers, sage-femme, religious leaders….These people will be trained to give such information and will cover various hours during the day to allow people to call whenever they feel comfortable speaking to one or the other person. To increase financial access to MH services, women groups or other community based organisations (men’s groups) will be encouraged to create mutuelles called for example “Mujj” to support cost related to pregnancy and delivery. Community contest: “Roy, mere modele” could be organised every year to reward publicly women who will have repected all their CPNs, planned their birth and went for post natal visits (for a FP method and child immunization). Finally, to reduce lack of FP use due to fear of side effects and poor CPC, service providers will be targeted with print and radio materials (see above) and women could be targeted with FP method-specific spots or infomercials. NEXT steps: Finalize the Behavior Change Communication strategy outline “Wer gi yaram jiggen are na njaboot” Conduct qualitative research Develop messages and materials INDICATORS The Behavior Change Communication indicators will be revised to include behavioral impact indicators. (To be assessed during ANNUAL SUPPLEMENTAL SURVEY) Increased access to quality services: -Percentage of clients who are satisfied with the services received (exit interviews) -Percentage of service providers who provide client-centered counseling: complete and accurate info on all methods, insist on client understanding, and promote decision by client (mystery client observation) -Percentage of service providers who can recognize pregnancy related risk factors and know what to do (3 delays) (interviews) Increased demand: percentage of: -Men who discussed FP or MH services with wives (interview/survey) -Religious leaders trained, (activity reports) -Religious leaders who gave public address in support of FP and MH services use (Interview/survey) 49 -Men reached by community based activities (number reached) -Men who recall messages and -women who completed their CPNs schedules - women who can list at least 6 pregnancy/ birth related risk factors (sensitize) -women with birth plans for current or previous pregnancy (3 delays) (sensitize) -women who delivered at health facility -Percentage of women who respect the post-natal visits calendar ADVOCACY Strategic Approach The Advocacy strategy was originally envisioned as part of the demand generation portion of the Project. It was envisioned that by application of the REDUCE model at National, Regional, District and Community levels, that leaders at these respective levels could be recruited as advocates on behalf of Maternal Health. This was seen as particularly important given Senegal’s high Maternal Mortality Rate. The application of the REDUCE model has been supplemented by the establishment of a new cadre of community health workers the Persuadeurs Communautaires (PC), to conduct outreach and to “persuade” and mentor individual pregnant women in the community to participate in Safe Motherhood programs. In addition, a carefully crafted program of using the media (newspapers, radio and television) has been developed to present project activities in such a way to advocate for both maternal health and family planning. However, the Advocacy volet has moved beyond its originally envisioned scope by the introduction of some extremely innovative interventions and the timely grasping of new opportunities. Most importantly, the Advocacy staff have provided significant input into increasing access to services and improving the quality of services through providing community-level channels for communication, and through recruitment of private sector providers. The Advocacy volet also has developed the most effective multi-sectoral collaboration in the Project through its work with both the Reseau des Parliamentaires and the Ministry of Education. Observations 1. Partnership with Reseau des parlementaires Over 80 Deputies constitute this network, about 25% being women. They have the mandate of working at a regional level to gather information on behalf of the legislature and bring that information back to inform and shape policy. The Project has become deeply involved in the Policy aspect through a request to assist in the preparation of Reproductive Health legislation. This request came about because of the fact that Senegal was behind many of its neighbors in developing this legislation, and needed technical and legal expertise. The legislation is now before the assembly. 50 The Project has also used the REDUCE model with the Parliamentarians on four occasions in Fatick, Kaolack, Louga and Thiès. At these Journées decentralisées the parliamentarians themselves have become advocates for reproductive health, and have engaged the relevant communities in data gathering about what constrains advances in reproductive health. This information is then fed back to the legislative assembly, and ultimately can feed back to the health bureaucracy. Strengths The relationship with the Parliamentarians has provided a powerful channel for the Project to influence national and local reproductive health policy, which the project has capitalized on. The outcome of the meetings held so far include commitments taken by participants to safe motherhood such as: related legislation, budget allocation, further advocacy for maternal health, the development of a system called "Structure de surveillance, d'alerte, et d'intervention rapide", and the active recruitment of multiple PCs. Recommendations The advocacy component can extend the work with parliamentarian further at the level of communauté rurale where specific actions can more readily be taken towards reduction of maternal mortality. The volet could also collaborate with the CEDPA ENABLE project to sensitize women to the global socio-economic impact of maternal deaths, to promote their leadership skills and encourage women to take responsibility at decision￾making levels. As was witnessed in Thiès, where women from various groupements de promotion feminine were encouraged to participate in local comité de santé elections and several ended up being elected president of the committee. The Project should also consider how the work with the Parliamentarians can be used effectively to advocate for altered policy around key issues identified elsewhere in the report: • Relaxing of fees for Maternal Health Services • Further subsidy of bednets • Relaxation of the law to allow social marketing of specific products More effective means for licensing private providers that will allow for practical skills and experience to count as well as paper qualifications. 2. Partnership with Direction de l'Enseignement Elementaire and the national education ministry At the instigation of the Director of Elementary Education, the Project has become a partner in the development of a remarkable new initiative that is potentially one of the most important for the long-term expansion of reproductive health programs. It is the DEE’s desire to ensure that all elementary teachers in the country are trained in reproductive health, and that reproductive health be added to the elementary school curriculum. The Project has worked with rapidly and effectively to develop a training program which will be introduced into the Teacher Training Schools in January 2004. This is significant 51 because 2,500 new teachers are trained every year, of which 300 are religious teachers (from the Arabic tradition). If every teacher become aware of reproductive health issues, and has a specific curriculum to teach this creates a broad, nationwide resource network. Teachers will not only teach the material, but will be encouraged to engage youth in extracurricular activities. Recommendation: The Project will have to play a strong role in the new year to support the DEE as it implements the new training, and to build the case for inclusion of reproductive health into the new curriculum for schools when the decennial review begins in May 2004 3. Persuadeurs Communautaires The idea to create a new form of community health worker charged with advocacy and individual support and mentoring is innovative and appears to be very effective. The Project has trained 220 PCs in 4 districts, centered on 19 health facilities. The PC’s role includes identifying every pregnant woman in their catchment area, visit them at home and coach them in the monitoring of their pregnancy, development of a birth preparedness plan, and follow up after birth. The short training time, and the use of two specially developed tools – the local register and the referral carnet greatly facilitate data collection and preventing drop-outs. At the two sites the evaluation team visited that had PCs present, the ANC completion rate was 95%. Facility nurses found the PCs helpful, and were increasingly using them for outreach activities, and there was eagerness on the behalf of the PCs to increase their knowledge through further training. Recommendation: This has been an excellent pilot program and shows the capacity of community members to take on social issues when they are adequately explained. All the PC’s volunteered as a result of being present at a presentation of the REDUCE model. There is enormous scope for the role that has been developed for the PC to become one of the major thrusts of the ARPV program, and this will need to be actively pursued in the coming months. There remains a serious challenge that needs to be addressed for the PCs. At present, the individuals volunteer their time willingly based on their individual sense of social responsibility. However, over the course of a year, the motivation has worn thin (as is to be expected). The Project staff have already come to the conclusion that the PCs need some form of external motivation if they are to remain engaged with the program for the long-term. There are a number of ways that this might be addressed, and to the greatest extent possible this motivation should take the form of non-financial rewards and benefits. However the Project must consider how the PCs can look at either income 52 replacement (where they give up other opportunities to conduct their work), or income generation (where income is needed to allow them to do the work). Recommendation: The project should work with DISC and ENABLE to explore how motivation could be provided to PCs, while at the same time determining how the PC concept can be incorporated into the ARPV. 4. Expanding access through involvement of the Private sector This topic is dealt with in detail in section 9 of the description of the clinical interventions of the Project. However, it has been managed under the direction of the technical leader for Advocacy and has been very effective in bridging the links between public and private sectors. Recommendation: The model developed with private sector providers should be replicated and scaled up throughout the project areas, and a similar approach developed for engaging private physicians at the large urban centers. 5. Creation of a network of journalists and media specialists The project has actively engaged journalists and media specialists to become advocates on behalf of reproductive health. By developing personal relationships with reporters and educating the press and media about reproductive health, it has proven possible to have active engagement when community events are held, and to have informed and intelligent commentary provided by the journalists. This has proven to be an enormous advantage to the project and has given the project exceptional visibility throughout the whole country. For events of national importance (those that involve the Minister, or members of the Legislature), coverage is provided as part of routine news-reporting. For smaller and local events, the project has found it helpful to either provide direct incentives, or alternatively pay for reporting services. Thus the services become a form of Public Service Advertisements paid for by the Project! The Project has established a formidable collection of newspaper and journal articles from the past couple years that report similar events or discuss efforts pertaining to promotion of safe motherhood and related health issues in Dakar, other keys cities or various provincial sites. This collection attests to some effectiveness in the work to build awareness among the coalescing network of journalists and media specialists so they inform the general public about these health issues and behaviors. Adjustment to indicators: As a result of discussions several additions to the indicators were proposed by the technical assistant responsible for Advocacy. These are included as Annex VII 53 PROJECT SYSTEMS, RESOURCES AND MOTIVATIONS SM/PF Project Work planning has become steadily and more effectively targeted on tracking to project goals. The reporting requirements to USAID have been met. The final reports for 2001, and 2002 have been reviewed, as well as the first two quarters of 2003 through August 2003. The early approach to assessing progress had been based on processes rather than results because there had not been a base line set across the various projects as a reference point. For 2001 70% of plans were realized, for 2002, 80.6% of plans were realized…the current rate for 2003 indicates that the end of year result will be higher than that, even 90%. These are interesting numbers in terms of understanding the scale of activity, they do not say much about the utility or impact of the work. In the past year, a robust set of indicators was developed by the team and confirmed by USAID. While needing some fine adjustments (notes are presented in an annex to this report) this table of indicators allows the Project to move forward with a new confidence in the ability to link activity to results. The 2003 Workplan has also been better utilized with partners in the field. It was dissemeniated to partners and presented by the component leaders with the Division Chief for Reproductive Health, to the medical personnel in the districts of Louga, Kaolack, Fatick, and Thies, in January 2003, Ziguinchor in February, and Dakar in April 2003. There has been broad and useful discussion among the project component leaders and more scrutiny of technical program links to budgeting to allow tracking and mapping the financial effects of mid-stream workplan adjustments. There has been a more systematic approach to the work planning process. As the project swung fully into gear the team remained highly motivated and dedicated to improving the system. More field experience and familiarity with our collaborative parties the DSR, Regional, District, individual Posts, and the supporting communities, and other CA’s have made the work planning process more inclusive of the needs of these groups. The Project human resource and operational equipment capacity, especially at the administrative level, is challenged by the ambitious scope of the plan for 2003, and will need to be considered carefully for 2004 and beyond. Future plans will have to cope with budgetary constraints (the loss in buying power of the US$, alone, has caused an effective reduction in the financial resource pool of the project of nearly 15%) as well as consider the reasonable limits of pressure that can be put on individual staff members. It is a pleasure to report that the spirit and dedication of the team has allowed the Project to implement a broader array of activities than would have been possible if staff were not willing to invest personal time and effort into the tasks of making the project effective and useful. The team is a well-integrated group that has effectively blended the understanding of operational and logistical requirements with the approach to technical implementation. Mutual respect is obvious and spirits and motivation are high. Planning will have to be carefully managed to assure that the activities projected can be reasonably 54 carried out. Implementation will need to be very closely monitored to assure that new commitments that arise almost daily do not overwhelm the strategic thrust of the workplan. It is recommended that for the implementation of the 2004 budget and planning cycle, a formal mechanism be established that will build in a priority review across volets to assure that changes or additions to the agreed-upon workplan can be accommodated within the financial resources of the project (and are reasonable in terms of the operational capacity of the team). Health Facilities Work Planning: During the field portion of the evaluation there were several sites where we had the opportunity to review annual work plans (Fissel, Joal, Gassane). All the plans illustrated the collaborative effort that had to have been made between the various elements within the medical health system at the regional, district, and Post level, the community health committees, and the clients to be served. The plans were detailed and depending on the location showed a variety of success. The importance of being a collaborative effort and reaching an understanding of the needs of all parties was essential to the process. In all locations the work planning exercise had concrete benefits that all could see, exemplified by the Joal Health Center that had identified the need for bathrooms in the maternity ward itself in individual rooms, as opposed to one outside area away from the ward itself. In the case of the Health Facilities the work planning process has been appropriate and effective in analyzing the facilities needs, helping them to prioritize those needs and strategies to achieve results, identifying those responsible for achieving the results, and tracking the success or failure to achieve the stated plan. There is an evaluation framework in place that puts emphasis on post-training monitoring and evaluation as measured by clearly identified indicators. The follow up activities, site visits, supervision conducted with Health Service Personnel, community outreach activities and communications, and auditing of medical facility logs and registries will provide the indicators that show the direction of the project. It is important to know that the work is effective. In discussions with Component leaders they all reported that individual evaluations were done for each activity. There are quarterly reports that track individual activities of the work plan. These reports are a clear report of the activity held (the issue’s presented, the number of participants, etc) but do not assess the technical quality or assess the utility of the activity. SM/PF staff, however, are not always able to observe directly activity implementation or to judge the quality and ultimate impact of project-supported work. Standardized observation and reporting forms could facilitate internal trip reporting for a staff that is almost never ‘back in the office’ long enough to file longer reports. M&E must deal with internal as well as external processes, and must include observations of quality. In 2002, Dr. Wayne Stinson was invited by the team to review the M&E system, with particular view to improving the linking of quality assessment to the routine activities of the project. He recommended (among several points) that staff should review and modify model tools for 55 trip reporting, facility assessment, and performance assessment (and provided several models). It is recommended that the team implement this recommendation both for the activities they directly observe and develop an instrument to be used by trainers and facilitators who are hired on a short-term basis to carry out specific SM/PF elements. Specifically for training activities, it is recommended that a standard process be used with participants to: confirm whether the objectives were understood and met, (Did the message get across to the participants?) Were there influencing factors, such as logistical problems, of the activity that impacted engagement? This feedback process can reinforce the engagement of the participants at the same time that it can help improve the next activity? Staff Levels and Procedures Project activities are provided for with good technical content, and good presentation by the technical component leaders. There are adequate administrative policies and procedures in place to assure that the technical activities are well supported materially, financially, and logistically. There is a regularly-updated Procedures Manual that is followed closely by the project staff. Proper procedures are followed for travel authorizations, procurement needs, contractual needs, and cash payouts, all in support of technical functions. The four technical components require multiple activities in widely dispersed areas. These activities are for both large and small groups of people (groups can range in number from a dozen to several tens of participants). The administrative unit is responsible for providing for the logistical needs of all of these events. The administrative staff consists of, exclusive of drivers, the Office Manager, Chief Accountant, Accounting Assistant, and the Administrative Assistant, and Financial Secretary. These five people, with the guidance of the Operations Manager, provide all the logistical support requested. It is impressive that they have been able to provide routine & high quality services over time, at feverish pace, without breakdown or dissent. It will be important to consider the specific roles of individuals and to review the capacity of this dedicated team to meet the goals that are set in the 2004 planning. It is recommended that the Project Director review the key roles and function of the Operations Manager and the general administrative staff and make any necessary adjustments and changes in role definition required to assure appropriate function of the project as demand for project staff time and supported activities continue to rise. Potential need for additional staff should be considered if the pace of activity is going be maintained or increased (the budgetary tension in this recommendation is acknowledged). Due to many outside constraints, internal coordination limitations, and tight schedules of activities, the administrative staff do not always learn precise activity dates until quite 56 late. It is also often the case that counterparts delay confirmation of a planned event…or request action from the team …on short notice. These late breaking events are good evidence of an active program that is growing in appreciation, but they also cause added strain and increase the likelihood of a breakdown in operating systems. Better advance scheduling and better communication between the technical and administration staff on changes in plans can alleviant some of the pressure. All project operations job descriptions were reviewed discussed with the individuals holding those positions. The job descriptions are appropriate for the tasks. There is an active PP&R system that is completed annually. Staff are appreciative of the process (which clarifies roles, responsibilities, performance standards, and progress) and the fact that there is an open dialogue with the project administration. Several staff notes that they are comfortable with working long hard hours dedicated to improving the lives of others, and pleased to be working in a safe environment where they are free to express opinions, and respected for the work performed. It is recommended that the staff address the complex processes of communication and activity scheduling in one of the next SM/PF team performance improvement sessions to identify specific steps that can be taken to anticipate events and ease pressures for last minute planning. Administrative, Financial, and Management Procedures Administration: The administration has an updated Procedures Manual that is comprehensive in scope that details procedures and processes necessary to provide support functions to the project. All of the procedures outlined in the Procedures Manual are related to standard MSH policy guidelines or are anchored in USAID Rules and Regulations as defined in Circular A110 and A122 as they relate to Non Profits. There are detail policy statements and processes covering a wide range of areas including: vehicle and travel policy, hiring and firing guidelines, MSH standards of work, a work place free of sexual harassment, drug free workplace, local holiday schedule, benefits package, health insurance administration, per diem policy, cash payments, petty cash management, integrity in procurement, and procurement. There are supporting documents and forms for all operations and activities, from time sheets, to vacation requests (list forms available). Processes to make advances, clear advances, and generally to manage money are clearly spelled out in the manual and are reinforced by periodic training. Finance: The financial system and the QuickBooks program used to manage it function efficiently through creative use by the Operations Manager of project coding linked directly to the work plans that tracks exact cost charged to each component. The Chief Accountant has gained a particular expertise in QuickBooks and has served MSH regionally to train projects opening in Guinea and Angola. The precision of the 57 accounting department has allowed the Operations Manager to provide valuable financial information and projected pipelines that create an accurate picture of the future expenditures. This information is critical to planning activities. The project has been able to produce accurate monthly financial reports to Boston in a timely manner that has allowed the project to be aware of its financial situation in real time. Logistics: Preparing for project activities is an ongoing challenge as already noted. The administrative unit meets this challenge regularly. One issue that has become an essential element of the services offered is the manner in which the funds used to pay for activity needs, per diems, transportation, site rental, are handled. Presently, SM/PF staff who are responsible for providing the necessary payments bring cash to activity sites. These sums can be considerable given that there are times the payments are made serially at several activity sites. The person responsible for the funds must keep the cash on their person from the time they leave the main office until they have made all the payments. Even then there may be considerable funds remaining to be returned to the main office at the end of the activities. The security risks to both personal safety and corporate liability are considerable. It is recommended that the team examine possibilities to mitigate risks to individuals. Options to consider include pre-specifying with local health authorities the individuals who will attend and be entitled to per diem and then pre-packaging those payments to avoid open displays of cash in semi-public places and placing safes for overnight deposit in regional or even district level spots. (Other specific security options have been discussed with the team and will also be reviewed in this context.) Transportation: There are clear vehicle policies in place and which are adhered to. The drivers are responsible for the preparedness of the vehicle. The drivers complete a required “Carnet du Bord” for each trip, recording the mileage, the destination, condition of the vehicle, and gas consumption. Vehicle allocation, assignment of drivers, calculation of gas usage, and driver per diem for particular missions are responsibilities of the Administrative Assistant. Complaints have been raised that some of the vehicles are in poor condition and do not have working air conditioners. Certain types of vehicles, particularly the pick-ups, have limited use and are uncomfortable (and given they have open backs make it difficult to safely secure baggage). One of the problems faced by the administration in addressing these complaints is that the vehicles are always on the road. Even when poor vehicle conditions are identified there is high pressure to get them back on the road. Rental of vehicles has been done periodically in the past, but has proven to increase risk, given that there is no assurance that proper maintenance has been carried out (the fatal accident of last year was due in a great part to a poorly maintained rental vehicle). It is recommended that a clear policy be established to “ground” unfit vehicles until repair even if it means stopping or changing the date of an activity. It will also be appropriate to consider whether the upcoming workplan requires replacement of 58 aging vehicles (some of which were inherited after years of service on previous projects). Context of Collaboration for Results and the Potential for Achievement: The Collaboration that is necessary for successful implementation of project goals operates at the technical and operational level, but also must effectively fit into the political structure and conform to socio-cultural norms. On the technical and operational level, the SM/PF Project works in a complicated arrangement in which it shares responsibilities for achieving SO3 goals with a range of other Contractors (BASICS, FHI, DA/DISC, along with CEDPA/ENABLE, ADVANCE Africa, and RPM+). The complexity of the interlocking project scopes of activity has been successfully managed to assure that the various projects are working with each other toward common goals and with reasonable clarity regarding who is responsible for what range of program elements. These working partners, however, are only part of the equation. Each project is working hard against an array of specific goals that require activities in conjunction with staff at all levels in the health service delivery system. The USAID SO3 team has expressed general satisfaction with the effectiveness of the collaboration of the SM/PF team with the other CAs and with its progress and contribution toward overall SO3 goals. Likewise, in meetings with the other CAs it is clear that there is a positive sense of collaboration across these components of the USAID-supported health program. From our meetings, it is reasonably clear that there is effective coordination among the CAs of activities at the central and regional levels, but a fairly consistent comment from colleagues in the health system at the district and health post levels (where 98% of SM/PF Project resources for activities are expended) was that it is sometimes difficult just manage daily life in an environment where so much programming is competing for time from the health system staff. The programs compete with each other and…with those who are seeking services. There are clearly some opportunities for the CA programs to schedule together and even co-sponsor activities (potential work with DISC staff to expand the useful impact of the MH/PF project’s innovative approach of bringing Médecins Chef’s for districts together with the local Mayors …and the infirmiers together with the presidents de communautés…. to help assure that the decisions around resource allocation that are made by the Mayors are informed by direct understanding of the requirements and importance…including political importance…of the system for health service delivery. Another area where additional collaboration will improve coordination is to more carefully link the strategies and implementation plans of the SM/PF Project for family￾planning counseling and the BASICS Project for child survival and nutrition counseling in the peri-natal period. 59 On this level, it is recommended that the SM/PF Chief of Party (or designated representative) meet or otherwise communicate more routinely on a one-to-one basis with the leadership of the other USAID-supported projects to facilitate early recognition of opportunities for joint efforts. The very good news is that the various programs are only bumping into each other because they are reaching their goals for engaging with communities and promoting the engagement of community forces in the support and management of health service delivery. The direct interactions at the local levels between the communities, the health systems, and the projects have created a powerful change. To promote transfer of full ‘ownership’ of program innovations to the health system (an example of where that is needed is the formative supervision initiative) and the communities served (examples are the health committees and the pilot program with Relais), the projects will have to consider together how to coordinate the full transformation from resource transfer mechansisms to catalytic influences in the system. The SM/PF team can work with DISC staff to promote development of mutuelles and other local mechanisms for income generation to provide material support to health service promotion and delivery and, perhaps, income replacement mechanisms for the Relais (and eventually the ARPV) from local resources. The recommendation here is not for a specific intervention change, but for a review with other CA partners of the changed context in the current project districts and engagement with the health and community leaders in those districts to consider how the dynamics have changed and what the most effective approach should now be to consolidate and build on advances. Referring to the PI model, this step would mean a shifting back during planning from a pure ‘implementation and replication’ cycle to a survey of the changed context and current situation against goals. Identifying the main strengths and features now driving the system and any new ‘root causes’ for factors that are now needing to be addressed in working toward improved availability & quality for health services and more confident demand for them should enable effective channeling of project resources. One area where there is some concern that the approach taken might not be the best strategy for sustainability is the current system of providing per diems for community members to engage in project-supported activities. It is clear that the provision of per diems for participation is welcome in the communities, the impact (after initial engagement) of the per diems themselves could turn out to be counter productive. If the goal is for the members of the communities to become engaged and feel a personal stake in the health service system, the mechanisms for involvement should be designed to maximize engagement while minimizing artificial resource transfers beyond those required for program initiation and support to innovation. It does seem appropriate to cover (for now) unusual transport costs, etc, but for people living in a community, the per diem now could cause a reaction later (after the end of the project support) causing individuals to question. It might be better to consider funding elements related to 60 gatherings of communities by providing a celebratory spread of food (and perhaps some entertainment) at the end of events in place of the per diem program. This is not a change that the SM/PF project can consider alone and should be a subject of discussion with USAID and the other CA partners. The provincial network of private health care providers represent a fine possibility for expanding access in the areas where they work. Individuals where clearly dedicated to their work and generally pleased to be associated with the Project and with the Ministry. It is also useful to be able to include their activities in service data collection. The system for certification of private practitioners seems to be understood and accepted, but it was noted that this system has not been able to certify many of those who had been in practice. This evidence of clear standards is good, but there may be an inconsistency with public sector standards and it is also possible that there are resources being sidelined who could play a useful role. One point for consideration might be to work with the central level in the MSPP to review the approach to certification of private health care providers to assure toward consistency with the standards for role allocation in service delivery in the public sector and if there would be any utility to a lower tier of certification. 61 ANNEX I – INDIVIDUALS CONSULTED USAID Brad Barker, SO3 Team Leader Elisabeth Benga-De, COTR Julia Henn, Michigan Fellow Ministère de la Santé et de la Prévention Division de la Santé de la Reproduction MSPP Medecin Colonel Adama Ndoye, Cheikh Ahmadou Bamba Diop Dr. El Hadj Ousseynou Faye Réseau des Parlementaires en Population et Developpement Honorable Député Famara Sarr Coordinateur Honorable Député Momar Lo, Coordinateur, Sahélien Parlementaire Réseau Ministère of Education Mamadou Aly Sall, Directeur, Direction de l’Enseignement Elementaire Abdourahim Gaye, National Staff Coordinateur pour Initial et Continué Training District of Thiadiaye Dr. Sokhna Sow, District Médecin Chef Et membres de son équipe (M. BA, Madam Cisse, Dr Gueye Radio PENC-mi de Fissel Talla Dieng, Directeur de la Station Mme. Khady Sene M. Modou Pouye Poste Santé de Fissel Ali Ndao, Infirmier Chef de Poste Kangou Sarr, Sage Femme N’Deye Yassine Diouf, Coordinateur Relais Nogaye N’Dao, Coordinateur Région Médicale Fatick Amath Mbaye, Médecin Chef de Région Family Health International (FHI) Dr. Abdoulaye Cire Anne, Charge des Opérations M. Massaer Gueye, Interim Director Dr. N’Deye Senn Nyiang M. Seydi Ba Gassama, IEC Specialist 62 Ademas Sedou Nourou Koita, Responsable Marketing Alain Kande, Gestionnaire Logisticien Aida Soumre’ Diop, Responsable I.E.C. Pharmacie Nationale d’Approvisionnement Pharmacien Lt Colonel Issa Diop, Directeur Papa Ibrahima Ndao, Directeur Administratif et Financier DISC Project (Development Associates) Dr. Vincent Joret, Chef d’Equipe BASICS II Dr Aboubacry Thiam Chef d’Equipe Pays SUD FM Thiès 102.2 Pape Ndao, Journaliste, Rédacteur en Chef Lamine Diatta, Chef de Station, Sud FM/Thies (other radio stations) Radio « Dunyaa », Thies, Mr. Diallo, Chef de Station RTS, Thies, Mme. Rokya M’Bengue, Health Specialist Mme. Michelle Diouf, Chef Station RTS/Thies Radio Joal « La Cotiere », M. Diakite, M. Diop Region de Louga Youssoufa Mbargane M’Baye, Troupe Theatre Troupe Theatre Artists SWAA : Youssou Mbargane Mbaye Arame Thaine Seck Seyni Seck Mame Khar Marone Ndongo Thiam Mbaye Ndiaye Mame Mor Samb Baytir Gueye Mor Sourang Seck Fatou MBengue District de Linguere Dr. Dione MCD Adjoint District Linguere Mme. Dia Khadiatou Beye, Coordinatrice SR Mme. Diouma Sow, Conseillere Mme. N’Deye Sy, Conseillere 63 Poste de Santé de Gassane (Communauté Rural de Gassane) M. Daouda Konaté, ICR Mor Samba, President Committee de Sante Sophie Samb, Presidente ARF-Persuadeur M. Samb, Chef de Village Abdou Diouf, PRC Matar Lo, Persuadeur Communitaire Pape Tair Top, Persuadeur Amy Toure, Matron Conseillere Dieynaba Ka, Persuader Houdj Mbengue, Persuader Programme Nationale Lutte contra Paludisme (PNLP) Dr. Pape Amadou Diack CEDPA Soukaye Dieng, ENABLE Senegal Programme Manager District de Guingueneo Dr. Mamadou Dieng, MCD Dr. Diokh, MCD/Adjoint Mamadou Ndiaye, Supervisor des Soins de Sante Primiaire Région Médical de Kaolack Dr. Issa Mbaye Médecin Chef de Région Amy Mbow, Coordinatrice Regionale SR de Kaolack Dr. Mame Coumba Faye Diouf, Médecin Chef de District de Kaolack Mr. Beye, Superviseur Soins de Sante Primiaire du District de Kaolack Mr. Sonko, Education pour La Sante Mme. N’Dao, Coordinatrice SR du District de Kaolack Abdou Diagne Gueye, President Relais Kaolack Prestataires Privés Ibrahima Ndao, Coordinateur Collectif des Infirmiers Prives de Kaolack Amadou Bamba Diop, Representant de la DSR dans la Mission d’Evaluation Dr. Diaam Sarr, Infirmier Prive Dr. Macodou N’Diaye, Infirmier Prive Bushra Diarara (Islamic Community) Imam Habib N’Diaye, Chef de Religioux Reunion Communitaire (Kaolack) Mr. Badji, ICP Parcelles Asscunie Kaolack Mbagnick Ndiaye, Coordonnature des Persuadeurs Communantaires de l’Association des Volontaires du District de Kaolack (AVODISK) et members 64 Infirmerie Camp Militaire Dr Kasse, Médecin Chef Adjoint Malang Sbuame, Infirmier Major Région Médical Thiès Dr. Idrissa Tallo, Médecin Chef Région Dr.. Fatou Nan, Médecin Chef Mekha Dr. Maty Diagne, Médecin Chef du District de Tivaounane Dr. Thior, Médecin Chef Popeguine Dr. Ibra Sene Médecin Chef Mbour (& membres d’équipe) Mme. Binta Gaye Mamadou Diop, District Sante de Thies Dr. Diouf, MCD Mme. Maimouna Dieng, Groupe Femmine de Thies District de Kebemer Dr Balla Mbacke Mboup, MCD/Kebemer Diago Tnadian, Chef du Poste/Ndande Poste w Santé de Ndande Diego Mbaye, Sage Femme, Chef de Poste 2 Community Representatives District de Joal M. Ibrahima Deme, President du Comite de Sante M. Thialy Faye, Gestionnaire Fatou Binetou Thioune, Sage-Femme Alioune Cisse, Responsable EPS Mariama N’Diaye Moussa Camara, President du Comite du suivre COPE M. Momar Talla Sene, 2eme adjoint Maire M. Toure Centre Sante Guediawaye Dr. Mamadou Diop, MCD Adjoint Mamadou Diallo, Superviseur SSP Aldiouma Dieng, Superviseur EPS Rokhaya Faye, Coordinatrice SR 65 ANNEX II - DOCUMENTS REVIEWED Maternal Health and Family Planning Project. Third Activity Report 2001 Maternal Health and Family Planning Project. Third Activity Report 2002 Projet Sante Maternelle et Planification Familiale. Rapport d’Activites: Janvier-Avril 2003 Projet Sante Maternelle et Planification Familiale. Rapport d’Activites: Mai-Aout 2003 Enquête dans 15 Districts sanitaires des regions de Fatick, Kaolack, Louga et Thiès. October 2003. Groupe SERDHA Supervision formative en SR, Introduction de l’approche COPE et de la prevention des infections. District de Joal – Fadiouth. 2003 MSH Proposal to USAID in response to RFA No: 685-00-A-004. 2000 Request for Application(s) for: Maternal Health/Family Planning (685-00-A-004 ) and STI/AIDS (685-00-A-005) for USAID/ Senegal Strategic Objectives (SO3) Plan Travail PSMPF, 2002 II Plan Travail PSMPF, 2003 USAID’s PMTCT Vision, 2003 Job Descriptions: all staff members Trip Reports: April 2002 Trip Report- McKenney July 2001 Trip Report- McKeown July 2002 Trip Report- Thomas June 2002 Trip Report- Stinson March 2002 Trip Report- Pollock May 2003 trip report- Bryant Nov 2001 Trip Report- Madden October 2000 Trip Report- McKeown September 2000 Trip Report- Konings Maternal Health and Family Planning Project: Local Procedures Manual Cooperative Agreement Award No. 685-A-00-00-00113-00 and amendments 1 through 6 Plan d’Action Sante de la Reproduction, Republique du Senegal, MSHP/DSR (2003-6) Mbour radio interventions evaluation report (April 2002) Keur Momar Sarr rural radio evaluation report (May 2003) Radio contracts 66 ANNEX III – DETAILED SCOPE OF WORK FOR THE PROJECT Specific task allocations: 6. Verify if the original strategy for the Project remains valid and has been implemented effectively. John Pollock, (Lead) Malcolm Bryant, John McKenney, Amelie Sow a. Review the original Scope of Work and its underlying assumptions as presented by USAID • Do these assumptions remain valid? If they have changed how will this affect the Project’s ability to achieve its goals • Is there a mechanism in place which monitors changes in Senegal that will impact the strategic imperatives? If not, how can this be developed. b. Review the rational and strategic approach proposed by MSH • Does the strategic approach described in MSH’s proposal remain valid in light of the experience of the first 3 years of implementation • Should the strategic approach be modified • What lessons learned can be taken that could be of value to other CAs or to other USAID/MSH Projects c. Review the conceptual framework proposed by MSH • How useful has the original framework been in directing activities of the Project. • How closely do the actual implementation activities of the Project map onto the framework? 7. Assess the progress towards achieving the Project Objectives and results Malcolm Bryant, (Lead) John Pollock, John McKenney, Amelie Sow d. Review the overall Goals and strategic objectives of the Project • Have the Project goals and results remained the same. If they have changed, what are these changes. • Will the project achieve its goals by the end of the project. If not, what needs to be modified? – the project goals, or the project? • Are the four technical interventions of the Project (clinical, logistics, IEC and Advocacy) the right mix of interventions or could a different mix lead to improved success • How effective has the Project been at achieving the geographic goals of reaching 29 districts, and if it has not achieved this goal, will it do so by the end of the project. • Has the addition of Malaria, Post abortion care and PMTCT activities altered the goals or overall scope of the project e. Review the PMP for KIR 3.1 Increased Access to services 67 • Has the Project implementation led to measurable improvements in access and quality of services? • Can the Project claim sole responsibility for improvements, or are they shared with other activities. If shared, what proportion rests with the project f. Review the PMP for KIR 3.2 Increased Demand for services • Has the Project implementation led to measurable improvements in access and quality of services? • Can the project claim sole responsibility for improvements, or are they shared with other activities. If shared, what proportion rests with the Project. 8. Evaluate the quality and implementation of the four Project technical volets at central, regional, district, and community levels Malcolm Bryant, (Lead) Amelie Sow (Focus on IEC/BCC and Advocacy) g. Review the specific approach being used in each volet • Is the approach used in the Volet linked with the Project’s strategic approach and likely to contribute to the goals and results of the project • What are the key elements of the volet. Are they consistent with good practice in this technical area. • How are cross-cutting issues such as training, and supervision being applied to the individual volets h. Review project workplans • How has the evolution of the workplanning process improved the project activities • How well are activities coordinated and efficiencies established • How is Performance Improvement being integrated into technical workplans • How are activities integrated at the different levels of service – central, regional, district and community i. Review project reports • Have reports been submitted in a timely fashion and do they meet contract requirements • Do reports accurately reflect technical accomplishments of the Project 9. Assess the overall technical and administrative management of the Project John McKenney, (Lead) John Pollock j. Examine Project workplans, evaluations and reports • Is the Project workplanning process appropriate and effective. • How has the newly developed evaluation framework been implemented • Do Project reports meet Project and partner needs 68 k. Review Staff levels and procedures • Is the staffing level appropriate to the work required • Are job descriptions appropriate • Do working conditions reflect MSH standards l. Review internal quality systems • What standards exist to determine quality of MSH work in the Project • What review and evaluation systems are in place to review and measure quality of MSH’s work. • Does the work done meet MSH’s worldwide standards m. Review administrative, financial, and management procedures • Do the financial, admin, logistics, transport, procurement and other admin systems function appropriately and efficiently. 10. Evaluate the effectiveness of the collaborative relationships between key partners (Reseau des partenaires du Senegal (REPS) John Pollock, (lead) Malcolm Bryant n. Review the overall relationships • Has the Project met the requirements for coordinating the overall inputs into SO3 • What have been the specific collaborative efforts and how effective have they been. • What more could be done o. Meet with SOT3/USAID • How does USAID view the overall partnership with MSH and the Proejct • Does the perception that MSH fails to provide effective home office support persist • What additional activities could be done to improve relationships with USAID p. Meet with DSR/MSHP • How does DSR view the overall partnership with MSH and the Proejct • How does MSH interact with the DSR at central, regional, district and community levels. • What points of friction exist – such as per-diem payments, honorarium, etc. How can these best be resolved • What additional activities could be done to improve relationships with DSR q. Meet with Cooperating Agencies • What are the relative roles of CAs and how are these roles respected • What should MSH do to improve coordination and leadership between the CAs. 69 ANNEX IV – PROPOSED EXPANSION OF THE PONT D’OR 70 The goal of the program is that all SDPs will achieve the minimum standards for being fully functional by the end of September 2006. However, motivation and continuous improvement are important. The proposal that the FFFSDP concept be used to supplement the Pont d”Or approach builds on Project experience, and also builds on the Performance Improvement Approach that has been applied internally, and in supervision at facility levels. By setting clear goals in both service delivery and community engagement that go beyond the minimum standards, the Project can establish a system of incentives to achieve excellence. By making goals concrete, attainable and realistic, and recognizing achievement with visible rewards (certification of Bronze, Silver or Gold standard), the strength of the bridge between community and facility can be reinforced, and the likelihood of sustainability in those high performing centers is enhanced. The process would involve at least the following steps: 1- Identification of the minimum project package at clinic level 2- Advocacy work at community level to get their buy in 3- Perform bridging exercise with community (bringing community onboard) 4- Formative supervision 5- Accreditation once a year (following a round of supervision) 6- Determination of a suitable mechanism for recognition of the achievement of sites – Bronze, Silver and Gold. Recognition must be of sufficient value to motivate and maintain interest. 7- Establishment of an approach for dealing with sites that lose their accreditation so that they remain motivated rather than becoming disillusioned. 8- Creation of mechanism by which high performing sites can “mentor” lower performing sites. 9- Determination of what to do for sites that perform at Gold level for multiple years – do they become champions Pont d’Or 10- Promote a sponsorship approach of the site to encourage wealthy people originating from the area to invest in their health structure This approach must ultimately be adopted by the MOH to scale-up to all service delivery points in Senegal with integrated health services. While we can refine the approach and process in the 3 remaining years of the SM/PF project, we cannot maintain it in the long-term. Development of Selection criteria The selection criteria will be based on the supervision tool and could include the following: At clinic level: 1- Staff performance in SOUB, contraceptive technology, CPN/delivery/Postnatal 2- Client-provider interaction for MH and FP 3- Contraceptive logistics and continuity of services: availability of methods, absence of stock outs 4- Infection Prevention At community level: 1- Contribution to health services improvement (financial and human) 2- Community involvement and overall leadership around the SDP 3- Active engagement of commission qualite and comite de gestion (regular meetings, or other community organizing in relation with DISC and ENABLE projects mandates) 4- Completion of consensual quality improvement action plan 71 Provision of technical assistance Communities and facility members will need to draw on external resources to assist them. This can be through project intervention, but would be preferable to be through DSR or other local resources. Incentives Careful thought needs to be given to what are the right incentives. What motivates one community may be quite different from what motivates a similar community in a different region. Elements of competition are always important motivators, as is public recognition – whether through political or media mechanisms. Incentives also must change regularly or 73 ANNEX V - PROJECT MONITORING TOOL No. Indicator Criteria Baseline Goal Responsible Data Source 1 Percent of PPS depot managers trained in stock management. Numerator = number of PPS depot managers trained in project-supported course or in another course of similar standards; denominator = number of PPS depot managers 0 100% Antoine Project reports P28 Grille de supervision 2 Percent of PPS depot managers who respect stocking norms Numerator = number of observed PPS depot managers who follow 80% of stocking norms; denominator = number of observed PPS depot managers 100% Antoine Supervision guide, page 68 3 Percent of PPS depots in which ‘PICS’ have been available on at least 90% of days during the past semester (PICS=pills, injections, condoms, spermicide) Numerator=number of PPS depots in which PICS have been available on at least 90 percent of days during the past semester; denominator= number of observed PPS depots 100% Antoine. Page 72 of supervision grille (remplacer trimestre par semester) 4 Percent of PPS depots in which anti-malarials have been available on at least 90 percent of days during the past semester Numerator=number of observed PPS depots in which anti-malarials have been available on at least 90 percent of days over the past semester; denominator= number of observed PPS depots 100% Antoine Page 78 of supervision grille liste medicaments:aj outer sulfadoxine pyri methamine 5 Percent of trained district and Numerator=number of 100% Antoine grille de 74 No. Indicator Criteria Baseline Goal Responsible Data Source regional depot managers who keep appropriate records up to date (Pourcentage de gestionnaires de produits formes qui tiennent a jour les outils de gestion) trained district and regional depot managers who keep appropriate records up to date; denominator=number of trained district and regional depot managers supervision logistique des districts 6 Percent of trained PPS depot managers who keep appropriate records up to date (Pourcentage de gestionnaires de produits formes qui tiennent a jour les outils de gestion) Numerator=number of trained PPS depot managers who keep appropriate records up to date; denominator=number of trained PPS depot managers 100% Antoine Grille de supervision Pages 79-80 7 Percent of district and regional depots without contraceptive stockouts during the past semester Numerator=number of district and regional depots reporting no contraceptive stockouts during the past semester; denominator= number of district and regional depots 100% Antoine Logistical supervision grille du district 8 Percent of district and regional depots without contraceptive stockouts due to the fault of the depot manager during the past semester (Pourcentage de depots qui n’ont pas connu de rupture de stocks de methods contraceptives imputables au gestionnaire de produits sur toute l’annee) Numerator=number of district and regional depots with no contraceptive stockout during the past semester which can be attributed to the depot manager; denominator=number of district and regional depots 100% Antoine District logistical supervision grille 9 Numerator=number of PPS depots with no Antoine Page 72 of supervision Comment: Note my discouraging comments Comment: Note my discouraging comments 75 No. Indicator Criteria Baseline Goal Responsible Data Source contraceptive stockout during the past semester which can be attributed to the depot manager; denominator=number of PPS depots grille 10 Numerator=number of first postnatal visits (defined as visit within 42 days after delivery) in past semester in the district; denominator= number of live births in past semester in the district 50% Philippe Item 12 on page 37 of supervision grille divided by item #3 at top of page 36 11 Numerator=number of PPS “offering” FP services at the last supervision visit; denominator=number of PPS 100% Philippe Item 3, 4 or 5 on page 30 of supervision grille 12 Numerator=number of FP service sites observed to respect 80 percent of administration norms; denominator=number of FP service sites in which FP services were observed during the most recent supervision visit) 100% Philippe pages 39 to 49 of supervision grille No. Criteria Baseline Goal Responsible Data Source Numerator=number of FP service sites observed to respect 80 percent of 100% Pages 65 to 67 of supervision grille Comment: Presumably the denominator is the number of recent mothers (must define ‘recent’). I don’t see how either the numerator or denominator can be determined through facilitative supervision. 76 No. Indicator Criteria Baseline Goal Responsible Data Source counseling steps; denominator=number of FP service sites in which FP services were observed during the most recent supervision visit 13 Numerator=number of PPS respecting 80% of norms and procedures for the prevention of infections, as determined during the most recent supervision visit; denominator=number of PPS 100% Philippe (targets to be determined from estimated baseline derived through data analysis of first supervision) Pages 61 to 64 of the supervision grille 14 Numerator: Number of PPSwhere COPE was introduced by project or similar source Denominator: total number of PPS 100% Philippe Project reports Page 30 of the supervision grille 15 Numerator=number of *CPN1 visits in past semester; denominator=estimated number of pregnancies during the past semester 90% Philippe Item #1 at bottom of page 36 divided by #3 at the top of the page 16 Numerator=number of CPN3 visits in past semester; denominator=number of CPN1 visits for whomen who are supposed to 60% Philippe Items #2 and 1 at the bottom of page 36 This indicator must be change as it is indicated Comment: Frankly, I think it would be better to focus on PPS practicing COPE rather than simply trained in it. Comment: Will supervisors do a record review? For what time period? 77 No. Indicator Criteria Baseline Goal Responsible Data Source complete their CPN3 visit during the same period (blue) or choose the “couverture adequate” during monitoring 17 Numerator=number of CPN visits in the past semester in which anti￾malarials were distributed; denominator=total number of CPN visits in past semester 100% Philippe from page 36 of supervision grille 18 Numerator: Number of PPS holding a follow up meeting in the last 3 months Denominator: Total number of PPS where COPE was introduced 100% Philippe Facilitative supervision 19 Numerator=number of PPS with at least one provider trained in CPN by the project; denominator=total number of PPS 100% Philippe Project reports page 28 of supervision grille 20 Numerator=number of PPS receiving two facilitative supervision visits during the last 12 months; denominator=number of PPS 100% Philippe page 36 of supervision grille Project Reports 21 Numerator=number of PPS observed to correctly 80% Philippe Index from page 32 of Comment: Rima says almost no one is getting TPI at this point, only prophylaxis. Clinic records don’t seem to measure anti-malarials properly. I doubt that this indicator can be validly at the present time. Comment: Should this be stated as a percentage? Comment: 78 No. Indicator Criteria Baseline Goal Responsible Data Source complete of all items on the FP register; denominator=total number of observed PPS supervision grille, “le remplissage du register PF.” 22 Numerator=number of PPS observed to correctly complete all items of the CPN register; denominator=total number of observed PPS 80% Philippe Index from page 33 of supervision grille 23 Numerator=number of PPS observed to correctly complete all items of the delivery register; denominator=total number of observed PPS 80% Index from page 34 of supervision grille 24 Numerator=number of PPS observed to correctly complete all items of the postnatal care register; denominator=total number of observed PPS 80% Index from page 35 of supervision grille 25 Numerator=number of PPS offering Norplant denominator=number of PPS with at least one service provider trained in Norplant 100% Philippe Index from page 28 & 30 of supervision grille 26 Numerator=number of PPS (including Level 2 hospitals) offering the SOUB or SOUC; 100% Philippe Item #6 and 7 on page 30 of the supervision grille 79 No. Indicator Criteria Baseline Goal Responsible Data Source denominator=number of PPS with delivery service 27 Numerator=number of PPS observed to meet 80 percent of standards for CPN procedures; denominator=number of PPS in which supervisors have observed CPN 100% Philippe Pages 51-54 of supervision grille 28 Numerator=number of maternities in which effective use of partograms has been confirmed; denominator=total number of maternities in which labor and delivery process has been observed 100% Philippe Pages 55-56 of supervision grille 29 Numerator: Number of PPS that meet 80% of supervisory norms for Group Education in reproductive health Denominator: Number of PPS with at least one trained counselors 100% Cheikh Fall Grille de Supervision Page 67 30 Numerator: Number of PPS that correctly follow counseling norms for the use of ITN during CPN Denominator: Total number of PPS that 100% Cheikh Fall Supervision Grille - TBD Comment: Yes, but how will this be measured? Comment: How many PPS offer urgent care (as opposed to normal labor and delivery)? Since urgent care is infrequent and noctural, how will we confirm performance according to norms? (I have never seen a feasible technique for confiming correct performance of even normal deliveries. 80 No. Indicator Criteria Baseline Goal Responsible Data Source conduct CPN 31 Numerator=number of PPS offering counselinng denominator=number of PPS with at least one service provider and one counselor 100% Cheikh Fall Index from Grille de supervision Numerator page 30 Denominator page 28 32 Number of RPV trained ??? Cheikh Fall Project Reports 33 Cheikh Fall USAID Annual Supplemental Survey 34 Cheikh Fall USAID Annual Supplemental Survey 35 Cheikh Fall USAID Annual Supplemental Survey 36 Numerator=number classified as Pont d’Or; denominator=number of PPS eligible to become Pont d’Or in target districts 100% Cheikh Fall Project Reports 37 Number de décideurs et de leaders sensibilisés et orientés comme PC 150 Dioum Policy Dialogue report and dans les zones d’intervention Project Reports 38 Numérateur: nombre de 75% Dioum Policy Dialogue 81 No. Indicator Criteria Baseline Goal Responsible Data Source décideurs et de leaders qui mettent en oeuvre des activites de promotion d’une maternité à moindre risque et de PF au sein de leur communauté Dénominateur : nombre de décideurs et de leaders sensibilisés et orientés comme PC follow up report , Project report and follow up survey Cahier des PC 39 Numérateur: nombre de décideurs et de leaders PC qui connaissent 2 avantages de la PF Dénominateur : nombre de décideurs et de leaders sensibilisés et orientés comme PC 75% Dioum PC report books, providers witness Follow up survey 40 Numérateur: nombre de décideurs et de leaders PC qui connaissent le calendrier de la CPN Dénominateur : nombre de décideurs et de leaders sensibilisés et orientés comme PC 75% Dioum Follow up survey 41 Numérateur: nombre de décideurs et de leaders PC qui connaissent 2 avantages Dénominateur : nombre de décideurs et de leaders sensibilisés et orientés 75% Dioum Follow up survey 82 No. Indicator Criteria Baseline Goal Responsible Data Source comme PC 42 Numérateur : nombre de femmes enceintes qui ont effectué au moins une CPN Dénominateur : nombre de femmes enceintes recensées 80% Dioum Cahier du persuadeur communautaire 43 Numérateur : nombre de lycée et collèges disposant d’un club EVF Dénominateur : nombre d’établissements scolaires dont les autorités ont été sensibilisées 80% Dioum Rapport de suivi des engagements des autorités scolaires 44 Numérateur : nombre de formateurs, de maîtres d’application et d’élèves maîtres orientés sur la SR Dénominateur : nombre de formateurs, de maîtres d’application et d’élèves maîtres des EFI (Ecoles de Formation des Instituteurs) ciblées 100% Rapport d’activités du volet politique 45 Propositions consensuelles des 6 EFI sur l’intégration de la SR dans le référentiel des compétences 100% Rapport de l’atelier national d’intégration 46 Numérateur : nombre de PPSP recensés et suivis qui transmettent des rapports d’activités aux structures 90% Dioum PV réunions de coordination des districts et régions 83 No. Indicator Criteria Baseline Goal Responsible Data Source publiques de santé Dénominateur : nombre de PPSP recensés et suivis médicales, rapports des PPSP transmis, enquêtes auprès des personnels des secteurs public et privés de santé 47 Numérateur : nombre de PPSP qui offrent des services de SR/PF Dénominateur : nombre de PPSP ayant des responsables orientés 100% Dioum Rapports des PPSP, rapport de supervision, rapports des districts 48 Nombre d’articles de presse diffusés pour relayer les activités de plaidoyer 35 par an Dioum Coupures de presse et éléménts audiovisuelles sur cassettes 84 ANNEX VI – DEMAND GENERATION DEMAND GENERATION: BEHAVIOR CHANGE COMMUNICATION + ADVOCACY COMPONENTS 1.1.1.1 METHODOLOGY 1.1.1.1.1 Review the MH/FP project design and guiding principles 1.1.1.1.2 Review the demand generation/Behavior Change Communication (BCC) interventions proposed under the various demand generation Intermediate Results (IRs): Can they effectively contribute to meeting the IRs 1.1.1.1.3 Review the various BCC (and advocacy) interventions in progress in terms of the IRs they fit under and the successes and challenges they present in view of meeting the IR specifically, the demand generation Key Intermediate Result (KIR), and SO3 through the MH/FP project mandate globally. We will also assess the extent to which each intervention adhere to the project guiding principles (client and community focus, quality, performance improvement, integration, leveraging, sustainability) 1.1.1.1.4 In addition to review of activity and evaluation reports, BCC related contracts, materials…data collection will be completed through qualitative discussions with key project, DSR, and district health personnel involved in demand generation activities. The following is an illustrative description of the review of select demand generation interventions with the target, item, and proposed methodology of review 85 Intervention Content: review what? Target: with whom Methodology How Rationale for strategy development Relevant project staff discussion Quality campaign development: formative research, mmd Project staff discussions 1.1.1.2 Pont d’Or 1.1.1.3 MH/FP services quality improvemen t and accreditatio n strategy Implementation*1 Project staff, relays, providers, community, commission qualite, ECD, radio station personel Focus groups? Interviews, discussions, observation, Project documents Content of programming Project staff, radio personnel, Review contracts Quality of/reaction on programming Écoutants, community, radio staff, commission qualité Interviews, radio feed-back from listeners Overall collaboration/impl ementation of radio *2 Radio staff, commission qualite Interview, discussions Mass-Media To increase konowledge of the benefits of MH/FP services and empower communities (radio communautaire de Fissel, Keur Momar Sarr?) Antennes SR décentralisées programming increases knowledge of mh/fp? Convinces to use services? Commissions qualite, communaute Interviews, discussions Evaluation reports 86 Organization, type, involvement, Monitoring, tools/CCC materials used Relays, ASC members, commissions qualite Interviews, discussions Community Activities (Thiadiaye, kaolack, Louga?) Community participation, benefit: knowledge, use of services Community Discussions Journées SR Benefit Providers, radio personnel, community Interviews Evaluation reports IPC/C training Helpful in improving performance? Equipment, materials for practice available? Proper follow￾up/supervision? suggestions Providers, project staff, DSR Interviews, supervision reports, curricula Progress? Problems? Suggestions? Script writing, review, translation, recording Project staff, radio partner Interview, discussion tapes Radio D L program As refresher for Relais polyvalents Content: usefulness, understanding, clarity Relays Pretest results 87 Others: MH/FP CCC strategy, BCC committee… Progress, challenges, suggestions Project staff, DSR Discussions MH/FP CCC messages & materials Understood, remembered, acted upon, effective? availability Community, providers Inventory, samples, interviews Activity reports Advocacy: PC, parliamentarians, EFI How effective? Practical results? suggestions Project staff, parliamentarians? pc, teachers? Activity reports, interviews Advocacy agent training Helpful in improving performance? Equipment, materials for practice available? Proper follow￾up/supervision? suggestions Advocacy agent trained District staff Activity reports, interviews Advocacy Forum sessions Understood, remembered, actions taken Community leaders/stakeholde rs, Partners, Activity reports, interviews Advocacy material Understood, remembered, availability Suggestions Community leaders/stakeholde rs, Partners, District staff Inventory, samples, interviews Activity reports Private sector involvement How effective? Practical results? Suggestions Private clinics District staff DSR Activity reports, interviews 88 *1 Review Implementation • Tool used for bridging exercice? • Training spcificities (service providers, relays…) • Launches (community and providers involvement?) • Selection Criteria for Pont d’Or sites? • Accreditation (community and providers involvement?) • Media (radio) campaign: how organised? Who is involved? What materials exactly are broadcast? How often? Is there a problem in such an arrangement? How could it work better? What can we do? • Community activities/commissions qualité: who are the members? Are they trained? who plans? Who implements? Who supervises? How are results accounted for? What planning/monitoring tools are available? Are they effectively used? Quels types d’activités exactement mènent-ils? Se sentent-ils capables de couvrir des activités de CIP/C sur la SM/PF et d’orienter les gens vers les services? • Pont d’Or relays: how comfortable are they in performing their role? Do they find their training helpful? Do they wish it were different? Do they have the necessary tools/materials? • La communauté est-elle consciente de l’existence des sites Pont d’Or? Comment l’ont-ils appris? S’y rendent-ils? Touvent-ils qu’il y a une différence? Ont-ils des suggestions? • Les prestataires:pensent-ils avoir les moyens de l’amélioration de leur performance? Ont-ils les outils/le matériel CCC approprié? Remarquent-ils une affluence comme conséquence aux activités communautaires et media? Si oui, ont-ils la capacité de les servir? • Is implementation (with planning and monitoring) participatory? • Other: Is there a sign directing to the clinic? Is there a sign on the clinic door/entrance? Do providers/relays have enough materials/IPC/C aids for MH/FP counseling or other activities *2 Is radio station happy with collaboration with project? What do community/commission qualité think about the radio programs? Do they work with radio stations? Do they work with the radio for their community activities? How involved are they in the programming? 89 ANNEX VII - Quelques résultats d’étape au regard de certains indicateurs I Mise en place (dans 20 sites urbains ruraux de 4 districts) d’un dispositif opérationnel de suivi de la grossesse, de l’accouchement et du post-partum grâce à l’ expérience Persuadeurs Communautaires (PC) = Approche : Plaidoyer en direction des élus locaux, leaders religieux, responsables des mouvements de femmes et de jeunes = Moyen de vérification : Rencontre avec les PC et exploitation du contenu de leurs outils (registres et carnets de référence) II Intégration de la Santé de la Reproduction dans le référentiel des compétences des Ecoles de Formation des Instituteurs (EFI) = Approche : Plaidoyer en direction des autorités scolaires (directeurs et chefs de services niveau central, chefs d’établissement, enseignants, parents d’élèves et élèves maîtres) pour une meilleure promotion de la SR à l’école = Moyen de vérification : Un module de formation initiale et continuée sur la Santé de la Reproduction disponible avec mise en œuvre à compter de l’année scolaire 20003/2004 III Collaboration entre structures publiques et privées de santé en nette amélioration et début d’implication des Points de Prestation de Services Privés (cabinets de soins infirmiers notamment) dans l’offre de services en SR dans le district de Kaolack = Approche : Mise en place de cadre de rencontre et de concertation entre prestataires publics et privés pour identification des axes d’une meilleure collaboration = Moyen de vérification : Rapports produits par les PPSP IV Processus de mise en place, au niveau régional, et local, d’une structure de veille, d’alertée et d’intervention rapide au profit d’une maternité à moindre risque = Approche : Tenue, en collaboration avec les Réseaux des Parlementaires, religieux et journalistes, de journées régionales du Parlement pour un plaidoyer en faveur de la réduction de la mortalité maternelle = moyen de vérification : Contact avec le bureau du Réseau des Parlementaires et exploitation des rapports des journées déjà tenues EL Hadji DIOUM, POLICY AEDVISOR Ministère de la Santé et de la Prévention Division de la Santé de la Reproduction Projet Santé Maternelle Planification Familiale (MSH) Interim Assessment November 3-21,2003 John Pollock Malcolm Bryant John McKenny Amelie Sow Introduction & Methodology The internal assessment is intended as a collaborative exercise with the field team to validate the approaches taken by the project, provide guidance on areas where success can be built on, identify areas where change or corrective action is needed, and identify unexpected changes in context or outcomes where a change in strategy may be required. •A Performance Improvement approach was used: •Full participation by the Project Team, •All stakeholders have been consulted •Recommendations target the stage of needed action (Goal definition, Root cause, Intervention Selection, Implementation) Performance Improvement Model GET and MAIN TAIN S TAKEHOLDER AGREEMENT AND PARTICIPATION IN THE PI PROCESS CONSI DER INSTITUTIONAL CONTEXT MISSION GOALS STRATEGIES CULTURE CLIE NT and COMM UNITY PERSPE CTIVES DESCRIBE ACTUAL PERFORMANCE DEFINE DESIRED PERFORMANCE FIND ROOT CAUSES Why does the performance gap exist? SELECT INTERVENTIONS What can be done to close the performance gap? IMPLEMENT INTERVENTIONS MONI TOR AND EVALUATE PERFORMANCE The Performance Improvement The Performance Improvement Process Process Strengths to build on and gaps C ON SULTAN T FAC TOR S – ROL E AS SU MED , C OMMU NICA TION SK I LLS Primary Strategies for Implementation The Three Primary Conceptual Strategy for the project has been based on: The “Fully Functional Service Delivery Point” Concept The “Systems Top” Model The “Stages of Programmatic Development” Grid Client S o c i o - c u l t u r a l E n v i r o n m e n t Supportive and representative political leadership Community ownership of services Community mobilization/involvement Mechanisms for local advocacy Local finance mechanisms IEC/BCC channels Communication Simultaneous ability to: Understand, Respond To & Act On Information Identify Health Needs Communicate Effectively Access Services HEALTH SERVICE USE AND DEMAND COMMUNITY PARTICIPATION AND SUPPORT Provider P o l i c y E n v i r o n m e n t Service Delivery Point Delivery of Defined Quality Services Responding to Client Needs HEALTH SERVICE SUPPLY Simultaneous existence of: Trained and Motivated Staff Sufficient Equipment, Drugs, & Supplies Adequate Infrastructure Functioning Referral System Access to Information HEALTH MANAGEMENT SYSTEM Coordination & Planning Quality Assurance Supervision & Training Information System Human Resources Financing Organizational Structure Drug Logistics Marketing Community Mobilization Has rights to: Clear information Access to services Choice of FP method Safe services Privacy Confidentiality Dignity, comfort Free expression of opinion Continuous supplies Has rights to: Good supplies Good working environment Good management and supervision Clear information and training Social environment Enables clients DEMAND & SUPPLY Provider performance Management & service Local ownership To act on Demand USE to accepted standards support systems Fully Functional Service Delivery Point THE SYSTEMS TOP COMMUNITY MUNICIPALITY DISTRICT REGION NATIONAL S.D.P. COORDINATION AND COMMUNICATION AND PLANNING HUMAN RESOURCES SUPE RVISION FINANCING INFORMATION ORGANIZATIONAL STRUCTURE QUALITY ASSURANCE LOGISTICS SYSTEM COMMUNITY MOBILIZATION FACILITY MUNICIPALITY DISTRICT REGION NATIONAL S.D.P. FACILITY COMMUNITY HUMAN RESOURCE MANAGEMENT AT NATIONAL LEVEL HUMAN RESOURCE MANAGEMENT AT DISTRICT LEVEL HUMAN RESOURCE MANAGEMENT AT FACILITY LEVEL ANALYZING THE SYSTEM Stages of Programmatic Development - Management systems robust, able to adapt to changes in the environment - Supervision integrated into all activities and is interactive and problem solving - Expanded focus on strategic planning, data for decison-making, control over resources - Management systems more flexible, efficient, sophisticated -Supervision remains intermittent but is supportive and facilitative - Increased attention to strategic planning, coordination - New focus on data for decision-making, efficiency, cost recovery - Management systems strengthened - Supervision takes place but is a “control” and is intermittent - Basic systems for planning, training, logistics, management info., financial management - Supervision rarely takes place Management and Leadership - Widespread awareness - Widespread expression of demand - Active community involvement - Demonstrated “behavior change” - Widespread awareness; harder-to￾reach populations beginning to express demand - Growing communi- ty involvement - BCC/IEC messages well targeted, using multimedia approaches - Expanded awareness - Nascent community involvement - BCC/IEC messages/materials more targeted, exploring alternative media - Awareness limited to more urban, more educated populations - IEC messages/ materials broadly focused Demand - Services perceived to be of high quality and “client focused” - Ongoing quality assurance program in place - Quality concerns expand to include choice, convenience, “client focus” - More attentionto monitoring of quality - Increasing attention to quality of care, specifically norms and standards, supervision, monitoring of quality - Major focus is provider training - Little monitoring of quality Quality - Widespread network of service delivery channels - Virtually no barriers to access Multiple service delivery channels, effective public-private collaboration - Few remaining barriers to access - Multiple service delivery channels - Still many barriers to access - Limited # service delivery channels - Huge barriers to access Access Stage IV Optimum Utilization of Services Stage III Good supply, good demand, good utilization Stage II Good supply, growing demand Improving utilization Stage I Low supply, low demand, poor utilization Assessment Goals • Strategy & Principles: How are they applied in the face of reality • Results, and progress toward results (IRs 3.1, 3.2, 3.3) • Context that influences results Collaboration……….. Systems Motivation Resources • What should be done now? (Current plans- Surprises) • How should we do it? (Strategic Adjustment) • How will we know we have done it? • How will we share our lessons and experience? Technical – DSR, CAS Political – MCR, MCD, Parlmentaire Socio-Cultural – Community, Religion Monitoring & Evaluation Are we effectively linking our plans to our results? Have we correctly identified the results we have achieved? PLANS RESULTS BETTER PLANS RESULTS MOTIVATION Priorities in Work Plan Review current work plan against Budget resources and set priorities and contingencies. Major factors: Current rates of expenditure Loss in buying power of US$ (14% appx.) Potential extension of technical plan through 9/06 800k 14% 01 02 03 04 05 06 07 Key Questions from USAID & MSP/DSR Alternative Approaches to Expand CBD IUD vs Norplant Role of Natural Family Planning Integration of FP into HIV/AIDS Discussion of IPT Missed Opportunities: (Men, Bednets) Use of Media (and Mass Media) Evolution of Pont d’Or Using FFSDP as base standard for all service delivery points: Basic Skill development Formative Supervision Community Cope/PI Broadcast IEC/BCC Community Advocacy Persuaders Commaunitaire ARPV (future) Use Pont d’Or to motivate through self selection +/or Competition and to sustain engagement of power of community Development of Selection Criteria At clinic level: Staff performance in SOUB, Contraceptive technology, CPN/delivery/Postnatal Client-provider interaction for MH and FP Contraceptive logistics and Continuity of services: Availability of methods, Absence of stock outs Infection Prevention At community level: Contribution to health services improvement (financial and human) Community involvement and overall leadership around the SDP Active engagement of commission qualite and comite de gestion Completion of consensual quality improvement action plan Relais Imrove Impact and Motivation through: Provision of materials Leaflets to distribute Resdesigned flip chart Hands-on kit Income generating projects Income replacement (ARPV to come) Women’s groups > Mutuelles>Maximize Access to MH/FP services & creation of a resource (work with DISC on this) Religious Leaders Updated/Revised/Complemented Religious Leaders’ Kit (with Arabic version) TOT program where leaders become advocates to target both leaders and lay people Public presentations by Religious Leaders Video TV Other Include in thinking about “hotline for men” Access to Men Consider “Hotline” with specific number and time slots for male Relais/Religious Leader/Sage Femme Focus on causerie and discussion with groups of men in community Media…..utilize local radio capacities & expertise for targeting audience & interactive programming Target groups: Truck drivers; fishermen; wrestlers, etc. IUD There is a lack of skilled staff (loss of skill) Dislike or distrust of method Negative reinforcement > complications > stopped using Need Health Education Need skills, but limited: Suggest MCD be trained & hold clinic during supervisions (advantage = focus for supervision new idea for commun. can also teach ICP Requires reinforcement of client/provider interface warm greeting all methods (build on & through formative supervision) Leadership MCD & Mayor Lack Technical skill & experience Lack leadership skills Administrative job description supervisory role for posts sante = control & not useful Solution: MCD can teach skills (e.g. IUD) Give Leadership Training (able to inspire & generate confidence) Follow leadership training with Distance Learning Same for ICP and commune leader Now Schools Reinforce program innovation with Ministry of Education by adding messages from leaders (Religious, etc) to curricula Malaria What are the mechanisms for: Distribution > Social Marketing > Vouchers > Groups Motivation > Formative research > Communication Monitoring Use Other Inclusion of PMTCT (strategy & policy in formation) Engagement with Private sector to expand Access Continuation of efforts with Parlementaire Operations Systems Clear policies are documented and adhered to Staff work to respond to evolving activity demands Resource constraints: vehicles staff level Security Set standards for # of participants in events Try to prepare travel & per diem payments in Envelopes (Travel time included in event) Safes in regional offices/locations Security guard?? Travel in daylight, lock doors, stay on main roads Results Review of Indicators to measure progress Project contribution in relation to other programs What are we Changing? How do we know? What does that mean? 89 ANNEX VII Quelques résultats d’étape au regard de certains indicateurs I Mise en place (dans 20 sites urbains ruraux de 4 districts) d’un dispositif opérationnel de suivi de la grossesse, de l’accouchement et du post-partum grâce à l’ expérience Persuadeurs Communautaires (PC) = Approche : Plaidoyer en direction des élus locaux, leaders religieux, responsables des mouvements de femmes et de jeunes = Moyen de vérification : Rencontre avec les PC et exploitation du contenu de leurs outils (registres et carnets de référence) II Intégration de la Santé de la Reproduction dans le référentiel des compétences des Ecoles de Formation des Instituteurs (EFI) = Approche : Plaidoyer en direction des autorités scolaires (directeurs et chefs de services niveau central, chefs d’établissement, enseignants, parents d’élèves et élèves maîtres) pour une meilleure promotion de la SR à l’école = Moyen de vérification : Un module de formation initiale et continuée sur la Santé de la Reproduction disponible avec mise en œuvre à compter de l’année scolaire 20003/2004 III Collaboration entre structures publiques et privées de santé en nette amélioration et début d’implication des Points de Prestation de Services Privés (cabinets de soins infirmiers notamment) dans l’offre de services en SR dans le district de Kaolack = Approche : Mise en place de cadre de rencontre et de concertation entre prestataires publics et privés pour identification des axes d’une meilleure collaboration = Moyen de vérification : Rapports produits par les PPSP IV Processus de mise en place, au niveau régional, et local, d’une structure de veille, d’alertée et d’intervention rapide au profit d’une maternité à moindre risque = Approche : Tenue, en collaboration avec les Réseaux des Parlementaires, religieux et journalistes, de journées régionales du Parlement pour un plaidoyer en faveur de la réduction de la mortalité maternelle = moyen de vérification : Contact avec le bureau du Réseau des Parlementaires et exploitation des rapports des journées déjà tenues EL Hadji DIOUM, POLICY AEDVISOR