LESSONS LEARNED FROM THE USAID/SÉNÉGAL COMMUNITY HEALTH PROGRAM (CHP) FINAL EVALUATION OF CHP, AS IMPLEMENTED BY ChildFund 2006-2011 MAY 2011 This publication was produced for review by the United States Agency for International Development. It was prepared by Initiatives Inc. LESSONS LEARNED FROM THE USAID/SÉNÉGAL COMMUNITY HEALTH PROGRAM (CHP) FINAL EVALUATION OF CHP, AS IMPLEMENTED BY ChildFund 2006-2011 Souleymane Barry Eliot Putnam Cheikh Tidiane Touré   The information contained in this report is the product of an evaluation conducted by Initiatives Inc. The report was prepared by Initiatives Inc. under Purchase Order No.: 685-O-00-11-00104-00 issued by the U.S. Agency for International Development. Initiatives Inc. 264 Beacon Street, 6th Floor Boston MA 02116 USA TEL (617) 262-0293 FAX (617) 262-2514 www.initiativesinc.com DISCLAIMER The opinions expressed in this report are those of the individual authors and do not necessarily represent the views of the United States Agency for International Development or the United States Government. i ACRONYMS ADEMAS Agence pour le Développement de Marketing Social ADC Agent de Développement Communautaire (Community development agent) APS Annual Program Statement of USAID ARI Acute respiratory infections (infections respiratoires aigues) ASC Agent de Santé Communautaire (or CHW) BR Bureau Regional (Regional coordination office, managed by Abt Associates) CHW Community Health Worker (or ASC) CA Cooperative Agreement CHP Community Health Program (Programme Santé Communautaire) CMO Chief District Medical Officer COP Chief of Party CPI Counterpart International CRS Catholic Relief Services DANSE Division de l’Alimentation, Nutrition et Survi d’Enfant (Child Survival/MOH) DLSI Div. de Lutte contre SIDA et Infections Sexuellement Transmises (AIDS/STI￾MOH) DSR Div. de la Santé de la Reproduction (Reproductive Health-MOH) DSSP Div. des Soins de Santé Primaires (Primary Care/MOH) ECS Educateurs Communautaires de Santé (fills same role as ADC) EGF Excisions genitals féminines (female genital mutilation) FHI Family Health International GMO Groupe de Mise en Oeuvre (program implementation unit in Plan regions) GOS Government of Sénégal HSS Health systems strengthening ICP Infirmière Chef de Poste (Chief Nurse, Health Post) IEC Information, education and communication IRA Infections respiratoires aigues (acute respiratory infection) MCD Médecin Chef du District (Chief District Medical Officer) ii MNCH Maternal, newborn and child health MOH Ministry of Health of Sénégal MSP Ministère de Santé Publique (MOH) NGO Non-governmental organization OIP Offre Initiale des Pilules (Pilot project to introduce contraceptive pill) ONG Organisation non-gouvernementale PMI President’s Malaria Initiative PNDS Plan National de Développement Sanitaire PNLP Programme National de Lutte contre le Paludisme (Malaria Program/MOH) PNT Programme National de Lutte contre la Tuberculose PSC Programme Santé Communautaire (ChildFund-led Community Health Program) RB Regional coordinating bureau (established by Abt Associates in Thiès and Kolda) SNEIPS Service National d’Education et Information pour la Santé (IEC for health) SPNN Santé pre-natal et neo-natal (pregnant and newborn health) SSP Soins de santé primaire (primary health care) Health Hut case de santé Trained birth attendant matrone    iii TABLE OF CONTENTS ACRONYMS ................................................................................................................................................. i TABLE OF CONTENTS ............................................................................................................................. iii ACKNOWLEDGEMENTS .......................................................................................................................... v EXECUTIVE SUMMARY .......................................................................................................................... 1 I. INTRODUCTION ........................................................................................................................... 2 II. PROGRAM CONTEXT .................................................................................................................. 4 III. METHODOLOGY OF THIS EVALUATION ................................................................................ 4 i. Methodological approach .............................................................................................................. 4 ii. Phases of data gathering ................................................................................................................ 5 iii. Sites and target groups .................................................................................................................. 6 iv. Reflections on evaluation methodology ........................................................................................ 6 IV. FINDINGS ....................................................................................................................................... 7 i. Overall approach and management of program ............................................................................ 7 ii. Quality and coverage of community health services .................................................................. 10 iii. Referral systems .......................................................................................................................... 12 iv. Community awareness and utilization of services ...................................................................... 12 v. Socio-cultural barriers limiting access to services ...................................................................... 14 vi. Respect for MOH norms and protocols ...................................................................................... 15 vii. Sustainability ........................................................................................................................... 15 V. RECOMMENDATIONS ............................................................................................................... 17 i. Service quality, referral and supervision ..................................................................................... 17 ii. Program coordination and management...................................................................................... 17 iii. Synergy between cases de santé and Health Posts ..................................................................... 18 iv. Sustainability ............................................................................................................................... 18 v. Operations research and evaluation ............................................................................................ 18 VI. LESSONS LEARNED ................................................................................................................... 19 VII. GENERAL CONCLUSION .......................................................................................................... 19 ANNEX A ................................................................................................................................................... 20 ANNEX B ................................................................................................................................................... 22 ANNEX C ................................................................................................................................................... 24 ANNEX D ................................................................................................................................................... 25 v ACKNOWLEDGEMENTS The members of the team selected to carry out the final evaluation of the USAID-funded Community Health Program in Sénégal wish to express their gratitude to Initiatives Inc. for entrusting such an important piece of work to them. We have been especially appreciative of the timely and responsive support given the team throughout the evaluation process by Initiatives’ staff members Jackie McConlogue and Nicole Dupré. We were most grateful for the time given the team by senior managers of key Divisions of the Senegalese Ministry of Health, and for their willingness to share their perceptions of and hopes for the community health program. We also wish to warmly thank the USAID/Sénégal staff for their assistance during the evaluation. Special thanks go to Ramatoulaye Dioume of the Health Office, who was our principal contact, and Sounka Ndiaye, USAID Monitoring and Evaluation Specialist, for their oversight and assistance in scheduling team briefings and debriefings. The team was warmly and supportively received throughout the evaluation by staff members at all levels of the ChildFund-led consortium that has implemented the Community Health Program since 2006. Chief of Party Mamadou Diagne welcomed us to a comprehensive briefing on the program at its Thiès headquarters as we began our work, and made himself and his staff accessible to us for the duration of the assignment. We were continually impressed by his good humor and passion for the program he leads. Finally, we express our deep admiration for the people who, supported by this program, actually do the work of making primary health care accessible to children and families in rural communities of Sénégal. The community health agents, matrones and outreach workers who, with minimal compensation, staff cases de santé and extend services to residents of villages who would otherwise have no health care, are the unsung heroes of this program. Souleymane Barry Eliot Putnam Cheikh Tidiane Touré May 2011 1 EXECUTIVE SUMMARY A cornerstone of USAID’s support for the health sector in Sénégal over the five-year period beginning July, 2006 has been implementation of a program to strengthen primary health care at the community level. This has been funded through a cooperative agreement (CA) with a consortium of six NGOs (four at the start, with two added later) led by ChildFund, previously known as the Christian Children’s Fund. The goal of the Community Health Program (CHP) has been to ensure widespread access to a basic package of primary health care services in rural communities, largely through revitalizing and staffing an existing but greatly underutilized network of health facilities known as “health huts”, or cases de santé1 . Although lacking its own resources to support these facilities, the Senegalese Ministry of Health considers them the essential base of the country’s “health care pyramid”. It has high hopes that this initiative will bring dependable primary care closer to rural populations, whose access to such services in government Health Posts and Health Centers has always been limited by distance and cost. This is a large and ambitious program, originally funded at about $13 million, but now, after expansion of its coverage from five to thirteen regions, valued at $26 million, with further expansion to virtual national coverage planned in a new five-year phase. Since a mid-term evaluation of the CHP was not performed, a final evaluation was considered essential to learning from the experience to date and informing program expansion. To this end, Initiatives Inc., a Boston-based consulting firm, was contracted by USAID to undertake the assignment, and a three-person team recruited to conduct the evaluation in March and April, 2011. The methodology of this exercise, described in detail in the report that follows, included meetings with stakeholders, individuals and groups at all levels of the health system. The evaluation team’s work was organized around a two-week series of visits to different areas of the country where the CHP is implemented. Care was taken to be sure the team visited Health Districts where each member of the program consortium was the implementing partner. Interviews and focus group discussions were guided by a data gathering instrument developed by the team, which is attached as an Annex to this report. It was designed to solicit information on a series of questions and issues specified in the Scope of Work for the evaluation: (1) quality and range of health services offered to communities; (2) referral and follow-up systems; (3) community awareness and utilization of services; (4) socio-cultural barriers to accessing services; (5) respect for norms and protocols of the MOH; (6) the soundness of program approaches and management; and (7) sustainability of services put in place by the program. Overall, the evaluation team found that the CHP has made significant headway in strengthening access to primary health care services in previously underserved rural communities in regions and districts across the country. Services offered through the program’s actions have become not only appreciated but expected by residents of rural communities served by functioning cases de santé, as attested to by numerous individual and focus group discussions held by the team. The CHP has made primary care of good quality more accessible to residents of the communities it reaches, in the process easing the patient load of government health facilities. Of note also is the enthusiasm for the program and its progress among leadership of key departments of the Ministry of Health. The MOH is anxious for this program to succeed because it considers community                                                             1 The evaluation team has chosen to use the French term for these facilities, one that is recognized throughout Sénégal, since the term “health huts” does not accurately capture their substance. 2 health to be an essential element of national health care strategy, one that it hopes that it will be able to absorb into its budget before too long. There are areas, however, in which the CHP requires significant strengthening. These are the subject of recommendations by the evaluation team described at length in this report.  Services. The team found that the range of services offered in the basic package by cases de santé is appropriate and should not now be expanded. But work needs to be done to systematize the process by which patients are referred to Health Posts, as well as to strengthen synergy between Health Posts and cases in other areas. Skills of Agents de Développement Communautaires (ADCs) recruited by the CHP to supervise cases and their teams and to interact with health committees and other community bodies, must be reinforced through additional training.  Program approach. USAID should insist on joint reviews and better coordination among the various health sector programs that it funds so as to maximize their inputs. It is also suggested that additional resources be sought for meeting the physical needs of cases.  Sustainability. Securing the benefits of this program for the long haul, when external support is reduced or no longer available, is on everyone’s mind and is the subject of several recommendations. These include drawing on models developed by individual consortium members and on ideas put forth by MOH officials at many levels, and developing approaches whereby resources within the community can be mobilized to strengthen and assure the sustainability of the services they so clearly value. Among other things, establishment of a national level working group on this subject is proposed. Again, the evaluation team makes several recommendations in these and other areas in this report (see pp. 17-19). It hopes that they and the efforts of all of the many individuals, government units, NGOs and donors that have such an interest in this program will help to further develop the CHP as a model for delivering health services to underserved communities. I. INTRODUCTION In July, 2006, USAID/Sénégal awarded a consortium led by the Christian Children’s Fund, since renamed ChildFund, a five-year Cooperative Agreement (CA # 685-A-00-06-00061-00) for the purpose of implementing the community health component of its overall support for the Senegalese health sector. The Community Health Program (CHP) responded to geographic and programmatic priorities spelled out in USAID’s “Annual Program Statement 2006-2011” (APS). It built on previous USAID-funded efforts to strengthen primary health care at the community level, and complemented other initiatives in areas such as health policy development, health systems strengthening (HSS) and social marketing. And it was designed in coordination with the Government of Senegal’s Plan National de Développement Sanitaire (PNDS). The goal of the CHP was to provide a paquet de services de base, or basic package of preventive and curative services, that would respond to the pressing primary health care requirements of rural communities – maternal, newborn and child health (MNCH) care, treatment of diarrhea and acute respiratory infections (ARI), malaria prevention and treatment, HIV/AIDS and TB prevention, family planning, nutrition and other community health needs. It would do so by equipping and staffing a network of rural cases de santé, typically two to three room concrete structures, usually with a room for patient consultations and a room for deliveries. They were built with support from USAID and other donors under previous projects, as well as by communities themselves, but were understaffed, underutilized, and, in many cases, closed altogether. Although cases de santé are not funded or staffed by the government, they are recognized by the Ministry of Health (MOH) as the essential base of Senegal’s health care pyramid, and are linked to public sector Health Posts and Health Centers through supervisory and supply systems. The MOH has made clear its desire that this base be strengthened and made more responsive to community health needs, but does not presently have resources to do so itself. 3 The program’s support for each case de santé centers around the recruitment and training of a team that includes (1) a Community Health Worker (CHW), or Agent de Santé Communautaire, to be in charge of treating patients in the facility and referring those requiring treatment by a trained clinician to the nearest Health Post; (2) one or more trained birth attendants, or matrones2 , whose role is to provide counseling about pregnancy, attend emergency deliveries, and offer family planning information to interested women; and (3) outreach workers. The role of the latter is to reach out to the community, organizing causeries, or discussion groups, with community members in their homes and villages to discuss health issues, promote healthy behaviors, and encourage use of the preventive and curative services available to them at the case. It is important to note that all members of the case team are volunteers. The program provides training and occasional refresher training of this team in delivery of the basic package of services. CHP inputs in support of the case de santé also include a small kit of birthing equipment (gloves, scissors, speculum, and pan), a beginning stock of medications and other supplies, such as ACT for malaria, diarrhea medicine, iron tablets, headache pills, and bandages, as well as informational materials. In a few instances support has also included repair or renovation of the case and provision of some furnishings, such as a delivery table. Some members of the program consortium (see below) also provide a small quarterly stipend to CHWs, matrons and outreach workers. The reasoning behind the CHP’s relatively modest level of direct support to the cases de santé and their teams is that sustainability of their work will depend on the engagement of communities in which they are located in generating this support, a strategy discussed further below. Supervision of the case de santé team and its function is the responsibility of a cadre of Agents de Développement Communautaire (ADC), roughly translated as Community Development Agents, who are recruited, trained and paid by the CHP. (In areas managed by some consortium members ADCs have different titles, but their role is the same.) That role is to monitor and support the work of the team at the case de santé, and oversee record keeping for treatments, deliveries and referrals. The ADC is also supposed to maintain linkages with government Health Posts and Health Centers, and liaise with community entities, such as village health committees, to encourage generation of financial support. A key supervisory role is also played by the Infirmier Chef de Poste (ICP), or Chief Nurse at the nearest Health Post. The ICP is responsible for visiting cases de santé in his/her jurisdiction on a regular basis to vaccinate children, give injections, etc. (i.e., services the CHW is not trained to provide), and monitor quality of services offered at the case. At its inception in 2006, the CHP CA was funded at a level of approximately $13 million. The implementing consortium, led by ChildFund, included Africare, Plan and World Vision, and the program at that time targeted five regions of the country for the strengthening of community health services. Midway through the program two additional NGOs, Catholic Relief Services (CRS) and Counterpart International (CPI), joined the consortium, at first focusing primarily on malaria prevention under the President’s Malaria Initiative, or PMI. As the program nears its completion in June of 2011, it has grown to a level where it is supporting 1,620 cases de santé in 13 regions and 65 health districts, and its total funding has risen to $26 million. A new, five-year phase of the program is presently under review by USAID/Sénégal. It is understood that the goal of that phase will be to enroll most, if not all, cases de santé in all regions of the country, including underserved urban areas of Dakar. Since a mid-term evaluation of the Community Health Program was not performed, a final evaluation of its success over the past five years in strengthening health services at the community level was considered essential by USAID/Sénégal. Although, as noted, a decision has already been made to continue and expand the program, conclusions drawn and lessons learned from activities during its first five-year phase will be critical to informing the program’s expansion. To this end, a three-person team was recruited by                                                             2 Matrone, the French term for “trained birth attendant”, is used throughout this report. 4 Initiatives Inc., a Boston-based consulting firm, to undertake the final evaluation of the CHP, using a scope of work (Annex D) developed by USAID that specified key issues it felt required priority attention. Field work for the evaluation began in early March, 2011, and proceeded as described below. II. PROGRAM CONTEXT Governments, NGOs and donors have for years, through implementation of pilot and full-scale programs and experimentation with different decentralized systems, sought effective and sustainable approaches to the provision of primary health care to rural communities in resource-poor settings. For such communities, hospital services are impossibly remote, and Health Centers and even Health Posts are often in locations so distant as to make access problematic for families unable to pay for transport. In such situations, even meager community-based primary health services, whether offered from some sort of facility or from volunteers’ homes, may be these families’ only lifeline. But maintaining such services, especially without external support, has proven to be difficult, and their coverage and quality are often suspect. The existence of a network of already constructed cases de santé in rural communities throughout Sénégal provided an opportunity to develop, under the ChildFund-led Community Health Program, a comprehensive approach to the provision of primary health care at the community level. Creation of the “basic package” of services, training of provider teams, and an overt dependence on communities to support their local case de santé, are all elements of an ambitious and innovative attempt to create a sustainable model of health care for residents of rural villages. If it can be shown to be working, and if it can be maintained as a viable model, even in the absence of significant government support, it could have significance for similar efforts in other countries. A second feature of this program of interest to others, and to the evaluation team, is the size and makeup of the implementing consortium. Led by ChildFund, it consists of six international NGOs, all with different operating philosophies, and their ability to “pull together” while retaining their individuality is critical to program success. One other contextual feature of the Community Health Program is that it is a centerpiece of the portfolio of health interventions described in USAID’s APS for the period 2006-2011. This in turn was designed to integrally support broad policy directions of the national health program as set forth by the Government of Sénégal in its PNDS. III. METHODOLOGY OF THIS EVALUATION i.   Methodological approach The purpose of this evaluation was to analyze the achievement of the Community Health Program in terms of key objectives for the improvement of community health in Sénégal. These included improvement in maternal, newborn and child health (MNCH), reduction in the incidence of infectious diseases such as TB and malaria, and improved treatment for common afflictions such as diarrhea and acute respiratory infections (ARI). As a qualitative evaluation, it was designed to gather information on changes that may have occurred, as a result of CHP interventions, in knowledge, attitudes and practices favorable to improved health in target populations. Throughout the evaluation process, a participatory approach was favored that involved interaction with NGO partners of the CHP consortium, field personnel responsible for program implementation and supervision (case de santé teams, ADCs, supervisors and coordinators), community leaders, and citizens of villages and communities visited by the evaluation team. Such an approach makes possible exposure to a wide range of perspectives, attitudes and opinions on the part of principal actors at different levels. It 5 creates opportunities for dialogue in the community that go beyond a simple appreciation of activities and results, and it highlights gaps and weaknesses that, if addressed, can positively affect future actions. ii.   Phases of data gathering Document review.  The evaluation team consulted a wide range of documents made available by USAID/Sénégal and the CHP consortium. Original program materials, quarterly and annual program reports, annual program plans and budgets, and other programmatic and technical documents were reviewed by the team, both before the evaluation began and while it was proceeding. (A bibliography of the various documents consulted by the team is contained in Annex A.) This ongoing review enabled the team to have an appreciation of the scope of the program and its various activities, and prepared it to effectively absorb information from its many interviews and discussions with individuals and groups at all levels. Meetings with key stakeholders. The team held a series of meetings in Dakar with different public and NGO sector stakeholders having direct or indirect interest in the strengthening of community health care in Sénégal. First and foremost, these included officials of Divisions of the MOH responsible for primary health care (DSSP), nutrition and child survival (DANSE), reproductive health (DSR), and health education and communication (SNEIPS). They also involved senior administrators of national programs to combat malaria (PNLP) and tuberculosis (PNT). These meetings enabled the team to gain an appreciation, through the perspectives of the different divisions and programs mentioned, of the high priority that the Ministry of Health places on community health in the context of its other national priorities. The team met in their Dakar offices with representatives of the individual members of the Community Health Program consortium (ChildFund, Africare, Plan, World Vision, CRS, and CPI) and had a full-day briefing from the CHP coordinating team at its main office in Thiès. It also met in Dakar with NGOs responsible for other elements of USAID-funded support for the Senegalese health sector, including IntraHealth, FHI, Abt Associates and ADEMAS. These meetings enabled the team to assess the level of integration, especially at field level, of different interventions, and the challenges the NGOs and USAID face in terms of ensuring effective coordination. Information gathering in the field. Data on which findings of this evaluation are based was derived from surveys conducted by the evaluation team between March 21 and March 30, 2011 in six regions of Sénégal. In localities visited, three techniques were used to gather information from different target groups:  Informal meetings, primarily in the form of unstructured conversations, were held with key informants who played various roles in the management of community affairs, and with project personnel at all levels. This permitted the gathering of general information in a spontaneous and non-threatening manner.  Semi-structured meetings, as their name implies, were used to gather qualitative information through semi-directive interviews, using an interview guide developed by the team. The guide (see Annex B) outlines themes and sub-themes to be used with specific target groups, based on the priority issues that the evaluation was asked by USAID to examine. Such meetings were held with District Chief Medical Officers (CMO), primary care supervisors, chief nurses at Health Posts (ICPs, or Infirmières Chef de Postes), teams of the cases de santé visited (CHWs, matrones and outreach workers), community leaders (both men and women) and ADCs and other program personnel.  Focus group meetings with 8-12 participants each were used to probe attitudes and opinions with respect to community health needs and concerns, as well as the work of the cases de santé, in settings allowing for lively exchange of views among participants. Focus groups were 6 organized for separate groups of men and women in communities visited by the evaluation team, usually at the location of the case de santé. iii.   Sites and target groups In selecting regions and sites to be visited, the evaluation team was guided by the need to visit areas in which each of the six members of the CHP consortium was the principal implementing partner, hopefully with some geographic diversity, and to do so over a 12-day period. Within each region selected, the team then chose a health district (district sanitaire) served by the program that was reasonably accessible and within that district a Health Post that served several cases de santé. In addition to visiting the Health Post and talking with its ICP, two cases were selected for visits by the team, one reasonably close to the Health Post and one at a distance. This selection was made so the team could assess issues of access, referral and counter-referral of patients between the case de santé and the Health Post, and frequency and quality of supervision of the case by the ICP. The following table summarizes the visits made by the team according to the various criteria outlined above: REGION DISTRICT HEALTH POST CASES VISITED NGO RESPONSIBLE TAMBACOUNDA Koumpentoum Mereto Keur Daouda Touba Sine AFRICARE KAFFRINE Kaffrine Ndiognick Louméne SegréSecco WORLD VISION KAOLACK Ndoffane Keur Baka/ Thiaré Ganda Wolof Koki CHILDFUND DIOURBEL Diourbel Ngohé Lagnar Mbacfasagal CRS SAINT-LOUIS Richard-Toll Savigne Ndiougue Diagambal PLAN MATAM Matam Ogo Hombo Diandioly CPI In all localities visited, the team made certain to hold meetings and discussions with the same target groups and individuals, namely:  District CMO and Primary Care Supervisor  Chief Nurse at the Health Post  Field staff of CHP implementing partner (ADCs, Coordinators, Supervisors)  CHWs, matrones and outreach workers at the case de santé  Members of the Health Committee overseeing the case de santé  Community leaders and elected officials  Groups of men and women of the community iv.   Reflections on evaluation methodology Considering limitations of time and geography, this evaluation could not hope to understand or describe the status of a program as large as the CHP in all of its areas of intervention throughout the country. The evaluation team’s mission was not to automatically extrapolate its findings to cover the entire program. However, the care given to selection of sites to be visited and target groups to be interviewed did, we feel, ensure that those findings would lend themselves to recommendations worthy of consideration for the program as a whole. 7 IV. FINDINGS i.   Overall approach and management of program Interventions of the Community Health Program directly address challenges set forth in the PNDS 2009- 2018 of the GOS, which calls for an acceleration of the battle against maternal, neo-natal and infant/juvenile morbidity and mortality, and specifically recognizes the necessity of a partnership with the community in promoting better health. The CHP is equally responsive to priorities defined in USAID/Senegal’s APS, which commits the agency to the reinforcement of technical assistance leading to more effective efforts to meet the country’s most critical health care needs-reducing maternal and infant mortality and combating infectious diseases. In its implementation, the CHP also takes into account recommendations of a technical review ordered by USAID in 2009 that promoted extension of family planning. These reference points are interesting in that they reinforce the overall coherence and l’alignement3 with respect to national priorities. Context for internal management of CHP The Community Health Program, as previously noted, is implemented by a ChildFund-led consortium of six international NGOs. Originally composed of four NGOs (ChildFund, Africare, Plan, World Vision), with the advent of the President’s Malaria Initiative (PMI) in 2007 CRS and CPI were added to the consortium, responsible for geographical areas previously uncovered by the CHP. At first the two new consortium members offered only malaria prevention and treatment services in cases de santé in their areas of intervention, contributing significantly, it should be said, to what is widely agreed to have been an effective national malaria campaign. Only in 2009 did CRS and CPI integrate the full basic service package into the cases de santé for which they are responsible. The evaluation team observed that people in these areas, in contrast to communities served by the original four consortium members, were as yet not fully aware of the expansion of services available at the case, or of their advantages to the community in terms of preventive and curative care. These problems of equity in accessing services should, in the team’s view, decrease over time. Internal coordination Early on, realizing that each consortium member has its own organizational style and equally distinct approach to work in the community, and seeking ways to use that diversity to the benefit of the program, the CHP instituted a system of quarterly coordination meetings. These gatherings give members the chance to inform each other on the evolution of indicators in their respective zones of intervention, discuss problems and issues affecting implementation of the program, and update each other on the execution of decisions taken at previous meetings. Coordination meetings are hosted in turn by each consortium NGO partner on its own “turf”, enabling the partner to organize group visits to the field where its approach can be demonstrated and all members have a chance to exchange ideas and experiences. These meetings also enable program coordinators at the national level to assess the need for program and policy adjustments and receive feedback from the operational level. Coordination with the MOH and its key partners At the mid-point of the CHP, when the focus had been almost entirely on interventions at the community level, it became clear that coordination at the central level, with MOH departments and programs such as DANSE, PNLP, DSR and PNT, needed strengthening. This was accomplished through a deliberate CHP effort to increase regular consultations, reviews and joint supervisory visits. This strategy has not only led                                                             3 The principle of alignement concerns the adequacy of donor policies and priorities in terms of their alignment with national priorities, as expressed by the country’s leadership. 8 to a spirit of expanded collaboration at the health district level but also has had a positive impact at the level of cases de santé. CHP’s efforts at broad-based collaboration at all levels have in fact been a positive alternative to more traditional vertical approaches that vested all coordinative responsibilities at the top. In addition to strengthened coordination with the central MOH, the program maintains strong linkages with health districts, which have been responsible for supervising training of the CHWs and matrones that staff cases de santé. In addition, in situations where more than one member of the consortium is represented in the same region, weaknesses in intra-regional program coordination have been minimized. Coordination with other USAID-funded agencies Despite the best intentions, efforts to maintain complementarity between the CHP and health sector programs implemented by other USAID-funded agencies, notably FHI, ADEMAS and IntraHealth, have not, in the view of the evaluation team, been entirely successful. Exchange between different NGO players at different levels of the health system is clearly inconsistent. In an effort to ensure good collaboration, USAID contracted with Abt Associates to establish regional bureaus (RB) in Thiès and Kolda, as a mechanism for reinforcing coordination, visibility and effectiveness of USAID-funded health initiatives at the health district level. Each program placed representatives in the RBs, with the exception of the CHP which obtained a waiver from USAID allowing them to keep staff in their central office in Thiès while maintaining functional linkages with the RB. This arrangement has not worked to everyone’s satisfaction, and it is clear that there is room for improvement in this area. Better coordination will help strengthen performance of all agencies, so that they, in turn, can help different levels of the health system more effectively contribute to positive accomplishments in the community. Approaches to the community of CHP consortium members In assembling a consortium of international NGOs, the CHP was aware of the need to make the most of each partner’s particular areas of experience and expertise while at the same time pursuing the common objective of strengthening community health. From an operational point of view, this meant that each member of the consortium would follow its own unique approach to development while staying true to the CHP’s mission. Differences were indeed noted by the evaluation team in strategies developed by members of the consortium, for instance in the manner in which they work with communities and the role of ADCs. As an example, in its implementation area Plan uses an approach to community development that involves signing a formal partnership agreement with the community and its leadership. The latter in turn sets up its own Groupe de Mise en Oeuvre (GMO) or implementing team, financed through the community structure by Plan’s program funds. (Other consortium members support their implementing teams, primarily the ADCs, directly.) The model is interesting in that the GMO, an entity of the community itself rather than the NGO, is responsible for oversight of cases de santé, including monitoring of services, resupply of medications, and coordination of the work of ADCs (called educateurs communautaires de santé in Plan areas of operation). The Plan approach has advantages in looking to long-term sustainability of community health initiatives. While still dependent on outside sources of funding, the GMOs assert that they, representing the community, are more aware of, and invested in, meeting needs and solving operational problems of the cases de santé. On the other hand, concerns have been raised as to whether the GMOs have consistently applied agreed-upon criteria in recruitment of ADCs, with the result that ADC performance has at times been seen to be less effective than in districts where other members of the consortium are involved. In areas covered by Africare, World Vision and CPI, the team noted that, while there was also ongoing collaboration with local community structures, effective involvement of community leaders in addressing pressing needs of the cases de santé, such as essential repairs to the facility, has been less apparent. In the 9 case of CRS and ChildFund, we noted an especially intense involvement by their ADCs in monitoring and supporting the work of the cases de santé. Individual ADCs had responsibility for supervising fewer cases (five to six) than, for instance, Africare’s ADCs (eight to nine), which enables them to spend more time working with individual cases and their teams. The evaluation team did not have sufficient contacts in the field to permit them to suggest in any detail that some consortium members are performing less effectively than others in their approaches to the community, but only that their different approaches reflect different levels of engagement in dealing with long and short term perspectives. The team does feel that the consistency of recruitment and oversight of ADCs, personnel who are so critical to the effective functioning of the CHP, deserves further attention and analysis, as does their training as discussed below. Supervision and follow-up of cases de santé Agents de Développement Communautaires (ADC). The supervisory system established by the CHP gives the ADC the central program role in supporting and monitoring the work of the cases de santé and their teams in his area. (All ADCs met by the evaluation team were male.) The ADC is, in turn, supervised by his program Coordinator and, from a technical perspective, the ICP, who is also responsible for clinical supervision of the cases linked to the ICP’s Health Post. The ADC’s supervisory role means that he is the primary person to oversee and work with the CHWs, matrones and outreach workers at the case de santé, ensuring that they are delivering the basic package of services at an acceptable level of quality, helping solve supply problems, reviewing their record keeping, helping develop their outreach plans, etc. The ADC’s role also involves working with the Comité de Santé that serves as the interface between the community and the case de santé. The Comité also controls an account that takes in funds from service fees and sale of medications at the case and is used to purchase replacement stocks of medications as needed from the Health Post. In particular, as the evaluation team understood it, the ADC is expected to help and encourage the Comité de Santé to mobilize financial support from the community to cover capital costs of such things as repairs or renovations of the case. Such generation of community support is of course a key to the future sustainability of the cases de santé and their services. On the basis of records available at the cases de santé, and discussions with different actors, the evaluation team concluded that the ADCs in general maintain close, supportive relationships with the cases and their teams for which they are responsible. In many zones, ADCs visit their different cases as much as once or twice per week, often under difficult circumstances, relying on local horse-drawn wagons to travel long distances. However, while frequency of ADC supervisory visits was admirable, the evaluation team had reservations about the quality of their supervision. For one thing, the absence in several cases de santé of fiches de references, referral forms specifically designed for the CHW to refer patients from the case to the Health Post when trained medical attention is required, and for the ICP to refer the client back to the case after consultation, indicated inattention to a basic supply problem. In some cases we were told that referral forms had been unavailable for many weeks, whereas the solution was as simple as having more copies of the form printed. In several instances ADCs’ supervision of the work of CHWs also seemed lacking. Patient registers were not always up-to-date, storage closets for medications were often dirty and disorganized, and stocks of medications themselves were often strikingly limited. While some stockouts of key medications, notably for malaria treatment, were systemic and thus out of the control of the case or the ADC, more competent management of supply and storage should have been expected of the case team, which in turn speaks poorly of the ADC’s supervision. 10 Finally, the team found little evidence of instances where ADCs had been able to mobilize Comités de Santé to themselves generate financial support for the case from the community, a subject that will be discussed further in this report in the context of sustainability of community health services. In short, the evaluation team found that, for the most part, ADCs recruited and trained by the CHP were hard-working and committed to their work, but were in need of better oversight and further training in the elements of competent, supportive supervision, as well as in community mobilization and organization. Infirmières Chef de Poste (ICP). The other key supervisory role with respect to cases de santé is that required of the Chief Nurse from the Health Post to which the case is linked. The evaluation team questions the degree to which this role is satisfactorily filled as well. In most of our visits to cases de santé, we were told that supervisory visits by the ICP were irregular. While intended to take place on a monthly basis, demand for the ICP’s services at the Health Post often made this unrealistic, especially in cases where it is linked to several cases de santé. ICPs are MOH employees, so their first priority must be to serve the Health Post. ICPs also noted that lack of transportation sometimes hindered their supervisory role. It was also apparent from our discussions that when these visits are made by the ICP, their time is taken up almost entirely with the weighing of infants and vaccination of children. Little time appears to be devoted to monitoring the work of the CHW and matrone, discussing clinical issues and concerns with them, and reviewing clinic records. The evaluation team noted that clinic registers rarely showed the signature of the ICP that would indicate his or her review. Supervision is intimately linked to program follow-up and evaluation. If effectively performed, it permits programs not only to thrive but to avail themselves of useful data at the community level. As an example, data generated by cases de santé supported by the CHP provided much anticipated evidence confirming the unmet demand for contraception on the part of village women, which led to the pilot initiative that made possible the initial pilot offering of contraceptive pills (OIP) by matrones. In general, the evaluation team feels that weaknesses in program supervision constitute an area in need of comprehensive attention by the CHP. At a minimum this should include refresher training of ADCs in supportive supervision. It should also include refresher training of ICPs to underline the importance, and remind them of the components, of proper clinical supervision. ii.      Quality and coverage of community health services One of the key considerations for this evaluation was an assessment of the quality and coverage of services offered by cases de santé supported by the CHP. Information gathered from interviews and observations at the cases visited by the evaluation team indicated that in most instances the basic package of services was in fact being offered. At some cases, notably those in Africare’s area of implementation, the full list of the services offered in the basic package was prominently displayed at the entrance. While the list is in French, a language that most rural villagers do not read, its prominence testified to the commitment of the CHW, matrones and outreach workers to provide comprehensive health information and services. It also attested to their confidence in being able to handle health questions and complaints whenever they are brought to them by members of the community. (This often occurs at times when the case itself is closed, at which time patients seek out the CHW or matrone in their nearby homes, which are well-known in the community.) Responses to our questions in the community, as well as information gathered from discussions with CMOs, ICPs and other MOH personnel at Health Posts and Health Centers, corroborated this impression that services offered at the cases de santé were of acceptable quality and greatly appreciated. 11 Yes, the CHW has been treating me for a long time and I have never had a problem with his services. Each time that I or one of my children is sick, I come directly to see him, no matter what the hour, and he or the midwife are always available. I thank God for them. (Focus group comment by married woman of 39 years.) In a study documenting the CHP pilot project in which contraceptive pills were offered to women by matrones at certain cases de santé (Offre initiale de pilules or OIP), information gathered from those women is instructive. Almost 98% of women questioned affirmed their satisfaction with the quality of services offered by the matrone. Table 1: Attitudes of women on quality of services offered by matrones at cases de santé Interviewees % Satisfactory 217 97,7 No opinion 4 1,8 Unsatisfactory 1 ,5 Total 222 100,0 Source: Etude de documentation de l’Offre initiale de pilules, ChildFund/FHI/CEFOREP, Avril 2010 When asked their opinion about the quality of the reception they received at the time of their visit to the case de santé for the OIP, 95% of the same group of women expressed their complete satisfaction. Information gathered by the evaluation team in the course of its many conversations in the field corroborated these results, in that it found a high level of satisfaction on the part of community members with the reception that they are accorded by CHWs and matrones at the case de santé whenever they visit. The team also concluded from its many interactions that the range of services offered at the case de santé, i.e. the services contained in the basic package, is appropriate for dealing with the complaints and illnesses that the CHW and matrones are asked to treat, up to a well-defined threshold of seriousness. Beyond that point, they are generally well-versed in the need to refer patients to the Health Post. Several CHWs expressed the desire to be allowed to give injections or take responsibility for other services that are presently the domain of the ICP. But the evaluation team feels that the limits placed on the services offered at the case are appropriate at the present time in the life of the Community Health Program. Family planning. Despite the generally positive view of services offered at the case, it is important to note the emergence of a strong demand for family planning services, as expressed by women during the team’s focus group discussions, primarily in areas not involved in the OIP pilot initiative. It seems not only appropriate but essential that family planning information and services be made available in all cases de santé, not only to meet existing demand but to stimulate demand where it does not yet exist. This will benefit the program as a whole, since documented results of the OIP have shown that its existence in cases de santé in Sénégal led to a marked increase in the number of women frequenting the cases for other services as well. Other concerns. Despite the demonstrated appropriateness of the range of services offered at the case de santé and the capacity of CHWs and matrones to manage them, the quality of available medications and other supplies and their proper maintenance remains a challenge. In cases visited by the evaluation team (1) certain medications were lacking, and (2) several of the medications on the shelves had passed their expiration date and should have been discarded. In others, the storage areas were in a very dirty and/or disorganized state. These concerns only served to accentuate for the evaluation team the weaknesses in supervision previously noted, and must be addressed if services are truly to be considered of acceptable quality. 12 iii. Referral systems Referrals from the case de santé to the Health Post commonly involve pregnant women referred for delivery (especially since an MOH directive limited deliveries by matrones in cases de santé to only those without means or time for transport to the Post), persons with persistent diarrhea, and other maladies too serious to be handled at the case. Actual practices under the CHP by which patients are referred vary considerably from one area to the next. Many CHWs with whom the team met in the course of its field work refer patients in ways that, while effective more often than not, do not correspond to the approach established by the CHP. It has been previously noted that several cases de santé lacked the forms by which patients are supposed to be referred and which also permit the ICP to confirm a visit and send the patient back to the case with a “counter-referral”. Some CHWs use the referral and counter-referral forms interchangeably (and thus inaccurately.) Some CHWs do not know how to write in French, and instead they accompany the patient to the Health Post or send him or her with another person, leaving no formal record of the referral. Some CHWs, lacking the proper referral form (a situation observed in about 50% of case de santés visited), simply use any scrap of paper available. Still others transmit their referrals to the ICP by cellphone. In sum, referrals from cases de santé are generally made when necessary, but not in a standardized way. This makes follow-up of referrals inconsistent, a problem compounded by the tendency of some ICPs not to share all pertinent information relating to a particular patient. Once again, the quality of program supervision is called into question. iv. Community awareness and utilization of services A more aware community. During visits to areas covered by the CHP, the evaluation team saw and heard ample evidence that inputs from the program are leading to heightened awareness, on the part of both women and men, of the causes of illness and the importance of healthy behaviors to the overall health of a community. It also observed that this awareness is reflected in a steadily increasing level of confidence in, and use of, services available at the case de santé. For example, women interviewed, whether individually or in focus groups, were able to identify at least three complaints or illnesses that are treated at the case de santé, and clearly understood the relative seriousness of different health issues in the community. This speaks to the success of the CHP in disseminating health information, in particular information on neonatal and infant health, in a manner accessible to all, whether through consultations with CHWs and matrones or discussions led by outreach workers in villages served by the cases. It overturns the classical theory that links knowledge and awareness to level of education, since the education of most residents of rural communities in Sénégal has been limited to Koranic studies. Clearly, the efforts of case de santé teams have led to a better informed community, whose members, especially women, better understand the causes of illness and the importance, including the financial importance, of prevention and early treatment. Service access and utilization. As a result of the positive impact the CHP has had on community awareness of health issues and services available at the case de santé, there has been a marked increase in utilization of services offered by the case, especially by women, as compared to previous behavior. In addition, ICPs told the evaluation team that increased confidence in the quality of services available at the case de santé, as well as its messages of prevention and early treatment, have led to a reduction of as much as 20-30% in the caseload at the Health Post. They can now focus more on treating serious cases of malaria, diarrhea and respiratory infections, along with performing deliveries. There are geographic and financial advantages as well to the availability of competent services at the level of the case de santé.  Geographic and financial access. In the past, the structure of the Senegalese health system, in which primary health services were only available at Health Posts or Health Centers in a particular health district, put rural populations at a serious disadvantage because of the long 13 distance of these facilities from their homes and villages. Transport to a health facility was time￾consuming and costly, if it was even available. People were often simply unable to access needed medical attention. By making a basic package of health services available to rural populations through cases de santé in or near their communities, the CHP has provided greater access to services at less cost in time and money. The ability to overcome these barriers to health care was clearly appreciated, as attested to by many of the groups and individuals contacted by the evaluation team. Our village is fortunate to have been blessed with a case de santé which can treat illnesses, especially during the rainy season. I no longer need to pay costs of transport to travel a long way for medical treatment. Before it cost me 200 CFA (about $.50) round trip to go to the Health Post, when transport was available at all. Now we save both time and money, and are healthier. (29 year old woman, married, 5 children)  Local capacity building. Another positive aspect of reinforcing community health services through the cases de santé was the fact that local people were recruited and trained to provide those services as CHWs, matrones and outreach workers. The USAID project has chosen and trained local people to staff the cases de santé, which is a source of pride for us. Personally speaking, my health has benefited, and this has enabled me to devote more time to my work. The population is content because maternal and child mortality is greatly reduced, and our husbands are happy because they spend less money on prescriptions. (42 year old woman and community leader) Reduction in Health Post workload and improvement in indicators. Inadequacies in coverage of the national health system, compounded by growing demand for primary health services at the community level, have created a service gap which the CHP has sought to fill. This led many government health providers, with which the evaluation team spoke to applaud the program, noting its impact on health indicators in its localities. From my point of view as a technician, I can say that it is an initiative to salute for at least two fundamental reasons: we are in an area where coverage is weak and villages distant from one another, and sometimes as much as 25 km from the Health Post. The (CHP) helps bridge this gap by bringing services closer to the people, improving access and reducing costs. But they (cases de santé) must be well￾supervised. (Chief Medical Officer, District of Richard Toll) Chief Nurses of Health Posts in areas visited by the evaluation team are strongly supportive of the strengthened role of cases de santé. Their position in the public health pyramid means that ICPs are the providers most affected by the work of the cases within their area of authority. As noted, strengthening the cases has led to a welcome reduction in the workload at the Health Post in terms of non-critical patient visits, such as treatment for non-acute diarrhea and ARIs, management of some deliveries, and counseling and provision of contraceptives. Here at the Health Post we are often overcrowded. Thus the ability of the matrone at the case de santé to give family planning counseling and provide the pill considerably reduces our workload, since all of the women who would have come here for that service can obtain it at their respective cases de santé. (Chief Nurse of a Health Post, or ICP) In fact, all primary health services taken over by the cases de santé revert to the benefit of the Health Post and, thus, the Health District. An increase in the use of modern contraceptive methods, a reduction in high-risk pregnancies (thanks to better pre-natal care from the midwife at the case), a reduction in 14 maternal mortality….at whatever level these are accomplished they directly benefit the health system in terms of improving health indicators. I would say that (the CHP) brings several advantages to the population and to us as service providers: access to the pill at the case de santé increases the chances of its use by women from the community and significantly reduces consultations for problems resulting from too-frequent pregnancies. It also improves our indicators for coverage of family planning. (Another ICP) In short, the Community Health Program and its strengthening of access to services in the cases de santé has not only led to improvements in health indicators, especially for maternal and child health, but has helped the caseload and working conditions for service providers at the Health Post level of the health system pyramid. v. Socio­cultural barriers limiting access to services From our discussions at various levels it became clear that there have been important reductions in socio￾cultural barriers limiting utilization of health services by individuals and families, especially in rural areas. Given the relatively conservative context in which it operates, the CHP has developed a number of innovative strategies to overcome these barriers. These have consisted largely of cultivating the support of key social groups, such as religious authorities, village chiefs and other community leaders. It is an approach that does not upset social equilibrium, but rather supports it by promoting changes, notably improvements in delivery of health services, that clearly benefit the life of the community. One of the most creative of these approaches has been the “grandmother strategy”, designed by the Grandmother Project and pilot-tested by ChildFund in Mbour in 2000-2001. The consortium agreed to use the strategy as a device to enlist the support of figures of enormous importance in local society in promoting improved health of their children and grandchildren. All consortium members have used the strategy in their respective areas of intervention. Perhaps in part because of such innovations, in localities visited by the team there were few indications that religious or other habits and customs were preventing people from accessing pre-natal counseling, baby weighing, vaccination and other services available at the case de santé. Even with respect to family planning, which in the past has been rejected in many communities on the basis of tradition or religion, the evolution has been such that there are now overt demands for contraception, especially in areas reached by the OIP pilot program. Before, it was extremely rare, even impossible, to hear men asking questions about family planning. But now, in addition to asking such questions, men are even urging their wives to ask the matrone for help in practicing contraception. (Matrone at a case de santé) The principal resistance to family planning encountered in the course of focus group meetings was found in the Matam region. This can be attributed in part to the fact that this region has been traditionally underserved by the health system and was only added to the CHP in 2008, with CPI as program implementer, and in part to its historically conservative nature. Social changes take time to be implanted in populations, and it seems reasonable to expect that, with time, cultural barriers to such services will come down. It should also be mentioned that, in all communities visited by the team, it was clear from its interviews that if the case de santé is located too centrally in its village, or if it is not cloturé, i.e., surrounded by a protective wall, utilization of its services tends to be limited. This is especially true for women, for whom privacy and discretion in seeking health services is essential. Most cases de santé visited by the team were 15 not centrally positioned, and most were in fact surrounded by a protective wall. Those that were not showed distinct signs of underutilization. This is an issue that should always be considered in connection with making services at the case as attractive as possible to residents of the community. vi.     Respect for MOH norms and protocols Respect for policies of the Ministry of Health of Sénégal seems to the evaluation team to have been one of the strengths of the Community Health Program. Officials and service providers encountered in Health Districts and Health Posts confirmed that CHWs and matrones observe quite systematically the norms and protocols set forth by the MOH in their delivery of services in cases de santé. This was confirmed by the CHWs and matrones themselves, who clearly understand the regulations placed by the Ministry on the services they can offer. It should be noted that the changing legislative and regulatory framework does not facilitate systematic observation of MOH norms and protocols, and slight differences are noted in interpretation of texts from one zone to another. A recent and significant case in point is the rule put in place by the MOH that states that only emergency deliveries, that is, deliveries where there is insufficient time for the woman to be transported safely to the Health Post, can be performed in cases de santé. In the Health Districts of Diourbel and Richard Toll, we were told that health teams and matrones are following the rule as mandated. On the other hand, in Matam we were told that this regulation had not as yet been received by local health authorities and service providers in the community. In fact there is general dissatisfaction with this particular regulation. In discussions with the team, ICPs as well as matrones expressed the view that it is normal for women who have presented with no particular problem during their pregnancy to be delivered by a matrone at the case de santé. Matrones in the cases and members of the community expressed astonishment at the rule, since they feel that the former, given their long experience delivering babies, are the people best qualified to continue to do so. Further, limiting the authority for matrones to do deliveries at the case seems at odds with the idea of de￾medicalization of services that is supported by other policies. It is important to note that this measure could lead to dissatisfaction with services in general offered by the case de santé, and to more home deliveries. These go against accepted medical practice, but are occurring in several communities. vii.    Sustainability In almost every office and site visited by the evaluation team the issue of the pérennisation or sustainability of the Community Health Program was in the forefront of peoples’ minds. In our first visit to the MOH, the head of the Division des Soins de Santé Primaires, or DSSP, which has the most direct interest in primary health services, voiced her concern that no time be lost in discussing policy and program options for permanent inclusion of community health services in the national system. In her view, and that of many others, all options need to be on the table. Direct CHP support. The evaluation team was struck by the relatively low financial investment by the Community Health Program in the cases de santé, considering that these facilities are at the core of the program. As noted above, that investment includes training and some refresher training of case teams, a small kit of midwifery equipment, IEC materials, and a beginning stock of medications. Funding for repairs and renovations was approved for a few facilities, and some members of the CHP consortium give stipends, of an average of $10/month, to otherwise unpaid CHWs, matrones and outreach workers. The absence of more substantial investment was apparent in many cases de santé visited by the team, where it observed broken delivery tables and other furniture, damaged walls or floors, and in one case a collapsed roof. The team noted poorly stocked (and often dirty) medicine cabinets and heard an almost universally expressed desire on the part of staff for greater financial “motivation.” 16 The evaluation team understands that the goal behind limiting direct support to cases de santé and their teams is to encourage the communities in which they are located to themselves raise and contribute funds for repairs and improvements, needed equipment or furnishings, and financial motivation. In the long term, this would be the most dependable way to ensure their sustainability. As the program continues and matures, and as communities increasingly appreciate the importance of access to services provided to them by the case de santé, the CHP hopes those communities will also begin to assume greater responsibility for their support. Community mobilization. But it is and will be difficult. Convincing already extremely poor villagers to participate in the support of a community service, however much that service is appreciated (and the evaluation team consistently saw evidence of that appreciation), is a challenge. One of the key community entities expected to spearhead a community mobilization effort is the Comité de Santé that oversees operations of the case de santé. One of the primary roles of this committee, generally numbering 3 to 5 persons, is management of funds generated at the case from service fees and sale of medications. The committee President receives these funds from the CHW who manages the case, and is responsible for approving their expenditure, usually for obtaining replacement stocks of medications from the Health Post to which the case is linked. A second role intended for the committee is to serve as an advocate for the case de santé in the community, making its services known and generating financial resources for its support. From discussions with members of these committees it became clear just what a difficult and lengthy process generating resources from the community can be. The collapsed roof mentioned above is a case in point. In a village visited by the evaluation team in the Health District of Matam, the roof over the delivery room in the case de santé had entirely fallen in, making it unusable, a condition which had clearly existed for some time. A discussion with the CHW and the President of the Comité de Santé revealed that, whereas the community was very committed to making the necessary repairs, and a local contractor had agreed to do the work for free, they had so far been unable to raise funds for materials needed for the work to begin. A similar situation was evident in the case de santé in a village visited in the Ndoffane Health District (Region of Kaolack, CHP implemented by ChildFund), where part of the floor of the patient examining room had broken through, leaving a large and unsightly hole, and there was evidence of heavy leakage in the roof. The health committee President and the CHW noted that funds in the case account were insufficient to both make needed repairs and purchase replacement medications, and had to be conserved for the latter. They hope to generate community support for those repairs before the rainy season starts in June, but so far have been unsuccessful. It became clear to the evaluation team that Comités de Santé are not, in most cases, as creative as desired in mobilizing community support for the case. They appear not to have a great deal of motivation to see beyond their role of managing the funds of the case de santé. Tools and strategies are needed to increase that motivation, and to involve other community entities (village development committees, village leadership, other civic groups) in building a sense of responsibility among community residents to see that this facility, which is so appreciated, is increasingly supported from local resources. (It was also clear to the team that situations such as those described above would benefit from much more rigorous supervision and follow-up on the part of the program’s ADCs, a subject which is discussed elsewhere.) Approaches to sustainability. Members of the CHP consortium have individually addressed the issue of sustainability with specific innovations worthy of replication. In the model of micro-financing schemes elsewhere in the world, World Vision has given seed funding to an Association of Outreach Workers, which in turn provides this cadre of workers with a source of financial assistance from which they can 17 borrow individually as needed at low interest. Similar associations of CHWs have been launched as well. In regions where Plan implements the CHP (and as described earlier in this report), it has signed formal agreements with the community and its GMO in the different health districts. Under these agreements it sends funds directly to the community itself, which in turn oversees the program and pays salaries of its ADCs (in Plan’s case called educateurs communautaires.) In this model funding still originates with the CHP, but is controlled by the public sector, which can be seen as a step towards taking control. The DSSP Supervisor in Matam Region was emphatic about the need to explore new ideas for generating resources that can be used to sustain community services. He suggested, for example, that women who bring their infants to the case de santé for check-ups and to be weighed could be asked for une cotisation, or small fee, which could generate significant income since this is a much-used service. Sénégal has an advantage as compared to some other countries in that people are accustomed to paying reasonable fees for services rather than none at all, and an additional fee for this purpose would thus not be seen as outlandish. In short, there are ideas and models of community mobilization and resource generation available, and the Community Health Program and others need to look at them all. What is clear is that the CHP, USAID and the MOH must address the issue of sustainability forthwith, even as this program is preparing to potentially pump increased sums of money into Senegal’s community health strategy, a resource that cannot be expected to last forever. V. RECOMMENDATIONS i. Service quality, referral and supervision  The range of services offered in the basic package by cases de santé is appropriate and should not, for the moment, be expanded. The basic package, including family planning services, should be available in all cases. Otherwise, mastery of the existing basic package on the part of CHWs and matrones should be reinforced through additional refresher training. Adding services, such as provision of injectable contraceptives (frequently suggested by case teams), should not be considered without assurance of the capacity of the providers and of regular supervision by ICPs.  The availability and use of referral/counter-referral forms created for the program should be systematized and assured in all cases de santé. This will ensure more consistent and uniform management and follow-up of patient referrals between the case de santé and the Health Post.  The skills of ADCs must be reinforced and broadened through additional training in supportive supervision. ADCs play the central role in overseeing management of cases de santé and mentoring CHWs, matrones and outreach workers. Because of the weaknesses in management of cases perceived by the evaluation team, additional training of ADCs in techniques of supportive supervision (as well as better oversight by their coordinators) is needed. ii. Program coordination and management  USAID should take the lead in systematizing periodic reviews and joint supervision, at both central and district levels, among the different health programs that it supports.  To achieve more effective coordination, the CHP should be represented in USAID-funded regional coordinating offices. It presently is not so represented, which has led to gaps and misunderstandings in program coordination.  Study the possibility of leveraging support from the USAID-funded HSS program for acquiring additional support for meeting physical needs of cases de santé, in terms of logistics, facility repair and rehabilitation, and equipment. The CHP investment in these areas of cases support is small, and more input is needed to ensure quality of services. 18  Review criteria for recruitment of case de santé teams, notably CHWs. The fact that some CHWs are not literate is often a problem in assuring competent oversight and effective use of management tools at the case. Literacy training as part of overall training might be considered as one solution to this issue.  Pay greater attention to reinforcing the capacities of case Health Committees, to enable them to acquire greater competence in fulfilling their leadership role in monitoring the services offered at the case and in mobilizing community support. iii. Synergy between cases de santé and Health Posts  Consider logistical support to facilitate oversight of cases de santé by Chief Nurses from the Health Post. ICPs are required to provide technical supervision of cases de santé in their areas on a regular basis, but this is rendered problematic by (among other things) lack of dependable transportation. Providing motorbikes might be one solution.  Improve coordination and problem-solving between the ICP, which supports cases de santé, and IntraHealth, implementer of the stratégie avancé in support of Health Posts. This is essential to achieving a more systematic, workable arrangement whereby ICPs fulfill their responsibility for case de santé oversight. iv. Sustainability    The ChildFund-led CHP consortium is urged to develop and put in place an operational mechanism for involving communities and community leaders in generating support for cases de santé and thus moving towards a sustainable community health model. To do this it must draw on all of the different models developed by its members for the purpose of sustaining their efforts, as well as the ideas put forth by MOH partners at national, regional and district levels. To do otherwise is to risk compromising the impressive accomplishments to date of the Community Health Program.  Reinforce the capacity of ADCs to work with case Health Committees, not only to be sure the committees properly oversee operations of the case de santé but also develop tools and strategies for mobilizing community support for meeting long-term needs of the case in terms of repairs, renovations and equipment. (This only underscores the central role of ADCs to the implementation of the CHP at the community level, and their need for additional training and other supports from the program.)  Establish a national-level working group on the sustainability of the national community health program as an essential element of national health services. This group should include as members senior officials of the Ministry of Health, USAID, and other implementing partners. It should meet quarterly, and discuss any and all ideas that can assure permanent support for community health services, whether through new and/or increased fees for service, deepened involvement of community organizations, new mechanisms for supporting staffs of cases de santé, expanded use of models developed by members of the Community Health Program consortium, etc. All such discussions would be intended to lead to recommendations for policy change or development that would in turn assure the sustainability of delivery of comprehensive primary health services in Senegalese communities. v. Operations research and evaluation  A baseline study should be conducted, drawing on data and statistics collected during the first five years of the CHP and against which future progress can more precisely be measured.  A mid-term evaluation should be conducted in the course of the next five-year phase of the CHP, to provide a basis for adjustments to program strategies as deemed necessary. 19 VI. LESSONS LEARNED 1. Maintaining linkages and mechanisms for coordination between government and donors at the central level and implementing partners at the regional and district levels invariably leads to an improvement in program performance. 2. When well-led and continuously involved, communities are capable of mobilizing resources and playing a significant role in the achievement of a program’s objectives and the assurance of its longevity. 3. When carefully selected and well-trained, community members are capable of playing key roles – CHW, midwife, outreach worker – in offering services in cases de santé appreciated by the community for their availability and their quality. VII. GENERAL CONCLUSION The Community Health Program, funded by USAID and implemented by a consortium of NGOs led by ChildFund, has succeeded in responding to the need, felt strongly by the MOH, to elevate community health to a high level of importance. It has achieved a level of coverage significantly greater than in similar community health programs in other countries with which the evaluation team is familiar. Services offered as a result of this program have become not only appreciated but expected in villages and communities served by functioning cases de santé. Of note also is the strong belief among leadership of different departments of the Ministry of Health in the importance of considering community health as an essential element of national health care strategy. There remains, however, an important challenge, that of sustaining these interventions into a future where external support will no longer be as substantial. All contacts made by the evaluation team were clear in their insistence that this is an obstacle that must be overcome. 20 ANNEX A Bibliography 1. Coles and All, “ Report of technical Review Team”, USAID, Dakar, Septembre-Octobre 2009 2. USAID, « RAPPORT D’ACTIVITES TRIMESTRIEL », Santé Communautaire, Dakar Juillet 2007 3. Santé Communautaire, «SYSTEME DE SUIVI-EVALUATION », Dakar Mars 2010 4. USAID « Orientation des ADC sur les stratégies d’intervention communautaire, Document technique », Santé Communautaire, Dakar Septembre 2007 5. USAID « ANNUAL ACTIVITIES REPORT OCTOBER 2006 TO SEPTEMBER 2007 », COMMUNITY HEALTH, Dakar October 2007 6. USAID « Santé Communautaire: plan d’action 5 », Dakar Octobre 2010-Septembre 2011-04-14 7. USAID « President’s Malaria Initiative (PSC/PMI) », rapport trimestriel, Dakar Janvier-Mars 2010 8. USAID « Quarterly Activities report », Community Health, Dakar July 2008 9. USAID « Santé Communautaire », rapport d’activités annuel, Dakar Octobre-Septembre 2010 10. USAID « Santé Communautaire », rapport trimestriel d’activités, Dakar Octobre-Décembre 2010 11. USAID « Santé Communautaire », rapport d’activités annuel, Dakar Octobre 2006 à Septembre 2007 12. USAID « Programme de Santé Communautaire », PSC/USAID, évaluation finale, Dakar 2011 13. USAID « Système de suivi-évaluation », santé communautaire, Dakar juin 2007 14. USAID « Quarterly activities report », Community Health, Dakar January-February-march 2007 15. USAID “ Community health”, Annual Activities report, Health program, Dakar October 2007- septembre 2008 16. USAID “ Community health project”, Dakar April-June 2007 17. USAID « Santé Communautaire », plan d’action AN 1, Dakar Octobre 2006 18. USAID « Santé Communautaire », rapport d’activités trimestriel, programme santé, Dakar Avril￾Mai-Juin 2010 19. USAID « rapport d’activités trimestriel », santé communautaire, Dakar Octobre-Novembre￾Décembre 2006 20. USAID « Quarterly activities report », Community Health, Dakar April-May-June 2007 21. USAID « Quarterly activities report », Community Health, Dakar October-November-December 2007 22. USAID « rapport d’activités trimestriel », santé communautaire, Dakar Janvier-Février-Mars 2007 21 23. USAID « Santé Communautaire », plan d’action AN 4, Dakar Octobre 2009-Septembre 2010 24. USAID « Santé Communautaire », rapport d’activités annuel, Dakar Octobre 08 à septembre 2009 25. USAID « Santé Communautaire », rapport d’activités trimestriel, programme santé, Dakar Avril￾Mai et Juin 2009 26. USAID « Santé Communautaire », rapport d’activités trimestriel, programme santé, Dakar Octobre-Novembre et Décembre 2009 27. USAID « Santé Communautaire », rapport d’activités trimestriel, programme santé, Dakar Janvier-Février et Mars 2009 28. USAID « Santé Communautaire », rapport d’activités trimestriel, Dakar 2009 29. USAID « Santé Communautaire », rapport d’activités trimestriel, Dakar 2008 30. USAID « Weer community health program », Dakar 2006 June 2011 31. USAID, « Santé communautaire », liste nominative des cases par région-district et poste de santé, Dakar 2009 32. USAID « Santé communautaire », cartographie des interventions PSSC et PMI ? Dakar Octobre 2007-Septembre 2008 33. CRS SENEGAL, « Protocole d’accord dans l’exécution du projet » : Président Malaria Initiative, Dakar Décembre 2010 34. USAID, « Santé communautaire- PMI », cadre logique des indicateurs, programme santé, Dakar Novembre 2007 22 ANNEX B Data collection instruments for interviews and focus groups 1. Guide d’entretien pour les différentes cibles par axe thématique Thème 1 : Perceptions du projet PSC [MCD, ICP/SF, MT/ASC, Communautés] - Perception du projet de santé communautaire - Quel apport pour la communauté ? - Implication des communautés dans ce projet ? (Niveau, comment, etc.) Thème 2 : Services (offerts, disponibles et/ou à rajouter éventuellement) [MCD/ICP, ADC, MT-ASC, communautés] - Connaissance des services offerts au niveau de la case (lesquels ?) - Perception/Appréciation des services offerts (utilisation, coûts, disponibilité, qualité, etc.) - Quels autres services pourraient être ajoutés ? Pourquoi - Comment s’y prendre pour éventuellement ces autres services Thème 3 : Références / Contre-références [MCD/ICP surtout et MT-ASC] - Gestion des références qui partent/viennent des cases de santé - La question des contre-références et du suivi de ces malades (difficultés) - Comment améliorer le système de référence et de contre-référence des malades ? Thème 4 : Barrières sociales et culturelles : religion, coutumes, appartenance sociale ou castuelle, etc [ICP, agents communautaires et communautés] - Sur la diffusion des connaissances & information des populations - Sur l’utilisation des services - Peurs, stigmatisations sociales (notamment pour la TB, le VIH, la PF, etc.) Thème 5 : Respect des protocoles du Ministère de la Santé [MCD, ICP, MT-ASC] - Connaissances des protocoles et normes du Ministère - Respect des protocoles et normes - Difficultés et/ou contraintes dans le respect de certains de ces normes et protocoles Thème 6 : Stratégies, Contraintes et difficultés [ADC, relais, MR-ASC] - Quelles stratégies mises en œuvre pour impliquer et sensibiliser les communautés (+ leaders) - Principales difficultés dans la mobilisation des communautés (rejets, refus, résistances) - Prise en compte du genre dans les stratégies et activités Thème 7: Points spécifiques - Supervision, - Encadrement, formation et recyclage [ICP, ADC, MR-ASC] - Taux et niveau de couverture - Gender sensitive : prise en compte des besoins des différents groupes dans les services offerts Succès & Echecs : les plus grands succès ; les choses qui ont moins bien réussi Défis actuels et futurs Recommandations 23 2. Guide d’entretien pour les membres du Consortium Axe 1: Cadre institutionnel - Partenariat entre différents membres du consortium (forces/ contraintes et défis) - Dans quelle mesure les compétences distinctives de chaque organisation ont elle été utilisées - Mécanisme de gouvernance interne (Rôle de chacun/ Redevabilité/ suivi - évaluation) - Coordination du programme au niveau de Diourbel/ impact sur la coordination de la non disponibilité d’un bureau de zone pour l’opérationnalisation Axe 2 : Coordination avec le Ministère de la Sante - Respect des protocoles et normes du ministère de la sante - Collaboration avec les postes de sante, les Districts et RMs (référence - contre-référence) Axe 3: Implication des communautés - Appropriation - Quel rôle les collectivités ont-elles joué dans ce projet - Statut de l’ASC, de la matrone, de l’ADC Axe 4: Enseignements tirés de cette expérience - Coordination interne - Coordination avec le Ministère - Implication des communautés/ des collectivités - Opportunités pour la pérennisation - Apport de la recherche opérationnelle? (Implication des universités et utilisation des paquets pilotes/ etc.) Vision pour un éventuel Programme de Sante Communautaire renouvelé 24 ANNEX C Liste des Contacts Dr. Ramatoulaye Dioume, USAID Dakar Sounka Ndiaye, USAID Dakar Isetha Simone, USAID Dakar Debbie Gaye, USAID Dakar Dr Amadou Mbow, USAID Dakar Dr Bocar Daff, DSR (MOH) Dr Aida Tall, DSR (MOH) Dr Ndeye Codou Lakh, DSSP (MOH) Dr Mame Bocar Lô, PNT (MOH) Dr Mbayam Dione, DANSE Mme Fatou Diouf Gaye, DANSE M. Seye, SNEIPS Dr Moussa Thior, PNLP (MOH) Mamadou Diagne, COP PSC/ ChildFund Ndeye Wade, ChildFund Medoune Diop, Counterpart International Salif Deme, Counterpart International Abdourahmane Traore, Counterpart International Dr Cheikh Sadbou Sarr, ADEMAS Dr Seynabou Mbengue, ADEMAS Ndeye Marie Rose Diaw, ADEMAS Agoussou Gomis, World Vision Elhadji Babacar Gueye, IntraHealth Laurence Mukanyindo, AFRICARE Ousseynou Samb, AFRICARE Gorgui Sène Diallo, AFRICARE Diaguily Koita, Plan Sénégal Dr Barbara Sow, FHI Sénégal Dr Malick Niang, Abt Associates Dr Sokhna Sow, Abt Associates/BR Thiès Malick Ndome, Catholic Relief Service (CRS) Dr Bouna Sall, MCD Diourbel Mme Niang, Superviseur SSP/Diourbel Babacar Sy Sarr, CRS/Diourbel Mamadou Jacques Ndour, ChildFund/Kaolack Boubacar Feye, ICP Ndoffane Dr Fatou Bereke Ndiaye, MCD Adjt Ndoffane Paul Waly Diouf, World Vision/Kaffrine Dr Diop, MCD Koumpentoum Prudence Nicole Mpamy, World Vision/Birkelane Alexandre Ndiaye, World Vision/Birkelane M. Niang, ICP Mereto M. Faye, ICP Thiaré M. Sarr, ICP Ndignick Amadou Moustapha Sow, Plan/Saint Louis Dibcor Sene, Plan/Saint Louis Dr Dieye, MCD Richard Toll Dr Ngom, MCD Adjt, Richard Toll Diadji Aly Sow, Superviseur SSP RM Matam Ousmane Diop, ICP Ogo Amadou Cherif Seck, ICP Savoigne ….and ADCs, CHWs, matrones, outreach workers, health committees, village leaders and community groups and individuals in all districts and communities visited. 25 ANNEX D Ordre de Travail (Scope of Work) USAID/Sénégal Community Health Program (ChildFund) Final Evaluation Work Plan March 14 – April 29, 2011 The evaluation will proceed according to the implementation plan included in the final evaluation technical proposal submitted by Initiatives, Inc. and approved by USAID/Sénégal, That plan is expanded below. March 14 – First evaluation team working meeting; briefing at USAID/Dakar March 15 – Team travels to program HQ in Thies for full day meeting with ChildFund program leadership and members of the program consortium. Community health program COP and senior program staff provide full presentation of project history, structure, scope and current status, followed by extensive period of questions and answers. March 16 – Team completes and submits evaluation work plan for USAID review; completes itinerary for field visits; arranges schedules for meetings with Dakar-based key informants, primarily within MOH, and with individual consortium members. March 17, 18 – Team holds meetings with Ministry of Health units with connection to or interest in the community health program. These include DANSE, DSP, DRS, SNEIPS, PNT, PNLP, CAS/PNDS, and others as deemed necessary. Issues/questions to be discussed will include but not be limited to:  extent to which the community health program has complemented MOH priorities  possible areas of conflict between MOH and program priorities  adherence of community health program to MOH protocols and policies  coordination/collaboration issues  potential for sustainability and eventual absorption of community health inputs by MOH. March 19 – Evaluation team meets for weekly progress review. March 21 – Visits to be made by one or more team members to Dakar offices of individual consortium members - World Vision, Plan, CRS, Africare, Counterpart International. Issues to be discussed will include:  perceived strengths and weaknesses of community health program partnership  extent to which individual partners’ technical capacities are used to advantage  appropriateness of geographic and technical division of labor, collaboration with MOH  goals for community health program if/when USAID support is renewed  hopes and expectations for findings of this evaluation. An expanded interview guide for meetings with consortium members is attached. 26 The team will set up meetings with other USAID-funded stakeholders involved in different ways with health programming in Sénégal that intersect with the community health program, such as ADEMAS, IntraHealth, Abt Associates, FHI and possibly RTI. March 20-25 – Two team members leave for Kaolack (where it will be rejoined after 3/21 by member making Dakar visits) to begin series of visits in regions of Tambacounda, Kaolack, Kaffrine, and Diourbel*. Objective: to observe program operations in areas where each of the consortium members is active, meet with area coordinators and MOH representatives, visit a broad range of “cases de santé”, and hold focus group discussions with ASCs, ICPs, midwives, community leaders, etc. A full schedule of visits is being developed and coordinated with the community health program COP. Interview guides for the various interactions are being prepared. These will be shared when completed**. March 25 – Full team returns to Dakar. March 26 – Team review meeting. March 27 – April 1 – Full team leaves for St. Louis to begin second week of field visits in regions of St. Louis and Matam*. Objective will be the same as previous week’s visits (above.) April 1 – Team returns to Dakar. April 2 – Team meets to review full field experience, begin drafting summary of key findings. April 4 – National holiday. April 5 – Debriefing for USAID/Dakar and consortium members. April 6 or 7 – Debriefing with combined MOH stakeholders. April 8 – Evaluation team finalizes individual writing assignments. April 9 – Team leader returns to U.S. April 10 – 29 – Preparation of draft evaluation report; USAID review; preparation and translation of final report, all as mandated in evaluation implementation plan. *Regions and districts have been selected to enable the evaluation team to visit areas where each member of the Community Health Program consortium has coordinating and oversight responsibility. **See attached “Cibles & Outils de Collecte”. 27 Guide d’entretien pour les membres du consortium I. Cadre institutionnel Partenariat entre différents membres du consortium (forces/ contraintes et défis) Dans quelle mesure les compétences distinctives de chaque organisation ont elle été utilisées Mécanisme de gouvernance interne au consortium (rôle de chacun/redevabilité/suivi-évaluation) II. Coordination avec le Ministère de la Santé Respect des protocoles et normes du ministère de la santé Collaboration avec les postes de santé, les Districts et RMs (référence/contre-référence/supervision) Coordination du programme au niveau de Diourbel/ impact sur la coordination de la non disponibilité d’un bureau de zone pour l’opérationnalisation III. Implication des communautés Appropriation Quel rôle les collectivités et comités de santé ont-elles joué dans ce projet Statut de l’ASC et de la matrone IV. Enseignements tirés de cette expérience Coordination interne Coordination avec le Ministère Implication des communautés/ des collectivités Opportunités et stratégies pour la pérennisation Apport et opportunités pour la recherche opérationnelle ? (Implication des universités et utilisation des paquets pilotes/ etc.) Vision pour un éventuel Programme de Santé Communautaire renouvelé