Management & Leadership Program Evaluation of the Leadership Capacity Strengthening Program (LCSP) for the Ministry of Public Health, Guinea July 2003 Linde Rachel David A. Goldenberg Assisted by: Fatoumata Diaby Karen Sherk Produced under funding from United States Agency for International Development Cooperative Agreement No. HRN￾A-00-00-00014-00 Management & Leadership Program Evaluation of the Leadership Capacity Strengthening Program (LCSP) for the Ministry of Public Health, Guinea July 2003 Linde Rachel David A. Goldenberg Assisted by: Fatoumata Diaby Karen Sherk Produced under funding from United States Agency for International Development Cooperative Agreement No. HRN￾A-00-00-00014-00 ACKNOWLEDGEMENTS This evaluation report would not have been possible without the dedicated effort and support of many contributors. We would like to take this opportunity to give our special thanks to the following persons and groups: At the Ministry of Public Health, Guinea: The Minister of Public Health, for his support, encouragement and leadership. The LCSP Participants and their team members, for giving so generously of their time and for sharing their thoughts with us. Oumar Diakite, for enthusiastically sharing his experience and insights. At PRISM: Alain Joyal, Director, for his hard and efficient work in preparation for the evaluation, and for his hospitality and generosity during our stay in Guinea. Alain also provided valuable and critical feedback to the first draft of this report. Mme Diaby Ilyassou Diallo, for a wide range of support, including the sharing of her thoughtful reflections on the implications of the LCSP for Guinea’s Health System and for fighting to get us on the ferry boat. Mamadou Keita , for providing us with key data and information. Fatoumata Diaby, for her companionship on our trek through La Haute Guinée, and her dedicated hard work as note-taker. Tanou Diallo, who hosted the team at dinner. The PRISM chauffeurs, for their professionalism, vigilance and practical knowledge. At MSH: Karen Sherk, for her conscientious support role and tireless notetaking and data entry. Sylvia Vriesendorp, who was a vigorous and attentive source of feedback and a reliable source of support when needed. Sallie Craig Huber, for her thorough and detailed feedback to the first draft of the report. Alison Ellis, another source of thoughtful input for the first draft. LCSP Evaluation Report – July 2003 i TABLE OF CONTENTS ACRONYMS...............................................................................................................................III EXECUTIVE SUMMARY ........................................................................................................IV 1 BACKGROUND AND OBJECTIVES................................................................................... 1 2 METHODOLOGY AND DATA COLLECTION PROCESS ............................................. 4 2.1 METHODOLOGY............................................................................................................... 4 2.2 DATA COLLECTION PROCESS ......................................................................................... 6 3 CONTEXTUAL FACTORS.................................................................................................... 7 4 FINDINGS: THE CHALLENGES...................................................................................... 10 4.1 PERSONAL CHALLENGES.............................................................................................. 12 4.2 CHALLENGES IN ORGANIZATIONAL PRACTICE ........................................................... 15 Job Descriptions.......................................................................................................... 17 Delegation and Information Sharing......................................................................... 17 Meetings....................................................................................................................... 18 Supervisory Systems.................................................................................................... 18 Decision-making ......................................................................................................... 18 4.3 CHALLENGES AT THE INSTITUTIONAL LEVEL.............................................................. 19 Relations with superiors.............................................................................................. 19 Relations with Peers.................................................................................................... 20 4.4 CHALLENGES INVOLVING SERVICE DELIVERY ............................................................. 20 5 FINDINGS: THE FOUR LEADERSHIP FUNCTIONS AND TEAM-BUILDING....... 21 5.1 SCANNING ...................................................................................................................... 22 5.2 FOCUSING ...................................................................................................................... 23 5.3 ALIGNING/MOBILIZING ................................................................................................ 24 5.4 INSPIRING ...................................................................................................................... 25 LCSP Evaluation Report – July 2003 ii 6 FINDINGS: FACTORS INFLUENCING THE RESULTS.............................................. 29 6.1 THE LCSP PROCESS .................................................................................................... 29 The Training Schedule and its Components.............................................................. 29 An Audience Geared for Success ............................................................................... 30 A Well thought out Training Structure ...................................................................... 31 A Trusted Training Team with a Clear Vision .......................................................... 31 Individual Action Plans and Follow-up Visits........................................................... 31 Key LCSP “Tools”....................................................................................................... 32 Differences in Experience: Regions vs. Central Staff.............................................. 33 Feedback and Support Networks................................................................................ 33 Teams’ Familiarity with LCSP Concepts................................................................... 34 7 CONCLUSIONS.....................................................................................................................35 7.1 EFFECTS AT THE PERSONAL LEVEL ............................................................................. 35 7.2 EFFECTS AT THE ORGANIZATIONAL LEVEL................................................................. 35 Participation ................................................................................................................ 36 Team Building............................................................................................................. 36 Generating and Seeking Feedback............................................................................. 36 The Concept and Use of “Challenge” ....................................................................... 37 From Efficiency-Focused to Team-Focused Orientation ......................................... 37 7.2 IMPACT AT THE INSTITUTIONAL LEVEL AND SERVICE DELIVERY LEVEL.................. 38 8 RECOMMENDATIONS FOR THE NEXT STAGES OF LEADERSHIP STRENGTHENING ...................................................................................................................40 8.1 REVIEW THE STRATEGIC IMPLICATIONS OF LEADERSHIP DEVELOPMENT................ 40 8.2 USE A TEAM-BASED APPROACH FOR EXTENDING LEADERSHIP TRAINING ................ 40 8.3 REPLICATE THE LCSP STRUCTURE............................................................................. 41 8.4 CONSIDER VARIATIONS IN THE COMPOSITION OF PARTICIPANT COHORTS. ............. 41 LCSP Evaluation Report – July 2003 iii 8.5 UNDERSTAND THE CHARACTERISTICS OF THE TARGET GROUPS FOR TRAINING............................................................................................................................... 41 8.6 REVIEW THE UTILITY OF THE LCSP TOOLS ............................................................... 41 8.7 CONTINUED REINFORCEMENT OF LEADERSHIP PRACTICES AT SENIOR LEVELS .................................................................................................................................. 42 8.8 ESTABLISH A POOL OF LCSP FACILITATORS.............................................................. 42 8.9 EXPLORE THE POSSIBILITIES FOR A SYNERGY OF EFFORT TO EXTEND LEADERSHIP PRACTICES....................................................................................................... 42 9 RECOMMENDATIONS FOR A MONITORING SYSTEM............................................ 43 9.1 IMPROVED MANAGEMENT PRACTICES AND SYSTEMS : THE MOST .......................... 45 9.2 IMPROVED WORK CLIMATE : THE CLIMATE ASSESSMENT TOOL............................. 45 9.3 M&L INDICATORS: ....................................................................................................... 46 REFERENCES............................................................................................................................ 47 APPENDICES............................................................................................................................. 48 LCSP Evaluation Report – July 2003 iv ACRONYMS AOP Annual Operational Plan CAFS Centre for African Family Studies DHS Demographic and Health Survey EPI/PHC/ED Extended Program of Immunization/Primary Health Care/Essential Drugs FY Fiscal Year GOG Government of Guinea GTZ Deutsche Gesellschaft für Technische Zusammenarbeit IGH Inspector General for Health LCSP Leadership Capacity Strengthening Program M&E Monitoring and Evaluation M&L Management and Leadership Program MOPH Ministry of Public Health MOST Management and Organizational Sustainability Tool MSH Management Sciences for Health NGO Non-Governmental Organization PEV/SSP/ME Programme Elargi de Vaccination/Soins de Santé Primaire/ Médicaments Essentiels PHC Primary Health Care PHD Préfectoral Health Director POA Plan of action PRISM Pour Renforcer les Interventions en Santé Reproductive et MST/SIDA (USAID’s bilateral project in Guinea, implemented by MSH) RHD Regional Health Director USAID US Agency for International Development LCSP Evaluation Report – July 2003 iii EXECUTIVE SUMMARY BACKGROUND AND METHODOLOGY PRISM (Pour Renforcer les Interventions en Santé Reproductive et MST/SIDA) provides the Government of Guinea with specialized technical assistance in strengthening health systems and promoting sustainable increases in the use of health services. Its interventions include a wide array of support functions, including the application of tools for the improvement of management capacity. In collaboration with the Management Sciences for Health M&L (Management and Leadership) Program in Boston, PRISM launched its leadership initiative in April 2002 with two Leadership Dialogue meetings in Conakry. These meetings became the basis for the design of the Leadership Capacity Strengthening Program (LCSP). Its curriculum consisted of three workshops to be implemented over the course of six months between April and November 2002. The first workshop focused on leadership and self￾knowledge, the second on leadership and organizational dynamics and the third on leadership and “changing the system”. Following the first two modules, participants produced action plans, and all received follow-up visits from their Guinean facilitators who doubled as coaches. The purpose of this evaluation was to obtain an initial appreciation of the effects of the LCSP, six months after the conclusion of its workshop curriculum. In addition, it was to provide recommendations for a possible extension of the program and for the monitoring of its effects. The program was expected to produce changes in behaviors, knowledge and capacities that are generally indicative of the M&L leadership functions (scanning, focusing, aligning/mobilizing and inspiring) and which would lead to improved performance at all levels. In order to anchor the expected changes in concrete activities, the evaluation focused on the challenges addressed by the participants within their immediate work environment. The evaluators adapted their approach to the evolving and partly tentative applications of the skills and knowledge obtained through the LCSP, and to a very broad understanding of challenges. Consequently, they used an inductive and open￾ended approach that was exploratory more than confirmatory. The returns are rewarding in that they provide a rich source of qualitative insight into the possible transition from individual focus to team orientation. The “challenges” identified and discussed by the participants covered primarily challenges at the personal level and those involving the immediate work teams. In some cases, the respondents also discussed the broader institutional challenges they faced as health professionals. . LCSP Evaluation Report – July 2003 iv FINDINGS AND CONCLUSIONS We have identified three basic assumptions underlying the LCSP approach that, on the basis of our findings, appear to be well founded and workable: a) the likelihood of achieving sustainable improvements in health service delivery is increased when managers have leadership capacity; b) leadership capacity is strengthened when managers know how to get their work teams to achieve results; c) in order to get their teams to achieve results, managers need to know how to promote and nurture a work climate in which team members are empowered and inspired. We believe that these assumptions can be scaled up to broader challenges as well. Effects at the Personal Level All participants greatly valued those aspects of the LCSP that had to do with personal issues. Only one participant considered the personal aspects less important than the organizational aspects. Through the various challenges that they set themselves, participants were able to make advances in the following areas of their personal behaviors: control of emotions such as anger and impatience, willingness to listen, greater accessibility to others, humility. As leaders, they understood that personal growth can provide a stronger basis for working with others in teams to face challenges and achieve results. However, this linkage’s actual realization varied considerably across the participants. Effects at the Organizational Level The LCSP inculcates a new approach to solving the management problems that stand in the way of resolving broader institutional challenges. This approach includes participation, transparency, risk-taking, and a purposive team-orientation. As a group, we found the LCSP participants to be visibly inspired and engaged in their quest for self-improvement. The introduction of the four leadership functions (i.e., scanning, focusing, aligning and inspiring) did not, in and of themselves, result in “new” behaviors or practices. However, combined and packaged within the LCSP program design, they helped to convey a new approach to existing management procedures (delegation, supervision, meetings, etc). The most noteworthy features of this new approach are: • Participation According to their colleagues, the program participants have introduced a new way of discussing issues and problems (scanning) -- collectively as a team. Team members clearly appreciate the greater clarity that comes through the sharing of information, improved means of communication and opportunities for input. However, we have little evidence that they feel included in major decision-making. Instilling the skills of LCSP Evaluation Report – July 2003 v aligning and mobilizing in team members is likely to become easier once they feel more directly included in the decisions that affect their work. • Team Building The participants had invested considerable effort in activities that can be used to build and strengthen their teams. They used improvements in such prosaic matters as job descriptions, meeting protocol, and delegation practices to put in practice a new orientation that promotes the engagement of subordinates and encourages their input. These are important elements in laying the groundwork needed for building effective teams. We found that some LCSP participants were more appreciative of this potential than others and would hope that there will be the opportunity to reinforce this important aspect of their LCSP experience and bring it to their conscious attention. • Generating and Seeking Feedback Providing and soliciting feedback promotes transparency and a sense of sharing. Participants sought to make it routine practice and, in some cases, used it to push the boundaries of risk. . The positions occupied by the members of this LCSP cohort are highly visible, potentially vulnerable and by tradition very lonely. There was a palpable sense of liberation conveyed by the participants when they discussed their discovery of new ways of sharing information and inviting feedback from peers, superiors and subordinates. • From Efficiency-Focused to Team-Focused Orientation Challenges addressed at the organizational level focused on the improvement of existing practices. All of the participants had addressed one or another of these and had been able to achieve improvements. What we think is noteworthy for this evaluation is the extent to which these improvements went beyond technical efficiency to investments in building teams that can address challenges at broader levels and achieve sustainable results. This requires attitudinal change. Based on our findings, we have developed a continuum of responses from those that are primarily efficiency-focused to those that manifests a team-development orientation.1 The continuum progresses from what we would call an efficiency-based model of a manager, to what M&L would call the “Manager Who Leads”. Most of the responses fell into the efficiency-oriented category, but we also noted a distinct effort to move toward the participatory mode and there is at least one participant that has consciously initiated practices that fall into the team-empowerment category. 1 See on page x. This is an inductively produced matrix, based on findings. It does not provide scores and numbers with regard to responses since our probings were not guided by this framework nor structured to obtain scores and/or counts. We simply reproduce the practices and approaches emphasized by the respondents in an entirely open-ended way. LCSP Evaluation Report – July 2003 vi Effects at the Institutional Level and Service Delivery Level It was not the intent of this evaluation to assess the impact of the LCSP on change at the broader institutional level or on service delivery. Still, there is some indication that the team-focused approach and certain individual behaviors promoted by the program have facilitated steps toward institutional change. For example, staff in one region were going against the standard practice of masking dismal service statistics. The respondent linked this initiative to the importance of honest feedback that had been stressed during the LCSP. RECOMMENDATIONS FOR THE NEXT STAGES OF LEADERSHIP STRENGTHENING • Review the Strategic Implications of Leadership Development • Use a team-based approach to extending leadership development • Consider the composition of participant cohorts • Understand the characteristics of the target groups for training • Review the utility and application of LCSP tools • Continued reinforcement of leadership practices at senior levels • Establish a pool of LCSP facilitators • Explore the possibilities for a synergy of effort to extend leadership training RECOMMENDATIONS FOR A MONITORING SYSTEM • The monitoring system should reinforce the fundamental characteristics and values of the LCSP. • To monitor progress on outputs and outcomes, we recommend a facilitated self￾assessment approach that becomes the basis for problem identification, action and learning. • Monitoring and Evaluation should be fully incorporated into project planning, where means-ends linkages are clearly spelled out and objectives are expressed in measurable terms. We therefore recommend a process that links monitoring directly with planning. • We recommend that PRISM’s training team adapt the MOST to the needs of the MSP and introduce it in Haute Guinée (Kankan, Faranah and N’Zérékoré) before making it available to other offices (regional or central). • We recommend a number of adaptations to make the Climate Assessment Tool useful as a monitoring tool that provides a learning opportunity and encourages self-diagnosis and planning for improvement. • We suggest two uses for the M&L Indicator Menu. First, its management indicators can be incorporated into the supervisory tools that are being developed at several of the MOPH regional offices. Second, it can provide a set of indicators for the four leadership functions that are consistent with our findings during this evaluation and which therefore offer a field-tested alternative to the current “Leadership Practices” questionnaire that has been used during training. LCSP Evaluation Report – July 2003 vii 1 BACKGROUND AND OBJECTIVES PRISM (Pour Renforcer les Interventions en Santé), a MSH program operating in Guinea since 1998, has been a key partner of the Ministry of Public Health (MOPH) in trying to achieve sustainable increases in the use of health services through systems strengthening. Included in the PRISM strategy for achieving this goal is the reinforcement of management capacity with a particular focus on team management. During the 2002 annual review of the Guinean Primary Health Care System conducted by the Ministry of Public Health (MOPH), weaknesses in the management of program teams and of the decentralization process in general were recognized as priority areas for improvement. The Ministry’s most senior officials agreed upon “the need to reinforce management skills at all levels of the system.” Consequently, the PRISM Project Director proposed a leadership strengthening program to the MOPH Secretary General, Dr. Momo Camara, and met with an enthusiastic response. Thus a program of reinforcement of leadership and management capacity was proposed and became integrated as a priority initiative in the FY02 workplan of EPI/PHC/ED (Extended Program of Immunization/Primary Health Care/Essential Drugs). In collaboration with the M&L (Management and Leadership) Program in Boston, PRISM launched its leadership initiative in April 2002 with two Leadership Dialogue meetings in Conakry. These meetings, attended by 21 senior MOPH officials, became the basis for the design of the Leadership Capacity Strengthening Program (LCSP). Its curriculum consisted of three workshops to be implemented over the course of six months between April and November 2002. The first workshop focused on leadership and self-knowledge, the second on leadership and organizational dynamics and the third on leadership and “changing the system”. Participants produced action plans, and all received follow-up visits from their Guinean facilitators who doubled as coaches. PRISM’s role in the LCSP process merits special mention. Because of its strong connections with the Ministry, PRISM was able to shepherd the process along through frequent conversations with key supporters of the program within the MOPH, schedule events and provide valuable administrative and logistical support throughout the program. Its commitment to integrate expansion of the program to the préfectoral level, at least in the regions covered by PRISM2 will be a significant factor in future impact of the program. In beginning to formulate an approach to address the weaknesses identified during the 2002 annual review, PRISM drew upon earlier training experiences with MOPH, adapting a course that had been particularly well received.3 2 Because PRISM (USAID) only covers the Région de Haute Guinée, ownership by the MOPH and its ability to align the initiative with other partners will be critical to the continuation of the program. 3 In 1999, PRISM developed and tested a course in team management for 65 of the 89 Heads of Health Centers in its program area. The trainers were surprised at the intensity of participants’ response to the Program. The trainees provided feedback that they weren’t just changing as managers, but as individuals LCSP Evaluation Report – July 2003 1 The initial dialogue meetings focused the attention of the participants on the most important leadership and management challenges currently facing the MOPH, the leadership capacities that are needed to overcome these challenges, and the weaknesses that need to be addressed. In the first workshop participants familiarized themselves with the M&L model of leadership functions and practice, undertook self-assessments with respect to identified skills, and explored implications for individual behavior and team dynamics. Participants produced action plans, and many of them were visited some time after the workshop by the Guinean trainers, who served as coaches. The second workshop, held in August, focused on leadership and the understanding of team and organizational dynamics, including conflict management, on negotiating effective agreements and on aligning various parties towards common goals. Again, action plans were produced and participants revisited prior to the third workshop. The objectives of the final (third) workshop were to consolidate the skills acquired in the application of leadership functions in central and regional offices, increase self￾confidence as leader and as change agent, reinforce communication capacities within the organizations, deepen the understanding and the dynamics of human interactions, extend the staff’s scope for decision-making within each organization, and develop a systematic way of thinking within each organization. The six-month training schedule, including follow-up activities, is summarized in Table 1. Table 1: LCSP Training Schedule Stage Dates ( # days) Location # of Participants DIALOGUE April 2002 (4 days) Conakry 21 MODULE 1 June 2002 (6 days) Conakry 17 FOLLOW-UP 1 July – 2 August 2002 (1- 2 days each) 7 regions & Conakry 16 MODULE 2 19 – 23 August 2002 (5 ) Dalaba 12 FOLLOW-UP 9 –19 Oct. 2002 (1 – 2 days) 7 regions & Conakry 12 MODULE 3 28 Oct. – 2 Nov. 2002 (5 ) Kindia 13 CATCH-UP & COACHING 28 April – 3 May, 2003 Conakry 9 + 12 At the end of the final workshop, the Minister requested the following activities to support the ongoing effect of the completed program: and that the training had an impact on their personal lives. “At the end of the training, the person who gave the thank you speech cried. In a culture where one must keep so many things to oneself and there is very limited, if any, transparency, this gave them the means to take risks” (Alain Joyal, 2003). LCSP Evaluation Report – July 2003 2 a) Ongoing informal support networking between the participants as they practice their leadership skills; b) A coaching workshop to help the participants become coaches of future participants (especially the PHD) c) Periodical visits by the Guinean trainers to support and follow up the changes that were being implemented by the participants d) Annual visits by the international training team members (MSH and CAFS) to reinforce the application of the lessons learned e) An evaluation of the effects of the workshop in the participants’ own work environments within the first six months of the final session; f) The development of a monitoring system that will allow the Ministry (with the support of PRISM) to discern the impact of this and future leadership strengthening and development programs on the performance as the MOPH as a whole. The objectives of the proposed evaluation were to: a) Determine the extent to which the training program has achieved its behavioral objectives.4 b) Determine the extent to which participants have achieved their own performance objectives, based on leadership challenges identified at their own level. c) Make recommendations for the establishment of a monitoring system that will permit the Ministry of Health to monitor progress on a regular basis. d) Formulate recommendations to the MOPH and PRISM on the best strategies and approaches to extend the program to the other organizational levels of the Ministry. 4 We refer here to behavioral outcomes related to the leadership functions. They are described in the LCSP program summary (see Appendix E) LCSP Evaluation Report – July 2003 3 2 METHODOLOGY AND DATA COLLECTION PROCESS 2.1 Methodology Though the LCSP was designed as a six-month program for capacity strengthening, it is clear that it has generated a living and evolving process that is far from over. Given its timing, approximately six months after the third module of the program, this evaluation must be viewed within this context. At this early stage, it aims to understand the process rather than pass judgment on final outcomes. What justifies this approach further is that an understanding of the process will be essential to the interpretation of outcomes. Evaluation informants included the participants themselves, members of their immediate work groups, and key stakeholders including program facilitators, donor representatives and non-participant senior staff of the MOPH having a direct interest in the program. It is important to note that the design of this evaluation was adapted to the time and resources that were available to the evaluation team, and did not include investments in sampling and direct observations of practices and procedures. At this relatively early stage of the participants’ implementation of what they had learned in the program, such an investment would have been premature. The principal methodology used for data collection was the semi-structured interview with selected individuals and groups. A thematic guide was established, ensuring that the discussions covered the same general topics even while they were adapted to local specificities and information flows (see Appendices B and C). Relying fully on the voluntary collaboration of key informants, it was important to conduct the evaluation in an unintrusive and flexible manner, adapting to the degree possible to local constraints and availabilities. The evaluation was conducted by two two-person teams, consisting of one interviewer and one note-taker each. Seven Regions were visited within a one-week period, from April 27 to May 2, according to the following schedule: Boké: Both teams jointly Kankan, Faranah and N’Zerekoré: Team A (Linde Rachel and Fatou Diaby) Labé, Mamou and Kindia: Team B (David Goldenberg and Karen Sherk). In addition, interviews were conducted with participants at the Ministry headquarters in Conakry between May 5 and 7. These interviews included: The Legal Counselor, the Counselor for International Cooperation, and the Director for Planning and Statistics & his team (Team A) The Inspector General & his team, the Director of Pharmacies & Laboratories & her team (Team B). LCSP Evaluation Report – July 2003 4 The total number of MOPH staff interviewed were, by rank category: Regional level: Directors* 7 Staff 59 Central level: Directors, General Inspector (heads of teams) * 3 Staff (team members) 20 Counselors* 2 Total 91 * LCSP participants Because the subject matter concerned personal behavior and perceptions, the interviews relied on open-ended questions that allowed the interviewee to express him- or herself as freely as possible. Interviews were audio-taped where possible, to provide backup verifications. In order to provide a more concrete context within which to examine essentially qualitative and broadly defined behavioral change outcomes, interview questions focused on specific challenges identified by the respondents. The challenges of the respondents work environment were described at two levels: a) the more general priority challenges of the respondents’ broader work environment; and b) specific performance improvement challenges based on the topics covered by the leadership program. The first category included improvements in the areas of job descriptions, the conduct and management of meetings, anger management, the management of interpersonal relationships. The second category included resource-related issues, accessibility to health services, relations with non-governmental and/or bilateral actors within the region, illicit drug sales, staffing issues. Guided by the interview schedules, answers to the following key questions were sought from the key respondent, i.e., the LCSP participant, him or herself: • Based on priority challenges identified, what was the respondent’s understanding of the situation (in his/her area of responsibility) and what were his or her responses? To what extent did these responses reflect the learning promoted by the leadership program? • What was the respondent’s training and job-related background that might influence his or her response to the leadership program? The participants’ responses were complemented and triangulated with information obtained from the members of his or her immediate work group and other collaborators LCSP Evaluation Report – July 2003 5 affiliated with the RHD office, such as the Regional Hospital Director and the Health Director of the Préfecture of the Regional seat. In addition to the interview process, each key respondent and his or her team members (where available) filled out a climate survey for his or her work group (Appendix D). Each participant also completed a brief self-evaluation questionnaire covering aspects of individual leadership practice. Given the logistical unfeasibility of sampling within the participants’ team environments, these questionnaires were applied for test purposes rather than as a systematic source of reliable data. 2.2 Data Collection Process The agenda of the data collection team was clearly ambitious, extending over seven regions in a large country with limited long-distance transportation options. Thanks to the invaluable support of PRISM staff, the agenda could be respected logistically. There was uniformly excellent cooperation and flexibility on the part of all participants, ensuring that all were available for the interviews, in spite of their busy agendas and the challenges imposed by less-than-perfect communication channels and travel schedules. On the other hand, the availability of the members of their teams was very mixed. At one regional office, nearly all staff were away from the office, participating in an important vaccination campaign. Another regional office was severely understaffed at the time of the evaluation visit. Thus the amount of information obtained from the participants’ team members, and the degree to which the participants’ responses could be triangulated with those of their colleagues varied a good deal from one site to another. Because the interviews were held only with those who were present and available at the time of the evaluation visit, we cannot claim, in most cases, that the responses obtained are fully representative of each participant’s entire team. The principal source for the findings reported here are responses obtained from program participants, their immediate work groups, and other stakeholders and key informants. Another important source is the documentation produced in the course of the leadership development program. While we can affirm consistencies and patterns in the responses received, we cannot affirm practices on the basis of direct observations or a review of data or records. Interviews conducted with the two local facilitators prior to the fieldwork were extremely helpful, and it is regrettable that we did not have time to review our findings with them after the fieldwork was completed.5 5 Both of them were fully involved in a catch-up and coaching workshops scheduled for that time. LCSP Evaluation Report – July 2003 6 3 CONTEXTUAL FACTORS There are several factors that both affected the LCSP and were affected by it: It is important to recognize that the senior staff – les cadres - of the MOPH constitute a small, interconnected world. There are only 900 doctors in the Ministry and half of them are located in Conakry. In our interviews we briefly touched upon such issues as the importance of informal networks, problems of favoritism, and the challenges of role confusion – e.g. managers who supervise former classmates and friends. Interpersonal relations are a critical, but formally unrecognized, element of the Ministry environment. The Leadership Development Program in Guinea challenges deep-seated cultural and historical patterns. The country experienced thirty years of autocracy during which initiative was strongly discouraged and obstacles simply accepted. Participants must also deal with “the often undiscussable clash between ‘modern’ management and the way things get done informally…(including) managing the expectations and concomitant pressure from friends and relatives to act on their behalf6 .” Internally, the MOPH also appears to be constituted of several different worlds. All the regional directors have spent their entire careers working at the district and regional level. They have no direct experience with central office dynamics. Conversely, with several notable exceptions, many central office staff had little experience of the realities and challenges of working at the periphery. In a similar fashion, there appear to be segregated career paths for physicians working in hospitals and those filling district and regional administrative functions. Senior health managers at the regional and préfectoral level work in relative isolation. They have no nearby peers.7 This is particularly true for the Préfectoral Health Directors (PHD) working in remote districts. In contrast, central office leaders are frequently in meetings with peers and have a chance to solicit advice and support. Conversely, the Regional Health Directors (RHD) have considerable autonomy and the power to shape their working environments. At the central level, department directors and counselors are sharply constrained in their freedom of action. As noted earlier, during the period October, 2001 – January, 2002, there was a major reshuffle of senior MSP management positions. Consequently, just prior to the initiation of the PRCL, every one of the 13 participants had been transferred to a new position. Five regional directors were transferred and two were promoted from PHD positions8 . Within the central office, several department directors were rotated to new departments or shifted to conseiller positions. As a consequence, these team leaders had only spent a few months (at most) with their new teams when the leadership training began. 6 PRISM LCSP Design Document, January 2002 7 Only one DRS mentioned sharing management and leadership concerns with directors from other ministries. 8 At the same time, their titles changed from IRS (Inspecteur Regionale de Sante) to RHD (Directeur). LCSP Evaluation Report – July 2003 7 The National Health Plan notes that the health system is beset by an uneven distribution of human resources. In particular, this applies to the disparity between Conakry and the rest of the country. Half of all doctors and mid-wives work in the capital, serving only 15% of the population. However, there are also differences in staffing patterns among the regions. Labe has seven senior staff members at the RHD level, while Faranah must make do with only two. The presence of particular partners in a region can have considerable impact upon a leader’s orientation. Several RHD who had worked with or were now working with GTZ were clearly stimulated by that organization’s commitment to quality. In the regions covered by PRISM, staff reflected on their previous trainings on leadership and team-building. All of the 12 LCSP participants had previous exposure to the basic concepts of management and leadership acquired in previous training. A number of them stated that the LCSP reinforced existing management and leadership practices and/or reminded them of previous learning. The National Health Plan lays great stress upon the management and leadership challenges posed by decentralization. However, the process of decentralization is not limited to the MOPH. It is affecting all operations of the Guinean government. Yet, it is unclear to what degree other ministries are supporting comparable forms of leadership training. Some of the participants have shared their leadership training experiences with peers from other ministries and with the local governors, and several expressed an interest in expanding the program beyond the MOPH. Resource-poor work environment9 . They have poor communication facilities. Most offices have to rely upon two-way radios which do not always function. Staff also complained of having to deal with sometimes overwhelming and uncoordinated requirements from the central level. The evaluation team The evaluation teams’ visits imposed strains on the DRS/DPS offices. Some had to simultaneously host an IMF delegation and accompanying journalists. In other cases, the offices were in the middle of vaccination campaigns. The LCSP sought to encourage leaders to rise above such inevitable constraints, to demonstrate that they could be effective despite resource challenges. The most influential individuals in the MOPH – the Minister and the Secretaire Generale – have strongly endorsed the LCSP. It was the Secretaire Generale who gave the initial go-ahead. Following the evaluation team’s debriefing session, the Minister was most emphatic in insisting that the program be extended to other staff levels within the Ministry. However, it is important to note that neither of them has completed the training 9 The results of the climate study suggest that this issue is perceived as a serious but unavoidable constraint. LCSP Evaluation Report – July 2003 8 modules10 They have not had the opportunity to absorb the LCSP’s concepts and values. To further their full “ownership” of the program, it is crucial that they become more intimately acquainted with its content 11. In the course of our interviews with the participants and with key informants, the evaluation team became aware of critical strategic issues that were not directly addressed by the leadership development program and its participants. While this topic falls outside our official evaluation mandate, we call attention to these matters because they will shape the capacity of leaders to affect real change in the health system. ♦ The health system is characterized by an informal system of hidden charges that must be borne by the client population. Our respondents called this “surtarification”. We understand that even at the lowest levels, health workers are charging for immunizations that should be free. Hospital clients must pay extra fees for services and medications. We believe that the LCSP participants were alluding to these matters in some of the ‘challenges’ that they defined: humanism, the alignment of individual and organizational interests, health sector financing, actual cost of essential drugs, control of health services, and staff motivation. ♦ Health information generated by lower levels is unreliable, and in many cases may simply be fabricated. The regional directors discussed their efforts to improve the timeliness and completeness of reporting from the periphery, but they avoided the issue of the quality of the information they were receiving. This problem undermines the system’s capacity to protect the population; leaders have no real basis for planning. 10 In contrast, a number of other senior officials who missed parts of the training did participate in a catch up session in May, 2003 as well as in a subsequent coaching training. 11 During our interviews, participants remarked that several of the most senior MSP officials could certainly benefit from improved leadership skills, particularly in the management of meetings. LCSP Evaluation Report – July 2003 9 4 FINDINGS: THE CHALLENGES M&L’s leadership framework (2002 version) considers the alignment of individual interests with the organizational mission a critical element in strengthening leadership capacity. According to its participants, what sets the Guinea experience apart from prior training experiences, is its acknowledgment of the personal and more introspective aspects of leadership practice. Module One of its three-module curriculum is organized under the heading of “Leadership and Self Knowledge” and includes explorations of individual learning styles, emotional intelligence and interpersonal relations, as well as group dynamics. Module Two addresses itself to organizational processes and structures, and Module Three to issues at the systemic and institutional level.12 Because the broader issues to be addressed at the higher level will require more than six months to take their course, the evaluation focused on challenges and outcomes within the more immediate environment of the leader-participant. The notion of challenges was introduced during the dialogue sessions held for senior Ministry staff in April 2002. Challenges are proactive restatements of problems that involve positive efforts to overcome in order to achieve desired results (adapted from J.Galer, 2003). In contrast to the concept of a problem that evokes the image of a burden imposing itself on its victim, a challenge implies taking charge and rising to the occasion. One of the activities during the dialogue meeting was the identification of the major challenges faced by the Ministry of Public Health in improving the health of the Guinea population. To a large degree, the result of this exercise reflects previous critical thinking that took place during the strategic planning process and the national reviews. The LCSP continued to work with and revisit this list of challenges throughout the training modules and produced a condensed set of challenges at the end of Module 3 in November, 2002. The evaluators propose that the identified challenges can be organized according to three levels: a) Challenges related to work climate and organizational culture How can we: • Correct the lack of humanism in provider-patient interactions? • Improve staff motivation in context of total absence of social security? • Protect those who exercise the health professions (e.g., via internal rules and procedures)? • Bring about true open-mindedness among health officials? • Get health and community teams to behave more responsibly with respect to their own health? • Increase the professional qualifications of staff? • Reconcile the personal/individual interests of staff with those of the organization? 12 Though it should be noted that each subsequent module revisited the personal/individual level, and that interim follow-up visits from the facilitators continued to discuss personal issues. LCSP Evaluation Report – July 2003 10 b) Challenges related to management practices in MOPH units How can we : • Ensure true communication between various levels of health system & between MOPH and outside? • Assure that existing facilities are equipped and maintained? • Inspect (control) and monitor health services in a decentralized setting? • Ensure true decentralization of services with health districts fully operational? • Develop research activities? • Improve the referral system up and down the health pyramid? • Carry out truly integrated and effective supervision? c) Challenges related to service delivery for Guinea’s population How can we: • Guarantee the availability and accessibility and quality of essential drugs for the population? • improve health sector financing? • shift from the manipulation of communities to community empowerment? • Fight against illegal medical practices and non-approved pharmaceuticals? • Reconcile the actual cost of essential drugs, posted prices, and the poverty of the population? • increase the population’s utilization of services? These challenges were reviewed throughout the LCSP and provided the backdrop to the development of individual challenges and action plans. While the concept of challenge is quite central to the LCSP, it is important to keep in mind that the use of challenges is a means to realizing training objectives and not an end. The participants’ definition of challenges was dynamic and constantly evolving. Consequently, the interviewers left it to each respondent to discuss those that preoccupied him or her at the time of the survey. The challenges identified by the participants can also be organized by level of application: challenges involving behavioral change on a personal level, challenges involving the immediate work team, and the broader organizational or institutional challenges that are faced by the respondents as health system professionals at the regional or national level. These are clearly not discrete categories, but parts of a continuum that constitutes the scope of leadership for the individuals concerned. The first two categories are related to the process by which challenges are transformed into desired outcomes at the (third) broader institutional level. All participants described steps taken to address personal and organizational challenges. Only a few also discussed their attempts to grapple with larger institutional or systemic challenges. LCSP Evaluation Report – July 2003 11 Challenges addressed at the individual level include self-knowledge, the management of emotions (particularly anger) and response to the unexpected, humility, listening, understanding the other, and awareness of individual learning styles. Challenges involving the immediate or core work team include issues of effective management operations: job descriptions (revision, clarification, communication), meetings (preparation, conduct and reporting), delegation (of ongoing duties, of interim authority), updated hospital bylaws, communication and transparency at all levels. Broader challenges include issues involving effective service delivery: systematization of supervision, integration of health services, coordination with external partners, improved vaccination coverage, illegal drug sales, the problem of charging non-approved fees to clients entitled to free services. Following the participant interview, each evaluator worked “outward” to the participant’s immediate work team and beyond, depending upon availability. In the case of a regional office, the director’s core team corresponds to a set of positions prescribed by the Ministry. In addition to basic support staff (secretary, etc.), the core structure includes, at a minimum, four section heads (primary health care section, “lutte contre la maladie” section, administration/finance section, and the planning, research and training section), the regional pharmacy inspector and the regional hospital director. The actual composition of the Regional Health Director’s (RHD) core team was found to vary considerably at the time of the evaluation, from only two key people in Faranah to seven in Labé. At the central level, the size of the core group depended on the support function of the office. In addition to size, the availability of core staff on the day of the interviews varied across the Regions, as mentioned earlier. At some of the sites, the interviews included préfectoral level staff and/or section heads from the regional hospital, (and in one region from préfectoral hospitals), providing input from a (slightly) more distant perspective. In the next section we will look at the effects of the leadership program at primarily two levels: the personal and the organizational. 4.1 Personal Challenges The objectives of the Program component that focused on self-knowledge are quite complex and it would have taken more than one or two interview encounters to probe all possible aspects. Given the limited time available, comments on the personal effects of the Leadership program were invited with one or two open-ended questions to capture the participants’ most immediate responses. A similar approach was taken on the topic with their colleagues/teams. In some cases, the respondents themselves opened the discussion on the personal effects of the program. The answers indicate that in the majority of cases, participants greatly valued the opportunity that the LCSP offered for personal reflection and change. LCSP Evaluation Report – July 2003 12 The challenges identified by the individual participants at the personal level included: management of emotions and interpersonal relations, interpersonal communication, self￾knowledge, humility and the art of listening. “Management of the boss” evoked a lot of interest among participants from the central office, possibly because the hierarchical relationships at the central office are very pronounced. By contrast, directors at the regional level are more autonomous, and considered the highest authority within their sector. The most frequent reference to change at the personal level was to the control of emotions, and that of anger in particular. Comments include: “I exercise better control of my emotions now. Before, I was blunt and aggressive, now I’m more reflective.”13 “I try to think before I do something.” “I now try to put myself in the place of the other and I won’t do anything without prior consultation.” 14 Taking time to consider the needs of the other was felt to be a positive development: “I try to know everything about the person concerned (with regard to transfers) and I’ll talk to them beforehand.”15 And so was the increased accessibility to others: “I find that my colleagues approach me more easily now.”16 Humility was an important area for attitudinal change: “It has allowed me to accept certain difficulties on my part, I acknowledge that it is my (own) weakness …”17 Participants from the central offices reported having a better sense of self-control in their relations with their superiors and with their peers. According to one, “I initiated a dialogue (with my boss), I now visit him. He was surprised. He asked ‘why this change?’ I share things (with him); things have become better (since then).”18 In most cases, there was a fairly strong degree of consistency between what the program participants themselves reported as behavioral change at the personal level and what the members of their immediate work teams observed. According to the subordinates, a notably positive development was their director’s willingness to listen and try to understand the views of others before making a decision: “before, he did not readily take an interest (in us / our standpoint). He publicly cut short our comments.19 Across the various comments from subordinates and colleagues noticed their directors’ increased patience, greater willingness to listen, and greater accessibility: “Today we have access to the boss in order to talk as a group or individually”. Most, though not all, of the team members appreciated the personal aspects of the program and its effects on their supervisor. 13 “Avant, j’étais un peu brute et violent. Maintenant, j’ai une façon plus réflective.” 14 « Je cherche maintenant me mettre à la place de la personne et je ne fait rien sans consulter. » 15 « Sur le plan de mutation, je cherche à savoir tout sur la personne et je la consulte avant. » 16 « Mes collègues me fréquent beaucoup plus qu’avant. » 17 “Ça m’a permit d’accepter certaines difficultés que j’ai à mon niveau; je reconnais que ça c’est ma faiblesse …” 18 “j’introduisais le dialogue, je lui rend visite maintenant, et il a été supris. Il m’a demandé pourquoi ce changement? Je partage des choses; les choses ce sont améliorées.” 19 « il s’interessait difficilement .. il nous a coupé la parole publiquement .. il te voit mais il ne te parle pas » LCSP Evaluation Report – July 2003 13 Given the highly personal nature of the change process that was introduced, it is not surprising that its effect varied considerably across individual participants. While the immediate effects of the personal aspects of the program were predominantly positive among the participants, they appeared to vary along a continuum of application of tools and concepts from self to group. In the most pro-active case linking the personal with team leadership, the respondent had gone on to study the learning styles of his team members, had invited their comments on his own style, had discussed the personal circumstances of individuals scheduled for transfer and had become actively involved in resolving personal conflicts among his staff. In other cases, the participants concentrated on gaining greater self-understanding and changing personal habits. Only in one case did a participant consider the personal aspects less important than the organizational aspects. According to one of the premises implicit in the Guinean LCSP, personal change is a means to improving leadership capacity that then plays itself out at the collective level. Thus, the evaluation process began with the participant-leader and then moved concentrically outward to focus on the program participant’s immediate work team in an attempt to detect effects of behavioral change. The diagram in Figure 1 depicts the movement of the evaluation focus from leader outward to the team level and beyond, and inventories the issues addressed at each level. Figure 1: Changes in Practices and Behaviors & Evaluation Focus Leader self-awareness & behaviors: Self-knowledge Management of own emotions Humility Understanding individual learning styles Managing one’s boss Understanding others Providing and requesting feedback Transparency (financial and other) Team processes: Meetings Delegation Job descriptions Supervision Information sharing / feedback Participatory decision making Institutionalizing feedback Institutional effectiveness: Broader challenges identified collectively Leader Self-awareness & Behaviors Team Processes Institutional Effectiveness F O C U S O F E V A L U A T I O N during the dialogue and other systemic challenges addressed individually LCSP Evaluation Report – July 2003 14 4.2 Challenges in Organizational Practice Team Management and decentralization management were recognized during the Guinean Primary Health Care Program (EPI/PHC/ED) annual review meeting as weaknesses that needed to be addressed actively. The idea of a Leadership/ Management Development course/program was received with enthusiasm by the MOPH senior staff during the review. The program has been accepted as a priority activity into the 2002 work plan of the EPI/PHC/ED.20 One of the most important reasons for launching the LCSP was the concern with team management in the context of decentralization. The National Plan of the MOPH found that: There are no mechanisms for participatory management in health services because of a poor division of tasks, the absence of team work, and the lack of coordination between the different actors. 21 The document also notes “the absence of planning for recruitment and redeployment, of career plans and plans for the improvement of living conditions for staff. There are no formalized framework or objective criteria for decisions in human resource management.” 22 The LCSP therefore put great emphasis on capacities related to team building and team management. Figure 1 summarizes the characteristics associated with cohesive, empowered and effective teams. These include qualities that promote a climate in which individuals feel involved, empowered, trusted and encouraged to be creative problem solvers. A second, related, set of characteristics represents key elements of operational effectiveness or good management. The challenges addressed by the participants are quite consistent with the concerns addressed above. From the standpoint of team building, most of these provided opportunities to promote the qualities associated with good team management and group climate: clarity, alignment, transparency. In this section, we review evidence of fairly specific transformations in management practice. Here we must emphasize their importance as means of change rather than 20 LCSP Dialogue Report, April 2002 21 Il n’y a pas de mécanisme de gestion participative des services de santé à cause de la mauvaise répartition des tâches, de l’absence de travail d’équipe et du manque de concertation entre différents acteurs. (Plan National de Développement Sanitaire, 2002, p.28) 22 Absence de plan de recrutement, de redéploiement, de plan de carrière, d’améliorations des conditions de vie du personnel. Pas de cadre formalisé et de critères objectifs pour les prises de décision en matière de gestion des ressources humaines. (Plan National de Développement Sanitaire, 2002) LCSP Evaluation Report – July 2003 15 ends in and of themselves, in that they provide leaders with the opportunity to practice and systematize new insights and skills. These changes in management procedures were therefore evaluated not simply as deliverables, but as elements of a different way of working that would bring about, eventually, positive impact on health services in Guinea. Figure 1 below attempts to map the inputs and effects that provide the conditions for achieving desired outcomes at the systemic level. Figure 2 From LCSP Inputs to Organizational Effects and Systemic Change LCSP inputs Practice and Organizational System-wide Systematization Effects Change + Individual insights and skills: Self-knowledge Management of Emotions Understanding of others (learning styles, etc.) Humility Transparency Sharing of infor- mation / power + Specific challenges at organizational level: Meetings Delegation Job descriptions Information sharing Feedback Supervision Group climate: Participation Empowerment Transparency Innovation Trust Operational effectiveness: Systematization Clarity Coordination Alignment Integration Critical analysis Feedback Achievement of desired outcomes: Improved health service delivery Increased demand for services Improved health status of client population The challenges involve management practices that are in and of themselves not new to the participants, at least not in their basic principles. The majority of the participants had had prior management training, some in the United States, and were familiar with good management practice. Furthermore, the Ministry was already focusing on the strengthening of a number of management practices and tools that required revision, such as job descriptions and hospital bylaws. What threw a new light on these and rendered them more interesting and meaningful was the human relations perspective introduced by the LCSP. As one participant noted, unlike prior courses, the LCSP course confronted him directly with the issue of behavioral change: “I understood that my management training lacked the management of human relations”. Addressing the challenges allowed the participants to exercise new attitudes and behaviors in leadership that had implications for broader issues as well. Most of the participants understood this, though some better than others. The reported performance in relation to these challenges varied between initiatives that were primarily technical and motivated by efficiency concerns to those that clearly leaned toward the introduction of team-building elements. LCSP Evaluation Report – July 2003 16 Job Descriptions # A majority of the participants identified job descriptions as one of the issues they needed to work on. Of these, most were concerned with matters of clarification and the need to ensure that each subordinate had a complete understanding of his or her tasks and responsibilities. It seems that this already constituted an improvement over a situation where staff lacked clarity on the existing division of responsibilities. As one RHD put it: “if the tasks are not stated clearly, everyone thinks its someone else’s job.”23 At a more progressive level, staff members had been asked to not only review their job descriptions, but to revise them, distribute them and share them with others, so that the team as a whole became more knowledgeable about their respective responsibilities. In one case, the RHD encouraged staff members to analyze the existing job description critically, especially the gap between Ministry expectation and actual performance. Each team member was then asked to propose ways for closing the gap. Thus the changes range from improved clarity to an encouragement of staff to critically analyze and propose solutions, i.e., from a more control-based, mechanistic approach to one of a manager who “leads” and “facilitates”. Delegation and Information Sharing Improved delegation of power and responsibility was another need recognized by a majority of the participants. Several RHD had already established the practice of circulating the position of interim director among their senior staff during their absences. Their concerns were largely with greater clarity, efficiency and transparency. Improvements here involved clearly detailed and posted notices of what had to be done by whom during the director’s absence. One RHD emphasized the need for the interim director to provide him with a detailed report on tasks undertaken during the period, and provided extensive feedback on the report. Taking it a step further, another RHD stressed the importance of conferring authority and ensuring recognition along with the delegation of responsibility. He delegated his presence at certain public events or training sessions, passing on the rewards that went with such representation. He noted the frustration caused by a purely task-based approach to delegation and encouraged more team involvement. He also felt that his staff had become more appreciative of the challenges of his own job since they had taken on delegated assignments: “They thought that the boss had everything. Since I’ve started to delegate they understand the situation better”.24 Another participant now allows his team members continuous access to his office whether he is absent or present. In one region, the team members commented that they were now much better at delegating among themselves - “we have each other’s keys”. Another regional director keeps a set of colored folders on his desk, one for each member of his team. As he receives memoranda from the Ministry he makes it a regular practice to place each item into the folder for the appropriate colleague. It is his manner of delegating. A department director at the Ministry now prepares with his entire team for his meetings with the Minister. 23 “Si les choses no sont pas précisées, tout le monde pense que c’est le travail des autres.” 24 “Ils pensaient que le chef avait tout, mais depuis que je délègue, ils ont compris la situation.” LCSP Evaluation Report – July 2003 17 Meetings The improved use of meetings became another vehicle to exercise newly acquired skills in leadership. This was an area of change that had caught the attention of subordinates in particular. Reported improvements included increased clarity, better planning and greater efficiency as well as those focusing on the process itself such as rotating the chairs of the meetings, seating arrangements, and being attentive to how the information flowed both during and after the meeting. One consistent departure from prior habits was the regularity and predictability of the meetings. In one case, the director’s staff had begun to schedule their own team meetings improving cross-communication, and so creating greater integration via the meetings. Supervisory Systems Some of the participants at the regional level mentioned their initiative in the area of supervision at the Préfecture level (health centers, hospitals, etc.) and of the regional hospital. While these initiatives had been planned independently of the LCSP, the training influenced the manner in which they were implemented. There was clearly more emphasis on a team approach. Instead of one single person conducting the supervision visits, a multidisciplinary team was established. For example, hospital staff served as members of supervisory teams for the Préfectoral Health Director (PHD) and vice-versa. More care is taken to ensure that the teams include a full range of skills. “We do everything together (now). We (even) develop the supervision agenda together.”25 One regional director noted that he now makes sure to summarize the results with the team before writing his report. “Before, I thanked people, I left, I wrote a report.”26 “The training has provided us with an approach that facilitates the work in the field, which creates trust between the supervisor and the supervisee.”27 In one case, the supervision report was sent directly to a Ministry director (not an immediate supervisor) to get feedback. This seemed an unusual occurrence involving the risk of a negative response, but it could be attributed to the importance of feedback that was stressed during the leadership program. In yet another case, staff noted that the supervisory process now used a team approach, while before, it was the responsibility of one single person. It is noteworthy that none of the RHD have them selves experienced supervision since they assumed their positions in October, 2001. At the same time, very few regions have been subjected to inspections. The IGH team was only able to perform 20% of its planned inspections in 2002.28 Decision-making 25 “On fait tout ensemble. On fait le calendrier de supervision ensemble.” 26 “Avant, j’ai remercié les gens, j’ai quitté, j’ai fait un rapport.” 27 “La formation nous a permis d’avoir une approche qui facilite même le travail sur le terrain, qui crée la confiance entre le superviseur et supervisee.” 28 This was partially due to the reassignment of its resources to intervening unplanned priorities. LCSP Evaluation Report – July 2003 18 There are certain decisions involving staff that have traditionally been made unilaterally at the director level or above without consultation with the staff. Included are decisions concerning transfers and temporary special assignments involving travel and per-diems. What staff appreciated particularly in one case, and attributed directly to the effects of the leadership program, was a new transparency in the way such decisions were made. According to the RHD, he examined closely and discussed each transfer case with the individual involved before making a decision. And he invited input from the members of his team before designating individuals for special assignments. In a number of group interviews, team members emphasized that they now reach decisions much more frequently through group consensus. 4.3 Challenges at the Institutional Level We will define the “institutional” as the MOPH structure beyond the team level within which the directors and their teams function. Issues related to the institutional context include: • Relations with superiors • Relations with peers belonging to other sections within the MOPH, other ministries, or non-governmental partner agencies • Relations with internal (MOPH) clients (for those agencies providing services and support at the central level) • Other challenges involving service delivery Relations with superiors The relationship with superiors was clearly an important issue with all participants, though it differed in the extent to which it seemed of immediate concern between the regional and the central participant groups. Geographical proximity played a role. While the Regional Health Directors had relatively limited direct contact with their superiors (the General Secretary and the Minister of Health), participants from the central agencies had to deal with their superiors from a much closer range. There was some evidence of initiatives to gain a better understanding of the person in the superior role, and to initiate more contact. The central office participants have become keen observers of the leadership practices of their superiors. They use meetings as opportunities for making observations and discuss these among themselves, garnering support for practices that are different and involve some measure of risk within the traditional ministry environment. This is an area where cultural expectations and traditional protocol are deeply ingrained and where change is likely to be very gradual.29 29 The participation of remaining senior Ministry staffers in the May catch-up and coaching sessions should help facilitate this cultural transformation. LCSP Evaluation Report – July 2003 19 “Retro-information”, or the giving and receiving of feedback, had become an important concept to all participants and, in some cases, had become a routine requirement within their own organizations. In one case, the practice clearly involved some risk when one of the regional directors sent a monitoring report showing negative figures to the Ministry departmental director for feedback. The normal practice is to make the figures look good before they were sent up through the hierarchy. When this director received a negative reaction to the reported situation, he issued a second report that insisted on the actual figures, emphasizing that they were the result of an effective monitoring system in his region. Coming spontaneously from this director, we consider this a fair indicator of success in the Guinean context where the willingness to take risks will be a necessary component of improvements at the systemic level. Relations with Peers The LCSP encouraged the participants to develop a network that would allow them to exchange ideas and become a vehicle for mutual support. Most participants expressed an appreciation of the value of such a network. While this initiative was in its very early stages, there was some evidence of using the network capacity: “we consult each other before we go to see the Minister”30 Other examples indicated a strengthened and mutually beneficial relationship with selected members of the LCSP cohort. One RHD cited a case where he was able to obtain rumored information from another network member that he would not have had access to otherwise. Another noted that relations with his peers (i.e., other RHDs) have become more friendly and open, and have shifted from purely professional to more collegial. 4.4 Challenges involving service delivery In our interviews, although the participants had not been asked specifically to elaborate on initiatives concerning broader issues at the level of service delivery, some went on to discuss them as challenges to be addressed rather than simply problems. One regional director recounted dealing with the issue of vaccination coverage in his region. Having conducted a survey that revealed a coverage rate of only six percent, he had called together his team to determine the cause. It was found that field agents were illegally charging mothers for the vaccines. He has since then implemented a revision of the remuneration system for field agents, noting: “it was critical to get these agents to understand that one can earn one’s living without making the mothers pay.”31 The illegal sale of drugs in local markets was mentioned on at least two occasions as particular challenges, though no specific actions were proposed. The problem of refugees flooding the local health delivery system led to stepped-up attempts to coordinate initiatives with non-Ministry agencies in the area. Though it is likely that these initiatives would have taken place without the LCSP, the greater attention paid to the quality of related 30 “On se consulte maintenant avant de voir le Ministre.” 31 “Il fallait que les gens comprennent qu’on peut gagner sa vie sans demander à la maman de payer.” LCSP Evaluation Report – July 2003 20 encounters may strengthen the effectiveness of these initiatives. The implementation of a fully “integrated health system” in his region was another such challenge that one of the regional directors had been actively involved. It is clearly difficult to relate these initiatives to the LCSP’s influence, but they do convey a spirit of embracing challenge rather than of submitting to complaining about that mirror’s the program’s challenge￾focused approach. 5 FINDINGS: THE FOUR LEADERSHIP FUNCTIONS AND TEAM￾BUILDING It is evident that the participants’ actions to address the identified challenges provided them with opportunities to practice their new interpersonal skills and behaviors. As first steps in a problem-solving process, these could be applied to broader issues and challenges as well. What was new about this process for the participants, given their historical and cultural context, was a focus on the inter-relational aspects of management as a basis for building a well-informed and well-motivated team and creating a group climate that encourages performance towards positive results. Thus their responses to the challenge become inputs into team-building capacities that set the stage for addressing the broader and more systemic issues. The scope and timing of the evaluation limited its primary focus to the assessment of immediate effects at individual and team levels. However, the findings also indicate the possible linkage between individual behavioral change, improved leadership capacity and the achievement of results. In one case, a new emphasis on the importance of open feedback motivated one of the regional directors to transcend the long-standing reluctance to publish negative findings without prior “massaging”. Taking a public stand permitted others to see that a) the move was not fatal, and b) the data becomes more meaningful. In a second case, the supervisory activities in the field intensified considerably after the chief of reproductive health had been encouraged to revise her own job description. In a third case, the practice of charging fees illegally was confronted and challenged by the director, using a team approach to problem solving. These are cases where we can glimpse the broader implications for service delivery of relatively small changes in routine practices. We will now revisit the first objective of the evaluation: Determine the extent to which the LCSP has achieved its behavioral objectives. Behavioral change objectives, defined at the outset of the LCSP (see Appendix E), were guided by the M&L framework. Subsequent to their identification of the challenges, the dialogue teams identified the leadership practices required to address those challenges. They then conducted self-assessments and ranked the functions accordingly those that needed the most strengthening and reinforcement. LCSP Evaluation Report – July 2003 21 Table 1: Ranking of M&L Functions based on the need to address them M & L FUNCTION COMMENTS 1. INSPIRING Must be improved at all levels; The deficit is important; It poses real problems 2. FOCUSING Need to follow up on an initiative and carry it to all levels; Not often done and not well done 3. ALIGNING/ MOBILISING Needs to be improved; Needs initiative; Affected by lack of resources 4. ORGANIZING It’s done, but not completely; It is a mid-level deficit; Not particularly well done 5. MONITORING & EVALUATING Needs to be improved, medium deficit; it is done, but not well 6. IMPLEMENTING There is implementation at all levels, but not at 100% because of a lack of resources; Things are done very well when there are sufficient resources 7. PLANNING It is done, if not in an excellent manner; The deficit is not important; It’s often done well 8. SCANNING Done informally; often done well The following conclusions are based on a review of the responses of the program participants. After the verbal interview which focused in particular on the challenges, we asked each respondent to fill out a self-evaluation questionnaire on the practices of the four functions that had been applied before and after the first LCSP training module. Their reliability is fairly weak due to the breadth of each item to be rated. The overall tendency was for ratings to remain approximately the same or to go down in some cases. This is partly because the self-ratings were quite high to begin with. We also noted a tendency toward lower self-ratings among those that had shown themselves in the interviews to be more analytical and self-critical. One conclusion that can be drawn from the results is that the less one knows the more highly one tends to rate oneself. To examine the extent to which the LCSP’s behavioral objectives have been achieved within the participant group, we return to the program’s original outcome statements: 5.1 Scanning Participants can talk with authority about their district or region, its health situation, its trends, needs, tendencies, the needs as expressed by key stakeholders; they can also (articulate) the challenges they face (especially insofar as decentralization is concerned) and know what are the assets that are available to address the challenges and move toward realization of the vision; participants know who are the key stakeholders in their region/district, know how to identify key stakeholders, and can articulate the needs and priorities of each of those groups. LCSP Evaluation Report – July 2003 22 Terminology aside, the basic concept of scanning was not new to the participants. In fact, dialogue participants had rated scanning as their lowest priority for improvement among the eight M & L functions. All of them demonstrated a thorough familiarity with the situation in which they worked and the challenges, resource needs and stakeholders involved. What was new was the notion of pooling knowledge and analyzing problems collectively as a team. As one participant put it: “No matter what the problem, I (now) try to bring everyone together to look at it and discuss it collectively. After that I can choose which course to take.”32 One aspect of scanning that needs to be strengthened is the synthesizing of information. 33 Documented as well as verbal evidence shows a tendency toward problem description and the creation of lists, but less in the form of a systematic and critical analysis that identifies underlying causes and patterns. The evaluation team identified this weakness in the course of the interviews as they reviewed some of the more complex “challenges” with respondents. 5.2 Focusing Participants have/or can articulate a shared vision for better health for their region or district ; they have a strategy* for helping each organization within the district/region to define its mission/strategy/priorities and their relationship to the realization of the shared vision (*strategy should be clearly defined (in form of a document) but could be at various stages of implementation (each organization’s work in this area could be recorded in some form and available for sharing); when each organization has gone through this process, district/regional leaders should lead an effort to harmonize the various organizational efforts. Some of the respondents spontaneously articulated their vision for better population health status and discussed their strategies voluntarily. Visions cited included an “integrated system” of health facilities (including private) across all Préfectures, the reduction of illegal pharmaceuticals in Guinea’s markets, and the improvement of the vaccination coverage rate for children of less than one year. It is possible that those who were preoccupied with personal issues such as the management of their emotions were less likely to be focused on a broader vision and strategy. Most of the participants had made attempts to get their staff familiarized with their responsibilities vis-à-vis Ministry expectations through a review and/or revision of the job descriptions with the members of their staff. The IGH team did expend considerable effort on reviewing their function within the Ministry and trying to communicate it to others. In terms of a documented strategy, the Ministry’s National Health Plan seemed to serve 32 “N’importe quel problème, j’essaie de réunir les gens, exposer le problème, discuter ensemble, après je peux choisir quelque chose.” 33 Which involves, of course, a certain element of focusing. This is an example of the artificiality imposed by an analytical separation of the four functions which need to be integrated in order to be effective. LCSP Evaluation Report – July 2003 23 that purpose. In this particular context, the challenge was to ensure cohesion with the Ministry’s plan. The evaluation teams did not have the time to review the various documents produced at each site and cannot confirm whether these constitute what might be considered clear strategic guidelines. We know that the Annual Operational Action Plans produced by each office include strategic elements. However, these usually come in the form of an enumeration that rarely provides clear evidence of a cohesive strategy vis-à-vis the National Health Plan. Evidence of focusing could also be found at less strategic levels such as clarifying team functions, focusing on quality issues and on improved ways of conducting meetings and sharing information. 5.3 Aligning/Mobilizing Participants have or can develop strategies for aligning/mobilizing the key stakeholders; they have led effort to coordinate district/regional goals with organizational goals and health personnel goals and to convert these goals into shared tasks (the results of this effort should be in the form of a document or series of documents); organizations throughout the district/region have plans that are aligned to support the overall district strategy; they also know how to mobilize the identified assets; have increased confidence in their ability to negotiate deals/partnerships for better health; they know how to turn a group of workers into a cohesive team and have proven their ability in their workplace and beyond (in the larger district/region). There were numerous references to coordination meetings (réunions de concertation) with the objective of aligning interests and/or action plans with local or national partners. One of the best examples of alignment was the integrated system described by one participant as follows: “…an integrated health system in all Préfectures of the region with the objective of creating a critical mass of health professionals with a similar vision of the organization of health care...”34 The planning of well-timed staggered meetings between the Regional and the Préfectoral offices can be seen as an indicator of the alignment of information and tasks between the two levels of administration. In one case, the Hospital Director, with the support of the RHD, was working through a local mediator to negotiate resources for a hospital addition with a multilateral agency. An alignment orientation seemed to be fairly endemic to RHDs who have the operational freedom to achieve alignment across sectors and health structures within their regions. In contrast, central office participants could achieve internal alignment within their own offices, but ran into territorial barriers when trying to work within the larger Ministry context. 34 “…l’approche global du système de santé integer de district dans les préfectures sanitaires de la region en vue de la mise en place d’une masse critique de prefessionnels de santé avec une vision similaire en matière d’organisation des soins de santé …” LCSP Evaluation Report – July 2003 24 Another important kind of alignment is the ability to bring together learning experiences. In two Regional Offices, participants discussed the relationship of the GTZ quality concepts with what they had learned in the LCSP. According to the participants’ ranking of aligning/mobilizing, the difficulty here was associated with a lack of resources. We felt that the regional group in particular demonstrated strong skills in that area, in part as a matter of survival. Their positions would become very difficult if they were not constantly aligning interests and demands both vertically and horizontally. However, the LCSPs notion of alignment goes beyond coping mechanisms for mere survival. There was some evidence of considering new alignment possibilities, such as the suggested initiative to consult the peer group before addressing the Minister. As peer networking habits increase, more possibilities for horizontal alignments are likely to emerge. The participants’ more immediate concern was the challenge of transferring the art and skill of alignment to the members of their teams. Some participants were trying to use delegation to encourage their staff to go beyond taking orders and to start connecting the dots for themselves. Enhanced supervision practices by several RHD are also being used to instill alignment skills in PHDs and the heads of health centers. Insistence on feedback as a routine mechanism is likely to encourage an alignment orientation further yet. 5.4 Inspiring Participants are attracting support from above and followers from behind/below because they are believable, congruent in their actions; organizations throughout the district or region are committed to the process of collaborative planning and action. During the initial dialogue, inspiring the team, as a leadership function, was ranked first in order of importance by the participants. A lot of emphasis was placed on the leader as role model. There was clear evidence of initiatives taken in this area. In two cases, participants had actively sought to get support or feedback from above. In one case, this resulted in a letter from the Minister, congratulating the participant (an RHD) for organizing a general coordination meeting with local partners. This official sign of recognition was mentioned with pride not only by the director, but also by the members of his team. Within their own teams, delegation proved to be an important instrument in inspiring staff. One staff member, having been entrusted by the RHD with the analysis of service statistics (normally the responsibility of a clinical section head), enthusiastically talked about the task and his achievements. Staff felt encouraged when they were given options, as in the case where the decision concerning travel assignments was opened up to the team as a whole. Comments made by the members of various teams express respect and LCSP Evaluation Report – July 2003 25 admiration for their leaders’ strengthened interpersonal skills: “He’s a good boss. His social behavior with the people he supervises – it’s extraordinary. He puts himself at the level of others. When there is a problem, He’s the first to be with you to help solve the problem. “35 And again: “This change – the feeling that one is valued as being responsible …. (he’s) always in the process of asking ‘what do you think of this?’ The director, he trusts us.”36 “He puts people at ease so they communicate (easily)…. he participates in the solution of problems”. 37 “Before him, there were other inspectors, but the level of information was in “zig-zags”. Now everyone has the same information in inspections. That’s new… all of this brings with it a certain trust at the team level. He knew how to best take on the important elements.” 38 “Before, things just sort of happened … now, I sense everyday that I’m needed. I have work to do.”39 And appreciative of a dynamic leader: “The director has a way of integrating things and finding ways of involving her section (across) the different programs, ensuring that pharmacists are well accepted in (all) sections. It opens new horizons for us”.40 These are some of the comments that represent the positive sentiments found in groups surrounding the leader, clearly indicating pride and enthusiasm. Understandably, it is more difficult to capture the less sanguine viewpoints in explicit statements. Suffice it to say, that there is clear evidence of the ability to inspire within the group of participants. A second evaluation objective focuses specifically on team building and team management: Determine the extent to which participants have achieved their performance objectives, based on leadership challenges identified at their own level.41 The individually identified challenges addressed by the participants are consistent with issues of team building, in as much as they are used as a means to that end. In most cases, the activities related to these challenges provided the participants with opportunities to promote the qualities associated with good team management and group climate: clarity, alignment, transparency. 35 “C’est un bon chef. Le comportement sociale qu’il adopte avec les gens qu’il gerent – extraordinaire. Quand il y a un problème, il est le premier d’être avec vous pour résoudre le problème.” 36 “Ce changement là – le sentiment qu’on est responsible …. toujours en train de dire, qu’est ce que vous pensez de ça? Le directeur, il fait confiance.” 37 “Il met les homes à l’aise pour communiquer … il participe à la solution du problème.” 38 “Avant lui, il y avait d’autres inspecteurs, mais le niveau d’information était en “zig-zag”. Maintenant, tout le monde est au même niveau d’information dans les inspections. Ça c’est nouveau… Tout ça a amener une certaine confiance dans l’équipe. Il a su profiter pour prendre les elements importants.” 39 “Avant, tout venait comme ça … maintenant, je sens chaque jour, qu’on a besoin de moi. J’ai du travail à remplir.” 40 Maintenant, la directrice façon d’intégrer toutes les directions. toujours trouver quelque chose à impliquer son secteur là dedans. Les différents programmes, sensibiliser les gens au point que les pharmaciens soient acceptés dans les secteurs. Ça nous ouvre des horizons.” 41 From evaluation scope of work (see Appendix A). LCSP Evaluation Report – July 2003 26 During the visits of the evaluation teams, a climate questionnaire that is currently being developed by M&L was administered to the leader-participants and to those team members that were available at the time of the visit.42 The climate instrument consists of fourteen statements on perceptions of group experience. Respondents were asked to provide their appreciation of each statement on a scale of 1 through 5 in terms of its importance to the respondent on the one hand, and perceived actual practice in their team on the other. The questionnaire, including guidelines, is available in Appendix X. Table 2 on the following page summarizes the score averages for group versus the leader scores on actual practice43 and importance44. When we review the scores in order of importance within the “Practice” column for the group, we see the highest scores (shaded and bolded) assigned to statements 3 and 8, both referring to issues of clarity. Statement 5, concerning access to resources, receives the lowest score (shaded only). There is a remarkable degree of agreement between group members and leaders in the assignment of these scores where clarity and resources are concerned. On the other hand, there is less agreement on the question of decision making between group members and leaders. This is also where the gap in perception of practice is the largest (4.5 – 3.7 = .8), in contrast to the remaining differences which are quite small (i.e., do not exceed .3). The gap between importance assigned and actual practice is greatest for statement 5, access to resources, though it should be noted that the “importance” rating is also relatively low, leading to the conclusion that staff members accept this as a condition of their work. The gap between importance and practice is also high for statement 6, referring to the development of capacity and knowledge. This, too, could be perceived as an issue involving resources. Comparing the scores assigned to importance, there is no remarkable gap in perception between group members and leader. Overall, the scores seem to describe an organizational climate that assigns high value to clarity, especially on the part of work group members. However, it should be noted that the team members are not evenly represented here. In the case of one of the more “progressive” (in terms of a team orientation) participants, almost the entire team was out on a vaccination campaign. Its availability might well have changed the pattern we describe here. 42 This questionnaire is in the process of being tested. One previously adapted version had been applied at an earlier time during the LCSP, but the exact conditions of its application were not known. Still, an attempt was made to bring the two applications in line for comparison on a number of common items. 43 Full column heading: “Is this currently practiced in your work group?” 44 Full column heading: “How important is this to your work group?” LCSP Evaluation Report – July 2003 27 Table 2: Summary of Climate Study Scores 45 Statement Group Leader Practice Importance Practice Importance 1 We are recognized for our individual contributions 3.5 4.4 3.8 4.3 2 We feel we are moving in the same direction 3.9 4.4 3.8 4.4 3 Our work group has clearly stated goals 4.2 4.7 4.4 5.0 4 We understand how our work contributes to the overall goals of the MOPH 4.0 4.5 3.7 4.1 5 We have access to the resources we need to do a good job 2.6 4.0 2.8 4.4 6 We develop our capacities and knowledge in a continuous manner to improve our performance 3.6 4.4 3.7 4.8 7 We are encouraged to explore and develop and experiment with new ideas 3.6 4.3 3.8 4.6 8 We have a plan that guides our activities 4.2 4.7 4.2 4.7 9 We are aware of our mutual capacities in implementing our activities 3.6 4.2 3.8 4.2 10 We know clearly what is expected of us 4.0 4.6 3.7 4.2 11 We can describe the needs of our clients on the basis of objective data 3.6 4.3 3.8 4.4 12 We participate in the decisions that influence the work group 3.7 4.5 4.5 4.9 13 We are proud of our work 4.0 4.6 3.6 4.5 14 We adapt easily to new circumstances 4.0 4.6 3.6 4.4 Among the leaders, there appears to be a more pronounced orientation toward participatory values. While we noted a clear concern with regard to clarity and information for a majority of participants across the various activities linked to the challenges, there was also a stated interest in making each of these activities more inclusive and collaborative. This preoccupation with the participatory process is quite consistent with the participants’ responses to the first LCSP Dialogue. Asked which aspects of the dialogue they had found the most striking, they responded with the following list of items: • The participatory method • Effective participation • The continuous search for consensus • Patience and productivity of the approach • The collective interest • Program planning46 45 A similar climate study tool (with nine items) was distributed to six of the same participants’ teams between the first and second modules. Comparing the current scores with those obtained for the matching six groups during the earlier application, we find a similar pattern with regard to clarity (high) and access to resources (low). The overall tendency is for the group scores to increase slightly for comparable items (e.g., statements 1, 3 and 7). The overall climate score for the six groups increases slightly. However, it is impossible to provide a meaningful interpretation of the original group findings without knowing more about the conditions under which they were completed. In all fairness, the first application of the questionnaire did not have the technical support it needed to provide a reliable baseline. LCSP Evaluation Report – July 2003 28 The results suggest that we are dealing with a participant cohort that is particularly receptive to the messages of participation communicated by the LCSP. The characteristics of this group as a factor that influences the course and outcomes of the program are discussed in Section 5. The appreciation of the more intrinsic features of the workplace, such as participation, is itself likely to be influenced by schooling and training. The climate instrument clearly requires more careful guidelines for a well-facilitated and consistent application. Because the items are complex and open to subjective interpretation, and because the samples are small, the instrument is very sensitive to the composition of the respondent group. One “loner” in the group can have a considerable impact on the scores, as was the case with one respondent who seemed to feel that certain group measures were not applicable to his situation. 6 FINDINGS: FACTORS INFLUENCING THE RESULTS 6.1 The LCSP Process The Training Schedule and its Components Beginning in April 2002, the PRCL process consisted of an introductory dialogue and three training ‘modules”. After both Modules 1 and 2, the local trainers – Oumar Diakite and Namoudou Keita – visited each regional director and the leader participants at the Ministry in Conakry. The follow-up visits planned after Module 3 were never carried out47. During the period, April 28 – May 3, 2003, while the evaluation team was visiting the regions, an “atelier de mise à niveau” (also referred to as “cours de rattrapage” or catch￾up session) was held for senior leaders who had missed all or portions of Modules 1, 2 and/or 3. The workshop took place over 4 days in half day segments. Participants included eight central office staffers and the RHD for Conakry, all of whom had been part of the dialogue session the year before. The following week a training session on coaching was held for all participants – the original 12 participants and the 9 of the catch￾up group. The Dialogue sessions were held in two two-day sessions with a split audience to permit for continued staff coverage at the Ministry. The Dialogue sought to establish: 1. Shared understanding of the challenges facing the MOPH 2. Awareness of the relationship between leadership practices and organizational performance 46 Response to the question: “qu’est ce qui vous a frappé le plus?” Report on Leadership Dialogue Meeting, April 02. 47 This is mainly due to the close out of PRISM I and the start up of PRISM II, which took place during the period when the follow-up should have taken place. LCSP Evaluation Report – July 2003 29 3. Clarity regarding the leadership practices needed to face those challenges 4. A Ministry-wide leadership development strategy Module 1 covered leadership and self-knowledge. Module 2 dealt with leadership and the dynamics of groups and organizations. Module 3 addressed leadership and changing ‘the system’. Participants rated each day’s session and filled out an evaluation form at the conclusion of each module. An Audience Geared for Success The original proposal targeted twenty participants48 twelve of whom completed all three modules. Some of the qualities of this first Guinean cohort to complete the LCSP may have contributed to its apparent depth of impact. ¾ The twelve participants who completed all three modules may have constituted a somewhat self-selected group. Despite busy schedules and heavy responsibilities, they persevered in finishing the program. The original 20 cohort members were specifically selected by the SG in consultation with the Chef de Cabinet and approved by the Minister – who insisted everyone attend all sessions. Those who didn’t had very valid justifications – generally special mandates in Guinea or overseas, or major events within their units or at the personal level. Some officials were unwilling to attend sessions out of Conakry and be cut off for extended periods from their offices. While it is notable that all seven decentralized DRS completed the modules, the Conakry DRS did not do so. ¾ This is a group composed of individuals with aspirations for further professional advancement. In contrast, the senior most officials of the Ministry (the Minister, the Secretaire Generale, and the Chef de Cabinet) were not part of this cohort49. Only one of the three Department directors completed the full set of modules. ¾ This is a generally confident group. While they claimed to be “shocked” by the manner in which the Program challenged them to examine themselves, they not only had the capacity to handle it, but they welcomed the challenge itself. ¾ All participants had considerable previous training experience, many in Europe and North America. They were familiar and comfortable with professional training processes. Several remarked that the LCSP did a good job of re-activating previously acquired knowledge. ¾ As a group, they exhibit good learning skills. They practiced recommended procedures such a keeping a journal or running through a reflection process. They studied the course materials and were active participants all through the program.. 48The 8 RHD, 3 National Directors (Public Health, Pharmacies & Laboratories, and Hospitals), the Secretaire-General, the National Coordinator of the PEV/SSP/ME, the Inspecteur General de la Sante, and the Minister Cabinet Director. 49.Obviously, had they participated, this may have presented serious challenges to the willingness of participants to be fully candid in expressing their views. LCSP Evaluation Report – July 2003 30 A Well thought out Training Structure The Program adhered closely to the design principles defined in the original design document: 1. “A very clear overall structure …that allows people to try out new things and make sense of abstract notions” 2. “The sequence of activities will permit frequent reflection …(and) allow participants to explore their own experiences.” 3. “Allows people to discuss emotional (gut) experiences with leadership concepts in the classroom with some measure of safety and enable the translation of those experiences into actionable ideas.” Many participants acknowledged that the structure of the training was particularly efficacious. Each module touched upon substantive issues that the individual leader could take back to his/her work environment and address with concrete actions. The subsequent follow-up visits from the facilitators presented opportunities for sharing and feedback with neutral agents. This training cohort had considerable previous training experiences, but asserted that this one had a particularly profound effect upon them. A Trusted Training Team with a Clear Vision It is notable that the Program maintained its focus and continuity even with a change in lead trainers between Modules 1 and 2. The clarity of definition of the M & L training approach eased the transition, but it is also a testimony to the strength of the West African training team. It is clear that the Program established a safe environment that permitted participants to stretch their boundaries, to experiment, and to explore new ideas. It is particularly remarkable the extent to which the high status participants entrusted themselves to the two (lower ranking) Guinean facilitators. The participants consistently rated the quality of their training experience as “excellent” and the majority would have preferred more extended modules. As one participant stated: “(Keita) helped me to recall many things (from the training). When I am confronted with a situation, I think of him and what we have discussed. (For me) it’s important that he is outside the Ministry of Health, but I have equal trust in Diakite, I can confide in him.” 50 Individual Action Plans and Follow-up Visits At the end of modules 1 and 2, each participant filled out a Plan of Action (POA) form, which was structured using the M&L framework. The individual identified a particular 50 Diakite is a staff member of the Ministry’s regional office in Kankan and Keita is a PRISM staff member. It is important to note that both are MDs. LCSP Evaluation Report – July 2003 31 challenge and defined, for each of the four leadership functions – scanning, focusing, aligning/mobilizing, and inspiring: 1) actions to be taken, 2) support or resources required and 3) evaluation criteria. The information provided in the individual POAs was extremely superficial and as a tool the POA does not appear to greatly influence the change process. While the evaluation team found that, in most cases, participants had taken steps to address these defined challenges, they also addressed a number of other challenge areas. In preparing for their follow-up visits to participants, the facilitators compiled lists of individual topics to review, e.g. the leader’s tone while speaking with collaborators, recognizing the learning styles of their team members, resource mobilization, how to influence senior officials at the MOPH, controlling one’s emotions. The objectives of the follow-up visits were: ♦ To evaluate with the leader the application of leadership practices specified in his/her plan of action. ♦ To strengthen the participant’s understanding and application of priority leadership concepts. ♦ To support the leader in resolving specific problems by applying leadership practices. ♦ If possible, to observe the leader in action (during meetings, supervision) ♦ To evaluate the level of collaboration of the leader with his colleagues, team members, and superiors. During each follow-up session, a facilitator was scheduled to meet first with the leader, then with team members, hold individual interviews with team members, and to conclude with synthesis session with the leader. Due to time constraints or leader preferences, this ambitious schedule was seldom followed. However, the facilitators were able to participate in a number of regular team meetings, to hold several “restitution” or debriefing sessions regarding LCSP concepts, and cover a variety of topics in different sites. However, several teams stated (to us) that “ (the facilitator) met privately with the leader.” In at least two instances, the facilitator found that the leader himself was unavailable and met only with his team. The follow-up sessions enabled facilitators to identify priority topics for reinforcement during the subsequent module. Key LCSP “Tools” Participants were provided with information binders for each module. During our interviews, many described frequently consulting these binders when struggling with a leadership problem. Some participants had distributed copies of materials on particular topics to their colleagues. The Program also provided each participant with small notebook to serve as a “journal intime” (intimate journal) in which to record thoughts or process emotions. When our interviews touched upon matters inter-personal relations, the participants would pull out their notebooks and flip the pages for us, demonstrating that they were using them. LCSP Evaluation Report – July 2003 32 The Program applied two assessment instruments. The “self-evaluation” questionnaire contains twenty items describing leadership practices (five for each leadership function). Each participant filled out the self-assessment during the Dialogue session in April, 2002. Subsequently, the members of participants’ immediate teams also rated the practices of their leaders. Following the first module, participants were asked to repeat the self￾rating. The team members that we interviewed recalled filling out the questionnaire, but said that the results were never shared with them. Several cited the very act of doing such a rating as significant. Each LDP participant viewed the aggregated scores from his/her own team. The evaluation team asked the participants to fill out the same self￾evaluation form during our visits in April and May 2003. Between Modules 1 and 2, a work climate questionnaire, comprised of nine items, was circulated to the staff directly supervised by the participants. This was not filled out by the leaders themselves, but only by their teams. In April and May, 2003, the evaluators had both the teams and their leaders fill out a revised climate study questionnaire containing fourteen items. 51 While the participants themselves were given some feedback concerning the results of the “self-evaluation,” it is not clear what further use the facilitators made of these two tools. Differences in Experience: Regions vs. Central Staff There were important differences in the quality of the application of their training for participants in the regions vs. those in the Central office. The RHD have effective control of their work environments. They can enact changes in the teams and organizations under their supervision – the RHD team, the hospitals, the PHD teams – and follow through to try to ensure that these result in broader systemic changes. In contrast, those housed in the Ministry building could stimulate team transformations, but then encounter barriers in the ability of their teams to operate in the larger institutional context. Another factor may be that roles and responsibilities in the regions are relatively well defined, whereas Central office staffers complained of ambiguity surrounding their functions. Feedback and Support Networks Most participants valued and benefited from feedback and opportunities to share their experiences as they progressed through the training. Aside from their contacts with the facilitators, participants sought out peer support. Central office participants became a close knit group that supported one another in difficult situations. They shared the experiences they were having with their own teams, in their personal lives, and with their superiors. Central staffers had the opportunity to assess the leadership skills of those above them. They described coming out of poorly conducted meetings and sharing their observations. The RHD, on the other hand, had fewer opportunities for such networking. 51 This information was presented earlier in Section 4.2. LCSP Evaluation Report – July 2003 33 In some instances, the RHD found a good audience in his planning, research, and training officer, his local PHD, or the DH of the regional hospital. Another RHD shared his experiences with peers from other ministries. In our interviews with them, the facilitators strongly emphasized the degree to which this first cohort had formed an active support network. In our interviews with the participants, we gave them the opportunity to discuss this point themselves. While they did affirm that mutual relations had certainly improved among the participants, they did not confirm the existence of strong support networks. Teams’ Familiarity with LCSP Concepts It is an established practice within the Ministry of Health that all individuals returning from trainings, should provide their colleagues with a “restitution” or feedback session, concerning their experience. However, we found considerable variation in how participants handled this with their teams, ranging from simple briefings/reports to mini training sessions. For example, some participants held formal sessions at which they provided their teams with a review of some of the key concepts. One RHD met individually with his team members and gave them assigned readings from the training. Others chose to wait for opportune moments to share their insights and new approaches. During our interviews with participants, they often pointed to the LCSP training materials and made references to frequently consulting them. One noted that he kept his office unlocked and that staff were free to look at the LCSP materials.52 In another’s office, we noticed that the materials were kept in a locked case. A long-term objective of the LCSP is obviously to transfer its concepts and fundamental values to as large an audience as possible, to achieve a “critical mass” of practitioners of good M&L practices. We asked each of those present at the group interview to define “leadership” and found most individuals were reasonably conversant with the key concepts. It should also be recalled that some of these team members had participated in the PRISM-supported M&L training for préfectoral level staff and some, as mentioned earlier, had had previous foreign management training (including the MSH training in Dakar). We found considerable variation among the teams regarding their sense of familiarity with the details of the LCSP. Some could describe its components in considerable detail; others stated, “he came back with many new terms,” or observed that the training had clearly had an impact upon their leader even if they did not fully understand it. In our interviews, team members stated they ended up with an impression that this was a very valuable training, but that they had no detailed knowledge of it. “We didn’t receive the training.” The facilitators contend that team members underplayed their exposure to 52 And we should add that this is a very uncommon practice in Africa. LCSP Evaluation Report – July 2003 34 the LCSP concepts (to the evaluation team) in order to strengthen the argument for their own future participation in the training. 7 CONCLUSIONS We have identified three basic assumptions underlying the LCSP approach that, on the basis of our findings, appear to be well founded and workable: a) the likelihood of achieving sustainable improvements in health service delivery is increased when managers have leadership capacity; b) leadership capacity is strengthened when managers know how to get their work teams to achieve results; c) in order to get their teams to achieve results, managers need to know how to promote and nurture a work climate in which team members are empowered and inspired. We believe that these assumptions can be scaled up to broader challenges as well. For the Guinean group of participants, the LCSP offered what was to them a unique and different approach to coping with challenges at all levels. Participants returned from their training experience with a new orientation to what were, for the most part, existing problems and issues. Consequently, they executed a number of changes in practice. The ultimate evaluation question is: will these micro-process changes contribute to a cumulative effect that can lead to more fundamental changes? At this point in time, it is clearly too early to determine such linkages, though we can conjecture their possible future manifestations. 7.1 Effects at the Personal Level All participants valued especially those aspects of the LCSP that had to do with personal issues. Only in one case did the participant consider the personal aspects less important than the organizational aspects. Through the various challenges that they set themselves, they were able to make advances in the following areas of their personal behaviors: control of emotions such as anger and impatience, willingness to listen, greater accessibility to others, humility. Potentially, these changes can provide a stronger basis for working with others in teams to face challenges and achieve results. However, the extent to which this potential linkage was established varied across the participants. 7.2 Effects at the Organizational Level In our discussions with the LCSP participants who were the subject of this study we found a group that was visibly inspired and engaged in its quest for self-improvement. While none of the four leadership functions (i.e., scanning, focusing, aligning and inspiring) in and of themselves introduced what we might consider “new” behaviors or practices, as a whole they helped to convey the message of a new approach and promoted a different way of conducting the same basic management procedures: delegation, meetings, etc. The most noticeable features of this new approach that we would identify are: LCSP Evaluation Report – July 2003 35 Participation According to the various responses, participants have introduced a new way of discussing issues and problems (scanning) -- collectively as a team. The comment quoted earlier in this section is a very apt expression of this new approach: “No matter what the problem, I (now) try to bring everyone together to look at it and discuss it collectively. After that I can choose which course to take.” Considering the climate study results, we suggest that the “new” approach remains to be communicated to, and fully understood by, team members, especially where decision-making is concerned. Team members clearly appreciate the greater clarity that comes through the sharing of information, the use of improved means of communication and opportunities for input. However, we have little evidence that they feel included in major decision-making processes. Instilling the skills of aligning and mobilizing in team members is likely to become easier once they feel more directly included in the decisions that affect their work. Team Building Across the participant cohort, a considerable amount of effort was invested in activities that will build and strengthen their teams. Job descriptions, meetings and delegation practices may seem prosaic as targets for improvement, but applying a new approach that promotes the engagement of subordinates and encourages their input lays the groundwork needed for building effective teams. We already see the positive response generated in subordinates through improved clarity and transparency. We found that some LCSP participants were more appreciative of this potential than others and would hope that there will be the opportunity to raise general awareness concerning this important aspect of their LCSP experience. Generating and Seeking Feedback The transparency and sense of sharing that resulted from this practice was quite evident and inspired attempts to make it routine practice and even push the boundaries of risk in some cases. The positions occupied by the members of this LCSP cohort are highly visible, potentially vulnerable and by tradition very lonely. There was a palpable sense of liberation conveyed by the participants when they discussed their discovery of new ways of sharing information and inviting feedback from peers, superiors and subordinates. LCSP Evaluation Report – July 2003 36 The Concept and Use of “Challenge” The evaluators struggled considerably to understand the sources and intentions of the individual challenges addressed by the participants. We found that the participants were not very clear about the concept of challenge and that perhaps we were forcing the use of the term in our interviews. However, our quest has clarified the value of using a challenge-focused approach. First, it replaces the traditional problem-focused approach that perpetuates the victim perspective (manifested in the ubiquitous grievance list (liste de doléances) one encounters in West Africa) with a more proactive approach. Second, associated with concrete activities at the team and organizational levels, challenges become a practical vehicle for applying a new approach and new ways of dealing with old procedures. From Efficiency-Focused to Team-Focused Orientation Challenges addressed at the organizational level focused on the improvement of existing practices. All of the participants had addressed one or another of these and had been able to achieve improvements. What we think is noteworthy for this evaluation is the extent to which these improvements went beyond technical efficiency to investments in team￾building as a basis for addressing challenges at broader levels and achieving sustainable results. This requires attitudinal change. The results of the climate study suggest a stronger emphasis on the importance of group participation among leaders than among group members. Such a difference would diminish as group members acquire a taste for a more participatory approach to processes such as decision-making. Among leaders, reported practices and achievements indicate that the importance placed on a participatory process varies. Based on these findings, we can develop a continuum of responses from those that are primarily efficiency-focused to those that manifests a team￾development orientation.53 53 This is an inductively produced matrix, based on findings. It cannot provide scores and numbers with regard to responses since our probings were not guided by this framework nor structured to obtain scores and/or counts. We simply reproduce the practices and approaches emphasized by the respondents in an entirely open-ended way. LCSP Evaluation Report – July 2003 37 Table 3: From efficiency-focused to team-focused orientation Managerial Practice / behavior Procedure / leadership function Clarity / efficiency Participation Empowerment Job Descriptions Individuals are asked to review their job descriptions and clarify them with their supervisor Job descriptions are shared with the team, inviting comments Individuals are encouraged to revise their job descriptions based on their own proposals Delegation Written delegation notice is posted and/or circulated within team Responsibilities are delegated on a rotational basis among members of the team The manager ensures that responsibility is delegated along with an appropriate level of authority Meetings Meetings are planned on regular basis, agendas posted in advance, minutes recorded and circulated Chairing of the meeting is rotated, opinions or feedback are invited Meetings are used to reach group consensus on important decisions Supervision Monitoring instruments are developed and used by individuals charged with the supervisory task A “multi-disciplinary” supervisory team is used Accurate results are published and used to improve performance Scanning Information is available at level of manager Information is available and shared within team Quality of information is improved by team through use of feedback and critique Focusing Priorities are established by leader or institution Priorities are reviewed and discussed with team Team established priorities through process of consensus Aligning / Mobilizing (objectives, resources, etc.) Manager coordinates, aligns and mobilizes Coordination meetings are used to align and mobilize as a team Each member of the team takes initiative to align and mobilize as a matter of routine practice Inspiring Manager communicates mission, ideas and invites support Team is inspired by manager’s ideas, behavior and achievements Team members generate ideas and are inspired by own achievements We can think of the first response category as that which applies to an efficiency-based model of a manager, and the third category as characteristic of what M&L would call the “Manager Who Leads”. Most of the responses we registered with regard to the challenges identified by the respondents we would categorize as efficiency-oriented, but we also noted a distinct effort to move toward the participatory mode and there is at least one participant that has initiated practices that contribute to team-empowerment. 7.2 Impact at the Institutional Level and Service Delivery Level It was not the intent of this evaluation to assess the impact of the LCSP on change at the broader institutional level or on service delivery. Still, there is some indication that the team-focused approach and certain individual behaviors promoted by the program have facilitated steps toward institutional change. The initiative to critically examine local vaccination coverage rates and bring illegal charging practices to light was based on a concerted team effort involving information sharing and transparency without which it LCSP Evaluation Report – July 2003 38 would not have been possible, according to the participant involved. Going against the standard practice of masking dismal service statistics was linked to the importance of honest feedback that had been stressed during the program. LCSP Evaluation Report – July 2003 39 8 RECOMMENDATIONS FOR THE NEXT STAGES OF LEADERSHIP STRENGTHENING There is general agreement that the investment made so far in the leadership initiative must be carried forward if it is to attain the critical mass needed for meaningful and sustainable change. One objective of the LCSP evaluation was to “formulate recommendations to the MOPH and PRISM on the best strategies and approaches to extend the program to the other organizational levels of the Ministry.” We make these recommendations based upon the Program’s strengths and weaknesses and an identification of the special qualities that contributed to its success. We have not addressed the crucial issue of funding for the extension of the effort. 8.1 Review the Strategic Implications of Leadership Development Leadership strengthening and team building are critical processes for the achievement of the strategic vision defined for Guinea’s health sector. As the Minister of Health stated: “Management (for) quality is an imperative. It is an imperative …for the struggle against poverty54.” It is vital that this strategic linkage be recognized throughout the Ministry and by its partners. The leadership development program needs to move beyond the operational level. We recommend that, with facilitation support, MOPH senior staff review the systemic challenges that they identified during the LCSP process with a more critical eye. These should be thoroughly analyzed so that clear linkages are defined on the manner in which the adoption of priority leadership practices can lead to effective management systems with an eventual impact upon the challenges facing Guinea’s health system. It is vital that leaders at all levels of the system share this understanding and vision. Such an understanding is required for an effective monitoring system. Leaders and managers should be in a position to reinforce practices that lead to long-term impact. 8.2 Use a team-based approach for Extending Leadership Training The participants were unanimous, often adamant, in recommending that the program be extended to a broader audience, especially within their teams. The notion of “critical mass” came up on many occasions, not just from the participants but also from their non￾participant colleagues. The need to speak a “common language” was another recurrent theme. Members of the leaders’ teams in the regions and at the Ministry emphasized repeatedly that they had an inadequate understanding of the concepts and the language involved, and that there needed to be coherence in this understanding before the benefits could become institutionalized 54 “La gestion, la qualité est un imperative, il est un imperative meme de la lutte contre la pauvrete” (Restitution session by the evaluation team at the MOPH, May 9, 2003) LCSP Evaluation Report – July 2003 40 Since the strengthening of team capacity is at the core of the leadership program, we feel that at least some portion of the LCSP training should be provided jointly to team members. In view of the logistical problems of moving regional teams out of their work environments, bringing facilitators to the site would seem to be a preferred option. In that case, a training agenda tailored to each site should be developed jointly with the facilitator(s). Having access to PRISM’s excellent resources in leadership training, the DRS of Haute Guinea could become trial sites for the development and testing of a team￾based curriculum that can then be adapted for use in other regional and central offices. 8.3 Replicate the LCSP Structure Participants greatly appreciated a form of training that alternated between theory and application and that was well adapted to their own challenging work agendas. They valued the opportunity to test new practices with their teams and then to receive support and reinforcement through facilitator visits and the subsequent training modules. 8.4 Consider Variations in the Composition of Participant Cohorts. Suggestions were solicited from the leaders and their groups concerning how best to structure any future leadership initiatives. There was general agreement that it would be difficult to group together participants of vastly different rank and seniority. Some felt that their staff should be trained together in order to enhance team-building. Others recognized the value of mixing individuals from different regions and departments. It may be worth considering taking participants out of their normal work environments for the first (personal growth) module in order to establish a neutral setting. Subsequent modules could then be applied for teams. 8.5 Understand the Characteristics of the Target Groups for Training The first LPD cohort had a number of qualities that may have enhanced the Program’s chances for success, notably their seniority, their previous management and leadership training experiences, their learning and study habits. Some PHD and DH share these characteristics, but many do not. Programs for other levels of leaders should be tailored to their particular strengths and experiences. 8.6 Review the Utility of the LCSP Tools The concept of the “challenge” needs to be better defined, both as a concept for systemic analysis and as a tool for leadership practice. Its use needs to be systematized. The POAs filled out by participants may have had value as starting points, but provided little LCSP Evaluation Report – July 2003 41 guidance for concrete actions. There did not appear to be a clear utilization path for the two data gathering tools – the work climate study and the self-evaluation instrument. As pointed out in the following section, these instruments can have value as a basis for participant and team discussion, for exploration of the fundamental concepts of leadership practice and team climate. However, their application must be carefully reviewed and planned if they are to yield useful information. 8.7 Continued Reinforcement of Leadership Practices at Senior Levels Given their full schedules and responsibilities, it is a challenge to find ways to familiarize the Minister and the Secretary General with the leadership functions, practices, terminology and mindsets with which their subordinates are now equipped. Their support is critical for the transformation in work climate that many of the 21 people who have been exposed, more or less, to the principles and practices of management and leadership, are pursuing. The Ministry should also support regular opportunities for its senior staffers to network and share their experiences in applying leadership practices. 8.8 Establish a Pool of LCSP Facilitators Many participants felt that the role of the facilitators was absolutely critical and recommended the creation of a facilitator core team within the MPS. Several felt that they themselves could also be instrumental in transferring concepts, techniques and skills with the help of didactic material through a mix of mentoring, modeling and discussions. However, it is critical to identify the characteristics necessary to successfully play the trainer & facilitator role. The capacity to foster a safe training environment is crucial. The current facilitators provide excellent models, even while they are considerably junior in rank and/or age to many of the participants of the program. It would be worthwhile to consider the recruitment of other graduates of the initial Training of Trainers, conducted several years ago by PRISM.55 8.9 Explore the Possibilities for a Synergy of Effort to Extend Leadership Practices At PRISM, the idea of coordinating a LCSP strategy with other interested partner agencies such as GTZ was discussed. PRISM and the MOPH should examine other agencies’ training and developmental orientations for possible convergence of interest. Perhaps the MOPH could approach other Ministries regarding their potential interest in a similar cross-sectoral program. 55 From which the current Guinean members of the LCSP facilitator team graduated. LCSP Evaluation Report – July 2003 42 9 RECOMMENDATIONS FOR A MONITORING SYSTEM A fourth and final objective of this evaluation was to: make recommendations for the establishment of a monitoring system that will permit the Ministry of Health to monitor progress on a regular basis56. Monitoring differs from evaluation in that it is: a) conducted on a recurring basis, b) done by staff internal to the organization, and c) used for self-corrective rather than judgmental purposes. Monitoring is basically a management function and should be designed so that it can be incorporated easily into routine operations. A monitoring system consists of a set of complementary mechanisms that, combined, permit the tracking of progress where a single monitoring tool is not sufficient. Monitoring systems should be parsimonious, i.e., focus on the most essential data only, if they are to be sustainable and well-utilized. The recommendations for a monitoring system are guided by the following assumption: LCSP inputs added to existing management capacity will, through mechanisms of team strengthening, improve team capacity to achieve observable improvements at the service delivery level. These assumptions are in line with those put forth by M&L’s results framework for Managers who Lead. The relationship between inputs, outputs and the longer term outcomes is presented in Figure 4. It is derived from the results framework and shows the role and function of the proposed monitoring system within a broader M&E scheme based on the stated assumptions. Evaluation looks at the hoped for end effects (of a truly effective health system) on the capacity to deliver quality health services, and at the ultimate impact on the health status of the client population. Monitoring is concerned with the means: the evidence of strong team-based management and leadership practices. 56 Evaluation Scope of Work, Appendix A. LCSP Evaluation Report – July 2003 43 Figure 4: Domain for Monitoring System within a Full M&E System Training Inputs Monitoring Evaluation Outputs Outcomes Results Impact Climate (IR2) Managers Who Lead Framework Leading* Managing** Scan Plan Focus Organize Align/ Implement Mobilize Inspire Monitor & Evaluate Management practices / leadership functions Organiza￾tional performance outcomes job descript. meetings, delegation, supervision … etc., tbd 1) Improved management systems 2) Improved work climate Improved Service Delivery Improved Health Status M&E data sources: MOST & Climate Study Service National Indicator-based monitoring statistics health surveys ________________ * Provided by LCSP ** Provided by prior management training Sources of evaluation data include service statistics, local or regional household surveys, and national surveys such as the DHS, disaggregated to the regional or zonal level. The monitoring system should reinforce the fundamental characteristics and values of the LCSP. It should: • be participatory • be evidence-based • be feasible and relatively simple to execute • encourage self-diagnosis and build evaluative skills • facilitate rapid feedback • complement current data collection structure To monitor progress on outputs and outcomes, we recommend a facilitated self￾assessment approach that becomes the basis for problem identification and action. The advantages of self-assessments are that they are participatory, transparent, and fairly easy to execute with the help of at least one skilled facilitator. The disadvantage is that the findings are difficult to compare across different settings and time points, since they are easily influenced by factors such as the composition of the respondent group (seniority, position, experience, etc.) and subjective interpretations of standards, terms and concepts, that are easier to control in a conventional more objective assessment. For the Guinean context where team work and transparency is very critical, we feel that the advantages LCSP Evaluation Report – July 2003 44 outweigh the disadvantages. This does not preclude the use of a more objective instrument based on selected indicators that have been standardized across the MOPH. Neither monitoring nor evaluation can be performed in a vacuum. They should be fully incorporated into project planning, where means-ends linkages are clearly spelled out and objectives are expressed in measurable terms. We therefore recommend a process that links monitoring directly with planning. 9.1 Improved Management Practices and Systems : the MOST MSH has developed a tool and a process, combining team-based self-assessment with planning for improved management. The Management and Organizational Sustainability Tool (MOST) process bring teams together to review systematically and critically their organizational functions, systems and processes. The tool provides guidelines for distinguishing between actual and desired performance, and helps groups to position themselves on a performance continuum. Its results can be used directly for a collaborative development of action plans. In addition, the MOST can provide an excellent introduction to the concepts of objectives and indicators and encourage evaluative thinking. While the focus of the MOST is on management procedures and systems, the process itself is team-based and fosters many of the qualities promoted by the LCSP, including participation, information exchange and feedback, transparency and learning within teams. It also facilitates the clarification of objectives and the identification of indicators MOST was introduced to the LCSP participants during Module Two, and during our evaluation visits a number of the respondents expressed an interest in learning more about the tool. We recommend that PRISM’s training team adapt the MOST to the needs of the MSP and introduce it in Haute Guinée (Kankan, Faranah and N’Zérékoré) before making it available to other offices (regional or central). 9.2 Improved Work Climate : the Climate Assessment Tool A test version of the climate tool was used for the evaluation. On the basis of our experience and in light of the criteria listed above, we recommend a number of adaptations to make it useful as a monitoring tool that provides a learning opportunity and encourages self-diagnosis and planning for improvement. Content: • The content of the statements is currently under revision at M&L • One column should be added to the right side of “Actual Practice” column for concrete examples of evidence LCSP Evaluation Report – July 2003 45 Process: • The climate assessment should be introduced by a skilled facilitator who ensures the opportunity to thoroughly discuss the concept of work climate and its characteristics. • Respondents are given at least 30 minutes to complete the Climate Study form. • The facilitator leads a small task group that analyzes the results according to the guidelines provided. • The results are discussed at length with the respondent group • Priority areas for improvement are identified and a 3-6 month improvement plan is drawn up. • The instrument is administered, in the same manner and to the same respondents, after 3 months and subsequently every 6 months. 9.3 M&L indicators: MSH has developed a “menu” of indicators for management and leadership for use by its technical advisors and clients, that is oriented specifically to its own approach. The leadership indicators were recently revised using input from a field inquiry in Brazil and from this evaluation in Guinea. The exploratory nature of the Guinea evaluation allowed us to describe detailed evidence of leadership characteristics that provided fertile materials for the indicator revisions. There are two uses we suggest for the indicator menu. First, it proposes output indicators for management practice that can be measured through observation. These, or an adaptation of them, can become incorporated into the current supervisory tools that have been (or are being) developed at several of the MOPH regional offices. Second, it proposes a set of indicators for the four leadership function which are consistent with our findings during this evaluation and which therefore offer a field-tested alternative to the current “Leadership Practices” questionnaire that has been used during training. LCSP Evaluation Report – July 2003 46 REFERENCES MSH, “PRISM Family Planning and Health Activities in Guinea”, Technical Proposal, 2002. MSH- PRISM, “Rapport de la conduite de l’atelier MODULE 2”, Août 2002. MSH-PRISM, “Rapport de la conduite de l’atelier MODULE 3”, November 2002. MSH-PRISM, “An Overview of a 6-Month Leadership Development Program (LDP) for Senior Health Managers in Guinea. MSH-PRISM, “Leadership Dialogue Meeting”, April 2002. PRISM, “Rapport du Suivi Post-Formation”, August 2002. PRISM II, “Plan d’Action Opérationnel 2003-2005”, 2002. République de Guinée, Ministère de la Santé Publique, Plan National de Développement Sanitaire 2003 – 2012 LCSP Evaluation Report – July 2003 47 APPENDICES A Scope of Work B Interview Guide for the Leader C Interview Guide for Group Members D Climate Study Questionnaire E An Overview of a 6-month Leadership Development Program (LDP) for Senior Health Managers in Guinea LCSP Evaluation Report – July 2003 48 APPENDIX A Management and Leadership Project (M&L) Management Sciences for Health (MSH) Evaluation of the LDP Intervention in Guinea Terms of Reference 1 Background and Objectives During the MOH’s last annual revue of the Guinean Primary Health Care System, weaknesses in the management of the (program team) and of the decentralization process in general were recognized as priority areas for improvement. The constraints of a long tradition of centrally controlled hierarchical structures where staff is used to receiving its orders from the upper reaches of the hierarchy have become clearly evident in the context of the country’s decentralization process and the development of good leadership and management skills at all levels (central, regional and local) more pressing than ever. Thus a program of reinforcement of leadership and management capacity was proposed and became integrated as a priority initiative in the FY02 workplan of PEV/SSP/ME. With the assistance of USAID, through PRISM57 in Guinea and the M&L Program in Boston, the PRCL (Programme de Renforcement des Capacités en Leadership) began in April 2002 with two leadership dialogue meetings at the hotel Camayenne in Conakry. This, then, became the basis for agreement to the PRCL program of a series of three workshops to be implemented over the course of several months, with the first workshop conducted in April and the final workshop in October-November 2002. The general goal of these workshops was to assist the participants in mastering and improving their skills in the exercise of the functions of leadership with a view of improving the performance of the Guinean health system. The initial dialogue meetings focused the attention of the participants (which represented the most senior levels of the GMOH) on the most important leadership and management challenges with which the MOH is currently faced, the leadership capacities that are needed to overcome these challenges, and the weaknesses that need to be addressed. In the first workshop participants familiarized themselves with a pragmatic model of leadership functions and practice, undertook self-assessments with respect to identified skills, and explored implications for individual behavior and team dynamics. Participants produced action plans, and many of them were visited some time after the workshop by the Guinean trainers, who served as coaches. The second workshop, held in August, focused on leadership and the understanding of team and organizational dynamics, including the ability to manage conflict productively, negotiate effective agreements and align various parties towards common goals. Again, action plans were produced and participants visited before the third workshop. 57 PRISM (Pour Renforcer les Interventions en Santé Reproductive et MST/SIDA) provides the Government of Guinea with specialized technical assistance in strengthening health systems and promoting sustainable increases in the use of health services. Its interventions include a wide array of support functions, including the application of tools for the improvement of management capacity. LCSP Evaluation Report – July 2003 49 The specific objectives of the final (third) workshop were to: • Consolidate the skills acquired in the application of leadership functions in central and regional offices • Increase self-confidence as leader and as change agent • Reinforce communication capacities within the organizations • Deepen the understanding and the dynamics of human interactions • Extend the staff’s scope for decision-making within each organization • Develop a systematic way of thinking within each organization At the end of the last workshop participants proposed the following activities to support the ongoing effect of the completed program: g) Ongoing informal support networking between the participants as they practice their leadership skills; h) An evaluation of the effects of the workshop in the participants’ own work environments within the first six months of the final session; i) The development of a monitoring system that will allow the Ministry (with the support of PRISM) to discern the impact of this and future leadership strengthening and development programs on the performance as the MOH as a whole The objectives of the proposed evaluation are to: e) Determine the extent to which the training program has achieved its behavioral objectives. f) Determine the extent to which participants have achieved their own performance objectives, based on leadership challenges identified at their own level. g) Make recommendations for the establishment of a monitoring system that will permit the Ministry of Health to monitor progress on a regular basis. h) Formulate recommendations to the MOH and PRISM on the best strategies and approaches to extend the program to the other organizational levels of the Ministry. An additional objective for M&L specifically is to use this study as an opportunity to test a series of leadership indicators in the field. 2 Proposed Methodology and Key Questions For the collection of data on the effects of leadership training, a well-triangulated qualitative approach will be used, using semi-structured interviews and other techniques such as concentric mapping. After making the necessary courtesy visits to the Mninister, the General Secretary and to the National Director for Public Health, the interview process will take place, beginning and ending with the direct beneficiaries of the leadership training program. These include the following: Seven Regional Health Directors, Two Special Advisors to the Minister of Health (Legal/International Cooperation) The National Coordinator of the PEV/SSP/ME, The General Inspector of Health, The National Director for Pharmaceutics and Laboratories The Director for Planning and Statistics LCSP Evaluation Report – July 2003 50 The combined total is 13 staff in key positions that include 7 regional and 6 national positions. In view of the broader objectives of the program, each of these individuals represents one specific work environment (or level) that is to be impacted by a new leadership style. The flow of inquiries will therefore move concentrically, from workshop participant to members of his or her work environments, immediate and beyond, then back to the participant. In order to determine the most appropriate informants to be queried within each work environment, the first interview should include a concentric mapping of the reach of influence by changes in leadership style and practices. Two sets of questions will guide the interviews, corresponding to objectives a) and b) above: a) Questions concerning changes in behavior and practices and their immediate effects on staff response and organizational climate. b) Questions concerning the extent to which specific leadership challenges have been addressed by each participant in the program. The leadership challenges defined at the outset of the Guinea leadership program are ambitious, because they are broad challenges for the system as a whole. They were revisited at the end of the last workshop in order to gauge whether any relevant new initiatives had been taken or other ones being planned. The objective of this part of the inquiry is to a) assess the determination of the participants in the face of the obstacles that are certain to present themselves in their own work environments; b) assess the sustainability potential of newly acquired attitudes and practices. In the course of the data collection phase, appropriate indicators will be identified, tested and revised for use in the Ministry’s leadership monitoring system. Selection of sites and participants for the evaluation: Because of the client’s interest in monitoring future progress, we recommend that all sites be included in this evaluation. This includes the seven regional sites and the Ministry of Health in Conakry. At each site, information will be collected on as many perspectives as is reasonable and practical. The selection of informants will be guided by a concentric map created in the course of the first interview with the workshop participant. This map can then become the basis for identifying additional informants, using a snowball approach to selection. The interviews will follow a thematic guide, using open-ended questions as a basis for guided probing. 3 Implementation Activities The interview schedules will be drafted by the Boston-based evaluation team, circulated to associated resource persons in Guinea (PRISM and MOH) and Boston, and revised for on-the￾ground testing. On-site evaluation activities will be scheduled for April 2003 (see calendar below). An evaluation team composed of two consultants will join relevant PRISM staff in a final review of the interview schedules in Conakry. These schedules will be pre-tested with selected local resource persons that are not included in the evaluation itself. LCSP Evaluation Report – July 2003 51 For the collection of the interview data, we propose a team of two: Interviewer (consultant) and note-taker. This will free up the interviewer to dedicate him/herself completely to the dynamics of the exchange and to the probing process. The pre-test will provide an opportunity to develop an productive team coordination between interviewer and support note-taker during the interviews. Indicators to be tested will be included in the interview schedules and special note will be taken according to a pre-defined list of criteria. Visits to the sites will include an investigation of monitoring capacity, guided by a standard set of questions. It will be important to permit sufficient time prior to leaving each site visit to process the information obtained and check for gaps or any additional information needed. 4 Deliverables • Interview guides • Criteria for testing indicators • Guide for assessing monitoring capacity • Report plan • Evaluation report • List of recommended monitoring indicators • ? (further deliverables for monitoring system?) 5 Division of Responsibilities a) M&L-Boston – Sylvia Vriesendorp, Linde Rachel, Karen Sherk Preparation of the terms of reference for the evaluation, selection and recruitment of evaluation consultants, development of key questions and draft interview guides, provision of background materials to consultants …… b) PRISM – Alain Joyal or delegate: Preparations in Guinea: review of terms of reference and interview guides, participation in the selection of groups and informants; selection of a local note-takers, ; logistics for preparatory visit and data collection; feedback on report. c) PRISM – (administrative support): Scheduling of interviews, travel arrangements, etc. d) Regional Consultant – (tbd): Background reading; participation in preparatory phase in Guinea; collection of data; analysis and report. e) Note-takers: Participation in preparatory phase in Guinea; collection of data; transcription of notes. LCSP Evaluation Report – July 2003 52 6 Proposed Calendar of Activities: Activity Dates /time intervals* Site of activity Person Respon￾sible Resource persons involved Communications, selection of consultants, other preparations Jan.-April 03 Boston SV LR; NL; KS Evaluation terms of reference Jan. 03 Boston LR SV; NL Draft interview guides Determine indicators to be tested March 24 - 28 Boston LR SV; NL In-country pre-study preparations: Review of terms of reference and interview guides; recruitment of note-takers; finalization of logistics April 03 Conakry LR AJ; consultant In-country preliminary consultant activities: Conduct preliminary interviews with key informants (trainers, key stakeholders); Pretest and finalize interview guides; Train team members April 21 - 25 Conakry LR Evaluation team; in-country support team Data collection April 28 – May 8 Guinea Region LR Evaluation team; in-country support team Processing and preliminary analysis of data; follow-up interviews with key informants in Conakry; preparation for presentation of preliminary results May 9-14 Conakry LR Evaluation team (including SV) Debriefings and formal presentation by evaluation team ( debriefing meetings with the MOH and with the Mission on the first day and formal public presentation on the second day) May 15-16 Guinea LR Evaluation team; (including SV) Analysis & draft report, including recommendations for indicators to be used and other aspects of the monitoring system May 19 – June 13 Boston LR Regional consultant Review of draft June 14-28 Boston / Guinea LR AJ; SV; NL; JG; other tbd Final report Due July 15 Residence / Boston LR Regional consultant; Report distributed July 16-18 Boston LR SV; NL; KS * proposed dates, tentative for now Legend: SV = Sylvia Vriesendorp (Principal MSH Trainer) LR = Linde Rachel (Senior Research & Evaluation Associate) NL = Nancy LeMay (Senior Program Officer, R&E Unit) AJ = Alain Joyal (PRISM person and intermediary) KS = Karen Sherk (Senior Program Assistant – R&E Unit) JG = Joan Galer (Director of Leadership Unit – M&L) 7 Deliverables • Interview guides • Recommendations for extension strategy and monitoring system, including a proposed set of monitoring indicators • Presentation of preliminary results – set of overheads/handouts • Evaluation report LCSP Evaluation Report – July 2003 53 APPENDIX B EVALUATION DU PROGRAMME DE LEADERSHIP GUIDE D’ENTRETIEN DIRIGEANTS D’EQUIPE Questions pour sondage / informations Objectif de question / instructions / commentaires 1 Identification / informations: Nom et titre Combien de temps dans le poste actuel? dans la region? avec MSP? au niveau central? Sur base d’une carte institutionnelle du MSP, indiquez les contacts et connexions principaux – interne et externe Quels sont les caractéristiques et défis speciaux de cette région/ce bureau : population, ressources, partenaires potentiels ? Comment diffère-t-ils de votre dernière région / dernier bureau (si applicable) ? Le processus de décentralisation, comment ça c’est passé dans votre région ? Quelle est la situation des ressources disponibles et potentielles dans votre région ? Combien de personnel y-a-t’il? Quelles sont les lacunes au niveau du personnel ? Sans aller trop dans les details de carriere, on aimerait savoir si l’individu a eu une experience de la région ainsi que du niveau central La cartography (mapping) nous permet de visualiser et capter les relations internes et externes du Directeur Ces informations fournirons des points de reférence pendant les discussions suivantes sur les défis du poste. 2 Historique de formation: Au cours des derniers trois ans, qu’est-ce que vous avez reçu comme formation en gestion et leadership (hors PRCL)? par type ou thème, fournisseur, période. Les expériences antérieures de formation pourront influencer les capacités du répondant. 3 Parlons du programme de leadership: C’est comment que vous-êtes venu au programme de leadership ? Quelles étaient vos attentes au début? Est-ce qu’elles ont changé? Si oui, dans quel sens? Quelles sont vos attentes à l’heure actuelle? Face aux développements stratégique et politique au C’est pour faciliter la memoire, clarifier le contexte, et préparer le répondant pour les questions suivantes. Nous aimerions savoir comment le répondant LCSP Evaluation Report – July 2003 54 niveau du MSP, le programme de leadership s’inscrit comment dans le plan global? conçoit l’intégration du PRCL avec les priorités du MSP. 4 La restitution avec le staff : Avez-vous partagé les idées et les acquis du programme de leadership avec votre staff ? Si oui, comment ? Avec qui ? Est-ce qu’il comprennent les concepts ? Qu’est ce qu’il leur faut encore pour qu’ils soient en mesure de bénéficier de ces idées ? Quel est le programme futur de communication de ces idées ? 5 Parlons des défis: Avant de lancer la discussion sur le “défi de référence” (le défi choisi pour cette évaluation), nous aimerions revoir votre expérience générale avec les activités autour les défis et les plans d’action élaborés au cours du PRLC. a) Le fond (background): Au cours du programme de leadership, vous avez identifié, collectivement, une liste des défis. Et nous avons vu l’évidence d’action à travers des rapports des encadreur et des plans d’action. Spécifiquement à votre niveau, qu’est ce qui s’est passé? b) Le défi choisi / de référence: Quel est le défi? Parmi d’autres, vous l’avez choisi pourquoi ? et comment ? avec qui ? Qui sont les autres qui collabore sur ce défi ? Est-ce qu’ils constituent une « équipe » ? Donnez-nous le contexte aussi complet que possible de ce défi Pour sondage : Scruter: Qui sont les concernés, les intéressés ? Quels sont les besoins spécifiques à adressés ? Vous avez travailler sur ce défi depuis quand ? Avec qui ? Quelles sont les questions clés dans ce défi ? Les questions clés ont-elles changées au cours de ce travail ? Quelles seront les conséquences possibles ? D’où viennent vos informations – de l’intérieur (au MSP), de l’extérieur? Expliquez comment vous / L’obejctif est d’établir le contexte du “défi de référence” en passant par l’expérience générale sur les défis et les plans d’action au cours du PRCL. C’est une question ouverte qui permettra au répondant de donner l’histoire de son expérience avec toutes les actions entreprises autour les défis et son plan d’action. I’l s’agit de déterminer le processus de reflexion dans le choix et l’inclusion des autres These are the key questions. We are trying to understand fully if and to what extent and how the leadership functions have been applied in meeting real challenges. Refer to the challenges listed in Annexe 4 of the Third Module Report. Are they different from what the director used before? If so, in what way? We want to know whether they are using networks and external sources for their information? LCSP Evaluation Report – July 2003 55 votre équipe ont fait pour avoir les informations appropriées par rapport à ce défi ? Que pensez-vous de la qualité et fiabilité de ces informations ? Que faites-vous pour vérifier les informations ? Qui s’engagent dans les recherches ? Focaliser: Pourquoi et comment le défi a-t-il été sélectionné ? Avec qui ? Est-ce qu’il existe un plan d’action ? Si oui, il a été dévéloppé par qui ? On peut le voir ? Est-ce les objectifs sont clairs ? raisonnables ? cohérents avec les objectifs du MSP ? Est-ce qu’on a fait une revue de la mission et la stratégie au cours de la formulation du plan d’action ? Est-ce qu’il y a eu des négotiations à faire au cours de ce processus ? Si oui, expliquez comment et avec qui ? Aligner/mobiliser : Parlons des ressources – humaines et autres – exigées pour aborder le défi. Avez-vous des ressources suffisantes pour l’atteinte des objectifs du défi. Sin non, que faites-vous pour aller à la rencontre du défi. Si oui, qu’avez-vous fait pour vous assurer les ressources nécéssaires ? Est-ce qu’il ya une équipe qui s’occupe de ce défi ? Si oui, comment l’avez-vous établi ? Est-ce qu’il y a un plan de travail pour cette équipe ? Les responsabilités, comment sont-elles distribuées à travers des members de cette équipe ? Avez-vous l’appui de vos supérieurs dans cette affaire ? De vos collaborateurs dans d’autres services ou agences ? Etes-vous satisfaite avec le progrès ? Si non ou oui, pourquoi ? Inspirer : Quel est le niveau de cooperation ou collaboration de votre staff par rapport à ce défi ? Avez-vous reçu des suggestions ou d’autre appui de la part d’autres participants du PRCL ? Est-ce qu’il y a d’autres sources d’inspiration et de feedback hors de la structure DRS ? Expliquez en détail. Fournissent-ils des conseils, suggestions, informations ? Utilisez-vous les commentaires et suggestions de votre staff et d’autres Nous cherchons à savoir s’il y a des inititiaves de développement de reseau. LCSP Evaluation Report – July 2003 56 collaborateurs ? Comment ? Donnez des exemples. Sont-ils capable de libérer le temps nécéssaire à cette collaboration ? Comment assurez-vous une amélioration continue dans ce domaine (du défi) ? A travers quels dispositifs ? Y a-t-il d’autres défis sur lesquels vous travaillez à l’heure actuelle ? Si oui, lesquels ? Quel sera votre prochain défi prioritaire ? Pensez-vous utiliser la même approche pour l’aborder ? Que ferais-vous de différent ? 6 Sur le plan personnel – observations de soi-même: Par rapport aux acquis personnels, quels ont été les choses (capacités, leçons, etc.) les plus importantes et utiles que vous avez obtenu à travers le PRCL? Spécifiquement, parlons des quatre fonctions de leadership: scruter, focaliser, aligner/mobiliser, et inspirer: comment s’appliquent elles dans vos activités quotidiennes ? Est-ce que vous agissez de manière différente depuis le PRCL? Si oui, comment ? 7 Au niveau du groupe: Au niveau du staff, est-ce qu’il a eu des améliorations dans leur performance depuis le PRCL ? Si oui, expliquez. Pour sondage: Scruter: est-ce que le staff s’informe activement & volontairement sur tous les aspects de leur travail ? Est-ce qu’il obtiennent les informations les plus importantes ? Est-ce qu’il prennent des iniatives pour obtenir les informations qu’il leur faut ? Est-ce qu’ils vérifient les données qu’ils reçoivent ? Focaliser: Est-ce que votre staff a une bonne compréhension, collectivement, de votre mission et stratégie ? Aligner/mobiliser: Le staff, est-il bien capable de travailler avec les ressources disponibles? Inspire: Est-ce qu’il y a eu des propositions ou des initiatives innovatrices de votre staff? Nous voulons connaître les perceptions du directeur sur la performance de son équipe et l’influence qu’il pense avoir eu depuis le programme de leadership. LCSP Evaluation Report – July 2003 57 8 Au niveau institutionnel (MSP): Au niveau de MSP, est-ce qu’il a eu des changements dans les relations entre la région et le niveau central ? Si oui, donnez des exemples. Y a-t-il eu des changements dans vos relations avec vos collègues dans les autres régions / bureaux? Anecdote pour le “leadership booklet survey”: s’assurer qu’il y ait les réponses à ces questions Si vous reflechissez sur votre expérience en tant que leader globalement, Comment fait-on pour emmener les autres: a) à identifier et/ou faire face à un tel défi b) à achever des résultats en surpassant ces défis Quelle compétence vous a été la plus utile en encouragant les autres de se mettre en face des défis et d’achêver des résultats ? Est-ce que vous pouvez nous donner un exemple? Appliquer l’étude de clima au directeur LCSP Evaluation Report – July 2003 58 APPENDIX C EVALUATION DU PROGRAMME DE LEADERSHIP GUIDE D’ENTRETIEN EQUIPE DE TRAVAIL Questions pour sondage / informations Objectif de question / instructions / commentaires 1 Identification / informations: Après introduction de l’équipe d’évaluation et de l’objectif de cet entretien, faire le tour du groupe : Nom et fonction Combien de temps dans le poste actuel? Formation reçue en gestion et/ou leadership. Sans aller trop dans les details de carriere, on aimerait savoir si l’individu a eu une experience de la région ainsi que du niveau central Have they had team mgt. or similar courses ? 2 Parlons du groupe: Vis-à-vis le directeur / du défi, où se situe ce groupe ? Comment se situe-t-elle dans la structure locale (du Bureau/Service Regional) ? A-t-elle un plan de travail ? Règlement intérieur ? Des réunions ? Des proces verbaux ? Nous supposons qu’il s’agit d’un groupe avec des obectifs en commun 3 Parlons du programme de leadership: Qu’avez-vous entendu du programme de leadership ? Avez-vous eu un exposé ou une restitution du directeur ? Selon vous, c’est quoi le leadership ? Pour évaluer le processus de restitution Pour avoir leur définition du leadership contre celle du directeur 4 Parlons du défi de ______________ (de référence): Il s’agit du même défi qu’on a abordé avec le DR. Note : Il faut vérifier le langage utilisé par rapport au « défi » et s’adapter. Selon vous, quel est le défi principal que vous êtes en train d’adresser ? C’est qui qui a décidé de se lancer sur cette initiative ? Parmi d’autres, ce défi a été choisi pourquoi ? et comment ? Donnez-nous le contexte aussi complet que possible de ce défi Pour sondage : Scruter: Qui sont les concernés, les intéressés ? Quels sont les besoins spécifiques à adressés ? Vous I’l s’agit de déterminer le processus de reflexion dans le choix et l’inclusion des autres These are the key questions. We are trying to understand fully if and to what extent and LCSP Evaluation Report – July 2003 59 avez travailler sur ce défi depuis quand ? Avec qui ? Quelles sont les questions clés dans ce défi ? Les questions clés ont-elles changées au cours de ce travail ? Quelles seront les conséquences possibles ? D’où viennent vos informations – de l’intérieur (au MSP), de l’extérieur? Expliquez comment vous / votre équipe ont fait pour avoir les informations appropriées par rapport à ce défi ? Que pensez-vous de la qualité et fiabilité de ces informations ? Que faites-vous pour vérifier les informations ? Qui s’engagent dans les recherches ? Focaliser: Pourquoi et comment le défi a-t-il été sélectionné ? Avec qui ? Est-ce qu’il existe un plan d’action ? Si oui, il a été dévéloppé par qui ? On peut le voir ? Est-ce les objectifs sont clairs ? raisonnables ? cohérents avec les objectifs du MSP ? Est-ce qu’on a fait une revue de la mission et la stratégie au cours de la formulation du plan d’action ? Est-ce qu’il y a eu des négotiations à faire au cours de ce processus ? Si oui, expliquez comment et avec qui ? Aligner/mobiliser: Parlons des ressources – humaines et autres – exigées pour aborder le défi. Avez-vous des ressources suffisantes pour l’atteinte des objectifs du défi. Sin non, que faites-vous pour aller à la rencontre du défi. Si oui, qu’avez-vous fait pour vous assurer les ressources nécéssaires ? Avez-vous l’appui de vos supérieurs dans cette affaire ? De vos collaborateurs dans d’autres services ou agences ? Etes-vous satisfaite avec le progrès ? Si non ou oui, pourquoi ? Inspirer: Quel est le niveau de cooperation ou collaboration de vos supérieurs et d’autres collègues par rapport à ce défi ? Expliquez en détail. Fournissent-ils des conseils, suggestions, informations ? Utilisez-vous les commentaires et suggestions d’autres collaborateurs ? Comment ? Donnez des exemples. Sont-ils capable de libérer le temps nécéssaire à cette collaboration ? how the leadership functions have been applied in meeting real challenges. Refer to the challenges listed in Annexe 4 of the Third Module Report. Are they different from what the director used before? If so, in what way? We want to know whether they are using networks and external sources for their information? LCSP Evaluation Report – July 2003 60 Comment assurez-vous une amélioration continue dans ce domaine (du défi) ? A travers quels dispositifs ? Y a-t-il d’autres défis sur lesquels vous travaillez à l’heure actuelle ? Si oui, lesquels ? Quel sera votre prochain défi prioritaire ? Pensez-vous utiliser la même approche pour l’aborder ? Que ferais-vous de différent ? Commentaires généraux / en resumé Selon vous, quelles sont les perspectives futures du programme de leadership au niveau de: - votre bureau ? - de la région ? - du plan national ? Avez-vous des récommandations pour : - le suivi de ce programme? - stratégie future de formation? pour vous même? les autres ? Défis (suite) – questions pour le “leadership booklet survey”: s’assurer qu’il y ait les réponses à ces questions Sur le plan plus global, parlez-nous de votre défi le plus important que vous avez du aborder dans les dernières annèes. Nous nous intéressons aux défis exigeant un travail en équipe ou en collaboration avec d’autres. Dans ces cas, selon votre expérience, comment fait-on pour emmener les autres a) à identifier et/ou faire face à un tel défi c) à achever des résultats en surpassant ces défis Quelle compétence vous a été la plus utile en encouragant les autres de se mettre en face des défis et d’achêver des résultats ? Que ferai-vous de différent la prochaine fois ? These are the key questions. The respondent should be allowed to express him/herself fully in response. The response should be recorded as much as possible in the respondent’s own words. A tape recorder might be used for this section – with the respondent’s consent. Sur le plan personnel – observations de soi-même: Par rapport aux acquis personnels, quels ont été les choses (capacités, leçons, etc.) les plus importantes et utiles que vous avez obtenu à travers le programme de leadership? Spécifiquement, parlons des quatre fonctions de LCSP Evaluation Report – July 2003 61 leadership: scruter, focaliser, aligner/mobiliser, et inspirer: comment s’appliquent elles dans vos activités quotidiennes ? Est-ce que vous agissez de manière différente depuis le PROGRAMME DE LEADERSHIP? Si oui, comment ? Au niveau du groupe: Au niveau du staff, est-ce qu’il a eu des améliorations dans leur performance depuis le programme de leadership ? Si oui, expliquez. [Pour sondage] Scruter: est-ce que le staff s’informe activement & volontairement sur tous les aspects de leur travail ? Est-ce qu’il obtiennent les informations les plus importantes ? Est-ce qu’il prennent des iniatives pour obtenir les informations qu’il leur faut ? Est-ce qu’il vérifient les données qu’ils reçoivent ? Focaliser: Est-ce que votre staff a une bonne compréhension, collectivement, de votre mission et stratégie ? Aligner/mobiliser: Le staff, est-il bien capable de ability to work with what is available to get results Inspire: encourage staff to find innovative ways for solving problems, go that extra mile, demonstrate commitment to staff] Nous voulons connaître les perceptions du directeur sur la performance de son équipe et l’influence qu’il pense avoir eu depuis le programme de leadership. General / summary comments Apply climate study individually to the Director Voir les instructions pour l’étude de clima LCSP Evaluation Report – July 2003 62 APPENDIX D Management Sciences for Health Développement du Management et du Leadership Evaluation du Climat des Groupes de Travail CARACTERE CONFIDENTIEL Vos réponses sont confidentielles. En aucun cas, vos classifications individuelles ne seront communiquées à d’autres dans l’organisation sans votre permission. Nom du Groupe de Travail : ______________________________________ Nom du Leader: __________________________________________ Prière de choisir l’option appropriée en cochant ci-après : † Je suis le leader du groupe de travail † Je suis un membre du groupe de travail LCSP Evaluation Report – July 2003 63 But L’enquête vise à évaluer le climat au sein d’un groupe de travail. Qu’est un « groupe de travail » ? Le groupe de travail comprend des personnes qui travaillent ensemble régulièrement pour arriver à des résultats donnés. Il peut s’agir d’une relation structurée telle qu’un service ou un centre ou alors, le groupe de travail peut être une équipe temporaire réunie pour un rôle ou une tâche spécifique. Qu’est un « climat de groupe de travail » ? Le climat est une mesure qui indique l’engagement et le dévouement des employés en vue de produire des résultats. • Le climat est mesuré à l’aune des perceptions des employés. • Les climats positifs sont caractérisés par la clarté, l’adaptabilité et l’engagement. • Les climats négatifs sont caractérisés par la confusion, la rigidité et le manque de confiance. • Le climat a un impact sur la productivité. • Quand le climat est positif, les gens vont au-delà de ce qu’on attend d’eux pour produire des résultats. • Le climat est influencé directement par les pratiques du leader. Caractère confidentiel • Vos réponses sont confidentielles. • En aucun cas, vos classifications individuelles ne seront communiquées à d’autres dans l’organisation sans votre permission. • Un rapport de feed-back ne sera réalisé que s’il existe au minimum trois ensembles de données des membres de l’équipe. Exactitude • Soyez aussi franc et honnête que possible en classant les éléments de cette enquête. Mettez ce que vous pensez. • Cette information ne servira que si vous faites une description authentique du climat dans votre groupe de travail. LCSP Evaluation Report – July 2003 64 Instructions 1. Marquez le nom de votre groupe de travail et également le nom de votre responsable sur la page de couverture. 2. Indiquez sur la page de couverture, si vous êtes le leader/responsable du groupe de travail ou un membre du groupe de travail en plaçant une coche (9) dans l’encadré. 3. Remplissez l’enquête et ensuite, mettez-la dans l’enveloppe de collection. LCSP Evaluation Report – July 2003 65 Comment est-ce que je classe les éléments ? Prière de lire chaque élément et de noter deux fois chaque élément. A gauche, notez (classez) l’Importance de l’élément A droite, décidez comment les choses se passent pour votre groupe de travail. réellement à l’heure actuelle. Prière d’utiliser cette échelle : 1 = Pas du tout 2 = Dans une petite mesure 3 = Dans une mesure modérée 4 = Dans une grande mesure 5 = Dans une très grande mesure Une fois que vous avez fait votre choix, indiquez votre sélection en marquant le nombre approprié dans la colonne, tel qu’indiqué ci-après. Importance – est-ce que c’est important ? Prière de noter chaque élément sur une échelle allant de 1 à 5 où : 1 = Pas du tout 2 = Dans une petite mesure 3 = Dans une mesure modérée 4 = Dans une grande mesure 5 = Dans une très grande mesure Evaluation du Groupe de Travail Réel– comment les choses sont￾elles actuellement ? Prière de noter chaque élément sur une échelle allant de 1 à 5 où : 1 = Pas du tout 2 = Dans une petite mesure 3 = Dans une mesure modérée 4 = Dans une grande mesure 5 = Dans une très grande mesure 4 1. C’est un exemple. 2 * Le groupe de travail comprend des personnes qui travaillent ensemble régulièrement pour arriver à des résultats donnés. Evaluation du Groupe de Travail LCSP Evaluation Report – July 2003 66 Prière de lire chaque élément et ensuite de remplir les deux côtés du tableau ci-dessous. Importance – est-ce que c’est important ? Prière de noter chaque élément sur une échelle allant de 1 à 5 où : 1 = Pas du tout 2 = Dans une petite mesure 3 = Dans une mesure modérée 4 = Dans une grande mesure 5 = Dans une très grande mesure Evaluation du Groupe de Travail Réel–comment les choses sont-elles actuellement ? Prière de noter chaque élément sur une échelle allant de 1 à 5 où : 1 = Pas du tout 2 = Dans une petite mesure 3 = Dans une mesure modérée 4 = Dans une grande mesure 5 = Dans une très grande mesure 1. Nous sommes reconnus pour nos contributions individuelles 2. Nous sentons engager dans la même direction. 3. Notre groupe de travail a des objectifs clairement définis. 4. Nous comprenons comment notre travail contribue à la réalisation de nos objectifs globaux du Ministère de la Santé. 5. Nous disposons des ressources dont nous avons besoin pour faire un bon travail 6. Nous développons nos compétences et connaissances de manière continue pour améliorer nos performances. 7. Nous sommes encouragés à explorer et expérimenter des nouvelles idées. 8. Nous avons un plan qui guide nos activités 9. Nous comprenons nos capacités mutuelles dans la mise en œuvre de nos activités. 10. Nous savons clairement ce que l’on attend de nous dans notre travail. 11. Nous pouvons décrire les besoins de nos clients sur la base des données objectives. 12. Nous participons aux décisions qui influencent le groupe de travail. 13. Nous sommes fiers de notre travail LCSP Evaluation Report – July 2003 67 Importance – est-ce que c’est important ? Prière de noter chaque élément sur une échelle allant de 1 à 5 où : 1 = Pas du tout 2 = Dans une petite mesure 3 = Dans une mesure modérée 4 = Dans une grande mesure Evaluation du Groupe de Travail Réel–comment les choses sont-elles actuellement ? Prière de noter chaque élément sur une échelle allant de 1 à 5 où : 1 = Pas du tout 2 = Dans une petite mesure 3 = Dans une mesure modérée 4 = Dans une grande mesure 5 = Dans une très grande mesure 5 = Dans une très grande mesure 14. Nous savons nous adapter aisément à de nouvelles circonstances • Le groupe de travail comprend des personnes qui travaillent ensemble régulièrement pour arriver à des résultats donnés. LCSP Evaluation Report – July 2003 68 APPENDIX E An overview of a 6-month leadership development program (LDP) for senior health managers in Guinea Considerations at the outset: The following challenges for the LDP relate to the traditional role of leaders in the Guinean setting and shaped the structure and content of the program: • The program participants (some central level and some regional level top managers) deal on a daily basis with the expectations of their followers regarding the behavior of a leader. At play are many culturally ingrained do’s and don’ts and the pressure is on the leader to conform with those. This will cramp his or her style to try out new things or behave differently, especially those at very senior positions. • The program has to accommodate the “translation” of imported notions and words about managing and leading into practices that can actually be adopted by staff. The particular challenge is finding a path between two extremes: gutting the new concepts from their power by Africanizing them to the point that all stays the same, on the one hand, and imposing a set of inflexible requirements that alienate staff because they break so abruptly with deeply entrenched ways of behaving, on the other hand. • All participants are enmeshed in a larger web of relationships, practices, traditions, some of which are good, while others are highly dysfunctional and problematic. The challenge is to help the participants find points of leverage that are under their control, so that they can engage in the right kind of advocacy, and pick the right battles. In order to do this well they will need support from others as well as self-assurance that what they are engaged in is right (self-confidence). • Questioning one’s own behavior and motives, and humility is not commonly portrayed among public figures, and although people talk about it as an important leadership asset, the actual behavior is not so commonly seen. In an environment of low trust, publicly examining one’s own style and need for control, and how that affects one’s ability to delegate to others and to trust others to do a job well is risky. • Under decentralization each level can expect some gains and some losses in the execution of its functions. The challenge is to help people articulate what is lost and gained and what comes in place, and helping self and others to re-define their roles. This may entail acknowledging one’s own competence in executing these new roles. This also includes managing participation and the consequences of inclusion (how to LCSP Evaluation Report – July 2003 69 maintain some degree of order and adhere to timelines when the voices of many people are heard) and exclusion (i.e. sabotage). • There is a significant gap between the stated principles of so-called modern management and the way things get done informally. This gap is not usually discussable in a public forum. Participants, like any other manager, experience the pressures of expectations and concomitant pressure from friends and relatives to act on their behalf, and have to frequently activate some form of ethical self-talk in order to deal effectively with frequent attempts by others to influence their decision-making. • Decentralization has shifted the competition for resources to the district, where a whole new set of political realities needs to be taken into account, and new strategies are needed to get one’s fair share. Political skills are needed to complement technical expertise, knowledge and professional status. With these challenges in mind a 4-part LDP was developed to strengthen leadership skills by stretching (confronting, questioning), giving feedback (personal feedback, observation, journaling) and providing support (coaching, visits, reference documents, personal support network, to produce the following outcomes: At the end of the program… • participants can talk with authority about their district or region, its health situation, its trends, needs, tendencies, the needs as expressed by key stakeholders; they can also (articulate) the challenges they face (especially insofar as decentralization is concerned) and know what are the assets that are available to address the challenges and move toward realization of the vision; participants know who are the key stakeholders in their region/district, know how to identify key stakeholders, and can articulate the needs and priorities of each of those groups (SCAN); • participants have/or can articulate a shared vision for better health for their region or district ; they have a strategy* for helping each organization within the district/region to define its mission/strategy/priorities and their relationship to the realization of the shared vision (*strategy should be clearly defined (in form of a document) but could be at various stages of implementation (each organization’s work in this area could be recorded in some form and available for sharing); when each organization has gone through this process, district/regional leaders should lead an effort to harmonize the various organizational efforts (FOCUS); • participants have or can develop strategies for aligning/mobilizing the key stakeholders; they have led effort to coordinate district/regional goals with LCSP Evaluation Report – July 2003 70 organizational goals and health personnel goals and to convert these goals into shared tasks (the results of this effort should be in the form of a document or series of documents); organizations throughout the district/region have plans that are aligned to support the overall district strategy; they also know how to mobilize the identified assets; have increased confidence in their ability to negotiate deals/partnerships for better health; they know how to turn a group of workers into a cohesive team and have proven their ability in their workplace and beyond (in the larger district/region) (ALIGN/MOBILIZE); • participants are attracting support from above and followers from behind/below because they are believable, congruent in their actions; organizations throughout the district or region are committed to the process of collaborative planning and action (INSPIRE). Challenges defined at the beginning of the Guinea leadership program The following is a summary of the challenges that were identified by the health system’s top managers at the beginning of the program: ƒ How do you reconcile the interest of the individuals with those of the organization? ƒ How do you do truly integrated and effective supervision (quality, job descriptions, motivation)? ƒ How do you deal with the lack of humanism in provider patient interactions (ethics, respect, etc.)? ƒ How do you increase the professional qualifications of staff (competence)? ƒ How do you improve the motivation of staff (including career plans, job security, benefits, etc) in a context of total absence of social security? ƒ How to you secure the availability and accessibility and quality of essential drugs for the population ƒ How do you improve health sector financing? ƒ How do you increase utilization of services? ƒ How do you go from community manipulation to community accountability? ƒ How do you get health and community teams to behave more responsibly with respect to their health? ƒ How do you make the various health facilities more operational if they were constructed without reference to the region’s health map? ƒ How do you reconcile the actual cost of essential drugs, the posted prices and the poverty of the population? ƒ How to assure that existing facilities are equipped and maintained? ƒ How to improve the referral system up and down the health pyramid? ƒ How to ensure complementarities of private and public sector health services? LCSP Evaluation Report – July 2003 71 ƒ How to bring about true communication between the various levels of the health system and improve the flow of information within and with the outside world (including between private and public sector) in order to implement concerted action? ƒ How do you ensure true decentralization of services with health districts fully operational? ƒ How do you maintain control of health services in a decentralized setting? ƒ How can you bring about true open-mindedness among health officials? ƒ How do you get to develop research activities? ƒ How do you fight against illegal medical practices and the sale of non￾approved or unchecked pharmaceuticals? ƒ How do you protect those who exercise the health professions? This list was reviewed again at the end of the program to ascertain what the participants had already been doing about them and what additional activities or strategies they intended to apply to the challenges. …and what has been accomplished (after 6 months) With respect to SCANNING, participants (and their coaches) reported that they had: • Started situational analysis at the prefectoral level (visited the field, read documents …) • Conducted a situation analysis • Evaluated epidemiological situation • Updated data • Organized consultative fora (CTRS, CTPS) With respect to FOCUSING, participants (and their coaches) reported that they had: • Reproduced and disseminated referral guidelines • Increased staff awareness about referral procedures • Lobbied with decision makers for increases in the health budget • Identified priority problems • Developed selection criteria for workers • Facilitated meetings • Made staff more aware of the importance of quality services With respect to ALIGNING and MOBILIZING, participants (and their coaches) reported that they had: • Supported putting in place community health insurance schemes (mutuelles) • Expanded the circle of people involved (PNDS) • Held coordination meetings (CTPS+CTRS) • Mobilized donors and other partners • Raised awareness at community level LCSP Evaluation Report – July 2003 72 • Lobbied for the establishment of a communication system between health centers and hospitals • Shared information at all levels, between providers and users • Shared the MOH vision with the districts and regions (support to the CTRS) • Shared the objectives of integrated supervision • Held dialogues with the community With respect to INSPIRING, participants (and their coaches) reported that they had: • Renewed management committees (COGES) • Supervised staff • Trained members of management committees (COGES) • Delegated tasks to subordinates • Invited staff to give (personal) feedback With respect to PLANNING, participants (and their coaches) reported that they had: • Organized planning workshops • Planned activities for each staff position • Developed indicators for performance • Developed supervisory tools With respect to ORGANIZING, participants (and their coaches) reported that they had: • Made staff more aware of the role of these committees • Moved personnel to other positions • Added staff • Trained management committees (at community level) • Developed a program of in-service training • Set up and trained supervision teams per level With respect to IMPLEMENTING, participants (and their coaches) reported that they had: • Promoted community health insurance schemes • Put COPE committees in place • Conducted multimedia campaigns • Increased cost recovery • Supervised services With respect to MONITORING and EVALUATING, participants (and their coaches) reported that they had: • Monitored hospital performance contracts • Set up a regional research committee • Provided feedback • Reported on trainings LCSP Evaluation Report – July 2003 73 Challenges defined at the beginning of the Guinea leadership program and intentions on what to do about these challenges Involve more people/stakeholders in planning and decision makers • Develop a strategy to mobilize all involved in essential drug program (donors, users, pharmacies, drug import companies, customs, etc.) • Develop strategies to increase ownership • Create mechanisms for consultations among the various actors ( private, public, donors, religious organizations, professional organizations) to review PHC • Elaborate supervision strategies including consultation at all levels • Develop a strategy to strengthen and expand community participation • Develop a shared vision • Continue to mobilize resources from donors Communicate with stakeholders • Formalize coordination mechanism for future interventions • Continue to hold meetings about this • Organize periodical meetings with professional groups • Organize meetings about referral practices (between regions and districts) • Coordination of resources (donors, collaborators) Raise awareness/teach • Raise awareness about health insurance schemes • Convince those who dispense prescriptions to protect the population’s health • Lobby for a review of prices in light of cost of purchase of ED • Make the population more aware • Organize the dissemination of ethical codes of conduct • Generate interest in operations research Monitor performance • Strengthen supervision and quality control • Strengthen supervision mechanisms at all levels • Evaluate staff performance • Evaluate the functioning of private facilities • Review the state of information about facilities outside the health map • Monitor performance of health managers Encourage good performance • Inspire people (public recognition plaques, study visits as a reward) • Inspire colleagues to exercise the leadership functions • Encourage people • Support professional groups • Continue and support current (good) practices Fix performance problems • Develop and monitor a maintenance plan LCSP Evaluation Report – July 2003 74 • Initiate the revision of the budget regarding maintenance costs • Redeploy personnel • Administer sanctions • Establish performance contracts with hospitals Provide professional development incentives • Continue to improve staff skills • Continue to develop leadership skills among other officials • Apply and disseminate lessons about leadership • Develop a regional maintenance and training system • Lobby for effective career planning • Put career plan into action • Lobby for the completion of the career plan policy • Negotiate redeployment to other sectors Conduct research • Conduct operations research • Determine the areas to research in teams • Develop and implement research protocols • Train research teams • Do an inventory of research conducted in the region Develop new initiatives • Create staff savings schemes (mutuelles) for covering health-related expenses • Establish a policy for indigents Push responsibility and accountability down • Lobby for the accelerated decentralization of inspection of services • Support the decentralization of basic health services package • Make the health districts more autonomous • Reinforce the responsibilities of the members of the management committees • Decentralize performance contracts with hospitals (to the region, to the district) Next steps: • Sharing addresses among participants (email, telephone and radio) to stay in touch • Follow-up visits once every four months by local trainers, and once a year by international trainers • External evaluation of the program • Annual meetings of the network of participants • Replicate the program for senior health managers who have dropped out LCSP Evaluation Report – July 2003 75 • Meeting with the Minister of Health to explore institutionalization of the program • Involvement of other donors in leadership development across the entire country LCSP Evaluation Report – July 2003 76