ASSESSMENT OF MANAGEMENT AND LEADERSHIP ACTIVITY Robert Blomberg Pauline Muhuhu Nancy J. Piet-Pelon Lizann Prosser June 2004 Submitted by: LTG Associates, Inc. Social & Scientific Systems, Inc. Submitted to: The United States Agency for International Development Under USAID Contract No. HRN–C–00–00–00007–00 This document is available in printed or online versions (POPTECH Publication Number 04–175–022). To review and/or obtain a document online, see the POPTECH web site at www.poptechproject.com. Documents are also available through the Development Experience Clearinghouse (www.dec.org). Printed copies and additional information about this and other POPTECH publications may be obtained from The Population Technical Assistance Project 1101 Vermont Avenue, NW, Suite 900 Washington, DC 20005 Telephone: (202) 898-9040 Fax: (202) 898-9057 admin@poptechproject.com Assessment of Management and Leadership Activity was made possible through support provided by the United States Agency for International Development (USAID) under the terms of Contract Number HRN–C–00–00– 00007–00, POPTECH Assignment Number 2004–175. The opinions expressed herein are those of the authors and do not necessarily reflect the views of USAID. ACRONYMS AND ABBREVIATIONS APROFAM La Asociación Pro-Bienestar de la Familia de Guatemala (Family Planning Association of Guatemala) ASHONPLAFA Asociación Hondureña de Planificación de la Familia (Honduras Family Planning Association) BPP Business Planning Program CA Cooperating agency CTO Cognizant technical officer FLEP Family Life Education Project (Uganda) FPMD Family Planning Management Development program FPMT Family Planning Management Training program GH/PRH/SDI Bureau for Global Health, Office of Population and Reproductive Health, Service Delivery Improvement Division Global Fund Global Fund To Fight AIDS, Tuberculosis and Malaria HCD Human capacity development HIV/AIDS Human immunodeficiency virus/acquired immune deficiency syndrome IMCI Integrated management of childhood illness IPPF International Planned Parenthood Federation IR Intermediate Result LAC/HSR Bureau for Latin America and the Caribbean Health Sector Reform Initiative M&L Management & Leadership Program MOH Ministry of Health MOST Management and Organizational Sustainability Tool MSH Management Sciences for Health NGO Nongovernmental organization NicaSalud A consortium of NGOs in Nicaragua PHN Population, health, and nutrition PROCOSI Programa de Coordinación en Salud Integral (The Collaborative Program for Integrated Health) PROFAMILIA Asociación Pro-Bienestar de la Familia Nicaragüense (Family Planning Association of Nicaragua PROSPEK MSH’s performance assessment and improvement process PVO Private voluntary organization PY Program year QAP Quality Assurance Program REACH Rural Expansion of Afghanistan’s Community-based Healthcare REDSO Regional Economic Development Services Office RFE Rapid Funding Envelope for HIV/AIDS SD Strategic Direction SILAIS Local System for Integrated Attention to Health (Sistema Local de Asención Integral a la Salud) TACAIDS Tanzania Commission for AIDS TB Tuberculosis TCNetwork Technical Cooperation Network USAID U.S. Agency for International Development VLDP Virtual Leadership Development Program CONTENTS Page Executive Summary....................................................................................................................... i I. Introduction..............................................................................................................................1 Methodology..............................................................................................................................1 II. Program Description ...............................................................................................................3 Use of Core Funds .....................................................................................................................5 Use of Field Support Funds .......................................................................................................7 Afghanistan..........................................................................................................................7 Angola..................................................................................................................................7 Bolivia..................................................................................................................................8 Brazil....................................................................................................................................8 Ghana ...................................................................................................................................8 Guatemala ............................................................................................................................8 Honduras..............................................................................................................................8 Indonesia..............................................................................................................................8 Malawi .................................................................................................................................8 Mozambique ........................................................................................................................9 Nicaragua .............................................................................................................................9 Nigeria..................................................................................................................................9 Peru ......................................................................................................................................9 Tanzania...............................................................................................................................9 Uganda .................................................................................................................................9 III. Summary Findings, Conclusions, and Recommendations.................................................10 Results and Accomplishments .................................................................................................10 Contribution of Core Funds to the Overall Success of the M&L Program .......................10 Contribution of Field Support Funds to the Overall Success of the M&L Program .........10 Systems and Management........................................................................................................11 Personnel and Staffing .......................................................................................................12 Relationships With USAID................................................................................................12 Monitoring and Evaluation Systems..................................................................................13 Management by USAID ....................................................................................................14 Lessons Learned.......................................................................................................................14 Multiplier Effect.................................................................................................................15 Instruments and Resources ................................................................................................16 Monitoring and Evaluation ................................................................................................16 Business Planning Program ...............................................................................................17 Future Strategic Directions ......................................................................................................18 Continuation of M&L ........................................................................................................18 Practicality of M&L...........................................................................................................18 Improvement of Marketing of M&L Concepts..................................................................19 Monitoring and Evaluation ................................................................................................19 Thought Leadership ...........................................................................................................19 Instruments and Resources ................................................................................................19 APPENDICES A. Scope of Work B. Schedule of Activities C. Persons Contacted D. Country Reports E. Summary of Mission Responses F. M&L Funding by Fiscal Year G. Guide to M&L Programs and Resources H. M&L Self-Assessment I. References TABLE The Evolving Approach to Management and Leadership ...............................................................4 FIGURE Leading & Managing Results Model...............................................................................................3 EXECUTIVE SUMMARY The Management & Leadership Program (M&L), which is implemented by Management Sciences for Health (MSH), is a five-year cooperative agreement in a 10–year Results Package. It builds on 15 years of MSH’s prior experience with Family Planning Management Training and Family Planning Management Development I and II. M&L was awarded on September 29, 2000, and is now in its fourth year. It is scheduled to end on September 29, 2005. The activity was designed to reinforce the U.S. Agency for International Development’s (USAID) global health programming by building capacity in organizations and teams of workers to manage and lead effective, sustainable health care systems. M&L accomplishes this goal by working with public and private organizations that provide primary and reproductive health care, family planning, AIDS–related services, and treatment for infectious diseases. M&L also provides technical assistance to national ministries of health and decentralized health services at various levels of government. M&L has been active in 18 countries and with the Bureau for Africa, the Bureau for Latin America and the Caribbean Health Sector Reform Initiative (LAC/HSR), and the Regional Economic Development Services Office (REDSO). The four-person assessment team conducted a collaborative inquiry to assess the performance of M&L and to provide guidance to USAID regarding the design of future activities. The assessment covered four basic areas: ƒ results and accomplishments, ƒ systems and management, ƒ lessons learned, and ƒ future strategic directions. The team conducted interviews at USAID/Washington, was briefed for 2 days by M&L in Boston, and went on field trips to Indonesia, Mozambique, Tanzania, and Nicaragua. These countries represent 48 percent of all the field support M&L receives in the countries in which it works. In each of these countries, M&L has in-country staff conducting the work. The assessment team found that M&L followed a logical sequence in working toward its goals. In the first year of the project, M&L developed a framework and delivery methods for its approach to developing sustainable organizations, and tested, applied, and refined those approaches in the field. In the second year, M&L developed and applied the M&L approach to capacity building and improving systems. Once this work had been completed, in the following two years, the team shifted priorities to capturing and improving the approaches most effective in enabling organizations to achieve results and creating means to replicate these approaches through a network of local technical assistance partners. RESULTS AND ACCOMPLISHMENTS This is a substantive program with many excellent results and accomplishments, from both core funding ($22,279,000) and field support ($22,646,000). With core funding, M&L has created instruments and resources (often called tools) used in the field and has refined and disseminated them. These resources were developed first in the field and then brought to the center for global application. Under M&L, the Virtual Leadership i Development Program and the Business Planning Program tools were developed. Both were created in Latin American countries and first applied there. Other resources have been developed for country-specific application, such as PROSPEK in Indonesia. Core funding also supported initial visits to countries for program startup before field support was available. Missions appreciated this, including Indonesia and Nicaragua, since it allowed a quick startup of needed activities. Core funds have also been used for monitoring and evaluation purposes, although the assessment team identified monitoring and evaluation as an area that needs focused attention from M&L during the remainder of the project. Field support increased rapidly as M&L progressed. From only $1,230,000 in federal fiscal year 2000, it increased to a total of $13,647,484 in 2004. This indicates that Missions are investing in M&L and believe that M&L has an important role in countries where decentralization and health sector reform are being implemented. USAID Missions have also made broad use of M&L. In several instances, M&L has been used as a contract vehicle of convenience, and Missions have operated programs through M&L that are not directly related to the core agenda. However, the core investments do not always seem to be targeted to address field needs. In some cases there has been insufficient complementarity between the two funding sources. Both operate well and do good work but the synergies are not as clear as they should be. The exception is Nicaragua, where core investments worked closely to launch the project before field funds were used for the majority of funding. SYSTEMS AND MANAGEMENT M&L is well managed. It received praise in the field for quality, timeliness, and relevance of reports, project deliverables, and invoicing. This perception is based on the high quality of the individuals working for M&L in the field. The chiefs of party in the field were technically competent and had the respect of their colleagues in the ministries of health as well as the Missions. There is a positive relationship between the in-country staff and those in M&L who provide technical assistance. Relations with USAID/Washington, the cognizant technical officer (CTO), and the senior technical advisor are very positive. Both parties work to make this relationship effective. The CTO and senior technical advisor have a high level of knowledge of M&L activities and issues. Relations between Missions and in-country M&L staff are generally positive. However, in some cases, there is a division between Missions and senior management of M&L in Boston. In these cases, senior management is not sufficiently accessible to the Missions. Although there are many reasons for this, including travel bans, M&L senior management needs to assertively find ways to be responsive to Missions. Monitoring and evaluation is a weakness identified by the team. M&L is not able to succinctly answer questions about what works, when and where it works, why it works, and how USAID and M&L should set priorities for efforts in the future. Monitoring and evaluation has focused on monitoring all field interventions, reporting progress and results in semiannual reports, annual Results Reviews prepared for USAID, and annual management reviews with its CTO. M&L does conduct some indepth evaluation of key, strategic field-based programs to capture both results and lessons learned. Eighteen evaluations1 have been completed or are in process. It is ii 1 Indepth evaluations conducted or in process to date: Application of the Management and Organizational Sustainability Tool (MOST) in four Latin American and Caribbean countries (2002); Family Life Education Project clear from the team’s visits in the field that M&L’s work is excellent and that it is highly appreciated. This message needs to be stated more clearly in well-documented information. LESSONS LEARNED When engaging a project like M&L, USAID needs to accept that management and leadership are not short-term undertakings. These issues require time and commitment. A commitment to an accepted implementation period is needed. M&L is more than a collection of tools. The program has developed a collection of instruments and resources that increase the knowledge base on how to improve management and leadership in organizations. The instruments and resources (tools) produced serve an important role. Those made during the Family Planning Management Development (FPMD) program, such as the Management and Organizational Sustainability Tool (MOST) and The Manager, are very popular in programs today. The team anticipates that electronic tools being produced today, in spite of connectivity problems, will be the popular tools of the future. The investment in these has been worthwhile from that viewpoint. Working with teams rather than training individuals is a strength of M&L’s work. The M&L approach focuses on problems and challenges that local teams have identified and helps them to solve those problems. M&L has developed a rational multiplier model, moving training out from regional centers to local service delivery points, using a cadre of locally trained facilitators. The team saw strong evidence of this in Nicaragua and preliminary evidence in Mozambique and Indonesia. As noted in the section, Systems and Management, monitoring and evaluation needs strengthening. First, this would help M&L participants recognize the fundamental importance of monitoring and evaluation. Second, M&L would be able to better understand what works in terms of interventions and tools. Third, monitoring and evaluation needs to contribute to a broader knowledge base about effective methods in efforts to foster increased M&L capacity. To reduce a perception among some USAID staff that M&L is a “collection of tools,” M&L needs to document the use of some of these resources for potential replication in other countries (i.e., PROSPEK in Indonesia and the Rapid Funding Envelope in Tanzania). The Business Planning Program (BPP) is another process to document, although it appears less applicable for organizations that need basic organizationwide business planning than for those seeking a new revenue source. iii (FLEP)/Uganda (2002); Lidernet/Brazil (2003); APROFAM/Guatemala (2003); Guinea Leadership Development Program (2003); Egypt Leadership Development Program (2003); Business Planning Program (BPP)/PROCOSI (2003); BPP/Ghana Social Marketing Foundation (2003); Peru/Manuela Ramos (2003); Bolivia/Chemonics (2003); Virtual Leadership Development Program (VLDP) I (2003); Health Managers Toolkit (2003); Guinea Leadership Development Program Follow-Up Inquiry (2004); Egypt Leadership Development Program Follow Up (evaluation conducted April 2004, report in process); Nicaragua Leadership Development Program (evaluation conducted March 2004, report in process); M&L–PRIME joint project in Armenia (midterm evaluation conducted May 2004); Follow Up of VLDP I and II cohorts (in process); and MSH Publications, including three issues of The Manager, funded by M&L (May 2004). FUTURE STRATEGIC DIRECTIONS This is an effective program performing important work; the concepts it espouses are critical for the future. Missions that have used M&L are aware of the added value of a focus on leadership at all levels. M&L should be continued through the second five-year period of the Results Package because both management and leadership are critical components of institutional capacity building. The assessment team considered the feasibility of incorporating M&L into other cooperative agreements but believed that this would diffuse its effect and benefit. There is a risk with the new Human Capacity Development Leader with Associates cooperative agreement that Missions will choose either HCD or M&L (but not both) to keep the number of cost centers at a manageable limit. This could mean that M&L will receive less field support. But this is a risk that USAID should take to be sure that the focus on both management and leadership is not lost. This focus is particularly important because many countries are involved in the difficult tasks of decentralization and health sector reform while others are rapidly losing staff to HIV/AIDS or other factors. The underlying principle of M&L of developing leaders at all levels is critical in these environments. M&L needs to improve the selling of its concepts. It does not have a simple message describing its work. Missions need to understand why M&L is important for their programs and how they will benefit if they use field support for M&L. This message is long overdue, considering M&L is in its fourth year of a five-year cooperative agreement. M&L needs to be more practical and mindful of the realities of Mission timetables. Missions need to show results each year. At the same time, Missions need to be more aware that programs such as M&L need time to show results. M&L is a long-term investment and should be taken on only when Missions recognize and support that time commitment. Monitoring and evaluation needs to be given high priority. There is sufficient time between now and September 2005 for M&L to document its contribution to the field programs. If this is not done, M&L risks losing important opportunities to show its accomplishments. M&L needs to exercise thought leadership by developing new monitoring and evaluation indicators for management and leadership development. The team recommends that M&L lead a group of experts in the field of leadership to codify what it is that makes effective leaders at all levels. Tools that are timeless, such as MOST and The Manager, should continue to be available globally as they are now. The emphasis on electronic tools should not overshadow the basics although the point is well taken that this is a time of experimentation for tools of the future (which electronic tools will be). The team recommends that no new electronic tools be created until stronger evaluation of existing electronic tools is available. iv I. INTRODUCTION The Management & Leadership Program (M&L) is implemented by Management Sciences for Health (MSH) under a cooperative agreement, with a budget of $53,426,000. The program aims to improve management and leadership performance in the health sector to meet the needs of health care consumers. It contributes to the Bureau for Global Health’s Management and Leadership Results Package. The program builds on previous work of the Family Planning Management Training (FPMT) program and Family Planning Management Development program (FPMD) I and II. The lessons about effective development have been learned from several decades of work in these two programs and other development activities. The M&L program was designed to add to these lessons. MSH is based in Boston and is a private, nonprofit organization with extensive experience worldwide in technical assistance, training, systems development, and applied research. Over the years, MSH and other organizations have developed many simple, practical instruments and resources in management and training. Under M&L, MSH has taken a holistic approach aimed at building sustainable organizations with an enduring culture of performance. They have integrated concepts and content from prior experience, and now focus on a systems approach, working on the systems that need to be in place as well as the training that people need to become “managers who lead.” This program provides an opportunity for validating the usefulness and impact of the instruments and resources that exist, in identifying gaps that still need to be filled with good resources and other technical applications, and evaluating how the technical assistance can best be packaged for ease of use in different environments. This project therefore builds on the practical work of MSH’s FPMT and FPMD I and II programs and continues to focus on improving management skills and systems development of leadership and capacity building. M&L has been implemented in 18 countries worldwide, building on the firm foundation of MSH’s 19 years of global experience in strengthening public health management and leadership. The M&L central and country teams have built M&L through consolidation of MSH’s work under three previous management projects funded by the U. S. Agency for International Development (USAID) and innovation that take USAID’s support for management and leadership to new heights. This report presents the assessment team’s findings in terms of results and accomplishments identified, lessons learned, and the team’s suggestions for future directions (see appendix A for the scope of work). METHODOLOGY A four-person core team conducted this assessment. The CTO (Nicaragua), the senior technical advisor (Indonesia), and a staff person from the Service Delivery Improvement Division, Office of Population and Reproductive Health, Bureau for Global Health (GH/PRH/SDI), who translated for the team in Mozambique, accompanied the team on country trips. In addition, a management specialist joined the Indonesia visit to address two specific questions posed by the Mission regarding M&L and future work. (Appendix B contains the schedule of activities and appendix C contains the persons contacted. Additional contacts are listed in appendix D, Country Reports, for Indonesia, Nicaragua, Mozambique, and Tanzania.) 1 Thirty-four questionnaires were sent in April 2004 to various countries as well as to the Bureau for Latin America and the Caribbean, Bureau for Africa, and the Regional Economic Development Services Office (REDSO). The 14 responses received were summarized and appear as appendix E. M&L provided its funding by fiscal year (appendix F), a guide to its programs and resources (appendix G), and a self-assessment that was conducted for this assessment in April 2004 (appendix H). Team meetings and interviews were conducted with key personnel in USAID/Washington and M&L at MSH in Boston. M&L prepared several presentations of its work over a 2–day period. Country trips were 2 weeks (Indonesia and Nicaragua) and 1 week each in Mozambique and Tanzania. The team was asked to assess the performance of M&L and provide guidance to USAID regarding the design of future activities in M&L. 2 II. PROGRAM DESCRIPTION In 2000, when USAID entered into this agreement, GH/PRH’s Strategic Objective was “to improve leadership, management, and sustainability of accessible, quality family planning and reproductive health programs.” M&L, as conceptualized by MSH, proposed the following three Intermediate Results (IRs): IR 1: Improved performance of management systems of organizations and programs IR 2: Improved performance of leaders and managers IR 3: Improved ability to anticipate and respond effectively to the changing external environment MSH has structured M&L around four Strategic Directions (SDs) that guide technical leadership in core activities and technical assistance in the field: SD 1: Developing Capacity of Individuals and Teams to Lead and Manage SD 2: Improving Management Systems SD 3: Partnering Locally for Sustainability SD 4: Capturing and Applying Knowledge According to MSH, the fundamentals of the M&L approach—building a sound foundation of management and leadership capacity and systems using a performance improvement methodology—constitute essential elements of building sustainable organizations. The Leading and Managing Results Model, initially developed in program year (PY) 1, maintains the focus on the Strategic Objective and IRs defined in the Request for Agreement. This model shows the connection of leading and managing practices to the Intermediate Results of improved work climate and management systems, to improved service delivery results, and eventually to health outcomes. The model was used in the development of M&L’s SDs and 3 shows the connections and synergy among the three IRs. It has been used in many M&L field projects to develop project-specific objectives, and it has served as a framework for the indicator development process. The table below, prepared by MSH, shows the differences between traditional approaches to the development of leadership and management and the approach taken under M&L. (A complete list of programs and resources completed or underway that constitute elements of the M&L initiative can be found in appendix G, Guide to M&L Programs and Resources.) The Evolving Approach to Management and Leadership Traditional MSH Approach Leadership Aimed at top leaders Aimed at managers at all levels Focus on individual’s leadership skill development Integrated approach to improved services, work climate, systems, and health outcomes Often reinforces the notion that leaders are “born” (Gandhi, Martin Luther King Jr.) Skills and competencies that anyone can learn (demystify leadership) Leadership transition: attention, if any, is given only to the top levels Transition preparation at all levels Often one-time, off-site workshops Process over time at clients’ workplaces Often separated from specific organizational or management challenges Connected to real organizational challenges Management Expert consultant model of assessment Participatory approach, ownership, and integration of improvements into organizational plans System-specific improvements led by outside or multiple experts Overall improvement plan with organizational outcomes; use of Leading and Managing framework Specific problems fixed in isolation Strengthening skills to address future not just current problems Priorities set by donors at times Organizational ownership of priorities and making lasting change happen Anticipate and Respond to Changing Environment Management and leadership treated separately System strengthening and individual/work group capacity building linked Work with one organization at a time and count results by organization Multiplier effect to reach multiple organizations and count program wide contributions In person technical assistance Innovative approaches using new technologies and in person technical assistance FP/RH vertical programs Integration with other vertical initiatives (HIV/AIDS, TB, malaria, child health) and multi-sectoral approaches Long time frame for demonstrating results Respond with speed, sustain with systems Monitoring and Evaluation Process indicators Indicators, including impact indicators, tailored to track client progress towards performance objectives and intended results One time or at project’s end Routine and integrated into project management Country or project specific Knowledge gained available for MSH and others; cross fertilization of applications from country to country Data required from above Data required by those who generate it Source: M&L During the first half of PY 1, the MSH focus was on developing the frameworks and delivery methods for its approach to developing sustainable organizations. It then took those approaches to the field, testing, applying, and refining them in demonstration projects. During PYs 1 and 2, 4 the priorities were more heavily focused on SDs 1 and 2—developing and applying the M&L approach to capacity building and improving systems. As M&L moved into PY 3 and had both core and field-support funded experience, the priorities shifted towards capturing and making small adjustments to the approaches most effective in enabling organizations to achieve results and creating the means to replicate these approaches through a network of local technical assistance partners. As the program matured, MSH priorities have evolved to focus increasingly on SDs 3 and 4—a natural evolution from a focus on what it does (leading and managing capacity building and systems improvement) to how it is replicated (through partners, continuous learning, and the development of a distributable body of knowledge). USE OF CORE FUNDS Core funds have been used for several purposes: ƒ creating programs to be used in the field or refining, defining the purpose, and disseminating tools which have been developed in field programs; ƒ monitoring and evaluation; and ƒ initial visits to countries to establish a field program. Two products developed during M&L use what MSH refers to as blended learning, that is, combining worksite team activities and homework with technical assistance and/or facilitation by M&L staff provided over the Internet or via e-mail. These are the Virtual Leadership Development Program (VLDP) and the Business Planning Program (BPP). The VLDP has it origins in the face-to-face Leadership Development Program for Ministry of Health municipal managers and staff undertaken in three Ministry of Health regions (Sistema Local de Asención Integral a la Salud [SILAIS]) in northern Nicaragua (i.e., Boaco, Matagalpa, and Jinotega), beginning in 2001. It is also based on the Pan American Health Organization’s request to MSH to develop a web-based leadership development program. The individuals responsible for creating and implementing that field program had central roles in the creation of the VLDP. The VLDP is an interactive, Internet-based program that helps health managers strengthen their leadership skills and competencies by working on challenges facing them in their own organizations. The program consists of an introductory module, five leadership development modules, and a final module for reflection and evaluation. Each team participates using the VLDP Internet site, a CD–ROM, and a print workbook. After completing the program, participants can become members of a virtual network, Leadernet, through which they receive coaching and support to advance further with their challenges. During the VLDP, the teams identify an actual challenge facing their organization. After careful analysis and discussion of the challenge, all structured by the facilitators and supported by materials in the modules, teams develop an action plan to work on the challenge. Experienced facilitators work closely together, rotating responsibility for facilitation throughout the 12–week course. The facilitators post daily announcements, drawing attention to a particular topic in the reading assigned, commenting on participants’ discussions, or raising provocative questions. They also review and respond to each team’s homework and provide feedback on each team’s progress in addressing its organizational challenge. 5 One member of the team is accountable for posting the homework of the group on the VLDP web site. All team members are invited to post their thoughts, issues, insights, and concerns on the café of the VLDP web site. Each day, the online M&L facilitator summarizes a few of the postings and develops a thoughtful commentary about issues that are emerging from the various teams. He/she sends this message to all members of all teams as an e-mail. This is intended to increase engagement in the VLDP process and to keep people curious about what is happening. Provision of the CD–ROM version of VLDP along with the workbooks increases the potential utility of the VLDP as it does not require participants to have access to the Internet. Whether it functions well and achieves similar outcomes without Internet participation and online facilitation has not been documented. As of December 2003, M&L reported that approximately 290 people had participated in the Spanish version of the VLDP course, representing 34 teams from 23 different organizations (an average of about 8 participants per team). Since January 2004, three more VLDP courses have been launched: Africa has 11 teams, the Caribbean has 15 teams, and Colombia, Ecuador, and Mexico, three countries that have graduated from USAID population and reproductive health assistance, have 6 teams. An indepth evaluation of VLDP I was conducted and a follow up on Latin American and Caribbean VLDP I, II, and III cohorts is in process. The BPP, developed by MSH and the Programa de Coordinación en Salud Integral (PROCOSI) in Bolivia, currently is being replicated in Nicaragua with teams from five nongovernmental organizations (NGOs), including NicaSalud, a consortium of NGOs that brings together Nicaraguan NGOs and private voluntary organizations (PVOs) working in the health sector, and four of its member organizations. The purpose of the BPP is to help NGOs learn how to identify a breakthrough idea and then to undertake a business planning process that results in the creation of a fundable service or product (assuming that the market study and feasibility analysis indicate that the project has viability). This is the first replication of the BPP by PROCOSI. The BPP variation on blended learning consists of an opening 5−day, face-to-face workshop conducted by PROCOSI instructors and M&L staff, followed by six homework modules provided on a CD−ROM. The six modules include ƒ a self-study of the agency’s strategic position within its sector and community, ƒ ƒ ƒ ƒ ƒ identification of a breakthrough idea that pinpoints what the agency needs to do to be competitive, a market study requiring interviews with members of the target market for the proposed product or service to see if a market exists, creation of a business plan defining individual roles, responsibilities, a timeframe, and the cost, an analysis of the financial return of the project, and an analysis of the social return of the project. The appropriateness of the thrust of the BPP—finding a breakthrough idea for a new business activity—was questioned by some program participants. They felt that for their agencies, learning business programming about their current operations might be a more useful place to 6 begin. They reported that having to think in terms of a new business activity seemed inappropriate for them at the time. They reported nonetheless that they had learned a more sophisticated way of approaching all of their agency planning work because of their participation in the BPP.2 The Technical Cooperation Network (TCNetwork) is a new initiative of M&L, which was launched in June 2003. It is designed to be a global community of technical assistance providers committed to mutual support, accountability, and excellence. The TCNetwork will address some of the barriers to linking local capacity with donors, government agencies, and other clients. Currently, there are nine members of the network from various countries. There are more than 50 applications for membership being vetted now. The TCNetwork is facilitated by M&L, although members believe that they have the potential to attract other donor support when M&L ends in September 2005. Some of the members of the TCNetwork have obtained work through their membership even though it is still at the beginning stages. A web site (tcnetwork.net) is already in place and allows its members to share information. It also serves as a directory for those organizations that are seeking consultants. The Global Exchange for Reproductive Health is an attempt by M&L, at USAID/Washington’s behest, to continue to encourage the graduated countries. Colombia, Mexico, Ecuador, Morocco, and Turkey are included. The purpose is to enable the countries to access information from the global community and to encourage them to continue their excellent work in population, family planning, and reproductive health. It is also an opportunity to keep these countries engaged with USAID and its work. USE OF FIELD SUPPORT FUNDS M&L has worked in 18 countries and three regions/bureaus with field support funds. Field support increased from $1,230,000 in FY 2000 to $13,647,484 in FY 2003. This is a summary of the work in 15 of the countries.3 Afghanistan Funding through M&L allowed for the rapid startup as USAID initiated a response to the health needs of this country emerging from war. M&L’s work preceded the development and award of the current bilateral project, Rural Expansion of Afghanistan’s Community-based Healthcare (REACH). Funding through M&L ceased in 2004. Angola This program was initiated in PY 4 as the Mission formulated a strategy to respond to the emerging AIDS crisis. M&L supported USAID/Angola and the Angolan government in ƒ carrying out a management and leadership needs assessment of the Angolan National HIV/AIDS Program, 2 For more information regarding both VLDP and BPP, see appendix D for the Nicaraguan country report. 7 3 This information is taken in part from Management and Leadership Program Self-Assessment, April 23, 2004. The self-assessment is included as appendix H. ƒ providing technical assistance to the World Bank in developing the institutional arrangements and a monitoring and evaluation framework for the Angola Multi￾Country AIDS Program, and ƒ preparing a summary of the National HIV/AIDS strategic plan. Bolivia MSH has worked in Bolivia since FPMD I. M&L works with two clients: PROSALUD and COMBASE. Brazil M&L implemented a country-based leadership development program using electronic technologies to deliver the course and to provide follow-up support to alumni (Lidernet). From the Lidernet work, Leadernet, a global adaptation, was developed. Ghana Initially using core funds, M&L conducted two workshops demonstrating the Leading and Managing Framework and the Human Resource Management Assessment Tool for the staff of the Human Resources Division, Ghana Health Services. The Mission then requested a comprehensive assessment of the human resources issues facing Ghana Health Services. The Mission approved the resulting recommendations but no follow-up work was conducted since neither the Mission nor the Human Resources Division responded. Guatemala MSH and now M&L have worked with La Asociación Pro-Bienestar de la Familia de Guatemala (APROFAM) since the 1990s. Honduras In program years 2 and 3, M&L implemented a technical assistance program with the Asociación Hondureña de Planificación de la Familia (ASHONPLAFA), the International Planned Parenthood Federation (IPPF) affiliate. The performance improvement approach was introduced and applied. Indonesia M&L began with core funds to initiate activities. This is a large and complex program with a well-developed field presence working with the Ministry of Health (MOH) and the National Family Planning Coordinating Board (see appendix D for the country report). Malawi M&L recently placed a resident technical advisor with the MOH in the HIV/AIDS unit. This is an example of strengthening the management and coordination capacity of an MOH to 8 implement a complex, non−USAID, Global Fund To Fight AIDS, Tuberculosis and Malaria (Global Fund) program. Mozambique M&L began in Mozambique with FPMD II. Based on this extensive work, M&L is developing a program to strengthen MOH leadership and management capacities at all levels of the health sector (see appendix D for the country report). Nicaragua MSH has been involved in-country for many years. It is currently providing assistance to strengthen leadership development at the Ministry of Health, the Ministry of the Family, and the Nicaraguan Social Security Institute as well as to the Asociación Pro-Bienestar de la Familia Nicaragüense (PROFAMILIA) and other NGOs through NicaSalud. It is also active in health sector reform at the MOH and in pilot efforts to develop social capital in rural communities (see appendix D for the country report). Nigeria M&L works with the National Primary Health Care Development Agency, supporting its strategic plan and assisting in the reengineering of its organizational structure and financial, human resource management, and programmatic service statistics systems. Peru This was a first-year activity to introduce the performance improvement approach. Tanzania Work with the Tanzania Commission for AIDS (TACAIDS) began with FPMD II. Currently, a resident advisor works with TACAIDS on capacity building and preparing proposals for funding through the Global Fund. This Rapid Funding Envelope for HIV/AIDS (RFE) mechanism is the first of its kind and is drawing considerable attention because of its success in accessing funds for the NGOs’ work (see appendix D for the country report). Uganda M&L work began in PY 3 and has grown to a funding level of approximately $5 million. The focus of this work is AIDS and is to assist with grants from the President’s AIDS Initiative, which are being disbursed by the Inter-Religious Council of Uganda. M&L also works with the Global Fund to establish its Project Management Unit in the MOH. In addition, M&L is adapting the Management and Organizational Sustainability Tool (MOST) for the National Tuberculosis and Leprosy Program to improve the national laboratory program and for the Joint Clinical Research Centre. In addition to these countries, M&L works with the Latin America Health Sector Reform Initiative. It was also working in Turkey, but this program closed when the Mission closed. 9 III. SUMMARY FINDINGS, CONCLUSIONS, AND RECOMMENDATIONS RESULTS AND ACCOMPLISHMENTS Contribution of Core Funds to the Overall Success of the M&L Program It is very early in the life of the project to achieve tangible results, especially at the service delivery level. The results presented here are therefore preliminary and require validation. These limited results are mainly process in nature. The M&L office in Boston spends time on the development and refinement of instruments and resources, some of which are adaptations from the FPMD project. Some of these originated from the field: ƒ PROSPEK, a performance-based planning and budgeting tool is in use in Indonesia; ƒ ƒ ƒ MOST, the management and organization strengthening tool has been adapted to suit the Mozambique environment and has been adopted by the MOH there as the national planning tool (MOSTambique); the Business Planning Program (BPP); and the Virtual Leadership Development Program (VLDP) is in use in Nicaragua. Some of these have long-term applications. There are countries that are using the earlier versions of some less sophisticated resources (such as The Manager) to build some basic, fundamental skills. At the same time, some of the sophisticated programs (such as the VLDP) that depend on Internet access are not appropriate in all settings. M&L used core funds to acquire field support in the large programs in Nicaragua and Indonesia. In both cases, the Missions felt that this was a very useful vehicle that allowed for quick startup. A management and leadership training package is in place and has been used in the countries visited except for Tanzania and Indonesia. There is evidence that M&L country teams are flexible and do select the most appropriate from the available training package to suit the country environment. These country teams are using an approach that promotes teamwork, ownership, commitment, and a high level of motivation, with results at different levels of the health care system. Preliminary field evidence shows improvements and increases in some service delivery in Mozambique and Nicaragua. However, since the programs have not been in the field for long, there is a need for validation of these increases. Contribution of Field Support Funds to the Overall Success of the M&L Program The RFE mechanism in which donors put funds was developed in Tanzania with technical support from an M&L advisor who conceived and directed its development. This mechanism allows NGOs to quickly access HIV/AIDS funds while awaiting the national mechanisms for the 10 flow of HIV/AIDS funds to be put in place. As part of this package, MSH, in partnership with Deloitte Touche Tohmatsu, assisted in the development of NGO capacity to prepare fundable proposals. This is a successful initiative that would justify the use of core funds to document, disseminate, and replicate in other countries.4 The provincial teams in Mozambique expressed the desire to emulate the capacity so far developed at the central level so that they too can quickly reach all districts and service facilities. The central level requested consolidation and expansion of the program throughout the country. The Nicaragua program may be the best example of M&L replicability, where the program has been replicated to become a nationwide effort within the MOH. The countries visited have acquired funds from different sources. Tanzania obtained significant funding from the Global Fund through the technical assistance and leadership of the M&L advisor. Mozambique obtained funding from the MOH common source of funds through a cost￾sharing system and community inputs in materials, while Nicaragua acquired funds from external, non−USAID sources. The process for management and leadership development in Mozambique and Nicaragua is very effective as it has positively changed the organizational climate at the national, provincial, and district levels. At the facility level in Mozambique and Nicaragua, the teamwork that includes community involvement in problem identification, priority setting, and the identification and implementation of interventions is very strong and is yielding preliminary results. Indonesia is beginning the expansion of PROSPEK even though there are no measured impact service delivery results yet. It is significant that other donors (the European Union, Catholic Relief Services, and the Asian Development Bank), who are all seeking mechanisms to help districts work more effectively, are discussing using PROSPEK with M&L. SYSTEMS AND MANAGEMENT In general, MSH’s management of M&L is quite good. At no time in the team’s conversations with USAID in Washington or the field were there any complaints about MSH’s management of M&L. This includes the quality, timeliness, and relevance of reports, project deliverable documents, and invoicing. MSH is a competent and capable cooperating agency (CA), and M&L is a well-implemented project, with a few exceptions. There did appear to be room for improvement on the visibility of MSH and its institutional capacity. In Tanzania, when asked about the applicability of some of the methodologies developed by MSH, both under M&L as well as under predecessor projects, the Population, Health, and Nutrition (PHN) officer indicated that he was not knowledgeable about MSH’s corporate capability and that there had been an absence of senior MSH management involved in the Tanzania program. It appears that this individual was not fully aware of the involvement of M&L senior management as would have been the program officer, who was out of the country during the team’s visit. Similarly, the Indonesia Mission mentioned the absence of senior M&L management. These comments also seem to unfairly represent the efforts of M&L to conduct management visits to Indonesia. In both cases, M&L has actively pursued management visits and involvement. However, the lack of awareness by the Missions of these efforts and the 11 4 See appendix D for the Tanzania country report for details on the RFE. continuing difficulty in receiving approval for management visits may indicate that MSH needs to rely on a wider variety of communication approaches. Quarterly e-mails or teleconferences inquiring about program progress and outstanding issues might substitute partially for the lack of in-person visits. Personnel and Staffing In both Boston and the field, the assessment team found the M&L staff to be of the highest quality. Before traveling to the field, the team spent over 2 days in Boston, receiving a briefing on the various interventions, methodologies, and programs developed and implemented by MSH through M&L. All of the program managers and technical staff were available to talk with the team. Program managers had a thorough understanding of the field programs and their history, accomplishments, and issues. Subsequent questions were handled competently and in a timely fashion. During the visit to Boston, a substantial amount of time was spent discussing some of the electronic learning platforms and networking forums that M&L is developing. These include Leadernet, TCNetwork, VLDP, Global Exchange for Reproductive Health, and Communities of Practice/Knowledge folders. During the discussions, the M&L staff demonstrated the knowledge, thoughtfulness, and experience that created and shaped the electronic learning programs. Although M&L should establish monitoring and evaluation systems to prove their appropriateness, these programs appear to be state-of-the-art designs. The chiefs of party in all countries visited were high quality, technically competent, respected by USAID Missions, and held in high regard by their implementing counterparts. In all cases, they appear to be highly effective in program design and management. The regard with which they are held by their counterparts has undoubtedly contributed to their success. In Indonesia, some communication problems exist between the M&L staff and some members of the USAID Mission, but given the pending closure of that program, there does not appear to be a need to analyze this further. M&L staff in the field appears to have easy and frequent access to MSH’s technical capacity, both in the form of senior technical advisors and in the form of MSH’s institutional knowledge. Several country programs use technical support from Boston on the design and implementation of specific program activities. Additionally, field staff has used and adapted MSH materials from previous projects for current use. For example, the team in Mozambique adapted the Management and Organization Sustainability Tool (MOST) for use with the MOH. This new adaptation has been named MOSTambique and enjoys a high level of ownership by its implementing audience. Similarly, the M&L staff in Tanzania cited its use of archived issues of The Manager for training and guidance. The awareness of these resources and their easy access by the field staff undoubtedly enhances the quality and ease of implementation of in-country programs. Relationships With USAID The relationship between MSH/Boston and USAID/Washington appears to be positive and effective. Despite inevitable turnover in program managers and senior technical advisors for USAID, there is a high level of knowledge about M&L’s activities and issues. The relationships between field personnel and Mission staff are solid and productive as well. On more than one occasion, when asked what M&L could do to improve its program, “Clone the chief of party,” was given as a response. 12 As mentioned above, the assessment team did note a separation between senior MSH management and some of the field Missions. Perceived separations appear to be the product of miscommunication and the inability to travel. MSH may want to be more proactive in identifying communication approaches other than in-person visits. Although the working relationship between MSH/Boston and USAID/Washington is quite good, there is room for improvement in the message that MSH communicates. During interviews with USAID/Washington, there was extensive awareness of the tools developed by MSH/M&L, but almost no awareness of their utility or effectiveness. Although there is a clear commitment by USAID to leadership development, the modus operandi for that development is not obvious. For M&L activities to continue in the future, USAID needs to be a better educated consumer. Monitoring and Evaluation Systems The greatest weakness identified by the assessment team was M&L’s monitoring and evaluation systems. Although the difficulty in fully absorbing the complexity of a monitoring and evaluation system in such a short period of time is acknowledged, the M&L system is lacking. M&L is in its fourth year. Although a thorough monitoring and evaluation system is in place in Nicaragua, there were no operational systems in Indonesia, Tanzania, or Mozambique. Indeed, when the chief of party in Tanzania was asked about the monitoring and evaluation system, she said that they were working on it. Monitoring and evaluation systems should be an integral component of every program (financed from both core and the field) from inception. When asked about monitoring and evaluation in Boston, numerous objective evaluation studies were cited and shown to the team. Although these were interesting and helpful, they cannot answer the pressing questions posed by USAID, including, “What works?” “When and where does it work?” “Why does it work?” and “How should USAID and M&L prioritize their efforts in the future?” A centralized monitoring and evaluation database has been developed. However, its utility for the field programs is unclear. One chief of party stated that she was asked to populate the database periodically, but was unable to access it. Another chief of party indicated that he was asked to supply data for the database, but that none of the data requested were data that he was collecting. This chief of party also mentioned his delight and resulting frustration when he was visiting Boston and found a compilation of indicators but could not find any that were applicable to his program. It is acknowledged that management and leadership do not lend themselves easily to evaluation indicators. However, M&L should be providing thought leadership on this subject by meeting with others who are delivering M&L programs and contributing to the state of knowledge regarding management and leadership indicators. One result of an inadequate monitoring and evaluation system is the lack of cost information. As a result, it is difficult to calculate the replicability cost of M&L interventions and to justify the continuation of M&L where resources are limited. As mentioned above, M&L would benefit from a more definitive, market-differentiating message. Good monitoring and evaluation systems are necessary for this. The development community is eager to learn from M&L, but that cannot happen unless M&L can quantify and articulate its achievements and the impact that results from its efforts. 13 Management by USAID There is also room for improvement in how M&L is used by USAID. Although USAID is to be applauded for the flexible design of M&L and MSH is to be applauded for being responsive to USAID’s needs in the field, USAID is not optimizing M&L’s implementation capacity. This lack of optimization hurts M&L’s ability to implement effectively and hinders its ability to communicate a cohesive message about its programs. Although these issues are not unique to M&L, they are worth identifying and discussing. USAID uses M&L as a contract vehicle of convenience and operates nonrelevant programs through it. Although it is possible to identify a management and leadership need in most programs, the Afghanistan program and the malaria and integrated management of childhood illness (IMCI) activities in Mozambique do not reflect the primary objective of M&L and could, arguably, be managed through any contract. In Tanzania, although the M&L program is addressing an immediate need (the coordination and production of proposals to access the President’s AIDS Initiative and Global Fund monies) and is being quite responsive to the Mission, it does not appear to be achieving M&L objectives. The program results in much work by the chief of party. Ideally, the Tanzanian counterparts will observe and learn from her efforts. However, the skills transfer and ownership of the management process by the locals that was observed in other countries is not evident in Tanzania. When engaging a project such as M&L, USAID needs to accept that management and leadership development are not like the social marketing of contraceptives. It requires time and commitment. Some of M&L’s programs are not receiving adequate attention because of personnel changes within the Missions and the resulting changes in strategy. When engaging M&L or similar projects, there needs to be a commitment to an accepted implementation period. Even when an M&L program is performing as it was designed to perform, engaging the full range of stakeholders, establishing ownership by the participants, involving the community, creating a leadership mentality, and acquiring funds (Mozambique), its efforts are not fully acknowledged by USAID. Part of this is M&L’s inability to articulate a clear message about its activities and their impact, but much of it is USAID’s inability to take the time to understand and fully appreciate a quality program. LESSONS LEARNED The products of M&L are more than a collection of tools. M&L is creating an array of resources that have increased the knowledge base of understanding how to improve management and leadership in organizations. While tools may be a way of describing some of the products, collectively they create a coherent whole that is greater than the sum of its (tool) parts. Core funding of M&L has created a resource for Missions that would not have been created without it. The institutional knowledge that is resident at MSH is a product not only of a long history of field experience, but of the intensive M&L conceptualization and development work that has been undertaken using core funds. This is a rich resource that benefits Missions beyond their field funding of M&L activities. 14 Focusing M&L activities on intact work teams produces greater benefits than training individuals from different worksites. Many previous training efforts have not paid attention to the gains achieved by training groups of people from a common worksite. The M&L approach has demonstrated these benefits, in some cases by having tried the traditional approach to training groups that are not composed of intact work teams. This training has not produced the same result on improving workplace climate, improved communication, and increased ownership of organizational goals. Having M&L teams in training focus on problems and challenges that they have identified in their organizations increases their ability to work as a team. Focusing on real life problems in the workplace, instead of an abstract problem-solving activity intended just for workshop training purposes, helps teams recognize their own efficacy. They come to understand their ability to identify and solve problems and to make change, thereby empowering themselves and creating personal ownership of the goals of their work units. M&L training results in increased collaboration across operating units, thereby improving quality of care. For example, referral of clients from health centers to hospitals and from hospitals back to health centers after client discharge is being improved in Nicaragua by having hospital directors and health center directors participate in the same M&L workshops. In another example, health centers with medication stock outs now find that neighboring health centers will loan medications until stocks are replenished, a behavior that did not occur before participation in M&L training. By selectively drawing from M&L materials, field staff can develop an M&L approach that is responsive to the needs of organizations in the country in which it is working. The M&L whole￾systems approach to working with organizations, that is, not addressing just one issue or system, gives M&L its special methodology, but that does not mean that every engagement is treated the same. Field staff can draw on the institutional knowledge and vast experience in this work at MSH to tailor a response to the work setting, institutional culture, and organizational needs of the clients. This flexibility increases the value of the institutional resources the CA has to offer; it is more than just a collection of toolsit is an approach to the work to be done. There is a value to persistent follow up with teams that have participated in training, both while they are in training and after training. M&L programs are not one-time workshops or interventions; the programs take place over time, allowing teams the opportunity to practice new learning and skills between sessions and to provide feedback and reflection on their experiences. New capabilities are built over time and are therefore more likely to endure. Multiplier Effect M&L has developed a rational multiplier model, moving training out from regional centers to local service delivery points, using a cadre of locally trained facilitators. Strong evidence of this approach was found in Nicaragua, and preliminary evidence of it was found in Mozambique and Indonesia. An important facet of this model is the accompanying by higher level (regional) trainers of lower level (municipal) trainers as they implement the training with local staff. This supervision and coaching increases the likelihood that the quality of the program will be maintained. This behavior is also found among M&L trainers as they accompany their first group of trainees in this multiplier model on the trainees’ first repetitions of the training program. 15 Instruments and Resources Use of instruments and resources, such as PROSPEK, are being picked up by other donors in Indonesia. Similar evidence was found in Nicaragua. The RFE project in Tanzania is another example with great potential. The core investments in M&L are acquiring funds from other donors to achieve greater impact at no additional cost to USAID. Just as instruments and resources developed under FPMD and earlier programs continue to have value, materials and tools developed under M&L will have future value. Many examples of the continued use of products developed under prior cooperative agreements were found to be valid and valuable for use in the current circumstances in many countries. Those materials that focus on fundamentals are useful for a long time. The assessment team believes that many of the resources that have been and are being developed under M&L have the potential for similar long-term use. A study of the instruments and resources that work (and those that do not) and why they work (or not) is needed. Many of the new resources created under M&L, such as the VLDP and the BPP, need to have a more thorough evaluation to determine whether they are producing results, the circumstances in which they are (most) effective, and the essential elements that must be present for them to achieve successful impact. This will help define the target markets for these programs and determine the readiness of organizations to participate. Connectivity and antiquated technology are major problems in all countries visited by the team, especially outside main urban areas. This limits the potential efficacy and utility of some of the new M&L Internet and web-based programs. There is evidence, however, that highly motivated participants can overcome these technological deficits by going to Internet cafés to engage with the training. The importance of participant motivation is another dimension that needs to be assessed. While electronic tools may have great promise, because they have implementation problems in developing countries, the development of new electronic tools should be given reduced priority. The focus at this time should be on evaluating and vetting the electronic tools that have been developed to date. Electronic access to fundamentals is more important than some newer applications. Evidence of the need for basic leadership and management skills, such as communication, leading teams, and supervision, emphasizes the importance of keeping these at the forefront of M&L work. These are topics that are always important and essential to effective organizational operations. As M&L moves to greater implementation of electronic resources, it is important that fundamentals are revitalized and receive adequate attention. Monitoring and Evaluation Monitoring and evaluation needs strengthening within the M&L program and needs to function at several levels. First, it can help M&L participants recognize the fundamental importance of monitoring and evaluation in guiding and improving their work effectiveness. In doing so, M&L needs to provide guidance and role modeling to show how these are integrated into operations and not added as an afterthought. Second, as an M&L effort, it can aid understanding of the interventions and tools that work that are developed under the program. In some cases, this 16 means contemplating operations research that compares the effectiveness of different approaches and interventions (e.g., comparing VLDP with traditional workshop-based, in-person training). At a third level, monitoring and evaluation needs to contribute to a broader knowledge base about what works in efforts to foster greater M&L capacity. There is an opportunity for M&L to increase its thought leadership in the development of innovative M&L indicators. Leadership indicators need to be anticipated as programs are designed. While M&L has developed an extensive listing of possible M&L indicators, they may not be adequate and/or appropriate for some M&L projects. In Mozambique, for example, it was found that none of the indicators listed measured what the project was attempting to accomplish. This is another area in which M&L needs to strengthen its capacity. Business Planning Program The BPP has less applicability to organizations that need basic, organizationwide business planning than for those seeking a new revenue source. The BPP seems to have much promise for organizations that are attempting to identify new ways or ventures to expand their operations and generate new income. It does not, however, address the needs of organizations that should understand their current operations in more businesslike terms. For example, many organizations (NGOs/PVOs) do not know the cost of delivering their individual products and services, and they do not know if these products and services generate net revenue or lose money. Because organizations increasingly need to focus on sustainability, helping them understand these fundamental issues would be an important variation on the BPP. Giving organizations the option of choosing a BPP whose purpose was analysis of basic operations versus analysis of the feasibility of implementing a breakthrough idea would increase the value of this activity. To reduce the specter of M&L being a collection of tools, there should be a decrease in effort to convert successfully implemented local programs (e.g., RFE currently operating in Tanzania and PROSPEK in Indonesia) into a package that implies some universal applicability. However, these examples, especially the RFE, should be documented for quick replication in other countries. That is, the processes that were used to create the programs need to be documented so that the lessons learned can be transferred. M&L should be evaluated on different measures than a health services delivery project. While the ultimate impact of M&L activities needs to demonstrate that it leads to increased service delivery and thereby increased health status of the population, these are long-term measures that may not be appropriate for the mediating role of effective M&L development. That is, it should be measured on the role it has in moving an organization from one stage of effectiveness to a higher one. Improvement of workplace climate and staff morale are examples of such measures. Others can be devised. Developing leadership enhances performance. When one Quality Assurance Program (QAP) director was asked what he had observed about the benefits of M&L training, he noted that in locations where M&L training had occurred, the implementation of QAP protocols was much quicker and smoother, and undertaken with greater eagerness, than where training had not occurred. This is a reflection of the adapting-to-change content of M&L and of workplace ownership of tasks and outcomes, and is evidence of how M&L can make a wide variety of programs more effective. 17 FUTURE STRATEGIC DIRECTIONS Continuation of M&L M&L is experiencing success in its field programs that should be commended and continued. The first recommendation of the assessment team is the continuation of M&L. It is clear that both management and leadership are critical components of institutional capacity building. In countries that are experiencing decentralization, health sector reform, or a combination of both, the need to develop managers who lead at all levels will continue to be critical. In countries where HIV/AIDS is decimating the population of health workers, learning to lead at all levels is a critical skill. The option of having M&L concepts embedded into other procurements rather than being a separate program was examined. It was concluded that this would not be an effective mechanism because M&L is a critical issue and needs to be seen as such. Embedding it in other programs will lessen its impact and potential influence on in-country programs. Even though it is being recommended that M&L be continued, the team recognizes that there is a risk. The Bureau for Global Health has a pending Leader with Associates cooperative agreement for human capacity development (HCD). This is a large procurement that seeks to “…improve human capacity to implement quality health programs.” It will be done through the achievement of the following results: improved workforce planning, allocation, and utilization; improved health worker skills; and strengthened systems for sustainable health worker performance. It does not include management and leadership per se. Therefore, a continuation of M&L is warranted. However, most Missions (wanting to keep cost centers within manageable limits) will not use two procurements on the general subject of human capacity development. It should be considered likely that HCD, as the broader procurement, will be used as it has the potential for covering a wide range of capacity issues. In hindsight, it may have been more effective to address M&L in the HCD procurement. As it stands now, there is a sizable risk that gains made in management and leadership issues will not be fostered and could be lost. Practicality of M&L M&L is not practical enough in its implementation approach and needs to be more mindful of USAID funding cycles and timetables. Indonesia is a good example. The PROSPEK model for district planning and budgeting takes a long time to fully implement. The PROSPEK model was introduced nearly 15 months ago and is not yet showing results in terms of impact on service delivery. There are two problems with this. The first is that district teams do not have concrete positive results to show their own hierarchy. It is possible that without demonstrated results, they will abandon the effort before it has matured. Second, USAID operates on a short timetable. A new procurement is being designed and a contractor is expected in the field by October 2004. PROSPEK has not had the data to influence that process. USAID Missions need to be more strategic and appreciate the time required when engaging M&L programs. Investing in the long-term results of management and leadership is a critical investment and worth a long-term approach. 18 Improvement of Marketing of M&L Concepts M&L performs excellent work in the field but it has not developed a clear, concise message that defines what it does. The assessment team challenged M&L to respond simply and clearly to the following questions: Why is M&L important? How do field Missions benefit if they use M&L? What will M&L do toward the success of the field programs? M&L’s answer does not make its product attractive. M&L needs to improve the management of its messages, remembering that Missions often think of short-term gains. Monitoring and Evaluation As has been mentioned in the System and Management and Lessons Learned sections, the evaluation part of monitoring and evaluation is weak. This is unfortunate since much of what is being done in the field programs is going well and deserves to be recognized. There is more than one year until the first five years of M&L ends (September 29, 2005), which is sufficient time to ensure that the contribution of each field program to M&L is documented. If monitoring and evaluation continues as planned, conducting evaluations of specific elements of field programs, it risks losing important opportunities to document the value of what has been accomplished. Indonesia is a case in point. Being able to work effectively in the decentralized and difficult environment and also to have created PROSPEK, which is attractive to other donors, is an important accomplishment and should be carefully documented. The RFE in Tanzania is an example of a mechanism that many countries need or are going to need in the near future. It is critical to document the development and implementation of the RFE for possible replication in other countries. This is a success story and deserves to be recognized as such. Thought Leadership M&L needs to exercise thought leadership by developing monitoring and evaluation indicators for management and leadership development. The team envisions M&L leading a group of thinkers in the field of leadership, especially to codify measures/indicators of what makes an effective leader. Instruments and Resources M&L and the predecessor projects FPMT and FPMD I and II have developed many instruments and resources. Some of these are timeless ones that deal with fundamental issues. These continue to have value and should be available globally. The Manager and MOST are two examples. The applicability of electronic tools seems to be premature in most settings. Connectivity is a problem in many countries. In others, where the use of the Internet is not allowed during office hours, the teams have to find other ways to complete their work. While going to an Internet café sounds like a good option, the assessment team was informed that most teams do not find that to be a viable option. M&L needs to study which electronic tools work and which do not and why. While doing that, no new tools should be developed. PROSPEK is a good tool/system for use in Indonesia. Documenting the process of creating and using PROSPEK is important. However, making PROSPEK a tool for global use is not valuable. 19 APPENDICES A. SCOPE OF WORK B. SCHEDULE OF ACTIVITIES C. PERSONS CONTACTED D. COUNTRY REPORTS E. SUMMARY OF MISSION RESPONSES F. M&L FUNDING BY FISCAL YEAR G. GUIDE TO M&L PROGRAMS AND RESOURCES H. M&L SELF–ASSESSMENT I. REFERENCES APPENDIX A SCOPE OF WORK (from USAID) ASSESSMENT of MANAGEMENT AND LEADERSHIP ACTIVITY SCOPE OF WORK April 12, 2004 I. PROGRAM INFORMATION Project Name: Management and Leadership (M&L) Cooperative Agreement Number: HRN-A-00-00-00014-00 Agreement value: $57,765,693.00 Obligation Date: 9/29/2000 to 9/29/2005 II. BACKGROUND The Management and Leadership activity (herein after referred to as M&L) is implemented by Management Sciences for Health (herein after referred to as MSH); cooperative agreement number HRN-A-00-00-00014-00. On September 29, 2000, USAID awarded a competitive five￾year cooperative agreement to MSH for the M&L activity ending September 29, 2005, with a ceiling of $57,765,693. The M&L activity is a five-year cooperative agreement, within a 10-year Results Package ending 2010. The following is funding through fiscal year 2003: Core $22,279,000 Field Support $22,646,000 Add-Ons $8,501,000 Total $53,426,000 The M&L activity was designed to reinforce USAID’s Global Health programming by building capacity in organizations and individuals to manage and lead effective, sustainable health care systems. M&L works with public and private organizations in developing countries that provide primary and reproductive health care, family planning, AIDS-related services, and treatment for infectious diseases. M&L provides technical services to national ministries of health, decentralized health services at various levels of government, international and local non￾governmental organizations; and participates in a variety of international programs and policy arenas. Field demand for this activity is substantial. M&L provides services to eighteen missions and regional bureaus to date and has been asked to design interventions in several more countries. In two thirds of the countries where M&L is active, the project has worked with clients for two or more years, attesting to mission satisfaction with their services. Due to the success of this project, M&L reached its ceiling in PY 4. A ceiling increase has been requested. The M&L activity contributes specifically to the Global Health Bureau’s Management and Leadership Development Results Package. The objective of the activity is to improve leadership, management and sustainability of accessible, quality family planning and reproductive health programs by focusing on strengthening systems and individuals in the M&L Assessment Scope of Work A–1 organization, and building organizational capacity through a performance improvement approach. Activity Objective: Improve leadership, management and sustainability of accessible, quality family planning and reproductive health programs. Intermediate Results: ƒ IR 1 - Improved performance of management systems of organizations and programs; ƒ IR 2 - Improved performance of leaders and managers; and ƒ IR 3 - Improved ability to anticipate and respond effectively to the changing external environment. To implement the M&L activity, the M&L team has adopted the following mission statement: To improve the health of women, men and children by: • Building capable, dynamic organizations and programs that deliver high quality integrated services • Capturing, applying and communicating management and leadership knowledge and solutions in the international health field. The M&L portfolio is divided into four interrelated Strategic Directions (SDs): SD 1: Developing the Capacity of Individuals and Teams to Lead and Manage SD 2: Improving Management Systems SD 3: Partnering Locally for Sustainability SD 4: Capturing and Applying Knowledge III. PURPOSE OF ASSESSMENT The purpose of this assessment is to assess the performance of M&L and provide guidance to USAID regarding the design of future activities in this area. Specifically, the assessment team is responsible for achieving the following objectives: Objective 1: Assess the progress made in achieving the three intermediate results both through core and field support funds and the effectiveness of the management structure and systems used for this activity. Objective 2: Identify lessons learned: a) how best to strengthen management and leadership of population/reproductive health and other health programs; and b) whether and how these improvements affect the overall performance of these programs. Objective 3: Make recommendations about future strategic directions for support to management and leadership in GH/PRH, relevant to both field implementation and global leadership. The assessment team will spend approximately one third of its effort on each objective. M&L Assessment Scope of Work A–2 IV. QUESTIONS TO BE ADDRESSED The following is a list of priority questions that the assessment team should address. In carrying out this assignment, the team should be aware that the assessment is taking place in year four of the activity, with over one year remaining for implementation. A) Results and Accomplishments 1. What progress has M&L made in improving management and leadership for health, and in demonstrating how improvements in management and leadership contribute to improved access, quality and sustained service delivery? 2. What are M&L’s primary accomplishments from the investment of Population core funds? How have core funds contributed to the overall success of M&L, e.g., have they provided innovation, tool development, scalability/replicability, field performance, leverage of field funding, other? 3. What are M&L’s primary accomplishments from the investment of field support? Are there specific accomplishments that have been achieved in a context of decentralized health services? B) Systems and Management 1. How effective is the M&L organizational and management structure in achieving results? How does the M&L structure maintain the quality of M&L’s work? 2. Is the M&L management team responsive and accountable to its key clients and partners: USAID Missions, USAID/GH, and host country partners (i.e. government and NGOs)? 3. Are the systems developed by M&L for monitoring, evaluation and knowledge application effective? How have these elements of the program supported the achievement of the overall project objective? 4. Have GH/PRH and relevant USAID Missions been effective in managing the M&L activity? C) Lessons Learned 1. What specific technical approaches or products of M&L have demonstrated the greatest impact in developing strong managers and leaders? 2. What is the value-added of the management and leadership tools developed or refined under the M&L program? Who uses these, why and how? Specific tools include the Developing Managers Who Lead Handbook, Health Manager’s Toolkit, The Manager, the Business Planning Program, MOST, Cost Management Tool, Financial Management Assessment Tool, Knowledge Folders. 3. One of the key approaches of M&L has been the introduction and application of electronic learning and exchange platforms. These include the Virtual Leadership Development Program, Business Planning Program, the Global Exchange – RH, TC Network, Leadernet, etc. How relevant and accessible are these e-learning tools to managers in low resource settings? Has the investment in these electronic platforms contributed substantially to M&L’s ability to replicate and scale up more effectively? M&L Assessment Scope of Work A–3 Do these e-tools and platforms contribute to good management practices and to overall sustainability of priority health programs? 4. The activities managed under Strategic Direction 3, Partnering for local Sustainability and Strategic Direction 4, Capturing and Applying Results, are designed to create a “multiplier effect- to reach the maximum number of managers and leaders”. How has M&L demonstrated the “multiplier effect”, and which approaches/activities provide the best examples? 5. How has M&L replicated and scaled up successful technical approaches and products? What lessons have been learned about the process of replication and scale up, particularly the transfer (applicability) of approaches and products to different cultural contexts? D) Future Strategic Directions 1. What are the priority areas for future Population Core investments within USAID’s Management and Leadership Development Results Package? What gaps and/or future opportunities exist for global technical leadership and field implementation in management and leadership? 2. Is there a justification for GH/PRH to treat management and leadership as a specific technical program to improve the delivery of population and reproductive health services and other priority health services? What are the advantages and disadvantages of maintaining it as a distinct activity versus combining different aspects of the program into other GH activities? 3. What components of the M&L portfolio should be maintained in approximately their current form? What components should be retained, but modified? Are there components or approaches that are no longer needed? 4. What are the prospects and the main challenges for continued utilization of tools developed or refined under M&L after the end of this cooperative agreement? 5. What are the prospects and the main challenges for maintenance and utilization of the different electronic platforms, managed discussion groups, etc. developed or refined under M&L after the end of this cooperative agreement? 6. Which of the activities developed to create a “multiplier effect” under Strategic Direction 3, Partnering for local Sustainability and Strategic Direction 4, Capturing and Applying Results,” are promising and merit continued support in a follow-on project? V. RESOURCES & METHODOLOGY A) Data Sources The assessment team will review relevant documentation, including but not limited to the following: M&L Cooperative Agreement; M&L Self-Assessment report; Management Review documents; research and technical reports; annual reports and other relevant documents. M&L Assessment Scope of Work A–4 B) Self Assessment Prior to the assessment, M&L will conduct a self-assessment (see Attachment 2). The completed M&L self-assessment will be sent to POPTECH on April 28, 2003 to serve as a data source for the assessment team. C) Mission Survey USAID/GH will send surveys to all Missions utilizing M&L, and to a subset of Missions not currently using M&L, to assess interest and experience related to management and leadership. Mission responses will be sent to POPTECH to serve as a data source for the assessment team on April 26, 2004. D) Background Materials/Documents ƒ M&L Cooperative Agreement, Amendments and Proposal ƒ M&L Activity Authorization (AAD)/Results Package ƒ M&L Semi Annual and Annual Reports ƒ M&L Annual Management Reviews ƒ Selected M&L Publications ƒ Selected M&L research and technical reports ƒ M&L Country Program Evaluations ƒ M&L Self Assessment report ƒ Results of USAID survey of Missions concerning the M&L activity E) Key Meetings A Team Planning Meeting will be held in Washington DC for USAID, POPTECH and the assessment team to ensure that the team members understand the assessment objectives. The assessment team will be briefed by the CTO/TA and POPTECH on the purpose, strategy and current status of the M&L activity. Background materials and other data sources will be provided, the timeline finalized and the team member responsibilities assigned. Report preparation guidelines will be provided and discussed. The team will review the outline for the assessment report (Attachment 1) and discuss any revisions with the CTO. The assessment team will travel to Boston to meet the M&L team, receive an orientation on the different strategic directions, approaches and tools developed under the M&L activity, and review in depth the results of the M&L self-assessment. F) Interviews While in Washington, the assessment team will meet with the CTO and TA and will conduct key informant interviews with select USAID Washington staff within the Office of Population and Reproductive Health, and with the former CTO of M&L, now in the Office of HIV/AIDS and with selected field staff as appropriate. A suggested list of interviewees is found in Attachment 4. M&L Assessment Scope of Work A–5 G) Field Visits To evaluate program performance and impact, the assessment team will divide up and conduct field visits in three or four countries. The preliminary plan is to have one or more team members travel to Indonesia, Nicaragua, Mozambique and Tanzania. The CTO and M&L staff will consult with the team on final country selection, and seek concurrence from the selected USAID missions. The criteria for selection of field visits include: large core and/or field support funding, regional diversity, and representation of the four M&L Strategic Directions. The team may also wish to query other missions that have received M&L assistance, as a complement to the information gained from the mission survey that will be undertaken by GH/PRH prior to the assessment itself. The M&L Activity Contact List and suggested consultations and interviews in the field are found in Attachment 3. VI. PROPOSED LEVEL OF EFFORT It is estimated that up to six weeks of effort will be required for each of the POPTECH consultants, and possibly an additional two weeks for the team leader. The consultants will perform some of the work at home prior to the team’s arrival in Washington, D.C. and after the country site visits are completed. The consultants are authorized to work a six-day week when in the field. The assessment will begin in late April. A total of six weeks will be needed for data collection, and approximately 13 weeks to complete the entire assignment. VII. DELIVERABLES A) Debriefings The assessment team will conduct separate debriefings for USAID and M&L team in Washington DC to discuss preliminary findings and recommendations. B) Draft Assessment Report The draft assessment report will be submitted to the CTO on or about June 7, 2004, to be shared with the TA and M&L team for corrections and comments. The draft assessment report will follow the Report preparation guidelines, contain clear findings, conclusions and recommendations, and address the priority questions above. The draft will be submitted in pdf format via email and, if so requested, in hard copy. C) Final Assessment Report The final Assessment Report will be no longer than 30 pages total excluding Annexes (Times New Roman font 12 point). The report will follow the attached outline, and any modifications to the outline should be discussed with USAID/GH. This report will be fully edited by POPTECH with approximately 12 hard copies and 10 CD ROMS distributed. M&L Assessment Scope of Work A–6 VIII. TEAM COMPOSITION The assessment team will consist of five members with the technical expertise described below. Language capability in Spanish is highly desirable for either the team leader or one of the program management specialists; language capability in Portuguese of one member of the team is desirable but not required. A) The team leader. This individual must have extensive experience in international health and management and in sound management and leadership practices. Additionally this individual should have excellent leadership, evaluation, writing, facilitation and interpersonal team skills. B) Four senior program management specialists. These persons should be familiar with GH programs and have experience in design, implementation and evaluation of FP/RH programs. They should have field experience and knowledge of GH results programming and strategic objectives. Note: One of the team members should be familiar with electronic platforms, E-learning and computer assisted learning technologies in low-resource settings. This individual will be responsible for addressing the electronic tools component of the assessment, and may call on outside expertise as needed. Note: A management specialist should be expressly employed to ensure that the assessment is responsive to the needs of USAID/Indonesia. The particular issues concern: a) the wisdom of investing in surveillance and/or local data collection to inform financial and budgetary decisions; and b) the usefulness of the M&L performance improvement-based model for public health planning and performance budgeting. IX. FUNDING AND LOGISTICS All funding and logistical support will be provided through POPTECH. POPTECH activities will include recruiting and supporting the assessment team (including travel, per diem and related team expenses), compiling Mission responses, providing logistical support including setting up meetings in Washington and the countries visited, possible translation and secretarial support, and producing and distributing the final report. The M&L team will assist POPTECH in making arrangements for the country site visits regarding logistics, scheduling of meetings and, if necessary, in-country travel. Week Activity Week 1 (April 26-30) a. Preparation (two days) at home beginning April 22 b. Arrive in Washington DC on April 28 c. Team planning meeting/meetings with USAID & others d. Travel to Boston on May 2 Week 2 (May 3-7) a. Interviews with M&L team at MSH on May 3 &4 b. Depart for field visits on May 5/6 Weeks 3-4 (May 10-21) a. Field visits to selected missions b. Return to Washington DC on May 23 M&L Assessment Scope of Work A–7 Week 5 (May 24-28) a. Team conducts post-field-visit review b. Team prepares for debriefings c. Debriefings on May 26 (USAID) and May 27 (M&L team) d. Travel to home evening of May 28 Week 6 (May 31-June 4) a. Team works at home b. TL consolidates draft report Week 7 (June 7-11) a. TL submits draft report to USAID/CTO on June 9 Week 8 (June 14-18) a. USAID/CTO sends consolidated comments to TL by June 16 b. TL revises draft report Weeks 9-12 (June 21-July 16) a. TL submits final draft to POPTECH by June 23 b. POPTECH edits report c. POPTECH sends clearance copy to USAID/CTO by July 15 Weeks 13-14 (July 19-30) a. POPTECH prints and distributes final assessment report within 3 days of receiving clearance by USAID M&L Assessment Scope of Work A–8 Attachment 1 Outline for Final Assessment Report I. Table of Contents II. Executive Summary (3 pages) – The Executive Summary should convey the important points of the report clearly and concisely. Because it may be distributed to a wider audience, it should be written as a stand-alone document that contains findings and conclusions related to all priority questions listed in the scope of work. III. Background IV. Methodology V. Program description VI. Summary Findings, Conclusions and Recommendations A. Results and Accomplishments B. Systems and Management C. Lessons Learned D. Future Strategic Directions Annexes A. Scope of Work B. Schedule C. List of Contacts D. List of interviewees E. Summary of M&L responses to self-assessment questions F. Summary of Mission Responses G. References M&L Assessment Scope of Work A–9 Attachment 2 M&L Activity: Proposed Self-Assessment Questions of MSH I. The role of management and leadership in improving service delivery and achieving results: A. What is being learned about leadership, management, and the ability to anticipate and respond to a changing environment as being relevant to results in population and reproductive health? B. What is being learned about leadership, management, and the ability to anticipate and respond to a changing environment as being relevant to other areas of public health, i.e., HIV, TB and malaria? II. Accomplishments and lessons learned: A. What are the primary accomplishments and lessons learned from M&L’s work in Strategic Directions 1-4? B. How is M&L building sustainability of technical interventions and programs? C. How effectively is M&L managing and disseminating knowledge? D. How has M&L used core funds to benefit the field? E. What have been M&L’s primary accomplishments form the investment of field support? F. How has M&L used the Cooperative Agreement mechanism to be responsive to the field? G. How has M&L designed and implemented programs to complement activities of other CAs? H. What ideas/interventions did M&L pursue that did not achieve anticipated results and have been dropped? What did we learn from this? III. Technical leadership and contributions to the state of the practice in management and leadership: A. What are noteworthy areas of technical leadership and contributions to development of the “state of practice”, including HCD and managing and disseminating knowledge? B. How does the M&L Program contribute to USAID’s goals? C. What is M&L’s ability to apply knowledge to a rapidly changing environment? IV. Work needed in the future: 1. What is the 10-year vision of the M&L Program? 2. What are some promising programs, approaches and strategies for improving management and leadership for improved health services? 3. How is M&L’s approach to partnerships and collaboration evolving? M&L Assessment Scope of Work A–10 Attachment 3 M&L Activity: Contact List Name Role Areas of Discussion Joseph Dwyer Project Director Project management, MAQ Management & Supervision Sub-committee, IBP Initiative Tim Allen Deputy Project Director Project management, blended learning activities, Global Exchange Program, knowledge management Susan Brinkert Director, Operations Project management, finance, contracts, reporting Alain Joyal Co-Director, Programs Unit Strategic Direction 2, Africa field programs Barbara Tobin Co-Director, Programs Unit Strategic Direction 3, Africa field programs Joan Galer Director, Management & Leadership Development Unit Strategic Direction 1 Alison Ellis Director, Monitoring & Evaluation Unit Strategic Direction 4, Indonesia program Sarah Johnson Senior Program Associate, Programs Unit Latin America field programs, Virtual Leadership Development Program (VLDP), Global Exchange Program Mary O’Neil Principal Program Associate, Programs Unit Human Capacity Development, M&L￾PRIME Joint Program/Armenia Judy Seltzer Principal Program Associate, Center for Health Reform and Finance Business Planning Program (BPP), Latin America Health Sector Reform Initiative Jennifer Braga Senior Project Officer Technical Cooperation Network, Mozambique program Marjut Korkiamaki Program Manager Mozambique Program Nancy LeMay Senior Program Officer Monitoring & evaluation Cary Perry Senior Program Officer Monitoring & evaluation Amber Oberc Administrative Coordinator Knowledge management, Communities of Practice/Knowledge Management, E-room James Wolff Principal Program Associate VLDP, blended learning Elena Decima Senior Program Officer Latin America programs Jennifer Rodine Director, Electronic Products Group Electronic products, distance learning Jude Griffin Electronic Products Group M&L electronic products/distance learning activities Sylvia Vriesendorp Leadership development specialist Kenya, Guinea, VLDP/Africa Janice Miller Director, MSH Publications M&L publications, evaluation of MSH/M&L publications Claire Bahamon MSH Publications, liaison to M&L M&L publications Riitta-Liisa Kolehmainen-Aitken Principal Program Associate, Health Finance & Reform Decentralization, Latin America Health Sector Reform Initiative Dr. Ron O’Connor CEO, MSH M&L Program, current/future challenges Dr. Jonathan Quick CEO-designate (if in Boston in May) M&L Program (has visited M&L programs in Guinea, Nicaragua, Indonesia), current/future challenges Sharon Stash Principal Program Associate, Center for Health Services and Systems HIV/AIDS M&L Assessment Scope of Work A–11 Key informants for Implementing Best Practices Initiative (IBP): Maggie Usher Patel and Susan Monaghan, WHO This list has been already discussed with Dr Abu of USAID/Maputo. Following your indications, we consider that at least two working days it would be necessary for the visit to M&L program in Nampula Province. The list of key informants below focuses on the Health Sector Support workplan implemented by M&L with field support funds. The M&L/Health Sector Support (HSS) subproject consists of a leadership development component, in which we provide technical assistance to managers at the central level and in selected provinces, districts and health units; and a management component, in which we work with central level managers and in selected provinces and districts to enable them to assess management performance and to implement improvement plans. USAID Evaluation of the Management and Leadership Program: Suggested Consultations and Interviews: MOZAMBIQUE Name Title USAID Sra Donna Strausser Deputy Director USAID Mozambique Dr. Abuchahama Saifodine Acting Technical Leader SO3 Sra. Lidia Cardoso USAID official of the M&L program Counterparts Dr Humberto Cossa Director of Planning and Cooperation Dr Gertrudes Machatine Director of Administration and Management Dr Antonio Jose Dabuka Acting Director of Human Resources Ing Moamed Sumargy Director Department of Maintenance Dr Alberto Vaquina Director of Nampula Health Province Dr. Tomas Anselmo Medical Chief of Nampula Health Province Sr Antonio Novela AIDs coordinator DPS Nampula Sr Calixto Maria Sampo Health District Director of Meconta sede Sra Nerina Jonn Health Center of Namialo Other Cooperating Agencies Dr. Moisés Ernesto Mazivila Common Fund and SWAP-MOH Sra. Greta Estima Coordinator Save the Children Nampula Partners Organizations (local or international) Dr Arturo Zanabria Coordinator ONGs Advance Africa Dra Claudia Guzman Coordinator Medicus Mundi Catalunha DPS Gaza M&L Assessment Scope of Work A–12 The following lists of key informants concerns two additional workplans M&L is implementing in Mozambique, malaria and IMCI: 1. The M&L/Malaria subproject assists the National Malaria Control Program (NMCP) by guaranteeing that USAID funds are managed in line with USAID rules and regulations, and that the activities financed by those funds are implemented in an efficient and cost￾effective manner. We will assist in the development of an adequate data collection and information management system for the NMCP that will support accurate decision￾making related to the program. Key informants: Malaria Dr Fransisco Saute _ Director Adjunct- National Malaria Control Program (NMCP) MS Evone Rongu - officer responsible for monitoring & evaluation _ NMCP If Dr Saute thinks that DR Avertino should be consulted, then Dr Avterino Barreto - Deputy Director, Dept Endemic and Epidemic Disease/MOH 2. IMCI: Support to the development of systems and materials for training medical and para-medical staff in "Integrated Management of Childhood Illnesses" (IMCI) through the IMCI program of the Ministry of Health of Mozambique (MISAU) Key informants: IMCI Dra Benedita Silva - Coordinator -IMCI program, MOH Mr Teotonio Fumo - Local Advisor - IMCI Data base, MSH/MOH Dr Bigirimana Zephrin - Local Advisor - IMCI training Depending on the discussion between Dra Benediata and Marinho Dr Martinho Dgedge - Deputy Director- Dept of Community Health- MOH The concern is that it might not be appropriate to contact Drs Martinho and Avertino as informants but as Supervisors or decision makers of the MOH. The M&L M&E Advisor for the NMCP (Chandana Mendis) has already started sensitizing malaria and IMCI programs about the evaluation, except at the level of Drs Avtertino and Martinho. M&L Team: Federico Rocuts, Coordinator and Senior Adviser; federico@msh.org.mz, or frocuts@msh.org Etelvina Mbalane, Local adviser in Leadership Orlando Melembe, Local Adviser in Management Jorge Tojais, Administrative and Finance Manager, jtojais@msh.org; or jtojais@msh.org.mz; principle contact for logistics arrangements (travel, hotel, appointments) Emphasis in local visits, health unit, District and Directorates of MOH M&L Assessment Scope of Work A–13 Note: Field visit will/should involve travel outside Managua, to SILAIS and municipalities noted below. USAID Evaluation of the Management and Leadership Program: Suggested Consultations and Interviews: NICARAGUA Name Title USAID Mr. Alonzo Wind Director of the Office of Human Investment Dr. Ivan Tercero Health Program Manager Dra. Claudia Evans Reproductive Health Program Manager Counterparts Dr. Rodolfo Correa Director of MOH Modernization Program Lic. Violeta Barreto Director of Human Resources, MOH/Virtual Leadership Development (VLDP) participant Dr. Freddy Cardenas, Dr. Carlos Jarquin Executive Director and Medical Director, respectively, PROFAMILIA PROFAMILIA: participant in the Business Planning Program that is ongoing in Nicaragua (see next page); Carndenas is past VLDP participant Dr. Zaira Pineda Dr. Alejandro Sánchez Director of Planning, Ministry of the Family Director, Social Protection Net, Ministry of the Family Dr. Henry Dávila Sub-Director, SILAIS Matagalpa Dra. Ninette Palacios Directora Municipal, Jinotega Dr. Armando Inser Director SILAIS Boaco Dr. Horacio Moreno Director Muncipal, San Lorenzo Lic. Eliseo Aráuz Director, Planning Division, MOH Other Cooperating Agencies NicaSalud: participant in the Business Planning Program that is ongoing in Nicaragua; see next page; a few past VLDP participants are also noted Quality Assurance Project Partners Organizations (local or international) Dr. Tom Bossert Harvard School of Public Health M&L Nicaragua Team: Dr. Barry Smith, Chief of Party Mario Lacayo, Deputy Director Noelia Gutierrez, Executive Secretary to Dr. Smith; can handle all local arrangements (hotel, travel, appointments) Below please find the list of participants who are currently participating in the Business Planning Program (BPP) in Nicaragua. (Past VLDP participants are noted in italic.) The "best bets" for interviews are shaded. All of their contact information is on the list. Another person worth contacting is Allan Hruska - he is the Director of Nicasalud, the organization which contracted PROCOSI/Bolivia to deliver the BPP. His e-mail is ahruska@nicasalud.org.ni. His phone number is the same as those participants noted on the list under Nicasalud. M&L Assessment Scope of Work A–14 Participant List: Business Planning Program, Nicaragua Launched February 2004, Facilitated by PROCOSI/Bolivia with coaching support from M&L Nombre Correo Electrónico Teléfono Organización Observaciones Cesar Ruiz cruizh@nicasalud.org.ni 2700099 NICASALUD Alejandro Uriza auriza@nicasalud.org.ni 2700099 NICASALUD Also VLDP participant: Fernando Campos fcampos@nicasalud.org.ni 2700099 NICASALUD CAPITAN Osmany Altamirano oaltamirano@nicasalud.org.ni 2700099 NICASALUD Gloria Gutiérrez G. division_medica@provadenic.org. ni 2508410 PROVADENIC Natan Brown finanza@provadenic.org.ni 2508410 PROVADENIC Laura Parajón director@provadenic.org.ni 2508410 PROVADENIC Martín Díaz adminstracion@provadenic.org.ni 2508410 PROVADENIC CAPITAN Gilbert Andino cecapro@provadenic.org.ni 2497733 PROVADENIC Enrique Beteta betetae@profamilia.org.ni 2701531 PROFAMILIA Also VLDP participant: Carlos Jarquin cjarquin@profamilia.org.ni 2701531 PROFAMILIA Jairo Narvaez admonmgo@ibw.com.ni 2400322 PROFAMILIA Also VLDP participant: Dr. Freddy Cardenas fcardenas@profamilia.org.ni 2701531 PROFAMILIA CAPITAN Guillermo Ramirez gramirez@profamilia.org.bo 2701531 PROFAMILIA Cósmar Siles pmnica@promujer.org.ni 3114148 PROMUJER Rosa Maria Porras inveco@ibw.com.ni 8338337 PROMUJER Walkiria M. Córdoba pmnica@promujer.org.ni 3114148 PROMUJER CAPITAN Martha García pmleon@promujer.org.ni 3110452 PROMUJER Maura Arostegui maurarey@promujer.org.ni 3110452 PROMUJER René Blandino cepresi@ibw.com.ni 2700652 CEPRESI José Thomas Morales the_232002@yahoo.com 2803632 - 06111140 CEPRESI Alvaro Roberto Rodríguez arodriguez_1541@hotmail.com 2700652 - 08847209 CEPRESI Patricia Gutiérrez oba_moro@yahoo.com 2700652 - 08847209 CEPRESI CAPITAN FACILITADORES/ REVISORES Claudia Muñoz Reyes cmunozreyes@procosi.org.bo (591-2) 2416061 PROCOSI NICASALUD - PROMUJER Fernando Unzueta funzueta@savechildren.org.bo (591-2) 2481615 Save The Children-USA CEPRESI Ma. Cecilia Boada cciboada@ceibo.entelnet.bo (591-2) 2440434 M.S.H./Bolivia PROFAMILIA - PROVADENIC Ervin Larico O. sistemas@procosi.org.bo (591-2) 2416061 PROCOSI ESPECIALIST A EN TI Judy Seltzer jseltzer@msh.org (617) 9429307 M.S.H./Boston M&L Assessment Scope of Work A–15 USAID Evaluation of the Management and Leadership Program: Suggested Consultations and Interviews: TANZANIA Most important are highlighted in bold Name Title USAID John Dunlop HPN Officer Lisbeth Loughran Health Sector Advisor Janice Timberlake Voluntary Sector Team Leader Lisa Baldwin IEC/BCC Officer James Allman Health Public Sector Team Leader Rene Berger HIV/AIDS Officer (just arrived) Counterparts Major-General (Rtd.) Herman C. Lupogo Executive Chairman, TACAIDS Dr. Joseph Temba Director, Policy Planning & National Response Mrs. Rustica Tembele Director, District & Community Response Mrs. Beng’I Issa Director, Finance, Administration & Resource Mobilization Mrs. Joyce Chonjo Director, M&E and Research, Acting Director, Advocacy and IEC Dr. Adeline Kimambo Vice Chairperson & Commissioner, TACAIDS and Chairperson of the Rapid Funding Envelope for HIV/AIDS Mrs. Halima Sharif Commissioner, TACAIDS and Member – Steering Committee of the Rapid Funding Envelope for HIV/AIDS Other Cooperating Agencies Mr. Dan Craun-Selka PACT Mr. Barry Chovitz Deliver Project Mr. Chuck Pill Policy Project Partners Organizations Joe Eshun Grants Manager, RFE, Deloitte & Touche Tanzania Bergis Schmidt-Ehry Chairman, Development Partners Group Dr. Elly Ndyetabura UNDP – Health/HIV/AIDS Dr. Calista Simbakalia, Mr. Peter Riwa, Dr. Justin Ngoma Healthscope Tanzania Dr. Kaushik Ramaiya, Mr. AS Hassim Secretary and President of the Association of Private Hospitals of Tanzania Ms. Ilaria Bernasconi Swiss Development Cooperation (member of RFE Steering Committee) Mrs. Dia Timmermans First Secretary, Royal Netherlands Embassy, member of GFCCM Dr. Roland Swai Head, National AIDS Control Programme, Ministry of Health M&L Assessment Scope of Work A–16 Additional key informants recommended as a result of discussions with Liz Loughran of USAID/Dar: 1. Dr. Peter Mmbuji, former Strategy 7 Coordinator, Ministry of Health (M&L main counterpart from 2000-2002). 2. Rapid Funding Envelope (RFE) projects to visit: TAYOA, PASADA, Shree Hindu Mandal Hospital, Counsenuth. All these are in Dar es Salaam and could be fitted in to other visits. M&L Team: Catherine Severo, Team Leader, csevero@msh.org Initial contact for logistical arrangements (she will likely delegate to administrative staff) USAID Evaluation of the Management and Leadership Program: Suggested Consultations and Interviews: INDONESIA Consultations will involve travel outside Jakarta, to East Java, West Java, and Cianjur District in West Java: Drs. Dwidjo Susono, Senior Advisor to the Minister for Decentralization Dr. Dini Latief, Director, National Research Institute for Health (LitBangKes) and formerly Senior Advisor to the Minister for Decentralization Dr. Azrul Azwar, Director General, Community Health Drs. Zainal, Pharmacist, Directorate of Pharmacy Drs. Mazwar Nurdin, Deputy for Family Information and Program Policy Coordination, BKKBN Molly Gingerich, Director, USAID/HPN Monica Kerrigan, Technical Advisor and CTO, USAID/HPN Dr. Bambang Giatno Director, Provincial Health Office, East Java Dr. Dedi Kuswenda, Director, District Health Office, Cianjur Drg. Titin, Head of Planning, District Health Office, Cianjur Dr. Lily Arianti, Family Health and Team Leader for PROSPEK, MojoKerto, East Java Dr. Siswantoro, Head, District Health Office, Probolinggo, East Java Dr. Farid, formerly staff of Provincial Health Office, South Sumatra; now Head, District Health Office, Ogan Komering Ulu Timur Mr. Russ Vogel, SOAG Administrator and STARH Dr. Gary Lewis, Team Leader, STARH Dr. Adrian Hayes, Policy Advisor, STARH Ms. Anne Hyre, Technical Advisor, MNH Project Dr. Djoko Sutigno, Depurty Team Leader, MNH Mr. John Palmer, Country Representative, HKI Mr. Mark Lediard, Technical Advisor, KuIS (Health Indonesia 2010) Mr. Joel Friedman, USAID/PERFORM Dr. Yin Yin Nwe, Senior Planning/Resource Mobilization Officer, UNICEF Dr. Pierre Claquin, Director, EU/SCHS M&L Team, Indonesia: Dr. Rob Timmons, Team Leader: rtimmons@msh.org OR rtimmons@msh.or.id Dr. Bob Bernstein, Deputy: rbernstein@msh.org Dr. Andy Barraclough, Senior Technical Advisor, Drugs Management: abarraclough@msh.org M&L Assessment Scope of Work A–17 Attachment 4 List of Interviewees at USAID/W 1. Margaret Neuse 2. Jim Shelton 3. Dana Vogel 4. Susan Wright 5. Barbara Addy 6. Estelle Quain 7. Kellie Stewart 8. LAC Bureau Representative 9. Sharon Rudy M&L Assessment Scope of Work A–18 APPENDIX B SCHEDULE OF ACTIVITIES SCHEDULE OF ACTIVITIES Date Location Purpose Team Members April 28 USAID/Washington Meeting with CTO Nancy Piet-Pelon April 29–30 Washington Team meetings, interviews Nancy Piet-Pelon, Lizann Prosser, Pauline Muhuhu May 2 Washington to Boston Travel to M&L office All team members May 3–4 M&L Boston Office Briefing by M&L All team members May 5 M&L Boston Office Briefing by M&L Bob Blomberg May 5–6 Travel Toward Indonesia Nancy Piet-Pelon May 5–6 Travel Toward Tanzania Lizann Prosser/Pauline Muhuhu May 6–20 Indonesia Briefings with M&L and partners/site visits Nancy Piet-Pelon, Barbara Addy, and Peter Connell May 6–13 Tanzania Briefings with M&L and partners/site visits Lizann Prosser/Pauline Muhuhu Mozambique Briefings with M&L and partners/site visits Lizann Prosser/Pauline Muhuhu May 9–20 Nicaragua Briefings with M&L and partners/site visits Bob Blomberg and Susan Wright May 24–28 Washington Team meetings, debriefings with USAID and M&L, report writing All team members May 31–June 4 Washington Finalizing draft report Nancy Piet-Pelon June 4–7 Team member locations Reviewing draft report All team members June 8–9 Amman, Jordan Finalizing report Nancy Piet-Pelon June 9 Amman, Jordan Submit report to USAID CTO and M&L Nancy Piet-Pelon June 9–16 USAID/Washington and M&L Review draft; collate and submit comments to team leader Susan Wright and Joseph Dwyer June 16–22 Amman, Jordan Team leader finalizes report Nancy Piet-Pelon June 23 Amman, Jordan Team leader submits report to POPTECH Nancy Piet-Pelon June 24–July 15 POPTECH Report is edited and prepared for submission Editorial staff July 15 POPTECH Clearance copy to USAID/CTO Editorial staff July 19–30 USAID/POPTECH Final copy cleared; POPTECH prints and distributes final assessment report Susan Wright for USAID, editorial staff for POPTECH APPENDIX C PERSONS CONTACTED PERSONS CONTACTED United States Agency for International Development Margaret Neuse, Director, Office of Population and Reproductive Health Scott Radloff, Deputy Director, Office of Population and Reproductive Health Jim Shelton, Senior Medical Advisor Dana Vogel, Team Leader, Service Delivery Improvement Division (SDI) Susan Wright, Cognizant Technical Officer, M&L/SDI Barbara Addy, Senior Technical Advisor, M&L/SDI Estelle Quain, former CTO for M&L Kellie Stewart, SDI James Griffin, SDI (telephone interview) Population Leadership Program Sharon Rudy Management Sciences for Health (MSH) Management & Leadership Program (M&L) Joseph Dwyer, Director Tim Allen, Deputy Director Alison Ellis S. Brinkert J. Griffin J. Rodine Joan Galer Barbara Tobin Alain Joyal K. Griffin Peg Hume Cary Perry Jennifer Braga Sarah Johnson Nancy LeMay Janice Miller Andrew Sharp Mary O’Neil Sharon Stash Sylvia Vriesendorp F. Nauseda J. Rodine John Wolff Judith Seltzer Marijut Korkaimoki (telephone interview) Technical Cooperation Network Chris Onyejekwe Prem Talwar Specific country contacts (Indonesia, Mozambique, Nicaragua, and Tanzania) are found in appendix D, Country Reports. APPENDIX D COUNTRY REPORTS COUNTRY REPORTS INDONESIA DATES OF VISIT May 5–20, 2004 (Peter Connell, a management specialist, joined the Indonesia visit to address two specific questions posed by the Mission regarding M&L and future work.) PRINCIPAL CONTACTS Ministry of Health, Republic of Indonesia Jakarta Dadi Sugandi Argadireja, Secretary General Azrul Azwar, Director, Community Health (USAID SOAG counterpart) Dini Latief, Head, Health Development Research Dwidjo Susono, Senior Adviser, Capacity Building and Decentralization Triono Soendoro, Director, Centre for Health Education and Training Agus Suwandono, Director, Centre for Disease Control Research and Development Kemas Muhammad Akib, Executive Secretary, Decentralization Unit Zainal Komar, Section Head, Directorate of Pharmacy Bandung Yudi Suprayudi, Health Chief, West Java Province Baniah Patriawati, Head, Health Planning, West Java Province Dedi Kuswendi, Head, Cianjur District Health Office Pak Dedi, Planning Division, Cirebon District Health Office Farid Fairuzi, Facilitator for Performance Improvement, South Sumatra Province Health Office Ketut Mendra, Performance Improvement Coordinator, Nusa Tenggara Barat Province Health Office Ibu Ninik, Head, Primary Health Care, West Java Province Titien Irawati, Head, Planning Division, Cianjur District Health Office Ibu Titin, Health Planning Division, West Java Province Ibu Yuli, Health Planning Division, West Java Province Surabaya Bambang Giatno, Health Chief, East Java Province H. Siswantoro, Head, Probolinggo District Health Office Lili Arianti Singgih, Head, Family Health, Mojokerto District Rissa Burham, Family Health, Mojokerto District Ari Suciati, Family Health, Probolinggo National Family Planning Coordinating Board (BKKBN) Mazwar Noerdin, Deputy for Family Information and Program Policy Coordination Aziz Wahab, Director, Data Processing and Information Technology Ida Bagus Permana, Director, Program Policy Integration Rahmat Sentoso, Director, Monitoring and Evaluation Wandri Muchtar, Director, Reporting and Statistics D–1 U.S. Agency for International Development (USAID)/Indonesia Molly Gingerich, Director, Office of Population and Health Lynn Adrian, Deputy Director, Office of Population and Health Monica Kerrigan, Senior Technical Adviser, M&L CTO in Indonesia Jessica Toludo, Team Leader, Democratic and Decentralized Government Ratna Kurniawati, Program Officer, Infectious Diseases Management Sciences for Health (MSH) Rob Timmons, Team Leader Robert Bernstein, Senior Technical Adviser Andy Barraclough, Senior Adviser, Pharmaceutical Management Asian Development Bank (ADB) Philip Stokoe, Team Leader, Public Health and Nutrition Harmein Harun, Team Member, Public Health and Nutrition Bob Tilden, Team Member, Public Health and Nutrition European Union (EU) Pierre Claquin, International Co-Director Helen Keller International (HKI) John Palmer, Country Director Anuraj Shankar, SUMMIT Program Manager The Royal Netherlands Tuberculosis Association (KNCV) Benson Hausman, Country Representative Jan Voskens, Senior Consultant Maternal and Neonatal Health Program (MNH) Anne Hyre, Midwifery Adviser PATH Iwan Ariawan, Director, Research and Evaluation Agus Sasmito, Deputy Project Director Yanti Triswan, Associate Representative Performance-Oriented Regional Management Project (PERFORM) Robert van der Hoff, Chief of Party Joel Friedman, Senior Urban Policy Adviser Sustaining Technical Achievements in Reproductive Health/Family Planning (STARH) Gary Lewis, Team Leader Adrian Hayes, Policy Adviser Russell Vogel, Technical Adviser United Nations Children’s Fund (UNICEF) Yin Yin Nwe, Senior Programme Officer Budi Subianto, Project Officer, Health Unit Scott Whollery, Project Officer, Health Unit D–2 World Health Organization (WHO ) Lokky Wai, Planning Officer SUMMARY OF FINDINGS Summary of Funding to Date, Indonesia (in US$) Funding Source Field Support Directive FY 00 FY 01 FY 02 FY 03 FY 04 (Received as of 4/28/04) Total (Received as of 4/28/04) Population 200,000 200,000 MAARD 1,300,000 3,535,000 2,025,000 6,860,000 Total 0 200,000 1,300,000 3,535,000 2,025,000 7,060,000 Indonesia radically decentralized responsibility for managing health services and family planning services to the district level in 2001 and 2004, respectively. USAID/Indonesia has positioned the M&L program as its principal provider of technical assistance to district health staff in handling the management consequences of the decentralization initiative. The M&L program now has five activities: ƒ helping the Ministry of Health (MOH) and the National Family Planning Coordination Board (BKKBN) to develop obligatory functions and minimum service standards for districts; ƒ providing a performance-based planning and budgeting tool, the Performance Assessment and Improvement Process (PROSPEK), for use by newly empowered district managers in identifying and resolving service delivery problems; ƒ identifying and tackling drug management and supply issues arising from decentralization; ƒ helping to develop an information-based early warning and rapid response system (EWRRS) for BKKBN to offset its loss of control over national family planning information flows; and ƒ assisting in the implementation of effective surveillance and outbreak control procedures at the district level. Results and Accomplishments Improving Management for Health There is evidence that the program has contributed to improving management for health. ƒ Obligatory functions and minimum service standards have provided an essential framework for setting management objectives in the newly decentralized environment. ƒ The PROSPEK model has helped tie management efforts to addressing service delivery problems in a logical, evidence-based manner. D–3 ƒ The drugs management activity has started to identify and tackle critical drug supply problems created by decentralization. ƒ The early warning system for family planning should supply useful management information for family planning policy setting. However, there is little explicit attention to or progress on the leadership issue. Importantly, the program has not yet demonstrated a clear linkage between management and leadership and improved access to and quality of sustained service delivery. This may just be a timing problem, because all of the PROSPEK projects are still under implementation and their health impact is not yet ready to be measuredalthough measurement will be a difficult challenge. Yet, it should be noted that leaders have emerged from M&L’s PROSPEK process. Two facilitators from districts in West and East Java are now active nationally as facilitators and national speakers on performance improvement in health services. They are exercising the important leadership role as champion. The Johns Hopkins University Gates Leadership Program is operating in Indonesia and was rapidly expanding in scope when the M&L program began in early 2002. It is the principal source of technical assistance and training in leadership development for health managers at the central, provincial, and district levels as well as for hospital administrators. Accomplishments from the Investment of Core Funds Almost all work in Indonesia has been field funded. Core money was used to prepare the initial country assessment and proposed work program. The initial drug management work in January 2002, before the field presence was established in February 2002, was funded with Mission field support funds. It is clear that Indonesia has benefited from past USAID investments of core funds in developing those drug management skills, specifically USAID’s support for the Rapid Pharmaceutical Management (RPM) and ongoing RPM+ programs. A new blended learning approach to delivering PROSPEK to district managers is being developed and should be launched before the end of 2004. The approach draws on the blended learning techniques developed with core money. Accomplishments from the Investment of Field Support Funds All five program activities are driven by the consequences of decentralization. The main accomplishments are: ƒ The lead role in the development of obligatory functions and minimum service standards for both the MOH and BKKBN. This has been well received by government clients. In addition, the BKKBN work represents good cooperation between M&L and another USAID contractor (Sustaining Technical Achievements in Reproductive Health/Family Planning [STARH]–JHU/CCP and JHPIEGO). ƒ PROSPEK was developed (with some out-of-country MSH assistance) and has been introduced to 14 districts. It responds to a clear need of newly empowered district managers, and the underlying process is proving to be robust, adaptable, and affordable. It is being institutionalized through the involvement of provincial government facilitators (50 to date, 75 by the end of the year) and is now starting to D–4 attract wide interest and new funding from the districts themselves and other donors (e.g., the European Union and the United Nations Children’s Fund [UNICEF]/Australian Agency for International Development [AusAID]). The remaining concern is that there is a lack of solid evidence of health impact in the face of the current expansion plans. The process needs to be modified to eliminate drawbacks in scope and implementation time, integration with complementary tools already in use, and M&L’s sustainability and exit strategies. ƒ The drug management activity has established the first post-decentralization picture of national drug supply and highlighted critical management issues. A particular accomplishment to date has been improved efficiency for the tuberculosis vertical program’s drug management system, lauded by both the government and the USAID contractor (Royal Netherlands Tuberculosis Association [KNCV]). ƒ EWRRS is nearing completion and should be introduced in June/July 2004. It is designed to respond to BKKBN’s need for management information, although BKKBN’s ability to provide or engender a rapid response to problems identified seems uncertain. The conclusion is that M&L has performed well for its clients and beneficiaries in Indonesia across most of its activities. The only exception seems to be under Impact Area 3information systems improved to support planning, management and implementation of the essential health packagewhere very little progress has been made to date, except under the auspices of PROSPEK. Systems and Management The government clients in Indonesia are generally very happy and appreciative of M&L’s work. The MOH is particularly pleased with its close working relationship with M&L, feels it has benefited directly from the program, and repeatedly stated its anxiety that M&L “should not stop” in the context of the Mission’s transition to a new country program. BKKBN is similarly appreciative. Provincial governments were supportive of the PROSPEK process but one of the two is concerned about the unwritten nature of its relationship with M&L and the continuing lack of evidence on health impacttwo important client relationship issues that will need to have improved management as the number of provincial clients grows. District managers are happy to have PROSPEK. They acknowledge that it is not perfect, but that it is one of the few available tools that helps with service delivery problems. The Mission is also generally happy with M&L. There are only three criticisms. M&L local management has on occasion proved inflexible in scope. The Mission has a primary focus on maternal, neonatal, and other child health issues and would like M&L to reflect that focus. M&L has a philosophical position that decentralized district management must decide its own focus, and that PROSPEK (as a generic tool for basic health services) should be allowed to follow district decisionseven if that means pursuing health issues beyond USAID’s primary focus. Similarly, M&L ƒ D–5 maintains that drug management issues beyond USAID’s focus (e.g., leprosy control) should also be pursued if they represent a major challenge nationally.1 ƒ ƒ PROSPEK is a minimally facilitated process, which is light on technical content and oversight. M&L management could do more to use the technical expertise of other USAID contractors to assist districts in pursuing the correct technical solutions. M&L is currently urging the expansion of PROSPEK at the expense of the quality of the process and the results. This concern is valid. However, it should be noted that the Mission has encouraged M&L to collaborate with other USAID–funded cooperating agencies (CAs) and donors to facilitate the replication of PROSPEK. Proposals to expand to one new province and new districts in the current focus provinces and to collaborate with the European Union, Catholic Relief Services, the Performance-Oriented Regional Management (PERFORM) project, and Building Institutions for Good Governance (BIGG), are in response to demand from these districts/organizations and have been consonant with Mission interests. Monitoring, Evaluation, and Knowledge Application M&L’s annual work planning and performance monitoring planningand the reporting for bothseem to work smoothly. The program has generally been faithful to its original objectives and shows solid progress. It is ahead of plan on most indicators in the 2003 performance monitoring plan. The exception is Impact Area 3 on information systems, where it is well behind plan (a clear strategy for rectifying this situation is not apparent). Plans and reports seem to be more of a head office issue than a local issue. Some of the M&L knowledge application tools are used in Indonesia. The Manager has been circulated. Other MSH publications on decentralization and drug management have been distributed to counterparts in the MOH, BKKBN, the districts, and other CAs. Relations With the Mission and GH/PRH Day-to-day management of M&L by the Mission seems to have been effective by global standards, despite the project’s cognizant technical officer (CTO) having the largest number of projects to oversee within the Indonesia Office of Population and Health. However, it was concluded that long-term strategic management of the activity from the Mission has been less effective for the following reasons: ƒ M&L was introduced to Indonesia with a timeframe of just 3 years and 9 months. The effective startup was January 2002. The current country program was planned to end in September 2005. The development of management and leadership skills probably requires a longer timeframe than most activities; it would have been ambitious to expect a health impact before the program ends, especially given the destabilizing impact of decentralization. 1 It should be noted that of the 14 districts, 12 did select an MCH topic of their focus, so the concern that USAID expressed may not be warranted. Rather, it is an indication of the lack of effective communication between M&L and the Mission. D–6 ƒ The USAID country program is currently being redesigned; the new basic human services program (including health) will now start as early as October 2004. This has caused confusion over the end date for M&L’s various activities and whether they will continue in another guise. The MOH seems to believe that PROSPEK is about to stop, whereas the team was informed that its continuation in the new program is being considered. The team had been told various end dates for PROSPEK assistance under the current program: December 2004, June 2005, and September 2005. MSH staff is preparing for close out in December 2004. ƒ There seem to be two schools of thought in the Mission about M&L over whether • PROSPEK should continue to expand or focus on improving its quality, • PROSPEK’s performance has been uniformly good or rather mixed, or • M&L should cease (with some exceptions) in December 2004 or sometime in 2005. It is of concern that the local M&L management is getting too close to these controversies. Impact of Technical Approaches PROSPEK is M&L’s greatest contribution to strong management in Indonesia because it has focused managers’ attention on specific service delivery problems in an organized and coherent manner, hopefully leaving behind a rigorous approach to management problem-solving. Its insistence on an evidence base for local decision-making is particularly important given the lack of experience of the district managers, who are its main client group. Furthermore, PROSPEK is a transferable tool. M&L’s other activities in-country have been more oriented toward consulting, coming up with tailor-made solutions to equally important problems, such as the need for performance standards within district health management or analyzing drug supply issues. The contribution to strengthening management skills is positive but much further removed. Only if one believes that strong managers become strong leaders can it be concluded that there has been a contribution to strengthening leadership in Indonesia. In fact, it is believed that leadership is more complex than this and has been somewhat slighted in the country program. Management and Leadership Tools PROSPEK has offered added value in Indonesia for the reasons cited above. None of M&L’s centrally developed tools have received much circulation in-country. Little evidence was seen that there has been much promotional effort for them, which may account for their low circulation. Alternatively, the original market research on which such tools’ development had been based may have been faulty in the assumption that Indonesia would be a target market. The shape of the M&L Indonesia program was proposed by MSH and refined and accepted by the Mission, and neither seemed to see much of a role for such tools as a response to the problems of decentralization. Electronic Learning These tools have not been adopted in Indonesia, except that a blended learning approach to delivering PROSPEK is being developed currently in Bahasa Indonesia and English. This may D–7 assist in accelerating the replication of PROSPEK in the future. Language and the lack of information technology resources in the government’s health system are constraints on the widespread adoption of electronic tools in Indonesia. Multiplier Effect Partnering for sustainability is central to M&L’s plans for PROSPEK. The program has now identified and trained 50 facilitators to carry the PROSPEK process forward and an additional 25 are to be trained before the end of 2004. These facilitators are almost all drawn from the public sector’s provincial health system, implying good prospects for sustainability in the long term. In the short term, it was concluded that this approach is too ambitious; to hand over facilitation from M&L’s own staff directly to provincial government staff seems risky when the long-term role of the provinces is still not clear. SD 4 (capturing and applying results) has been addressed in Indonesia through PROSPEK. This methodology is a good example of M&L using existing, documented knowledge from within MSH and elsewhere and adapting/applying this knowledge to a new country. Replicating PROSPEK To date, the only piece of M&L relevant to replication is PROSPEK. It has so far been replicated in 14 districts, mainly with M&L in-house resources, which is not sustainable. M&L did not facilitate the workshops in all 14 districts. Provincial personnel served as the facilitators, while M&L staff served as coaches. Additional widespread replication of the program will depend on ƒ documentation of the process (completed), ƒ attracting and proving a strong pool of third-party facilitators (focused mostly on government facilitators at this point and far from proven), and ƒ persuading district governments to adopt PROSPEK and implement it with their own funds (adoption is going well and some progress is being made on attracting district funding). The main lessons so far seem to be the following: ƒ Replication takes much longer than expected (PROSPEK is complicated and M&L is not prepared to let MOH facilitators operate alone for some time yet). ƒ Transferring the model directly from the contractor to the client may not be feasible, since the government client has little time for facilitation on a part-time basis, and no budget for adding staff facilitators. An intermediate step may be needed, possibly involving funded universities, NGOs, or other agencies. ƒ PROSPEK seems to be a pioneering tool in a decentralized system; it may justify adoption as part of M&L’s global tool set for application in other countries that have decentralized. D–8 CONCLUSIONS AND ISSUES Future Strategic Directions Priority Areas for Future Population Core Investments From the Indonesian perspective, a clear priority is to strengthen M&L’s leadership component, which is relatively invisible today. This may need some reconsideration on the nature and origins of leadership. The two models that seem to be acknowledged in the program today are that ƒ leadership qualities are innate in a few individuals and ƒ strong leaders emerge from a pool of strong managers. While management skills are probably necessary for leadership, they are not sufficient. At a minimum, style and personality issues need to be tackled to engender the respect and trust that make people follow a leader. On the management side of the program, the top priorities in the development sector today seem to be the following: ƒ human resource management (especially performance motivation/appraisal and incentive systems, which offset to some extent the often low financial compensation in public health); ƒ marketing (especially recognizing who the clients are and understanding their needs; being driven by the client); ƒ planning (both strategic/long-term thinking and short-term action planning; maintaining a strong sense of direction); ƒ information management (especially understanding the differences between data and information, and between computers and information systems); and ƒ financial management (especially financial sustainability: surpluses, deficits, and how to fund them in the long term). M&L needs to have excellent capacity to understand and transfer all five of these priorities. An inventory needs to be taken of M&L’s products and tools against a checklist of this sort. Interviews with the leader of the Gates leadership program in Indonesia revealed its attachment to systems thinking and principles from The Fifth Discipline. This represents a good framework for containing the management tools above; it is unknown whether M&L concurs. Maintaining and Sustaining M&L Management and leadership are sufficiently vague collective conceptsby public health standardsthat they justify a stand-alone program. Leaving individual contractors to develop responses to such topics would lead to an unmanageable collection of approaches that would confuse clients, especially when programs and contractors change and new approaches are introduced or assumed. D–9 Of the two, management principles are perhaps less vague and better documented than is leadership, so that management will tend to dominate in a program that combines both. If M&L believes that leadership primarily emerges from management, then this is acceptable; if not, leadership may need to be given a more separate status in the program to ensure equal treatment. The global M&L portfolio is not readily apparent in Indonesia, so there is little evidence to support continuation or another course. The absence of many of the tools here may indicate either poor marketing (understanding what the client wants) or poor selling (persuading the client that M&L has what it wants), especially with respect to the electronic-based tools. The prospects for PROSPEKthe only readily replicable legacy in Indonesiaare quite good; users are generally enthusiastic and it is affordable. The biggest challenge is to prove that it actually works. It is too early to tell at this time, but if no solid evidence is available within 18 months, it will need a radical redesign. In the meantime, its sustainability can be improved by diversifying the sources of third-party facilitators, perhaps through partnerships with universities or NGOs. Only SD 3 (partnering for sustainability) is relevant in Indonesia, and the approach M&L is currently taking is not necessarily the best. D–10 MOZAMBIQUE DATES OF VISIT May 14–22, 2004 PRINCIPAL CONTACTS Management Sciences for Health (MSH) Team Federico Rocuts, Coordinator and Senior Advisor, M&L Etelvina Mbalane, Local Advisor in Leadership, M&L Olando Melembe, Local Management Advisor Jorge Tojais, Finance and Administration Chadana Mendis, Monitoring and Evaluation Adviser, Malaria Project Benedita Silva, Coordinator, IMCI, Ministry of Health Arturo Zanabria, Nongovernmental Organization Coordinator, Advance Africa U.S. Agency for International Development/Mozambique Abuchahama Saifodine, Public Health Advisor and Acting Technical Leader, SO 3 Lidia Cardoso, Project Management Specialist and Official of M&L Titus Angi, Health, Population, and Nutrition Specialist Mozambique Ministry of Health Maputo Moses Ernesto Mazivila, Deputy Director, Planning Department Gertrudes Machatine, Director of Finance Naomade Sumagaly, Director, Maintenance Department Benedita Silva, Coordinator, IMCI, Ministry of Health Francisco Saute, Coordinator, Malaria, Ministry of Health Nampula Province Augusto Morgado, Director, Lumbo Health Center Adelina Daniel Soane, Maternal and Child Health, Revieria Health Center Calisto Samyo, Deputy Director, Meconta District Health Services Buanali Mussa, Director, Namialo Health Center Mguuela A., Maternal and Child Health, Lumbo Health Center Xovela, Medical Technician, Namyulo Gaza Province Dario Sacur, Health Provincial Director, Xai Xai Alberto Ferreira da Silva, Portuguese Corporation Advisor Luke Verder Gelkien, UNFPA Advisor Francisco Paulo Mahiqa, Department of Finance Castigo Novela, Chief Department of Information, Planning and Cooperation Manuel Victorino, Epidemiology Andre Constantino Paulo, Chief of Planning Claudia Gufman, Outgoing Coordinator, Medicos Sem Fronteiras Natalia Cabrera, Incoming Coordinator, Medicos Sem Fronteiras Samuel Jose, Human Resources Department D–11 SUMMARY OF FINDINGS Summary of Funding to Date, Mozambique (in US$) Funding Source Field Support Directive FY 00 FY 01 FY 02 FY 03 FY 04 Total Health Sector Support Project MAARD 1,360, 000 1,360,000 Child Survival 1, 025,000 586,000 (expected) 1,611,000 Malaria MAARD 1, 640,480 1,640,480 Population 500,000 (expected) 500,000 Total 0 0 1,360, 000 2, 665,480 1,086,000 5,111,480 In Mozambique, the M&L activities are managed by an in-country team of three persons who are accommodated within the Ministry of Health (MOH). Two of the three M&L staff are Mozambican nationals. The M&L activities include three unrelated programs. The funds to support the malaria and integrated management of childhood illness (IMCI) programs that are greater than that of M&L are channeled through M&L; however, the in-country M&L team has no technical responsibility for these programs. The IMCI and malaria programs are both related and relevant to USAID goals and are consistent with M&L’s mandate, as these are concerned with the development of management capacity. For the malaria project, M&L has an in-country technical expert in monitoring and evaluation, who is also an infectious disease expert. One third of the budget in the malaria work plan is allocated toward technical assistance provided for the development of an adequate data collection and information management system for the National Malaria Control Program (NMCP). This will support accurate decision￾making related to the NMCP (approximately $380,000). Another third of the budget is for providing logistics support, mainly for management activities: staffing the NMCP, training health personnel to diagnose malaria, training personnel in epidemiological vigilance (using databases for decision-making), and developing a situational analysis room. The remaining third is for administrative, logistic, and management support for the activities. The IMCI program is a management project. M&L is producing six IMCI training materials that will be distributed in the provinces. M&L also has local, professional managers who are providing technical assistance to introduce the revised IMCI curriculum used in Mozambique to health facilities, and a local, professional database manager who is training provincial-level MOH staff in the use of the IMCI database to improve decision-making. A monitoring and evaluation advisor provides monitoring and evaluation technical assistance to the malaria and IMCI programs; this report does not address those activities. The M&L program that focuses on strengthening the health sector is recent, having begun in February 2003. It has potential for expansion and for having an impact on service delivery for the Mozambique population of 17,479,266. However, this program is threatened by possible delays in the flow of field support funds as a result of the delayed finalization of the USAID– Government of Mozambique strategic plan for the next plan period. The program’s overall objective is to strengthen MOH leadership and management capability in order to enable the health care systems to provide quality health services in accordance with the D–12 vision and established principles of the Strategic Plan for the Health Sector (PESS) 2001–2010. The program focuses its activities at all levels of the health sector, at departments of planning and finance at the central level, and at provincial, district, and health care delivery facility levels. Improved Performance of Leaders and Managers A systematic management and leadership capacity-building effort in the two directorates of the MOH resulted in global directorate-level work plans that have attracted multidonor, common source funding. This has been achieved through leadership training in teams drawn from central, provincial, and district levels using selected MSH tools adapted for the Mozambican environment. The process allowed for the analysis of the work environment in terms of its health problems, capability (human and material resources) to respond to the health problems, and the support from all involved, including the community. This process has also resulted in high levels of commitment, comradeship, and the energy to move ahead. The tools and approaches developed with core funds and the experiences of other countries, such as Brazil and Egypt, were adapted and successfully applied in Mozambique. The MOST tool has been adopted by the MOH as the national planning tool and has been renamed MOSTambique. Human capacity at the district level was described as a major human resource challenge in terms of skills, numbers, and distribution. There is only one doctor for a population of more than 500,000. The services are mainly provided by lower level nurses, midwives, and medical assistants (referred to as technicians). There are also many infrastructural changes facing this group. Before M&L training, these staff members believed that they had technical responsibility only; they did not perceive themselves as leaders or managers. Although they may have been aware of problems, they had previously looked to headquarters or the provincial level to address the problems. Exposure to management and leadership training, which is only in the infancy stage, has resulted in the development of district-level skills for systematic problem-solving and planning by using MOSTambique. High levels of community involvement in problem identification, priority setting, identification, and implementation of interventions were reported by interviewees at provincial, district, and facility levels. Evidence exists of plans and some immediate outcomes in implementation at the facility level. Some facilities have mobilized resources locally to address the priority problems in the districts. Financial support has been obtained from NGOs in some facilities, with material and labor support from the community. To address the maternal and infant mortality/morbidity problem related to complications of unattended deliveries because of distances to the health facility (women deliver on the way to the health center), one health center is in the process of constructing a waiting facility for antenatal mothers who have to travel long distances. The community is contributing materials for this intervention. The health center staff is also educating the community to recognize and determine when women should go to the waiting facility. In the same facility, the community is constructing a well to address water shortages. In another facility, the unhygienic dumping of hospital waste was identified by the community as a priority environmental problem. The facility was able to obtain funds locally from an NGO to construct a placenta disposal pit and another for other hospital waste. In another facility visited, family planning use had increased from 419 in 2003 to 399 in 3 months of 2004 following interventions to improve family planning knowledge and use. D–13 A high level of interest, motivation, and commitment to carry through with the plans developed by the teams was observed in all the sites visited. The team approach to capacity development has resulted in a number of outcomes, but some of these may be subjective. Some outcomes include the first all-inclusive strategic plan formulated at the national level in two departments. It was learned that this strategic plan facilitates equitable distribution of the resources, buy-in, and ownership at all levels. A number of MOH central and provincial personnel have the capability to facilitate the processes with minimal support from the M&L country team. There is a strong potential for continuity in leadership in the field, as the skills are within a team rather than an individual. There is now a better understanding of staff abilities and potential within the teams, with potential for better use of human resources. Systems and Management The effectiveness of the M&L organizational and management structure in achieving results is reflected in the availability of a pool of instruments and tools used as protocols for application to country situations. This successful structure helped initiate the Mozambique country program. M&L used successful country project personnel to assist the Mozambique program, and thus provided a cross-fertilization of ideas and approaches. VLDP and BPP have not been used in Mozambique, and the chances for using VLDP are limited, given the connectivity capability of the country. Acquiring Funds The M&L team in Mozambique has acquired about $3,229,000 for project implementation. The funds come from the MOH sector-wide approach (SWAP) common fund and district-based agencies. In addition, the MOH provides office space for the M&L team, covering all expenses related to office operations, including a desktop computer for the senior advisor and transportation for the team. The common fund pays for participant costs (travel, per diem, and transportation) at the national and provincial levels, and the salaries, per diem, and transportation for MOH facilitators. In addition to acquiring funds, the M&L team has developed the capacity of the personnel involved to successfully lobby for resources, and has created ownership of the program by the MOH. A USAID team shared concerns that the reduction of its funding and the delayed finalization of the country strategy has forced the Mission to advise the M&L team to prepare for project closure by September 2004. CONCLUSIONS AND ISSUES The M&L team is highly competent and highly regarded by the MOH. The immediate future of this successful program is in question, and urgent intervention is required, both to sustain the investments already made and to move forward. There is a need to complete the planned activities in Mozambique in order to move the teams to the next stage, where they can monitor their efforts and plan for the future. Failure to complete M&L work plan activities will result in wasted efforts of these teams, and the process of capacity building will not be completed. D–14 The following areas need attention: completion of planned activities, consolidation of learning and documentation of achievements and lessons learned, and expansion to more districts, provinces, and departments. An inventory of the in-country human resources developed during this project could serve as a resource for the expansion of M&L activities in-country, and the processes applied in the implementation of the project should be created. The processes applied in management and leadership capacity building in the modification of training and tools should be documented for future reference. Management and leadership is critical in all health programs. There is a need for a discussion between the MOH, USAID, and other donors to explore and agree upon the best mechanisms for integrating M&L capacity in all health activities, and which is supported by all donors since the MOH has adopted MOSTambique as a planning tool. D–15 NICARAGUA DATES OF VISIT May 9–19, 2004 PRINCIPAL CONTACTS Management Sciences for Health (MSH), Nicaragua Barry Smith, Chief of Party Mario Lacayo, Deputy Director Diony Fuentes, Reproductive Health Specialist Alba Luz Solorzano, Financial Manager Coordinator Manuel Rodriguez, Coordinator, Fully Functional Service Delivery Points (FFSDP) Julio Ortega, MCH/Community Development Advisor Argentina Parajon, MCH/Community Development Advisor Mary Luz Dussan, Community Health Specialist Claritza Morales, Coordinator, Health Training Carlos Saenz, Coordinator, Ministry of the Family USAID/Nicaragua Alonzo Wind, Director, Office of Human Investment Ivan Tercero, Manager, Health Program Claudia Evans, Manager, Reproductive Health Program Centro para la Educación y Prevención del SIDA (CEPRESI) Patricia Gutierrez, Administrative Director (participant in BPP) PROFAMILIA Carolos Jarquin, Medical Director (participant in BPP) Freddy Cardenas, Executive Director (participant in BPP) Nicaragua Ministry of the Family (MIFAMILIA) Zaira Pineda, Director of Planning Alejandro Sanchez, Director of Social Protection Net Nicaragua Ministry of Health Jose Antonio Alvarado, Minister of Health Violeta Barreto, Director, Human Resources (MOH/VLDP participant) Myrna Somarriba, Coordinator, National Health Plan Pro Mujer Nicaragua Cosmar Siles, Administrative Director (participant in BPP) NicaSalud Fernando Campos, Deputy Director (participant in BPP and VLDP) Allan Hruska, Chief of Party (participant in VLDP) Sistema Local de Atención Integral en Salud (SILAIS) Boaco Armando Incer, General Director D–16 Municipality of San Lorenzo Horacio Moreno, Director, Municipality of San Lorenzo SILAIS Matagaipa Sergio Gutierrez, Subdirector, Training and Investigations Municipality of Jinotega Ninette Palacios, General Director Vaccination and Communal Development Project of Nicaragua (PROVADENIC) Laura Parajon, Executive Director (participant in BPP) Martin Diaz, Financial Director (participant in BPP) Quality Assurance Project Oscar Nuñez, Chief of Party SUMMARY OF FINDINGS Summary of Funding to Date, Nicaragua (in US$) Funding Source Total Field Support Directive FY 00 FY 01 FY 02 FY 03 FY 04 (Received as of 4/28/04) (received only) 60,000 200,000 974,000 1,234,000 Child Survival 400,000 100,000 500,000 Population 100,000 1,345,000 1,445,000 Harvard School of Public Health Infectious Disease 281,000 281,000 Bringing Information to Decisionmakers for Global Effectiveness (BRIDGE) MAARD 500,000 170,000 670,000 Education 0 Total 0 560,000 300,000 3,100,000 170,000 4,130,000 MSH has had a presence in Nicaragua since 1994. The USAID/Nicaragua PHN officer noted that the M&L activity has built on the significant previous experience that the contractor has with local partners and conditions. This has facilitated the implementation of activities under M&L. The PHN team described M&L as highly responsive and effective in carrying out key activities under USAID/Nicaragua’s current strategy. The MSH chief of party is highly regarded in USAID/Nicaragua. His extensive tenure in Central America and the diversity of his work experience in this context results in effective working relationships at many different levels. He is credited by the PHN officer with building a very dynamic and cohesive MSH in-country team, which is considered very effective. Only the chief of party is an expatriate, although several external consultants come from nearby Latin American countries. Many of the in-country staff previously worked on Nicaraguan MOH D–17 activities that are M&L targets. This insider knowledge facilitates relationships and approaches to implementing the work within those organizations. Management and Leadership Development M&L in Nicaragua provides technical assistance to the MOH, PROFAMILIA, the Ministry of the Family, the Social Security Institute (INSS), NicaSalud, and other organizations participating in the Business Planning Program. It also works with rural leaders in Waslala in developing their leadership capacity to improve their communities. The principal area of technical assistance to all these institutions, with the exception of the rural leaders program, is overall institutional reform and strengthening. Many of the tools that have been developed under M&L for use worldwide have their origins in work initially conducted in Nicaragua under earlier funding mechanisms (e.g., PROSALUD). These earlier field-based and field-tested experiences have contributed significantly to the design and implementation of M&L products. Historically, the development of tools has been in the opposite direction: designed in the central office and then tested in the field. The focus or target of the M&L effort is on institutional modernization (reengineering) and reform in the MOH, Ministry of the Family, PROFAMILIA, and the INSS. M&L works side by side with teams in each institution. Using a learning-by-doing approach, M&L works together with the teams to refine policies, models of care, and management systems, and to improve the organizational climate, strengthen leadership, and develop sound business plans. In most instances, the M&L development program is delivered to intact work teams. The philosophical underpinnings and curriculum focus emphasize shared leadership within the work setting: scanning, focusing, aligning, and inspiring. Some training cycles have included groups that do not work together on a day-to-day basis. Participants in these groups have reported less success in achieving desired changes in work relationships, processes, and engagement. As a result, MSH has emphasized working with intact teams whenever possible. Those interviewed at various governmental ministries and NGOs were nearly uniformly enthusiastic about the benefits of their participation in M&L workshops. Some were extremely enthusiastic. Most were able to cite ways in which the workplace climate had improved as a result of participation in the program. Fewer were able to provide evidence of its impact on work team output. (That does not mean that work team output had not changed, but rather that quantifiable evidence was not available at the time of the interview.) M&L Within the Ministry of Health Although it began in 3 of the country’s 17 regional health regions (known as SILAIS), at present the M&L program operates at all MOH levels. The minister has been the enthusiastic executive sponsor of expansion of the leadership development program throughout the MOH, explaining that this cultural transformation (not just of the ministry, but of individuals in all aspects of their lives) was what he wanted to leave as his legacy. He noted that he has 159 labor unions with which he has to work within the MOH, and M&L influences have had an impact on his achieving effective relationships with the unions. Currently there is an intensive focus on delivering M&L training and technical assistance to a cross-departmental working group that is restructuring the entire ministry in anticipation of the D–18 MOH’s implementation of a new 15–year strategy, along with a launching of M&L training at all levels. There is an urgency about this activity because of the likely departure in mid–2005 of the minister who has mandated it. A Guatemalan physician and management and leadership expert, who helped create the M&L participatory methodology for institutional modernization/ reengineering, is the lead M&L technical assistance person for this national MOH reform activity. In multiple interviews, respondents expressed great enthusiasm for the M&L processes they experienced, and the transformations that had resulted from training as a team. Among these were improved work climate (empirically measured), the breaking down of departmental barriers and vertical programming behavior, improved communication and negotiation skills, and a sense of empowerment and job ownership that resulted in the team taking responsibility for achieving its work plans and goals. As part of its replication work, M&L has provided training and follow-on technical assistance to the MOH Department of Human Resources, which has just completed publication of a self￾instruction manual and a facilitator’s manual to replicate the M&L leadership development municipal-level training program nationwide within the MOH. A guide for implementing the program is in its final stages of preparation, with the help of M&L consultants. The guide was funded with M&L core funds. These activities have gone forward through the efforts of other donors, including the German Technical Cooperation (GTZ), the Inter-American Development Bank (IDB), and the World Bank. The World Bank and the IDB are cofinancing the MOH restructuring process. The model for implementing this training is a complex but logical, tiered strategy, intended to acquire resources. Each SILAIS has a training director who participates in M&L training with peers. These SILAIS training directors then work with training directors at each of the MOH municipalities within their SILAIS to prepare them to deliver the M&L training to local teams. The SILAIS trainers accompany the municipality trainers in the implementation of the local workshops. The process of training all MOH employees in municipalities that had participated under the early phases of M&L had not concluded at the time of the visit. According to representatives of different municipalities interviewed, the proportion of employees at different municipalities that had received the training varied widely. When asked why all employees had not yet been trained (given that they had been in the program for over two years), the most common response was that they were awaiting materials from the Department of Human Resources (publications mentioned above) before moving forward. In some instances, however, training had efficiently moved forward without having these resources. Replicating the Fully Functional Service Delivery Points (FFSDP) Under PROSALUD, MSH developed a health unit monitoring instrument that has been adopted by the MOH and applied to over 1,000 health posts and centers. The monitoring mechanism has been praised by the World Bank for its contribution to the Health System Strengthening Policy at the SILAIS level. Known as AMAS under the MOH, the instrument allows the ministry to monitor the status of the service delivery points and with a monitoring software application developed under M&L, to see the results in a graphic format. D–19 Multisectoral Expansion of M&L to the Ministry of the Family and the Ministry of Education The PHN officer has requested that M&L program activities be expanded to include the newly created Ministry of the Family (MIFAMILIA). A full-time M&L staff person is helping the leaders of the former Ministry of Social Affairs and other agencies brought together under MIFAMILIA to work together in the development of their strategic vision and plan for the new ministry. It is anticipated that a ministry planning office will be created and funded at the conclusion of this phase, resulting in an ongoing team-based planning process. M&L is also providing support for the agency’s model of care and in leadership strengthening. Work with the Ministry of Education, again at the request of the PHN officer, had not been initiated at the time of the team’s visit, but exploratory conversations were being planned. However, the focus of the engagement will be built on the social capital development project currently being carried out in 11 communities of Waslala. Under this intervention, M&L is applying its approach to the training of community leaders to build community empowerment, governance, and citizenship. Communities are frequently divided on political and religious lines. Through the M&L intervention, staff members hope to determine if the social capital of communities can, in fact, be enhanced. This activity is a result of USAID–funded research undertaken by Harvard University that aimed at measuring social capital in different communities. When the Harvard activity was added to the M&L project, M&L was forced to create an intervention that would result in the strengthening of social capital in different communities, now that the baseline levels had been measured. The Ministry of Education’s interest is to have all of its rural teachers trained to be able to work with leaders in the communities where they teach. Assistance to PROFAMILIA (Nicaraguan IPPF Affiliate) M&L has provided technical assistance to PROFAMILIA in the design of new roles, functions, and systems for the agency’s management systems strengthening and decentralization plan. The specific focus of the assistance has been on reengineering administrative, management information, and financial processes and systems. These interventions are intended to help the agency achieve increasing levels of self-sustainability as USAID funds are no longer being provided for direct support. The agency has experienced substantial turnover at the top administrative levels due to reductions in the salary scale. A new chief executive officer, a physician formerly with the MCH program at the MOH, had been in his job only 2 months at the time of the visit. He was able to provide a report showing the last quarter results, which indicated that 2 of the agency’s 16 clinics had met costs for the quarter (for the first time), and that others had made improvements. M&L has also been helping the agency understand its cost structure, and determine which of its services have the best net revenue-generating potential. In addition, M&L has worked to strengthen the board of directors and contributed assistance in the development of a strategic plan. The technical assistance to PROFAMILIA from M&L has been critical to the future of the agency. Virtual Leadership Development Program The Virtual Leadership Development Program (VLDP) has it origins in the face-to-face Management and Leadership Development Program (LDP) undertaken in municipalities in three SILAIS in northern Nicaragua (Boaco, Matagalpa, and Jinotega), beginning in 2001. The individuals responsible for creating and implementing that field program had central roles in the creation of the VLDP. D–20 The VLDP is operated entirely electronically; that is, there is no face-to-face session with facilitators at any point in the implementation of the program. As with the traditional LDP, the VLDP focuses on teams, not individuals. In the 12–week program, participants complete homework assignments based on a series of modules. One member of the team is accountable for posting the homework of the group on the VLDP web site. All team members are invited to post their thoughts, issues, insights, concerns, and other items on the café of the VLDP web site. Each day, the facilitator summarizes a few of the postings, develops a thoughtful commentary about issues that are emerging from the various teams, and sends this message to all members as an e-mail. This is intended to increase engagement in the VLDP process and to keep people curious about what is happening. The use of the VLDP version of the Management and Leadership Development Program does not appear to be practicable in some Nicaraguan settings, according to respondents. Connectivity in rural areas is not yet well enough developed to make routine interaction feasible. The MOH prohibits Internet access with its telephone lines. When asked if Internet cafés would be an option (they existed in communities where the question was asked), there was general skepticism about government health workers being willing to put forth the effort to participate if that was a required condition. Business Planning Program The Business Planning Program (BPP), developed by MSH and PROCOSI in Bolivia, currently is being replicated in Nicaragua with teams from five NGOs, including NicaSalud, the umbrella NGO that brings together Nicaraguan NGOs and PVOs working in the health sector, and four of its member organizations. The purpose of the BPP is to help NGOs learn how to identify a breakthrough idea and then to undertake a business planning process that results in the creation of a fundable service or product (assuming that the market study and feasibility analysis indicate that the project has viability). This is the first replication of the BPP by PROCOSI. The BPP consists of an opening 5–day face-to-face workshop conducted by the PROCOSI instructors and M&L staff, followed by six homework modules provided on a CD–ROM. The six modules include 1. self-study of the agency’s strategic position within its sector, 2. identification of a breakthrough idea that identifies what the agency needs to do to be competitive, 3. a market study requiring interviews with members of the target market for the product or service to see if a market exists, 4. creation of a business plan defining individual roles, responsibilities, a time line, and the cost to make the idea a reality, 5. an analysis of the financial return of the project, and 6. an analysis of the social return of the project. D–21 The sequenced business plan development modules and homework are scheduled over 5 months. PROCOSI instructors/experts are available via the Internet to help teams with the homework and to comment and make recommendations on the homework when it is sent as a draft. At the time of the visit, teams were at various stages of finishing the third module, a market study that requires the collection of survey data from target markets. The BPP program will close in June with a final face-to-face workshop, where teams will review their BPP experiences and practice marketing to a potential funding organization. There were differences reported in the success of team participation in the BPP. Of the five teams, two seemed to be on schedule and three were behind schedule (one may no longer be participating). Two of the organizations had experienced or were experiencing substantial turnover of team members due to resignations. Teams were of varying sizes. NicaSalud staff reported that limiting the size of its team was a barrier to successful participation and achievement of the BPP goal. Ten professional staff members were allowed to participate, but there are 14 professional staff in the organization. They felt that this was detrimental to team building and joint ownership of the business plan. This is one of the teams that is behind schedule due to staff turnover and other organizational priorities and crises (and may not complete the planning process due to the new responsibility of managing the Global Fund). As a result of staff resignations, one of the teams (PROFAMILIA) had only two of its original members but it was attempting to continue. The team was attempting to use the BPP to extend the planning practice to each of the 16 centers, with the idea of including them in an organizationwide plan. This was a daunting prospect in the timeframe of the training program. Whether the BPP is suited for general business planning of an organization rather than for studying the feasibility of a breakthrough idea (or idea revolucionaria, as it is referred to in the Spanish version of the program), is not clear. The three remaining NicaSalud member NGOs had made the most progress and had used the experience to strengthen teamwork and workplace interpersonal effectiveness. CEPRESI, PROVADENIC, and Pro Mujer had formulated their breakthrough ideas, but only CEPRESI had conducted a market study for its idea of establishing a men’s clinic targeting men for the testing, diagnosis, and treatment of STIs, including HIV/AIDS and other health issues. Because these organizations are smaller in terms of staff and budget, they seemed to have better success in bringing staff together to do the homework. All of the agencies reported that PROCOSI staff had been extremely responsive and eager to help. Responses to inquiries were almost immediate; it was even suggested that PROCOSI staff members wanted more interaction with participants than they were actually receiving. The appropriateness of the thrust of the BPPfinding a breakthrough idea for a new business activitywas questioned by some program participants. They felt that for their agencies, learning business programming about their current operations might be a more useful place to begin. They reported that having to think in terms of a new business activity seemed inappropriate for them. They nonetheless reported that they felt they learned a more sophisticated way of approaching all of their agency planning work because of their participation in the BPP. D–22 CONCLUSIONS AND ISSUES M&L has been highly responsive to the needs of USAID/Nicaragua in the implementation of its strategy. MSH staff members are highly competent and professional, with a depth of experience that makes them uniquely competent for their work in the country. The enthusiastic response to M&L activity in the health sector (MOH, NicaSalud, and INSS) has resulted in the request to extend interventions to other sectors. Expanding to additional sectors will require an assessment of current levels of effort and the rate at which current staff can be transitioned from current activities to new activities without jeopardizing/diminishing the successful implementation of M&L work to date. There is gathering evidence of the M&L impact on improved workplace climate and worker relations. The M&L impact on these intermediate measures of outcome are not uniform, however, and a best practice analysis of variables accounting for improved team performance would be useful. The evidence of M&L impact on service delivery effectiveness is still quite preliminary. Some evidence was presented (e.g., increased prenatal visit rates, newborn follow-up visit rates), but with the array of diverse inputs in the health sector, attribution of these effects to M&L interventions will require further analysis and substantiation. Additional findings should be forthcoming in an M&L evaluation activity undertaken in April 2004, with results available in June. The train-the-trainers strategy for replication of the MOH M&L development effort requires the evaluation of replication activities using the new self-instructional modules. Leveraging efforts such as these can suffer from a diminution in quality and therefore impact. It will be important to identify the best practices in successful replications in order to reinforce these with other groups responsible for replication and dissemination. Under some circumstances, the VLDP holds much promise as a way in which M&L development work can be undertaken on a broad scale at low cost. For a variety of reasons (e.g., technological skills, Internet connectivity, and competing workgroup priorities), the appropriate use of VLDP requires further study to determine when it does and does not work. Because VLDP appears to offer great promise, long-term impact comparisons with classroom￾based equivalents should be undertaken, including return on investment. Even at a lower success rate, VLDP may prove more cost-effective in many circumstances. Replication needs to happen faster to amortize more quickly the investment made in many of the resources (tools, programs, and activities) developed to date under M&L. Cost analysis needs to accompany all of the implementation efforts (i.e., determining the marginal cost of each repetition). Understanding how and when to most effectively implement the BPP also needs further analysis. Increased exchanges are needed between M&L and the agencies to which the program is being offered to assure better understanding of the concepts and purposes. Also, earlier marketing of the program would allow agencies to put it in their annual work plans, thereby reducing the competition of other priorities in committing to full participation. Outcomes of the BPP need to continue to be monitored and reasons for success and failure analyzed. An analysis of technical D–23 assistance solicited by participants, the frequency of exchanges, the quality and depth of the advice provided, and feedback on drafts should be undertaken to see if it is possible to identify critical levels of interaction that account for more or less successful business plans and (funding) outcomes. Interconnectedness of Efforts/Activities Despite the wide variety of activities and agencies participating under M&L in Nicaragua, there is a certain interconnectedness and reinforcement of experience and learning that ties them together. It is clear that some of the new assignments are taking M&L beyond its health sector work, but this is directly in response to USAID/Nicaragua. All of the work builds on MSH’s extraordinary experience in Nicaragua and true expertise in the development of management and leadership capacity and implementation of systems. D–24 TANZANIA DATES OF VISIT May 6–13, 2004 PRINCIPAL CONTACTS USAID/Tanzania John Dunlop, Health, Population and Nutrition Officer Rene Berger, HIV/AIDS Officer Lizbeth Loughran, PHN Officer (telephone interview) Management and Leadership Program (M&L) Catherine Severo, Team Leader Phyllis Craun-Selka, Local Consultant, Organization Development Tanzania Commission for AIDS (TACAIDS) J.M.V. Temba, Director, Policy Planning and National Response Rustica Tembele, Director, District and Community Response Beng’i Mazana Issa, Director, Finance, Administration, and Resource Mobilization National AIDS Control Program (NACP) Roland Swai, Program Manager Rapid Funding Envelope (RFE) Steering Committee Ilaria Dali-Bernasconi, Deputy Country Director, Swiss Agency for Development and Cooperation (SDC) Pius Wanzala, Program Officer, Norwegian Agency for Development (NORAD) Tanzania Youth Aware Trust Fund (TAYOA) Peter Masika, Executive Director Zanzibar AIDS Commission (ZAC) Asha Abdulla, Executive Director Zanzibar International Film Festival (ZIFF) Fatma Kassim, Administrative Secretary Mwanajuma Kiloko, Accountant Zanzibar NGO Cluster (ZANGOC) Asha Ahmed, Secretary Hassan Kh. Juma, Coordinator Marie Stopes Tanzania Benard Katyetye, Clinical Officer Omar Ibrahim, Peer Counselor D–25 Deloitte & Touche/Tanzania Simon Mponji, Country Managing Partner Joe Eshun, Grants Manager, Rapid Funding Envelope for HIV/AIDS United Nations Development Programme (UNDP) Elly Felix Ndyetabura, National Program Specialist Pastoral Activities and Services for People with AIDS (PASADA) Mary Ash, Executive Director Tanzania Public Health Association (TPHA) Adeline Kimambo, Program Manager, TACAIDS Commissioner and Vice-Chairperson, and Chair, RFE Steering Committee Pact Tanzania Dan Craun-Selka, Country Director African Youth Alliance, Tanzania (AYA) Halima Shariff , Country Coordinator, TACAIDS Commissioner, and TACAIDS RFE Steering Committee Healthscope Tanzania Calista Simbakalia, Associate Director Japanese International Cooperation Agency (JICA) Michiko Tajima, Health Cooperation Planning Advisor Shree Hindu Mandal Hospital Kaushik L. Ramaiya, Medical Officer, and Secretary, Association of Private Hospitals of Tanzania Strategies for Enhancing Access to Medicines (SEAM) Project, MSH William Mfuko, Senior Technical Advisor SUMMARY OF FINDINGS Summary of Funding to Date, Tanzania (in US$) Funding Source FY 04 Field Support Directive FY 00 FY 01 FY 02 FY 03 Received Expected Total Population 150,000 125,000 275,000 HIV 400,000 685,000 1,000,000 2,085,000 President’s AIDS Initiative 750,000 750,000 Total 550,000 0 685,000 1,125,000 0 750,000 3,110,000 D–26 M&L is engaged in three general activities in Tanzania: ƒ support to the Tanzania Commission for AIDS (TACAIDS), ƒ support for the development and administration of the Rapid Funding Envelope (RFE), and ƒ public/private partnerships (PPP) with the Association of Private Hospitals. These activities constitute approximately 70, 25, and 5 percent, respectively, of M&L’s portfolio in Tanzania. The PPP activity is currently dormant due to the departure of the counterpart individual and the inability to engage his replacement. Most of the team’s interviews, conversations, and observations regarded support to TACAIDS and the RFE. The presence of M&L and MSH in Tanzania is almost entirely represented by the M&L team leader. Many of those interviewed were not cognizant of the organizational relationships among MSH, M&L, and the team leader, and generally referred to her efforts as an individual. Repeatedly, when asked what additional assistance would be helpful, the respondent would say “clone [the team leader].” Indeed, those who currently benefit from her involvement want more of her time, and those who do not benefit from her involvement want her to be assigned to their organization. She is doing what the Mission wants her to do and is extremely satisfied with her performance. The coaching and mentoring approach that she uses is much appreciated. However, when inquiring about VDLP and other forms of electronically based development, concerns such as lack of accessibility, lack of time, and lack of discipline to follow through, were raised The need for human capacity development in Tanzania is tremendous. There is a debilitating shortage of leaders and the lack of what USAID/Tanzania calls a “critical citizenry,” that is, a citizenry that demands quality and performance and is critical when it does not occur. There are many reasons for these deficiencies and they will not be corrected quickly, but the need for continued support and investment in management and leadership training is inferred. Support for TACAIDS The team leader receives extraordinary approval from both the Mission and counterparts for her efforts with TACAIDS. She is credited with the success that Tanzania has had in its pursuit of monies from both the President’s AIDS Initiative and the Global Fund. She is further credited with supporting TACAIDS in a critical way through her facilitation of meetings, development of strategic plans, preparation of documents, and capacity-development workshops. She is careful to highlight the support and technical assistance that she receives from M&L/Boston and its technical staff, but what happens in the field is because of the team leader. In fact, the Mission indicated that it had not heard from anyone in M&L/Boston and was disappointed that there did not seem to be greater interest by M&L senior management. Conversely, widespread criticism of TACAIDS was found regarding a number of issues, including lack of strategic focus, inappropriate organizational structure, lack of clarity on roles, inadequate staff, underused commissioners, and inability of senior management to challenge the politically appointed executive director. M&L is not responsible for these organizational issues, but they do highlight some of the challenges of working with TACAIDS and might influence the type of assistance that is most appropriate. D–27 The Rapid Funding Envelope (RFE) mechanism in Tanzania is extremely successful and universally acclaimed. Donors and recipients alike spoke highly of the RFE, which was constructed as a short-term pooling and distribution mechanism for donor funds until the World Bank’s Multi-Country AIDS Program in Tanzania was established. This mechanism was established in collaboration with Deloitte & Touche/Tanzania; M&L is primarily responsible for the development and implementation of the RFE. This mechanism has allowed for the transparent distribution of donor funding intended for HIV/AIDS. Originally intended as a stopgap measure while the country waited for the World Bank’s Multi-Country AIDS Program launch, the RFE has used and distributed funds from eight bilateral donors. The RFE is considered innovative by USAID and its participants and an effective answer to the funding gap created by the delay of the Multi-Country AIDS Program. USAID and others are considering whether this short-term mechanism should evolve into something with greater longevity and application. The RFE appears to be appreciated by all parties and was to be presented by USAID at the state-of-the-art workshop (SOTA) in South Africa (summer 2004) as a funding mechanism that might be of value to other countries. The management of the RFE appears to be trusted, transparent (Deloitte & Touche/Tanzania), democratic (through the use of a donor-staffed steering committee), and efficient (proposals are turned around in a timely manner and screening criteria are equitable). The M&L team leader is credited with the creation, design, and critical support for the RFE. Although intended to be temporary, there is broad interest in its continuation. The steering committee is currently assessing this possibility and the attendant implications on the RFE’s structure and organizational design. Concerns The RFE and its management have relied heavily on donors. Any discussions of an extension will require major reworking of its management to ensure local ownership. Also, the use of Deloitte & Touche (one of the reasons funds management is perceived as transparent and trustworthy) may not be a realistic expense for the long term. This appears to be appropriate for core funds. As a type of common fund, the RFE is a best practice that should be documented and disseminated. The challenges faced by Tanzania in effectively and efficiently directing available donor funding are not unique. Other Missions and countries would undoubtedly benefit by reviewing and adopting the system used in Tanzania. As mentioned above, the public/private partnership activities are largely dormant. Discussions on this topic revolved primarily around the inadequate capacity in the public sector and the need to engage the private sector (generally interpreted to be the not-for-profit private sector) as partners or as an outside source of services. NGOs expressed frustration with the inability of the public sector to become partners with them, even when they were a more effective, less expensive source of services. CONCLUSIONS AND ISSUES There is great support and enthusiasm for the work that M&L (in the form of the team leader) has played in Tanzania. As mentioned above, all stakeholders would like to see her involvement continued, if not expanded. She has committed to staying in Tanzania through March 2005 but is unlikely to continue beyond that point. D–28 The design of her interventions raises questions about the sustainability and endurability of her efforts. There is an acknowledged shortage of qualified managers and leaders in Tanzania as well as a shortage of talented personnel that could be developed into managers and leaders. Despite the high acclaim for her work, her efforts do not appear to be addressing these shortages. Her activities do provide an opportunity for Tanzanians to observe a leader at work and to benefit from her mentoring and coaching. However, she may be doing more than is ideal and the resulting transfer of technical skills is limited and unlikely to be sustained. It is strongly recommended that M&L assistance in Tanzania be reconfigured to encourage greater participation and ownership by Tanzanians so that a reservoir of talent and ability is established in advance of her departure. The RFE is understandably donor driven. Should it become a permanent institution, a greater emphasis will need to be placed on skills transfer, with more Tanzanian involvement in its management and administration. D–29 APPENDIX E SUMMARY OF MISSION RESPONSES SUMMARY OF MISSION RESPONSES Thirty-four questionnaires were sent to various countries as well as to the Bureau for Latin America and the Caribbean (GH/LAC) and the Bureau for Africa and the Regional Economic Development Services Office of the U.S. Agency for International Development. Fourteen responses were returned. This summary describes the general responses to each question and highlights areas of concern. In general, the responses were positive. Missions felt that M&L provided the technical assistance needed and was timely in its work. 1. Purpose: What is the main reason you decided to use the M&L program in your country? Decentralization and health sector reform issues have changed the management and leadership dynamic in many ministries of health. Missions were looking for an existing cooperative agreement that could work to improve management and leadership in these situations. Management Sciences for Health (MSH) has an established reputation through its extensive work with the Family Planning Management Training program (FPMT) and the Family Planning Management Development programs (FPMD) I and II. Its reputation for quality work over several years made it an attractive option to Missions. Missions highlighted the following attributes: ƒ M&L provided uniformly high-quality work, ƒ ƒ ƒ ƒ M&L has the ability to recruit quality technical assistance providers, M&L has technical expertise, M&L was willing to take on long-term and difficult assignments, and there is the capacity to innovate. 2. Implementation Results: Have the M&L program activities in your country been useful in improving management and leadership of priority public health programs? Please explain briefly. In most countries, M&L is being asked to work with the government institutions, mainly ministries of health. Specifically in Indonesia, M&L technical assistance has been extremely timely and important to both the Ministry of Health (MOH) and the National Family Planning Coordinating Board (BKKBN). It helped both institutions learn how to position the central offices to be responsive to decentralization and to reach out and provide guidance/leadership to the districts. It also facilitated very practical hands-on training, or “public health 101,” as it is referred to in Indonesia. Specifically, M&L helped the MOH respond to a legal requirement established by the Ministry of Home Affairs (MHA) that all sectoral ministries establish minimum service standards. M&L led its counterparts through the development of Minimum Service Standards for Essential Public Health Care Functions, which gained a lot of credit for the MOH/BKKBN, not only with the MHA but with international donors and districts as well. 3. Scale and Importance: Targeted Institutions: What are the principal organizations that received M&L assistance in your country? The principal organizations include ministries of health, family planning programs (e.g., the BKKBN in Indonesia), the Commission for AIDS (Tanzania), and private sector organizations E–1 (e.g., APROFAM in Guatemala and NicaSalud in Nicaragua). Assistance was also provided to decentralized health offices at the district levels. 4. Positive Outcomes: What do you consider the most effective activities or aspects of the assistance you have received from M&L? Please explain briefly. The Missions highlighted the timeliness of the assistance provided and the fact that M&L was willing to work on long-term concerns. 5. Challenges and Constraints: What do you consider the least effective activities or aspects of their assistance? Please explain briefly. There were only a few negative comments. One country was concerned at the lack of responsiveness from M&L headquarters, although it did not cite specific examples. Another country where M&L began work but is no longer active felt that its country concerns were “lost in the larger program…M&L could have played a larger role if they had been more aggressive in following up and providing ideas for [technical assistance].” 6. Strategic Fit: Were the M&L interventions in your country specifically designed and tailored to support the Mission strategy? Host government or other sector strategy? In all cases, the countries acknowledged that the M&L interventions did support the Mission and host government strategy. 7. Timeliness, Technical Approach, and Responsiveness: Please comment on M&L’s timeliness, technical proficiency and general responsiveness to the Mission and to host country organizations with which they work. In general, the Missions acknowledge that M&L has always been timely, technically accurate, and exceedingly responsive to the Mission and the government. An example is Indonesia, where, following the Mission’s request to have an M&L planning team in Indonesia in early 2001, the Mission and the M&L CTO worked extremely quickly to field a team, develop, vet, and negotiate a strategic plan, and subsequently to field a high-quality technical assistance team. Government counterparts have always lauded M&L’s work and the high-quality technical assistance provided. One Mission was not happy with the financial report and described it as “inadequate.” 8. Management of Resources: How has M&L managed its personnel and resources in your country? The Missions acknowledged a good use of resources by M&L. M&L is commended for submitting high-quality reports to the Missions on time. 9. USAID/Washington Support: Have the CTO and other staff in GH supported you effectively in your use of M&L? Please explain briefly. Both the former and present CTOs received high marks from most of the Missions for being available, facilitative, and interested in the concerns of the Missions. Two Missions did report E–2 that they had no contact from the CTO at all. Another acknowledged that support from SDI appears to be “hands off.” However, neither of these countries cited a specific problem that had not been addressed because of relations with the CTO. 10. Future Directions: In the future, should there be another global health program similar in scope and approach to the current M&L program? Why or why not? If you feel a follow-on program would be helpful, are there aspects that you would change, add, or eliminate? Please explain briefly. In general, the Missions feel that the work being tackled by M&L needs to be continued in their countries. Issues of decentralization and health sector reform both require new ways of approaching management and leadership at all levels. 11. Additional Comments: Please add any further comments you would like to make concerning M&L work in your country—past, present, or future. Even though the Missions see this as an important future activity, some felt that M&L needs to be more “results oriented” in future programming. E–3 APPENDIX F M&L FUNDING BY FISCAL YEAR (from MSH) M&L FUNDING BY FISCAL YEAR Funding Source FY 00 FY 01 FY 02 FY 03 FY 04 Project to Date Received as of 4/28/04 Totals (Rec'd Only) Core Directive Programmable funds Population 5,000,000 5,094,000 4,000,000 19,444,000 MAQ Population 75,000 25,000 50,000 80,000 230,000 Contraceptive Security Population 50,000 50,000 HIV/AIDS HIV 425,000 425,000 450,000 1,300,000 Providers Guide Population 300,000 300,000 Providers Guide HHS (Interagency) 140,000 140,000 Distance Learning Op Expense 70,000 70,000 Pop/Environ Special Initiative Population 150,000 150,000 Graduated Countries Population 150,000 150,000 Total Core 5,425,000 5,640,000 6,019,000 4,750,000 0 21,834,000 Field Support Directive Afghanistan* Other 5,000,000 5,000,000 Afghanistan* Population 3,000,000 1,200,000 4,200,000 Africa Bureau Child Survival 100,000 200,000 250,000 550,000 Africa Bureau HIV 0 Angola HIV 200,000 200,000 Bolivia Population 200,000 100,000 220,000 250,000 770,000 Bolivia Child Survival 40,000 50,000 90,000 Bolivia Inf Disease 40,000 40,000 Brazil HIV 1,220,000 200,000 1,420,000 Brazil TB 554,631 554,631 Ghana Population 50,000 50,000 Guatemala Population 265,000 32,000 297,000 Guatemala Child Survival 145,000 17,000 162,000 Honduras Population 50,000 64,000 114,000 Indonesia Population 200,000 200,000 Indonesia MAARD 1,300,000 3,535,000 2,025,000 6,860,000 Kenya HIV 0 LAC/HSR Child Survival 78,750 50,000 237,000 365,750 LAC/HSR HIV 96,250 50,000 91,000 237,250 LAC/HSR Population 130,000 130,000 LAC/HSR TB 20,000 20,000 Malawi MAARD 327,884 327,884 Morocco Other 100,000 100,000 Mozambique (HSS) MAARD 1,136,000 1,136,000 Mozambique (Malaria) MAARD 1,640,484 1,640,484 Mozambique Child Survival 1,025,000 1,025,000 Mozambique Population 0 5,350,000 Nicaragua Child Survival 60,000 200,000 974,000 1,234,000 Nicaragua (HSPH) Child Survival 400,000 100,000 500,000 Nicaragua Population 100,000 1,345,000 1,445,000 Nicaragua Inf Disease 281,000 281,000 Nicaragua (Bridge) MAARD 500,000 170,000 670,000 Nicaragua Education 0 Nigeria TB 150,000 150,000 Nigeria Population 100,000 100,000 Nigeria HIV 150,000 150,000 Peru Population 55,000 55,000 REDSOE HIV 200,000 280,000 480,000 REDSOE Child Survival 150,000 15,000 165,000 REDSOE Population 100,000 100,000 Tanzania Population 150,000 125,000 275,000 Tanzania HIV 400,000 685,000 1,000,000 2,085,000 Tanzania PEPFAR 2.0 0 Turkey * Population 325,000 260,000 585,000 Uganda HIV 200,000 200,000 Uganda PEPFAR 1.5 500,000 500,000 Uganda PEPFAR 2.0 0 Uganda MAARD 150,000 500,393 650,393 Total FS 1,230,000 3,319,631 13,395,000 13,647,484 3,523,277 35,115,392 TOTAL 6,655,000 8,959,631 19,414,000 18,397,484 3,523,277 56,949,392 * Turkey and Afghanistan: Core funds treated as Field Support APPENDIX G GUIDE TO M&L PROGRAMS AND RESOURCES (from MSH) Guide to M&L Programs and Resources This guide demonstrates the interrelationships and differences among the various types of programs and resources available in the Management Sciences for Health’s Management and Leadership (M&L) Program funded by the United States Agency for International Development. The resources are organized into three main categories: Building Sustainable Leadership and Management Development Programs; Communities and Networks; and Access to Information and Tools. Within the categories are brief descriptions of each program or resource. Building Sustainable Leadership and Management Development Programs • Leadership Dialogue—The Leadership Dialogue presents an opportunity to engage health managers and leaders in a focused discussion on the challenges and practices of leading and managing. The Dialogue can be used as a catalyst to begin leadership development efforts aimed at addressing organizational challenges. The process, which can take either one or two days, aligns stakeholders around a shared understanding of their challenges and the leading and managing practices needed to address those challenges. • Leadership Development Program (LDP) and Guide—The LDP is a mid-level program that develops the skills and competencies of managers to lead their groups to face challenges and achieve results. Working in the client organization and with local facilitators, participants work in teams on specific performance challenges, often focusing on improving service delivery. Over time, they participate in a series of workshops and local meetings to set and meet challenges together and incorporate this process into the ongoing work of their teams. The LDP was delivered in a face-to-face approach in Egypt, Guinea, Senegal, Kenya, and Nicaragua, and through the blended￾learning approach of the Virtual Leadership Development Program. The Leading Performance Improvement (LPI) Guide, developed in Egypt and expanded to Mozambique and Kenya, is geared to help managers and leaders implement the LDP. The facilitator’s guide enables local organizations to assume responsibility for the program. • Work Group Climate Assessment (WCA)—The WCA is a survey process that enables work groups to assess their current perception of climate, providing insights and strategies for improvement to work group managers on areas requiring attention. This survey complements the LPI Guide and helps teams improve their work climate while they improve performance. WCA has been applied in Brazil, Egypt, Guinea, Kenya, Mozambique, and other Latin American countries via the Virtual Leadership Development Program. • The Senior Leadership Development Program (SLP)—The SLP is a leadership program geared toward national senior level leadership of the public and NGO sectors. The SLP benefits national level and multi￾sectoral coordination. A facilitator’s manual will be developed to transfer implementation to the TCNetwork and other partners. Guide to Programs and Resources M&L—May 2004 1 • Virtual Leadership Development Program (VLDP)—The VLDP is a 12-week blended-learning approach to leadership development which follows the leadership development program described under the LDP. Teams are enrolled in the course, which combines individual work on the VLDP Web site (additionally supported by CDs and workbooks that all participants receive) and on-site team meetings in the organizations. Resolving actual workplace challenges identified by the participating teams forms the basis of instruction. The program is designed to reach multiple organizations and can enroll up to one hundred participants at a time. Since its inception, participants from eight Latin American countries and seven African countries have participated in the program. VLDP can be accessed through the Virtual Center for Leadership and Management. 2 • Business Planning Program (BPP)—The BBP is a three- to five-month program that utilizes both face-to-face and electronic methodologies to help participating organizations build expertise in capturing and packaging breakthrough ideas, identifying target markets, understanding financial and social returns, and configuring human, material, and financial resources results. The BPP was successfully launched in Latin America and Africa where local Partner NGOs (PROCOSI and the Ghana Social Marketing Foundation) are delivering the program. BPP can also be accessed through the Virtual Center for Leadership and Management. • Management and Organizational Sustainability Tool (MOST)—MOST was developed under the Family Planning Management Development (FPMD) Program, M&L’s predecessor. MOST is a participatory management diagnostic process that enables managers in NGOs/FBOs and public sector institutions to develop a management capacity profile for their organization and a prioritized action plan for improvement. By the end of PY5, there will be a “suite” consisting of the overall organizational MOST instrument and specialized in-depth MOST in three areas; Human Resource Management, Financial Management (FIMAT), and Health Information Systems. COMMUNITIES AND NETWORKS • LeaderNet—LeaderNet is a Community of Practice that provides opportunities for ongoing learning and support for managers who lead, and facilitators of management and leadership programs. Members have the opportunities to connect, develop, and gain support through the LeaderNet Web site, email, print, fax, CD-ROM, and phone as well as through face-to-face meetings. LeaderNet can be accessed through the Virtual Center for Leadership and Management (VCLM). • Technical Cooperation Network (TCNetwork)—The TCNetwork is a global community of technical assistance (TA) providers committed to mutual support, accountability, and improving health services. The TCNetwork increases accessibility for clients and donors to quality technical assistance provided by effective, well￾managed, and locally owned members of the community. Through networking and branding among high performing TA providers, the TCNetwork will address some of the barriers to linking local capacity with donors, government agencies, and other clients. TCNetwork can be accessed through the Virtual Center for Leadership and Management. 2 The Virtual Center for Leadership and Management (VCLM) is a Web-based center that integrates and supports M&L’s electronic information resources, online communities of practice, and virtual programs. It provides easy cross￾links for members of the TCNetwork, the Communities of Practice, LeaderNet, Global Exchange for Reproductive Health, the Electronic Resource Center, the Virtual Leadership Development Program, and the Business Planning Program. Guide to Programs and Resources M&L—May 2004 2 • Communities of Practice/ Knowledge Folders (CoPs/ KFs)— The Communities of Practice and their related Knowledge Folders represent a mechanism to synthesize and present useful and up-to-date knowledge, tools, approaches, and information in specific areas of management, leadership, and organizational performance improvement. Four CoPs have been formed—Performance Improvement, Developing Managers Who Lead, Human Resource Management, and Health Information Systems. Using the approach and technology developed, tested, and systematized by M&L, other content areas can be added. The CoPs and their “products” will be available through the Virtual Center for Leadership and Management, the Electronic Resource Center, The Manager, the Handbook, and/or CD-ROMs. • Global Exchange for Reproductive Health—The Global Exchange is an initiative to promote the exchange of information and best practices among countries which have “graduated” from USAID population assistance. It will enable members to contribute and access information from the global community. The current countries are Colombia, Ecuador, Mexico, Morocco, and Turkey. This network, supported by a Web site, will be managed by its members. The Global Exchange for Reproductive Health can be accessed through the Virtual Center for Leadership and Management. Access to Information and Tools • The Manager’s Electronic Resource Center (ERC)—The ERC is a trilingual electronic information resource that shares management experience through an international network of health professionals. The ERC Web site offers more than 10,000 pages of quick-loading information and over 150 ready-to-use management tools, in English, Spanish, and French. The ERC can be accessed through the Virtual Center for Leadership and Management. • The Health Manager’s Toolkit—The Health Manager’s Toolkit is an electronic compendium of tools designed to assist health managers around the world to provide accessible high-quality and sustainable health services. It features technical resources for leading and managing health service delivery collected from a range of USAID￾funded Cooperative Agencies and PVOs/NGOs. New tools from the M&L program will also be added after testing and validation. The Health Manager’s Toolkit is part of the ERC and can be accessed through the Virtual Center for Leadership and Management. • The Manager—The Manager is MSH’s award winning quarterly reaching more than 15,000 people in 192 countries. The Manager provides health professionals at every level with the practical knowledge and tools they need to manage their health care programs. It helps managers improve the quality and sustainability of their health services while addressing financial challenges and constraints. Each issue of this popular periodical focuses on a specific management topic. The Manager includes tools and techniques for solving management problems; working solutions from the field; “how-to” guidelines; and case studies and analyses to be used for staff development and training. The Manager can be accessed through the Virtual Center for Leadership and Management. • Leading and Managing at All Levels: A Handbook for Improving Health Services—This handbook will help build the capacity of managers at all levels of the health system to lead their teams to achieve results and transform their organizations into high-performing organizations. It will include “how-to” guidelines, checklists, and real-life examples. The Handbook will be available in CD-ROM in 2005. Guide to Programs and Resources M&L—May 2004 3 APPENDIX H M&L SELF–ASSESSMENT (from MSH) Management & Leadership Program: Self-Assessment April 23, 2004 Funding provided by the US Agency for International Development under the Management and Leadership Program, cooperative agreement number HRN-A-00-00-00014-00 TABLE OF CONTENTS List of Acronyms i Introduction 1 I. The role of management and leadership in improving service delivery and achieving results 3 A. What is being learned about leadership, management, and the ability to anticipate and respond to a changing environment in relation to results in population and reproductive health and other areas of public health 3 Build on experience and success 3 The M&L Approach 3 Strengthening Management to Achieve Results 6 Why Leadership Is Important 6 The Relationship of Management and Leadership to Improved Service Delivery 6 What Differentiates the M&L Approach 7 II. Accomplishments and Lessons Learned 10 A. What are the primary accomplishments and lessons learned from M&L’s work in Strategic Directions 1–4 10 Strategic Direction 1: Developing Capacity of Individuals and Teams to Lead and Manage 10 Strategic Direction 2: Improving Management Systems 12 Strategic Direction 3: Partnering Locally for Sustainability 14 Strategic Direction 4: Capturing and Applying Knowledge 15 B. What are the primary accomplishments and lessons learned from the investment of field support funds? 17 C. How is M&L building sustainability of technical interventions and programs? 21 D. How effectively is M&L managing and disseminating knowledge? 23 E. How has M&L used core funds to benefit the field? 24 F. How has M&L used the centrally funded cooperative agreement mechanism to be responsive to the field? 25 G. How has M&L designed and implemented programs to complement activities of other CAs? 26 H. What ideas/interventions did M&L pursue that did not achieve anticipated results and have been dropped? What did we learn from this? 27 I. What criteria were used by M&L for selecting countries to use core funds? 28 III. Technical leadership and contributions to the state of the practice in management and leadership 29 A. What are noteworthy areas of technical leadership and contributions to development of the “state of the practice,” including Human Capacity Development (HCD) and managing and disseminating knowledge? 29 B. How does the M&L Program contribute to USAID’s goals? 32 C. What is M&L’s ability to apply knowledge to a rapidly changing environment? 33 IV. Worked needed in the future 34 A. What is the ten-year vision for improving management and leadership? 34 B. What are some promising programs, approaches and strategies for improving management and leadership for improved organizational performance and health services? 35 C. How is M&L’s approach to partnerships and collaboration evolving? 38 LIST OF ACRONYMS BKKBN National Family Planning Coordinating Board (Indonesia) BPP Business Planning Program CAFS Centre for African Family Studies CTO Cognizant Technical Officer CBD Community Based Distribution CoP Community of Practice CfR Consulting for Results CA Cooperating Agency CYP Couple Years of Protection ERC Electronic Resource Center ELCO Eligible Couple FBO Faith Based Organization FHI Family Health International FLEP Family Life Education Program (Uganda) FP Family Planning FPMT Family Planning Management Training FPMD Family Planning Management Development FFY Federal fiscal year FIMAT Financial Management Assessment Tool GSMF Ghana Social Marketing Foundation HIS Health information systems HCD Human capacity development HRM Human resource management IBP Implementing Best Practices (Initiative) IR Intermediate Result ICASA International Conference on AIDS and STIs in Africa IMF International Monetary Fund IRCU Inter-religious Council of Uganda JCRC Joint Clinical Research Centre (Uganda) KF Knowledge Folder LAC Latin America and Caribbean LACHSRI Latin America and Caribbean Health Sector Reform Initiative LCSP Leadership Capacity Strengthening Program (Guinea) LDP Leadership Development Program LDPE Leadership Development Program of Egypt LEAG Leadership Evaluation Advisory Group L&M Leading and Managing LPI Leading Performance Improvement M&L Management and Leadership Program MOST Management and Organizational Sustainability Tool MSH Management Sciences for Health MAQ Maximizing Access and Quality (Initiative) MOH Ministry of Health M&E Monitoring and evaluation NGO Non-governmental organization OPRH Office of Population and Reproductive Health PAHO Pan American Health Organization PHR+ Partners in Health Reform (Plus) project PPD Partners in Population and Development PI Performance improvement PICG Performance Improvement Consultative Group PLP Population Leadership Program PY Program Year M&L Self-Assessment, April 23, 2004 i RPM+ Rational Pharmaceutical Management (Plus) project REDSO Regional Office for Eastern and Southern Africa RH Reproductive Health SAR Semi-Annual Report SSK Social Insurance Organization (Turkey) SD Strategic Direction SO Strategic Objective TA Technical assistance TCNetwork Technical Cooperation Network TDY Temporary Duty TB Tuberculosis UNFPA United Nations Fund for Population Activities USAID United States Agency for International Development VCLM Virtual Center for Leadership and Management VLDP Virtual Leadership Development Program WCA Work Group Climate Assessment WHO World Health Organization M&L Self-Assessment, April 23, 2004 ii INTRODUCTION Three and a half years into a five-year project, the Management & Leadership Program (M&L) continues to expand a successful program, contributing to the realization of the Office of Population/Reproductive Health’s (OPRH) Strategic Objective: To improve leadership, management, and sustainability of accessible, quality family planning and reproductive health programs. The skills and competencies required for managing and leading have never been more crucial to the organizations and national programs with which we work. Their challenges span the gamut from declining couple years of protection (CYP), to the AIDS epidemic and decentralization. Further complicating the challenge is the ever changing environment in which they work; being able to effectively respond to that changing context and to anticipate it raises a high bar for organizational performance. What the M&L Program offers can be applied to family planning (FP) and other reproductive health (RH) settings, indeed in any health care setting; we have worked in vertical and integrated health programs at the clinic, district, and national levels. Results in the face of the organization’s challenges are what matter. The fundamentals of the M&L approach—building a sound foundation of management and leadership capacity and systems using a performance improvement methodology—is a longer-term approach to building sustainable organizations. The value of this approach, and its long-term nature, was recognized in USAID’s investment in a ten-year Results Package. The Leading and Managing Results Model (below), initially developed in Program Year (PY) 1, maintains our focus on the Strategic Objective and Intermediate Results (IRs) defined in the Request for Agreement (RFA). This model shows the connection of leading and managing practices to the intermediate results of Improved Work Climate and Management Systems, to improved service delivery results, and eventually to health outcomes. The model was used in the development of M&L’s Strategic Directions (SDs) and shows the connections and synergy among the three IRs. It has been used in many of our field projects to develop project-specific objectives, and it has served as a framework for the indicator development process. Its simplicity, however, belies the complexity of the real world dynamics of implementing programs in management and leadership. Management and leadership are comprehensive bodies of knowledge. The application of this knowledge to health care in the developing world has been spearheaded by USAID through the M&L Program. M&L Self-Assessment, April 23, 2004 1 Developing capable managers who can lead well and manage well, and who have internalized the eight practices from scanning to evaluating, improves the capacity of organizations to respond to a rapidly changing environment. The connection to M&L’s three Intermediate Results (sidebar) is apparent. IR 1 Improved performance of management systems of organizations and programs IR 2 Improved performance of leaders and managers IR 3 Improved ability to anticipate and respond effectively to the changing external environment. The M&L Program is structured around four Strategic Directions which guide technical leadership in core activities and technical assistance in the field: Strategic Direction 1: Developing Capacity of Individuals and Teams to Lead and Manage Strategic Direction 2: Improving Management Systems Strategic Direction 3: Partnering Locally for Sustainability Strategic Direction 4: Capturing and Applying Knowledge SDs 1 and 2 focus on building high performing individuals, teams, and organizations or programs. SDs 3 and 4 focus on the continuous cycle of knowledge synthesis, dissemination, and application—extending the reach of a single cooperative agreement through the “multiplier” effect of a network of partners. These four Strategic Directions are interdependent, each supporting and expanding the benefits of the others, and all contributing to the realization of the program’s overall Strategic Objective. During the first half of PY1 our focus was on developing the frameworks and delivery methods for our approach to developing sustainable organizations. Then we took those approaches to the field, testing, applying, and refining them in demonstration projects. During PY1 and PY2 our priorities were more heavily focused on SD1 and SD2—developing and applying the M&L approach to capacity building and improving systems. As we moved into PY3, with more and more experience to draw from—both core- and field-support funded—our priorities shifted towards capturing and fine tuning the approaches most effective in enabling organizations to achieve results, and creating the means to scale up and replicate these approaches through a network of local technical assistance (TA) partners. As the Program matured, our priorities have evolved to focus increasingly on SDs 3 and 4—a natural evolution from a focus on “what” we do (leading and managing capacity building and systems improvement) to “how” we replicate it (through partners, continuous learning, and development of a distributable body of knowledge). In the end we feel one of the most valid measures of M&L’s effectiveness is the results our clients are able to achieve and their satisfaction with our work. We have been fortunate to work with many USAID Missions that recognize that these kinds of interventions are multi-year undertakings, and they have bought into the M&L program with multi-year funding. Our work to date in over eighteen countries is evidence that M&L’s approach is immediately relevant to field needs, responsive to USAID, Missions, and clients, and produces results that are replicable across countries. We look forward to the USAID assessment, and begin the process in this self-assessment, which provides background on M&L to date, including lessons learned (Sections I and II), accomplishments (Section II), our contributions to the state-of-the-practice (Section III), and our opinions on what’s needed in the future (Section IV). M&L Self-Assessment, April 23, 2004 2 I. THE ROLE OF MANAGEMENT AND LEADERSHIP IN IMPROVING SERVICE DELIVERY AND ACHIEVING RESULTS A. What is being learned about leadership, management, and the ability to anticipate and respond to a changing environment in relation to results in population and reproductive health and other areas of public health? Build on Experience and Success With over thirty years’ global experience in management development in the health sector, Management Sciences for Health (MSH) has learned that an organization’s ability to improve its FP/RH services, as well as other health services, and to achieve results is significantly improved by an integrated focus on two areas: strengthened work group capacities and management systems. The Family Planning Management Training (FPMT) and Family Planning Management Development (FPMD) I and II projects in particular recognized that: • good management goes hand in hand with good leadership; • good management systems are vital for sustainable results; • good systems and good management alone are not enough. Strong leadership at all levels of an organization is essential, as are the values and efforts of people who are directed and energized by good leadership and a positive work climate; • a holistic approach over time achieves results more effectively than isolated system-specific interventions or “one-off” trainings; • user-friendly, field tested tools and innovative approaches can be adapted for multiple applications at various levels of a health system, reducing time and resource intensive “reinventing the wheel”; • management and leadership approaches and tools cannot remain static, but must adapt to changing conditions just as client organizations must. The M&L Approach With this foundation of experience, M&L developed an integrated approach to addressing its three IRs. The key conceptual pieces for the M&L Program include: Leading and Managing Results Model; Leading and Managing Framework and Process; and Principles for Developing Managers Who Lead. Leading and Managing Results Model—The Leading and Managing Results Model (see the Introduction) describes the relationships among effective leading and managing practices and the improvements in work climate and management systems which ultimately contribute to changes in health services and health outcomes. Organizations that strengthen their management and leadership practices with a focus on results can build a solid foundation for effectively responding to the health needs of populations. Leading and Managing (L&M) Framework and Process—The L&M Framework (next page) of eight leading and managing practices is the foundation of all Program work. Based on the best thinking and practice in leadership and management, the framework is a simple, usable guide which is being adopted and adapted by M&L client organizations and health care workers around the world. The framework defines organizational outcomes for each practice area. M&L Self-Assessment, April 23, 2004 3 M&L Self-Assessment, April 23, 2004 4 The Leading and Managing Process (below) shows the integrated, non-linear, interrelationships among the leading and managing functions, and the need of health managers and teams for continuous inspiration and monitoring and evaluation to both face challenges and achieve results. M&L’s initiatives to “develop managers who lead” are designed to enable work groups or teams and organizations to face challenges and achieve results in complex conditions. This purpose is the same irrespective of the level of the health system at which a program is implemented—central, regional or provincial, district, and health facility level. This process has resonated with managers at all levels of health systems, and has contributed to FP/RH results in M&L projects in countries as diverse as Egypt, Mozambique, Nicaragua, and Guinea. Principles for Developing Managers Who Lead—Five principles underlying M&L activities embody the values, premises, and beliefs for developing managers who lead. They are based on research and field experience and represent a break with a “training” mentality of leadership development. All of our programs, face-to-face and virtual, follow these principles. 1. Leadership enables groups of people to face challenges and achieve results in complex conditions. Results are the true measure of leadership commitment. 2. Leading and managing are commitments and practices that are carried out by people at all levels of organizations and programs. 3. Developing “managers who lead” is a process that takes place over time. This process works best when it is owned by the client organization and addresses critical organizational challenges. 4. You can learn to lead. Leadership commitments and practices improve through a process of facing challenges and receiving feedback and support. 5. Positive changes in commitments and practices are sustained when they are part of the organization’s routine systems. M&L Self-Assessment, April 23, 2004 5 Strengthening Management to Achieve Results Good management is the glue that holds the internal parts of an organization together. It contributes to a positive work climate, supports high-quality services, and helps operationalize the organization’s vision, mission, and strategy. Improving management practices is a means to improving services and making organizations sustainable in three ways: programmatically, financially, and institutionally. • Programmatic sustainability—A well-managed organization delivers products and services that respond to its clients’ needs and anticipate new areas of need. As a result of its success, it is able to expand its client base. • Financial sustainability—A well-managed organization draws on various sources of revenue, which allows it to support its ongoing efforts and to undertake new initiatives. • Institutional sustainability—A well-managed organization has a strong yet flexible structure. Its structure allows it to respond to the shifting priorities of its supporters and to new responsibilities towards its clients, while supporting a positive work environment for its staff. Management systems are not static. As organizations grow, their services expand and client needs shift, an organization needs to evolve. Strong leadership enables an organization to effectively use its management systems, to recognize the need to update and modify these systems without depending on outside consultants. Even well-managed, successful organizations must constantly assess and adapt their management practices as new demands arise and environments change. An organization’s leaders play a critical role in fostering a climate of continuous assessment and improvement to put in motion a change process that involves staff from all the major divisions of the organization, as well as other key stakeholders, to identify and address both internal and external challenges. Why Leadership Is Important Challenges such as the AIDS epidemic and the resulting declining profile of FP in many countries, the continuing effect of poverty on peoples’ health, poor quality services, the instability of funding, and low morale among health workers require managers with leadership abilities. Organizations look to managers who can lead to provide guidance and inspiration under challenging conditions. Managers who can lead are critical to achieving sustainable results, adapting to change, and strengthening their institutions to improve the health of those they serve. Leadership and management are both necessary. “Leadership and management are two distinctive, complementary systems of action, each having its own characteristic practice and activities, but both are necessary for success in an increasingly complex and volatile setting.” 3 The Relationship of Management and Leadership to Improved Service Delivery With each major activity, M&L is gathering lessons about the most effective approaches to strengthen the leading and managing capacities of groups or teams and organizations, whether through short-term training programs, technical assistance, or distance learning approaches. Recognizing that improved management and leadership capacity is a means to achieving the desired goal of improved health service delivery, M&L distinguishes between the intermediate outcomes of technical support (improved work climate and management systems) for improved service delivery and, wherever possible, the expected longer-term relationship between organizational effectiveness and improved health services. Service delivery results are most evident with teams at the district level or below (Egypt, Nicaragua, and Indonesia). At the regional or provincial level and above, the association is more difficult to establish and requires more time. When M&L works with teams at the regional, provincial or central levels, we help 3 Kotter, John, “What Leaders Really Do” Harvard Business Review, May-June, 1990 M&L Self-Assessment, April 23, 2004 6 them make the association with service delivery by guiding their selection of organizational or work group challenges on which they will focus new or strengthened leading and managing practices. Throughout the developing world, forward thinking organizations and governments are working to strengthen their leadership and management capacities. While some countries struggle with limited resources and sustainability, governments and health organizations in countries affected by the AIDS pandemic face increased challenges, most significantly a lack of absorptive capacity. Organizations with limited or even decreasing numbers of staff, and fragile or nascent management systems are now managing significant amounts of new funding and are struggling to demonstrate and report on results to multiple donors within tight time frames. With its clearly defined results model, framework, and process that links management and leadership with improved service delivery, M&L has attracted considerable USAID interest and funding for initiatives beyond family planning and reproductive health and at all levels of the health system. Examples include: • Program/National Coordination—M&L works in Tanzania, Uganda, and Malawi with national AIDS programs and Global Fund programs. • Multi-level—In Mozambique, M&L works with the Ministry of Health on integrated leadership and management development at national, provincial, district, and service delivery levels. The focus at district and service delivery levels is on HIV/AIDS. M&L has also placed an advisor to assist the National Malaria Control Program to develop a management information system. In Indonesia, M&L works with central, provincial, and district managers on a variety of essential systems (policy development, drug management, improving performance in planning and budgeting) to address essential public health services, including FP, maternal and child health services, TB, and malaria. • District—In Kenya, 75 health managers from 15 districts are participating in leadership development in response to the decentralized government’s HIV/AIDS program. • Service Delivery Providers—At the service delivery level, M&L continues to work with managers of integrated FP/RH programs (Egypt, Nicaragua, Indonesia) as well as those responsible for vertical health programs, such as AIDS, TB, and Malaria (Brazil [AIDS/TB]; Mozambique [AIDS/malaria]; Indonesia [TB, malaria]) • Networks and cross organizational—NicaSalud, Partners in Population and Development (PPD), Inter-Religious Council of Uganda (IRCU). The increasing amount of non-population funding that M&L is receiving from Missions indicates their recognition that management and leadership development is relevant and critical for all health service delivery. What Differentiates the M&L Approach? The following table summarizes approaches to: • leadership and management development; • strategies to help an organization respond to changes in its internal and external environment; • monitoring and evaluation practices. The traditional approaches pursued are listed and contrasted with M&L’s approaches, which reflect the lessons we have learned over the years and which we believe differentiates us from some other health development programs. M&L Self-Assessment, April 23, 2004 7 Traditional M&L Program Leadership Aimed at top leaders Aimed at managers at all levels Focus on individual’s leadership skill development Integrated approach focused on improved services, work climate, systems, and health outcomes Often reinforces the notion that leaders are “born” (Gandhi, Martin Luther King Jr.) Skills and competencies that anyone can learn (demystify leadership) Leadership transition: attention, if any, is given only to the top levels Transition preparation at all levels (“the leadership engine”) Often “one-off,” off-site workshops Modularized and facilitated process over time at clients’ workplaces Often separated from the work environment, and unconnected to specific organizational or management challenges Connected to the work environment, to work group or team, and real organizational challenges the client is facing in the workplace Management Expert consultant model of assessment Participatory approach ensures ownership and integration of improvements into organizational plans System-specific improvements led by outside or multiple experts Conducted within context of overall improvement plan; connected to Leading and Managing Framework and organizational outcomes Specific problems fixed in isolation Strengthening organizational skills to identify and address future not just current problems Priorities often set by donors Organizational ownership of identifying challenges and initiating lasting change in cooperation with donors Anticipate and respond to changing environment Management and leadership treated separately Linkages between system strengthening and individual/work group capacity building to achieve organizational outcomes Work with one organization at a time “Multiplier effect” to reach multiple organizations and programs simultaneously through effective local TA providers In person technical assistance Innovative approaches including “blended learning” using new technologies as available and appropriate FP/RH vertical programs FP/RH plus other areas of public health, HIV/AIDS, TB, malaria; Identifying synergies among programs Results by organization Program wide contributions to international efforts, e.g., Cairo Declaration; WHO 3x5; President’s Emergency Plan 2-7-10 Long time frame for demonstrating results Respond with speed, sustain with systems Monitoring and Evaluation Process indicators Indicators tailored to needs and desires of clients to track progress towards achieving their performance objectives and intended results “One-off” or at project’s end Routine and integrated into project management Country specific Knowledge gained available for application by MSH and local technical assistance providers as part of the multiplier effect, including cross fertilization of applications from country to country M&L Self-Assessment, April 23, 2004 8 Lessons Learned 1. Health challenges facing the world are too great to be addressed through interventions or approaches tailored to one organization at a time. It is necessary to develop programs, tools, and interventions which can be applied to more than one organization simultaneously yet still meet their individual needs. 2. While funding directives are often focused on single issues (HIV/AIDS, malaria, child survival, etc.) the organizations which deal with these challenges face a reality which is multi-service oriented. Program interventions and tools and approaches must be flexible enough to be used in more than one specific health service. 3. Local technical assistance is much more available than it was in the past. Often this assistance provides much needed perspectives and expertise. Program interventions are of higher quality when M&L partners with local technical assistance. 4. The role of the international technical assistance agency is becoming better suited as a knowledge manager/provider than as a technical assistance provider. The provision of ways and means to facilitate the exchange, synthesis, and application of good practices, lessons learned, tools, etc. is a preferred role for international (mostly northern) agencies. M&L is moving from being a doer to being an enabler. 5. There is a growing need for rapid assessment, rapid start-up, and rapid institutional capacity building. The days of working with one organization over 2–6 years are waning. M&L has begun to develop ways to provide rapid assistance which is coupled with short-term TA and medium term organizational development through low-cost Web-based management and leadership programs. 6. While insufficient funding and the resultant sustainability questions were of paramount importance in the past, and are still important, today in many countries there is the challenge of absorptive capacity, especially in the HIV/AIDS area. Good leadership and management systems are essential for sustainability not only for organizations which are dependent on single sources of revenue, but for organizations which now must rapidly transform themselves to substantially increase services. M&L Self-Assessment, April 23, 2004 9 II. ACCOMPLISHMENTS AND LESSONS LEARNED A. What are the primary accomplishments and lessons learned from M&L’s work in Strategic Directions 1–4? Strategic Direction 1: Developing Capacity of Individuals and Teams to Lead and Manage The M&L program established the following objectives for this Strategic Direction (SD): By 2004, we will see: • Six hundred health managers and/or practitioners around the world trained to practice effective leadership and management skills to achieve and sustain improved service indicators in their organizations. (By 2005, we will reach 800 managers.) • Six FP/RH service delivery organizations able to respond more effectively to changing conditions. These organizations will be characterized by favorable work climate and continuous performance improvements. They will have processes in place to continuously develop managers to lead. (By 2005, we will reach 12 organizations.) • Six hundred health managers from organizations around the world who maintain effective leadership and management practices in the face of competing demands; they will be able to access ongoing support and learning through virtual and face-to-face formats. (By 2005, we will reach 800 managers.) M&L has accomplished the objectives set for 2004. Health managers at central, regional/provincial and district levels from FP/RH service delivery organizations have had an opportunity to develop their leadership and management skills by focusing on real organizational and service delivery challenges facing them. The objectives have been achieved through country-based, face-to-face programs such as the Egypt and Nicaragua Leadership Development Programs as well as regionally based blended￾learning programs, such as the Virtual Leadership Development Program. Background information and a description of results achieved are available in materials received by the Assessment Team, as follows: Egypt Leadership Development Program—The Leadership Development Program of Egypt was a one￾year pilot program co-led by the Ministry of Health and Population (MOHP) and M&L. Funded with core funds, the program was implemented between June 2002 and June 2003. Its purpose was to demonstrate how to improve the quality and accessibility of FP and maternal health services in three districts of Aswan Governorate, and to increase the capacity of managers to lead others to achieve results. See Evaluation Note; 2003 Results Review; June 2003 Semi-Annual Report (SAR) Highlight; December 2002 SAR Highlight; October 2001 SAR Highlight. Nicaragua Leadership Development Program—The Leadership Development Program with the MOH began as a core-funded pilot program in PY2. It has been extended widely around the country over the subsequent years, and is now a program “owned” by the MOH. Leadership development fits into the overall effort to reform and modernize the MOH. See December 2003 SAR Highlight; October 2001 SAR. Guinea Leadership Development Program—Responding to the MOPH’s expressed need to “reinforce management skills at all levels of the health system,” PRISM (Pour Renforcer les Interventions en Santé Reproductive et MST/SIDA, USAID’s bilateral program implemented by MSH) launched a leadership initiative in April 2002 with two Leadership Dialogue meetings in collaboration with M&L. These meetings became the basis for M&L’s pilot Leadership Capacity Strengthening Program (LCSP). The LCSP consisted of three workshops conducted over six months in 2002. Participants produced action plans to address their actual, immediate work challenges, and received follow-up visits from Guinean facilitators who doubled as coaches. See Evaluation Note; December 2002 SAR Highlight; October 2001 SAR. M&L Self-Assessment, April 23, 2004 10 Virtual Leadership Development Program (VLDP)— VLDP is a 12-week blended learning approach to leadership development, combining individual work on the VLDP Web site (additionally supported by CDs and workbooks that all participants receive) and on-site team meetings in the organizations. Organizations must enroll in teams and work on actual organizational challenges—the modules are built around these challenges and corresponding action plans. The VLDP is designed to reach multiple organizations and can enroll up to one hundred participants at a time. Since its inception, participants from eight Latin American countries and seven African countries have participated in the program. VLDP can be accessed through the Virtual Center for Leadership and Management. See Evaluation Note; 2003 Results Review; December 2003 SAR Highlight; June 2003 SAR Highlight; December 2002 SAR Highlight; 2002 Results Review. Lidernet—A Web-based support mechanism for alumni of the Brazil Leadership Development Program. See December 2003 SAR Highlight; December 2002 SAR Highlight; 2002 Results Review. LeaderNet4 —A Web-based support mechanism for alumni and facilitators of leadership programs. See December 2003 SAR Highlight. Lessons Learned 1. There is significant need and demand to develop the skills and competencies in managing and leading and organizational performance improvement among personnel at all levels of the health system in the public and NGO sectors. 2. M&L’s Leading and Managing Framework, notably the practical definitions of leading and managing functions and practices, demystifies the concepts and resonates with both cooperating agency (CA) colleagues and health managers. 3. By focusing on real organizational challenges and organizing participants into work teams in their home institutions, participants have an opportunity to immediately apply new knowledge, skills, and practices. 4. The commitment of senior leadership to staff participation in programs such as the VLDP and country￾based leadership development programs is absolutely critical to success, both during the programs and afterwards. Nevertheless, the continuing demands of “regular work” will always impinge upon some participants’ ability to focus and continue to practice new skills and competencies. Other lessons learned include: 5. Managers who lead foster motivation, team spirit, and the commitment to perform and innovate. In order to sustain knowledge and skills, participants require continuous challenges, feedback and support, whether from their own organizations, or from such mechanisms as Lidernet and LeaderNet that are designed to provide such support. 6. Work teams that normally work together (“intact” teams) as opposed to teams organized for the purposes of a blended-learning course, are a more effective mechanism to sustain practices and performance. We have therefore added a criterion for “intact teams” to our leadership development program offerings. 7. There is tremendous value in creating shared products and processes, that is, in collecting the shared learning and experience from facilitators around the world to create better programs. 8. Action plans are an effective mechanism to structure, monitor, and implement the organizational challenge chosen by the team. Most teams require more assistance than is planned in courses to define measurable objectives, to track performance indicators, and to analyze and interpret results. In response to this finding, Monitoring and Evaluation Unit staff assist program facilitators to review and strengthen action plans. 4 Lidernet is a Web-based support mechanism for alumni of the Brazil Leadership Development Program. LeaderNet is a Community of Practice that provides opportunities for ongoing learning and experience for managers who lead and facilitators of management and leadership programs. Members can connect, develop, and gain support through the LeaderNet Web site, email, print, fax, CD-ROM, phone as well as through face-to-face meetings. M&L Self-Assessment, April 23, 2004 11 Strategic Direction 2: Improving Management Systems Organizations must have strong management systems in order to deliver high-quality FP/RH and other health services and to enable managers and service providers to perform effectively. Strong systems are necessary for organizational sustainability. M&L has been delivering technical assistance and training and applying practical tools to address local management problems, strengthening the ability of organizations and programs to: • plan and organize efficient management strategies, structures, and systems; • implement work; • monitor and evaluate to be sure customer needs are met and that the organization is performing and continually learning. The M&L program established the following objectives for this SD: By 2005, M&L client organizations will have and will use effective, transparent, documented management practices and systems to facilitate their work and achieve improved performance. Members of the Technical Cooperation Network (TCNetwork) will also be able to provide technical assistance in improving management systems. Work in SD2 is coordinated with our work in SD1. Developing the capacity of individuals and teams to lead and manage is inextricably linked with improving management systems. All of M&L’s management development activities respond to needs identified in the field, and focus on opportunities to capture best practices and to globally disseminate proven technical approaches and technical resources to improve organizational performance in the delivery of FP and other RH services. M&L is accomplishing this objective. Background information and a description of results achieved by key programs implemented under this SD are available in materials received by the Assessment Team, as follows: Business Planning Program (BPP)—The BPP is a blended learning program designed to enable NGOs/ FBOs to write business plans to attract nontraditional donor funding and to improve their on-going management. Presently PROCOSI/Bolivia and the NGOs belonging to the PROCOSI network have received approximately $345,000 in new funds for their business plans. Funding sources include: GlobalGiving.com; EngenderHealth; Embassy of the Netherlands; Embassy of Belgium; Embassy of Spain; Embassy of Italy; M&L/Nicaragua. Worth noting is that some of these funders represent first-time funding sources for these NGOs. GlobalGiving represents not only a new funding stream, but also offers a completely new funding model: it posts abstracts of the business plans on its Web site and serves to “broker” relationships between organizations looking for funding and ones ready to give funding. See Evaluation Note; 2003 Results Review; December 2003 SAR Highlight; June 2003 SAR Highlight; 2002 Results Review; December 2002 SAR Highlight; October 2001 SAR Highlight Human Resource Management Assessment Tool adapted for HIV/AIDS Contexts—See October 2001 SAR Highlight. A copy of this resource material was provided to the Assessment Team. Family Life Education Program (FLEP), Uganda—FLEP was a project of the Busoga Diocese of Uganda, a multiservice RH agency that operated in five Districts of Uganda, two of which fell within the scope of the Delivery for Improved Services for Health (DISH II) project funded by USAID from 1999–2002. In response to findings and recommendations of assessments commissioned by DISH II, M&L was invited in June 2001 to facilitate an integrated strategic planning and organizational assessment and to provide TA to help FLEP determine its strategic directions and to streamline its operations. See Evaluation Note; 2002 Results Review; October 2001 SAR M&L Self-Assessment, April 23, 2004 12 Management and Organizational Sustainability Tool (MOST)—Developed under the predecessor FPMD II project, MOST is a participatory management diagnostic tool that enables managers in NGOs and MOHs to develop a management capacity profile for their organization and a prioritized action plan for improvement.5 Based on feedback from MSH facilitators on the process as well as a follow-up review of the use of MOST in five countries since FPMD II, the MOST facilitator manual was revised. A copy of this resource material was provided to the Assessment Team. Health Manager’s Toolkit—Initially developed under FPMD II in collaboration with Family Health International (FHI), this global electronic resource continues to provide health manager’s around the world with field-tested, practical tools to address FP/RH and other service delivery challenges. See Evaluation Note. Financial Management Assessment Tool (FIMAT)—The most recent issue of The Manager focuses on financial management and includes the FIMAT which is based on experience under FPMD II and in the Egypt bilateral (POP IV) applying a “MOST-like” process to determining the stage of development of all components of public and NGO-sector financial management systems. Human Capacity Development (HCD) Initiative—M&L is a key contributor to USAID’s HCD Initiative with core funds from the Office of HIV/AIDS. In collaboration with other CA partners in the HCD Initiative, M&L developed a framework assessing overall human capacity development needs—including leadership, policy and legislative recommendations—across sectors and within individual organizations and programs. TA has been provided to improve management of the health care workforce and address human resource issues (staff shortages, inadequate supervision, etc.) caused by the epidemic in several countries in Africa. See December 2003 SAR Highlight; 2002 Results Review Lessons Learned 1. Human resource management is now recognized as a critical management system that has been too long ignored (apart from strengthening supervision systems). The effective use, deployment, and retention of employees at all levels of the health system are as essential to service delivery as is the availability of commodities. 2. To respond to the increasing demand for FP/RH services as well as AIDS, TB, and other infectious diseases there is significant need and demand for technical assistance in public and NGO sector organizations in all management areas: organizational capacity in mission development or revision; strategic and operational planning; review organizational structure to implement strategic and operational plans; build and adjust management systems (planning, information management, human resource management, quality assurance, financial management, revenue generation, and logistics management). 3. As the external environment continues to change more frequently and often dramatically due to burgeoning diseases, withdrawal or rapid growth of donor funding, and shifting donor or government strategies, the capacity of organizations to adjust strategy, structure, and management systems independent of external technical assistance has become more critical. M&L’s principle that positive changes in commitments and practices are sustained when they are part of the organization’s routine systems has been validated many times over. Procedures and protocols for management systems must be well documented and managers trained in their use and adjustment in order to assure sustainability. Other lessons learned include: 5 By the end of PY5 there will be a “Suite” consisting of the primary MOST instrument and specialized in-depth MOST in three areas: Human Resource Management, Financial Management (FIMAT), and Health Information Systems. M&L Self-Assessment, April 23, 2004 13 4. The absorptive capacity of the counterpart organization determines how many management systems may be developed at the same time. 5. Effective management systems are essential for the delivery of health services. The HIV/AIDS epidemic places increasing importance on the following systems: human resource management, logistics management, management information systems, and quality assurance. 6. The past publications of MSH on management development (FP Manager’s Handbook, numerous issues of The Manager, Lessons Learned series on strategic planning and management information systems) are as relevant today as when they were published. They contain practical approaches and tools that may be introduced and adapted to the local context. Strategic Direction 3: Partnering Locally for Sustainability The purpose of Strategic Direction 3 is to extend the impact of USAID’s M&L program. This is being achieved by building a network of strong southern TA agencies with virtual connections, strengthening their technical capacities, and fostering strategic partnerships through the implementation of the Technical Cooperation Network (TCNetwork). The TCNetwork is a global community of technical assistance providers committed to mutual support, accountability, and excellence. It increases exposure for talented local consultants, providing greater opportunities and recognition. Members of the TCNetwork have access to many resources including Web links to donor sites offering new program and procurement information. The TCNetwork also offers high quality tools, training opportunities, and chat rooms where collaboration and information sharing can take place. See the TCNetwork Web site at: www.tcnetwork.net. M&L is also building on the sustainable performance improvements demonstrated by a set of “graduated countries” through the Global Exchange on Reproductive Health project. The Global Exchange is an initiative to promote the exchange of information and best practices among countries which have “graduated” from USAID population assistance. It will also enable members to access information from the global community. The current countries are Colombia, Ecuador, Mexico, Turkey, and Morocco. This network, supported by a Web site, will be managed by its members. The M&L program established the following objectives for this SD: By 2005, M&L will have launched a range of practical partnering opportunities to multiply the impact of our program. The TCNetwork, will be adding measurable value to its members, and donors and client organizations will recognize TCNetwork membership as an indicator of high-quality technical assistance and accountability. By June 2004, we expect to have at least 25 members in the TCNetwork and 40 members by June 2005. Through access to the resources of the Virtual Center for Leadership and Management (VCLM)6 , M&L staff, consultants, and TCNetwork members will be able to incorporate and deliver approaches and know￾how and to develop strategic partnerships among themselves. Managers in “graduated countries” will be able to build on innovations and successes, share information accurately and efficiently, and receive timely information on approaches and technical advances that improve service delivery. The TCNetwork is on track to achieve its 2004 objective: There are currently nine initial members, two new members, with 30 applications being processed. See December 2003 SAR Highlight and October 2001 SAR for additional information. 6 The VCLM is a Web-based center that integrates and supports M&L’s electronic information resources, online communities of practice, and virtual programs. It provides easy cross-links for members of the TCNetwork, the Communities of Practice, LeaderNet, Global Exchange on Reproductive Health, the Electronic Resource Center, the VLDP, and the Business Planning Program. M&L Self-Assessment, April 23, 2004 14 Centre for African Family Studies (CAFS)—CAFS was named as M&L’s formal partner in its proposal to USAID. This partnership has not worked out as planned for a variety of reasons, including CAFS’ absorptive capacity and consistently overloaded workplan. Over the years, M&L has allocated fewer financial and human resources to the CAFS partnership. Our chief lesson learned is that the partnership worked very effectively when CAFS was able to work side-by-side on a discrete field project. Our work with FLEP/Uganda is a good example. Through this partnership, CAFS brought its strategic planning expertise while the MSH facilitator introduced the MOST process and the HRM Assessment Tool. Together, M&L and CAFS provided FLEP with the package of technical assistance it needed to make significant progress with its organizational challenges. This approach to knowledge transfer has been more effective than periodic face-to-face meetings or events to share tools, lessons learned, etc. See October 2001 SAR. Global Exchange on Reproductive Health—Focus groups implemented by M&L in 2003 confirmed the need and interest to remain part of the global international FP community. For further information see: December 2003 SAR Highlight Lessons Learned 1. Now and in the future, management and leadership challenges will benefit significantly by drawing upon the skills, experience, and local expertise of technical assistance providers around the world. 2. The use of core funds to develop self-governing networks is a timely and much needed approach to strengthening collaboration across regions among TA providers and “graduated countries.” The use of new information technology to share good practices, FP/RH information, and relevant aspects of consulting and management and leadership is a timely and cost-effective undertaking. 3. Focusing on TCNetwork and Global Exchange sustainability from the outset tests and strengthens commitment of members and ensures value-added of the Networks to clients. 4. CAFS: The benefits of partnership felt so strongly in the initial stages of planning and implementation must be continually revisited as the rationale and urgency may change over time. Learning must be focused on field-based opportunities to work together. 5. When “graduating” from donor assistance, ministries of health must devote resources to periodic meetings of key FP/RH stakeholders and service delivery organizations, an activity that donors normally fund. In this way, achievements and lessons learned may continue to be shared among peers. Strategic Direction 4: Capturing and Applying Knowledge Capturing and applying knowledge means organizing activities to produce and manage knowledge gained in M&L programs—sharing “what we know and how we know it” for improving management and leadership practices worldwide. The M&L program established the following objectives for this SD: To enable internal and external health professionals to access and apply evidence-based knowledge to inform their strategies for improving management systems and work climate in health programs. By 2005, M&L will capture, synthesize, share, and disseminate knowledge that contributes to improvements in FP/RH service delivery. M&L Self-Assessment, April 23, 2004 15 Our efforts in Strategic Direction 4 are to measure the impact of M&L interventions in the three intermediate results of the M&L program: improved performance of management systems, improved performance of leaders and managers, and an improved capacity to respond to a changing environment. The M&L Program has made many significant accomplishments to date. Background information and a description of results achieved by key programs implemented under this SD are available in materials received by the Assessment Team, as follows: The December 2003 Management Review—This document provides a comprehensive overview of our key strategies and results in: Planning Monitoring & Evaluation, Special Studies, and Knowledge Management. In-depth evaluations—Various in-depth evaluations have been completed: Business Planning Program/PROCOSI, Egypt, Guinea, Guatemala, Health Manager’s Toolkit, Peru, Uganda/FLEP, Nicaragua Leadership Development Program, and the Virtual Leadership Development Program I. Results are summarized in the Evaluation Notes provided to the Assessment Team. MSH Publications—An in-depth evaluation has recently been undertaken on MSH Publications including issues of The Manager funded by M&L. Future In-depth Evaluations—Will be completed or undertaken in 2004 on the Nicaragua Leadership Development Program, follow-up of the Egypt Leadership Development Program since M&L’s withdrawal in June 2003, interim and final evaluations of the M&L-PRIME joint project in Armenia, cross-Program review of the achievement of performance objectives by M&L field-based clients; and the Mozambique program (if it ends in September 2004). Communications Products—M&L has developed a number of print and electronic materials providing an overview of the Program, its technical capacity, and results achieved. See M&L’s Web site at http://www.msh.org/projects/mandl. Several sample communication materials were provided to the Assessment Team. MSH-PAHO-GHC Technical Seminars—M&L continues to participate annually in these seminars. See October 2002 Results Review and October 2001 SAR. Lessons Learned 1. Monitoring and evaluation need to be built into programs during the design and budgeting stages so that they become an integral part of the program. Monitoring progress should be the responsibility of managers implementing a program so that adjustments or refinements may be made during program implementation. Evaluations should be done by those not involved in implementing the program. A separate M&E unit should be available to partner with program staff to develop monitoring plans and indicators, and to design and carry out evaluations. An objective view of program achievements and impact is critical to extracting the lessons learned. 2. Core funds are required to build and sustain an effective internal mechanism for technical support to program staff in monitoring program progress and results, and for maintaining and refining a centralized database to store information. Missions will not pay for this capacity or system. Core funds are also frequently needed to conduct in-depth evaluations of country programs. Missions prefer to use field support funds to support the implementation of activities and some monitoring, not the in-depth evaluation. 3. It is easier to show the link or association between M&L inputs and service delivery outcomes when the management and leadership interventions are focused at the district level and below. It is not M&L Self-Assessment, April 23, 2004 16 impossible to demonstrate an association at the provincial level and above, however the challenges work teams focus on need to be well defined, related to service delivery, and more time is needed for implementation so that results “trickle down” to the service delivery level. 4. Public and NGO sector organizations need assistance in defining “SMART” objectives, effective performance indicators, data collection, analysis, use of data, and reporting. This is especially true in light of the massive funding for HIV/AIDS and the requirements of funding mechanisms such as the President’s Emergency Plan. The M&E Unit therefore needs to allocate more human resources to providing field-based TA in strengthening management information systems and use of data for decision-making. 5. Knowledge must be synthesized or it will not be used. There is too much information, too many tools, etc. Technical assistance providers are too busy to read all there is even within their own organizations. Without easy access to synthesized knowledge reinventing the wheel will likely continue. B. What are the primary accomplishments and lessons learned from the investment of field support funds? Over the course of implementation of the M&L Program and to date, we have worked in 18 countries and with three regions/Bureaus with field support funds. One of the key accomplishments of the M&L Program is the substantial increase in field support funding received. Field support has increased from $1,230,000 in Federal Fiscal Year (FFY) 2000 to $13,647,484 in FFY 2003. This indicates both that management and leadership is immediately relevant to field needs and that the Missions feel that M&L performance has been very good. It is difficult to briefly summarize the primary accomplishments of the numerous and complex field programs supported with field support funds. Background information and a description of results achieved are available in materials received by the Assessment Team. Brief summaries of field support programs and references to further information follow: Afghanistan—An example of M&L and MSH’s capacity to respond rapidly to USAID, Missions, and client needs is seen in Afghanistan. Funding through M&L allowed for the rapid start-up as well as for rapid results as USAID initiated its response to the health needs of this country emerging from war. M&L’s work preceded the development and award of the Mission’s current bilateral project, Rural Expansion of Afghanistan's Community-based Healthcare (REACH) being implemented by MSH. Descriptions of the accomplishments may be found in the following documents: June 2003 SAR Highlight; December 2002 SAR Highlight; 2002 Results Review. Angola—This program was initiated in PY4 as the mission formulated its strategy to respond to the emerging AIDS crisis. M&L has provided key support to USAID/Angola and the Angolan Government by carrying out a management and leadership needs assessment of the Angolan National HIV/AIDS Program; providing TA to the World Bank in developing the institutional arrangements and a monitoring and evaluation framework for the Angola Multi-Country AIDS Program (MAP); and preparing a summary of the National HIV/AIDS strategic plan to be disseminated nationally. Preparations are underway for a series of workshops to strengthen HIV/AIDS leadership and operational planning at the provincial level. Bolivia—MSH has a long experience working in Bolivia since FPMD I. The focus of M&L technical assistance with field support funds has been with two clients: PROSALUD and COMBASE, a faith-based organization. See October 2001 SAR (PROSALUD); October 2001 SAR and December 2003 SAR Highlight (COMBASE). Brazil—M&L first implemented a country-based leadership development program using electronic technologies to deliver the course and to provide follow-up support to alumni (Lidernet). DFID took over M&L Self-Assessment, April 23, 2004 17 funding of the leadership development program when Brazil was no longer eligible for USAID population funds. The content of the course has since been adapted and transferred to Mozambique through a south-to-south exchange. Brazil also marked the first time that M&L expertise was tapped to address management challenges in HIV/AIDS and TB. Information on accomplishments may be found in: December 2003 SAR Highlight and October 2001 SAR. Ghana—In another example of a core funds investment leading directly to Mission field support, M&L conducted two workshops demonstrating the Leading and Managing Framework and the Human Resource Management (HRM) Assessment Tool for the staff of the Human Resources Division, Ghana Health Services, MOH. As a result, the Mission requested M&L to conduct a comprehensive assessment of the HR issues facing the Ghana Health Services. The Mission approved the resulting recommendations; however no further work was conducted due to lack of response from the HR Division or the USAID Mission. See October 2001 SAR Guatemala—M&L’s main client has been APROFAM, with which MSH has been working since the 90s. Key achievements of M&L’s TA may be found in: Evaluation Note; December 2003 SAR Highlight; October 2001 SAR. Honduras—In PY2 and 3, M&L implemented a small TA program with ASHONPLAFA, the IPPF affiliate. Honduras is one of the countries where M&L successfully introduced and applied its Performance Improvement approach at the organizational and systems levels. See the October 2001 SAR Highlight for information. Indonesia—This program is another example of the use of core funds to respond rapidly to the Mission’s need for immediate assistance, followed by substantial field support investments. The Indonesia program is large and complex, with a well developed field presence working side-by-side with the MOH and the National Family Planning Coordination Board. Accomplishments are summarized in: December 2003 SAR Highlight; June 2003 SAR Highlight; October 2001 SAR Highlight. Malawi—M&L recently placed a Resident Technical Advisor in the HIV/AIDS Unit of the MOH. This is a example in the M&L portfolio of a Mission using the cooperative agreement mechanism to strengthen the management and coordination capacity of a MOH to implement a large, complex, non-USAID, Global Fund-funded program. Mozambique—At the request of USAID/Mozambique, FPMD II conducted a series of health finance studies in preparation for the development of the MOH strategic plan. Following this FPMD II conducted management capacity assessments at the provincial level as part of the Ministry’s decentralization effort. MOST was adapted for use in the public sector and FIMAT was developed. Both “MOSTambique” and FIMAT were adopted as official MOH tools and the health finance studies continue to be considered foundational documents for the health sector. Based on this work, M&L was invited to return to Mozambique in 2002 to develop a program to strengthen MOH leadership and management capacities at all levels of the health sector, including national, provincial, district, and clinics. This program includes coordinated applications of MOST and leadership development. M&L also provides management development to the MOH malaria and child survival programs. See 2003 Results Review; December 2003 SAR Highlight; June 2003 SAR Highlight. Nicaragua—Another comprehensive program, Nicaragua began with a combination of field support for assistance to PROFAMILIA and a pilot Leadership Development Program supported initially with core funds. Today M&L’s work in Nicaragua is nearly fully funded by field support. The Mission looked to M&L to continue to implement its highly successful bilateral program, PROSALUD, implemented by MSH. For information on PROFAMILIA, see October 2001 SAR. For information on the Leadership Development Program, see June 2003 SAR Highlight, 2002 and 2003 Results Reviews. Nigeria—M&L is working with the National Primary Health Care Development Agency in support of its strategic plan and to re-engineer its organizational structure, financial, human resource management, and programmatic service statistics systems. As of this writing, recommended improvements in the financial management systems are being implemented. Human resource activities are focusing on the development and use of a performance-based planning, monitoring, and evaluation system for field staff. A three-month pilot project to gather service statistics was initiated in October 2003. M&L Self-Assessment, April 23, 2004 18 REDSO—The Regional Office for Eastern and Southern Africa (REDSO) strategic plan has a strategic objective to strengthen the capacity of three regional partner organizations. M&L works with three organizations: the Commonwealth Regional Health Community Secretariat (CRHCS); the Regional Centre for Quality Health Care (RCQHC); and the Centre for African Family Studies (CAFS) on management challenges, including strategic planning and strengthening their financial and information management systems. Latin America Health Sector Reform Initiative (LACHSRI)—M&L has been a partner in this initiative since FPMD II. The initiative has allowed M&L to make several practical tools available to a wide range of Latin American MOHs who are struggling with decentralization and other health sector reform initiatives. This includes MOST and the Decentralization Mapping Tool (DMT). Annual regional workshops have provided the means for countries to share experiences, challenges, and lessons learned in a structured forum. Many countries have continued their conversations and exchange of information following the workshops through e-mail listservs established by participants. See October 2001 SAR. Peru—With one small earmark from the Mission in PY1, M&L successfully introduced its Performance Improvement approach to address several key challenges at the central level of Manuela Ramos. See Evaluation Note. Tanzania—This is another large, well-funded, comprehensive, and complex program. M&L’s work is the direct result of a management needs assessment undertaken in FPMD II of the moribund national AIDS commission. That assessment led to the creation of TACAIDS (the Tanzanian AIDS Commission). M&L has worked on management strengthening with TACAIDS since its inception and in 2003, placed a Resident Advisor in Tanzania. In addition to capacity building of TACAIDS, M&L has assisted in proposal preparation for two rounds of the Global Fund. In collaboration with Deloitte & Touche, M&L developed and implemented an innovative mechanism—Rapid Funding Envelope—that allows for the rapid distribution of Global Fund resources with complete transparency and accountability. With M&L support, Tanzania has won significant funding—including $87 million from the Global Fund and $70 million from the World Bank—to roll out large HIV/AIDS programs. The lessons learned on the importance and challenges of national, cross sector coordination from Tanzania are being shared in new publication: “Going to Scale in Planning Multisectoral HIV/AIDS Programs” (published in April 2004, supported with funding from the USAID/W Office of HIV/AIDS). For additional information on key achievements to date, see: December 2003 SAR Highlight; December 2002 SAR Highlight; October 2001 SAR. Turkey—M&L assistance to the MOH and Social Security Organization ended in March 2002 when USAID graduated Turkey from population assistance. M&L successfully closed-out its assistance in the development of key management systems to these two counterparts. M&L also documented and disseminated to USAID Missions the results of the rapid assessments of quality in USAID/Turkey focus provinces through its Quality Surveys. The assistance provided to USAID/Turkey in developing its country monitoring and evaluation plan, and the methodology developed for the rapid quality surveys were published by MSH and MEASURE Evaluation in Best Practices in Monitoring and Evaluation: Lessons from the USAID Turkey Population Program (Mathis, Jill et al., October 2001), and Monitoring Quality of Care in Family Planning by the Quick Investigation of Quality (QIQ) (Sullivan, Tara M. et al, Editors, MEASURE Evaluation, July 2000). See 2002 and 2003 Results Reviews; October 2001 SAR. Uganda—M&L began work in PY3 with an initial $200,000 earmark. From that small starting point, the program has grown steadily to a funding level of approximately $5 million, including substantial funding for grants to NGOs and FBOs from the President’s Emergency Plan to be dispersed by the Inter-Religious Council of Uganda (IRCU). In addition to organizational capacity building and grants management with IRCU, the M&L portfolio includes working with the Global Fund to establish its Project Management Unit in the MOH and reviewing the Uganda AIDS Commission’s strategy. M&L is also adapting the MOST for the National TB and Leprosy Program to improve the national laboratory program (NTLP) and for the Joint Clinical Research Centre (JCRC) for use by AIDS labs. See the December 2003 SAR Highlight. M&L Self-Assessment, April 23, 2004 19 Lessons Learned 1. M&L’s ability to rapidly start-up technical assistance activities is greatly facilitated when we have had a past presence and positive reputation among our counterparts in the country and at the Mission. This is the case for most M&L field support-funded programs, except Angola, Ghana, Indonesia, Nigeria, and with REDSO: • Angola—M&L’s CTO corresponded with the Mission Officer regarding M&L’s ability to respond to Mission needs. Through the use of forward funding with core funds, followed by the field support earmark, M&L was able to get on the ground quickly. • Ghana—Initially core funded, Ghana was one of the countries identified by M&L as a priority. Correspondence with the Mission led to an invitation and subsequent field support funds. • Indonesia—One reason the Mission sought M&L assistance was the Health, Population and Nutrition (HPN) Officer’s knowledge of M&L’s work from assignments in other countries. Moreover, the HPN Officer and other CAs working in Indonesia recognized MSH’s technical expertise in managing decentralization from the numerous MSH publications on the topic. • Nigeria—The request for assistance in Nigeria was received through the BASICS II project, in which MSH is a partner. • REDSO—Since the REDSO strategy had a Strategic Objective for capacity building, it saw a compelling and logical connection to invite M&L to work with their three regional partner organizations. 2. The organizational performance and contextual challenges facing all of the field support-funded countries are numerous and varied but can be categorized by region. Latin America—Public sector programs are challenged by health sector reform, notably decentralization (Brazil, Nicaragua). The need for developing the leadership and management capacity of individuals, workgroups, and organizations is critical. For NGO sector organizations, the principal challenge is sustainability in response to the withdrawal of donor support and increasingly, governments providing FP and other RH services for free. This is the case for NGOs in Bolivia, Guatemala, Honduras, Nicaragua, and Peru. The response needs to pay attention to the efficiency and effectiveness of key management systems, including strategic and operational planning, human resource management, quality assurance, financial and information management, revenue generation, and NGOs’ ability to market their unique, high quality services. Africa—Throughout the continent, less well developed and under-funded public and NGO sector organizations struggle to face significant challenges. While coping with environmental factors such as International Monetary Fund (IMF) structural adjustments, health sector reform, decentralization, changing donor priorities and uncertain future funding, the magnitude of the AIDS pandemic hit with full force. The already existing need for assistance in leading and managing skills and management systems was magnified to deal with the massive influx of donor funding to prevent, treat and care for HIV positive patients and the public at large (Uganda, Tanzania, and Malawi). Asia—Although MSH has a long history in Afghanistan, our experience there is one of starting over to build the health infrastructure from A to Z, from policy to the basic package of primary health services the government will provide, to informing donors of geographic areas in desperate need of health facilities. Indonesia, on the other hand, with its model of decentralization at the “most extreme” may be characterized by the urgent need to maintain the gains of the primary health care system over the past 30 years. The system is jeopardized as it has been fractured by decentralization due to the lack of clarity about roles and responsibilities at the various levels of the health system. As importantly, local politicians are disinclined to invest their scarce resources in public health as it does not bring them easily apparent political advantage. M&L Self-Assessment, April 23, 2004 20 To meet these varied challenges, a broad range of technical expertise is required and must be deployed. MSH has the necessary broad range of technical expertise, including local TA partners, which M&L successfully draws upon. 3. M&L is able to achieve greater results and impact if multiple years of funding are foreseen by the Mission at the time project activities are designed. The best examples include Indonesia, Mozambique, Tanzania, and Uganda. With the assurance of continued field support funding, technical assistance was placed in the field, rather than relying on short-term temporary duty (TDY) assignments which are less effective for rapid and sustained capacity development of NGO and public sector organizations. 4. M&L must continually maintain flexibility in program design in the event that Mission country strategies change, and be prepared to close-out early. This means that the fifth principle (positive changes in commitments and practices are sustained when they are a part of the organization’s routine systems) must always be present and on the top of M&L’s agenda (Brazil, Indonesia, and Turkey). 5. Technical approaches which worked well in one country can be successfully adapted and introduced to another country. Examples of this transfer include: the competency-based approach to leadership development from Brazil to Mozambique; the performance-based grants mechanism from Haiti to Afghanistan; various tools and methodologies developed under USAID’s Rapid Pharmaceutical Management program to Indonesia; district-level capacity building approaches from the Philippines to Indonesia; and successful leadership development programs from Nicaragua and Egypt to Kenya and Senegal. 6. The most effective coordination and cooperation with other CAs occurs at the field level where plans, challenges, and lessons learned in working with counterparts can be continually exchanged. C. How is M&L building sustainability of technical interventions and programs? There are several fundamental principles based on years of experience and that of CA colleagues which M&L applies, irrespective of whether the counterpart is a public or NGO sector organization. These include: • M&L’s work must respond to an immediate felt need of the counterpart organization. In the past and in instances where the Mission solely, without the collaboration of the counterpart, has identified the primary problem (“go fix this” syndrome) M&L’s work has been less successful and sometimes, regrettably, has not been sustainable. • Continuously ensure the commitment and engagement of the counterpart, especially at the senior level, throughout the course of program implementation. A one-time “ok” from the Executive Director is not sufficient, as priorities and senior personnel may change. This is most often true in the public sector, but also in NGOs where changes occur in senior leadership. For example, changes in executive leadership in Nicaragua (PROFAMILIA) and Honduras (ASHONPLAFA) have impacted progress in the implementation of activities and their results. The importance of senior management commitment is a critical success factor in MOST. The revised manual contains a new section devoted to this. • Consolidate consensus on the best mechanisms to sustain program interventions from the inception of program design and implementation. This principle is seen in several examples: the MOH in Mozambique continued to use the MOST (MOSTambique) after the departure of FPMD II; the institutionalization of the Nicaragua Leadership Development Program in the MOH, based on the leadership of its HR Director, and later the Minister and Secretary General of Health. M&L Self-Assessment, April 23, 2004 21 • The replication of the Leadership Development Program to additional districts in Aswan Governorate, Egypt, without technical assistance or financial support from M&L or any other CA/donor is an example of M&L’s fifth principle, ensuring that our collaborative work is part of an organization’s routine systems. The program is also being replicated in other governorates with UNFPA funding, again without M&L technical assistance. This development underscores that the program is not solely about performance improvement—it is about leading performance improvement, which means getting commitment and inspiration to carry on without M&L and without external donor support. Other good examples of this principle in action include ongoing M&L work with management systems in Bolivia (PROSALUD, COMBASE), Guatemala (APROFAM); and Indonesia (drug management and the issuance of Ministerial Degrees on Essential Public Health Functions and Services). M&L has pursued several strategies to build sustainability of its work: • In Brazil, the success and effectiveness of the LiderNet program sparked the interest of another donor, DFID, which is funding the program. • M&L has implemented a deliberate strategy of building strong working relationships with the initial recipients of the Business Planning Program (BPP)—PROCOSI/Bolivia and the Ghana Social Marketing Foundation (GSMF). These two organizations are M&L Program Partners, capable of replicating and delivering the BPP in their respective regions, charging tuition to cover the operational costs of making the Program available. PROCOSI is currently delivering the program to NGO members of NicaSalud/Nicaragua. GSMF has just completed the delivery of the Program to a second internal BPP team; they are currently in discussions with AWARE-RH/West Africa to offer the BPP to a number of regional institutions during the second half of 2004. • Another example of a deliberate strategy was the development of the TCNetwork. Based on initial research of key success and failure factors for networks, M&L determined at the outset that the TCNetwork should be a self-developed and self-governing “virtual” global network. Therefore, M&L devoted the necessary time to check the validity of the idea through face-to-face meetings with local consulting firms and to work with TCNetwork members to develop governance procedures. Their ownership will ensure that the TCNetwork continues and succeeds in its efforts to become a sustainable, independent organization that will continue beyond the life of M&L. • Partnering with private sector organizations which will have a continuing presence in-country is yet another strategy. An example of this is the partnership with Deloitte Touche in the implementation of the Rapid Funding Envelope mechanism for distribution of Global Fund resources in Tanzania. • Another effective strategy employed is the integration of a pilot program into an existing bilateral. The most recent example is the replication and roll-out of the Guinea Leadership Capacity Strengthening Program in the PRISM Project, USAID’s bilateral FP/RH program implemented by MSH. • Finally, “commoditization” through print and electronic publications is an effective means to both sustain M&L work and scale-up. As always, issues of The Manager are based on real field experiences. The forthcoming publication, “Leading and Managing at All Levels: A Handbook for Improving Health Services,” will document and disseminate practical tools and knowledge collected from M&L’s varied experiences with developing managers who lead in the field. The forthcoming “Facilitator’s Guide to Leading Performance Improvement” is another example of taking an effective approach, documenting it, and adding guidance for facilitators so that skilled facilitators with organizational development and management experience, anywhere in the world, will be able to apply this approach. M&L Self-Assessment, April 23, 2004 22 D. How effectively is M&L managing and disseminating knowledge? M&L defines Knowledge Application as the explicit sharing of what we know, how we know it, and how it can be applied for improving management and leadership practices worldwide. The M&L Program and MSH have developed a diversified approach for managing and disseminating knowledge internally and externally: • MSH Institutional Memory—Developed under FPMD II with USAID support, Institutional Memory provides all MSH employees access to MSH's documents and other internal information resources. It is a powerful reference for lessons learned, successes, failures, and information gathering. M&L routinely submits its documents to Institutional Memory so that all MSH staff can access them. Institutional Memory is fully supported by MSH and managed by the MSH Information Center. • Staff development—In PY1 and PY2 M&L developed staff knowledge and skills about the phases, tools and approaches of effective consulting and programming using the Performance Improvement Framework through the development and implementation of the "Well-Rounded Consultant” and "Programming for Results" workshops. These workshops were subsequently merged into a program called “Consulting for Results” which was offered to other MSH technical staff, with MSH corporate support. The program was also delivered to CAFS consultants. All materials have been turned over to MSH’s Human Resource and Administration Department so that MSH may offer the program periodically to new MSH professional staff. Additionally, M&L has adapted Consulting for Results for independent consultants and is offering it twice during PY4 for Partners in Population and Development. Members of the TCNetwork are attending as “facilitators in training” so they in turn can offer the course in a model similar to the BPP. TCNetwork members are organizing courses in Nigeria, the Philippines, and India. • The Manager’s Electronic Resource Center (ERC)—Available on the Web at http://erc.msh.org, the ERC is a tri-lingual electronic information resource that shares management experience through an international network of health professionals. The ERC Web site offers over 10,000 pages of quick-loading information and more than 150 ready-to-use management tools, in English, French, and Spanish. The ERC has an average of 21,000 unique visitors per month who visit at least once for a total of 40,000 visits per month. This resource was also developed under FPMD and is fully supported by MSH. • The Health Manager’s Toolkit—A key feature of the ERC, the Toolkit is located at: www.erc.msh.org/toolkit. The Toolkit is an electronic compendium of 57 (as of this date) practical, proven tools designed to assist health professionals at all levels of an organization to provide accessible, high-quality, and sustainable health services. Collected from 23 USAID-funded Cooperating Agencies and international PVO/NGOs, the site includes resources in 10 categories, including leadership and management development, quality assurance, gathering and analyzing data, guidelines for improving organizational performance, and self-assessment tools for evaluating management systems. The average number of user sessions per month in calendar year 2003 was 1,200. However, in January 2004 there were 7,310 user sessions, with 1,805 “hits” from Latin American and Caribbean countries. An evaluation of the use and impact of the Toolkit conducted by M&L in 2003 confirmed that it is reaching its target audience of health professionals in developing countries. See Evaluation Note. • Print and electronic publications—MSH's publications communicate effective management and leadership practices in the field of international health through a portfolio of books, periodicals, instructional manuals, monographs, tools, and journal articles. MSH's print and electronic publications support managers of health programs and colleagues in international development, to strengthen the performance and sustainability of health care programs and organizations around the world. To date, the M&L Program has developed and published the following: M&L Self-Assessment, April 23, 2004 23 o MOST Management and Organizational Sustainability Tool: A Guide for Users and Facilitators o Human Resource Management Rapid Assessment Tool for HIV/AIDS Environment: A Guide for Strengthening HRM Systems o Management Strategies for Improving Health and Family Planning Services: A Compendium of The Manager Series, Vols. V-IX o Going to Scale in Planning Multisectoral HIV/AIDS Programs (just published in April 2004) o The Manager: six issues to date via print and electronic media in three languages. (See Question E below for further information) o Leading and Managing at All Levels: A Handbook for Improving Health Services (forthcoming, 2005) These publications can be seen at www.msh.org/resources/publications/index.html. Copies have been provided to the Assessment Team. • Communities of Practice (CoP)— The face-to-face and virtual meetings of the CoPs are a valuable mechanism for technical experts to share information and learn from one another on “what works” and “what doesn’t” in the field and within the context of the rapidly changing and highly politicized context of health service delivery. There are four CoPs in M&L focusing on a specific practice area relevant to management and leadership—Performance Improvement, Developing Managers Who Lead, Human Resource Management, and Health Information Systems. Each CoP has two deliverables: a synthesis of internal lessons learned for technical staff in MSH and TCNetwork members, and resources for dissemination to the public, such as Leading and Managing at All Levels: A Handbook for Improving Health Services. The CoPs have provided M&L with valuable lessons on how to approach knowledge synthesis. Background information is available in the M&L Management Review with USAID, December 15-16, 2003, Section 4, pages 28-30. E. How has M&L used core funds to benefit the field? The majority of activities are described in other Sections of this self-assessment. M&L has used core funds to: • Develop, deliver, and refine innovative approaches to developing managers who lead, such as the Virtual Leadership Development Program, the Leadership Development Program in Egypt, and the Business Planning Program. • Leverage the scope of M&L’s involvement beyond focused assistance in specific countries, through the partnership with CAFS, the TCNetwork, and print and electronic publications. • Monitor and evaluate key, strategic, field-based programs, using the results to refine programs and disseminating results to USAID, other Cooperating Agencies, and counterparts through the Evaluation Notes. • Developed a well-functioning Monitoring & Evaluation Unit with a centralized database to store monitoring data collected by program managers. This database is available to most field-based staff. • Participate in several USAID-sponsored global initiatives, such Maximizing Access and Quality (MAQ), Implementing Best Practices (IBP) Consortium, and the Performance Improvement Consultative Group (PICG). Contributions to these initiatives include: effective approaches to the “organization of work”; Leading Change to Adapt and Apply Best Practices; and application of Performance Improvement to management systems and organizational sustainability. M&L Self-Assessment, April 23, 2004 24 • The Manager, MSH’s award-winning quarterly publication provides health professionals at every level with practical knowledge and tools they need to manage their health care programs. The following issues have been developed and translated into French and Spanish with core funds: o Managing Reproductive Health Services with a Gender Perspective o Planning for Leadership Transition o Developing Managers Who Lead o Exercising Leadership to Make Decentralization Work o Creating a Work Climate that Motivates Staff and Improves Performance o Assessing Your Organization’s Capacity to Manage Finances o Crafting Business Plans for Social Return (forthcoming, 2004) M&L recently completed an evaluation of the use and impact of three issues of The Manager through a reader’s survey. The results confirm past assessments of this well-received material: the overwhelming majority of respondents rate the issues as useful, very useful or extremely useful. The Manager is used in a variety of ways including: to increase personal knowledge, as a reference, improve management and leadership skills, improve staff performance, and self-assessment. F. How has M&L used the centrally funded cooperative agreement mechanism to be responsive to the field? Several aspects of the centrally funded cooperative agreement mechanism contribute to M&L’s ability to be responsive to needs in the field: 1. Rapid procurement for Missions without the need to go through the lengthy competitive procurement process. It’s important for USAID to maintain centrally funded cooperative agreements in its portfolio in order to provide rapid, technical assistance to the field. M&L’s program in Uganda, funded by the President’s Emergency Fund is an excellent example of this, as was M&L’s program in Afghanistan. 2. The ability to use core funds to leverage field support funds. The shortage of field support funds in the first year of the Program created the imperative that M&L actively engage with Missions to communicate its capacity to respond to local needs and challenges. Using core funds, M&L was able to get into the field to offer demonstration projects to Missions and prospective clients. These early core fund investments led to field support funding from: Indonesia, Nicaragua, Ghana, and REDSO. By PY3 M&L worked in 18 field support countries and with three regions/Bureaus, seven of which were new buy-ins. The dramatic increase in the percentage of field support is objective evidence of the value of M&L to the field, and demonstrates that M&L understands and can address Mission priorities. Field support expenditures have grown from 13 percent of total expenditures in PY1 to 67 percent in PY4 (as of March 2004), from $624,709 to $8,887,250. 3. Working with multiple countries simultaneously allows M&L to readily draw on similar experiences, promote south-to-south exchanges, and replicate successful programs. 4. Another positive feature of the centrally funded cooperative agreement is the capacity to “forward fund,” that is, to “borrow” from core funding for rapid response or start-up of field programs while awaiting the arrival of field support funds. The core funds are reimbursed once field support funds are obligated. This has been particularly appreciated by Missions/Bureaus including Afghanistan, Angola, Bolivia, Brazil, Guatemala, Honduras, LACHSRI, Malawi, Nicaragua, Nigeria, REDSO, and Uganda. 5. The comprehensive mandate of the cooperative agreement and the relevance of developing management and leadership capacity to all health programs, permit M&L to receive funds from a variety of sources in addition to population, including child survival, HIV/AIDs, and Infectious Diseases. M&L Self-Assessment, April 23, 2004 25 6. In general, a cooperative agreement is more appropriate for a program providing technical assistance and is more flexible than a contract. The substantial involvement of the USAID Cognizant Technical Officer (CTO) allows the cooperating agency and CTO to craft strategies to realign with changing circumstances. Moreover, in these times of heightened security, the cooperative agreement has allowed M&L to continue field work. For example, in October 2002 following the bombing in Bali, Indonesia, M&L was able to continue working in the country, whereas contractors were required to evacuate for more than 6 months. 7. Finally, the cooperative agreement has permitted “earmarks” for M&L participation in global leadership initiatives, such as MAQ, PICG, IBP, “graduated countries,” and other joint USAID-CA undertakings. G. How has M&L designed and implemented programs to complement activities of other CAs? M&L has pursued a diversified approach to ensuring that activities both complement and benefit those of other CAs as well as the field. Some examples are: • Routine coordination on specific country programs. AdvanceAfrica in Angola and Mozambique; FHI in Tanzania and the Caribbean Regional HIV/AIDS Program; Rational Pharmaceutical Management in Indonesia; DELIVER and Partners in Health Reform Plus (PHR+) in Uganda; EngenderHealth in Bolivia; and numerous health and non-health CAs in Indonesia. • Cooperation to produce a coordinated technical vision. Population Leadership Program (PLP); FHI, JHPIEGO, and Synergy Project on the HCD framework; MEASURE Evaluation on indicators for USAID OPRH’s new Strategic Objectives and IRs, and for its Compendium of Indicators for Evaluating Reproductive Health Programs (August 2002); measures numerous CAs in the MAQ Sub-Committees on Management and Supervision, and Organization of Work; and with the Leadership Evaluation Advisory Group (LEAG), sponsored by PLP, on approaches to measuring the results and impact of leadership development programs. • Collaboration with Chemonics in Bolivia to build the capacity of municipalities to determine the cost per case of the Government’s new basic package of health services. See October 2001 SAR. • Work with NGO Networks for Health on the development of the BPP in Bolivia, a seminar on community-based distribution (CBD) in Kenya, and the introduction of Eligible Couple (ELCO) mapping in Uganda (see October 2001 SAR Highlight). • The M&L results framework and principles of developing managers who lead was presented at the Latin America/Caribbean (LAC) PEAK Fellows conference in Honduras, April 2002. See October 2001 SAR. • Collaboration with the Pan American Health Organization (PAHO) for the initial development of a leadership module for its LAC Virtual Public Health Campus. (PAHO has yet to launch this program.) • Work with JHPIEGO, EngenderHealth, and the Policy Project in the design of performance indicators for the Turkey country strategy. With funding and technical support from MEASURE Evaluation, M&L conducted Quality Surveys in Turkey during 1998-2002 to measure the impact of all CAs’ technical assistance on counterparts’ FP/RH programs. • M&L is currently implementing a joint project with PRIME II in Armenia. M&L is focusing on building an effective human resource management system in the MOH while PRIME is improving the performance of front-line providers. • M&L also collaborates with CAs and others through presentations in the U.S. and overseas. For example, see the list of conference presentations in the 2002 Results Review. M&L Self-Assessment, April 23, 2004 26 H. What ideas/interventions did M&L pursue that did not achieve anticipated results and have been dropped? What did we learn from this? There are a few core-funded programs from the initial years of the M&L program implementation that were dropped: • The Management Link (see October 2001 SAR)—A periodic e-mail newsletter on management and leadership topics was dropped since more and more counterparts in developing countries were getting access to M&L resources via the Web. This activity was therefore deemed a duplication of effort. • Community Health_L (see October 2001 SAR)—This listserv was initiated in FPMD II. Although an excellent resource to exchange information and knowledge among peers around the world working in community health, M&L determined that it had run its course and that the focus should be on providing support to program alumni via LeaderNet. • Joint activities with the Packard Foundation and various USAID-funded Fellows programs—Initial coordination meetings with Packard indicated an interest in collaboration; however Packard lost substantial funding and put less emphasis on its organizational effectiveness work. M&L hosted a few Fellows, orienting them to our approaches and tools before they took up their assignments in D.C. and overseas. However, it proved to be far too much work to continually request implementers of Fellows programs to include M&L in their orientation programs. This routine was never institutionalized or “regularized” in the various Fellows programs. • Internal systems—Following the adage “practice what we teach,” M&L applied its Performance Improvement approach to strengthen several internal systems during PY1 through PY3. In addition to improving system efficiency (file management, expenditure reporting, memos of engagement), the goal was to build in-house capacity to apply PI practices and tools among staff, especially more “junior” staff. Despite the best intentions, work on internal systems mostly remained on the back burner due to many other priorities, including the rapid expansion of field support funding, addressed by the small Operations Unit. Specific funding for this activity ended at the end of PY3. There have been a few activities proposed in PY1 and PY2 whose conception evolved over time through implementation: • The M&L Program had a Partnership Unit in its organizational structure in PY1—Good work was done researching experiences and knowledge about effective partnerships and a useful set of principles was developed. However, it became clear as we implemented the PY1 core and field￾supported workplans and in discussions with the CTO, that partnerships should take a different form as compared to the description in USAID’s RFA for the M&L Program. The Partnership Unit was eliminated when M&L restructured in 2001. Partnerships became the focus of field-based activities in SD3, especially the activities of the TCNetwork. • Leadership Booklet (October 2001 SAR)—M&L determined that a separate booklet on leadership was less practical than documenting and sharing learning through existing publications, such as The Manager and the forthcoming publication: Leading and Managing at All Levels: Handbook for Improving Health Services. All of the research conducted for the Leadership Booklet will be reflected in the forthcoming Handbook. • E-learning pilot of Consulting for Results (CfR) and Consultant Support Center (see October 2001 SAR)—The Consultant Support Center evolved into the TCNetwork Virtual Hub, which has been critical in the development of TCNetwork, and more broadly into the Virtual Center for Leadership and Management (VCLM). CfR has now been revised for external audiences and is being offered to two groups of consultants of Partners in Population and Development. CfR is also generating M&L Self-Assessment, April 23, 2004 27 significant interest among TCNetwork members and potential members. A virtual version of the course may be undertaken in the future. The excellent idea of a virtual course was simply premature. • CAFS—As noted previously, the implementation of our partnership with CAFS has evolved considerably. I. What criteria were used by M&L for selecting countries to use core funds? In the first year M&L identified Bolivia, Brazil, Cambodia, India, Indonesia, Nicaragua, South Africa, Tanzania, and Uganda as strategic countries for the use of core funds to implement demonstration projects. In the second year we targeted Cambodia, the Dominican Republic, India, Malawi, Senegal, and Uganda. The criteria used for identifying strategic countries included: • past presence and relationships in a country under FPMD II (Brazil, Bolivia, Tanzania); • existence of an MSH bilateral project (Nicaragua, Senegal, South Africa, Uganda); • existence of another MSH Cooperative Agreement, such as BASICS II and RPM+, and the Gates-funded Strategies for Enhancing Access to Medicines (SEAM) Project (India, Cambodia); • opportunity to leverage field support funds. This involved studying USAID Mission plans for assistance in health and population — whether a Mission had chosen to program all available funds in a bilateral, thus diminishing the opportunity for an earmark to a central cooperative agreement, and also whether a Mission was limiting the number of CAs operating in a country due to staffing constraints; • with CTO support, respond to a request by a MOH or USAID Mission (Angola, Armenia, Egypt, Indonesia, and Mozambique). M&L Self-Assessment, April 23, 2004 28 III. TECHNICAL LEADERSHIP AND CONTRIBUTIONS TO THE STATE OF THE PRACTICE IN MANAGEMENT AND LEADERSHIP A. What are noteworthy areas of technical leadership and contributions to development of the ‘‘state of the practice,” including Human Capacity Development (HCD) and managing and disseminating knowledge? The M&L Program has contributed the following: • Principles that are the foundation for all of our work in client engagement, program design, implementation and evaluation. • The Leading and Managing Framework. • Leading Performance Improvement to improve organizational performance. This includes a process and materials based on the principles and L & M Framework mentioned above. • An effective approach to measuring changes in services, systems and workgroup climate. • The effective use of “blended learning” and electronic reinforcement to support client’s efforts to improve their ability to identify challenges and achieve results. • A tested process for capturing, synthesizing, applying and disseminating knowledge. • Effective approaches and tools to improve human resource management. • An effective and comprehensive approach to addressing the crisis in HCD to implement health services, including family planning and reproductive health. • A new model of networking to draw upon skilled local consultants, through the Technical Cooperation Network. Principles The M&L Program design includes three distinct IRs: 1. Improved performance of management systems of organizations and programs; 2. Improved performance of leaders and managers; and 3. Improved ability to anticipate and respond effectively to the changing external environment. The principles that evolved from the initial struggle to integrate M&L work in these three areas, with a focus on the primary audience—managers in health care—led to the M&L principles that now are fundamental to our work. In summary, the principles include: working with managers at all levels in organizations or programs, in their own settings, with their work teams; addressing challenges they identify that are barriers to improved services. M&L uses a process over time, based on performance improvement, with the measurement of success being improved services, systems and/or work climate. Improved skills or awareness of individuals is a means to an end, not an end in itself in this approach. Finally the M&L program aims to “demystify” leadership development imparting practices in leadership and management that anyone can improve. (The full text of the five principles is included in Section I.) The M&L principles and collaboration with PLP are contributing to the state of practice in leadership development, with a focus on linking leadership development directly to improving health services. The principles also are foundational to M&L work with the WHO/USAID/IPPF/UNFPA Consortium on Implementing Best Practices (IBP) and the Maximizing Access and Quality (MAQ) Initiative’s Management and Supervision Working Group’s focus on “Leading Change—Adapting and Applying Good Practices.” M&L Self-Assessment, April 23, 2004 29 The Leading and Managing Framework This framework includes the leading and managing practices described in Section I. The change in the “state of practice” is that of a focus on both leading well and managing well at the same time. M&L does not divide people, some as leaders and others as managers. Instead M&L asks managers, at all levels in organizations and programs, if a part of their job is to lead groups of people to achieve results. This helps them to see that their role as leaders is as important as their management role, wherever they are in the system. Leading Performance Improvement (LPI) The LPI brings together the Leading and Managing Framework approach, materials and tools in a sequenced process, over time, addressing real challenges while learning better ways to lead and manage. This process is an evolution in the state of practice from what has previously been separate training in management and separate training in leadership often focused on hypothetical case studies. Electronic Support and “Blended Learning” These components are essential to M&L work because sustaining improved leadership and management for results cannot be achieved through “one-off” workshops. M&L has been committed to reinforcing organizational improvement efforts with electronic and distance support from the beginning of this Cooperative Agreement. The evaluation of the VLDP I program and cohort and the BPP/PROCOSI blended learning approaches (see Section II, A. and Evaluation Notes) demonstrates that M&L can reach a wider audience than previously thought possible, and maintain participants’ commitment to achieving results while, at the same time, improving their leadership and management practices. All blended learning, like the face-to-face work, involves working in teams with real organizational challenges, with performance measurements related to improvements in services, systems and /or work climate. USAID has recognized the power of this approach and is using the M&L experience and platforms for their internal USAID On-Line Learning Program. The U.S. Department of Health and Human Services is now asking to use this approach to reach 20,000 health care providers in the U.S. Medicaid Program to improve “cultural competence” for improved health outcomes. M&L continues to learn while documenting improved results. Such blended learning approaches can be successfully used to rapidly scale-up. Knowledge Application (KA) KA is the M&L term for a tested process to capture, synthesize, apply and disseminate knowledge. M&L has modeled its approach to KA on successful practices of private sector consulting firms including McKinsey, A.T. Kearney, and Bain. The M&L process of monitoring and evaluation, Plans for Performance Improvement, in-depth evaluations, Communities of Practice, synthesis of what we know into “Knowledge Folders” and dissemination/communication are the components of this process. Knowledge Folders contain “what we know, how we know it and how it can be applied.” Effective KA is fundamental to extending the reach of M&L’s work to others in MSH, the TCNetwork, clients and other CAs. M&L staff participate in the work of the INFO Project and have made important contributions to the thinking in this important area as the INFO Project works to prepare an issue of Population Reports on knowledge management. Monitoring and Evaluation (M&E) The approaches, systems, and materials are a part of KA described above. The development sector has had no effective approach to evaluating return on investment in leadership or management training beyond tracking improved awareness and skills of individuals participating primarily in “one-off” workshops. The private sector, even in U.S. industry, is still struggling with ways to measure the effectiveness of investments in improved management and leadership. M&L has twice presented its evolving approach to M & E to the LEAG and has received a very favorable response. In addition, a Menu of Indicators is now available to all MSH staff, consultants and others involved in leadership and management development. The Menu assists staff in selecting measurable indicators right up-front in the program design stage. M&L is still learning but participants in LEAG acknowledge that this approach, aimed at evaluating achievements by teams related to improvements in services, systems and/or work climate, is on the right track. M&L Self-Assessment, April 23, 2004 30 Effective approaches in Human Resource Management (HRM) M&L is contributing to the state of practice for HRM by continuing work started under the FPMD Program. In 1998 FPMD developed the first HRM Assessment Tool which has since been applied, under FPMD and M&L, in public and private sector organizations in over 20 countries. M&L work focused on participatory HRM Assessment, developing HR strategies, and strengthening HRM systems. From this experience M&L learned first hand the importance of both “management and leadership.” HRM systems are only as effective as the hands of the leadership that guides them. Human Capacity Development (HCD) In response to the request from the USAID Office for HIV/AIDS, M&L has contributed significantly to the development of the HCD framework through a Task Force to address the impact of HIV/AIDS on the health workforce. In its role on the Task Force, M&L provided technical leadership in defining the term “HCD” and also developing a framework for implementing it (see below). M&L also contributed to the dissemination of the framework with donors and health managers globally by helping to shape satellite sessions at the Barcelona AIDS Conference in 2002 and the International Conference on AIDS and STIs in Africa (ICASA) Conference in 2003. A meeting was held in London in 2002 to present the HCD framework to a group of 12 donors. As a result, the HCD framework is now being used as the basis for workshops and TA by several CAs. FHI is using the framework in developing district workplans. UNAIDS e-workspace features the framework in ongoing work in HCD. UNAIDS also sponsored an HCD conference in Zambia that focused on the Community “sphere.” The framework is also being used by USAID, the Centers for Disease Control and Prevention, and HRSA (Health Resources and Services Administration of the U.S. Department of Health and Human Services) as a basis for developing HR policy for the implementation of the President’s Emergency Plan. M&L is uniquely qualified to provide technical leadership in this area due to its extensive experience in human resource management. Framework for Human Capacity Development M&L Self-Assessment, April 23, 2004 31 Within each of these spheres, there are four critical components: 1. Legal, Policy, and Financial Requirements 2. Leadership 3. HRM System 4. Partnerships B. How does the M&L Program contribute to USAID’s goals? The M&L Program contributes to the three Intermediate Results (IRs) that USAID has defined for this Cooperative Agreement in this ten year Results Package. This is done by integrating work across the three IRs and evaluating this work using the IRs. USAID Missions support organizations, programs and institutions to provide improved FP, RH and other health services. M&L’s work with these same client agencies enables them to plan for and receive donor funding, implement programs and account for the effective use of funds. M&L efforts enhance the absorptive capacity for USAID program specific funding. The descriptions of the link between improved management and leadership and improved services, work climate and management systems in Section I, along with the description of achievements in Section II, provide examples of improved organizational capacities. In addition to the direct technical assistance of M&L related to its work in Strategic Directions 1 and 2, M&L has also set the foundation to "multiply" these efforts through: Strategic Direction 3 - Partnering for Sustainability; and Strategic Direction 4 - Knowledge Application M&L knows that there are hundreds of organizations and institutions that must be able to receive, effectively utilize and account for their own and donor resources to scale up and meet the needs in FP/RH, HIV/AIDS, maternal health, child health and infectious disease. One international program in management and leadership cannot meet all these needs. M&L’s work in SDs 3 and 4 is to put successful approaches and tools into the hands of experienced TA providers and CAs with expertise in organizational development, all over the world. This includes continuous learning from M&L’s own work as well as learning from what works for members of the TCNetwork and other CAs. M&L set the foundation for this transfer in PYs 1 to 3 and has begun the transfer process in PY4. M&L also contributes to the three results in the new OPRH Strategic Objective 1 Results Framework for Global Health/OPRH. This is done through the evolving state of practice described earlier in this Section. IR1 – Global Leadership: M&L contributes evidence-based practices in improving management, leadership and sustainability. M&L is building partnerships, especially through the TCNetwork, and influencing other donors. These donors include DFID, who has picked up funding for leadership development in Brazil, and the Kellogg and Gates Foundations, who are seeing ways to link leadership development to improved services through discussions in the LEAG meetings mentioned earlier. IR2 – Knowledge Generated: As noted earlier, M&L is contributing its learning in knowledge application to the INFO Project and to USAID. This includes a systematic approach to M & E, in-depth evaluations, synthesis of what we know, how we know it, and packaging this in Knowledge Folders and issues of The Manager. IR3 – Support to the Field: M&L takes a customer orientation to all work with USAID Missions to be as responsive as possible to their needs and the needs of their in-country client organizations. M&L uses "client engagement" to assist all stakeholders to clearly describe priority challenges and desired performance in overcoming the challenges. This helps to clarify the expectations of clients and the donor as to what M&L can offer. M&L Self-Assessment, April 23, 2004 32 C. What is M&L’s ability to apply knowledge to a rapidly changing environment? M&L believes that a key to using knowledge in changing environments is the approach of "Consulting for Results." All technical staff, as well as the members of the TCNetwork, must be able to engage with clients as effective consultants. As mentioned above, M&L uses client engagement to rapidly assess the needs and expectations of clients and the USAID Missions. M&L approaches and tools are participatory—the clients themselves define their challenges and expected outcomes. Through this approach M&L has been able to successfully use experience gained initially in FPMD, and build on this for M&L work in such countries as Nicaragua and Brazil, and then adapt the approach for introduction to programs in Guinea and Egypt. The LPI approach that has evolved from such country experiences is being adapted for use in all of M&L’s work. It is being replicated in Egypt without technical assistance or financial support from M&L and has been transferred to M&L’s Mozambique and Senegal programs as well as the Kenya (HIV/AIDS core funded) program. Moreover it is nearing the stage for transfer to others, including the TCNetwork. This example of the documentation and wide dissemination of a proven approach is key to M&L’s ability to scale up across countries, and within countries. M&L has similar examples of replication and scale-up, such as the VLDP, MOST, and BPP. Examples of how M&L disseminates knowledge through publications, and documented approaches and tools are described in Section II. M&L Self-Assessment, April 23, 2004 33 IV. WORK NEEDED IN THE FUTURE As a ten-year Results Package, more work remains to be done in management and leadership—on both M&L’s and OPRH’s IRs. The first four years of the M&L program built a strong and flexible base to address the continuing needs. Building on the legacy of previous USAID investments in programs, approaches, tools, and principles, there is a continued need to contribute to the body of knowledge on management, leadership and organizational performance improvement and multiply the number of organizations and programs across countries and regions which make use of this know-how. A. What is the 10-year vision for improving management and leadership? Many past efforts in the field of management and leadership development have tended to be geared toward “one-off” events, often to individuals. Millions of dollars have been spent on training programs, how-to manuals, etc. TA was usually provided by international experts in visits of limited duration, one organization at a time. In establishing the M&L program, USAID recognized that this is insufficient in the face of the enormous need for improved performance in hundreds of organizations. To make a lasting difference on a large scale, interventions are needed which can be applied to multiple organizations. These interventions need to be sustainable and locally applied, adapted and updated with changing times, not relying solely on international experts. Given the challenges, worsened by the human capacity problems created by HIV/AIDS, managers and leaders need to inspire commitment and energy of their teams—good systems and procedures alone are not enough. In the face of these challenges the vision below is offered as a way to guide the work which remains to be done by the year 2010. Hundreds of programs and organizations are led and managed in ways that inspire the commitment and energy needed to make a positive and lasting change in the health of men, women, and children. Work needed to achieve this vision will continue to build on these strategic activities: • Develop, test, and provide simple, replicable methods for institutionalizing management, leadership, and organizational performance improvement capabilities. • Demonstrate concretely that improvements in management, leadership and organizational performance are essential to achieving sustainable results in accessible, quality FP/RH and service delivery as well as in addressing maternal and child health, and infectious diseases. • Expand awareness of donors and counterparts of the need to invest in management, leadership and organizational performance improvement in order to make lasting improvements in accessible, quality FP/RH programs, and other health services. • Work with local technical assistance providers and partners to expand the number of organizations that employ proven methods to improve management, leadership and organizational performance. • Create and refresh a body of knowledge on leading organizational service improvement which is systematically applied by MSH, its partners, policy makers, and practitioners around the world. • Build the inter-relationships between health and other sectors in program planning and implementation. M&L Self-Assessment, April 23, 2004 34 B. What are some promising programs, approaches and strategies for improving management, leadership and organizational performance? The current M&L programs that contribute to this vision for 2010 are mentioned below and grouped according to the strategic activities of the vision. 1. Develop, test, and provide simple, replicable methods for institutionalizing management, leadership, and organizational performance improvement capabilities The Virtual Center for Leadership and Management (VCLM) 7—The VCLM will become a means to reach hundreds of organizations with the best of M&L’s online communities of practice, electronic information sources, and virtual programs. The Leading Performance Improvement 8 and the Senior Leadership Development Program (SLP) 9 — Having facilitators’ manuals available in print and electronically through the ERC and the VCLM, M&L can assist TCNetwork members and others to use these programs. M&L’s role will be to continue implementing some programs, as well as updating and maintaining the programs based on lessons learned. LeaderNet will be available for support to facilitators. The Work Group Climate Assessment (WCA)—is a survey process that enables work groups to assess their current perception of their work environment that is, climate, and includes strategies for implementing needed improvements. Pre-Service Training—Learning from successful work in the United States integrating the basics of management and leadership into pre-service medical training, and experiences gained in Egypt 10 will enable M&L to expand the number of institutions which offer its programs to medical personnel who are called upon to lead and manage. Building on this, approaches can be adapted to reach a greater numbers of doctors and nurses in other countries. VLDP—This program is designed to reach multiple organizations and can enroll up-to one hundred participants at a time. FHI has recently contracted with MSH to use VLDP, adapted to the HIV/AIDS context, across the Caribbean, demonstrating one way it can be scaled up. Current VLDP work in East Africa is demonstrating that virtual learning and support can be effective in all parts of the world. Variations of the VLDP—In addition to the know-how developed in on-line coaching and facilitation, M&L has created the basic electronic architecture to host Web-based programs such as the VLDP. This knowledge and technology can readily be expanded to include other content areas such as strategic planning, operational planning, budgeting, etc. Human Capacity Development (HCD)—Expand the application of M&L’s Human Capacity Development Framework with a focus on leadership development and cross-sector strategic and operational planning. Human Resource Management (HRM)—Transfer application of the M&L Human Resource Management Assessment and Implementation approach and tools to members of the TCNetwork to multiply and the improved performance of organizations. 7 The VCLM is a Web-based center that integrates and supports M&L’s electronic information resources, online communities of practice, and virtual programs. It provides easy cross-links for members of the TCNetwork, the Communities of Practice, LeaderNet, Global Exchange on Reproductive Health, the Electronic Resource Center, the VLDP, and the Business Planning Program. 8 The LPI program, developed in Egypt and expanded to Mozambique and Kenya, is geared to help managers and leaders identify and overcome challenges. The facilitator’s guide enables the local organization to assume responsibility for the program. 9 Geared toward national senior level leadership of the public and NGO sectors, the SLP benefits national level and multi-sectoral coordination. A facilitator’s manual can also be developed to transfer its implementation to the TCNetwork and other partners. 10 In PY4 M&L will initiate support to the medical and nursing faculties in the development of a sustainable management and leadership curriculum at the Medical Faculty of Menoufia and the Nursing Faculty at Alexandria University. M&L Self-Assessment, April 23, 2004 35 Such virtually supported programs can support NGOs now entrusted with the implementation of large programs like the President’s Emergency Plan. Immediate initial TA can be provided on site with the longer term assistance in management and leadership provided through on-line facilitation. MOST Suite—The MOST programs can be added to the VCLM and made available to numerous organizations. M&L’s role would be one of implementing a number of MOSTs per year, updating the MOST suite based on lessons learned, and providing advice and assistance to other internal and external users through the VCLM. In addition, support to organizations in implementing the MOST can be provided by LeaderNet and by TCNetwork members. Business Planning Program (BPP)—The approach is to work with local partners trained and commissioned by M&L to implement the BPP in a defined region. M&L will draw on their learning, make modifications to the program and maintain a master program of high quality on the VCLM for use by local Program Partners. 2. Demonstrate concretely that improvements in management, leadership and organizational performance are essential to achieving sustainable results in accessible, quality FP/RH service delivery. Monitoring and Evaluation (M&E) for results—Related to the program planning process, M&L’s M&E system and know-how could be adapted for NGOs; M&L could offer technical assistance in this area. Indeed, through our work in the VLDP and other leadership development programs using action plans to document teams’ organizational challenges, it is clear that most NGOs and MOHs need significant assistance in strengthening their management information systems and as importantly, building managers’ capacity to use their own data for decision-making. Health Information Systems (HIS)—When completed in PY5, the M&L CoP and Knowledge Folder on HIS will be made available to the public through the ERC. As noted above, TA in this area is a growing need which could be supported by M&L. Menu of Indicators on Management and Leadership—Based on M&L’s three IRs, the Menu can offer a valuable head start to organizations interested in measurement in these areas. With further testing and validation through field application, the Menu could become a resource for CAs and other organizations through M&L technical assistance as well as the VCLM and the TCNetwork. 3. Expand awareness of donors and counterparts of the need to invest in management, leadership and organizational performance improvement in order to make lasting improvements in accessible, quality FP/RH programs. M&L’s work in MAQ, LeaderNet, the TCNetwork, and The Manager and other publications are important ways that M&L can assist in generating awareness among donors and local governments and private sector partners of the need to invest in management, leadership, and organizational performance improvement. M&L’s participation in MAQ as Co-chair of the Management and Supervision Subcommittee and Co-Chair of the Organization of Work Subcommittee supports the expansion of the latest thinking to other organizations, including donors. M&L will also continue to contribute to U.S. and field-based activities of the Implementing Best Practices Consortium of WHO/USAID. Through online discussions in LeaderNet, managers and leaders from M&L and other programs, and shortly, other CAs, can share information and challenges. Alumni of face-to-face and online programs can exchange information. Facilitators of these programs can exchange ideas and common challenges. The Manager is also a primary means of knowledge dissemination and application. It is published in English, French, and Spanish, reaching over 15,000 subscribers in 192 countries. M&L Self-Assessment, April 23, 2004 36 4. Work with local technical assistance providers and partners to expand the number of organizations that employ proven methods to improve management, leadership and organizational performance. Highly qualified local TA is more widely available than in the past. M&L and major donors recognize this and place a premium on tapping the TCNetwork to provide an increasing share of the TA. M&L will continue to work with the TCNetwork as a way to further evolve from hands-on work to identifying and engaging local consultants. The TCNetwork and other local partners will be able to obtain support and feedback through the VCLM. VCLM will offer a single entry point to all of M&L’s Communities of Practice, electronic information sources, and programs. Moreover, the Global Exchange on Reproductive Health community will contribute to expanding the number of organizations that employ proven methods to improve management, leadership, and organizational performance in the areas of FP/RH. Several member countries of the Global Exchange are also TCNetwork members; both networks will serve as a source of TA for M&L and others. Finally, M&L will continue to work with service delivery CAs such as IntraHealth, EngenderHealth, and FHI to strengthen leading and managing practices, work group climate, and management systems to improve organizational performance and health service delivery. 5. Create and refresh a body of knowledge on leading organizational service improvement which is systematically applied by MSH, its partners, policy makers, and practitioners around the world. The TCNetwork represents a primary means to develop, assess, and disseminate knowledge. Other mechanisms include: VCLM and ERC, including the Health Manager’s Toolkit—these are primary venues for sharing lessons learned, practices, and tools. M&L will also continue to contribute to the corporate-funded Institutional Memory system. Knowledge Management—With further refinement, the CoPs and their related Knowledge Folders represent a mechanism to synthesize and present useful and up-to-date knowledge, tools, approaches, and information in specific areas of management, leadership, and organizational performance improvement. Using the approach and technology developed, tested, and systematized by M&L, other content areas can be added. The CoPs and their “products” will be available on the VCLM, ERC, through The Manager, in handbooks, or CD-ROMs. 6. Build the inter-relationships between health and other sectors in program planning and implementation. In the design and implementation of M&L interventions, care is taken to show the link between accessible, quality FP/RH programs and other USAID programs that focus on areas such as HIV/AIDS and maternal and child health. Management, leadership and organizational performance improvement are relevant to FP/RH as well as to HIV/AIDS, TB, malaria and other areas, such as education. M&L is in a position to act as a link between FP/RH and other sectors. This is especially apparent in leadership for HCD. Leadership for HCD—M&L’s desired outcome in leadership development in HCD is: people at all levels are enabled to face challenges and achieve results in complex conditions. The key elements needed are visionary leadership, leadership development for managers at all levels, and motivating work climates. M&L Self-Assessment, April 23, 2004 37 Visionary Leadership—Countries and programs with initial success confronting the HIV/AIDS crisis have had visionary leadership. It takes courageous and committed leaders to face the challenge of this crisis and mobilize others for effective results. Since leadership to respond to this crisis must come from across programs, sectors, and organizations, a critical task is the coordination of national efforts. Unless this coordination is well led and organized, responses to the crisis will be scattered and mismanaged. M&L’s strategy is to develop visionary leadership at senior levels by aligning all key individuals around a coordinated national strategy and plan, assessing organizational sustainability, and diagnosing areas for improvement. Programs and activities used for this are the Senior Leadership Development Program, national coordination strategic planning activities, and the MOST. Leadership Development for Managers at all levels—In order to improve the performance of basic health services, it is critical that managers at all levels learn how to lead others to face challenges and achieve results in complex conditions. This is as important at the health unit level as it is at the national level. M&L’s strategy is accomplished by working with local organizations and facilitators to ensure ownership, teaching them to transfer the program widely. Create work climates of motivation and performance—M&L’s strategy is to support managers to develop work climates that value employees and build motivation by providing the correct levels of challenge, clarity and support. Activities for this strategy include: the WCA tool and process to determine which leading and managing practices need strengthening; work with teams to strengthen these practices and set and monitor goals; and conduct a follow-up WCA to monitor progress. C. How is M&L’s approach to partnerships and collaboration evolving? Partnerships and collaboration are essential to promote improvements in the leadership, management and sustainability of accessible, quality FP/RH programs. M&L bases its program design and implementation on the premise that to achieve its vision, it must work with other organizations, programs, and firms—especially those located in developing countries. This is the foundational principle behind the TCNetwork. Creation of the VCLM and LeaderNet also manifest this core conviction. The Global Exchange on Reproductive Health is based on the certainty that cooperation is essential. The PY5 Workplan has collaborative activities with EngenderHealth, FHI, IntraHealth, ADRA, and CARE. M&L Self-Assessment, April 23, 2004 38 APPENDIX I REFERENCES REFERENCES Management and Leadership Program. Highlights of Results. Management Sciences for Health, July 2002–June 2003. __________. Management & Leadership Program: Self-Assessment. April 23, 2004. __________. Cooperative Agreement and Program Description. __________. M&L Activity Authorization (AAD)/Results Package. __________. M&L Semiannual Reports (six reports covering October 2000–December 2003). __________. M&L Annual Management Reviews (for 2001, 2002, and 2003). __________. M&L Results Review reports (for 2002 and 2003). USAID Bureau for Global Health. HCDP Program Description. 2004. (Draft) M&L Informational Materials ƒ M&L Leading and Managing Results Model ƒ Leading and Managing Framework and Process ƒ Principles for Developing Managers Who Lead ƒ Human Resource Management ƒ Building and Sharing Knowledge to Advance Health Reform in Latin America ƒ Supporting Decentralizing Health Services in Indonesia ƒ Decentralizing Health and Family Planning Programs ƒ The Human Resource Crisis in Health in Sub-Saharan Africa ƒ M&L HIV/AIDS, Tuberculosis and Malaria Programs ƒ Technical Cooperation Network ƒ LeaderNet ƒ The Virtual Leadership Development Program ƒ Rapid Funding Envelope for HIV/AIDS (Tanzania) ƒ For Disabled Tanzanians, Informational Materials about HIV/AIDS ƒ Lidernet (Brazil) ƒ New Thinking, New Funding Options (Business Planning Program) M&L Country Program Evaluations ƒ APROFAM, Guatemala ƒ ƒ ƒ ƒ ƒ ƒ ƒ Leadership Development Program of Egypt Leadership Capacity Strengthening Program, Guinea Movimiento Manuela Ramos, Peru Family Life Education Program, Uganda Business Planning Program with PROCOSI, Bolivia Health Manager’s Toolkit Virtual Leadership Development Program M&L Tools ƒ MOST Management and Organizational Sustainability Tool: A Guide for Users and Facilitators ƒ Human Resource Management Rapid Assessment Tool for HIV/AIDS Environment: A Guide for Strengthening HRM Systems ƒ Management Strategies for Improving Health and Family Planning Services: A Compendium of The Manager Series, Vols. V–IX ƒ Issues of The Manager: • “Managing Reproductive Health Services with a Gender Perspective” • “Planning for Leadership Transition” • “Developing Managers Who Lead” • “Exercising Leadership to Make Decentralization Work” • “Creating a Work Climate that Motivates Staff and Improves Performance” • “Assessing Your Organization’s Capacity to Manage Finances” POPULATION TECHNICAL ASSISTANCE PROJECT 1101 Vermont Ave., NW Suite 900 Washington, DC 20005 Phone: (202) 898-9040 Fax: (202) 898-9057 www.poptechproject.com