1 FOCUS ON RESULTS: ENHANCING CAPACITY ACROSS SECTORS IN TRANSITION (FORECAST) A Participant Training and Human Capacity Development Program FINAL EVALUATION REPORT December 2, 2011 This publication is made possible by the generous support of the American people through the United States Agency for International Development (USAID/Liberia) under the Liberia Monitoring and Evaluation Program (L-MEP), Contract Number 669-C-00-10-00181-00. The author’s views expressed in this publication do not necessarily reflect the views of the United States Agency for International Development or the United States Government. 2 Focus on Results: Enhancing Capacity across Sectors in Transition A Participant Training and Human Capacity Development Program FORECAST CONTRACT NUMBER: 669-C-00-10-00181-00 Final Evaluation Report Submitted to: USAID/Liberia December 2, 2011 3 TABLE OF CONTENTS ABBREVIATIONS..............................................................................................................................................4 EXECUTIVE SUMMARY...................................................................................................................................6 1. INTRODUCTION......................................................................................................................................11 1.1. Background and Purpose of the Assessment..................................................................................11 1.2. Design and Conduct of the Assessment .........................................................................................11 1.3. Presentation of the Findings............................................................................................................12 2. THE LIBERIA FORECAST PROJECT .........................................................................................................13 2.1. The Situation Before the FORECAST Project................................................................................13 2.2. Establishment of the Liberia FORECAST Project ...........................................................................13 2.3. General Description of the FORECAST Project ............................................................................14 3. FINDINGS OF THE ASSESSMENT ............................................................................................................15      3.1     Impact of the Intervention on the Operational Effectiveness of Targeted Organizations...........15 3.1.1 Achievements Reported by the Project.......................................................................................15 3.1.2.   Stakeholders’ Assessment............................................................................................................21 3.1.3.   Conclusion and Lessons Learned.................................................................................................27     3.2.     Meeting the Eight FORECAST Objectives....................................................................................29     3.3.     Effectiveness of FORECAST’s Management Structure and Processes.........................................32 4. THE FUTURE OF USAID’S CAPACITY BUILDING SUPPORT...........................................................36 5. APPENDICES ..........................................................................................................................................40 Appendix 1: Short‐Term Training Support Provided Departments/Units/Divisions as at the End of 2010….37 Appendix 2: Update of Training Events Status (by Sector) – as at 03/31/11…….……………………….…...40 Appendix 3: Institutions and Persons Interviewed ………………………………………………………….54 Appendix 4: Assessment Tools/Instruments ………………………………………………………………...56 Appendix 5: FORECAST Implementation Steps …………………………………………………………….64 Appendix 6: Letter from the Ministry of Gender & Development on Achievements about FORECAST......65 Appendix 7: Report Submitted by Ethel Brooks, PT/HICD Officer, USAID/Washington EGAT Bureau…..67 Appendix 8: The Future of FORECAST: Perspective of USAID/Liberia Education Advisor James Gray........72 Appendix 9: FORECAST Field Trip Report February to March 2010…………………………………….... 78 4 ABBREVIATIONS AED Academy for Educational Development AEDE AGC Agency for Economic Development and Empowerment Associates for Global Change AMREF AUTOCAD African Medical and Research Foundation A CAD (Computer Aided Design or Computer Aided Drafting) Software application for 2D and 3D design and drafting BWI Booker Washington Institute COP Chief of Party CSET Center for Sustainable Energy Technology CTO DPs Cognizant Technical Officer Development Partners FGD Focus Group Discussion FORECAST Focus on Results Enhancing Capacity Across Sectors in Transition FY Fiscal Year GOL Government of Liberia GSA HICD General Services Agency Human and Institutional Capacity Building HRM IBI Human Resources Management International Business Initiative IMES IT International M&E Specialist Information Technology ITS International Training Specialist KII Key Informant Interview LECBS Liberia Emergency Capacity Support LIBAAM Liberia Improved Budget and Asset Management Program LIPA Liberia Institute of Public Administration L-MEP Liberia Monitoring and Evaluation Program MADET Mano Training and Development Foundation M&E Monitoring and Evaluation MOA Ministry of Agriculture MOE Ministry of Education MOH & SW Ministry of Health and Social Welfare MOGD Ministry of Gender and Development MPW MOYS Ministry of Public Work Ministry of Youth and Sports MPEA Ministry of Planning and Economic Affairs MPH MPCHS MPW NCDS NGO Master of Public Health Mother Patern College of health Sciences Ministry of Public Work National Capacity Development Strategy Non-Governmental Organization PMP PT Performance Management Plan Participants Training 5 SBA Subah - Belleh Associates SES Senior Executive Services TA Technical Assistance TC Third Country TOKTEN Transfer of Knowledge Through Expatriate Nationals TOR Terms of Reference TOT Training of Trainers UL University of Liberia USAID United States Agency for International Development 6 EXECUTIVE SUMMARY Background and Purpose of the Assessment The FORECAST Project, implemented by the Academy for Educational Development (AED), was designed to build human and institutional capacity within Government of Liberia (GOL) ministries, public institutions, NGOs, and other organizations. The Project, which commenced in September 2008, ended in May 2011. USAID/Liberia requested that the Liberia Monitoring and Evaluation Program (L-MEP) conduct an End-of￾Project Stakeholder Assessment toward the goal of making recommendations regarding whether USAID should continue to support Participant Training and Human and Institutional Capacity Development in Liberia and, if so, what the focus should be. The purpose of this assessment is to:  Determine whether FORECAST has achieved its goal of increased operational effectiveness of targeted GOL ministries, public institutions, NGOs, and other partner organizations.  Assess the management structure and processes FORECAST employed to determine if these were the most effective approach to implementing a human capacity project.  Make recommendations to USAID/Liberia regarding whether a future human capacity development project should be implemented and if so, what areas it should focus on, and how best it should be managed. Methodology The methodology for the assessment was generally participatory. It involved interviews and consultations with all categories of stakeholders, including individual beneficiaries, through focus group discussions, individual and key informant interviews, spot checks, and observations. The Assessment Team was comprised of representatives from FORECAST, L-MEP, USAID/Liberia, USAID/Washington D.C., and led by Subah-Belleh Associates. Key Findings and Recommendations 1. Impact of the Intervention on Operational Effectiveness of Targeted Organizations Over 30 public and private organizations received training and technical assistance that helped them improve their operational effectiveness in specific areas. In addition, some (Health, Education, Public Works, LIPA, and other) pointed to some aspects of improvement and change in certain departments/units of their institutions, as a result of the FORECAST Project intervention. Some provided specific success stories, which are included in the findings of this report. However, it was found that, overall, not much impact has been made in terms of operational effectiveness and other changes expected. Many returned participants complained that equipment and operational changes in their organizations that should have followed their training and return to work have not been forthcoming Perhaps, given that most staff benefitting from training were just returning, whle others were still in training, it was a little too early to expect any wholesale operational improvements. Through facilitated cross-assessments, individual assessments, training, and other capacity development approaches, FORECAST Project also sought to build the capacities of selected local private/NGO partners in HICD assessment methodologies and approaches. The Agency for Economic Development and Empowerment (AEDE) and Mano Training and Development Foundation (MADET) benefited from institutional capacity building assessments. Other private/NGO partners also benefitted from training capacity development in order to be able to plan and deliver training in specific fields. Stella Maris Polytechnic also benefitted, to a limited extent, in a ToT-type exercise supported by the Project. Several more examples of improvements are provided in the report. 7 2. Meeting the Eight FORECAST Objectives There are mixed results as far as the achievement of the eight FORECAST outcomes is concerned. FORECAST Liberia’s own assessment is that it was most successful in delivering management and technical training and supporting the development of local capacities to provide HICD services, including the conduct of needs assessments and the delivery of training in such areas as monitoring and evaluation, general management, computer skills training, etc. According to FORECAST, these accomplishments have manifested themselves in improved performance in the targeted organizations. Many of these successes were confirmed by the various stakeholders covered by the assessment. However, there were areas of much difficulty in accessing evidence of outcome achievement. This was particularly true for the following outcomes:  Operational effectiveness of targeted departments increased.  Transparent and accountable financial practices adopted and enforced by targeted departments  Increased capacity of Liberian workers to provide skilled labor to support public sector infrastructure  Improved managerial and organizational performance of targeted local HCD partner institutions The report goes into detail on each of the eight objectives. 3. Consistency of Activities with USAID Task Order USAID’s Task Order to AED for the FORECAST Liberia Project contained four major tasks the Project was expected to undertake:  Improve management and technical skills, and create transparent, accountable practices that will impact the operational level within targeted GOL ministries, public institutions, NGOs, and other partner organizations;  Provide basic and advanced vocational training to help rebuild and maintain the public sector’s physical infrastructure;  Improve the performance of targeted local institutions through training and technical assistance; and  Develop the capacity of local organizations to perform similar human capacity development activities. In implementing USAID’s Task Order, the FORECAST Liberia Project undertook these key activities:  Facilitation of Participant Training for targeted public and private/NGO institutions.  Follow-up of training and related technical support to partner organizations.  Support for local capacity building to carry HICD activities, particularly needs assessment. Though stakeholders took a generally positive view of the necessity of and support provided by, the FORECAST Project, their lack of adequate understanding of the Project, as expressed by them, created an inadequate basis for their fair assessment of the Project. For many, it was thought that FORECAST should have done more (in terms of resource provision) in translating training into practice and provided more training opportunities, among others. 4. Effectiveness of FORECAST’s Management Structure and Processes The AED team was small but highly effective in working with USAID and Liberian partner organizations to identify training opportunities and to work with training institutions in many different countries. They did an excellent job searching for appropriate training programs around the world. Their effectiveness is 8 illustrated by the fact that they exceeded their training targets. In addition, the participant follow-up initiated by the FORECAST Monitoring and Evaluation (M&E) Specialist was highly effective, according to participants, in helping them apply their newly acquired skills. The training plan concept worked well since it focused participants on the skills they intended to achieve and on their achievement. Having a sub-contractor provide institutional capacity building technical assistance to a number of organizations also illustrated an effective management approach. However, the management structure and the number of staff were too small to be able to provide institutional technical assistance to government ministries and organizations. 5. Recommendations for the Future of USAID’s Capacity Building Support The Assessment Team concluded that USAID should support a FORECAST 2 project with changes made based on the findings included in this report. FORECAST 2 should be designed to complement rather than overlap the human and institutional capacity building activities of other USAID projects, and should fill gaps in capacity that USAID feels are critical to achieving its overall development objectives. Specific recommendations are the following: Strategic Recommendations a. Establishing Effective Partnership FORECAST 2 must be designed as a more formal strategic partnership between the Government and USAID that establishes commitments on both sides. It should reflect priorities associated with the National Capacity Development Strategy, but must also be a bridging effort that fills gaps rather than attempt to meet all perceived needs of beneficiary institutions. A way of ensuring this is to insist that Government implement the National Civil Service Training Policy as a basis for collaboration in capacity building. The Policy places binding responsibilities on public entities to develop and implement annual training plans, with agreed levels of budgetary commitment to training. The Civil Service Agency is the manager of the Policy and must be involved in any strategic partnership with USAID on capacity building support. b. Priorities for FORECAST 2 The capacity challenges in Liberia are huge and span all sectors, particularly the public sector and institutions, where there are less competitive recruitment practices and very little access to training and other skills building opportunities. While the FORECAST Project represented a tremendous effort to address the problem, the needs still abound. However, FORECAST 1 support provided more substantive support to the health and education sector than all other sectors, even though huge needs still exist in these sectors. As such, FORECAST 2, while continuing to deliver support to the health and education sectors, should step up support for training and capacity building in the Economic Growth and Governance and Democracy Sectors. As for the private sector, a more strategic assessment is required to determine critical needs within the sector. The selection of private sector entities with which to collaborate should be based on their overall value across sectors and their level of institutional development that can enable them to adequately absorb and utilize technical support for the benefit of other sectors. c. Approach to Awarding/Delivering FORECAST 2 The approach to delivering FORECAST 1 was generally effective. However, the duration of the project, which was understandably driven by the need to make some “quick impact” on the capacity situation at the time, was considered by all stakeholders to have been too short to allow more effective planning and execution as well as achieve the desired impact.. Going forward, it will be more effective to design and implement a more long-term strategy and program of support to capacity building across sectors in Liberia, building on the orientation and achievements of FORECAST 1. A more long-term delivery period will allow for more effective planning and implementation, follow- 9 up/technical support, and monitoring, and evaluation. Perhaps an initial 5-year program can be formulated for FORECAST 2. d. Management Arrangements As already stated above, the management arrangement for FORECAST 2 should build on the experiences of FORECAST 1. One critical area for strengthening should be the capacity to provide post-training follow-up and technical support. In FORECAST 1, a single M&E specialist supported virtually all sectors with follow-up/technical support, while at the same time undertaking project M&E activities and supporting the development and management of M&E structures within partner organizations. It will be more useful in FORECAST 2 to build a more technically ready and competent project organization. This may be achieved in great measures by including in the project organization more sector and subject specialists that can effectively support follow-up of training and technical support to partner organizations, while maintaining an independent M&E function within the project. FORECAST 2 should also consider the possibility of sub-contracting the management of certain services to local experts and organization. For instance, post-training technical support in specific areas/sectors/organizations may be sub-contracted to a farm, while FORECAST monitor and evaluate progress and achievements. Programmatic Recommendations a. Ensure an active interagency/cross-sector working group (or collaborative framework) that can assure a fair reflection of priority needs of participating institutions and continuing focus on the purpose of the FORECAST project. This will enhance a positive response from participating institution avoid the kinds of negative perception experienced in FORECAST 1 that the project was bottomless scholarship pit and an end in itself for capacity building. This can be achieved by: o Including partners in project design. o Hosting a workshop with key agency staff to clarify, identify, and reaffirm goal commitments with the USAID, the contractor, and within and among the partner organizations. o Regularly communicating with partners on the aims and objectives of the FORECAST Project and expected roles and contributions of the partners in meeting said goals, objectives, and outcomes. o Ensuring specific responsibilities/obligations for partners in realizing the intended outcomes the FORECAST Project. b. Any continuation of Project activities must be flexible to accommodate changes in planning and implementation. This can be achieved by: o Periodic reflection gatherings to review issues and possible changes. o Strategic discussions about cross-sector capacity building needs. o Set appropriate technical support strategies and more realistic targets and timing for impact and the measurement of said impact. c. The engagement of external HICD partners such as those FORECAST worked with in the past should become a more formalized element of FORECAST 2’s strategy, particularly to ensure close partnering in FORECAST’s implementation activities as well as strengthen FORECAST’s outreach. This can be accomplished by, for example: o Consultation with partners in project design and planning. o Inclusion of partners in field implementation activities. o Inclusion of partners in communication planning. o The use of formal agreements or MOUs with partner organizations. 10 o Defining indicators of success for each of the above by hosting a joint session to define these indicators. d. More innovative strategies to training/capacity building should be considered, while strengthening the technical support component to ensure hands-on experience or learning by doing. This can be accomplished through: o Study visits to third countries, particularly African countries that have successfully implemented holistic strategies and programs for capacity development. o Use of experts/consultants, including individuals from within participating institutions who can contribute knowledge and experience to others. e. Assure realistic expectations of time and resources that are needed for project enactment and activities. This can be accomplished by: o Initial face-to-face meetings with initiative/objective teams whereby direction, activities, and indicators are mapped out more concretely. o Providing for reflection points at periodic stages in project implementation to address issues, modify objectives, and revise activities and workloads if necessary. o Allowing flexibility if agency or staff cannot commit to an activity or certain timeline because of other pressures. f. For continued FORECAST activities, more clearly defined measurable outcomes and indicators for the overall goal and objectives as well as for each individual institution; ensure consensus among all as individual institutions and as a cross-sector group. This can be achieved by: o Holding an initial strategic workshop with all stakeholders to define goal(s) and objectives and assist agencies to set their own specific indicators of success, based on the broad indicators set for the project. 11 1. INTRODUCTION 1.1.Background and Purpose of the Assessment Background The FORECAST Project was designed to build human and institutional capacity within Government of Liberia (GOL) ministries, public institutions, NGOs, and other organizations. The Project, which commenced in September 2008, ended in May 2011. USAID/Liberia requested that the Liberia Monitoring and Evaluation Program (L-MEP) conduct an End-of-Project Stakeholder Assessment toward the goal of making recommendations regarding how USAID can continue to support Participant Training and Human and Institutional Capacity Development in Liberia. 1.2. Design and Conduct of the Assessment The End-of-Project assessment was designed to answer some key research questions, using a participatory methodology.  The Key Research Questions Based on the design, objectives, and expected outputs of the FORECAST Liberia Project, the following research questions were developed to guide the Stakeholder Assessment: 1. To what extent has the operational effectiveness of targeted organizations improved as a result of FORECAST training, technical assistance, and capacity building efforts? 2. Have the eight FORECAST outcomes been achieved and if not, what were the constraints to fully achieving them? 3. Were the capacity building activities of FORECAST consistent with the Task Order and if not, why not? 4. Was the management structure and processes of FORECAST appropriate for achieving project results, and if not, what changes should be made to any future management system? 5. Should USAID fund a future participant training and/or capacity building project, and if so, for what sectors, what balance between public and private sector beneficiaries, what balance between training and capacity building, what type of mechanism, and what management structure?  Methodology The methodology for the assessment was generally participatory. It employed a combination of key informant interviews (KIIs), individual interviews, focus group discussions (FGDs), and observations, based on the classification and participation of stakeholders in the implementation of the FORECAST project. For stakeholders within the upper tier of the FORECAST design and management, including USAID Contractors, relevant Mission staff, sub-contractors of FORECAST, and private sector/semi-autonomous agencies key informant interviews were used. For participants (stakeholders) falling within the larger tier of training participants, the assessment used stratified random sampling to select participants by organization, training duration and sector. Candidates for interview were drawn through stratified random sampling, so as to achieve a sample representative that will bear the characteristics of each stratum. Additionally, participants were drawn from a cross sectoral dimension to participate in a more heterogeneous focus group discussions, with the aim of drawing out lessons learned from FORECAST implementation from a broader perspective. Data gathering was guided by specific data collection tools, developed by Subah-Belleh Associates and administered by the assessment team. 12 Conduct of the Assessment  Organization and Management The Assessment was led by SBA, operating under sub-contract from the L-MEP. The team included professional staff of SBA, the L-MEP, and USAID.  Duration of the Assessment and People and Places Covered SBA designed and developed the field tools; they included questionnaire for individual and institutional interviews and focus group discussion. The field exercise commenced in March 2011, mainly in the Monrovia area, and moved out into five rural counties—Nimba, Bong, Margibi, Grand Bassa, and Bomi Counties—between April 17 and 23. Individual and group consultations and discussions were held with 66 persons. As a result, a total of 66 individual and group consultations were held with various stakeholders. See Appendix A for the list of persons and groups consulted and places visited. 1.3.Presentation of the Findings The presentation of the findings of the assessment is designed to provide answers to the key research questions. Accordingly, it looks at the following key areas: a. Impact of the Intervention on Operational Effectiveness of Targeted Organizations; b. Meeting the Eight FORECAST Objectives; c. Consistency of Activities with USAID Task Order; d. Effectiveness of FORECAST’s Management Structure and Processes; and e. Recommendations for the Future of USAID’s Capacity Building Support. 13 2. THE LIBERIA FORECAST PROJECT 2.1. The Situation before the FORECAST Project When the Liberia FORECAST Project was designed, Liberia was just emerging from 14 years of civil war, which brought widespread physical destruction to public and private infrastructure and created a major problem of “brain drain” across all sectors of society, particularly the civil service. The civil service was not in the position to manage post-war recovery and reconstruction as well as deal with long standing problems that impeded the effective management of public resources and the growth of the economy. In this post-war environment, human resource development was recognized at the time as a key to recovery. The UN, USAID and other support programs intervened generally by attracting skilled Liberians living abroad to return home and join the civil service, through the provision of topped up salaries and other incentives. Some of the major programs included the Senior Executive Service (SES), Transfer of Knowledge through Expatriate Nationals (TOKTEN), Liberia Emergency Capacity Building Support (LECBS), and the Scott Fellows Programs, among others. Capacity building interventions also included the offer of mostly short term training within public sector institutions and at the county and local levels for government personnel. Trainings covered such areas as computers skills, procurement, planning and management. While all of these interventions helped bridge the huge capacity gaps that existed, they were generally short-term, expensive, unsustainable, and uncoordinated and limited with regard to addressing, in a consensual way, priority capacity building need. They were based on the discretion of the institutions requesting the assistance and not necessarily supported by an organized needs assessment. However, later, a National Capacity Development Strategy (NCDS)1 was formulated. The HICD assessment, carried out by the FORECAST Project, significantly informed start-up activities of the NCDS. FORECAST also actively participated on the Technical Working Group of the NCDS, contributing inputs to the development of the strategy. 2.2.Establishment of the Liberia FORECAST Project It was in the environment described above that the FORECAST project sought to integrate USAID’s sector-wide capacity building funds so as to produce maximum and sustained performance improvement in critical social and economic sectors. At the time, most post-war development documents from the GOL and development partners stressed the need for quick action to restore basic services and give people a palpable sense that peace is bringing hope for a better future. The Liberia FORECAST Project was then launched in the early stages of the transition from relief to development and at a time when countervailing pressures to “act quickly” and “act strategically” had to be balanced. The GOL wanted quick action since it needed to build human capacity across all sectors as rapidly as possible. FORECAST was then designed to (1) fill post-war institutional performance gaps by building the capacity of mid-level professionals at the national and county levels; (2) increase operational effectiveness of targeted Government of Liberia (GOL) ministries, public institutions, NGOs and other partner institutions; (3) upgrade technical skills and improves efficiency and effectiveness of professionals and institutions in Liberia in the short and long term, with 30% target beneficiaries from the counties and the remainder from the capital. 1 The National Capacity Development Strategy (NCDS) is the Government of Liberia’s 10-year strategy/agenda for coordinating and advancing capacity development investments in the public, private, and civil society sectors. The aim of the NCDS is “to guide investments in capacity development across sectors in a coherent and coordinated manner and to facilitate the exchange of good practices in order to promote effective investments in the people and institutions that drive Liberia’s development agenda”. …… 14 2.3. General Description of the FORECAST Project The FORECAST Liberia Project is a Human and Institutional Capacity Development (HICD)/Participant Training Program. The task order for the Project was managed by the USAID/Liberia Education Office and implemented by the AED, in partnership with the GOL through the Ministry of Planning and Economic Affairs. Initiated in September 2008, the project ended in May 2011. The FORECAST Task Order states that the overall objective of FORECAST is to provide the training and technical services required to strengthen the base of skilled, high performing professionals and institutions in Liberia. As the Task Order states, these individuals and entities must be able to implement and support policies; exercise equity, accountability and transparency in managing public resources; provide better service delivery; participate more actively in and contribute more broadly to the country’s economic and social development; and generally impact all of USAID’s priority sectors, including education, health, economic growth, and democracy and governance. The major tasks FORECAST was expected to undertake, according to the Task Order included:  Improve management and technical skills, and create transparent, accountable practices that will impact the operational level within targeted GOL ministries, public institutions, NGOs, and other partner organizations;  Provide basic and advanced vocational training to help rebuild and maintain the public sector’s physical infrastructure;  Improve the performance of targeted local institutions through training and technical assistance;  Develop the capacity of local organizations to perform similar human capacity development activities. FORECAST was implemented by AED which was tasked with conducting an assessment to determine and prioritize human capacity development and workforce skills needs for the education, economic growth, democracy and governance teams2 and to mentor and develop the capacity of local organizations to manage similar training, human capacity and workforce development programs including technical assistance interventions to local key organizations for these three teams. AED was then to implement participant training and human capacity building programs for all four USAID sectors: health, education, economic growth, and democracy and governance. The FORECAST project goal, according to their Performance Monitoring Plan, is: Increased operational effectiveness of targeted GOL ministries, public institutions, NGOs, and other partner organizations. To achieve this goal, FORECAST was to achieve eight outcomes: 1. Management and technical skills of targeted departments improved. 2. Operational effectiveness of targeted departments increased. 3. Transparent and accountable practices adopted and enforced by targeted departments. 4. Increased capacity of Liberian workers to provide skilled labor to support public sector infrastructure. 5. Improved managerial and organizational performance of targeted local institutions. 6. Improved technical capability of targeted local institutions in the provision of key services. 7. Tailored HCD activities designed and delivered by targeted local organizations. 8. HCD assessments conducted by targeted local organizations. 2 Health was not included because a detailed capacity needs assessment of the health sector had already been done under the auspices of the Ministry of Health and Social Welfare. 15 3. FINDINGS OF THE ASSESSMENT The findings of the assessment respond directly to the Key Research Questions, which are formulated around the expected outcomes of the Liberia FORECAST Project. Generally, they provide the project’s stakeholders’ views on the impact of the Project, the relevance of the intervention, and the future of USAID capacity building support to Liberia’s development. Added to the stakeholders’ views are the analyses and conclusions of the assessment team. 3.1.Impact of the Intervention on the Operational Effectiveness of Targeted Organizations Research Question: To what extent has the operational effectiveness of targeted organizations improved as a result of FORECAST training, technical assistance, and capacity building efforts? The Project was designed on the basic assumption that the provision of training, follow-up/technical support, and other capacity building efforts would lead to improvement in the operational effectiveness of the targeted institutions/departments/units. 3.1.1. Achievements Reported by the Project Training and Technical/Follow-up Support Training: According to various reports and statistics produced by the FORECAST Project, the Project far exceeded its cumulative training targets across all sectors. The following table shows cumulative 2009 and 2010 trainings completed against planned targets. The total of 839 completed trainees for this period more than doubled the 385 targeted by the FORECAST Project for the same period. Adjustments in strategy and rationalization of priorities among partner/beneficiary organizations allowed for more and diverse short-term trainings that were critical to the management and technical skills needs to the institutions involved. Project Outcome and Output-level Indicators FY 2009 Total FY 2010 Total Cumm. to Date Target % Achieved Project Outcome: Management and technical skills of targeted departments improved Indicator 1.1.1.1: # of staff of targeted departments trained in management practices 46 21 67 60 112% Indicator 1.1.1.2: # of departments benefiting from basic management practices training 33 13 46 20 230% Indicator 1.1.2.1: # of staff of targeted departments completing technical skills training 364 239 603 285 212% Indicator 1.1.2.2: # of departments benefiting from technical skills training 117 60 177 20 885% Total 560 333 893 385 In 2011, additional trainees completed training, further increasing the number of actual trainees significantly beyond the targeted numbers. It is estimated, from data produced (but not verified) by the Project, that individuals who had completed training and were in training at the time of the assessment combined to take the total number of trainees to around 983, as shown in the following table 16 Training and Technical Assistance Beneficiaries (All Sectors) (Summarized from FORECAST Project Reports – Details in Appendix 2) Sector No. of Participants by Type of Training Employer/Beneficiaries Long-Term* Short-Term* Total Training Health Sector 64 55 119 MOHSW/TNIMA/JFK, Phebe, ELWA, Redemption, MGD, GOL/Senator, Handicap International, Cuttington Education Sector 14 157 171 MOE, UL, SMP, WAEC, RTTIs Governance and Democracy 0 458 458 MLME, RIA, FDA, NGO (LIWOMAC), GSA, NEC, CBOs, LACC, LIPA Economic Growth 22 149 171 MPW, MLME, MOA, GoL/Private/Tech Providers, Others Cross-Cutting 3 61 64 GOL (MOE, MOH, MPW, MPEA, MOA, MOF, LIPA, BWI) Total 103 880 983 Technical Support Governance & Economic Growth Budget Advisor Ministry of Finance SunSystems Expert Trainer Forestry Curriculum revision Forestry Training Institute M&E Capacity Development Follow-up Support (Provided by FORECAST Project’s M&E Specialist) M&E Staff of GoL and M&E Partners MOH, MOE, MPW, MOA, MOPEA * Short-Term Training was considered to be any training lasting 6 months or less, while Long-Term Training was defined as training with duration of more than six months. Follow-up of Training/Technical Support: Follow-up and technical support was intended to assist returned trainees and beneficiary/partner organizations improve their systems, processes, accountability, and general organizational effectiveness. In this direction, the Project undertook a number of support/technical assistance activities within partner institutions. The Project also assisted in the setting up of trainee network groups—in the areas of computer skills training, basic management, and monitoring and evaluation. These were post-training professional support groups in which members formed a “community of practice” to assist each other apply their learned skills and further develop their professional competence and standards. FORECAST utilized the network groups as critical vehicles for the provision of technical support and the monitoring of the application of learned skills in their respective work environments. According to FORECAST, technical assistance activities were provided on a demand basis and activities took the form of mentoring and coaching sessions. The Project’s 2010 Annual Portfolio/Performance Review provided some examples of the kinds of technical assistance activities undertaken:  MOH&SW: Review of assessment recommendations implementation was carried out and the report presented to the appropriate authorities for necessary actions to be taken. Support was also provided to train newly recruited county M&E officers.  MPEA: The National M&E Framework has been drafted and is currently being edited by the respective ministers. Introductory M&E sessions are being conducted for new staff members. (FORECAST provided technical support).  NCDU: The final draft of the National Capacity Development Strategy (NCDS) has been completed and is to be presented to Cabinet. (FORECAST provided technical support)  MOE: Introductory M&E sessions are being conducted for staff members preparing to undertake M&E activities within their units. 17  MOA: The M&E system assessment has been conducted. Worked with M&E team to operationalize the national MOA M&E system.  MOGD: On-going assistance in M&E system development and data management.  LIPA: M&E training course package drafted and currently being reviewed/completed by the working group. The Project also provided technical support to institutions in a number of other ways. For instance, support was provided in helping to apply computer/IT knowledge and skills to day-to-day work activities. (Further in this section, (3.1.1) there are examples of support provided and the impact of such support). Capacity Development A key strategic requirement of the FORECAST Project was to build the capacity of local institutions to carry on after the Project. According to USAID’s Task Order, the Project was specifically tasked to: “Develop the capacity of local organizations to perform similar human capacity development activities.” In fulfilling this task, FORECAST built two levels of partnerships for human and institutional capacity building (HICD). FORECAST’s HICD partners included local non-governmental organizations and public partners. The Private/NGO partners included the Agency for Economic Development Empowerment (AEDE) and Mano Training and Development Foundation (MADET), while the public partners included the Liberia Institute of Public Administration (LIPA), Booker Washington institute (BWI), and the General Services Agency (GSA).  Private/NGO Partners Through facilitated cross-assessments, individual assessments, training, and other capacity development approaches, the Liberia FORECAST Projects sought to build the capacities of the local private/NGO partners in HICD assessment methodologies and approaches. Some private/NGO partners also benefitted from training capacity development in order to be able to plan and deliver training in specific fields. The following matrix shows the private/NGO HIDC partners supported by the FORECAST Project and the areas of capacity enhancement they received from the Project. It also includes Stella Maris Polytechnic, which benefitted, to a limited extent, in a Training of Trainers (TOT)-type exercise supported by the Project. 18 Private/NGO HIDC Partner Area of Capacity Development Supported by FORECAST 1. Agency for Economic Development and Empowerment (AEDE)  Provided institutional assessment of AEDE and supported AEDE’s own internal capacity building to carry out institutional capacity building needs assessment  Provided with resources to assist partner build capacity to carry out assessment 2. Mano Training and Development Foundation (MADET)  Provided institutional assessment of MADET and supported MADET’s own internal capacity building to carry out institutional capacity building needs assessment  Provided with resources to assist partner build capacity to carry out assessment 3. Stella Maris Polytechnic (SMP)  External training provided for 2 teaching staff: CISCO Training and Certification (CCNA Professionals)—to build institution’s capacity to provide advanced computer training to the public (all sectors).  Public Partners The FORECAST Project supported institutional capacity building for public partners in order to strengthen and expand their service capacities: Booker Washington Institute FORECAST provided support for the training of four drafting instructors at BWI in the use of computer aided design (AUTOCAD). AUTOCARD is a set of three-dimensional drafting software used by construction design engineers that allow calculations and revisions to be more precise, quick and easy as compared with the manual technical drafting technique. The support was intended to build the capacity of four instructors to deliver training and practice using AUTOCAD. As a result of this support, BWI now has four trained instructors who run all of their drafting training and practice using AUTOCAD. Liberia Institute of Public Administration (LIPA) The Project assisted LIPA to build its capacity to deliver training in monitoring and evaluation and human resource management. Two LIPA staff underwent training with the African Medical and Research Foundation (AMREF) in Kenya, followed by TOT, in M&E. A second TOT was conducted in Human Resource Management, with a focus on performance-based capacity planning. The HRM TOT included heads of human resource departments from seven partner ministries, the Civil Service Agency, selected LIPA training staff, and representatives from project partner NGOs AEDE and MADET. In order to understand and more effectively target its support to LIPA, the Project also carried out, through AED, a comprehensive institutional assessment of the organization. The support to LIPA enabled the institution organize and manage training events (in M&E and HRM) it had not offered in many years. It even placed LIPA firmly in a position to support the implementation of the National M&E Policy for Liberia as well as strengthen performance management and M&E across the public sector. General Services Agency (GSA) Prior to the FORECAST Project, the General Services Agency (GSA) felt the need for information technology (IT)/computer skills training capacity building and approached the International Business Initiative (IBI), the implementers of the USAID-funded Liberia Improved Budget and Asset Management Program (LIBAAM), for support. IBI assisted the GSA build a limited infrastructure. It was soon realized, however, that the problem of the GSA was the same across all sectors of the Government. Thus, the GSA again engaged the IBI to strengthen its capacity to provide training to others. IBI responded with the setting 19 up of a fuller IT and training infrastructure. A program of asset management for the government was started. With the end of the IBI LIBAAM Program, the FORECAST project worked with the GSA to further strengthen its capacity to provide computer and IT training as well as skills to manage the IT and training program. Two of GSA’s staff were trained in Ghana in IT Service Management. Four major activities followed, which constituted support in this sector. Beginning with Fundamental IT training for 181 Government of Liberia employees, the Project not only provided support for the GSA to become an internationally certified computer/IT training institution, the first in the country, but also assisted 17 ministries and institutions provide basic skills for use on the job. The IT facilities at GSA were the location of the trainings. Impact on Operational Effectiveness of Targeted Institutions The following table (produced by FORECAST) shows the outcomes expected from training and capacity building activities under the FORECAST Project. It also shows the indicators intended to measure the outcomes and the Project’s assessment of progress towards achievement of the expected outcomes. Much of the information provided here were gathered through the administration of post-training assessment questionnaires to the beneficiaries as part of the FORECAST Project’s regular monitoring, evaluation, and post-training follow-up and technical support activities. Performance Indicator FY 2009 Total FY 2010 Total Cumm. to Date Target % Achieved Project Outcomes: o Operational effectiveness of targeted departments increased. o Transparent and accountable financial practices adopted and enforced by targeted departments Outcome level Indicators Indicator 1.1.1: % of targeted departments that demonstrate improved skills in management practices. - 60% 60% 70% 86% Indicator 1.1.2: % of targeted departments that demonstrate improved technical skills in service provision. - 67% 67% 75% 96% FORECAST Liberia’s reports (annual reports and M&E assessment reports) provided information on some specific activities undertaken, in addition to training, to arrive at the level of impact reported by the Project. The following extract from a Project report shows some of the interventions undertaken by the Project and their assessment of the progress made by the beneficiaries in impacting the effectiveness of their various organizations: The establishment and operations of professional networking groups have helped translate training into practice and provide a reference point for FORECAST’s own assessment of progress, though in a limited way. The following are examples of outcomes reported by the Project: Basic Computer Skills Networking Group Information gathered from the interaction revealed that beneficiaries were generally pleased with their improved abilities to perform tasks on time and with improved accuracy. This, they said, reflected in increased satisfaction expressed by their supervisors for their work done since returning from the FORECAST training. Achievements generally included reduced time for their departments and in the preparation of official documents and with improved quality. A summary of achievements by the use new skills include:  Working in MS-Excel to do calculations,  Using in MS-Word to do calculations, 20  Very effective in document preparation,  Being able to train their colleagues in the use of the computer for effective work,  Developing PowerPoint presentations for their supervisors to use during workshops. Consensus was reached that beneficiaries needed functioning and reliable computers to work with. Most of them had not fully utilized their skills either because they did not have computer (or the appropriate software) or their supervisors did not give them enough assignments to warrant the use of their improved skills. Unreliable electricity supply and Internet services as well as the menace of viruses were also significant challenges faced by beneficiaries. (See network meeting report for full details) Basic Management Skills Networking Group Since returning from the FORECAST training, beneficiaries have put a number of measures in place towards improving administration and management practices at their various work places. These include: o Introducing weekly and monthly work plan and reporting schedules. o Conducting regular staff meetings to plan/discuss activities of their units/departments. o Replicating the training in the form of in-house briefings for their colleagues. Two of the beneficiaries had also been transferred from their unit to another unit as a result of their new skills acquired from the FORECAST training. The initial effects of these initiatives on the overall performance of their units/departments are anticipated in the coming months. The major challenge faced by the beneficiaries was the failure of LIPA to deliver the course materials to them as promised, thus leaving the beneficiaries with very limited resources for reference. Other challenges include inadequate materials/equipment to effectively do their work, non-cooperation by some co-workers who saw the beneficiaries as threats due to their new skills acquired, the slow response of co￾workers to the innovations being introduced, and insufficient transportation to reach out to the counties. According to FORECAST, with regard to management skills, 60% of the departments / units / divisions supported recorded improvement, while technical skills in service provision recorded 67% improvement. FORECAST’s results are presented in the following charts (produced by FORECAST Project): Managerial Improvements 21 Technical Improvements FORECAST Project’s Criteria for Assessment of Improvements in Operations of Departments/Units Management Skills  A framework has been set for participatory decision-making.  There are policies that guide performance of activities.  The guiding policies are up to date.  The guiding policies are understood by all staff.  Plans are created to cover relevant areas of the unit’s operations.  Created plans are implemented effectively.  Work procedures are developed for the department.  It is ensured that the work procedures are followed.  Official needs of staff members are provided. Technical Skills  With the use of skills acquired, the department has aligned its operations with the service provided by its umbrella institution.  The department tracks how they do their work with respect to new skills acquired  The department sets the right targets with use of the new skills, tracks the right outcomes and takes appropriate action to see if the two tally.  The organization hires, develops and keeps the right people and provides them with incentives to succeed as regards the use of the skills supported by FORECAST. 3.1.2. Stakeholders’ Assessment Training and Follow-up Technical Support Stakeholders generally assessed the FORECAST support to training to be successful in building managerial and technical skills in a variety of ways across the various sectors. Both individual and institutional beneficiaries spoke extremely positively. However, perhaps the most positive expression of satisfaction came from beneficiaries of the Health Services Management Training Course, delivered by Mother Patern College of Health Sciences. All health workers who benefitted, particularly county health officers, said the training provided the management skills they needed to effectively run health programs. With only their 22 medical training, most indicated that they had struggled to manage health services because they did not have the requisite managerial training. But they now assert that they are better managers and their institutions and they are benefitting enormously from their new skills. In the words of a medical director interviewed, “I know now that I can effectively run the hospital and still find time for myself. In all my years as a doctor, I have not taken a vacation, but this year, I will be able to take one because I have all the systems and processes in place for the facility to run well in my absence”. In their own words, various stakeholders assessed the FORECAST training and development support in the following ways:  “FORECAST was one of the most successful projects in which the Ministry was ever engaged.” - Deputy Minister, for Administration, Ministry of Education  “A lot of what I am doing and am capable of doing in M&E come from FORECAST. There are things I never would have been able to do without the training I received. Being called by my former institution (employer) to work as a consultant shows the success of my training experience.” - a former Governance Commission (GC) employee who benefitted from overseas M&E training.  “FORECAST gave the best services and support possible to us while in training.” - a University of Liberia staff, who benefitted from Master’s Program in Public Sector Management at GIMPA (Ghana)  “We gained a lot of experience from the training. For instance, it was the first time for most Eos to play with computers, develop work plans, and do PowerPoint Presentations.” - a County Education Officer, who attended the Management, Supervision, & M&E Course for CEOs and DEOs, delivered by MDI (Swaziland) in Gbarnga, Bong County.  “The training was useful. My technicians who were trained are now better workers. For me, as a coordinator, in the past, I did not have the technical skills to supervise properly; I had only managerial skills. Now, I can supervise my people more effectively.” - a County Pump Maintenance Supervisor, who, along with his pump maintenance technician, benefitted from technical training in pump maintenance provided through FORECAST.  “The training program was a tremendous help. It has helped me to achieve speed and accuracy. It has presented us with a very good future here at BWI.” - a BWI Drafting Instructor, who benefitted from training in computer-aided designs (AUTOCAD) and is now training his students to use it.  “FORECAST provided opportunities that did not exist anywhere else. When I was informed that I will be taking over a new hospital as medical director, I went scouting for some sort of short-term management training course. Cuttington told me, for instance, that if I needed to get management training, I would have had to enroll in a 4-year college program. But that was not what I needed. The Health Services Management Course provided by Mother Pattern, through the FORECAST Project’s support, was exactly what I needed…..it met both my needs and my expectations.” - a Hospital Medical Director who benefitted from the Health Services Management Course provided by Mother Pattern College of Health Sciences, through FORECAST’s support. Capacity Building All stakeholders (HICD partners) acknowledged the support of the FORECAST Project in building their capacities to strengthen the environment for human and institutional capacity development in Liberia. They were generally positive about the support provided by FORECAST in strengthening their capacities to carry out local training and HICD assessment and process management. There overall impressions were captured in the following comments:  “The M&E aspect of FORECAST’s support was particularly successful. FORECAST put a lot into making this work. FORECAST assisted in bringing M&E actors together….” - an executive of LIPA 23  “Working with FORECAST was smooth….” - a manager of the GSA IT Center  “AEDE is now stronger, better with FORECAST’s assistance. AEDE is even embarking institutional capacity building needs assessment of other organizations.” - Maxwell Nimely of AEDE representative at Stakeholders Retreat at Kendeja Resort Despite their overall positive commentaries, HICD partners had some concerns over the way the project was organized and communicated with partners, at least in the beginning, as expressed in the following comments of two of the partners:  “For a long time, there was a lack of understanding about how far FORECAST could go. In the end, selection was done only by FORECAST, without the involvement of LIPA. Even the criteria used for selection were not known.” - Prof. Harold Monger (LIPA)  “We were not very clear for a long time what FORECAST was all about.” - Sis. Barbara Brillant (MPCHS) Impact of Training and Technical Support on Organizational Improvements The impact of training and other capacity building activities on individual and institutional beneficiaries (departments and units) was the main focus of the Stakeholders’ Assessment. However, because the Stakeholders’ Assessment was not a full—fledged evaluation of the Project, our findings are based on the stakeholders’ own assessment and cannot necessarily validate the FORECAST Project’s own assessment. A more detailed evaluation would have sought to independently assess changes in the beneficiary institutions, measured against certain baseline data and based on agreed measurements and criteria for change. It would have also systematically sought the perspectives of service users in assessing progress towards change in the level of effectiveness and efficiency in service delivery. The Assessment Team sought stakeholders’ own assessment of what had changed and how such changes could be attributed to the FORECAST Project. The views of returned trainees and managers and organizational leaders were consolidated into the findings presented here. According to the stakeholders, the interventions made by the Project have had significant impact on their staff and their organizations as a whole. However, there were sometime mixed views between trainees and managers over the extent of progress that had been made with institutional improvements. The following presents a select sample of outcomes, comments, and success stories intended to capture the general views of stakeholders regarding the impact of the FORECAST support on their institutions. Ministry of Education The Ministry of Education was perhaps the biggest beneficiary of the FORECAST Project’s support in terms of both expenditure and short and long-term training benefits. At least 166 employees benefitted long- and short-term training. Appendixes 1 and 2 contain details of the numbers and types of short and long-term training provided by sector and beneficiary institutions. Comments Regarding the FORECAST Support According to the Ministry’s top officials:  “FORECAST is one of the most successful projects in which the Ministry has ever engaged. The only problem, though, was that the process (selection of trainees) was too long.”  “The trainees have been doing very well.”  “We need incentives for retaining those trained.” 24 According to Returned Trainees:  “The training has given us new and much needed skills and confidence.”  “Resource challenges are preventing us from putting into practice what we learned.” This was particularly true for the M&E trainees and the Guidance and Counseling Trainees—who visibly lacked the basics to do their day-to-day work. 25 Success Story: The Internal Audit Section Prior to the FORECAST intervention, the Internal Audit Section of the Ministry was dead. However, as a result of the training provided for its staff, the Section is now completely revived and functioning effectively. The Ministry has provided the Section with the basic resources it needs to do its work and a more professional internal audit is now in place. The training gave the staff the knowledge, skills, and confidence they needed to take on their responsibilities. Ministry of Health and Social Welfare The Ministry of Health ranks next to the Ministry of Education in terms of the substantive nature of FORECAST’s Training support. The project delivered at least 85 short and long-term training opportunities to central and county-level staff of the Ministry, including staff of the JFK Hospital, the Redemption Hospital, and TNIMA. See Appendixes 1 and 2 details of the numbers and types short and long-term training provided by sector and beneficiary institutions. Comments Regarding the FORECAST Support According to the Ministry’s top officials:  “Through the training FORECAST supported for our HR staff, the Ministry has now transitioned from the use of SES staff to the Ministry’s own staff in the leadership of the HR Section of the Ministry.”  “Counter-parting with external consultants and experts is now possible—with the increased capacity of newly trained staff under the FORECAST support. For example, when the Ministry’s new 5-Year Plan was being developed, Ministry’s staff, trained through FORECAST (particularly in health financing and M&E) were actively engaged, easing the workloads of deputy and assistant ministers.” According to Returned Trainees:  “The training was good—the kind Liberians need. We are seeing the difference what we did before and what we were supposed to do. Therefore, we are now constantly trying to improve our management practices since we have learned the skills to do so.”  “The Ministry is very supportive in helping us put into practice what we have learned, though some resource challenge do exist that limit what we can do.”  “Before, I, as head of the hospital, did everything. I hired, fired, purchased, etc. But now, I realize that those are not my responsibilities. The training made me realize that most things I did before were wrong. Now, I can wisely use my time. We now practice work planning and monitoring. There are now regular early morning meetings to review the previous day’s work, weekly strategic planning meetings with sr. staff, and monthly general staff meetings. These are management practices that I have put in place since I returned from training.” Success Story: The Government Hospital, Tubmanburg, Bomi County The current Medical Director took over the hospital amidst huge challenges: There was no management system in place, an uncomfortable work environment, poor patient record system, etc. But the training received by the medical director, through the FORECAST support has changed things, according to the medical director. There are now improved management practices in place; planning and performance management activities (Gantt Charts, for instance, are being used to track progress of planned activities) are helping the medical director to more effectively manage the hospital. In addition, the quality of patient records has been improved, through the introduction of an improved records management system. In the words of the medical director, “I am learning 26 and actively practicing what I learn. Today, Bomi is the proud winner of the Ministry’s performance award. This is what the training has helped us to achieve.” 27 Ministry of Gender & Development The Ministry of Gender and Development benefitted in a limited way from FORECAST’s support, but the Ministry’s own assessment is that the impact of the FORECAST support was enormous. No specific interviews were conducted with the Ministry of Gender (the former M&E Officer of the Ministry participated in group discussion), but the Minister of Gender officially communicated the Ministry’s assessment of the impact of the FORECAST support on the Ministry. It is explained in the following success story: Success Story: The M&E Unit According to the Minister of Gender, in a letter to FORECAST: “Since your organization engaged the M&E Unit of the Ministry of Gender and Development, we have seen remarkable improvement in the performance of the M&E Unit and related divisions. The establishment of the M&E system at the Ministry has begun to take root for the first time since the creation of the Ministry nine years ago.” University of Liberia The University of Liberia benefitted from limited numbers of training opportunities (6 short-term and 2 long-term). The institution also served as trainers of a FORECAST-sponsored training event (Science Education Training for Science Instructors of public schools from around the country), though some unanswered questions surround the actual delivery of the training, as explained in Section 3.1.3 below) Comments Regarding the FORECAST Support The assessment team held only one group consultation with returned trainees at the University of Liberia. Their assessment of the impact of the training on them and the University of Liberia was assessed in the following words:  “Staff returning from long-term training have been provided with increased salaries and resettlement allowances.”  “The training enhanced our research skills, which we are utilizing in university teaching and research.”  “Communication and working relationships have improved between us and our colleagues and students.” 3.1.3. Conclusion and Lessons Learned Training The training opportunities delivered through the FORECAST Project provided undoubtedly the most impressive and effective undertaking in capacity development in across sectors in Liberia for a long time. It helped build critical technical and managerial skills that will help to professionalize and strengthen the capacities of the targeted institutions on a long-term basis once they commit to providing the needed resources and platform for returned trainees to translate their skills into practice. For the vast majority of beneficiaries, the opportunity for training was their first real capacity building experience--after years of service in the public sector. For the beneficiary institutions, the impact was not immediately obvious as it takes time, resources, and commitment to translate skills and capacities into operational improvements. Despite the reported high levels of intake (and achievement, as far as training targets were concerned) and the positive assessment of the stakeholders, two important observations were made: 28  Some beneficiary institutions complained that they thought they should have had more slots for training. Despite working with them to prioritize their needs, many institutions had not taken care to link their requests for training to their most critical needs and priorities and had submitted to FORECAST numerous requests for training, particularly long-term overseas training, some of which were denied by the Project. There was a common belief in the Project that some public sector institutions had wrongly viewed the project to be an open-ended opportunity for scholarship and, thus, made numerous and unjustifiable requests for staff training.  A set of trainees could not be located. Some 60 (58 males and 2 females) science teachers from schools all over the country were reportedly trained in a Science Education Workshop in Kakata, Margibi County, by the University of Liberia, but not a single one of the beneficiaries could be identified at the various schools they were said to have come from. In all cases investigated in the counties covered by the assessment, neither the schools, nor the District Education Offices, could identify the names and individuals reported to have attended the training. Most or all had no idea that any such training had taken place or that any of their teachers had attended. The FORECAST Assessment Team investigated the reasons that the team was unable to locate any of the registered participants in the training. The team first asked the FORECAST Chief-of-Party (COP). She told the team that the training event had not been organized by them, but instead had been organized by USAID. She also said that the FORECAST M&E staff had made an unannounced visit to the venue at the start of the training and found “unverified” participants in attendance, all carrying name tags. The Assessment Team was informed that the Ministry of Education, which should have been the institution making the request for such a training, was not aware of the training. The Team was also informed that this issue had, at some point in time, come to the attention of the Ministry of Education and that the Ministry at the time, through Former Deputy Minister for Instruction, Mrs. Hester Williams-Katakaw, had demanded that the funds expended on the event, which had not been authorized by the Ministry, be returned to the Ministry’s “allocation” under the FORECAST Project. However, the issue apparently became was dropped after Deputy Minister Williams-Katakaw was replaced. When Mrs. Katakaw was contacted by the Assessment Team, she said she could not fully remember what the situation was at the time, but did recall “problems” with that training, which she said was delivered by persons from the University of Liberia. She clearly stated that she, as Deputy Minister for Instruction, had not been aware of the training and had not recommended the persons involved for training. As facilitators of the said training course, the University of Liberia’s Department of Physics, located in the College of Science and Technology, was also contacted for comments on the issue. According to Mr. Emmanuel Johnson, the then Coordinator of the Course, the University’s Physics Department only served as facilitators of the course and cannot speak to such issues as the origin and authenticity of the participants. He said someone claiming to be from the Ministry of Education did provide them with a list of trainees and persons on that list were the ones who claimed to have been in the training. He said all other aspects of the training, including payments to service providers and others, were outside their interest and control. Mr. Johnson then referred us to one Mr. Koko Gray, formerly of USAID, who he said might have an idea how the program was organized, since he had some oversight at the time over the FORECAST Project. Mr. Koko Gray was contacted by the Assessment Team. He claimed to have no knowledge of the details of the training in question. He said his role at USAID at the time did not include determining training priorities for institutions, or selecting or recommending their trainees and service providers. He said the FORECAST Project should have the answers being sought on this question since the project fully funded the event and made financial payment to all involved. The Assessment Team then contacted Mr. Thomas Wobill who had been the FORECAST M&E Specialist at the time of the subject training. Mr. Wobill told the Assessment Team that he had identified the fact that the actual trainees were not the ones officially registered for the course. He drafted a field monitoring report which is attached as Appendix 9 to this report. In that report, he stated that It was evident that the organizers of the science education training presented false information to FORECAST to collect per diem, T&T and accommodation allowances. The teachers were all non-existent, even in the county where the 29 training activity took place.” In an email to Mark Bassie data, L-MEP M&E Specialist for the Education Sector, Mr. Wobill said that “this report was submitted to Rebecca Cusic (COP/Liberia) with cc to Susan Bouldin (Project Director/Washington) and I was told by Rebecca that they were working on it. This was supposed to be a confidential report to Rebecca and Susan. Rebecca told me that she and Susan were responsible to appropriately inform the respective officials in AED, USAID, and MOE for the necessary actions to be taken. She even informed us that the AED lawyer was working on the issue.” The Assessment Team could find no evidence that FORECAST followed up on Mr. Wobill’s concerns. One of the staff who has worked in the Training Department of FORECAST during the time of the subject training reported that the training was not processed through that department as other training has been. These generally positive, but mixed, views seem to suggest some fundamental challenges with planning, partnering, and communication affected the management of the FORECAST Project. The Management of the Project did admit that the pressure of time and the desire to show early and quick results had an impact on process and the level of communication and collaboration established during project implementation. Capacity Building The approach to building local capacity to deliver training, which would have otherwise been accessed externally, was a huge contributor to the FORECAST effort, particularly in the midst of resource constraints. The sheer numbers of persons trained and the establishment of new/additional public sector training capacity dedicated to serving all sectors presented an opportunity for continuous training and development over time. A post-training assessment of some trainers showed that a “big change” occurred and the demands for training were coming in from all sources in the public sector. In the words of the head of the GSA facility, Joe Tuah, “ FORECAST met our expectations”. For private/NGO partners benefitting from HICD capacity building, the outcome contains a huge potential for the future of capacity building as it establishes local capacity to assist institutions identify their capacity development needs—a key step to addressing institutional capacity needs. Impact on Performance Generally, it was difficult to obtain a fair assessment of the impact of training and other capacity building activities on institutional systems, processes, and performance. Three challenges made this difficult:  It usually takes a long time after capacity building activities to measure the impact of interventions. In the case of the FORECAST Project, the Stakeholders’ Assessment took place at a time when many trainees were just returning from training and many others still in training. This was basically a period for settling down and assessing what changes and improvements were possible.  No institution—specific set of indicators were established to measure the impact of training on performance prior to the commencement of the FORECAST intervention.  Most institutions are saddled with huge resource challenges that the cost of introducing change was a matter of major concern.  Some institutions still held the view that the FORECAST Project should have gone beyond training and provided support to organizational improvements that were expected to follow training. Despite these challenges, though, there was a general impression that all institutions were better off with improved technical and managerial skills and that over time, these should have a positive impact on the performance of the organizations. However, the level of commitment to change has to be firmly established and resource commitments made to overall organizational performance improvements. 3.2. Meeting the Eight FORECAST Objectives 30 Research Question: Have the eight FORECAST outcomes been achieved and if not, what were the constraints to fully achieving them? There are mixed results as far as the achievement of the eight FORECAST outcomes is concerned. FORECAST Liberia’s own assessment is that it was most successful in delivering management and technical training and supporting the development of local capacities to provide HICD services, including the conduct of needs assessments and the delivery of training in such areas as monitoring and evaluation, general management, computer skills training, etc. According to FORECAST, these accomplishments have manifested themselves in some ways in improved performance in the targeted organizations. The various stakeholders covered by the assessment confirmed much of these successes. However, there were areas of much difficulty in accessing evidence of outcome achievement. This was particularly true for the following outcomes:  Operational effectiveness of targeted departments increased.  Transparent and accountable financial practices adopted and enforced by targeted departments  Increased capacity of Liberian workers to provide skilled labor to support public sector infrastructure  Improved managerial and organizational performance of targeted local HCD partner institutions Expected Outcomes Level of Achievement Comments Outcome 1: Management and technical skills of targeted departments improved. Evidence found of persons trained. An estimated 983 individuals from both public and private/NGO institutions trained in various areas of technical and managerial skills. Outcome 2: Operational effectiveness of targeted departments increased. Little and scattered evidence of changes found. Some success stories are documented in 3.1.2 of this report Some level of improvements in individual and institutional capacities recorded. Outcome 3: Transparent and accountable financial practices adopted and enforced by targeted departments No specific evidence identified to support the achievement of this ooutome. T was even not so obvious how the support of the project would have achieved this or how the change would have been measured. No specific reports showed what specific challenges existed in this area of transparency and accountability. However, it was hoped that training in management and technical skills would have influenced the development and implementation of improved management practices, including the introduction of transparency and accountability in systems and processes. Outcome 4: Increased capacity of Liberian workers to provide skilled labor to support public sector infrastructure Not much reportedly achieved. Only one main training event was recorded in support of this outcome—technical training for pump technicians (MPW). Indirect support included technical training in computer-aided designs (AUTOCAD) for BWI and support for the development of a national policy on technical and vocational education. Outcome 5: Improved managerial and organizational performance of targeted local HCD partner institutions No evidence There was no information available by which a fair assessment of organizational management improvements could be measured. It was not even clear what specific project activity (ies) were designed to achieve this result. Outcome 6: Improved technical capability of targeted local HCD institutions in the provision of key services. Some evidence There was evidence that local HICD partners had undergone training and practical exercises in organizational capacity assessment. However, there was no evidence that the institutions had successfully undertaken any independent HICD activity for which they were trained. Outcome 7: Targeted local HCD organizations design and deliver HCD activities No evidence to some evidence As part of th training process, HICD partners (AEDE and MADET) participated in cross HICD assessments of each other’s organizations. There was no evidence that the partners had carried 31 Expected Outcomes Level of Achievement Comments out any independent external organizational assessments, though one reported preparing to embark on organizational assessment activities. Conclusion and Lessons Learned The FORECAST Project did not achieve all eight of its expected outcomes, though there were clear links between the activities undertaken and the possibilities of impact on several or all outcomes. Judging from FORECAST Liberia’s own statement of the challenges the Project faced, it was always going to be difficult for a small project organization, with limited resources, over a short period of time to accomplish all eight targeted outcomes. Given the nature of the capacity challenges in the partner organizations that the project had to deal with and their (partners) general misconceptions about the Project, FORECAST Liberia spent an enormous amount of time and concentration on educating the partners about the objectives of the Project, getting partners organized in order to work with them and meet the Project’s requirements for participant training, rationalizing training requests, organizing local and overseas training for successful candidates, and managing trainees’ stay in training. Besides the challenges associated with organizing and facilitating training for well over 900 persons, measurement of impact was difficult for two main reasons: 1. No clear and realistic performance targets and current baselines were set for some outcomes. For example, for Outcome 3 (transparent and accountable financial practices), it was not particularly obvious why this target was set or how the FORECAST interventions would have successfully led to this outcome and in what particular institutions. It was not also clear whether the inclusion of this particular Project Outcome was a result of the notion that the lack of transparent and accountable financial systems and practices was largely due to the lack of adequate skills and capacities within the institutions. Another example of an unclear outcome is Outcome 5 (Improved managerial and organizational performance of targeted local HCD partner). While this was a realistic target—in terms of possibility for achievement—it was not clear how the support activities provided by the Project would have led to this outcome. It seemed that the focus of working with HCD partners was on building their capacities to perform HICD activities, particularly institutional needs assessments. 2. The time between the actual beginning of training and other capacity building activities and the introduction of change was so short that it was inconceivable that a lot would have happened in terms of impact by the expiration date of the Project. In fact, some trainees were still in training at the time of this assessment, while other would still be in training by the time the Project finally ends. Consistency of Activities with USAID Task Order Research Question: Were the capacity building activities of FORECAST consistent with the Task Order and if not, why not? FORECAST Project - Results Framework - Outcomes Outcome 1: Management and technical skills of targeted departments improved. Outcome 2: Operational effectiveness of targeted departments increased. Outcome 3: Transparent and accountable financial practices adopted and enforced by targeted departments Outcome 4: Increased capacity of Liberian workers to provide skilled labor to support public sector infrastructure Outcome 5: Improved managerial and organizational performance of targeted local HCD partner institutions Outcome 6: Improved technical capability of targeted local HCD institutions in the provision of key services. Outcome 7: Targeted local HCD organizations design and deliver HCD activities Outcome 8: Targeted local HCD organizations conduct assessments 32 USAID’s Task Order to AED for the FORECAST Liberia Project contained four (4) major tasks the Project was expected to undertake:  Improve management and technical skills, and create transparent, accountable practices that will impact the operational level within targeted Government of Liberia (GOL) ministries, public institutions, NGOs, and other partner organizations;  Provide basic and advanced vocational training to help rebuild and maintain the public sector’s physical infrastructure;  Improve the performance of targeted local institutions through training and technical assistance;  Develop the capacity of local organizations to perform similar human capacity development activities. In implementing USAID’s Task Order, the FORECAST Liberia Project undertook these key activities:  Facilitation of Participant Training for targeted public and private/NGO institutions.  Follow-up of training and related technical support to partner organizations  Support for local capacity building to carry HICD activities, particularly needs assessment. Although stakeholders took a generally positive view of the necessity of, and support provided by, the FORECAST Project, their clear lack of adequate understanding of the Project, created an inadequate basis for their fair assessment of the Project. For many, it was thought that FORECAST should have done more (in terms of resource provision) in translating training into practice and providing more training opportunities. Conclusion and Lessons Learned In comparing FORECAST Liberia Project’s activities against the USAID Task order for the Project, sufficient compatibility was found, though the depth of the activities were not adequate enough for achieving the full results expected from implementing the Task Order. For instance, while training was provided across all sectors and capacities of HICD partners were built to some extent, follow-up of training and technical support was grossly inadequate--mainly limited to M&E activities. In addition, as already stated elsewhere in this report, it was not always clear how training and technical support interventions would have facilitated change and on what scale, or what beneficiary institution were required to do to ensure success of the effort. On the other hand, there were areas in which the scale of activities undertaken made little or no impact on fulfilling certain elements of the Task Order. A particular case in point was the task order to provide basic and advanced vocational training to help rebuild and maintain the public sector’s physical infrastructure. In this area, only very little support was provided. The only directly related training recorded was one provided for county pump technicians. However, the Ministry of Public Works also benefitted from management and M&E training, which is helping them better perform their leadership role in the infrastructure sector. 3.3.Effectiveness of FORECAST’s Management Structure and Processes Research Question: Was the management structure and processes of FORECAST appropriate for achieving project results, and if not, what changes should be made to any future management system? The Project Results Framework (Cross-Cutting Issues) identified the basic activities and outputs expected in the management of the Project. These generally represented internal requirements for ensuring the proper management of the Project and formed the basis for assessing project management practices and outcomes. They are presented in the following table: The FORECAST Project 33 Results and Indicators (Revised) Results Assessments Training and capacity development needs identified, necessary interventions designed Performance Management Plan (PMP) PMP used as a management tool and adjusted accordingly Work plan & Training plan Work plan and Training Plan used for project implementation in a timely manner Project Organization AED established a relatively small project organization to manage the FORECAST Project. The Chief of Party was assisted by a small team. A Training Unit provided all of the basic support required to manage training requests and deliver training opportunities to beneficiary institutions. The Training Unit also provided follow-up support for trainees while they were in training and maintained a database of trainees in and out of training. In addition to the training function was an M&E function, managed by an international M&E Specialist. The M&E specialist generally provided all of the basic follow-up of training as well as technical support with all beneficiaries—individuals and institutions. Project Management Process  HICD Assessment and Performance Gap Analysis for Education and Economic Growth The design of the FORECAST Project essentially began with the HICD Assessment and Performance Gap Analysis, carried out across the education and economic growth sectors for the Academy for Educational Development by Associates for Global Change. It also addressed cross-cutting issues, including gender issues, building local capacity (including training capacity--ToT), and building leadership capacity. This Assessment launched a process designed to better identify some of the issues, gaps, needs and types of capacity building initiatives that FORECAST can fund in order to achieve its objectives. In other sectors, including health, capacity assessments had already been done and FORECAST then relied on the outcomes of such assessments to plan interventions in the sectors and/or institutions concerned. 34 More Detailed Institutional Assessments In addition to the assessments, the FORECAST Project undertook more detailed needs assessments among the various partner/beneficiary organizations in order to more clearly focus their needs and identify their priorities for training and other capacity building interventions. It was on this basis that training and other capacity building support were requested.  Development of Performance Management Plans The Project put in place quarterly performance management plans. The plans generally outlined the performance plans for the management of training and other capacity building support planned for partner organizations. Quarterly evaluations were also done to assess the outcome/status of implementation of the various performance management plans. Closely related to performance management planning, the Project also developed data collection tools for project information management. These included:  Individual follow-up checklist for project beneficiaries,  Baseline / continuous assessment forms for departments / units / organizations,  Performance achievement form.  The Use of Work and Training Plans Work Planning. The Project developed and utilized work plans on a regular basis as part of its overall management processes. Annual work plans were developed, out of which quarterly work plans were drawn for the various functions of the Project, particularly training and monitoring and evaluation. Training Plans. Annual training plans and targets were also developed by the project to guide training commitments on an annual basis. Annual training plans also included plans for follow-up of training activities in support of completed/returned trainees. In support of training plans, an operational handbook (Training Procedures Guide) was developed by the FORECAST Project to provide stakeholders with critical actions and timelines as well as ensure commitment to uniform standards, including openness and transparency, in the identification and nomination of candidates for training. Conclusion and Lessons Learned The Contractor created a small, but generally effective organization for managing the FORECAST Project. The Project was largely successful in assisting partners to access training opportunities for their staff. Various project reports even indicate that, in terms of numbers, the Project far exceeded agreed training targets. Generally, all partners interviewed during the Assessment and in previous assessments (including the November 2009 Mid-Term Evaluation) spoke positively about their relationship with FORECAST and wanted the work of the Project continued. TRAINING PRE-PROCESS: (From Training Procedures Guide)  Nomination letter; (Sent by the Project Partner to the Project International Training Specialist with copy to USAID/COTR)  FORECAST Technical form must be filled out and sent along with the nomination letter  Nomination discussed; (among the project, USAID, and the nominating partner) and approved by Advisory Board Members  Training Budget established; (By the project)  Bio Data Form filled out with a copy of Passport; (Between the project and the nominee)  Contact training Provider to confirm Training Information & make necessary Training Arrangements; (By the project)  Training Implementation Plan developed; (By the project)  English Language Proficiency (for UST);  Visa Process (for TCT); (By the project)  DS 2019 process initiated (for UST); (By Home office & COTR)  PSRFI in process; (By COTR)  Conditions of sponsorship for J1 visa holders (for UST); (By COTR)  Medical clearance and HAC insurance (plan C if possible); (By the project)  Pre-departure orientation checklist; (By the project)  Conditions of training; (Signed by the nominating entity and participant)  Participant Objective memo; (Signed by Project and participant)  Participant action plan. (Drafted and signed by participant) 35 Despite its relative success, though, there were challenges. One main challenge identified was the Project’s limited scope and capacity when it came to follow-up of training and associated technical support to partner organizations. Most of the responsibilities for these activities rested solely with the International M&E Specialist. Besides the obviously huge limitation of virtually a single individual, much of the support provided were in areas in which he had expertise. It was found, for example, that excellent follow-up support, including technical support, was most successful in the areas of M&E and computer applications than they were in other areas. This was because of the M&E and basic IT background of the M&E Specialist. To understand the broader challenges that affected the management of the Project, the following comment of one of the Project’s own reports (Annual Report - October 1, 2009-September 30, 2010) stated:  Change in administration at the Ministry of Education, which delayed decision-making regarding outstanding or recently changed requests. The first official meeting with the new administration took place on September 29, 2010.  The change in application requirement for J-1 visas which now must be initiated through the completion of the DS-160, which can take anywhere from 2-5 days of constant redoing before successful completion and submission is possible.  The last minute requests for training and other activities from the GOL partners posed major difficulties throughout the course of the year, particularly in light of the fact that project administration was set up to handle the number of activities and locations initially targeted. This interfered tremendously with time needed to focus on HICD activities.  Last minute responses from universities (US and Third Country) continued to be a challenge throughout the year, as much as those with no response whatsoever, in spite of the project staff’s ongoing communication to solicit a response.  Visas continued to be problematic, in particular for those travelling to the US, UK, and additionally Swaziland due to a recent change in regulations for transit visas in South Africa. The additional cost of travel to Ghana for these visas was highly visible for those already accepted to universities in the UK.  It is still a challenge, in spite of all efforts, for supervisors and their employees receiving support from the project to fully engage in follow up activities once participants return from trainings. There is still a tendency for many to interpret capacity building as only “training’ or “scholarships”, which implies there is no further intervention required once a training event is completed. The project continues efforts to help beneficiary institution leaders and supervisors to understand the importance of “post training” activities, emphasizing that this is not a scholarship program or a training program. This potential challenge was foreshadowed by Gary Walker in his initial performance gap analysis in November 2008 of project activities.  The onslaught of demands for academic programs both in and out of Liberia that will not be completed until months or in some cases years after the project has closed presents some extremely important concerns. Not only will there be no follow up with these individuals to determine any impact project activities have had due to this support, but just as importantly, it is not clear who or what entity will be responsible to provide continuing support for these individuals, including payment of their monthly stipends, renewal of lodging arrangements, payment of upcoming tuition fees, book fees, ongoing insurance costs, return travel arrangements and other matters that the project has handled. 36 4. THE FUTURE OF USAID’S CAPACITY BUILDING SUPPORT Research Question: Should USAID fund a future participant training and/or capacity building project, and if so, for what sectors, what balance between public and private sector beneficiaries, what balance between training and capacity building, what mechanism (Task Order, Project, etc), and what management structure? The capacity building opportunities delivered through the FORECAST Project represented an unprecedented and somewhat successful effort to build capacities across sectors in Liberia. At no time in Liberia has such capacity building support opportunities been provided on such a scale and scope and delivered through a single effort (as the FORECAST Project) over a short period of time. For the vast majority of beneficiaries, the opportunity for training was their first real capacity building experience. However, there still remain so many unmet needs that need to be attended, leaving the overwhelming opinion among all stakeholders that there is a need to continue the FORECAST Project. Concentrating cross-sector capacity building resources in one place allowed easier access to training and other support and the opportunity to build consensus on priorities and needs across sectors as well as track resource flows to the various sectors. Accordingly, the following recommendations have emerged from the Stakeholder Assessment. They are presented with the intention of being applicable to a continuation and/or expansion of the FORECAST Liberia Project in the future. They are as follows: Strategic Recommendations d. Establishing Effective Partnership FORECAST 2 must be designed as a more formal strategic partnership between the Government and USAID that establishes commitments on both sides. It should reflect priorities associated with the National Capacity Development Strategy, but must also be a bridging effort that fills gaps rather than attempt to meet all perceived needs of beneficiary institutions. A way of ensuring this is to insist that Government implement the National Civil Service Training Policy as a basis for collaboration in capacity building. The Policy places binding responsibilities on public entities to develop and implement annual training plans, with agreed levels of budgetary commitment to training. The Civil Service Agency is the manager of the Policy and must be involved in any strategic partnership with USAID on capacity building support. The National Civil Service Training and Development Policy The policy, developed in 2009, outlines the processes the Government wants ministries and other civil service agencies to follow in making decisions on training, particularly in the area of funding, type of training, sourcing training, and beneficiaries of training, to address the capacity challenges facing public sector institutions. The training policy essentially promotes good practices in the area of training and development in government, especially in the civil service. Goal To help professionalise the civil service, to increase its institutional capacity to deliver high quality services to the public. Key Policies g. Focus Training on Development of Organizations All civil service organizations will annually review their training needs and identify training programs for the coming year, particularly in line with organizational development and reform activities planned and underway. - The CSA will issue new training procedures to this effect, as part of a wider set of civil service regulations. - This will be supported by a new Training and Learning Coordinators network to be established across the civil service. 37 - New procedures for scholarships will require civil servants to agree to resume employment in relevant positions once the relevant study program has been completed and to actively transfer their new skills to others. h. Accelerate Civil Service Capacity Building - Civil service capacity building will be accelerated by increasing the use of short-term training in Liberia. This increased focus on short term training will not necessarily decrease the number of overseas scholarships. - Capacity building will also be accelerated by making greater use of existing expertise among Liberian civil servants, especially those who have studied overseas on government or donor-funded scholarships. - Capacity building will be accelerated by increasing the allocation of Government funds to training. i. Ensure Transparent Training Decisions - Ensuring that each organization’s training plan is approved by CSA and LIPA and is made available to staff, including information on how staff will be selected. - Steps must be taken to advise staff of training opportunities within the organization and externally. - New procedures will apply to selection of scholarships offered to civil servants, especially for long term overseas study. - Greater transparency will also be required for the selection of training providers. j. Build Capacity to Build Capacity - Steps are already being taken to build LIPA’s capacity to design and manage training programs for government organizations, with fundamental changes being made in training methodologies. - LIPA will be established as a centre of excellence in civil service training methodology, not only as the major provider of training, but through much greater support for training provided by others. - The Government will also require all future donor projects to include a component for building Liberian training capacity. k. Move towards a Competency-Based Training System Over a 10 year period the Government will move towards establishment of a comprehensive competency-based training system, under which practical competencies will be identified for each category of civil service positions, and then used consistently for staff selection, performance management and the design of training programs. l. Priorities for FORECAST 2 The capacity challenges in Liberia are huge and span all sectors. In the public sector and institutions, where there are less competitive recruitment practices and very little access to training and other skills building opportunities, capacity building is especially challenging. While the FORECAST Project represented a tremendous effort to address the problem, the needs still abound. However, FORECAST 1 support provided more substantive support to the health and education sector than all other sectors, even though huge needs still exist in these sectors. As such, FORECAST 2, while continuing to deliver support to the health and education sectors, should conduct a needs assessment in the Economic Growth and Governance and Democracy Sectors and determine whether more training and capacity building is warranted in these sectors under FORECAST 2. As for the private sector, a more strategic assessment is required to determine critical needs within the sector. The selection of private sector entities with which to collaborate should be based on their overall value across sectors and their level of institutional development that can enable them to adequately absorb and utilize technical support for the benefit of other sectors. m. Approach to Awarding/Delivering FORECAST 2 The approach to delivering FORECAST 1 was generally effective. However, the duration of the project, which was understandably driven by the need to make some “quick impact” on the capacity situation at the time, was considered by all stakeholders to have been too short to allow more effective planning and execution as well as achieve the desired impact.. Going forward, it will be more effective to design and implement a more long-term strategy and program of support to capacity building across sectors in Liberia, building on the orientation and achievements of FORECAST 1. A more long-term delivery period will allow for more effective planning and implementation, follow￾up/technical support, and monitoring, and evaluation. Perhaps an initial 5-year program can be 38 formulated for FORECAST 2. Such a program would carefully build on both the initial Needs Assessment that FORECAST conducted, the National Training Plan, and further needs assessments, and would obtain Government of Liberia commitment to support capacity building by providing returned trainees more on-the-job support to apply their new skills. n. Management Arrangements As already stated above, the management arrangement for FORECAST 2 should build on the experiences of FORECAST 1. One critical area for strengthening should be the capacity to provide post-training follow-up and technical support. In FORECAST 1, a single M&E specialist supported virtually all sectors with follow-up/technical support, while at the same time undertaking project M&E activities and supporting the development and management of M&E structures within partner organizations. It will be more useful in FORECAST 2 to build a more technically ready and competent project organization. This may be achieved in great measure by including in the project organization more sector and subject specialists that can effectively support follow-up of training and technical support to partner organizations, while maintaining an independent M&E function within the project. Another option would be to sub-contract the management of certain services to local experts and organizations. For instance, post-training technical support in specific areas/sectors/organizations may be sub-contracted to a farm, while FORECAST monitor and evaluate progress and achievements. Programmatic Recommendations o. Ensure an active interagency/cross-sector working group (or collaborative framework) that can assure a fair reflection of priority needs of participating institutions and continuing focus on the purpose of the FORECAST project. This will enhance a positive response from participating institution avoid the kinds of negative perception experienced in FORECAST 1 that the project was bottomless scholarship pit and an end in itself for capacity building. This can be achieved by: o Including partners in project design. o Hosting a workshop with key agency staff to clarify, identify, and reaffirm goal commitments with the USAID, the contractor, and within and among the partner organizations. o Regularly communicating with partners on the aims and objectives of the FORECAST Project and expected roles and contributions of the partners in meeting said goals, objectives, and outcomes. o Ensuring specific responsibilities/obligations for partners in realizing the intended outcomes the FORECAST Project. p. Any continuation of Project activities must be flexible to accommodate changes in planning and implementation. This can be achieved by: o Periodic reflection gatherings to review issues and possible changes. o Strategic discussions about cross-sector capacity building needs. o Set appropriate technical support strategies and more realistic targets and timing for impact and the measurement of said impact. q. The engagement of external HICD partners such as those FORECAST worked with in the past should become a more formalized element of FORECAST 2’s strategy, particularly to ensure close partnering in FORECAST’s implementation activities as well as strengthen FORECAST’s outreach. This can be accomplished by, for example: o Consultation with partners in project design and planning. o Inclusion of partners in field implementation activities. o Inclusion of partners in communication planning. o The use of formal agreements or MOUs with partner organizations. 39 o Defining indicators of success for each of the above by hosting a joint session to define these indicators. r. More innovative strategies to training/capacity building should be considered, while strengthening the technical support component to ensure hands-on experience or learning by doing. This can be accomplished through: o Study visits to third countries, particularly African countries that have successfully implemented holistic strategies and programs for capacity development. o Use of experts/consultants, including individuals from within participating institutions who can contribute knowledge and experience to others. s. Assure realistic expectations of time and resources that are needed for project enactment and activities. This can be accomplished by: o Initial face-to-face meetings with initiative/objective teams whereby direction, activities, and indicators are mapped out more concretely. o Providing for reflection points at periodic stages in project implementation to address issues, modify objectives, and revise activities and workloads if necessary. o Allowing flexibility if agency or staff cannot commit to an activity or certain timeline because of other pressures. t. For continued FORECAST activities, more clearly defined measurable outcomes and indicators for the overall goal and objectives as well as for each individual institution; ensure consensus among all as individual institutions and as a cross-sector group. This can be achieved by: o Holding an initial strategic workshop with all stakeholders to define goal(s) and objectives and assist agencies to set their own specific indicators of success, based on the broad indicators set for the project. 40 5. APPENDIXES Section 5 Appendix 1: Short‐Term Training Support Provided Departments/Units/Divisions as at the End of 2010 (Source: FORECAST Project) Institution / Organization Location of Activity Capacity Building Activities Category # of Beneficiaries FY 2009 FY 2010 FY 2011 1 MPEA STT / IC Basic Computer Skills Training Technical skills 5 0 STT / TC Monitoring and Evaluation Technical skills 2 1 STT / IC Outcome Mapping Technical skills 0 1 STT / IC Monitoring and Evaluation TOT Technical skills 0 1 2 MOHSW STT / IC Basic Computer Skills Training Technical skills 5 0 STT / TC Managing HIV/AIDS programs Technical skills 2 0 STT / TC Medical Equipment Technology Technical skills 0 1 STT / IC Basic Management Skills Training Management practices 5 0 STT / US Making Decentralization Work Technical skills 3 0 STT / TC Model community based FP design Technical skills 1 0 STT / IC M&E systems assessment Technical skills 5 0 STT / US Public financial management Technical 0 1 STT / TC Introduction to Public Health surveillance Technical 0 1 LTT / TC Health Economics Technical 0 2 STT / TC Psychology, guidance and counseling Technical skills 0 3 STT / IC Outcome Mapping Technical skills 0 3 STT / TC Monitoring and Evaluation Technical skills 2 0 3 MOE STT / IC Basic Computer Skills Training Technical skills 2 0 STT / IC Science Education Training Technical skills 46 0 STT / TC Monitoring and Evaluation Technical skills 1 2 STT / IC Basic Management Skills Training Management practices 5 0 STT / IC Leadership, Supervision, Monitoring and Evaluation Technical skills 0 48 41 Institution / Organization Location of Activity Capacity Building Activities Category # of Beneficiaries FY 2009 FY 2010 FY 2011 STT / TC Financing the health related MDGs Technical skills 0 1 STT / US Making Decentralization Work Technical skills 1 0 STT / TC Management and Supervision Skills Technical skills 1 0 STT / TC Business administration Technical skills 0 1 STT / TC IT service management Technical skills 0 1 STT / TC Leadership skills for women supervisors Technical skills 0 1 STT / TC Network administration and website design Technical skills 0 1 STT / TC Institutional development & organizational strategy Technical skills 0 1 STT / TC School Health Nutrition and HIV Prevention Technical skills 0 1 STT / TC Psychology, guidance and counseling Technical skills 0 3 STT / TC Auditing for Internal Auditors Technical skills 0 4 4 MPW STT / IC Basic Computer Skills Training Technical skills 4 0 STT / IC Water and Sanitation Technical skills 41 0 STT / IC Performance Based Capacity Building Management practices 0 1 STT / IC Basic Management Skills Training Management practices 4 0 STT / TC Monitoring and Evaluation Technical skills 1 0 5 MOA STT / IC Basic Management Skills Training Management practices 4 0 STT / IC Outcome Mapping Technical skills 0 5 STT / IC Performance Based Capacity Building Management practices 0 1 STT / TC Monitoring and Evaluation Technical skills 1 0 6 NEC STT / IC Boundary Delimitation Technical skills 143 0 7 UL STT / IC Administration and Management Management practices 4 0 STT / IC Science Education Technical skills 20 0 STT / TC Human Resource Management Management practices 0 2 8 LIWOMAC STT / US National Endowment for Democracy Technical skills 1 0 9 MLME STT / US Renewable energy Technical skills 1 0 10 LIPA STT / TC Monitoring and Evaluation Technical skills 0 2 STT / IC Monitoring and Evaluation TOT Technical skills 0 0 STT / IC Performance Based Capacity Building Management practices 0 0 11 MOGD STT / TC Monitoring and Evaluation Technical skills 0 1 STT / IC Monitoring and Evaluation TOT Technical skills 0 1 STT / TC Financing the health related MDGs Technical skills 0 1 STT / IC Performance Based Capacity Building Management practices 0 1 42 Institution / Organization Location of Activity Capacity Building Activities Category # of Beneficiaries FY 2009 FY 2010 FY 2011 12 House of Representatives STT / TC Financing the health related MDGs Technical skills 0 1 13 Mother Pattern STT / TC CISCO training and certification Technical skills 0 2 310 96 43 Appendix 2: Update of Training Events Status (by Sector) – as at 03/31/11 (Source: FORECAST Project) a. Detailed Training Information HEALTH SECTOR LONG TERM TRAINING Training Type Major Field Of Study Participant Name Training Provider Training Location Trng Dates (Tentative) Actual Trng Dates Number of Participants Employer Beneficiaries TRNG STATUS Comments LTT Financial Management Amos Pailey University of Seattle Seattle, Washington USA Oct 1 2009 –Aug. 2011 Departed September 27, 2009 1 Comptroller JFK Completes in May 2011 Grades received – doing well LTT Health System Management Garfee Williams Cuallau Jabbeh Institute of Tropical Medicine Antwerp, Belgium Sep 9, 2009 – Aug 2010 Departed September 2, 2009 2 Medical Doctors MOHSW Bong & Nimba Completed follow up in progress LTT Health Economics Justine Korvayan Dominic Togba Chulalong-korn University Bangkok, Thailand June 1 2009 – May 2010 May 25, 2009 departed 2 Health Executives MOHSW Completed follow up in progress LTT Nursing/Ed Mgnt, MPH, Psych nursing, Health Ed, Health Systems Mgmt, Repro Health, Nursing Ed Munah Tarpeh Rachel Jarteh Varwo Gbassie Linda Birch Sarah Kolllie Peter Yarkpawolo Cecilia Nuta Cecilia Flomo Royal Institute of Medicine Amsterdam, Netherlands 21 Sep 09 – Aug 2010 Departed September 4, 2009 (for 2 week prep courses & full program 8 Health Practitioners/Managers MOHSW/ TNIMA/JFK 7 completed follow up in progress Cecilia Flomo Kpangbala stopped in 2009 and restarted in Oct 2010. Will continue until Sep 2011. LTT MPH Ramata Yaoda University of Ghana Accra, Ghana Aug 2009 – July 2010 Departed August 1, 2009 1 Health Practitioner MOHSW/ TNIMA/JFK completed LTT MPH-Planning & Management, Health Ed Promotion, Pop & Repro Health Pharmaceutical Microbiology Bob Singbeh Olive Dwana Kou Baawo Bentoe Tenhoungue Kerson Saykor Sando Sirleaf Duredoh George Kumasi University Kumasi, Ghana Aug 2009 – Feb 7, 2011 D George – until July 2011 6 Departed Aug. 12, 2009 1 departed Aug 19, 2009 7 Health Practitioners, Managers MOHSW/ TNIMA/JFK completed LTT Human Resource Management (Health James Beyan Mawolo Kollie University of Uganda Kampala, Uganda Aug 19, 2009- Departed September 28, 2 Managers MOHSW in progress to be handed over 44 Systems) December 2011 2009 Tentative 21 months Training Type Major Field Of Study Participant Name Training Provider Training Location Trng Dates (Tentative) Actual Trng Dates Number of Participants Employer Beneficiaries TRNG STATUS Comments LTT Lab Technician Multiple MPCHS Monrovia Oct 2010-Aug 2011 Oct 2010-Aug 2011 17 Multiple In progress to be handed over STT Health Systems management Multiple MPCHS Monrovia Nov 2010-Jan 2011 Nov 2010-Jan 2011 30 MOH (all counties) completed LTT Nursing Management; Pharmaceutical Mgt Joseph Jimmy Mary Howard Bendu Sarno IIHMR Jaipur, India July 6, 2009 – May 30, 2011 Departed July 1, 2009 3 Health Executives JFK/MOHSW in progress research in progress in country LTT/STT Social Work Guidance Counselling STT TCT Victoria Zawa Elain Doe Tinisi Saytue TBD Ghana and Kwazulu applications rejected LTT TBD STT – DTI Swaziland TBD LTT STT April 12-30, 2010 3 MOH Technicians MOHSW Completed follow up in progress LTT MPH/M&E George Jacobs Janjay Jones Jimma University Jimma, Ethiopia December 2009- 6/20/2011 Departed December 20, 2009 2 M&E officers MOH field research in progress to be handed over LTT Maternal Child Health Edith Tellewoyan U of Ghana – no response Submitted application to KIT Amsterdam, Netherlands September 2010 – Sep 2011 Instructor Phebe in progress to be handed over LTT Pediatric Nursing Oral Siakor September 2010 – Sep 2011 CU Clinical Instructor Cuttington University LTT Supply Chain/Pharmaceutical Mgt John Harris Muhimbili University Tanzania November 8 2010 – September 2012 Pharmacist MOH In progress to be handed over LTT MPH Multiple MPCHS Liberia September 2010- Dec 2011 16 from various Institutions in Liberia Multiple in progress to be handed over 45 HEALTH SECTOR – SHORT TERM Training Type Major Field Of Study Participant Name Training Provider Training Location Trng Dates (Tentative) Actual Trng Dates Number of Participants Employer Beneficiaries TRNG STATUS Comments STT HIV/AIDS & RH Programmes Joseph Gbassie 1 female CAFS Nairobi TCT 11/10-28/08 11/10-28/08 2 GOL staff MOH, GOL Completed follow up in progress STT Decentralization DM Varpilah Dr. Dahn Harvard University Boston 6/8-16/09 6/8-16/09 4 MOH officials MOH, GOL Completed follow up in progress STT Model Community￾based FP design Dr. Baawo MACRO Intl Accra TCT 2/9-20/09 2/9-20/09 1 Director of FP division MOH, GOL Completed follow up in progress STT Monitoring & Evaluation TOT Workshop FORECAST TOT Liberia ICT 2/5-7/09 2/5-7/09 5 MOH M&E Technicians MOHSW Completed follow up in progress STT M&E Coursework AMREF Nairobi June 22-July 17 June 22-July 17 2 M&E Officers MOH Completed follow up in progress STT Dental Technician Arthur Mulbah (tech) UMU - Ganta Ganta, Liberia October 2010 – December 2011 1 Dental Tech (Phebe) 1 Dental Assit ELWA MOH in progress to be handed over STT Medical Equipment Training Peterson Greaves Med Eq. Trng (MET) Valley View University Accra Ghana JUNE, 2010 (2 months) August 1, 2010 1 technician Redemption; MOH; Phebe completed follow up in progress STT Policy Making in Family Planning Honorable Gaye Patricia Camara William Twehweh Bill & Melinda Gates + WHO, WB, UNICEF etc. Kampala, Uganda November 15-18, 2009 3 (1) DM, (1) AM, & (1)Senator MOE, MGD, + GOL Completed follow up in progress STT Physical Therapy Flomo Forkpa Togo – changing to Tanzania TOTCOM Tanzania 9/2010- 7/2013 TBD 1 technician Handicap International in progress to be handed over STT Public Financial Management Karzon Harvard U Business School Boston, USA 7/5 – 7/23 2010 TBD 1 comptroller MOH completed follow up in progress 46 EDUCATION SECTOR - SHORT-TERM Training Type Major Field of Study Participant Name Training Provider Training Location Trng Dates (Tentative) Actual Trng Dates Number of Participants Employer Beneficiaries TRNG STATUS Comments STT Information Tech & Web Design (too many to name here) Educare, Stella Maris GSA, MDI, CISCO, other TBD Monrovia; Swaziland; South Africa; Too numerous to include here multiple 15 + MOE (13)/ UL/Stella Maris(2) completed follow up in progress STT Management Training Certificate (off the shelf) LIPA – Certificate in Administration and Management Liberia ICT April 6, 2009 – June 6, 2009 April 6 – June 29, 2009 4 University of Liberia Admin/Finance Supervisors UL Completed Post event monitoring in progress STT M&E Training Francis Wayne GIMPA Ghana May 11 – May 15, 2009 May 11 – May 15, 2009 4 M&E staff MPW, MOE, MOA Completed Post event monitoring in progress STT Guidance & Counseling Momo Fahnbulleh Jomo Broderick MacArthur King William Kpawulu Patrick Logan Peter G. Roberts Dev. Trng Int’l Swaziland April 12, 2010 – April 30 (3 weeks) Group 2 – October 2010 Oct 1-30 2010 6 MOE / Instruction/ PTA Coordinators Completed follow up in progress STT Mathematics Rufus Duo Heartland International Chicago, IL August 16- Sep 162010 1 WRTTI Instructor completed follow up in progress 47 EDUCATION SECTOR CONTINUED Training Type Major Field of Study Participant Name Training Provider Training Location Trng Dates (Tentative) Actual Trng Dates Number of Participants Employer Beneficiaries TRNG STATUS Comments STT Financial Management Esther Kollie/ Henrietta Baysah Albert Dabweson Momoh Soumah MDI LIPA MDI ICT Swaziland January 15– April 15, 2010 February 8 – April 9, 2010 3 MOE Instruction, admin, planning All Completed STT Testing and Evaluation Dale Gbotoe (LTT) Martin Poqoi (STT) Heartland International Chicago, IL USA August 16 – September 16 2010 2 RTTI/WAEC/L completed follow up in progress STT Education Management/Supervisio n/Admin. Gabriel Nelson Felicia Sacko Cecilia Reeves TBD + Management Dvpt Inst. TBD + Swaziland Sep & Oct.& Nov 2009 1-Sep 3- 30, 2009 1 Sep 23-Oct 20, 2009 Nov 7 – Dec 4, 2009 3 MOE-Instruction & Admin All completed Follow up in progress LTT Human Resources (MPA) Nathaniel Gibson Cuttington University ICT Began September 2010-June 2012 1 MOE In progress to be handed over 48 EDUCATION SECTOR – LONG TERM Training Type Major Field Of Study Participant Name Training Provider Training Location Trng Dates (Tentative) Actual Trng Dates Number of Participants Employer Beneficiaries TRNG STATUS Comments LTT Civil Engineering Adam Yaba University of Tennessee UST Sept/09-June /2011 Sept/09- May/2011 1 Deputy Director MOE May 2011 completion date Grades received-doing very well LTT Financial Management Samuel Zulon Colorado state U UST Aug 2009 – Sep 2010 Aug 2009 – May 2011 1 Comptroller MOE May 2011 completion date Grades received-doing well LTT Finance and Banking James Andrews Southern New Hampshire U UST Sep /09- May /2011 Sep /09- May /2011 1 Assistant Minister, 1 Analyst 2 MOE/Admin May 2011 completion date Grades received-doing well LTT Human Resource Management Samuel Hodge Southern New Hampshire U UST Sep/ 09- May /2011 Sep/ 09- May /2011 1 MOE/Admin May 2011 completion date Grades received – doing well LTT Procurement/Tra nsport Management Ericson Gbarmore Christine Sirleaf Magnus Mabadna Joseph Kpannah Cambridge Distance Learning LIPA London Oct 9,2009- Oct. Sep 2010 4 MOE/Admin Completed STT Supply Chain Management Monrovia (12 weeks) March 8, 2010 Completed LTT Education Administration Charles Dennis Patrick Davies Lawrence Tweh UL ICT November 2010- August 2012 3 MOE:Admin, Instruction, in progress to be handed over STT Auditing Olivia Mannah Michael Quaye Mensah Gonlakpor Rabshakeh Giplaye Management Devlpmt Inst. Swaziland Jan 4 – April 30 2010 Jan 4 – April 30 2010 4 MOE Admin Completed LTT MPA Abu Sherriff Josephus Dixon GIMPA Accra, Ghana Feb 13, 2010- Dec 2010 2 UL Completed 49 Training Type Major Field Of Study Participant Name Training Provider Training Location Trng Dates (Tentative ) Actual Trng Dates Number of Participants Employer Beneficiaries TRNG STATUS Comments LTT Early Childhood Education/Elementary education Anita Cooper University of Arkansas, Fayettville US September 16 2010- August 30, 2011 1 LTT MOE/Instructi on In progress to be handed over STT Science Education Multiple UL/Consulta nts ICT Kataka June1-6, 2009 Jun 1-6, 2009 60 UL science ed staff, MOE science ed teachers Completed Verification follow up in progress STT Supervision/Leadershi p/Management All CEO’s plus select DEO’s MDI ICT with external facilitating institution Jan 11-22, 2020 50 District and County Education officers Completed follow up in progress STT Human Resources 2 UL professors GIMI Israel July 1-12 2010 2 UL Completed follow up in progress STT School Health, Nutrition and HIV Prevention Program Comfort Somerville The Partnership for Child Dvlpmnt Accra Ghana June 8-17, 2010 1 MOE School health nutrition unit Completed follow up in progress 50 GOVERNING JUSTLY AND DEMOCRACTICALLY SHORT TERM Training Type Major Field Of Study Participant Name Training Provider Training Location Trng Dates (Tentative) Actual Trng Dates Number of Participants Employer Beneficiaries TRNG STATUS Comments STT The Future of Media in Liberia Estella Nelson National Endowment for Democracy Washington DC 2/17-20/09 2/17-20/09 1 person from LIWOMAC NGO Completed Post event monitoring in progress STT Organizational Management and Performance Skills Training Multiple (too many to name here) TechMedia ICT Liberia Rolling 3 month courses beginning November 16, 2009 181 participants from multiple GOL agencies MLME, RIA, FDA, multiple completed Post event monitoring in progress STT IT Training Certification Kessellee Johnson John Tuah SoftTech Ghana February 2010 July 2010 (Tuah) 2 GSA both completed Post event monitoring in progress STT National Elections Commission Trainings Multiple (too many to name here) (consultants) ICT Liberia May 7 – June 8 2009 193+ Commissioners, Magistrates, Assistant Magistrates Completed STT CBO/CSO Boundary Delimitation Training Multiple (too many to name here) consultants ICT Liberia May/June August 28, 29, 2009 70 Representatives of CBO/CSOs Completed 8/29/09 STT Investigation Techniques Multiple GSA, NPA & Basel Governance ICT, ICT & TCT (Basel, Switzerland) May 2010 ongoing NPA-8/16- 11/8 2010 Advanced Investigator Tech (3/21- 4/1 2011) 7 investigators plus other members of dept. Liberian Anti￾Corruption Commission Police Training Completed Prep for final training (April 6- 15, 2011 advanced investigator training) STT FINCAB 5 trainers LIPA Monrovia Jan-Apr 2011 4 LIPA instructors in progress STT Budget Advisor Short Term Technical Assistance consultant Monrovia Oct 2010- April 2011 Bureau of Budget Ministry of Finance DOB completed STT SunSystems Expert Trainer Short Term Technical Assistance consultant Monrovia Feb-April 2011 Bureau of Expenditures Ministry of Finance DOE in progress 51 ECONOMIC GROWTH LONG AND SHORT TERM Strategic Objective Training Type MFOS Training Provider Training Location Trng Dates (Tentative) Actual Trng Dates Number of Participants Employer Beneficiaries TRNG STATUS Comments Economic Growth STT Community Services TBD Liberia ICT May 2009 June 1-6, 2009 41Technicians and Coordinators MPW Completed Post event monitoring in progress Economic Growth STT Renewable Energy (Augustus Goanoe) USEA US August 31 – Sept 4, 2009 August 31 – Sept 4, 2009 1 Director of Renewable Energy Completed follow up in progress Economic Growth STT Nat’l Working Group-TVET SKD/consulta nts In country August 5-7 August 5-7 100 high level partners GOL, Private, Tech Providers Completed Final Report received 3/2010 Economic Growth LTT Agriculture Cuttington Unversity Gbarnga Liberia Oct 2010- Aug 2011 22 ag students multiple completed Economic Growth STTA Forestry Curriculum revision AGC Felipe Tejeda Liberia January 2011 - March 2011 April multiple instructors Forestry Training Institute prep in progress CAADP Workshop STT Agriculture CAADP Senegal Dec 1-3 2010 multiple 7 Completed 52 CROSS CUTTING SHORT TERM & TECHNICAL ASSISTANCE All Partners (Cross Cutting) STT Basic computer Skills IICS/MPD Monrovia ICT 2/16-3/6/09 2/16-3/6/09 20 Middle officials GOL (MOE, MOH, MPW, MOPEA) Completed Post event monitoring in progress All Partners (Cross Cutting) STT Basic Management Skills (Basic Business Skills Training) LIPA Monrovia ICT 3/9-27/09 20 Middle officials GOL (MOE, MOH, MPW, MOA) Completed Post event monitoring in progress All Partners (Cross Cutting) TA M & E FORECAST M&E Specialist Liberia (Monrovia and Counties) ongoing 3-40 MOH, MOE, MPW, MOA, MOPEA Ongoing (until end of project) All Sectors LTT Policy and Planning Steve Marvie University of Manchester Manchester, England September 2010 - 2011 1 DM Assistant MPEA in progress to be handed over Education and Economic Growth STT TA ICT AUTOCADd consultant Kakata June-August 2010 TBD 4 BWI Instructors BWI Completed All Sectors (or TBD) STT TCT M&E AMREF Nairobi June 21– July 16 2010 2 Directors of Departments LIPA completed LIPA TOT STT ICT M&E Modules and Human Resource Development AGC Consultant Monrovia (LIPA Office) August 2 – August 13, 2010 15 combination LIPA trainers & GOL M&E participants completed Finance LTT TCT Masters in Finance and Economic Development Henley University Reading England September 2010 - 2011 2 Analysts Ministry of Finance in progress to be handed over 53 b. Summary Training & Technical Assistance Information Health Sector Training Beneficiaries Major Field Of Study Number of Participants Employer Beneficiaries TRNG STATUS Long-Term Training Financial Management 1 JFK Completes in May 2011 Health System Management 2 MOHSW Bong & Nimba Completed Health Economics 2 MOHSW Completed Nursing/Ed Mgnt, MPH, Psych nursing, Health Education, Health Systems Mgmt, Repro Health, Nursing Education 8 MOHSW/TNIMA/JFK 7 completed MPH 1 MOHSW/TNIMA/JFK completed MPH-Planning & Management, Health Ed Promotion, Pop & Repro Health, Pharmaceutical Microbiology 7 MOHSW/ TNIMA/JFK completed Human Resource Management (Health Systems) 2 MOHSW in progress Lab Technician 17 Multiple In progress Nursing Management; Pharmaceutical Mgt 3 JFK/MOHSW in progress MPH/M&E 2 MOH field research in progress Maternal Child Health 1 Phebe in progress Pediatric Nursing 1 Cuttington University Supply Chain/Pharmaceutical Mgt 1 MOH In progress MPH 16 Multiple in progress Total-Long-Term 64 Short-Term Training Health Systems management 30 MOH (all counties) completed HIV/AIDS & RH Programs 2 MOH, GOL Completed Social Work Guidance Counseling 3 MOHSW Completed Decentralization 4 MOH, GOL Completed Model Community-based FP design 1 MOH, GOL Completed Monitoring & Evaluation TOT Workshop 5 MOHSW Completed M&E Coursework 2 MOH Completed Dental Technician 2 MOH/Phebe, ELWA in progress Medical Equipment Training 1 Redemption; MOH; Phebe completed Policy Making in Family Planning 3 MOE, MGD, GOL/Senator Completed Physical Therapy 1 Handicap International in progress Public Financial Management 1 MOH completed Total-Short-Term 55 Total (Health Sector) 119 54 Education Sector Training Beneficiaries (Summarized from FORECAST Project Reports – Details in Appendix 2) Major Field Of Study Number of Participants Employer Beneficiaries TRNG STATUS Long-Term Training Human Resources (MPA) 1 MOE In progress Civil Engineering 1 MOE May 2011 completion date Financial Management 1 MOE May 2011 completion date Finance and Banking 2 MOE May 2011 completion date Human Resource Management 1 MOE May 2011 completion date Procurement/Transport Management 2 MOE Completed Education Administration 3 MOE in progress MPA 2 UL Completed Early Childhood Education/Elementary education 1 MOE In progress Total-Long-Term 14 Short-Term Training Information Tech & Web Design 15 MOE (13)/ UL/SMP (2) completed Management Training Certificate (off the shelf) 4 UL Completed M&E Training 4 MPW, MOE, MOA Completed Guidance & Counseling 6 MOE Completed Mathematics 1 WRTTI Instructor completed Financial Management 3 MOE All Completed Testing and Evaluation 2 RTTI/WAEC/L completed Education Management/Supervision/Admin. 3 MOE All completed Auditing 4 MOE Completed Supply Chain Management 2 Science Education 60 MOE Completed Supervision/Leadership/Management 50 MOE (DEOs/CEOs) Completed Human Resources 2 UL Completed School Health, Nutrition and HIV Prevention Program 1 MOE Completed Total-Short-Term 157 Total (Health Sector) 171 55 Governance and Democracy Sector Training Beneficiaries Major Field Of Study Number of Participants Employer Beneficiaries TRNG STATUS Long-Term Training None Total-Long-Term 0 Short-Term Training The Future of Media in Liberia 1 NGO (LIWOMAC) Completed Organizational Management and Performance Skills Training 181 MLME, RIA, FDA, multiple Completed IT Training Certification 2 GSA both completed National Elections Commission Trainings 193 NEC Completed CBO/CSO Boundary Delimitation Training 70 CBO/CSOs Completed 8/29/09 Investigation Techniques 7 LACC (Training for the Police) Completed FINCAB 4 LIPA instructors in progress Total-Short-Term 458 Total (Health Sector) 458 Technical Assistance Budget Advisor Ministry of Finance completed SunSystems Expert Trainer Ministry of Finance in progress Economic Growth Sector Training Beneficiaries Major Field Of Study Number of Participants Employer Beneficiaries TRNG STATUS Long-Term Training Agriculture 22 Multiple (Agric. students) completed Total-Long-Term 22 Short-Term Training Community Services 41 MPW Completed Renewable Energy 1 MLME Completed Nat’l Working Group-TVET (Meeting) 100 GOL, Private, Tech Providers Completed Agriculture (CAADP Workshop) 7 MOA/Others Completed Total-Short-Term 149 Total (Health Sector) 171 Technical Assistance Forestry Curriculum revision multiple instructors Forestry Training Institute prep in progress 56 Cross-Cutting -- Training and Technical Assistance Major Field Of Study Number of Participants Employer Beneficiaries TRNG STATUS Long-Term Training Policy and Planning 1 MPEA in progress Masters in Finance and Economic Development 2 Ministry of Finance in progress Total-Long-Term 3 Short-Term Training Basic computer Skills 20 GOL (MOE, MOH, MPW, MOPEA) Completed Basic Management Skills (Basic Business Skills Training) 20 GOL (MOE, MOH, MPW, MOA) Completed M&E Modules and Human Resource Development 15 combination LIPA trainers & GOL M&E participants completed AUTOCADd 4 BWI Completed M&E 2 LIPA completed Total-Short-Term 61 Total (Health Sector) 64 Technical Assistance M&E Capacity Development Follow-up Support (Provided by FORECAST Project’s M&E Specialist) 40 M&E Staff MOH, MOE, MPW, MOA, MOPEA Ongoing 57 Appendix 3: Institutions and Persons Interviewed Institutions Persons Positions USAID Economic Growth 1. Michael Boyd Senior Economic Growth Officer/Team Leader, USAID 2. Dan Whyner Forestry Advisor, USAID 3. Vickie Sigman, Agriculture Development Advisor, USAID Health 4. Randolph Augustin Health Development Officer/Team Leader, USAID 5. Sophie T. Parwon Health Management Specialist, USAID Democracy and Governance 6. Sean Hall DG Team Leader, USAID Education 7. Julia Richards Education Officer/Team Leader, USAID 8. Luann Gronhovd Education Development Officer, USAID 9. Miriam White Education Specialist, USAID 10. Mardea Nyumah Education Specialist, USAID 11. Anthony Davis Education Program Management Assistant Liberia FORECAST Project 12. Rebecca Cusic Chief of Party, FORECAST 13. Thierno Kane International Training Specialist, FORECAST 14. Thomas Wobill International M&E Specialist, FORECAST Ministry of Education 15. Matthew Zarzar Deputy Minister for Administration, MOE 16. Kadiker Rex Dahn Deputy Minister for Planning, MOE 17. Anthony Nimely Dept. of Planning, MOE 18. Lorpu Mannah Director, Girls Education Unit, MOE 19. Gabriel Nelson 20. Francis Wayes Project Coordinator, Division of Finance (formerly in Dept. of Instruction), MOE 21. McArthur King Guidance & Counseling, MOE 22. Kpawulu Williams Guidance & Counseling, MOE 23. Wehyi S. Kollah Guidance & Counseling, MOE 24. George Wuo County Education Officer, Nimba Co. 25. William Gizi County Education Officer, Margibi Co. 26. Amos K.G. S. Kapu County Education Officer, Bong Co. 27. Chapman Adams County Education Officer, Grand Bassa Co. 28. Augustine Karyor District Education Officer, Grand Bassa Co. 29. Amos A. Fully County Education Officer, Bomi Co. Ministry of Health & Social Welfare 30. C. Sanford Wesseh Assistant Minister for Vital Health Statistics 31. James Beyan Actg. Human Resource Director, MOH 32. Mawolo Kollie HR Policy, MOH 33. Banda Ebba Assistant, Office of the Minister 34. Dr. Rhoda Roberts￾Peters Medical Director, Government Hospital, Tumbanburg, Bomi County 35. Dr. Samson Azor￾Arzoaquoi Medical Director, C. B. Dunbar Hospital, Gbarnga, Bong County 36. Jerry Manneh Accountant, Nimba County Health Services 37. Monah W. Young Human Resource Manager, C. H. Rennie Hospital, Kakata, Margibi Co. Ministry of Planning & 38. Patrick Krah M&E Unit, MOA 58 Institutions Persons Positions Economic Affairs 39. S. Wilberforce Davis Dept. of Administration 40. Diana S.T. Cooper Dept. of Regional & Sectoral Planning Ministry of Public Works 41. Andrew Kieh Human Resource Director 42. Prince Tugbeh Asst. Director, Organizational Dev. & Training 43. George Yango Asst. Minister for Community Services 44. Andrew Boikpah Community Services Dept. 45. T. Maxwell Ricks County Coordinator, WASH, Gbarnga, Bong Co. Ministry of Gender & Development 46. James Whawhen Former M&E Director (now Deputy COP, L￾MEP) 47. Ishmael R. Pajibo M&E Officer Liberia Institute of Public Administration (LIPA) 48. Prof. Harold J. Monger Director-General, LIPA 49. Richard S. Panton Deputy Director-General, Training, LIPA 50. Harris F. Tarnue Deputy Director-General, Research & Consultancy 51. Christian Tabla Special Asst. to Director-General, LIPA 52. Esigbemi Kperegbey Director of Training, LIPA General Services Agency (GSA) 53. John Tuah Director, Bureau of MIS, GSA 54. Kesseley Johnson Assistant – Bureau of MIS, GSA University of Liberia 55. Josephus Dixon Instructor of Sociology, Liberia College, UL 56. John Soffa Accountant, Business & Finance Office 57. Sylvester Nyentah Accountant, Business & Finance Office 58. Bockarie A. Sheriff Accounting Instructor (formerly internal Auditor), College of Business 59. Theodore Roberts College of Business & Public Administration Booker Washington Institute (BWI) 60. Christian Jones Head, Architecture Drafting Section AEDE 61. Esther Paegar Executive Director, AEDE 62. Maxwell Nimely Research Specialist, AEDE Mother Pattern College of Health Sciences, Stella Maris Polytechnic 63. Sis. Barbara Brilliant Head, MPCHS Non-Direct Beneficiaries 64. Rancy Leesela County Health Services Administrator, Nimba 65. Jackson Sengbeh DEO, Gbarnga School District 66. Edwin G. Kwakpea Principal, Doloken Gboveh High, Bong County 59 Appendix 4: Assessment Tools/Instruments SUBAH-BELLEH ASSOCIATES (MANAGEMENT CONSULTANTS) FORECAST Stakeholder Assessment Institutional Key Informants Interview ‐ General Name of Organization: __________________________________________________________________ Date of Interview: _____________________________ Persons Present Positions Part 1: Assessing FORECAST’s Performance 1. Did you have a good understanding of the FORECAST Project? 2. Did the project perform to your organization’s expectations? 3. What were the successes? 4. Were there any barriers to the achievement of expected outcomes? In your opinion, was there anything FORECAST and/or USAID could have done to overcome these barriers? 5. Where there gaps that were overlooked or were deemed not to be priorities? Can they or should they be addressed in a future project? 6. What kinds of commitments would your organization make to any future effort of this nature, especially those you may not have made while working with FORECAST? Section 2: Assessing the Beneficiary Organization’s Participation 60 7. How is PT related to HICD in your organization? How is training linked to improving the institutional capacity of your organization? 8. What determines who gets selected for training in your organization? 9. Were there any prerequisites for selecting those your organization selected for training? Degree, Years or service, County of origin, Male or female, Pre-test results, Expression of Interest, etc. 10. How were decision made to determine priority gap? 11. Looking back, are you satisfied with the priorities chosen by your organization and the selections made of staff to benefit? Part 3: Measuring Changes in Beneficiary Organization 12. In what ways did you benefit from the FORECAST Project? 13. How did the project contribute to the improvement of the operational effectiveness of your organization? 14. What were the project’s principal successes? 15. What were the barriers to the project success, if any? 16. What are those things that FORECAST /USAID and/or your organization could have done to overcome the barrier? 17. What are participants doing now? 18. What kinds of support has your organization provided the participants to assist them utilize their learned skills as well as pass them on to others in the organization? 19. Are their performances being monitored? Part 3: The Next Steps for Forecast 20. Should HICD and PT be managed by a single project as was done in the current FORECAST project, or should these tasks be managed by individual projects working in the different sectors? 21. What strategies should be employed to avoid problems that arose under FORECAST?  More dedicated stakeholder involvement?  Stronger USAID sectorial inputs?  Etc. 61 SUBAH-BELLEH ASSOCIATES (MANAGEMENT CONSULTANTS) FORECAST PROJECT STAKEHOLDER ASSESSMENT Key Informant Interview Guide FORECAST & USAID Thank you for agreeing to participate in this interview. We are grateful for your time. L-MEP and Subah-Belleh Associates are conducting a stakeholder assessment of the FORECAST project. As a stakeholder, we are interviewing you to learn your important perspective on this project and the activities with which you were involved. Name of Respondent: _____________________________ Institution: ______________________ Position: ______________________________ Date: ____________________ A. ASSESSING ACHIEVEMENT OF PLANNED OUTCOMES 1. Have the eight FORECAST Outcomes been achieved and if not, what were the constraints? If yes what are the achievements? 2. To what extent has FORECAST contributed to filling human capacity development gaps? What has been the most effective? Why? 3. To what extent was the National Human Capacity Development Plan achieved (NCDS) based on FORECAST experience at the time? What types of assistance provided by FORECAST in the development of the National Human Capacity Development Strategic Plan were the most useful, if any? What lessons learned within FORECAST contributed to the NCDS? 4. What assessments were utilized or carried out by FORECAST to determine priority gaps? What is the utility of each assessment presently in determining future priority capacity building gaps? What recent assessments or evaluations exist now that might better inform in designing future capacity building activities? 5. How were decisions made to determine priority gaps? To determine which entities in the sector should become beneficiary institutions? To determine which section within an institution should be prioritized? To determine which employees should benefit from capacity building? Was anything missing in this process? How would you do this differently? What affect did this have on institutions/departments/employees that did not benefit? 62 6. What was the Liberian context when the FORECAST project was designed? What is still constant? What has changed? What may still change? How best can USAID support capacity building within these constants and changes? B. DETERMINING WHETHER ACTIVITIES WERE CONSISTENT WITH PLAN 7. Were the capacity building activities of FORECAST consistent with the Task Order and if not, why? If yes how? 8. What assessment were utilized or carried out by FORECAST to determine priority gap? Was it necessary and sufficient? How could this be done differently? 9. How were decision made to determine priority gap? Determine entities and employees to benefit from the training? 10. What resources were or were not provided to ensure the training or capacity building? 11. What do the organizations with which you work with need in order to succeed? What problems are those organization facing C. ASSESSING EFFECTIVENESS OF FORECAST’S MANAGEMENT STRUCTURE FOR IMPLEMENTING THE PROJECT. 12. What is the management structure of FORECAST? 13. Was the management structure and processes of FORECAST appropriate for achieving project results, and if not, what changes should be made to any future management system? 14. How was the FORECAST project managed and what lessons learned can be garnered from their management approach that can be applied to a future project? 15. What are other Donors doing in the area of human capacity building and how can USAID best complement their focus? 16. What is the most appropriate project? 17. How could we do this same project differently in the future? D. DETERMINING FUTURE DIRECTION FOR USAID CAPICITY BUILDING PROJECT: 18. Should USAID fund a future participant training and/or capacity building project? For what sectors? For what sector beneficiaries? What balance between public and private sector beneficiaries? What balance between training and capacity building? What mechanism (Task order project)? What management structure? 19. What was the Liberian context when the FORECAST project was design? What is still constant? What has change? What may still change? How best can USAID support capacity building within these constants and changes? 63 SUBAH-BELLEH ASSOCIATES (MANAGEMENT CONSULTANTS) FORECAST Stakeholder Assessment Individual Interview Questionnaire Name of Returned Trainee Telephone/E-mail Institution (pre-training) Dept./Unit & Position (pre-training) Organization & Position (if changed since training) Training Received and Duration Training Provider Date of Interview Instruction Please assist USAID assess the outcome of the FORECAST Liberia Project. As a beneficiary of the project, please give us your frank assessment of the outcome of the program, based on your experience. Pre-Course/Training 1. How were you selected for training? ______________________________________________________________ _________________________________________________________________________________ 2. Were you aware of or participate in the process that led to your selection? ______________________________ 3. Did your manager participate in it? In what way(s)? Please explain briefly? _______________________________ _________________________________________________________________________________ 4. What knowledge, skills or other benefits did you expect to gain from the course? What goals did you set for yourself? Please be as specific as possible. _________________________________________________________________________________ _________________________________________________________________________________ ___________ 64 Post Course/Training About the Training 5. Now that you completed your course, did you achieve the goals/expectations you set? Why or why not? _________________________________________________________________________________ __________ _________________________________________________________________________________ __________ 6. Was the training designed to match your need in terms of what you needed to know to improve the way you do your job? Could it have been done differently? _________________________________________________________________________________ _______ _________________________________________________________________________________ _______ _________________________________________________________________________________ _______ 7. Was the training duration and content adequate for your needs? (For only for short-course beneficiaries) _________________________________________________________________________________ ________ 8. Was the training provider adequate – in terms of knowledge, methods, and training environment? Please describe briefly. _________________________________________________________________________________ ________ _________________________________________________________________________________ _______ _________________________________________________________________________________ _______ 9. What would you had changed (or done differently) about your training if you had the authority to decide? __________________________________________________________________________________________ __ Utilization of Learned Skills 10. How are you making use of the training you received? Do you have any particular “success story”? _________________________________________________________________________________ ___________ 65 _________________________________________________________________________________ ___________ _________________________________________________________________________________ ___________ 11. Which elements have been favorable and which elements have not been favorable in implementing/using the knowledge and skills gained? - Support from manager: Y / N - Appropriate equipment available: Y / N - Support from colleagues: Y / N - Relevancy of the acquired skills: Y / N - Relevant advice from others: Y / N - Other: ______________________________________________________________________________ __ 12. Of the knowledge skills that you acquired in the course, which do you believe has been/will be most useful to you on the job? You can chose up to three knowledge/skills areas _________________________________________________________________________________ ________ _________________________________________________________________________________ _______ _________________________________________________________________________________ _______ 13. How often do you make use of learned knowledge/skills? Example: Daily, Weekly, Only when needed, Never? _________________________________________________________________________________ _______ 14. Are there specific skills and concepts that you are finding very useful? Please describe briefly. _________________________________________________________________________________ _______ _________________________________________________________________________________ _______ 15. Is your manager aware of what you were taught in training? If YES, How did you ensure that? If NO, why not? _________________________________________________________________________________ _______ 16. What would you recommend for helping you apply learned skills in your workplace? 66 _________________________________________________________________________________ _______ Perceived Impact Being Made 17. Please describe briefly whether and what impacts you/your training has had/is having on your unit/department/organization today. Do you now do things differently? Please give details. _________________________________________________________________________________ _______ _________________________________________________________________________________ _______ _________________________________________________________________________________ _______ _________________________________________________________________________________ _______ The FORECAST Project/Approach 18. Please give us your general opinion, based on your experience interacting with The FORECAST Project, of the approach of the FORECAST Project to delivering training. What worked well? What kinds of improvements will you suggest? _________________________________________________________________________________ _______ _________________________________________________________________________________ _______ _________________________________________________________________________________ _______ _________________________________________________________________________________ _______ _________________________________________________________________________________ _______ Thanks for your time. 67 Appendix 5: FORECAST Implementation Steps 68 Appendix 6: Letter from the Minister of Gender & Development on Achievements Through FORECAST 69 70 Appendix 7: Report Submitted by Ethel Brooks, PT/HICD Officer, USAID/Washington EGAT Bureau Liberia FORECAST Participant Training - Capacity Development Assessment March 26 – April 1, 2011 I. Purpose: This end-of-project assessment of USAID/ Liberia FORECAST Participant Training / Human Capacity Development (PT/HCD) Project is intended to help inform the Mission's decisions about the future of its capacity development and participant training initiatives. In doing so, it takes a brief backward glance at operational challenges, successes, opportunities and focuses on areas for consideration in moving forward over the next five years. The recommended next steps are based on discussions with USAID technical officers and external stakeholders. The theme that emerged from meetings and discussions is the unrelenting need to ensure that future training and other interventions are strongly based on the Work Units’ performance gaps. This is a crucial focus for measuring and then sustaining results. Further, the discussions led to the conclusion that the Mission will be well served by continuing a FORECAST project to implement training and capacity development II. Background: The concept and statement of work for the FORECAST PT/HCD Project were developed in November 2007, when the country and the mission were emerging from a period of transition. The SOW requested the contractor to deliver three broad categories of services, including: 1. Conduct an assessment to determine and prioritize human capacity development and workforce skills needs. 2. Implement USAID/Liberia’s Participant Training and Human Capacity Development (PT/HCD) program including providing technical assistance interventions to local key organizations, and, 3. Mentor and develop the capacity of local organizations to manage similar training, human capacity and workforce development programs, including providing technical assistance interventions to local key organizations. The following outcomes were expected to be realized through implementation of the PT/HICD project: √New skills, knowledge and attitudes developed and sustained within the participating institutions or government agencies leading to improved performance through the adoption or support of productive work habits and operational practices that address priority goals and challenges of the respective participating organizations and GOL entities. √ Good will and cultural understanding will be engendered. √ Participants commit themselves to 1) timely completion of their program and applying their skills at their work sites and, 2) sharing their new skills, knowledge and perceptions with colleagues, thereby maximizing the impact of training and creating a 71 climate of growth, wider access to information and an on-going learning environment in their institutions, as well as the broader community. √ Participants form networking teams and linkages with other groups with similar interests to mutually enhance capabilities, participate in community service activities, reinforce leadership potentials and promote positive trends within the country. USAID Management established a three-year base period of performance for the FORECAST Project contractor with expectations that during this period local NGO capacity would be developed to assume responsibility for on-going management of the project goals. The assessment revisited the original assumptions and goals that were made in 2007 to determine whether they remained realistic and viable beyond the three-year base period of the project. III. Methodology In carrying out the assessment, information was drawn from a pre-arrival briefing with Africa/SD; a review of work that is being on the Mission’s draft strategic framework, program lists and other USAID and project documents; meetings with the USAID Mission Director, meetings with technical team leaders and the implementing contractor’s staff; and, interviews and discussions with ministry officials, returned participants and their supervisors, an NGO beneficiary and other stakeholders (see Appendix I). IV. Discussions and Findings Within USAID: The first internal discussions with technical team leaders led to the following mixed conclusions: The FORECAST project was under-utilized during the base period, due largely to: (1) a lack of understanding about the nature and scope of the project, and (2) the perception that FORECAST was not user friendly. The in-house FORECAST management was perceived as distracted and overshadowed by other mission priorities, thereby diluting effectiveness, and negatively affecting the coherent flow of useful information available to technical staff, especially those who were new to Post. However, in instances where FORECAST was used, the respondents concluded that the programs were handled well. Team leaders noted that capacity development and training will be embedded in all of the technical projects and, to a large extent, will be managed by technical assistance contractors to ensure that sector-specific needs are appropriately addressed by subject matter experts. Nevertheless, it was acknowledged that a FORECAST-type mechanism offers advantages in implementing cross-cutting programs (such as management, ICT, finance, etc).It was recognized that such a mechanism can facilitate strategic coordination across sectors, enhance uniform standards of performance, and minimize duplication of administrative, processing, logistical, monitoring and data reporting functions, thereby increasing cost efficiencies for the Mission. FORECAST is also valued for its flexibility in enabling the mission to respond quickly to impromptu needs. EXTERNAL: Discussions external to USAID were almost unanimously positive about the advantages and gains under FORECAST. Anecdotally, stakeholders indicated that the FORECAST project has made a difference in the performance, self esteem and morale of persons who have completed programs. 72 The Ministry of Health and Social Welfare official effusively cited improved performance in two areas that he directly attributed to FORECAST intervention, namely Monitoring & Evaluation and Human Resources Policy & Management. Both units within the Ministry are now headed or strongly influenced by persons who received training under FORECAST, and both are mentoring colleagues and cascading the skills they acquired. Previously there were no Health Economists in Liberia; now thanks to FORECAST there are two who are nearing the completion of their program, which will enable the tracking of healthcare costs. The Ministry official also reported that their decentralization efforts will benefit and be accelerated as a result of FORECAST training. He was emphatic in noting that FORECAST training definitely has a catalytic role in bringing about changes. The Ministry of Education’s Deputy Minister for Administration elaborated on the improved performance of Monitoring & Evaluation and recognized the performance of other staff that are now more resourceful and who carry out their work with greater confidence as a result of FORECAST training. The Agency for Economic Development & Empowerment (AEDE) Director attributed its newly acquired capacity to assess internal performance, as well as conduct capacity development assessments for other NGOs, to FORECAST training and related interventions. In addition to a backward glance, the internal and external discussions and interviews, focused on ways to strengthen a successor FORECAST, or a similar mechanism. The discussions and interviews acknowledged that FORECAST was put in place to address the Government of Liberia’s and USAID’s first critical transitional needs with a sense of urgency at a time when public and private sector capacity gaps were broad and deep. Capacity needs are still prevalent and are projected to be so for the foreseeable future; however the needs will be developed and addressed more strategically in an environment that differs significantly from the last 5-7 years. Based on this realization, it was suggested that the next capacity building and training project should: 1) Establish clear project goals and objectives; disseminate the goals broadly and often; and adhere to priorities but with enough flexibility to support unanticipated or emerging needs; 2) Involve beneficiaries and stakeholders in the project design; 3) Allow more time for start-up; 4) Include an outreach and knowledge-sharing component in the project to continuously inform mission staff and other stakeholders of available programs, new possibilities and outcomes; 5) Ensure adequate in-house management; 6) Solidify coordination with the Ministry of Planning, the Civil Service Agency, and Human Resources as the entry point for capacity development assessments across sectors; 7) Be realistic about what can reasonably be accomplished, and develop a five-year period of performance, rather than three years ; 73 8) Ensure a good balance between in-country and external training and other capacity development interventions; draw upon in-country training providers to the extent feasible, and supplement them with outside specialists that conduct training in-country; 9) Build upon, rather than duplicate what has been accomplished under FORECAST I; and, 10) Coordinate with other donors’ capacity development initiatives. Other consistent themes emerged during discussions that have implications for the design and management of a future project. ● All of the stakeholders cited emphatically that the FORECAST PT/HCD Project is a valuable mechanism, is providing high demand services, is being managed in a responsive manner, and for various reasons, should be continued. Ministry officials and staff commended the contractor’s planning and concerted efforts to get participants identified and placed in training programs, with a sense of urgency and under taxing circumstances. They were particularly pleased with USAID’s and AED’s overall collaboration with the ministries’ priorities in selecting candidates for training; several mentioned their satisfaction with AED’s far-reaching efforts to find solutions to time sensitive challenges. They specifically noted AED’s efforts to identify schools and arrange placements for long-term programs in third countries and in the U.S., notwithstanding delayed receipt of applications from ministries, and the fact that university admissions deadlines had passed. This was reiterated as a carryover point from the mid-term evaluation to make a case for flexibility when unforeseen circumstances occur. Other accomplishments were re-visited, including: ●The development of an operational handbook and procedures by the contractor that provide stakeholders with critical actions and timelines, and is expected to strengthen their commitment to uniform standards, openness and transparency. This handbook should be considered as a reference for future use. ● Targets for in-country training exceeded the plan by more than 300%, while the target for training in U.S. universities was not met. Future planning should factor in the difficulty of getting visas for travel to the U.S.A Ministry of Education official elaborated on his numerous, yet unsuccessful efforts to help rectify the issue with visas. ● Basic management practices, computer training, and monitoring & evaluation were singled out as particularly effective. Training in these cross cutting areas is projected as an on-going need. ● Inclusion of policy makers on the FORECAST PTHCD Steering Committee was mentioned as a contributing factor in the effectiveness of the program, and is recommended for future planning. ● USAID and Ministry of Planning coordination of integrated capacity development monitoring and evaluation, with a view toward greater involvement of line ministries, is viewed as a valuable approach to minimizing duplication and providing support to help supervisors track results. ●A networking structure has been established for persons from the various ministries who have completed training, although the levels of engagement vary between ministries. The Ministry of Education beneficiaries have initiated mini-workshops and other activities to share their newly acquired knowledge and skills with colleagues. Most of the other ministries are yet 74 to initiate such activities. This structure should be further developed and broadened during the next project period. Going forward, a successor project should not only address in greater depth the areas cited above, but should also stress: • Closer adherence to deadlines for selecting program participants. Delays experienced at the start up were, reportedly, due to information gaps and inconsistent understandings about the nature of capacity development, versus academic scholarships. Although the delays were somewhat unavoidable, they nevertheless limited options for university placements of degree￾seeking participants. • More attention to creating a work place culture of follow up, and documenting post-training results. • More on-site technical assistance to help supervisors develop greater capacity to make more effective use of, and thereby retain staff after they complete training. • Provisions for monitoring the appropriateness of programs, methodologies and progress of participants in third country long-term training, especially in non-Africa region institutions. In addition to tracking and ensuring the well being of current participants, the monitoring should also provide useful insights about the universities’ practical, applied instructional approaches versus predominantly theoretical instruction, and will help guide future placements. Considering the difficulty in getting visas for U.S. long term training, and the absorptive capacity in selected disciplines in African universities, placements in other non-African countries will become increasingly important. •Seek and expand opportunities for linkage between FORECAST PT/HCD and other USAID interventions, especially training that is implemented under technical projects. •Develop a short-term, intermediate, and longer term follow up plan An issue that remains to be resolved is whether the Mission will be better served by having its own contractor or using the services of a central IQC contract. 75 Appendix 8: The Future of FORECAST: Perspective of USAID/Liberia Education Advisor, James Gray The Future of FORECAST April 25, 2011 In the SOW for FORECAST I, the contractor was to: 1. Conduct an assessment to determine and prioritize human capacity development and workforce skills needs. 2. Implement USAID/Liberia’s Participant Training and Human Capacity Development (PT/HCD) program including providing technical assistance interventions to local key organizations, and, 3. Mentor and develop the capacity of local organizations to manage similar training, human capacity and workforce development programs, including providing technical assistance interventions to local key organizations. Whether USAID/Liberia decides to do PT/HICD as a part of its other projects or as a separate project, these tasks will still have to be performed. Any new FORECAST-like project: 1. Should be predicated on in-depth, official needs assessments conducted, in conjunction with the USAID-funded contractor, by all ministries and agencies that will benefit from USAID￾funded PTor HICD.For GOL organizations, these assessments should be done in cooperation with the Civil Service Agency and the Ministry of Planning. 2. Should aim at critical gaps—those that if not filled could result in the failure of efforts to fill other gaps. The PT and HICD gaps need to be clearly identified and a clear line drawn between the intervention (training, mentoring, or workshop) and the resultant benefit to the organization. The contractor and the organization should be able to describe exactly how the PT/HICD intervention would help fill the gap. The existence of a gap does not necessarily justify an intervention to close it. The gap must be critical or important to the mission of the organization. 3. Should work with beneficiary organizations to establish clearly recognizable indicators that the gap identified before the PT or HICD intervention was actually filled or whether some subsequent, minor intervention is required. Post intervention mentoring could be key to achieving this goal. 4. Should formally review PT and HICD activities of other USAID funded projects and activities funded by other donors such as the World Bank, UNDP, UNICEF, EU, DfID, SIDA, etc. The new project should complement and leverage other interventions. 5. Should ensure that any training efforts or workshops involve individuals from relevant sections (human resources, training, or management) of the organizations undergoing training, even if these individuals are not being trained. In order to reinforce internal 76 capacity, these sections should be involved at every stage of the process: planning, logistics, implementation, evaluation, post-training follow-up, mentoring …). 6. Should require that those who benefit from training, especially from long-term training, commit to formally sharing what they learned with others in the organization. As mentioned again below, the project should identify mentors who can support these individuals not only in their own work but also in sharing the information they learned. 7. Should require the organizations who benefit from PT or HICD to make formal commitments to providing support, both physical (e.g. desk or computer) and institutional (e.g. possibility of promotion or other incentives or a clear career path opened by the training), for the participant after the training is complete. In FORECAST I, participants often complained around their inability to be effective after training because their organization was unable to provide necessary support. Other participants complained that they were stuck at their old level despite their newly acquired skills. Incentives are important to motivate everyone to do their best. If participants are not rewarded after training, there is a greater likelihood that they will seek their rewards elsewhere. Thus the gap they were intended to fill will remain unfilled in the organization. 8. Should be flexible enough to respond to immediate PT and HICD needs that may not have been recognized in the initial needs assessment. This was a strength of FORECAST I. FORECAST I worked with a few local partners (NGOs like AEDE and MADET or official organizations such as LIPA) to assist in its PT and HICD efforts. Unfortunately these partners required training and capacity building themselves before they could be totally effective. Although these organizations still need assistance, they are better equipped today to assist any subsequent PT/HICD efforts. 9. Should continue to utilize local partners to implement PT and HICD. First, it is cheaper and eliminates the major logistical effort to send people out of country and then support them while abroad. As a result of the reduced costs, many more participants can be trained. It may be necessary, however, to bring in experts for some specialized training. However, even this may be cost effective. Second, by utilizing local partners, the sustainability of USAID’s efforts will be enhanced. 10. Should work with local organizations (perhaps AEDE or MADET) to support the project by performing services for participant that were done by project staff in FORECAST I such as a. Obtaining visas b. Making travel local and international travel arrangements including accommodations c. Identifying schools or training providers outside Liberia who can provide competent service d. Maintaining regular contact with participants while they are outside Liberia and being capable of offering limited levels of personal support. 77 One of the successes of FORECAST was in monitoring and evaluation (M&E). Participants attributed much of the success to the presence of Thomas Wobill, the M&E specialist on the FORECAST team. Mr. Wobill’s expertise allowed him to provide direct monitoring of those trained in M&E throughout the life of the project although this mentoring was not formally apart of the FORECAST project. Since he has no special expertise in other areas, Mr. Wobill was unable to provide similar support to participants in other areas (e.g. IT or HR). 11. Should consider the provision of technical support for areas in which there are a significant number of beneficiaries. This technical assistance could be done by specifically placing mentors within the project either on a permanent or short-term basis. Mentoring could be episodic. Alternatively or in conjunction with FORECAST provided mentoring, the project could identify local experts to provide mentoring after participants had undergone training. 12. Should support the formation of formal networking groups. Although this was done in FORECAST I, members of these groups noted that work related time conflicts often prevented them from getting together. Perhaps efforts should be made convince organizations of the usefulness of networking groups to the operation of the organization itself. Current and future USAID-funded projects will certainly involve PT and HICD. However, in many cases these efforts will be specific to the project’s sector (e.g. nurses, engineers, or human rights specialists.) 13. Should focus on organizations that can facilitate the efforts of USAID projects or leverage their effectiveness just as building roads enhances agricultural interventions. For example, a. PT and HICD efforts could be directed through the Ministry of Internal Affairs at traditional leaders such as chiefs or zoes who, if given the tools, could assist in the delivery of USAID project outputs such as health and sanitation, education, democracy and governance, or economic growth b. As the Government of Liberia begins to decentralize, increasing the capacity of individuals in the various districts and counties may be important even if these individuals are not directly involved in a particular project. c. Throughout Liberia, the only organizations that are in every village are the churches mosques. Through training and the provision of training material, these organizations could maximize the output of projects. For example, churches and mosques could spread the effectiveness of washing ones hands, encourage the use of mosquito nets, explain the germ theory of disease, promote breast feeding, provide civic education, encourage the participation of citizens on the management of schools, etc. The outputs and effectiveness of FORECAST I were measured by the following:  # of staff of targeted departments trained in management practices  # of departments benefiting from basic management practices training 78  # of staff of targeted departments completing technical skills training (More than a thousand people in a combination of long-term and short-term courses  # of departments benefiting from technical skills training 14. Should not be overly concerned with numbers but with the impact of those trained on the effectiveness of the organization. 79 Appendix 9: FORECAST Field Trip Report Liberia FORECAST Project Monitoring and Evaluation Unit Field Visit Report February - March, 2010 Submitted by Thomas A. N. Wobill International M&E Specialist 80 1.0 Introduction As a continuation of the FORECAST capacity building package, beneficiaries are followed up by the M&E specialist to undertake a baseline study of the situation on the ground before they begin to use their skills for the performance improvement their units/departments. To accomplish this task for four capacity building activities3 sponsored by FORECAST, a field trip was undertaken in February and March to the beneficiaries stationed in the counties to obtain first hand information about their work status. The objectives of this monitoring activity are listed below:  To conduct baseline studies of each partner unit/department supported and develop action plans with the beneficiaries on how they would use their new knowledge/skills to improve departmental/unit performance.  To ascertain the extent of skill use and the consequent effects on the unit/department and to make recommendations for strengthening their efforts to improve the teaching of science at the high school level. In all, fourteen (14) out of the fifteen (15) counties4 were visited. Information was gathered about all the teachers, County Education Officers (CEOs) and District Education Officers (DEOs) of the Ministry of Education (MOE) in all fifteen (15) counties. See appendix for list of places visited and persons met. Beneficiaries of the other capacity building activities monitored were not distributed in all the counties as the MOE. They were also not present at the time of visit as a result of having been retired or having been called to Monrovia. 3 I. Water and Sanitation training for MPW staff, II. Science Education training conducted by the physics department of the University of Liberia for Ministry of Education teachers, and III. Management & Supervision training conducted by MDI for MOE County and District Education Officers IV. Decentralization 4 The team did not go to the Lofa county due to the conflict situation there 81 2.0 Findings 2.1 Management and Supervision Training for CEOs and DEOs by MDI Most of the beneficiaries had already started using their new skills according to their action plans developed during the training. The M&E baseline checklist was filled by the participants with the guidance of the M&E specialist and in consonance with their action plans developed during their training. While the supervisors of the DEOs (i.e., the CEOs) were present to consent to items on the action plans, it was not possible to have the supervisor of the CEOs (i.e., the Deputy Minister for Instruction) sit in the discussions because she does not reside within the districts with them. Information about their action plans will however be sent to the Deputy Minister for instruction. The beneficiaries indicated that the only obstacle to the implementation of their action plans is the lack of adequate resources for all their activities. They also asked the FORECAST team to indicate when the laptop computers, as promised, would be sent to them. 2.2 Science Education Training for Science Teachers by the University of Liberia With the exception of Voinjama Multilateral High school in the Lofa county5, the team visited all the schools indicated by beneficiaries of the science education as being tutors there. Interaction with the principals, vice principals, DEOs and CEOs proved this assertion to be false. The principals, CEOs and DEOs that were contacted all confirmed that those names were strange in their schools/districts/counties. They were unhappy that FORECAST had sponsored teachers for training without first consulting with their school principals, county officials or the Ministry of Education. 2.3 Water and Sanitation Training – Ministry of Public Works MPW hardly had offices within the counties at the time of visit. The special assistants to the Superintendents could not confirm the existence of MPW staff at the county level. Those who were confirmed to be present at the county level had been retired or called to Monrovia for a meeting by the MPW minister (or their supervisors). In some of the counties, the FORECAST team was informed that the MPW personnel were not resident there. They only came to undertake tasks and went back to Monrovia or other places of residence not known to the respondents. The team did not meet with any MPW personnel. Only one of them was spoken to by phone and she indicated that she was in Monrovia at the time. 5 The CEO and the Principal were contacted on phone. 82 2.4 Decentralization Training The beneficiaries of the decentralization training were mostly stationed in Monrovia, and have already been followed up. One of them was stationed at the county level6. Interestingly, the day of the FORECAST team’s arrival at his county coincided with the day he departed for Monrovia and only returned when the team had already begun their journey back to Monrovia. 3.0 Conclusions The field trip was worthwhile as it provided evidence on the ground about how FORECAST supports performance improvement of partner institutions at the county level. However, given the nature of the roads, it is advisable to undertake field trips during the dry season as was done this time. The CEOs and DEOs were appreciative of the management, leadership and supervision training they received and were enthusiastic about implementing their action plans. It was evident that the organizers of the science education training presented false information to FORECAST to collect perdiem, T&T and accommodation allowances. The teachers were all non-existent, even in the county where the training activity took place. 4.0 Recommendations Management, Leadership and Supervision Skills Training Work should be expedited with the MOE to create the enabling environment for the CEOs and DEOs to effectively use their new skills and knowledge to implement their action plans to improve performance at their various work stations. FORECAST should work on providing the laptop computers as promised. FORECAST should work with the MOE M&E unit/staff through the Deputy Minister for Planning and Development to operationalize a monitoring system (also within each department) to effectively follow up on training beneficiaries. Science Education Training Appropriate action should be taken to identify the source(s) of error and elements of fraud in the planning and execution of this training activity. Those found culpable should be made liable for all costs incurred as well as necessary actions taken against them. FORECAST should also work with the authorities from the University of Liberia and MOE to make the organizers produce the teachers who attended the training and ensure that they use their skills effectively. Water and Sanitation Training 6 Zwedru in the Grand Gedeh County. 83 MPW should clearly indicate the employment status and locations of their staff members for effective follow up on their performance. 5.0 Field visit itinerary Routes Days County Locations Dates (2010) 1 1 Cape Mount, Bomi Feb 18 – 19 2 Gparpulu 2 1 Buchanan Feb 25 – 26 2 River Cess 3 1 Kakata, Gbarnga 2 Saniquelle Mar 3 – 5 3 Kakata, Gbarnga 4 1 Grand Gedeh Mar 15 – 20 2 River Gee 3 Maryland 4 Grand Kru 5 Sinoe Annex I: MDI training beneficiaries County Location(s) Beneficiaries Result 1 Grand Cape Mount Sinje 1. William G. S. Johnson (CEO) Baseline checklist to be filled and submitted to 84 County Location(s) Beneficiaries Result 2. Moses M. Paul (DEO) 3. Mad. Hannah B.C. Tamba (DEO)7 FORECAST. 2 Bomi Tubmanburg 1. Amos A. Fully (CEO) 2. Moses Flomo (DEO) 3. Allemanuel V. S. Kayee (DEO) Checklist filled and submitted. 3 Gbarpulu Bopulu 1. Boimah Q. Taweh (CEO) 2. Wellington Y. Kerkulah (DEO) 3. Kelvin J. Lawrence (DEO) Baseline checklist to be filled and submitted to FORECAST. 4 Grand Bassa Buchanan 1. Chapman L. Adams (CEO) 2. Clarence O. Reeves (DEO) 3. Augustus N. Karyor (DEO) Checklist filled and submitted. 5 River Cess Cestos City 1. James G. Baryoegar (CEO), 2. Dallas A. V. Gueh (DEO), 3. Jacob R. Kouviakoe (DEO), Checklist filled and submitted. 6 Margibi Kakata 1. William F. Gizi (CEO) 2. James G. Gaye (DEO) 3. Mohammed J. Bayoh (DEO) Baseline checklist to be filled and submitted to FORECAST. 7 Bong Gbarnga 1. Amos Kwelleegbo G.S. Kapu (DEO) 2. Jackson N. Sengbey (DEO) 3. David T. Haye (DEO) 4. George M. Garmeche (DEO) 5. Joe F. Weedor (DEO) Baseline checklist to be filled and submitted to FORECAST. 8 Nimba Sanniquelle 1. George Weo (CEO) Baseline checklist to be filled and submitted to 7 Mad Hannah Tamba not yet working in the county 85 County Location(s) Beneficiaries Result 2. Joseph N. Kulah (DEO) 3. Clifford N. Konah (DEO) 4. Samuel Paye (DEO) 5. Emmanuel K. Dan (DEO) FORECAST. 9 Grand Gedeh Zwedru 1. Rev. David N. Towah, Jr (DEO) 2. Bestman R. Charpy (CEO) Baseline checklist to be filled and submitted to FORECAST. 10 River Gee Fish Town 1. Joseph D. Bohlem (CEO) 2. Alfred T. Sackor (DEO) 3. Theodore T. Walker (DEO) Baseline checklist to be filled and submitted to FORECAST. 11 Maryland Harper 1. James N. Barti (CEO) 2. Joel M. Jackson 3. Mad. L. Siede Williams-Kopah (DEO) Baseline checklist to be filled and submitted to FORECAST. Joel had been assigned to the county but had not reported. 12 Grand Kru Barclayville 1. Harry D. Doe (CEO) 2. N. Juwle Toe (DEO) 3. Jacob Oati Nyewan (DEO) Baseline checklist to be filled and submitted to FORECAST. 13 Sinoe Greenville 1. Stanley B. Tuolee (DEO) 2. Kieh W. Wisseh (DEO) 3. Mike S. Naklen (DEO) Baseline checklist to be filled and submitted to FORECAST. 14 Lofa Voinjama 1. Augustus Y. Smith (CEO) 2. John K. F. Mulbah (DEO) 3. Anthony Arzoaquoi (DEO) Baseline checklist to be filled and submitted to FORECAST. 15 Monsterrado Monrovia 1. Wilfred N. Dahn (DEO) 2. Kolubah H. Flomo (DEO) 3. Malayee S. Cheyard (DEO) Baseline checklist to be filled and submitted to FORECAST. 86 Annex II: Science Education training beneficiaries County School(s) Beneficiaries (Teachers) Result Person(s) contacted 1 Grand Cape Mount St. John High School Isaac Massaquoi Teachers not known Mr. Roland S. Mendimasa Sr. (Principal) Robertsport High 1. Edward Gbessie 2. Zobon Barmadia Teachers not known 1. Mr. Jivrius Moore (principal) 2. Mr. James G. Kiazolu (vice principal) 2 Bomi CH Dewey 1. Charles Rogers 2. Raymond Sieh Teachers not known 1. Mr. Faikai F. Dorley (principal) 2. Mr. Amos A. Fully (CEO) 3 Gbarpulu Bopulu 1. Augustine Bundoo 2. Abraham Kromah Teachers not known Mad. Korlu Okwumuo (Central DEO) 4 Grand Bassa Bassa High School 1. Emmanuel Paye 2. Thomas Browne Teachers not known Mr. Vee Moilluoh Sherif (Principal) St. Peter Claver Catholic High School Stephen T. Tarr Teachers not known Sister Grace Boro (Principal) 5 River Cess Cesstos High 1. Abel Bloyue 2. George Cole 3. Daniel Giah Teachers not known 1. Mr. James G. Baryoegar (CEO) 2. Mr. Dallas A. V. Gueh (DEO) 3. Mr. Jacob R. Kouviakoe (DEO) 4. Mr. George Cole (Teacher) 6 Margibi Habel Multilateral High 1. Bobby Karpeh 2. Anthony S. Wolo Teachers not known 1. Mr. Larry D. Nimely (Principal) 7 Bong Gboveh High 1. Thomas Bonokollie Teachers not Mr. Edwin G. Kwakpae 87 County School(s) Beneficiaries (Teachers) Result Person(s) contacted 2. Victor Tweh 3. Abul Bah 4. Moses Togba known (Principal), 8 Nimba Sanniquelle High 1. Sylvester Dahn 2. Johnson Doupu 3. Isaac Gaye 4. Dakuu Duyenku Teachers not known 1. Mr. J. Parkinson Suweh (Vice Principal) 9 Grand Gedeh Zwedru Multilateral 1. Morris Kyne 2. Roosevelt Wright 3. Moses Doe Teachers not known H. Shard Sanny (Vice Principal, Administration) 10 River Gee Fishtown Pubic8 1. Chester Nehwun 2. William Tarpeh Teachers not known Daniel D. Wesseh (Principal) 11 Maryland PleeboHigh, Cape Palmas High 1. Zack Greene 2. Gbemie Bedell 3. Francis Nyenator Teachers not known Charles Floyd Davies (Principal, Cape Palmas High); Mr. Wleh Wah, Sr (Principal, Pleebo High) 12 Grand Kru Barclayville Central 1. Victor Weah 2. Borbor Sieh Teachers not known Harry D. Doe (CEO) 13 Sinoe Sinoe High 1. Richard Seator 2. Alfred Keah 3. Augustine Tarpeh Teachers not known Cheslie S. Mennoh (Dean of Students) 14 Lofa Voinjama Multilateral High 1. Yarkpazuo Flomo 2. Kokolo Wolobah 3. Moses Suahkollie Teachers not known James Zogbo Karbbar (Principal); Augustine Y. Smith (CEO) 8 There was no school with the name ‘Fishtown Public.’ There were only two schools namely ‘Fishtown Demonstration’ and ‘Tweh Jaikaly Elementary & Junior High’ schools. 88 County School(s) Beneficiaries (Teachers) Result Person(s) contacted 15 Monsterrado University of Liberia Teaching / Lab Assistants & Lab Demonstrators Personnel were not available