Evaluation of the USAID/Georgia 
FORECAST Program 
Evaluation of the USAID/Georgia 
FORECAST Program 
December 2010 
This report was produced for internal review by the United States Agency for International Development 
(USAID). The evaluation was conducted by Mr. Jeffrey Shahan and Mr. Stephen Kowal of 
USAID/Washington, Bureau for Economic Growth, Agriculture, and Trade, Office of Education 
(EGAT/ED), Mr. Ivica Vasev of USAID/Macedonia, Office of Program Development, Mr. George 
Simongulashvili of the GeoEduConsulting Foundation, with assistance from Mr. David Dzebisashvili of 
USAID/Georgia, Office of Program Development. 
Evaluation of the USAID/Georgia FORECAST Program i
LIST OF ABBREVIATIONS 
COP Chief of Party 
COTR Contracting Officer’s Technical Representative 
CPT Certified Performance Technologist 
DRC Danish Refugee Council 
EGAT/ED Bureau for Economic Growth, Agriculture, and Trade, Office of Education 
FORECAST Focus On Results: Enhancing Capacity Across Sectors In Transition 
HICD Human and Institutional Capacity Development 
HPI Human Performance Improvement 
HPT Human Performance Technology 
IDP Internally Displaced Person 
ISPI International Society for Performance Improvement 
KPI Key Performance Indicator 
MOU Memorandum of Understanding 
ONA Organizational Needs Assessment 
PA Performance Assessment 
PI Performance Improvement 
PMP Performance Monitoring Plan 
PT Participant Training 
RCA Root Cause Analysis 
RFP Request for Proposals 
SOW Scope of Work 
UNHCR United Nations High Commissioner for Refugees 
USAID United States Agency for International Development 
WL World Learning 
Evaluation of the USAID/Georgia FORECAST Program ii
ACKNOWLEDGEMENTS 
The evaluation team wishes to express appreciation to the staff of World Learning – 
Georgia for their time, consideration, expertise, and practical insights. The Georgian 
officials, officials of other organizations, and their respective staff, with whom we met, 
provided an informed appreciation of what relative progress means along the path of 
achieving meaningful and lasting performance gains. Particular thanks go to USAID 
Georgia for logistical assistance and for helping the team to better understand the “big 
picture” of the Georgian context. 
Evaluation of the USAID/Georgia FORECAST Program iii
CONTENTS 
EXECUTIVE SUMMARY ................................................................................................ 1
Recommendations ....................................................................................................... 2
Agency HICD Initiative and FORECAST Program ....................................................... 4
USAID/Georgia FORECAST Program ......................................................................... 5
Scope of Evaluation ..................................................................................................... 6
FINDINGS AND RECOMMENDATIONS ........................................................................ 6
Institutional Performance ............................................................................................. 6
International School of Economics (ISET) ................................................................ 7
National Assessment and Examination Center (NAEC) ........................................... 8
National Center for Education Quality Enhancement (NCEQE) ............................. 10
The Ministry of Internally Displaced Persons (IDP) from the Occupied Territories, 
Accommodation, and Refugees .............................................................................. 11
FORECAST Program Implementation ....................................................................... 12
Mission Program Management and Coordination ................................................... 12
Institution Structural and Management Fluctuations ............................................... 13
Change Management ............................................................................................. 13
Application of HICD Model Components ................................................................ 14
Fewer Institutions, More Depth ............................................................................... 20
Withdrawal of HICD Assistance .............................................................................. 20
Future Program Design .............................................................................................. 21
A Distinct Programmatic Niche ............................................................................... 21
Two Distinct Program Components ........................................................................ 22
Partial-HICD ........................................................................................................... 22
Performance Monitoring Plans ............................................................................... 23
Implementer Staff Caseload and Documentation Requirements ............................ 23
Accessible Performance Improvement Expertise ................................................... 24
HICD Know-How Transfer ...................................................................................... 25
GENERAL CONCLUSIONS ......................................................................................... 26
APPENDICES ............................................................................................................... 27
APPENDIX A. ORIGINAL SCOPE OF WORK ............................................................. 28
APPENDIX B. THE HICD PROCESS ........................................................................... 33
APPENDIX C. MEETINGS HELD ................................................................................. 37
APPENDIX D. INSTITUTIONS STUDIED ..................................................................... 42
APPENDIX E. DOCUMENTS REVIEWED ................................................................... 45
Evaluation of the USAID/Georgia FORECAST Program 1
EXECUTIVE SUMMARY 
The Government of Georgia’s continued commitment to reform following the 2003 Rose 
Revolution provided an opportunity for USAID to address the issue of limited host 
country institutional capacity, which remained a significant barrier to the reform process, 
and served as a contextual catalyst for the founding of the FORECAST Program. The 
program is based on the Agency’s HICD approach which advocates a holistic focus on 
organizational and individual performance through comprehensive assessments 
followed by targeted interventions to improve performance related to valued outputs. 
The USAID Mission in Georgia is currently in the process of strategic and programmatic 
planning and has asked the USAID Bureau for Economic Growth, Agriculture, and 
Trade, Office of Education (EGAT/ED) to evaluate its FORECAST Program. 
With the roll-out of a new Agency initiative for Human and Institutional Capacity 
Development (HICD) in 2004, the United States Agency for International Development 
(USAID) in Georgia proceeded with an informed approach to more holistically address 
sustainable institutional performance. The Mission’s Focus On Results – Enhancing 
Capacity Across Sectors in Transition (FORECAST) Program was established in 
September 2007. The Government of Georgia’s continued commitment to reform 
following the 2003 Rose Revolution provided an opportunity for the program to address 
limited institutional capacity which remained a significant barrier to the reform process. 
The USAID Mission in Georgia is currently in the process of strategic and programmatic 
planning and has asked the USAID Bureau for Economic Growth, Agriculture, and 
Trade, Office of Education (EGAT/ED) to evaluate its FORECAST Program. 
Since the Georgia FORECAST program began in September 2007, the program has 
implemented full HICD initiatives with nine individually selected organizations and 
numerous stand-alone training and related interventions. The impact of the FORECAST 
program is far reaching across the USAID Georgia portfolio providing valued outcomes 
to partner organizations in meeting institutional objectives and thereby to USAID 
Georgia in meeting its strategic objectives in economic growth, democracy and 
governance, energy and environment, health, and social transition. Virtually all of the 
Mission’s technical offices reported finding great value in the program and wish to retain 
the services offered by the program in the future. 
This evaluation addresses performance improvement of institutions deemed key to the 
development of the Republic of Georgia. It addresses the validity and effectiveness of 
implementation practices underlying USAID’s Human and Institutional Capacity 
Development and participant training activities in Georgia. The report also provides a 
proposed roadmap for future Human and Institutional Capacity Development initiatives 
and discrete participant training interventions designed to improve the performance of 
institutions key to USAID’s assistance objectives. 
The findings and recommendations provided by this evaluation are based on desk￾reviews of FORECAST documentation and interviews with key partner organization, 
Evaluation of the USAID/Georgia FORECAST Program 2
implementer, and Mission staff. Repeated efforts were made to verify as many of the 
findings as possible. 
Recommendations 
Key recommendations include: 
 Incorporating an “Institutional Context Check” in the partner organization 
selection process to better gauge the degree to which nominated institutions 
meet HICD selection criteria;
 Continuing building local Human Performance Technology expertise in order to 
allow for local organizations to be able to fully and independently provide HICD 
services;
 Transferring Human and Institutional Capacity Development know-how to partner 
organizations and ensuring formal position descriptions allow for full utilization of 
this knowledge to better equip partner organizations to implement ongoing cycles 
of continued performance improvement;
 Establishing criteria for graduation from USAID Human and Institutional Capacity 
Development assistance so as not to diminish or reverse any performance gains 
already achieved; 
 Requiring the HICD implementer’s requests for proposals (RFPs) for 
performance assessments express a straightforward desire for CPT-certified or 
HPI-certified experts and request core HPT/HPI methodologies, but not place 
limitations on methodologies to be proposed as additional, supplemental tools; 
 Addressing frequent institutional structure and management fluctuations with 
host country partner organizations in order to mitigate disruptive consequences. 
Other recommendations include: 
 Maintaining a quality-assurance balance for optimal implementer staff caseload;
 Reducing excessive documentation and Mission approvals requirements;
 Seeking the establishment and use of internal host country partner organization 
performance monitoring and management systems, earlier in the Human and 
Institutional Capacity Development process;
 Establishing performance scoring systems that do not depend on top 
management to provide key mid-level performers with needed performance data 
related to outputs of the organization that are of value to the new management; 
Evaluation of the USAID/Georgia FORECAST Program 3
 Including core Human Performance Technology methodologies in performance 
assessments;
 Issuing multiple MOUs, each at critical junctures in the HICD process;
 Requiring detailed descriptions of analysis of performance assessments to better 
inform all present and future stakeholders;
 Exploring management models that would allow Mission technical office staff to 
serve an activity manager role, providing further opportunity for activity 
integration and enhanced program activities management; 
 Utilizing the FORECAST implementer’s HICD Plan as a living document to be 
modified on an ongoing basis to reflect current Mission needs and priorities; 
 Limiting the use of technical assistance experts as performance assessment 
team leaders, instead favoring Human Performance Technology experts;
 Discontinuing to practice of having the implementer produce program outputs￾oriented performance monitoring plans, especially under the HICD component of 
the program. 
Specific recommendations for the design of a potential follow-on cross-cutting program 
are also provided.
Evaluation of the USAID/Georgia FORECAST Program 4
INTRODUCTION
With the roll-out of a new Agency initiative for Human and Institutional Capacity 
Development (HICD) in 2004, the United States Agency for International Development 
(USAID) in Georgia proceeded with an informed commitment to more holistically 
address sustainable institutional performance. The Mission’s Focus On Results – 
Enhancing Capacity Across Sectors in Transition (FORECAST) Program was 
established in September 2007. The Government of Georgia’s continued commitment to 
reform following the 2003 Rose Revolution provided an opportunity for the program to 
address limited institutional capacity which remained a significant barrier to the reform 
process. The program is based on the Agency’s HICD approach which advocates a 
holistic focus on organizational and individual performance through comprehensive 
assessments followed by targeted interventions to improve performance related to 
valued outputs. The USAID Mission in Georgia is currently in the process of strategic 
and programmatic planning and has asked the USAID Bureau for Economic Growth, 
Agriculture, and Trade, Office of Education (EGAT/ED) to evaluate its FORECAST 
Program. 
Agency HICD Initiative and FORECAST Program 
USAID has historically made large investments in training, each year 
supporting hundreds of thousands of host country nationals taking 
part in USAID-sponsored participant training programs worldwide. 
Increasingly, USAID has recognized that training does not have an 
impact until the knowledge or skills acquired by the trainees have 
been successfully applied to a specific work situation, which, in turn, 
results in a measurable improvement in organizational performance. Knowledge and 
skills addressed by training, however, is only one piece of the organizational 
performance puzzle. Most performance improvement experts recognize at least five 
additional factors that affect performance in any organization.1
In 2004, the Agency’s Human and Institutional Capacity Development (HICD) model 
and technical initiative was formulated in recognition that a more holistic approach to 
sustainable organizational performance is needed. HICD is a series of structured and 
integrated processes designed to remove significant barriers to the achievement of an 
institution’s goals and objectives. HICD is a USAID model of structured and integrated 
processes designed to identify root causes of performance gaps in host country partner 
institutions, address those gaps through a wide array of performance solutions in the 
context of all human performance factors, and enable cyclical processes of continuous 
performance improvement through the establishment of performance monitoring and 
management systems. 
 
1
 Chevalier, Roger. "Updating the Behavior Engineering Model." Performance Improvement 42.5 (2003): 
8-14. 
Evaluation of the USAID/Georgia FORECAST Program 5
The FORECAST Program was established and made available to USAID Missions 
worldwide in 2005 as a follow-on mechanism for the Agency to continue its long￾standing tradition of sponsoring participant training activities, but also provided Agency 
sponsoring units with a contract mechanism and access to the technical expertise 
needed to undertake individual HICD initiatives with selected host country 
organizations. 
For more information on The HICD Process, Components, Agency Policy, Mechanisms, 
and Implementation Guidance, and its components, refer to Appendix B of this report. 
USAID/Georgia FORECAST Program 
USAID/Georgia’s FORECAST program can be accurately characterized as 
one program with two very distinct components. USAID/Georgia continues to 
place high value on FORECAST as an extremely flexible mechanism that 
allows Mission management and technical offices to request discrete 
trainings and accompanying technical assistance – this first component of the 
program is the traditional approach to implementing USAID participant training 
programs, referred to by USAID/Georgia as partial-HICD. Some technical and program 
officers recognize that training addresses only individual knowledge and skills, even 
when conducted in the context of the individual’s institution, so when a participant 
returns from the training he or she still works in the same unchanged institution that is 
often incapable of supporting the new knowledge and skills sets. The Mission also 
utilizes FORECAST as a means for implementing the second component of the 
program, longer-term HICD initiatives with individual host country organizations. With 
HICD, the Mission in Georgia more systematically and systemically pursues sustainable 
institutional performance with its host country partner institutions. 
Since the Georgia FORECAST program began in September 2007, the program has 
implemented full HICD initiatives with nine individually selected organizations as well as 
numerous stand-alone training and related technical assistance. The impact of the 
FORECAST program is far reaching across the USAID Georgia portfolio providing 
valued outcomes to partner organizations in meeting institutional objectives and thereby 
to USAID Georgia in meeting its strategic objectives in economic growth, democracy 
and governance, energy and environment, health, and social transition. Virtually all of 
the Mission’s technical offices reported finding great value in the program and wish to 
retain the services offered by the program in the future. 
The FORECAST program is managed by the Program and Project Support Office (PPS) 
of the USAID Mission in Georgia, rather than any one technical office in the Mission. It 
is a cross-cutting program – designed to contribute in terms of outcomes and strategic 
impact to any of the technical sectors included in the Mission’s portfolio. The program 
been implemented by World Learning with offices in Tbilisi, Georgia, since its inception. 
Evaluation of the USAID/Georgia FORECAST Program 6
Scope of Evaluation 
The purpose of this evaluation is to inform the Mission’s decision regarding whether to 
continue a cross-cutting capacity development program beyond May 2011 by providing 
USAID/Georgia with a rapid analysis of the effectiveness and relevance of the 
methodologies utilized, lessons learned from the experiences of previous years of 
providing assistance under the program, and insights into necessary adjustments to 
improve the efficiency and effectiveness of the program in the future. The evaluation did 
not focus on the participant training aspect of the program. 
The objectives set forth in the Scope of Work (SOW) for this evaluation of USAID 
Georgia’s FORECAST Program included: 
1. Evaluation of progress in relation to the initial project objectives made in four 
selected target institutions that FORECAST has provided HICD and partial-HICD 
support for. 
2. Evaluation of the relevance of the FORECAST methodology of providing HICD 
services and interventions and monitoring their effectiveness (including 
FORECAST PMP) in relation to the project objectives. 
3. Recommendations on improving the management and substance of future HICD 
and partial-HICD interventions deriving from the above. 
FINDINGS AND RECOMMENDATIONS 
The focus of the evaluation and this report is on HICD and partial-HICD aspects of the 
FORECAST Program in Georgia, rather than the discrete training and related technical 
assistance interventions requested by the Mission. This report provides findings and 
recommendations regarding the performance improvement of certain institutions 
deemed key by USAID to the development of the Republic of Georgia. It addresses the 
validity of the methodologies underlying USAID’s on-going HICD activities. The report 
also provides a proposed roadmap for future HICD initiatives in improving the 
performance of institutions key to USAID’s assistance objectives, as well as other 
requests for discrete participant training and related technical assistance. 
The findings and recommendations provided in this report are based on desk reviews of 
program documentation and interviews with key host country partner organization, 
implementer, and Mission staff. Repeated efforts were made to verify as many of the 
findings as possible. 
Institutional Performance 
Since it began in 2005, the FORECAST program in Georgia has begun HICD or partial￾HICD initiatives with fifteen institutions representing all technical areas of 
USAID/Georgia’s assistance portfolio, conducting nine performance assessments (PA) 
and providing numerous performance solutions. Observations were made about all 
fifteen of these institutions, with a focus on four selected institutions. The selected focus 
institutions were: National Center for Education Quality Enhancement (NCEQE) 
Evaluation of the USAID/Georgia FORECAST Program 7
[formerly known as, National Education Accreditation Center (NEAC)], National 
Assessment and Examination Center (NAEC), Ministry of Refugees and 
Accommodation (MRA), and International School of Economics at Tbilisi State 
University (ISET). The remaining institutions were: Tbilisi State University (TSU), 
Ministry of Environment Protection and Natural Resources (MEPNR), Municipal Service 
Providers Association (MSPA) and Municipal Finance Officers Association(MFOA), 
National Center for Disease Control (NCDC) Persons with Disabilities, Elders and 
Orphanages Service Agency at the Ministry of Labor, Health and Social Affairs, 
Georgian Hospital Association (GHA), Georgian Accreditation Center (GAC), National 
Bank of Georgia – Human Resources Department (NBG), Kipshidze Central University 
Hospital (KCUH), Health Insurance Mediation Service (HIMS), and Educational and 
Scientific Infrastructure Development Agency (ESIDA). For more information regarding 
these institutions, please refer to Appendix D of this report.
When looking broadly on issues of institutional performance of the organizations 
involved in HICD initiatives, one must keep the idea of “relative progress” in mind. With 
the transition from the former communist system, structures that had provided a daily 
context to institutional leaders and key performers began to evolve. This evolution 
continues to force leaders and performers to manage the practical consequences of 
these underlying structural changes taking place around them. In terms of mind-set, a 
fundamental change in basic terms about the way performance is viewed is beginning 
to occur. Moreover, there are many instances where improved performance can be 
observed as a direct result of the recommended performance solutions by the 
FORECAST program. 
A distinction must be made between the differences in achievement of objectives for full 
HICD initiatives and so-called “partial HICD” initiatives, which mainly target short-term 
tactical objectives rather than full scale performance improvement of an organization. In 
general, because of their nature, partial HICD projects have achieved these short-term 
technical assistance objectives. The International School of Economics is an example 
of such an initiative. 
International School of Economics (ISET) 
The International School of Economics at Tbilisi State University (ISET) opened its 
doors to students in 2006 to offer the first Master's Degree Program in Economics at 
international standards in the South Caucasus. In addition to the two-year Master's 
Program in Economics adhering to international academic standards with English 
language instruction by visiting and permanent international faculty, ISET offers a 
research program providing students with hands-on experience in theoretical and 
empirical research and producing research and policy analysis for local needs. The 
school also has an outreach program to re-train faculty, help upgrade teaching and 
research standards in economics at partner universities in the Caucasus. 
USAID’s assistance to ISET under the FORECAST program began in January 2009 
with a goal of contributing to the growth and sustainability of ISET as a premiere 
Evaluation of the USAID/Georgia FORECAST Program 8
economic institute of higher learning in the Caucasus region.2
 The largest challenges 
facing ISET from the onset were the small size of the institution, relative inability to 
expand its role in the region, and struggles with fiscal sustainability. 
The FORECAST program collaborated with ISET to provide a package of three key 
interventions. The first was in strategic and business planning enlisting an expert 
business strategist to help ISET address issues of fiscal sustainability, financial 
management systems, its relationship with Tbilisi State University, curriculum growth 
areas, and regional outreach. The second and third intervention stemmed from the 
strategic planning sessions and consisted of the establishment of new accounting 
systems and audit strategy, and participant training in Jordan on USAID procurement 
regulation. Progress can be observed in the major areas challenging ISET: size, 
regional mandate, and funding. The size of the faculty has doubled from three to six 
permanent faculty members. Both contributing to its size and its visibility in the region, 
ISET has established a visiting faculty program. The program focuses on engaging 
faculty from Azerbaijan and Armenia through teaching opportunities and retreats. 
Further collaboration with similar education institutions in the region has resulted in 
increased attendance of students from Armenia from zero in 2009 to ten in November 
2010. While ISET continues to face a financial shortfall for its fiscal year 2011, a less 
expensive, internationally-accepted accounting and auditing system is now in place. In 
addition, ISET’s short-term strategy includes plans to begin applying for grants from 
international donors and the academic community to help address the future funding 
situation. 
USAID’s assistance to ISET under the FORECAST program is viewed as very 
successful by all stakeholders in terms of outcomes, but contributing factors to this 
success appear to be more related to ISET leadership and collaboration, rather than the 
application of HICD principles. In fact, while USAID refers to this FORECAST initiative 
as partial-HICD, it is the only focus initiative of this evaluation that cannot be considered 
HICD because none of the key components of the Agency’s HICD model were applied. 
That said, with its manageable size and performance-minded leadership, ISET stands 
as strong candidate for true future HICD assistance. 
Strictly in terms of applying HICD principles, USAID/Georgia should consider continued 
assistance to ISET in follow-on to the strategic planning that has already taken place in 
order to help ISET establish and integrate an ongoing performance monitoring and 
management system based on the school’s business plan. Other performance 
solutions might be considered to help ISET to sustainably address its anticipated fiscal 
challenges. 
National Assessment and Examination Center (NAEC) 
The National Assessment and Examination Center (NAEC) in Georgia cites its origins 
with the initial implementation of the Georgian Education System Realignment and 
 
2
 World Learning. "FORECAST Caucasus Fiscal Year 2010 Semi-Annual Report." 2010. 
Evaluation of the USAID/Georgia FORECAST Program 9
Strengthening Program by the Georgian Ministry of Education and Science (MES) in 
1999. NAEC was established in 2002 by the MES, with national examinations being 
carried out in Georgia for the first time the same year. NAEC serves to administer 
standardized tests for undergraduate and graduate admissions, as well as standardized 
teacher certification across the country. 
USAID set out to contribute to the overall improved performance of NAEC by requesting 
that a full HICD initiative be carried-out through its FORECAST program beginning in 
October 2008. NAEC’s organizational goals included the rollout of a general graduate 
examination and the building of internal capacity to meet its expanding organizational 
mandate and workload brought on by a public focus on the role of NAEC by the 
Georgian Government.3
A performance assessment was carried out through the FORECAST program in 
October/November 2008 by international performance improvement consultants and a 
local FORECAST sub-vendor for HICD expertise, with findings that included 
organizational performance gaps that were related to factors of human resources, 
physical environment and equipment, and coordination planning. The assessment 
resulted in a performance solutions package that included technical assistance in 
developing a professional management and planning system, as well as assistance 
related to human resources management and personnel development. In March 2009, 
the first intervention of a workload planning and coordination tool was developed by an 
international performance improvement consultant with daily, on-the-ground cooperation 
of NAEC management and staff. The Director of NAEC approved and accepted the 
final delivered tool with cited enthusiasm. But, the full tool has never been used as it 
was designed to be due to a lack of sincere buy-in by department heads. The second 
planned intervention was to be implemented only after the NAEC established an actual 
HR department. The second intervention was cancelled at the request of the NAEC 
Director citing a need to concentrate on other urgent organizational output priorities and 
an impending move to new facilities before an HR department could be established. A 
viable connection between the additional workspace afforded by the new facilities and 
the hiring of an HR manager could not be ascertained by the evaluators.
USAID’s assistance to NAEC under the FORECAST program can be best viewed as a 
challenging lessons-learned. NAEC appears to operate with an informal “family” culture 
that is accompanied by resistance to formalizing internal processes and 
communications. The center is experiencing the growing pains of staff expansions, and 
increased mandate and visibility by the Government of Georgia that such an informal 
“family” structure strains to support. There certainly appeared to be genuine initial 
enthusiasm and willingness on the part of top management to introduce change, but 
middle management and the rest of the staff do not see any value in replacing current, 
more informal means of communication and coordination. It should be stated that a 
small, modified component of the workload planning and coordination tool in the form of 
 
3
 World Learning. "FORECAST Caucasus Fiscal Year 2009 Quarterly Report." 2009. 
Evaluation of the USAID/Georgia FORECAST Program 10
a simple process timeline and checklist is being used and appreciated for its relative 
value by many at NAEC.
USAID should explore the possibility of working with the National Assessment and 
Examination Center after they move to the new premises and establish the much 
needed HR department, but buy-in from middle-managers must be ascertained. 
National Center for Education Quality Enhancement (NCEQE) 
Formerly a department within the Ministry of Education and Science, the National 
Center for Education Quality Enhancement (NCEQE) has grown into an entity that 
administers institutional and program accreditation for both private and state owned 
educational institutions in Georgia. 
At the time when the collaboration with USAID’s FORECAST program first began, the 
Center was known as the National Education and Accreditation Center (NEAC). The 
goal of the full HICD initiative with NEAC was to improve the overall performance of the 
Center in order to foster the development of an accreditation system and enhancing its 
service delivery, especially in training schools in self-assessment and external 
evaluators in evaluation methodologies. 
The performance assessment was conducted in two phases during July and October 
2008 and the performance solutions package included developing a strategic plan, 
personnel system, and a database as well as redesigning their core processes and 
establishing management systems for planning and evaluation. These interventions 
were implemented from February 2009 to August 2010 and while the core processes 
were mapped successfully and management system was established, there was a low 
motivation to contribute to development of the action plan due to the director’s 
resignation in June 2009. 
During the interview with the new Director of the Center, the evaluators found out that 
apart from the database that they still use, the other performance solutions were no 
longer in place. NCEQE has undergone a tremendous increase in staff, competencies 
and budget and although the management recognizes the success of the HICD 
program, most of the solutions appear to be inapplicable to the present situation. 
Regardless of the fact that the performance solutions were implemented successfully, 
the fluctuations in top management and the increased mandate of the Center point to a 
lack in the case of this HICD initiative of two very important features of the Agency’s 
HICD methodology: obtaining buy-in at the highest level possible and incorporating 
performance solutions features designed to address potential management changes 
into the package. 
Evaluation of the USAID/Georgia FORECAST Program 11
The Ministry of Internally Displaced Persons (IDP) from the Occupied 
Territories, Accommodation, and Refugees 
During the first half of 2008, and prior to the conflict with Russia, the Government of 
Georgia and the Ministry of IDPs from the Occupied Territories, Accommodation and 
Refugees (formerly known as MRA), with donor assistance, put in place an Action Plan 
in order to integrate the IDPs into mainstream society and thus make operational the 
State Strategy for IDPs. 
In February 2009, what started as a discrete request for technical assistance to 
increase the capacities of MRA to respond to the needs of the IDPs from the conflict in 
2008, become a true HICD initiative. The performance assessment and its findings and 
recommendations were endorsed not only by MRA but also by the international donors 
who based their own capacity building support to the Ministry on those 
recommendations. United Nations High Commissioner for Refugees (UNHCR), the 
Danish Refugee Council (DRC), the Swiss Development Center, European Union and 
the World Bank invested resources based on the solutions package of the HICD 
initiative under the FORECAST program.4
The Steering Committee, a formal coordination body within MRA, inspired high level 
participation by all donors and interested stakeholders and drew more than €50 million 
from EU. It was conceptualized and established as a coordination mechanism by the 
FORECAST implementer and stands as one of the program’s major successes. The 
Steering Committee resembles the HICD Stakeholder Group although in this case 
formalized and structured even more so in order to address the need of this particularly 
vulnerable population of IDPs. The other three areas of focus for FORECAST were the 
database and information technology strategy development, the communication 
strategy, and the human resources management. Currently, the remaining 
interventions from the performance solution package are being implemented by DRC 
thus fully assuming the performance solutions from the HICD initiative under the 
FORECAST program. 
One of the main lessons learned in this process is that coordination among various 
stakeholders is extremely important; not only to avoid overlap but even more so to 
coordinate and complement each other’s activities – gaining vital synergies needed to 
build true sustainable institutional performance. In that sense, DRC and UNHCR have 
been excellent partners of USAID and the FORECAST program in assisting the Ministry 
address IDP needs. 
 
4
 Mooney, E., & Hovey, G. (2010). Technical Assistance to the Ministry of Refugees and Accommodation 
- Phase III Final Report. 
Evaluation of the USAID/Georgia FORECAST Program 12
FORECAST Program Implementation 
The September 2007 start of the new FORECAST in Georgia under the newly awarded 
FORECAST contract marked a significant shift for USAID implementer staff. The 
FORECAST Program is designed to have the implementer respond to flexible requests 
from the Mission to implement both HICD initiatives and traditional participant training, 
with accompanying technical assistance, referred to as partial-HICD. The shift from 
providing only participant training services to a full systematic and systemic HICD 
approach required staff to become familiar with the principles of the Agency’s new HICD 
approach. One forward-edge component of the program calls for HICD activities to be 
carried-out exclusively by select local organizations, after working with these 
organizations to build their internal capacity in the area of Human Performance 
Improvement (HPI) and Human Performance Technology (HPT). The utilization of local 
performance improvement experts serves as a model for the Agency in building local 
capacity of performance improvement expertise. 
Mission Program Management and Coordination 
Management of the FORECAST program by the Program Office of USAID as a cross￾cutting activity serves to ensure that the services offered through the program are 
available equally to all technical offices without any technical sector cliques or bias. 
Most stakeholders hold the view that it is best for HICD to be a cross-cutting program, 
managed by the program office, citing synergies in methods, implementation, expertise, 
and procurement. There is also some interest by specific stakeholders in the Energy 
and Environment Office in perhaps exploring the integration of HICD principles into 
certain technical assistance programs that they manage directly. 
The evaluation team conducted fact-finding meetings with staff representing all of the 
Mission’s technical offices. Most staff members were very familiar with the purpose, 
services, and flexible-value of the participant training component of the FORECAST 
Program, but most did not share an awareness or understanding of the Agency’s HICD 
initiative or the program’s HICD component. The Mission program office should 
consider organizing activities to reinforce practical knowledge on the part of technical 
office staff of the entire Agency HICD model and sub-components, as well as the 
implementer’s role in providing expertise and services to promote awareness and 
understanding of the program’s HICD component. USAID will be offering an online 
course on HICD in 2011. Mission and implementer staff are encouraged to complete 
this course. 
Another reoccurring theme in technical office staff feedback suggests that improvement 
could be made by enhancing the technical office’s role in intervention or activity 
oversight. While the value of having the program managed by the Program Office 
seemed evident, sector experts in technical offices suggested they are asked to provide 
program approvals for interventions or activities that they are not aware of or involved 
with in terms of oversight. Ideally, FORECAST interventions should be integrated within 
Evaluation of the USAID/Georgia FORECAST Program 13
the technical offices’ portfolios. The Mission might explore enhanced management 
models that would allow staff in cognizant technical offices to be appointed to an activity 
manager role, while the program office retains role of contracting officer’s technical 
representative (COTR) and overall management responsibilities for the program. Such 
a management arrangement may also provide further opportunity for FORECAST 
program activities to be integrated with technical assistance implementer activities. 
Institution Structural and Management Fluctuations 
The often disruptive consequences of frequent institutional restructuring and turnover of 
management and key stakeholders due to political reasons have been a recurring issue 
encountered by the FORECAST Program implementer. Often new ministers, directors, 
or administrators arrive at post with their own ideas of how the institution should be 
altered and have little or no exposure or sense of ownership of any ongoing HICD 
initiative (for example, the new Director of NCEQE and the Head of Administration at 
MRA).5
 One obvious way of addressing such situations is to institutionalize process 
changes and other performance improvements at the permanent performer-levels, so 
that at least new individual practices will likely not revert to previous performance (such 
as the Personnel Development System at the National Center for Education Quality 
Enhancement, the Steering Committee at the Ministry for Refugees and 
Accommodation, and the establishment of alumni office at the Tbilisi State University). 
Establishing internal scoring systems that do not depend on top management can be 
effective in providing key mid-level performers with needed performance data related to 
outputs of the organization that are of value to the new management. It may also be 
useful to institutionalize performance changes, changing by-laws, based on changes to 
primary laws. Establishing internal standard operating procedures, and providing job 
aids aimed at the individual performer level can also be extremely effective. 
Change Management 
To address the issue of management turnover, and moreover, broader issues of 
openness and readiness for change, USAID should request that the FORECAST 
implementer reinforce its efforts to explicitly include change management principles as a 
standard HICD component for the FORECAST program. Change management is 
designed to help an organization’s leaders and performers transition their thinking 
toward true performance based outcomes and management. It is a set of processes 
implemented to ensure that significant changes are enforced in an orderly, controlled 
and systematic manner to effect organizational and individual performer change. The 
most effective way to make needed changes is to pinpoint existing resistances to 
change and then focus efforts on eliminating or reducing as many of those incidences 
and types as possible. Change management might be included as one of the 
responsibilities of the host country partner organizations as laid out in the initial 
memorandum of understanding, and possibly further specified in any later MOUs (see 
 
5
 World Learning. "FORECAST Caucasus Fiscal Year 2009 Annual Report." 2009. 
Evaluation of the USAID/Georgia FORECAST Program 14
Identify Partner Organizations
Obtain Partner Commitment
Form Stakeholder Group
Conduct Performance Assessment
Prepare Performance Solutions Package
Implement Performance Solutions
Monitor Change in Performance
the Application of HICD Model Components section of this report). Effective mitigation 
of resistance to change, recognition of past implementation problems, and a strong 
communication plan can help to ensure that HICD activities are better able to weather 
any managerial or structural changes. 
Application of HICD Model Components 
Using the Agency’s HICD process, illustrated on the right, USAID/Georgia can better 
focus on strengthening host country partner organizations that are key to achieving the 
Mission’s assistance objectives.
Identify Host Country Partner Organization 
The selection process undertaken for the HICD 
initiatives under the FORECAST Program has 
resulted in the identification of certain highly 
suitable institutions for HICD, but also several 
less so (e.g., NAEC). One major factor is the 
high level of continued fluctuation in institutional 
structure and leadership across public 
institutions in Georgia. Currently, the Mission 
program office annually solicits nominations 
from the technical offices based on a 
predetermined set of selection criteria. Final 
selection is made by the Mission front office. 
The FORECAST implementer is informed of 
selected institutions only after final selection, 
and nomination documentation is not shared 
with the implementer. 
The Mission should consider the FORECAST implementer’s HICD Plan to be a living 
document to be modified on an ongoing basis to reflect current Mission needs and 
priorities, thus institutions can be selected for HICD initiatives at any time throughout the 
fiscal year based on obligated funds available. 
The selection process would be aided by providing the FORECAST implementer with 
Mission nomination documentation and tasking the FORECAST implementer, including 
local HICD expert sub-contractors, with conducting a pre-selection institutional context 
check of the degree to which nominated institutions meet HICD selection criteria. Such 
an analysis would involve any USAID technical assistance providers and other donors 
or stakeholders, as appropriate. The results of the pre-selection institutional context 
check could also be provided to the PA team to serve as a basis for the performance 
assessment. The analysis must delve into a number of fundamental selection factors, 
including: 
 direct alignment between the organization’s and USAID’s objectives for the 
sector;
Evaluation of the USAID/Georgia FORECAST Program 15
 degree of buy-in to the HICD process from all levels of the organization, top￾down, but not just the top leadership; 
 likelihood of changes in organizational structural or leadership that may have an 
affect initiative implementation; 
 degree of collaboration with technical assistance providers. 
Having the experience of an already existing technical assistance provider working with 
a given institution can be invaluable during the selection and initiative implementation, 
so consideration should be given to revising or redefining the selection criteria related to 
“over-crowding” to allow for appropriate synergies and leveraging of resources. 
Selection criteria should also be revised to remove confusing or redundant criteria (and 
one that’s counterproductive). 
Assessing the partner’s initial commitment to undertaking an HICD initiative with the 
FORECAST implementer should become a fundamental element of the Mission’s 
selection process and may be accomplished through a series of meetings explaining the 
HICD methodology and mutually exploring performance areas to be targeted. In 
assessing the partner’s commitment to an HICD initiative, USAID should ensure direct 
alignment between the partner organization’s own internal objective and USAID’s 
assistance objectives for the sector, and that the initiative directly addresses areas of 
greatest concern to the partner organization.
Obtain Partner Commitment 
Securing partner commitment is crucial to the success of any HICD initiative. While 
gauging initial commitment of the partner must be part of the host country partner 
organization selection process, memorandums of understanding (MOU) can be used as 
a vital tool to capture the agreement of parties in writing to jointly undertake an HICD 
initiative. The current practice of the FORECAST Georgia Program is to issue one MOU 
at the beginning of a new initiative with a partner organization prior to conducting the 
performance assessment. These MOUs are signed by the FORECAST implementer 
and the leadership of the host country partner organization. 
There are examples of initial host country partner organization commitment, that later 
diminishes due to subsequent institutional or leadership changes – but, the HICD 
initiative moves forward without regard to this shift, resulting in wasted resources (e.g., 
NAEC). USAID/Georgia would be better served adopting the practice of having its 
HICD implementer issue multiple MOUs, each at critical junctures in the HICD process. 
For example, an MOU could be signed at the point of initial partner organization 
selection, prior to the performance assessment, in mutual agreement to the 
performance solutions package, prior to any performance solution of high fiscal value, 
or to address subsequent cycles of performance improvement assistance. As stated in 
an earlier section, establishing change management as a responsibility of the host 
country partner organization could help to establish buy-in and commitment from 
multiple levels of the organization early on in the activity. 
Evaluation of the USAID/Georgia FORECAST Program 16
An MOU should also be executed when leadership changes occur within the partner 
organization. Meetings should be held with the new leadership explaining the HICD 
methodology and mutually exploring performance areas to be targeted. Even if the 
HICD initiative is well underway and many performance solutions are already in place, it 
is important to engage new leadership to ensure continued usage of the introduced 
performance principles before receiving further HICD assistance. The Mission may 
consider negotiating a compromise that involves the continuation of HICD principles in 
situations where new leaders arrive with a new agenda for the organization. 
USAID/Georgia should also consider adopting the practice of including a USAID 
representative and the highest practical stakeholder leadership as stakeholder 
signatories to these MOUs in order to raise the probability of successful implementation 
of HICD programs, through increased motivation for principled performance 
improvement within selected host country partner organizations. Initial MOU signing 
may even be carried-out as public and media relations events. 
Form Stakeholder Group 
Appropriately functioning stakeholder groups do not exist for most HICD initiatives 
undertaken by the FORECAST Program (the MRA initiative being a notable exception). 
A primary function of the stakeholder group is to provide feedback to the partner on the 
HICD process, including the selection and funding of recommended performance 
solutions and evaluation of HICD results. 
The stakeholder group can also be used to communicate the value of the HICD 
initiatives to new institutional leadership and possibly mitigate the effect of 
organizational leadership changes. Functioning stakeholder groups can help to provide 
continuity and communication, stability, given often tumultuous organizational 
circumstances that were not foreseen at the beginning of the HICD activity, as well (see 
also the Institution Structural and Management Fluctuations section of this report). The 
stakeholder group formed for the MRA initiative is one of the strongest aspects of 
USAID’s development assistance to MRA under the FORECAST Program – showing 
that a proactive approach to coordination is necessary and yields positive, synergistic 
results. 
The Mission should require that the implementer begin consistently implementing this 
important HICD component. A stakeholder group should be established for each overall 
HICD initiative with a given host country partner organization. The primary members of 
the stakeholder group are management and key staff from the partner organization. 
Other members may include: Mission staff, Assistance Objectives team leaders, 
Program Officer and/or mission’s HICD point of contact, and activity manager(s); HICD 
Contractor – key staff most familiar with partner organization and/or sector and Country 
Director or other HICD expert staff; and, Technical Assistance Provider(s) – Chief of 
Party (COP) or designee, and key staff most familiar with partner organization and/or 
sector.
Evaluation of the USAID/Georgia FORECAST Program 17
Conduct Performance Assessment
At the very heart of the Agency’s HICD model is the organizational performance 
assessment (PA). This vital component designed to identify core causes of performance 
gaps in host country partner organizations, suggest a wide array of performance 
solutions in the context of all human performance factors (see appendix B) in order to 
address those gaps, and enable cyclical processes of continuous performance 
improvement through the recommended establishment of a performance monitoring and 
management system, if not already in place. As a result of the PA, the host country 
partner organization will receive a written report detailing the findings of the assessment 
team, along with the team’s recommendations for performance solutions. 
The evaluation team found that PAs under the FORECAST program are sometimes not 
being carried out by certified performance improvement experts (for example, KCUH 
and NCDC), but rather by subject-area experts in the fields primarily handled by each 
institution targeted for an HICD initiative. 
The evaluation team reviewed the HICD implementer’s requests for proposals (RFP) 
used to hire experts to carry-out the PAs that have been or will be conducted thus far 
under the program. Evidence suggests that the FORECAST implementer is not 
consistently requesting (or selecting) CPT/HPT certified experts, nor is the implementer 
consistently requesting core, fundamental HICD model component approaches. The 
desk review of these ten (10) RFPs and resulted in the following findings: 
 zero (0) of the RFPs requested that offers be CPT/HPT-certified; 
 only one (1) RFP mentioned the Agency’s HICD model to give context to the 
assessment;
 only one (1) RFP mentioned the six recognized human performance factors in 
the context of recommending performance solutions;
 only six (6) RFPs mentioned the HPT/HPI models as the technical basis for the 
performance assessment;
 only five (5) RFPs requested the establishment of performance indicators at 
various level of the organization by performance consultants in support of 
measuring performance on an ongoing basis in relation to organizational goals;
 none (0) of the RFPs highlighted the need for the assessment to address the 
establishment of an internal performance monitoring and management system to 
continually monitoring these very same performance indicators (see previous 
item) that should be identified in the assessment – another hallmark of the 
Agency’s HICD model.
The Mission should require that implementer requests for proposals (RFPs) for PAs 
express a straightforward desire for CPT-certified or HPI-certified experts and request 
core HPT/HPI methodologies, but not place limitations on methodologies to be 
proposed as additional, supplemental tools. HPT represents proven approaches in the 
science of human performance and are a founding, core component of the Agency’s 
HICD model. The consistent use of certified performance improvement experts will help 
ensure that what is being implemented under the program is actually HICD. Core 
components of the HICD and PI models (see Appendix B), include: 
Evaluation of the USAID/Georgia FORECAST Program 18
 Identification of performance gaps, based on alignment with institutional 
objectives; 
 Performing a cause analysis (i.e. -- root cause analysis, opportunity analysis; 
bridging between performance technology and systematic change; as well as 
opportunity inquiry, ideal performance system design, performance system 
creation, and appreciative evaluation); 
 Observation of the six recognized human performance factors in the context of 
the RCA when recommending appropriate performance solutions; 
 Establishment of performance indicators at various level of the organization by 
performance consultant in support of measuring performance on an ongoing 
basis in relation to organizational goals; 
 Establishment of an internal institutional performance monitoring and 
management system to allow host country partner organizations to continually 
monitoring their performance indicators as defined in the PA. 
The implementer is to be commended for constantly searching for new ways of solving 
often complex performance issues in the context of the technical sector of the partner 
organization. One cited implementation practice is procuring the services of an HPT￾expert that is also a sector-expert in the same field as the partner organization. HPT 
expertise should not be negatively influenced, overcrowd, or infused with undue 
influences by other sector-specific methodologies. When possible, the practice of 
involving separate practitioners for HPT and sector expertise, respectively, should be 
maintained. USAID’s core approach to performance improvement is contained in the 
HICD model – these are the officially recognized Agency approaches; the holistic, 
systemic, systematic characteristics of the PI model is what sets HICD apart from other 
Agency technical assistance efforts in the area of institutional capacity building. 
One important function of a PA report that is presented to the host country partner 
organization is to record the technical approaches used and the analyses performed by 
the PA team for the clear utility of all present and future stakeholders involved in an 
HICD initiative. PA reports produced on the FORECAST program varied widely in these 
terms. Requirements for the contents of PAs should also include a detailed description 
of the analysis that was conducted, as well as any instruments or surveys that were 
used to carry out the assessment. These would most likely reside in an annex of the 
report and would then be available for possible use by the partner organization itself 
and other stakeholders. 
A recurring theme in stakeholder feedback suggests that the practice of allowing 
performance improvement experts to frequent, but short, trips in and out of the country 
to work with host country partner organizations in conducting assessment or 
implementing performance solutions can be disruptive to the Georgia-based 
stakeholders. Whenever feasible and practical, HPT international consultants’ scopes 
of work should call for fewer, but longer-term, engagements in-country. This issue can 
be further addressed with the expanded use of the FORECAST implementer’s sub￾contracted local HICD expert organizations to conduct PAs, as these organizations 
Evaluation of the USAID/Georgia FORECAST Program 19
become better equipped to do so without the assistance or oversight of international 
consultants. 
A last observation regarding assessment of institutional performance is that there often 
appears to be significant lag-time between the completion of the PA and implementation 
of the first performance solutions. Implementer staff suggested this can often be an 
issue of re-connecting with the partner organization which may not have had regular 
contact with FORECAST staff during the duration of the performance assessment. To 
help address this issue HICD implementer staff could be included on the performance 
assessment team to facilitate the transition from the performance assessment phase to 
the performance solution implementation stage. 
Establish Performance Monitoring and Management System - Monitor Change in Performance 
Performance is ideally described according to measurable indicators that align with the 
mandate and objectives of the institution. However, a desk review of HICD PAs 
revealed that the methods for setting levels of desired performance and measuring 
actual performance were at times less than clear. Many of the assessments did not 
focus on descriptions of performance that came from measurable indicators linked to 
institutional goals. This can cause a disconnect between the way performance is being 
described in the assessment and the overall measurement of organizational 
performance. 
Measurable key performance indicators (KPIs) that are in full alignment with Mission 
and partner organization objectives should be established as part of the PA or early in 
the implementation of the HICD initiative. Not only would this allow for more accurate 
and objective descriptions of actual and desired performance, but it would also provide 
a baseline set of data to which post-intervention performance measurements could be 
compared. It must be noted that the institutions’ staff must understand the relevance of 
KPIs and be able to actively participate in their development. The ability to develop 
KPIs is directly related to the commitment and readiness to use them beyond program 
graduation and make them part of a broader performance management system. 
A review of PA expert RFPs and PA reports indicates that there is insufficient emphasis 
being placed by the implementer on the value USAID holds in assisting host country 
partner institutions to enable internal cyclical processes of continuous performance 
improvement through the establishment of performance monitoring systems. The 
establishment and institutionalization of performance monitoring systems (e.g., 
performance scorecard, dashboards) at the organizational-level is a hallmark of the 
HICD approach and should be a more prominent component in all future HICD 
initiatives. Notwithstanding the expert view that all processes must be mapped before 
developing unit or performer-level performance indicators, organizational-level 
performance monitoring systems can form an integral component of the PA itself and 
should not come as the last, perhaps never implemented recommendation of the PA. 
Evaluation of the USAID/Georgia FORECAST Program 20
Fewer Institutions, More Depth 
Generally, an HICD initiative implemented by the FORECAST Program in Georgia 
consists of a performance assessment and select performance solutions to address one 
cycle of performance improvement, and then the HICD initiative for a given institution is 
finished. True institutional performance improvement and capacity development is a 
long-term approach of building sustainable cycles of continuous performance. The 
HICD component of the follow-on cross-cutting program should be designed to be more 
selective and focus on longer-term, more in-depth HICD assistance, even if doing so 
results in the selection of fewer institutions. This will allow for greater engagement 
through more focused collaboration, and more targeted assistance. 
Withdrawal of HICD Assistance 
Generally, an HICD initiative implemented by the FORECAST Program in Georgia 
consists of a performance assessment and select performance solutions to address one 
cycle of performance improvement, and then the HICD initiative for a given institution is 
finished. True institutional performance improvement is a long-term approach of building 
sustainable cycles of continuous performance within the institutions themselves. 
Moreover, initiating and managing change in organizational culture and performer 
mindset takes time, often more time than simply the introduction of new technology or 
contributing to a more enabling business environment (i.e. - drafting/passing laws, etc.). 
Careful consideration should be given before withdrawing HICD support from those host 
country partner organizations that remain active, productive partners with USAID in 
improving their performance, so as not to diminish or reverse any performance gains 
already achieved – in effect squandering the resources already expended on these 
partner organizations. Criteria for graduation from USAID HICD assistance, oriented at 
sustainable performance improvement should be developed, along the following lines: 
 USAID strategic of objectives related to the institution have been met or are no 
longer linked to institution outputs; 
 The institution has demonstrated sustained, measurable levels of relative 
performance improvement that directly supports institutional goals; 
 The institution has basic levels of in-house HICD know-how and access to 
external sources as needed; 
 The institution has a meaningful, functioning performance measurement system; 
 The institution has an ongoing plan to develop and implement interventions in 
response to measurement system; 
 The institution has a strategy for obtaining resources for ongoing future 
interventions. 
Potential benefits of establishing criteria for graduation from USAID HICD assistance 
may also warrant a revision of the Agency’s overall HICD guidance to provide a general 
framework for the inclusion of this important consideration. 
Evaluation of the USAID/Georgia FORECAST Program 21
Future Program Design 
USAID/Georgia technical office staff almost universally hailed the flexibility of the 
FORECAST Program to enable them to respond to unexpected technical needs or 
requests as absolutely unique and invaluable to their program portfolios. The design of 
the program will continue to drive demand for FORECAST-type services and the need 
for a follow-on program. The Mission program office has informed the evaluation team 
of its intention to maintain one program in the future to ease management burdens, as 
opposed to de-bundling services along natural lines of distinct expertise between 
participant training and HICD. With this information in mind, the team offers the 
following findings and recommendations for future programmatic planning. 
A Distinct Programmatic Niche 
Fact-finding meetings with staff representing all of the Mission’s technical offices 
provided the evaluation team with an opportunity to gauge perceived value of the 
FORECAST Program throughout the Mission, but also the opportunity to identify any 
overarching administrative issues. Certain staff members were very familiar with the 
purpose, services of the FORECAST Program, but universally, Mission staff struggled 
to identify the distinct purpose or niche of the program, aside from its flexible 
accessibility, relative to the Mission’s sector-specific technical assistance programs. 
USAID/Georgia should consider that there are two distinct design characteristics that 
would define a follow-on program to FORECAST – a true cross-cutting program 
designed to integrate adult-learning and institutional performance improvement 
expertise across all sectors, regardless of the sector: 
 The first is indeed the intended flexible accessibility in responding to discrete 
requests for participant training and accompanying technical assistance. 
USAID’s assistance to ISET under the FORECAST program is an excellent 
example of this type of programming that is possible given the flexible design of 
the FORECAST program which is so valued by the USAID/Georgia. 
 The second is the delivery of expertise and services related to the Agency’s 
Human and Institutional Capacity Development (HICD) model and initiative. 
HICD is longer-term, more in-depth approach to address the overall 
organizational “health” of institutions, regardless of the sector, for an 
organizational performance improvement approach. Utilizing certified 
performance technologists (CPT) to provide organizational performance 
technology is absolutely unique from most technical assistance programs offering 
capacity development via sector-specific experts. 
The evaluation team observed a perceived general lack of knowledge, understanding, 
or recognition on the part of many Mission staff between HICD and discrete participant 
training and technical assistance requests (a.k.a. - partial-HICD). In fact, during 
discussions with Mission staff, the term HICD was often used interchangeable to refer to 
either of these distinct program components. USAID/Georgia will need to address this 
finding when engaging Mission staff in the design of a follow-on program. 
Evaluation of the USAID/Georgia FORECAST Program 22
Two Distinct Program Components 
The two distinct design characteristics that would define a cross-cutting follow-on 
program to FORECAST addressed in the previous section present the Mission with 
program design challenges. Participant training and HICD are two very different 
program components that require very different types of implementer expertise. The 
challenge for USAID/Georgia will be to obtain the two very different types of 
implementer expertise from a single procurement. One way of addressing this issue 
would be to design the program procurement so that a consortium of these two types of 
implementers is logically led to join forces. Another might be a sub-contractor/grantee 
relation where the areas of expertise between the prime and subordinate implementers 
are clearly defined along the lines of these distinct areas of expertise. 
Partial-HICD 
Human and Institutional Capacity Development (HICD) is a USAID model of structured 
and integrated processes designed to identify root causes of performance gaps in host 
country partner institutions, address those gaps through a wide array of performance 
solutions in the context of all human performance factors, and enable cyclical processes 
of continuous performance improvement through the establishment of performance 
monitoring and management systems. The term partial-HICD is a term uniquely used by 
USAID/Georgia to refer to packages of participant training and technical assistance 
implemented by the FORECAST implementer. Theoretically, the design of these 
packages of interventions would be based on pre-existing institutional capacity 
assessments or the pre-existing knowledge of technical assistance implementers, 
USAID staff, or other donors – but in practice, it almost always based a simply a few 
meetings with the host country partner organization to make a peripheral determination 
of the major capacity needs. 
At the core of the Agency’s HICD model are certain elements of principled performance 
improvement: direct alignment of partner organization objectives to USAID objectives 
for the sector; a completely systematic and systemic assessment of gaps in institutional 
performance; performance solutions (a.k.a. – interventions) designed to address the 
human performance factor that is most closely associated with the core causes of the 
gaps; and an institutionalized performance monitoring system that allows host country 
partners to manage their own performance on an ongoing basis (while also providing 
USAID with much needed hard data regarding its development assistance efforts). The 
evaluation team approached the term “partial-HICD” with an open-mind and sought to 
identify the core HICD elements in the partial-HICD initiatives implemented under the 
current FORECAST program. These elements could not be identified at a level that 
would suggest that these activities are even partially HICD. The so-called partial-HICD 
initiatives are, in fact, simply discrete requests for packages of participant training 
and/or related technical assistance – traditional participant training programming as it 
has been done by the Agency worldwide for many years. This finding was corroborated 
by both FORECAST implementer staff and certain Mission staff. The participant 
Evaluation of the USAID/Georgia FORECAST Program 23
training component of the current FORECAST program is highly valued by the Mission 
and therefore any potential follow-on program should be designed to acknowledge the 
participant training component of the program, and the term “partial-HICD” should no 
longer be used in order to mitigate confusion. 
Performance Monitoring Plans 
The development of Performance Monitoring Plans (PMP) is a standard practice of the 
current FORECAST Program. A desk review of the PMPs identified issues with the 
manner in which the PMPs are developed, the information itself that is collected in 
PMPs, the manner of information collection, and the way the information is being used.
First, each PMP is formulated by each intervention or solution provider, and each 
provider views the purpose, usefulness, design of their PMP differently, resulting in 
mismatched, sometimes significantly different, PMPs for the same package of 
interventions for the same host country partner organization. FORECAST PMPs attempt 
to collect information regarding outcomes of specific interventions or solutions, rather 
than collecting information on outputs directly related to an established KPI over time in 
correlation with the implementation of an intervention or solutions package. Even in 
cases of full HICD initiatives, such as NAEC and NCEQE, organizational performance 
monitoring was not explicitly tied into the PMPs that were used both at the 
organizational level and at the activity level. The PMP data is collected by the 
FORECAST implementer, mostly via telephone calls and emails to the host country 
beneficiaries, but not used as an internal tool by the beneficiaries themselves. Finally, 
the PMPs themselves are delivered to the USAID program office as a contract 
deliverable, but stakeholder feedback suggests that the PMPs are not widely used by 
Mission staff for Agency Reporting purposes. 
USAID/Georgia is encouraged to discontinue to practice of having the implementer 
produce such PMPs, especially under the HICD component of the program. Rather, 
value for USAID would be realized in assisting host country partner institutions to 
establish performance monitoring and management systems, resulting in performance 
data that informs cyclical processes of continuous performance improvement within the 
partner institution. It is also this performance data collected within our host country 
partner organizations that would be truly useful to USAID for Agency reporting 
purposes. Also, mindful of the Agency’s focus toward use of host country systems, 
actual performance data from partner organizations could better inform decisions 
related to pursuing direct host country organization contracts.
Implementer Staff Caseload and Documentation Requirements 
As implementer staff continue to provide HICD support to existing and new partner 
organizations, the Mission should be mindful of the need to find a quality-assurance 
balance between the number of institutions and implementer staff caseload. This issue 
of implementer workload is interrelated with an ongoing issue of excessive 
documentation and Mission approval requirements under the current FORECAST 
program. 
Evaluation of the USAID/Georgia FORECAST Program 24
At higher management and implementation levels of the program, the current 
FORECAST contract is required to develop an operations manual detailing how its staff 
will conduct all aspects of program implementation. The implementer is also responsible 
for maintaining an HICD/Participant Training Plan detailing the technical activities of the 
program. It is the operations plan and HICD/Participant Training Plan, together, that 
have traditionally served as the implementer’s work plan for cross-cutting programs 
such as FORECAST. In recent years, the Mission has requested that the implementer 
also produce another work plan, akin to those used by technical assistance 
implementers, resulting in yet another document that must be produced by the 
implementer. The development of this extra work plan is not part of the original design 
of the FORECAST program and it is not a requirement of any other FORECAST 
program elsewhere. In order to reduce excessive documentation requirements, and in 
order to ensure program flexibility, USAID/Georgia should consider in the design of a 
follow-on cross-cutting program, the exclusive use of HICD Solutions Packages and a 
flexible Training/T.A. Plan, and discontinuing the practice of requiring an extra work 
plan. 
Reporting, documentation, and Mission approval requirements of the current 
FORECAST program are described as excessive by both USAID and the implementer. 
Implementer program staff reported spending much of their day cutting and pasting 
information from one report or form to the next. These multi-layered, overlapping, and 
somewhat redundant requirements include weekly, monthly, quarterly, and annual 
reports on program implementation and results. A series of forms that are designed to 
provide both USAID and its implementer with technical design and implementation tools 
are observed in practice to be overwhelming and sometimes redundant in the context of 
the FORECAST program in Georgia. These forms include intervention concept forms, 
intervention request forms, request for provider proposals templates, intervention 
implementation plans, budget worksheets, and more. Four of these forms require 
Mission approval, involving Mission program and/or technical office staff attention each 
time for the same intervention. The Mission is encouraged to ensure that the design 
process of a follow-on cross-cutting program includes a thorough review involving all 
Mission stakeholders of the Mission’s practical needs in terms of project reporting and 
management approvals. 
Accessible Performance Improvement Expertise 
A core component of the Agency’s HICD model is the organizational performance 
assessment (PA) which is intended to be conducted by certified performance 
improvement experts. 
Local Certified Experts 
The design of the current FORECAST program to build local capacity to carry-out HICD 
initiatives, including the PA, has been found by the evaluation team to be a forward-
Evaluation of the USAID/Georgia FORECAST Program 25
edge and successful approach that could stand as a model for the Agency in building 
the local capacity of performance improvement expertise. 
More so than any other HICD activities to date, FORECAST/Georgia utilized the 
services of local consultants and organizations in conducting performance assessments 
and in implementing solutions. This was done through the issuance of sub-IQCs by 
FORECAST/Georgia to local business service providers. FORECAST/Georgia 
organized training sessions on HICD and performance improvement specifically for 
these local partners. The first sessions were held before the beginning of any HICD 
activities. This paved the way for the first two years of the program, during which, 
according the implementer, “FORECAST IQC …local partners had a secondary role in 
providing support functions to the international performance improvement experts, while 
learning the HICD methodology through coaching and mentoring that was provided on￾site.”6
 The next phase beginning now is for the local partner to take the lead in 
conducting performance assessments under the oversight international performance 
improvement experts. This is the exact type of dual-capacity-development approach 
that the evaluation team commends the FORECAST/Georgia program for taking. Not 
only are the capacities of the targeted host country organizations strengthened, but the 
capacities of the local implementer partners are strengthened. 
Consideration should also be given to including the following components in the design 
of the follow-on cross-cutting program: 
 require current (category I) local partners to obtain CPT or HPI certification; 
 reduce the need for limited competitions amongst sub-IQC holders, especially if 
successful relationships with host country partner organizations have already 
been established; and 
 continue to seek to cultivate new rounds of additional of local business service 
providers in providing HPT services. 
In-House Certified Experts 
Another recommendation is for the Mission to consider designing the follow-on program 
to include a requirement that implementer Chief of Party and Program Managers be 
certified HPT experts, and participate in PAs and other roles of certified experts. This 
would also help address this issue of lag-time from the performance assessment phase 
to the performance solution implementation stage by allowing staff to better facilitate the 
transition (see Perform Performance Assessment section of this report). 
HICD Know-How Transfer 
Sustainable institutional capacity building requires a continuous cyclical process of 
performance improvement. It is not realistic for an HICD initiative to run through one set 
of performance improvement cycles (as contained in the original PA and intervention 
package), then withdraw HICD support from the institution and expect most of the 
 
6
 World Learning. "FORECAST Caucasus Fiscal Year 2010 Semi-Annual Report." 2010. 
Evaluation of the USAID/Georgia FORECAST Program 26
performance gains to be sustainable. Ultimately, HICD initiatives will be successful over 
the long-term if the partner organizations can integrate HICD approaches into their 
operations and culture. This must be a principal goal of HICD practitioners. One way 
for USAID to help its partner organizations achieve this is to include in the design of the 
follow-on cross-cutting program the need for the program implementer to transfer HICD 
know-how to key performers within each host country partner institution or to an 
established staff person to serve as the institution’s resident HICD coordinator to carry 
out HPT with a continuous performance monitoring system (scorecard), reassessing 
gaps, RCA, and other component. Most western organizations utilizing performance 
improvement implement the methodologies with in-house staff supported only by 
outside experts. This should be the end goal for the HICD component of the follow-on 
cross-cutting program in Georgia. The design of a follow-on program should 
purposefully include: 
 Transfer of HICD know-how from program implementer staff to partner 
organization staff, with updated job descriptions to allow for ongoing utilization of 
these skills at partner institutions; 
 Ongoing HICD coaching; and 
 Membership in HPT/HPI and capacity development professional organizations 
(e.g., International Society for Performance Improvement or American Society for 
Training and Development). 
GENERAL CONCLUSIONS 
USAID/Georgia’s FORECAST program can be accurately characterized as one 
program with two very distinct components. USAID/Georgia continues to place high 
value on FORECAST as a flexible mechanism that allows Mission management and 
technical offices to request discrete training interventions and associated technical 
assistance, as well as HICD, to better address the pursuit of sustainable institutional 
performance with its host country partner institutions. The impact of the FORECAST 
program is far reaching, across the Mission’s entire portfolio, providing valued outcomes 
to partner organizations in meeting institutional objectives, as well as to USAID Georgia 
in meeting its strategic objectives in economic growth, democracy and governance, 
energy and environment, health, and social transition. The findings and 
recommendations conveyed in this evaluation report are provided with a spirit of 
appreciation and admiration. EGAT/ED remains a full and willing partner in supporting 
future USAID/Georgia HICD and participant training endeavors. 
Evaluation of the FORECAST Program in Georgia 27
APPENDICES 
 A. ORIGINAL SCOPE OF WORK 
 B. THE HICD PROCESS 
 C. MEETINGS HELD 
 D. INSTITUTIONS STUDIED 
 E. DOCUMENTS REVIEWED 
Evaluation of the FORECAST Program in Georgia 28
APPENDIX A. ORIGINAL SCOPE OF WORK 
STATEMENT OF WORK 
Evaluation of USAID’S FORECAST Program in Georgia 
 
Summary 
USAID/Caucasus-Georgia seeks to carry out an evaluation of the full and/or partial7
 human and 
institutional capacity development (HICD) interventions implemented under the FORECAST activity in 
Georgia since September 2007. The current FORECAST activity ends in May 2011. The mission may 
initiate a follow-on HICD activity. Therefore, the evaluation will help the mission (a) take stock of the 
effectiveness and relevance of the current activity (b); make needed improvements in the management 
and substance of interventions; and (c) make an informed decision about the continuation of the future 
activity starting following May 2011. The main components of the evaluation will be as follows: 
1. Evaluation of progress in relation to the initial project objectives made in 3 selected target institutions 
that FORECAST has provided HICD and partial-HICD support for. 
2. Evaluation of the relevance of FORECAST methodology of providing HICD services and interventions 
and monitoring their effectiveness (including the FORECAST PMP) in relation to the project objectives. 
3. Recommendations on improving the management and substance of future HICD and/or partial-HICD 
interventions deriving from the above. 
Background 
Focus on Results: Enhancing Capacity Across Sectors in Transition (FORECAST) is designed by USAID 
to promote the usage of HICD in the E&E region in order to complement E&E’s on-going development 
assistance activities by helping USAID partner organizations to integrate performance improvement (PI) 
technologies into their organizational processes. 
The overall development hypothesis is as follows: an institution cannot be effective and efficient unless 
its performance is driven by (a) a skilled workforce, and (b) an effective organizational structure 
(including well-designed systems, policies, work processes and procedures). Human capacity 
development (i.e.: training) is only one aspect of an intervention aimed at PI of an institution. Other 
aspects aim to close identified gaps in organizational structures that cause a misuse of an institution’s 
human and material resources. It is believed that only the combination of the two approaches is most 
likely to lead to tangible improvements in the overall performance of an institution.
In Georgia, FORECAST (preceded by the START program implemented in 2002-2006 years) is 
implemented since September 2007. It complements the implementation of USAID/Caucasus-Georgia’s 
assistance objectives (AO) in democracy and governance, economic growth, energy and environment, and 
health and social development. This is achieved through the development of individual skills and 
institutional strengthening of select public, private and civil society organizations (CSO) that play a 
 
7 performance improvement interventions with which the mission responds to the specific needs of institutions that have already 
carried out a performance assessment.
Evaluation of the FORECAST Program in Georgia 29
strategic role in the implementation of the Georgian government’s reform program and the mission’s 
AOs. Furthermore, it addresses performance gaps within select organizations that have yet to overcome 
internal management, organization and skills barriers to effective and efficient implementation. Thus, 
FORECAST employs a holistic approach of improving the performance of institutions from a point of 
view of organizational structure and processes, as well as individual employees. 
The current government is strongly committed to human and institutional capacity building and has 
requested such assistance in several areas including vocational and higher education, health reform and 
hospital administration, rule-of-law, local government, public administration, and business climate reform 
to mention a few. USAID has successfully responded to many requests in these areas since 2007. The 
majority of entities FORECAST has worked with have been public institutions. 
Under its Task Order FORECAST is mandated to provide a comprehensive package of HICD and non￾HICD services and interventions, including, but not limited to, the following: 
Services 
 Ongoing expert consultation services related to the Mission’s overall HICD efforts across its 
portfolio; 
 HICD planning and coordination on the development of annual work plans in which organizations are 
identified to receive HICD interventions and individuals selected for training, mission HICD plans 
and assistance to technical office teams to develop or revise individual HICD plans, as requested, and 
partnering with extended AO Teams and stakeholders; 
 Procurement of training and non-training interventions and logistics providers; 
 Intervention implementation services: participant recruitment and processing, pre-departure 
orientation, and participants tracking, reporting and monitoring; 
 Program monitoring, evaluation and follow-up; 
 Participant training processing services (US and third-country). 
Interventions 
 Performance assessments (PA); 
o Organizational performance 
o Individual or workgroup performance
 Participant training/Academic education 
 Technical assistance (TA) 
 Small grants 
 Local capacity building 
Intended use and objectives of evaluation 
The intended use of the evaluation is to inform the mission’s decision regarding the continuation of 
FORECAST beyond May 2011. The evaluation is, therefore, expected to provide the mission with a rapid 
assessment of the effectiveness and relevance of the mission’s HICD methodology, lessons learned from 
the experiences of previous years and insights into necessary adjustments to improve the efficiency and 
effectiveness of HICD interventions in future. Given the time and resource constraints the evaluation will 
not examine all interventions to date. Nor will it focus on the participant training aspect of the program. 
Rather, it will engage with a selected number of institutions (at least three), for which FORECAST 
assistance has been perceived (by the mission, the contractor and the target institution) as either 
successful or unsuccessful. The list of institutions will be provided by the mission. 
Evaluation of the FORECAST Program in Georgia 30
The objectives of the evaluation are as follows: 
1. Evaluation of progress in relation to the initial project objectives made in 3 selected target institutions 
that FORECAST has provided HICD and partial-HICD support for. 
2. Evaluation of the relevance of FORECAST methodology of providing HICD services and interventions 
and monitoring their effectiveness (including the FORECAST PMP) in relation to the project objectives. 
3. Recommendations on improving the management and substance of future HICD and partial-HICD 
interventions deriving from the above. 
Tasks 
1. Evaluation of progress in relation to the initial project objectives made in 3 selected target 
institutions that FORECAST has provided HICD and partial-HICD support for. 
 Describe the present level of institutional performance of the target institutions: 
o Describe an overview of progress made in relation to the original objectives of respective 
intervention packages in 3 selected target institutions; 
o Provide an overall overview of the degree to which the target institutions use the newly 
developed and established work processes, systems, structures and policies. 
 Where appropriate, identify any barriers that have hindered the target institutions from achieving the 
initial objectives; 
 Indicate the key internal and external factors responsible for either success or failure with a 
description of respective lessons to be learnt; 
 Assess the degree and likelihood of sustainability of FORECAST assistance with a view to the 
performance of the institution following the completion of FORECAST support. 
2. Evaluation of the relevance of FORECAST methodology of providing HICD services and 
interventions and monitoring their effectiveness (including the FORECAST PMP) in relation to the 
project objectives. 
Areas to explore: 
 Assess the relevance of FORECAST assistance in relation to observed progress as captured 
under objective 1; 
 Review the extent to which the HICD services, interventions and methodology are perceived 
as effective by the target institutions; 
 Review the level of effectiveness and relevance of the current administrative procedures, as 
well as the existing institutional performance measurement system for the program (the 
FORECAST PMP); 
 Identify the aspects of the administrative procedures used that are unduly labor-intensive or 
time-consuming and, therefore, least effective in terms of achieving the intended objectives; 
Evaluation of the FORECAST Program in Georgia 31
 Review the extent to which FORECAST has ascertained an appropriate balance between the 
roles of international and local consultants in relation to the progress achieved; 
3. Recommendations on improving the management and substance of future HICD and 
partial-HICD interventions deriving from the above. 
Areas to explore: 
 Identify those critical factors of success or failure of the evaluated interventions (institutional, 
individual, financial, etc.) that need to be taken into account in future programs; 
 Indicate how some of the best practices established in FORECAST’s successful interventions 
can be drawn on in future programs; 
 Identify main areas for improving the currently used HICD services, methodology and tools 
and provide respective recommendations; 
 Provide recommendations for a more practical of the PMP as an effective performance 
measurement system; 
 Provide ideas for ensuring sustainability of HICD interventions; 
 Indicate any possible risk factors or critical assumptions to be taken into account in future 
interventions. 
Methodology 
The evaluators, in collaboration with USAID/Caucasus-Georgia, will finalize the overall evaluation 
methodology prior to their arrival in Georgia (before June 7, 2010). Given the nature of FORECAST 
activity most of the data produced could be qualitative but the evaluators will be encouraged to provide 
quantitative data where possible. A standard questionnaire will be developed by the evaluation experts 
prior to the start of the fieldwork. It will be used for all institutions evaluated. In addition, the evaluators 
will draw on open-ended qualitative questions specific to each institution’s experience. 
The mission expects that, at a minimum, the evaluators will: 
- Review and analyze the relevant project documentation (IRFs, IIPs, monitoring reports, weekly 
activity reports, annual reports). These will be provided in advance to the start of the fieldwork as 
part of the preparation stage; 
- Conduct workplace visits; 
- Meet and interview a) the supervisors and key management staff, b) customers and clients, and c) 
other stakeholders (donor agencies, CSOs, media, etc.) as deemed relevant; 
- Meet and consult relevant mission teams; 
- Meet local service providers. 
A list of target institutions to be evaluated will be provided at a later stage. Given the time constraints it is 
proposed to evaluate only three interventions. They will be selected as per the mission’s perception of 
their effectiveness (i.e. a successful intervention, an unsuccessful intervention and an intervention with no 
ostensibly negative or positive outcomes). 
Evaluation of the FORECAST Program in Georgia 32
The evaluators will also work with the mission to develop a list of illustrative questions referring 
to respective areas for exploration under the TASKS section. 
Logistics/Translator/Administrative Staff: 
Logistics will be set up in advance of the field work. Meetings will be set up in advance of the 
arrival of the evaluators. One interpreter will be required to accompany the evaluator, arrange 
meetings, organize transportation, and interpret as necessary. 
Deliverables 
The final report including: 
An Executive Summary – (2-3 pages) containing a clear, concise summary of the most critical 
elements of the report, including the recommendations. 
Table of contents. 
An evaluation report (no more than 15 pages) including major findings and related issues and 
questions. 
Appendices (the statement of work; the list of documents reviewed; the list of meetings held). 
Evaluation of the FORECAST Program in Georgia 33
Identify Partner Organizations
Obtain Partner Commitment
Form Stakeholder Group
Conduct Performance Assessment
Prepare Performance Solutions Package
Implement Performance Solutions
Monitor Change in Performance
APPENDIX B. THE HICD PROCESS 
The HICD Process 
Human and Institutional Capacity Development 
(HICD) encompasses several methodological 
approaches. The process includes components to 
place performance in the context of transitioning 
country, the Performance Improvement model, and 
performance monitoring (ex: the Balanced 
Scorecard model); other models and methods of 
improving institutional performance improvement 
can also be incorporated as supplementary tools to 
arrive at the best improved performance. 
A major HICD framework is based on the 
Performance Improvement (PI) model developed by ISPI, which provides a systematic 
process for analyzing and improving performance. The HICD process allows Missions 
to focus on strengthening partner organizations that are key to achieving the Mission’s 
strategic goals and assisting these organizations to integrate/institutionalize a 
continuous cycle of performance improvement processes to ensure continued success 
in future performance cycles. The process is depicted in the figure to the right. 
Updated Behavioral Engineering Model & Performance Improvement (PI) Model 
The Updated Behavioral Engineering Model was developed by ISPI in order to identify 
and categorize the factors that affect organizational performance. It consists of six 
factors that are divided into individual and environmental factors. 
CONSIDER INSTITUTIONAL
CONTEXT
Mission
Goals
Strategies
Culture
Client and Community
Perspectives
DEFINE DESIRED
PERFORMANCE
DESCRIBE ACTUAL
PERFORMANCE
MONITOR AND EVALUATE PERFORMANCE
OBTAIN AND MAINTAIN STAKEHOLDER AGREEMENT
FIND ROOT CAUSES
Why does the
performance gap exist?
SELECT PERFORMANCE
SOLUTIONS
What can be done to
close the performance
gap?
IMPLEMENT
PERFORMANCE
SOLUTIONS
What solutions match
root causes?
GAP
Evaluation of the FORECAST Program in Georgia 34
The ISPI Performance Improvement model is a systematic process that involves eight 
distinct steps for analyzing and improving performance: 
1. Consider the institutional context of the performance problem and get 
stakeholder agreement. 
2. Define desired performance in measurable terms if possible. 
3. Describe actual performance. 
4. Measure or describe performance gap. 
5. Find the root causes of the performance gap. 
6. Select intervention. 
7. Implement performance solutions (interventions). 
8. Monitor and evaluate performance. 
The PI model allows for a comprehensive analysis of organizational performance based 
on the performance factors identified in the Updated Behavioral Engineering Model. 
Through use of the model, appropriate performance solutions can be designed 
depending on which of the six performance factors lie at the root causes of performance 
gaps. 
Performance Management Systems 
A hallmark of HICD is its results-oriented nature. In order to gauge the success of an 
HICD project, it is important to establish measurable performance goals for each area of 
Updated Behavior Engineering Model ENVIRONMENTAL 
Information 
 Roles and performance 
expectations are clearly defined; 
employees are given relevant and 
frequent feedback about the 
adequacy of performance. 
 Clear and relevant guides are 
used to describe the work 
process. 
 The performance management 
system guides employee 
performance and development. 
Resources and Tools 
 Materials, tools, expert support, 
and time needed to do the job are 
present. 
 Processes and procedures are 
clearly defined in reference 
documentation. 
 Overall physical and 
psychological work environment 
contributes to improved 
performance; work conditions are 
safe, clean, organized, and 
conducive to performance. 
Incentives 
 Financial and non-financial 
incentives are present; 
measurement and reward 
systems reinforce positive 
performance. 
 Jobs are enriched to allow for 
fulfillment of employee needs. 
 Overall work environment is 
positive, where employees 
believe they have an opportunity 
to succeed; career development 
opportunities are present. 
INDIVIDUAL 
Knowledge and Skills 
 Employees have the necessary 
knowledge, experience and skills 
to do the desired behaviors. 
 Employees with the necessary 
knowledge, experience and skills 
are properly placed to use and 
share what they know. 
 Employees are cross-trained to 
understand each other’s roles. 
Capacity 
 Employees have the individual 
capacity to learn and do what is 
needed to perform successfully. 
 Employees are recruited and 
selected to match the realities of 
the work situation. 
 Employees are free of emotional 
limitations that would interfere 
with their performance. 
Motives 
 Motives of employees are aligned 
with the work and the work 
environment. 
 Employees desire to perform the 
required jobs. 
 Employees are recruited and 
selected to match the realities of 
the work situation. 
Evaluation of the FORECAST Program in Georgia 35
performance targeted through the HICD initiative. If a baseline of performance data 
does not already exist, the HICD contractor should work with the partner organization, 
as part of the HICD initiative, to establish some form of performance scorecard. 
Institutionalizing a performance scorecard enables an organization to regularly monitor 
performance in critical areas that are leading indicators of the organization’s overall 
performance success. A comprehensive scorecard establishes performance measures 
at every level of performance: individual worker level, process level, and organizational 
level with indicators at each level rolling up to the next highest level and ultimately 
providing a consolidated performance report card for senior managers to use in 
planning strategies and allocating the organization’s resources to meet needs with the 
highest priority. 
Policy 
It is USAID policy to integrate a HICD approach when conducting regional, country, or 
sector strategic planning and when the designing new developmental activities. 
Regional and pillar bureaus must address the integration of HICD when approving 
USAID-specific regional, country, or sector strategic plans. 
For more information please refer to the HICD Policy Paper, a mandatory reference to 
Agency policy ADS chapter 201. 
Mechanisms 
Offered through EGAT/ED, Focus On Results: Enhancing Capacity Across Sectors In 
Transition (FORECAST) consists of two 5-year IQC contracts offering a fully integrated, 
flexible package of cross-cutting services for HICD, including the next generation of 
Participant Training (PT). FORECAST is implemented by The Academy for Educational 
Development and World Learning for International Development. USAID/Georgia’s 
HICD activities are implemented by World Learning (WL). 
One way to achieve sustainability in its programs is for USAID to increasingly focus its 
development assistance efforts on strengthening the abilities of its host country partner 
organizations that are in a position to effect change or have a central role in delivering 
commodities and services to the actual host country end users. FORECAST reflects 
the evolution of traditional participant training into a more holistic capacity development 
approach by providing a full range of HICD services focused on providing targeted 
interventions to address performance gaps within USAID partner organizations. 
Implementation Guidance 
The HICD chapter of the FORECAST Practitioner’s Handbook was developed by the 
Bureau for Economic Growth, Agriculture and Trade (EGAT) to help USAID integrate 
HICD into its development assistance programs. It provides information on the 
Evaluation of the FORECAST Program in Georgia 36
evolution of HICD, implementation guidance, and tools for implementing HICD 
initiatives. 
HICD can be integrated into USAID programs at any point in a Mission’s strategic 
planning cycle. For Missions at the beginning of the strategic planning cycle, a Cross￾Sector Assessment may be used to integrate HICD into the Mission strategy, programs, 
and activities. If introduced mid-cycle, HICD may be used to complement ongoing 
initiatives or provide support to partners not already directly benefiting from other 
Mission activities. 
Evaluation of the FORECAST Program in Georgia 37
APPENDIX C. MEETINGS HELD 
Evaluation of USAID/Georgia FORECAST Program 
Evaluation Team Schedule 
Week of October 25 – 30 
Monday 
10.00-12.00: Evaluation Team meeting at the Radisson Hotel
13.00-14.00: Evaluation Team meeting: with the USAID Program office
14.00-14.30: Evaluation Team meeting: with the USAID Mission Director and Deputy Mission Director.
15.00 onwards: Meetings at WL
 World Learning Office Staff Group Meeting -- Jeff/Ivica/George
 World Learning Country Director -- Jeff
 World Learning Deputy Director for Programming -- Ivica/George
Tuesday 
9:00AM-9:45AM: Evaluation Team Group Meeting (Radisson) 
 World Learning Program Officers 
Time Evaluators Institution Program Staff Comments 
10:00AM￾11:00AM 
(Ivica/George) TSU Eka 
Leonidze/Roman 
Tsustkiridze 
The program staff 
assigned for this 
program has left WL 
Jeff/David GHA Sopo 
Motsonelidze 
11:45AM￾12:45PM 
(Ivica/George) NEAC Tako 
Toroshelidze / 
Roman 
Tsutskiridze 
The program staff 
assigned for this 
program has left WL
Jeff/David MEPNR Eka Leonidze FORECAST staff 
working on PA has left
2:00PM￾2:45PM 
(Ivica/George) HIMS Sopo 
Motosonelidze 
(Jeff/David) NAEC Eka Leonidze / 
Roman 
The program staff 
assigned for this 
Evaluation of the FORECAST Program in Georgia 38
Tsutskiridze program has left WL
3:30PM￾4:30PM 
(Ivica/George) MRA Eka 
Leonidze/Roman 
Tsustkiridze 
The program staff 
assigned for this 
program has left WL
Jeff/David ESIDA Sopo 
Motsonelidze 
Wednesday 
9:00AM-9:45AM: Evaluation Team Group Meeting (Radisson) 
 World Learning Program Officers (continued from Tuesday) 
Time Evaluators Institution Program Staff Comments 
10:00AM￾11:00AM 
 
Jeff/David ISET Tako 
Toroshelidze 
/Roman 
Tsutskiridze 
The program staff 
assigned for this program 
has left WL
11:45AM￾12:45PM 
(Ivica/George) NCDC Roman 
Tsutskiridze 
Jeff/David Service 
Agency 
Eka Leonidze 
2:00PM￾2:45PM 
(Ivica/George) GAC Roman 
Tsutskiridze / 
Dea Pagava 
 
3:30PM￾4:30PM 
(Ivica/George) NBG Roman 
Tsutskiridze 
 
Thursday
9:00AM-9:45AM: Evaluation Team Group Meeting (Radisson) 
 World Learning Program Officers (continued from Tuesday and Wednesday) 
Time Evaluators Institution Program Staff Comments 
10:00AM￾11:00AM 
Jeff/David KCUH Roman 
Tsutskiridze 
 
11:45AM￾12:45PM 
Jeff/David MSPA&MFOA Roman 
Tsutskiridze 
 
Evaluation of the FORECAST Program in Georgia 39
 Focus Institution meetings 
Time Evaluation 
Team 
Institution Meeting with Comments 
11:00-
12:00 
(Ivica/George) NCEQ David 
Kereselidze 
Address: 1, Merab Aleksidze 
Str. 
16:00 
– 
17:00 
(Ivica/George) MRA Valeri 
Kopaleishvili 
Address: 15a, Tamarashvili 
Street 
Friday 
9:00AM-9:45AM: Evaluation Team Group Meeting (Radisson) 
10:00AM-11:00AM: WL Program Officers, re: paperwork/reporting -- Jeff
 Focus Institution meetings (continued from Friday) 
Time Evaluation 
Team 
Institution Meeting with Comments 
10:00 
– 
11:00 
(Ivica/George) MRA Paul Mackintosh 
from DRC 
Address: 15a, Tamarashvili 
Street 
13:00-
14:00 
(Ivica/George) MRA UNHCR TBD 
 
14.00-15.00 Meeting with Joakim Parker, Deputy Mission Director -- Jeff/David 
15:00-16:00 Mid-Evaluation Meeting with Program Office 
16:00-16:30 Evaluation Team Group Meeting (at Mission) 
Saturday 
 Evaluation Team Work 
o report outline finalized 
o desk review assignments on schedule 
o report components in draft 
Week of November 1 – 5 
Monday 
Evaluation of the FORECAST Program in Georgia 40
 Focus Institution meetings (continued from Friday) 
Time Evaluation 
Team 
Institution Meeting with Comments 
10:00 -
11:00 
Jeff/David ISET Eric Livney Address: 16, Zandukeli St. 
 
12.45 – 13.30: Evaluation Team Group Meeting (Mission) 
13.30 - 15.00: Meeting with Medea Kakachia (HSD), focal point for TSU, NEAC, NAEC, 
ESIDA and Service Agency: 
 
16.00 - 17.00 Meeting with George Khechinashvili (HSD) focal point for HMS, GHA and 
KCUH 
Tuesday 
 Focus Institution meetings (continued from Friday and Monday) 
Time Evaluation 
Team 
Institution Meeting with Comments 
10:00-
11:00 
Jeff/David/Ivica IQC 
Category I 
Local 
Partners 
UNA-Georgia Address: Dolidze 2, 
floor 4 
12:00-
13:00 
Jeff/David/Ivica IQC 
Category I 
Local 
Partners 
Partners Georgia Address: Paliashvili 
11b 
14:00-
15:00
Jeff/David NAEC Giorgi Martashvili (899 28 
90 44) 
Address: 60, Iv. 
Javkhishvili Street 
Wednesday 
 Other donors and stakeholders 
Time Evaluation 
Team 
Institution Meeting with Comments 
11:00AM￾12:00PM 
Jeff/David/Ivica IQC 
Category I 
Local 
Partners 
ARC Address: Dolidze 2, 
floor 4 (interpreter 
needed)
Thursday 
Evaluation of the FORECAST Program in Georgia 41
9.30 - 10.30: Meeting with EG team (Douglas Balko, Revaz Ormotsadze and Nino Metreveli), to discuss 
GAC, ISET, Bank of Georgia projects and also future synergies between EG and FORECAST on some 
key projects (EPI, NEO)
 
11.00 - 12.00: Meeting with DG team (Michelle Logsdon, George Vashakidze and Khatuna Khvichia) to 
discuss MSPA/MOFA and also future synergies with DG projects (e.g. G3) 
14.00 - 15.00: Meeting with Tamara Sirbiladze (HSD) to discuss the NCDC project 
*15.00-15.45: Meeting with Jill Kelley, Director of Energy and Environment (E&E) Office 
Friday 
15.30-16.30: Mission-wide debrief – Jeff/Ivica/George/Davi
Evaluation of the FORECAST Program in Georgia 42
APPENDIX D. INSTITUTIONS STUDIED 
FORECAST FULL HICD PROGRAMS 
Tbilisi State University: The goal of the Tbilisi State University (TSU) HICD program was to 
contribute to TSU’s transition to an independent, self-financed and sustainable institution 
through the establishment of decentralized administrative and management structures and the 
development of a business plan for the university. The program commenced in July 2008 with 
performance assessment activities carried out by a team of international performance assessment 
consultants, with the support of the local field of activity expert and three representatives from 
the local FORECAST partner consortium. 
National Education Accreditation Center: The goal of the National Education Accreditation 
Center (NEAC) HICD program was to improve the overall performance of the center in meeting 
its short- and medium-term goals in further developing the general education accreditation 
system and service delivery (training schools in self-assessment and training of external 
evaluators in evaluation methodologies). The performance assessment of NEAC was carried out 
in two phases (July and October, 2008) by the international performance improvement 
consultants together with the FORECAST local partner organization. 
National Assessment and Examination Center: The goal of the NAEC program was to 
improve the overall performance of the NAEC in meeting its short- and medium-term goals of 
further developing and rolling out the general graduate examination, and building their 
organizational capacities through the introduction of management tools and techniques that 
would allow the institution to better cope with its expanding organizational mandate and 
workload, including the development of clear job descriptions, a performance system, and 
professional development mechanisms for staff. 
Ministry of Environment Protection and Natural Resources: The performance assessment 
and TA activities at the Ministry of Environment Protection and Natural Resources (MEPNR) 
aim to define management and programming gaps, to provide technical assistance to strategically 
plan for and address those gaps, and to recommend an intervention package to address priority 
performance shortcomings/gaps. 
Municipal Service Providers Association and Municipal Finance Officers Association: The 
goal of the MSPA & MFOA HICD program is to improve the overall performance of the 
associations in meeting their short- and medium-term goals. The first phase of the program, 
assessment and the technical assistance, aimed to define management and programming gaps at 
the MSPA and MFOA, to provide technical assistance to strategically plan for and address those 
gaps, and to recommend an intervention package to address priority performance 
shortcomings/gaps. 
Evaluation of the FORECAST Program in Georgia 43
Ministry of Refugees and Accommodation: The long-term TA at MRA started in February 
2009 and one component of the program is still in progress. The overall goal of the program is 
contribute to building the capacities of the MRA in responding to the immediate needs of the 
IDP population from the recent conflict. 
National Center for Disease Control: The objective of the NCDC performance assessment and 
technical assistance was to define performance gaps at the center, and to provide technical 
assistance to strategically plan for and address these gaps. The activity was conducted by a team 
of international and local consultants in November 2009 and January 2010. Although it was 
initially planned that the performance assessment would be followed by the development of an 
intervention package based on the recommendations provided by the assessment provider, it has 
been decided to cancel all further activities related to this program. This recommendation came 
as a direct result of the performance assessment, followed by a serious lack of understanding and 
cooperation on the part of the target institution. According to the final assessment report, as well 
as FORECAST’s observation, NCDC, as currently managed, is indeed unwilling and unable to 
absorb any future FORECAST assistance. This is due to the leadership of the institution’s almost 
absolute reluctance to consider a) the potential that change may be needed at the institution, and 
b) even the most basic level of engagement with the FORECAST program over potential 
institutional development and capacity-building activities. 
Persons with Disabilities, Elders and Orphanages Service Agency at the Ministry of Labor, 
Health and Social Affairs: The overall goal of the Service Agency HICD program is to 
establish a well-functioning and well-known entity capable of engaging individuals, businesses, 
sponsor organizations and communities in protecting and assisting vulnerable children, and 
adults with special needs.
Georgian Hospital Association: The GHA HICD program aims to produce a strong strategy 
and a road map that will calibrate its strategic objectives with its tactical operations, the 
necessary funding levels and sources of funds, and the right organizational structure. Such 
strategy and the road map would create the basis on which the GHA programs will be built and 
executed over a foreseeable future. 
OUTSTANDING PARTIAL-HICD PROGRAMS AND TA’s 
Georgian Accreditation Center: The GAC partial-HICD program aimed to facilitate the pre￾conditions for the successful integration and recognition of a Georgian accreditation and 
(eventual) certification regime for good bound for international markets. 
National Bank of Georgia, HR Department: The goal of the program was to assist the 
National Bank of Georgia (NBG) to strengthen (and where appropriate develop) its HR 
management systems. The activity was conducted by a team of international and local experts 
Evaluation of the FORECAST Program in Georgia 44
from November 2008 to February 2009. As a direct result of the program, the NBG HR Policy 
Manual has been developed, together with the organizational HR matrix, job descriptions and 
other HR documents. The program is completed and closed out. 
Kipshidze Central University Hospital: The goal of the KCUH partial-HICD program was to 
create a standardized hospital employee recruitment and performance appraisal system for 
professional and technical staff. The program represented a two-stage activity, where the first 
one, implemented in May 2008, was an institutional assessment of the hospital, and the second – 
TA and training (March-April 2009), resulted in development of the medical staff bylaws, 
KCUH draft policies and procedures, the key position descriptions, and the sample performance 
appraisal framework. The program is completed and closed out 
International School of Economics: The goal of the partial-HICD program was to contribute to 
the growth and sustainability of ISET as a premiere economic institute of higher learning in the 
Caucasus. The program is completed and closed out. 
Health Insurance Mediation Service: The goal of the partial-HICD program is to support 
HIMS in achieving optimal operating efficiency for delivering mediation services and 
information to the population. 
Educational and Scientific Infrastructure Development Agency: The program aims to 
improve the general education infrastructure and ensure a more fair, transparent and fact-based 
decision making process in school construction and infrastructure rehabilitation in Georgia 
according to modern standards. 
Evaluation of the FORECAST Program in Georgia 45
 APPENDIX E. DOCUMENTS REVIEWED 
The DOCUMENT CATEGORY column refers to documents requested by the evaluation team.
The FILE NAMES column refers to documents received and reviewed by the evaluation team.
FORECAST Georgia
DOCUMENT CATEGORY FILE NAMES
Contractor Quarterly Reports FY 2008 FORECAST Quarterly Report 1
FY 2008 FORECAST Quarterly Report 2
FY 2009 FORECAST Quarterly Report 1
FY 2009 FORECAST Quarterly Report 2
FY 2010 FORECAST Quarterly Report 1
FY 2010 FORECAST Quarterly Report 2
Contractor Annual and Semi‐Annual Reports FY 2008 FORECAST Annual Report
FY 2008 FORECAST Semi‐Annual Report
FY 2009 FORECAST Annual Report
FY 2009 FORECAST Semi‐Annual Report
FY 2010 FORECAST Annual Report
FY 2010 FORECAST Semi‐Annual Report
Contractor Operations Manual (for FORECAST
local office)
FORECAST OPERATIONS MANUAL (revised 07‐25‐
08).doc
Other information or documentation FORECAST Activity‐Level PMP (4‐10).doc
Educational and Scientific Infrastructure Development Agency (ESIDA)
DOCUMENT CATEGORY FILE NAMES
Partner organization commitment
documentation, including memorandums of
understanding (MOUs)
1. MEMORANDUM OF UNDERSTANDING ESIDA.doc
Solutions (interventions) Package 3. IIP ESIDA Intervention I.doc
6. IIP ESIDA Intervention II.doc
7. ESIDA Intervention II Final Report.pdf
8. ESIDA Intervention II Final Report. Appendix A‐
G.pdf
12. IIP ESIDA Intervention III.doc
13. ESIDA Intervention III Final Report.pdf
Contractor procurement documentation for
outsourcing solutions (interventions)
2. IRF ESIDA.doc
5. RecMemo ESIDA Intervention II.doc
9. RFP ESIDA Intervention III (international
experts).doc
Evaluation of the FORECAST Program in Georgia 46
10. RFP ESIDA Intervention III (local experts).doc
11. RecMemo ESIDA Intervention III.doc
Georgian Hospital Association (GHA)
DOCUMENT CATEGORY FILE NAMES
Solutions (interventions) Package 5. GAC Component III Final Report.doc
6. GAC Component III Final Report. Annex I.DOC
7. GAC Component III Final Report. Annex II.doc
8. GAC Component III Final Report. Annex III.doc
9. GAC Component III Final Report. Annex IV.doc
Contractor procurement documentation for
outsourcing solutions (interventions)
1. IRF ‐ IP GAC Components I and II.doc
2. IRF GAC Component III.doc
3. RFP GAC Component III.doc
4. RecMemo GAC Component III.doc
Health Insurance Mediation Service (HIMS)
DOCUMENT CATEGORY FILE NAMES
Solutions (interventions) Package 4. IIP HIMS Intervention I.doc
5. IIP HIMS Intervention II.doc
Contractor procurement documentation for
outsourcing solutions (interventions)
1. IRF HIMS.doc
2. RFP HIMS Intervention I.doc
3. RecMemo  HIMS Intervention I.doc
International School of Economics at Tbilisi State University (ISET)
DOCUMENT CATEGORY FILE NAMES
Solutions (interventions) Package 4. IIP ISET Component I.doc
5. ISET Comp I Final Report.doc
6. ISET Comp I Final Report. ISET Business Plan.doc
7. ISET Comp I Final Report. Strategic Options for ISET
‐ A Discussion Document.doc
8. ISET Comp I Final Report. Strategic Plan.doc
9. ISET Comp I Final Report. Transition.doc
13. IIP ISET Comp II.doc
14. ISET Comp II Final Report.doc
15. IRF‐IP ISET Comp III Workshop on USAID Rules and
Regulations.doc
Evaluation of the FORECAST Program in Georgia 47
Contractor procurement documentation for
outsourcing solutions (interventions)
1. IRF ISET.doc
2. RFP ISET Comp I.doc
3. RecMemo ISET Comp I.doc
10. RFP ISET Comp II International Expert.doc
11. RFP ISET Comp II Local Expert.doc
12. RecMemo ISET Comp II.doc
15. IRF‐IP ISET Comp III Workshop on USAID Rules and
Regulations.doc
FORECAST PMP 16. PMP ‐ ISET Component I.xls
17. PMP ‐ ISET Component II.xls
Kipshidze Central University Hospital (KCUH)
DOCUMENT CATEGORY FILE NAMES
Contractor procurement documentation for
outsourcing performance assessment
1. PAR KCUH.doc
Performance Assessment (PA) report 2.  KCUH Assessment Final Report.doc
3. KCUH Assessment Final Report Presentation.ppt
Solutions (interventions) Package 7. IIP KCUH TA.doc
8. KCUH TA Final Report.doc
Contractor procurement documentation for
outsourcing solutions (interventions)
4. IRF KCUH TA.doc
5. RFP KCUH TA.doc
6. RecMemo KCUH TA.doc
Ministry of Environment Protection and Natural Resources (MEPNR)
DOCUMENT CATEGORY FILE NAMES
Partner organization commitment
documentation, including memorandums of
understanding (MOUs)
1. MOU MEPNR.doc
Stakeholder group formation documentation 2. Stakeholder Group MEPNR.doc
Contractor procurement documentation for
outsourcing performance assessment
3. PAR MEPNR.doc
4. RFP PA and TA MEPNR.doc
5. RecMemo PA and TA MEPNR.doc
Performance Assessment (PA) report 6. MEPNR Assessment Final Report.pdf
7. MEPNR Assessment Final Report. Work
Processes.doc
Solutions (interventions) Package 8. Intervention Package MEPNR.doc
10. IIP MEPNR Intervention I. Workshop.doc
9. MEPNR PA and TA Final Report. Appendix1.xls
11. MEPNR PA and TA Final Report. Appendix2.ppt
12. MEPNR PA and TA Final Report. Appendix3.ppt
Evaluation of the FORECAST Program in Georgia 48
Contractor procurement documentation for
outsourcing solutions (interventions)
13. RFP MEPNR Intervention II.doc
14. RFP MEPNR Intervention IV.doc
Ministry of Refugees and Accomodation (MRA)
Phase 1
DOCUMENT CATEGORY FILE NAMES
Partner organization commitment
documentation, including memorandums of
understanding (MOUs)
1. MEMORANDUM OF UNDERSTANDING ‐ MRA.doc
Contractor procurement documentation for
outsourcing performance assessment
2. IRF ‐ MRA Inst.Assessment and TA (Phase I).doc
3. TOR TA International TA Expert (Phase I).doc
4. TOR Local Project Officer (Phase I).doc
Performance Assessment (PA) report 6. MRA Phase I Final Report.doc
Solutions (interventions) Package 5. IIP MRA Inst.Assessment and TA (Phase I).doc
6. MRA Phase I Final Report.doc
Phase 2
DOCUMENT CATEGORY FILE NAMES
Solutions (interventions) Package 6. IIP MRA IT Strategy.doc
11. IIP MRA Communications Strategy.docx
12. Final Report MRA Communications Strategy.pdf
13. MRA Communications Strategy Attachment #1.pdf
14. MRA Communications Strategy Attachment #2.pdf
15. MRA Communications Strategy Attachment #3.pdf
19. IIP MRA HR Management.doc
20. Final Report MRA HR Management.doc
Contractor procurement documentation for
outsourcing solutions (interventions)
1. IRF‐IIP MRA TA II.doc
2. MRA TA II Final Report.doc
3. IRF MRA IT Strategy.doc
4. RFP MRA IT Strategy.doc
5. RecMemo MRA IT Strategy.doc
8. IRF MRA Communications Strategy.doc
9. RFP MRA Communications Strategy.doc
10. RecMemo MRA Communications Strategy.doc
16. IRF MRA HR Management.doc
17. RFP MRA HR Management.doc
18. RecMemo MRA HR Management.doc
FORECAST PMP 21. PMP ‐ MRA HR.doc
22. PMP ‐ MRA TA.doc
23. PMP ‐ MRA Communications Strategy.xls
Evaluation of the FORECAST Program in Georgia 49
24. PMP ‐ MRA Database.xls
Phase 3
DOCUMENT CATEGORY FILE NAMES
Solutions (interventions) Package 2. MRA III TA III Final report (Confidential for
USAID).doc
5. IIP MRA III Restructuring.doc
6. MRA III Restructuring Final Report.doc
7. IIP MRA III Legal Department.doc
10. IIP MRA III Legal Department.doc
11. MRA III Legal Department Final Report.doc
14. IIP MRA III Finance Department TA.doc
15. MRA III Finance Department TA Final Report.doc
16. IIP MRA III Finance Manegement Training.doc
Contractor procurement documentation for
outsourcing solutions (interventions)
1. IRF MRA TA III.doc
3. RFP MRA III Restructuring.doc
4. RecMemo MRA III Restructuring.doc
8. RFP MRA III Legal Department.doc
9. RecMemo MRA III Legal Department.doc
12. RFP MRA III Finance Department TA.doc
13. RecMemo MRA III Finance Department TA.doc
Institutional Performance Data
FORECAST PMP 17. PMP MRA III.doc
18. PMP MRA III Legal Department.doc
19. PMP MRA III Finance Department.doc
Municipal Service Providers Association  (MSPA) and Municipal Finance Officers Association (MFOA)
DOCUMENT CATEGORY FILE NAMES
Partner organization commitment
documentation, including memorandums of
understanding (MOUs)
1. MEMORANDUM OF UNDERSTANDING ‐ MFOA and
MSPA.doc
Contractor procurement documentation for
outsourcing performance assessment
2. PAR ‐ MFOA and MSPA.doc
3. RFP ‐ PA and TA at MSPA and MFOA.doc
4. RecMemo PA and TA ‐ MFOA  MSPA.doc
Performance Assessment (PA) report 5. MSPA and MFOA FocusGroups Report.doc
Evaluation of the FORECAST Program in Georgia 50
Solutions (interventions) Package 8. MSPA and MFOA Intervention Package ‐ IIP
Intervention I (MSPA).doc
9. MSPA and MFOA Intervention I (MSPA) Final
Report.doc
10. MSPA and MFOA Intervention Package
Intervention II (MFOA).doc
13. IIP Intervention II (MFOA).doc
14. MSPA and MFOA Intervention Package
Intervention III(MSPA).doc
17. IIP Intervention III (MSPA).doc
Contractor procurement documentation for
outsourcing solutions (interventions)
11. RFB Intervention II (MFOA).doc
12. RecMemo Intervention II (MFOA).doc
15. RFP Intervention III (MSPA).doc
16. RecMemo ‐ Intervention III (MSPA).doc
FORECAST PMP 18. PMP ‐ MSPA.doc
19. PMP ‐ MFOA.doc
National Assessment and Examination Center (NAEC)
DOCUMENT CATEGORY FILE NAMES
Partner organization commitment
documentation, including memorandums of
understanding (MOUs)
1. MEMORANDUM OF UNDERSTANDING NAEC.doc
Stakeholder group formation documentation 2. Stakeholder Group NAEC.doc
Contractor procurement documentation for
outsourcing performance assessment
3. PAR NAEC.doc
4. RFP NAEC PA.doc
5. RFP NAEC FoA.doc
6. RecMemo NAEC.doc
7. Revised Budget NAEC PA.doc
Performance Assessment (PA) report 8. NAEC PA Final Report.pdf
9. NAEC PA Final Report. Annex 4.pdf
10. NAEC PA Final Report. Annex 6.pdf
11. NAEC PA Final Report. Annex 11.pdf
12. NAEC PA Final Repor. All other annexes.pdf
Solutions (interventions) Package 13. Intervention Package NAEC.doc
14. IIP NAEC Intervention I.rtf
15. NAEC Intervention I Final Report.doc
FORECAST PMP 16. PMP ‐ NAEC.xls
National Bank of Georgia (NBG)
DOCUMENT CATEGORY FILE NAMES
Evaluation of the FORECAST Program in Georgia 51
Solutions (interventions) Package 4. IIP NBG.doc
5. NBG Final Report (International Service
Provider).doc
6. NBG Final Report (Local Service Provider).doc
Contractor procurement documentation for
outsourcing solutions (interventions)
1. IRF NBG.doc
2. RFP NBG.doc
3. RecMemo NBG.doc
National Center for Disease Control (NCDC)
DOCUMENT CATEGORY FILE NAMES
Partner organization commitment
documentation, including memorandums of
understanding (MOUs)
1. MEMORANDUM OF UNDERSTANDING NCDC.doc
Contractor procurement documentation for
outsourcing performance assessment
2. PAR NCDC.doc
3. RFP NCDC.doc
4. RecMemo PA NCDC.doc
Performance Assessment (PA) report 5. NCDC PA Final Report ‐ Cover.doc
6.  NCDC PA Final Report. Section 1. PMP.doc
7. NCDC PA Final Report. Section 2 ‐ Conclusions and
Recommendations.doc
8. NCDC PA Final Report. Section 3a ‐ NCDC Org
Chart.pdf
9. NCDC PA Final Report. Section 3b ‐ Organizational
Structure,  Focus Group Results.doc
10. NCDC PA Final Report. Section 3c ‐ NCDC Structure
modified.doc
Solutions (interventions) Package 6.  NCDC PA Final Report. Section 1. PMP.doc
7. NCDC PA Final Report. Section 2 ‐ Conclusions and
Recommendations.doc
National Education Accreditation Center
DOCUMENT CATEGORY FILE NAMES
Partner organization commitment
documentation, including memorandums of
understanding (MOUs)
1. MEMORANDUM OF UNDERSTANDING ‐ NEA.doc
Stakeholder group formation documentation 2. Stakeholder group NEA.doc
Contractor procurement documentation for
outsourcing performance assessment
3. PAR NEA.doc
4. RFP NEA (PA).doc
5. RecMemo NEA (PA).doc
6. Revised budget (includes revised  budget for the
Evaluation of the FORECAST Program in Georgia 52
TSU project).pdf
Performance Assessment (PA) report 7. NEAC PA Final Report.pdf
8. NEAC PA Final Report Annex 1‐3.ppt
09. NEAC PA Final Report. Annex 4.ppt
10. NEAC PA Final Report. Annex 5.ppt
11. NEAC PA Final Report. Annex 6.ppt
12. NEAC PA Final Report. Annex 7.ppt
Solutions (interventions) Package 13. Intervention Package NEA.doc
14. IIP NEAC Intervention I & II.doc
15. NEA Intervention I&II Final Report.pdf
18. IIP NEAC Intervention III.doc
19. NEA Intervention III Final Report.pdf
22. IIP NEAC Intervention IV.doc
23. NEAC Intervention IV Final Report.doc
26. NEAC Intervention V Final Report.pdf
Contractor procurement documentation for
outsourcing solutions (interventions)
16. RFP NEAC Intervention III.doc
17. RecMemo NEAC Intervention III.doc
20. RFP NEAC Intervention IV.doc
21. RecMemo NEAC Intervention IV.doc
24. RFP NEAC Intervention V.doc
25. RecMemo NEAC Intervention V.doc
FORECAST PMP 27. PMP NEA.xls
Persons with Disabilities, Elders and Orphanages Service Agency at the Ministry of Labor, Health and
Social Affairs of Georgia
DOCUMENT CATEGORY FILE NAMES
Partner organization commitment
documentation, including memorandums of
understanding (MOUs)
1. MOU Service Agency.doc
Contractor procurement documentation for
outsourcing performance assessment
2. PAR Service Agency.doc
3. RFP to Local Partners on Service Agency.doc
4. Rec Memo Service Agency.doc
Tbilisi State University (TSU)
DOCUMENT CATEGORY FILE NAMES
Partner organization commitment
documentation, including memorandums of
understanding (MOUs)
1. MEMORANDUM OF UNDERSTANDING ‐ TSU.doc
Evaluation of the FORECAST Program in Georgia 53
Stakeholder group formation documentation 2. Stakeholder Group TSU.doc
Contractor procurement documentation for
outsourcing performance assessment
3. PAR ‐ TSU.doc
4. RFP TSU PA (PA).doc
5. RFP TSU PA (FoA).doc
6. Rec. Memo TSU PA.doc
7. PAR Budget Revision Request ‐ TSU.doc
Performance Assessment (PA) report 8. TSU PA Final Report.pdf
Solutions (interventions) Package 9. Intervention Package TSU.doc
10. IIP TSU Intervention I and II phase 1.doc
16. TSU Intervention III Final Report.doc
11. IIP TSU Intervention I and II phase 2.rtf
15. IIP TSU Intervention III.rtf
Contractor procurement documentation for
outsourcing solutions (interventions)
13. RFP TSU Intervention III.doc
14. RecMemo TSU Intervention III.doc
FORECAST PMP 17. PMP TSU.xls