[June 2013]
This publication was produced at the request of the United States Agency for International Development. It was 
prepared independently by Mendez, England & Associates.
Performance Evaluation of the USAID/Georgia Access 
to Mechanization Project
PERFORMANCE EVALUATION OF THE 
USAID/GEORGIA ACCESS TO MECHANIZATION 
PROJECT (AMP) 
 
Final Report 
June 2013 
Prepared under RFTOP No. SOL-114-13-000001 
Submitted to: 
USAID/Caucasus 
Submitted by: 
Colin Maclean 
Gaga Nikabadze 
Contractor: 
Mendez England & Associates 
4300 Montgomery Avenue, Suite 103 
Bethesda, MD 20814 
Tel: 301- 652 -4334
www.mendezengland.com
DISCLAIMER 
The authors’ views expressed in this publication do not necessarily reflect the views of the 
United States Agency for International Development or the United States Government
iii | P a g e
TABLE OF CONTENTS 
EXECUTIVE SUMMARY ......................................................................................... 1
Evaluation Purpose and Evaluation Questions .......................................................... 1
Project Background.................................................................................................... 1
Evaluation Design, Methods and Limitations ............................................................. 2
Findings ....................................................................................................................... 3
Conclusions ................................................................................................................ 3
Recommendations ..................................................................................................... 4
1.0 EVALUATION PURPOSE & EVALUATION QUESTIONS ................... 5
1.1 Evaluation Purpose ........................................................................................ 5
1.2 Evaluation Questions..................................................................................... 5
2.0 PROJECT BACKGROUND ......................................................................... 6
3.0 EVALUATION METHODS & LIMITATIONS .......................................... 8
3.1 Evaluation Methodology................................................................................. 8
3.1.1 Quantitative Research and Analysis....................................................... 9
3.1.2 Qualitative Research and Analysis........................................................ 10
3.2 Evaluation Limitations.................................................................................. 11
4.0 FINDINGS, CONCLUSIONS & RECOMMENDATIONS ..................... 11
4.1 Assess the Operating Conditions of MSCs................................................. 11
4.1.1 Question 1: Can each MSC manage its business operations?........ 11
4.1.1.1 Findings .................................................................................................. 12
4.1.1.2 Conclusions .......................................................................................... 13
4.1.1.3 Recommendations ............................................................................... 14
4.1.2 Question 2: Can each MSC operate and maintain its equipment 
and can such equipment be repaired by MSC operators or 
appropriate service providers?.............................................................. 14
iv | P a g e
4.1.2.1 Findings .................................................................................................. 15
4.1.2.2 Conclusions .......................................................................................... 15
4.1.2.3 Recommendations ............................................................................... 16
4.1.3 Question 3: Is the maintenance and repair of equipment funded 
by MSCs in accordance with service schedules and repair needs?16
4.1.3.1 Findings .................................................................................................. 17
4.1.3.2 Conclusions .......................................................................................... 17
4.1.3.3 Recommendations ............................................................................... 18
4.1.4 Question 4: Has the establishment of competing Georgian 
government mechanization centers impacted the MSCs 
established by AMP?................................................................................. 18
4.1.4.1 Findings .................................................................................................. 18
4.1.4.2 Conclusions .......................................................................................... 19
4.1.4.3 Recommendations ............................................................................... 20
4.2 Assess the Effectiveness of Providing Large Grants to Private 
Enterprises or Individuals to Establish or Expand MSCs ......................... 20
4.2.1 Question 5: Did USG-funded grants mitigate perceived or real 
financial or other investment risks, causing enterprises or 
individuals to co-invest in the project?................................................ 20
4.2.1.1 Findings .................................................................................................. 21
4.2.1.2 Conclusions .......................................................................................... 21
4.2.1.3 Recommendations ............................................................................... 21
4.2.2 Question 6: How did the establishment or expansion of MSCs 
harm or help competing mechanization service providers 
including government owned centers?................................................ 22
4.2.2.1 Findings .................................................................................................. 22
4.2.2.2 Conclusions .......................................................................................... 23
4.2.2.3 Recommendations ............................................................................... 23
4.3 Assess whether or not smallholder farmer productivity and incomes have 
improved as a result of training provided by the project and access to and 
utilization of mechanization services............................................................24
4.3.1 Question 7: Did training & utilization of mechanization services 
v | P a g e
contribute to an increase in collective household incomes? .......... 24
4.3.1.1 Findings .................................................................................................. 24
4.3.1.2 Conclusions .......................................................................................... 25
4.3.1.3 Recommendations ............................................................................... 25
4.4 Identify opportunities to improve impact and enhance the 
implementation and management of future similar projects such as gender 
quality and sustainability................................................................................25
4.4.1 Sustainability .............................................................................................. 25
4.4.2 Gender Quality ......................................................................................... 26
5.0 LESSONS LEARNED .................................................................................. 26
 
ANNEXES 
A. Statement of Work 
B. Work Plan - Evaluation Design and Methodology 
C. List of Persons Interviewed 
D. List of Documents Reviewed 
E. Overview of the Evaluation Team’s Mission Findings 
F. MSC Interviewee Questionnaire 
 
 FIGURES 
Figure 1: Geographic Distribution of MSCs 
Figure 2: Question 1 – Summary of Evaluation Design and Methods 
Figure 3: Question 2 – Summary of Evaluation Design and Methods 
Figure 4: Question 3 – Summary of Evaluation Design and Methods 
Figure 5: Question 4 – Summary of Evaluation Design and Methods 
Figure 6: Question 5 – Summary of Evaluation Design and Methods 
Figure 7: Question 6 – Summary of Evaluation Design and Methods 
Figure 8: Question 7 – Summary of Evaluation Design and Methods
vi | P a g e
ACRONYMS 
AOR
COP
Agreement Officer’s Representative
Chief of Party
COR Contracting Officer’s Representative 
DCOP Deputy Chief of Party
EU European Union
FSC Government owned Farm Service Centers
IDP Internally Displaced People
MCG Millennium Challenge Georgia
ME&A Mendez England & Associates
MoA Ministry of Agriculture
MOU Memorandum of Understanding
NGO Non-Governmental Organization
PMP Performance Management Plan
PSD Private Sector Development
SME Small and Medium Enterprise
SOW Statement of Work
TOR Terms of Reference
UN United Nations
USAID United States Agency for International Development
WB World Bank
FINAL PERFORMANCE EVALUATION OF AMP 1 | P a g e
EXECUTIVE SUMMARY 
EVALUATION PURPOSE AND EVALUATION QUESTIONS 
This is a report on the final performance evaluation of the Access to Mechanization Project 
(AMP) funded by the United States Agency for International Development (USAID) Mission in 
Georgia. The AMP project was implemented during the period September 2009 - December 
2012, by CNFA. 
The evaluation of AMP was conducted during the period April – May, 2013, by a team 
assembled by Mendez, England & Associates (ME&A) with headquarters in Bethesda, Maryland. 
The team consisted of two experts - one international and one local – with experience in 
agricultural development and the private sector. The main objective of the evaluation was to 
determine the effects of the AMP program on increasing productivity and incomes for 
smallholder farmers by improving access to machinery services. The main thrust of the 
evaluation was to determine the extent to which AMP was successful in establishing 21 
mechanization service centers (MSCs) throughout Georgia and increasing the incomes of at 
least 12,500 small farmer beneficiaries. An integral part of the evaluation mission was to 
answer a set of pre-determined questions in order for USAID/Georgia to obtain a greater 
understanding of the impact that AMP has had on intended beneficiaries. The findings and 
conclusions reached by the Evaluation Team could be used by USAID/Georgia to design future 
interventions in the field of agriculture development in the country. 
The evaluation questions focused primarily on the performance of the MSCs themselves, as well 
as the impact they have had on local farming communities, especially with regard to increases in 
household incomes. These questions are depicted in table format in the body of this report, 
followed by sections related to findings, conclusions and recommendations. In general, they are 
concerned with the extent to which MSCs manage their business operations effectively, the 
impact they have on competing mechanization centers, and whether or not they would have 
been established without AMP’s matching grant initiative. Supporting data and information can 
be found in the annexes attached to this report. 
PROJECT BACKGROUND 
AMP was designed to address Georgia’s severe shortage of agricultural machinery by using a 
commercially-sustainable, market-oriented methodology for the development of machinery 
service providers. The methodology employed to achieve project objectives involved providing 
matching grants and leveraging commercial finance, business, extension training and technical 
assistance via the Farmer-to-Farmer (F2F) Volunteer program. The ultimate result of AMP’s 
interventions would lead to specific indicator targets being met, including jobs created and 
additional net income for farmers. In addition, one of the project’s other objectives was to 
enable farmers to cultivate land that would have otherwise been left fallow. Finally, the 
enhanced efficiency of newer equipment provided to the farmer clients was expected to reduce 
costs for both service providers and farmers, while allowing a greater area to be served by each 
unit of machinery. In summary, small-scale farmers in particular would become more 
productive, knowledgeable, and better off after improving their working practices as a direct 
result of AMP’s interventions.
FINAL PERFORMANCE EVALUATION OF AMP 2 | P a g e
EVALUATION DESIGN, METHODS AND LIMITATIONS
As the AMP program ended in December 2012, this post-evaluation was tasked with 
concentrating on gathering qualitative data rather than analyzing the quantitative results 
highlighted in CFNA’s End-of-Project Final Report. The methodology that was ultimately 
adopted by the Evaluation Team to secure the required information and data that would 
support its findings, centered on conducting face-to-face interviews with pre-selected 
stakeholders with first-hand knowledge of AMP, and with the owners and other key personnel 
of the MSCs. In addition, it was deemed important that focus group discussions (FGDs) be 
conducted with farmers in MSC jurisdictions in order to obtain feedback from the project’s 
other main beneficiaries. Even though the time allocated for site visits was short, the evaluation 
team was able to visit all 21 MSCs created by AMP and conduct 17 FGDs, which provided 
invaluable insight into MSCs operations and their impact on the rural farming communities they 
serve. At the same time, the Evaluation Team also examined AMP-related documents and 
materials, such as annual and quarterly reports, performance management plans (PMPs), and 
other sources of reference that would validate their findings and conclusions. 
The Evaluation Team encountered few limitations during its fieldwork in Georgia. Some of the 
more relevant limitations are listed below: 
1. Quite a few of the MSCs were established during the latter stages of AMP; therefore, 
the effects of their impact on local farming communities would only be apparent post￾project. Consequently, the Evaluation Team was not able to comment conclusively on 
the actual statistics regarding job creation resulting from AMP’s interventions or on the 
real extent of increases in collective household incomes. 
2. The evaluation took place at the beginning of the spring agriculture season, when 
farmers were busy preparing land for seeding and planting. It was initially anticipated 
that this might affect their availability to attend FGDs; however, this proved not to be 
the case, with the exception of five locations where farmers were unfortunately too 
busy to attend. 
3. Since AMP ended in December 2012, it was agreed with USAID that the Evaluation 
Team would essentially limit its review to qualitative rather than quantitative analysis in 
order to concentrate on finding meaningful answers to the pre-determined questions 
established by USAID/Georgia. Therefore, the Evaluation Team will not comment on 
the achievement of performance indicator targets; instead, it has limited its remarks to 
AMP’s data gathering and reporting procedures, as well as the extent to which they did 
or did not contribute to effective self-monitoring and evaluation. 
4. One of the more significant and problematic issues that confronted the Evaluation Team 
was the difficulty in trying to correlate increases in farmer household incomes with the 
activities of the MSCs established by AMP. This is largely due to the fact that other 
factors, that cannot be isolated, come into play when determining the true extent of 
such increases. Therefore, it cannot be said that the impact of the project’s 
interventions were the sole factor. However, it was clear from the farmer FGDs that all 
had experienced some increase in their incomes, a significant part of which was 
attributed to the MSCs. 
FINAL PERFORMANCE EVALUATION OF AMP 3 | P a g e
While important, the above limitations did not prevent the Evaluation Team from gathering the 
information and data needed to draw conclusions and make recommendations for similar, 
USAID-funded projects in the future. 
FINDINGS 
1. Most MSCs are currently managing their businesses profitably, even without high 
standards of management practices.
2. There is no evidence to suggest that MSCs in general are negatively impacted by 
government centers or vice versa.
3. Without AMP, two thirds of the MSCs would not have been established and the 
additional one third would have started, but at a later time and with less servicing 
capacity.
4. AMP’s contribution to rural farming is substantial. In real terms, over 16,500 rural 
households are estimated to have benefited from AMP’s interventions, with an 
approximate $5.33 million additional annual income accrued by farmers over the 
project’s lifetime. Furthermore, the over 2,000 Internally Displaced People (IDP)
families that also benefited, would have not been able to do so without AMP’s 
involvement.
5. Data gathered from FGDs indicated that most farmers experienced an increase in 
household income, although this could not be directly correlated to MSC services. 
However, following the implementation of two successive annual surveys conducted by 
AMP it was estimated that, on average, small-scale farmer’s incomes increased by just 
over $300. 
6. From discussions with MSCs owners, the Evaluation Team found that few were able to 
prepare basic financial statements such as cash flow statements, profit and loss accounts 
or balance sheets. 
7. One of the more pressing issues emanating from discussions with MSCs and farmers 
was the lack of available spare parts to repair machinery and equipment. 
8. Most MSCs were able to carry out minor repairs and maintenance of machinery and 
equipment with some having the technical expertise to carry out repairs that were 
more serious. However, the Evaluation Team found that few MSCs were capable of 
carrying out repairs on modern machinery and equipment with more sophisticated and 
new technologies. 
CONCLUSIONS 
1. Demand for MSC services far outweighs supply thus ensuring that most remain 
profitable in the short- to medium-term. 
2. Competition from government centers is not a barrier to growth for AMP MSCs and 
their operations; nor do AMP MSCs pose a threat to government centers.
3. AMP’s intervention through matching grants was a vital decision-making factor in 
establishing MSCs.
4. Income increases from the MSCs’ mechanization services could have been higher if the 
farmers had access to value chain services and had not been subject to cheap foreign 
imports.
FINAL PERFORMANCE EVALUATION OF AMP 4 | P a g e
5. It is difficult to correlate the impact of AMP on farm household incomes since any 
increases could also have been effected by other factors not directly related to AMP. 
For example income derived from other sources of activity, from other family members, 
etc., may play a role in increased household income. 
6. Most MSCs neither prepare nor understand how to use financial statements as a 
management tool. 
7. If the lack of available spare parts to repair machinery and equipment continues, it is 
highly likely that farmers will continue to experience unwanted down time with 
machinery lying idle in the fields waiting repair. The worst-case scenario is that income 
would be lost if farmers miss the planting ‘window’ at the start of the season when 
seeds must be sown. 
8. Most MSCs do not feel they have sufficient knowledge, skills or expertise in new 
technologies to be able to carry out needed repairs or maintenance on new machinery 
and equipment. 
RECOMMENDATIONS 
1. In the future, any similar project should consider a more value chain approach to widen 
the scope of available MSC services – e.g. input supplies, extension and marketing. This is 
important because: 1) evidence suggests that there is a demand from farmers for such 
services; and 2) it will increase the potential of MSCs to increase income all-year-round 
instead of being limited to sales on a seasonal basis, as is currently the case for most 
MSCs. 
2. A brief study on the effect that the privatization of government-owned Farm Service 
Centers (FSCs) will have on the farming sector should be conducted.
3. Any future similar project should seriously consider including matching grants as a major 
component.
4. The scope of assistance for any future project should consider a more diversified 
approach (entire value chain) to agricultural development.
5. Any future similar project might be advised not to include quantifiable results such as 
increase in household income that cannot be readily or scientifically verified.
6. Plans for any future interventions should consider building the capacity of MSC key 
personnel to prepare, at a minimum, cash flow statements with one year forecasts, in 
order for them to be more aware of cash flows in and out of the business. 
7. The issue of lack of spare parts is a challenging one with no easy solution. Nevertheless, 
future projects should consider using the project as a catalyst between dealers, MSCs 
and farmers and a mechanism for establishing procedures and guidelines for how and 
when each party should act if/when it becomes apparent that the availability of spare 
parts needs attention. 
8. It is recommended that the inclusion of practical training workshops in new 
technologies be considered for any follow-on project working with the MSCs.
FINAL PERFORMANCE EVALUATION OF AMP 5 | P a g e
1.0 EVALUATION PURPOSE &
EVALUATION QUESTIONS
1.1 EVALUATION PURPOSE 
This is a report on the final performance evaluation of the Access to Mechanization Project (AMP) 
project funded by the United States Agency for International Development (USAID) Mission in 
Georgia. The project was implemented by CNFA between September 2009 and December 
2012. 
According to the Statement of Work (SOW) for this assignment (see Annex A), the main goal of 
the evaluation was to “determine the effects of the program on increasing productivity and 
incomes for smallholder farmers by improving access to machinery services.” Essentially, the 
Evaluation Team was tasked with assessing the results of AMP in terms of the above-mentioned 
list of objectives. The underlying rationale for AMP was that it should support USAID/Caucasus’ 
Development Objective – “Inclusive and Sustainable Economic Growth.”
The Evaluation Team consisted of two key experts: international expert, Colin Maclean (Team 
Leader), and local expert, Gaga Nikabadze (Agribusiness Specialist). In addition, the team was 
assisted by Ms. Lisa Basishvili who served as the interpreter and translator. 
1.2 EVALUATION QUESTIONS 
As per the Scope of Work (SOW), the evaluation team was asked to answer a number of 
specific evaluation questions, outlined below: 
1. Assess the operating condition of MSCs. 
a. Can each MSC manage its business operations (e.g., positive cash flow from 
operations, record keeping, designation of employee duties, staff coverage, sales 
and marketing capacity) when evaluated against standards determined by the 
evaluators?
b. Can each MSC operate and maintain its equipment, and can such equipment be 
repaired by MSC operators or appropriate service providers? 
c. Is the maintenance and repair of equipment regularly funded by MSCs in 
 accordance with applicable service schedules and repair needs? 
 d. Has the establishment of competing Georgian government mechanization 
 centers impacted the MSCs established by AMP? 
2. Assess the effectiveness of providing large grants to private enterprises or individuals to 
establish or expand MSCs; Assess whether or not grant assistance caused market 
distortions by answering the following two questions: 
a. Did the United States Government (USG)-funded grants mitigate perceived or 
real financial or other investment risks, causing enterprises or individuals to co-
FINAL PERFORMANCE EVALUATION OF AMP 6 | P a g e
invest with the project? 
 b. How did the establishment or expansion of MSCs harm or help competing 
mechanization service providers, including Government of Georgia (GoG)-
owned centers? 
3. Assess whether or not smallholder farmer productivity and incomes have improved as a 
result of training provided by the project and access to and utilization of mechanization 
services. 
a. Did training and utilization of mechanization services contribute to an increase in 
 collective household incomes? 
4. Identify opportunities to improve impact and enhance the implementation and 
management of future similar projects such as gender equity and sustainability. 
2.0 PROJECT BACKGROUND
AMP was specifically designed to help address Georgia’s severe shortage of agricultural 
machinery and services available to small-scale farmers1
 with a view to improving their farming 
operations through increased productivity and, ultimately, leading to increased household 
incomes. The underlying rationale for such an initiative was that by adopting a more 
commercially sustainable, market-oriented methodology for the development of machinery 
service providers, beneficiary farmers would be able to improve the quality and yield of farming 
output resulting from improved technical know-how and production methods (plowing, 
planting, cultivating and harvesting). In addition, by facilitating the potential for farmers to 
cultivate land that would have otherwise been left fallow, the opportunity to benefit from 
additional income assumed even greater significance. 
Although other agribusiness-related initiatives, including the Millennium Challenge Georgia 
(MCG) Agribusiness Development Activity, assisted in the establishment of Farm Service 
Centers (FSCs) through the provision of grants, the demand for such services in the mid to 
late-2000s was (and remains even in 2013) significantly higher than available supply. By May 
2008, 12 FSCs had been established. In addition, the GoG, under the auspices of the Ministry 
of Agriculture (MoA), financially supports 12 FSCs throughout the country2
, offering machinery 
services as well as technological advice on soil testing, plant diseases and fertilizers. 
However, according to the MoA, these FSCs were inherited by the present government, which 
has no interest in retaining them and, at the time of writing, was planning to privatize them. In 
summary, by the time the AMP started in September 2009, the dire lack of mechanization 
equipment and services for small-scale farmers - identified as the gap that needed to be 
addressed - remained as relevant as initially perceived by USAID/Georgia during the project 
design stage. Given that nearly all persons interviewed during the evaluation agreed that this 
 
1
For the purposes of this report defined as farming less than 2 hectares 
2
Gurjaani, Dedoflistskaro, Kareli, Marneuli, Bolnisi, Akhaltsikhe, Zestafoni, Samtredia, Abasha, Ozurgeti and Ambrolauri
FINAL PERFORMANCE EVALUATION OF AMP 7 | P a g e
gap still remains as of April 2013, it is clear that the decision to initiate AMP was justified and 
necessary. 
In brief, AMP was a $5.1million initiative implemented using a combination of matching grants, 
leveraged commercial finance, business and extension training and volunteer technical assistance 
to establish 21 MSCs3
 throughout Georgia to provide fee-based, custom machinery services to 
small-scale farmers (for a geographic distribution of MSCs, see Figure 1, below). 
Figure 1: Geographic Distribution of MSCs 
The project equipped the centers with an array of agricultural machinery using a combination of 
grant funding (up to a maximum of $200,000) and local partner matching investment (with a 
minimum matching investment of 1:1). The project adopted a flexible approach during 
implementation to ensure that each MSC housed the appropriate mix of machinery and 
equipment in line with their individual target markets. A contractual requirement was that all 
machinery should be purchased locally from already established equipment dealers in order to 
ensure the provision of proper training, maintenance and long-term supply of parts. 
Specifically, AMP was tasked with achieving a number of key objectives (see Annex E for the full 
list of quantitative indicator targets), outlined below: 
1. Establish 21 MSCs offering improved mechanization services to over 12,500 smallholder 
farmers. 
 
3
The original number was 25-30 which later officially modifed USAID / CNFA to 21 
FINAL PERFORMANCE EVALUATION OF AMP 8 | P a g e
2. Provide technical assistance to these MSCs using a combination of local consultants and 
farmer-to-farmer (F2F) volunteers to ensure the safe, sustainable and profitable 
operation of the businesses and long-term availability of services for farmers. 
3. Leverage USAID grant funds with local partner matching investment, including large￾scale involvement of local and international commercial finance, to maximize investment 
in the rural economy. 
4. Improve the competitive environment for machinery services by reducing the cost to 
farmers as a result of the increased supply of machinery and availability of dedicated 
service providing businesses. 
5. Facilitate the return of more agricultural land into production. 
6. Contribute to increased productivity and income of smallholder farmers. 
7. Provide discounted machinery services to about 3,000 resettled Internally Displaced 
Persons (IDPs). 
Following the completion of AMP in December 2012, USAID/Georgia decided to conduct an 
end-of-project performance evaluation, which is the subject of this evaluation report. The 
findings, conclusions and recommendations, described in detail both in the body of the report 
and its attachments (see 1.2 above and relevant annexes), are the work of the Evaluation Team 
tasked with carrying out the evaluation. The result of the evaluation should allow 
USAID/Georgia to identify lessons learned, assess strengths and weaknesses associated with the 
project, and refer to suggested recommendations for future project planning purposes. 
3.0 EVALUATION METHODS &
LIMITATIONS 
3.1 EVALUATION METHODOLOGY
To design the methodology for this evaluation, the Evaluation Team took into account a 
number of key factors including quantitative and qualitative aspects of the mission which could 
influence or place intended/unintended limitations on its strategy for answering the pre￾determined evaluation questions. The more important of these factors are outlined below: 
1. Key Factor: A significant part of AMP’s mission was to meet or exceed a wide range of 
indicators and quantitative targets against which the project’s performance would be 
measured. 
Issue: As AMP had already ended in December 2012, the Evaluation Team concluded 
that the results depicted in the CNFA End-of-Project Report relating to quantitative 
targets had already been approved by USAID and, therefore, there would be no need to 
verify the accuracy of the data reported in project documentation including whether or 
not quantitative results had actually been achieved. 
Outcome: Given the above, the only requirement incumbent on the Evaluation Team 
was to report on AMP’s data gathering and reporting methodology to determine its 
efficiency and relevance from an operational point of view and thus, the extent to which 
it contributed to the effectiveness or otherwise of the project’s internal monitoring and 
FINAL PERFORMANCE EVALUATION OF AMP 9 | P a g e
evaluation (M&E) system.
2. Key Factor: The gathering and analysis of data and information from key informants 
during the evaluation took place at the beginning of the spring planting season in 
Georgia. 
Issue: There was every possibility that some of the FGDs organized with the 
collaboration of each MSC might not have high attendance rates (if any) due to farmers 
being occupied in their fields. As the evaluation was particularly concerned with 
gathering the views of women and IDPs, it was acknowledged that this might present a 
challenge. 
Outcome: The Evaluation Team was able to conduct 17 FGDs from 15 – 22 April 
2013, including interviews with women and IDPs. MSCs, who already had the contact 
details of farmers within their catchment areas, invited them to participate in FGDs. This 
significantly factored into the majority of FGDs being well supported and attended (on 
average nine farmers per FGD). During these sessions, MSCs and farmers advised the 
Evaluation Team that those who turned up in fact constituted a representative sample of 
the wider farming community as most farms in fact were of a similar size and produced 
a similar range of products. Finally, while it was initially envisaged that site visits to 
review machinery and equipment might be possible at the same time as visits to MSCs, 
this was unfortunately not possible for two reasons. First, in most locations farmers 
were already heavily involved in seasonal preparations with much of the MSCs’ 
machinery and equipment in actual use. Second, many of the farms were not easily 
accessible making site visits difficult to conduct within the available timeframe. However, 
the Team was able to view machinery and equipment, either in storage or undergoing 
repair, on MSC premises which provided a useful insight as to how those particular 
MSCs operated.
3. Key Factor: One of the Evaluation Team’s tasks was to research and comment on the 
sustainability of future similar projects. This was carefully considered throughout the 
evaluation.
Issue: In order to address this issue, it was important for the Evaluation Team to 
identify a number of meaningful criteria against which the sustainability of MSCs could be 
calculated.
Outcome: After serious deliberation, it was decided that such criteria should include 
the following: 1) a review of financial performance over time including cash flow, debt 
servicing, management of accounts receivable and payable; 2) assessment of an MSC’s 
client base (growing or otherwise); 3) prevailing and forecasted demand for services 
from farming community; and 4) brief analysis of local and regional market conditions. 
3.1.1 Quantitative Research and Analysis 
The Evaluation Team reviewed a wide range of project-related documentation from several 
sources, including USAID (during pre-mobilization) and CNFA (upon arrival in Georgia). In 
particular, the Evaluation Team reviewed AMP’s M&E plans and related documents, such as 
Performance Management Plan (PMP) excel spreadsheets, as the main reference point for 
statistical data across the life of the project. The team’s analysis of this material formed the 
basis of observations regarding the methodology employed by AMP to track its performance 
relative to pre-determined indicator targets (see Annex E – Overall M&E Matrix). The 
FINAL PERFORMANCE EVALUATION OF AMP 10 | P a g e
verification of statistical results actually achieved by the end of AMP, however, did not fall 
within the purview of this evaluation. 
This led the Evaluation Team to draw conclusions and make recommendations regarding 
options that USAID might wish to consider when preparing future Request for Task Order 
Proposals (RFTOPs) regarding the M&E methodology procedures expected to be implemented 
throughout the duration of a long-term project. 
Finally, their impact on intended beneficiaries was reviewed by cross-referencing achievements 
(or otherwise) against the Evaluation Team’s findings emanating from evidence gathered from 
its qualitative research approach. This included findings from interviews with pre-identified 
respondents with direct knowledge of AMP, as well as from feedback gained during FGDs with 
MSCs (see Annex E as a source of reference). 
3.1.2 Qualitative Research and Analysis 
Initially, it was anticipated that a sample of the 21 MSCs would be selected for site visits, 
although no specific criteria had yet been established as to how this sample would be identified. 
This was further complicated by the fact that the proposed visits would be conducted 
effectively at the start of the plowing/planting season, thus potentially affecting the number of 
farmers willing to participate in FGDs. Also to be taken into consideration was the need for 
representatives of women farmers and IDPs (AMP was tasked with supporting 2,000-3,000 IDP 
families to benefit from improved mechanization) to attend FGDs. 
However, following a thorough review of the geographic locations of all 21 MSCs, the 
Evaluation Team decided that it would be possible within the time parameters of the mission to 
visit and host FGDs in all 21 MSC locations. In fact, it was possible to host 17 FGDs around the 
country with an average attendance level of 10 farmers (12 is the internationally accepted level 
for a focus group session), which was an excellent turnout given that most farmers were busy 
with pre-season preparations. In those cases where turnout was less than average this was 
directly related to farmers being unable to spare the time due to their being out in the field. In 
some cases, attendance significantly exceeded expectations either because there was an 
exceptionally high level of interest in the Evaluation Team’s mission or because farmers in that 
particular location had been unable to work on their farms due to weather conditions. 
While the focus of the evaluation centered on eliciting feedback from AMP’s direct 
beneficiaries, such as the MSCs themselves and the smallholder farming community, it was also 
important to ascertain the views and opinions of third party stakeholders, such as the GoG, 
MoA, local municipalities, industry, business and farming associations, financial institutions and 
the international community. The Evaluation Team met with representatives from each of these 
groups (see Annex 3, List of Interviews, for a complete list). The primary purpose of these 
meetings was to ascertain the perceptions of AMP held by individuals or organizations that 
could enhance the quality of recommendations for any future planned interventions by 
USAID/Georgia. 
FINAL PERFORMANCE EVALUATION OF AMP 11 | P a g e
3.2 EVALUATION LIMITATIONS
The Evaluation Team encountered few limitations during its fieldwork in Georgia. Some of the 
more relevant limitations are listed below: 
1. Quite a few of the MSCs were established during the later stages of AMP; therefore, the 
effects of their impact on local farming communities would only be apparent post￾project. Consequently, the Evaluation Team was not able to comment conclusively on 
the actual statistics regarding job creation resulting from AMP’s interventions or on the 
real extent of increases in collective household incomes. 
2. The evaluation took place at the beginning of the spring agriculture season, when 
farmers were busy preparing land for seeding and planting. Initially, it was anticipated 
that this might affect availability to attend FGDs; however, this proved not to be the 
case with the exception of five locations where farmers were unfortunately too busy to 
attend. 
3. Since AMP ended in December 2012, it was agreed with USAID that the Evaluation 
Team would essentially limit its review to qualitative analysis rather than quantitative 
research in order to concentrate on finding meaningful answers to the pre-determined 
questions established by USAID/Georgia. Therefore, the Evaluation Team will not 
comment on the achievement of performance indicator targets but instead has limited 
its remarks to AMP’s data gathering and reporting procedures, as well as the extent to 
which they contributed (or otherwise) to effective self-monitoring and evaluation. 
4. One of the more significant and problematic issues that confronted the Evaluation Team 
was the difficulty in trying to correlate increases in farmer household incomes with the 
activities of the MSCs established by AMP. This is largely due to the fact that other 
factors come into play when determining the true extent of such increases. Therefore, 
changes in household income cannot be attributed exclusively to the project’s 
interventions. However, it was clear from the farmer FGDs that all had experienced 
some increase in their incomes and that they attributed a significant part of that to the 
MSCs. 
In summary, while important, the above limitations did not prevent the Evaluation Team from 
gathering the information and data needed to draw conclusions and make recommendations for 
similar USAID projects funded in the future. 
4.0 FINDINGS, CONCLUSIONS & 
RECOMMENDATIONS
4.1 ASSESS THE OPERATING CONDITIONS OF MSCS 
4.1.1 Question 1: Can each MSC manage its business operations? 
FINAL PERFORMANCE EVALUATION OF AMP 12 | P a g e
Figure 2: QUESTION 1 - SUMMARY OF EVALUATION DESIGN AND METHODS
Evaluation 
Question 1
Type of Analysis 
Conducted
Data Sources and 
Methods Used
Type and Size 
Sample
Limitations
1. Can each MSC 
manage its 
business 
operations (e.g. 
positive cash 
flow, record 
keeping, staff 
coverage, sales 
and marketing 
capacity) to be 
evaluated by 
standards 
determined by 
the evaluators
Description – based 
on content analysis 
of expert opinions
Key informant 
interviews with key 
personnel in 21 
MSCs 
Interviewees 
identified by 
Evaluation Team, 
AMP & USAID
None encountered
Description – based 
on content analysis 
of secondary data
Review of MSC 
company sales and 
financial records
100% sample of 
MSCs decided by 
Evaluation Team
Insufficient financial 
data supplied by the 
MSCs for scrutiny
Description - based 
on content analysis 
of primary data
Review of AMP M & 
E records & relevant 
reports
All requested 
documents provided
Understanding some 
of the documentation 
in terms of clarity and 
relevance
4.1.1.1 Findings 
MSC Operating Activities 
In order to gain a clear understanding of how MSCs were managing their respective operations, 
the Evaluation Team prepared a structured list of questions specifically designed to provide 
answers directly related to their profitability and long-term sustainability. In this particular 
instance, the decision was taken that the standard that would need to be met was whether 
MSCs had any specific operating functions in place (or not) that would facilitate proper 
management of their businesses. While not an ideal form of measurement, the Evaluation Team 
decided that as most MSCs were essentially owner managed this could be evaluated through 
responses to a pre-prepared questionnaire employed by the Evaluation Team.
Drawing on information gathered from MSC owners, directors and managers, the Evaluation 
Team found that all MSCs appeared to be managing their affairs sufficiently well enough to 
remain profitable in the short- to medium-term. In CNFA’s End-of-Project Report, it was 
calculated that the increased annual gross profit of all 21 MSCs (collective and cumulative) was 
in the region of $424,000, or a mean average of $20,000 per MSC. However, this needs to be 
qualified by the fact that, with the exception of a few MSCs, most prepared no meaningful 
financial records (cash-flow statements, balance sheets or income statements). However, they 
did forward the required financial information required by contract and requested by AMP 
throughout the project. 
From a financial management perspective, MSCs prepared documents such as invoices and 
receipts solely for the tax authorities and not as an operating tool to assist in the effective 
management of the business. A brief review of these documents presented to the Evaluation 
Team, confirmed that no real financial reporting or forecasting was practiced by most of the 
MSCs although some stated that they had received instruction from a F2F volunteer financial 
expert on how to prepare financial statements, including cash-flow statements. Nevertheless, 
comments made by owners or their accountants revealed that only a few MSCs possessed the 
skills and/or resources required to prepare such documents. In short, the evidence suggests 
FINAL PERFORMANCE EVALUATION OF AMP 13 | P a g e
that implementing what they had learned about financial reporting was beyond most MSCs’
understanding and capabilities. 
In terms of how MSCs manage their human resources, it was apparent from interview 
comments that there were no specific or common issues or features that could be identified as 
being particularly troublesome for them. MSCs are mainly small businesses with relatively few 
full-time staff. Most are dependent on seasonal workers whom they employ when required. In 
terms of staff coverage, feedback from MSC owners indicated that this was not a problem for 
them. In terms of marketing capacity, it is important to underline the fact that demand for 
services considerably exceeds supply and that their target audience is well defined. Therefore, 
at present there is little need for a sophisticated marketing strategy. The only issue that might 
improve the ability of MSCs to improve their income stream all-year-round would be to offer 
additional products and services such as input supplies and extension facilities. 
Farmers’ Impact on MSCs
One of the main criteria to assess how well a business is performing is the nature of its client 
base and the extent of on-going, repeat sales to those clients. In the above-mentioned CNFA 
End-of-Project Report, which commented on AMP’s indicator targets, sales of services to 
farmers was calculated at just over $1.78m accumulated over the lifetime of the project (against 
an initial planned target of $2.5-$4.0m4
). Taken as a mean average, each of the 21 MSCs would 
therefore have benefited from around $85,000 in income from farmers as a direct result of 
AMP’s interventions. In addition, from the information that the Evaluation Team gathered 
during FGDs with farmers, since the end of AMP in December 2012, most of their business 
relations with MSCs remained on a sound footing and they believed it would continue to do so 
for the foreseeable future. Because of this positive development, it can be assumed that owners 
and managers of MSCs are to some degree demonstrating certain management abilities that are 
effectively serving to cement relations with their main client basis – the local farming 
community. 
4.1.1.2 Conclusions 
All available evidence5
 suggests that most MSCs possess the basic skills and expertise to operate 
their respective businesses profitably – at least in the short- to medium-term. However, at the 
start of each new season there is always the worry about adverse weather and other external 
factors that could affect the successful cultivation and harvesting of crops. 
The first season of 2013 was underway as this report was being prepared. The success or 
failure of this season could potentially enhance or undermine income and thus the livelihoods of 
the entire sector. For the MSCs, this is one of their busiest periods - when demand for their 
services substantially exceeds what most are capable of providing due to the continued lack of 
proper machinery and equipment for the tasks. 
 
4
 A number of factors were suggested for the non-achievement of this particular indicator target including land not being 
cultivated properly or on time resulting from natural causes such as drought or floods. 
5
 Includes evidence gathered from interviews with MSC owners, managers and other key personnel, focus groups sessions with 
their main customers (farmers), AMP records including cash-flow, income statemenst and balance sheets and MSC financial 
documents provided to the Evaluation team
FINAL PERFORMANCE EVALUATION OF AMP 14 | P a g e
Nevertheless, feedback from most stakeholders interviewed, including MSCs and farmers, 
clearly indicates that as there is very little direct competition for their particular services from 
the 12 government-owned mechanization service centers around the country. Thus, business is 
likely to remain stable in 2013. Therefore, while short- to medium-term survival seems assured, 
it is difficult to conclude that current management abilities and/or prevailing market conditions 
will remain stable in the longer term primarily because MSCs make most of their income on a 
pre-determined seasonal basis with lengthy periods of inactivity during the rest of the year. 
4.1.1.3 Recommendations 
To ensure the longer-term survival and sustainability of the 21 MSCs established by AMP, two 
important issues need to be addressed. First, financial management skills need to be improved 
so that owners and managers can undertake forward planning and are able to forecast income 
and expenditure. This can be best achieved by introducing and training owners and managers in 
the art of basic cash-flow analysis and the preparation of cash flow statements. While some of 
the MSCs recalled receiving some financial planning advice from F2F volunteers, the Evaluation 
Team did not find this to have been a particularly successful endeavor since only a few of the 
MSCs actually prepare such statements. Second, the range of services currently offered by most 
MSCs is not expansive enough to ensure real long-term growth. Therefore, any similar future 
project being contemplated by USAID/Georgia should consider the following: 
1. Include a component that ensures that MSCs prepare cash-flow statements that can be 
monitored by a project team in real time (monthly, quarterly, annually) to track 
performance of actual results against planned. This will help owners/managers operate 
their businesses more effectively. 
2. Provide the support needed for MSCs to expand the scope of their activities to include 
extension services, input supplies and marketing. This would enable MSCs to earn 
income year-round rather than seasonally. 
Adopting these initiatives should significantly contribute towards MSCs becoming more 
sustainable and, indeed, more professional in the way they operate their businesses. 
4.1.2 Question 2: Can each MSC operate and maintain its equipment and can such 
equipment be repaired by MSC operators or appropriate service providers? 
Figure 3: QUESTION 2-SUMMARY OF EVALUATION DESIGN AND METHODS
Evaluation 
Question
Type of Analysis 
Conducted
Data Sources and 
Methods Used
Type and Size 
Sample
Limitations
2. Can each MSC 
operate and 
maintain its 
equipment and 
can such 
equipment be 
repaired by MSC 
operators or 
appropriate 
service providers
Description – based 
on content analysis 
of expert opinions
Key informant 
interviews with key 
personnel in 21 
MSCs 
Interviewees 
identified by 
Evaluation Team, 
AMP (ex-team 
members) & USAID 
Limited effective 
means of verifying 
responses
Description and 
comparison of 
ratings by MSCs & 
service providers
Key informant 
interviews with key 
personnel in MSCs 
and from farming 
community
16 focus group 
sessions nationwide
None encountered 
– farmer’s views 
accepted as 
reflectively actual 
events
FINAL PERFORMANCE EVALUATION OF AMP 15 | P a g e
4.1.2.1 Findings 
During interviews with the owners/managers of MSCs, one of the specific questions asked 
related to the ability of the organization to properly operate and maintain its equipment. In 
addition, questions about the ability of in-house technicians or external service providers to 
repair such equipment and machinery were also asked. The outcome of those enquiries 
revealed a number of observations (see Annex F – Overview of the Evaluation Team’s Mission 
Findings for a more in-depth analysis). In creating the chart in Annex F, the Evaluation Team 
identified responses to questions using a high, medium or low risk assessment format in order 
to conduct a statistical analysis of responses and thus draw meaningful conclusions. The results 
indicated that 8 of the 21 MSCs were highly likely to have the necessary skills and suitable 
qualified technicians to carry out most types of repair and maintenance. The others, with the 
exception of one, appeared to have adequate resources to conduct most minor repairs, while 
repairs that were more complicated were generally outsourced to local service providers who 
were better equipped to handle such work. 
It should be noted though that most of the MSC machinery and equipment is relatively new 
since much was purchased during the start-up phase through the AMP matching-grant 
arrangement. This effectively means that most MSCs have not actually experienced many 
problems to date and, if they did and were unable to handle the matter themselves, would 
simply return the equipment or machinery in question to the dealer for repair as this would fall 
within the sales warranty agreement. However, one of the major issues affecting both MSCs 
and occasionally equipment dealers was the lack of available spare parts which, according to 
MSCs as well as farmers, often resulted in inactivity while waiting for spare parts to arrive. A 
number of farmers explained that this situation could lead to their missing the plowing and 
planting seasonal window, the short timeframe within which they had to prepare their land 
plots for seed sowing. If this happened, farmers would miss an entire season’s crops with the 
obvious ramification of lost income. 
One issue mentioned on a number of occasions by MSCs as well as farmers was the difficulty of 
repairing and maintaining new technology. Many of the interviewees mentioned that owning or 
using new machinery and equipment, that you are not accustomed to, often means that you 
need training to be able to properly repair and maintain it. 
4.1.2.2 Conclusions 
For the moment, there appears to be little evidence to suggest that the repair and maintenance 
of machinery and equipment is an issue for most MSCs, a conclusion supported by the farmers. 
However, lack of access to spare parts is an issue that could result in essential machinery and 
equipment, such as tractors and other agricultural implements, lying idle in the fields while 
farmers wait in hope and frustration. For farmers affected by the lack of functioning machinery 
or equipment, the impact can be immediate and potentially damaging in terms of lost income. If 
land is not prepared and seeded on time it is possible that farmers will not be able to make a 
reasonable living since they will lose all or most of the income they depend on to survive. 
Occasionally, this situation has resulted in some farmers utilizing government mechanization 
center services (if they are located within close proximity) in order to avoid missing the above￾mentioned seasonal window. 
FINAL PERFORMANCE EVALUATION OF AMP 16 | P a g e
The longer term impact of this situation could compromise the sustainability of MSCs as 
farmers look for alternative service providers able to meet their demands for the right 
equipment at the right time and at the right price. While available evidence currently suggests 
that farmers’ customer loyalty towards their local MSCs remains high, it cannot be guaranteed 
in the long-term, since farmers cannot afford any downtime just before the start of a season. 
For many farmers, particularly those that grow perennial, open-field crops such as wheat, barley 
and maize, a significant proportion of a season’s yield is often used for household consumption 
and livestock feed rather than marketed for profit. If these crops are compromised in any way, 
the effects on the overall farming community can be devastating. 
4.1.2.3 Recommendations 
In response to the above findings and conclusions, similar projects planned for the future should 
focus on addressing the concerns described in the following ways: 
1. Adopt a more practical approach to upgrading the technical skill and expertise of MSCs 
and farmers. This could involve repair and maintenance workshops using experienced 
local rather than international technicians made available through the project. 
Instruction manuals should be translated into the Georgian language so that farmers and 
technicians can understand them (a real issue for anyone but especially important for 
rural MSCs). 
2. Work more closely with the recognized dealers from whom the MSCs purchase their 
machinery and equipment with a view to identifying processes and procedures to 
alleviate obstacles, i.e., the lack of spare parts. Organize and deliver practical workshops 
with MSC dealers and farmers, jointly, on repair and maintenance required to minimize 
downtime.
3. Offer customized and practical training courses to selected MSC technicians to upgrade 
their skills, knowledge and ability to operate and maintain new technologies. 
Implementation of the measures mentioned above should help alleviate the two primary 
challenges currently affecting the entire rural farming sector and should lead to reduced loss of 
income that results when land remains idle and is not cultivated productively, due to downtime 
resulting from lack of knowledge in the use and maintenance of new machinery and equipment.
4.1.3 Question 3: Is the maintenance and repair of equipment funded by MSCs in 
accordance with service schedules and repair needs? 
Figure 4: QUESTION 3 -SUMMARY OF EVALUATION DESIGN AND METHODS
Evaluation 
Question
Type of Analysis 
Conducted
Data Sources and 
Methods Used
Type and Size 
Sample
Limitations
3. Is the 
maintenance and 
repair of 
equipment 
funded by MSCs 
in accordance 
with service 
schedules and 
repair needs?
Description – based 
on content analysis 
of expert opinions
Key informant 
interviews with key 
personnel in 21 
MSCs 
Interviewees 
identified by 
Evaluation Team, 
AMP (ex-team 
members) & USAID 
Limited effective 
means of verifying 
responses
Description – based 
on feedback from 
MSCs & service 
providers
Key informant 
interviews with key 
personnel in MSCs 
and from farming 
17 focus group 
sessions nationwide
None encountered 
– farmer’s views 
accepted as 
reflectively actual 
FINAL PERFORMANCE EVALUATION OF AMP 17 | P a g e
Figure 4: QUESTION 3 -SUMMARY OF EVALUATION DESIGN AND METHODS
Evaluation 
Question
Type of Analysis 
Conducted
Data Sources and 
Methods Used
Type and Size 
Sample
Limitations
community events
Description based 
on content analysis 
of primary data
Review of AMP M & 
E records and 
relevant reports
The Evaluation 
Team unable to 
track actual 
performance against 
plan in M&E-related 
documents
4.1.3.1 Findings 
Out of 21 total MSCs, 17 confirmed that they serviced and maintained all of their machinery 
and equipment, including combine harvesters, tractors, seeders, rot tillers, cultivators, etc., in 
accordance with the manufacturers’ operating manual instructions. The remaining 4 tried to 
follow the requirements but did not understand the language of instruction. All MSCs advised 
that failure to follow operating manual instructions was foolish - in the extreme - because their 
livelihoods depend on their machinery and equipment being ready and available for rental. 
Machinery and equipment is imported from different countries and have varying lifespan and 
capabilities which must be taken into account in terms of servicing requirements. Other than 
the lack of available spare parts, mentioned earlier, none of the MSCs identified specific or 
recurring problems that were a cause for concern from a technical point of view. 
An additional factor to consider is the extent to which AMP’s interventions, as reported at the 
end of the project, significantly altered the playing field in which MSCs operate. For example, 
AMP has invested around $5.42 million in agricultural machinery through the use of grant funds 
and matching investments, which made it possible for an additional 82 tractors operating in 
target areas, and 235 additional pieces of related equipment (mainly attachable farming 
implements). Furthermore, AMP conducted field demonstrations for MSCs and farmers 
regarding no-till technology and using different types of machinery for different soils. The fact 
that those farmers who attended the FGDs had no major complaints about the operating 
trustworthiness of machinery and equipment can be viewed as testament to the MSCs’ ability 
to adhere to service schedules and manufacturers’ operating instructions. 
At the end of the project, nearly 16,500 farmers were reported to have benefited from the 
provision of increased farm services through AMP’s assistance that largely included using the 
MSCs for plowing, cultivating and seeding at the start of each agricultural seasonal cycle. The 
final testimony to the effectiveness of MSCs management of their hardware assets is that 
farmers using the services of 15 MSCs reported a productivity improvement rate of between 
15% and 70% in crop output as a direct result of having fully functioning machinery and 
equipment at their disposal. 
4.1.3.2 Conclusions 
There is very little evidence to suggest that MSCs are not taking care of their machinery and 
equipment in accordance with manufacturers’ guidelines. As most owners/managers pointed 
out during interviews, not doing so would be highly detrimental to their businesses and 
FINAL PERFORMANCE EVALUATION OF AMP 18 | P a g e
ultimately to their personal livelihoods, since both largely depend on offering professional 
services to the farmers within their respective catchment areas. This fact is supported by 
farmers’ near universal view that MSCs maintain their machinery and equipment to the highest 
standards. It is further supported by the fact that demand for MSC services is so high, they 
must maintain a constant vigil to preserve the proper working order of machinery and tools in 
order for them to be ready and available when needed. The Evaluation Team had the 
opportunity to view stock held by MSCs during its field site visits around the country and, 
noted that, indeed, most machinery and equipment was actually out in the field, with few items 
available for scrutiny (although neither of the experts on the team were mechanical engineers.) 
In addition, at several MSC workshops, the team was able to observe repairs being carried out 
to ready machinery for use. 
4.1.3.3 Recommendations 
In addition to ameliorating the lack of available spare parts needed for repairs, another 
recommendation would be to ensure that MSC technicians and mechanics have access to 
instructions in a language they understand (Georgian, Russian) so they can quickly determine 
what is required. Two suggestions for a future project are to: 1) simply offer a translation 
service and; 2) offer practical training workshops in servicing machinery and equipment hosted 
by trained local mechanical engineers, as referred to above, with instruction manual handouts. 
4.1.4 Question 4: Has the establishment of competing Georgian government 
mechanization centers impacted the MSCs established by AMP? 
Figure 5: QUESTION 4 - SUMMARY OF EVALUATION DESIGN AND METHODS
Evaluation 
Question
Type of Analysis 
Conducted
Data Sources and 
Methods Used
Type and Size 
Sample
Limitations
4. Has the 
establishment of 
competing 
Georgian 
government 
mechanization 
centers impacted 
the MSCs 
established by 
AMP
Description based 
on content analysis 
of primary data
Review of AMP 
M & E records and 
relevant reports
The Evaluation 
Team unable to 
track or verify 
actual performance 
against plan in M&E￾related documents
Description based 
on content analysis 
of expert opinions
Key informant 
interviews with key 
personnel in MSCs
Farmers’ focus 
groups 
All 21 MSCs
16 of the 21 MSC 
areas
None encountered
None encountered
4.1.4.1 Findings 
In overall terms, the Evaluation Team found that the establishment of government FSCs did not 
adversely affect the business activities of most MSCs. Interviews with MSC owners/managers 
indicated that because FSCs and MSCs operated in different segments of the market, there is 
little competition between them. While MSC clients tended to be smallholder farmers with 
land plots averaging less than 2 hectares, FSCs serviced larger farms often with several hundred 
hectares of arable land. Supported by the government, FSCs also tended to have a wider range 
of machinery and equipment better suited to servicing larger land plots with, for example, 
combine harvesters for perennial crops. 
FINAL PERFORMANCE EVALUATION OF AMP 19 | P a g e
MSCs universally attested to the fact that 
they were more flexible than FSCs in 
terms of price and payment. FSCs 
operate with an up-front payment policy 
whereas MSCs offer special terms to 
farmers, particularly those who appear to 
be struggling. Farmers also cited the 
location of some FSCs as problematic 
because, in some cases, they were located 
on main roads outside of towns or 
villages, too far from farms to be useful. 
During its field visits to the MSCs, the Evaluation Team observed, first hand, several of these 
FSCs and indeed took the opportunity to note the types of machinery and equipment in stock. 
In several cases, tractors, harvesters and implements appeared in pristine condition confirming 
comments from MSCs and farmers that FSCs were not created with small farmers in mind 
because some farmers have complained that sometimes even MSC equipment is too large for 
their land plots and the crops they cultivate. 
In addition, farmers clearly indicated during FGDs that the local MSC would always be their first 
stop for agriculture services. Only in particular circumstances would they seek assistance from 
FSCs, for example, when their local MSC was unable to provide specific equipment due to 
unavailability or being under repair or maintenance. However, these tended to be rare 
occasions. While MSCs and FSCs largely served different clientele, some evidence indicates 
that when both centers were located close to one another, a degree of collaboration did exist. 
For example, Akhalkalaki MSC stated that it had a good working relationship with the nearby 
FSC from which it could source spare parts when needed. Similarly, Bolnisi MSC indicated that 
its relations with its local FSC were solid enough to allow both to send clients to one another 
where this was deemed appropriate. 
4.1.4.2 Conclusions 
The Evaluation Team found no evidence to suggest that the establishment of government FSCs 
has negatively affected MSCs. Where direct competition appeared to exist, MSCs believed that 
they were better placed to win the business based on lower prices and offering terms that are 
more flexible for smaller farmers. In reviewing the entire sector, it appears that MSCs and 
FSCs service different client bases. It can therefore be concluded that the existence of FSCs (12 
exist nationwide) does not represent a barrier to growth for MSCs, in large part because 
demand for the services of both far exceeds supply and is likely to remain so for the 
foreseeable future. 
In terms of sustainability, there is sufficient evidence to indicate that with current levels of 
demand (even with limited management skills or financial know-how) all 21 MSCs can look 
forward to a relatively secure future. Finally, although women farmers accounted for less than 
10% of attendees at FGDs, their contribution to local farming communities should not be 
underestimated. Although women explained that they did not experience any sort of 
Typical Government Farm Service Center
FINAL PERFORMANCE EVALUATION OF AMP 20 | P a g e
discrimination in their daily activities, they are under-represented in the industry and more 
could be done in future projects to enhance their opportunities and increase their participation. 
It is worth noting that the current Minister of Agriculture commented during an interview that 
all 12 government-owned FSCs would eventually be privatized (no further information or dates 
provided) because the government has no interest in retaining them. Although no further 
information or dates were provided about the privatization, the Minister mentioned that an 
announcement would be made in due course regarding how the sale process will be conducted. 
4.1.4.3 Recommendations 
Based on review and reflection of the findings and conclusions presented above, the following 
might be worth considering for planning similar projects in the future: 
1. Conduct a brief study on the effect of the impending sale of government-owned FSCs 
on the farming sector. Try to establish in real terms how the sale would impact the 
MSCs with 12 additional larger mechanization centers operating as private sector 
operators. 
2. Conduct a more precise inventory of machinery and equipment in all MSCs and assess 
the extent to which availability is aligned with farmers’ needs. 
In adopting the above, any future project would be better informed regarding the nature of the 
playing field in which it would be a key player. 
4.2 ASSESS THE EFFECTIVENESS OF PROVIDING LARGE GRANTS TO 
PRIVATE ENTERPRISES OR INDIVIDUALS TO ESTABLISH OR EXPAND 
MSCs 
4.2.1 Question 5: Did USG-funded grants mitigate perceived or real financial or 
other investment risks, causing enterprises or individuals to co-invest in the 
project? 
Figure 6: QUESTION 5 - SUMMARY OF EVALUATION DESIGN AND METHODS
Evaluation 
Question
Type of Analysis 
Conducted
Data Sources and 
Methods Used
Type and Size 
Sample
Limitations
5. Did USG-funded 
grants mitigate 
perceived or real 
financial or other 
investment risks, 
causing 
enterprises or 
individuals to co￾invest in the 
project?
Description – based
on content analysis 
of expert opinions
Key informant 
interviews with key 
personnel in 21 
MSCs 
Interviewees 
identified by 
Evaluation Team, 
AMP (ex-team 
members) & USAID
None encountered
Description - based 
on content analysis 
of primary data
Review of AMP M & 
E records & relevant 
reports
All requested 
documents provided
Understanding some 
of the 
documentation in 
terms of clarity and 
relevance
FINAL PERFORMANCE EVALUATION OF AMP 21 | P a g e
4.2.1.1 Findings 
With the exception of the Akhalkalaki MSC, which would have started regardless of AMP’s 
assistance, all other MSCs stated that they would not have been able to begin operations 
without the matching-grant project component. Two thirds indicated that they would not have 
opened at all, while the remainder suggested that they would have started but much later and 
with fewer resources. This information clarifies, without a doubt, the significance of the role 
that AMP matching grants played in establishing the MSCs. MSCs also stated that the availability 
of AMP grants considerably reduced risks. While some MSCs were able to obtain credit from a 
bank for their matching grant contributions, others financed their start-ups and machinery and 
equipment purchases with personal funds. To date, no MSCs have defaulted on their loans, 
many have already paid off the loan, and some have even borrowed additional funds for 
development purposes. 
4.2.1.2 Conclusions 
The AMP matching grant component was the catalyst for the establishment and further 
development of MSCs. Its significance is apparent in AMP’s end-of-project achievements in 
December 2012 as reflected in the performance indicators and targets (see Annex G – ‘Overall 
M&E Matrix’ for full list). In short, without the matching grants, few MSCs would have started 
on their own and the targeted farming communities in the 21 MSC jurisdictions would not have 
benefited from job creation, increased household incomes, or higher crop yields.
Feedback from MSC owners/managers 
and farmers nationwide confirmed that 
economic activity in their respective areas 
has increased as a direct result of AMP 
program interventions, which in turn was
only possible following the establishment 
of MSCs and the matching grant 
component. As a result, AMP’s 
contribution to rural farming cannot be 
underestimated. In real terms, over 
16,500 rural households are estimated to 
have benefited from AMP’s interventions, 
with an approximate $5.33million6
additional annual income accrued by 
farmers over the project’s lifetime. Furthermore, over 2,000 IDP families that also benefited
would have not been able to do so without AMP’s involvement. 
4.2.1.3 Recommendations 
The above findings and conclusions clearly show that matching grants act as a stimulus to rural 
economic development and, in this case, directly contributed to the creation of 21 much 
 
6
 Calculated on the basis of 2 annual farm surveys conducted by AMP from information provided by a sample of farmers in each 
of the 21 MSC areas 
Example of tractors purchased through AMP matching 
grant component
FINAL PERFORMANCE EVALUATION OF AMP 22 | P a g e
needed mechanization centres nationwide. In the future, similar project may consider the 
following: 
1. Employ a similar strategy if the aim is to expand on the achievements of AMP in 
furthering the improvement of the livelihoods of farmers, their families and 
communities. 
2. Ensure, in those areas where IDPs are prevalent, that their needs are adequately 
addressed. IDPs who attended FGDs expressed an improvement in their lives through 
AMP interventions but still lacked market outlets for their products. 
3. Consider initiatives that could enhance the livelihoods of female farmers, some of whom 
are also IDPs and war widows. Take specific actions to help soft loans. 
These suggested measures could build on the progress made to date from AMP’s interventions.
4.2.2 Question 6: How did the establishment or expansion of MSCs harm or help 
competing mechanization service providers including government owned 
centers? 
4.2.2.1 Findings 
Government-run FSCs were established after the MSCs were created (see Question 4 above 
for comments on the reverse). However, since the FSCs were created, there is still no tangible 
evidence that MSCs affect their operating activities. The two client bases are essentially quite 
different and address the needs of different target audiences. For most smaller farmers, the 
MSCs provide most of the machinery and equipment that suits their smaller plots of land 
(usually less than 2 hectares). In some instances there is some evidence to suggest that even 
where farmers use the services of their local MSC this might be restricted to mechanization 
services only if that particular MSC does not offer additional services such as input supplies. In 
this case the farmer will buy such supplies from an FSC if there is one based locally. 
It is difficult to categorize the impact of MSCs on FSCs or other service providers in terms of 
‘help or harm’ as all seem able to meet the demands of their own customers. As mentioned 
above, there appears to be a degree of collaboration between some MSCs and some FSCs 
where competition is not seen as a threat to either. 
Figure 7: QUESTION 6 - SUMMARY OF EVALUATION DESIGN AND METHODS
Evaluation 
Question
Type of Analysis 
Conducted
Data Sources and 
Methods Used
Type and Size 
Sample
Limitations
6. How did the 
establishment or 
expansion of 
MSCs harm or 
help competing 
mechanization 
service providers 
including 
government 
owned centers?
Description – based 
on content analysis 
of expert opinions
Key informant 
interviews with key 
personnel in 21 
MSCs 
Interviewees 
identified by 
Evaluation Team, 
AMP (ex-team 
members) & USAID
None encountered
Description based 
on content analysis 
of primary data
Review of AMP M & 
E records & relevant 
reports
All requested 
documents provided
Understanding some 
of the 
documentation in 
terms of clarity and 
relevance
FINAL PERFORMANCE EVALUATION OF AMP 23 | P a g e
In terms of whether or not other local service providers were affected by the emergence of 
MSCs there appears little evidence to prove or disprove this hypothesis as the Evaluation Team 
did not meet with any other local service provider. According to MSCs, farmers did not want 
to use local service providers because their equipment was generally believed to be old, 
outdated, and in poor working order. In any event, the fact that demand for mechanization 
services far surpassed supply suggests that there exists enough business for all providers, even 
those offering limited services. The main criteria that farmers adopt regarding which service 
provider to use is largely based on availability of the right equipment at the right time and the 
right price. This fact was frequently mentioned during FGDs because farmers cannot miss the 
window of opportunity before each season for preparing and seeding the land. Most 
smallholder farmers cultivate and harvest non-perennial, horticulture crops such as tomatoes, 
beans, cabbage etc. – therefore timeliness is vital. 
4.2.2.2 Conclusions 
The main conclusion drawn is that MSCs have not interfered with free market competition. 
Demand in the marketplace for both small and large farmers remains high and is still significantly 
greater than supply. Informed commentators have calculated that there would be enough room 
in the market to create up to 200 more mechanization centers. One main weakness of the 
current MSCs is that most do not offer extension services nor agricultural input supplies, the 
availability of which would help farmers address needs not associated with mechanization. For 
example, soil testing and crop diversification were issues often raised at FGDs together with 
acquiring a better understanding of use of fertilisers and pesticides. Marketing was another 
issue raised as was irrigation and the current state of existing irrigation systems, many of which 
were deemed as ineffective or useless. Both MSCs and farmers complained about the effects 
on their livelihood of cheap Turkish imports (mainly horticulture crops) which, according to 
them, were often dumped on Georgian markets at below cost simply to obtain market share. 
While not directly related to this evaluation, the Minister of Agriculture did acknowledge these 
concerns and explained that the government has plans to address the situation in due course. 
4.2.2.3 Recommendations 
There is clear evidence that expanding AMP’s assistance to establishing MSCs in the design of a 
similar long-term agriculture project in the future would be a positive step toward rural 
economic development in Georgia. Such a project should: 
1. Include a component to expand the MSC network nationwide based on locations most 
likely to benefit from such centers. 
2. Conduct research to identify the optimum range of services that MSCs should offer 
prospective clients based on the findings of this evaluation report. 
3. Collaborate closely with other international donors supporting agriculture in Georgia –
i.e. Mercy Corps has been operating in the country since 2000 and has 3 on-going 
projects related to agriculture (livestock and value chain) – to avoid duplication of effort. 
They are already interacting with some of the MSCs so need to be consulted. 
FINAL PERFORMANCE EVALUATION OF AMP 24 | P a g e
4. Review the MoA’s forthcoming long-term agriculture strategy for Georgia and identify 
gaps that need to be filled that fit within the context of the strategy.
5. Collaborate with the Georgian Farmers’ Associations, which acts as a farmers’ lobby 
between the private sector and government. It plays an integral role in the country 
regarding all aspects of agriculture and serves as a donor coordination medium to 
ensure the focus of international interventions. 
These measures should significantly contribute towards gaining a more in-depth understanding 
of the agriculture sector in Georgia and help focus future interventions that can make a 
meaningful contribution toward rural economic development in the country. 
4.3 ASSESS WHETHER OR NOT SMALLHOLDER FARMER PRODUCTIVITY 
AND INCOMES HAVE IMPROVED AS A RESULT OF TRAINING 
PROVIDED BY THE PROJECT AND ACCESS TO AND UTILIZATION OF 
MECHANIZATION SERVICES
4.3.1 Question 7: Did training and utilization of mechanization services contribute 
to an increase in collective household incomes? 
4.3.1.1 Findings 
One of the main difficulties in answering this question is the need to correlate MSC activities 
with increases in household income, particularly as a result of training and improved working 
practices by farmers, leading to increased productivity. As mentioned earlier in the report, it 
has been estimated that collective household incomes for small-scale farmers increased by 
$5.33 million by the end of AMP in December 2012. If this figure is divided by 16,539 
households that reportedly benefited from AMP’s interventions, this is an average amount of 
$322 per household which, although not a substantial amount, nevertheless helps to alleviate 
some of the financial difficulties of some of the smaller farmers which of course is one of the 
aims of the project. Additionally, farmers’ incomes depend on more than one factor including 
the weather, crop prices and market demand, among others. What appears to be known 
however is that farmers most certainly associate MSCs with positively influencing their lives by 
providing mechanization services and trainings, as well as by contributing to increases in their 
household incomes, although they were unable to quantify the extent of those increases. 
Figure 8: QUESTION 7 - SUMMARY OF EVALUATION DESIGN AND MEHODS
Evaluation 
Question
Type of Analysis 
Conducted
Data Sources and 
Methods Used
Type and Size 
Sample
Limitations
7. Did training & 
utilization of 
mechanization 
services 
contribute to an 
increase in 
collective 
household 
incomes?
Description – based 
on content analysis 
of expert opinions
Focus group 
sessions in 16 MSC 
locations
Interviewees 
identified by 
Evaluation Team, 
AMP (ex-team 
members) & USAID
None encountered
Description based 
on content analysis 
of primary data
Review of AMP M & 
E records & relevant 
reports
All requested 
documents provided
Understanding some 
of the 
documentation in 
terms of clarity and 
relevance
FINAL PERFORMANCE EVALUATION OF AMP 25 | P a g e
4.3.1.2 Conclusions 
From the evidence gathered during the evaluation mission it is clear that AMP made a 
contribution towards improving the lives of farmers in those rural communities where MSCs 
were established. Although two project surveys were conducted in an effort to determine the 
exact nature of increases in farmer household incomes, precise data cannot be quantified due 
to the existence of other influencing factors. Conversely, what appears evident is that had the 
MSCs not been established, farmers would, in all likelihood, not have experienced any rise in 
income due to the competition from cheap foreign imports and the continued lack of access to 
agricultural input supplies. 
4.3.1.3 Recommendations 
One of the major considerations for any future USAID intervention in agriculture in Georgia is 
to widen the scope of MSCs to potentially include extension services and input supplies. 
Access to both would significantly increase the knowledge base for farmers so that more 
informed decisions could be made regarding new technologies, crop diversification and 
marketing. The following is therefore suggested: 
1. Assist the current 21 MSCs expand their services in line with farmers’ demands. 
2. Create MSCs in new locations based on the matching grant proposition to include 
extension and /or input supplies, i.e. entire value chain. 
3. Develop a set of criteria that can correlate a project-related action or activity with a 
measureable result for monitoring and evaluation purposes. 
4.4 IDENTIFY OPPORTUNITIES TO IMPROVE IMPACT AND ENHANCE 
THE IMPLEMENTATION AND MANAGEMENT OF SIMILAR PROJECTS 
BY ACTIVELY ADDRESSING ISSUES SUCH AS GENDER EQUALITY 
AND SUSTAINABILITY 
4.4.1 Sustainability 
There are many factors that can influence whether or not a business can be sustainable or not 
and MSCs are no exception. The Evaluation Team developed a number of criteria to calculate 
the probability of MSCs’ sustainability including an analysis of financial statements, market 
demand for products and services, the timely repayment of bank loans by MSC owners, and 
employment (see results in Annex F – ‘Overview of the Evaluation Team’s Mission Findings’). 
Following a thorough review of these criteria for each MSC, the Evaluation Team determined
that all should have little difficulty remaining profitable in the short- to medium-term. MSC 
sustainability owes primarily to the fact that in the current trading conditions, market demand 
for agriculture services across the board far exceeds available supply, and less to the abilities of 
owners or other key personnel to manage their businesses effectively. This is particularly true 
with respect to the almost complete lack of MSC financial management even after they were 
required to submit quarterly reports to AMP focusing on cash-flow performance. In addition, 
although an F2F volunteer assisted MSCs to develop financial reporting skills to prepare cash￾flow statements, balance sheets, and profit and loss accounts, the evidence suggests that most 
MSCs still do not understand nor are capable of preparing or even using this information as a 
decision-making management tool for their enterprises. While this fact does not appear to 
FINAL PERFORMANCE EVALUATION OF AMP 26 | P a g e
affect immediate sustainability because of strong market demand, longer-term sustainability 
could be compromised for those MSCs that carry out no form of forecasting as markets 
become increasingly competitive.
4.4.2 Gender Quality 
Feedback from female farmers during FGDs indicated that they did not experience any specific 
discrimination in their daily activities. However, a future project should further increase
women’s participation by offering a wider range of technical support including soft grants, in 
particular for IDP war widows. Although women represented only 9% of the farmers who
attended FGDs, their views are relevant and important for rural communities. In some cases,
women are the only breadwinner in the family so their needs must be addressed in any future 
long-term agriculture project. In the few cases where the women were war widows, addressing 
their needs in a future project assumes even greater relevance as they are de facto more 
disadvantaged than other farmers due to circumstances beyond their control. War widows told 
the Evaluation Team that they need a voice to represent their interests as they lack the basic 
skills to run a business by themselves. Survival in the short-term may be assured but in the 
longer term they will surely require more support. At the present time USAID supports female 
farmers through its Economic Prosperity Initiative (EPI) and New Economic Opportunities 
(NEO) projects. 
The Evaluation Team was not aware of any specific on-going initiatives providing support to 
female farmers, so this should be pursued at the design stage of any further USAID 
interventions in Georgia. 
5.0 LESSONS LEARNED
Below are several lessons learned from AMP: 
1. During the project design stage there needs to be a closer alignment between the 
setting of quantifiable target indicators and a verifiable means of measuring their 
achievement, i.e. ensure that correlation exists between one and the other and that 
performance can be monitored by data gathering methods that are understandable and 
easy to analyze.
2. Consideration should be given to monitoring the performance of target indicators that 
can be measured by using ‘at least’ to be achieved, rather than providing a range of 
parameters which can sometimes lead to the manipulation of statistics to fit within 
those parameters (although there is no evidence that the AMP team adopted this 
practice.)
3. It is advisable to clarify and specify the precise M&E system that should be adopted for 
reporting purposes to USAID for the duration of the project (simplify the M&E
procedure to be one document per reporting period supported by one Excel 
spreadsheet (actual versus plan)).
4. Where project beneficiaries are required to submit supporting documentation as 
evidence of performance, ensure that project management has a specific rationale and 
FINAL PERFORMANCE EVALUATION OF AMP 27 | P a g e
strategy for this information or data, so that feedback is provided to those beneficiaries 
on future actions required should any deviation from plan occur. 
5. When considering sustainability as a measurement of success, identify the criteria to be 
used in advance so that proper reporting systems can be put in place to track progress.
6. Establish an awareness of what other international donors, multilaterals and NGOs are 
doing on the ground in agriculture to avoid duplication of effort and resources and, in 
particular, to avoid delivering similar services to the same clients or beneficiaries.
7. Long-term technical assistance projects should be in tune with the host government’s 
long-term strategy for the agriculture sector. This is vital to ensure that priority areas 
are addressed in line with government policy and that intended beneficiaries receive the 
type and level of support that has been identified as important and relevant.
FINAL PERFORMANCE EVALUATION OF AMP 28 | P a g e
ANNEXES
1
ANNEX A: STATEMENT OF WORK
2
STATEMENT OF WORK 
Performance Evaluation of Access to Mechanization Project
Summary
Name of the Project: Access to Mechanization Project (AMP) 
Project Numbers: Associate Cooperative Agreement No.AID-114-LA-09-00001 
Project Dates: September 29, 2009 – December 1, 2012 
Project Funding: $5,110,607 
Implementing Organization/s: CNFA Agreement Officer’s Representative (AOR): David 
 Tsiklauri 
USAID/Caucasus plans to conduct a performance evaluation of the Access to Mechanization Project 
(AMP). The purpose of evaluating this project is to assess its contributions toward achieving the 
USAID/Caucasus’s Development Objective - “Inclusive and Sustainable Economic Growth.” Specifically,
the goal of evaluating AMP is to assess the effects of the program on increasing productivity and incomes 
for smallholder farmers by improving access to machinery services. The USAID/Caucasus Economic 
Growth Office is currently changing its portfolio of activities and the results of this evaluation will be 
used to design future projects and adjust current projects if needed. Therefore, this evaluations is to 
identify “lessons learned,” assess strengths and weaknesses of strategies and activities performed under 
these projects, and provide recommendations to USAID for project planning purposes for the next five 
years. 
Summary of Specific Technical Requirements
The Contractor shall: 
 Provide draft evaluation design and work plan for review and comment prior to arrival 
 in country. 
 Meet with USAID within three days of arrival in country and provide deliverables (final 
 detailed evaluation design and the work plan). 
 Conduct evaluations in accordance with the USAID-approved evaluation design and 
 the work plan. 
 Provide evaluation reports to USAID in accordance with Reporting Guidelines. 
 Meet with USAID for out brief. 
 Develop draft and final evaluation reports. 
1. Activities to be evaluated
3
1.2 In September 2009, USAID/Caucasus awarded the Access to Mechanization Project to CNFA as an 
associate award under John Ogonowski and Doug Bereuter Farmer-to-Farmer Program Leader with 
Associates Cooperative Agreement EDH-A-00-08-00019, which will end in December 1, 2012. The 
ultimate objective of the project was to increase productivity and incomes for smallholder farmers by 
improving access to machinery services. 
2. Background
Access to Mechanization Project
During its implementation, the AMP project used a combination of matching investments, leveraged 
commercial finance, business and extension training, and volunteer technical assistance was to establish 
21 Machinery Service Centers (MSCs). The primary goal of the AMP was to increase the productivity 
and incomes of smallholder farmers by improving access to machinery services. The project had the 
following specific objectives: 
 Establish 21 MSCs offering improved mechanization services to over 12,500 smallholder farmers. 
 Provide technical assistance to these enterprises, using a combination of local consultants and 
Farmer-to-Farmer (FtF) volunteers, to ensure the safe, sustainable and profitable operation of the 
businesses and long-term availability of services for farmers. 
 Leverage USAID grant funds with local partner matching investment, including large- scale 
involvement of local and international commercial finance, to maximize investment in the rural 
economy.
 Improve the competitive environment for machinery services by reducing the cost to farmers, as a 
result of increased supply of machinery and the availability of dedicated service providing businesses. 
 Facilitate the return of more agricultural land into production. 
 Contribute to increased productivity and income of smallholder farmers. 
 Provide discounted machinery services to about 3,000 resettled Internally Displaced Persons (IDPs). 
The project assisted 21 MSC’s with an array of agricultural machinery, using a combination of grant 
funding and local partner matching investment (with a minimum match requirement of 1:1). AMP 
provided the assortment of machinery and implements responsive to the needs in each MSC’s target 
market, with an appropriate mix of tractors of various horsepower and implements required to serve 
the area’s main crops. Flexibility was crucial to ensure the commercial viability of each MSC. 
Matching grants were complemented by rigorous business and management trainings; extension 
trainings, field days and demonstrations; and technical assistance provided by local consultants and FtF 
volunteers. AMP used this combination of financing, business training, and technical assistance to ensure 
sustainability and proper management of the enterprises providing machinery services. 
Through the implementation of the AMP project, 215 new jobs were created and 84 units of agricultural 
machinery and 238 implements were purchased with combined grant funding and matching investment. 
During the life of the AMP project cumulatively more than 2,400 farmers participated in extension 
trainings and 400 – 1,000 farmers have been served by each assisted service center with different 
agricultural services resulting in increased household incomes for MSC beneficiaries by more than $6.25 
4
million. 
3. Purpose of the evaluation and key evaluation questions to be addressed
 Access to Mechanization Project
The final evaluation will assess the results of AMP in terms of establishing or expanding 21 
mechanization service centers (MSCs), and increasing the incomes of at least 12,500 small farmer 
beneficiaries. The evaluation will cover the life of the project from September 2009 through November 
2012. The conclusions of the evaluation will be used by USAID/Caucasus to design future interventions 
in the area of agriculture sector development. With similar purpose, USAID will share the results of the 
study with other donors, host country government stakeholders and partners working in this area. The 
evaluation team should review and summarize the implementation and results achieved by this project 
to address the following purposes and answer the following key evaluation questions.
1. Assess the operating condition of MSCs. 
Q. Can each MSC manage its business operations (e.g., positive cash flow from 
operations, record keeping, designation of employee duties, staff coverage, sales 
 and marketing capacity) to be evaluated against standards determined by the 
 evaluators? 
Q. Can each MSC operate and maintain its equipment, and can such equipment be 
 repaired by MSC operators or appropriate service providers? 
Q. Are the maintenance and repairs of equipment regularly funded by MSCs in 
 accordance with applicable service schedules and repair needs? 
Q. Has the establishment of competing Georgian government mechanization 
Centers impacted the MSCs established by AMP? 
2. Assess the effectiveness of providing large grants to private enterprises or individuals to 
establish or expand MSCs; and assess whether or not grant assistance caused market distortions 
by answering the following two questions: 
Q. Did USG-funded grants mitigate perceived or real financial or other investment risks, 
causing enterprises or individuals to co-invest with the project? 
Q. How did the establishment or expansion of MSCs harm or help competing 
 mechanization service providers, including Government of Georgia owned 
 centers? 
3. Assess whether or not smallholder farmer productivity and incomes have improved as a result 
of training provided by the project, and access to and utilization of mechanization services. 
Q. Did training and utilization of mechanization services contribute to an increase in 
 collective household incomes? 
5
4. Identify opportunities to improve impact and enhance the implementation and management of 
future similar projects such as gender equity and sustainability. 
4. Methodology
The Methodology will be used as proposed in the Mendez England & Associates Technical Proposal as of 
January 31, 2013. 
5. Work Location
Tbilisi and selected Georgian regions and the U.S. 
The teams will travel outside the capital as needed (for AMP to all 8 regions with MSCs: Kakheti, Shida 
Kartli, Kvemo Kartli, Samtskhe-Javakheti, Imereti, Racha, Samegrelo, and Guria) in order to meet with 
key players in diverse parts of the country and to get a better sense of the overall context within 
Georgia. 
6. Projects Documents for Review 
The COR, through the Mission’s Economic Growth office and respective projects AOR and COR will 
put the contractor in contact with its implementing partner and might provide help with a small number 
of meetings (such as meeting with USG agencies). To the extent possible, relevant reports and other 
project documentation will be provided by the Mission to the contractor prior to travel to Georgia. 
These documents are: 
1. Program Description for AMP 
2. AMP annual and quarterly reports 
3. AMP work plans 
4. Studies/assessments produced under both projects. 
5. Other projects documents which will be provided by the EG office in Tbilisi 
The Mission’s EG Team will brief the evaluation team on their perceptions of political dynamics. 
6
ANNEX B: WORK PLAN, EVALUATION DESIGN & 
METHODOLOGY
7
Work Plan, Evaluation Design and Methodology 
1. INTRODUCTION
In the preparation of this Work Plan and Evaluation Research Design, the Evaluation Team has 
followed the guidelines outlined in USAID’s Request for Task Order Proposal (RFTOP) and 
accompanying Scope of Work (SOW) (Annex 1), together with ME&A’s Technical Proposal, for 
the performance evaluation of the ‘Access to Mechanization Project’ (AMP) implemented in 
Georgia between September 2009 and December 2012 
In overall terms, the purpose of evaluating this project according to the SOW is, ‘to assess its 
contribution toward achieving the USAID/Caucasus’s Development Objective – “Inclusive and 
Sustainable Economic Growth”. The main goal of evaluating AMP is to determine the effects of the 
program on increasing productivity and incomes for smallholder farmers by improving access to 
machinery services. 
The main thrust of the Evaluation Team’s mission is to determine the extent to which the AMP 
was successful in establishing or expanding 21 mechanization service centers (MSCs) and 
increasing the incomes of at least 12,500 small farmer beneficiaries. In pursuit of these findings 
the Evaluation Team (see 2. below) is further tasked with answering a specific set of evaluation 
questions posed by USAID/Caucasus (Annex 2). The evaluation will cover the duration of the 
AMP from September 2009 – December 2012. Finally, the conclusions reached and suggested 
recommendations will be used by USAID/Caucasus to help design future interventions in the 
field of agriculture sector development. Additional attached Annexes (3 to 9) include a revised 
Mission Schedule, List of Stakeholder Interviewees, Evaluation Design Chart, Evaluation Design 
& Methodology Matrix, Overall Monitoring & Evaluation Matrix, Interviewee Profile & Illustrative 
Questions and Draft Outline of the Final Evaluation Report (Table of Contents). 
2. EVALUATION TEAM 
The evaluation of AMP will be conducted by a team of two experts: international expert Mr
Colin Maclean (Team Leader) and local expert Dr Gaga Nikabadze (Agribusiness Specialist). In 
addition, the team will be assisted by Ms Marika Gorgadze as interpreter. Colin Maclean will 
assume overall responsibility for the overall management of the evaluation in collaboration with 
USAID/Caucasus. In short, this encompasses all activities specified in the Mission Schedule 
(Annex 3) related to the evaluation mission, including pre-mobilization, on-site implementation 
and end-of-assignment deliverables. Gaga Nikabadze will also contribute to the evaluation 
mission by assisting in the scheduling of meetings, conducting interviews and carrying out 
additional research as identified by the Team Leader. He will further participate in the planning, 
organization and implementation of regional field visits including interviews and focus group 
sessions (where deemed appropriate) as well as contributing towards the preparation of the 
draft and final evaluation reports. Finally, oversight of the evaluation mission will fall under the 
remit of Ms Mirela McDonald, Evaluation IQC Manager with ME&A and Ms Audra Stark, Project 
Manager with ME&A. 
8
3. EVALUATION TASKS and SUB-TASKS 
3.1 Pre-Mobilization Activities 
Conference Calls 
On Monday, 1 April 2013, a conference call took place between Meredith Kummings (ME&A), 
Colin Maclean (Team Leader), and (USAID/Caucasus). A number of issues were discussed 
including clarification of the evaluation questions to be answered, the contact details for key 
personnel who managed AMP, the Co-operative Agreement adjusting some of AMP’s indicator 
targets, the location of the MSCs by region and the organization of focus groups in the regions 
in order to ensure adequate coverage of a representative sample of farmers and their views 
regarding AMP. Following the conference call, a document - Modification 02 (AID-114-LA-09-
00001) - was forwarded to the Team Leader together with a copy of AMP’s grantees with 
contact details. It was agreed that any outstanding issues would further be discussed at the in￾briefing with USAID scheduled for Thursday, 11 April 2013. 
Literature Review 
Both members of the Evaluation Team were provided with a selection of project-related 
documentation by ME&A, all of which were reviewed prior to the in-country start of the 
mission. These included AMP Work Plans, Annual Reports, Quarterly Reports, PMPs and 
Training Materials among others. The documents reviewed provided a useful insight into the 
planning and operating activities of AMP and will be the source of reference for the Evaluation 
Team for data and information related to project objectives, indicators and targets. 
Preparation of Draft Work Plan and Evaluation Design 
This activity was completed prior to the mobilization of the Team Leader in collaboration with 
the local expert and ME&A Evaluation Project Manager. Minor adjustments were made to the 
Work Plan following the Team Leader in-briefing with USAID in Tbilisi. 
3.2 In-Country Activities 
Initial Meetings
On Wednesday, 10 April 2013, a meeting was held between the Evaluation Team members –
Colin Maclean, Team Leader and Gaga Nikabadze, Agribusiness Specialist – and 
USAID/Caucasus in Tbilisi. This was essentially an in-briefing for both parties to review the 
evaluation’s objectives and the draft Work Plan. Issues regarding the proposed methodology to 
be used for data gathering and analysis were discussed together with the proposed list of site 
visits and content of the Final Evaluation Report. The Evaluation Team had two days to 
incorporate any requested amendments before final submission of the Work Plan on Friday, 12 
April 2013. 
 Data Gathering 
9
Meetings with stakeholders began on Thursday,11 April, 2013, and will continue through to 
Monday, 22 April, 2013. These will include not only AMP’s direct beneficiaries but a cross￾section of other relevant individuals and organizations with an interest in the project’s activities 
and outcomes – (see Annex 4 for suggested list of interviewees). In some instances, required 
information will be sourced via the adoption of quantitative research methods (review and 
analysis of facts that can be statistically verified from recorded data e.g. AMP records such as 
PMP’s and Annual Reports). In other cases, qualitative research methods (opinions, observations 
and explanations gathered from participating interviewees) will be employed to act as a support 
mechanism (or otherwise) quantitative i.e. numerical data – (see Section 5. below for a more in￾depth explanation of the Evaluation Team’s planned methodology). 
Data Analysis 
This activity involves digesting and analyzing data and information gathered during the in-country 
mission. Findings will be tabulated both within the body of the final report (see Annex 5 –
Evaluation Design & Methodology Matrix) which will summarize the qualitative aspect of the 
evaluation and as an attachment to the main report (see Annex 7 – Overall Monitoring & 
Evaluation Matrix) which will address the quantitative elements of the evaluation. Section 5 
below further elaborates on both approaches. On Wednesday, 24 April 2013, the Evaluation 
Team will have an out-briefing with USAID/Caucasus during which initial findings will be 
presented in summary format together with some of the key issues arising from the evaluation. 
At this stage, comments and/or suggestions offered by USAID/Caucasus will be acknowledged 
and addressed in the Draft Final Report. 
3.3 End-of-Mission Activities 
Completion and Submission of Draft Report to USAID 
On completion of the in-country mission and following the Team Leader’s return to home base, 
a Draft Final Report will be prepared and submitted to USAID/Caucasus on Thursday, 2 May, 
2013. 
Final Report submitted to USAID with integrated comments 
No later than Monday, 20 May, 2013, the Team Leader will receive their comments for 
integration into the Final Report which will subsequently be re-submitted to USAID/Caucasus 
on Thursday, 23 May, 2013 
4. EVALUATION DESIGN PLAN 
In the Technical Proposal that ME&A submitted to USAID/Caucasus for the AMP evaluation an 
illustrative Evaluation Design chart was included (below) which highlighted the approach and 
sequence of events that the Evaluation Team would undertake in implementing the assignment 
i.e. Development of Methodology – Data Collection – Results Analysis and Reporting. This 
remains our underlying approach for this evaluation. 
5. METHODOLOGY 
1
0
The Evaluation Team recognizes that the credibility of an evaluation’s findings, conclusions and 
recommendations rests for the most part on the quality of the research design as well as data 
collection methods and analysis used. Given the wide scope of activities undertaken by AMP 
only a combination of both quantitative and qualitative research methods will suffice if a clear 
picture is to emerge as to how AMP performed over the lifetime of the project. 
In the Pre-Mobilization Activity stage of the evaluation process during which the Evaluation 
Team conducted an in-depth review of the documents made available to them, it became 
apparent that the evaluation mission would be addressing two separate though inter-connected 
issues: first, what AMP was tasked with achieving from a quantitative point of view e.g. number 
of MSCs created, number of jobs created, increases in farmers’ incomes etc., and second, the 
extent to which the accomplishment of these numeric parameters impacted on the intended 
beneficiaries (in this case smallholder farmers and farming communities throughout the regions 
in Georgia). The following approach is therefore how the Evaluation Team envisages tackling the 
evaluation process. 
5.1 Quantitative Research and Analysis 
In summary, quantitative data (e.g. number of actions / activities to be accomplished against plan) 
will be sourced from AMP Annual Work Plans and other project-related periodic reports. In 
addition, particular reference will be made to the project’s annual Performance Monitoring Plans 
(PMPs) designed to provide ongoing analysis and commentary on actual results achieved against 
the plan including actions and strategies that were taken to address anticipated deviation from 
the plan. The collective outcome of this evaluation will be a thorough assessment of the 
performance of AMP at the end of the project relative to the planned objectives as envisaged in 
the original SOW and elaborated and/or revised in subsequent AMP Annual Work Plans. 
In this instance, as the project was tasked with accomplishing a significant number of target 
indicators7 (as highlighted in AMP’s End-of-Project Report submitted by CNFA) verification of 
performance from a statistical point of view can essentially only focus on a review of AMP 
records which the Evaluation Team can only assume accurately reflects whether a numeric 
indicator was achieved or not. We will therefore comment on this within the body of the final 
report in the relevant sections supported by an annex for tabulated reference purposes (see 
Annex 7 – Overall Monitoring & Evaluation Matrix in this Work Plan as a proposed format). The 
findings here will be cross-referenced with findings accumulated from our qualitative research 
approach to determine the extent to which evidence gathered contributes towards the 
Evaluation Team’s understanding of AMP’s impact on its intended beneficiaries. This in turn will 
enhance the team’s prospect of being able to provide satisfactory and meaningful answers to 
USAID’s pre-determined evaluation questions. 
5.2 Qualitative Research and Analysis 
 
7
IDP-Specific Indicators; Project Indicators – Programmatic; Project Indicators – MSC Performance; Project Indicator – Farmer 
Impact
1
1
The required approach under this activity assumes particular importance given the geographic 
spread of AMP’s direct and indirect beneficiaries and the time available to the Evaluation Team 
to conduct its enquiries. Here, the team’s approach will be to identify, locate and meet with as 
representative a body as possible with direct or indirect knowledge and/or experience of the 
project throughout its lifetime (see Annex 4 for a proposed list of potential interviewees). In 
effect, there is less concern here as to whether or not a particular action has been implemented 
or not (verifiable or not from AMP records) as opposed to whether the result of that action 
met the needs and aspirations of the intended recipients. 
In so doing, the Evaluation Team will pay particular attention to avoiding subjective opinion and 
hearsay as these effectively add little or no value to understanding the facts. On the other hand, 
perceptions are valid and will be included in the team’s findings as they may enhance the quality 
of recommendations for any future planned interventions by USAID/Caucasus. In order to 
effectively carry out the qualitative research needed the following approach is suggested 
regarding the Evaluation Team’s in-country activities the analysis of which will be elaborated in 
the body of the Draft Final Report in the proposed format (see Annex 6 - Summary of 
Evaluation Design & Methodology). Here, in line with USAID’s published TIPS practical advice 
and suggestions for constructing an evaluation report, we address each evaluation question one￾by-one depicting how the relevant research was conducted together with any identified 
limitations. The accompanying text outlines the Evaluation Team’s findings, conclusions drawn, 
and then recommendations suggested emanating from both: 
1. MSCs 
The Mission Schedule prepared for this evaluation originally indicated site visits to 4 
regions although there are in fact 8 regions in Georgia. This was re-visited at the in￾briefing and the evaluation team’s recommendation to visit all 21 MSCs was accepted by 
the Mission. This change will ensure the coverage needed to draw meaningful 
conclusions. 
 Data Gathering Approach: Structured interview with MSC key personnel
2. Smallholder Farmers
Given the number of farmers benefiting (12,500) from the provision of increased farm 
services made available as a result of project assistance, focus group sessions will be 
needed in all regions (subject to participants’ attendance) in order to acquire a 
representative opinion of AMP’s performance. 
Data Gathering Approach: Focus group sessions in all regions subject to enough 
participants attending 
3. Internally Displaced Persons (IDPs)
Similarly to farmers, the project was to assist the number of IDP families benefiting from 
improved mechanization (2000-3000) which means that to reach some sort of 
1
2
representative sample face-to face interviews with individuals is neither practical nor 
realistic. 
Data Gathering Approach: Focus group sessions in all regions subject to enough 
participants attending
 4. Central and Local Government
Needed to gather feedback from the political arena as to how AMP was perceived 
during its lifetime and to date. 
 Data Gathering Approach: Face-to face interviews with identified parties
5. Financial Institutions 
Due to their involvement in disbursement of funds through various aspects of the 
project (grants and investments) the views of the banks in question will add value to the 
Evaluation Team’s understanding of the business performance of AMP beneficiaries.
Data Gathering Approach: Direct interviews with key personnel involved
6. Industry, Business and Farming Associations 
Clearly have a role to play in representing the interests of members so their views will 
add value to the bigger picture. 
Data Gathering Approach: Direct interviews with key personnel familiar with AMP 
7. International Community 
The effects and impact of AMP will be known to various members of the international 
donor community with representative offices in Georgia (World Bank, UN, EC, GIZ, 
Mercy Corps etc.) and their views will clearly enrich the Evaluation Team’s 
understanding of how AMP was/is perceived today. 
Data Gathering Approach: Direct interviews with key personnel familiar with AMP 
5.3 Limitations 
Every evaluation poses its own inherent limitations as to what can realistically be carried 
out within the scope of the mission and this assignment is no exception. Below are a 
number of issues that need to be taken into account that could impact on the required 
results being achieved: 
1. The evaluation will take place during planting season for farmers throughout 
Georgia so their availability and/or willingness to attend interviews and/or 
meetings needs to be taken into account 
1
3
2. Many of the MSCs were established during the later stages of the AMP project 
which effectively means that the effects of their impact on local farming 
communities will only be apparent in the coming months and in some cases 
years following the end of the project. The evaluation team will therefore not 
be able to comment on the real impact and/or contribution to, for example, job 
creation or how household incomes have been affected post-project. 
3. One of the inherent difficulties in the AMP evaluation concerns the time 
constraints within which the team is tasked with carrying out its duties. 
However, having identified who the team wants to meet and where, it is 
confident that the required coverage of potential interviewees will be met. 
Nevertheless, there remains the risk that requested individuals and 
organizations may not be available at relatively short notice. This could 
negatively impact on findings. 
4. In terms of the quantitative analysis falling within the scope of the AMP 
evaluation it should be noted that as the project has now ended and been signed 
off by USAID, the Evaluation Team will not comment on the achievement or 
otherwise of performance indicator targets. Instead, it will limit its remarks to 
offering comment only on the data gathering process and the extent to which it 
contributed (or otherwise) to the project’s self- monitoring and evaluation. 
However, qualitative analysis will be performed, including reviewing the 
indicator data reported in AMP’s PMP to help answer “why” the interventions 
worked or did not work and to assess the extent to which AMP positively or 
negatively impacted on Georgia’s economy (e.g. increased productivity, market 
distortion, etc.). 
5. Some of the USAID evaluation questions regarding the maintenance of 
machinery and equipment strongly indicate that the Evaluation Team is able to 
some extent to view some of these items. However, it now being planting 
season there is some question as to whether or not tractors and planting 
machinery will be available for viewing given that they may well be out in the 
fields already. This could therefore limit the team’s understanding of the true 
state and condition of some of those items 
6. PROPOSED SITE VISITS AND MEETINGS 
Further to the above-mentioned research methodologies and following discussions with 
CNFA and USAID at the in-briefing session for the Evaluation Team, there has been a 
re-alignment in the schedule devised for the proposed field site visits – details outlined 
in the revised Mission Schedule (Annex 3). In short, the Evaluation team will travel as 
one unit in order to optimize time in-country and to facilitate the preparation of the 
Final Draft Report at the end of the mission due to the tight schedule.
1
4
STATEMENT OF WORK 
Performance Evaluation of Access to Mechanization Project
Summary
Name of the Project: Access to Mechanization Project (AMP) 
Project Numbers: Associate Cooperative Agreement No.AID-114-LA-09-00001 
Project Dates: September 29, 2009 – December 1, 2012 
Project Funding: $5,110,607 
Implementing Organization/s: CNFA Agreement Officer’s Representative (AOR): David 
 Tsiklauri 
USAID/Caucasus plans to conduct a performance evaluation of the Access to Mechanization Project 
(AMP). The purpose of evaluating this project is to assess its contributions toward achieving the 
USAID/Caucasus’s Development Objective - “Inclusive and Sustainable Economic Growth.” Specifically,
the goal of evaluating AMP is to assess the effects of the program on increasing productivity and incomes 
for smallholder farmers by improving access to machinery services. The USAID/Caucasus Economic 
Growth Office is currently changing its portfolio of activities and the results of this evaluation will be 
used to design future projects and adjust current projects if needed. Therefore, this evaluations is to 
identify “lessons learned,” assess strengths and weaknesses of strategies and activities performed under 
these projects, and provide recommendations to USAID for project planning purposes for the next five 
years. 
Summary of Specific Technical Requirements
The Contractor shall: 
 Provide draft evaluation design and work plan for review and comment prior to arrival 
 in country. 
 Meet with USAID within three days of arrival in country and provide deliverables (final 
 detailed evaluation design and the work plan). 
 Conduct evaluations in accordance with the USAID-approved evaluation design and 
 the work plan. 
 Provide evaluation reports to USAID in accordance with Reporting Guidelines. 
 Meet with USAID for out brief. 
 Develop draft and final evaluation reports. 
1. Activities to be evaluated
1.2 In September 2009, USAID/Caucasus awarded the Access to Mechanization Project to CNFA as an 
associate award under John Ogonowski and Doug Bereuter Farmer-to-Farmer Program Leader with 
Associates Cooperative Agreement EDH-A-00-08-00019, which will end in December 1, 2012. The 
1
5
ultimate objective of the project was to increase productivity and incomes for smallholder farmers by 
improving access to machinery services. 
2. Background
Access to Mechanization Project
During its implementation, the AMP project used a combination of matching investments, leveraged 
commercial finance, business and extension training, and volunteer technical assistance was to establish 
21 Machinery Service Centers (MSCs). The primary goal of the AMP was to increase the productivity 
and incomes of smallholder farmers by improving access to machinery services. The project had the 
following specific objectives: 
 Establish 21 MSCs offering improved mechanization services to over 12,500 smallholder farmers. 
 Provide technical assistance to these enterprises, using a combination of local consultants and 
Farmer-to-Farmer (FtF) volunteers, to ensure the safe, sustainable and profitable operation of the 
businesses and long-term availability of services for farmers. 
 Leverage USAID grant funds with local partner matching investment, including large- scale 
involvement of local and international commercial finance, to maximize investment in the rural 
economy.
 Improve the competitive environment for machinery services by reducing the cost to farmers, as a 
result of increased supply of machinery and the availability of dedicated service providing businesses. 
 Facilitate the return of more agricultural land into production. 
 Contribute to increased productivity and income of smallholder farmers. 
 Provide discounted machinery services to about 3,000 resettled Internally Displaced Persons (IDPs). 
The project assisted 21 MSC’s with an array of agricultural machinery, using a combination of grant 
funding and local partner matching investment (with a minimum match requirement of 1:1). AMP 
provided the assortment of machinery and implements responsive to the needs in each MSC’s target 
market, with an appropriate mix of tractors of various horsepower and implements required to serve 
the area’s main crops. Flexibility was crucial to ensure the commercial viability of each MSC. 
Matching grants were complemented by rigorous business and management trainings; extension 
trainings, field days and demonstrations; and technical assistance provided by local consultants and FtF 
volunteers. AMP used this combination of financing, business training, and technical assistance to ensure 
sustainability and proper management of the enterprises providing machinery services. 
Through the implementation of the AMP project, 215 new jobs were created and 84 units of agricultural 
machinery and 238 implements were purchased with combined grant funding and matching investment. 
During the life of the AMP project cumulatively more than 2,400 farmers participated in extension 
trainings and 400 – 1,000 farmers have been served by each assisted service center with different 
agricultural services resulting in increased household incomes for MSC beneficiaries by more than $6.25 
million. 
3. Purpose of the evaluation and key evaluation questions to be addressed
 Access to Mechanization Project
The final evaluation will assess the results of AMP in terms of establishing or expanding 21 
1
6
mechanization service centers (MSCs), and increasing the incomes of at least 12,500 small farmer 
beneficiaries. The evaluation will cover the life of the project from September 2009 through November 
2012. The conclusions of the evaluation will be used by USAID/Caucasus to design future interventions 
in the area of agriculture sector development. With similar purpose, USAID will share the results of the 
study with other donors, host country government stakeholders and partners working in this area. The 
evaluation team should review and summarize the implementation and results achieved by this project 
to address the following purposes and answer the following key evaluation questions.
5. Assess the operating condition of MSCs. 
Q. Can each MSC manage its business operations (e.g., positive cash flow from 
operations, record keeping, designation of employee duties, staff coverage, sales 
 and marketing capacity) to be evaluated against standards determined by the 
 evaluators? 
Q. Can each MSC operate and maintain its equipment, and can such equipment be 
 repaired by MSC operators or appropriate service providers? 
Q. Are the maintenance and repairs of equipment regularly funded by MSCs in 
 accordance with applicable service schedules and repair needs? 
Q. Has the establishment of competing Georgian government mechanization 
centers impacted the MSCs established by AMP? 
6. Assess the effectiveness of providing large grants to private enterprises or individuals to 
establish or expand MSCs; and assess whether or not grant assistance caused market distortions 
by answering the following two questions: 
Q. Did USG-funded grants mitigate perceived or real financial or other investment risks, 
causing enterprises or individuals to co-invest with the project? 
Q. How did the establishment or expansion of MSCs harm or help competing 
 mechanization service providers, including Government of Georgia owned 
 centers? 
7. Assess whether or not smallholder farmer productivity and incomes have improved as a result 
of training provided by the project, and access to and utilization of mechanization services. 
Q. Did training and utilization of mechanization services contribute to an increase in 
 collective household incomes? 
8. Identify opportunities to improve impact and enhance the implementation and management of 
future similar projects such as gender equity and sustainability. 
4. Methodology
1
7
The Mission is looking for the offer or to suggest the best methods that minimize bias and provide 
strong evidence. 
The offer or is expected to suggest the use of various data collection and analysis methods, both 
quantitative and qualitative, including document review, key informant interviews with wide range of 
representatives from international community, the GOG, political parties and non- governmental 
organizations, focus group discussions, survey instruments, and others. The methodology for any 
evaluation process that involves the selection of participants (e.g. surveys, focus groups, interviews) must 
be clearly explained and justified. Selected respondents should be representative of women, youth and 
vulnerable groups, where appropriate. The contractor will conduct a two days review for AMP to 
provide a general overview of the current situation in Georgia, and will help identify areas that merit 
closer attention once the team begins its fieldwork. Reading materials will be available to the team prior 
to their arrival. 
The offerer will develop a detailed evaluation design (to be included in the proposal), including data 
collection plan and drafts of data collection tools. A draft of the plan and design will be shared with 
USAID for review and comment prior to in-country arrival. The final plan will then be presented to the 
Mission during the in brief in more detail. The evaluation design should include the evaluation matrix (an 
illustrative evaluation matrix for this study is given below). The offeror should also explain in details 
limitations and weaknesses of the methodology. 
The offeror will also describe a data analysis plan that details the analysis of information collected; what 
procedures will be used to analyze qualitative data collected through key informant and other 
stakeholder interviews; and how the evaluation will analyze and use quantitative data. 
Again, the methods described herein are only illustrative and USAID expects that the offeror will 
suggest the best methods that would generate most reliable and evidence-based answers to the key 
evaluation questions. 
Illustrative evaluation matrix for AMP:
Research Question Data Source Methodology
Can each MSC manage its
business operations (e.g., positive cash flow from 
operations, appropriate record keeping, 
designation of employee duties, staff coverage, 
sales and marketing capacity)?
MSC
staff/managers/owners
Farmers
Document review
Can each MSC operate and
maintain its equipment, and can such equipment 
be repaired by MSC operators or appropriate 
service providers?
MSC staff
Farmers
MSC documents review
and on-site observation
Are the maintenance and
repairs of equipment regularly funded by MSCs in 
accordance with applicable service schedules and 
repair needs?
MSC staff Documents and interviews
1
8
Research Question Data Source Methodology
Did USG-funded grants
mitigate perceived or real financial or other
investment risks, causing enterprises or individuals
to co-invest with the project?
Farmers/clients
Other donors
Local government
Interviews
Focus groups
Has the establishment of competing Georgian 
government mechanization centers impacted the 
MSCs established by AMP?
Farmers
Other donors
Local government
Interviews
Did training and utilization
of mechanization services increase collective 
household incomes? (Use of proxy questions 
if/when necessary)
Farmers Focus groups
Interviews
Identify opportunities to
improve impact and enhance the implementation 
and management of future
similar projects, if any.
Stakeholders Focus groups
Interviews
5. Work Location
Tbilisi and selected Georgian regions and the U.S. 
The teams will travel outside the capital as needed (for AMP to all 8 regions with MSCs: 
Kakheti, Shida Kartli, Kvemo Kartli, Samtskhe-Javakheti, Imereti, Racha, Samegrelo, and Guria) in order 
to meet with key players in diverse parts of the country and to get a better sense of the overall context 
within Georgia. 
6. Performance Period
The following levels of effort are illustrative and should serve only as an example of the staff which may 
be mobilized under this Task Order. These levels may not reflect the actual level of effort contracted, 
and the contractor will be expected to submit its own estimate of the level of effort needed to fulfill the 
objectives. 
Total number
of work days
Number of
work days in the US
Number of work 
days in
Georgia
International
Technical Expert (team 
leader) – Level I
22 7 (two before
and five after visit)
15
1
9
Total number
of work days
Number of
work days in the US
Number of work 
days in
Georgia
Local
Economic and Evaluation 
Consultant – Level II
17 0 17 (including 3 days 
to finalize
the evaluation)
Translator –
Level III
12 0 12
The contractor shall initiate Washington-based work by reading reports and familiarizing him/herself 
with the projects, beginning the effective date of the contract for two days for AMP. 
The contractor will be required to visit Georgia once. The team leader’s visit will commence no later 
than on November 5, 2012, for approximately 34 workdays (15 days for AMP). USAID will leave up to 
the contractor to decide on the sequencing of the two evaluations, however it is expected that the team 
leader will accompany technical staff during both evaluations. It is expected that most of the visits for 
AMP will be in the regions outside of the capital.
The contractor shall finalize both reports in Washington. The Contractor can use up to two workdays 
for AMP. 
A six-day work week will be authorized in Georgia with no premium pay. 
11. Projects Documents for Review
The COR, through the Mission’s Economic Growth office and respective projects AOR and COR will 
put the contractor in contact with its implementing partner and might provide help with a small number 
of meetings (such as meeting with USG agencies). To the extent possible, relevant reports and other 
project documentation will be provided by the Mission to the contractor prior to travel to Georgia. 
These documents are: 
1. Program Description for AMP 
2. AMP annual and quarterly reports 
3. AMP work plans 
4. Studies/assessments produced under both projects. 
5. Other projects documents which will be provided by the EG office in Tbilisi 
The Mission’s EG Team will brief the evaluation team on their perceptions of political dynamics. 
2
0
Evaluation Questions 
1. Assess the operating condition of MSCs. 
Q. Can each MSC manage its business operations (e.g., positive cash flow from 
operations, record keeping, designation of employee duties, staff coverage, sales 
 and marketing capacity) to be evaluated against standards determined by the 
 evaluators? 
Q. Can each MSC operate and maintain its equipment, and can such equipment be 
 repaired by MSC operators or appropriate service providers? 
Q. Are the maintenance and repairs of equipment regularly funded by MSCs in 
 accordance with applicable service schedules and repair needs? 
Q. Has the establishment of competing Georgian government mechanization 
centers impacted the MSCs established by AMP? 
2. Assess the effectiveness of providing large grants to private enterprises or individuals to 
establish or expand MSCs; and assess whether or not grant assistance caused market distortions 
by answering the following two questions: 
Q. Did USG-funded grants mitigate perceived or real financial or other investment risks, 
causing enterprises or individuals to co-invest with the project? 
Q. How did the establishment or expansion of MSCs harm or help competing 
 mechanization service providers, including Government of Georgia owned 
 centers? 
3. Assess whether or not smallholder farmer productivity and incomes have improved as a result 
of training provided by the project, and access to and utilization of mechanization services. 
Q. Did training and utilization of mechanization services contribute to an increase in 
 collective household incomes? 
4. Identify opportunities to improve impact and enhance the implementation and management of 
future similar projects such as gender equity and sustainability. 
2
1
Mission Schedule
◄ February 2013 ~ March 2013 ~ April 2013 ►
Sun Mon Tue Wed Thu Fri Sat
24 25 26 27 28 1 2
3 4 5 6 7 8 9
10 11 12 13 14 15 16
17 18 19 20 21 22 23
24
31
25 26 27 
Review Materials
Research Design and 
Work Plan Preparation
Home
28 
Phone Call 
w/USAID/Georgia
Review Materials
Research Design and 
Work Plan Preparation
Home
29
Team Member 
Conference Call
Review Materials
Research Design and 
Work Plan Preparation
Home
30
2
2
◄ March 2013 ~ April 2013 ~ May 2013 ►
Sun Mon Tue Wed Thu Fri Sat
31 1
Review Materials
Research Design and 
Work Plan Preparation
Home
2
Review Materials
Research Design and 
Work Plan Preparation
Home
3
Submit Draft 
Research Design and 
Work Plan to USAID
Home
4 5 6
7 8
Team Leader departs 
for Georgia
9
Team Leader Arrives in 
Georgia
Holiday in Georgia
10
In-Briefing with 
USAID
Work Plan and 
Research Discussion
Tbilisi
11
Meetings and Interviews 
with Stakeholders
Tbilisi
12
Submit Final 
Research Design and 
Work Plan
Meetings and Interviews 
with Stakeholders
Tbilisi
13
Meetings and Interviews 
with Stakeholders
Tbilisi
14 15
Meetings and Interviews 
with Stakeholders
Team 1: Tbilisi
Team 2: Guria
16 
Meetings and Interviews 
with Stakeholders
Team 1: Kvemo Kartli
Team 2: Guria
17 
Meetings and Interviews 
with Stakeholders
Team 1: Kakheti
Team 2: Samegrelo
18
Meetings and Interviews 
with Stakeholders
Team 1: Kakheti
Team 2: Samegrelo
19 
Meetings and Interviews 
with Stakeholders
Team 1: Shida Kartli
Team 2: Imereti
20
Meetings and Interviews 
with Stakeholders
Team 1: Samtskhe￾Javakheti
Team 2: Racha
21 22 
Meetings and Interviews 
with Stakeholders
Team 1: Samtskhe￾Javakheti
Team 2: Racha
23
Prepare for out-briefing
Tbilisi
24 
Out-Briefing with 
USAID 
Tbilisi
25 
Team Leader Returns 
home
26
Write Draft Report
27
28 29
Write Draft Report
30
Write Draft Report
1 2 3 4
◄ April 2013 ~ May 2013 ~ June 2013 ►
Sun Mon Tue Wed Thu Fri Sat
28 29 30 1
Write Draft Report
2
Submit Draft Report 
to
USAID
3 4
5 6 7 8 9 10 11
12 13 14 15 16 17 18
19 20
Receive USAID 
Comments
21
Integration of USAID 
Comments
22
Integration of USAID 
Comments
23
Submit Final Report 
to USAID
24 25
26 27 28 29 30 30 1
PROPOSED LIST OF AMP STAKEHOLDERS TO BE INTERVIEWED
AMP 
1. MSCs 
2. Local farmers 
Public Sector 
1. Ministry Of Agriculture (current and former officials) 
2. Ministry Of Regional Development And Infrastructure 
3. Ministry of Economy & Sustainable Development 
4. Local Governments / Regional Municipalities 
Donor Organizations 
1. World Bank 
2. Mercy Corp 
3. Care International Caucasus 
4. SDC 
5. UNDP 
6. USAID/ NEO Project 
7. USAID/ EPI Project 
8. European Commission 
9. GIZ
Private Sector (Agricultural Input Supplying Companies and Others) 
1. Agro Geo+ 
2. Borun 
3. Kartlisi 
4. Zadeni 
5. LDT Mekanizatori 
6. Non-USAID funded MSCs 
7. Equipment Dealers 
Associations and Chambers of Commerce 
1. American Chamber Of Commerce (Agribusiness Committee) 
2. Georgian Farmers Association 
3. Georgian Agricultural Corporation 
Banks 
1. Bank of Georgia 
2. ProCredit Bank 
3. TBC 
4. VTB Bank 
5. Kor Standard Bank 
Evaluation Design Chart 
Stage 1: Development of Evaluation Methodology Stage 2: Data 
Collection Stage 3: Results Analysis and Reporting Kickoff Meeting Field Investigation Administration and 
Data Collection Analysis Final Report
Work Products Meet with client to 
finalize requirements 
and discuss needs 
and evaluation 
methodology
Initial Review of 
FAI materials
Methodology Plan Work Plan
Draft Final Report
Submit for Revision
Final Report
Create Data 
Analysis Plan
Analyze Data using 
descriptive and 
inferential statistics
Final Data Files
Meetings, interviews with 
stakeholders
Monitor data 
collection, adjust 
as needed for 
optimal quality
Raw Data File
Set of questions developed
Samples of 
beneficiaries 
selected
Final Methodology
Final Questions
Team Activities
Summary of Evaluation Design and Methodology 
Evaluation Question Type of Analysis 
Conducted
Data Sources and 
Methods Used
Interview Selection 
Process and Size of 
Sample 
Limitations / 
Concerns
Research Area 1. Assess the operating condition of MSCs
Q.1 Can each MSC 
manage its 
business 
operations (e.g. 
positive cash flow, 
record keeping, 
designation of 
employee duties, 
staff coverage, 
sales and 
marketing 
capacity)?
Description – based 
on content analysis 
of expert opinions
Key informant 
interviews with key 
personnel in MSCs 
Interviewees identified 
and agreed by Evaluation 
Team, CNFA/AMP 
personnel and USAID
Description – based 
on content analysis 
of secondary data
Review of MSC 
company sales and 
financial records 
Evaluation team will visit 
all 21 MSCs asking in 
advance for access to 
financial statements
MSCs provide 
financial 
records for 
scrutiny
Description based 
on content analysis 
of primary data
Review of AMP M & E 
records and relevant 
reports
Relevant AMP 
documentation provided to 
Evaluation Team
Q.2 Can each MSC 
operate and 
maintain its 
equipment and can 
such equipment be 
repaired by MSC 
operators or 
appropriate service 
providers?
Description – based 
on content analysis 
of expert opinions
Key informant 
interviews with key 
personnel in MSCs 
Interviewees identified 
and agreed by Evaluation 
Team, CNFA/AMP 
personnel and USAID 
Description and 
comparison of 
ratings by MSCs & 
service providers
Key informant 
interviews with key 
personnel in MSCs & 
selected service 
providers (possible 
focus group session if 
enough participants in 
selected area)
Interviewees identified 
and agreed by Evaluation 
Team, CNFA/AMP 
personnel and USAID
MSCs to assist in 
selection of service 
providers Will service 
providers 
include those 
not 
necessarily 
having a 
positive view 
of AMP
Q.3 Are the 
maintenance & 
repairs of 
equipment funded 
Description – based 
on content analysis 
of expert opinions
Key informant 
interviews with key 
personnel in MSCs and 
selected service 
Interviewees identified 
and agreed by Evaluation 
Team, CNFA/AMP 
Need to define 
service 
schedules & 
Evaluation Question Type of Analysis 
Conducted
Data Sources and 
Methods Used
Interview Selection 
Process and Size of 
Sample 
Limitations / 
Concerns
by MSCs in 
accordance with
service schedules 
& repair needs?
providers personnel and USAID repair needs
Description – based 
on content analysis 
of expert opinions
Focus group sessions 
with farmers in selected 
regions
To be organized by 
Evaluation Team and 
MSCs in sample regions 
Description based 
on content analysis 
of primary data
Review of AMP M & E 
records and relevant 
reports
Relevant AMP 
documentation provided to 
Evaluation Team 
Q.4 Has the 
establishment of 
competing 
Georgian 
government 
mechanization 
centers impacted 
the MSCs 
established by 
AMP?
Description based 
on content analysis 
of primary data
Review of AMP M & E 
records and relevant 
reports 
Relevant AMP 
documentation provided to 
Evaluation Team 
Criteria 
needed to 
address this 
unclear and 
requires 
clarification in 
order avoid 
yes/no answer
Description – based 
on content analysis 
of expert opinions
Key informant 
interviews with key 
personnel in MSCs and 
government 
mechanization centers
Interviewees identified 
and agreed by Evaluation 
Team, CNFA/AMP 
personnel and USAID 
Criteria 
needed to 
address this 
unclear and 
requires 
clarification in 
order avoid 
yes/no answer
Research Area 2. Assess the effectiveness of providing large grants to private enterprises or individuals to 
establish or expand MSCs and assess whether or not this caused market distortions
Q.1 Did USG-funded 
grants mitigate 
perceived or real 
financial or other 
investment risks, 
causing 
enterprises or 
individuals to co￾invest with the 
project?
Description based 
on content analysis 
of primary data
Review of AMP M & E 
records and relevant 
reports 
Relevant AMP 
documentation provided to 
Evaluation Team
Description – based 
on content analysis 
of expert opinions
Key informant 
interviews with key 
personnel in MSCs, 
financial institutions and 
relevant government 
ministries
Interviewees identified by 
Evaluation Team, 
CNFA/AMP personnel and 
USAID 
Difficulty in 
triangulating 
responses 
from 
respondents 
to reach 
meaningful 
conclusion
 Q.2 How did the Description based Review of AMP M & E Relevant AMP 
Evaluation Question Type of Analysis 
Conducted
Data Sources and 
Methods Used
Interview Selection 
Process and Size of 
Sample 
Limitations / 
Concerns
establishment or 
expansion of MSCs 
harm or help 
competing 
mechanization 
service providers, 
including 
government owned 
centers?
on content analysis 
of primary data
records and relevant 
reports 
documentation provided to 
Evaluation Team
Description – based 
on content analysis 
of expert opinions
Key informant 
interviews with key 
personnel in MSCs and 
government 
mechanization centers
Interviewees identified 
and agreed by Evaluation 
Team, CNFA/AMP 
personnel and USAID
Evidence 
based 
responses 
needed but 
question 
closer to 
opinion and / 
or hearsay
Research Area 3. Assess whether or not smallholder farmer productivity and incomes have improved as a result of 
training provided by the project and access to and utilization of mechanization services
 Q.1 Did training and 
utilization of 
mechanization 
services contribute 
to an increase in 
collective 
household 
incomes?
Description based 
on content analysis 
of primary data
Review of AMP M & E 
records and relevant 
reports 
Relevant CNFA/AMP 
documentation provided to 
Evaluation Team
Description – based 
on content analysis 
of expert opinions
Focus group sessions 
with farmers in selected 
regions
To be organized by 
Evaluation Team and 
MSCs in sample regions –
target = 8-10 per focus 
group session including 
representatives from all 
identified participants
No tested 
methodology 
available to 
correlate 
training with 
household 
income
Description – based 
on content analysis 
of expert opinions
Key informant 
interviews with key 
personnel in relevant 
government ministries 
Interviewees identified by 
Evaluation Team, AMP 
personnel and USAID
Research Area 4. Identify opportunities to improve impact and enhance the implementation and management of 
future similar projects such gender equality and sustainability 
Response to appear in text of Final Report with particular reference to recommendations
Overall Monitoring and Evaluation Matrix 
AMP Overall Monitoring and Evaluation Matrix
Period of Performance: September 2009 –December 2012
Indicators Target
Level of 
Achievemen
t (Actual)8
Source(s) 
of 
Verification
Comments/ Reasons for 
Deviation (if any)
1. IDP Specific Indicators:
1.1
No. of IDP families benefiting from 
improved mechanization 2000-3000 2,098 PMPs, quarterly & 
annual reports Achieved
1.2
No. of hectares farmed by IDPs 
covered by mechanization due to 
project
800 801.8 PMPs, quarterly & 
annual reports Achieved
1.3 Value of assistance provided by IDPs $145,000 $120,358 PMPs, quarterly & 
annual reports
Almost achieved – short support timeframe –
April to June 2010
2. Project Indicators –
Programmatic:
2.1 No. of MSCs established 25-30
21
PMPs, quarterly & 
annual reports
Target realigned in Grant Cooperative Agreement 
March 2012
2.2 No. of additional tractors operating in 
target areas
60-100
82
PMPs, quarterly & 
annual reports Achieved
2.3
No. of additional pieces of related 
farm equipment available in target 
areas
240-400
235
PMPs, quarterly & 
annual reports Almost achieved
2.4
Total new investment in agricultural 
machinery due to project (grant funds 
+ matching investment) $3.98m
$5.42m9
PMPs, quarterly & 
annual reports
Significantly exceeded although method of 
calculation not statistically robust
 
8
Total aggregated amount for Years 1,2&3 as of 19.11.12
9
Actual amount rounded up from 7 figure total – all other similar amounts thereafter rounded up in chart
AMP Overall Monitoring and Evaluation Matrix
Period of Performance: September 2009 –December 2012
Indicators Target
Level of 
Achievemen
t (Actual)8
Source(s) 
of 
Verification
Comments/ Reasons for 
Deviation (if any)
2.5 Amount of financing mobilized in 
support of machinery services $1m - $2m
$1.08m
PMPs, quarterly & 
annual reports Achieved
2.6
No. of rural service providing 
enterprises receiving business skills 
training
75
47
PMPs, quarterly & 
annual reports Not achieved – availability of service providers
2.7
No. of agriculture-related firms 
benefiting directly from USG 
supported interventions (incl. both 
grantees and non-grantee service 
providers receiving bus. training
75
47
PMPs, quarterly & 
annual reports
Not achieved – late start of some MSCs means 
benefits accrue post-project
2.8 No. of extension trainings conducted 
by AMP 125
119
PMPs, quarterly & 
annual reports Almost achieved
2.9 No. of field days and demonstrations 
conducted by MSCs 75
78
PMPs, quarterly & 
annual reports Achieved
3. Project Indicators – MSC 
Performance
3.1 New jobs created 225-270
195
PMPs, quarterly & 
annual reports, 
MSCs
Anticipated to be met post-project monitoring
3.2 Increased household income from 
new jobs created £$1.0m - $1.2m
$377,067
PMPs, quarterly & 
annual reports
Anticipated to be met post-project monitoring
3.3 Additional hectares covered by 
mechanization services due to project 9,500 – 15,000
30,388
PMPs, quarterly & 
annual reports Exceeded
3.4 Sales of services to farmers $2.5m - $4.0m
$1.78m
PMPs, quarterly & 
annual reports Anticipated to be met post-project monitoring
3.5 Increased annual gross profit of 
machinery service providers due to 
$1.25m - £2.0m PMPs, quarterly & Not achieved – problems with sourcing accurate 
AMP Overall Monitoring and Evaluation Matrix
Period of Performance: September 2009 –December 2012
Indicators Target
Level of 
Achievemen
t (Actual)8
Source(s) 
of 
Verification
Comments/ Reasons for 
Deviation (if any)
USG assistance $424,021 annual reports data
3.6
%age increase in gross profit of 
machinery service providers due to 
USG assistance
10%
N/a
PMPs, quarterly & 
annual reports Anticipated to be met post-project monitoring
3.7 %age decrease in MSC service prices 
over 2009 baselinerates 20%
N/a
PMPs, quarterly & 
annual reports n/a
3.8 No. of women owned businesses 
assisted 5
2
PMPs, quarterly & 
annual reports
Not achieved – issue regarding availability of 
businesses to participte
4. Project Indicators –
Farmer Impact:
4.1 Ave. plot size serviced by assisted 
farm service providers (incl. IDPs) 0.97 ha
0.41ha
PMPs, quarterly & 
annual reports Not achieved – difficult to assess
4.2
No. of farmers benefiting from the 
provision of increased farm services 
made available due to AMP assistance 
(incl. IDPs)
9,250 – 14,000 12,819
PMPs, quarterly & 
annual reports Achieved
4.3
Percentage, average and total 
increased annual income to small￾scale farmers due to AMP (calculated
through annual farm survey at 
calendar year end
$4.6m - $9.8m
$5.33m
PMPs, quarterly & 
annual reports
Achieved
4.4 No. of women provided training on 
business or agricultural practices 300
164
PMPs, quarterly & 
annual reports
Not achieved – identifying women farmers an 
issue
4.5 No. of farmers participating in field 
days and demonstrations 1,125
1,207
PMPs, quarterly & 
annual reports
Achieved
4.6 No. of farmers participating in AMP 
extension training
1,875
2,318
PMPs, quarterly & 
annual reports
Achieved
AMP Overall Monitoring and Evaluation Matrix
Period of Performance: September 2009 –December 2012
Indicators Target
Level of 
Achievemen
t (Actual)8
Source(s) 
of 
Verification
Comments/ Reasons for 
Deviation (if any)
4.7
No. of farmers, processors and 
others who have adopted new 
technologies or management 
practices due to USG assistance 
(MSC clients, IDPs, service providers, 
training beneficiaries)
12,275 – 17,000
16,539
PMPs, quarterly & 
annual reports
Achieved
4.8
No. of additional hectares nder 
improved technologies or 
management practices due to USG 
assistance
9,500 – 15,000 31,190
PMPs, quarterly & 
annual reports Exceeded
4.9
No. of rural households benefiting 
directly from USG assistance (MSC 
clients, IDPs, service providers, 
training beneficiaries)
12,500 – 17,300
16,539
PMPs, quarterly & 
annual reports
Achieved
Interviewee Profile & Illustrative Questions 
Background Information of Interviewee 
1 Name of interviewee
2 Name of organization
3 Main activities of business
4 Position of person interviewed
5 Region and district of the organization
6 Legal status of organization
7 When business started
8 Interviewee have direct experience working with 
AMP
9 Nature of interviewee’s relationship with AMP
10 Nature of organizations relationship with AMP
Illustrative Questions for Interviewees 
Direct Beneficiaries - Farmers
Q.1 What were the benefits to you of MSC services? For example:
 more timely planting, cultivation and harvest;
 greater acreage under cultivation;
 higher productivity and income per hectare;
reduced costs; increased income).
1. Do you attribute these benefits to project activities?
A.1
Q.2 Did AMP improve the quantity and quality of agricultural services available to you?
A.2
Q.3 Did AMP increase the level of competition among businesses supplying agricultural products and 
services?
A.3
Q.4 Have your costs of using farm machinery been reduced as a result of the AMP/MSC?
A.4
Q.5 Did you or your employees participate in any training events sponsored by AMP or an AMP 
assisted MSC?
A.5
Q.6 What training was received?
A.6
Q.7 How effective was the training? What in your opinion were the best aspects of the training?
A.7
Q.8 In which areas would you like to see more training?
A.8
Q.9 Did you receive any other technical assistance from AMP or an AMP-assisted MSC? What
was the nature of the technical assistance? 
A.9
Q.10 How do you rate the quality of technical assistance provided to you?
A.10
Q.11 In terms of the quality of training/technical assistance, how do you rate the quality of:
 - AMP/MSC consultants?
 - Farmer to Farmer Volunteers?
A.11
Q.12 Did training and utilization of mechanization services provided by AMP-assisted MSCs
increase your income? 
A.12
Q.13 Did the program address the priorities of the participating farmers for agricultural services?
A.13
Q.14 How do you rate the AMP-assisted MSC in terms of responsiveness and timely delivery of
services? 
A.14
Q.15 How in particular did AMP assist IDPs?
Q.15 
Direct Beneficiaries - MSCs
Q.1 Does the MSC have:
 - positive cash flow from operations,
 - appropriate record keeping,
 - designation of employee duties,
 - staff coverage,
 - sales and marketing capacity?
A.1
Q.2 Is this organization sufficiently profitable to assure sustainability?
A.2
Q.3 What is the likelihood that you would have created this organization without AMP assistance?
A.3 
Q.4 Is this organization properly operating and maintaining its equipment? Is the equipment
repaired by MSC operators or appropriate service providers?
A.4
Q.5 Is maintenance and repair of equipment regularly funded by this organization in accordance
with applicable service schedules of the equipment?
A.5
Q.6 Did you or members of your staff participate in any AMP training events, seminars, etc.?
A.6
Q.7 What training was received?
A.7
Q.8 How effective was AMP training? What in your opinion were the best aspects of the AMP
 training? 
A.8
Q.9 In which areas would you like to see more training? 
A.9
Q.10 Did you receive any other technical assistance from AMP or an AMP-assisted MSC? What
 was the nature of the technical assistance? 
A.10
Q.11 How do you rate the quality of technical assistance provided to you? 
A.11
Q.12 In terms of the quality of training/technical assistance, how do you rate the quality of:
 - AMP/MSC consultants?
 - Farmer to Farmer Volunteers? 
A.12
Local and Central Government Officials
Q.1 In your opinion was providing large grants to private enterprises or individuals to establish or 
expand MSCs an effective strategy? Why or why not? What were the alternatives?
A.1
Q.2 Did the MSC program create new jobs? 
A.2
Q.3 Did the establishment or expansion of MSCs harm or help competing mechanization service
providers? If so, how? 
A.3 
Q.4 Did USG-funded grants mitigate perceived or real financial or other investment risks, causing
enterprises or individuals to co-invest with the project? 
A.4
Q.5 Did AMP leverage USAID grant funds with local partner matching investments, including
large-scale involvement of local and international commercial finance, to maximize impact and
investment in the rural economy?
A.5
Q.6 Did AMP improve the competitive environment for machinery services? If so, how?
A.6
Q.7 Did AMP contribute to returning more agricultural land to production? If so, how? 
A.7
DRAFT REPORT OUTLINE 
LIST OF ACRONYMS AND ABBREVIATIONS………………………………………...…
EXECUTIVE SUMMARY ..........................................................................................................
1.0 INTRODUCTION: BACKGROUND AND CONTEXT ...................................................
1.1 Description of the Access to Mechanization Project (AMP)........................................................................
1.2 Purpose of the Evaluation .....................................................................................................................................
1.3 Methodology of the Evaluation ............................................................................................................................
2.0 REVIEW 1: ASSESS THE OPERATING CONDITION OF MSCs……………. 
2.1 Summary of Evaluation Questions - Design and Methods............................................................................
2.2Findings……………………………………………………………………... …………….... 
2.3 Conclusions……….................................................................................................................................................
2.4 Recommendations......................................................................................................................
3.0 REVIEW 2: ASSESS THE EFFECTIVENESS OF PROVIDING LARGE GRANTS 
TO ENTERPRISES AND INDIVIDUALS TO ESTABLISH OR EXPAND MSCs…… 
3.1 Summary of Evaluation Questions - Design and Methods............................................................................
3.2 Findings…………………………………………………………………………………….. ..........
3.3 Conclusions……………………………………………………………………………….. ..........
3.4 Recommendations………………………………………………………………………… ..........
4.0 REVIEW 3: ASSESS WHETHER OR NOT SMALLHOLDER FARMER 
PRODUCTIVITY AND INCOMES HAVE IMPROVED AS A RSULT OF TRAINING ....…
4.1 Summary of Evaluation Questions – Design and Methods ...........................................................................
4.2 Findings…………………………………………………………………………………... 
4.3 Conclusions………………………………………………………………………………..............
4.4 Recommendations .................................................................................................................................................
5.0 REVIEW 4: IDENTIFY OPPORTUNITIES TO IMPROVE IMPACT AND 
ENHANCE IMPLEMENTATION OF FUTURE SIMILAR PROJECTS ................................. 
5.1 Findings………………………. ...........................................................................................................................
5.2 Conclusions ……....................................................................................................................................................
5.3 Recommendations………..........................................................................................................................................
6.0 REVIEW 4: LESSONS LEARNED .....................................................................................
Annexes 
 A. Statement of Work 
 B. Evaluation Design and Methodology 
 C. List of Persons Interviewed 
 D. List of Documents Reviewed 
 E. Overall Monitoring and Evaluation Matrix 
ANNEX C: LIST OF MEETINGS 
List of Meetings 
N CONTACT PERSON POSITION ORGANIZATION LOCATION DATE
1 Louisa Namicheishvili CNFA, Country Director CNFA Tbilisi 10/04/2013
2 Giorgi Tkeshelashvili CNFA, AMP Project Field Officer CNFA Tbilisi 10/04/2013
3 Otar Chigladze CNFA, AMP Project M&E Expert CNFA Tbilisi 10/04/2013
4 Davidt Tsiklauri USAID, Project Manager USAID Tbilisi 10/04/2013
5 Stephen M. Haykin USAID, Mission Director USAID Tbilisi 10/04/2013
6 Keti Chogovadze Project Development Specialist USAID Tbilisi 10/04/2013
7 Roy Plucknett USAID, Deputy Mission Director USAID Tbilisi 10/04/2013
8 Douglas Balko USAID, Economic Growth Office 
Specialist
USAID Tbilisi 10/04/2013
9 Maia Khatiashvili USAID, GIS Expert USAID Tbilisi 10/04/2013
10 Patrick Norrell CNFA, Vice President CNFA Tbilisi 11/04/2013
11 Louisa Namicheishvili CNFA, Country Director CNFA Tbilisi 11/04/2013
12 Giorgi Tkeshelashvili CNFA, AMP Project Field Officer CNFA Tbilisi 11/04/2013
13 Irakli Kasrashvili Mercy Corps, Director Mercy Corps Tbilisi 11/04/2013
14 Nino Zambakhidze GFA, Director Georgian Farmers Association Tbilisi 11/04/2013
15 Shalva Pipia Deputy Minister Ministry Of Agriculture Tbilisi 12/04/2013
16 David Lee Head, Agribusiness Committee American Chamber Of Commerce Tbilisi 12/04/2013
17 Ilia Gogichaisvili Agribusiness Lending Coordinator TBC Bank Tbilisi 12/04/2013
18 Dima Kostarov Head of The Strategic Development 
Department
Agro Geo+ Tbilisi 13/04/2013
19 Valeri Kopaleishvili Ex-Deputy Minister Ministry of Internally Displaced Persons 
from the Occupied Territories, 
Accommodation and Refugees
Tbilisi 13/04/2013
20 Davit Tvaliashvili Owner MSC Gori 15/04/2013
21 Malkhaz Nakhutrishvili Owner MSC Kareli 15/04/2013
22 Ioseb Gogitadze Owner MSC Sachkere 15/04/2013
23 Pavle Lursmanashvili Owner MSC Zestaponi 16/04/2013
24 Avtandil Mikiashvili Owner MSC Ambrolauri 16/04/2013
25 Merab Nadaria Owner MSC Khobi 17/04/2013
26 David Khurodze Owner MSC Senaki 17/04/2013
27 Gela Gamkrelidze Owner MSC Kvemo 
Natanebi
17/04/2013
28 Mamuka Nasidze Owner MSC Akhaltsikhe 18/04/2013
29 Teodradze Otari Owner MSC Akhalkalaki 18/04/2013
30 Nugzar Londaridze Owner MSC Aspindza 18/04/2013
35 Davit Khmelidze Owner MSC Bolnisi 19/04/2013
36 Givi Kaikhosroshvili Owner MSC Marneuli 19/04/2013
37 Mamuka Kharadze Owner MSC Teleti 19/04/2013
38 Davit Petriashvili Owner MSC Tetritskaro 19/04/2013
39 Ioseb Okruashvili Owner MSC Sartichala 20/04/2013
40 Giorgi Zirakishvili Owner MSC Akhmeta 20/04/2013
41 Bezhan Gonashvili Owner MSC Dedoplisckaro 20/04/2013
42 Zaza Juxaridze Manager MSC Tsnori 22/04/2013
43 David Baindurashvili Owner MSC Lagodekhi 22/04/2013
44 Kakhaber Mosemxvdishvli Manager MSC Gurjaani 22/04/2013
45 David Kirvalidze Minister Ministry of Agriculture Tbilisi 22/04/2013
46 Berdia Asanishvili Deputy Head Signagi Municipality Signagi 22/04/2013
47 Ramaz Managadze Agriculture Program Coordinator Lagodekhi Municipality Lagodekhi 22/04/2013
ANNEX D: LIST OF MATERIALS CONSULTED 
List of Materials Consulted 
USAID / CNFA 
Name of Document Date
Cooperative Agreement AID-114-LA-09-00001 28/09/09
Grant & Cooperative Agreement Amendment 09/03/12
AMP Work Plans 
Name of Document Date
Year 1 Work Plan 16/11/09
Year 2 Work Plan 03/12/10
Year 3 Work Plan 31/10/11
Year 1 WP – Annex 1 – Farmer Application Forms --
Year 1 WP – Annex 2 – Evaluation Form --
Year 1 WP – Annex 3 – CNFA Organization Chart --
Year 1 WP – Annex 4 - Unified Implementation Timeline --
 
AMP Annual Reports
Name of Document Date
Year 1 – Annual Report 31/10/10
Year 2 – Annual Report 25/10/11
AMP Quarterly Reports
Name of Document Date
Q1 Report – Annex 1 – Application Form --
Q1 Report – Annex 2 – Evaluation Form --
Q1 Report – Annex 3 – Applicant’s Guide 26/11/09
Q1 Report – Annex 4 – General Eligibility Criteria --
Q1 Report – Annex 5 – Environmental & Worker Safety Guidelines 28/11/09
Q1 Report – Annex 6 – Guidelines for the Procurement of AMP Goods & 
Services
25/12/09
Q1 Report – Annex 7 – Outreach Strategy for AMP --
Q2 Report – (01/01/10 – 31/03/10) 30/04/10
Q3 Report – (01/04/10 – 30/06/10) 30/07/10
Q5 Report - (01/10/10 – 31/12/10) 29/01/11
Q6 Report - (01/01/11 – 31/03/11) 29/04/11
Q7 Report – (01/04/11 – 30/06/11) 29/07/11
Q9 Report – (01/10/11 – 31/12/11) 31/01/12
Q10 Report – (01/01/12 – 31/03/12) 30/04/12
Q11 Report – (31/03/12 – 30/06/12) 31/07/12
AMP Performance Monitoring & Evaluation Plans (PMEPs) & docs
Name of Document Date
PMEP 19/11/09
Annex 1 – Quarterly Reporting Form 19/11/09
Annex 2 – Training beneficiary Record 19/11/09
Annex 3 – Client Survey Form 13/12/09
Annex 4 – Project Monitoring Plan 13/12/09
PMP Data Collection Spreadsheet 29/10/09
PMP Data Collection Spreadsheet 05/03/10
Initial Indicators 22/10/09
Performance Indicator Reference Sheets 05/03/10
Performance Indicator Reference Sheet – Short Version 05/03/10
Narrative Achievements – AMP? 11/11
List of Georgia Indicators 17/11/11
AMP Project Papers
Name of Document Date
Crop Production Handbook 2010
Modern Machinery System for Crop Production
Food Safety
AMP Business Training & Extension Training Materials
Name of Document Date
Business Planning
Extension Training – Assorted brochures and hand- outs
AMP Miscellaneous Documents
Name of Document Date
AMP – End-of Project Report
Modification – Attachment 1 
AMP Grantees Contact Details
AMP – Grant Summary Database (Electronically)
Strategy of Agriculture Development of Georgia (Ministry of 
Agriculture) 2012 - 2022
2012
AMP – Household Income Survey
AMP – Impact Data for all Grantees
MSC Cash-Flow Statements (prepared by AMP) Dates
1. Laba+ LLC 2010 & 2011
2. Malkhaz Nakhutsrishvili I/E 2010 & 2011
3. Bezhan Gonashvili I/E 2010 & 2011
4. Davit Petriashvili I/E 2010 & 2011
5. Davit Tvaliashvoli I/E 2010 & 2011
6. Alaverdi LLC 2010 & 2011
7. Dorani LLC 2010 & 2011
8. Geo Nut LLC 2010 & 2011
9. Agronominali LLC 2010 & 2011
10. Nugzar Londaridze I/E 2010 & 2011
11. Mamuli 96 LLC 2010 & 2011
12. Alva LLC 2010 & 2011 
13. DV+ LLC 2010 & 2011
14. Levan Aroshidze I/E 2010 & 2011
15. Ruka Mapping LLC 2011 & 2012
16. Gela Gamkrelidze I/E 2011 & 2012
17. Mamuka Kharadze I/E 2011 & 2012
18. Avtandil Mikiashvili I/E 2011 & 2012
19. Energia 777 LLC 2011 & 2012 
20. Lursmanashvili SLC 2011 & 2012
21. Daviti LLC 2011 & 2012
MSC Balance Sheets (prepared by AMP) Date
1. Laba+ LLC 2010 & 2011
2. Malkhaz Nakhutsrishvili I/E 2010 & 2011
3. Bezhan Gonashvili I/E 2010 & 2011
4. Davit Petriashvili I/E 2008 - 2011
5. Davit Tvaliashvoli I/E 2010 & 2011
6. Alaverdi LLC 2010 & 2011
7. Dorani LLC 2010 & 2011
8. Geo Nut LLC 2010 & 2011
9. Agronominali LLC 2010 & 2011
10. Nugzar Londaridze I/E 2010 & 2011
11. Mamuli 96 LLC 2010 & 2011
12. Ruka Mapping LLC 2011 & 2012
13. Gela Gamkrelidze I/E 2011 & 2012
14. Alva LLC 2010 & 2011
15. Levan Aroshidze I/E 2010 & 2011
16. Mamuka Kharadze I/E 2011 & 2012
17. Avtandil Mikiashvili I/E 2010 & 2011
18. Energia 777 LLC 2011 & 2012
19. Lursmanashvili SLC 2010 & 2011
20. DV+ LLC 2008 - 2011
21. Daviti LLC 2011 & 2012
ANNEX E: OVERVIEW OF THE EVALUATION TEAM’S MISSION FINDINGS
OVERVIEW OF THE EVALUATION TEAM’S MISSION FINDINGS
MSC Start 
date
Increased 
productivity 
resulting 
from MSC 
services
Household 
Income 
Increase?
Jobs 
Created
MSCs/ 
farmers 
benefit 
from 
capacity 
building 
training?
Farmer Focus 
Group Sessions
Barriers / 
restrictions for 
farmers to 
become more 
competitive
Sustainability 
Likelihood 
MSCs 
(Market 
development 
capacity) *
Yes % 
age
Yes % 
age
Yes Nu
m
Yes Male Fem
ale
ID
P
Identified 
Problem 
Yes Reaso
n
1 Gori Jan 2011  20-25  7-15  16  10 - - Price instability  D > S
2 Kareli Oct 2010  15-20  5-10  12  8 - - Lack of spare 
parts
 D > S
3 Sachkhere Dec 2011  15-20  5-10  8  7 1 - Lack of proper 
machinery
 D > S
4 Zestaponi Oct 2012  n/a  n/a  12  10 - - No agri -. 
insurance 
 D > S
5 Ambrolauri May 2012  n/a n/a  13  (MSC 
only)
No focus group Poor prod’n 
planning
 D > S
6 Khobi Sept 2011  30-40  10-15  17  7 - - Machine  D > S
availability
7 Senaki May 2011  n/a  n/a  9  (MSC 
only)
No focus group Land 
fragmentation
 D > S
8 Natanebi Nov 2011  50  20-25  11  9 - - Lack of proper 
machinery
 D > S
9 Akhaltsikhe July 2010  25-50  None  45  9 4 - Cheap Turkish 
imports
 D > S
10 Aspindza July 2011  20-30  None  11  10 3 - Cheap Turkish 
imports
 D > S
11 Akhalkalaki Apr 2012  20  None  12  6 - - Cheap Turkish 
imports
 D > S
12 Bolnisi Nov 2010  20-40  10  15  9 - - Lack combine 
harvester
 D > S
13 Marneuli Mar 2011  20-30  5-10  15  17 3 2** Cheap Turkish 
imports
 D > S
14 Tetritskaro Nov 2010  20  5-10  12  10 3 4 Irrigation channels 
need excavated
 D > S
15 Teleti Apr 2012  50-70  30-35  6  6 - - Fertility of land  D > S
16 Sartichala Aug 2011  20-25  20  46  19 2 - Lack of input 
supplies
 D > S
17 Akhmeta Apr 2011  n/a  n/a  9  (MSC 
only)
No focus group Timely access to 
spare parts
 D > S
18 Gurjaani Dec 2011  n/a  n/a  8  (MSC 
only)
No focus group Soil testing  D > S
19 Tsnori June 2011  30-40  10-15  9  8 - - Lack of proper 
machinery
 D > S
20 Lagodekhi Dec 2011  n/a  n/a  7  18 - -
Lack of proper 
machinery
 D > S
21 Dedoplisck
aro
Nov 2010  25-35  15-20  11  15 - - Cheap Import  D > S 
TOTAL 293 178 16 6
* Criteria for sustainability also includes analysis of financial statements, market demand, timely repayment of bank loans, employment
** 2 = female IDPs 
NOTES 
1. Jobs created = full time + seasonal part-time 
2. Household income increase - calculated by show of hands 
3. n/a = not available 
OVERVIEW OF THE EVALUATION TEAMS ANSWERS TO QUESTIONS 
MSC
Q1
Can each MSC 
manage 
business 
operations 
(finance, sales 
& mktg, staff..)
Q2
Ability to 
operate, 
maintain & 
repair 
equipment?
Q3
Equipment 
maintenance 
& repair in 
line with 
service 
schedules?
Q4
Do the gov’t 
mechaniz’n 
centres 
Impact the 
AMP MSCs
Q5
Was availability 
of matching 
grant key 
decision to co￾invest?
Q6
Did creation 
of MSCs harm 
gov’t or 
competing 
mechaniz’n 
service 
providers?
Q7
Did training & 
use of MSCs 
contribute to 
an increase in 
household 
incomes? *
H M L H M L H M L H M L H M L H M L H M L
1 Gori       
2 Kareli       
3 Sachkhere       
4 Zestaponi       No focus group
5 Ambrolauri       No focus group
6 Khobi       
7 Senaki       No focus group
8 Natanebi       
9 Akhaltsikhe       
10 Aspindza       
11 Akhalkalaki       
12 Bolnisi       
13 Marneuli       
14 Tetritskaro       
15 Teleti       
16 Sartichala       
17 Akhmeta       No focus group
18 Gurjaani       No focus group
19 Tsnori       
20 Lagodekhi      
21 Dedopliscka
ro
      
Legend: H = High 
 M = Medium 
 L = Low 
* Legend H = > 40% 
 M = 20% - 40% 
 L = < 20% 
ANNEX F: MSC INTERVIEWEE QUESTIONNAIRE 
Interviewee Background and Questionnaire 
MSC – Akhalkalaki Machinery Service Center
1 Local Partner Energia 777 LLC
2 Name of interviewee
3 Position of interviewee
4 Main activities of business
5 Region and district of the organization
6 Legal status of organization
7 When business started
8 Interviewee have direct experience 
working with AMP
MSC Interviewee Questionnaire
Q.1 Does the MSC have:
 - positive cash flow from operations,
 - properly defined employee duties,
 - staff coverage,
 - sales and marketing capacity?
A.1
Q.2 Is the business sufficiently profitable to assure sustainability? What financial records do you 
keep to track and monitor performance?
A.2
Q.3 What is the likelihood that you would have created this organization without AMP 
assistance?
A.3 
Q.4 Is the organization properly operating and maintaining its equipment? Can the equipment
Be repaired by MSC operators or appropriate service providers?
A.4
Q.5 Is maintenance and repair of equipment regularly funded by this organization in accordance 
with applicable service schedule and repair needs?
A.5
Q.6 Did you or members of your staff participate in any AMP training events, seminars, etc.?
A.6
Q.7 If so, how effective was the training in addressing your needs?
A.7
Q.8 What in your opinion were the best aspects of the AMP training and in which areas would 
you like to see more training?
A.8
Q.9 Did you receive any other technical assistance from AMP? What was the nature of the 
technical assistance and how did you rate it in terms of quality?
A.9
Q.10 In terms of the quality of training/technical assistance, how do you rate the quality of:
 - AMP/MSC consultants?
 - Farmer to Farmer Volunteers?
A.10
Q.11 Was the availability of an AMP matching-grant a major factor in your decision to set up 
the business in terms of reducing your risk exposure?
A.11
Q.12 Since the establishment of the MSC has your business affected the activities of either 
government centers or competing mechanization service providers?
A.12